Example ==================================== After the Smartflow framework installation, you can couple the Smartflow framework with different CFD solvers for reinforcement learning (RL) applications. This example demonstrates how to train a reinforcement learning agent using the SmartFlow framework on a turbulent channel flow case at a friction Reynolds number of Ret = 5200. The case is located in the ``/your/smartflow/path/SmartFlow/examples/train_retau_05200`` directory. The training process is based on the MlpPolicy, and the trained agent is evaluated on the same case. Flow Case Configuration ---------------------------- The CFD solver used in this case is `CaLES `_, configured specifically for channel flow at Ret = 5200. For details on the CaLES case setup, please refer to the `flow_cases `_ directory. Training Configuration --------------------------- The following YAML-style settings define the training environment and parameters as an example, and you may need to adjust them according to your specific requirements. In general, each case has its own configuration file, i.e., `config.yaml`, which contains the settings for the environment, runner, and other parameters.Taking the `retau_05200` case as an example, the configuration file is located in the ``/your/samrtflow/path/SmartFlow/examples/train_retau_05200`` directory. The configuration file is as follows: config.yaml ^^^^^^^^^^^^^^^^^^ .. code-block:: yaml wandb: project: "channel" run_name: "train-retau-05200" mode: "online" sync_tensorboard: true save_code: true environment: # The following settings are used for the environment. You may need to adjust them according to your specific requirements. # The number of CFD running instances parallelly n_cfds: 2 # The number of agents per CFD environment agents_per_cfd: 48 # The number of tasks (processors) per CFD, which is the number of MPI ranks # Please note it should not exceed the number of cores your server has tasks_per_cfd: 4 # Your desired state, action, and reward dimensions per your project requirements cfd_state_dim: 2 cfd_action_dim: 1 cfd_reward_dim: 2 # The number of agent state, action, and reward dimensions per your project requirements agent_state_dim: 2 agent_action_dim: 1 # The number of CFD steps between two consecutive actions cfd_steps_per_action: 10 # The number of cells between two consecutive agents in x-, y-, and z-directions agent_interval: 4 neighbor_count: 0 # The number of polling for polling the database poll_time: 360000000 verbosity: "debug" # whether to save the trajectories save_trajectories: true trajectory_path: "trajectories" cfd_dtype: "float64" # You don't need to change the action bounds unless you have a specific requirement action_bounds: [-1.0, 1.0] # The reward beta value, which is used to balance the global reward and local rewards in this step reward_beta: 0.2 # The case names for the CFD case case_names: ["retau_05200"] # You may need to adjust the CaLES executable path and case folder according to your setup below: executable_path: "/scratch/maochao/code/CaLES/build/cales" # Path to the CaLES executable case_folder: "../flow_cases" runner: mode: "train" # Whether to restart the training from the last checkpoint # If you want to restart the training, set it to true restart: false policy: "MlpPolicy" # Action network type # Whether to reset the number of timesteps for the training reset_num_timesteps: false total_cfd_episodes: 3200 # The total number of CFD cases # The number of actions per CFD episode for each agent steps_per_episode: 120 hidden_layers: [128, 128] # The learning rate for the training learning_rate: 5e-4 log_interval: 1 summary_interval: 1 seed: 16 ckpt_num: 1000000 ckpt_interval: 1 batch_size: 1 logging: save_dir: '../runs' smartsim: n_dbs: 1 network_interface: "lo" run_command: "mpirun" launcher: "local" extras: n_cells: 16 tauw_min_percent: 0.8 tauw_max_percent: 1.2 hwm_min: 0.075 hwm_max: 0.150 kap_log: 0.41 train.sh ^^^^^^^^^^^^ Before starting the training, remove any previous logs and caches: .. code-block:: bash rm -r __pycache__ trajectories envs tensorboard_logs wandb models/checkpoints Run the training script with the following command: .. code-block:: bash python -u ../../src/smartflow/main.py \ runner.mode=train \ runner.total_cfd_episodes=400 \ runner.restart=False \ runner.reset_num_timesteps=True \ > out 2> err Evaluation Configuration --------------------------- The following settings are used for evaluating a trained policy. Similar to the training configuration, you may need to adjust them according to your specific requirements. Taking the case above as the example, the evaluation configuration file is located in the ``/your/smartflow/path/SmartFlow/examples/eval_retau_05200`` directory. The configuration file is as follows: config.yaml ^^^^^^^^^^^^^^^^^^ .. code-block:: yaml wandb: project: "channel" run_name: "eval-retau-05200" mode: "online" sync_tensorboard: true # group: null # Optional # tags: [] # Optional save_code: true environment: n_cfds: 1 agents_per_cfd: 48 tasks_per_cfd: 4 cfd_state_dim: 2 cfd_reward_dim: 2 cfd_action_dim: 1 agent_state_dim: 2 agent_action_dim: 1 cfd_steps_per_action: 10 agent_interval: 4 neighbor_count: 0 poll_time: 360000000 verbosity: "debug" save_trajectories: true trajectory_path: "trajectories" cfd_dtype: "float64" action_bounds: [-1.0, 1.0] reward_beta: 0.2 case_names: ["retau_05200"] # You may need to adjust the CaLES executable path according to your specific dictionary. executable_path: "/scratch/maochao/code/CaLES/build/cales" case_folder: "../flow_cases" runner: mode: "eval" restart: false policy: "MlpPolicy" reset_num_timesteps: true total_cfd_episodes: 1 steps_per_episode: 120 hidden_layers: [128, 128] learning_rate: 5e-4 log_interval: 1 summary_interval: 1 seed: 16 ckpt_num: 1000000 ckpt_interval: 1 batch_size: 1 logging: save_dir: '../runs' smartsim: n_dbs: 1 network_interface: "lo" run_command: "mpirun" launcher: "local" extras: n_cells: 16 tauw_min_percent: 0.8 tauw_max_percent: 1.2 hwm_min: 0.075 hwm_max: 0.150 kap_log: 0.41 eval.sh ^^^^^^^^^^^^ Before starting the evaluation, remove any previous logs and caches: .. code-block:: bash rm -r __pycache__ trajectories envs tensorboard_logs wandb Run the evaluation script with the following command: .. code-block:: bash python -u ../../src/smartflow/main.py \ # You may need to adjust the path to the trained model according to your setup. runner.model_load_path="/scratch/maochao/code/SmartFlow/experiments/train_retau_05200/models/final/yjkxqlf3" \ runner.steps_per_episode=3600 \ > out 2> err .. note:: When you run the case code, if you see something like ``Autnentication required, XXX`` in the error file, please check your operating system and ``DISPLAY`` status. You may need to set the ``DISPLAY`` environment variable to none and restart your server. After your training and evaluation, you can check the results in the ``wandb`` directory. You can also visualize the results using TensorBoard in ``wandb`` website under your account.