mirror of
https://gitlab.rlp.net/mobitar/ReCo.jl.git
synced 2024-12-30 16:03:32 +00:00
No description
analysis | ||
graphics | ||
performance | ||
scripts | ||
src | ||
test | ||
visualization | ||
.gitignore | ||
.JuliaFormatter.toml | ||
LICENSE | ||
Manifest.toml | ||
Project.toml | ||
README.adoc |
= ReCo.jl :source-highlighter: highlight.js :highlightjs-languages: bash, julia, latex image:https://img.shields.io/badge/code%20style-blue-4495d1.svg[Code Style: Blue, link=https://github.com/invenia/BlueStyle] **Re**inforcement learning of **co**llective behavior. == Setup The following steps from the setup have to be followed every time before running anything in the following sections. An exception is installing the dependencies which has to be done only the first time. === Launch Julia To activate the environment, navigate to the main directory `ReCo.jl` and then run the following to launch Julia: [source,bash] ---- cd ReCo.jl julia --threads auto ---- `auto` automatically sets the number of threads to use. If you want to use a specific number `N` of threads, replace `auto` with `N`. === Activating environments After launching Julia, the package environment has to be activated by running the following in the REPL: [source,julia] ---- using Pkg Pkg.activate(".") ---- `"."` stands for the current path which has to be the path of the package directory `ReCo.jl`. If specified otherwise, an environment in a subdirectory has to be activated instead of the package environment. To activate an environment of a subdirectory `SUBDIRECTORY`, run the following: [source,julia] ---- using Pkg Pkg.activate("SUBDIRECTORY") ---- === Install/update dependencies After activating the package environment, run the following to install/update the package dependencies: [source,julia] ---- Pkg.update() ---- This step has to be done by running the line above for the first time using the package to install its dependencies. Running the line above after the first time will update the installed dependencies which is optional. This step has to be done **additionally** after activating an environment in a subdirectory. === Import the package You can import the package by running: [source,julia] ---- using ReCo ---- This will export the package's functions that are intended to be used by the end user. == Help mode To access the documentation of the presented package functions further in this README, run `using ReCo` first. Then, enter the help mode by pressing `?` in the REPL. Now, enter the function's name followed by enter to see its documentation. == Run a simulation Initialize a simulation with 100 particles having a self-propulsion velocity of 40.0 and return the relative path to the simulation directory: [source,julia] ---- sim_dir = init_sim(; n_particles=100, v₀=40.0) ---- Run the simulation: [source,julia] ---- run_sim(sim_dir; duration=20.0) ---- The values for the number of particles, self-propulsion velocity and simulation duration are used here as an example. For more information about possible values and other optional arguments, see the documentation of `init_sim` or `run_sim`. == Simulation visualization For visualization, the environment of the subdirectory `visualization` has to be activated. === Animation To generate an animation of a simulation, run the following: [source,julia] ---- include("visualization/Animation.jl") using .Animation animate(sim_dir) ---- The function's documentation includes all possible optional arguments and where the output can be found. === Snapshot plot To plot only one snapshot of a simulation, run the following: [source,julia] ---- include("visualization/SnapshotPlot.jl") using .SnapshotPlot plot_snapshot(sim_dir) ---- This will ask for the number of the snapshot to plot out of the total number of snapshots. The function's documentation includes all possible optional arguments and where the output can be found. == Run a reinforcement learning process Run a reinforcement learning process and return the environment helper and the the relative path of the process directory: [source,julia] ---- env_helper, rl_dir = run_rl(ENVTYPE) ---- `ENVTYPE` has to be replaced by one of the environments named after the file names in the directory `RL/Envs`, for example: `LocalCOMEnv`. A short description of an environment is included at the beginning of the corresponding file. The documentation of `run_rl` includes all possible optional arguments. === Q-matrix `env_helper` has the abstract type `EnvHelper`. To access the Q-matrix, enter the following: [source,julia] ---- env_helper.shared.agent.policy.learner.approximator.table ---- To generate a LaTeX table with the states and actions combintation names for the Q-matrix, activate the environment of the subdirectory `analysis` and run the following: [source,julia] ---- include("analysis/Q_matrix_latex_export.jl") export_q_matrix(env_helper, FILENAME_WITHOUT_EXTENSION) ---- `FILENAME_WITHOUT_EXTENSION` has to be replaced by the wanted file name without extension of the `.tex` file. The function's documentation explains where the output is placed. The output file can be used in a LaTeX document: [source,latex] ---- \input{FILENAME_WITHOUT_EXTENSION} ---- === Rewards To access the rewards, run the following: [source,julia] ---- env_helper.shared.hook.rewards ---- To plot the rewards, activate the environment of the subdirectory `visualization` and run the following: [source,julia] ---- include("visualization/RewardsPlot.jl") using .RewardsPlot plot_rewards(rl_dir) ---- The function's documentation explains where the output is placed. === Mean kappa To plot the mean of kappa as the ratio of the eigenvalues of the gyration tensor, activate the environment of the subdirectory `analysis` and run the following: [source,julia] ---- include("analysis/mean_kappa.jl") plot_mean_κ(; rl_dir=rl_dir, n_last_episodes=N_LAST_EPISODES) ---- `N_LAST_EPISODES` is the number of the last episodes of the learning process to average over. The function's documentation explains where the output is placed. == Run analysis First, activate the environment of the subdirectory `analysis`. === Mean squared displacement and random walk [source,julia] ---- include("analysis/mean_squared_displacement.jl") run_msd_analysis() run_random_walk() ---- The output is `exports/graphics/mean_squared_displacement.pdf` and `exports/graphics/random_walk.pdf`. === Radial distribution function [source,julia] ---- include("analysis/radial_distribution_function/radial_distribution_function.jl") run_radial_distribution_analysis() ---- The output is `exports/graphics/radial_distribution.pdf` and `exports/graphics/radial_distribution_all_vs.pdf`. === Reward discount analysis [source,julia] ---- include("analysis/reward_discount_analysis.jl") run_reward_discount_analysis() ---- The output is `exports/graphics/reward_discount_analysis.pdf`. == Graphics The subdirectory `graphics` has some Julia files that generate graphics related to this package. The function in every file that has to be run to generate the corresponding graphics starts with `plot_` or `gen_`. The output is placed in `exports/graphics`. The subdirectory of a file has an environment that has to be activated first.