1
0
Fork 0
mirror of https://gitlab.rlp.net/mobitar/ReCo.jl.git synced 2024-10-11 20:34:22 +00:00
ReCo.jl/README.adoc
2022-08-05 14:00:08 +02:00

211 lines
6.2 KiB
Text

= ReCo.jl
:source-highlighter: highlight.js
:highlightjs-languages: bash, julia
image:https://img.shields.io/badge/code%20style-blue-4495d1.svg[Code Style: Blue, link=https://github.com/invenia/BlueStyle]
**Re**inforcement learning of **co**llective behavior.
== Setup
The following steps from the setup have to be followed every time before running anything in the following sections. An exception is installing the dependencies which has to be done only the first time.
=== Launch Julia
To activate the environment, navigate to the main directory `ReCo.jl` and then run the following to launch Julia:
[source,bash]
----
cd ReCo.jl
julia --threads auto
----
`auto` automatically sets the number of threads to use. If you want to use a specific number `N` of threads, replace `auto` with `N`.
=== Activating environments
After launching Julia, the package environment has to be activated by running the following in the REPL:
[source,julia]
----
using Pkg
Pkg.activate(".")
----
`"."` stands for the current path which has to be the path of the package directory `ReCo.jl`.
If specified otherwise, an environment in a subdirectory has to be activated instead of the package environment. To activate an environment of a subdirectory `SUBDIRECTORY`, run the following:
[source,julia]
----
using Pkg
Pkg.activate("SUBDIRECTORY")
----
=== Install/update dependencies
After activating the package environment, run the following to install/update the package dependencies:
[source,julia]
----
Pkg.update()
----
This step has to be done by running the line above for the first time using the package to install its dependencies. Running the line above after the first time will update the installed dependencies which is optional.
This step has to be done **additionally** after activating an environment in a subdirectory.
=== Import the package
You can import the package by running:
[source,julia]
----
using ReCo
----
This will export the package's functions that are intended to be used by the end user.
== Help mode
To access the documentation of the presented package functions further in this README, run `using ReCo` first. Then, enter the help mode by pressing `?` in the REPL. Now, enter the function's name followed by enter to see its documentation.
== Run a simulation
Initialize a simulation with 100 particles having a self-propulsion velocity of 40.0 and return the relative path to the simulation directory:
[source,julia]
----
sim_dir = init_sim(; n_particles=100, v₀=40.0)
----
Run the simulation:
[source,julia]
----
run_sim(sim_dir; duration=20.0)
----
The values for the number of particles, self-propulsion velocity and simulation duration are used here as an example. For more information about possible values and other optional arguments, see the documentation of `init_sim` or `run_sim`.
== Simulation visualization
For visualization, the environment of the subdirectory `visualization` has to be activated.
=== Animation
To generate an animation of a simulation, run the following:
[source,julia]
----
include("visualization/Animation.jl")
using .Animation
animate(sim_dir)
----
The function's documentation includes all possible optional arguments and where the output can be found.
=== Snapshot plot
To plot only one snapshot of a simulation, run the following:
[source,julia]
----
include("visualization/SnapshotPlot.jl")
using .SnapshotPlot
plot_snapshot(sim_dir)
----
This will ask for the number of the snapshot to plot out of the total number of snapshots. The function's documentation includes all possible optional arguments and where the output can be found.
== Run a reinforcement learning process
Run a reinforcement learning process and return the environment helper and the the relative path of the process directory:
[source,julia]
----
env_helper, rl_dir = run_rl(ENVTYPE)
----
`ENVTYPE` has to be replaced by one of the environments named after the file names in the directory `RL/Envs`, for example: `LocalCOMEnv`. A short description of an environment is included at the beginning of the corresponding file.
The documentation of `run_rl` includes all possible optional arguments.
=== Q-matrix
`env_helper` has the abstract type `EnvHelper`. To access the Q-matrix, enter the following:
[source,julia]
----
env_helper.shared.agent.policy.learner.approximator.table
----
=== Rewards
To access the rewards, run the following:
[source,julia]
----
env_helper.shared.hook.rewards
----
To plot the rewards, activate the environment of the subdirectory `visualization` and run the following:
[source,julia]
----
include("visualization/RewardsPlot.jl")
using .RewardsPlot
plot_rewards(rl_dir)
----
The function's documentation explains where the output is placed.
=== Mean kappa
To plot the mean of kappa as the ratio of the eigenvalues of the gyration tensor, activate the environment of the subdirectory `analysis` and run the following:
[source,julia]
----
include("analysis/mean_kappa.jl")
plot_mean_κ(; rl_dir=rl_dir, n_last_episodes=N_LAST_EPISODES)
----
`N_LAST_EPISODES` is the number of the last episodes of the learning process to average over. The function's documentation explains where the output is placed.
== Run analysis
First, activate the environment of the subdirectory `analysis`.
=== Mean squared displacement and random walk
[source,julia]
----
include("analysis/mean_squared_displacement.jl")
run_msd_analysis()
run_random_walk()
----
The output is `exports/graphics/mean_squared_displacement.pdf` and `exports/graphics/random_walk.pdf`.
=== Radial distribution function
[source,julia]
----
include("analysis/radial_distribution_function/radial_distribution_function.jl")
run_radial_distribution_analysis()
----
The output is `exports/graphics/radial_distribution.pdf` and `exports/graphics/radial_distribution_all_vs.pdf`.
=== Reward discount analysis
[source,julia]
----
include("analysis/reward_discount_analysis.jl")
run_reward_discount_analysis()
----
The output is `exports/graphics/reward_discount_analysis.pdf`.
== Graphics
The subdirectory `graphics` has some Julia files that generate graphics related to this package. The function in every file that has to be run to generate the corresponding graphics starts with `plot_` or `gen_`. The output is placed in `exports/graphics`. The subdirectory of a file has an environment that has to be activated first.