Experiments

A few ready-to-run experiments using the command line interface.

Warning

Make sure you have the experiments folder from the Github repository salmanazarr/torchrl.

Note

See CLI Reference for further extensions of these commands.

List all Problems

This lists each problem and associated hyperparameter set.

torchrl --usr-dirs experiments list -o yaml problems
a2c_cartpole:
- a2c_cartpole
ddpg_pendulum:
- ddpg_pendulum
dqn_cartpole:
- double_dqn_cartpole
- dqn_cartpole
per_cartpole:
- per_cartpole
ppo_pendulum:
- ppo_pendulum

We use one of the problems from above as an example.

Run DQN on CartPole

torchrl --usr-dirs experiments run dqn_cartpole \
        --hparam-set dqn_cartpole \
        --seed 1 \
        --log-dir log/dqn \
        --progress

Resume Experiment

To resume this experiment, we simply point to the log directory,

torchrl --usr-dirs experiments resume log/dqn --progress

Extra Hyperparameters

Extra hyperparameters can be provided as arbitrary key value pairs multiple times and can be accessed inside the Problem

torchrl --usr-dirs experiments run dqn_cartpole \
        --hparam-set dqn_cartpole \
        --extra-hparams "num_total_steps=3000" \
        --extra-hparams "lr=0.0001" \
        --seed 1 \
        --log-dir log/dqn \
        --progress