torchrl.utils package

Submodules

torchrl.utils.gym_utils module

torchrl.utils.gym_utils.append_run_history(history, *transition)[source]

Extend history into numpy arrays.

This is an extension function for init_run_history. Transition is a 5-tuple containing observation, action, reward, next_state and done flag.

torchrl.utils.gym_utils.get_gym_spaces(make_env_fn: Callable[..., gym.core.Env]) → Tuple[gym.core.Space, gym.core.Space][source]

A utility function to get observation and actions spaces of a Gym environment

torchrl.utils.gym_utils.init_run_history(observation_space: gym.core.Space, action_space: gym.core.Space) → list[source]

Initialize history for numpy-based Gym environments.

The function returns numpy arrays in the following order * obs_history: T x observation_shape * action_history: T x action_shape

action_shape is 1 for discrete environments
  • reward history: T x 1
  • next_obs_history: T x observation_shape
  • done_history: T x 1

torchrl.utils.misc module

torchrl.utils.misc.import_usr_dir(usr_dir)[source]
torchrl.utils.misc.minibatch_generator(*args, minibatch_size=5)[source]
torchrl.utils.misc.set_seeds(seed)[source]

Set the seed value for PyTorch, NumPy and Python. Important for reproducible runs! :param seed: seed value :return:

torchrl.utils.misc.to_camel_case(name: str)[source]

torchrl.utils.multi_envs module

class torchrl.utils.multi_envs.MultiGymEnvs(make_env_fn, n_envs: int = 1, base_seed: int = 0, daemon: bool = True, autostart: bool = True)[source]

Bases: torchrl.utils.multi_proc_wrapper.MultiProcWrapper

A utility class which wraps around multiple environments and runs them in subprocesses

close()[source]
render(env_ids: list)[source]
reset(env_ids: list)[source]
step(env_ids: list, actions: list)[source]

torchrl.utils.multi_proc_wrapper module

class torchrl.utils.multi_proc_wrapper.MultiProcWrapper(obj_fns, daemon=True, autostart=True)[source]

Bases: object

A generic wrapper which takes a list of functions to be run inside a process. Each function must return an object, see target_fn for how it is used. Communication between each new process and the parent process happens via Pipes.

exec_remote(fn_string, proc_list=None, args_list=None, kwargs_list=None)[source]
start()[source]
stop()[source]
torchrl.utils.multi_proc_wrapper.target_fn(conn, obj_fn)[source]

torchrl.utils.nop module

class torchrl.utils.nop.Nop[source]

Bases: object

A NOP class. Give it anything.

nop(*args, **kwargs)[source]

torchrl.utils.schedule module

class torchrl.utils.schedule.ExpDecaySchedule(start=1.0, end=0.01, num_steps=1000)[source]

Bases: torchrl.utils.schedule.Schedule

step()[source]

Step the wrapped_value

wrapped_value

Access wrapped value without side-effect

class torchrl.utils.schedule.LinearSchedule(min_val=0.0, max_val=1.0, num_steps=1000, invert=False)[source]

Bases: torchrl.utils.schedule.Schedule

step()[source]

Step the wrapped_value

wrapped_value

Access wrapped value without side-effect

class torchrl.utils.schedule.Schedule[source]

Bases: object

step()[source]

Step the wrapped_value

value

Step schedule and return the old value

wrapped_value

Access wrapped value without side-effect

torchrl.utils.sum_tree module

class torchrl.utils.sum_tree.SumTree(capacity: int = 16)[source]

Bases: object

Implements a Sum Tree data structure.

A Sum Tree data structure is a binary tree with leaf nodes containing data and internal nodes containing sum of the tree rooted at that node. The binary tree here is represented by an array.

add(value)[source]
clear()[source]
get(value)[source]
max_value
next_free_index
sum_value
update(index: int, value)[source]

Module contents