Core Concepts

This document is a glossary for core concepts of TorchRL framework.

Agent

BaseAgent is an abstract class which defines the learning agent in the given Environment.

Environment

Environment is the system which provides feedback to the Agent. Currently, Open AI gym.Env environments are being used. The system is flexible enough to extend to any other environment kind.

Problem

Any task is defined by extending the abstract class Problem. A problem’s entrypoint is run() which generates the trajectory rollout and call’s the Agent’s learn() method with appropriate rollout information.

Hyper-Parameter Set

A HParams set is a class of arbitrary key-value pairs that contain the hyper-parameters for the problem. Keeping these as first-class objects in the code base allow for easily reproducible experiments.

Runner

A MultiEpisodeRunner takes in a method which returns a constructed environment and creates multiple subprocess copies for parallel trajectory rollouts via the collect() method. Each Problem internally creates a MultiEpisodeRunner and executes the collection process.