RLlib Utilities

A collection of utilities at bsk_rl.utils.rllib.

• Semi-MDP Discounting in RLlib - For semi-MDP discounting with GAE.

• RLlib Callbacks - For logging data at the end of each episode.

Two environments are added to the ray.tune.registry with this import. They are "SatelliteTasking-RLlib" and "ConstellationTasking-RLlib". These environments are wrapped with the unpack_config() function to make them compatible with RLlib’s API, and they are wrapped with the EpisodeDataWrapper to allow for data logging at the end of each episode during training. These environments can be selected by name when setting config.environment(env="SatelliteTasking-RLlib"). Callback functions that are arguments to EpisodeDataWrapper can be set in the env_config dictionary.

unpack_config(env)

Create a wrapped version of an env class that unpacks env_config from Ray into kwargs.

Necessary when setting

config.environment(
    env=unpack_config(SatelliteTasking),
    env_config=env_args
)

which generates environments that look like

SatelliteTasking(**env_args)

since RLlib expects the environment to take a dictionary called kwargs instead of the actual arguments.

load_torch_mlp_policy(policy_path: Path, policy_name: str = 'default_agent') → Callable

Load a PyTorch policy from a saved model.

Parameters:

• policy_path (Path) – Path to the directory containing the policy checkpoint.

• policy_name (str) – Name of the policy to load from the checkpoint.

Return type:

Callable