Multi-Agent Environments
Two multiagent environments are given in the package:
GeneralSatelliteTasking, a Gymnasium-based environment and the basis for all other environments.
ConstellationTasking, which implements the PettingZoo parallel API.
The latter is preferable for multi-agent RL (MARL) settings, as most algorithms are designed for this kind of API.
Configuring the Environment
For this example, a multisatellite target imaging environment will be used. The goal is to maximize the value of unique images taken.
As usual, the satellite type is defined first.
[1]:
from typing import ClassVar
from bsk_rl import act, comm, data, obs, sats, scene
from bsk_rl.sim import dyn, fsw
class ImagingSatellite(sats.ImagingSatellite):
observation_spec: ClassVar[list[obs.Observation]] = [
obs.OpportunityProperties(
dict(prop="priority"),
dict(prop="opportunity_open", norm=5700.0),
n_ahead_observe=10,
)
]
action_spec: ClassVar[list[act.Action]] = [act.Image(n_ahead_image=10)]
dyn_type = dyn.FullFeaturedDynModel
fsw_type = fsw.SteeringImagerFSWModel
Satellite properties are set to give the satellite near-unlimited power and storage resources. To randomize some parameters in a correlated manner across satellites, a sat_arg_randomizer is set and passed to the environment. In this case, the satellites are distributed in a trivial single-plane Walker-delta constellation.
[2]:
from bsk_rl.utils.orbital import walker_delta_args
sat_args = dict(
imageAttErrorRequirement=0.01,
imageRateErrorRequirement=0.01,
batteryStorageCapacity=1e9,
storedCharge_Init=1e9,
dataStorageCapacity=1e12,
u_max=0.4,
K1=0.25,
K3=3.0,
omega_max=0.087,
servo_Ki=5.0,
servo_P=150 / 5,
)
sat_arg_randomizer = walker_delta_args(altitude=800.0, inc=60.0, n_planes=1)
Gym API
GeneralSatelliteTasking uses tuples of actions and observations to interact with the environment.
[3]:
from bsk_rl import GeneralSatelliteTasking
env = GeneralSatelliteTasking(
satellites=[
ImagingSatellite("EO-1", sat_args),
ImagingSatellite("EO-2", sat_args),
ImagingSatellite("EO-3", sat_args),
],
scenario=scene.UniformTargets(1000),
rewarder=data.UniqueImageReward(),
communicator=comm.LOSCommunication(), # Note that dyn must inherit from LOSCommunication
sat_arg_randomizer=sat_arg_randomizer,
log_level="INFO",
)
env.reset()
env.observation_space
2026-09-02 14:52:19,249 gym INFO Resetting environment with seed=20012762
2026-09-02 14:52:19,251 scene.targets INFO Generating 1000 targets
2026-09-02 14:52:19,316 sats.satellite.EO-1 INFO <0.00> EO-1: Finding opportunity windows from 0.00 to 600.00 seconds
2026-09-02 14:52:19,344 sats.satellite.EO-1 INFO <0.00> EO-1: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-09-02 14:52:19,370 sats.satellite.EO-2 INFO <0.00> EO-2: Finding opportunity windows from 0.00 to 600.00 seconds
2026-09-02 14:52:19,400 sats.satellite.EO-3 INFO <0.00> EO-3: Finding opportunity windows from 0.00 to 600.00 seconds
2026-09-02 14:52:19,429 gym INFO <0.00> Environment reset
[3]:
Tuple(Box(-1e+16, 1e+16, (20,), float64), Box(-1e+16, 1e+16, (20,), float64), Box(-1e+16, 1e+16, (20,), float64))
[4]:
env.action_space
[4]:
Tuple(Discrete(10), Discrete(10), Discrete(10))
Consequently, actions are passed as a tuple. The step will stop the first time any satellite completes an action.
[5]:
observation, reward, terminated, truncated, info = env.step([7, 9, 8])
2026-09-02 14:52:19,442 gym INFO <0.00> === STARTING STEP ===
2026-09-02 14:52:19,443 sats.satellite.EO-1 INFO <0.00> EO-1: target index 7 tasked
2026-09-02 14:52:19,444 sats.satellite.EO-1 INFO <0.00> EO-1: Target(tgt-434) tasked for imaging
2026-09-02 14:52:19,445 sats.satellite.EO-1 INFO <0.00> EO-1: Target(tgt-434) window enabled: 582.7 to 728.4
2026-09-02 14:52:19,445 sats.satellite.EO-1 INFO <0.00> EO-1: setting timed terminal event at 728.4
2026-09-02 14:52:19,446 sats.satellite.EO-2 INFO <0.00> EO-2: target index 9 tasked
2026-09-02 14:52:19,447 sats.satellite.EO-2 INFO <0.00> EO-2: Target(tgt-648) tasked for imaging
2026-09-02 14:52:19,448 sats.satellite.EO-2 INFO <0.00> EO-2: Target(tgt-648) window enabled: 455.3 to 600.0
2026-09-02 14:52:19,448 sats.satellite.EO-2 INFO <0.00> EO-2: setting timed terminal event at 600.0
2026-09-02 14:52:19,449 sats.satellite.EO-3 INFO <0.00> EO-3: target index 8 tasked
2026-09-02 14:52:19,449 sats.satellite.EO-3 INFO <0.00> EO-3: Target(tgt-406) tasked for imaging
2026-09-02 14:52:19,450 sats.satellite.EO-3 INFO <0.00> EO-3: Target(tgt-406) window enabled: 591.9 to 600.0
2026-09-02 14:52:19,450 sats.satellite.EO-3 INFO <0.00> EO-3: setting timed terminal event at 600.0
2026-09-02 14:52:19,558 sats.satellite.EO-2 INFO <458.00> EO-2: imaged Target(tgt-648)
2026-09-02 14:52:19,559 data.base INFO <458.00> Total reward: {'EO-2': 0.4059009590411402}
2026-09-02 14:52:19,560 sats.satellite.EO-2 INFO <458.00> EO-2: Satellite EO-2 requires retasking
2026-09-02 14:52:19,560 sats.satellite.EO-1 INFO <458.00> EO-1: Finding opportunity windows from 1200.00 to 1800.00 seconds
2026-09-02 14:52:19,597 sats.access_satellite WARNING <458.00> initial_generation_duration is shorter than the maximum window length; some windows may be neglected.
2026-09-02 14:52:19,598 sats.satellite.EO-2 INFO <458.00> EO-2: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-09-02 14:52:19,627 sats.satellite.EO-3 INFO <458.00> EO-3: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-09-02 14:52:19,668 gym INFO <458.00> Step reward: 0.4059009590411402
[6]:
observation
[6]:
(array([ 0.08389423, -0.00195657, 0.48904243, 0.00809647, 0.47668937,
0.00354655, 0.18146897, 0.00827156, 0.98276213, 0.0334346 ,
0.06155021, 0.02187101, 0.48887743, 0.03613643, 0.67097138,
0.04890894, 0.76293069, 0.15682642, 0.84941348, 0.15777831]),
array([ 0.33218314, -0.02195802, 0.85141495, -0.02041991, 0.82451313,
-0.01834931, 0.39691044, 0.01210964, 0.29632816, 0.01404744,
0.16463084, 0.01767325, 0.82806094, 0.02241321, 0.2255959 ,
0.06441106, 0.78520319, 0.07507394, 0.01819296, 0.07801373]),
array([ 0.28958045, -0.01199367, 0.78604207, 0.01573225, 0.0185567 ,
0.02853442, 0.7534716 , 0.01985243, 0.81722567, 0.02349048,
0.83212766, 0.03541335, 0.16297575, 0.04365021, 0.1332045 ,
0.05939289, 0.10342463, 0.07936322, 0.68878215, 0.09488842]))
At this point, either every satellite can be retasked, or satellites can continue their previous action by passing None as the action. To see which satellites must be retasked (i.e. their previous action is done and they have nothing more to do), look at "requires_retasking" in each satellite’s info.
[7]:
info
[7]:
{'EO-1': {'requires_retasking': False},
'EO-2': {'requires_retasking': True},
'EO-3': {'requires_retasking': False},
'd_ts': 458.00000000000006}
Based on this list, we decide here to only retask the satellite that needs it.
[8]:
actions = [0 if info[sat.name]["requires_retasking"] else None for sat in env.unwrapped.satellites]
actions
[8]:
[None, 0, None]
[9]:
observation, reward, terminated, truncated, info = env.step(actions)
2026-09-02 14:52:19,688 gym INFO <458.00> === STARTING STEP ===
2026-09-02 14:52:19,689 sats.satellite.EO-2 INFO <458.00> EO-2: target index 0 tasked
2026-09-02 14:52:19,690 sats.satellite.EO-2 INFO <458.00> EO-2: Target(tgt-288) tasked for imaging
2026-09-02 14:52:19,690 sats.satellite.EO-2 INFO <458.00> EO-2: Target(tgt-288) window enabled: 332.8 to 473.3
2026-09-02 14:52:19,691 sats.satellite.EO-2 INFO <458.00> EO-2: setting timed terminal event at 473.3
2026-09-02 14:52:19,697 sats.satellite.EO-2 INFO <474.00> EO-2: timed termination at 473.3 for Target(tgt-288) window
2026-09-02 14:52:19,698 data.base INFO <474.00> Total reward: {}
2026-09-02 14:52:19,698 sats.satellite.EO-2 INFO <474.00> EO-2: Satellite EO-2 requires retasking
2026-09-02 14:52:19,701 gym INFO <474.00> Step reward: 0.0
In this environment, the environment will stop if any agent dies. To demonstrate this, one satellite is forcibly killed.
[10]:
from Basilisk.architecture import messaging
def isnt_alive(log_failure=False):
"""Mock satellite 0 dying."""
self = env.unwrapped.satellites[0]
death_message = messaging.PowerStorageStatusMsgPayload()
death_message.storageLevel = 0.0
self.dynamics.powerMonitor.batPowerOutMsg.write(death_message)
return self.dynamics.is_alive(log_failure=log_failure) and self.fsw.is_alive(
log_failure=log_failure
)
env.unwrapped.satellites[0].is_alive = isnt_alive
observation, reward, terminated, truncated, info = env.step([6, 7, 9])
2026-09-02 14:52:19,707 gym INFO <474.00> === STARTING STEP ===
2026-09-02 14:52:19,707 sats.satellite.EO-1 INFO <474.00> EO-1: target index 6 tasked
2026-09-02 14:52:19,708 sats.satellite.EO-1 INFO <474.00> EO-1: Target(tgt-662) tasked for imaging
2026-09-02 14:52:19,709 sats.satellite.EO-1 INFO <474.00> EO-1: Target(tgt-662) window enabled: 664.0 to 867.9
2026-09-02 14:52:19,709 sats.satellite.EO-1 INFO <474.00> EO-1: setting timed terminal event at 867.9
2026-09-02 14:52:19,710 sats.satellite.EO-2 INFO <474.00> EO-2: target index 7 tasked
2026-09-02 14:52:19,710 sats.satellite.EO-2 INFO <474.00> EO-2: Target(tgt-718) tasked for imaging
2026-09-02 14:52:19,711 sats.satellite.EO-2 INFO <474.00> EO-2: Target(tgt-718) window enabled: 902.7 to 1110.0
2026-09-02 14:52:19,711 sats.satellite.EO-2 INFO <474.00> EO-2: setting timed terminal event at 1110.0
2026-09-02 14:52:19,713 sats.satellite.EO-3 INFO <474.00> EO-3: target index 9 tasked
2026-09-02 14:52:19,713 sats.satellite.EO-3 INFO <474.00> EO-3: Target(tgt-107) tasked for imaging
2026-09-02 14:52:19,714 sats.satellite.EO-3 INFO <474.00> EO-3: Target(tgt-107) window enabled: 998.9 to 1149.2
2026-09-02 14:52:19,714 sats.satellite.EO-3 INFO <474.00> EO-3: setting timed terminal event at 1149.2
2026-09-02 14:52:19,761 sats.satellite.EO-1 INFO <666.00> EO-1: imaged Target(tgt-662)
2026-09-02 14:52:19,762 data.base INFO <666.00> Total reward: {'EO-1': 0.48887742657016553}
2026-09-02 14:52:19,763 sats.satellite.EO-1 INFO <666.00> EO-1: Satellite EO-1 requires retasking
2026-09-02 14:52:19,764 sats.satellite.EO-1 WARNING <666.00> EO-1: failed battery_valid check
2026-09-02 14:52:19,766 gym INFO <666.00> Step reward: -0.5111225734298345
2026-09-02 14:52:19,766 gym INFO <666.00> Episode terminated: True
2026-09-02 14:52:19,767 gym INFO <666.00> Episode truncated: False
PettingZoo API
The PettingZoo parallel API environment, ConstellationTasking, is largely the same as GeneralSatelliteTasking. See their documentation for a full description of the API. It tends to separate things into dictionaries keyed by agent, rather than tuples.
[11]:
from bsk_rl import ConstellationTasking
env = ConstellationTasking(
satellites=[
ImagingSatellite("EO-1", sat_args),
ImagingSatellite("EO-2", sat_args),
ImagingSatellite("EO-3", sat_args),
],
scenario=scene.UniformTargets(1000),
rewarder=data.UniqueImageReward(),
communicator=comm.LOSCommunication(), # Note that dyn must inherit from LOSCommunication
sat_arg_randomizer=sat_arg_randomizer,
log_level="INFO",
)
env.reset()
env.observation_spaces
2026-09-02 14:52:19,774 WARNING Creating logger for new env on PID=4475. Old environments in process may now log times incorrectly.
2026-09-02 14:52:19,776 gym INFO Resetting environment with seed=547872858
2026-09-02 14:52:19,778 scene.targets INFO Generating 1000 targets
2026-09-02 14:52:19,822 sats.satellite.EO-1 INFO <0.00> EO-1: Finding opportunity windows from 0.00 to 600.00 seconds
2026-09-02 14:52:19,856 sats.satellite.EO-2 INFO <0.00> EO-2: Finding opportunity windows from 0.00 to 600.00 seconds
2026-09-02 14:52:19,887 sats.satellite.EO-3 INFO <0.00> EO-3: Finding opportunity windows from 0.00 to 600.00 seconds
2026-09-02 14:52:19,921 gym INFO <0.00> Environment reset
[11]:
{'EO-1': Box(-1e+16, 1e+16, (20,), float64),
'EO-2': Box(-1e+16, 1e+16, (20,), float64),
'EO-3': Box(-1e+16, 1e+16, (20,), float64)}
[12]:
env.action_spaces
[12]:
{'EO-1': Discrete(10), 'EO-2': Discrete(10), 'EO-3': Discrete(10)}
Actions are passed as a dictionary; the agent names can be accessed through the agents property.
[13]:
observation, reward, terminated, truncated, info = env.step(
{
env.agents[0]: 7,
env.agents[1]: 9,
env.agents[2]: 8,
}
)
2026-09-02 14:52:19,933 gym INFO <0.00> === STARTING STEP ===
2026-09-02 14:52:19,933 sats.satellite.EO-1 INFO <0.00> EO-1: target index 7 tasked
2026-09-02 14:52:19,934 sats.satellite.EO-1 INFO <0.00> EO-1: Target(tgt-496) tasked for imaging
2026-09-02 14:52:19,935 sats.satellite.EO-1 INFO <0.00> EO-1: Target(tgt-496) window enabled: 257.6 to 412.0
2026-09-02 14:52:19,935 sats.satellite.EO-1 INFO <0.00> EO-1: setting timed terminal event at 412.0
2026-09-02 14:52:19,937 sats.satellite.EO-2 INFO <0.00> EO-2: target index 9 tasked
2026-09-02 14:52:19,937 sats.satellite.EO-2 INFO <0.00> EO-2: Target(tgt-243) tasked for imaging
2026-09-02 14:52:19,938 sats.satellite.EO-2 INFO <0.00> EO-2: Target(tgt-243) window enabled: 360.4 to 517.4
2026-09-02 14:52:19,938 sats.satellite.EO-2 INFO <0.00> EO-2: setting timed terminal event at 517.4
2026-09-02 14:52:19,939 sats.satellite.EO-3 INFO <0.00> EO-3: target index 8 tasked
2026-09-02 14:52:19,939 sats.satellite.EO-3 INFO <0.00> EO-3: Target(tgt-438) tasked for imaging
2026-09-02 14:52:19,940 sats.satellite.EO-3 INFO <0.00> EO-3: Target(tgt-438) window enabled: 319.3 to 512.1
2026-09-02 14:52:19,941 sats.satellite.EO-3 INFO <0.00> EO-3: setting timed terminal event at 512.1
2026-09-02 14:52:20,010 sats.satellite.EO-1 INFO <260.00> EO-1: imaged Target(tgt-496)
2026-09-02 14:52:20,011 data.base INFO <260.00> Total reward: {'EO-1': 0.22361688537972357}
2026-09-02 14:52:20,012 sats.satellite.EO-1 INFO <260.00> EO-1: Satellite EO-1 requires retasking
2026-09-02 14:52:20,013 sats.satellite.EO-1 INFO <260.00> EO-1: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-09-02 14:52:20,041 sats.satellite.EO-2 INFO <260.00> EO-2: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-09-02 14:52:20,073 sats.satellite.EO-3 INFO <260.00> EO-3: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-09-02 14:52:20,103 gym INFO <260.00> Step reward: {'EO-1': 0.22361688537972357}
[14]:
observation
[14]:
{'EO-1': array([ 0.07562008, -0.02598129, 0.22996627, -0.02785367, 0.07136504,
0.00755213, 0.05484768, -0.01710429, 0.10100791, 0.00502686,
0.31621494, 0.01607201, 0.1768264 , 0.03599422, 0.4398006 ,
0.04204823, 0.38719559, 0.07397254, 0.57964512, 0.06621403]),
'EO-2': array([ 0.53474044, -0.00426528, 0.95837205, -0.0146927 , 0.2414678 ,
0.02057079, 0.43278159, 0.01760692, 0.69691807, 0.05174167,
0.25368445, 0.05318077, 0.49330349, 0.05244175, 0.98047479,
0.05294192, 0.64491908, 0.06531905, 0.27988181, 0.06063213]),
'EO-3': array([ 0.69045643, -0.02567906, 0.19462184, -0.02835644, 0.30674601,
-0.00716302, 0.29828441, 0.01040045, 0.00473951, 0.02197465,
0.5373621 , 0.03081877, 0.14627486, 0.03598493, 0.65820259,
0.05734681, 0.63719485, 0.06634786, 0.05880865, 0.078448 ])}
Other than compatibility with MARL algorithms, the main benefit of the PettingZoo API is that it allows for individual agents to fail without terminating the entire environment.
[15]:
# Immediately kill satellite 0
env.unwrapped.satellites[0].is_alive = isnt_alive
env.agents
[15]:
['EO-1', 'EO-2', 'EO-3']
[16]:
observation, reward, terminated, truncated, info = env.step({
env.agents[0]: 7,
env.agents[1]: 9,
}
)
2026-09-02 14:52:20,119 gym INFO <260.00> === STARTING STEP ===
2026-09-02 14:52:20,119 sats.satellite.EO-1 INFO <260.00> EO-1: target index 7 tasked
2026-09-02 14:52:20,120 sats.satellite.EO-1 INFO <260.00> EO-1: Target(tgt-394) tasked for imaging
2026-09-02 14:52:20,120 sats.satellite.EO-1 INFO <260.00> EO-1: Target(tgt-394) window enabled: 499.7 to 703.5
2026-09-02 14:52:20,122 sats.satellite.EO-1 INFO <260.00> EO-1: setting timed terminal event at 703.5
2026-09-02 14:52:20,122 sats.satellite.EO-2 INFO <260.00> EO-2: target index 9 tasked
2026-09-02 14:52:20,123 sats.satellite.EO-2 INFO <260.00> EO-2: Target(tgt-887) tasked for imaging
2026-09-02 14:52:20,124 sats.satellite.EO-2 INFO <260.00> EO-2: Target(tgt-887) window enabled: 605.6 to 799.5
2026-09-02 14:52:20,124 sats.satellite.EO-2 INFO <260.00> EO-2: setting timed terminal event at 799.5
2026-09-02 14:52:20,139 sats.satellite.EO-3 INFO <322.00> EO-3: imaged Target(tgt-438)
2026-09-02 14:52:20,140 data.base INFO <322.00> Total reward: {'EO-3': 0.2982844056917848}
2026-09-02 14:52:20,141 sats.satellite.EO-3 INFO <322.00> EO-3: Satellite EO-3 requires retasking
2026-09-02 14:52:20,144 gym INFO <322.00> Step reward: {'EO-1': -1.0, 'EO-3': 0.2982844056917848}
2026-09-02 14:52:20,144 gym INFO <322.00> Episode terminated: ['EO-1']