Multi-Agent Environments
Two multiagent environments are given in the package:
GeneralSatelliteTasking, a Gymnasium-based environment and the basis for all other environments.
ConstellationTasking, which implements the PettingZoo parallel API.
The latter is preferable for multi-agent RL (MARL) settings, as most algorithms are designed for this kind of API.
Configuring the Environment
For this example, a multisatellite target imaging environment will be used. The goal is to maximize the value of unique images taken.
As usual, the satellite type is defined first.
[1]:
from typing import ClassVar
from bsk_rl import act, comm, data, obs, sats, scene
from bsk_rl.sim import dyn, fsw
class ImagingSatellite(sats.ImagingSatellite):
observation_spec: ClassVar[list[obs.Observation]] = [
obs.OpportunityProperties(
dict(prop="priority"),
dict(prop="opportunity_open", norm=5700.0),
n_ahead_observe=10,
)
]
action_spec: ClassVar[list[act.Action]] = [act.Image(n_ahead_image=10)]
dyn_type = dyn.FullFeaturedDynModel
fsw_type = fsw.SteeringImagerFSWModel
Satellite properties are set to give the satellite near-unlimited power and storage resources. To randomize some parameters in a correlated manner across satellites, a sat_arg_randomizer is set and passed to the environment. In this case, the satellites are distributed in a trivial single-plane Walker-delta constellation.
[2]:
from bsk_rl.utils.orbital import walker_delta_args
sat_args = dict(
imageAttErrorRequirement=0.01,
imageRateErrorRequirement=0.01,
batteryStorageCapacity=1e9,
storedCharge_Init=1e9,
dataStorageCapacity=1e12,
u_max=0.4,
K1=0.25,
K3=3.0,
omega_max=0.087,
servo_Ki=5.0,
servo_P=150 / 5,
)
sat_arg_randomizer = walker_delta_args(altitude=800.0, inc=60.0, n_planes=1)
Gym API
GeneralSatelliteTasking uses tuples of actions and observations to interact with the environment.
[3]:
from bsk_rl import GeneralSatelliteTasking
env = GeneralSatelliteTasking(
satellites=[
ImagingSatellite("EO_1", sat_args),
ImagingSatellite("EO_2", sat_args),
ImagingSatellite("EO_3", sat_args),
],
scenario=scene.UniformTargets(1000),
rewarder=data.UniqueImageReward(),
communicator=comm.LOSCommunication(), # Note that dyn must inherit from LOSCommunication
sat_arg_randomizer=sat_arg_randomizer,
log_level="INFO",
)
env.reset()
env.observation_space
2026-10-21 00:25:28,580 gym INFO Resetting environment with seed=602585588
2026-10-21 00:25:28,583 scene.targets INFO Generating 1000 targets
2026-10-21 00:25:28,658 sats.satellite.EO_1 INFO <0.00> EO_1: Finding opportunity windows from 0.00 to 600.00 seconds
2026-10-21 00:25:28,701 sats.satellite.EO_2 INFO <0.00> EO_2: Finding opportunity windows from 0.00 to 600.00 seconds
2026-10-21 00:25:28,743 sats.satellite.EO_3 INFO <0.00> EO_3: Finding opportunity windows from 0.00 to 600.00 seconds
2026-10-21 00:25:28,787 gym INFO <0.00> Environment reset
[3]:
Tuple(Box(-1e+16, 1e+16, (20,), float64), Box(-1e+16, 1e+16, (20,), float64), Box(-1e+16, 1e+16, (20,), float64))
[4]:
env.action_space
[4]:
Tuple(Discrete(10), Discrete(10), Discrete(10))
Consequently, actions are passed as a tuple. The step will stop the first time any satellite completes an action.
[5]:
observation, reward, terminated, truncated, info = env.step([7, 9, 8])
2026-10-21 00:25:28,802 gym INFO <0.00> === STARTING STEP ===
2026-10-21 00:25:28,803 sats.satellite.EO_1 INFO <0.00> EO_1: target index 7 tasked
2026-10-21 00:25:28,804 sats.satellite.EO_1 INFO <0.00> EO_1: Target(tgt-16) tasked for imaging
2026-10-21 00:25:28,805 sats.satellite.EO_1 INFO <0.00> EO_1: Target(tgt-16) window enabled: 395.9 to 599.6
2026-10-21 00:25:28,805 sats.satellite.EO_1 INFO <0.00> EO_1: setting timed terminal event at 599.6
2026-10-21 00:25:28,807 sats.satellite.EO_2 INFO <0.00> EO_2: target index 9 tasked
2026-10-21 00:25:28,808 sats.satellite.EO_2 INFO <0.00> EO_2: Target(tgt-559) tasked for imaging
2026-10-21 00:25:28,809 sats.satellite.EO_2 INFO <0.00> EO_2: Target(tgt-559) window enabled: 210.6 to 418.3
2026-10-21 00:25:28,809 sats.satellite.EO_2 INFO <0.00> EO_2: setting timed terminal event at 418.3
2026-10-21 00:25:28,811 sats.satellite.EO_3 INFO <0.00> EO_3: target index 8 tasked
2026-10-21 00:25:28,811 sats.satellite.EO_3 INFO <0.00> EO_3: Target(tgt-869) tasked for imaging
2026-10-21 00:25:28,812 sats.satellite.EO_3 INFO <0.00> EO_3: Target(tgt-869) window enabled: 128.8 to 310.5
2026-10-21 00:25:28,813 sats.satellite.EO_3 INFO <0.00> EO_3: setting timed terminal event at 310.5
2026-10-21 00:25:28,847 sats.satellite.EO_3 INFO <131.00> EO_3: imaged Target(tgt-869)
2026-10-21 00:25:28,849 data.base INFO <131.00> Total reward: {'EO_3': 0.27045703989264347}
2026-10-21 00:25:28,850 sats.satellite.EO_3 INFO <131.00> EO_3: Satellite EO_3 requires retasking
2026-10-21 00:25:28,850 sats.satellite.EO_1 INFO <131.00> EO_1: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-10-21 00:25:28,892 sats.satellite.EO_3 INFO <131.00> EO_3: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-10-21 00:25:28,932 gym INFO <131.00> Step reward: 0.27045703989264347
[6]:
observation
[6]:
(array([ 0.8940089 , -0.00701555, 0.23873156, 0.00678615, 0.57203609,
0.01372953, 0.96278808, 0.04827396, 0.54374278, 0.04445407,
0.91384117, 0.04648001, 0.09529316, 0.06651036, 0.89237846,
0.06269924, 0.29809928, 0.06382673, 0.52632214, 0.09272779]),
array([ 0.86909793, -0.0132896 , 0.69642123, -0.01828058, 0.87037131,
0.0031345 , 0.08277049, 0.02077608, 0.35528928, 0.0139607 ,
0.15242718, 0.04278509, 0.80882893, 0.05097881, 0.35338252,
0.07668309, 0.62508577, 0.07791942, 0.44658013, 0.06593744]),
array([ 0.15416228, -0.02298246, 0.11870998, -0.00950178, 0.5772644 ,
0.01008369, 0.93110649, 0.02956784, 0.34331456, 0.03810418,
0.48480952, 0.0364297 , 0.91454342, 0.06308395, 0.26584297,
0.07454483, 0.04433428, 0.09919842, 0.96158203, 0.07721853]))
At this point, either every satellite can be retasked, or satellites can continue their previous action by passing None as the action. To see which satellites must be retasked (i.e. their previous action is done and they have nothing more to do), look at "requires_retasking" in each satellite’s info.
[7]:
info
[7]:
{'EO_1': {'requires_retasking': False},
'EO_2': {'requires_retasking': False},
'EO_3': {'requires_retasking': True},
'd_ts': 131.0}
Based on this list, we decide here to only retask the satellite that needs it.
[8]:
actions = [
0 if info[sat.name]["requires_retasking"] else None
for sat in env.unwrapped.satellites
]
actions
[8]:
[None, None, 0]
[9]:
observation, reward, terminated, truncated, info = env.step(actions)
2026-10-21 00:25:28,956 gym INFO <131.00> === STARTING STEP ===
2026-10-21 00:25:28,958 sats.satellite.EO_3 INFO <131.00> EO_3: target index 0 tasked
2026-10-21 00:25:28,958 sats.satellite.EO_3 INFO <131.00> EO_3: Target(tgt-112) tasked for imaging
2026-10-21 00:25:28,959 sats.satellite.EO_3 INFO <131.00> EO_3: Target(tgt-112) window enabled: 0.0 to 184.6
2026-10-21 00:25:28,960 sats.satellite.EO_3 INFO <131.00> EO_3: setting timed terminal event at 184.6
2026-10-21 00:25:28,976 sats.satellite.EO_3 INFO <185.00> EO_3: timed termination at 184.6 for Target(tgt-112) window
2026-10-21 00:25:28,977 data.base INFO <185.00> Total reward: {}
2026-10-21 00:25:28,978 sats.satellite.EO_3 INFO <185.00> EO_3: Satellite EO_3 requires retasking
2026-10-21 00:25:28,981 gym INFO <185.00> Step reward: 0.0
In this environment, the environment will stop if any agent dies. To demonstrate this, one satellite is forcibly killed.
[10]:
from Basilisk.architecture import messaging
def isnt_alive(log_failure=False):
"""Mock satellite 0 dying."""
self = env.unwrapped.satellites[0]
death_message = messaging.PowerStorageStatusMsgPayload()
death_message.storageLevel = 0.0
self.dynamics.powerMonitor.batPowerOutMsg.write(death_message)
return self.dynamics.is_alive(log_failure=log_failure) and self.fsw.is_alive(
log_failure=log_failure
)
env.unwrapped.satellites[0].is_alive = isnt_alive
observation, reward, terminated, truncated, info = env.step([6, 7, 9])
2026-10-21 00:25:28,988 gym INFO <185.00> === STARTING STEP ===
2026-10-21 00:25:28,989 sats.satellite.EO_1 INFO <185.00> EO_1: target index 6 tasked
2026-10-21 00:25:28,989 sats.satellite.EO_1 INFO <185.00> EO_1: Target(tgt-53) tasked for imaging
2026-10-21 00:25:28,990 sats.satellite.EO_1 INFO <185.00> EO_1: Target(tgt-53) window enabled: 510.1 to 666.2
2026-10-21 00:25:28,991 sats.satellite.EO_1 INFO <185.00> EO_1: setting timed terminal event at 666.2
2026-10-21 00:25:28,992 sats.satellite.EO_2 INFO <185.00> EO_2: target index 7 tasked
2026-10-21 00:25:28,992 sats.satellite.EO_2 INFO <185.00> EO_2: Target(tgt-147) tasked for imaging
2026-10-21 00:25:28,993 sats.satellite.EO_2 INFO <185.00> EO_2: Target(tgt-147) window enabled: 575.1 to 600.0
2026-10-21 00:25:28,994 sats.satellite.EO_2 INFO <185.00> EO_2: setting timed terminal event at 600.0
2026-10-21 00:25:28,995 sats.satellite.EO_3 INFO <185.00> EO_3: target index 9 tasked
2026-10-21 00:25:28,995 sats.satellite.EO_3 INFO <185.00> EO_3: Target(tgt-367) tasked for imaging
2026-10-21 00:25:28,997 sats.satellite.EO_3 INFO <185.00> EO_3: Target(tgt-367) window enabled: 628.7 to 804.5
2026-10-21 00:25:28,997 sats.satellite.EO_3 INFO <185.00> EO_3: setting timed terminal event at 804.5
2026-10-21 00:25:29,084 sats.satellite.EO_1 INFO <513.00> EO_1: imaged Target(tgt-53)
2026-10-21 00:25:29,085 data.base INFO <513.00> Total reward: {'EO_1': 0.09529316123085063}
2026-10-21 00:25:29,086 sats.satellite.EO_1 INFO <513.00> EO_1: Satellite EO_1 requires retasking
2026-10-21 00:25:29,088 sats.satellite.EO_1 WARNING <513.00> EO_1: failed battery_valid check
2026-10-21 00:25:29,090 gym INFO <513.00> Step reward: -0.9047068387691494
2026-10-21 00:25:29,090 gym INFO <513.00> Episode terminated: True
2026-10-21 00:25:29,091 gym INFO <513.00> Episode truncated: False
PettingZoo API
The PettingZoo parallel API environment, ConstellationTasking, is largely the same as GeneralSatelliteTasking. See their documentation for a full description of the API. It tends to separate things into dictionaries keyed by agent, rather than tuples.
[11]:
from bsk_rl import ConstellationTasking
env = ConstellationTasking(
satellites=[
ImagingSatellite("EO_1", sat_args),
ImagingSatellite("EO_2", sat_args),
ImagingSatellite("EO_3", sat_args),
],
scenario=scene.UniformTargets(1000),
rewarder=data.UniqueImageReward(),
communicator=comm.LOSCommunication(), # Note that dyn must inherit from LOSCommunication
sat_arg_randomizer=sat_arg_randomizer,
log_level="INFO",
)
env.reset()
env.observation_spaces
2026-10-21 00:25:29,099 WARNING Creating logger for new env on PID=4630. Old environments in process may now log times incorrectly.
2026-10-21 00:25:29,102 gym INFO Resetting environment with seed=1124061386
2026-10-21 00:25:29,104 scene.targets INFO Generating 1000 targets
2026-10-21 00:25:29,156 sats.satellite.EO_1 INFO <0.00> EO_1: Finding opportunity windows from 0.00 to 600.00 seconds
2026-10-21 00:25:29,200 sats.satellite.EO_1 INFO <0.00> EO_1: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-10-21 00:25:29,242 sats.satellite.EO_2 INFO <0.00> EO_2: Finding opportunity windows from 0.00 to 600.00 seconds
2026-10-21 00:25:29,283 sats.satellite.EO_3 INFO <0.00> EO_3: Finding opportunity windows from 0.00 to 600.00 seconds
2026-10-21 00:25:29,331 gym INFO <0.00> Environment reset
[11]:
{'EO_1': Box(-1e+16, 1e+16, (20,), float64),
'EO_2': Box(-1e+16, 1e+16, (20,), float64),
'EO_3': Box(-1e+16, 1e+16, (20,), float64)}
[12]:
env.action_spaces
[12]:
{'EO_1': Discrete(10), 'EO_2': Discrete(10), 'EO_3': Discrete(10)}
Actions are passed as a dictionary; the agent names can be accessed through the agents property.
[13]:
observation, reward, terminated, truncated, info = env.step(
{
env.agents[0]: 7,
env.agents[1]: 9,
env.agents[2]: 8,
}
)
2026-10-21 00:25:29,345 gym INFO <0.00> === STARTING STEP ===
2026-10-21 00:25:29,346 sats.satellite.EO_1 INFO <0.00> EO_1: target index 7 tasked
2026-10-21 00:25:29,347 sats.satellite.EO_1 INFO <0.00> EO_1: Target(tgt-144) tasked for imaging
2026-10-21 00:25:29,348 sats.satellite.EO_1 INFO <0.00> EO_1: Target(tgt-144) window enabled: 439.9 to 614.7
2026-10-21 00:25:29,349 sats.satellite.EO_1 INFO <0.00> EO_1: setting timed terminal event at 614.7
2026-10-21 00:25:29,350 sats.satellite.EO_2 INFO <0.00> EO_2: target index 9 tasked
2026-10-21 00:25:29,350 sats.satellite.EO_2 INFO <0.00> EO_2: Target(tgt-874) tasked for imaging
2026-10-21 00:25:29,352 sats.satellite.EO_2 INFO <0.00> EO_2: Target(tgt-874) window enabled: 198.0 to 407.8
2026-10-21 00:25:29,353 sats.satellite.EO_2 INFO <0.00> EO_2: setting timed terminal event at 407.8
2026-10-21 00:25:29,353 sats.satellite.EO_3 INFO <0.00> EO_3: target index 8 tasked
2026-10-21 00:25:29,354 sats.satellite.EO_3 INFO <0.00> EO_3: Target(tgt-571) tasked for imaging
2026-10-21 00:25:29,355 sats.satellite.EO_3 INFO <0.00> EO_3: Target(tgt-571) window enabled: 452.8 to 600.0
2026-10-21 00:25:29,355 sats.satellite.EO_3 INFO <0.00> EO_3: setting timed terminal event at 600.0
2026-10-21 00:25:29,406 sats.satellite.EO_2 INFO <201.00> EO_2: imaged Target(tgt-874)
2026-10-21 00:25:29,407 data.base INFO <201.00> Total reward: {'EO_2': 0.7886105411095861}
2026-10-21 00:25:29,408 sats.satellite.EO_2 INFO <201.00> EO_2: Satellite EO_2 requires retasking
2026-10-21 00:25:29,411 sats.satellite.EO_3 INFO <201.00> EO_3: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-10-21 00:25:29,456 gym INFO <201.00> Step reward: {'EO_2': 0.7886105411095861}
[14]:
observation
[14]:
{'EO_1': array([0.39930841, 0.00279877, 0.41754475, 0.02420564, 0.15893471,
0.04191728, 0.52463356, 0.07099838, 0.46693405, 0.09836153,
0.59288132, 0.12332311, 0.58077539, 0.1014984 , 0.52791724,
0.14447099, 0.58144194, 0.14121156, 0.87716701, 0.13965945]),
'EO_2': array([ 0.43492891, -0.03070845, 0.2054845 , -0.02652328, 0.91685564,
-0.02016505, 0.03049002, -0.01573901, 0.00750496, -0.01304151,
0.84308463, -0.01287091, 0.64187453, -0.0061946 , 0.49510098,
0.00586469, 0.2709099 , 0.06915823, 0.87697463, 0.03922305]),
'EO_3': array([ 0.30373095, -0.03526316, 0.68571603, -0.02396368, 0.88870479,
-0.00303529, 0.82648424, -0.01221933, 0.72934507, 0.04499284,
0.42800512, 0.04417563, 0.01494808, 0.07062578, 0.22595312,
0.05140626, 0.24791442, 0.0628036 , 0.46770611, 0.08372981])}
Other than compatibility with MARL algorithms, the main benefit of the PettingZoo API is that it allows for individual agents to fail without terminating the entire environment.
[15]:
# Immediately kill satellite 0
env.unwrapped.satellites[0].is_alive = isnt_alive
env.agents
[15]:
['EO_1', 'EO_2', 'EO_3']
[16]:
observation, reward, terminated, truncated, info = env.step(
{
env.agents[0]: 7,
env.agents[1]: 9,
}
)
2026-10-21 00:25:29,473 gym INFO <201.00> === STARTING STEP ===
2026-10-21 00:25:29,474 sats.satellite.EO_1 INFO <201.00> EO_1: target index 7 tasked
2026-10-21 00:25:29,474 sats.satellite.EO_1 INFO <201.00> EO_1: Target(tgt-970) tasked for imaging
2026-10-21 00:25:29,475 sats.satellite.EO_1 INFO <201.00> EO_1: Target(tgt-970) window enabled: 1024.5 to 1080.3
2026-10-21 00:25:29,476 sats.satellite.EO_1 INFO <201.00> EO_1: setting timed terminal event at 1080.3
2026-10-21 00:25:29,477 sats.satellite.EO_2 INFO <201.00> EO_2: target index 9 tasked
2026-10-21 00:25:29,478 sats.satellite.EO_2 INFO <201.00> EO_2: Target(tgt-294) tasked for imaging
2026-10-21 00:25:29,479 sats.satellite.EO_2 INFO <201.00> EO_2: Target(tgt-294) window enabled: 424.6 to 600.0
2026-10-21 00:25:29,479 sats.satellite.EO_2 INFO <201.00> EO_2: setting timed terminal event at 600.0
2026-10-21 00:25:29,536 sats.satellite.EO_2 INFO <427.00> EO_2: imaged Target(tgt-294)
2026-10-21 00:25:29,537 data.base INFO <427.00> Total reward: {'EO_2': 0.8769746338089947}
2026-10-21 00:25:29,538 sats.satellite.EO_2 INFO <427.00> EO_2: Satellite EO_2 requires retasking
2026-10-21 00:25:29,540 sats.satellite.EO_2 INFO <427.00> EO_2: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-10-21 00:25:29,586 gym INFO <427.00> Step reward: {'EO_1': -1.0, 'EO_2': 0.8769746338089947}
2026-10-21 00:25:29,586 gym INFO <427.00> Episode terminated: ['EO_1']