Multi-Agent Environments
Two multiagent environments are given in the package:
GeneralSatelliteTasking, a Gymnasium-based environment and the basis for all other environments.
ConstellationTasking, which implements the PettingZoo parallel API.
The latter is preferable for multi-agent RL (MARL) settings, as most algorithms are designed for this kind of API.
Configuring the Environment
For this example, a multisatellite target imaging environment will be used. The goal is to maximize the value of unique images taken.
As usual, the satellite type is defined first.
[1]:
from typing import ClassVar
from bsk_rl import act, comm, data, obs, sats, scene
from bsk_rl.sim import dyn, fsw
class ImagingSatellite(sats.ImagingSatellite):
observation_spec: ClassVar[list[obs.Observation]] = [
obs.OpportunityProperties(
dict(prop="priority"),
dict(prop="opportunity_open", norm=5700.0),
n_ahead_observe=10,
)
]
action_spec: ClassVar[list[act.Action]] = [act.Image(n_ahead_image=10)]
dyn_type = dyn.FullFeaturedDynModel
fsw_type = fsw.SteeringImagerFSWModel
Satellite properties are set to give the satellite near-unlimited power and storage resources. To randomize some parameters in a correlated manner across satellites, a sat_arg_randomizer is set and passed to the environment. In this case, the satellites are distributed in a trivial single-plane Walker-delta constellation.
[2]:
from bsk_rl.utils.orbital import walker_delta_args
sat_args = dict(
imageAttErrorRequirement=0.01,
imageRateErrorRequirement=0.01,
batteryStorageCapacity=1e9,
storedCharge_Init=1e9,
dataStorageCapacity=1e12,
u_max=0.4,
K1=0.25,
K3=3.0,
omega_max=0.087,
servo_Ki=5.0,
servo_P=150 / 5,
)
sat_arg_randomizer = walker_delta_args(altitude=800.0, inc=60.0, n_planes=1)
Gym API
GeneralSatelliteTasking uses tuples of actions and observations to interact with the environment.
[3]:
from bsk_rl import GeneralSatelliteTasking
env = GeneralSatelliteTasking(
satellites=[
ImagingSatellite("EO-1", sat_args),
ImagingSatellite("EO-2", sat_args),
ImagingSatellite("EO-3", sat_args),
],
scenario=scene.UniformTargets(1000),
rewarder=data.UniqueImageReward(),
communicator=comm.LOSCommunication(), # Note that dyn must inherit from LOSCommunication
sat_arg_randomizer=sat_arg_randomizer,
log_level="INFO",
)
env.reset()
env.observation_space
2026-07-28 23:39:22,839 gym INFO Resetting environment with seed=2196937167
2026-07-28 23:39:22,841 scene.targets INFO Generating 1000 targets
2026-07-28 23:39:22,911 sats.satellite.EO-1 INFO <0.00> EO-1: Finding opportunity windows from 0.00 to 600.00 seconds
2026-07-28 23:39:22,932 sats.satellite.EO-1 INFO <0.00> EO-1: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-07-28 23:39:22,953 sats.satellite.EO-2 INFO <0.00> EO-2: Finding opportunity windows from 0.00 to 600.00 seconds
2026-07-28 23:39:22,975 sats.satellite.EO-3 INFO <0.00> EO-3: Finding opportunity windows from 0.00 to 600.00 seconds
2026-07-28 23:39:22,996 gym INFO <0.00> Environment reset
[3]:
Tuple(Box(-1e+16, 1e+16, (20,), float64), Box(-1e+16, 1e+16, (20,), float64), Box(-1e+16, 1e+16, (20,), float64))
[4]:
env.action_space
[4]:
Tuple(Discrete(10), Discrete(10), Discrete(10))
Consequently, actions are passed as a tuple. The step will stop the first time any satellite completes an action.
[5]:
observation, reward, terminated, truncated, info = env.step([7, 9, 8])
2026-07-28 23:39:23,008 gym INFO <0.00> === STARTING STEP ===
2026-07-28 23:39:23,008 sats.satellite.EO-1 INFO <0.00> EO-1: target index 7 tasked
2026-07-28 23:39:23,009 sats.satellite.EO-1 INFO <0.00> EO-1: Target(tgt-265) tasked for imaging
2026-07-28 23:39:23,009 sats.satellite.EO-1 INFO <0.00> EO-1: Target(tgt-265) window enabled: 629.7 to 814.6
2026-07-28 23:39:23,010 sats.satellite.EO-1 INFO <0.00> EO-1: setting timed terminal event at 814.6
2026-07-28 23:39:23,011 sats.satellite.EO-2 INFO <0.00> EO-2: target index 9 tasked
2026-07-28 23:39:23,012 sats.satellite.EO-2 INFO <0.00> EO-2: Target(tgt-996) tasked for imaging
2026-07-28 23:39:23,012 sats.satellite.EO-2 INFO <0.00> EO-2: Target(tgt-996) window enabled: 526.1 to 600.0
2026-07-28 23:39:23,013 sats.satellite.EO-2 INFO <0.00> EO-2: setting timed terminal event at 600.0
2026-07-28 23:39:23,014 sats.satellite.EO-3 INFO <0.00> EO-3: target index 8 tasked
2026-07-28 23:39:23,014 sats.satellite.EO-3 INFO <0.00> EO-3: Target(tgt-45) tasked for imaging
2026-07-28 23:39:23,015 sats.satellite.EO-3 INFO <0.00> EO-3: Target(tgt-45) window enabled: 399.6 to 542.6
2026-07-28 23:39:23,015 sats.satellite.EO-3 INFO <0.00> EO-3: setting timed terminal event at 542.6
2026-07-28 23:39:23,082 sats.satellite.EO-3 INFO <402.00> EO-3: imaged Target(tgt-45)
2026-07-28 23:39:23,082 data.base INFO <402.00> Total reward: {'EO-3': 0.4821728341531314}
2026-07-28 23:39:23,083 sats.satellite.EO-3 INFO <402.00> EO-3: Satellite EO-3 requires retasking
2026-07-28 23:39:23,084 sats.satellite.EO-2 INFO <402.00> EO-2: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-07-28 23:39:23,104 sats.satellite.EO-3 INFO <402.00> EO-3: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-07-28 23:39:23,127 gym INFO <402.00> Step reward: 0.4821728341531314
[6]:
observation
[6]:
(array([ 9.59173544e-01, -2.57064604e-02, 6.66005014e-01, -7.63423863e-04,
1.95688313e-01, 2.79292416e-02, 5.05761485e-01, 3.06905515e-02,
8.76436248e-01, 3.99556960e-02, 2.95654635e-01, 5.79785228e-02,
3.63347981e-01, 6.25178087e-02, 9.31281674e-01, 7.84246944e-02,
9.61370897e-01, 9.05452736e-02, 8.77855516e-01, 1.20450698e-01]),
array([ 0.68840307, -0.01320415, 0.51689527, 0.00552527, 0.81898587,
0.01178171, 0.53830038, 0.00622084, 0.49629636, 0.01701365,
0.71662814, 0.03617185, 0.83158196, 0.02177033, 0.91649269,
0.07700274, 0.13631026, 0.1207788 , 0.75932505, 0.11714337]),
array([ 0.87018655, -0.0278871 , 0.79703585, -0.01675315, 0.32161433,
0.01322907, 0.22717107, 0.02344416, 0.21055424, 0.02373596,
0.07355229, 0.05676325, 0.17151241, 0.05867112, 0.63630265,
0.06617649, 0.13289427, 0.06590531, 0.82214669, 0.09459811]))
At this point, either every satellite can be retasked, or satellites can continue their previous action by passing None as the action. To see which satellites must be retasked (i.e. their previous action is done and they have nothing more to do), look at "requires_retasking" in each satellite’s info.
[7]:
info
[7]:
{'EO-1': {'requires_retasking': False},
'EO-2': {'requires_retasking': False},
'EO-3': {'requires_retasking': True},
'd_ts': 402.0}
Based on this list, we decide here to only retask the satellite that needs it.
[8]:
actions = [0 if info[sat.name]["requires_retasking"] else None for sat in env.unwrapped.satellites]
actions
[8]:
[None, None, 0]
[9]:
observation, reward, terminated, truncated, info = env.step(actions)
2026-07-28 23:39:23,145 gym INFO <402.00> === STARTING STEP ===
2026-07-28 23:39:23,147 sats.satellite.EO-3 INFO <402.00> EO-3: target index 0 tasked
2026-07-28 23:39:23,147 sats.satellite.EO-3 INFO <402.00> EO-3: Target(tgt-670) tasked for imaging
2026-07-28 23:39:23,148 sats.satellite.EO-3 INFO <402.00> EO-3: Target(tgt-670) window enabled: 243.0 to 432.5
2026-07-28 23:39:23,149 sats.satellite.EO-3 INFO <402.00> EO-3: setting timed terminal event at 432.5
2026-07-28 23:39:23,155 sats.satellite.EO-3 INFO <433.00> EO-3: timed termination at 432.5 for Target(tgt-670) window
2026-07-28 23:39:23,156 data.base INFO <433.00> Total reward: {}
2026-07-28 23:39:23,157 sats.satellite.EO-3 INFO <433.00> EO-3: Satellite EO-3 requires retasking
2026-07-28 23:39:23,159 gym INFO <433.00> Step reward: 0.0
In this environment, the environment will stop if any agent dies. To demonstrate this, one satellite is forcibly killed.
[10]:
from Basilisk.architecture import messaging
def isnt_alive(log_failure=False):
"""Mock satellite 0 dying."""
self = env.unwrapped.satellites[0]
death_message = messaging.PowerStorageStatusMsgPayload()
death_message.storageLevel = 0.0
self.dynamics.powerMonitor.batPowerOutMsg.write(death_message)
return self.dynamics.is_alive(log_failure=log_failure) and self.fsw.is_alive(
log_failure=log_failure
)
env.unwrapped.satellites[0].is_alive = isnt_alive
observation, reward, terminated, truncated, info = env.step([6, 7, 9])
2026-07-28 23:39:23,165 gym INFO <433.00> === STARTING STEP ===
2026-07-28 23:39:23,165 sats.satellite.EO-1 INFO <433.00> EO-1: target index 6 tasked
2026-07-28 23:39:23,166 sats.satellite.EO-1 INFO <433.00> EO-1: Target(tgt-598) tasked for imaging
2026-07-28 23:39:23,166 sats.satellite.EO-1 INFO <433.00> EO-1: Target(tgt-598) window enabled: 849.0 to 1057.4
2026-07-28 23:39:23,166 sats.satellite.EO-1 INFO <433.00> EO-1: setting timed terminal event at 1057.4
2026-07-28 23:39:23,167 sats.satellite.EO-2 INFO <433.00> EO-2: target index 7 tasked
2026-07-28 23:39:23,167 sats.satellite.EO-2 INFO <433.00> EO-2: Target(tgt-382) tasked for imaging
2026-07-28 23:39:23,168 sats.satellite.EO-2 INFO <433.00> EO-2: Target(tgt-382) window enabled: 840.9 to 1033.5
2026-07-28 23:39:23,168 sats.satellite.EO-2 INFO <433.00> EO-2: setting timed terminal event at 1033.5
2026-07-28 23:39:23,169 sats.satellite.EO-3 INFO <433.00> EO-3: target index 9 tasked
2026-07-28 23:39:23,170 sats.satellite.EO-3 INFO <433.00> EO-3: Target(tgt-840) tasked for imaging
2026-07-28 23:39:23,171 sats.satellite.EO-3 INFO <433.00> EO-3: Target(tgt-840) window enabled: 830.3 to 1012.4
2026-07-28 23:39:23,172 sats.satellite.EO-3 INFO <433.00> EO-3: setting timed terminal event at 1012.4
2026-07-28 23:39:23,240 sats.satellite.EO-3 INFO <833.00> EO-3: imaged Target(tgt-840)
2026-07-28 23:39:23,241 data.base INFO <833.00> Total reward: {'EO-3': 0.6302168780912606}
2026-07-28 23:39:23,242 sats.satellite.EO-3 INFO <833.00> EO-3: Satellite EO-3 requires retasking
2026-07-28 23:39:23,243 sats.satellite.EO-1 INFO <833.00> EO-1: Finding opportunity windows from 1200.00 to 1800.00 seconds
2026-07-28 23:39:23,266 sats.satellite.EO-2 INFO <833.00> EO-2: Finding opportunity windows from 1200.00 to 1800.00 seconds
2026-07-28 23:39:23,291 sats.satellite.EO-1 WARNING <833.00> EO-1: failed battery_valid check
2026-07-28 23:39:23,292 gym INFO <833.00> Step reward: -0.3697831219087394
2026-07-28 23:39:23,293 gym INFO <833.00> Episode terminated: True
2026-07-28 23:39:23,293 gym INFO <833.00> Episode truncated: False
PettingZoo API
The PettingZoo parallel API environment, ConstellationTasking, is largely the same as GeneralSatelliteTasking. See their documentation for a full description of the API. It tends to separate things into dictionaries keyed by agent, rather than tuples.
[11]:
from bsk_rl import ConstellationTasking
env = ConstellationTasking(
satellites=[
ImagingSatellite("EO-1", sat_args),
ImagingSatellite("EO-2", sat_args),
ImagingSatellite("EO-3", sat_args),
],
scenario=scene.UniformTargets(1000),
rewarder=data.UniqueImageReward(),
communicator=comm.LOSCommunication(), # Note that dyn must inherit from LOSCommunication
sat_arg_randomizer=sat_arg_randomizer,
log_level="INFO",
)
env.reset()
env.observation_spaces
2026-07-28 23:39:23,299 WARNING Creating logger for new env on PID=4334. Old environments in process may now log times incorrectly.
2026-07-28 23:39:23,301 gym INFO Resetting environment with seed=2659541256
2026-07-28 23:39:23,303 scene.targets INFO Generating 1000 targets
2026-07-28 23:39:23,341 sats.satellite.EO-1 INFO <0.00> EO-1: Finding opportunity windows from 0.00 to 600.00 seconds
2026-07-28 23:39:23,365 sats.satellite.EO-1 INFO <0.00> EO-1: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-07-28 23:39:23,397 sats.satellite.EO-2 INFO <0.00> EO-2: Finding opportunity windows from 0.00 to 600.00 seconds
2026-07-28 23:39:23,419 sats.satellite.EO-3 INFO <0.00> EO-3: Finding opportunity windows from 0.00 to 600.00 seconds
2026-07-28 23:39:23,440 gym INFO <0.00> Environment reset
[11]:
{'EO-1': Box(-1e+16, 1e+16, (20,), float64),
'EO-2': Box(-1e+16, 1e+16, (20,), float64),
'EO-3': Box(-1e+16, 1e+16, (20,), float64)}
[12]:
env.action_spaces
[12]:
{'EO-1': Discrete(10), 'EO-2': Discrete(10), 'EO-3': Discrete(10)}
Actions are passed as a dictionary; the agent names can be accessed through the agents property.
[13]:
observation, reward, terminated, truncated, info = env.step(
{
env.agents[0]: 7,
env.agents[1]: 9,
env.agents[2]: 8,
}
)
2026-07-28 23:39:23,451 gym INFO <0.00> === STARTING STEP ===
2026-07-28 23:39:23,452 sats.satellite.EO-1 INFO <0.00> EO-1: target index 7 tasked
2026-07-28 23:39:23,452 sats.satellite.EO-1 INFO <0.00> EO-1: Target(tgt-258) tasked for imaging
2026-07-28 23:39:23,453 sats.satellite.EO-1 INFO <0.00> EO-1: Target(tgt-258) window enabled: 679.1 to 773.0
2026-07-28 23:39:23,453 sats.satellite.EO-1 INFO <0.00> EO-1: setting timed terminal event at 773.0
2026-07-28 23:39:23,454 sats.satellite.EO-2 INFO <0.00> EO-2: target index 9 tasked
2026-07-28 23:39:23,454 sats.satellite.EO-2 INFO <0.00> EO-2: Target(tgt-215) tasked for imaging
2026-07-28 23:39:23,455 sats.satellite.EO-2 INFO <0.00> EO-2: Target(tgt-215) window enabled: 225.7 to 434.0
2026-07-28 23:39:23,455 sats.satellite.EO-2 INFO <0.00> EO-2: setting timed terminal event at 434.0
2026-07-28 23:39:23,456 sats.satellite.EO-3 INFO <0.00> EO-3: target index 8 tasked
2026-07-28 23:39:23,457 sats.satellite.EO-3 INFO <0.00> EO-3: Target(tgt-990) tasked for imaging
2026-07-28 23:39:23,457 sats.satellite.EO-3 INFO <0.00> EO-3: Target(tgt-990) window enabled: 405.4 to 491.8
2026-07-28 23:39:23,458 sats.satellite.EO-3 INFO <0.00> EO-3: setting timed terminal event at 491.8
2026-07-28 23:39:23,503 sats.satellite.EO-2 INFO <228.00> EO-2: imaged Target(tgt-215)
2026-07-28 23:39:23,504 data.base INFO <228.00> Total reward: {'EO-2': 0.33553527135725913}
2026-07-28 23:39:23,505 sats.satellite.EO-2 INFO <228.00> EO-2: Satellite EO-2 requires retasking
2026-07-28 23:39:23,507 gym INFO <228.00> Step reward: {'EO-2': 0.33553527135725913}
[14]:
observation
[14]:
{'EO-1': array([ 0.21100774, -0.02707085, 0.08031594, -0.01352156, 0.24824216,
0.00597969, 0.19898935, 0.04531546, 0.81679262, 0.05339555,
0.34202282, 0.07154226, 0.6588438 , 0.07914378, 0.21692025,
0.09127459, 0.25861208, 0.09078203, 0.16034766, 0.09470916]),
'EO-2': array([ 0.08201499, -0.03190404, 0.85419247, -0.03145766, 0.48810691,
-0.00930605, 0.65013097, -0.01474836, 0.86232814, -0.01264183,
0.96105491, -0.01031333, 0.5845807 , 0.00113286, 0.28027491,
0.01271403, 0.36995942, 0.03193396, 0.24268463, 0.02711125]),
'EO-3': array([ 0.04785237, -0.01768634, 0.74436816, -0.02096046, 0.60428724,
-0.00277222, 0.33106912, 0.02443722, 0.23243698, 0.02825707,
0.05228779, 0.03111735, 0.60504502, 0.01191079, 0.17438103,
0.02866509, 0.76383074, 0.06374862, 0.72870737, 0.05955632])}
Other than compatibility with MARL algorithms, the main benefit of the PettingZoo API is that it allows for individual agents to fail without terminating the entire environment.
[15]:
# Immediately kill satellite 0
env.unwrapped.satellites[0].is_alive = isnt_alive
env.agents
[15]:
['EO-1', 'EO-2', 'EO-3']
[16]:
observation, reward, terminated, truncated, info = env.step({
env.agents[0]: 7,
env.agents[1]: 9,
}
)
2026-07-28 23:39:23,520 gym INFO <228.00> === STARTING STEP ===
2026-07-28 23:39:23,521 sats.satellite.EO-1 INFO <228.00> EO-1: target index 7 tasked
2026-07-28 23:39:23,522 sats.satellite.EO-1 INFO <228.00> EO-1: Target(tgt-153) tasked for imaging
2026-07-28 23:39:23,522 sats.satellite.EO-1 INFO <228.00> EO-1: Target(tgt-153) window enabled: 748.3 to 889.2
2026-07-28 23:39:23,523 sats.satellite.EO-1 INFO <228.00> EO-1: setting timed terminal event at 889.2
2026-07-28 23:39:23,523 sats.satellite.EO-2 INFO <228.00> EO-2: target index 9 tasked
2026-07-28 23:39:23,524 sats.satellite.EO-2 INFO <228.00> EO-2: Target(tgt-659) tasked for imaging
2026-07-28 23:39:23,525 sats.satellite.EO-2 INFO <228.00> EO-2: Target(tgt-659) window enabled: 382.5 to 577.9
2026-07-28 23:39:23,525 sats.satellite.EO-2 INFO <228.00> EO-2: setting timed terminal event at 577.9
2026-07-28 23:39:23,555 sats.satellite.EO-2 INFO <385.00> EO-2: imaged Target(tgt-659)
2026-07-28 23:39:23,556 data.base INFO <385.00> Total reward: {'EO-2': 0.242684634172877}
2026-07-28 23:39:23,557 sats.satellite.EO-2 INFO <385.00> EO-2: Satellite EO-2 requires retasking
2026-07-28 23:39:23,558 sats.satellite.EO-2 INFO <385.00> EO-2: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-07-28 23:39:23,578 sats.satellite.EO-3 INFO <385.00> EO-3: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-07-28 23:39:23,598 gym INFO <385.00> Step reward: {'EO-1': -1.0, 'EO-2': 0.242684634172877}
2026-07-28 23:39:23,598 gym INFO <385.00> Episode terminated: ['EO-1']