Multi-Agent Environments
Two multiagent environments are given in the package:
GeneralSatelliteTasking, a Gymnasium-based environment and the basis for all other environments.
ConstellationTasking, which implements the PettingZoo parallel API.
The latter is preferable for multi-agent RL (MARL) settings, as most algorithms are designed for this kind of API.
Configuring the Environment
For this example, a multisatellite target imaging environment will be used. The goal is to maximize the value of unique images taken.
As usual, the satellite type is defined first.
[1]:
from bsk_rl import sats, act, obs, scene, data, comm
from bsk_rl.sim import dyn, fsw
class ImagingSatellite(sats.ImagingSatellite):
observation_spec = [
obs.OpportunityProperties(
dict(prop="priority"),
dict(prop="opportunity_open", norm=5700.0),
n_ahead_observe=10,
)
]
action_spec = [act.Image(n_ahead_image=10)]
dyn_type = dyn.FullFeaturedDynModel
fsw_type = fsw.SteeringImagerFSWModel
Satellite properties are set to give the satellite near-unlimited power and storage resources. To randomize some parameters in a correlated manner across satellites, a sat_arg_randomizer is set and passed to the environment. In this case, the satellites are distributed in a trivial single-plane Walker-delta constellation.
[2]:
from bsk_rl.utils.orbital import walker_delta_args
sat_args = dict(
imageAttErrorRequirement=0.01,
imageRateErrorRequirement=0.01,
batteryStorageCapacity=1e9,
storedCharge_Init=1e9,
dataStorageCapacity=1e12,
u_max=0.4,
K1=0.25,
K3=3.0,
omega_max=0.087,
servo_Ki=5.0,
servo_P=150 / 5,
)
sat_arg_randomizer = walker_delta_args(altitude=800.0, inc=60.0, n_planes=1)
Gym API
GeneralSatelliteTasking uses tuples of actions and observations to interact with the environment.
[3]:
from bsk_rl import GeneralSatelliteTasking
env = GeneralSatelliteTasking(
satellites=[
ImagingSatellite("EO-1", sat_args),
ImagingSatellite("EO-2", sat_args),
ImagingSatellite("EO-3", sat_args),
],
scenario=scene.UniformTargets(1000),
rewarder=data.UniqueImageReward(),
communicator=comm.LOSCommunication(), # Note that dyn must inherit from LOSCommunication
sat_arg_randomizer=sat_arg_randomizer,
log_level="INFO",
)
env.reset()
env.observation_space
2026-07-23 20:39:39,386 gym INFO Resetting environment with seed=4102373661
2026-07-23 20:39:39,389 scene.targets INFO Generating 1000 targets
/home/runner/work/bsk_rl/bsk_rl/src/bsk_rl/sim/dyn/base.py:330: BSKDeprecationWarning: Basilisk.utilities.unitTestSupport.np2EigenMatrix3d will be removed after 2027-06-23: Use simHelpers.np2EigenMatrix3d instead.
self.scObject.hub.IHubPntBc_B = unitTestSupport.np2EigenMatrix3d(self.I_mat)
/home/runner/work/bsk_rl/bsk_rl/src/bsk_rl/sim/dyn/base.py:335: BSKDeprecationWarning: Basilisk.utilities.unitTestSupport.np2EigenVectorXd will be removed after 2027-06-23: Use simHelpers.np2EigenVectorXd instead.
self.scObject.hub.r_CN_NInit = unitTestSupport.np2EigenVectorXd(rN)
/home/runner/work/bsk_rl/bsk_rl/src/bsk_rl/sim/dyn/base.py:336: BSKDeprecationWarning: Basilisk.utilities.unitTestSupport.np2EigenVectorXd will be removed after 2027-06-23: Use simHelpers.np2EigenVectorXd instead.
self.scObject.hub.v_CN_NInit = unitTestSupport.np2EigenVectorXd(vN)
/home/runner/work/bsk_rl/bsk_rl/src/bsk_rl/sim/dyn/base.py:701: BSKDeprecationWarning: Basilisk.utilities.unitTestSupport.np2EigenVectorXd will be removed after 2027-06-23: Use simHelpers.np2EigenVectorXd instead.
unitTestSupport.np2EigenVectorXd(nHat_B),
2026-07-23 20:39:39,455 sats.satellite.EO-1 INFO <0.00> EO-1: Finding opportunity windows from 0.00 to 600.00 seconds
2026-07-23 20:39:39,498 sats.satellite.EO-2 INFO <0.00> EO-2: Finding opportunity windows from 0.00 to 600.00 seconds
2026-07-23 20:39:39,540 sats.satellite.EO-3 INFO <0.00> EO-3: Finding opportunity windows from 0.00 to 600.00 seconds
2026-07-23 20:39:39,579 gym INFO <0.00> Environment reset
[3]:
Tuple(Box(-1e+16, 1e+16, (20,), float64), Box(-1e+16, 1e+16, (20,), float64), Box(-1e+16, 1e+16, (20,), float64))
[4]:
env.action_space
[4]:
Tuple(Discrete(10), Discrete(10), Discrete(10))
Consequently, actions are passed as a tuple. The step will stop the first time any satellite completes an action.
[5]:
observation, reward, terminated, truncated, info = env.step([7, 9, 8])
2026-07-23 20:39:39,594 gym INFO <0.00> === STARTING STEP ===
2026-07-23 20:39:39,594 sats.satellite.EO-1 INFO <0.00> EO-1: target index 7 tasked
2026-07-23 20:39:39,595 sats.satellite.EO-1 INFO <0.00> EO-1: Target(tgt-233) tasked for imaging
2026-07-23 20:39:39,596 sats.satellite.EO-1 INFO <0.00> EO-1: Target(tgt-233) window enabled: 232.8 to 439.6
2026-07-23 20:39:39,596 sats.satellite.EO-1 INFO <0.00> EO-1: setting timed terminal event at 439.6
2026-07-23 20:39:39,598 sats.satellite.EO-2 INFO <0.00> EO-2: target index 9 tasked
2026-07-23 20:39:39,598 sats.satellite.EO-2 INFO <0.00> EO-2: Target(tgt-770) tasked for imaging
2026-07-23 20:39:39,599 sats.satellite.EO-2 INFO <0.00> EO-2: Target(tgt-770) window enabled: 461.7 to 600.0
2026-07-23 20:39:39,600 sats.satellite.EO-2 INFO <0.00> EO-2: setting timed terminal event at 600.0
2026-07-23 20:39:39,601 sats.satellite.EO-3 INFO <0.00> EO-3: target index 8 tasked
2026-07-23 20:39:39,601 sats.satellite.EO-3 INFO <0.00> EO-3: Target(tgt-324) tasked for imaging
2026-07-23 20:39:39,602 sats.satellite.EO-3 INFO <0.00> EO-3: Target(tgt-324) window enabled: 399.4 to 600.0
2026-07-23 20:39:39,603 sats.satellite.EO-3 INFO <0.00> EO-3: setting timed terminal event at 600.0
2026-07-23 20:39:39,661 sats.satellite.EO-1 INFO <235.00> EO-1: imaged Target(tgt-233)
2026-07-23 20:39:39,662 data.base INFO <235.00> Total reward: {'EO-1': 0.5263842651237525}
2026-07-23 20:39:39,663 sats.satellite.EO-1 INFO <235.00> EO-1: Satellite EO-1 requires retasking
2026-07-23 20:39:39,663 sats.satellite.EO-1 INFO <235.00> EO-1: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-07-23 20:39:39,701 gym INFO <235.00> Step reward: 0.5263842651237525
[6]:
observation
[6]:
(array([ 0.29614877, -0.02143865, 0.46287004, -0.00657994, 0.5446678 ,
0.00556678, 0.07649471, 0.01835521, 0.47403504, 0.01938431,
0.25143396, 0.03124141, 0.33224538, 0.06943352, 0.52200892,
0.09525983, 0.49279627, 0.09017059, 0.61876797, 0.12934802]),
array([ 0.62140258, -0.02045264, 0.1357805 , 0.00226611, 0.69619035,
0.0118423 , 0.74042679, 0.01597507, 0.61170298, 0.02714764,
0.02409966, 0.03822101, 0.68216053, 0.05948057, 0.09327898,
0.05363798, 0.50594752, 0.0397735 , 0.46749935, 0.05540091]),
array([ 0.1062439 , -0.02662127, 0.06880159, -0.01955847, 0.35172964,
0.03018645, 0.96131493, 0.02457413, 0.28996382, 0.03400198,
0.52909536, 0.02883931, 0.86323029, 0.0504642 , 0.13546505,
0.05418909, 0.76908779, 0.04775114, 0.83417772, 0.05539686]))
At this point, either every satellite can be retasked, or satellites can continue their previous action by passing None as the action. To see which satellites must be retasked (i.e. their previous action is done and they have nothing more to do), look at "requires_retasking" in each satellite’s info.
[7]:
info
[7]:
{'EO-1': {'requires_retasking': True},
'EO-2': {'requires_retasking': False},
'EO-3': {'requires_retasking': False},
'd_ts': 235.00000000000003}
Based on this list, we decide here to only retask the satellite that needs it.
[8]:
actions = [0 if info[sat.name]["requires_retasking"] else None for sat in env.unwrapped.satellites]
actions
[8]:
[0, None, None]
[9]:
observation, reward, terminated, truncated, info = env.step(actions)
2026-07-23 20:39:39,724 gym INFO <235.00> === STARTING STEP ===
2026-07-23 20:39:39,724 sats.satellite.EO-1 INFO <235.00> EO-1: target index 0 tasked
2026-07-23 20:39:39,725 sats.satellite.EO-1 INFO <235.00> EO-1: Target(tgt-734) tasked for imaging
2026-07-23 20:39:39,726 sats.satellite.EO-1 INFO <235.00> EO-1: Target(tgt-734) window enabled: 112.8 to 306.5
2026-07-23 20:39:39,727 sats.satellite.EO-1 INFO <235.00> EO-1: setting timed terminal event at 306.5
2026-07-23 20:39:39,745 sats.satellite.EO-1 INFO <292.00> EO-1: imaged Target(tgt-734)
2026-07-23 20:39:39,746 data.base INFO <292.00> Total reward: {'EO-1': 0.2961487672530666}
2026-07-23 20:39:39,746 sats.satellite.EO-1 INFO <292.00> EO-1: Satellite EO-1 requires retasking
2026-07-23 20:39:39,749 gym INFO <292.00> Step reward: 0.2961487672530666
In this environment, the environment will stop if any agent dies. To demonstrate this, one satellite is forcibly killed.
[10]:
from Basilisk.architecture import messaging
def isnt_alive(log_failure=False):
"""Mock satellite 0 dying."""
self = env.unwrapped.satellites[0]
death_message = messaging.PowerStorageStatusMsgPayload()
death_message.storageLevel = 0.0
self.dynamics.powerMonitor.batPowerOutMsg.write(death_message)
return self.dynamics.is_alive(log_failure=log_failure) and self.fsw.is_alive(
log_failure=log_failure
)
env.unwrapped.satellites[0].is_alive = isnt_alive
observation, reward, terminated, truncated, info = env.step([6, 7, 9])
2026-07-23 20:39:39,755 gym INFO <292.00> === STARTING STEP ===
2026-07-23 20:39:39,756 sats.satellite.EO-1 INFO <292.00> EO-1: target index 6 tasked
2026-07-23 20:39:39,756 sats.satellite.EO-1 INFO <292.00> EO-1: Target(tgt-192) tasked for imaging
2026-07-23 20:39:39,757 sats.satellite.EO-1 INFO <292.00> EO-1: Target(tgt-192) window enabled: 778.0 to 789.3
2026-07-23 20:39:39,757 sats.satellite.EO-1 INFO <292.00> EO-1: setting timed terminal event at 789.3
2026-07-23 20:39:39,759 sats.satellite.EO-2 INFO <292.00> EO-2: target index 7 tasked
2026-07-23 20:39:39,759 sats.satellite.EO-2 INFO <292.00> EO-2: Target(tgt-607) tasked for imaging
2026-07-23 20:39:39,760 sats.satellite.EO-2 INFO <292.00> EO-2: Target(tgt-607) window enabled: 540.7 to 600.0
2026-07-23 20:39:39,761 sats.satellite.EO-2 INFO <292.00> EO-2: setting timed terminal event at 600.0
2026-07-23 20:39:39,761 sats.satellite.EO-3 INFO <292.00> EO-3: target index 9 tasked
2026-07-23 20:39:39,762 sats.satellite.EO-3 INFO <292.00> EO-3: Target(tgt-974) tasked for imaging
2026-07-23 20:39:39,763 sats.satellite.EO-3 INFO <292.00> EO-3: Target(tgt-974) window enabled: 550.8 to 600.0
2026-07-23 20:39:39,763 sats.satellite.EO-3 INFO <292.00> EO-3: setting timed terminal event at 600.0
2026-07-23 20:39:39,835 sats.satellite.EO-2 INFO <543.00> EO-2: imaged Target(tgt-607)
2026-07-23 20:39:39,836 data.base INFO <543.00> Total reward: {'EO-2': 0.0932789759495607}
2026-07-23 20:39:39,837 sats.satellite.EO-2 INFO <543.00> EO-2: Satellite EO-2 requires retasking
2026-07-23 20:39:39,838 sats.satellite.EO-1 INFO <543.00> EO-1: Finding opportunity windows from 1200.00 to 1800.00 seconds
2026-07-23 20:39:39,890 sats.satellite.EO-2 INFO <543.00> EO-2: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-07-23 20:39:39,966 sats.satellite.EO-3 INFO <543.00> EO-3: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-07-23 20:39:40,009 sats.satellite.EO-1 WARNING <543.00> EO-1: failed battery_valid check
2026-07-23 20:39:40,011 gym INFO <543.00> Step reward: -0.9067210240504393
2026-07-23 20:39:40,012 gym INFO <543.00> Episode terminated: True
2026-07-23 20:39:40,012 gym INFO <543.00> Episode truncated: False
PettingZoo API
The PettingZoo parallel API environment, ConstellationTasking, is largely the same as GeneralSatelliteTasking. See their documentation for a full description of the API. It tends to separate things into dictionaries keyed by agent, rather than tuples.
[11]:
from bsk_rl import ConstellationTasking
env = ConstellationTasking(
satellites=[
ImagingSatellite("EO-1", sat_args),
ImagingSatellite("EO-2", sat_args),
ImagingSatellite("EO-3", sat_args),
],
scenario=scene.UniformTargets(1000),
rewarder=data.UniqueImageReward(),
communicator=comm.LOSCommunication(), # Note that dyn must inherit from LOSCommunication
sat_arg_randomizer=sat_arg_randomizer,
log_level="INFO",
)
env.reset()
env.observation_spaces
2026-07-23 20:39:40,020 WARNING Creating logger for new env on PID=4685. Old environments in process may now log times incorrectly.
2026-07-23 20:39:40,022 gym INFO Resetting environment with seed=443517169
2026-07-23 20:39:40,024 scene.targets INFO Generating 1000 targets
2026-07-23 20:39:40,067 sats.satellite.EO-1 INFO <0.00> EO-1: Finding opportunity windows from 0.00 to 600.00 seconds
2026-07-23 20:39:40,114 sats.satellite.EO-2 INFO <0.00> EO-2: Finding opportunity windows from 0.00 to 600.00 seconds
2026-07-23 20:39:40,151 sats.satellite.EO-3 INFO <0.00> EO-3: Finding opportunity windows from 0.00 to 600.00 seconds
2026-07-23 20:39:40,191 gym INFO <0.00> Environment reset
[11]:
{'EO-1': Box(-1e+16, 1e+16, (20,), float64),
'EO-2': Box(-1e+16, 1e+16, (20,), float64),
'EO-3': Box(-1e+16, 1e+16, (20,), float64)}
[12]:
env.action_spaces
[12]:
{'EO-1': Discrete(10), 'EO-2': Discrete(10), 'EO-3': Discrete(10)}
Actions are passed as a dictionary; the agent names can be accessed through the agents property.
[13]:
observation, reward, terminated, truncated, info = env.step(
{
env.agents[0]: 7,
env.agents[1]: 9,
env.agents[2]: 8,
}
)
2026-07-23 20:39:40,204 gym INFO <0.00> === STARTING STEP ===
2026-07-23 20:39:40,205 sats.satellite.EO-1 INFO <0.00> EO-1: target index 7 tasked
2026-07-23 20:39:40,206 sats.satellite.EO-1 INFO <0.00> EO-1: Target(tgt-716) tasked for imaging
2026-07-23 20:39:40,206 sats.satellite.EO-1 INFO <0.00> EO-1: Target(tgt-716) window enabled: 213.5 to 413.6
2026-07-23 20:39:40,207 sats.satellite.EO-1 INFO <0.00> EO-1: setting timed terminal event at 413.6
2026-07-23 20:39:40,209 sats.satellite.EO-2 INFO <0.00> EO-2: target index 9 tasked
2026-07-23 20:39:40,209 sats.satellite.EO-2 INFO <0.00> EO-2: Target(tgt-782) tasked for imaging
2026-07-23 20:39:40,210 sats.satellite.EO-2 INFO <0.00> EO-2: Target(tgt-782) window enabled: 397.8 to 600.0
2026-07-23 20:39:40,211 sats.satellite.EO-2 INFO <0.00> EO-2: setting timed terminal event at 600.0
2026-07-23 20:39:40,212 sats.satellite.EO-3 INFO <0.00> EO-3: target index 8 tasked
2026-07-23 20:39:40,212 sats.satellite.EO-3 INFO <0.00> EO-3: Target(tgt-272) tasked for imaging
2026-07-23 20:39:40,213 sats.satellite.EO-3 INFO <0.00> EO-3: Target(tgt-272) window enabled: 598.0 to 600.0
2026-07-23 20:39:40,214 sats.satellite.EO-3 INFO <0.00> EO-3: setting timed terminal event at 600.0
2026-07-23 20:39:40,269 sats.satellite.EO-1 INFO <216.00> EO-1: imaged Target(tgt-716)
2026-07-23 20:39:40,270 data.base INFO <216.00> Total reward: {'EO-1': 0.9016371067985927}
2026-07-23 20:39:40,271 sats.satellite.EO-1 INFO <216.00> EO-1: Satellite EO-1 requires retasking
2026-07-23 20:39:40,272 sats.satellite.EO-1 INFO <216.00> EO-1: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-07-23 20:39:40,313 sats.satellite.EO-3 INFO <216.00> EO-3: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-07-23 20:39:40,365 gym INFO <216.00> Step reward: {'EO-1': 0.9016371067985927}
[14]:
observation
[14]:
{'EO-1': array([ 0.97715355, -0.03392982, 0.47559523, -0.01254435, 0.99697058,
0.00286245, 0.41840993, 0.0062253 , 0.34252184, 0.03704592,
0.45430779, 0.02940397, 0.24929601, 0.0591923 , 0.01230719,
0.06643121, 0.73088295, 0.05213808, 0.0434921 , 0.07178733]),
'EO-2': array([ 0.30931429, -0.0296477 , 0.25151083, -0.01132397, 0.00956444,
-0.0147433 , 0.45115322, -0.01436648, 0.24755746, -0.01057436,
0.41269271, 0.00397858, 0.39996858, -0.00432535, 0.42943811,
0.03685536, 0.13883212, 0.05800028, 0.47205264, 0.03189739]),
'EO-3': array([ 0.31464509, -0.02547089, 0.57416934, -0.02325096, 0.96081066,
-0.01170343, 0.62622023, -0.00949071, 0.63614861, -0.00597942,
0.50153916, 0.00306494, 0.48302546, 0.02247722, 0.72194452,
0.05112856, 0.50340111, 0.06701346, 0.46261963, 0.07125547])}
Other than compatibility with MARL algorithms, the main benefit of the PettingZoo API is that it allows for individual agents to fail without terminating the entire environment.
[15]:
# Immediately kill satellite 0
env.unwrapped.satellites[0].is_alive = isnt_alive
env.agents
[15]:
['EO-1', 'EO-2', 'EO-3']
[16]:
observation, reward, terminated, truncated, info = env.step({
env.agents[0]: 7,
env.agents[1]: 9,
}
)
2026-07-23 20:39:40,382 gym INFO <216.00> === STARTING STEP ===
2026-07-23 20:39:40,383 sats.satellite.EO-1 INFO <216.00> EO-1: target index 7 tasked
2026-07-23 20:39:40,384 sats.satellite.EO-1 INFO <216.00> EO-1: Target(tgt-993) tasked for imaging
2026-07-23 20:39:40,385 sats.satellite.EO-1 INFO <216.00> EO-1: Target(tgt-993) window enabled: 594.7 to 707.7
2026-07-23 20:39:40,386 sats.satellite.EO-1 INFO <216.00> EO-1: setting timed terminal event at 707.7
2026-07-23 20:39:40,386 sats.satellite.EO-2 INFO <216.00> EO-2: target index 9 tasked
2026-07-23 20:39:40,387 sats.satellite.EO-2 INFO <216.00> EO-2: Target(tgt-782) window enabled: 397.8 to 600.0
2026-07-23 20:39:40,388 sats.satellite.EO-2 INFO <216.00> EO-2: setting timed terminal event at 600.0
2026-07-23 20:39:40,450 sats.satellite.EO-2 INFO <400.00> EO-2: imaged Target(tgt-782)
2026-07-23 20:39:40,450 data.base INFO <400.00> Total reward: {'EO-2': 0.47205264314687556}
2026-07-23 20:39:40,451 sats.satellite.EO-2 INFO <400.00> EO-2: Satellite EO-2 requires retasking
2026-07-23 20:39:40,453 sats.satellite.EO-2 INFO <400.00> EO-2: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-07-23 20:39:40,513 gym INFO <400.00> Step reward: {'EO-1': -1.0, 'EO-2': 0.47205264314687556}
2026-07-23 20:39:40,513 gym INFO <400.00> Episode terminated: ['EO-1']