Multi-Agent Environments
Two multiagent environments are given in the package:
GeneralSatelliteTasking, a Gymnasium-based environment and the basis for all other environments.
ConstellationTasking, which implements the PettingZoo parallel API.
The latter is preferable for multi-agent RL (MARL) settings, as most algorithms are designed for this kind of API.
Configuring the Environment
For this example, a multisatellite target imaging environment will be used. The goal is to maximize the value of unique images taken.
As usual, the satellite type is defined first.
[1]:
from bsk_rl import sats, act, obs, scene, data, comm
from bsk_rl.sim import dyn, fsw
class ImagingSatellite(sats.ImagingSatellite):
observation_spec = [
obs.OpportunityProperties(
dict(prop="priority"),
dict(prop="opportunity_open", norm=5700.0),
n_ahead_observe=10,
)
]
action_spec = [act.Image(n_ahead_image=10)]
dyn_type = dyn.FullFeaturedDynModel
fsw_type = fsw.SteeringImagerFSWModel
Satellite properties are set to give the satellite near-unlimited power and storage resources. To randomize some parameters in a correlated manner across satellites, a sat_arg_randomizer is set and passed to the environment. In this case, the satellites are distributed in a trivial single-plane Walker-delta constellation.
[2]:
from bsk_rl.utils.orbital import walker_delta_args
sat_args = dict(
imageAttErrorRequirement=0.01,
imageRateErrorRequirement=0.01,
batteryStorageCapacity=1e9,
storedCharge_Init=1e9,
dataStorageCapacity=1e12,
u_max=0.4,
K1=0.25,
K3=3.0,
omega_max=0.087,
servo_Ki=5.0,
servo_P=150 / 5,
)
sat_arg_randomizer = walker_delta_args(altitude=800.0, inc=60.0, n_planes=1)
Gym API
GeneralSatelliteTasking uses tuples of actions and observations to interact with the environment.
[3]:
from bsk_rl import GeneralSatelliteTasking
env = GeneralSatelliteTasking(
satellites=[
ImagingSatellite("EO-1", sat_args),
ImagingSatellite("EO-2", sat_args),
ImagingSatellite("EO-3", sat_args),
],
scenario=scene.UniformTargets(1000),
rewarder=data.UniqueImageReward(),
communicator=comm.LOSCommunication(), # Note that dyn must inherit from LOSCommunication
sat_arg_randomizer=sat_arg_randomizer,
log_level="INFO",
)
env.reset()
env.observation_space
2026-07-18 03:51:25,924 gym INFO Resetting environment with seed=1655655415
2026-07-18 03:51:25,927 scene.targets INFO Generating 1000 targets
/home/runner/work/bsk_rl/bsk_rl/src/bsk_rl/sim/dyn/base.py:330: BSKDeprecationWarning: Basilisk.utilities.unitTestSupport.np2EigenMatrix3d will be removed after 2027-06-23: Use simHelpers.np2EigenMatrix3d instead.
self.scObject.hub.IHubPntBc_B = unitTestSupport.np2EigenMatrix3d(self.I_mat)
/home/runner/work/bsk_rl/bsk_rl/src/bsk_rl/sim/dyn/base.py:335: BSKDeprecationWarning: Basilisk.utilities.unitTestSupport.np2EigenVectorXd will be removed after 2027-06-23: Use simHelpers.np2EigenVectorXd instead.
self.scObject.hub.r_CN_NInit = unitTestSupport.np2EigenVectorXd(rN)
/home/runner/work/bsk_rl/bsk_rl/src/bsk_rl/sim/dyn/base.py:336: BSKDeprecationWarning: Basilisk.utilities.unitTestSupport.np2EigenVectorXd will be removed after 2027-06-23: Use simHelpers.np2EigenVectorXd instead.
self.scObject.hub.v_CN_NInit = unitTestSupport.np2EigenVectorXd(vN)
/home/runner/work/bsk_rl/bsk_rl/src/bsk_rl/sim/dyn/base.py:701: BSKDeprecationWarning: Basilisk.utilities.unitTestSupport.np2EigenVectorXd will be removed after 2027-06-23: Use simHelpers.np2EigenVectorXd instead.
unitTestSupport.np2EigenVectorXd(nHat_B),
2026-07-18 03:51:25,997 sats.satellite.EO-1 INFO <0.00> EO-1: Finding opportunity windows from 0.00 to 600.00 seconds
2026-07-18 03:51:26,037 sats.satellite.EO-1 INFO <0.00> EO-1: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-07-18 03:51:26,080 sats.satellite.EO-2 INFO <0.00> EO-2: Finding opportunity windows from 0.00 to 600.00 seconds
2026-07-18 03:51:26,129 sats.satellite.EO-3 INFO <0.00> EO-3: Finding opportunity windows from 0.00 to 600.00 seconds
2026-07-18 03:51:26,167 sats.satellite.EO-3 INFO <0.00> EO-3: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-07-18 03:51:26,227 gym INFO <0.00> Environment reset
[3]:
Tuple(Box(-1e+16, 1e+16, (20,), float64), Box(-1e+16, 1e+16, (20,), float64), Box(-1e+16, 1e+16, (20,), float64))
[4]:
env.action_space
[4]:
Tuple(Discrete(10), Discrete(10), Discrete(10))
Consequently, actions are passed as a tuple. The step will stop the first time any satellite completes an action.
[5]:
observation, reward, terminated, truncated, info = env.step([7, 9, 8])
2026-07-18 03:51:26,245 gym INFO <0.00> === STARTING STEP ===
2026-07-18 03:51:26,246 sats.satellite.EO-1 INFO <0.00> EO-1: target index 7 tasked
2026-07-18 03:51:26,246 sats.satellite.EO-1 INFO <0.00> EO-1: Target(tgt-357) tasked for imaging
2026-07-18 03:51:26,247 sats.satellite.EO-1 INFO <0.00> EO-1: Target(tgt-357) window enabled: 307.7 to 501.8
2026-07-18 03:51:26,249 sats.satellite.EO-1 INFO <0.00> EO-1: setting timed terminal event at 501.8
2026-07-18 03:51:26,250 sats.satellite.EO-2 INFO <0.00> EO-2: target index 9 tasked
2026-07-18 03:51:26,250 sats.satellite.EO-2 INFO <0.00> EO-2: Target(tgt-727) tasked for imaging
2026-07-18 03:51:26,251 sats.satellite.EO-2 INFO <0.00> EO-2: Target(tgt-727) window enabled: 209.9 to 399.0
2026-07-18 03:51:26,252 sats.satellite.EO-2 INFO <0.00> EO-2: setting timed terminal event at 399.0
2026-07-18 03:51:26,253 sats.satellite.EO-3 INFO <0.00> EO-3: target index 8 tasked
2026-07-18 03:51:26,254 sats.satellite.EO-3 INFO <0.00> EO-3: Target(tgt-903) tasked for imaging
2026-07-18 03:51:26,255 sats.satellite.EO-3 INFO <0.00> EO-3: Target(tgt-903) window enabled: 599.0 to 791.3
2026-07-18 03:51:26,255 sats.satellite.EO-3 INFO <0.00> EO-3: setting timed terminal event at 791.3
2026-07-18 03:51:26,310 sats.satellite.EO-2 INFO <212.00> EO-2: imaged Target(tgt-727)
2026-07-18 03:51:26,312 data.base INFO <212.00> Total reward: {'EO-2': 0.8546892501587753}
2026-07-18 03:51:26,312 sats.satellite.EO-2 INFO <212.00> EO-2: Satellite EO-2 requires retasking
2026-07-18 03:51:26,316 gym INFO <212.00> Step reward: 0.8546892501587753
[6]:
observation
[6]:
(array([ 0.1088312 , -0.02431215, 0.35554287, -0.01968277, 0.98397863,
-0.00419493, 0.36606182, 0.00423861, 0.4412684 , 0.00615503,
0.54228435, 0.01678689, 0.10450404, 0.04602908, 0.82468336,
0.1031751 , 0.39897213, 0.0983513 , 0.55515587, 0.12160578]),
array([ 0.95757864, -0.01817219, 0.23912658, -0.01695447, 0.42017361,
-0.01599159, 0.44244882, -0.01167557, 0.24471272, 0.01442911,
0.42673351, 0.011557 , 0.22599841, 0.03050565, 0.32998391,
0.01610372, 0.47516584, 0.03674701, 0.46918454, 0.04048104]),
array([ 9.11064806e-01, -1.34699926e-02, 1.93301890e-01, -2.70988738e-02,
4.65058462e-01, -8.61637294e-03, 9.41660991e-02, -1.31560315e-02,
2.04904054e-01, -9.66900616e-03, 4.29437673e-01, 6.97736606e-04,
1.59401348e-01, 3.94113968e-02, 5.59194185e-01, 8.80410975e-02,
5.22814308e-01, 6.79004637e-02, 2.70834284e-01, 8.53427474e-02]))
At this point, either every satellite can be retasked, or satellites can continue their previous action by passing None as the action. To see which satellites must be retasked (i.e. their previous action is done and they have nothing more to do), look at "requires_retasking" in each satellite’s info.
[7]:
info
[7]:
{'EO-1': {'requires_retasking': False},
'EO-2': {'requires_retasking': True},
'EO-3': {'requires_retasking': False},
'd_ts': 212.0}
Based on this list, we decide here to only retask the satellite that needs it.
[8]:
actions = [0 if info[sat.name]["requires_retasking"] else None for sat in env.unwrapped.satellites]
actions
[8]:
[None, 0, None]
[9]:
observation, reward, terminated, truncated, info = env.step(actions)
2026-07-18 03:51:26,339 gym INFO <212.00> === STARTING STEP ===
2026-07-18 03:51:26,340 sats.satellite.EO-2 INFO <212.00> EO-2: target index 0 tasked
2026-07-18 03:51:26,341 sats.satellite.EO-2 INFO <212.00> EO-2: Target(tgt-301) tasked for imaging
2026-07-18 03:51:26,341 sats.satellite.EO-2 INFO <212.00> EO-2: Target(tgt-301) window enabled: 108.4 to 277.4
2026-07-18 03:51:26,342 sats.satellite.EO-2 INFO <212.00> EO-2: setting timed terminal event at 277.4
2026-07-18 03:51:26,360 sats.satellite.EO-2 INFO <265.00> EO-2: imaged Target(tgt-301)
2026-07-18 03:51:26,361 data.base INFO <265.00> Total reward: {'EO-2': 0.9575786388417618}
2026-07-18 03:51:26,361 sats.satellite.EO-2 INFO <265.00> EO-2: Satellite EO-2 requires retasking
2026-07-18 03:51:26,365 gym INFO <265.00> Step reward: 0.9575786388417618
In this environment, the environment will stop if any agent dies. To demonstrate this, one satellite is forcibly killed.
[10]:
from Basilisk.architecture import messaging
def isnt_alive(log_failure=False):
"""Mock satellite 0 dying."""
self = env.unwrapped.satellites[0]
death_message = messaging.PowerStorageStatusMsgPayload()
death_message.storageLevel = 0.0
self.dynamics.powerMonitor.batPowerOutMsg.write(death_message)
return self.dynamics.is_alive(log_failure=log_failure) and self.fsw.is_alive(
log_failure=log_failure
)
env.unwrapped.satellites[0].is_alive = isnt_alive
observation, reward, terminated, truncated, info = env.step([6, 7, 9])
2026-07-18 03:51:26,372 gym INFO <265.00> === STARTING STEP ===
2026-07-18 03:51:26,372 sats.satellite.EO-1 INFO <265.00> EO-1: target index 6 tasked
2026-07-18 03:51:26,373 sats.satellite.EO-1 INFO <265.00> EO-1: Target(tgt-345) tasked for imaging
2026-07-18 03:51:26,374 sats.satellite.EO-1 INFO <265.00> EO-1: Target(tgt-345) window enabled: 474.4 to 664.2
2026-07-18 03:51:26,375 sats.satellite.EO-1 INFO <265.00> EO-1: setting timed terminal event at 664.2
2026-07-18 03:51:26,376 sats.satellite.EO-2 INFO <265.00> EO-2: target index 7 tasked
2026-07-18 03:51:26,376 sats.satellite.EO-2 INFO <265.00> EO-2: Target(tgt-819) tasked for imaging
2026-07-18 03:51:26,378 sats.satellite.EO-2 INFO <265.00> EO-2: Target(tgt-819) window enabled: 421.5 to 519.1
2026-07-18 03:51:26,378 sats.satellite.EO-2 INFO <265.00> EO-2: setting timed terminal event at 519.1
2026-07-18 03:51:26,379 sats.satellite.EO-3 INFO <265.00> EO-3: target index 9 tasked
2026-07-18 03:51:26,380 sats.satellite.EO-3 INFO <265.00> EO-3: Target(tgt-465) tasked for imaging
2026-07-18 03:51:26,381 sats.satellite.EO-3 INFO <265.00> EO-3: Target(tgt-465) window enabled: 743.0 to 907.9
2026-07-18 03:51:26,382 sats.satellite.EO-3 INFO <265.00> EO-3: setting timed terminal event at 907.9
2026-07-18 03:51:26,426 sats.satellite.EO-2 INFO <424.00> EO-2: imaged Target(tgt-819)
2026-07-18 03:51:26,427 data.base INFO <424.00> Total reward: {'EO-2': 0.47516584388583294}
2026-07-18 03:51:26,428 sats.satellite.EO-2 INFO <424.00> EO-2: Satellite EO-2 requires retasking
2026-07-18 03:51:26,430 sats.satellite.EO-2 INFO <424.00> EO-2: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-07-18 03:51:26,488 sats.satellite.EO-1 WARNING <424.00> EO-1: failed battery_valid check
2026-07-18 03:51:26,490 gym INFO <424.00> Step reward: -0.5248341561141671
2026-07-18 03:51:26,491 gym INFO <424.00> Episode terminated: True
2026-07-18 03:51:26,491 gym INFO <424.00> Episode truncated: False
PettingZoo API
The PettingZoo parallel API environment, ConstellationTasking, is largely the same as GeneralSatelliteTasking. See their documentation for a full description of the API. It tends to separate things into dictionaries keyed by agent, rather than tuples.
[11]:
from bsk_rl import ConstellationTasking
env = ConstellationTasking(
satellites=[
ImagingSatellite("EO-1", sat_args),
ImagingSatellite("EO-2", sat_args),
ImagingSatellite("EO-3", sat_args),
],
scenario=scene.UniformTargets(1000),
rewarder=data.UniqueImageReward(),
communicator=comm.LOSCommunication(), # Note that dyn must inherit from LOSCommunication
sat_arg_randomizer=sat_arg_randomizer,
log_level="INFO",
)
env.reset()
env.observation_spaces
2026-07-18 03:51:26,500 WARNING Creating logger for new env on PID=4457. Old environments in process may now log times incorrectly.
2026-07-18 03:51:26,503 gym INFO Resetting environment with seed=2234260136
2026-07-18 03:51:26,505 scene.targets INFO Generating 1000 targets
2026-07-18 03:51:26,552 sats.satellite.EO-1 INFO <0.00> EO-1: Finding opportunity windows from 0.00 to 600.00 seconds
2026-07-18 03:51:26,606 sats.satellite.EO-1 INFO <0.00> EO-1: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-07-18 03:51:26,649 sats.satellite.EO-2 INFO <0.00> EO-2: Finding opportunity windows from 0.00 to 600.00 seconds
2026-07-18 03:51:26,689 sats.satellite.EO-3 INFO <0.00> EO-3: Finding opportunity windows from 0.00 to 600.00 seconds
2026-07-18 03:51:26,757 gym INFO <0.00> Environment reset
[11]:
{'EO-1': Box(-1e+16, 1e+16, (20,), float64),
'EO-2': Box(-1e+16, 1e+16, (20,), float64),
'EO-3': Box(-1e+16, 1e+16, (20,), float64)}
[12]:
env.action_spaces
[12]:
{'EO-1': Discrete(10), 'EO-2': Discrete(10), 'EO-3': Discrete(10)}
Actions are passed as a dictionary; the agent names can be accessed through the agents property.
[13]:
observation, reward, terminated, truncated, info = env.step(
{
env.agents[0]: 7,
env.agents[1]: 9,
env.agents[2]: 8,
}
)
2026-07-18 03:51:26,771 gym INFO <0.00> === STARTING STEP ===
2026-07-18 03:51:26,772 sats.satellite.EO-1 INFO <0.00> EO-1: target index 7 tasked
2026-07-18 03:51:26,773 sats.satellite.EO-1 INFO <0.00> EO-1: Target(tgt-483) tasked for imaging
2026-07-18 03:51:26,774 sats.satellite.EO-1 INFO <0.00> EO-1: Target(tgt-483) window enabled: 453.5 to 659.5
2026-07-18 03:51:26,775 sats.satellite.EO-1 INFO <0.00> EO-1: setting timed terminal event at 659.5
2026-07-18 03:51:26,776 sats.satellite.EO-2 INFO <0.00> EO-2: target index 9 tasked
2026-07-18 03:51:26,776 sats.satellite.EO-2 INFO <0.00> EO-2: Target(tgt-508) tasked for imaging
2026-07-18 03:51:26,777 sats.satellite.EO-2 INFO <0.00> EO-2: Target(tgt-508) window enabled: 554.6 to 600.0
2026-07-18 03:51:26,778 sats.satellite.EO-2 INFO <0.00> EO-2: setting timed terminal event at 600.0
2026-07-18 03:51:26,779 sats.satellite.EO-3 INFO <0.00> EO-3: target index 8 tasked
2026-07-18 03:51:26,779 sats.satellite.EO-3 INFO <0.00> EO-3: Target(tgt-80) tasked for imaging
2026-07-18 03:51:26,780 sats.satellite.EO-3 INFO <0.00> EO-3: Target(tgt-80) window enabled: 42.3 to 221.2
2026-07-18 03:51:26,780 sats.satellite.EO-3 INFO <0.00> EO-3: setting timed terminal event at 221.2
2026-07-18 03:51:26,803 sats.satellite.EO-3 INFO <83.00> EO-3: imaged Target(tgt-80)
2026-07-18 03:51:26,804 data.base INFO <83.00> Total reward: {'EO-3': 0.5896769280338586}
2026-07-18 03:51:26,805 sats.satellite.EO-3 INFO <83.00> EO-3: Satellite EO-3 requires retasking
2026-07-18 03:51:26,809 gym INFO <83.00> Step reward: {'EO-3': 0.5896769280338586}
[14]:
observation
[14]:
{'EO-1': array([ 0.58727191, 0.00089239, 0.32454008, -0.00442502, 0.74930722,
0.00527911, 0.09052917, 0.00671534, 0.74686536, 0.0103355 ,
0.60579407, 0.01738331, 0.18197546, 0.06030357, 0.6538727 ,
0.06500696, 0.69457981, 0.10502861, 0.7945001 , 0.13757465]),
'EO-2': array([ 0.70917299, -0.0145614 , 0.03297885, -0.00844334, 0.08828336,
-0.0145614 , 0.12982499, 0.00178257, 0.46531881, 0.01645117,
0.10679989, 0.04176311, 0.96287015, 0.07588112, 0.41673879,
0.07062561, 0.66653506, 0.08272823, 0.13188968, 0.0778784 ]),
'EO-3': array([ 0.72382597, -0.0145614 , 0.54084839, -0.0145614 , 0.25537293,
-0.00715845, 0.21734468, 0.01156485, 0.36168292, -0.00810215,
0.47262755, 0.00755228, 0.40316352, -0.00144273, 0.34226539,
0.04860005, 0.03058945, 0.02907199, 0.16426302, 0.04423624])}
Other than compatibility with MARL algorithms, the main benefit of the PettingZoo API is that it allows for individual agents to fail without terminating the entire environment.
[15]:
# Immediately kill satellite 0
env.unwrapped.satellites[0].is_alive = isnt_alive
env.agents
[15]:
['EO-1', 'EO-2', 'EO-3']
[16]:
observation, reward, terminated, truncated, info = env.step({
env.agents[0]: 7,
env.agents[1]: 9,
}
)
2026-07-18 03:51:26,827 gym INFO <83.00> === STARTING STEP ===
2026-07-18 03:51:26,828 sats.satellite.EO-1 INFO <83.00> EO-1: target index 7 tasked
2026-07-18 03:51:26,829 sats.satellite.EO-1 INFO <83.00> EO-1: Target(tgt-483) window enabled: 453.5 to 659.5
2026-07-18 03:51:26,830 sats.satellite.EO-1 INFO <83.00> EO-1: setting timed terminal event at 659.5
2026-07-18 03:51:26,831 sats.satellite.EO-2 INFO <83.00> EO-2: target index 9 tasked
2026-07-18 03:51:26,832 sats.satellite.EO-2 INFO <83.00> EO-2: Target(tgt-532) tasked for imaging
2026-07-18 03:51:26,833 sats.satellite.EO-2 INFO <83.00> EO-2: Target(tgt-532) window enabled: 526.9 to 600.0
2026-07-18 03:51:26,833 sats.satellite.EO-2 INFO <83.00> EO-2: setting timed terminal event at 600.0
2026-07-18 03:51:26,835 sats.satellite.EO-3 WARNING <83.00> EO-3: Requires retasking but received no task.
2026-07-18 03:51:26,875 sats.satellite.EO-3 INFO <222.00> EO-3: timed termination at 221.2 for Target(tgt-80) window
2026-07-18 03:51:26,876 data.base INFO <222.00> Total reward: {}
2026-07-18 03:51:26,877 sats.satellite.EO-3 INFO <222.00> EO-3: Satellite EO-3 requires retasking
2026-07-18 03:51:26,879 sats.satellite.EO-2 INFO <222.00> EO-2: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-07-18 03:51:26,924 sats.satellite.EO-3 INFO <222.00> EO-3: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-07-18 03:51:26,967 gym INFO <222.00> Step reward: {'EO-1': -1.0}
2026-07-18 03:51:26,968 gym INFO <222.00> Episode terminated: ['EO-1']