Multi-Agent Environments
Two multiagent environments are given in the package:
GeneralSatelliteTasking, a Gymnasium-based environment and the basis for all other environments.
ConstellationTasking, which implements the PettingZoo parallel API.
The latter is preferable for multi-agent RL (MARL) settings, as most algorithms are designed for this kind of API.
Configuring the Environment
For this example, a multisatellite target imaging environment will be used. The goal is to maximize the value of unique images taken.
As usual, the satellite type is defined first.
[1]:
from typing import ClassVar
from bsk_rl import act, comm, data, obs, sats, scene
from bsk_rl.sim import dyn, fsw
class ImagingSatellite(sats.ImagingSatellite):
observation_spec: ClassVar[list[obs.Observation]] = [
obs.OpportunityProperties(
dict(prop="priority"),
dict(prop="opportunity_open", norm=5700.0),
n_ahead_observe=10,
)
]
action_spec: ClassVar[list[act.Action]] = [act.Image(n_ahead_image=10)]
dyn_type = dyn.FullFeaturedDynModel
fsw_type = fsw.SteeringImagerFSWModel
Satellite properties are set to give the satellite near-unlimited power and storage resources. To randomize some parameters in a correlated manner across satellites, a sat_arg_randomizer is set and passed to the environment. In this case, the satellites are distributed in a trivial single-plane Walker-delta constellation.
[2]:
from bsk_rl.utils.orbital import walker_delta_args
sat_args = dict(
imageAttErrorRequirement=0.01,
imageRateErrorRequirement=0.01,
batteryStorageCapacity=1e9,
storedCharge_Init=1e9,
dataStorageCapacity=1e12,
u_max=0.4,
K1=0.25,
K3=3.0,
omega_max=0.087,
servo_Ki=5.0,
servo_P=150 / 5,
)
sat_arg_randomizer = walker_delta_args(altitude=800.0, inc=60.0, n_planes=1)
Gym API
GeneralSatelliteTasking uses tuples of actions and observations to interact with the environment.
[3]:
from bsk_rl import GeneralSatelliteTasking
env = GeneralSatelliteTasking(
satellites=[
ImagingSatellite("EO_1", sat_args),
ImagingSatellite("EO_2", sat_args),
ImagingSatellite("EO_3", sat_args),
],
scenario=scene.UniformTargets(1000),
rewarder=data.UniqueImageReward(),
communicator=comm.LOSCommunication(), # Note that dyn must inherit from LOSCommunication
sat_arg_randomizer=sat_arg_randomizer,
log_level="INFO",
)
env.reset()
env.observation_space
2026-11-09 20:09:18,383 gym INFO Resetting environment with seed=3606055840
2026-11-09 20:09:18,385 scene.targets INFO Generating 1000 targets
2026-11-09 20:09:18,451 sats.satellite.EO_1 INFO <0.00> EO_1: Finding opportunity windows from 0.00 to 600.00 seconds
2026-11-09 20:09:18,469 sats.satellite.EO_2 INFO <0.00> EO_2: Finding opportunity windows from 0.00 to 600.00 seconds
2026-11-09 20:09:18,485 sats.satellite.EO_2 INFO <0.00> EO_2: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-11-09 20:09:18,504 sats.satellite.EO_3 INFO <0.00> EO_3: Finding opportunity windows from 0.00 to 600.00 seconds
2026-11-09 20:09:18,520 gym INFO <0.00> Environment reset
[3]:
Tuple(Box(-1e+16, 1e+16, (20,), float64), Box(-1e+16, 1e+16, (20,), float64), Box(-1e+16, 1e+16, (20,), float64))
[4]:
env.action_space
[4]:
Tuple(Discrete(10), Discrete(10), Discrete(10))
Consequently, actions are passed as a tuple. The step will stop the first time any satellite completes an action.
[5]:
observation, reward, terminated, truncated, info = env.step([7, 9, 8])
2026-11-09 20:09:18,530 gym INFO <0.00> === STARTING STEP ===
2026-11-09 20:09:18,530 sats.satellite.EO_1 INFO <0.00> EO_1: target index 7 tasked
2026-11-09 20:09:18,530 sats.satellite.EO_1 INFO <0.00> EO_1: Target(tgt-394) tasked for imaging
2026-11-09 20:09:18,531 sats.satellite.EO_1 INFO <0.00> EO_1: Target(tgt-394) window enabled: 256.5 to 403.7
2026-11-09 20:09:18,531 sats.satellite.EO_1 INFO <0.00> EO_1: setting timed terminal event at 403.7
2026-11-09 20:09:18,532 sats.satellite.EO_2 INFO <0.00> EO_2: target index 9 tasked
2026-11-09 20:09:18,532 sats.satellite.EO_2 INFO <0.00> EO_2: Target(tgt-645) tasked for imaging
2026-11-09 20:09:18,533 sats.satellite.EO_2 INFO <0.00> EO_2: Target(tgt-645) window enabled: 601.1 to 787.9
2026-11-09 20:09:18,533 sats.satellite.EO_2 INFO <0.00> EO_2: setting timed terminal event at 787.9
2026-11-09 20:09:18,534 sats.satellite.EO_3 INFO <0.00> EO_3: target index 8 tasked
2026-11-09 20:09:18,534 sats.satellite.EO_3 INFO <0.00> EO_3: Target(tgt-129) tasked for imaging
2026-11-09 20:09:18,535 sats.satellite.EO_3 INFO <0.00> EO_3: Target(tgt-129) window enabled: 438.7 to 600.0
2026-11-09 20:09:18,535 sats.satellite.EO_3 INFO <0.00> EO_3: setting timed terminal event at 600.0
2026-11-09 20:09:18,585 sats.satellite.EO_1 INFO <259.00> EO_1: imaged Target(tgt-394)
2026-11-09 20:09:18,586 data.base INFO <259.00> Total reward: {'EO_1': 0.36892830976860636}
2026-11-09 20:09:18,586 sats.satellite.EO_1 INFO <259.00> EO_1: Satellite EO_1 requires retasking
2026-11-09 20:09:18,586 sats.satellite.EO_1 INFO <259.00> EO_1: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-11-09 20:09:18,605 gym INFO <259.00> Step reward: 0.36892830976860636
[6]:
observation
[6]:
(array([ 8.09785271e-01, -2.98701347e-02, 1.52669546e-01, 7.80429453e-03,
2.95286520e-01, -1.17229219e-04, 5.27661857e-01, 3.98056205e-02,
2.45192650e-01, 6.50691148e-02, 9.43596094e-01, 5.47223877e-02,
7.34058977e-01, 5.27225362e-02, 8.67882159e-01, 6.10165586e-02,
3.27600602e-01, 8.75052488e-02, 2.77466324e-01, 6.48726950e-02]),
array([0.05509023, 0.02672192, 0.57981207, 0.01320986, 0.07878765,
0.02966948, 0.72707732, 0.03486494, 0.74247227, 0.05047854,
0.13080247, 0.03782862, 0.54099147, 0.05248683, 0.80078723,
0.06002517, 0.42167886, 0.07441973, 0.61076104, 0.06634833]),
array([ 0.2107735 , -0.01805952, 0.85071984, -0.0167406 , 0.0554047 ,
-0.0135674 , 0.49775243, 0.00913266, 0.40946674, -0.00657428,
0.86948191, 0.02807432, 0.31814359, 0.05482079, 0.55147529,
0.03153203, 0.52596466, 0.03560323, 0.63126932, 0.03783113]))
At this point, either every satellite can be retasked, or satellites can continue their previous action by passing None as the action. To see which satellites must be retasked (i.e. their previous action is done and they have nothing more to do), look at "requires_retasking" in each satellite’s info.
[7]:
info
[7]:
{'EO_1': {'requires_retasking': True},
'EO_2': {'requires_retasking': False},
'EO_3': {'requires_retasking': False},
'd_ts': 259.0}
Based on this list, we decide here to only retask the satellite that needs it.
[8]:
actions = [
0 if info[sat.name]["requires_retasking"] else None
for sat in env.unwrapped.satellites
]
actions
[8]:
[0, None, None]
[9]:
observation, reward, terminated, truncated, info = env.step(actions)
2026-11-09 20:09:18,619 gym INFO <259.00> === STARTING STEP ===
2026-11-09 20:09:18,619 sats.satellite.EO_1 INFO <259.00> EO_1: target index 0 tasked
2026-11-09 20:09:18,619 sats.satellite.EO_1 INFO <259.00> EO_1: Target(tgt-512) tasked for imaging
2026-11-09 20:09:18,620 sats.satellite.EO_1 INFO <259.00> EO_1: Target(tgt-512) window enabled: 88.7 to 283.2
2026-11-09 20:09:18,620 sats.satellite.EO_1 INFO <259.00> EO_1: setting timed terminal event at 283.2
2026-11-09 20:09:18,627 sats.satellite.EO_1 INFO <284.00> EO_1: timed termination at 283.2 for Target(tgt-512) window
2026-11-09 20:09:18,628 data.base INFO <284.00> Total reward: {}
2026-11-09 20:09:18,628 sats.satellite.EO_1 INFO <284.00> EO_1: Satellite EO_1 requires retasking
2026-11-09 20:09:18,630 gym INFO <284.00> Step reward: 0.0
In this environment, the environment will stop if any agent dies. To demonstrate this, one satellite is forcibly killed.
[10]:
from Basilisk.architecture import messaging
def isnt_alive(log_failure=False):
"""Mock satellite 0 dying."""
self = env.unwrapped.satellites[0]
death_message = messaging.PowerStorageStatusMsgPayload()
death_message.storageLevel = 0.0
self.dynamics.powerMonitor.batPowerOutMsg.write(death_message)
return self.dynamics.is_alive(log_failure=log_failure) and self.fsw.is_alive(
log_failure=log_failure
)
env.unwrapped.satellites[0].is_alive = isnt_alive
observation, reward, terminated, truncated, info = env.step([6, 7, 9])
2026-11-09 20:09:18,634 gym INFO <284.00> === STARTING STEP ===
2026-11-09 20:09:18,634 sats.satellite.EO_1 INFO <284.00> EO_1: target index 6 tasked
2026-11-09 20:09:18,635 sats.satellite.EO_1 INFO <284.00> EO_1: Target(tgt-195) tasked for imaging
2026-11-09 20:09:18,635 sats.satellite.EO_1 INFO <284.00> EO_1: Target(tgt-195) window enabled: 606.8 to 769.7
2026-11-09 20:09:18,635 sats.satellite.EO_1 INFO <284.00> EO_1: setting timed terminal event at 769.7
2026-11-09 20:09:18,636 sats.satellite.EO_2 INFO <284.00> EO_2: target index 7 tasked
2026-11-09 20:09:18,637 sats.satellite.EO_2 INFO <284.00> EO_2: Target(tgt-645) window enabled: 601.1 to 787.9
2026-11-09 20:09:18,637 sats.satellite.EO_2 INFO <284.00> EO_2: setting timed terminal event at 787.9
2026-11-09 20:09:18,637 sats.satellite.EO_3 INFO <284.00> EO_3: target index 9 tasked
2026-11-09 20:09:18,638 sats.satellite.EO_3 INFO <284.00> EO_3: Target(tgt-686) tasked for imaging
2026-11-09 20:09:18,638 sats.satellite.EO_3 INFO <284.00> EO_3: Target(tgt-686) window enabled: 474.6 to 600.0
2026-11-09 20:09:18,639 sats.satellite.EO_3 INFO <284.00> EO_3: setting timed terminal event at 600.0
2026-11-09 20:09:18,672 sats.satellite.EO_3 INFO <477.00> EO_3: imaged Target(tgt-686)
2026-11-09 20:09:18,673 data.base INFO <477.00> Total reward: {'EO_3': 0.6312693237075131}
2026-11-09 20:09:18,673 sats.satellite.EO_3 INFO <477.00> EO_3: Satellite EO_3 requires retasking
2026-11-09 20:09:18,674 sats.satellite.EO_3 INFO <477.00> EO_3: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-11-09 20:09:18,695 sats.satellite.EO_1 WARNING <477.00> EO_1: failed battery_valid check
2026-11-09 20:09:18,697 gym INFO <477.00> Step reward: -0.3687306762924869
2026-11-09 20:09:18,697 gym INFO <477.00> Episode terminated: True
2026-11-09 20:09:18,697 gym INFO <477.00> Episode truncated: False
PettingZoo API
The PettingZoo parallel API environment, ConstellationTasking, is largely the same as GeneralSatelliteTasking. See their documentation for a full description of the API. It tends to separate things into dictionaries keyed by agent, rather than tuples.
[11]:
from bsk_rl import ConstellationTasking
env = ConstellationTasking(
satellites=[
ImagingSatellite("EO_1", sat_args),
ImagingSatellite("EO_2", sat_args),
ImagingSatellite("EO_3", sat_args),
],
scenario=scene.UniformTargets(1000),
rewarder=data.UniqueImageReward(),
communicator=comm.LOSCommunication(), # Note that dyn must inherit from LOSCommunication
sat_arg_randomizer=sat_arg_randomizer,
log_level="INFO",
)
env.reset()
env.observation_spaces
2026-11-09 20:09:18,702 WARNING Creating logger for new env on PID=4719. Old environments in process may now log times incorrectly.
2026-11-09 20:09:18,704 gym INFO Resetting environment with seed=3926971607
2026-11-09 20:09:18,705 scene.targets INFO Generating 1000 targets
2026-11-09 20:09:18,742 sats.satellite.EO_1 INFO <0.00> EO_1: Finding opportunity windows from 0.00 to 600.00 seconds
2026-11-09 20:09:18,761 sats.satellite.EO_2 INFO <0.00> EO_2: Finding opportunity windows from 0.00 to 600.00 seconds
2026-11-09 20:09:18,785 sats.satellite.EO_3 INFO <0.00> EO_3: Finding opportunity windows from 0.00 to 600.00 seconds
2026-11-09 20:09:18,802 gym INFO <0.00> Environment reset
[11]:
{'EO_1': Box(-1e+16, 1e+16, (20,), float64),
'EO_2': Box(-1e+16, 1e+16, (20,), float64),
'EO_3': Box(-1e+16, 1e+16, (20,), float64)}
[12]:
env.action_spaces
[12]:
{'EO_1': Discrete(10), 'EO_2': Discrete(10), 'EO_3': Discrete(10)}
Actions are passed as a dictionary; the agent names can be accessed through the agents property.
[13]:
observation, reward, terminated, truncated, info = env.step(
{
env.agents[0]: 7,
env.agents[1]: 9,
env.agents[2]: 8,
}
)
2026-11-09 20:09:18,811 gym INFO <0.00> === STARTING STEP ===
2026-11-09 20:09:18,811 sats.satellite.EO_1 INFO <0.00> EO_1: target index 7 tasked
2026-11-09 20:09:18,812 sats.satellite.EO_1 INFO <0.00> EO_1: Target(tgt-717) tasked for imaging
2026-11-09 20:09:18,812 sats.satellite.EO_1 INFO <0.00> EO_1: Target(tgt-717) window enabled: 287.4 to 386.3
2026-11-09 20:09:18,813 sats.satellite.EO_1 INFO <0.00> EO_1: setting timed terminal event at 386.3
2026-11-09 20:09:18,813 sats.satellite.EO_2 INFO <0.00> EO_2: target index 9 tasked
2026-11-09 20:09:18,814 sats.satellite.EO_2 INFO <0.00> EO_2: Target(tgt-17) tasked for imaging
2026-11-09 20:09:18,814 sats.satellite.EO_2 INFO <0.00> EO_2: Target(tgt-17) window enabled: 528.3 to 600.0
2026-11-09 20:09:18,814 sats.satellite.EO_2 INFO <0.00> EO_2: setting timed terminal event at 600.0
2026-11-09 20:09:18,815 sats.satellite.EO_3 INFO <0.00> EO_3: target index 8 tasked
2026-11-09 20:09:18,815 sats.satellite.EO_3 INFO <0.00> EO_3: Target(tgt-230) tasked for imaging
2026-11-09 20:09:18,816 sats.satellite.EO_3 INFO <0.00> EO_3: Target(tgt-230) window enabled: 339.8 to 513.0
2026-11-09 20:09:18,816 sats.satellite.EO_3 INFO <0.00> EO_3: setting timed terminal event at 513.0
2026-11-09 20:09:18,867 sats.satellite.EO_1 INFO <290.00> EO_1: imaged Target(tgt-717)
2026-11-09 20:09:18,868 data.base INFO <290.00> Total reward: {'EO_1': 0.9209024340623988}
2026-11-09 20:09:18,869 sats.satellite.EO_1 INFO <290.00> EO_1: Satellite EO_1 requires retasking
2026-11-09 20:09:18,870 sats.satellite.EO_3 INFO <290.00> EO_3: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-11-09 20:09:18,890 gym INFO <290.00> Step reward: {'EO_1': 0.9209024340623988}
[14]:
observation
[14]:
{'EO_1': array([ 6.64321627e-01, -2.55512694e-02, 8.57998755e-01, -2.88804314e-02,
5.32344549e-01, -8.59502111e-04, 8.72928795e-01, -1.77151599e-02,
1.58364657e-01, -1.20932253e-02, 6.46498675e-01, 3.48702454e-04,
8.69128970e-01, -2.56295572e-03, 7.40723187e-01, 5.17955478e-03,
9.24484757e-02, 3.68039578e-02, 6.49702772e-01, 2.03249175e-02]),
'EO_2': array([ 0.91430095, -0.01970588, 0.44260039, 0.00110623, 0.43977249,
0.00904016, 0.80074181, 0.04181161, 0.92254218, 0.04026864,
0.65726644, 0.01877416, 0.78278333, 0.04993459, 0.38193856,
0.04680721, 0.37613004, 0.04293931, 0.86562859, 0.0422813 ]),
'EO_3': array([ 0.39774145, -0.03362813, 0.85488674, -0.02707705, 0.51697409,
-0.02699223, 0.73377192, -0.01868338, 0.64317729, 0.0107266 ,
0.06004222, 0.00566809, 0.81460099, 0.00872921, 0.88797246,
0.02909121, 0.91679252, 0.0339307 , 0.5116157 , 0.07612259])}
Other than compatibility with MARL algorithms, the main benefit of the PettingZoo API is that it allows for individual agents to fail without terminating the entire environment.
[15]:
# Immediately kill satellite 0
env.unwrapped.satellites[0].is_alive = isnt_alive
env.agents
[15]:
['EO_1', 'EO_2', 'EO_3']
[16]:
observation, reward, terminated, truncated, info = env.step(
{
env.agents[0]: 7,
env.agents[1]: 9,
}
)
2026-11-09 20:09:18,900 gym INFO <290.00> === STARTING STEP ===
2026-11-09 20:09:18,901 sats.satellite.EO_1 INFO <290.00> EO_1: target index 7 tasked
2026-11-09 20:09:18,901 sats.satellite.EO_1 INFO <290.00> EO_1: Target(tgt-206) tasked for imaging
2026-11-09 20:09:18,902 sats.satellite.EO_1 INFO <290.00> EO_1: Target(tgt-206) window enabled: 319.5 to 518.5
2026-11-09 20:09:18,902 sats.satellite.EO_1 INFO <290.00> EO_1: setting timed terminal event at 518.5
2026-11-09 20:09:18,903 sats.satellite.EO_2 INFO <290.00> EO_2: target index 9 tasked
2026-11-09 20:09:18,903 sats.satellite.EO_2 INFO <290.00> EO_2: Target(tgt-461) tasked for imaging
2026-11-09 20:09:18,903 sats.satellite.EO_2 INFO <290.00> EO_2: Target(tgt-461) window enabled: 531.0 to 600.0
2026-11-09 20:09:18,904 sats.satellite.EO_2 INFO <290.00> EO_2: setting timed terminal event at 600.0
2026-11-09 20:09:18,913 sats.satellite.EO_1 INFO <338.00> EO_1: imaged Target(tgt-206)
2026-11-09 20:09:18,914 data.base INFO <338.00> Total reward: {'EO_1': 0.740723186595938}
2026-11-09 20:09:18,914 sats.satellite.EO_1 INFO <338.00> EO_1: Satellite EO_1 requires retasking
2026-11-09 20:09:18,916 sats.satellite.EO_1 INFO <338.00> EO_1: Finding opportunity windows from 600.00 to 1200.00 seconds
2026-11-09 20:09:18,931 gym INFO <338.00> Step reward: {'EO_1': -0.25927681340406195}
2026-11-09 20:09:18,932 gym INFO <338.00> Episode terminated: ['EO_1']