Time-Discounted GAE
In semi-MDPs, each step has an associated duration. Instead of the usual value equation
\begin{equation} V(s_1) = r_1 + \gamma r_2 + \gamma^2 r_3 + ... \end{equation}
one discount based on step duration
\begin{equation} V_{\Delta t}(s_1) = \gamma^{\Delta t_1} r_1 + \gamma^{\Delta t_1 + \Delta t_2} r_2 + \gamma^{\Delta t_1 + \Delta t_2 + \Delta t_3} r_3 + ... \end{equation}
using the convention that reward is given at the end of a step.
The generalized advantage estimator can be rewritten accordingly. In our implementation, the exponential decay lambda is per-step (as opposed to timewise).
RLlib Version
RLlib is actively developed and can change significantly from version to version. For this script, the following version is used:
[1]:
from importlib.metadata import version
version("ray") # Parent package of RLlib
[1]:
'2.35.0'
Define the Environment
A simple single-satellite environment is defined, as in :doc:examples/rllib_training.
[2]:
import numpy as np
from bsk_rl import act, data, obs, sats, scene
from bsk_rl.sim import dyn, fsw
class ScanningDownlinkDynModel(
dyn.ContinuousImagingDynModel, dyn.GroundStationDynModel
):
# Define some custom properties to be accessed in the state
@property
def instrument_pointing_error(self) -> float:
r_BN_P_unit = self.r_BN_P / np.linalg.norm(self.r_BN_P)
c_hat_P = self.satellite.fsw.c_hat_P
return np.arccos(np.dot(-r_BN_P_unit, c_hat_P))
@property
def solar_pointing_error(self) -> float:
a = (
self.world.gravFactory.spiceObject.planetStateOutMsgs[self.world.sun_index]
.read()
.PositionVector
)
a_hat_N = a / np.linalg.norm(a)
nHat_B = self.satellite.sat_args["nHat_B"]
NB = np.transpose(self.BN)
nHat_N = NB @ nHat_B
return np.arccos(np.dot(nHat_N, a_hat_N))
class ScanningSatellite(sats.AccessSatellite):
observation_spec = [
obs.SatProperties(
dict(prop="storage_level_fraction"),
dict(prop="battery_charge_fraction"),
dict(prop="wheel_speeds_fraction"),
dict(prop="instrument_pointing_error", norm=np.pi),
dict(prop="solar_pointing_error", norm=np.pi),
),
obs.OpportunityProperties(
dict(prop="opportunity_open", norm=5700),
dict(prop="opportunity_close", norm=5700),
type="ground_station",
n_ahead_observe=1,
),
obs.Eclipse(norm=5700),
]
action_spec = [
act.Scan(duration=180.0),
act.Charge(duration=120.0),
act.Downlink(duration=60.0),
act.Desat(duration=60.0),
]
dyn_type = ScanningDownlinkDynModel
fsw_type = fsw.ContinuousImagingFSWModel
sat = ScanningSatellite(
"Scanner-1",
sat_args=dict(
# Data
dataStorageCapacity=5000 * 8e6, # bits
storageInit=lambda: np.random.uniform(0.0, 0.8) * 5000 * 8e6,
instrumentBaudRate=0.5 * 8e6,
transmitterBaudRate=-50 * 8e6,
# Power
batteryStorageCapacity=200 * 3600, # W*s
storedCharge_Init=lambda: np.random.uniform(0.3, 1.0) * 200 * 3600,
basePowerDraw=-10.0, # W
instrumentPowerDraw=-30.0, # W
transmitterPowerDraw=-25.0, # W
thrusterPowerDraw=-80.0, # W
panelArea=0.25,
# Attitude
imageAttErrorRequirement=0.1,
imageRateErrorRequirement=0.1,
disturbance_vector=lambda: np.random.normal(scale=0.0001, size=3), # N*m
maxWheelSpeed=6000.0, # RPM
wheelSpeeds=lambda: np.random.uniform(-3000, 3000, 3),
desatAttitude="nadir",
),
)
duration = 5 * 5700.0 # About 5 orbits
env_args = dict(
satellite=sat,
scenario=scene.UniformNadirScanning(value_per_second=1 / duration),
rewarder=data.ScanningTimeReward(),
time_limit=duration,
failure_penalty=-1.0,
terminate_on_time_limit=True,
)
RLlib Configuration
The configuration is mostly the same as in the standard example.
[3]:
import bsk_rl.utils.rllib # noqa To access "SatelliteTasking-RLlib"
from ray.rllib.algorithms.ppo import PPOConfig
N_CPUS = 3
training_args = dict(
lr=0.00003,
gamma=0.999,
train_batch_size=250,
num_sgd_iter=10,
model=dict(fcnet_hiddens=[512, 512], vf_share_layers=False),
lambda_=0.95,
use_kl_loss=False,
clip_param=0.1,
grad_clip=0.5,
reward_time="step_end",
)
config = (
PPOConfig()
.env_runners(num_env_runners=N_CPUS - 1, sample_timeout_s=1000.0)
.environment(
env="SatelliteTasking-RLlib",
env_config=env_args,
)
.reporting(
metrics_num_episodes_for_smoothing=1,
metrics_episode_collection_timeout_s=180,
)
.checkpointing(export_native_model_files=True)
.framework(framework="torch")
.api_stack(
enable_rl_module_and_learner=True,
enable_env_runner_and_connector_v2=True,
)
)
Rewards can also be distributed at the start of the step by setting reward_time="step_start".
The additional setting that must be configured is the appropriate learner class. This uses the d_ts key from the info dict to discount by the step length, not just the step count.
[4]:
from bsk_rl.utils.rllib.discounting import TimeDiscountedGAEPPOTorchLearner
config.training(learner_class=TimeDiscountedGAEPPOTorchLearner)
[4]:
<ray.rllib.algorithms.ppo.ppo.PPOConfig at 0x7f04d693e450>
Training can then proceed as normal.
[5]:
import ray
from ray import tune
ray.init(
ignore_reinit_error=True,
num_cpus=N_CPUS,
object_store_memory=2_000_000_000, # 2 GB
)
# Run the training
tune.run(
"PPO",
config=config.to_dict(),
stop={"training_iteration": 2}, # Adjust the number of iterations as needed
)
# Shutdown Ray
ray.shutdown()
2026-07-23 20:41:01,105 INFO worker.py:1783 -- Started a local Ray instance.
2026-07-23 20:41:04,850 INFO tune.py:616 -- [output] This uses the legacy output and progress reporter, as Jupyter notebooks are not supported by the new engine, yet. For more information, please see https://github.com/ray-project/ray/issues/36949
/opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/gymnasium/spaces/box.py:130: UserWarning: WARN: Box bound precision lowered by casting to float32
gym.logger.warn(f"Box bound precision lowered by casting to {self.dtype}")
/opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/gymnasium/utils/passive_env_checker.py:164: UserWarning: WARN: The obs returned by the `reset()` method was expecting numpy array dtype to be float32, actual type: float64
logger.warn(
/opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/gymnasium/utils/passive_env_checker.py:188: UserWarning: WARN: The obs returned by the `reset()` method is not within the observation space.
logger.warn(f"{pre} is not within the observation space.")
Tune Status
| Current time: | 2026-07-23 20:42:14 |
| Running for: | 00:01:09.43 |
| Memory: | 5.0/15.6 GiB |
System Info
Using FIFO scheduling algorithm.Logical resource usage: 3.0/3 CPUs, 0/0 GPUs
Trial Status
| Trial name | status | loc | iter | total time (s) | num_env_steps_sample d_lifetime | num_episodes_lifetim e | num_env_steps_traine d_lifetime |
|---|---|---|---|---|---|---|---|
| PPO_SatelliteTasking-RLlib_d3271_00000 | TERMINATED | 10.1.0.7:6176 | 2 | 51.8174 | 8000 | 42 | 8000 |
(SingleAgentEnvRunner pid=6234) /home/runner/work/bsk_rl/bsk_rl/src/bsk_rl/sim/dyn/base.py:330: BSKDeprecationWarning: Basilisk.utilities.unitTestSupport.np2EigenMatrix3d will be removed after 2027-06-23: Use simHelpers.np2EigenMatrix3d instead.
(SingleAgentEnvRunner pid=6234) self.scObject.hub.IHubPntBc_B = unitTestSupport.np2EigenMatrix3d(self.I_mat)
(SingleAgentEnvRunner pid=6234) /home/runner/work/bsk_rl/bsk_rl/src/bsk_rl/sim/dyn/base.py:335: BSKDeprecationWarning: Basilisk.utilities.unitTestSupport.np2EigenVectorXd will be removed after 2027-06-23: Use simHelpers.np2EigenVectorXd instead.
(SingleAgentEnvRunner pid=6234) self.scObject.hub.r_CN_NInit = unitTestSupport.np2EigenVectorXd(rN)
(SingleAgentEnvRunner pid=6234) /home/runner/work/bsk_rl/bsk_rl/src/bsk_rl/sim/dyn/base.py:336: BSKDeprecationWarning: Basilisk.utilities.unitTestSupport.np2EigenVectorXd will be removed after 2027-06-23: Use simHelpers.np2EigenVectorXd instead.
(SingleAgentEnvRunner pid=6234) self.scObject.hub.v_CN_NInit = unitTestSupport.np2EigenVectorXd(vN)
(SingleAgentEnvRunner pid=6234) /home/runner/work/bsk_rl/bsk_rl/src/bsk_rl/sim/dyn/base.py:701: BSKDeprecationWarning: Basilisk.utilities.unitTestSupport.np2EigenVectorXd will be removed after 2027-06-23: Use simHelpers.np2EigenVectorXd instead.
(SingleAgentEnvRunner pid=6234) unitTestSupport.np2EigenVectorXd(nHat_B),
(SingleAgentEnvRunner pid=6233) unitTestSupport.np2EigenVectorXd(nHat_B),
(PPO pid=6176) unitTestSupport.np2EigenVectorXd(nHat_B),
(PPO pid=6176) Install gputil for GPU system monitoring.
(SingleAgentEnvRunner pid=6233) 2026-07-23 20:41:23,707 sats.satellite.Scanner-1 WARNING <10440.00> Scanner-1: failed battery_valid check
(PPO pid=6176) /home/runner/work/bsk_rl/bsk_rl/src/bsk_rl/sim/dyn/base.py:701: BSKDeprecationWarning: Basilisk.utilities.unitTestSupport.np2EigenVectorXd will be removed after 2027-06-23: Use simHelpers.np2EigenVectorXd instead. [repeated 8x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.)
(PPO pid=6176) self.scObject.hub.IHubPntBc_B = unitTestSupport.np2EigenMatrix3d(self.I_mat) [repeated 2x across cluster]
(PPO pid=6176) self.scObject.hub.r_CN_NInit = unitTestSupport.np2EigenVectorXd(rN) [repeated 2x across cluster]
(PPO pid=6176) self.scObject.hub.v_CN_NInit = unitTestSupport.np2EigenVectorXd(vN) [repeated 2x across cluster]
(SingleAgentEnvRunner pid=6233) 2026-07-23 20:41:28,931 sats.satellite.Scanner-1 WARNING <19740.00> Scanner-1: failed battery_valid check [repeated 7x across cluster]
(SingleAgentEnvRunner pid=6233) 2026-07-23 20:41:34,151 sats.satellite.Scanner-1 WARNING <26880.00> Scanner-1: failed battery_valid check [repeated 2x across cluster]
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,296 utils.orbital WARNING <0.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,306 utils.orbital WARNING <60.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,324 utils.orbital WARNING <240.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,333 utils.orbital WARNING <300.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,341 utils.orbital WARNING <360.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,355 utils.orbital WARNING <480.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,364 utils.orbital WARNING <540.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,373 utils.orbital WARNING <600.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,385 utils.orbital WARNING <720.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,397 utils.orbital WARNING <780.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,420 utils.orbital WARNING <960.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,429 utils.orbital WARNING <1020.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,436 utils.orbital WARNING <1080.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,457 utils.orbital WARNING <1260.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,469 utils.orbital WARNING <1380.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,478 utils.orbital WARNING <1440.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,489 utils.orbital WARNING <1500.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,499 utils.orbital WARNING <1560.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,510 utils.orbital WARNING <1620.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,519 utils.orbital WARNING <1680.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,530 utils.orbital WARNING <1740.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,541 utils.orbital WARNING <1800.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,552 utils.orbital WARNING <1920.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,561 utils.orbital WARNING <1980.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,573 utils.orbital WARNING <2100.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,587 utils.orbital WARNING <2280.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,594 utils.orbital WARNING <2340.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,609 utils.orbital WARNING <2520.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,616 utils.orbital WARNING <2580.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,630 utils.orbital WARNING <2760.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,637 utils.orbital WARNING <2820.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,647 utils.orbital WARNING <2940.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,654 utils.orbital WARNING <3000.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,669 utils.orbital WARNING <3180.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,676 utils.orbital WARNING <3240.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,683 utils.orbital WARNING <3300.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,697 utils.orbital WARNING <3480.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,704 utils.orbital WARNING <3540.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,715 utils.orbital WARNING <3660.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,722 utils.orbital WARNING <3720.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,729 utils.orbital WARNING <3780.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,736 utils.orbital WARNING <3840.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,750 utils.orbital WARNING <4020.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,757 utils.orbital WARNING <4080.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,764 utils.orbital WARNING <4140.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,774 utils.orbital WARNING <4260.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,789 utils.orbital WARNING <4380.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,798 utils.orbital WARNING <4440.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,808 utils.orbital WARNING <4500.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,819 utils.orbital WARNING <4560.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,827 utils.orbital WARNING <4620.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,835 utils.orbital WARNING <4680.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,844 utils.orbital WARNING <4740.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,853 utils.orbital WARNING <4800.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,863 utils.orbital WARNING <4860.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,884 utils.orbital WARNING <5040.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,901 utils.orbital WARNING <5220.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,912 utils.orbital WARNING <5340.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,921 utils.orbital WARNING <5400.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,931 utils.orbital WARNING <5460.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,940 utils.orbital WARNING <5520.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,951 utils.orbital WARNING <5580.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,962 utils.orbital WARNING <5640.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,983 utils.orbital WARNING <5820.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,994 utils.orbital WARNING <5880.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,014 utils.orbital WARNING <6060.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,029 utils.orbital WARNING <6180.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,041 utils.orbital WARNING <6300.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,055 utils.orbital WARNING <6420.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,063 utils.orbital WARNING <6480.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,071 utils.orbital WARNING <6540.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,086 utils.orbital WARNING <6660.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,094 utils.orbital WARNING <6720.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,103 utils.orbital WARNING <6780.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,113 utils.orbital WARNING <6840.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,128 utils.orbital WARNING <6960.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,143 utils.orbital WARNING <7080.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,152 utils.orbital WARNING <7140.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,161 utils.orbital WARNING <7200.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,180 utils.orbital WARNING <7380.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,197 utils.orbital WARNING <7560.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,204 utils.orbital WARNING <7620.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,219 utils.orbital WARNING <7800.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,229 utils.orbital WARNING <7920.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,244 utils.orbital WARNING <8100.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,251 utils.orbital WARNING <8160.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,266 utils.orbital WARNING <8340.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,276 utils.orbital WARNING <8460.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,283 utils.orbital WARNING <8520.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:38,201 sats.satellite.Scanner-1 WARNING <26160.00> Scanner-1: failed battery_valid check [repeated 4x across cluster]
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,298 utils.orbital WARNING <8700.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,309 utils.orbital WARNING <8820.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,319 utils.orbital WARNING <8940.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,330 utils.orbital WARNING <9060.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,340 utils.orbital WARNING <9180.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,347 utils.orbital WARNING <9240.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,362 utils.orbital WARNING <9420.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,369 utils.orbital WARNING <9480.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,376 utils.orbital WARNING <9540.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,387 utils.orbital WARNING <9660.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,397 utils.orbital WARNING <9720.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,404 utils.orbital WARNING <9780.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,411 utils.orbital WARNING <9840.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,422 utils.orbital WARNING <9960.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,430 utils.orbital WARNING <10020.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,441 utils.orbital WARNING <10140.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,448 utils.orbital WARNING <10200.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,462 utils.orbital WARNING <10380.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,472 utils.orbital WARNING <10440.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,482 utils.orbital WARNING <10500.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,490 utils.orbital WARNING <10560.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,501 utils.orbital WARNING <10620.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,510 utils.orbital WARNING <10680.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,528 utils.orbital WARNING <10860.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,546 utils.orbital WARNING <11040.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,558 utils.orbital WARNING <11160.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,573 utils.orbital WARNING <11340.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,579 utils.orbital WARNING <11400.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,594 utils.orbital WARNING <11580.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,605 utils.orbital WARNING <11640.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,626 utils.orbital WARNING <11820.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,636 utils.orbital WARNING <11880.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,644 utils.orbital WARNING <11940.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,654 utils.orbital WARNING <12000.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,664 utils.orbital WARNING <12060.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,674 utils.orbital WARNING <12120.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,688 utils.orbital WARNING <12240.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,710 utils.orbital WARNING <12420.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,725 utils.orbital WARNING <12540.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,740 utils.orbital WARNING <12660.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,755 utils.orbital WARNING <12780.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,764 utils.orbital WARNING <12840.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,773 utils.orbital WARNING <12900.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,793 utils.orbital WARNING <13080.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,803 utils.orbital WARNING <13140.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,825 utils.orbital WARNING <13320.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,836 utils.orbital WARNING <13380.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,856 utils.orbital WARNING <13560.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,864 utils.orbital WARNING <13620.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,881 utils.orbital WARNING <13800.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,900 utils.orbital WARNING <13980.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,911 utils.orbital WARNING <14040.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,920 utils.orbital WARNING <14100.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,928 utils.orbital WARNING <14160.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,937 utils.orbital WARNING <14220.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,953 utils.orbital WARNING <14400.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,966 utils.orbital WARNING <14520.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:39,991 utils.orbital WARNING <14700.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,001 utils.orbital WARNING <14760.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,012 utils.orbital WARNING <14820.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,022 utils.orbital WARNING <14880.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,030 utils.orbital WARNING <14940.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,041 utils.orbital WARNING <15000.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,059 utils.orbital WARNING <15180.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,073 utils.orbital WARNING <15360.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,082 utils.orbital WARNING <15420.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,091 utils.orbital WARNING <15480.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,100 utils.orbital WARNING <15540.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,119 utils.orbital WARNING <15720.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,135 utils.orbital WARNING <15840.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,144 utils.orbital WARNING <15900.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,153 utils.orbital WARNING <15960.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,167 utils.orbital WARNING <16080.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,175 utils.orbital WARNING <16140.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,184 utils.orbital WARNING <16200.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,202 utils.orbital WARNING <16380.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,220 utils.orbital WARNING <16500.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,232 utils.orbital WARNING <16560.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,250 utils.orbital WARNING <16740.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,258 utils.orbital WARNING <16800.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,265 utils.orbital WARNING <16860.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,273 utils.orbital WARNING <16920.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,280 utils.orbital WARNING <16980.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,290 utils.orbital WARNING <17100.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,297 utils.orbital WARNING <17160.00> Could not find eclipse transitions in next 12000.0 seconds
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:40,305 utils.orbital WARNING <17220.00> Could not find eclipse transitions in next 12000.0 seconds
Trial Progress
| Trial name | env_runners | fault_tolerance | learners | num_agent_steps_sampled_lifetime | num_env_steps_sampled_lifetime | num_env_steps_trained_lifetime | num_episodes_lifetime | perf | timers |
|---|---|---|---|---|---|---|---|---|---|
| PPO_SatelliteTasking-RLlib_d3271_00000 | {'num_env_steps_sampled_lifetime': 16000, 'num_episodes': 21, 'num_module_steps_sampled': {'default_policy': 4000}, 'episode_duration_sec_mean': 1.43597144499995, 'num_module_steps_sampled_lifetime': {'default_policy': 12000}, 'num_agent_steps_sampled_lifetime': {'default_agent': 12000}, 'episode_len_min': 109, 'num_env_steps_sampled': 4000, 'sample': np.float64(21.380969537555455), 'agent_episode_returns_mean': {'default_agent': -0.7757017543859648}, 'episode_return_max': -0.7223508771929823, 'episode_return_min': -0.8290526315789473, 'episode_len_max': 169, 'episode_return_mean': -0.7757017543859648, 'num_agent_steps_sampled': {'default_agent': 4000}, 'episode_len_mean': 139.0, 'module_episode_returns_mean': {'default_policy': -0.7757017543859648}, 'time_between_sampling': np.float64(4.760398875000021)} | {'num_healthy_workers': 2, 'num_in_flight_async_reqs': 0, 'num_remote_worker_restarts': 0} | {'default_policy': {'curr_kl_coeff': 0.20000000298023224, 'mean_kl_loss': 0.016711968928575516, 'num_non_trainable_parameters': 0.0, 'num_module_steps_trained': 4000, 'num_trainable_parameters': 139013.0, 'entropy': 1.3517364263534546, 'policy_loss': 0.1501941829919815, 'total_loss': 0.15378421545028687, 'vf_explained_var': 0.03400951623916626, 'default_optimizer_learning_rate': 5e-05, 'vf_loss': 0.00024764236877672374, 'vf_loss_unclipped': 0.00024764236877672374, 'curr_entropy_coeff': 0.0}, '__all_modules__': {'num_env_steps_trained': 4000, 'total_loss': 0.15378421545028687, 'num_non_trainable_parameters': 0.0, 'num_module_steps_trained': 4000, 'num_trainable_parameters': 139013.0}} | {'default_agent': 8000} | 8000 | 8000 | 42 | {'cpu_util_percent': np.float64(46.34324324324324), 'ram_util_percent': np.float64(31.64054054054055)} | {'env_runner_sampling_timer': 21.63686972948001, 'learner_update_timer': 4.462519843529986, 'synch_weights': 0.0056780599600551795, 'synch_env_connectors': 0.0064777804100106095} |
(SingleAgentEnvRunner pid=6233) 2026-07-23 20:41:42,654 utils.orbital WARNING <22860.00> Could not find eclipse transitions in next 12000.0 seconds [repeated 341x across cluster]
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:49,008 sats.satellite.Scanner-1 WARNING <17280.00> Scanner-1: failed battery_valid check [repeated 4x across cluster]
(SingleAgentEnvRunner pid=6234) /home/runner/work/bsk_rl/bsk_rl/src/bsk_rl/sim/dyn/base.py:330: BSKDeprecationWarning: Basilisk.utilities.unitTestSupport.np2EigenMatrix3d will be removed after 2027-06-23: Use simHelpers.np2EigenMatrix3d instead.
(SingleAgentEnvRunner pid=6234) self.scObject.hub.IHubPntBc_B = unitTestSupport.np2EigenMatrix3d(self.I_mat)
(SingleAgentEnvRunner pid=6234) /home/runner/work/bsk_rl/bsk_rl/src/bsk_rl/sim/dyn/base.py:335: BSKDeprecationWarning: Basilisk.utilities.unitTestSupport.np2EigenVectorXd will be removed after 2027-06-23: Use simHelpers.np2EigenVectorXd instead.
(SingleAgentEnvRunner pid=6234) self.scObject.hub.r_CN_NInit = unitTestSupport.np2EigenVectorXd(rN)
(SingleAgentEnvRunner pid=6234) /home/runner/work/bsk_rl/bsk_rl/src/bsk_rl/sim/dyn/base.py:336: BSKDeprecationWarning: Basilisk.utilities.unitTestSupport.np2EigenVectorXd will be removed after 2027-06-23: Use simHelpers.np2EigenVectorXd instead.
(SingleAgentEnvRunner pid=6234) self.scObject.hub.v_CN_NInit = unitTestSupport.np2EigenVectorXd(vN)
(SingleAgentEnvRunner pid=6234) /home/runner/work/bsk_rl/bsk_rl/src/bsk_rl/sim/dyn/base.py:701: BSKDeprecationWarning: Basilisk.utilities.unitTestSupport.np2EigenVectorXd will be removed after 2027-06-23: Use simHelpers.np2EigenVectorXd instead.
(SingleAgentEnvRunner pid=6234) unitTestSupport.np2EigenVectorXd(nHat_B),
(SingleAgentEnvRunner pid=6233) unitTestSupport.np2EigenVectorXd(nHat_B),
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:41:54,610 sats.satellite.Scanner-1 WARNING <25740.00> Scanner-1: failed battery_valid check [repeated 3x across cluster]
(SingleAgentEnvRunner pid=6233) /home/runner/work/bsk_rl/bsk_rl/src/bsk_rl/sim/dyn/base.py:701: BSKDeprecationWarning: Basilisk.utilities.unitTestSupport.np2EigenVectorXd will be removed after 2027-06-23: Use simHelpers.np2EigenVectorXd instead. [repeated 4x across cluster]
(SingleAgentEnvRunner pid=6233) self.scObject.hub.IHubPntBc_B = unitTestSupport.np2EigenMatrix3d(self.I_mat)
(SingleAgentEnvRunner pid=6233) self.scObject.hub.r_CN_NInit = unitTestSupport.np2EigenVectorXd(rN)
(SingleAgentEnvRunner pid=6233) self.scObject.hub.v_CN_NInit = unitTestSupport.np2EigenVectorXd(vN)
(SingleAgentEnvRunner pid=6233) 2026-07-23 20:42:01,132 sats.satellite.Scanner-1 WARNING <26100.00> Scanner-1: failed battery_valid check [repeated 3x across cluster]
(SingleAgentEnvRunner pid=6233) 2026-07-23 20:42:06,301 sats.satellite.Scanner-1 WARNING <23940.00> Scanner-1: failed battery_valid check [repeated 2x across cluster]
2026-07-23 20:42:14,338 INFO tune.py:1009 -- Wrote the latest version of all result files and experiment state to '/home/runner/ray_results/PPO_2026-07-23_20-41-04' in 0.0163s.
2026-07-23 20:42:14,998 INFO tune.py:1041 -- Total run time: 70.15 seconds (69.41 seconds for the tuning loop).
(SingleAgentEnvRunner pid=6234) 2026-07-23 20:42:09,330 sats.satellite.Scanner-1 WARNING <18900.00> Scanner-1: failed battery_valid check [repeated 5x across cluster]