Hybrid imaging with time-varying image quality
This example shows how to set up an imaging environment with targets that have heterogeneous, time-varying rewards. The underlying environment accepts hybrid actions consisting of a target index and a desired acquisition delay. Here, a custom actor produces categorical logits over target/timing pairs, and a separate critic estimates the state value. Action wrappers convert the selected pair into a target index and delay in seconds. The final section demonstrates multi-satellite coordination through shared acquisition information and task intents. This example is part of an upcoming publication.
[1]:
from collections.abc import Callable, Iterable
from typing import Any, ClassVar
import gymnasium as gym
import numpy as np
from Basilisk.architecture import bskLogging
from Basilisk.utilities import orbitalMotion
from scipy.interpolate import make_lsq_spline
from bsk_rl import ConstellationTasking, SatelliteTasking, act, obs, sats
from bsk_rl.data.unique_image_data import (
GlobalReward,
UniqueImageData,
UniqueImageReward,
UniqueImageStore,
)
from bsk_rl.scene.targets import CityTargets, Target
from bsk_rl.sim import dyn, fsw
from bsk_rl.utils.orbital import random_circular_orbit
bskLogging.setDefaultLogLevel(bskLogging.BSK_WARNING)
ALTITUDE_KM = 800.0
EARTH_RADIUS = orbitalMotion.REQ_EARTH * 1e3
ORBIT_PERIOD = (
2
* np.pi
* np.sqrt((orbitalMotion.REQ_EARTH + ALTITUDE_KM) ** 3 / orbitalMotion.MU_EARTH)
)
N_AHEAD = 40
SIM_RATE = 0.5
IMAGING_FRACTIONS = (0.0, 0.5, 1.0)
SPLINE_DEGREE = 2
SPLINE_KNOTS = np.r_[np.zeros(3), [1 / 3, 2 / 3], np.ones(3)]
/Users/lorenzzo/Local/repo_updates/issue_363/.venv_issue_363/lib/python3.12/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm
Targets and quality
Acquisition quality can depend on viewing geometry or a stakeholder preference for the acquisition time. This example includes the paper’s off-nadir and periodic quality models. The distance and glare models are omitted to keep the example compact.
QualityTargets extends CityTargets by assigning a quality model and weight to each target. Setting quality_weight=1.0 makes reward equal to priority times quality, while quality_weight=0.0 makes reward depend only on priority. Setting quality_weight=(0.0, 1.0) samples a separate weight for each target on every reset. Choose a single model with quality_types=("quality_off_nadir_angle",) or quality_types=("quality_periodic",), or sample uniformly from
quality_types=("quality_off_nadir_angle", "quality_periodic"). The absolute acquisition reward is
\[r_i(t)=p_i\big[1-w_{q,i}+w_{q,i}q_i(t)\big],\]
where \(p_i\) is target priority, \(w_{q,i}\) is its quality weight, and \(q_i(t)\in[0,1]\) is the quality at acquisition time. The rewarder below determines whether an acquisition receives this absolute reward or only its improvement over the best completed reward.
Off-nadir quality uses the angle between the target’s local vertical and its viewing direction: \(q=\mathrm{clip}(1-2\theta/\pi,0,1)\). Future samples use predicted geometric viewing directions; at the current time, realized instrument pointing is used. Periodic quality represents a known, time-varying acquisition preference, using one or two sinusoidal components with periods between 150 and 570 seconds. Frequencies in the code are angular frequencies in radians per second. These are illustrative acquisition-quality models, not a radiometric sensor simulation.
[2]:
def periodic_quality_function(t, amplitudes, frequencies, phases):
"""Evaluate a window-independent sum of periodic components.
``frequencies`` are angular frequencies in radians per second.
"""
t = np.asarray(t, dtype=float)
amplitudes = np.asarray(amplitudes, dtype=float)
frequencies = np.asarray(frequencies, dtype=float)
phases = np.asarray(phases, dtype=float)
if amplitudes.ndim != 1 or frequencies.ndim != 1 or phases.ndim != 1:
raise ValueError("amplitudes, frequencies, and phases must be 1-D arrays.")
if not (len(amplitudes) == len(frequencies) == len(phases)):
raise ValueError(
"amplitudes, frequencies, and phases must have the same length."
)
if len(amplitudes) == 0:
raise ValueError("At least one periodic component is required.")
if not (
np.all(np.isfinite(t))
and np.all(np.isfinite(amplitudes))
and np.all(np.isfinite(frequencies))
and np.all(np.isfinite(phases))
):
raise ValueError("Periodic-function inputs must all be finite.")
if np.any(frequencies <= 0):
raise ValueError("Periodic frequencies must be greater than zero.")
amplitude_sum = np.sum(np.abs(amplitudes))
if amplitude_sum <= 0:
raise ValueError("At least one periodic amplitude must be nonzero.")
quality = np.sum(
np.array(
[
amplitudes[i] * np.sin(frequencies[i] * t + phases[i])
for i in range(len(amplitudes))
]
),
axis=0,
)
quality = (quality + amplitude_sum) / (2 * amplitude_sum)
return quality.item() if np.ndim(quality) == 0 else quality
def quality_periodic(sat, tgt, t, amplitudes, frequencies, phases, **kwargs):
return periodic_quality_function(
t,
amplitudes=amplitudes,
frequencies=frequencies,
phases=phases,
)
def _positions_at_times(sat, times, trajectory_r_BP_P=None):
"""Return predicted positions, using dynamics for the realized current time."""
times = np.asarray(times, dtype=float)
current_mask = np.abs(times - sat.simulator.sim_time) <= 1e-6
if times.ndim == 0:
if current_mask:
return np.asarray(sat.dynamics.r_BN_P, dtype=float)
elif np.all(current_mask):
return np.broadcast_to(
np.asarray(sat.dynamics.r_BN_P, dtype=float),
times.shape + (3,),
).copy()
if trajectory_r_BP_P is None:
trajectory_r_BP_P = sat.trajectory.r_BP_P
positions = np.asarray(trajectory_r_BP_P(times), dtype=float)
if times.ndim > 0 and np.any(current_mask):
positions = positions.copy()
positions[current_mask] = sat.dynamics.r_BN_P
return positions
def quality_off_nadir_angle(sat, tgt, t, trajectory_r_BP_P=None, **kwargs):
"""Return quality based on the target-centered instrument viewing angle.
At the current simulation time, use the realized instrument direction so
the reward reflects the actual imaging attitude. For future profile samples,
where the attitude is not yet known, use the geometric target-to-satellite
direction. Quality decreases linearly from one for a target directly below
the instrument to zero for a target viewed at its local horizon.
"""
t = np.asarray(t, dtype=float)
scalar_input = t.ndim == 0
r_BN_P = _positions_at_times(sat, t, trajectory_r_BP_P)
target_position = np.asarray(tgt.r_LP_P, dtype=float)
target_radius = np.linalg.norm(target_position)
target_to_camera = r_BN_P - target_position
view_norm = np.linalg.norm(target_to_camera, axis=-1, keepdims=True)
if (
np.any(~np.isfinite(r_BN_P))
or not np.all(np.isfinite(target_position))
or target_radius <= 0.0
or np.any(view_norm <= 0.0)
):
raise ValueError("Off-nadir geometry must contain finite, nonzero vectors.")
local_vertical_hat_P = target_position / target_radius
view_hat_P = target_to_camera / view_norm
realized_mask = np.abs(t - sat.simulator.sim_time) <= 1e-6
if np.any(realized_mask):
c_hat_P = np.asarray(sat.fsw.c_hat_P, dtype=float)
if c_hat_P.shape != (3,) or not np.all(np.isfinite(c_hat_P)):
raise ValueError("c_hat_P must be a finite three-component vector.")
pointing_norm = np.linalg.norm(c_hat_P)
if pointing_norm <= 0.0:
raise ValueError("c_hat_P must be nonzero.")
if scalar_input:
view_hat_P = -c_hat_P / pointing_norm
else:
view_hat_P = view_hat_P.copy()
view_hat_P[realized_mask] = -c_hat_P / pointing_norm
off_nadir = np.arccos(
np.clip(np.sum(view_hat_P * local_vertical_hat_P, axis=-1), -1.0, 1.0)
)
quality = np.clip(1 - off_nadir / (np.pi / 2), 0.0, 1.0)
return quality.item() if scalar_input else quality
QUALITY_FUNCTIONS = {
"quality_off_nadir_angle": quality_off_nadir_angle,
"quality_periodic": quality_periodic,
}
def get_randomized_quality_function_params(quality_function_name):
if quality_function_name in ["quality_off_nadir_angle"]:
return {}
elif quality_function_name == "quality_periodic":
n_components = np.random.randint(1, 3)
amplitudes = np.random.uniform(0.1, 1.0, size=n_components)
frequencies = (
2
* np.pi
* np.random.uniform(
10.0 / 5700.0,
1.0 / 150.0,
size=n_components,
)
)
phases = np.random.uniform(0.0, 2 * np.pi, size=n_components)
return {
"amplitudes": amplitudes,
"frequencies": frequencies,
"phases": phases,
}
else:
raise ValueError(f"Unknown quality function: {quality_function_name}")
Assign quality models to targets
QualityTargets extends CityTargets: it generates city-based targets through the parent class and then attaches a quality weight, a selected quality model, and that model’s randomized parameters to each target. A scalar weight applies to all targets; a two-element range samples a separate weight for each target. A one-element tuple of quality names selects the same model for every target.
On each reset, the scenario initializes the best obtained reward to 0.0 and clears the per-satellite capture timestamps. These fields support marginal rewards for re-imaging and the cooldown filter used in the coordination example below.
[3]:
from copy import deepcopy
class QualityTargets(CityTargets):
"""City-based target scenario with per-target quality models and weights."""
def __init__(
self,
n_targets: int | tuple[int, int],
n_select_from: int | None = None,
location_offset: float = 0,
priority_distribution: Callable | None = None,
radius: float = orbitalMotion.REQ_EARTH * 1e3,
quality_weights: float | list[float] = 0.5,
quality_function_name: list[str] | None = None,
quality_function_weights: None | list[float] = None,
) -> None:
super().__init__(
n_targets, n_select_from, location_offset, priority_distribution, radius
)
self.quality_weights = quality_weights
if quality_function_name is None:
self.quality_function_name = [
"quality_off_nadir_angle",
"quality_periodic",
]
self.quality_function_name = deepcopy(quality_function_name)
self.quality_function_weights = quality_function_weights
if self.quality_function_weights is not None:
assert len(self.quality_function_name) == len(quality_function_weights), (
"quality_function_weights must match quality_functions"
)
self.quality_function_weights = np.array(quality_function_weights) / np.sum(
np.array(quality_function_weights)
)
def regenerate_targets(self) -> None:
super().regenerate_targets()
for tgt in self.targets:
if isinstance(self.quality_weights, (list, tuple, np.ndarray)):
quality_weight = np.random.uniform(
self.quality_weights[0], self.quality_weights[1]
)
else:
quality_weight = self.quality_weights
quality_function = np.random.choice(
self.quality_function_name, p=self.quality_function_weights
)
quality_function_params = get_randomized_quality_function_params(
quality_function
)
tgt.quality_weight = quality_weight
tgt.quality_function_name = quality_function
tgt.quality_function_params = quality_function_params
tgt.highest_quality_obtained = None
tgt.highest_quality_time = None
tgt.highest_reward_obtained = 0.0
tgt.last_img_time_per_sat = {}
Rewarding physical captures
QualityImageStore detects a completed acquisition from an increase in stored instrument data and evaluates quality using the target’s assigned model and the satellite state at capture time. Requested acquisition time and realized capture time are recorded separately because pointing constraints can delay capture.
QualityImageReward converts these acquisition events into step rewards. With incremental_score=False, a target is rewarded only for its first completed acquisition. With incremental_score=True, a capture earns \(\max(r_i(t)-r_{i,\mathrm{max}},0)\), where \(r_{i,\mathrm{max}}\) is the best absolute acquisition reward completed before that capture. Pending intents affect predicted marginal profiles, but do not increase the completed-reward baseline used for actual reward
allocation.
[4]:
class QualityImageData(UniqueImageData):
def __init__(self, imaged=None, duplicates=0, known=None, image_events=None):
super().__init__(imaged=imaged, duplicates=duplicates, known=known)
if image_events is None:
image_events = []
self.image_events = [event.copy() for event in image_events]
def __add__(self, other: "QualityImageData") -> "QualityImageData":
imaged = self.imaged | other.imaged
image_events = self.image_events + other.image_events
duplicates = (
self.duplicates
+ other.duplicates
+ len(self.imaged)
+ len(other.imaged)
- len(imaged)
)
known = self.known | other.known
return self.__class__(
imaged=imaged,
duplicates=duplicates,
known=known,
image_events=image_events,
)
def get_quality(sat, tgt, t=None, **function_context):
"""Evaluate the quality function assigned to a target."""
if t is None:
t = sat.simulator.sim_time
quality_function = tgt.quality_function_name
quality_function_params = {
**tgt.quality_function_params,
**function_context,
}
return QUALITY_FUNCTIONS[quality_function](
sat,
tgt,
t=t,
**quality_function_params,
)
class QualityImageStore(UniqueImageStore):
"""Record quality-aware target acquisitions, including repeated captures."""
data_type = QualityImageData
def __init__(
self,
*args,
record_image_profiles: bool = False,
image_profile_samples: int = 25,
**kwargs,
) -> None:
"""Initialize image-quality logging options.
Args:
record_image_profiles: Sample quality over the acquisition access window.
This is intended for post-test visualization and is disabled by default
to avoid adding work to training environments.
image_profile_samples: Number of evenly spaced samples in each profile.
"""
if image_profile_samples < 2:
raise ValueError("image_profile_samples must be at least 2")
self.record_image_profiles = record_image_profiles
self.image_profile_samples = image_profile_samples
super().__init__(*args, **kwargs)
def _access_window(self, target: "Target", time: float) -> tuple[float, float]:
"""Return a containing window, otherwise the nearest known window or NaNs."""
candidates = []
for opportunity in getattr(self.satellite, "opportunities", []):
if opportunity.get("type") != "target":
continue
opportunity_target = opportunity.get("object")
if opportunity_target is not target and getattr(
opportunity_target, "id", None
) != getattr(target, "id", None):
continue
window = opportunity.get("window")
if window is None or len(window) != 2:
continue
window_open, window_close = map(float, window)
if window_open <= time <= window_close:
return window_open, window_close
candidates.append((window_open, window_close))
if candidates:
return min(
candidates,
key=lambda window: min(
abs(time - window[0]),
abs(time - window[1]),
),
)
return np.nan, np.nan
def _requested_imaging_time(self) -> float:
"""Read the controller's absolute requested imaging time, when available."""
try:
return float(self.satellite.fsw.insControl.imagingTime) * 1e-9
except (AttributeError, TypeError, ValueError):
return np.nan
def _quality_profile(
self,
target: "Target",
window_open: float,
window_close: float,
) -> tuple[np.ndarray, np.ndarray]:
"""Sample raw target quality over an access window."""
if (
not self.record_image_profiles
or not np.all(np.isfinite([window_open, window_close]))
or window_close <= window_open
or not all(
hasattr(target, attribute)
for attribute in (
"quality_function_name",
"quality_function_params",
)
)
):
return np.asarray([], dtype=float), np.asarray([], dtype=float)
profile_times = np.linspace(
window_open,
window_close,
self.image_profile_samples,
)
profile_qualities = np.asarray(
get_quality(
self.satellite,
target,
t=profile_times,
w_o=window_open,
w_c=window_close,
),
dtype=float,
)
if profile_qualities.shape != profile_times.shape or not np.all(
np.isfinite(profile_qualities)
):
return np.asarray([], dtype=float), np.asarray([], dtype=float)
return profile_times, np.clip(profile_qualities, 0.0, 1.0)
def get_log_state(self) -> float:
"""Log the instantaneous storage unit state at the end of each step.
Returns:
float: storedData from satellite storage unit
"""
msg = self.satellite.dynamics.storageUnit.storageUnitDataOutMsg.read()
return msg.storedData[0]
def compare_log_states(
self, old_state: float, new_state: float
) -> QualityImageData:
"""Record an acquisition when stored instrument data increases.
Args:
old_state: Previous scalar stored-data reading.
new_state: Current scalar stored-data reading.
Returns:
QualityImageData containing the captured target and acquisition event,
or an empty record when storage does not increase. Repeated captures
of the same target are recorded as separate events.
"""
data_increase = new_state - old_state
if data_increase <= 0:
return QualityImageData()
else:
assert self.satellite.latest_target is not None
target = self.satellite.latest_target
self.update_target_colors([target])
image_time = float(self.satellite.simulator.sim_time)
window_open, window_close = self._access_window(target, image_time)
quality = get_quality(
self.satellite,
target,
t=image_time,
w_o=window_open,
w_c=window_close,
)
target.last_quality_obtained = quality
target.last_quality_time = image_time
if (
target.highest_quality_obtained is None
or quality > target.highest_quality_obtained
):
target.highest_quality_obtained = quality
target.highest_quality_time = image_time
target.last_img_time_per_sat[self.satellite.name] = image_time
profile_times, profile_qualities = self._quality_profile(
target,
window_open,
window_close,
)
image_event = {
"time": image_time,
"requested_time": self._requested_imaging_time(),
"target_id": str(target.id),
"target_name": str(target.name),
"satellite_name": str(self.satellite.name),
"priority": float(target.priority),
"quality": float(quality),
"quality_function_name": str(
getattr(target, "quality_function_name", "legacy_distance")
),
"quality_function_params": dict(
getattr(target, "quality_function_params", {})
),
"access_window_open": window_open,
"access_window_close": window_close,
"imaging_time_fraction": (image_time - window_open)
/ (window_close - window_open),
"profile_times": profile_times,
"profile_qualities": profile_qualities,
"quality_weight": np.nan,
"reward": 0.0,
"rewarded": False,
}
return QualityImageData(
imaged={target},
image_events=[image_event],
)
class QualityImageReward(UniqueImageReward):
"""Reward initial captures or positive re-imaging improvements."""
data_store_type = QualityImageStore
def __init__(
self,
reward_fn: Callable,
record_image_profiles: bool = False,
image_profile_samples: int = 25,
incremental_score: bool = False,
):
"""Initialize the rewarder.
Args:
reward_fn: Callback evaluated as
reward_fn(priority, quality, quality_weight=target.quality_weight).
The quality weight is supplied by each target.
record_image_profiles: Sample target quality over each successful image's
access window for post-test visualization.
image_profile_samples: Number of samples stored in each quality profile.
incremental_score: If True, reward only positive improvements over
the best completed reward for each target. If False, reward only
the first completed acquisition of each target.
"""
super().__init__(reward_fn)
self.data_store_kwargs.update(
record_image_profiles=record_image_profiles,
image_profile_samples=image_profile_samples,
)
self.incremental_score = incremental_score
def create_data_store(self, satellite) -> None:
"""Create the store, bypassing the unique-target filter for re-imaging."""
if self.incremental_score:
GlobalReward.create_data_store(self, satellite)
else:
super().create_data_store(satellite)
def _update_event_reward_info(
self,
event: dict[str, Any],
target: "Target",
quality_weight: float,
reward: float,
rewarded: bool,
) -> None:
"""Attach reward information to an acquisition event in place."""
event["quality_weight"] = float(quality_weight)
event["reward"] = float(reward)
event["rewarded"] = rewarded
profile_qualities = np.asarray(event["profile_qualities"], dtype=float)
if profile_qualities.size == 0:
event["profile_rewards"] = np.asarray([], dtype=float)
return
try:
profile_rewards = np.asarray(
self.reward_fn(
target.priority,
profile_qualities,
quality_weight=quality_weight,
),
dtype=float,
)
if profile_rewards.ndim == 0:
profile_rewards = np.full_like(profile_qualities, profile_rewards)
except (TypeError, ValueError):
profile_rewards = np.asarray(
[
self.reward_fn(
target.priority,
quality,
quality_weight=quality_weight,
)
for quality in profile_qualities
],
dtype=float,
)
event["profile_rewards"] = profile_rewards
def calculate_reward(
self, new_data_dict: dict[str, QualityImageData]
) -> dict[str, float]:
"""Allocate step rewards for completed acquisitions.
Evaluate reward_fn(priority, quality, quality_weight=target.quality_weight).
Nonincremental scoring rewards only the first acquisition of a target;
incremental scoring rewards positive improvements over its best completed
reward. For simultaneous captures, allocate reward among satellites tied
for the highest realized quality and give the others zero.
Args:
new_data_dict: Acquisition data keyed by satellite name.
Returns:
Dictionary mapping each satellite name to its total reward this step.
"""
event_per_target = {}
targets_by_id = {}
for sat_id, new_data in new_data_dict.items():
targets_by_id.update({str(target.id): target for target in new_data.imaged})
for event in new_data.image_events:
target_id = event["target_id"]
if target_id not in event_per_target:
event_per_target[target_id] = []
event_per_target[target_id].append((sat_id, event))
reward = {sat_id: 0.0 for sat_id in new_data_dict}
for tgt_id, events in event_per_target.items():
target = targets_by_id.get(tgt_id)
if target in self.data.imaged and not self.incremental_score:
for sat_id, event in events:
self._update_event_reward_info(
event,
target,
target.quality_weight,
reward=0.0,
rewarded=False,
)
continue
time_list = np.array([event["time"] for _, event in events])
time_events = set(time_list)
time_events_sorted = sorted(time_events)
for time_event in time_events_sorted:
events_at_time = [
(sat_id, event)
for sat_id, event in events
if event["time"] == time_event
]
max_quality = max(event["quality"] for _, event in events_at_time)
sat_ids_max_quality = [
sat_id
for sat_id, event in events_at_time
if event["quality"] == max_quality
]
scaling_conflict_score = {
sat_id: (
1.0 / len(sat_ids_max_quality)
if sat_id in sat_ids_max_quality
else 0.0
)
for sat_id, _ in events_at_time
}
if self.incremental_score:
if target.highest_reward_obtained is None:
highest_reward_obtained = 0
else:
highest_reward_obtained = target.highest_reward_obtained
highest_new_reward = None
for sat_id, event in events_at_time:
quality = event.get("quality", 0.0)
step_reward = self.reward_fn(
target.priority, quality, quality_weight=target.quality_weight
)
if self.incremental_score:
if (
highest_new_reward is None
or step_reward > highest_new_reward
):
highest_new_reward = step_reward
step_reward = max(0.0, step_reward - highest_reward_obtained)
step_reward = step_reward * scaling_conflict_score.get(sat_id, 0.0)
reward[sat_id] += step_reward
self._update_event_reward_info(
event,
target,
target.quality_weight,
step_reward,
rewarded=step_reward > 0.0,
)
if (
self.incremental_score
and highest_new_reward > highest_reward_obtained
):
target.highest_reward_obtained = highest_new_reward
target.highest_reward_time = time_event
return reward
Satellite and observations
Following the paper’s observation representation, a quadratic B-spline approximates the estimated marginal reward over each remaining observation window. Its five coefficients are passed to the agent.
The observation has 32 satellite/context features (12 spacecraft properties and 20 priority-density bins), followed by 40 target tokens with 12 features each: five reward-spline coefficients, relative position (3), pointing angle, angular rate, and opening and closing offsets. Both the spline and action wrapper parameterize the remaining window, using the current time as the start when the window is already open. The action wrapper enforces a minimum delay of 0.5 seconds. Opening offsets in the observation remain signed and can therefore be negative for an already-open window.
PriorityDensity is a custom observation class that summarizes future imaging opportunities in 20 consecutive 180-second bins. Each target contributes its priority, divided by five, to the bin containing the closing time of its next available window. This provides context about upcoming opportunities beyond the 40 targets represented individually.
TargetBookkeeping stores the best completed reward known for each target and, when the communication mode uses them, competing intended rewards. Its combined baseline is the larger of the completed and intended values. An intent changes the expected marginal benefit of an acquisition; it does not count as a completed image or increase the obtained reward.
estimate_quality_profile samples absolute predicted reward at 25 times over the remaining window, subtracts the satellite’s current bookkeeping baseline, and clips the result to [0, 1] before fitting the spline. With empty bookkeeping, the baseline is zero. Absolute predictions are cached, but the baseline is read on every call. The coordination loop updates bookkeeping and clears the observation cache before requesting a new observation, so the spline values change while the observation
dimensions remain fixed.
[5]:
class PriorityDensity(obs.Observation):
"""Priority sums in twenty future 180-second opportunity-closing bins."""
def __init__(self):
super().__init__(name="priority_density")
def get_obs(self):
now = self.simulator.sim_time
self.satellite.calculate_additional_windows(
now + 21 * 180 - self.satellite.window_calculation_time
)
upcoming = self.satellite.upcoming_opportunities_dict(types="target")
density = np.zeros(20)
for target, windows in upcoming.items():
index = int(np.floor((windows[0][1] - now) / 180))
if 0 <= index < len(density):
density[index] += target.priority / 5.0
return density
class TargetBookkeeping:
def __init__(self):
self._reset()
def _reset(self):
self.bookkeeping = {}
self.bookkeeping_imaged = {}
self.bookkeeping_intended = {}
def update(
self,
tgt_id: str,
obtained_reward: float,
time: float,
) -> None:
if tgt_id not in self.bookkeeping_imaged:
self.bookkeeping_imaged[tgt_id] = {"reward": obtained_reward, "time": time}
else:
if obtained_reward > self.bookkeeping_imaged[tgt_id]["reward"]:
self.bookkeeping_imaged[tgt_id] = {
"reward": obtained_reward,
"time": time,
}
self._update()
def update_intended(
self,
tgt_id: str,
expected_reward: float,
time: float,
) -> None:
if (
tgt_id in self.bookkeeping_imaged
and expected_reward <= self.bookkeeping_imaged[tgt_id]["reward"]
):
return
if tgt_id not in self.bookkeeping_intended:
self.bookkeeping_intended[tgt_id] = {
"reward": expected_reward,
"time": time,
}
else:
if expected_reward > self.bookkeeping_intended[tgt_id]["reward"]:
self.bookkeeping_intended[tgt_id] = {
"reward": expected_reward,
"time": time,
}
self._update()
def delete_intended(self):
self.bookkeeping_intended = {}
self._update()
def _update(self):
self.bookkeeping = copy(self.bookkeeping_imaged)
tgt_ids_imaged = set(self.bookkeeping_imaged.keys())
tgt_ids_intended = set(self.bookkeeping_intended.keys())
for tgt_id in tgt_ids_intended:
if tgt_id not in tgt_ids_imaged:
self.bookkeeping[tgt_id] = self.bookkeeping_intended[tgt_id]
else:
if (
self.bookkeeping_intended[tgt_id]["reward"]
> self.bookkeeping_imaged[tgt_id]["reward"]
):
self.bookkeeping[tgt_id] = self.bookkeeping_intended[tgt_id]
def get_reward(self, tgt_id: str) -> float:
if tgt_id not in self.bookkeeping:
return 0.0
else:
return self.bookkeeping[tgt_id]["reward"]
def _quality_profile_context(sat):
"""Cache trajectory data shared by all quality profiles at one simulation step."""
trajectory = sat.trajectory
cache_key = (
id(trajectory),
sat.simulator.sim_time,
trajectory.sim_time,
len(trajectory.times),
)
cache = getattr(sat, "_quality_profile_context", None)
if cache is None or cache["key"] != cache_key:
cache = {
"key": cache_key,
"trajectory_r_BP_P": trajectory.r_BP_P,
"full_reward_profiles": {},
}
sat._quality_profile_context = cache
return cache
def estimate_absolute_reward(sat, tgt, times):
"""Predict acquisition reward without completed or intended reward baselines.
``times`` contains absolute simulator times. Use the same trajectory and
quality assumptions as the observation profiles, but do not infer reward
from their clipped marginal splines.
"""
profile_context = _quality_profile_context(sat)
raw_quality = QUALITY_FUNCTIONS[tgt.quality_function_name](
sat,
tgt,
np.asarray(times, dtype=float),
**tgt.quality_function_params,
trajectory_r_BP_P=profile_context["trajectory_r_BP_P"],
)
return np.asarray(
reward_function(tgt.priority, raw_quality, quality_weight=tgt.quality_weight),
dtype=float,
)
def estimate_quality_profile(sat, opp, return_rewards: bool = False):
tgt = opp["object"]
window = opp["window"]
window_open = np.max([window[0], sat.simulator.sim_time])
window_close = window[1]
profile_context = _quality_profile_context(sat)
profile_key = (
str(tgt.id),
float(window_open),
float(window_close),
)
profile = profile_context["full_reward_profiles"].get(profile_key)
if profile is None:
times = np.linspace(window_open, window_close, 25)
full_reward = estimate_absolute_reward(sat, tgt, times)
profile = {
"times": times,
"full_reward": full_reward,
}
profile_context["full_reward_profiles"][profile_key] = profile
obtained_reward = (
sat.bookkeeping.get_reward(tgt.id) if hasattr(sat, "bookkeeping") else 0.0
)
marginal_reward = np.clip(
profile["full_reward"] - obtained_reward,
a_min=0.0,
a_max=1.0,
)
if return_rewards:
return marginal_reward
# Normalize sample times to [0, 1] over the remaining access window.
x_vector = (profile["times"] - window_open) / (window_close - window_open)
spline = make_lsq_spline(x_vector, marginal_reward, t=SPLINE_KNOTS, k=SPLINE_DEGREE)
return spline.c
Define the satellite
QualitySatellite combines the spacecraft dynamics and flight software with the observations defined above. Its observation specification includes spacecraft properties, PriorityDensity, and the next N_AHEAD target opportunities. The opportunity callback uses estimate_quality_profile to supply the five reward-spline coefficients alongside each target’s geometry and window offsets. Each satellite has its own bookkeeping, which is cleared when the environment resets.
The action specification uses act.Image(n_ahead_image=N_AHEAD, hybrid=True). This enables a native imaging action containing a target-opportunity index and a desired acquisition delay in seconds. The wrappers below expose normalized timing fractions and, optionally, a categorical action over target/timing pairs. satellite_args supplies the orbit, pointing constraints, timing tolerances, and spacecraft resource settings shared by training and evaluation.
[6]:
from functools import partial
class QualitySatellite(sats.ImagingSatellite):
dyn_type = dyn.FullFeaturedDynModel
fsw_type = fsw.SteeringImagerFSWModel
observation_spec: ClassVar = [
obs.SatProperties(
dict(prop="omega_BN_B", norm=0.03),
dict(prop="c_hat_H"),
dict(prop="r_BN_P", norm=EARTH_RADIUS),
dict(prop="v_BN_P", norm=7616.5),
),
PriorityDensity(),
obs.OpportunityProperties(
dict(fn=partial(estimate_quality_profile)),
dict(prop="r_LB_H", norm=EARTH_RADIUS),
dict(prop="target_angle", norm=np.pi / 2),
dict(prop="target_angle_rate", norm=0.03),
dict(prop="opportunity_open", norm=300),
dict(prop="opportunity_close", norm=300),
n_ahead_observe=N_AHEAD,
),
]
action_spec: ClassVar = [act.Image(n_ahead_image=N_AHEAD, hybrid=True)]
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.bookkeeping = TargetBookkeeping()
def reset_pre_sim_init(self) -> None:
super().reset_pre_sim_init()
self.bookkeeping._reset()
def satellite_args():
return QualitySatellite.default_sat_args(
oe=lambda: random_circular_orbit(alt=ALTITUDE_KM, i=45),
imageAttErrorRequirement=0.01,
imageRateErrorRequirement=0.01,
imageTargetMinimumElevation=np.arctan(800 / 500),
timeToleranceLower=1.0, # Seconds; BSK-RL converts to nanoseconds.
timeToleranceUpper=0.0, # Zero disables the upper bound, allowing late capture.
u_max=0.4,
K1=0.25,
K3=3.0,
servo_P=30,
omega_max=np.radians(5),
rwBasePower=20,
disturbance_vector=np.zeros(3),
batteryStorageCapacity=80 * 3600 * 2 * 1000,
storedCharge_Init=0.99 * 80 * 3600 * 2 * 1000,
maxWheelSpeed=15000,
wheelSpeeds=lambda: np.random.uniform(-1, 1, 3),
dataStorageCapacity=2 * 8e6 * 25 * 1000,
storageInit=0,
transmitterPacketSize=0,
)
Minimal wrappers and environment factory
With hybrid=True and at least one imaging time tolerance enabled, the native imaging action accepts a target index and a desired acquisition delay in seconds. This delay specifies when to try to acquire the image; it is not the exposure duration. If both tolerances are zero, the continuous value instead specifies the maximum imaging action duration in seconds, capped by max_duration and with a minimum of one simulator time step. The wrappers let a policy select a target and a normalized
position within its remaining observation window.
From a discrete option to a target and timing fraction
Let \(N\) be the number of upcoming target opportunities and let \(F=(F_0,\ldots,F_{D-1})\) contain \(D\) predefined timing fractions. The policy selects one categorical action \(a\in\{0,\ldots,ND-1\}\). WrapperDiscreteToHybridAction decodes it as
\[i=\left\lfloor\frac{a}{D}\right\rfloor,\qquad
k=a\bmod D,\qquad f=F_k.\]
Here \(i\) is the zero-based target-opportunity index and \(k\) is the timing-option index. The inverse mapping is \(a=iD+k\). Timing options vary fastest within each target. With N_AHEAD=40 and IMAGING_FRACTIONS=(0.0, 0.5, 1.0), each satellite has Discrete(120):
Discrete action |
Target index |
Timing fraction |
|---|---|---|
6 |
2 |
0.0 |
7 |
2 |
0.5 |
8 |
2 |
1.0 |
The actor therefore needs a categorical distribution over 120 choices. Although these choices are translated into real-valued delays, the policy only chooses among the three listed fractions; it does not output an arbitrary continuous timing value.
From a timing fraction to seconds
For current simulation time \(t_{\mathrm{now}}\) and the selected opportunity’s physical window \([t_{\mathrm{open}},t_{\mathrm{close}}]\), define the remaining opening and closing offsets
\[u=\max(0,t_{\mathrm{open}}-t_{\mathrm{now}}),\qquad
v=t_{\mathrm{close}}-t_{\mathrm{now}}.\]
WrapperActionDuration clips \(f\) to \([0,1]\) and returns
\[\Delta t_{\mathrm{img}}=\max\bigl(0.5,\;u+f(v-u)\bigr),\qquad
t_d=t_{\mathrm{now}}+\Delta t_{\mathrm{img}}.\]
The native action receives (i, delta_t_img) and the controller uses the absolute desired acquisition time \(t_d\). For example, if the current time is 100 seconds and target opportunity 2 spans [120, 180] seconds, action 7 selects the midpoint: a 50-second delay and a desired acquisition time of 150 seconds. If the window instead spans [90, 140] seconds, the same fraction selects the midpoint of the remaining window: a 20-second delay and a desired acquisition time of 120 seconds.
Fraction zero requests the earliest remaining time, subject to the 0.5-second minimum delay.
The wrapper stack processes actions in this order:
discrete action a
-> WrapperDiscreteToHybridAction: (target index i, fraction f)
-> WrapperActionDuration: (target index i, delay in seconds)
-> WrapperActionLogging
-> native imaging action: absolute desired time = now + delay
For a constellation, this conversion is applied separately to each supplied entry in the satellite-name action dictionary.
Acquisition tolerances and action limits
Acquisition also requires the pointing-angle and angular-rate constraints to be satisfied. When both timing bounds are enabled, the allowed acquisition times satisfy
\[t_d-\tau_{\mathrm{lower}}\leq t\leq t_d+\tau_{\mathrm{upper}},\]
where the positive tolerances are configured in seconds through timeToleranceLower and timeToleranceUpper. A zero tolerance disables its corresponding bound independently. The current settings, timeToleranceLower=1.0 and timeToleranceUpper=0.0, allow acquisition from one second before \(t_d\) onward, without an instrument upper timing bound. A late acquisition is therefore possible if the pointing requirements are met later. Setting both tolerances to zero removes both timing
gates and makes the continuous action value a maximum imaging duration. The action can finish sooner if an image is captured. The mode is selected once per environment reset from the initialized controller tolerances, which must remain fixed during an episode; enabling either bound emits a warning explaining the capture-time interpretation.
Physical access, action-duration limits, and simulation stepping still apply. A positive upper tolerance also lets the native imaging action limit its timeout using the requested delay plus that tolerance. With the upper tolerance disabled, this example retains the physical window and existing action/environment limits; the fraction does not impose a new upper deadline.
[7]:
from bsk_rl.gym import is_no_action
def clean_action(
action: tuple[int, float | np.ndarray | list],
) -> tuple[int, float]:
if action is None:
return None
discrete = action[0].item() if isinstance(action[0], np.generic) else action[0]
val = action[1][0] if isinstance(action[1], (list, np.ndarray)) else action[1]
continuous = val.item() if isinstance(val, np.generic) else val
return (discrete, continuous)
class WrapperActionDuration(gym.ActionWrapper):
"""Convert a fraction into an acquisition delay over the remaining window.
The returned delay is in seconds and has a minimum of 0.5 seconds.
"""
def __init__(
self,
env,
) -> None:
super().__init__(env)
self.env = env
self._n_ahead = {}
for sat in self.env.satellites:
obs_class = next(
item
for item in sat.observation_spec
if isinstance(item, (obs.OpportunityProperties))
and item.type == "target"
)
self._n_ahead[sat.name] = obs_class.n_ahead_observe
def get_image_window(
self, target_index: int, sat_name: str | None
) -> tuple[float, float]:
if sat_name is not None:
sat = next(sat for sat in self.unwrapped.satellites if sat.name == sat_name)
else:
sat = self.unwrapped.satellite
opps = sat.find_next_opportunities(
n=self._n_ahead[sat.name],
types="target",
pad=True,
)
if isinstance(target_index, (int, np.integer)):
opp = opps[target_index]
else:
opp = [
tgt_i
for tgt_i in opps
if not isinstance(tgt_i["object"], str)
and tgt_i["object"].id == target_index
]
if len(opp) > 0:
opp = opp[0]
else:
for i in range(5):
opps = sat.find_next_opportunities(
n=self._n_ahead[sat.name] * (i + 1),
types="target",
pad=True,
)
opp = [
tgt_i
for tgt_i in opps
if not isinstance(tgt_i["object"], str)
and tgt_i["object"].id == target_index
]
if len(opp) > 0:
opp = opp[0]
break
window_start = opp["window"][0] - self.unwrapped.simulator.sim_time
window_start = np.max([window_start, 0.0])
window_end = opp["window"][1] - self.unwrapped.simulator.sim_time
return (window_start, window_end)
def get_imaging_time(
self, target_index: int, window_fraction: float, sat_name: str | None
) -> float:
window_start, window_end = self.get_image_window(target_index, sat_name)
window_fraction = np.clip(window_fraction, 0.0, 1.0)
return np.clip(
float(window_start + (window_end - window_start) * window_fraction),
0.5,
np.inf,
)
def _set_action_duration(
self, action, sat_name: str | None
) -> str | None | tuple[int, float]:
if isinstance(action, str) or action is None:
return action
action = clean_action(action)
target_index = action[0]
duration = self.get_imaging_time(target_index, action[1], sat_name)
return (action[0], float(duration))
def action(
self, action: tuple[int, float]
) -> tuple[int, float] | dict | str | None:
if isinstance(action, dict):
action_dict = {}
for sat_name, sat_action in action.items():
action_dict[sat_name] = self._set_action_duration(sat_action, sat_name)
return action_dict
else:
return self._set_action_duration(action, None)
class WrapperDiscreteToHybridAction(gym.ActionWrapper, gym.Wrapper):
"""Decode a categorical choice into a target index and imaging fraction.
Timing options vary fastest within each target: action = target * D + timing.
Args:
env: Imaging environment with a native hybrid action space.
discrete_imaging_fraction: Predefined timing fractions for each target.
"""
def __init__(
self,
env,
discrete_imaging_fraction: Iterable[float],
) -> None:
super().__init__(env)
self.discrete_imaging_fraction = list(discrete_imaging_fraction)
self.discretization = len(self.discrete_imaging_fraction)
self.env = env
if not isinstance(env.action_space, gym.spaces.Tuple):
action_space = env.action_spaces
# Assumes all satellites have the same action space
self.n_hybrid_actions = action_space[env.satellites[0].name][0].n
else:
action_space = env.action_space[0]
self.n_hybrid_actions = action_space.n
self.n_discrete_actions = self.n_hybrid_actions * self.discretization
space = gym.spaces.Discrete(self.n_discrete_actions)
if isinstance(env.action_space, gym.spaces.Tuple):
self.action_space = space
else:
self.action_space = lambda sat_name: space
def _split_action(self, action):
if action is None or is_no_action(action):
return None
action = int(action)
if not 0 <= action < self.n_discrete_actions:
raise ValueError(
f"Action index {action} out of range for {self.n_discrete_actions} choices."
)
target_index, continuous_action = divmod(action, self.discretization)
return (target_index, self.discrete_imaging_fraction[continuous_action])
def action(
self, action: int | dict
) -> (
tuple[int, float]
| tuple[str, float]
| dict[str, tuple[int, float]]
| dict[str, tuple[str, float]]
| None
):
if isinstance(action, dict):
action_dict = {}
for sat_name, sat_action in action.items():
action_dict[sat_name] = self._split_action(sat_action)
return action_dict
else:
return self._split_action(action)
class WrapperActionLogging(gym.Wrapper):
def __init__(self, env):
super().__init__(env)
self.env = env
if not isinstance(env.action_space, gym.spaces.Tuple):
action_space = env.action_spaces
self.n_discrete = action_space[env.satellites[0].name][0].n
else:
action_space = env.action_space
self.n_discrete = action_space[0].n
self._initialize_action_logger()
def _initialize_action_logger(self):
self.action_duration_logger_list = {
sat.name: {
"duration_image": [],
}
for sat in self.env.satellites
}
self.action_log_discrete_count = {i: 0 for i in range(self.n_discrete)}
self.action_duration_log = {i: 0.0 for i in range(self.n_discrete)}
def return_loggers(self):
duration_list_image_all = [
self.action_duration_logger_list[sat.name]["duration_image"]
for sat in self.env.satellites
]
return {
"action_log_discrete_count": self.action_log_discrete_count,
"action_duration_log": self.action_duration_log,
"action_duration_logger_list": self.action_duration_logger_list,
"action_duration_image_list_all": np.concatenate(duration_list_image_all),
}
def reset(self, *, seed: int | None = None, options: dict[str, Any] | None = None):
self._initialize_action_logger()
return self.env.reset(seed=seed, options=options)
def _store_action_info(self, action: tuple[int, float], sat_name: str):
action = clean_action(action)
if action is None:
return
self.action_duration_logger_list[sat_name]["duration_image"].append(action[1])
if isinstance(action[0], str):
return
discrete_action = int(action[0])
self.action_log_discrete_count[discrete_action] += 1
self.action_duration_log[discrete_action] += (
action[1] - self.action_duration_log[discrete_action]
) / self.action_log_discrete_count[discrete_action]
def step(self, action: dict | tuple[int, float]):
if isinstance(action, dict):
for sat_name, sat_action in action.items():
self._store_action_info(sat_action, sat_name)
else:
self._store_action_info(action, self.env.satellite.name)
return self.env.step(action)
ENV_CONFIG = dict(
n_targets=(100, 3000),
orbits=3,
quality_weight=(0.0, 1.0), # Scalar for a common weight; 0.0 disables quality.
quality_types=("quality_off_nadir_angle", "quality_periodic"),
)
def reward_function(priority, quality, quality_weight=0.5):
quality_weight = np.clip(quality_weight, 0.0, 1.0)
quality = (1 - quality_weight) + quality_weight * np.clip(
quality, a_min=0.0, a_max=1.0
)
reward = priority * quality
return reward
Create consistent training and evaluation environments
make_env combines the target scenario, rewarder, and simulation settings from a common configuration. In evaluation mode it returns a ConstellationTasking environment with the requested number of satellites, including a per-satellite dictionary interface when only one satellite is used. With train=True, it returns the construction arguments for a single-satellite SatelliteTasking environment; the RLlib wrapper below uses these arguments to instantiate the training environment.
setup_wrapper_stack then adds duration logging and converts a normalized imaging fraction into a delay in seconds. Setting discrete_action_space=True also adds the outer wrapper that decodes an integer into a target and one of the supplied timing fractions. Keeping environment construction and action conversion in these helpers makes the settings and action convention explicit when creating a training or evaluation instance.
[8]:
def make_env(
env_config=None,
n_sats=1,
incremental_score=False,
n_targets: int | None = None,
train: bool = False,
time_limit: float | None = None,
):
config = {**ENV_CONFIG, **dict(env_config or {})}
env_config = dict(
scenario=QualityTargets(
n_targets=config["n_targets"] if n_targets is None else n_targets,
quality_weights=config["quality_weight"],
quality_function_name=config["quality_types"],
),
rewarder=QualityImageReward(
reward_fn=reward_function, incremental_score=incremental_score
),
sim_rate=SIM_RATE,
max_step_duration=500.0,
time_limit=config["orbits"] * ORBIT_PERIOD
if time_limit is None
else time_limit,
failure_penalty=0.0,
log_level="INFO",
)
if train:
# Training uses a single satellite configuration
env_config.update(
dict(satellite=QualitySatellite("EO", sat_args=satellite_args()))
)
return env_config
else:
env = ConstellationTasking(
satellites=[
QualitySatellite(f"EO_{i}", sat_args=satellite_args())
for i in range(n_sats)
],
**env_config,
)
return env
def setup_wrapper_stack(
env,
discrete_action_space=False,
discrete_imaging_fraction=None,
):
env = WrapperActionLogging(env)
env = WrapperActionDuration(env)
if discrete_action_space:
env = WrapperDiscreteToHybridAction(
env,
discrete_imaging_fraction=discrete_imaging_fraction,
)
return env
Try a few imaging actions
This short rollout selects target opportunity 0, the first upcoming opportunity, and requests the midpoint of its remaining access window with (0, 0.5). It uses discrete_action_space=False, so the action contains a target index and a timing fraction directly. The action is supplied in a dictionary keyed by satellite name, and the duration wrapper converts the fraction into seconds. This exercises the physical environment without RLlib or a trained checkpoint. Increase the loop length to
run a longer rollout.
[9]:
env = make_env()
env = setup_wrapper_stack(
env,
discrete_action_space=False,
discrete_imaging_fraction=IMAGING_FRACTIONS,
)
observation, info = env.reset(seed=3)
try:
for _ in range(5):
# Each satellite receives (target index, window fraction).
# With the discrete wrapper, each dictionary value is an action index.
action = {env.satellites[0].name: (0, 0.5)}
observation, reward, terminated, truncated, info = env.step(action)
if all(terminated.values()) or all(truncated.values()):
break
finally:
env.close()
2026-10-26 21:02:03,844 gym INFO Resetting environment with seed=3
2026-10-26 21:02:03,846 scene.targets INFO Generating 1742 targets
2026-10-26 21:02:04,129 sats.satellite.EO_0 INFO <0.00> EO_0: Finding opportunity windows from 0.00 to 18600.00 seconds
2026-10-26 21:02:04,419 gym INFO <0.00> Environment reset
2026-10-26 21:02:04,421 gym INFO <0.00> === STARTING STEP ===
2026-10-26 21:02:04,422 sats.satellite.EO_0 INFO <0.00> EO_0: target index 0 tasked
2026-10-26 21:02:04,422 sats.satellite.EO_0 INFO <0.00> EO_0: Target(Villa Aberastain, AR) imaging timing: requested capture at 417.2
2026-10-26 21:02:04,423 sats.satellite.EO_0 INFO <0.00> EO_0: Target(Villa Aberastain, AR) tasked for imaging
2026-10-26 21:02:04,424 sats.satellite.EO_0 INFO <0.00> EO_0: Target(Villa Aberastain, AR) window enabled: 374.8 to 459.7
2026-10-26 21:02:04,425 sats.satellite.EO_0 INFO <0.00> EO_0: setting timed terminal event at 459.7
2026-10-26 21:02:04,503 sats.satellite.EO_0 INFO <417.00> EO_0: imaged Target(Villa Aberastain, AR)
2026-10-26 21:02:04,505 data.base INFO <417.00> Total reward: {'EO_0': np.float64(0.024999692474987952)}
2026-10-26 21:02:04,506 comm.communication INFO <417.00> Optimizing data communication between all pairs of satellites
2026-10-26 21:02:04,506 sats.satellite.EO_0 INFO <417.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:04,537 gym INFO <417.00> Step reward: {'EO_0': np.float64(0.024999692474987952)}
2026-10-26 21:02:04,538 gym INFO <417.00> === STARTING STEP ===
2026-10-26 21:02:04,539 sats.satellite.EO_0 INFO <417.00> EO_0: target index 0 tasked
2026-10-26 21:02:04,542 sats.satellite.EO_0 INFO <417.00> EO_0: Target(Recreo, AR) imaging timing: requested capture at 458.8
2026-10-26 21:02:04,544 sats.satellite.EO_0 INFO <417.00> EO_0: Target(Recreo, AR) tasked for imaging
2026-10-26 21:02:04,547 sats.satellite.EO_0 INFO <417.00> EO_0: Target(Recreo, AR) window enabled: 366.0 to 500.7
2026-10-26 21:02:04,547 sats.satellite.EO_0 INFO <417.00> EO_0: setting timed terminal event at 500.7
2026-10-26 21:02:04,561 sats.satellite.EO_0 INFO <461.00> EO_0: imaged Target(Recreo, AR)
2026-10-26 21:02:04,562 data.base INFO <461.00> Total reward: {'EO_0': np.float64(0.49907471357643646)}
2026-10-26 21:02:04,563 comm.communication INFO <461.00> Optimizing data communication between all pairs of satellites
2026-10-26 21:02:04,564 sats.satellite.EO_0 INFO <461.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:04,594 gym INFO <461.00> Step reward: {'EO_0': np.float64(0.49907471357643646)}
2026-10-26 21:02:04,595 gym INFO <461.00> === STARTING STEP ===
2026-10-26 21:02:04,596 sats.satellite.EO_0 INFO <461.00> EO_0: target index 0 tasked
2026-10-26 21:02:04,596 sats.satellite.EO_0 INFO <461.00> EO_0: Target(Huinca Renancó, AR) imaging timing: requested capture at 500.4
2026-10-26 21:02:04,597 sats.satellite.EO_0 INFO <461.00> EO_0: Target(Huinca Renancó, AR) tasked for imaging
2026-10-26 21:02:04,598 sats.satellite.EO_0 INFO <461.00> EO_0: Target(Huinca Renancó, AR) window enabled: 467.5 to 533.3
2026-10-26 21:02:04,598 sats.satellite.EO_0 INFO <461.00> EO_0: setting timed terminal event at 533.3
2026-10-26 21:02:04,608 sats.satellite.EO_0 INFO <510.00> EO_0: imaged Target(Huinca Renancó, AR)
2026-10-26 21:02:04,608 data.base INFO <510.00> Total reward: {'EO_0': np.float64(0.3474319715528675)}
2026-10-26 21:02:04,609 comm.communication INFO <510.00> Optimizing data communication between all pairs of satellites
2026-10-26 21:02:04,610 sats.satellite.EO_0 INFO <510.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:04,642 gym INFO <510.00> Step reward: {'EO_0': np.float64(0.3474319715528675)}
2026-10-26 21:02:04,643 gym INFO <510.00> === STARTING STEP ===
2026-10-26 21:02:04,644 sats.satellite.EO_0 INFO <510.00> EO_0: target index 0 tasked
2026-10-26 21:02:04,645 sats.satellite.EO_0 INFO <510.00> EO_0: Target(Romang, AR) imaging timing: requested capture at 525.3
2026-10-26 21:02:04,645 sats.satellite.EO_0 INFO <510.00> EO_0: Target(Romang, AR) tasked for imaging
2026-10-26 21:02:04,646 sats.satellite.EO_0 INFO <510.00> EO_0: Target(Romang, AR) window enabled: 463.8 to 540.6
2026-10-26 21:02:04,647 sats.satellite.EO_0 INFO <510.00> EO_0: setting timed terminal event at 540.6
2026-10-26 21:02:04,657 sats.satellite.EO_0 INFO <541.00> EO_0: timed termination at 540.6 for Target(Romang, AR) window
2026-10-26 21:02:04,658 data.base INFO <541.00> Total reward: {}
2026-10-26 21:02:04,658 comm.communication INFO <541.00> Optimizing data communication between all pairs of satellites
2026-10-26 21:02:04,659 sats.satellite.EO_0 INFO <541.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:04,686 gym INFO <541.00> Step reward: {}
2026-10-26 21:02:04,687 gym INFO <541.00> === STARTING STEP ===
2026-10-26 21:02:04,687 sats.satellite.EO_0 INFO <541.00> EO_0: target index 0 tasked
2026-10-26 21:02:04,688 sats.satellite.EO_0 INFO <541.00> EO_0: Target(Rufino, AR) imaging timing: requested capture at 556.2
2026-10-26 21:02:04,688 sats.satellite.EO_0 INFO <541.00> EO_0: Target(Rufino, AR) tasked for imaging
2026-10-26 21:02:04,689 sats.satellite.EO_0 INFO <541.00> EO_0: Target(Rufino, AR) window enabled: 457.1 to 571.5
2026-10-26 21:02:04,690 sats.satellite.EO_0 INFO <541.00> EO_0: setting timed terminal event at 571.5
2026-10-26 21:02:04,700 sats.satellite.EO_0 INFO <571.50> EO_0: timed termination at 571.5 for Target(Rufino, AR) window
2026-10-26 21:02:04,701 data.base INFO <571.50> Total reward: {}
2026-10-26 21:02:04,701 comm.communication INFO <571.50> Optimizing data communication between all pairs of satellites
2026-10-26 21:02:04,702 sats.satellite.EO_0 INFO <571.50> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:04,731 gym INFO <571.50> Step reward: {}
Categorical policy over target/timing pairs
Because timing choices are discrete, the policy only needs RLlib’s standard TorchCategorical. The actor produces one logit per target and timing option, then flattens them into the same target-major order as the wrapper. A single softmax represents the joint distribution \(\pi(i,k\mid o)\), so timing preferences can depend on the target without a custom conditional distribution class.
A MultiDiscrete([N, D]) policy with separate categorical factors would impose independence between target and timing choices. The flattened categorical avoids that restriction. A custom distribution would become useful if capture times were continuous and conditioned on the chosen target.
[10]:
from ray.rllib.algorithms.ppo import PPOConfig
from ray.rllib.algorithms.ppo.torch.ppo_torch_rl_module import PPOTorchRLModule
from ray.rllib.core import Columns
from ray.rllib.core.rl_module.rl_module import RLModuleSpec
from ray.rllib.models.torch.torch_distributions import TorchCategorical
from ray.rllib.utils.framework import try_import_torch
from bsk_rl.utils.rllib.discounting import TimeDiscountedGAEPPOTorchLearner
torch, nn = try_import_torch()
2026-10-26 21:02:05,406 INFO util.py:154 -- Missing packages: ['ipywidgets']. Run `pip install -U ipywidgets`, then restart the notebook server for rich notebook output.
2026-10-26 21:02:07,388 INFO util.py:154 -- Missing packages: ['ipywidgets']. Run `pip install -U ipywidgets`, then restart the notebook server for rich notebook output.
Configurable actor, critic, and combined RL module
The following definitions construct a custom actor, which outputs \(ND\) logits, and a custom critic:
Actor: separate target and satellite MLPs; multi-head target self-attention with a residual connection and LayerNorm; mean/max target pooling; and one logit per target/timing pair.
Critic: a separate target MLP, mean/max pooling, and an MLP combining these embeddings with the raw satellite/context features to predict a scalar value.
Both networks accept keyword arguments for their dimensions, depths, activation, and dropout. HybridQualityModule combines them and reads those arguments from RLlib’s model_config_dict; critic arguments use the critic_ prefix. The module checks that its target count and timing discretization agree with the environment. The training forward pass returns logits and values so PPO can evaluate the actions stored in its rollout batch.
[11]:
def build_mlp(
input_size,
width,
depth,
output_size,
*,
act_function="ReLU",
dropout=0.0,
dropout_before_output=0.0,
output_activation=False,
):
"""Configurable hidden layers, preserving the research parameter layout."""
if depth < 0 or width <= 0 or output_size <= 0:
raise ValueError("MLP dimensions must be positive and depth nonnegative.")
activation = getattr(nn, act_function)
layers = []
for _ in range(depth):
layers.extend([nn.Linear(input_size, width), activation()])
if dropout:
layers.append(nn.Dropout(dropout))
input_size = width
if dropout_before_output:
layers.append(nn.Dropout(dropout_before_output))
layers.append(nn.Linear(input_size, output_size))
if output_activation:
layers.append(activation())
return nn.Sequential(*layers)
class TargetSelfAttention(nn.Module):
def __init__(self, input_dim, attention_dim, num_heads):
super().__init__()
if num_heads <= 0 or attention_dim <= 0 or attention_dim % num_heads:
raise ValueError(
"attention_dim must be positive and divisible by num_heads."
)
self.num_heads = num_heads
self.head_dim = attention_dim // num_heads
self.attention_dim = attention_dim
self.qkv = nn.Linear(input_dim, 3 * attention_dim, bias=False)
self.out = nn.Linear(attention_dim, input_dim, bias=False)
def forward(self, tokens):
batch_size, n_targets, _ = tokens.shape
qkv = self.qkv(tokens).view(
batch_size, n_targets, 3, self.num_heads, self.head_dim
)
q, k, v = qkv.permute(2, 0, 3, 1, 4).unbind(dim=0)
attended = nn.functional.scaled_dot_product_attention(
q, k, v, dropout_p=0.0, is_causal=False
)
attended = (
attended.transpose(1, 2)
.contiguous()
.view(batch_size, n_targets, self.attention_dim)
)
return self.out(attended)
class ImagingActor(nn.Module):
def __init__(
self,
target_size,
context_size,
n_times,
*,
width_phi=256,
depth_phi=2,
tgt_encoded_dim=64,
attention_dim=64,
num_heads=2,
width_phi_sat=256,
depth_phi_sat=2,
sat_encoded_dim=64,
width_psi_img=128,
depth_psi_img=4,
act_function="ReLU",
dropout=0.0,
psi_img_dropout_last_layer=0.1,
):
super().__init__()
self.phi = build_mlp(
target_size,
width_phi,
depth_phi,
tgt_encoded_dim,
act_function=act_function,
dropout=dropout,
)
self.self_attention = TargetSelfAttention(
tgt_encoded_dim, attention_dim, num_heads
)
self.norm_self_attention = nn.LayerNorm(tgt_encoded_dim)
self.psi_img = build_mlp(
3 * tgt_encoded_dim + sat_encoded_dim,
width_psi_img,
depth_psi_img,
tgt_encoded_dim,
act_function=act_function,
dropout=dropout,
dropout_before_output=psi_img_dropout_last_layer,
output_activation=True,
)
self.out_layer_psi_img = nn.Linear(tgt_encoded_dim, n_times)
self.phi_sat = build_mlp(
context_size,
width_phi_sat,
depth_phi_sat,
sat_encoded_dim,
act_function=act_function,
dropout=dropout,
)
def forward(self, context, targets):
tokens = self.phi(targets)
satellite = self.phi_sat(context)
tokens = self.norm_self_attention(tokens + self.self_attention(tokens))
n_targets = tokens.shape[1]
mean_targets = tokens.mean(dim=1, keepdim=True).expand(-1, n_targets, -1)
max_targets = tokens.max(dim=1, keepdim=True).values.expand(-1, n_targets, -1)
features = torch.cat(
[
tokens,
mean_targets,
max_targets,
satellite.unsqueeze(1).expand(-1, n_targets, -1),
],
dim=-1,
)
logits = self.out_layer_psi_img(self.psi_img(features))
return logits.flatten(start_dim=1)
class ImagingCritic(nn.Module):
def __init__(
self,
target_size,
context_size,
*,
width_phi=256,
depth_phi=2,
tgt_encoded_dim=128,
width_psi=64,
depth_psi=3,
act_function="ReLU",
dropout=0.0,
):
super().__init__()
self.phi = build_mlp(
target_size,
width_phi,
depth_phi,
tgt_encoded_dim,
act_function=act_function,
dropout=dropout,
)
self.psi = build_mlp(
context_size + 2 * tgt_encoded_dim,
width_psi,
depth_psi,
1,
act_function=act_function,
dropout=dropout,
)
def forward(self, context, targets):
tokens = self.phi(targets)
features = torch.cat(
[context, tokens.mean(dim=1), tokens.max(dim=1).values], dim=-1
)
return self.psi(features).squeeze(-1)
class HybridQualityModule(PPOTorchRLModule):
"""Combine independently configurable actor and critic heads for PPO."""
def setup(self):
config = self.config.model_config_dict
self.n_targets = int(config["n_targets"])
self.context_size = int(config["obs_sat"])
self.n_times = int(config["discretization"])
target_features = self.config.observation_space.shape[0] - self.context_size
if (
self.n_targets <= 0
or self.context_size < 0
or target_features <= 0
or target_features % self.n_targets
):
raise ValueError(
"Observation size must match context plus equal target tokens."
)
self.target_size = target_features // self.n_targets
if (
not isinstance(self.config.action_space, gym.spaces.Discrete)
or self.n_times <= 0
or self.config.action_space.n != self.n_targets * self.n_times
):
raise ValueError("Expected Discrete(n_targets * discretization) actions.")
actor_keys = (
"width_phi",
"depth_phi",
"tgt_encoded_dim",
"attention_dim",
"num_heads",
"width_phi_sat",
"depth_phi_sat",
"sat_encoded_dim",
"width_psi_img",
"depth_psi_img",
"act_function",
"dropout",
"psi_img_dropout_last_layer",
)
self.pi_head = ImagingActor(
self.target_size,
self.context_size,
self.n_times,
**{key: config[key] for key in actor_keys if key in config},
)
self.vf = ImagingCritic(
self.target_size,
self.context_size,
width_phi=config.get("critic_width_phi", 256),
depth_phi=config.get("critic_depth_phi", 2),
tgt_encoded_dim=config.get("critic_tgt_encoded_dim", 128),
width_psi=config.get("critic_width_psi", 64),
depth_psi=config.get("critic_depth_psi", 3),
act_function=config.get("act_function", "ReLU"),
dropout=config.get("critic_dropout", config.get("dropout", 0.0)),
)
self.action_dist_cls = TorchCategorical
self._inference_only_state_dict_keys = {}
def get_initial_state(self):
return {}
def split_observation(self, batch):
values = batch[Columns.OBS].float()
context = values[:, : self.context_size]
targets = values[:, self.context_size :].reshape(
-1, self.n_targets, self.target_size
)
return context, targets
def compute_values(self, batch, embeddings=None):
return self.vf(*self.split_observation(batch))
def _forward_inference(self, batch, **kwargs):
return {
Columns.ACTION_DIST_INPUTS: self.pi_head(*self.split_observation(batch))
}
def _forward_exploration(self, batch, **kwargs):
return self._forward_train(batch)
def _forward_train(self, batch, **kwargs):
context, targets = self.split_observation(batch)
return {
Columns.ACTION_DIST_INPUTS: self.pi_head(context, targets),
Columns.VF_PREDS: self.vf(context, targets),
}
PPO configuration
The configuration below supplies the model dimensions and PPO hyperparameters for three timing options per target. Env_wrapped adapts the custom environment for RLlib: it accepts env_config, constructs the single-satellite environment, applies the action-wrapper stack, and initializes the optional episode-data logging callbacks. Passing the class directly as env=Env_wrapped lets RLlib create each environment instance without registering a separate Gym environment ID.
TimeDiscountedGAEPPOTorchLearner uses the reported step duration d_ts, in seconds, to apply the duration-dependent discount \(\gamma^{\Delta t}\). Here gamma=0.995 is the per-second factor, and the learner discounts raw capture rewards at the end of each step. The evaluation loop accumulates undiscounted rewards, as in the paper’s reported comparisons.
Defining ppo_config does not build or train an algorithm. The inference demonstration at the end of the cell creates a separate wrapped environment and a fresh module with random initial weights, then executes one categorical action. Its output checks the inference interface and does not represent trained-policy performance.
[12]:
from bsk_rl.utils.rllib.callbacks import EpisodeDataLogger
MODEL_CONFIG = {
"n_targets": N_AHEAD,
"obs_sat": 32,
"discretization": len(IMAGING_FRACTIONS),
"width_phi": 256,
"depth_phi": 2,
"tgt_encoded_dim": 64,
"attention_dim": 64,
"num_heads": 2,
"width_phi_sat": 256,
"depth_phi_sat": 2,
"sat_encoded_dim": 64,
"width_psi_img": 128,
"depth_psi_img": 4,
"act_function": "ReLU",
"dropout": 0.0,
"psi_img_dropout_last_layer": 0.1,
"critic_tgt_encoded_dim": 128,
"critic_width_phi": 256,
"critic_depth_phi": 2,
"critic_width_psi": 64,
"critic_depth_psi": 3,
}
TRAINING_CONFIG = {
"gamma": 0.995,
"lr": [[0, 0.00012122027544761331], [117126, 3.707168645103191e-06]],
"entropy_coeff": [[0, 0.047466805372264546], [734487, 0.0]],
"train_batch_size": 9000,
"num_sgd_iter": 15,
"lambda_": 0.8713548569911232,
"use_kl_loss": False,
"clip_param": 0.24943882616708635,
"grad_clip": 0.3104924935285628,
"vf_clip_param": 2.0,
}
module_spec = RLModuleSpec(
module_class=HybridQualityModule, model_config_dict=dict(MODEL_CONFIG)
)
class Env_wrapped(EpisodeDataLogger, gym.Wrapper):
def __init__(self, env_config):
env_config = dict(env_config)
episode_data_callback = env_config.pop("episode_data_callback", None)
satellite_data_callback = env_config.pop("satellite_data_callback", None)
env_args = make_env(env_config, n_sats=1, train=True)
env = SatelliteTasking(**env_args)
env = WrapperActionLogging(
env,
)
env = WrapperActionDuration(
env,
)
env = WrapperDiscreteToHybridAction(
env,
discrete_imaging_fraction=IMAGING_FRACTIONS,
)
EpisodeDataLogger.__init__(self, episode_data_callback, satellite_data_callback)
gym.Wrapper.__init__(self, env)
ppo_config = (
PPOConfig()
.environment(
env=Env_wrapped,
env_config=dict(ENV_CONFIG),
normalize_actions=False,
clip_actions=False,
)
.framework("torch")
.api_stack(
enable_rl_module_and_learner=True,
enable_env_runner_and_connector_v2=True,
)
.env_runners(num_env_runners=0, rollout_fragment_length=32)
.training(learner_class=TimeDiscountedGAEPPOTorchLearner, **TRAINING_CONFIG)
.rl_module(rl_module_spec=module_spec)
)
env = make_env()
env = setup_wrapper_stack(
env,
discrete_action_space=True,
discrete_imaging_fraction=IMAGING_FRACTIONS,
)
env.reset(seed=7)
module = RLModuleSpec(
module_class=HybridQualityModule,
observation_space=env.observation_space(env.satellites[0].name),
action_space=env.action_space(env.satellites[0].name),
model_config_dict=dict(MODEL_CONFIG),
).build()
def policy(satellite, *args, **kwargs):
obs = torch.as_tensor(satellite.get_obs())
module.eval()
with torch.no_grad():
output = module.forward_inference({Columns.OBS: obs[None]})
distribution = module.get_inference_action_dist_cls().from_logits(
output[Columns.ACTION_DIST_INPUTS]
)
action = distribution.to_deterministic().sample()
return int(action.item())
action = policy(env.satellites[0])
_, reward, _, _, info = env.step({env.satellites[0].name: action})
target_index, timing_index = divmod(action, len(IMAGING_FRACTIONS))
print(
"Untrained policy:",
action,
"Target:",
target_index,
"Imaging fraction:",
IMAGING_FRACTIONS[timing_index],
"Reward:",
reward,
)
2026-10-26 21:02:08,848 WARNING Creating logger for new env on PID=71679. Old environments in process may now log times incorrectly.
2026-10-26 21:02:08,851 gym INFO Resetting environment with seed=7
2026-10-26 21:02:08,855 scene.targets INFO Generating 390 targets
2026-10-26 21:02:08,986 sats.satellite.EO_0 INFO <0.00> EO_0: Finding opportunity windows from 0.00 to 18600.00 seconds
2026-10-26 21:02:09,092 gym INFO <0.00> Environment reset
2026-10-26 21:02:09,141 gym INFO <0.00> === STARTING STEP ===
2026-10-26 21:02:09,142 sats.satellite.EO_0 INFO <0.00> EO_0: target index 38 tasked
2026-10-26 21:02:09,143 sats.satellite.EO_0 INFO <0.00> EO_0: Target(Bougara, DZ) imaging timing: requested capture at 3638.7
2026-10-26 21:02:09,144 sats.satellite.EO_0 INFO <0.00> EO_0: Target(Bougara, DZ) tasked for imaging
2026-10-26 21:02:09,144 sats.satellite.EO_0 INFO <0.00> EO_0: Target(Bougara, DZ) window enabled: 3589.0 to 3688.4
2026-10-26 21:02:09,145 sats.satellite.EO_0 INFO <0.00> EO_0: setting timed terminal event at 3688.4
2026-10-26 21:02:09,234 sim.simulator INFO <500.00> Max step duration reached
2026-10-26 21:02:09,235 data.base INFO <500.00> Total reward: {}
2026-10-26 21:02:09,235 comm.communication INFO <500.00> Optimizing data communication between all pairs of satellites
2026-10-26 21:02:09,267 gym INFO <500.00> Step reward: {}
Untrained policy: 115 Target: 38 Imaging fraction: 0.5 Reward: {'EO_0': 0.0}
Multi-satellite coordination through communication
A policy trained for one satellite can be evaluated independently on each satellite, with coordination handled by the decision loop below. This example models ideal, instantaneous sharing of completed acquisitions and intended tasks. It does not simulate message transmission delays or communication outages. The policy function below uses deterministic inference from the untrained module created above; load trained weights to evaluate a learned RL policy.
An intent identifies a target another satellite is currently tasked to image. Reward-aware intents also carry the expected absolute reward at the requested acquisition time. This prediction is not an obtained reward and does not guarantee capture at the requested time.
The human-readable mode names correspond to these multisat_comm_config["mode"] values:
Mode |
Code value |
Coordination rule |
|---|---|---|
Intent |
|
Exclude globally imaged targets and targets currently intended by other satellites. A new decision selects from the remaining targets; there is no comparison of competing reward predictions. |
Intent-incremental |
|
Keep re-imaging candidates when their predicted reward can exceed both the best completed reward and the largest relevant competing intent reward. Store this maximum in bookkeeping and use the clipped marginal profile to test whether an opportunity has a useful improvement. Several satellites may retain intents for the same target; this mode does not forcibly choose a single claimant. |
Intent-greedy |
|
Exclude globally imaged targets. If a satellite selects an already intended target, compare the two expected absolute rewards at their respective requested acquisition times. A strictly larger prediction replaces the previous claimant and schedules that satellite to choose again. Otherwise, temporarily exclude the contested target for the challenger and ask it to choose again. Equal predictions retain the existing claimant. |
Intent-greedy-incremental |
|
Use the same conflict-resolution rule as intent-greedy while allowing useful re-imaging. The marginal profile is measured against the best completed reward; pending intents are handled by the explicit greedy comparison rather than subtracted from this baseline. |
The additional "imaged" mode shares completed acquisitions only: it excludes globally imaged targets without reserving targets through intents. Nonincremental modes are intended to use incremental_score=False; the two incremental modes require incremental_score=True. Select rewarder scoring explicitly when creating the environment—the communication-mode string does not change it automatically.
For a given target and the satellite currently choosing an action, intent-incremental computes:
baseline = max(best_obtained_reward, highest_competing_intent_reward)
marginal_reward = max(absolute_reward_at_t - baseline, 0)
With no competing intent, the baseline is simply the best obtained reward. Intent-greedy-incremental uses that completed-reward baseline and resolves competing intents separately. Advertising an intent does not increase obtained reward. Actual incremental reward is the improvement over the best completed reward, not over an intent prediction.
Before each decision, the loop refreshes the satellite’s completed/intended bookkeeping and clears the cached observation. Intent-based exclusion changes which target opportunities are available. Incremental modes additionally use estimate_quality_profile(..., return_rewards=True) to retain opportunities whose maximum predicted marginal reward exceeds 0.001. This is an opportunity-level check; it does not guarantee a positive reward at the policy’s chosen timing option. cooldown_s is
a separate intended restriction on rapid captures of the same target by the same satellite.
Observation wiring: QualitySatellite binds its spline observation to estimate_quality_profile. After the loop updates bookkeeping and clears the observation cache, the next observation fits the marginal profile using that satellite’s current completed/intended reward baseline. The estimator caches absolute predictions but subtracts the current bookkeeping baseline on every call, so same-time bookkeeping changes affect the spline coefficients. This changes observation values while
preserving the 512-value observation shape. The successful-capture handler also records last_img_time_per_sat, which supplies the cooldown filter.
Helpers for decoding and comparing imaging actions
get_imaging_action decodes an action into a target index and normalized timing fraction, or returns None when no imaging action is supplied. get_time uses the duration wrapper to convert that fraction into a delay in seconds for the selected satellite and target opportunity.
get_expected_absolute_reward adds the current simulation time to this delay and evaluates the predicted absolute reward at the requested acquisition time. The coordination loop uses this prediction to advertise reward-aware intents and compare competing tasks. It is evaluated independently of the bookkeeping baseline: marginal reward is computed separately from completed or intended rewards. The requested time remains a prediction point; it does not guarantee that the instrument will capture
the image at that time.
[13]:
def get_expected_absolute_reward(env, satellite, target_index, action_fraction):
"""Predict absolute reward at the wrapper-decoded desired acquisition time.
The wrapper returns a delay, whereas quality functions expect absolute
simulator time. The prediction is independent of reward bookkeeping and
does not guarantee acquisition at the desired time or within the window.
"""
target = satellite.parse_target_selection(target_index)
imaging_delay = float(
env.get_imaging_time(target_index, action_fraction, satellite.name)
)
if not np.isfinite(imaging_delay) or imaging_delay < 0.0:
raise ValueError(f"Invalid desired imaging delay: {imaging_delay}")
desired_time = satellite.simulator.sim_time + imaging_delay
predicted_reward = estimate_absolute_reward(
satellite, target, np.array([desired_time], dtype=float)
)
if predicted_reward.size != 1 or not np.all(np.isfinite(predicted_reward)):
raise ValueError(
f"Invalid expected acquisition reward for target {target.id}: "
f"{predicted_reward}"
)
return float(predicted_reward.item())
def get_imaging_action(env, action):
"""Decode a target index, or return None for a non-imaging action."""
action_parsed = (
env._split_action(action)
if isinstance(env, WrapperDiscreteToHybridAction)
else action
)
if action_parsed is None:
return None
tgt_index, fraction = action_parsed
return tgt_index, fraction
def get_time(env, action: int | tuple[int, float], sat_name: str) -> float:
action_parsed = get_imaging_action(env, action)
if action_parsed is None:
raise ValueError("Non-imaging actions do not have an imaging time.")
action_time = env.get_imaging_time(action_parsed[0], action_parsed[1], sat_name)
return action_time
Configure satellite access filters
The following block creates the constellation and configures each satellite’s access filters for the selected communication mode. These filters determine which target opportunities are available when forming observations and selecting actions. Depending on the mode, they exclude completed or competing intended targets, temporarily exclude targets lost in greedy comparisons, or retain opportunities with useful predicted marginal reward.
The displayed configuration uses two satellites, 1000 targets, and incremental_intent with incremental scoring enabled. Its filters require a maximum predicted marginal reward greater than 0.001 over the remaining window and more than cooldown_s=15.0 seconds since the same satellite last captured that target. These checks select eligible opportunities; they do not guarantee positive reward at the chosen timing fraction. The 15-second cooldown is an explicit setting of this example;
the current paper draft does not specify a cooldown value.
[ ]:
from copy import copy
setup_env_args = {}
incremental_score = True
cooldown_s = 15.0 # set to 0 to disable cooldown
seed = 42
n_targets = 1000
n_sats = 2
multisat_comm_config = {"mode": "incremental_intent"}
env = make_env(
setup_env_args,
n_sats=n_sats,
incremental_score=incremental_score,
n_targets=n_targets,
time_limit=ORBIT_PERIOD * 0.1,
)
env = setup_wrapper_stack(
env, discrete_action_space=True, discrete_imaging_fraction=IMAGING_FRACTIONS
)
seed_case = seed + n_targets
_, info = env.reset(seed=seed_case)
sat_last_decision_time = {sat.name: 0.0 for sat in env.satellites}
reward_cumulative = 0.0
intents = {sat.name: None for sat in env.satellites}
intents_quality = {sat.name: None for sat in env.satellites}
comm_mode = multisat_comm_config["mode"]
if len(env.satellites) > 1:
def filter_intent(opp, sat):
return opp["object"] not in sat.intent_set_shared
def filter_imaged(opp, sat):
return opp["object"] not in sat.imaged_set_shared
def filter_minimum_reward(opp, sat):
rewards = estimate_quality_profile(sat, opp, return_rewards=True)
minimum_quality = np.max(rewards) > 1e-3
return minimum_quality
def filter_time(opp, sat):
tgt = opp["object"]
current_time = sat.simulator.sim_time
time_diff = current_time - tgt.last_img_time_per_sat.get(sat.name, -np.inf)
return time_diff > cooldown_s
def filter_lower_reward(opp, sat):
return opp["object"] not in sat.temp_filter_set
for sat in env.satellites:
sat.intent_set_shared = set()
sat.imaged_set_shared = set()
if comm_mode == "imaged":
for sat in env.satellites:
sat.add_access_filter(lambda opp, sat=sat: filter_imaged(opp, sat))
elif comm_mode == "intent":
for sat in env.satellites:
sat.add_access_filter(lambda opp, sat=sat: filter_intent(opp, sat))
sat.add_access_filter(lambda opp, sat=sat: filter_imaged(opp, sat))
elif comm_mode == "incremental_intent":
for sat in env.satellites:
if multisat_comm_config.get("filter_minimum_quality", True):
sat.add_access_filter(
lambda opp, sat=sat: filter_minimum_reward(opp, sat)
)
if cooldown_s > 0:
sat.add_access_filter(lambda opp, sat=sat: filter_time(opp, sat))
elif comm_mode == "greedy":
# Retask an existing claimant when another satellite predicts higher reward.
for sat in env.satellites:
sat.temp_filter_set = set()
sat.add_access_filter(lambda opp, sat=sat: filter_imaged(opp, sat))
sat.add_access_filter(lambda opp, sat=sat: filter_lower_reward(opp, sat))
elif comm_mode == "greedy_incremental":
for sat in env.satellites:
sat.temp_filter_set = set()
sat.add_access_filter(lambda opp, sat=sat: filter_lower_reward(opp, sat))
if multisat_comm_config.get("filter_minimum_quality", True):
sat.add_access_filter(
lambda opp, sat=sat: filter_minimum_reward(opp, sat)
)
if cooldown_s > 0:
sat.add_access_filter(lambda opp, sat=sat: filter_time(opp, sat))
2026-10-26 21:02:09,304 WARNING Creating logger for new env on PID=71679. Old environments in process may now log times incorrectly.
2026-10-26 21:02:09,307 gym INFO Resetting environment with seed=1042
2026-10-26 21:02:09,311 scene.targets INFO Generating 1000 targets
2026-10-26 21:02:09,499 sats.satellite.EO_0 INFO <0.00> EO_0: Finding opportunity windows from 0.00 to 1800.00 seconds
2026-10-26 21:02:09,561 sats.satellite.EO_1 INFO <0.00> EO_1: Finding opportunity windows from 0.00 to 1800.00 seconds
2026-10-26 21:02:09,647 sats.satellite.EO_0 INFO <0.00> EO_0: Finding opportunity windows from 1800.00 to 4200.00 seconds
2026-10-26 21:02:09,703 sats.satellite.EO_1 INFO <0.00> EO_1: Finding opportunity windows from 1800.00 to 4200.00 seconds
2026-10-26 21:02:09,837 gym INFO <0.00> Environment reset
Run the coordination loop
At each step, the loop identifies satellites that require a new task, refreshes their shared acquisition information and mode-specific bookkeeping, and clears cached observations before selecting actions. It records new intents and their expected rewards; greedy modes can also displace a competing intent and request another decision. The action dictionary advances the constellation, rewards are accumulated across satellites, and the loop stops when the environment reports termination or truncation.
The current policy runs the actor defined above with randomly initialized weights and selects its highest-logit target/timing pair through a deterministic categorical distribution. It therefore demonstrates coordination with an untrained policy, rather than uniformly sampling random actions. Load trained module weights to evaluate a learned policy with the same loop.
[15]:
while True:
actions = {}
current_time = env.simulator.sim_time
old_intents = copy(intents)
intents = {}
intents_quality_old = copy(intents_quality)
intents_quality = {}
imaged_set = env.rewarder.data.imaged
satellites_needing_retask = []
for satellite in env.satellites:
needs_retask = (
satellite.name in info and info[satellite.name]["requires_retasking"]
)
exceeded_time_limit = (
current_time - sat_last_decision_time[satellite.name] >= 500.0
)
intent_imaged = (
old_intents[satellite.name] in imaged_set
if old_intents[satellite.name] is not None
else False
) and comm_mode not in ["incremental_intent", "greedy_incremental"]
if needs_retask or exceeded_time_limit or intent_imaged:
satellites_needing_retask.append(satellite.name)
intents[satellite.name] = None
intents_quality[satellite.name] = None
else:
intents[satellite.name] = old_intents[satellite.name]
intents_quality[satellite.name] = intents_quality_old[satellite.name]
while len(satellites_needing_retask) > 0:
for satellite in env.satellites:
if satellite.name in satellites_needing_retask:
if comm_mode == "imaged":
satellite.imaged_set_shared = imaged_set
if comm_mode == "intent":
satellite.imaged_set_shared = imaged_set
satellite.intent_set_shared = set(intents.values()) - {None}
if comm_mode == "incremental_intent":
satellite.bookkeeping.delete_intended()
for tgt in env.rewarder.data.imaged:
satellite.bookkeeping.update(
tgt.id, tgt.highest_reward_obtained, current_time
)
for intent_quality in intents_quality.values():
if intent_quality is not None:
satellite.bookkeeping.update_intended(
intent_quality[0].id,
intent_quality[1],
intent_quality[2],
)
if comm_mode == "greedy":
satellite.temp_filter_set = set()
satellite.imaged_set_shared = imaged_set
if comm_mode == "greedy_incremental":
satellite.temp_filter_set = set()
satellite.bookkeeping.delete_intended()
for tgt in env.rewarder.data.imaged:
satellite.bookkeeping.update(
tgt.id, tgt.highest_reward_obtained, current_time
)
satellite.observation_builder.obs_dict_cache = (
None # Clear cache to force re-evaluation of the observation
)
sat_last_decision_time[satellite.name] = current_time
action = policy(satellite)
imaging_action = get_imaging_action(env, action)
if (
comm_mode in ["greedy", "greedy_incremental"]
and imaging_action is not None
):
target_index, action_fraction = imaging_action
target_parsed = satellite.parse_target_selection(target_index)
while target_parsed in intents.values():
expected_absolute_reward = get_expected_absolute_reward(
env, satellite, target_index, action_fraction
)
intent_rewards = {
val[0].id: (key, val[1])
for key, val in intents_quality.items()
if val is not None
}
prev_data = intent_rewards.get(target_parsed.id)
prev_sat_name = prev_data[0]
prev_reward = prev_data[1]
if expected_absolute_reward > prev_reward:
# Assigns the new satellite's intent since it has a better expected reward than the previous satellite's intent
intents[satellite.name] = target_parsed
intents_quality[satellite.name] = (
target_parsed,
expected_absolute_reward,
get_time(env, action, satellite.name),
)
# Removes old satellites intent if the new satellite has a better expected reward
intents[prev_sat_name] = None
intents_quality[prev_sat_name] = None
actions[prev_sat_name] = None
satellites_needing_retask.append(prev_sat_name)
break
else:
satellite.temp_filter_set.add(target_parsed)
satellite.observation_builder.obs_dict_cache = None # Clear cache to force re-evaluation of the observation
action = policy(satellite)
imaging_action = get_imaging_action(env, action)
if imaging_action is None:
break
target_index, action_fraction = imaging_action
target_parsed = satellite.parse_target_selection(
target_index
)
actions[satellite.name] = action
imaging_action = get_imaging_action(env, action)
if imaging_action is not None:
target_index, action_fraction = imaging_action
target_parsed = satellite.parse_target_selection(target_index)
intents[satellite.name] = target_parsed
if comm_mode in [
"incremental_intent",
"greedy",
"greedy_incremental",
]:
expected_absolute_reward = get_expected_absolute_reward(
env, satellite, target_index, action_fraction
)
intents_quality[satellite.name] = (
target_parsed,
expected_absolute_reward,
get_time(env, action, satellite.name),
)
satellites_needing_retask.remove(satellite.name)
_, reward, terminated, truncated, info = env.step(actions) # type: ignore
reward_cumulative += sum(reward.values())
if any(terminated.values()) or any(truncated.values()):
break
2026-10-26 21:02:09,946 gym INFO <0.00> === STARTING STEP ===
2026-10-26 21:02:09,948 sats.satellite.EO_0 INFO <0.00> EO_0: target index 3 tasked
2026-10-26 21:02:09,948 sats.satellite.EO_0 INFO <0.00> EO_0: Target(Māhta, IN) imaging timing: requested capture at 11.8
2026-10-26 21:02:09,949 sats.satellite.EO_0 INFO <0.00> EO_0: Target(Māhta, IN) tasked for imaging
2026-10-26 21:02:09,950 sats.satellite.EO_0 INFO <0.00> EO_0: Target(Māhta, IN) window enabled: 0.0 to 23.7
2026-10-26 21:02:09,950 sats.satellite.EO_0 INFO <0.00> EO_0: setting timed terminal event at 23.7
2026-10-26 21:02:09,952 sats.satellite.EO_1 INFO <0.00> EO_1: target index 0 tasked
2026-10-26 21:02:09,953 sats.satellite.EO_1 INFO <0.00> EO_1: Target(San Sebastián de la Gomera, ES) imaging timing: requested capture at 430.7
2026-10-26 21:02:09,953 sats.satellite.EO_1 INFO <0.00> EO_1: Target(San Sebastián de la Gomera, ES) tasked for imaging
2026-10-26 21:02:09,954 sats.satellite.EO_1 INFO <0.00> EO_1: Target(San Sebastián de la Gomera, ES) window enabled: 371.5 to 489.9
2026-10-26 21:02:09,955 sats.satellite.EO_1 INFO <0.00> EO_1: setting timed terminal event at 489.9
2026-10-26 21:02:09,967 sats.satellite.EO_0 INFO <24.00> EO_0: timed termination at 23.7 for Target(Māhta, IN) window
2026-10-26 21:02:09,968 data.base INFO <24.00> Total reward: {}
2026-10-26 21:02:09,969 sats.satellite.EO_0 INFO <24.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:10,037 gym INFO <24.00> Step reward: {}
2026-10-26 21:02:10,066 gym INFO <24.00> === STARTING STEP ===
2026-10-26 21:02:10,067 sats.satellite.EO_0 INFO <24.00> EO_0: target index 0 tasked
2026-10-26 21:02:10,069 sats.satellite.EO_0 INFO <24.00> EO_0: Target(Nawāshahr, IN) imaging timing: requested capture at 31.5
2026-10-26 21:02:10,070 sats.satellite.EO_0 INFO <24.00> EO_0: Target(Nawāshahr, IN) tasked for imaging
2026-10-26 21:02:10,071 sats.satellite.EO_0 INFO <24.00> EO_0: Target(Nawāshahr, IN) window enabled: 0.0 to 38.9
2026-10-26 21:02:10,072 sats.satellite.EO_0 INFO <24.00> EO_0: setting timed terminal event at 38.9
2026-10-26 21:02:10,080 sats.satellite.EO_0 INFO <39.00> EO_0: timed termination at 38.9 for Target(Nawāshahr, IN) window
2026-10-26 21:02:10,082 data.base INFO <39.00> Total reward: {}
2026-10-26 21:02:10,083 sats.satellite.EO_0 INFO <39.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:10,159 gym INFO <39.00> Step reward: {}
2026-10-26 21:02:10,190 gym INFO <39.00> === STARTING STEP ===
2026-10-26 21:02:10,191 sats.satellite.EO_0 INFO <39.00> EO_0: target index 0 tasked
2026-10-26 21:02:10,192 sats.satellite.EO_0 INFO <39.00> EO_0: Target(Gadarpur, IN) imaging timing: requested capture at 66.3
2026-10-26 21:02:10,192 sats.satellite.EO_0 INFO <39.00> EO_0: Target(Gadarpur, IN) tasked for imaging
2026-10-26 21:02:10,193 sats.satellite.EO_0 INFO <39.00> EO_0: Target(Gadarpur, IN) window enabled: 16.6 to 93.6
2026-10-26 21:02:10,193 sats.satellite.EO_0 INFO <39.00> EO_0: setting timed terminal event at 93.6
2026-10-26 21:02:10,209 sats.satellite.EO_0 INFO <83.00> EO_0: imaged Target(Gadarpur, IN)
2026-10-26 21:02:10,211 data.base INFO <83.00> Total reward: {'EO_0': np.float64(0.5571039239604911)}
2026-10-26 21:02:10,211 sats.satellite.EO_0 INFO <83.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:10,288 gym INFO <83.00> Step reward: {'EO_0': np.float64(0.5571039239604911)}
2026-10-26 21:02:10,323 gym INFO <83.00> === STARTING STEP ===
2026-10-26 21:02:10,324 sats.satellite.EO_0 INFO <83.00> EO_0: target index 3 tasked
2026-10-26 21:02:10,325 sats.satellite.EO_0 INFO <83.00> EO_0: Target(Koini, IN) imaging timing: requested capture at 149.3
2026-10-26 21:02:10,326 sats.satellite.EO_0 INFO <83.00> EO_0: Target(Koini, IN) tasked for imaging
2026-10-26 21:02:10,327 sats.satellite.EO_0 INFO <83.00> EO_0: Target(Koini, IN) window enabled: 98.0 to 200.7
2026-10-26 21:02:10,327 sats.satellite.EO_0 INFO <83.00> EO_0: setting timed terminal event at 200.7
2026-10-26 21:02:10,348 sats.satellite.EO_0 INFO <149.00> EO_0: imaged Target(Koini, IN)
2026-10-26 21:02:10,349 data.base INFO <149.00> Total reward: {'EO_0': np.float64(0.724379742733913)}
2026-10-26 21:02:10,349 sats.satellite.EO_0 INFO <149.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:10,421 gym INFO <149.00> Step reward: {'EO_0': np.float64(0.724379742733913)}
2026-10-26 21:02:10,466 gym INFO <149.00> === STARTING STEP ===
2026-10-26 21:02:10,466 sats.satellite.EO_0 INFO <149.00> EO_0: target index 6 tasked
2026-10-26 21:02:10,467 sats.satellite.EO_0 INFO <149.00> EO_0: Target(Jitpur, NP) imaging timing: requested capture at 180.2
2026-10-26 21:02:10,468 sats.satellite.EO_0 INFO <149.00> EO_0: Target(Jitpur, NP) tasked for imaging
2026-10-26 21:02:10,469 sats.satellite.EO_0 INFO <149.00> EO_0: Target(Jitpur, NP) window enabled: 80.5 to 211.5
2026-10-26 21:02:10,469 sats.satellite.EO_0 INFO <149.00> EO_0: setting timed terminal event at 211.5
2026-10-26 21:02:10,482 sats.satellite.EO_0 INFO <180.00> EO_0: imaged Target(Jitpur, NP)
2026-10-26 21:02:10,483 data.base INFO <180.00> Total reward: {'EO_0': np.float64(0.005747950399001029)}
2026-10-26 21:02:10,484 sats.satellite.EO_0 INFO <180.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:10,551 gym INFO <180.00> Step reward: {'EO_0': np.float64(0.005747950399001029)}
2026-10-26 21:02:10,582 gym INFO <180.00> === STARTING STEP ===
2026-10-26 21:02:10,583 sats.satellite.EO_0 INFO <180.00> EO_0: target index 8 tasked
2026-10-26 21:02:10,584 sats.satellite.EO_0 INFO <180.00> EO_0: Target(Bāsopatti, IN) imaging timing: requested capture at 204.2
2026-10-26 21:02:10,585 sats.satellite.EO_0 INFO <180.00> EO_0: Target(Bāsopatti, IN) tasked for imaging
2026-10-26 21:02:10,586 sats.satellite.EO_0 INFO <180.00> EO_0: Target(Bāsopatti, IN) window enabled: 104.0 to 228.5
2026-10-26 21:02:10,587 sats.satellite.EO_0 INFO <180.00> EO_0: setting timed terminal event at 228.5
2026-10-26 21:02:10,598 sats.satellite.EO_0 INFO <204.00> EO_0: imaged Target(Bāsopatti, IN)
2026-10-26 21:02:10,600 data.base INFO <204.00> Total reward: {'EO_0': np.float64(0.6400027666600737)}
2026-10-26 21:02:10,601 sats.satellite.EO_0 INFO <204.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:10,680 gym INFO <204.00> Step reward: {'EO_0': np.float64(0.6400027666600737)}
2026-10-26 21:02:10,715 gym INFO <204.00> === STARTING STEP ===
2026-10-26 21:02:10,716 sats.satellite.EO_0 INFO <204.00> EO_0: target index 8 tasked
2026-10-26 21:02:10,717 sats.satellite.EO_0 INFO <204.00> EO_0: Target(Simaria, IN) imaging timing: requested capture at 217.5
2026-10-26 21:02:10,717 sats.satellite.EO_0 INFO <204.00> EO_0: Target(Simaria, IN) tasked for imaging
2026-10-26 21:02:10,718 sats.satellite.EO_0 INFO <204.00> EO_0: Target(Simaria, IN) window enabled: 126.9 to 231.0
2026-10-26 21:02:10,718 sats.satellite.EO_0 INFO <204.00> EO_0: setting timed terminal event at 231.0
2026-10-26 21:02:10,728 sats.satellite.EO_0 INFO <227.00> EO_0: imaged Target(Simaria, IN)
2026-10-26 21:02:10,730 data.base INFO <227.00> Total reward: {'EO_0': np.float64(0.6273504569748937)}
2026-10-26 21:02:10,731 sats.satellite.EO_0 INFO <227.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:10,810 gym INFO <227.00> Step reward: {'EO_0': np.float64(0.6273504569748937)}
2026-10-26 21:02:10,878 gym INFO <227.00> === STARTING STEP ===
2026-10-26 21:02:10,879 sats.satellite.EO_0 INFO <227.00> EO_0: target index 14 tasked
2026-10-26 21:02:10,882 sats.satellite.EO_0 INFO <227.00> EO_0: Target(Kasba, IN) imaging timing: requested capture at 242.3
2026-10-26 21:02:10,882 sats.satellite.EO_0 INFO <227.00> EO_0: Target(Kasba, IN) tasked for imaging
2026-10-26 21:02:10,884 sats.satellite.EO_0 INFO <227.00> EO_0: Target(Kasba, IN) window enabled: 128.7 to 257.6
2026-10-26 21:02:10,884 sats.satellite.EO_0 INFO <227.00> EO_0: setting timed terminal event at 257.6
2026-10-26 21:02:10,893 sats.satellite.EO_0 INFO <249.00> EO_0: imaged Target(Kasba, IN)
2026-10-26 21:02:10,894 data.base INFO <249.00> Total reward: {'EO_0': np.float64(0.03568579182674939)}
2026-10-26 21:02:10,895 sats.satellite.EO_0 INFO <249.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:10,967 gym INFO <249.00> Step reward: {'EO_0': np.float64(0.03568579182674939)}
2026-10-26 21:02:10,999 gym INFO <249.00> === STARTING STEP ===
2026-10-26 21:02:11,000 sats.satellite.EO_0 INFO <249.00> EO_0: target index 0 tasked
2026-10-26 21:02:11,000 sats.satellite.EO_0 INFO <249.00> EO_0: Target(Rāghopur, IN) imaging timing: requested capture at 250.0
2026-10-26 21:02:11,001 sats.satellite.EO_0 INFO <249.00> EO_0: Target(Rāghopur, IN) tasked for imaging
2026-10-26 21:02:11,002 sats.satellite.EO_0 INFO <249.00> EO_0: Target(Rāghopur, IN) window enabled: 134.3 to 250.9
2026-10-26 21:02:11,003 sats.satellite.EO_0 INFO <249.00> EO_0: setting timed terminal event at 250.9
2026-10-26 21:02:11,007 sats.satellite.EO_0 INFO <251.00> EO_0: timed termination at 250.9 for Target(Rāghopur, IN) window
2026-10-26 21:02:11,008 data.base INFO <251.00> Total reward: {}
2026-10-26 21:02:11,009 sats.satellite.EO_0 INFO <251.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:11,077 gym INFO <251.00> Step reward: {}
2026-10-26 21:02:11,112 gym INFO <251.00> === STARTING STEP ===
2026-10-26 21:02:11,113 sats.satellite.EO_0 INFO <251.00> EO_0: target index 3 tasked
2026-10-26 21:02:11,113 sats.satellite.EO_0 INFO <251.00> EO_0: Target(Mohanpur Gaughāta, IN) imaging timing: requested capture at 254.0
2026-10-26 21:02:11,114 sats.satellite.EO_0 INFO <251.00> EO_0: Target(Mohanpur Gaughāta, IN) tasked for imaging
2026-10-26 21:02:11,114 sats.satellite.EO_0 INFO <251.00> EO_0: Target(Mohanpur Gaughāta, IN) window enabled: 136.2 to 256.9
2026-10-26 21:02:11,115 sats.satellite.EO_0 INFO <251.00> EO_0: setting timed terminal event at 256.9
2026-10-26 21:02:11,119 sats.satellite.EO_0 INFO <257.00> EO_0: timed termination at 256.9 for Target(Mohanpur Gaughāta, IN) window
2026-10-26 21:02:11,120 data.base INFO <257.00> Total reward: {}
2026-10-26 21:02:11,120 sats.satellite.EO_0 INFO <257.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:11,207 gym INFO <257.00> Step reward: {}
2026-10-26 21:02:11,238 gym INFO <257.00> === STARTING STEP ===
2026-10-26 21:02:11,239 sats.satellite.EO_0 INFO <257.00> EO_0: target index 2 tasked
2026-10-26 21:02:11,240 sats.satellite.EO_0 INFO <257.00> EO_0: Target(Mālior, IN) imaging timing: requested capture at 261.2
2026-10-26 21:02:11,240 sats.satellite.EO_0 INFO <257.00> EO_0: Target(Mālior, IN) tasked for imaging
2026-10-26 21:02:11,241 sats.satellite.EO_0 INFO <257.00> EO_0: Target(Mālior, IN) window enabled: 139.0 to 265.4
2026-10-26 21:02:11,242 sats.satellite.EO_0 INFO <257.00> EO_0: setting timed terminal event at 265.4
2026-10-26 21:02:11,246 sats.satellite.EO_0 INFO <265.50> EO_0: timed termination at 265.4 for Target(Mālior, IN) window
2026-10-26 21:02:11,247 data.base INFO <265.50> Total reward: {}
2026-10-26 21:02:11,248 sats.satellite.EO_0 INFO <265.50> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:11,327 gym INFO <265.50> Step reward: {}
2026-10-26 21:02:11,353 gym INFO <265.50> === STARTING STEP ===
2026-10-26 21:02:11,354 sats.satellite.EO_0 INFO <265.50> EO_0: target index 2 tasked
2026-10-26 21:02:11,355 sats.satellite.EO_0 INFO <265.50> EO_0: Target(Pemagatshel, BT) imaging timing: requested capture at 276.3
2026-10-26 21:02:11,355 sats.satellite.EO_0 INFO <265.50> EO_0: Target(Pemagatshel, BT) tasked for imaging
2026-10-26 21:02:11,356 sats.satellite.EO_0 INFO <265.50> EO_0: Target(Pemagatshel, BT) window enabled: 166.6 to 287.1
2026-10-26 21:02:11,357 sats.satellite.EO_0 INFO <265.50> EO_0: setting timed terminal event at 287.1
2026-10-26 21:02:11,359 sats.satellite.EO_0 INFO <266.00> EO_0: imaged Target(Pemagatshel, BT)
2026-10-26 21:02:11,360 data.base INFO <266.00> Total reward: {'EO_0': np.float64(0.011399764508251285)}
2026-10-26 21:02:11,361 sats.satellite.EO_0 INFO <266.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:11,438 gym INFO <266.00> Step reward: {'EO_0': np.float64(0.011399764508251285)}
2026-10-26 21:02:11,464 gym INFO <266.00> === STARTING STEP ===
2026-10-26 21:02:11,465 sats.satellite.EO_0 INFO <266.00> EO_0: target index 0 tasked
2026-10-26 21:02:11,466 sats.satellite.EO_0 INFO <266.00> EO_0: Target(Bāwāli, IN) imaging timing: requested capture at 266.5
2026-10-26 21:02:11,466 sats.satellite.EO_0 INFO <266.00> EO_0: Target(Bāwāli, IN) tasked for imaging
2026-10-26 21:02:11,467 sats.satellite.EO_0 INFO <266.00> EO_0: Target(Bāwāli, IN) window enabled: 214.2 to 266.0
2026-10-26 21:02:11,467 sats.satellite.EO_0 INFO <266.00> EO_0: setting timed terminal event at 266.0
2026-10-26 21:02:11,471 sats.satellite.EO_0 INFO <266.50> EO_0: timed termination at 266.0 for Target(Bāwāli, IN) window
2026-10-26 21:02:11,472 data.base INFO <266.50> Total reward: {}
2026-10-26 21:02:11,473 sats.satellite.EO_0 INFO <266.50> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:11,551 gym INFO <266.50> Step reward: {}
2026-10-26 21:02:11,578 gym INFO <266.50> === STARTING STEP ===
2026-10-26 21:02:11,581 sats.satellite.EO_0 INFO <266.50> EO_0: target index 0 tasked
2026-10-26 21:02:11,581 sats.satellite.EO_0 INFO <266.50> EO_0: Target(Bagulā, IN) imaging timing: requested capture at 276.2
2026-10-26 21:02:11,582 sats.satellite.EO_0 INFO <266.50> EO_0: Target(Bagulā, IN) tasked for imaging
2026-10-26 21:02:11,583 sats.satellite.EO_0 INFO <266.50> EO_0: Target(Bagulā, IN) window enabled: 184.7 to 286.0
2026-10-26 21:02:11,584 sats.satellite.EO_0 INFO <266.50> EO_0: setting timed terminal event at 286.0
2026-10-26 21:02:11,594 sats.satellite.EO_0 INFO <286.00> EO_0: timed termination at 286.0 for Target(Bagulā, IN) window
2026-10-26 21:02:11,595 data.base INFO <286.00> Total reward: {}
2026-10-26 21:02:11,596 sats.satellite.EO_0 INFO <286.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:11,672 gym INFO <286.00> Step reward: {}
2026-10-26 21:02:11,712 gym INFO <286.00> === STARTING STEP ===
2026-10-26 21:02:11,713 sats.satellite.EO_0 INFO <286.00> EO_0: target index 0 tasked
2026-10-26 21:02:11,716 sats.satellite.EO_0 INFO <286.00> EO_0: Target(Gobārdānga, IN) imaging timing: requested capture at 286.8
2026-10-26 21:02:11,716 sats.satellite.EO_0 INFO <286.00> EO_0: Target(Gobārdānga, IN) tasked for imaging
2026-10-26 21:02:11,718 sats.satellite.EO_0 INFO <286.00> EO_0: Target(Gobārdānga, IN) window enabled: 196.6 to 287.6
2026-10-26 21:02:11,719 sats.satellite.EO_0 INFO <286.00> EO_0: setting timed terminal event at 287.6
2026-10-26 21:02:11,721 sats.satellite.EO_0 INFO <288.00> EO_0: timed termination at 287.6 for Target(Gobārdānga, IN) window
2026-10-26 21:02:11,722 data.base INFO <288.00> Total reward: {}
2026-10-26 21:02:11,723 sats.satellite.EO_0 INFO <288.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:11,786 gym INFO <288.00> Step reward: {}
2026-10-26 21:02:11,813 gym INFO <288.00> === STARTING STEP ===
2026-10-26 21:02:11,814 sats.satellite.EO_0 INFO <288.00> EO_0: target index 1 tasked
2026-10-26 21:02:11,814 sats.satellite.EO_0 INFO <288.00> EO_0: Target(Faradābād, BD) imaging timing: requested capture at 306.2
2026-10-26 21:02:11,815 sats.satellite.EO_0 INFO <288.00> EO_0: Target(Faradābād, BD) tasked for imaging
2026-10-26 21:02:11,815 sats.satellite.EO_0 INFO <288.00> EO_0: Target(Faradābād, BD) window enabled: 191.8 to 324.4
2026-10-26 21:02:11,816 sats.satellite.EO_0 INFO <288.00> EO_0: setting timed terminal event at 324.4
2026-10-26 21:02:11,828 sats.satellite.EO_0 INFO <318.00> EO_0: imaged Target(Faradābād, BD)
2026-10-26 21:02:11,829 data.base INFO <318.00> Total reward: {'EO_0': np.float64(0.4171109179135972)}
2026-10-26 21:02:11,830 sats.satellite.EO_0 INFO <318.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:11,895 gym INFO <318.00> Step reward: {'EO_0': np.float64(0.4171109179135972)}
2026-10-26 21:02:11,927 gym INFO <318.00> === STARTING STEP ===
2026-10-26 21:02:11,928 sats.satellite.EO_0 INFO <318.00> EO_0: target index 2 tasked
2026-10-26 21:02:11,929 sats.satellite.EO_0 INFO <318.00> EO_0: Target(Kakching, IN) imaging timing: requested capture at 333.0
2026-10-26 21:02:11,929 sats.satellite.EO_0 INFO <318.00> EO_0: Target(Kakching, IN) tasked for imaging
2026-10-26 21:02:11,930 sats.satellite.EO_0 INFO <318.00> EO_0: Target(Kakching, IN) window enabled: 222.9 to 348.1
2026-10-26 21:02:11,931 sats.satellite.EO_0 INFO <318.00> EO_0: setting timed terminal event at 348.1
2026-10-26 21:02:11,946 sats.satellite.EO_0 INFO <348.50> EO_0: timed termination at 348.1 for Target(Kakching, IN) window
2026-10-26 21:02:11,947 data.base INFO <348.50> Total reward: {}
2026-10-26 21:02:11,948 sats.satellite.EO_0 INFO <348.50> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:12,026 gym INFO <348.50> Step reward: {}
2026-10-26 21:02:12,053 gym INFO <348.50> === STARTING STEP ===
2026-10-26 21:02:12,054 sats.satellite.EO_0 INFO <348.50> EO_0: target index 0 tasked
2026-10-26 21:02:12,055 sats.satellite.EO_0 INFO <348.50> EO_0: Target(Chandanais, BD) imaging timing: requested capture at 351.0
2026-10-26 21:02:12,055 sats.satellite.EO_0 INFO <348.50> EO_0: Target(Chandanais, BD) tasked for imaging
2026-10-26 21:02:12,056 sats.satellite.EO_0 INFO <348.50> EO_0: Target(Chandanais, BD) window enabled: 226.4 to 353.6
2026-10-26 21:02:12,057 sats.satellite.EO_0 INFO <348.50> EO_0: setting timed terminal event at 353.6
2026-10-26 21:02:12,062 sats.satellite.EO_0 INFO <354.00> EO_0: timed termination at 353.6 for Target(Chandanais, BD) window
2026-10-26 21:02:12,063 data.base INFO <354.00> Total reward: {}
2026-10-26 21:02:12,063 sats.satellite.EO_0 INFO <354.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:12,124 gym INFO <354.00> Step reward: {}
2026-10-26 21:02:12,152 gym INFO <354.00> === STARTING STEP ===
2026-10-26 21:02:12,154 sats.satellite.EO_0 INFO <354.00> EO_0: target index 2 tasked
2026-10-26 21:02:12,155 sats.satellite.EO_0 INFO <354.00> EO_0: Target(Ban Pae, TH) imaging timing: requested capture at 420.2
2026-10-26 21:02:12,155 sats.satellite.EO_0 INFO <354.00> EO_0: Target(Ban Pae, TH) tasked for imaging
2026-10-26 21:02:12,156 sats.satellite.EO_0 INFO <354.00> EO_0: Target(Ban Pae, TH) window enabled: 349.6 to 486.4
2026-10-26 21:02:12,157 sats.satellite.EO_0 INFO <354.00> EO_0: setting timed terminal event at 486.4
2026-10-26 21:02:12,182 sats.satellite.EO_0 INFO <420.00> EO_0: imaged Target(Ban Pae, TH)
2026-10-26 21:02:12,184 data.base INFO <420.00> Total reward: {'EO_0': np.float64(0.10437116422746319)}
2026-10-26 21:02:12,184 sats.satellite.EO_0 INFO <420.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:12,252 gym INFO <420.00> Step reward: {'EO_0': np.float64(0.10437116422746319)}
2026-10-26 21:02:12,292 gym INFO <420.00> === STARTING STEP ===
2026-10-26 21:02:12,296 sats.satellite.EO_0 INFO <420.00> EO_0: target index 1 tasked
2026-10-26 21:02:12,297 sats.satellite.EO_0 INFO <420.00> EO_0: Target(Ban San Phak Wan Luang, TH) imaging timing: requested capture at 452.0
2026-10-26 21:02:12,298 sats.satellite.EO_0 INFO <420.00> EO_0: Target(Ban San Phak Wan Luang, TH) tasked for imaging
2026-10-26 21:02:12,299 sats.satellite.EO_0 INFO <420.00> EO_0: Target(Ban San Phak Wan Luang, TH) window enabled: 348.2 to 484.0
2026-10-26 21:02:12,299 sats.satellite.EO_0 INFO <420.00> EO_0: setting timed terminal event at 484.0
2026-10-26 21:02:12,306 sats.satellite.EO_1 INFO <430.50> EO_1: imaged Target(San Sebastián de la Gomera, ES)
2026-10-26 21:02:12,308 data.base INFO <430.50> Total reward: {'EO_1': np.float64(0.795573591049628)}
2026-10-26 21:02:12,308 sats.satellite.EO_1 INFO <430.50> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:12,312 sats.satellite.EO_0 INFO <430.50> EO_0: Finding opportunity windows from 4200.00 to 4800.00 seconds
2026-10-26 21:02:12,359 sats.satellite.EO_1 INFO <430.50> EO_1: Finding opportunity windows from 4200.00 to 4800.00 seconds
2026-10-26 21:02:12,418 gym INFO <430.50> Step reward: {'EO_1': np.float64(0.795573591049628)}
2026-10-26 21:02:12,458 gym INFO <430.50> === STARTING STEP ===
2026-10-26 21:02:12,460 sats.satellite.EO_1 INFO <430.50> EO_1: target index 0 tasked
2026-10-26 21:02:12,460 sats.satellite.EO_1 INFO <430.50> EO_1: Target(Machico, PT) imaging timing: requested capture at 488.3
2026-10-26 21:02:12,461 sats.satellite.EO_1 INFO <430.50> EO_1: Target(Machico, PT) tasked for imaging
2026-10-26 21:02:12,462 sats.satellite.EO_1 INFO <430.50> EO_1: Target(Machico, PT) window enabled: 420.9 to 546.1
2026-10-26 21:02:12,462 sats.satellite.EO_1 INFO <430.50> EO_1: setting timed terminal event at 546.1
2026-10-26 21:02:12,470 sats.satellite.EO_0 INFO <452.00> EO_0: imaged Target(Ban San Phak Wan Luang, TH)
2026-10-26 21:02:12,472 data.base INFO <452.00> Total reward: {'EO_0': np.float64(0.19947173638520796)}
2026-10-26 21:02:12,473 sats.satellite.EO_0 INFO <452.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:12,543 gym INFO <452.00> Step reward: {'EO_0': np.float64(0.19947173638520796)}
2026-10-26 21:02:12,572 gym INFO <452.00> === STARTING STEP ===
2026-10-26 21:02:12,573 sats.satellite.EO_0 INFO <452.00> EO_0: target index 4 tasked
2026-10-26 21:02:12,574 sats.satellite.EO_0 INFO <452.00> EO_0: Target(Ratchaburi, TH) imaging timing: requested capture at 496.5
2026-10-26 21:02:12,574 sats.satellite.EO_0 INFO <452.00> EO_0: Target(Ratchaburi, TH) tasked for imaging
2026-10-26 21:02:12,575 sats.satellite.EO_0 INFO <452.00> EO_0: Target(Ratchaburi, TH) window enabled: 440.2 to 541.0
2026-10-26 21:02:12,576 sats.satellite.EO_0 INFO <452.00> EO_0: setting timed terminal event at 541.0
2026-10-26 21:02:12,589 sats.satellite.EO_1 INFO <488.00> EO_1: imaged Target(Machico, PT)
2026-10-26 21:02:12,591 data.base INFO <488.00> Total reward: {'EO_1': np.float64(0.38370711429960774)}
2026-10-26 21:02:12,591 sats.satellite.EO_1 INFO <488.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:12,656 gym INFO <488.00> Step reward: {'EO_1': np.float64(0.38370711429960774)}
2026-10-26 21:02:12,685 gym INFO <488.00> === STARTING STEP ===
2026-10-26 21:02:12,699 sats.satellite.EO_1 INFO <488.00> EO_1: target index 0 tasked
2026-10-26 21:02:12,703 sats.satellite.EO_1 INFO <488.00> EO_1: Target(San Sebastián de la Gomera, ES) imaging timing: requested capture at 488.9
2026-10-26 21:02:12,704 sats.satellite.EO_1 INFO <488.00> EO_1: Target(San Sebastián de la Gomera, ES) tasked for imaging
2026-10-26 21:02:12,709 sats.satellite.EO_1 INFO <488.00> EO_1: Target(San Sebastián de la Gomera, ES) window enabled: 371.5 to 489.9
2026-10-26 21:02:12,714 sats.satellite.EO_1 INFO <488.00> EO_1: setting timed terminal event at 489.9
2026-10-26 21:02:12,718 sats.satellite.EO_1 INFO <490.00> EO_1: timed termination at 489.9 for Target(San Sebastián de la Gomera, ES) window
2026-10-26 21:02:12,719 data.base INFO <490.00> Total reward: {}
2026-10-26 21:02:12,719 sats.satellite.EO_1 INFO <490.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:12,798 gym INFO <490.00> Step reward: {}
2026-10-26 21:02:12,839 gym INFO <490.00> === STARTING STEP ===
2026-10-26 21:02:12,841 sats.satellite.EO_1 INFO <490.00> EO_1: target index 0 tasked
2026-10-26 21:02:12,842 sats.satellite.EO_1 INFO <490.00> EO_1: Target(Amdel, MA) imaging timing: requested capture at 569.7
2026-10-26 21:02:12,842 sats.satellite.EO_1 INFO <490.00> EO_1: Target(Amdel, MA) tasked for imaging
2026-10-26 21:02:12,843 sats.satellite.EO_1 INFO <490.00> EO_1: Target(Amdel, MA) window enabled: 529.1 to 610.3
2026-10-26 21:02:12,845 sats.satellite.EO_1 INFO <490.00> EO_1: setting timed terminal event at 610.3
2026-10-26 21:02:12,852 sats.satellite.EO_0 INFO <501.50> EO_0: imaged Target(Ratchaburi, TH)
2026-10-26 21:02:12,853 data.base INFO <501.50> Total reward: {'EO_0': np.float64(0.019042621930721183)}
2026-10-26 21:02:12,853 sats.satellite.EO_0 INFO <501.50> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:12,865 sats.satellite.EO_0 INFO <501.50> EO_0: Finding opportunity windows from 4800.00 to 5400.00 seconds
2026-10-26 21:02:12,971 gym INFO <501.50> Step reward: {'EO_0': np.float64(0.019042621930721183)}
2026-10-26 21:02:13,011 gym INFO <501.50> === STARTING STEP ===
2026-10-26 21:02:13,012 sats.satellite.EO_0 INFO <501.50> EO_0: target index 3 tasked
2026-10-26 21:02:13,013 sats.satellite.EO_0 INFO <501.50> EO_0: Target(Samrong, TH) imaging timing: requested capture at 529.5
2026-10-26 21:02:13,014 sats.satellite.EO_0 INFO <501.50> EO_0: Target(Samrong, TH) tasked for imaging
2026-10-26 21:02:13,014 sats.satellite.EO_0 INFO <501.50> EO_0: Target(Samrong, TH) window enabled: 440.3 to 557.4
2026-10-26 21:02:13,015 sats.satellite.EO_0 INFO <501.50> EO_0: setting timed terminal event at 557.4
2026-10-26 21:02:13,025 sats.satellite.EO_0 INFO <529.00> EO_0: imaged Target(Samrong, TH)
2026-10-26 21:02:13,026 data.base INFO <529.00> Total reward: {'EO_0': np.float64(0.3456253471201201)}
2026-10-26 21:02:13,027 sats.satellite.EO_0 INFO <529.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:13,127 gym INFO <529.00> Step reward: {'EO_0': np.float64(0.3456253471201201)}
2026-10-26 21:02:13,159 gym INFO <529.00> === STARTING STEP ===
2026-10-26 21:02:13,161 sats.satellite.EO_0 INFO <529.00> EO_0: target index 2 tasked
2026-10-26 21:02:13,162 sats.satellite.EO_0 INFO <529.00> EO_0: Target(Ban Bang Lamung, TH) imaging timing: requested capture at 547.8
2026-10-26 21:02:13,163 sats.satellite.EO_0 INFO <529.00> EO_0: Target(Ban Bang Lamung, TH) tasked for imaging
2026-10-26 21:02:13,164 sats.satellite.EO_0 INFO <529.00> EO_0: Target(Ban Bang Lamung, TH) window enabled: 453.9 to 566.6
2026-10-26 21:02:13,164 sats.satellite.EO_0 INFO <529.00> EO_0: setting timed terminal event at 566.6
2026-10-26 21:02:13,171 sats.satellite.EO_0 INFO <547.50> EO_0: imaged Target(Ban Bang Lamung, TH)
2026-10-26 21:02:13,172 data.base INFO <547.50> Total reward: {'EO_0': np.float64(0.1966497718953449)}
2026-10-26 21:02:13,173 sats.satellite.EO_0 INFO <547.50> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:13,248 gym INFO <547.50> Step reward: {'EO_0': np.float64(0.1966497718953449)}
2026-10-26 21:02:13,276 gym INFO <547.50> === STARTING STEP ===
2026-10-26 21:02:13,277 sats.satellite.EO_0 INFO <547.50> EO_0: target index 1 tasked
2026-10-26 21:02:13,278 sats.satellite.EO_0 INFO <547.50> EO_0: Target(Koh Kong, KH) imaging timing: requested capture at 580.8
2026-10-26 21:02:13,278 sats.satellite.EO_0 INFO <547.50> EO_0: Target(Koh Kong, KH) tasked for imaging
2026-10-26 21:02:13,279 sats.satellite.EO_0 INFO <547.50> EO_0: Target(Koh Kong, KH) window enabled: 492.1 to 614.1
2026-10-26 21:02:13,280 sats.satellite.EO_0 INFO <547.50> EO_0: setting timed terminal event at 614.1
2026-10-26 21:02:13,310 sats.satellite.EO_1 INFO <569.50> EO_1: imaged Target(Amdel, MA)
2026-10-26 21:02:13,311 data.base INFO <569.50> Total reward: {'EO_1': np.float64(0.7745063964760883)}
2026-10-26 21:02:13,312 sats.satellite.EO_1 INFO <569.50> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:13,395 gym INFO <569.50> Step reward: {'EO_1': np.float64(0.7745063964760883)}
2026-10-26 21:02:13,435 gym INFO <569.50> === STARTING STEP ===
2026-10-26 21:02:13,451 sats.satellite.EO_1 INFO <569.50> EO_1: target index 0 tasked
2026-10-26 21:02:13,456 sats.satellite.EO_1 INFO <569.50> EO_1: Target(’Ali Ben Sliman, MA) imaging timing: requested capture at 595.4
2026-10-26 21:02:13,460 sats.satellite.EO_1 INFO <569.50> EO_1: Target(’Ali Ben Sliman, MA) tasked for imaging
2026-10-26 21:02:13,464 sats.satellite.EO_1 INFO <569.50> EO_1: Target(’Ali Ben Sliman, MA) window enabled: 567.9 to 621.4
2026-10-26 21:02:13,465 sats.satellite.EO_1 INFO <569.50> EO_1: setting timed terminal event at 621.4
2026-10-26 21:02:13,471 sats.satellite.EO_0 INFO <583.00> EO_0: imaged Target(Koh Kong, KH)
2026-10-26 21:02:13,473 data.base INFO <583.00> Total reward: {'EO_0': np.float64(0.00857099491719181)}
2026-10-26 21:02:13,474 sats.satellite.EO_0 INFO <583.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:13,569 gym INFO <583.00> Step reward: {'EO_0': np.float64(0.00857099491719181)}
2026-10-26 21:02:13,606 gym INFO <583.00> === STARTING STEP ===
2026-10-26 21:02:13,606 sats.satellite.EO_0 INFO <583.00> EO_0: target index 3 tasked
2026-10-26 21:02:13,607 sats.satellite.EO_0 INFO <583.00> EO_0: Target(Cần Thơ, VN) imaging timing: requested capture at 628.3
2026-10-26 21:02:13,608 sats.satellite.EO_0 INFO <583.00> EO_0: Target(Cần Thơ, VN) tasked for imaging
2026-10-26 21:02:13,609 sats.satellite.EO_0 INFO <583.00> EO_0: Target(Cần Thơ, VN) window enabled: 540.1 to 673.5
2026-10-26 21:02:13,610 sats.satellite.EO_0 INFO <583.00> EO_0: setting timed terminal event at 673.5
2026-10-26 21:02:13,618 sats.satellite.EO_1 INFO <595.00> EO_1: imaged Target(’Ali Ben Sliman, MA)
2026-10-26 21:02:13,620 data.base INFO <595.00> Total reward: {'EO_1': np.float64(0.48662061623566244)}
2026-10-26 21:02:13,621 sats.satellite.EO_1 INFO <595.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:13,733 gym INFO <595.00> Step reward: {'EO_1': np.float64(0.48662061623566244)}
2026-10-26 21:02:13,771 gym INFO <595.00> === STARTING STEP ===
2026-10-26 21:02:13,773 sats.satellite.EO_1 INFO <595.00> EO_1: target index 0 tasked
2026-10-26 21:02:13,774 sats.satellite.EO_1 INFO <595.00> EO_1: Target(Oulad Yaïch, MA) imaging timing: requested capture at 617.4
2026-10-26 21:02:13,774 sats.satellite.EO_1 INFO <595.00> EO_1: Target(Oulad Yaïch, MA) tasked for imaging
2026-10-26 21:02:13,775 sats.satellite.EO_1 INFO <595.00> EO_1: Target(Oulad Yaïch, MA) window enabled: 581.5 to 639.7
2026-10-26 21:02:13,776 sats.satellite.EO_1 INFO <595.00> EO_1: setting timed terminal event at 639.7
2026-10-26 21:02:13,786 sats.satellite.EO_1 INFO <617.00> EO_1: imaged Target(Oulad Yaïch, MA)
2026-10-26 21:02:13,789 data.base INFO <617.00> Total reward: {'EO_1': np.float64(0.6838681854948269)}
2026-10-26 21:02:13,789 sats.satellite.EO_1 INFO <617.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:13,873 gym INFO <617.00> Step reward: {'EO_1': np.float64(0.6838681854948269)}
2026-10-26 21:02:13,901 gym INFO <617.00> === STARTING STEP ===
2026-10-26 21:02:13,902 sats.satellite.EO_1 INFO <617.00> EO_1: target index 4 tasked
2026-10-26 21:02:13,903 sats.satellite.EO_1 INFO <617.00> EO_1: Target(Ain Kansara, MA) imaging timing: requested capture at 656.4
2026-10-26 21:02:13,904 sats.satellite.EO_1 INFO <617.00> EO_1: Target(Ain Kansara, MA) tasked for imaging
2026-10-26 21:02:13,905 sats.satellite.EO_1 INFO <617.00> EO_1: Target(Ain Kansara, MA) window enabled: 594.7 to 695.7
2026-10-26 21:02:13,905 sats.satellite.EO_1 INFO <617.00> EO_1: setting timed terminal event at 695.7
2026-10-26 21:02:13,913 sats.satellite.EO_0 INFO <628.00> EO_0: imaged Target(Cần Thơ, VN)
2026-10-26 21:02:13,915 data.base INFO <628.00> Total reward: {'EO_0': np.float64(0.3663438367949623)}
2026-10-26 21:02:13,918 sats.satellite.EO_0 INFO <628.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:13,985 gym INFO <628.00> Step reward: {'EO_0': np.float64(0.3663438367949623)}
2026-10-26 21:02:14,011 gym INFO <628.00> === STARTING STEP ===
2026-10-26 21:02:14,012 sats.satellite.EO_0 INFO <628.00> EO_0: target index 0 tasked
2026-10-26 21:02:14,032 sats.satellite.EO_0 INFO <628.00> EO_0: Target(Rạch Giá, VN) imaging timing: requested capture at 645.3
2026-10-26 21:02:14,036 sats.satellite.EO_0 INFO <628.00> EO_0: Target(Rạch Giá, VN) tasked for imaging
2026-10-26 21:02:14,038 sats.satellite.EO_0 INFO <628.00> EO_0: Target(Rạch Giá, VN) window enabled: 534.7 to 662.7
2026-10-26 21:02:14,039 sats.satellite.EO_0 INFO <628.00> EO_0: setting timed terminal event at 662.7
2026-10-26 21:02:14,045 sats.satellite.EO_0 INFO <645.00> EO_0: imaged Target(Rạch Giá, VN)
2026-10-26 21:02:14,047 data.base INFO <645.00> Total reward: {'EO_0': np.float64(0.6520300383357149)}
2026-10-26 21:02:14,048 sats.satellite.EO_0 INFO <645.00> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:14,134 gym INFO <645.00> Step reward: {'EO_0': np.float64(0.6520300383357149)}
2026-10-26 21:02:14,175 gym INFO <645.00> === STARTING STEP ===
2026-10-26 21:02:14,176 sats.satellite.EO_0 INFO <645.00> EO_0: target index 0 tasked
2026-10-26 21:02:14,177 sats.satellite.EO_0 INFO <645.00> EO_0: Target(Bình Minh, VN) imaging timing: requested capture at 659.0
2026-10-26 21:02:14,177 sats.satellite.EO_0 INFO <645.00> EO_0: Target(Bình Minh, VN) tasked for imaging
2026-10-26 21:02:14,178 sats.satellite.EO_0 INFO <645.00> EO_0: Target(Bình Minh, VN) window enabled: 539.2 to 673.0
2026-10-26 21:02:14,179 sats.satellite.EO_0 INFO <645.00> EO_0: setting timed terminal event at 673.0
2026-10-26 21:02:14,184 sats.satellite.EO_1 INFO <656.00> EO_1: imaged Target(Ain Kansara, MA)
2026-10-26 21:02:14,186 data.base INFO <656.00> Total reward: {'EO_1': np.float64(0.5837788834463559)}
2026-10-26 21:02:14,186 sats.satellite.EO_1 INFO <656.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:14,258 gym INFO <656.00> Step reward: {'EO_1': np.float64(0.5837788834463559)}
2026-10-26 21:02:14,305 gym INFO <656.00> === STARTING STEP ===
2026-10-26 21:02:14,307 sats.satellite.EO_1 INFO <656.00> EO_1: target index 7 tasked
2026-10-26 21:02:14,313 sats.satellite.EO_1 INFO <656.00> EO_1: Target(Algeciras, ES) imaging timing: requested capture at 688.9
2026-10-26 21:02:14,314 sats.satellite.EO_1 INFO <656.00> EO_1: Target(Algeciras, ES) tasked for imaging
2026-10-26 21:02:14,316 sats.satellite.EO_1 INFO <656.00> EO_1: Target(Algeciras, ES) window enabled: 586.9 to 721.9
2026-10-26 21:02:14,316 sats.satellite.EO_1 INFO <656.00> EO_1: setting timed terminal event at 721.9
2026-10-26 21:02:14,319 sats.satellite.EO_0 INFO <658.50> EO_0: imaged Target(Bình Minh, VN)
2026-10-26 21:02:14,321 data.base INFO <658.50> Total reward: {'EO_0': np.float64(0.6409562025459951)}
2026-10-26 21:02:14,321 sats.satellite.EO_0 INFO <658.50> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:14,390 gym INFO <658.50> Step reward: {'EO_0': np.float64(0.6409562025459951)}
2026-10-26 21:02:14,417 gym INFO <658.50> === STARTING STEP ===
2026-10-26 21:02:14,418 sats.satellite.EO_0 INFO <658.50> EO_0: target index 0 tasked
2026-10-26 21:02:14,419 sats.satellite.EO_0 INFO <658.50> EO_0: Target(Long Thành, VN) imaging timing: requested capture at 667.9
2026-10-26 21:02:14,420 sats.satellite.EO_0 INFO <658.50> EO_0: Target(Long Thành, VN) tasked for imaging
2026-10-26 21:02:14,421 sats.satellite.EO_0 INFO <658.50> EO_0: Target(Long Thành, VN) window enabled: 541.8 to 677.3
2026-10-26 21:02:14,421 sats.satellite.EO_0 INFO <658.50> EO_0: setting timed terminal event at 677.3
2026-10-26 21:02:14,431 sats.satellite.EO_0 INFO <677.50> EO_0: timed termination at 677.3 for Target(Long Thành, VN) window
2026-10-26 21:02:14,432 data.base INFO <677.50> Total reward: {}
2026-10-26 21:02:14,433 sats.satellite.EO_0 INFO <677.50> EO_0: Satellite EO_0 requires retasking
2026-10-26 21:02:14,509 gym INFO <677.50> Step reward: {}
2026-10-26 21:02:14,548 gym INFO <677.50> === STARTING STEP ===
2026-10-26 21:02:14,549 sats.satellite.EO_0 INFO <677.50> EO_0: target index 20 tasked
2026-10-26 21:02:14,551 sats.satellite.EO_0 INFO <677.50> EO_0: Target(Tena, CO) imaging timing: requested capture at 3964.8
2026-10-26 21:02:14,552 sats.satellite.EO_0 INFO <677.50> EO_0: Target(Tena, CO) tasked for imaging
2026-10-26 21:02:14,553 sats.satellite.EO_0 INFO <677.50> EO_0: Target(Tena, CO) window enabled: 3901.7 to 4027.9
2026-10-26 21:02:14,554 sats.satellite.EO_0 INFO <677.50> EO_0: setting timed terminal event at 4027.9
2026-10-26 21:02:14,559 sats.satellite.EO_1 INFO <688.50> EO_1: imaged Target(Algeciras, ES)
2026-10-26 21:02:14,561 data.base INFO <688.50> Total reward: {'EO_1': np.float64(0.025730269188953946)}
2026-10-26 21:02:14,562 sats.satellite.EO_1 INFO <688.50> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:14,661 gym INFO <688.50> Step reward: {'EO_1': np.float64(0.025730269188953946)}
2026-10-26 21:02:14,703 gym INFO <688.50> === STARTING STEP ===
2026-10-26 21:02:14,706 sats.satellite.EO_1 INFO <688.50> EO_1: target index 0 tasked
2026-10-26 21:02:14,706 sats.satellite.EO_1 INFO <688.50> EO_1: Target(Ain Kansara, MA) imaging timing: requested capture at 692.1
2026-10-26 21:02:14,707 sats.satellite.EO_1 INFO <688.50> EO_1: Target(Ain Kansara, MA) tasked for imaging
2026-10-26 21:02:14,708 sats.satellite.EO_1 INFO <688.50> EO_1: Target(Ain Kansara, MA) window enabled: 594.7 to 695.7
2026-10-26 21:02:14,708 sats.satellite.EO_1 INFO <688.50> EO_1: setting timed terminal event at 695.7
2026-10-26 21:02:14,712 sats.satellite.EO_1 INFO <696.00> EO_1: timed termination at 695.7 for Target(Ain Kansara, MA) window
2026-10-26 21:02:14,713 data.base INFO <696.00> Total reward: {}
2026-10-26 21:02:14,715 sats.satellite.EO_1 INFO <696.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:14,804 gym INFO <696.00> Step reward: {}
2026-10-26 21:02:14,846 gym INFO <696.00> === STARTING STEP ===
2026-10-26 21:02:14,847 sats.satellite.EO_1 INFO <696.00> EO_1: target index 0 tasked
2026-10-26 21:02:14,848 sats.satellite.EO_1 INFO <696.00> EO_1: Target(Teroual, MA) imaging timing: requested capture at 699.4
2026-10-26 21:02:14,848 sats.satellite.EO_1 INFO <696.00> EO_1: Target(Teroual, MA) tasked for imaging
2026-10-26 21:02:14,849 sats.satellite.EO_1 INFO <696.00> EO_1: Target(Teroual, MA) window enabled: 585.4 to 702.9
2026-10-26 21:02:14,849 sats.satellite.EO_1 INFO <696.00> EO_1: setting timed terminal event at 702.9
2026-10-26 21:02:14,854 sats.satellite.EO_1 INFO <701.00> EO_1: imaged Target(Teroual, MA)
2026-10-26 21:02:14,854 data.base INFO <701.00> Total reward: {'EO_1': np.float64(0.07431902323793338)}
2026-10-26 21:02:14,854 sats.satellite.EO_1 INFO <701.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:14,945 gym INFO <701.00> Step reward: {'EO_1': np.float64(0.07431902323793338)}
2026-10-26 21:02:14,991 gym INFO <701.00> === STARTING STEP ===
2026-10-26 21:02:14,992 sats.satellite.EO_1 INFO <701.00> EO_1: target index 2 tasked
2026-10-26 21:02:14,992 sats.satellite.EO_1 INFO <701.00> EO_1: Target(Dar El Kebdani, MA) imaging timing: requested capture at 714.0
2026-10-26 21:02:14,993 sats.satellite.EO_1 INFO <701.00> EO_1: Target(Dar El Kebdani, MA) tasked for imaging
2026-10-26 21:02:14,996 sats.satellite.EO_1 INFO <701.00> EO_1: Target(Dar El Kebdani, MA) window enabled: 618.3 to 727.1
2026-10-26 21:02:15,000 sats.satellite.EO_1 INFO <701.00> EO_1: setting timed terminal event at 727.1
2026-10-26 21:02:15,030 sats.satellite.EO_1 INFO <725.50> EO_1: imaged Target(Dar El Kebdani, MA)
2026-10-26 21:02:15,031 data.base INFO <725.50> Total reward: {'EO_1': np.float64(0.046522390704619956)}
2026-10-26 21:02:15,033 sats.satellite.EO_1 INFO <725.50> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:15,135 gym INFO <725.50> Step reward: {'EO_1': np.float64(0.046522390704619956)}
2026-10-26 21:02:15,165 gym INFO <725.50> === STARTING STEP ===
2026-10-26 21:02:15,167 sats.satellite.EO_1 INFO <725.50> EO_1: target index 5 tasked
2026-10-26 21:02:15,169 sats.satellite.EO_1 INFO <725.50> EO_1: Target(Atarfe, ES) imaging timing: requested capture at 739.6
2026-10-26 21:02:15,170 sats.satellite.EO_1 INFO <725.50> EO_1: Target(Atarfe, ES) tasked for imaging
2026-10-26 21:02:15,171 sats.satellite.EO_1 INFO <725.50> EO_1: Target(Atarfe, ES) window enabled: 617.5 to 753.7
2026-10-26 21:02:15,171 sats.satellite.EO_1 INFO <725.50> EO_1: setting timed terminal event at 753.7
2026-10-26 21:02:15,188 sats.satellite.EO_1 INFO <754.00> EO_1: timed termination at 753.7 for Target(Atarfe, ES) window
2026-10-26 21:02:15,190 data.base INFO <754.00> Total reward: {}
2026-10-26 21:02:15,190 sats.satellite.EO_1 INFO <754.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:15,272 gym INFO <754.00> Step reward: {}
2026-10-26 21:02:15,304 gym INFO <754.00> === STARTING STEP ===
2026-10-26 21:02:15,307 sats.satellite.EO_1 INFO <754.00> EO_1: target index 0 tasked
2026-10-26 21:02:15,308 sats.satellite.EO_1 INFO <754.00> EO_1: Target(Maracena, ES) imaging timing: requested capture at 754.5
2026-10-26 21:02:15,309 sats.satellite.EO_1 INFO <754.00> EO_1: Target(Maracena, ES) tasked for imaging
2026-10-26 21:02:15,311 sats.satellite.EO_1 INFO <754.00> EO_1: Target(Maracena, ES) window enabled: 618.1 to 754.2
2026-10-26 21:02:15,312 sats.satellite.EO_1 INFO <754.00> EO_1: setting timed terminal event at 754.2
2026-10-26 21:02:15,314 sats.satellite.EO_1 INFO <754.50> EO_1: timed termination at 754.2 for Target(Maracena, ES) window
2026-10-26 21:02:15,315 data.base INFO <754.50> Total reward: {}
2026-10-26 21:02:15,315 sats.satellite.EO_1 INFO <754.50> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:15,424 gym INFO <754.50> Step reward: {}
2026-10-26 21:02:15,460 gym INFO <754.50> === STARTING STEP ===
2026-10-26 21:02:15,462 sats.satellite.EO_1 INFO <754.50> EO_1: target index 4 tasked
2026-10-26 21:02:15,464 sats.satellite.EO_1 INFO <754.50> EO_1: Target(Alhama de Murcia, ES) imaging timing: requested capture at 770.5
2026-10-26 21:02:15,465 sats.satellite.EO_1 INFO <754.50> EO_1: Target(Alhama de Murcia, ES) tasked for imaging
2026-10-26 21:02:15,466 sats.satellite.EO_1 INFO <754.50> EO_1: Target(Alhama de Murcia, ES) window enabled: 651.4 to 786.4
2026-10-26 21:02:15,466 sats.satellite.EO_1 INFO <754.50> EO_1: setting timed terminal event at 786.4
2026-10-26 21:02:15,468 sats.satellite.EO_1 INFO <755.00> EO_1: imaged Target(Alhama de Murcia, ES)
2026-10-26 21:02:15,470 data.base INFO <755.00> Total reward: {'EO_1': np.float64(0.002593261933429352)}
2026-10-26 21:02:15,471 sats.satellite.EO_1 INFO <755.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:15,558 gym INFO <755.00> Step reward: {'EO_1': np.float64(0.002593261933429352)}
2026-10-26 21:02:15,590 gym INFO <755.00> === STARTING STEP ===
2026-10-26 21:02:15,592 sats.satellite.EO_1 INFO <755.00> EO_1: target index 1 tasked
2026-10-26 21:02:15,593 sats.satellite.EO_1 INFO <755.00> EO_1: Target(Almería, ES) imaging timing: requested capture at 759.0
2026-10-26 21:02:15,594 sats.satellite.EO_1 INFO <755.00> EO_1: Target(Almería, ES) tasked for imaging
2026-10-26 21:02:15,595 sats.satellite.EO_1 INFO <755.00> EO_1: Target(Almería, ES) window enabled: 632.4 to 763.1
2026-10-26 21:02:15,596 sats.satellite.EO_1 INFO <755.00> EO_1: setting timed terminal event at 763.1
2026-10-26 21:02:15,602 sats.satellite.EO_1 INFO <763.50> EO_1: timed termination at 763.1 for Target(Almería, ES) window
2026-10-26 21:02:15,603 data.base INFO <763.50> Total reward: {}
2026-10-26 21:02:15,604 sats.satellite.EO_1 INFO <763.50> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:15,703 gym INFO <763.50> Step reward: {}
2026-10-26 21:02:15,739 gym INFO <763.50> === STARTING STEP ===
2026-10-26 21:02:15,741 sats.satellite.EO_1 INFO <763.50> EO_1: target index 1 tasked
2026-10-26 21:02:15,742 sats.satellite.EO_1 INFO <763.50> EO_1: Target(Villarrobledo, ES) imaging timing: requested capture at 772.5
2026-10-26 21:02:15,744 sats.satellite.EO_1 INFO <763.50> EO_1: Target(Villarrobledo, ES) tasked for imaging
2026-10-26 21:02:15,745 sats.satellite.EO_1 INFO <763.50> EO_1: Target(Villarrobledo, ES) window enabled: 650.0 to 781.6
2026-10-26 21:02:15,745 sats.satellite.EO_1 INFO <763.50> EO_1: setting timed terminal event at 781.6
2026-10-26 21:02:15,752 sats.satellite.EO_1 INFO <782.00> EO_1: timed termination at 781.6 for Target(Villarrobledo, ES) window
2026-10-26 21:02:15,753 data.base INFO <782.00> Total reward: {}
2026-10-26 21:02:15,754 sats.satellite.EO_1 INFO <782.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:15,836 gym INFO <782.00> Step reward: {}
2026-10-26 21:02:15,873 gym INFO <782.00> === STARTING STEP ===
2026-10-26 21:02:15,875 sats.satellite.EO_1 INFO <782.00> EO_1: target index 4 tasked
2026-10-26 21:02:15,876 sats.satellite.EO_1 INFO <782.00> EO_1: Target(Valencia, ES) imaging timing: requested capture at 797.2
2026-10-26 21:02:15,876 sats.satellite.EO_1 INFO <782.00> EO_1: Target(Valencia, ES) tasked for imaging
2026-10-26 21:02:15,882 sats.satellite.EO_1 INFO <782.00> EO_1: Target(Valencia, ES) window enabled: 676.7 to 812.4
2026-10-26 21:02:15,894 sats.satellite.EO_1 INFO <782.00> EO_1: setting timed terminal event at 812.4
2026-10-26 21:02:15,915 sats.satellite.EO_1 INFO <810.50> EO_1: imaged Target(Valencia, ES)
2026-10-26 21:02:15,917 data.base INFO <810.50> Total reward: {'EO_1': np.float64(0.060854626884254755)}
2026-10-26 21:02:15,918 sats.satellite.EO_1 INFO <810.50> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:16,004 gym INFO <810.50> Step reward: {'EO_1': np.float64(0.060854626884254755)}
2026-10-26 21:02:16,042 gym INFO <810.50> === STARTING STEP ===
2026-10-26 21:02:16,044 sats.satellite.EO_1 INFO <810.50> EO_1: target index 0 tasked
2026-10-26 21:02:16,046 sats.satellite.EO_1 INFO <810.50> EO_1: Target(Benifayó, ES) imaging timing: requested capture at 811.0
2026-10-26 21:02:16,048 sats.satellite.EO_1 INFO <810.50> EO_1: Target(Benifayó, ES) tasked for imaging
2026-10-26 21:02:16,051 sats.satellite.EO_1 INFO <810.50> EO_1: Target(Benifayó, ES) window enabled: 674.3 to 810.6
2026-10-26 21:02:16,065 sats.satellite.EO_1 INFO <810.50> EO_1: setting timed terminal event at 810.6
2026-10-26 21:02:16,071 sats.satellite.EO_1 INFO <811.00> EO_1: timed termination at 810.6 for Target(Benifayó, ES) window
2026-10-26 21:02:16,074 data.base INFO <811.00> Total reward: {}
2026-10-26 21:02:16,075 sats.satellite.EO_1 INFO <811.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:16,165 gym INFO <811.00> Step reward: {}
2026-10-26 21:02:16,198 gym INFO <811.00> === STARTING STEP ===
2026-10-26 21:02:16,200 sats.satellite.EO_1 INFO <811.00> EO_1: target index 4 tasked
2026-10-26 21:02:16,201 sats.satellite.EO_1 INFO <811.00> EO_1: Target(Binisalem, ES) imaging timing: requested capture at 832.4
2026-10-26 21:02:16,201 sats.satellite.EO_1 INFO <811.00> EO_1: Target(Binisalem, ES) tasked for imaging
2026-10-26 21:02:16,202 sats.satellite.EO_1 INFO <811.00> EO_1: Target(Binisalem, ES) window enabled: 717.7 to 853.9
2026-10-26 21:02:16,202 sats.satellite.EO_1 INFO <811.00> EO_1: setting timed terminal event at 853.9
2026-10-26 21:02:16,205 sats.satellite.EO_1 INFO <811.50> EO_1: imaged Target(Binisalem, ES)
2026-10-26 21:02:16,207 data.base INFO <811.50> Total reward: {'EO_1': np.float64(0.8723150559752924)}
2026-10-26 21:02:16,207 sats.satellite.EO_1 INFO <811.50> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:16,287 gym INFO <811.50> Step reward: {'EO_1': np.float64(0.8723150559752924)}
2026-10-26 21:02:16,326 gym INFO <811.50> === STARTING STEP ===
2026-10-26 21:02:16,328 sats.satellite.EO_1 INFO <811.50> EO_1: target index 3 tasked
2026-10-26 21:02:16,329 sats.satellite.EO_1 INFO <811.50> EO_1: Target(Calviá, ES) imaging timing: requested capture at 830.2
2026-10-26 21:02:16,331 sats.satellite.EO_1 INFO <811.50> EO_1: Target(Calviá, ES) tasked for imaging
2026-10-26 21:02:16,332 sats.satellite.EO_1 INFO <811.50> EO_1: Target(Calviá, ES) window enabled: 712.8 to 848.9
2026-10-26 21:02:16,333 sats.satellite.EO_1 INFO <811.50> EO_1: setting timed terminal event at 848.9
2026-10-26 21:02:16,347 sats.satellite.EO_1 INFO <843.00> EO_1: imaged Target(Calviá, ES)
2026-10-26 21:02:16,349 data.base INFO <843.00> Total reward: {'EO_1': np.float64(0.06112811158338784)}
2026-10-26 21:02:16,351 sats.satellite.EO_1 INFO <843.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:16,418 gym INFO <843.00> Step reward: {'EO_1': np.float64(0.06112811158338784)}
2026-10-26 21:02:16,506 gym INFO <843.00> === STARTING STEP ===
2026-10-26 21:02:16,508 sats.satellite.EO_1 INFO <843.00> EO_1: target index 0 tasked
2026-10-26 21:02:16,509 sats.satellite.EO_1 INFO <843.00> EO_1: Target(Binisalem, ES) imaging timing: requested capture at 848.4
2026-10-26 21:02:16,510 sats.satellite.EO_1 INFO <843.00> EO_1: Target(Binisalem, ES) tasked for imaging
2026-10-26 21:02:16,511 sats.satellite.EO_1 INFO <843.00> EO_1: Target(Binisalem, ES) window enabled: 717.7 to 853.9
2026-10-26 21:02:16,511 sats.satellite.EO_1 INFO <843.00> EO_1: setting timed terminal event at 853.9
2026-10-26 21:02:16,517 sats.satellite.EO_1 INFO <852.00> EO_1: imaged Target(Binisalem, ES)
2026-10-26 21:02:16,519 data.base INFO <852.00> Total reward: {}
2026-10-26 21:02:16,520 sats.satellite.EO_1 INFO <852.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:16,595 gym INFO <852.00> Step reward: {}
2026-10-26 21:02:16,639 gym INFO <852.00> === STARTING STEP ===
2026-10-26 21:02:16,641 sats.satellite.EO_1 INFO <852.00> EO_1: target index 9 tasked
2026-10-26 21:02:16,643 sats.satellite.EO_1 INFO <852.00> EO_1: Target(Noceto, IT) imaging timing: requested capture at 907.4
2026-10-26 21:02:16,644 sats.satellite.EO_1 INFO <852.00> EO_1: Target(Noceto, IT) tasked for imaging
2026-10-26 21:02:16,645 sats.satellite.EO_1 INFO <852.00> EO_1: Target(Noceto, IT) window enabled: 855.4 to 959.4
2026-10-26 21:02:16,645 sats.satellite.EO_1 INFO <852.00> EO_1: setting timed terminal event at 959.4
2026-10-26 21:02:16,664 sats.satellite.EO_1 INFO <907.00> EO_1: imaged Target(Noceto, IT)
2026-10-26 21:02:16,666 data.base INFO <907.00> Total reward: {'EO_1': np.float64(0.7179888850051652)}
2026-10-26 21:02:16,666 sats.satellite.EO_1 INFO <907.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:16,772 gym INFO <907.00> Step reward: {'EO_1': np.float64(0.7179888850051652)}
2026-10-26 21:02:16,817 gym INFO <907.00> === STARTING STEP ===
2026-10-26 21:02:16,818 sats.satellite.EO_1 INFO <907.00> EO_1: target index 0 tasked
2026-10-26 21:02:16,819 sats.satellite.EO_1 INFO <907.00> EO_1: Target(Volvera, IT) imaging timing: requested capture at 910.3
2026-10-26 21:02:16,820 sats.satellite.EO_1 INFO <907.00> EO_1: Target(Volvera, IT) tasked for imaging
2026-10-26 21:02:16,821 sats.satellite.EO_1 INFO <907.00> EO_1: Target(Volvera, IT) window enabled: 839.4 to 913.7
2026-10-26 21:02:16,821 sats.satellite.EO_1 INFO <907.00> EO_1: setting timed terminal event at 913.7
2026-10-26 21:02:16,828 sats.satellite.EO_1 INFO <914.00> EO_1: timed termination at 913.7 for Target(Volvera, IT) window
2026-10-26 21:02:16,830 data.base INFO <914.00> Total reward: {}
2026-10-26 21:02:16,834 sats.satellite.EO_1 INFO <914.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:16,919 gym INFO <914.00> Step reward: {}
2026-10-26 21:02:16,958 gym INFO <914.00> === STARTING STEP ===
2026-10-26 21:02:16,960 sats.satellite.EO_1 INFO <914.00> EO_1: target index 1 tasked
2026-10-26 21:02:16,960 sats.satellite.EO_1 INFO <914.00> EO_1: Target(Terralba, IT) imaging timing: requested capture at 916.3
2026-10-26 21:02:16,962 sats.satellite.EO_1 INFO <914.00> EO_1: Target(Terralba, IT) tasked for imaging
2026-10-26 21:02:16,964 sats.satellite.EO_1 INFO <914.00> EO_1: Target(Terralba, IT) window enabled: 800.6 to 918.6
2026-10-26 21:02:16,964 sats.satellite.EO_1 INFO <914.00> EO_1: setting timed terminal event at 918.6
2026-10-26 21:02:16,970 sats.satellite.EO_1 INFO <919.00> EO_1: timed termination at 918.6 for Target(Terralba, IT) window
2026-10-26 21:02:16,971 data.base INFO <919.00> Total reward: {}
2026-10-26 21:02:16,972 sats.satellite.EO_1 INFO <919.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:17,055 gym INFO <919.00> Step reward: {}
2026-10-26 21:02:17,098 gym INFO <919.00> === STARTING STEP ===
2026-10-26 21:02:17,100 sats.satellite.EO_1 INFO <919.00> EO_1: target index 0 tasked
2026-10-26 21:02:17,100 sats.satellite.EO_1 INFO <919.00> EO_1: Target(Gallarate, IT) imaging timing: requested capture at 919.5
2026-10-26 21:02:17,101 sats.satellite.EO_1 INFO <919.00> EO_1: Target(Gallarate, IT) tasked for imaging
2026-10-26 21:02:17,102 sats.satellite.EO_1 INFO <919.00> EO_1: Target(Gallarate, IT) window enabled: 871.6 to 920.0
2026-10-26 21:02:17,103 sats.satellite.EO_1 INFO <919.00> EO_1: setting timed terminal event at 920.0
2026-10-26 21:02:17,106 sats.satellite.EO_1 INFO <920.50> EO_1: timed termination at 920.0 for Target(Gallarate, IT) window
2026-10-26 21:02:17,107 data.base INFO <920.50> Total reward: {}
2026-10-26 21:02:17,108 sats.satellite.EO_1 INFO <920.50> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:17,187 gym INFO <920.50> Step reward: {}
2026-10-26 21:02:17,219 gym INFO <920.50> === STARTING STEP ===
2026-10-26 21:02:17,220 sats.satellite.EO_1 INFO <920.50> EO_1: target index 5 tasked
2026-10-26 21:02:17,221 sats.satellite.EO_1 INFO <920.50> EO_1: Target(Castelfranco di Sotto, IT) imaging timing: requested capture at 946.5
2026-10-26 21:02:17,222 sats.satellite.EO_1 INFO <920.50> EO_1: Target(Castelfranco di Sotto, IT) tasked for imaging
2026-10-26 21:02:17,222 sats.satellite.EO_1 INFO <920.50> EO_1: Target(Castelfranco di Sotto, IT) window enabled: 844.1 to 972.4
2026-10-26 21:02:17,223 sats.satellite.EO_1 INFO <920.50> EO_1: setting timed terminal event at 972.4
2026-10-26 21:02:17,233 sats.satellite.EO_1 INFO <946.00> EO_1: imaged Target(Castelfranco di Sotto, IT)
2026-10-26 21:02:17,235 data.base INFO <946.00> Total reward: {'EO_1': np.float64(0.11166066216117564)}
2026-10-26 21:02:17,236 sats.satellite.EO_1 INFO <946.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:17,322 gym INFO <946.00> Step reward: {'EO_1': np.float64(0.11166066216117564)}
2026-10-26 21:02:17,354 gym INFO <946.00> === STARTING STEP ===
2026-10-26 21:02:17,355 sats.satellite.EO_1 INFO <946.00> EO_1: target index 1 tasked
2026-10-26 21:02:17,356 sats.satellite.EO_1 INFO <946.00> EO_1: Target(La Spezia, IT) imaging timing: requested capture at 952.3
2026-10-26 21:02:17,356 sats.satellite.EO_1 INFO <946.00> EO_1: Target(La Spezia, IT) tasked for imaging
2026-10-26 21:02:17,357 sats.satellite.EO_1 INFO <946.00> EO_1: Target(La Spezia, IT) window enabled: 839.7 to 958.5
2026-10-26 21:02:17,358 sats.satellite.EO_1 INFO <946.00> EO_1: setting timed terminal event at 958.5
2026-10-26 21:02:17,365 sats.satellite.EO_1 INFO <955.00> EO_1: imaged Target(La Spezia, IT)
2026-10-26 21:02:17,366 data.base INFO <955.00> Total reward: {'EO_1': np.float64(0.5782403085024662)}
2026-10-26 21:02:17,367 sats.satellite.EO_1 INFO <955.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:17,452 gym INFO <955.00> Step reward: {'EO_1': np.float64(0.5782403085024662)}
2026-10-26 21:02:17,485 gym INFO <955.00> === STARTING STEP ===
2026-10-26 21:02:17,487 sats.satellite.EO_1 INFO <955.00> EO_1: target index 1 tasked
2026-10-26 21:02:17,488 sats.satellite.EO_1 INFO <955.00> EO_1: Target(Santa Croce sull’ Arno, IT) imaging timing: requested capture at 963.9
2026-10-26 21:02:17,488 sats.satellite.EO_1 INFO <955.00> EO_1: Target(Santa Croce sull’ Arno, IT) tasked for imaging
2026-10-26 21:02:17,489 sats.satellite.EO_1 INFO <955.00> EO_1: Target(Santa Croce sull’ Arno, IT) window enabled: 844.5 to 972.7
2026-10-26 21:02:17,490 sats.satellite.EO_1 INFO <955.00> EO_1: setting timed terminal event at 972.7
2026-10-26 21:02:17,495 sats.satellite.EO_1 INFO <964.50> EO_1: imaged Target(Santa Croce sull’ Arno, IT)
2026-10-26 21:02:17,496 data.base INFO <964.50> Total reward: {'EO_1': np.float64(0.6116193943991023)}
2026-10-26 21:02:17,497 sats.satellite.EO_1 INFO <964.50> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:17,579 gym INFO <964.50> Step reward: {'EO_1': np.float64(0.6116193943991023)}
2026-10-26 21:02:17,608 gym INFO <964.50> === STARTING STEP ===
2026-10-26 21:02:17,610 sats.satellite.EO_1 INFO <964.50> EO_1: target index 1 tasked
2026-10-26 21:02:17,611 sats.satellite.EO_1 INFO <964.50> EO_1: Target(Castelvetro di Modena, IT) imaging timing: requested capture at 968.7
2026-10-26 21:02:17,612 sats.satellite.EO_1 INFO <964.50> EO_1: Target(Castelvetro di Modena, IT) tasked for imaging
2026-10-26 21:02:17,615 sats.satellite.EO_1 INFO <964.50> EO_1: Target(Castelvetro di Modena, IT) window enabled: 856.9 to 972.9
2026-10-26 21:02:17,616 sats.satellite.EO_1 INFO <964.50> EO_1: setting timed terminal event at 972.9
2026-10-26 21:02:17,624 sats.satellite.EO_1 INFO <973.00> EO_1: timed termination at 972.9 for Target(Castelvetro di Modena, IT) window
2026-10-26 21:02:17,625 data.base INFO <973.00> Total reward: {}
2026-10-26 21:02:17,626 sats.satellite.EO_1 INFO <973.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:17,703 gym INFO <973.00> Step reward: {}
2026-10-26 21:02:17,742 gym INFO <973.00> === STARTING STEP ===
2026-10-26 21:02:17,744 sats.satellite.EO_1 INFO <973.00> EO_1: target index 6 tasked
2026-10-26 21:02:17,745 sats.satellite.EO_1 INFO <973.00> EO_1: Target(Marano di Napoli, IT) imaging timing: requested capture at 984.0
2026-10-26 21:02:17,746 sats.satellite.EO_1 INFO <973.00> EO_1: Target(Marano di Napoli, IT) tasked for imaging
2026-10-26 21:02:17,747 sats.satellite.EO_1 INFO <973.00> EO_1: Target(Marano di Napoli, IT) window enabled: 879.8 to 995.1
2026-10-26 21:02:17,748 sats.satellite.EO_1 INFO <973.00> EO_1: setting timed terminal event at 995.1
2026-10-26 21:02:17,757 sats.satellite.EO_1 INFO <995.50> EO_1: timed termination at 995.1 for Target(Marano di Napoli, IT) window
2026-10-26 21:02:17,759 data.base INFO <995.50> Total reward: {}
2026-10-26 21:02:17,759 sats.satellite.EO_1 INFO <995.50> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:17,852 gym INFO <995.50> Step reward: {}
2026-10-26 21:02:17,896 gym INFO <995.50> === STARTING STEP ===
2026-10-26 21:02:17,898 sats.satellite.EO_1 INFO <995.50> EO_1: target index 1 tasked
2026-10-26 21:02:17,899 sats.satellite.EO_1 INFO <995.50> EO_1: Target(Potenza, IT) imaging timing: requested capture at 1001.4
2026-10-26 21:02:17,899 sats.satellite.EO_1 INFO <995.50> EO_1: Target(Potenza, IT) tasked for imaging
2026-10-26 21:02:17,901 sats.satellite.EO_1 INFO <995.50> EO_1: Target(Potenza, IT) window enabled: 907.4 to 1007.2
2026-10-26 21:02:17,902 sats.satellite.EO_1 INFO <995.50> EO_1: setting timed terminal event at 1007.2
2026-10-26 21:02:17,910 sats.satellite.EO_1 INFO <1007.50> EO_1: timed termination at 1007.2 for Target(Potenza, IT) window
2026-10-26 21:02:17,912 data.base INFO <1007.50> Total reward: {}
2026-10-26 21:02:17,913 sats.satellite.EO_1 INFO <1007.50> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:17,982 gym INFO <1007.50> Step reward: {}
2026-10-26 21:02:18,018 gym INFO <1007.50> === STARTING STEP ===
2026-10-26 21:02:18,020 sats.satellite.EO_1 INFO <1007.50> EO_1: target index 5 tasked
2026-10-26 21:02:18,021 sats.satellite.EO_1 INFO <1007.50> EO_1: Target(Rogatec, SI) imaging timing: requested capture at 1017.1
2026-10-26 21:02:18,021 sats.satellite.EO_1 INFO <1007.50> EO_1: Target(Rogatec, SI) tasked for imaging
2026-10-26 21:02:18,022 sats.satellite.EO_1 INFO <1007.50> EO_1: Target(Rogatec, SI) window enabled: 932.6 to 1026.7
2026-10-26 21:02:18,023 sats.satellite.EO_1 INFO <1007.50> EO_1: setting timed terminal event at 1026.7
2026-10-26 21:02:18,036 sats.satellite.EO_1 INFO <1027.00> EO_1: timed termination at 1026.7 for Target(Rogatec, SI) window
2026-10-26 21:02:18,037 data.base INFO <1027.00> Total reward: {}
2026-10-26 21:02:18,038 sats.satellite.EO_1 INFO <1027.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:18,078 sats.satellite.EO_1 INFO <1027.00> EO_1: Finding opportunity windows from 4800.00 to 5400.00 seconds
2026-10-26 21:02:18,151 gym INFO <1027.00> Step reward: {}
2026-10-26 21:02:18,191 gym INFO <1027.00> === STARTING STEP ===
2026-10-26 21:02:18,193 sats.satellite.EO_1 INFO <1027.00> EO_1: target index 10 tasked
2026-10-26 21:02:18,194 sats.satellite.EO_1 INFO <1027.00> EO_1: Target(Ljubovija, RS) imaging timing: requested capture at 1055.8
2026-10-26 21:02:18,195 sats.satellite.EO_1 INFO <1027.00> EO_1: Target(Ljubovija, RS) tasked for imaging
2026-10-26 21:02:18,196 sats.satellite.EO_1 INFO <1027.00> EO_1: Target(Ljubovija, RS) window enabled: 948.3 to 1084.5
2026-10-26 21:02:18,197 sats.satellite.EO_1 INFO <1027.00> EO_1: setting timed terminal event at 1084.5
2026-10-26 21:02:18,209 sats.satellite.EO_1 INFO <1062.50> EO_1: imaged Target(Ljubovija, RS)
2026-10-26 21:02:18,213 data.base INFO <1062.50> Total reward: {'EO_1': np.float64(0.005514410220247971)}
2026-10-26 21:02:18,216 sats.satellite.EO_1 INFO <1062.50> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:18,295 gym INFO <1062.50> Step reward: {'EO_1': np.float64(0.005514410220247971)}
2026-10-26 21:02:18,334 gym INFO <1062.50> === STARTING STEP ===
2026-10-26 21:02:18,336 sats.satellite.EO_1 INFO <1062.50> EO_1: target index 13 tasked
2026-10-26 21:02:18,338 sats.satellite.EO_1 INFO <1062.50> EO_1: Target(Rankovce, MK) imaging timing: requested capture at 1082.7
2026-10-26 21:02:18,339 sats.satellite.EO_1 INFO <1062.50> EO_1: Target(Rankovce, MK) tasked for imaging
2026-10-26 21:02:18,340 sats.satellite.EO_1 INFO <1062.50> EO_1: Target(Rankovce, MK) window enabled: 987.4 to 1102.8
2026-10-26 21:02:18,340 sats.satellite.EO_1 INFO <1062.50> EO_1: setting timed terminal event at 1102.8
2026-10-26 21:02:18,352 sats.satellite.EO_1 INFO <1099.50> EO_1: imaged Target(Rankovce, MK)
2026-10-26 21:02:18,353 data.base INFO <1099.50> Total reward: {'EO_1': np.float64(0.023765420073078768)}
2026-10-26 21:02:18,354 sats.satellite.EO_1 INFO <1099.50> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:18,429 gym INFO <1099.50> Step reward: {'EO_1': np.float64(0.023765420073078768)}
2026-10-26 21:02:18,460 gym INFO <1099.50> === STARTING STEP ===
2026-10-26 21:02:18,462 sats.satellite.EO_1 INFO <1099.50> EO_1: target index 10 tasked
2026-10-26 21:02:18,463 sats.satellite.EO_1 INFO <1099.50> EO_1: Target(Covasna, RO) imaging timing: requested capture at 1134.5
2026-10-26 21:02:18,465 sats.satellite.EO_1 INFO <1099.50> EO_1: Target(Covasna, RO) tasked for imaging
2026-10-26 21:02:18,466 sats.satellite.EO_1 INFO <1099.50> EO_1: Target(Covasna, RO) window enabled: 1039.9 to 1169.4
2026-10-26 21:02:18,467 sats.satellite.EO_1 INFO <1099.50> EO_1: setting timed terminal event at 1169.4
2026-10-26 21:02:18,487 sats.satellite.EO_1 INFO <1145.50> EO_1: imaged Target(Covasna, RO)
2026-10-26 21:02:18,489 data.base INFO <1145.50> Total reward: {'EO_1': np.float64(0.0030979265085962784)}
2026-10-26 21:02:18,489 sats.satellite.EO_1 INFO <1145.50> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:18,555 gym INFO <1145.50> Step reward: {'EO_1': np.float64(0.0030979265085962784)}
2026-10-26 21:02:18,590 gym INFO <1145.50> === STARTING STEP ===
2026-10-26 21:02:18,592 sats.satellite.EO_1 INFO <1145.50> EO_1: target index 0 tasked
2026-10-26 21:02:18,593 sats.satellite.EO_1 INFO <1145.50> EO_1: Target(Boldeşti-Scăeni, RO) imaging timing: requested capture at 1157.3
2026-10-26 21:02:18,593 sats.satellite.EO_1 INFO <1145.50> EO_1: Target(Boldeşti-Scăeni, RO) tasked for imaging
2026-10-26 21:02:18,594 sats.satellite.EO_1 INFO <1145.50> EO_1: Target(Boldeşti-Scăeni, RO) window enabled: 1033.4 to 1169.1
2026-10-26 21:02:18,597 sats.satellite.EO_1 INFO <1145.50> EO_1: setting timed terminal event at 1169.1
2026-10-26 21:02:18,604 sats.satellite.EO_1 INFO <1157.00> EO_1: imaged Target(Boldeşti-Scăeni, RO)
2026-10-26 21:02:18,606 data.base INFO <1157.00> Total reward: {'EO_1': np.float64(0.2270585561858818)}
2026-10-26 21:02:18,607 sats.satellite.EO_1 INFO <1157.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:18,685 gym INFO <1157.00> Step reward: {'EO_1': np.float64(0.2270585561858818)}
2026-10-26 21:02:18,721 gym INFO <1157.00> === STARTING STEP ===
2026-10-26 21:02:18,722 sats.satellite.EO_1 INFO <1157.00> EO_1: target index 2 tasked
2026-10-26 21:02:18,723 sats.satellite.EO_1 INFO <1157.00> EO_1: Target(Bilozerka, UA) imaging timing: requested capture at 1199.7
2026-10-26 21:02:18,724 sats.satellite.EO_1 INFO <1157.00> EO_1: Target(Bilozerka, UA) tasked for imaging
2026-10-26 21:02:18,724 sats.satellite.EO_1 INFO <1157.00> EO_1: Target(Bilozerka, UA) window enabled: 1120.7 to 1242.4
2026-10-26 21:02:18,725 sats.satellite.EO_1 INFO <1157.00> EO_1: setting timed terminal event at 1242.4
2026-10-26 21:02:18,744 sats.satellite.EO_1 INFO <1203.50> EO_1: imaged Target(Bilozerka, UA)
2026-10-26 21:02:18,746 data.base INFO <1203.50> Total reward: {'EO_1': np.float64(0.10296013415656079)}
2026-10-26 21:02:18,746 sats.satellite.EO_1 INFO <1203.50> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:18,814 gym INFO <1203.50> Step reward: {'EO_1': np.float64(0.10296013415656079)}
2026-10-26 21:02:18,880 gym INFO <1203.50> === STARTING STEP ===
2026-10-26 21:02:18,881 sats.satellite.EO_0 INFO <1203.50> EO_0: target index 17 tasked
2026-10-26 21:02:18,882 sats.satellite.EO_0 INFO <1203.50> EO_0: Target(Aquitania, CO) imaging timing: requested capture at 3995.3
2026-10-26 21:02:18,882 sats.satellite.EO_0 INFO <1203.50> EO_0: Target(Aquitania, CO) tasked for imaging
2026-10-26 21:02:18,884 sats.satellite.EO_0 INFO <1203.50> EO_0: Target(Aquitania, CO) window enabled: 3929.0 to 4061.5
2026-10-26 21:02:18,884 sats.satellite.EO_0 INFO <1203.50> EO_0: setting timed terminal event at 4061.5
2026-10-26 21:02:18,885 sats.satellite.EO_1 INFO <1203.50> EO_1: target index 0 tasked
2026-10-26 21:02:18,886 sats.satellite.EO_1 INFO <1203.50> EO_1: Target(Zelenodolsk, UA) imaging timing: requested capture at 1224.4
2026-10-26 21:02:18,887 sats.satellite.EO_1 INFO <1203.50> EO_1: Target(Zelenodolsk, UA) tasked for imaging
2026-10-26 21:02:18,887 sats.satellite.EO_1 INFO <1203.50> EO_1: Target(Zelenodolsk, UA) window enabled: 1146.8 to 1245.4
2026-10-26 21:02:18,888 sats.satellite.EO_1 INFO <1203.50> EO_1: setting timed terminal event at 1245.4
2026-10-26 21:02:18,899 sats.satellite.EO_1 INFO <1228.50> EO_1: imaged Target(Zelenodolsk, UA)
2026-10-26 21:02:18,901 data.base INFO <1228.50> Total reward: {'EO_1': np.float64(0.525948564551845)}
2026-10-26 21:02:18,902 sats.satellite.EO_1 INFO <1228.50> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:18,971 gym INFO <1228.50> Step reward: {'EO_1': np.float64(0.525948564551845)}
2026-10-26 21:02:19,002 gym INFO <1228.50> === STARTING STEP ===
2026-10-26 21:02:19,004 sats.satellite.EO_1 INFO <1228.50> EO_1: target index 1 tasked
2026-10-26 21:02:19,004 sats.satellite.EO_1 INFO <1228.50> EO_1: Target(Pokrovsk, UA) imaging timing: requested capture at 1248.3
2026-10-26 21:02:19,005 sats.satellite.EO_1 INFO <1228.50> EO_1: Target(Pokrovsk, UA) tasked for imaging
2026-10-26 21:02:19,006 sats.satellite.EO_1 INFO <1228.50> EO_1: Target(Pokrovsk, UA) window enabled: 1206.6 to 1268.1
2026-10-26 21:02:19,006 sats.satellite.EO_1 INFO <1228.50> EO_1: setting timed terminal event at 1268.1
2026-10-26 21:02:19,023 sats.satellite.EO_1 INFO <1263.00> EO_1: imaged Target(Pokrovsk, UA)
2026-10-26 21:02:19,026 data.base INFO <1263.00> Total reward: {'EO_1': np.float64(0.38900071434681843)}
2026-10-26 21:02:19,026 sats.satellite.EO_1 INFO <1263.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:19,105 gym INFO <1263.00> Step reward: {'EO_1': np.float64(0.38900071434681843)}
2026-10-26 21:02:19,140 gym INFO <1263.00> === STARTING STEP ===
2026-10-26 21:02:19,142 sats.satellite.EO_1 INFO <1263.00> EO_1: target index 0 tasked
2026-10-26 21:02:19,143 sats.satellite.EO_1 INFO <1263.00> EO_1: Target(Yuvileine, UA) imaging timing: requested capture at 1267.7
2026-10-26 21:02:19,143 sats.satellite.EO_1 INFO <1263.00> EO_1: Target(Yuvileine, UA) tasked for imaging
2026-10-26 21:02:19,144 sats.satellite.EO_1 INFO <1263.00> EO_1: Target(Yuvileine, UA) window enabled: 1248.0 to 1272.4
2026-10-26 21:02:19,145 sats.satellite.EO_1 INFO <1263.00> EO_1: setting timed terminal event at 1272.4
2026-10-26 21:02:19,151 sats.satellite.EO_1 INFO <1272.50> EO_1: timed termination at 1272.4 for Target(Yuvileine, UA) window
2026-10-26 21:02:19,152 data.base INFO <1272.50> Total reward: {}
2026-10-26 21:02:19,153 sats.satellite.EO_1 INFO <1272.50> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:19,223 gym INFO <1272.50> Step reward: {}
2026-10-26 21:02:19,258 gym INFO <1272.50> === STARTING STEP ===
2026-10-26 21:02:19,261 sats.satellite.EO_1 INFO <1272.50> EO_1: target index 1 tasked
2026-10-26 21:02:19,262 sats.satellite.EO_1 INFO <1272.50> EO_1: Target(Sokhumi, GE) imaging timing: requested capture at 1313.0
2026-10-26 21:02:19,263 sats.satellite.EO_1 INFO <1272.50> EO_1: Target(Sokhumi, GE) tasked for imaging
2026-10-26 21:02:19,263 sats.satellite.EO_1 INFO <1272.50> EO_1: Target(Sokhumi, GE) window enabled: 1229.6 to 1353.6
2026-10-26 21:02:19,264 sats.satellite.EO_1 INFO <1272.50> EO_1: setting timed terminal event at 1353.6
2026-10-26 21:02:19,283 sats.satellite.EO_1 INFO <1323.50> EO_1: imaged Target(Sokhumi, GE)
2026-10-26 21:02:19,285 data.base INFO <1323.50> Total reward: {'EO_1': np.float64(0.20551218533976295)}
2026-10-26 21:02:19,288 sats.satellite.EO_1 INFO <1323.50> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:19,348 gym INFO <1323.50> Step reward: {'EO_1': np.float64(0.20551218533976295)}
2026-10-26 21:02:19,381 gym INFO <1323.50> === STARTING STEP ===
2026-10-26 21:02:19,382 sats.satellite.EO_1 INFO <1323.50> EO_1: target index 1 tasked
2026-10-26 21:02:19,383 sats.satellite.EO_1 INFO <1323.50> EO_1: Target(Saatlı, AZ) imaging timing: requested capture at 1396.6
2026-10-26 21:02:19,384 sats.satellite.EO_1 INFO <1323.50> EO_1: Target(Saatlı, AZ) tasked for imaging
2026-10-26 21:02:19,385 sats.satellite.EO_1 INFO <1323.50> EO_1: Target(Saatlı, AZ) window enabled: 1384.9 to 1408.3
2026-10-26 21:02:19,385 sats.satellite.EO_1 INFO <1323.50> EO_1: setting timed terminal event at 1408.3
2026-10-26 21:02:19,416 sats.satellite.EO_1 INFO <1396.50> EO_1: imaged Target(Saatlı, AZ)
2026-10-26 21:02:19,419 data.base INFO <1396.50> Total reward: {'EO_1': np.float64(0.7923814938276732)}
2026-10-26 21:02:19,419 sats.satellite.EO_1 INFO <1396.50> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:19,495 gym INFO <1396.50> Step reward: {'EO_1': np.float64(0.7923814938276732)}
2026-10-26 21:02:19,520 gym INFO <1396.50> === STARTING STEP ===
2026-10-26 21:02:19,522 sats.satellite.EO_1 INFO <1396.50> EO_1: target index 2 tasked
2026-10-26 21:02:19,523 sats.satellite.EO_1 INFO <1396.50> EO_1: Target(Corat, AZ) imaging timing: requested capture at 1426.1
2026-10-26 21:02:19,523 sats.satellite.EO_1 INFO <1396.50> EO_1: Target(Corat, AZ) tasked for imaging
2026-10-26 21:02:19,524 sats.satellite.EO_1 INFO <1396.50> EO_1: Target(Corat, AZ) window enabled: 1368.1 to 1455.7
2026-10-26 21:02:19,525 sats.satellite.EO_1 INFO <1396.50> EO_1: setting timed terminal event at 1455.7
2026-10-26 21:02:19,535 sats.satellite.EO_1 INFO <1426.00> EO_1: imaged Target(Corat, AZ)
2026-10-26 21:02:19,537 data.base INFO <1426.00> Total reward: {'EO_1': np.float64(0.8375055149095487)}
2026-10-26 21:02:19,538 sats.satellite.EO_1 INFO <1426.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:19,614 gym INFO <1426.00> Step reward: {'EO_1': np.float64(0.8375055149095487)}
2026-10-26 21:02:19,648 gym INFO <1426.00> === STARTING STEP ===
2026-10-26 21:02:19,650 sats.satellite.EO_1 INFO <1426.00> EO_1: target index 0 tasked
2026-10-26 21:02:19,651 sats.satellite.EO_1 INFO <1426.00> EO_1: Target(Xudat, AZ) imaging timing: requested capture at 1438.9
2026-10-26 21:02:19,651 sats.satellite.EO_1 INFO <1426.00> EO_1: Target(Xudat, AZ) tasked for imaging
2026-10-26 21:02:19,652 sats.satellite.EO_1 INFO <1426.00> EO_1: Target(Xudat, AZ) window enabled: 1337.2 to 1451.9
2026-10-26 21:02:19,652 sats.satellite.EO_1 INFO <1426.00> EO_1: setting timed terminal event at 1451.9
2026-10-26 21:02:19,660 sats.satellite.EO_1 INFO <1448.00> EO_1: imaged Target(Xudat, AZ)
2026-10-26 21:02:19,663 data.base INFO <1448.00> Total reward: {'EO_1': np.float64(0.7590256629768871)}
2026-10-26 21:02:19,664 sats.satellite.EO_1 INFO <1448.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:19,735 gym INFO <1448.00> Step reward: {'EO_1': np.float64(0.7590256629768871)}
2026-10-26 21:02:19,770 gym INFO <1448.00> === STARTING STEP ===
2026-10-26 21:02:19,773 sats.satellite.EO_1 INFO <1448.00> EO_1: target index 0 tasked
2026-10-26 21:02:19,777 sats.satellite.EO_1 INFO <1448.00> EO_1: Target(Əmircan, AZ) imaging timing: requested capture at 1452.9
2026-10-26 21:02:19,778 sats.satellite.EO_1 INFO <1448.00> EO_1: Target(Əmircan, AZ) tasked for imaging
2026-10-26 21:02:19,779 sats.satellite.EO_1 INFO <1448.00> EO_1: Target(Əmircan, AZ) window enabled: 1374.4 to 1457.8
2026-10-26 21:02:19,779 sats.satellite.EO_1 INFO <1448.00> EO_1: setting timed terminal event at 1457.8
2026-10-26 21:02:19,784 sats.satellite.EO_1 INFO <1458.00> EO_1: timed termination at 1457.8 for Target(Əmircan, AZ) window
2026-10-26 21:02:19,789 data.base INFO <1458.00> Total reward: {}
2026-10-26 21:02:19,789 sats.satellite.EO_1 INFO <1458.00> EO_1: Satellite EO_1 requires retasking
2026-10-26 21:02:19,865 gym INFO <1458.00> Step reward: {}
2026-10-26 21:02:19,901 gym INFO <1458.00> === STARTING STEP ===
2026-10-26 21:02:19,905 sats.satellite.EO_1 INFO <1458.00> EO_1: target index 2 tasked
2026-10-26 21:02:19,908 sats.satellite.EO_1 INFO <1458.00> EO_1: Target(Gurlan, UZ) imaging timing: requested capture at 1541.6
2026-10-26 21:02:19,908 sats.satellite.EO_1 INFO <1458.00> EO_1: Target(Gurlan, UZ) tasked for imaging
2026-10-26 21:02:19,910 sats.satellite.EO_1 INFO <1458.00> EO_1: Target(Gurlan, UZ) window enabled: 1474.4 to 1608.8
2026-10-26 21:02:19,910 sats.satellite.EO_1 INFO <1458.00> EO_1: setting timed terminal event at 1608.8
2026-10-26 21:02:19,934 data.base INFO <1513.50> Total reward: {}
2026-10-26 21:02:20,005 gym INFO <1513.50> Step reward: {}
2026-10-26 21:02:20,006 gym INFO <1513.50> Episode truncated: ['EO_0', 'EO_1']