{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Multi-Agent Environments\n",
    "\n",
    "Two multiagent environments are given in the package:\n",
    "\n",
    "* [GeneralSatelliteTasking](../api_reference/index.rst#bsk_rl.GeneralSatelliteTasking), \n",
    "  a [Gymnasium](https://gymnasium.farama.org)-based environment and the basis for all other environments.\n",
    "* [ConstellationTasking](../api_reference/index.rst#bsk_rl.ConstellationTasking), which\n",
    "  implements the [PettingZoo parallel API](https://pettingzoo.farama.org/api/parallel/).\n",
    "\n",
    "The latter is preferable for multi-agent RL (MARL) settings, as most algorithms are designed\n",
    "for this kind of API.\n",
    "\n",
    "## Configuring the Environment\n",
    "\n",
    "For this example, a multisatellite target imaging environment will be used. The goal is\n",
    "to maximize the value of unique images taken.\n",
    "\n",
    "As usual, the satellite type is defined first."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-09-12T21:06:02.814396Z",
     "iopub.status.busy": "2024-09-12T21:06:02.814248Z",
     "iopub.status.idle": "2024-09-12T21:06:03.752600Z",
     "shell.execute_reply": "2024-09-12T21:06:03.752328Z"
    }
   },
   "outputs": [],
   "source": [
    "from bsk_rl import sats, act, obs, scene, data, comm\n",
    "from bsk_rl.sim import dyn, fsw\n",
    "\n",
    "class ImagingSatellite(sats.ImagingSatellite):\n",
    "    observation_spec = [\n",
    "        obs.OpportunityProperties(\n",
    "            dict(prop=\"priority\"), \n",
    "            dict(prop=\"opportunity_open\", norm=5700.0),\n",
    "            n_ahead_observe=10,\n",
    "        )\n",
    "    ]\n",
    "    action_spec = [act.Image(n_ahead_image=10)]\n",
    "    dyn_type = dyn.FullFeaturedDynModel\n",
    "    fsw_type = fsw.SteeringImagerFSWModel"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Satellite properties are set to give the satellite near-unlimited power and storage resources. To randomize some parameters in a correlated manner across satellites, a ``sat_arg_randomizer`` is set and passed to the environment. In this case, the satellites are distributed in a trivial single-plane Walker-delta constellation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-09-12T21:06:03.754468Z",
     "iopub.status.busy": "2024-09-12T21:06:03.754318Z",
     "iopub.status.idle": "2024-09-12T21:06:03.756458Z",
     "shell.execute_reply": "2024-09-12T21:06:03.756253Z"
    }
   },
   "outputs": [],
   "source": [
    "\n",
    "from bsk_rl.utils.orbital import walker_delta_args\n",
    "\n",
    "sat_args = dict(\n",
    "    imageAttErrorRequirement=0.01,\n",
    "    imageRateErrorRequirement=0.01,\n",
    "    batteryStorageCapacity=1e9,\n",
    "    storedCharge_Init=1e9,\n",
    "    dataStorageCapacity=1e12,\n",
    "    u_max=0.4,\n",
    "    K1=0.25,\n",
    "    K3=3.0,\n",
    "    omega_max=0.087,\n",
    "    servo_Ki=5.0,\n",
    "    servo_P=150 / 5,\n",
    ")\n",
    "sat_arg_randomizer = walker_delta_args(altitude=800.0, inc=60.0, n_planes=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Gym API\n",
    "\n",
    "GeneralSatelliteTasking uses tuples of actions and observations to interact with the\n",
    "environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-09-12T21:06:03.757739Z",
     "iopub.status.busy": "2024-09-12T21:06:03.757645Z",
     "iopub.status.idle": "2024-09-12T21:06:04.047740Z",
     "shell.execute_reply": "2024-09-12T21:06:04.047439Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:03,760 \u001b[0m\u001b[mgym                            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[mResetting environment with seed=1879338696\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:03,761 \u001b[0m\u001b[mscene.targets                  \u001b[0m\u001b[mINFO       \u001b[0m\u001b[mGenerating 1000 targets\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:03,920 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mFinding opportunity windows from 0.00 to 600.00 seconds\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:03,943 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mFinding opportunity windows from 600.00 to 1200.00 seconds\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:03,964 \u001b[0m\u001b[92msats.satellite.EO-2            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[92mEO-2: \u001b[0m\u001b[mFinding opportunity windows from 0.00 to 600.00 seconds\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:03,982 \u001b[0m\u001b[92msats.satellite.EO-2            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[92mEO-2: \u001b[0m\u001b[mFinding opportunity windows from 600.00 to 1200.00 seconds\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,000 \u001b[0m\u001b[34msats.satellite.EO-3            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[34mEO-3: \u001b[0m\u001b[mFinding opportunity windows from 0.00 to 600.00 seconds\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,020 \u001b[0m\u001b[34msats.satellite.EO-3            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[34mEO-3: \u001b[0m\u001b[mFinding opportunity windows from 600.00 to 1200.00 seconds\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,043 \u001b[0m\u001b[mgym                            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[mEnvironment reset\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Tuple(Box(-1e+16, 1e+16, (20,), float64), Box(-1e+16, 1e+16, (20,), float64), Box(-1e+16, 1e+16, (20,), float64))"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from bsk_rl import GeneralSatelliteTasking\n",
    "\n",
    "env = GeneralSatelliteTasking(\n",
    "    satellites=[\n",
    "        ImagingSatellite(\"EO-1\", sat_args),\n",
    "        ImagingSatellite(\"EO-2\", sat_args),\n",
    "        ImagingSatellite(\"EO-3\", sat_args),\n",
    "    ],\n",
    "    scenario=scene.UniformTargets(1000),\n",
    "    rewarder=data.UniqueImageReward(),\n",
    "    communicator=comm.LOSCommunication(),  # Note that dyn must inherit from LOSCommunication\n",
    "    sat_arg_randomizer=sat_arg_randomizer,\n",
    "    log_level=\"INFO\",\n",
    ")\n",
    "env.reset()\n",
    "\n",
    "env.observation_space"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-09-12T21:06:04.049293Z",
     "iopub.status.busy": "2024-09-12T21:06:04.049199Z",
     "iopub.status.idle": "2024-09-12T21:06:04.051452Z",
     "shell.execute_reply": "2024-09-12T21:06:04.051204Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Tuple(Discrete(10), Discrete(10), Discrete(10))"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "env.action_space"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Consequently, actions are passed as a tuple. The step will stop the first time any\n",
    "satellite completes an action."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-09-12T21:06:04.052785Z",
     "iopub.status.busy": "2024-09-12T21:06:04.052709Z",
     "iopub.status.idle": "2024-09-12T21:06:04.184093Z",
     "shell.execute_reply": "2024-09-12T21:06:04.183843Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,053 \u001b[0m\u001b[mgym                            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[93;1m=== STARTING STEP ===\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,053 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mtarget index 7 tasked\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,053 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mTarget(tgt-195) tasked for imaging\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,054 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mTarget(tgt-195) window enabled: 462.8 to 548.3\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,054 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[msetting timed terminal event at 548.3\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,055 \u001b[0m\u001b[92msats.satellite.EO-2            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[92mEO-2: \u001b[0m\u001b[mtarget index 9 tasked\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,055 \u001b[0m\u001b[92msats.satellite.EO-2            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[92mEO-2: \u001b[0m\u001b[mTarget(tgt-680) tasked for imaging\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,056 \u001b[0m\u001b[92msats.satellite.EO-2            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[92mEO-2: \u001b[0m\u001b[mTarget(tgt-680) window enabled: 646.1 to 833.3\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,056 \u001b[0m\u001b[92msats.satellite.EO-2            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[92mEO-2: \u001b[0m\u001b[msetting timed terminal event at 833.3\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,056 \u001b[0m\u001b[34msats.satellite.EO-3            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[34mEO-3: \u001b[0m\u001b[mtarget index 8 tasked\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,057 \u001b[0m\u001b[34msats.satellite.EO-3            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[34mEO-3: \u001b[0m\u001b[mTarget(tgt-211) tasked for imaging\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,057 \u001b[0m\u001b[34msats.satellite.EO-3            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[34mEO-3: \u001b[0m\u001b[mTarget(tgt-211) window enabled: 986.2 to 1077.6\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,057 \u001b[0m\u001b[34msats.satellite.EO-3            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[34mEO-3: \u001b[0m\u001b[msetting timed terminal event at 1077.6\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,148 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<465.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mimaged Target(tgt-195)\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,152 \u001b[0m\u001b[mdata.base                      \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<465.00> \u001b[0m\u001b[mData reward: {'EO-1': 0.8230500147347414, 'EO-2': 0.0, 'EO-3': 0.0}\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,156 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<465.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mSatellite EO-1 requires retasking\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,156 \u001b[0m\u001b[92msats.satellite.EO-2            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<465.00> \u001b[0m\u001b[92mEO-2: \u001b[0m\u001b[mFinding opportunity windows from 1200.00 to 1800.00 seconds\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,182 \u001b[0m\u001b[mgym                            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<465.00> \u001b[0m\u001b[mStep reward: 0.8230500147347414\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "observation, reward, terminated, truncated, info = env.step([7, 9, 8])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-09-12T21:06:04.185562Z",
     "iopub.status.busy": "2024-09-12T21:06:04.185474Z",
     "iopub.status.idle": "2024-09-12T21:06:04.187695Z",
     "shell.execute_reply": "2024-09-12T21:06:04.187440Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([ 0.21602972, -0.00403203,  0.28284637,  0.00356078,  0.4518415 ,\n",
       "         0.04613867,  0.47992008,  0.05650256,  0.64224289,  0.06190224,\n",
       "         0.74877742,  0.07368187,  0.59800648,  0.05895398,  0.30639417,\n",
       "         0.06799013,  0.08390608,  0.0861771 ,  0.47193369,  0.10286262]),\n",
       " array([0.88676028, 0.00289549, 0.26382734, 0.02413358, 0.33591011,\n",
       "        0.03177945, 0.7800974 , 0.05823664, 0.76553764, 0.05932064,\n",
       "        0.48517132, 0.09174613, 0.49501144, 0.11767415, 0.60871352,\n",
       "        0.16188445, 0.53565483, 0.15520731, 0.7778392 , 0.15127552]),\n",
       " array([1.92145237e-01, 7.09354487e-04, 7.69811097e-02, 9.43758064e-03,\n",
       "        8.96589817e-01, 4.59254776e-03, 9.05020147e-01, 2.69390689e-02,\n",
       "        9.13063309e-01, 9.14312097e-02, 9.30135351e-01, 8.71906659e-02,\n",
       "        1.55938107e-01, 7.51445464e-02, 2.97161675e-01, 8.92334327e-02,\n",
       "        8.99333303e-01, 1.10183857e-01, 1.88123050e-01, 1.27600018e-01]))"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "observation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point, either every satellite can be retasked, or satellites can continue their\n",
    "previous action by passing `None` as the action. To see which satellites must be\n",
    "retasked (i.e. their previous action is done and they have nothing more to do), look at\n",
    "`\"requires_retasking\"` in each satellite's info."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-09-12T21:06:04.188988Z",
     "iopub.status.busy": "2024-09-12T21:06:04.188894Z",
     "iopub.status.idle": "2024-09-12T21:06:04.190832Z",
     "shell.execute_reply": "2024-09-12T21:06:04.190612Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'EO-1': {'requires_retasking': True},\n",
       " 'EO-2': {'requires_retasking': False},\n",
       " 'EO-3': {'requires_retasking': False},\n",
       " 'd_ts': 465.00000000000006}"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "info"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Based on this list, we decide here to only retask the satellite that needs it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-09-12T21:06:04.192183Z",
     "iopub.status.busy": "2024-09-12T21:06:04.192087Z",
     "iopub.status.idle": "2024-09-12T21:06:04.194149Z",
     "shell.execute_reply": "2024-09-12T21:06:04.193911Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, None, None]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "actions = [0 if info[sat.name][\"requires_retasking\"] else None for sat in env.unwrapped.satellites]\n",
    "actions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-09-12T21:06:04.195362Z",
     "iopub.status.busy": "2024-09-12T21:06:04.195269Z",
     "iopub.status.idle": "2024-09-12T21:06:04.216314Z",
     "shell.execute_reply": "2024-09-12T21:06:04.216082Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,195 \u001b[0m\u001b[mgym                            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<465.00> \u001b[0m\u001b[93;1m=== STARTING STEP ===\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,195 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<465.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mtarget index 0 tasked\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,196 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<465.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mTarget(tgt-457) tasked for imaging\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,196 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<465.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mTarget(tgt-457) window enabled: 442.0 to 532.9\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,197 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<465.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[msetting timed terminal event at 532.9\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,210 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<529.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mimaged Target(tgt-457)\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,212 \u001b[0m\u001b[mdata.base                      \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<529.00> \u001b[0m\u001b[mData reward: {'EO-1': 0.21602971982103714, 'EO-2': 0.0, 'EO-3': 0.0}\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,213 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<529.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mSatellite EO-1 requires retasking\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,214 \u001b[0m\u001b[mgym                            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<529.00> \u001b[0m\u001b[mStep reward: 0.21602971982103714\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "observation, reward, terminated, truncated, info = env.step(actions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this environment, the environment will stop if any agent dies. To demonstrate this,\n",
    "one satellite is forcibly killed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-09-12T21:06:04.217867Z",
     "iopub.status.busy": "2024-09-12T21:06:04.217758Z",
     "iopub.status.idle": "2024-09-12T21:06:04.314449Z",
     "shell.execute_reply": "2024-09-12T21:06:04.314221Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,218 \u001b[0m\u001b[mgym                            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<529.00> \u001b[0m\u001b[93;1m=== STARTING STEP ===\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,219 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<529.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mtarget index 6 tasked\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,219 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<529.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mTarget(tgt-507) tasked for imaging\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,220 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<529.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mTarget(tgt-507) window enabled: 852.5 to 1057.2\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,220 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<529.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[msetting timed terminal event at 1057.2\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,221 \u001b[0m\u001b[92msats.satellite.EO-2            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<529.00> \u001b[0m\u001b[92mEO-2: \u001b[0m\u001b[mtarget index 7 tasked\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,221 \u001b[0m\u001b[92msats.satellite.EO-2            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<529.00> \u001b[0m\u001b[92mEO-2: \u001b[0m\u001b[mTarget(tgt-353) tasked for imaging\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,222 \u001b[0m\u001b[92msats.satellite.EO-2            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<529.00> \u001b[0m\u001b[92mEO-2: \u001b[0m\u001b[mTarget(tgt-353) window enabled: 1349.7 to 1528.5\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,222 \u001b[0m\u001b[92msats.satellite.EO-2            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<529.00> \u001b[0m\u001b[92mEO-2: \u001b[0m\u001b[msetting timed terminal event at 1528.5\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,222 \u001b[0m\u001b[34msats.satellite.EO-3            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<529.00> \u001b[0m\u001b[34mEO-3: \u001b[0m\u001b[mtarget index 9 tasked\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,223 \u001b[0m\u001b[34msats.satellite.EO-3            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<529.00> \u001b[0m\u001b[34mEO-3: \u001b[0m\u001b[mTarget(tgt-261) tasked for imaging\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,223 \u001b[0m\u001b[34msats.satellite.EO-3            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<529.00> \u001b[0m\u001b[34mEO-3: \u001b[0m\u001b[mTarget(tgt-261) window enabled: 1192.3 to 1200.0\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,223 \u001b[0m\u001b[34msats.satellite.EO-3            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<529.00> \u001b[0m\u001b[34mEO-3: \u001b[0m\u001b[msetting timed terminal event at 1200.0\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,285 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<855.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mimaged Target(tgt-507)\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,287 \u001b[0m\u001b[mdata.base                      \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<855.00> \u001b[0m\u001b[mData reward: {'EO-1': 0.30639416945804554, 'EO-2': 0.0, 'EO-3': 0.0}\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,290 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<855.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mSatellite EO-1 requires retasking\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,291 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<855.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mFinding opportunity windows from 1200.00 to 1800.00 seconds\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,312 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[93mWARNING    \u001b[0m\u001b[33m<855.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[93mfailed battery_valid check\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,312 \u001b[0m\u001b[mgym                            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<855.00> \u001b[0m\u001b[mStep reward: -0.6936058305419545\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,312 \u001b[0m\u001b[mgym                            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<855.00> \u001b[0m\u001b[mEpisode terminated: True\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,312 \u001b[0m\u001b[mgym                            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<855.00> \u001b[0m\u001b[mEpisode truncated: False\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "from Basilisk.architecture import messaging\n",
    "\n",
    "def isnt_alive(log_failure=False):\n",
    "    \"\"\"Mock satellite 0 dying.\"\"\"\n",
    "    self = env.unwrapped.satellites[0]\n",
    "    death_message = messaging.PowerStorageStatusMsgPayload()\n",
    "    death_message.storageLevel = 0.0\n",
    "    self.dynamics.powerMonitor.batPowerOutMsg.write(death_message)\n",
    "    return self.dynamics.is_alive(log_failure=log_failure) and self.fsw.is_alive(\n",
    "        log_failure=log_failure\n",
    "    )\n",
    "\n",
    "env.unwrapped.satellites[0].is_alive = isnt_alive\n",
    "observation, reward, terminated, truncated, info = env.step([6, 7, 9])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## PettingZoo API\n",
    "\n",
    "The [PettingZoo parallel API](https://pettingzoo.farama.org/api/parallel/) environment, \n",
    "ConstellationTasking, is largely the same as GeneralSatelliteTasking. See their\n",
    "documentation for a full description of the API. It tends to separate things into\n",
    "dictionaries keyed by agent, rather than tuples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-09-12T21:06:04.316002Z",
     "iopub.status.busy": "2024-09-12T21:06:04.315886Z",
     "iopub.status.idle": "2024-09-12T21:06:04.728081Z",
     "shell.execute_reply": "2024-09-12T21:06:04.727819Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,318 \u001b[0m\u001b[m                               \u001b[0m\u001b[93mWARNING    \u001b[0m\u001b[93mCreating logger for new env on PID=96022. Old environments in process may now log times incorrectly.\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,501 \u001b[0m\u001b[mgym                            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[mResetting environment with seed=2620988696\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,502 \u001b[0m\u001b[mscene.targets                  \u001b[0m\u001b[mINFO       \u001b[0m\u001b[mGenerating 1000 targets\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,653 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mFinding opportunity windows from 0.00 to 600.00 seconds\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,674 \u001b[0m\u001b[92msats.satellite.EO-2            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[92mEO-2: \u001b[0m\u001b[mFinding opportunity windows from 0.00 to 600.00 seconds\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,699 \u001b[0m\u001b[34msats.satellite.EO-3            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[34mEO-3: \u001b[0m\u001b[mFinding opportunity windows from 0.00 to 600.00 seconds\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,725 \u001b[0m\u001b[mgym                            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[mEnvironment reset\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'EO-1': Box(-1e+16, 1e+16, (20,), float64),\n",
       " 'EO-2': Box(-1e+16, 1e+16, (20,), float64),\n",
       " 'EO-3': Box(-1e+16, 1e+16, (20,), float64)}"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from bsk_rl import ConstellationTasking\n",
    "\n",
    "env = ConstellationTasking(\n",
    "    satellites=[\n",
    "        ImagingSatellite(\"EO-1\", sat_args),\n",
    "        ImagingSatellite(\"EO-2\", sat_args),\n",
    "        ImagingSatellite(\"EO-3\", sat_args),\n",
    "    ],\n",
    "    scenario=scene.UniformTargets(1000),\n",
    "    rewarder=data.UniqueImageReward(),\n",
    "    communicator=comm.LOSCommunication(),  # Note that dyn must inherit from LOSCommunication\n",
    "    sat_arg_randomizer=sat_arg_randomizer,\n",
    "    log_level=\"INFO\",\n",
    ")\n",
    "env.reset()\n",
    "\n",
    "env.observation_spaces"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-09-12T21:06:04.729530Z",
     "iopub.status.busy": "2024-09-12T21:06:04.729429Z",
     "iopub.status.idle": "2024-09-12T21:06:04.731480Z",
     "shell.execute_reply": "2024-09-12T21:06:04.731258Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'EO-1': Discrete(10), 'EO-2': Discrete(10), 'EO-3': Discrete(10)}"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "env.action_spaces"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Actions are passed as a dictionary; the agent names can be accessed through the `agents`\n",
    "property."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-09-12T21:06:04.732963Z",
     "iopub.status.busy": "2024-09-12T21:06:04.732747Z",
     "iopub.status.idle": "2024-09-12T21:06:04.852141Z",
     "shell.execute_reply": "2024-09-12T21:06:04.851891Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,734 \u001b[0m\u001b[mgym                            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[93;1m=== STARTING STEP ===\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,734 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mtarget index 7 tasked\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,735 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mTarget(tgt-539) tasked for imaging\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,735 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mTarget(tgt-539) window enabled: 288.7 to 440.8\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,736 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[msetting timed terminal event at 440.8\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,736 \u001b[0m\u001b[92msats.satellite.EO-2            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[92mEO-2: \u001b[0m\u001b[mtarget index 9 tasked\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,736 \u001b[0m\u001b[92msats.satellite.EO-2            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[92mEO-2: \u001b[0m\u001b[mTarget(tgt-937) tasked for imaging\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,737 \u001b[0m\u001b[92msats.satellite.EO-2            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[92mEO-2: \u001b[0m\u001b[mTarget(tgt-937) window enabled: 357.5 to 473.6\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,737 \u001b[0m\u001b[92msats.satellite.EO-2            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[92mEO-2: \u001b[0m\u001b[msetting timed terminal event at 473.6\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,737 \u001b[0m\u001b[34msats.satellite.EO-3            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[34mEO-3: \u001b[0m\u001b[mtarget index 8 tasked\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,738 \u001b[0m\u001b[34msats.satellite.EO-3            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[34mEO-3: \u001b[0m\u001b[mTarget(tgt-926) tasked for imaging\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,738 \u001b[0m\u001b[34msats.satellite.EO-3            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[34mEO-3: \u001b[0m\u001b[mTarget(tgt-926) window enabled: 353.2 to 451.1\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,738 \u001b[0m\u001b[34msats.satellite.EO-3            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<0.00> \u001b[0m\u001b[34mEO-3: \u001b[0m\u001b[msetting timed terminal event at 451.1\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,794 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<291.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mimaged Target(tgt-539)\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,796 \u001b[0m\u001b[mdata.base                      \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<291.00> \u001b[0m\u001b[mData reward: {'EO-1': 0.662026254515778, 'EO-2': 0.0, 'EO-3': 0.0}\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,799 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<291.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mSatellite EO-1 requires retasking\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,800 \u001b[0m\u001b[36msats.satellite.EO-1            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<291.00> \u001b[0m\u001b[36mEO-1: \u001b[0m\u001b[mFinding opportunity windows from 600.00 to 1200.00 seconds\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,824 \u001b[0m\u001b[34msats.satellite.EO-3            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<291.00> \u001b[0m\u001b[34mEO-3: \u001b[0m\u001b[mFinding opportunity windows from 600.00 to 1200.00 seconds\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,850 \u001b[0m\u001b[mgym                            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<291.00> \u001b[0m\u001b[mStep reward: {'EO-1': 0.662026254515778, 'EO-2': 0.0, 'EO-3': 0.0}\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,850 \u001b[0m\u001b[mgym                            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<291.00> \u001b[0m\u001b[mEpisode terminated: {'EO-1': False, 'EO-2': False, 'EO-3': False}\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,850 \u001b[0m\u001b[mgym                            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<291.00> \u001b[0m\u001b[mEpisode truncated: {'EO-1': False, 'EO-2': False, 'EO-3': False}\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "observation, reward, terminated, truncated, info = env.step(\n",
    "    {\n",
    "        env.agents[0]: 7,\n",
    "        env.agents[1]: 9,\n",
    "        env.agents[2]: 8,\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-09-12T21:06:04.853619Z",
     "iopub.status.busy": "2024-09-12T21:06:04.853505Z",
     "iopub.status.idle": "2024-09-12T21:06:04.855808Z",
     "shell.execute_reply": "2024-09-12T21:06:04.855572Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'EO-1': array([ 3.54079768e-01,  1.43722174e-04,  3.41016950e-01, -1.19984827e-02,\n",
       "         4.59862409e-01, -3.73977166e-04,  1.57034165e-01,  4.09939092e-02,\n",
       "         4.77186531e-01,  1.85235817e-02,  7.89599807e-01,  5.51169539e-02,\n",
       "         4.91062428e-01,  6.76157713e-02,  4.91219311e-01,  5.56755562e-02,\n",
       "         6.77410361e-01,  7.07679283e-02,  2.94042355e-01,  1.12199602e-01]),\n",
       " 'EO-2': array([ 9.14702578e-01, -2.65200788e-02,  4.95920248e-01, -2.44466556e-02,\n",
       "         3.61190392e-01, -5.08222404e-03,  7.70723330e-01, -2.27018729e-02,\n",
       "         3.24277884e-01,  5.20187817e-04,  6.73082171e-01,  1.16583843e-02,\n",
       "         9.45554102e-01,  1.19332716e-02,  3.68596535e-01,  5.59413877e-03,\n",
       "         2.98544255e-01,  3.60677170e-02,  3.48486992e-01,  3.10203217e-02]),\n",
       " 'EO-3': array([ 0.52332492, -0.01215394,  0.98737445,  0.01091435,  0.4451887 ,\n",
       "         0.00316024,  0.0034789 ,  0.0223582 ,  0.76490943,  0.01442445,\n",
       "         0.51863226,  0.02706901,  0.92739003,  0.0306299 ,  0.83661002,\n",
       "         0.04446559,  0.73678664,  0.03750987,  0.85066234,  0.11141505])}"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "observation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Other than compatibility with MARL algorithms, the main benefit of the PettingZoo API\n",
    "is that it allows for individual agents to fail without terminating the entire environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-09-12T21:06:04.857180Z",
     "iopub.status.busy": "2024-09-12T21:06:04.857087Z",
     "iopub.status.idle": "2024-09-12T21:06:04.859415Z",
     "shell.execute_reply": "2024-09-12T21:06:04.859180Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['EO-2', 'EO-3']"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Immediately kill satellite 0\n",
    "env.unwrapped.satellites[0].is_alive = isnt_alive\n",
    "env.agents"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-09-12T21:06:04.860730Z",
     "iopub.status.busy": "2024-09-12T21:06:04.860636Z",
     "iopub.status.idle": "2024-09-12T21:06:04.883021Z",
     "shell.execute_reply": "2024-09-12T21:06:04.882791Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,861 \u001b[0m\u001b[mgym                            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<291.00> \u001b[0m\u001b[93;1m=== STARTING STEP ===\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,862 \u001b[0m\u001b[92msats.satellite.EO-2            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<291.00> \u001b[0m\u001b[92mEO-2: \u001b[0m\u001b[mtarget index 7 tasked\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,862 \u001b[0m\u001b[92msats.satellite.EO-2            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<291.00> \u001b[0m\u001b[92mEO-2: \u001b[0m\u001b[mTarget(tgt-835) tasked for imaging\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,863 \u001b[0m\u001b[92msats.satellite.EO-2            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<291.00> \u001b[0m\u001b[92mEO-2: \u001b[0m\u001b[mTarget(tgt-835) window enabled: 322.9 to 532.5\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,863 \u001b[0m\u001b[92msats.satellite.EO-2            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<291.00> \u001b[0m\u001b[92mEO-2: \u001b[0m\u001b[msetting timed terminal event at 532.5\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,863 \u001b[0m\u001b[34msats.satellite.EO-3            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<291.00> \u001b[0m\u001b[34mEO-3: \u001b[0m\u001b[mtarget index 9 tasked\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,863 \u001b[0m\u001b[34msats.satellite.EO-3            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<291.00> \u001b[0m\u001b[34mEO-3: \u001b[0m\u001b[mTarget(tgt-686) tasked for imaging\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,864 \u001b[0m\u001b[34msats.satellite.EO-3            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<291.00> \u001b[0m\u001b[34mEO-3: \u001b[0m\u001b[mTarget(tgt-686) window enabled: 926.1 to 1098.0\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,864 \u001b[0m\u001b[34msats.satellite.EO-3            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<291.00> \u001b[0m\u001b[34mEO-3: \u001b[0m\u001b[msetting timed terminal event at 1098.0\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,873 \u001b[0m\u001b[92msats.satellite.EO-2            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<332.00> \u001b[0m\u001b[92mEO-2: \u001b[0m\u001b[mimaged Target(tgt-835)\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,875 \u001b[0m\u001b[mdata.base                      \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<332.00> \u001b[0m\u001b[mData reward: {'EO-1': 0.0, 'EO-2': 0.36859653472551324, 'EO-3': 0.0}\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,876 \u001b[0m\u001b[92msats.satellite.EO-2            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<332.00> \u001b[0m\u001b[92mEO-2: \u001b[0m\u001b[mSatellite EO-2 requires retasking\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,881 \u001b[0m\u001b[mgym                            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<332.00> \u001b[0m\u001b[mStep reward: {'EO-2': 0.36859653472551324, 'EO-3': 0.0}\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,881 \u001b[0m\u001b[mgym                            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<332.00> \u001b[0m\u001b[mEpisode terminated: {'EO-2': False, 'EO-3': False}\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[90;3m2024-09-12 15:06:04,881 \u001b[0m\u001b[mgym                            \u001b[0m\u001b[mINFO       \u001b[0m\u001b[33m<332.00> \u001b[0m\u001b[mEpisode truncated: {'EO-2': False, 'EO-3': False}\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "observation, reward, terminated, truncated, info = env.step({\n",
    "        env.agents[0]: 7,\n",
    "        env.agents[1]: 9,\n",
    "    }\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv_refactor",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}