Optional Performance Benchmarks

Basilisk performance benchmarks are developer utilities for measuring selected implementation choices. They are not numerical validation tests. Run them when investigating execution speed, comparing algorithms, or checking whether a local change affects a performance-sensitive code path.

Benchmark timing depends on the processor, compiler, build type, system load, and power management state. Use a release build, run the benchmark several times from a quiet terminal, and compare results from the same machine and build configuration.

Dynamics Effector Benchmark

The dynamics effector benchmark measures selected spacecraft, state effector, and dynamic effector execution speed with a shared benchmark harness, including the NDOF spinning-body BSM multi-body dynamics case.

Source script: benchmark_state_effectors.py

Benchmark selected Basilisk spacecraft dynamics effectors.

This optional developer benchmark runs compact spacecraft simulations that isolate selected dynamics effectors. Each trial builds a fresh spacecraft.Spacecraft object, attaches the requested effector load, disables recorders and plotting, runs the default RK4 integration loop, and reports wall-clock timing.

The benchmark is intended for local before/after comparisons while optimizing state effectors, dynamic effectors, or the spacecraft integration path. It is not a numerical validation test and it is not run by CI.

Running the Benchmark

Run the benchmark from the Basilisk repository root directory after building the Basilisk Python package:

cd /path/to/basilisk
.venv/bin/python benchmarks/dynamics/benchmark_state_effectors.py --case all

Run a single case with a longer measurement:

.venv/bin/python benchmarks/dynamics/benchmark_state_effectors.py \
   --case SpinningBodyTwoDOFStateEffector --steps 10000 --trials 7

Use --components to scale the number of repeated devices in cases that support repetition. Use --segments to scale the per-effector body count in the NDOF translator and hinged-body cases:

.venv/bin/python benchmarks/dynamics/benchmark_state_effectors.py \
   --case LinearTranslationNDOFStateEffector --components 2 --segments 8

Supported Cases

The --case option accepts either the canonical benchmark names or the corresponding effector class-style aliases:

  • spacecraft

  • spacecraft-point-mass

  • spacecraft-orbit

  • spacecraft-point-mass-orbit

  • dual-hinged-rigid-body

  • gravity-gradient

  • hinged-rigid-body

  • linear-spring-mass-damper

  • linear-translation-ndof

  • linear-translation-one-dof

  • n-hinged-rigid-body

  • prescribed-motion

  • reaction-wheel

  • spherical-pendulum

  • spinning-body-ndof

  • spinning-body-one-dof

  • spinning-body-two-dof

  • vscmg

Expected Terminal Output

The exact timings will vary by machine. An abridged representative all-case run looks like:

Dynamics effector benchmark
Cases: all
Steps per trial: 2000
Time step: 0.010000 s
Simulated time per trial: 20.000 s
Trials: 5
Warmup steps: 200
Components per case: 4
Segments per NDOF case: 4
Gravity: disabled except orbit and gravity-gradient cases
Recorders: disabled
Spacecraft integrator: RK4 default

Case                                      median [s]   median us/step      min [s]    min us/step
spacecraft                                  0.012345            6.173     0.012120          6.060
hinged-rigid-body                           0.045678           22.839     0.045100         22.550

Interpreting the Columns

median [s]

Median wall-clock time across measured trials for one ExecuteSimulation() call.

median us/step

Median wall-clock microseconds per integration step.

min [s]

Fastest measured trial wall-clock time.

min us/step

Fastest measured trial wall-clock microseconds per integration step.

Eigen and linearAlgebra Benchmark

The benchmark_eigen_linear_algebra executable compares common Eigen vector and matrix operations against the Basilisk linearAlgebra.c/h utility functions. It includes dot products, cross products, fixed-size matrix-vector products, fixed-size matrix-matrix products, dynamic matrix products, and inverse operations.

Source file: benchmark_eigen_linear_algebra.cpp

This benchmark is excluded from the normal build with EXCLUDE_FROM_ALL. Build it explicitly from the Basilisk repository root directory:

cd /path/to/basilisk
.venv/bin/cmake -S src -B dist3
.venv/bin/cmake --build dist3 --target benchmark_eigen_linear_algebra --config Release

Then run the benchmark executable:

dist3/benchmarks/utilities/benchmark_eigen_linear_algebra

To run a longer benchmark, pass a positive integer scale factor. For example, the following command multiplies the default loop counts by 5:

dist3/benchmarks/utilities/benchmark_eigen_linear_algebra 5

To run only a brief executable smoke check, use --smoke:

dist3/benchmarks/utilities/benchmark_eigen_linear_algebra --smoke

Expected Terminal Output

The exact timings will vary by machine. A representative run looks like:

Eigen versus linearAlgebra benchmark
Build and run this benchmark target in Release mode for meaningful timings.
The ratio column is linearAlgebra time divided by Eigen time.

Operation                           Iterations    Eigen ns/op     linearAlg ns/op      linear/Eigen     checksum diff
v3 dot                                20000000          2.512               2.623             1.044             0.000
v3 cross                              10000000          1.701               3.458             2.032             0.000
3x3 multiply                           5000000          2.386               3.538             1.483             0.000
3x3 transpose multiply                 5000000          2.761               4.241             1.536             0.000
3x3 vector multiply                    8000000          2.213               4.164             1.881             0.000
3x3 inverse                            1000000          2.475               4.849             1.959             0.000
4x4 inverse                             200000          4.401              22.896             5.202             0.000
6x6 multiply                            800000          8.622              13.384             1.552             0.000
6x6 vector multiply                    2000000          2.387               5.347             2.240             0.000
dynamic 6x6 multiply                    500000         68.503              50.585             0.738             0.000
dynamic 12x12 multiply                  100000        154.105             650.041             4.218             0.000
generic 6x6 inverse                        200        573.330           54896.460            95.750             0.000

checksum sink: -1791071704.319

Interpreting the Columns

Eigen ns/op

Average Eigen execution time for the listed operation, in nanoseconds per operation.

linearAlg ns/op

Average linearAlgebra execution time for the same operation, in nanoseconds per operation.

linear/Eigen

Ratio of linearAlg ns/op divided by Eigen ns/op. Values greater than 1.0 mean linearAlgebra was slower for that operation. Values less than 1.0 mean linearAlgebra was faster.

checksum diff

Relative difference between the Eigen and linearAlgebra checksums. This is a basic guard that both loops performed comparable numerical work; it is not a replacement for a unit test.

Benchmark Smoke Tests

The benchmark smoke tests are pass/fail freshness checks. They do not evaluate performance; they only verify that the benchmark entry points still build or run with a minimal workload.

Source test: test_benchmark_smoke.py

The benchmark smoke tests are included in the default pytest collection paths, so they run when a developer invokes pytest from the Basilisk repository root directory or from the src directory. To run only the benchmark smoke tests from the repository root directory:

.venv/bin/pytest -q benchmarks/tests

Or from the src directory:

../.venv/bin/pytest -q ../benchmarks/tests

The Python smoke test runs the dynamics benchmark with one integration step, one trial, no warmup, one component, and one segment. The C++ smoke test builds and runs the benchmark_smoke_tests CMake target, which executes the Eigen versus linearAlgebra benchmark in --smoke mode with capped iteration counts. The C++ smoke test requires an already configured dist3 build directory that includes the benchmark targets; otherwise it is skipped with a reconfigure hint.

Developers can also run the C++ smoke target directly:

.venv/bin/cmake --build dist3 --target benchmark_smoke_tests --config Release

Guidelines for Future Benchmarks

Keep performance benchmarks opt-in unless there is a specific reason to run them in CI. New benchmarks should document:

  • The target name and source file.

  • The build configuration used for meaningful results.

  • The command used to run the benchmark.

  • Representative terminal output.

  • How to interpret each reported metric.

  • Any known limitations that affect comparison quality.