#import "importer/main.typ": *
#import "helpers.typ": *


= Validation <validation>
We perform several validation tests to ensure the accuracy and reliability of our simulation results. This section details how we check that the new implementation remains robust and consistent with previous versions.

== Code validation
#let notebook = json("../workdir/11_visualization/validation_simple_run.ipynb")
#figure(
  image_cell(notebook, cell_id: "validation_signal_comparison_old_v_new"),
  caption: [
    Validation signal comparison between the new implementation of #beorn (blue) and the old version (green, dashed). From _left_ to _right_:
    Global evolution of the differential brightness temperature $d T_"b"$.
    Evolution of the mean kinetic temperature of the gas $T_k$.
    Mean ionization fraction history $x_"HII"$.
    Dimensionless power spectrum of the $d T_"b"$ field as a function of redshift at $k = 0.12 "Mpc"^(-1)$.
  ],

) <fig:validation_signal_comparison_old_v_new>

We ensure consistency of the updated BEoRN code with previous versions by running a series of simulations under identical conditions. We compare key outputs starting from the profiles of individuall sources, to the ionization maps, and finally to the global reionization signals. This step-by-step comparison allows us to identify any discrepancy that may arise from the code changes. In @fig:validation_signal_comparison_old_v_new we present a comparison of the central observables of reionization between the previous and current versions of BEoRN. The underlying simulation parameters are kept identical or updated to an equivalent setting where necessary. The results show essentially unchanged evolution of the global brightness temperature, kinetic temperature, and ionization fraction. Note that this run uses an artifical halo catalog and an unphysically high star formation efficiency to ensure a rapid reionization within the simulation range. As such this setup is not representative of a realistic reionization scenario.
// but ...

Similarly we maintain backward compatibility with the input format used in previous BEORN runs (i.e. snapshots generated by pkdgrav or 21cmfast). This allows us to reproduce the earlier runs and match the results as described by @Schaeffer_2023.

== Convergence tests
#let notebook = json("../workdir/11_visualization/validation_convergence.ipynb")
#figure(
  image_cell(notebook, cell_id: "validation_signal_comparison"),
  caption: [
    Validation signal comparison between runs using a grid resolution of $128^3$ (blue) and $256^3$ (green) as well as a run using the #thesandark 1 simulation ($2100^3$ particles) with a grid resolution of $128^3$ (yellow).
    // TODO - specfify each color
    From _left_ to _right_:
    Global evolution of the differential brightness temperature $d T_"b"$.
    Evolution of the mean kinetic temperature of the gas $T_k$.
    Mean ionization fraction history $x_"HII"$.
    Dimensionless power spectrum of the $d T_"b"$ field as a function of redshift at $k = 0.12 "Mpc"^(-1)$.
  ],

) <fig:validation_signal_comparison>

To ensure that our results are not sensitive to the numerical resolution of the simulation, we perform convergence tests. We compare the following variations of resolution: Firstly we compare effects of the grid resolution by running simulations with $128^3$ and $256^3$. Secondly we investigate the impact of the mass resolution by comparing the results obtained from the #smallcaps[Thesan-Dark] 1 and 2 simulations, which have different particle masses, as mentioned in @procedure.

@fig:validation_signal_comparison shows that there is no significant difference when using a finer grid resolution. This is expected since the ionized regions rapidly become larger than the cell size. The comparison between the different mass resolutions sees a deviation in the timing of reionization. This effect is expected and documented by @Kannan_2021: The lowest mass halos which are not resolved by #thesandark 2 form small bubbles quickly (as early as $z=10$) and contribute to the ionization budget at early times. We account for this by artificially increasing the lower mass cutoff during these validation runs but an imbalance between the two simulations remains. Globally the shapes of the signals are very similar and the power spectra match closely.

An additional convergence test is performed by varying the binning of the accretion rate alpha parameter. Keeping the overall range fixed between $alpha = 0$ and $alpha = 5.0$ we compare the signal generated when using $n = 5, 25, 50$ bins. Since a bulk of the halos where the accretion rate cannot be fitted is assigned a fallback value which then falls into one of the bin, there is a global shift to the signal when changing the binning. However, the overall shape and features of the signal remain unchanged from $n = 25$ bins onwards.
