remoll/master-thesis-report
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

detailed most of the procedure

Browse Source
This commit is contained in:
Remy Moll
2025-09-12 23:41:31 +02:00
parent 9045c95ddd
commit c43b64ecb7
13 changed files with 413 additions and 114 deletions
Download Patch File Download Diff File Expand all files Collapse all files

46
validation.typ
View File

@@ -1,19 +1,47 @@
= Validation <validation>
We perform several validation tests to ensure the accuracy and reliability of our simulation results. This section details both code-level tests and scientific validations against established results.
#import "importer/main.typ": *
#import "helpers.typ": *
- Comparison to old (simplified) run!
- Comparison to THESAN
= Validation <validation>
We perform several validation tests to ensure the accuracy and reliability of our simulation results. This section details how we check that the new implementation remains robust and consistent with previous versions.
== Code validation
We ensure consistency of the updated BEoRN code with previous versions by running a series of simulations under identical conditions. We compare key outputs starting from the profiles of individuall sources, to the ionization maps, and finally to the global reionization signals. This step-by-step comparison allows us to identify any discrepancy that may arise from the code changes.
#let notebook = json("../workdir/11_visualization/validation_simple_run.ipynb")
#figure(
image_cell(notebook, cell_id: "validation_signal_comparison_old_v_new"),
caption: [
Validation signal comparison between the new implementation of #beorn (blue) and the old version (green, dashed). From _left_ to _right_:
Global evolution of the differential brightness temperature $d T_"b"$.
Evolution of the mean kinetic temperature of the gas $T_k$.
Mean ionization fraction history $x_"HII"$.
Dimensionless power spectrum of the $d T_"b"$ field as a function of redshift at $k = 0.12 "Mpc"^(-1)$.
],
) <fig:validation_signal_comparison_old_v_new>
We ensure consistency of the updated BEoRN code with previous versions by running a series of simulations under identical conditions. We compare key outputs starting from the profiles of individuall sources, to the ionization maps, and finally to the global reionization signals. This step-by-step comparison allows us to identify any discrepancy that may arise from the code changes. In @fig:validation_signal_comparison_old_v_new we present a comparison of the central observables of reionization between the previous and current versions of BEoRN. The underlying simulation parameters are kept identical or updated to an equivalent setting where necessary. The results show essentially unchanged evolution of the global brightness temperature, kinetic temperature, and ionization fraction. Note that this run uses an artifical halo catalog and an unphysically high star formation efficiency to ensure a rapid reionization within the simulation range. As such this setup is not representative of a realistic reionization scenario.
// but ...
Similarly we maintain backward compatibility with the input format used in previous BEORN runs (i.e. snapshots generated by pkdgrav or 21cmfast). This allows us to reproduce the earlier runs and match the results as described by @Schaeffer_2023.
== Convergence tests
To ensure that our results are not sensitive to the numerical resolution of the simulation, we perform convergence tests. We compare the following variations of resolution: Firstly we compare effects of the grid resolution by running simulations with $128^3$, $256^3$, and $512^3$ cells. Secondly we investigate the impact of the mass resolution by comparing the results obtained from the #smallcaps[Thesan-Dark] 1 and 2 simulations, which have different particle masses, as mentioned in @procedure.
#let notebook = json("../workdir/11_visualization/validation_convergence.ipynb")
#figure(
image_cell(notebook, cell_id: "validation_signal_comparison"),
caption: [
Validation signal comparison between runs using a grid resolution of $128^3$ (blue) and $256^3$ (green) as well as a run using the #thesandark 1 simulation ($2100^3$ particles) with a grid resolution of $128^3$ (yellow).
// TODO - specfify each color
From _left_ to _right_:
Global evolution of the differential brightness temperature $d T_"b"$.
Evolution of the mean kinetic temperature of the gas $T_k$.
Mean ionization fraction history $x_"HII"$.
Dimensionless power spectrum of the $d T_"b"$ field as a function of redshift at $k = 0.12 "Mpc"^(-1)$.
],
) <fig:validation_signal_comparison>
To ensure that our results are not sensitive to the numerical resolution of the simulation, we perform convergence tests. We compare the following variations of resolution: Firstly we compare effects of the grid resolution by running simulations with $128^3$ and $256^3$. Secondly we investigate the impact of the mass resolution by comparing the results obtained from the #smallcaps[Thesan-Dark] 1 and 2 simulations, which have different particle masses, as mentioned in @procedure.
== Scientific validation?
Against THESAN runs
// Probably not happening!
@fig:validation_signal_comparison shows that there is no significant difference when using a finer grid resolution. This is expected since the ionized regions rapidly become larger than the cell size. The comparison between the different mass resolutions sees a deviation in the timing of reionization. This effect is expected and documented by @Kannan_2021: The lowest mass halos which are not resolved by #thesandark 2 form small bubbles quickly (as early as $z=10$) and contribute to the ionization budget at early times. We account for this by artificially increasing the lower mass cutoff during these validation runs but an imbalance between the two simulations remains. Globally the shapes of the signals are very similar and the power spectra match closely.
An additional convergence test is performed by varying the binning of the accretion rate alpha parameter. Keeping the overall range fixed between $alpha = 0$ and $alpha = 5.0$ we compare the signal generated when using $n = 5, 25, 50$ bins. Since a bulk of the halos where the accretion rate cannot be fitted is assigned a fallback value which then falls into one of the bin, there is a global shift to the signal when changing the binning. However, the overall shape and features of the signal remain unchanged from $n = 25$ bins onwards.