remoll/master-thesis-report
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

report completed and refined to my personal satisfaction

Browse Source
This commit is contained in:
Remy Moll
2025-09-15 03:26:26 +02:00
parent cafaf38ddf
commit 221bdcda07
14 changed files with 354 additions and 215 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
validation.typ
View File

@@ -3,7 +3,7 @@
= Validation <validation>
We perform several validation tests to ensure the accuracy and reliability of our simulation results. This section details how we check that the new implementation remains robust and consistent with previous versions.
We perform several validation tests to ensure the accuracy and reliability of our simulation results. This section details how we verify that the new implementation remains robust and consistent with previous versions.
== Code validation
#let notebook = json("../workdir/11_visualization/validation_simple_run.ipynb")
@@ -14,14 +14,12 @@ We perform several validation tests to ensure the accuracy and reliability of ou
Global evolution of the differential brightness temperature $d T_"b"$.
Evolution of the mean kinetic temperature $T_k$.
History of the mean ionization fraction $x_"HII"$.
Dimensionless power spectrum of the $d T_"b"$ field as a function of redshift at $k = 0.12 "Mpc"^(-1)$.
21-cm power spectrum as a function of redshift at $k = 0.12 "cMpc"^(-1)$.
],
) <fig:validation_signal_comparison_old_v_new>
We ensure consistency of the updated #beorn code with previous versions by running a series of simulations under identical conditions. We compare key outputs starting from the profiles of individual sources, to the ionization maps, and finally to the global reionization signals. This step-by-step comparison allows us to identify any discrepancy that may arise from the code changes. In @fig:validation_signal_comparison_old_v_new we present a comparison of the central observables of reionization between the previous and current versions of #beorn. The underlying simulation parameters are kept identical or updated to an equivalent setting where necessary. The results show an essentially unchanged evolution of the global brightness temperature, kinetic temperature, and ionization fraction. Note that this run uses an artifical halo catalog and an unphysically high star formation efficiency to ensure a rapid reionization within the simulation range. As such this setup is not representative of a realistic reionization scenario.
// but ..
// TODO - comment on the power spectra!
We ensure the consistency of the updated #beorn code with previous versions by running a series of simulations under identical conditions. We compare key outputs starting from the profiles of individual sources, to the ionization maps, and finally to the global reionization signals. This step-by-step comparison allows us to identify any discrepancy that may arise from the code changes. In @fig:validation_signal_comparison_old_v_new we present a comparison of the central observables of reionization between the previous and current versions of #beorn. The underlying simulation parameters are kept identical or updated to an equivalent setting where necessary. The results show an essentially unchanged evolution of the global brightness temperature, kinetic temperature, and ionization fraction. Note that this run uses an artifical halo catalog and an unphysically high star formation efficiency to ensure a rapid reionization within the simulation range. As such this setup is not representative of a realistic reionization scenario.
Similarly we maintain backward compatibility with the input format used in previous #beorn runs (i.e. snapshots generated by #pkdgrav or `21cmfast`). This allows us to reproduce the earlier runs and match the results as described by @Schaeffer_2023.
@@ -32,18 +30,19 @@ Similarly we maintain backward compatibility with the input format used in previ
image_cell(notebook, cell_id: "validation_signal_comparison"),
caption: [
Validation signal comparison between runs using a grid resolution of $128^3$ (blue) and $256^3$ (green) as well as a run using the #thesandark 1 simulation ($2100^3$ particles) with a grid resolution of $128^3$ (yellow).
// TODO - specfify each color
From _left_ to _right_:
Global evolution of the differential brightness temperature $d T_"b"$.
Evolution of the mean kinetic temperature $T_k$.
History of the mean ionization fraction $x_"HII"$.
Dimensionless power spectrum of the $d T_"b"$ field as a function of redshift at $k = 0.12 "Mpc"^(-1)$.
Dimensionless power spectrum of the $d T_"b"$ field as a function of redshift at $k = 0.12 "cMpc"^(-1)$.
],
) <fig:validation_signal_comparison>
To ensure that our results are not sensitive to the numerical resolution of the simulation, we perform convergence tests. We compare the following variations of resolution: Firstly, we scrutinize the effect of the grid resolution by running simulations with $128^3$ and $256^3$ cells. Secondly, we investigate the impact of the mass resolution by comparing the results obtained from the #smallcaps[Thesan-Dark] 1 and 2 simulations, which have different particle masses, as mentioned in @procedure.
To ensure that our results are not sensitive to the numerical resolution of the simulation, we perform convergence tests. We compare the following variations of resolution: Firstly, we scrutinize the impact of the grid resolution by running simulations with $128^3$ and $256^3$ cells. Secondly, we investigate how changing the mass resolution affects the results by comparing the outputs obtained from the #smallcaps[Thesan-Dark] 1 and 2 simulations, which have different particle masses, as mentioned in @procedure.
@fig:validation_signal_comparison shows that there is no significant difference when using a finer grid resolution. This is expected since the ionized regions rapidly become larger than the cell size. The comparison between the different mass resolutions exhibits a deviation in the timing of reionization. This effect is expected and documented by @Kannan_2021: The lowest mass halos which are not resolved by #thesandark 2 form small bubbles quickly (as early as $z=10$) and contribute to the ionization budget at early times. We account for this by artificially increasing the lower mass cutoff during these validation runs but an imbalance between the two simulations remains. Globally the shapes of the signals are very similar and the power spectra match closely.
@fig:validation_signal_comparison shows that there is no significant difference when using a finer grid resolution. This is expected since the ionized regions rapidly become larger than the cell size. The comparison between the different mass resolutions exhibits a deviation in the timing of reionization. This effect is expected and documented by @Kannan_2021: The lowest mass halos which are not resolved by #thesandark 2 form small bubbles quickly (as early as $z=10$) and contribute to the ionization budget at early times. We account for this by artificially increasing the lower mass cutoff during these validation runs but an imbalance between the two simulations remains. Additionally, the higher particle count per halo seems to enable a more consistent tree finding. The lack of good trees at the earliest snapshots is clearly visible as a dip in the $d T_b$ signal of the #thesandark 2 runs. When the accretion rate cannot be inferred from the tree, a fallback value is used instead. This leads to a systematic shift in the signal that abruptly appears when the fallback value is no longer needed. After observing this effect we tune the fallback value to be in line with the expected accretion rate as shown in @fig:alpha_evolution_vs_redshift. This largely removes the jump in the subsequent runs.
Globally the shapes of the signals are very similar and, apart from the initially delayed cosmic dawn, the power spectra match closely after the discrepancy during the first few snapshots.
An additional convergence test is performed by varying the binning of the accretion rate $alpha$ parameter. Keeping the overall range fixed between $alpha = 0$ and $alpha = 5.0$, we compare the signal generated when using $n = 5, 25, 50$ bins. A bulk of the halos where the accretion rate cannot be fitted is assigned a fallback value. This then falls into one of the bins, causing a global shift of the signal when changing the binning. However, the overall shape and features of the signal remain unchanged from $n = 25$ bins onwards.