Utilities

This directory contains a number of useful tools and utilities for handling the data from the ASTRO3D Genesis simulations. We have provided a number of example scripts to run these tools in the run_scripts directory.

adjust_spec

This utility adjusts the data specification of the default VELOCIraptor``+``Wherewolf trees to match the required specification of the LHaloTree strucutre. See LHaloTreeReader for an overview of the LHalo tree structure.

forest_sorter

This utility takes input HDF5 merger trees that have not been saved in any specific order and sorts them on a number of fields; usually an at least an ID field and a mass field. The user can specify on which fields the sorting should occur. In the default case, the trees are sorted first on the ForestID, then the hostHaloID and finally in descending order of 200mean mass of the halo. This results in halos that are sorted in ascending order according to their ForestID, then within each Forest, all halos within a single FoF group are grouped and finally within each FoF-group, the most massive halo is sorted first.

Tests

First please run the basic tests on the default test data provided by invoking pytest. If this default test does not pass, please email jseiler@swin.edu.au

Included in the main function of tests/forest_sorter_test.py is an example of running tests with customized settings. Of particular note is the gen_data variable. If this is set to 1, a small set of sorted trees will be generated from the specified unsorted HDF5 trees. The number of halos tested on is handled by the NHalos_test variable.

If you wish to test your fully sorted trees after running forest_sorter(), set gen_data=0, fname_in to the path of the original unsorted trees and fname_out to the path of the sorted trees.

If the default test passes but your specific test fails please ensure that your data file is not corrupt. Importantly, check that the snapshot keys are named appropriately. We require the snapshot fields to include the word snap (case insensitive) and assume that the snapshot number corresponding to the snapshot key is included as a single cluster towards the end of the key; snap53_04 should correspond to snapshot number 04 for example.

If the snapshot fields are named correctly and your data can be otherwise read in via `h5py`, please email jseiler@swin.edu.au

convert_indices

This utility takes the sorted HDF5 trees from forest_sorter and adjusts the halo IDs to match the requirements of LHalo trees. LHalo trees requires that these IDs are tree local and are the indices of the halos (rather than unique temporal IDs).

The resulting LHalo compatible trees are saved as a HDF5 file with all other fields identical to the input trees.

treefrog_to_lhalo

This function takes trees with the LHalo corrected indices (from convert_indices()) and writes LHalo tree format. Under this format, the tree pointers used to walk the tree are tree local. See LHaloTreeReader for an overview of the LHalo tree style pointers.

The output format for this function can be either binary (write_binary_flag == 1), HDF5 (write_binary_flag == 0) or both (write_binary_flag == 2).

The binary files will have the following data format:

  • 32-bit integer: NTrees, describing the number of trees in the file,
  • 32-bit integer: TotNHalos, describing the total number of halos within the file,
  • NTrees 32-bit integers: TreeNHalos, describing the number of halos within each tree.

Following this header is TotNHalos halo entries with data format:

  • Descendant, 32-bit integer,
  • FirstProgenitor, 32-bit integer,
  • NextProgenitor, 32-bit integer,
  • FirstHaloInFOFgroup, 32-bit integer,
  • NextHaloInFOFgroup, 32-bit integer,
  • Len, 32-bit integer,
  • M_Mean200, 32-bit float,
  • Mvir, 32-bit float,
  • M_TopHat, 32-bit float,
  • Posx, 32-bit float,
  • Posy, 32-bit float,
  • Posz, 32-bit float,
  • Velx, 32-bit float,
  • Vely, 32-bit float,
  • Velz, 32-bit float,
  • VelDisp, 32-bit float,
  • Vmax, 32-bit float,
  • Spinx, 32-bit float,
  • Spiny, 32-bit float,
  • Spinz, 32-bit float,
  • MostBoundID, 64-bit integer,
  • SnapNum, 32-bit integer,
  • Filenr, 32-bit integer,
  • SubHaloIndex, 32-bit integer,
  • SubHalfMass, 32-bit integer.

The function is MPI compatible and the final number of files written is equivalent to the number of processors used to call the function. These files are load balanced such that each one will have a similar number of halos (but not necessarily number of trees). For example,

$ mpirun -np 4 python run_scripts/run_treefrog_to_lhalo.py

Would generate 4 LHalo tree binary files.