Zoom Region Snapshots

[JBorrow]

Zoom regions have a smaller (not-entirely-box-covering) grid. There will also be particles that are outside of any cell.

These will be tracked by:

• NumParticles_InCells
• NumParticles_OutsideCells

All other particles will be at the end of the array. We need to be able to read these.

• Read background particles only (i.e. outside of any cells)
• Read cell particles only (i.e. only the zoom region)
• Read sub-cells (already handled..?)
• Read cells and outside of cells (e.g. when looking at a halo right on the zoom boundary)

[JBorrow]

@MatthieuSchaller @WillJRoper

[MatthieuSchaller]

I will add NumParticles_InCells_Total and NumParticles_InCells_ThisFile. Same for outside. This may be helpful for distributed snapshots.

[kyleaoman]

From what I've heard of the zoom code it translates the zoom region to be in the box centre. This is a little bit annoying to compare to the parent box. There are a few ways to address this, but my suggestion after a quick think would be to (1) store some metadata in the snapshot files that contains the offset from the parent box and (2) allow a swiftsimio user to set a flag (parent_box_coordinates=True or something like that). This could either be once on initialisation, and then everything is assumed to happen either in the parent box frame or in the zoom box frame depending on the value, or it could be allowed to be specified on any relevant function call (e.g. anywhere a region is passed, etc).

Note that I'd want to be able to both provide input (e.g. specify a region) in my coordinate frame of choice, and receive output (e.g. coordinate arrays) in my frame of choice. Alternatively, it could be decided that the parent box frame is the "true" frame and the recentering of the zoom region becomes an implementation detail that is handled internally. Needs some thought/discussion, I guess.

[WillJRoper]

The current plan is to automatically shift the coordinates (and zoom region metadata) back to the parent coordinates so the outputs will always be in the same frame as the ICs (including any user-specified shift which is already stored in the snapshots). For me, this is the simplest approach since it removes any questions about the frame.

The bigger wrinkle is that a lot of IC generation codes will already perform the shift at which point the information is already lost (at least in my experience, I don't know if this has been addressed on more recent pipelines).

[MatthieuSchaller]

We forgot to write it here but we will undo any shift that the code adds behind the user's back. This will happen at i/o time and will put back the particles where they belong based on the ICs.

I think that would fix the point you bring up?

[kyleaoman]

Ok, if the snapshots are written in the parent box frame then there's nothing to do in swiftsimio, I think. I just wasn't sure if that was going to be straightforward with the cells involved.

[MatthieuSchaller]

@JBorrow: We currently already write the cell centre as well as the cell width. If, in the case of a zoom, I write out the centres of the zoom-region cells and their width then we have everything we need.

I can add the location of the bottom-left-front corner of the zoom region if it helps. (i.e. the (0,0,0) in the non-zoom case).

[WillJRoper]

I think we will need the bottom-left-front corner to shift during the cell index look-up? Although that could admittedly be derived from the centres and widths.

[JBorrow]

Yeah that bottom left corner will help. We won't be able to get away without any code changes here because of the background particles at the end of the array, but that's ok.

[MatthieuSchaller]

Ok. I'll add that. There might be little code changes needed to access just the zoom-in region particles though.

[WillJRoper]

@MatthieuSchaller has made some of the necessary changes (minus those needed in MPI land) on the zoom_io branch of SWIFT. I've tested this and generated a snapshot on cosma which can be found here:

/cosma8/data/dp004/dc-rope1/SWIFT/test_snaps/eagle_dmo_0000.hdf5

I've run a test on this and everything appears to be working as expected. This test can be found here which also shows how to use the "Origin" attribute to extract a cell containing a position.