Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When using file-level compression, allow setting compression parameters #422

Merged
merged 5 commits into from
Feb 23, 2025

Conversation

dstndstn
Copy link
Contributor

In the legacypipe code, we use mem:// files very extensively -- this allows us to do in-memory checksums before writing to disk.

We also use compression extensively, which we set at the file level via extended filename syntax - eg we create files like

fits = fitsio.FITS('mem://[compress R 100,100; qz -1e-4]', 'rw')

However, we find that when doing this, when you go to write an image HDU, the compression parameters don't get passed through because compress= is not set.

This patch fixes that.

Test case: these two files should get the same ZDITHER0 set.

import numpy as np
import fitsio
A = ((1+np.arange(10000)) * 0.1).astype(np.float32)

fitsio.write('dither1.fits', A.copy(), dither_seed='checksum', compress='RICE', qlevel=1e-4,
             tile_dims=100, clobber=True)
hdr = fitsio.read_header('dither1.fits', ext=1)
print('File #1: ZDITHER0:', hdr['ZDITHER0'])

fits = fitsio.FITS('mem://[compress R 100,100; qz -1e-4]', 'rw')
fits.write(A.copy(), dither_seed='checksum')
data = fits.read_raw()
fits.close()
f = open('dither2.fits', 'wb')
f.write(data)
f.close()
hdr = fitsio.read_header('dither2.fits', ext=1)
print('File #2: ZDITHER0:', hdr['ZDITHER0'])

@esheldon
Copy link
Owner

Thanks Dustin. Can you add a unit test for this one?

Copy link
Collaborator

@beckermr beckermr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@beckermr beckermr merged commit 273a00c into esheldon:master Feb 23, 2025
13 checks passed
@dstndstn dstndstn deleted the dither_compressed_file branch February 24, 2025 01:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants