Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setting manual raw_pattern #217

Open
mandulaj opened this issue Jan 18, 2024 · 10 comments
Open

Setting manual raw_pattern #217

mandulaj opened this issue Jan 18, 2024 · 10 comments

Comments

@mandulaj
Copy link

mandulaj commented Jan 18, 2024

I am loading a Raw image (stored as tiff) however the default Raw pattern RGGB:

array([[0, 1],
       [3, 2]], dtype=uint8)

is not the correct one. I would need GBRG corresponding to something like:

array([[1, 3],
       [0, 2]], dtype=uint8)

However I haven't found a way how to set this manually in rawpy.

@letmaik
Copy link
Owner

letmaik commented Jan 29, 2024

Can you post your complete code and upload an example image?

@mandulaj
Copy link
Author

Hey @letmaik, thanks for getting back to me.

Let me show you an example. This is the RAW un-debayerd TIFF (I can also provide the original file if needed). The Sensor has the following bayer pattern order GBRG:

Screenshot_2024-01-30_10-24-21

Ignoring the green tint, this is what the correct debayering using https://github.com/cheind/pytorch-debayer and the GBRG bayer pattern results in:
Screenshot_2024-01-30_10-20-56

I am using the RawPy library in a very simple way:

import rawpy

with rawpy.imread("image.tif") as raw_img:
    img = raw_img.postprocess()
    plt.imshow(img)
    plt.show()

However this results in the following results:
Screenshot_2024-01-30_10-27-43

Clearly rawpy is using a different bayer pattern for the debayering process. This can be seen by accessing the raw_pattern property showing it uses RGGB. Since raw_pattern is read only, I haven't found a way to change it to GBRG.

Bayer_patterns

@mandulaj
Copy link
Author

Here is the RAW TIFF:
image.tif.zip

@letmaik
Copy link
Owner

letmaik commented Jan 30, 2024

It's unusual to have RAW images in TIFF format. The issue is that there's not enough metadata in the file I think. The underlying library libraw then doesn't know which Bayer pattern to apply. If I may ask, where do you get this image from? Is it produced by a camera directly?

@mandulaj
Copy link
Author

Yes, I know, the TIFFs are in fact written by myself, so I could even add the necessary metadata if I knew what it was.

I don't want to go into details, but the summary is, for performance reasons, TIFF is the fastest format we can write the data to disk. We tried several Debayering and post-processing algorithms, including the standard OpenCV, 5x5 convolutions, but libraw yielded the best results so far (on a different camera model that has the default RGGB pattern).

So let me get this straight, the underlying libraw library is expecting some kind of metadata that indicates the bayer pattern, however, there is no API to set it manually.

Since you are probably more familiar with the Libraw library, would you happen to know which metadata would have to be added? Perhaps if you point me in the right direction, I can try to add this feature to rawpy to set the pattern manually.

@letmaik
Copy link
Owner

letmaik commented Jan 31, 2024

libraw seems to have ways to read from image data without metadata as well, e.g. https://www.libraw.org/docs/API-CXX.html#open_bayer. Maybe that would suit your use case. This is not currently implemented in rawpy but it seems like a useful thing to have. There is normally quite a bit of metadata in RAW camera files that influence the postprocessing, but maybe that's not needed in your case and the Bayer pattern is enough.

@mandulaj
Copy link
Author

I will look into it once I have time, but I am not that familiar with making cpython bindings with a C library so not sure if I will be able to do this myself. 😅

@letmaik
Copy link
Owner

letmaik commented Jan 31, 2024

I gave it a try and got it to work. I'll give you something to test soon.

@letmaik
Copy link
Owner

letmaik commented Feb 1, 2024

This is not officially released yet as I'd like you to try it out first. You can manually download Python wheels with this feature from the workflow artifacts (bottom right) from here: https://github.com/letmaik/rawpy/actions/runs/7734024290

This is how you would use it with imageio:

import rawpy
import imageio

arr = imageio.imread("rawbayer.tiff")
with rawpy.RawPy() as raw:
    raw.open_bayer(arr.tobytes(), arr.shape[1], arr.shape[0], "GBRG")
    rgb = raw.postprocess()

And here with pillow:

import rawpy
from PIL import Image

img = Image.open("rawbayer.tiff")
with rawpy.RawPy() as raw:
    raw.open_bayer(img.tobytes(), img.width, img.height, "GBRG")
    rgb = raw.postprocess()

For now, I'm not exposing it under a high-level interface like rawpy.imread since it is quite advanced and I haven't made up my mind yet on how that interface would look like. For your purposes, this doesn't matter though.

Please try it out and let me know if it's working as expected.

@kmilos
Copy link
Contributor

kmilos commented Feb 6, 2024

I don't want to go into details, but the summary is, for performance reasons, TIFF is the fastest format we can write the data to disk.

So... just throw in some basic metadata on top (CFA pattern, color matrix, etc.) and you have yourself a DNG. 😉 See e.g. https://www.libraw.org/comment/5771#comment-5771 and https://github.com/schoolpost/PiDNG

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants