Tiled tif write support

[jamesra]

I would like to write some BigTiff 4GB+ images as tiled tiffs to export data to an Omero server. Pillow appears to support reading tiled tiff but not writing.

Being optimistic I naively added the TileWidth and TileLength tags to a tifinfo dictionary passed to the save function the result is an invalid tif file. My test code can be found on github here:

https://github.com/jamesra/random/blob/master/pillow_tif_tiles.py

TiffInfo shows the tags correctly in the resulting tif file, but it fails to load in Photoshop or any of the built-in Windows viewers I tried.

[wiredfool]

The libtiff support is pretty much only for writing a single tile. IIRC, It's going to take working on some of the low level code to make it work with tiles.

[jamesra]

There appear to be write functions in libtiff:
http://www.libtiff.org/man/TIFFWriteTile.3t.html

I may play with a crude implementation tomorrow if I can reach that function from Pillow. I'm still trying to wrap my head around what needs to be done.

[wiredfool]

The encoding is (helpfully enough) in TiffDecode.c, with the setup at the bottom of encode.c.

The setup code is in TiffImagePlugin.py -- if you're running with libtiff it actually does the save in _save instead of calling out to ImageFile._save. ImageFile._save does the tiles, save is hardwired for one.

TiffWriteTile may be a near drop in replacement to the shuffle and TiffWriteScanline function call in TiffDecode.c:ImagingLibTiffEncode. If that's the case, and it works in the single tile case, then it may just be a case of removing the hardwired encode call in TiffImagePlugin and using ImageFile._save instead.

I've meant to go back and clean up some of the tiff support for a while, If you can't get your head around it, I'll take a look at it at some point.

[jamesra]

Do all paths lead to LibTiff eventually? I'm trying to setup a debug environment for VS 2008. I'd like my system to work without installing additional dependencies. My users and I utilize the Gohlke installer to obtain Pillow for Windows. When I debug python the trace indicates that libtiff is not installed. However when I follow the chain through ImageFile._save the encoder seems to reach TiffDecode.c:LibTiffEncoderNew both paths appear to use libitff.

Thanks for offering to look. If you have npt seen a pull request by about 6/6/2014 I've probably had to work on other projects for a bit. I would still love and use a fix.

[wiredfool]

More and more roads are leading to libtiff. I think cgohilke's installers use statically linked libraries, so you'll have the tiff that it's compiled against.

And I do seem to remember that it's a pain to setup the debugging libs on windows.

[jamesra]

Yes it is. I've made it close to a semi-functional debug setup. I can pass selftest.py and the libtiff tests are either slow or frozen. Others such as JPeg seem to be passing.

One change change to the source of TiffDecode.h to compile on Windows was changing unistd.h to io.h

#ifndef _UNISTD_H
    #if defined(_MSC_VER)
        #include \<io.h>
        #define _UNISTD_H
    #else
        #include \<unistd.h>
    #endif  
#endif

Would you like that as a separate pull request? Is there a better way for me to structure that #if test to be cross-platform?

[aclark4life]

Is there any associated PR for this one yet?

[jamesra]

No sorry. I got stuck on the Windows dev environment and had to set it aside to work on something else for a bit. If you want to take a shot at it that would be great.

[jamesra]

Oh... I didn't read the post above yours. If you are asking about the #include change I didn't create a pull request yet because even though the alteration allows the build to complete I am not able to pass the tif tests with the resulting build. Some of the other tests do pass though.

The change in my branch is here:
jamesra@13c39cb

It would be easy to create the pull if it is wanted.

[aclark4life]

Sure please do send, thanks

[OniDaito]

Hi all. I've just started looking at the source as I'm really keen on BigTIFF (not necessarily tiled tiff) support. Has there been any movement on this at all or shall I just crack on with looking at libtiff? Cheers

[jamesra]

I haven’t had a chance to make progress.

It has been a while but I vaguely remember a Pillow API issue where I could not read/write specific tiles. I believe Pillow wanted to read or write the entire file. I need a method where I can update individual tiles. Like many BigTIFF users our images are too large to entirely load into memory and get decent performance.

[radarhere]

Does the image size limit imposed by #2583 mean that this scenario is no longer viable?

[bthorsted]

Will this ever be implemented in Pillow?

[aclark4life]

@bjtho08 Probably no time soon, unfortunately, unless you are offering to do it for us 😄 And it also may be a "wontfix" according to @radarhere 3 years ago, sorry.