Overflow Exception in MakeBigTIFF

[mmckean]

I'm using your library to create very large tiffs from stitched microscopy images. Your code has been working great! I really appreciate this project! I recently ran into a problem that I've been unable to figure out a solution for. When writing the output for one particularly large scan, I'm running into an Overflow Exception in MakeBigTiff.

For this problematic scan, the details are that I'm writing out a 81,920 x 57,344 tiled tiff. (16bit, single channel) I then create a second tiled tiff that's 40,960 x 28672. I open the larger tiff in "a" mode and add the second image as a page by copying one tile at a time. After all of the tiles have been copied, as the tiff objects are disposed, the exception occurs in the MakeBigTiff method. This code has been working fine now with many other smaller files.

I've stepped through the source code for MakeBigTiff a few times, but so far I'm unable to identify exactly what the issue is. I've had a tough time following the intended logic in this method. I see at least a couple of places where values appear to be read into variables that are too small to hold them. I'm unsure though if some of this is intentional.

I made one attempt at creating a unit test that could reproduce the issue. That first attempt didn't trigger the exception, but it did create a tiff file that couldn't be opened. I'm going to try another attempt using the same exact dimensions as the problematic image - I'll update this ticket with that test, if it works.

Here's the exception text:
System.OverflowException occurred
HResult=-2146233066
Message=Arithmetic operation resulted in an overflow.
Source=BitMiracle.LibTiff.NET40
StackTrace:
at BitMiracle.LibTiff.Classic.Tiff.MakeBigTIFF() in C:\Root\GitHub\Repos\libtiff.net\LibTiff\Internal\Tiff_DirWrite.cs:line 490
InnerException:

[mmckean]

I was able to create a simple unit test to reproduce the problem. In the attached zip, there are two files. Add BigTiff.cs to the UnitTests project and copy the tiff file to the TestCase folder.

I suspect that the problem may be in adding a non-bigtiff image as a page to the bigtiff image. The problematic code seems to be called in that case.

TestCase.zip

[Bobrovsky]

Hello!

Thank you for the detailed bug report and the test code.

Unfortunately, we are very busy right now so we won't be able to take a look at it in the next few days.

[mmckean]

I wanted to let you know that I've found a workaround that I can use for now. If I open my output tiffs as BigTiffs, regardless of their size, I don't hit this bug.

For example:
using (Tiff dest = Tiff.Open(TiledTiffPath, "a8"))

I'm still interested in the fix for the underlying issue, and I'd be happy to help out in any way possible.

Thanks!

[Bobrovsky]

Hello,

I converted your test code to C++ and checked it with libtiff 4.0.7 (latest version of the original library). The code fails with "Maximum TIFF file size exceeded" when creating output image for the first time. It does so because TIFF_BIGTIFF flag is not set.

So, for now I see two options:

1. Make LibTiff.Net throw an exception in a similar case.
2. Fix LibTiff.Net so it will execute your test without any issues.

The first option will make the library more explicit in case of an error but you will still need to use your workaround.

The second option will probably require more work and I am not sure we will find time for it.

Please let me know if you see an other option.

Here is the code I used:

int _tmain(int argc, _TCHAR* argv[])
{
	int height = 81920;
	int width = 57344;
	int tileSize = 256;
	int tileRows = height / tileSize;
	int tileCols = width / tileSize;

	char* fn = "verybig.tiff";
	remove(fn);

	// Use a 256x256 16bit single-channel tiff as the input.
	TIFF* baseImg = TIFFOpen("tile-gray-16.tiff", "r");
	// Read the base image into a buffer
	int baseImgDataLength = tileSize * tileSize * 2;
	unsigned char* baseImgData = new unsigned char[baseImgDataLength];
	TIFFReadRawTile(baseImg, 0, baseImgData, baseImgDataLength);

	// Write a new tiff, using the input image as a tile...
	TIFF* outputImg = TIFFOpen(fn, "w");
	TIFFSetField(outputImg, TIFFTAG_IMAGEWIDTH, width);
	TIFFSetField(outputImg, TIFFTAG_IMAGELENGTH, height);
	TIFFSetField(outputImg, TIFFTAG_COMPRESSION, COMPRESSION_NONE);
	TIFFSetField(outputImg, TIFFTAG_PHOTOMETRIC, PHOTOMETRIC_MINISBLACK);
	TIFFSetField(outputImg, TIFFTAG_FILLORDER, FILLORDER_MSB2LSB);

	TIFFSetField(outputImg, TIFFTAG_ROWSPERSTRIP, height);

	TIFFSetField(outputImg, TIFFTAG_XRESOLUTION, 96);
	TIFFSetField(outputImg, TIFFTAG_YRESOLUTION, 96);

	TIFFSetField(outputImg, TIFFTAG_BITSPERSAMPLE, 16);
	TIFFSetField(outputImg, TIFFTAG_SAMPLESPERPIXEL, 1);

	TIFFSetField(outputImg, TIFFTAG_PLANARCONFIG, PLANARCONFIG_CONTIG);

	TIFFSetField(outputImg, TIFFTAG_TILEWIDTH, tileSize);
	TIFFSetField(outputImg, TIFFTAG_TILELENGTH, tileSize);

	int numberOfTiles = tileCols * tileRows;
	for (int tileNumber = 0; tileNumber < numberOfTiles; tileNumber++)
	{
		TIFFWriteRawTile(outputImg, tileNumber, baseImgData, baseImgDataLength);
	}
	TIFFClose(outputImg);

	TIFFClose(baseImg);
}

[mmckean]

Thanks for following up. I think given the constraints on time available to fix the issue, it would be totally acceptable to mirror the behavior of libtiff, and throw an exception with a descriptive message.