Question: Dicom file appending

[bfokin]

Hello,

As a little background. I want to generate a dicom file that stores an image stack from a group of jpgs.
On a small scale, I am able to implement this by creating an array that stores np.arrays (where one np.array is from calling np.array() on a single jpg. From here, I just pass the frame array into ds.PixelData when creating a dicom file.

However, this is not doable on a large scale due to memory constraints (the ram would be all used up).

Question:
Is it possible to append np.arrays in chunks to a dicom file rather than passing the full np.array into ds.PixelData?

Thank you very much!

[darcymason]

I can't think of any way to do this directly in pydicom presently, but I can think of one possible workaround:

• copy the write_dataset() code (it's quite short) in filewriter.py to another function name in your own module, say my_write_dataset
• edit my_write_dataset to check for your pixel data tag, and when found, call your own function(s) to write the pixel data DataElement (see the code near the end of filewriter.write_dataelement() for examples of how to write the tag, VR if not implicit VR, and the 4-byte length), then write your value in chunks
• somewhere before calling dataset.save_as or dcmwrite, do:

import pydicom.filewriter
pydicom.filewriter.write_dataset = my_write_dataset

[bfokin]

Is it possible, within write_data_element to set the elem data to a the pixel data array?

So we have (7fe0, 0010) Pixel data OB:None.

Could I use the pixel array to replace None?

[darcymason]

I'm not sure if I understand. You can set the pixel data, but your original question was that you couldn't do that because it is too large to do all the data at once.

Another thought just occurred to me - have you thought about using memmap'd files? This essentially allows the program to use virtual memory larger than the physical memory, without your code having to worry about the actual size.

[bfokin]

I'm sorry if my question wasn't clear.

Within my_write_data_element, I was planning on, in batches, write frame data to a buffer which would then write to the fp.

when my_write_data_element is called, it is only when we are writing to pixel data.
One of the parameters of the function is elem which contains (7f30, 001) Pixel Data with OB: None in this case (since there hasn't been any frames stored in database.PixelData).

Within write_data_element, the function write_OBvalue() will write to the buffer fp.write(cast(bytes, elem.value))

So I thought that I could store the nparray data from the batch within elem.value so then it would be written to the file. However, I got an error saying a bytes-like object is requred, not 'MultiValue' when the line fp.write(cast(bytes, elem.value)) within write_OBvalue is executed.

When I said, "Could I use the pixel array to replace None?" I was referring to, could I use the nparray data into elem.value in batches. I am trying to use the library as vanilla as possible

[darcymason]

Within write_data_element, the function write_OBvalue()

I don't see a need to call a pydicom function like write_OBvalue - it is basically a single line. It would be much easier just to call fp.write() in your own code, for each chunk of your pixel data.

[bfokin]

memmap was just the implementation I was looking for. Thank you for pointing me in the right direction! I appreciate your help!