how to input to "imread" using uint8 vector that is equivalent to binary data of image file from fread

Hi, I am trying to read many 512x512 jpg files that extracted from a .db database as uint8 vector for each file.

The vector is not the pixel intensity matrix, but exactly the binary data of file. So I need to fwrite the vector into a temp jpg file to hard disk, then imread that file as matrix in memory.

Because the number of jpg can be hundreds of thousands, so the frequent io from hard disc make the process very slow.

Is there a way to "recognize" the uint8 vector as if it is on the hard drive, then read it as a image file into imread?

Thanks.

Answer 1

[Walter Roberson]

For jpeg in particular, imread() invokes a function that calls upon a .mex* file to do the reading. It is likely that the file invokes libjpeg .

I believe that there is a route to do what you want by using Java. Java permits defining a Java.io.FileInputStream from memory (Byte array), and ImageIO.read() can be used to read images from FileInputStream .

https://javaconceptoftheday.com/read-and-write-images-in-java/

I do not have any code that puts all this together; I have not used Java much.

[raym]

I have used the java way to do the rgb binary reading, and use imwrite to write jpg. the speed increased from 3 round/s to 15 round/s. The SSD is even faster: 18~19 round/s.

Answer 2

[Image Analyst]

Of course imread() will "recognize" the image file on disk as a uint8 jpg image if you got it written out properly. I don't know of anyway to decompress a JPG array that's in binary in memory so writing it out to disk to read back in may be the way you need to do it. If that's too slow, you can use solid state drive instead of a hard drive. Exactly how slow is it? For me, it typically takes less than a second to read in an image from my SSD.

[raym]

I have more than 8000 (512x512) jpg data and each is split into R,G, and B in a SQLite .db file. I need to query three jpg's raw binary data into Matlab first, then fwrite the three to disk, then read three jpg using imread, then concatenate into a 3D matrix, then imwrite as a final jpeg, and repeat all the steps for 8000 timies. I installed a rampdisk software to speed it up, and it can process three round per second, so in total would need 45 minutes.

[Image Analyst]

Not bad. It would take you more than 45 minutes to figure out code to decompress the data to avoid a roundtrip to ramdisk.

[raym]

Yes. It is interesting to know why so many funciton/command/API that receive a filepath input cannot also accept a binary stream of the same binary data in hard disk.

[Image Analyst]

You can put it in as a suggestion but I think it would rarely be used by others so it probably will never rise to the top of the priority list. In over 40 years of image processing I've never had to do that.