Provide a passthrough access to image data from scene importers

[hugoam]

Hello !

We would like to thread the importing of images attached to an imported scene file, that we currently get mainly through AbstractImporter::image2D() function. Being able to thread it would be cool as it turns out this is one of the main bottlenecks when loading different kinds of scenes.

As a simple solution, we thought a image2DData() function could be provided as an alternative to image2D(), but maybe you have other better ideas. Giving a quick try this approach seemed easy enough to implement and can be made to work with both embedded files and separate files, although the ownership of what it returns is an open question in that case 😄.

If you have a precise idea of how to approach this but don't have time, I could help, so let us know 😅

Cheers

[mosra]

[a hour-long call happened in the middle of writing this, sorry]

I have a precise idea how to do this, already discussed it briefly with @Squareys some time ago. Coincidentally I need it as well (for optimizing gigabyte-sized scenes without image recompression), so I'll try my best to prioritize it. If you have a hard deadline for this, please let me know ;)

It'd be essentially about this:

• The overall API stays the same (so, still returning an ImageData*D from image*D(), and similarly for the image/scene converters). That has a nice advantage of various proxy importers not having to do anything in order to support the data passthrough.
• The ImageData would then contain ... the image file data, verbatim. Like, the PNG file stored in there. And to distinguish it from regular images, it'd use an implementation-specific pixel format, pixel size of zero and zero row / slice strides as well. Similar approach is used for supporting the meshopt compression in Mesh compression magnum-plugins#129. There's still some minor bits to figure out but I have the idea pretty clear in my head already.

And besides allowing you to fan out the import to a worker pool, it'll allow these features:

• Converting between scene formats (FBX -> glTF, say) while keeping the embedded images in whatever original format they were in, avoiding unnecessary work and/or data loss.
• Ability to transcode images without data loss (say, optimizing already-encoded PNGs with optipng, or JPEG -> JPEG XL conversion that makes the image bit-exact but signficantly smaller).
• Importing just image metadata (sizes, formats, other things) but not the data. (This isn't fully clear in my head yet, though, as it conflicts slightly with the idea of using an implementation-specific pixel format from above.)
• And reusing the existing ImageData interface means it can take advantage of things like zero-copy import (Zero-copy importer plugin APIs #240, wait, 2018?! sigh I'm working on way too many things in parallel), where the returned ImageData instance would be just a non-owning view on a range of the input file.

It depends on a bunch of changes to the ImageData internals (getting rid of the nasty GL-specific PixelStorage class in favor of something more flexible that can handle also Y up / Y down differences for Vulkan & WebGPU, which is another thing I desperately need), and similar work block-compressed images.

I could help

At least initially it'll be quite an annoying piece of work touching obscure internals so probably not something you can help me with, but if I'll come across something that's possible to "outsource", I'll get in touch :)