Segmentation mask of reflection

[cs-mshah]

Describe the issue

I have an interesting use case:
I have a mirror and an object in front of it. I am able to get the rendering and segmentation mask of the scene. But I also want the segmentation mask of the reflection of the object in the mirror. How should I achieve this using blenderproc?

Minimal code example

No response

Files required to run the code

No response

Expected behavior

An additional data key called "reflection" would be beneficial after calling the blenderproc render function. This should contain the mask of the reflection that would be visible in the mirror of the object in front of it.

BlenderProc version

v2.7

[cornerfarmer]

Hey @cs-mshah,

interesting question. This definitely does not work out of the box.
You could try to do it similar to how the old segmap renderer worked (https://github.com/DLR-RM/BlenderProc/blob/main/blenderproc/python/renderer/SegMapRendererUtility.py):

• for the object you want to segment, use an emission shader (https://github.com/DLR-RM/BlenderProc/blob/main/blenderproc/python/renderer/SegMapRendererUtility.py#L258)
• for the mirror use your reflective material
• set the number of rays to 1, so you hopefully get not antialiasing (https://github.com/DLR-RM/BlenderProc/blob/main/blenderproc/python/renderer/SegMapRendererUtility.py#L50)

However, I am not fully sure this works. It might be that due to the mirror you get blur or not uniform coloring of the object of interest