Is it possible to output cost volume?

[topriss]

Hi,

It is amazing how good the disp map is even tested on challenging unseen image pairs.
I was wondering if anyone could also output the cost volume beside the disp map.
I checked the costVolumePlugin provided in lib, but didn't really understand how it works.

Thanks!

[Alexey-Kamenev]

That's an interesting question! In theory you should be able to get the output of the cost volume plugin by either modifying the TensorRT model generation script (like this one) which produces TRT C++ model files or directly auto-generated files themselves (like this one).
The idea is to make cost volume layer (cost_vol) an output layer by calling network->markOutput with appropriate parameters. Then you need to change the binding code in the sample app like here and other places where assumption are made about output layer.

Note that the cost volume values that you'll get will be in feature space, not pixel space. Also, in case of 3D model each value will actually be a vector as the output of cost volume plugin will be 5D tensor (see our paper for details).