Reproduce result & Evaluation on Matterport dataset

[tsunghan-wu]

Hi,

I'm trying to reproduce the result on matterport dataset. However, I have encountered difficulties in it. The following lines are the script I ran. However, after I ran the script, I saw some blur depth images which is not the as clear as that in the paper and the evaluation is not the same. Could anyone tell me whether my script is incorrect ? Thanks !!!

cd('../torch/');

% Boundary detection
cmd = 'lua main_test_bound_matterport.lua -test_model ../pre_train_model/bound.t7 -test_file ./data_list/mp_test_list_horizontal.txt -root_path <my_root_path>';
system(cmd); % the result should be in ../torch/result/

% Surface normal estimation
cmd = 'lua main_test_matterport.lua -test_model ../pre_train_model/normal_matterport.t7 -test_file ./data_list/mp_test_list_horizontal.txt -root_path <my_root_path>';
system(cmd);

% cd('../matlab/');

% Get occlusion boundary (the 2nd channel) and convert to weight
GenerateOcclusionWeight('../torch/result/bound_matterport_test_bound/', '../torch/result/bound_matterport_weight/');

% Compose depth by global optimization
composeDepth('<my_root_path>', '../torch/result/normal_matterport_matterport_test', '../torch/result/bound_matterport_weight', 'mp_render', '../results/matterport', [1001, 0.001, 1]);

[yindaz]

The result quality may varies from case to case. Did you run through the demo successfully and get reasonable result?

[tsunghan-wu]

Thanks for your kindly response.
In your DeepCompletionRelease/matlab/evalDepth.m, line 60 ~ 70 :

metric.rmse = sqrt(median(vertcat(example_error.sqr)));
dt = vertcat(example_error.dt);
metric.dt = [sum(dt<1.05)/length(dt) sum(dt<1.10)/length(dt) sum(dt<1.25)/length(dt) sum(dt<1.5625)/length(dt) sum(dt<1.9531)/length(dt)];

metric.rel_raw = median(vertcat(example_error.rel_raw));
metric.rmse_raw = sqrt(median(vertcat(example_error.sqr_raw)));
dt = vertcat(example_error.dt_raw);
metric.dt_raw = [sum(dt<1.05)/length(dt) sum(dt<1.10)/length(dt) sum(dt<1.25)/length(dt) sum(dt<1.5625)/length(dt) sum(dt<1.9531)/length(dt)];

metric.rel_ukn = median(vertcat(example_error.rel_ukn));
metric.rmse_ukn = sqrt(median(vertcat(example_error.sqr_ukn)));

I think your implementation is not correct. Your code calculate "root median square error" rather than "root mean square error".

[suklasatapathy]

How can I run this code for my depth images ? The matlab code is running only for saved results ? nUnable to generate new results%%%%%%%%%
% Visualize results
input = imread('../data/realsense/030_depth_open.png');
output = imread('../results/realsense/realsense_030_1.png');
colormap = jet(double(max([input(:);output(:)])));
figure;
subplot(1,2,1); imshow(label2rgb(input, colormap)); title('Input');
subplot(1,2,2); imshow(label2rgb(output,colormap)); title('Output');

[suklasatapathy]

In the result folder: results/realsense/: realsense_004_1.png, realsense_012_1.png, realsense_026_1.png realsense_030_1.png, realsense_046_1.png are available. So If I am running demo code of matlab then only these files can be produced. How to generate result for other depth maps. Kindly reply.

[suklasatapathy]

hdf5 files are not found. From where I have to download it ?

[Darker-zq]

@tsunghan-mama Do you have any datasets provided by the author? If so, please send me a copy.Thanks!
My Email is [email protected]