I cannot download the zip file from google drive

[Helicopt]

I cannot download the zip file from google drive

every time i try to download it , it stucked at 20 MB and then failed as 'Network error'.

Can you check it or provide another link?

By the way, do you think it is fair to use a more powerful detector, like some private detector in MOT16 private to make tracktors in MOT17 better? Then it seems that it is very easy to improve MOT17 performance to 60% MOTA via your method because many MOT16 private detectors can boost trackers to 70% MOTA. Why you choose FPN & FRCNN but not such private one? If you use such private detector to refine your tracktor, can you get better MOTA ?

[timmeinhardt]

I just checked the link and was able to download the zip file without any problem. Please check your internet connection.

What you are suggesting is definitely possible but not considered as working with public detections anymore. We only use the regressor of the FRCNN+FPN but apply it to the public detections. You could of course choose an even better detector and use its regressor but that doesn`t mean that its superior detection performance directly translates to tracking performance.

[Helicopt]

Thanks! Maybe there is something wrong with my network, I will check it later.

But rethinking this work, I am still with questions. Anyway, even if you are using the only part of regressor in the detector, it works like a real detector. For example, it takes time as adding a new detector. So in real application of MOT, if I am able to use FPN & FRCNN, I won't use a poor detector as DPM, why not directly use FPN & FRCNN as detection input? So if the tracktor can only improve performance on DPM, it is very wired, it does not contribute anything but just like replacing the detector. But for MOT17, improving DPM can boost the overall result. So have you compare to other methods on FRCNN or SDP? To prove it, I guess more results on SDP are needed. For example, if we use the tracktor on SDP and the result of SDP become better than all other methods with SDP, that will be more awesome.

[Helicopt]

I just investigate with MOTDT17 and the Tracktor17 in SDP detection, and I found that MOTDT17 is much better than Tracktor17 with SDP input. But it is true that Tracktor17 has higher MOTA on average with all 3 detector than MOTDT17. In the sumplementary of paper you only report you are better with no-fpn than others. So the fact is that when it comes to better input than FRCNN, the tracktor seems meaningless.

[timmeinhardt]

Regarding your two last posts I also wanted to refer you to the no-fpn comparison in Table 2 of the supplementary and agree with your observation that we are worse than MOTDT on SDP detections. However, as Table 2 shows, Tracktor (without FPN) is superior on Faster R-CNN detections. This demonstrates the potential of Tracktor, in other words, the application of a regressor to perform frame-to-frame bounding box tracking. But clearly, MOTDT benefits more from the rich set of SDP object detections. However, we argue that this does not render Tracktor meaningless but motivates the combination of methods like MOTDT on top of Tracktor. Apply Tracktor for its robustness to missing detections (see results Table 2), online applicability and its little computational cost to all the easy tracking scenarios and extend it with additional tracking specific methods such as MOTDT.

[Helicopt]

Thank you for the detailed reply.

Sure, I notice that Tracktor without FPN is superior on FRCNN. I believe tracktor is helpful as a regressor, but I still don't think using another FRCNN after FRCNN is 'little computational cost', expecially in real world applications. That's why I think it is weired. It means we need to use two-pass detection, and the second pass only improve a little bit on MOTA.

Anyway, thanks for your work. I think it's another way to change the MOT community.

[timmeinhardt]

In a real data (private detections) setting Tracktor is not considered as another FRCNN after a FRCNN, but just one object detector providing the detections and the regressions of the previous bounding boxes to the current frame. This can be achieved with a single forwarding of the image and bounding boxes through the regressor. In that sense, Tracktor provides indeed a lightweight multi-object tracker with no need for additional training on tracking ground truth data.

[Helicopt]

That sounds right about the computational cost. However, the problem is still there. If the first-pass detector( detection input) is identical with the second-pass(regress ), how much will the improvement be. This concern is based on your SDP result. And your FRCNN is not identical with the public FRCNN, they are not the same model, right? So I wonder IF you use the same FRCNN, as both detection input and regressor, will it improves? Maybe in the supplementary the improvement is from your better FRCNN(better than the public one). And it seems that if they are same, or the first-pass ( detection input) is better, then the second-pass (regressor or Tracktor) cannot make the result better.

[Helicopt]

I mean whatever detector you use , whatever one-pass or two-pass, the ceiling is there, it is the best detector you use.

If you use DPM as detection and FRCNN as Tracktor, the result is similar with use FRCNN as detection input and no Tracktor. If you use FRCNN(worse version) as detection input and FRCNN (better version) as Tracktor, the result is similar with only better FRCNN as detection input and no Tracktor. And if you use SDP as detection input and FRCNN as Tracktor, the result even becomes worse than using SDP only and no Tracktor.

So I wonder if you use same FRCNN as detection input and Tracktor, what is the different with use use FRCNN as detection input only. This is not the case in supplementary, I mean the same FRCNN checkpoint model, but not just same method. If it is still better when use Tracktor, I think it is more convincing. But now I do not see such a important experiment in the paper and the supplementary is not such convincing. If the fact is that same FRCNN as detection input and Tracktor can not bring improvement than FRCNN as detection input , i think the Tracktor is not so helpful as proposed. It is just a very simple fact, the better detection you use , the better result you gain.

[timmeinhardt]

I totally agree that Tracktor supports the underlying principle of tracking-by-detection, which is: the better the detection (detector) the better is the tracking. But I think you are missing a fundamental point of Tracktor. For SPD detection and FRCNN Tracktor vs. SDP only and no Tracktor , the first will be substantially better. The latter does not tackle the multi-object problem at all. You will only have tracks with a length of one for each frame because the data association between frames is not there. The regression of Tracktor is not to improve on the detections, but to form tracks from frame to frame by regressing the old bounding box of the previous frame to the current frame and the object`s new position. Our contribution lays in the fact that we can do this by only using the object detector which is already a part of the tracking-by-detection pipeline anyway.

[Helicopt]

Wait, I do not totally agree that SDP detection with FRCNN tracktor is substantially better. Is this proved by any experiment? If so, I am glad to see that.

Then, you say no tracktor does not tackle MOT problem. Well, I found a lot SOT methods in MOT, I think they are also some kind of regressors. I guess the only advantage is that we need not to train another regressor. But it is not so important afterall.

The key difference between Tracktor and SOT is that, SOT can tracks everything, the proposed tracktor only tracks humans. So sometimes SOT cannot distinguish a false positive, and the tracktor can do this. But the tracktor cannot track everything.

So you mean the tracktor is another sort of SOT and the contribution is the work proposes such a new 'SOT' in MOT?

[kaikaizhu]

I'm sorry that there is a problem with my network here. I can't connect to Google drive. Can you send the required weight files to my email? Thank you very much. My email is [email protected].

[XiongDei]

Dear author,
I also encounter the download problem. No matter I download via browser or use command wget, the download will be interrupted with a 'Service Unavailable' error as shown in images.

[timmeinhardt]

@XiongDei @kaikaizhu I just reuploaded the model weights in a new zip file. I hope this download works better.

@Helicopt Of course there are others trackers that tackle the MOT problem and some of them apply SOT to the multi-object problem. But you claimed that Tracktor does not improve on SDP detections compared to no Tracktor. But without Tracktor there is no tracking so only detections which is definitely worse than SDP + Tracktor. However, I agree that there are other trackers that might be better on SDP. But that was not the point of our paper.
Consider Tracktor as an extension of the tracking-by-detection paradigm which already tackles most easy tracking scenarios and then you can put MOTDT or other trackers on top.

[Helicopt]

I think using FRCNN as tracktor with SDP, will be worse than SDP with no tracking. The reason is that FRCNN has a worse recall and precision, so how can it make SDP better? Or I hope there is experiment to prove it. But i cannot find relative in your paper. Actually I have make a try, and applying tracktor on my SDP input will lead to worse result(than no tracking). Perhaps it is my fault, because I am not so familiar with your code. If you can release further experiment result, it will be great.

Besides, I still confused about whether Tracktor is more like a private detector. You can say it isn't. But there is not strict definition. What about using extended public proposals? you use public detections as proposals, but if I use two times of the proposal area and find the best in it ? If the area continues to expand, finally we get best ones in the whole image, then there is no difference with private detector.

[timmeinhardt]

SDP with no tracking will not tackle the tracking problem at all. You will only have a lot of tracks with length 1 because without tracking the entire data association step is missing. However, Tracktor applies regression not just to refine BBs in a single frame but to generate tracks from frame to frame, i.e., doing the data association. Without providing quantitative results Tracktor FRCNN + SDP must be better than just SDP in terms of tracking. Please revisit the paper to check your understanding of Tracktor.

I agree with your second point of the difference between public and private detections and if you add more proposals to the public detections it becomes private. However, we are only reclassifying (and regressing) the given proposals with our detector. This is something that other trackers have done before.

[Helicopt]

The improvement of the tracktor is not from data association, it does not provide better association results. The mainly improvement is from less FN and FP. Of course we have hundreds of methods to do association without tracktor. I mean SDP with FRCNN tracktor is not better than SDP with a basic association, e.g. deepSORT.
That is exactly what I mean. I think I fully understand what is the tracktor in your paper.

Besides, such things like a tracktor is not new at all. We all know it but we think such method is not valid. Any SOT in MOT method is doing so. But we won't include the detection checkpoint and algorithm in public MOT at all.

[Helicopt]

For SOT + SDP, it is better than SDP + basic association, but for Tracktor + SDP, there is not evidence that it is better than SDP + basic association.

[timmeinhardt]

In your previous post you compared Tracktor + SDP with SDP + no tracking and in that scenario Tracktor + SDP is of course better. If you compare it with SDP + basic association it probably very much depends on the basic association. Simple Hungarian or DeepSort are possible associations and yes currently we do not provide explicit comparisons with those. Different basic associations have possible different merits and might excel in different scenarios.

Tracktor is just another type of basic association which works surprisingly well and relies solely on the object detector which is part of the tracking-by-detection pipeline anyway. In addition, the Tracktor regression data association is differentiable which opens up a lot of different research directions and it is more robust to missing detections.

The main argument of our paper is not to discredit existing tracking solutions but to motivate the exploitation of the object detector to its fullest by solving easy tracking scenarios with regression and extending Tracktor with tracking specific methods.

[Helicopt]

I find the best solution to prove the tracktor is working. You may use a DPM tracktor along with DPM public detection, FRCNN tracktor with FRCNN public detection and SDP tracktor with SDP input. Then if your result is still "surprisingly good" and reach "state of the art", that proves it is really working.

If you use FRCNN for all three, I believe it not fair to compare with other results, or, in other word, lots of existing works can go beyond the tracktor17 by simply replacing all detections input by your FPN refined proposals. In this situation, as you just said, all these replacements are based on public detection proposals, is not "as private". So your idea actually is not new and not leading the community to better direction. Actually it is not something surprising nor constructive. For such replacement, a lot of researchers know it serveral years ago, but did not try it because they think it is not valid. But now it is in proceeding of ICCV.

[Helicopt]

Anyway, it is just a discussion. I hope to see better development for the comunity. I just want to prove tracktor is working well and apply it to my own work. But after serveral try, I find it is not working well on SDP. So I came here to discuss with you.