retrain original wights.h5

[ClemoMUL]

Hello everyone,

I am currently working on applying your tracking model to a digit sensor. During this process, I tried to retrain the original weights (weights.h5) with the original train.py file, and I started the example_tracking_sim.py with the newly obtained weights. However, when I started example_tracking_sim.py with the newly obtained weights, the results were quite different from when I started the simulation with the weights.h5 file. This was quite surprising as I was expecting the original weights.h5 file to have been trained with the original train.py file.
Has anyone tried the same thing and gotten weights that give the same/similar result as the original weights?

The picture simulation_original weights.png shows the example_tracking_sim.py with the original weights.
simulation_trained weights_B1.png shows the example_tracking_sim.py with the weights obtained withthe train.py file.

An interesting observation is that - in the case of weights trained on my own - the distortions of the points are registered with the arrows on the wrong side of the field -> wenn distorting the points on the bottom left side, the arrows on the top rights side are distorted. When distorting the points on the top right side the arrows on the bottom left side are distorted. (simulation_trained weights_B2.png and Simulation_trained weights_B3.png).

As the simulation works with the original weights my conclusion was that an error might be in the train.py file. As a consequence I visualised the training data -> however as it seems the pictures with the markers seem to be generated correctly, as well as the distortion of the markers is generated correctly (training_B1.jpg and training_B2.jpg).

The file "training_B1.jpg" shows the image with the original marker positions and the distortions into which the markers are distorted (displayed in training_B2.jpg).

Best regards,
Clemens

Simulation_original weights.png:

simulation_trained weights_B1.png:

Simulation_trained weights_B2.png:

Simulation_trained weights_B3.png:

training_B1.jpg:

training_B2.jpg:

[Zhanggengming-china]

I'm also experiencing this problem, have you solved it yet?

[wx405557858]

Hi @ClemoMUL @Zhanggengming-china , it looks like a image scale visualization issue, can you try when visualize the marker displacement, multiply each (x, y) by (H, W) and see whether that resolved it?

[JunJie-zhang-o]

Hi @ClemoMUL @Zhanggengming-china , it looks like a image scale visualization issue, can you try when visualize the marker displacement, multiply each (x, y) by (H, W) and see whether that resolved it?嗨，它看起来像一个图像刻度可视化问题，当可视化标记位移，将每个（x，y）乘以（h，w）时，您可以尝试一下吗？

I think this is not a visualization problem, but the difference in the Y label in train.py.

Because the original weight.h5 should directly output the absolute position of the marker point when simulating. The Y label in train.py is a relative position.

If I change the Y label in the train.py file to an absolute position, I can get the class effect. But this raises a new question, does the initial image of X input during training still make sense?

In addition, I also found that the definition of the initial marker point in train.py and generate_data.py is different

I think the idea of ​​this project is very good, and I found many interesting techniques in it. At the same time, I also have some questions mentioned above, and I hope to get the author's reply.