Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Loss is NaN #117

Open
Qiuxingzl opened this issue Oct 8, 2022 · 1 comment
Open

Loss is NaN #117

Qiuxingzl opened this issue Oct 8, 2022 · 1 comment

Comments

@Qiuxingzl
Copy link

Qiuxingzl commented Oct 8, 2022 •
edited
Loading

Ubuntu20.04/Python2.7/Tensorflow1.15

Run script train_Sony.py, training starts but from the log it seems the Loss is not correct? Does anyone konw the issue?

333 154 Loss=nan Time=0.040
333 155 Loss=nan Time=0.041
333 156 Loss=nan Time=0.043
333 157 Loss=nan Time=0.036
333 158 Loss=nan Time=0.042
333 159 Loss=nan Time=0.040
333 160 Loss=nan Time=0.036
333 161 Loss=nan Time=0.042
334 1 Loss=nan Time=0.044
334 2 Loss=nan Time=0.042
334 3 Loss=nan Time=0.036
@Qiuxingzl Qiuxingzl changed the title Loss is nan Loss is NaN Oct 8, 2022
@AndrewKeYanzhe
Copy link

Hi @Qiuxingzl , have you found out whether NaN loss is the expected output?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Qiuxingzl @AndrewKeYanzhe