Difference between Logger.save_dir, Logger.root_dir, and Logger.log_dir

[daniellepintz]

I find these three properties confusing - would appreciate if someone could clarify them for me.

I noticed that save_dir is the only one of the three on the Base API. root_dir and log_dir are only present in TensorboardLogger and CSVLogger

https://github.com/PyTorchLightning/pytorch-lightning/blob/8c07d8bf905e395bbd2142b5df7b185b8e936c41/pytorch_lightning/loggers/tensorboard.py#L112-L146

https://github.com/PyTorchLightning/pytorch-lightning/blob/8c07d8bf905e395bbd2142b5df7b185b8e936c41/pytorch_lightning/loggers/csv_logs.py#L143-L173

@awaelchli @edward-io @Borda @ananthsub @rohitgr7 @ethanwharris

[awaelchli]

Hello
Good question.

I will explain from the top of my head and someone will remind me if a detail is wrong :)

Example path:

/home/danielle/lightning_logs/version_22

• /home/danielle/: This is the working directory, known as default_root_dir in Trainer.
• lightning_logs: This is the name of the logging project, referred to self.name in some loggers
• version_22: The version of the experiment, self.version in some loggers

Together:

• logger.save_dir = /home/danielle/. In case of TB, this gets set to trainer.default_root_dir (because TB is the default logger)
• logger.root_dir = /home/danielle/lightning_logs. This is the combination of save_dir and name.
• logger.log_dir = /home/danielle/lightning_logs/version_22. This is the whole path, where logs eventually will end up in. This is also the same as trainer.log_dir.

I tried to standardize the log dir properties a while ago and miserably failed because things break easily and it is a nightmare to have backward compatibility. Can't say whether this situation has improved, but I expect not.

[rohitgr7]

I was thinking of a direction where we can have a consistent property across loggers, in case we choose to remove others.

[daniellepintz]

Can you expand a bit, what would this property be?

[rohitgr7]

I am suggesting instead of having different names which tell where the logger logs exactly for the experiment, it should have just one. We have log_dir for tensorboard, for some it's save_dir.

[daniellepintz]

@rohitgr7 I completely agree!

[mcorroyer]

Hi @awaelchli @rohitgr7, I think all loggers should indeed have the same property semantics. It's especially troublesome when using the ModelCheckpoint callback with a custom logger because the Trainer will use logger.save_dir as its log directory but the ModelCheckpoint will use logger.save_dir/logger.name/logger.version. If it's hard to standardize the log dir properties, Trainer.log_dir and ModelCheckpoint.dirpath should at least be defined using the same logic.