Source code for the paper Do Deep Neural Network Solutions form a Star Domain?
-
Clone this repository:
$ git clone https://github.com/aktsonthalia/starlight --recursive
-
Create a new conda environment and install the requirements:
$ conda create -n starlight python=3.9 $ conda activate starlight $ pip install -r requirements.txt
- Set up your WandB account. Note down the
entity
andproject
values for use in the experiments.
- You can download pretrained models as zip files. Once they have been downloaded, extract them.
- Run the script
model_paths/create_model_paths.py
:$ cd model_paths/ $ python create_model_paths.py -f <PATH_TO_EXTRACTED_MODELS> $ cd ..
- You will find config files in
configs/
. Open the configuration file for the experiment you wish to run (names are self-explanatory).cifar10_resnet18_star.yaml
cifar100_resnet18_star.yaml
cifar10_densenet_star.yaml
cifar100_densenet_star.yaml
- Change the wandb
entity
andproject
values. - Go to
dataloaders/constants.py
and change any dataset paths that you might need to. Otherwise, default values will be used.
- Run the training script:
$ python main.py <PATH_TO_CONFIG_FILE>
You can plot the loss barrier between two given models using:
$ cd postprocessing
$ python plot_barriers.py -c <CONFIG_FILE from configs/> -a <PATH_TO_MODEL_A_CHECKPOINT> -b <PATH_TO_MODEL_B_CHECKPOINT>
-
Thanks to my collaborator, Alexander Rubinstein for the STAI-tuned tool that made running the experiments so much easier. If you're interested, you can run more extensive experiments using this tool, too. Follow this README.
-
Thanks to Arnas Uselis for testing the code prior to release.
-
Thanks to the ML Cloud at Universität Tübingen for providing and maintaining the compute resources that made the experiments possible.
-
Thanks to WandB for their academic researchers plan.
-
Thanks to the following open-source repositories, code from which was used in this one.
Feel free to open an issue, if you find a bug, or would like to see a new feature.
Check the issues for planned improvements. If you wish, you may also open pull requests addressing them.
If you find this work useful, please consider citing:
@misc{sonthalia2024deep,
title={Do Deep Neural Network Solutions Form a Star Domain?},
author={Ankit Sonthalia and Alexander Rubinstein and Ehsan Abbasnejad and Seong Joon Oh},
year={2024},
eprint={2403.07968},
archivePrefix={arXiv},
primaryClass={cs.LG}
}