NASBoost

This repository contains the official code for the following paper:

FILL HERE

We fork the implementations of naszilla repository, which includes the official implementation of the following papers:

Paper	Blog Post
A Study on Encodings for Neural Architecture Search	Blog Post
BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search	Blog Post
Exploring the Loss Landscape in Neural Architecture Search	Blog Post

Installation

Clone this repository and install its requirements (which includes nasbench, nas-bench-201, and nasbench301). It may take a few minutes.

git clone https://github.com/dani3l125/NASBoost
cd NASBoost
cat requirements.txt | xargs -n 1 -L 1 pip install
pip install -e .

Next, download the nas benchmark dataset we use (either with the terminal command below, or from their website) nas-bench-201.

# these files are 0.5GB, 2.1GB, and 1.6GB, respectively
wget https://ndownloader.figshare.com/files/25506206?private_link=7d47bf57803227af4909 -O NAS-Bench-201-v1_0-e61699.pth

Place the downloaded benchmark data file in ~/nas_benchmark_datasets (or choose another directory and edit line 15 of naszilla/nas_benchmarks.py accordingly).

Now you have successfully installed all the requirements to reproduce our experiments and visualizations.

Running experiments

To reproduce the results we present, define a config.yaml file with the parameters that matches the desired experiment. For example, the following commands will reproduce the results for random search given the initial configuration:

cd naszilla
python naszilla/run_experiments.py --search_space nasbench_201 --algo_params random --queries 300 --trials 10 --k_alg 1 --cfg
'naszilla/knas_config.yaml' --dataset cifar10
python naszilla/run_experiments.py --search_space nasbench_201 --algo_params random --queries 300 --trials 10 --k_alg 1 --cfg
'naszilla/knas_config.yaml' --dataset cifar100
python naszilla/run_experiments.py --search_space nasbench_201 --algo_params random --queries 300 --trials 10 --k_alg 1 --cfg
'naszilla/knas_config.yaml' --dataset ImageNet16-120

These experiments should finish running within a few minutes.

Run Visualization

To reproduce the study visualizations we reported, run the script visualize.py. You may want to change the zoom in CLUSTERS constant.

Citation

Please cite our paper if you use code from this repo:

FILL HERE