some snakemake principles -

[ctb]

• if you didn’t create the file, snakemake did; you can safely remove it and snakemake will recreate it!

• if running with --use-conda, don’t run snakemake/genome-grist/charcoal/sgc in a new directory every time; snakemake puts the conda environment stuff under .snakemake/conda in the current working directory, and this will be recreated every time you start in a new directory.

• -k is a good default - it will run all steps possible, before exiting with errors. This prevents a few errors from stopping a multi-day run!

• if after running with snakemake -k ... you get an error exit and you want to find the error message, rerun with -j 1 -k and you will see the error message much more clearly!

• TODO: discuss how -j and memory-intensive steps interact, and how you can specify available (memory) resources on command line.

• note that genome-grist, charcoal, and spacegraphcats all are built around snakemake and accept any valid snakemake argument on the command line.

[ctb]

Mostafa Shokrof

Add resources tag to the jobs with the expected memory and cpu and time. It makes it easier when using cluster

Mostafa Shokrof

Also, separate the input file paths from the script so it will easier to run on different datasets

[SichongP]

if running with --use-conda, don’t run snakemake/genome-grist/charcoal/sgc in a new directory every time; snakemake puts the conda environment stuff under .snakemake/conda in the current working directory, and this will be recreated every time you start in a new directory.

Also --conda-prefix allows you to set a shared conda directory for all snakemake projects. Any rule with same env.yaml files can then re-use prebuilt conda envs

if you didn’t create the file, snakemake did; you can safely remove it and snakemake will recreate it!

Not the .snakemake directory. It's where all run-related metadata are kept :)

[ctb]

if you didn’t create the file, snakemake did; you can safely remove it and snakemake will recreate it!

Not the .snakemake directory. It's where all run-related metadata are kept :)

(well, you can remove it but snakemake might rerun stuff that you'd prefer not to be rerun ;)