Sfitz update call sSNV to 8.0.0-rc.1

[sorelfitzgibbon]

• I have read the code review guidelines and the code review best practice on GitHub check-list.

• The name of the branch is meaningful and well formatted following the standards, using [AD_username (or 5 letters of AD if AD is too long)-[brief_description_of_branch].

• I have set up or verified the branch protection rule following the github standards before opening this pull request.

• I have added my name to the contributors listings in the
  metadata.yaml and the manifest block in the nextflow.config as part of this pull request, am listed
  already, or do not wish to be listed. (This acknowledgement is optional.)

• I have added the changes included in this pull request to the CHANGELOG.md under the next release version or unreleased, and updated the date.

• [x]] I have updated the version number in the metadata.yaml and manifest block of the nextflow.config file following semver, or the version number has already been updated. (Leave it unchecked if you are unsure about new version number and discuss it with the infrastructure team in this PR.)

• I have tested the pipeline on at least one A-mini sample.

• Update to use call-sSNV v8.0.0-rc.1, including removing id from the call-sSNV input yamls.

• nftest expected files were update to new versions after careful inspection to ensure no variant changes. This was to deal with sample order changes.

• nftest.yaml paths were updated to use wild cards for the actual file paths and to reflect the new sample ID (extracted from BAMs) in the expected file paths.

Strangely!!!, a new variant showed up for MuSE in one of the test runs.
MuSE-2.0.4_LUAD0000034_CPT0053040010_SNV.vcf.gz:
chr21 8033328 . G T . PASS SOMATIC GT:DP:AD:BQ:SS 0/1:29:22,7:29,30:2 0/0:12:12,0:28,0:.
I kept the run in the output directory, under extra-MuSE-variant-chr21-8033328

Closes #123

Testing Results

nftest run test-call-sSNV
output dir: /hot/software/pipeline/metapipeline-DNA/Nextflow/development/unreleased/sfitz-update-call-sSNV
log: /hot/software/pipeline/metapipeline-DNA/Nextflow/development/unreleased/sfitz-update-call-sSNV/log-nftest-20231221T222740Z.log

[sorelfitzgibbon]

Testing completed after merging in main. Test results updated with new log. Note the weird MuSE variant described above.

[yashpatel6]

Looks good! Any concerns @zhuchcn @nwiltsie @tyamaguchi-ucla ?

[yashpatel6]

The new variant is interesting, the only MuSE change is from 2.0.3 to 2.0.4; this sample is different from the one used for testing in call-sSNV so we didn't see it there?

[nwiltsie]

The new variant is interesting, the only MuSE change is from 2.0.3 to 2.0.4;

There's also virtually no change between those two versions: wwylab/MuSE@v2.0.3...v2.0.4 ... very weird.

[yashpatel6]

Also, the variant only showed up in one of the test runs?

[sorelfitzgibbon]

Also, the variant only showed up in one of the test runs?

I think so, although I can't say for sure. NFtest assertions weren't working for much of the testing and I dutifully deleted all the failed runs for that day, on that day.

[zhuchcn]

Don't have any concern, also no idea about the new variant! Does MuSE use any random process so the output isn't always reproducible (I doubt it does)?

[sorelfitzgibbon]

It sure looks stochastic! One run had the extra variant, the next run did not. I concatenated all of the log.command.sh for the MuSE processes and found no difference between the two runs. I launched with nftest run test-call-sSNV both times.

[nwiltsie]

It sure looks stochastic! One run had the extra variant, the next run did not.

Oh I do not like that! Would you mind posting the output paths for those two runs?

[sorelfitzgibbon]

/hot/software/pipeline/metapipeline-DNA/Nextflow/development/unreleased/sfitz-update-call-sSNV/test-call-sSNV
/hot/software/pipeline/metapipeline-DNA/Nextflow/development/unreleased/sfitz-update-call-sSNV/extra-MuSE-variant-chr21-8033328/test-call-sSNV

[yashpatel6]

I ran the test just now and the output did not contain the new variant: /hot/software/pipeline/metapipeline-DNA/Nextflow/development/unreleased/yashpatel-test-call-ssnv-update 🤔

[tyamaguchi-ucla]

If there's stochasticity in the algorithm, we can suggest that our collaborator (Shuangxi) implement an option to set a seed.

[yashpatel6]

Agreed, it would be worth confirming with Shuangxi whether stochasticity is expected in MuSE and then request a seeding option if there is

[tyamaguchi-ucla]

Looks generally good to me!
I'm not sure if there's been any discussion about updating the final output structure of Intersect-BCFtools-1.17, but I think organizing it a bit more would be helpful for end users. For instance, should we create a new directory for storing figures, or can they be placed under /QC, considering the increasing number of pipelines generating figures?

[yashpatel6]

Intersect-BCFtools-1.17, but I think organizing it a bit more would be helpful for end users. For instance, should we create a new directory for storing figures, or can they be placed under /QC, considering the increasing number of pipelines generating figures?

We've had a little bit of discussion about what should be in the QC directory and what shouldn't but not fully formalized yet; we'll formalize it in the next NF WG

[yashpatel6]

I confirmed that the commands bring run are the same between the two tests so if no other concerns from @nwiltsie, we can merge this and follow up with Shuangxi about stochasticity?

[nwiltsie]

I confirmed that the commands bring run are the same between the two tests so if no other concerns from @nwiltsie, we can merge this and follow up with Shuangxi about stochasticity?

Sounds good to me!

[sorelfitzgibbon]

OK merging! I have sent an email to Shuangxi.

[tyamaguchi-ucla]

OK merging! I have sent an email to Shuangxi.

Also, maybe consider creating a GitHub issue in the MuSE repo?