Gridss environment files / port to nf-test

[famosab]

PR checklist

Closes #5508

• This comment contains a description of changes (with reason).
• If you've fixed a bug or added code that should be tested, add tests!
• If you've added a new tool - have you followed the module conventions in the contribution docs
• If necessary, include test data in your PR.
• Remove all TODO statements.
• Emit the versions.yml file.
• Follow the naming conventions.
• Follow the parameters requirements.
• Follow the input/output options guidelines.
• Add a resource label
• Use BioConda and BioContainers if possible to fulfil software requirements.
• Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
  • For modules:
    • nf-core modules test <MODULE> --profile docker
    • nf-core modules test <MODULE> --profile singularity
    • nf-core modules test <MODULE> --profile conda
  • For subworkflows:
    • nf-core subworkflows test <SUBWORKFLOW> --profile docker
    • nf-core subworkflows test <SUBWORKFLOW> --profile singularity
    • nf-core subworkflows test <SUBWORKFLOW> --profile conda

[famosab]

Changes need to be finished after #5933 is merged since that influences the setup section in main.nf.test.

[famosab]

@SPPearce Somehow the versions are not accessed correctly with conda - any idea why?

[SPPearce]

It is failing with an error message before it gets that far

[SPPearce]

https://github.com/nf-core/modules/actions/runs/10634458696/job/29481867188?pr=5932 :

Test Process GRIDSS_SOMATICFILTER

  Test [40d75c38] 'human - vcf' 
    > N E X T F L O W  ~  version 24.04.4
    > Launching `/opt/actions-runner/_work/modules/modules/.nf-test-40d75c3865392257efc32c38a8ae25a2.nf` [hopeful_sammet] DSL2 - revision: 52fa2580f4
    > Creating env using conda: /opt/actions-runner/_work/modules/modules/modules/nf-core/gridss/somaticfilter/tests/../../../bwa/index/environment.yml [cache /home/ubuntu/tests/40d75c3865392257efc32c38a8ae25a2/work/conda/bwa_index-e04957aeeb5615827c[51](https://github.com/nf-core/modules/actions/runs/10634458696/job/29481867188?pr=5932#step:14:52)54[52](https://github.com/nf-core/modules/actions/runs/10634458696/job/29481867188?pr=5932#step:14:53)1bb90048]
    > [6a/7c4040] Submitted process > BWA_INDEX (genome.fasta)
    > WARN: Input tuple does not match tuple declaration in process `GRIDSS_GRIDSS` -- offending value: [[id:test], /nf-core/test-datasets/modules/data/genomics/homo_sapiens/illumina/bam/test.paired_end.sorted.bam, []]
    > Creating env using conda: /opt/actions-runner/_work/modules/modules/modules/nf-core/gridss/somaticfilter/tests/../../../gridss/gridss/environment.yml [cache /home/ubuntu/tests/40d75c386[53](https://github.com/nf-core/modules/actions/runs/10634458696/job/29481867188?pr=5932#step:14:54)92257efc32c38a8ae25a2/work/conda/gridss_gridss-716d62033d0cd1f43531d3f9d4d204fe]
    > [6f/83bdfb] Submitted process > GRIDSS_GRIDSS (test)
    > Creating env using conda: /opt/actions-runner/_work/modules/modules/modules/nf-core/gridss/somaticfilter/tests/../environment.yml [cache /home/ubuntu/tests/40d75c3865392257efc32c38a8ae25a2/work/conda/gridss_somaticfilter-179f560e4ee1afe2a0c14c43fb3ca9dc]
    > [f2/21a256] Submitted process > GRIDSS_SOMATICFILTER (test)
    > ERROR ~ Error executing process > 'GRIDSS_SOMATICFILTER (test)'
    > 
    > Caused by:
    >   Process `GRIDSS_SOMATICFILTER (test)` terminated with an error exit status (1)
    > 
    > 
    > Command executed:
    > 
    >   gridss_somatic_filter \
    >       --input test.vcf.gz \
    >        \
    >       --output test.high_confidence_somatic.vcf \
    >       --fulloutput test.all_somatic.vcf \
    >       --normalordinal 0
    >   
    >   cat <<-END_VERSIONS > versions.yml
    >   "GRIDSS_SOMATICFILTER":
    >       gridss: 2.13.2
    >   END_VERSIONS
    > 
    > Command exit status:
    >   1
    > 
    > Command output:
    >   Test passed 🌈
    >   Test passed 🥇
    > 
    > Command error:
    >   Attaching package: ‘Matrix’
    >   
    >   The following object is masked from ‘package:S4Vectors’:
    >   
    >       expand
    >   
    >   The following objects are masked from ‘package:tidyr’:
    >   
    >       expand, pack, unpack
    >   
    >   
    >   Attaching package: ‘MatrixGenerics’
    >   
    >   The following objects are masked from ‘package:matrixStats’:
    >   
    >       colAlls, colAnyNAs, colAnys, colAvgsPerRowSet, colCollapse,
    >       colCounts, colCummaxs, colCummins, colCumprods, colCumsums,
    >       colDiffs, colIQRDiffs, colIQRs, colLogSumExps, colMadDiffs,
    >       colMads, colMaxs, colMeans2, colMedians, colMins, colOrderStats,
    >       colProds, colQuantiles, colRanges, colRanks, colSdDiffs, colSds,
    >       colSums2, colTabulates, colVarDiffs, colVars, colWeightedMads,
    >       colWeightedMeans, colWeightedMedians, colWeightedSds,
    >       colWeightedVars, rowAlls, rowAnyNAs, rowAnys, rowAvgsPerColSet,
    >       rowCollapse, rowCounts, rowCummaxs, rowCummins, rowCumprods,
    >       rowCumsums, rowDiffs, rowIQRDiffs, rowIQRs, rowLogSumExps,
    >       rowMadDiffs, rowMads, rowMaxs, rowMeans2, rowMedians, rowMins,
    >       rowOrderStats, rowProds, rowQuantiles, rowRanges, rowRanks,
    >       rowSdDiffs, rowSds, rowSums2, rowTabulates, rowVarDiffs, rowVars,
    >       rowWeightedMads, rowWeightedMeans, rowWeightedMedians,
    >       rowWeightedSds, rowWeightedVars
    >   
    >   
    >   Attaching package: ‘S4Arrays’
    >   
    >   The following object is masked from ‘package:abind’:
    >   
    >       abind
    >   
    >   The following object is masked from ‘package:base’:
    >   
    >       rowsum
    >   
    >   Test passed 🌈
    >   Test passed 🥇
    >   No reference genome supplied using --ref. Not performing variant equivalence checks.
    >   2024-08-30 14:52:[55](https://github.com/nf-core/modules/actions/runs/10634458696/job/29481867188?pr=5932#step:14:56).635999 Reading test.vcf.gz
    >   Tumour samples: 
    >   Matched normals: 
    >   Error: array-like subscript has more than one effective dimension
    >   Execution halted
    >   

worth noting it is complaining that the tuple doesn't match at the top:
> WARN: Input tuple does not match tuple declaration in processGRIDSS_GRIDSS -- offending value: [[id:test], /nf-core/test-datasets/modules/data/genomics/homo_sapiens/illumina/bam/test.paired_end.sorted.bam, []]

[famosab]

I am also confused by the error message. I tried running it locally but that did not work because of the M1 chip and then I did not have any more time to configure that the run with conda works.

[SPPearce]

I just use gitpod for all my development work, has all three container engines available

[SPPearce]

Ok, there is still a fundamental issue here.
Conda fails on GRIDSS/SOMATICFILTER, with an unhelpful error.
I think this is because the input is empty, because the input is currently the output of GRIDSS on a bam file that doesn't contain any SVs.
The solution is either:

• Use better test-data, but I can't immediately see any in the GRIDSS github repo.
• Only use a stub
• Dig in and figure out what is different between the conda and docker images.
• Use a seqera container (although that may break the docker/singularity test instead potentially ;) )

Also I've adjusted the output channels to only use meta, not meta and meta2, which I think is correct but could do with verification.

[famosab]

Maybe we can go for using the stub and write a small warning into the module? Because judging from the last release the tool does not seem to be maintained? And if they ever get back to me then we can activate the test again?

[SPPearce]

I've also tried an actual GRIDSS vcf, that failed with a different error.
I've tried generating a new seqera container, that causes the docker test to fail as well.

[SPPearce]

I think the bioconda recipe just doesn't currently work

[SPPearce]

I have added a workaround in the gridss/somaticfilter code itself, which appears to be working.
I also added (part of) a proper gridss vcf file to the test-datasets repo to use.