Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update diff subworkflows #7414

Open
wants to merge 15 commits into
base: master
Choose a base branch
from
Open

Update diff subworkflows #7414

wants to merge 15 commits into from

Conversation

suzannejin
Copy link
Contributor

@suzannejin suzannejin commented Jan 31, 2025

Some small changes to coordinate them with differentialabundance pipeline:

PR checklist

Closes #XXX

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the module conventions in the contribution docs
  • If necessary, include test data in your PR.
  • Remove all TODO statements.
  • Emit the versions.yml file.
  • Follow the naming conventions.
  • Follow the parameters requirements.
  • Follow the input/output options guidelines.
  • Add a resource label
  • Use BioConda and BioContainers if possible to fulfil software requirements.
  • Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
    • For modules:
      • nf-core modules test <MODULE> --profile docker
      • nf-core modules test <MODULE> --profile singularity
      • nf-core modules test <MODULE> --profile conda
    • For subworkflows:
      • nf-core subworkflows test <SUBWORKFLOW> --profile docker
      • nf-core subworkflows test <SUBWORKFLOW> --profile singularity
      • nf-core subworkflows test <SUBWORKFLOW> --profile conda

@suzannejin
Copy link
Contributor Author

Hey there @pinin4fjords, while updating the subworkflows, I just realised that the snapshots for gprofiler2 don't work anymore. I initially thought it might be related to the update of their database today, but I tried to set to an old archive, and same. Any thoughts?

@pinin4fjords
Copy link
Member

  • Remove mergeMaps. Otherwise the file format of ${contrast_id}_${exp_id} lead to problems with the Rmarkdown rendering.

Could you clarify that please? The mergeMaps was done for a good reason, it might be better to fix the markdown logic.

@suzannejin
Copy link
Contributor Author

suzannejin commented Feb 4, 2025

  • Remove mergeMaps. Otherwise the file format of ${contrast_id}_${exp_id} lead to problems with the Rmarkdown rendering.

Could you clarify that please? The mergeMaps was done for a good reason, it might be better to fix the markdown logic.

Could you explain why do you prefer to use mergeMaps?

@pinin4fjords
Copy link
Member

  • Remove mergeMaps. Otherwise the file format of ${contrast_id}_${exp_id} lead to problems with the Rmarkdown rendering.

Could you clarify that please? The mergeMaps was done for a good reason, it might be better to fix the markdown logic.

Could you explain why do you prefer to use mergeMaps?

Yep, sure. If you look at the function:

def mergeMaps(meta, meta2){
    (meta + meta2).collectEntries { k, v ->
        meta[k] && meta[k] != v ? [k, "${meta[k]}_${v}"] : [k, v]
    }
}

It's designed such that non-identical values of common keys are concatenated. So e.g. if you merged maps like [id: 'matrix_1', foo: 'one'] and [id: 'contrast_1', bar: 'two'], you'd get [id: 'matrix_1_contrast_1', foo: 'one', bar: 'two'].

As such it's not the same as the simple map addition you're proposing, which would be lossy, and simply over-write common keys with values from the second map. That was not the intention when I wrote this.

@suzannejin
Copy link
Contributor Author

  • Remove mergeMaps. Otherwise the file format of ${contrast_id}_${exp_id} lead to problems with the Rmarkdown rendering.

Could you clarify that please? The mergeMaps was done for a good reason, it might be better to fix the markdown logic.

I reintroduced the mergeMaps, and fixed the markdown logic, which basically was defining the differential and report files based on contrast id only

@suzannejin
Copy link
Contributor Author

Hey there @pinin4fjords, while updating the subworkflows, I just realised that the snapshots for gprofiler2 don't work anymore. I initially thought it might be related to the update of their database today, but I tried to set to an old archive, and same. Any thoughts?

Never mind :) it is actually the problem, and consistent snapshots with the previous one were obtained.

@suzannejin
Copy link
Contributor Author

suzannejin commented Feb 7, 2025

Some issues with gprofiler2/gost module -> #7451

Running gprofiler2::gost on archived database seems a way to keep reproducibility, but given issues 2 and 4, this option seems undesirable for testing purposes.

Alternatively, one can upload the achived gmt through 'upload_GMT_file' and run tests on that. However, I encountered with the issue 1. This issue might relate to how the original package gprofiler2::gost treat custom gmt vs their database. I am waiting for the authors of gprofiler2 package to confirm about this... If this is true, then we should treat them as two different "methods".

In the meanwhile, I updated the snapshots to match the new database.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants