Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test count #141

Merged
merged 29 commits into from
May 29, 2024
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
97ffcd5
Add test for count table
LaurenceKuhl May 24, 2024
2558ab9
Fix path
LaurenceKuhl May 24, 2024
48b834d
Revert to the old nomenclature for now
LaurenceKuhl May 24, 2024
01b7057
Temporarily use my path until i fix the github issue
LaurenceKuhl May 24, 2024
bd49779
Current tries to make the input non mandatory
LaurenceKuhl May 24, 2024
481a0bb
Fix curly brackets
LaurenceKuhl May 24, 2024
43bd503
Fix curly brackets
LaurenceKuhl May 24, 2024
a065a28
fix curly brackets..
LaurenceKuhl May 24, 2024
e9acfb1
Trying to add empty channels
LaurenceKuhl May 24, 2024
676a940
Fix input count file
LaurenceKuhl May 24, 2024
b2844dc
revert back download_pipeline to fix linting issue
LaurenceKuhl May 24, 2024
ba2b4c3
Fix linting
LaurenceKuhl May 24, 2024
df969dc
Fixl inting
LaurenceKuhl May 24, 2024
31de28a
Merge branch 'dev' into test_count
LaurenceKuhl May 27, 2024
59fd55f
Update conf/test_screening_count_table.config
LaurenceKuhl May 27, 2024
fd9e43c
Update conf/test_screening_count_table.config
LaurenceKuhl May 27, 2024
bb48406
Update conf/test_screening_count_table.config
LaurenceKuhl May 27, 2024
7248dda
Update conf/test_screening_count_table.config
LaurenceKuhl May 27, 2024
1c88a22
Update subworkflows/local/utils_nfcore_crisprseq_pipeline/main.nf
LaurenceKuhl May 27, 2024
e39eb47
Remove if statement
LaurenceKuhl May 28, 2024
6d228d4
Fix link
LaurenceKuhl May 28, 2024
148010f
Fix the count test fail
LaurenceKuhl May 28, 2024
429d2f8
Merge branch 'dev' into test_count
LaurenceKuhl May 28, 2024
bfe2142
Fix typos
LaurenceKuhl May 28, 2024
4be630d
Merge branch 'test_count' of https://github.com/LaurenceKuhl/crisprse…
LaurenceKuhl May 28, 2024
387fb9b
Ran pre commit
LaurenceKuhl May 28, 2024
e7935bd
Update download pipeline
LaurenceKuhl May 28, 2024
155c62e
Merge branch 'dev' of https://github.com/nf-core/crisprseq into test_…
mirpedrol May 28, 2024
af1cbe4
move validation of input channels to the main if block
mirpedrol May 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ jobs:
- "test_screening_rra"
- "test_targeted"
- "test_umis"
- "test_screening_count_table"

steps:
- name: Check out pipeline code
Expand Down
9 changes: 3 additions & 6 deletions .github/workflows/download_pipeline.yml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the changes on this file come from the template update, the previous versions are the correct ones, the most recent.

Original file line number Diff line number Diff line change
Expand Up @@ -30,14 +30,11 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Install Nextflow
uses: nf-core/setup-nextflow@v2
uses: nf-core/setup-nextflow@v1

- name: Disk space cleanup
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1

- uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5
- uses: actions/setup-python@0a5c61591373683505ea898e09a3ea4f39ef2b9c # v5
with:
python-version: "3.12"
python-version: "3.11"
architecture: "x64"
- uses: eWaterCycle/setup-singularity@931d4e31109e875b13309ae1d07c70ca8fbc8537 # v7
with:
Expand Down
28 changes: 28 additions & 0 deletions conf/test_screening_count_table.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
/*
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Nextflow config file for running minimal tests
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Defines input files and everything required to run a fast and simple pipeline test.

Use as follows:
nextflow run nf-core/crisprseq -profile test_screening,<conda/docker/singularity> --outdir <OUTDIR>
LaurenceKuhl marked this conversation as resolved.
Show resolved Hide resolved

----------------------------------------------------------------------------------------
*/

params {
config_profile_name = 'Test screening profile with an input count table'
config_profile_description = 'Minimal test dataset to check pipeline function'

// Limit resources so that this can run on GitHub Actions
max_cpus = 2
max_memory = '6.GB'
max_time = '6.h'

// Input data
count_table = "https://raw.githubusercontent.com/LaurenceKuhl/test-datasets/crisprseq/testdata/count_table.tsv"
LaurenceKuhl marked this conversation as resolved.
Show resolved Hide resolved
analysis = 'screening'
mle_design_matrix = "https://raw.githubusercontent.com/nf-core/test-datasets/crisprseq/testdata/design_matrix.txt"
LaurenceKuhl marked this conversation as resolved.
Show resolved Hide resolved
contrasts = "https://raw.githubusercontent.com/nf-core/test-datasets/crisprseq/testdata/rra_contrasts.txt"
LaurenceKuhl marked this conversation as resolved.
Show resolved Hide resolved

}
19 changes: 10 additions & 9 deletions nextflow.config
Original file line number Diff line number Diff line change
Expand Up @@ -204,15 +204,16 @@ profiles {
executor.cpus = 4
executor.memory = 8.GB
}
test { includeConfig 'conf/test_targeted.config' }
test_targeted { includeConfig 'conf/test_targeted.config' }
test_full { includeConfig 'conf/test_targeted_full.config' }
test_targeted_full { includeConfig 'conf/test_targeted_full.config' }
test_umis { includeConfig 'conf/test_umis.config' }
test_screening_full { includeConfig 'conf/test_screening_full.config' }
test_screening { includeConfig 'conf/test_screening.config' }
test_screening_paired { includeConfig 'conf/test_screening_paired.config' }
test_screening_rra { includeConfig 'conf/test_screening_rra.config' }
test { includeConfig 'conf/test_targeted.config' }
test_targeted { includeConfig 'conf/test_targeted.config' }
test_full { includeConfig 'conf/test_targeted_full.config' }
test_targeted_full { includeConfig 'conf/test_targeted_full.config' }
test_umis { includeConfig 'conf/test_umis.config' }
test_screening_full { includeConfig 'conf/test_screening_full.config' }
test_screening { includeConfig 'conf/test_screening.config' }
test_screening_paired { includeConfig 'conf/test_screening_paired.config' }
test_screening_rra { includeConfig 'conf/test_screening_rra.config' }
test_screening_count_table { includeConfig 'conf/test_screening_count_table.config' }
}

// Set default registry for Apptainer, Docker, Podman and Singularity independent of -profile
Expand Down
75 changes: 49 additions & 26 deletions subworkflows/local/utils_nfcore_crisprseq_pipeline/main.nf
Original file line number Diff line number Diff line change
Expand Up @@ -77,42 +77,65 @@ workflow PIPELINE_INITIALISATION {
//
validateInputParameters()



reads_targeted = Channel.empty()
reads_screening = Channel.empty()
fastqc_screening = Channel.empty()
reference = Channel.empty()
protospacer = Channel.empty()
template = Channel.empty()
versions = Channel.empty()

//
// Create channel from input file provided through params.input
//
Channel
.fromSamplesheet("input")
.multiMap {
meta, fastq_1, fastq_2, reference, protospacer, template ->
if (fastq_2) {
files = [ fastq_1, fastq_2 ]
} else {
files = [ fastq_1 ]
}
reads_targeted: [ meta.id, meta - meta.subMap('condition') + [ single_end:fastq_2?false:true, self_reference:reference?false:true, template:template?true:false ], files ]
reads_screening:[ meta + [ single_end:fastq_2?false:true ], files ]
reference: [meta - meta.subMap('condition') + [ single_end:fastq_2?false:true, self_reference:reference?false:true, template:template?true:false ], reference]
protospacer: [meta - meta.subMap('condition') + [ single_end:fastq_2?false:true, self_reference:reference?false:true, template:template?true:false ], protospacer]
template: [meta - meta.subMap('condition') + [ single_end:fastq_2?false:true, self_reference:reference?false:true, template:template?true:false ], template]
}
.set { ch_input }
if(params.input) {
Channel
.fromSamplesheet("input")
.multiMap {
meta, fastq_1, fastq_2, reference, protospacer, template ->
if (fastq_2) {
files = [ fastq_1, fastq_2 ]
} else {
files = [ fastq_1 ]
}
reads_targeted: [ meta.id, meta - meta.subMap('condition') + [ single_end:fastq_2?false:true, self_reference:reference?false:true, template:template?true:false ], files ]
reads_screening:[ meta + [ single_end:fastq_2?false:true ], files ]
reference: [meta - meta.subMap('condition') + [ single_end:fastq_2?false:true, self_reference:reference?false:true, template:template?true:false ], reference]
protospacer: [meta - meta.subMap('condition') + [ single_end:fastq_2?false:true, self_reference:reference?false:true, template:template?true:false ], protospacer]
template: [meta - meta.subMap('condition') + [ single_end:fastq_2?false:true, self_reference:reference?false:true, template:template?true:false ], template]
}
.set { ch_input }
fastqc_screening = ch_input.reads_screening
fastqc_screening = ch_input.reads_screening
reference = ch_input.reference
protospacer = ch_input.protospacer
template = ch_input.template
LaurenceKuhl marked this conversation as resolved.
Show resolved Hide resolved
} else {
ch_input = Channel.empty()
}

//
// Validate input samplesheet
//
ch_input.reads_targeted
.groupTuple()
.map {
validateInputSamplesheet(it)
if(params.input) {
LaurenceKuhl marked this conversation as resolved.
Show resolved Hide resolved
ch_input.reads_targeted
.groupTuple()
.map {
validateInputSamplesheet(it)
}
.set { reads_targeted }
} else {
reads_targeted = Channel.empty()
}
.set { reads_targeted }

emit:
reads_targeted = reads_targeted
fastqc_screening = ch_input.reads_screening
reference = ch_input.reference
protospacer = ch_input.protospacer
template = ch_input.template
reads_targeted
fastqc_screening
reference
protospacer
template
versions = ch_versions
}

Expand Down
7 changes: 6 additions & 1 deletion templates/template_fluteMLE.R
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,12 @@
FluteMLE(mle, treatname= i, proj=i, pathview.top=5)
}
} else {
FluteMLE(mle, treatname= "${prefix}", proj="${prefix}", ${args}, pathview.top=5)
beta_strings <- grep("\\\\.beta", colnames(mle), value = TRUE)
before_beta <- sub("\\\\.beta.*", "", beta_strings)
unique_strings <- unique(before_beta)
for(i in unique_strings) {
FluteMLE(mle, treatname= i, proj=i, ${args}, pathview.top=5)
}
}

version_file_path <- "versions.yml"
Expand Down
10 changes: 6 additions & 4 deletions workflows/crisprseq_screening.nf
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ include { BAGEL2_PR } from '../modules/local/
include { BAGEL2_GRAPH } from '../modules/local/bagel2/graph'
include { MATRICESCREATION } from '../modules/local/matricescreation'
include { MAGECK_FLUTEMLE } from '../modules/local/mageck/flutemle'
include { MAGECK_FLUTEMLE as MAGECK_FLUTEMLE_CONTRASTS } from '../modules/local/mageck/flutemle'
include { MAGECK_FLUTEMLE as MAGECK_FLUTEMLE_DAY0 } from '../modules/local/mageck/flutemle'
include { VENNDIAGRAM } from '../modules/local/venndiagram'
// nf-core modules
include { FASTQC } from '../modules/nf-core/fastqc/main'
Expand Down Expand Up @@ -277,8 +279,8 @@ workflow CRISPRSEQ_SCREENING {
ch_mle = MATRICESCREATION.out.design_matrix.combine(ch_counts)
MAGECK_MLE (ch_mle)
ch_versions = ch_versions.mix(MAGECK_MLE.out.versions)
MAGECK_FLUTEMLE(MAGECK_MLE.out.gene_summary)
ch_versions = ch_versions.mix(MAGECK_FLUTEMLE.out.versions)
MAGECK_FLUTEMLE_CONTRASTS(MAGECK_MLE.out.gene_summary)
ch_versions = ch_versions.mix(MAGECK_FLUTEMLE_CONTRASTS.out.versions)
ch_venndiagram = BAGEL2_PR.out.pr.join(MAGECK_MLE.out.gene_summary)
VENNDIAGRAM(ch_venndiagram)
ch_versions = ch_versions.mix(VENNDIAGRAM.out.versions)
Expand All @@ -287,8 +289,8 @@ workflow CRISPRSEQ_SCREENING {
ch_mle = Channel.of([id: "day0"]).merge(Channel.of([[]])).merge(ch_counts)
MAGECK_MLE_DAY0 (ch_mle)
ch_versions = ch_versions.mix(MAGECK_MLE_DAY0.out.versions)
MAGECK_FLUTEMLE(MAGECK_MLE_DAY0.out.gene_summary)
ch_versions = ch_versions.mix(MAGECK_FLUTEMLE.out.versions)
MAGECK_FLUTEMLE_DAY0(MAGECK_MLE_DAY0.out.gene_summary)
ch_versions = ch_versions.mix(MAGECK_FLUTEMLE_DAY0.out.versions)
}
}

Expand Down