Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added 2FAST2Q module #7318

Merged
merged 65 commits into from
Jan 30, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
9e7f287
Added 2FAST2Q module
afombravo Jan 16, 2025
11263b5
added tests to 2fast2q
afombravo Jan 19, 2025
28620cc
added tests to 2fast2q
afombravo Jan 19, 2025
6327252
added tests to 2fast2q
afombravo Jan 19, 2025
f505542
fixed tests to 2fast2q
afombravo Jan 19, 2025
323062a
fixed tests to 2fast2q
afombravo Jan 19, 2025
eec55df
Merge branch 'master' into 2FAST2Q
SPPearce Jan 19, 2025
eafe98e
Update modules/nf-core/fast2q/tests/main.nf.test
afombravo Jan 19, 2025
48a87b5
Update modules/nf-core/fast2q/tests/main.nf.test
afombravo Jan 19, 2025
96dd572
added optional input - hope it is being parsed correctly
afombravo Jan 19, 2025
d9982e3
added optional input - hope it is being parsed correctly
afombravo Jan 19, 2025
00ed681
fix
afombravo Jan 19, 2025
9de9c4c
fix lack of optional input on tests
afombravo Jan 19, 2025
210bba4
fixing in progree
afombravo Jan 19, 2025
749ccf7
fixing in progress
afombravo Jan 19, 2025
b4a4e3f
fixing in progress
afombravo Jan 19, 2025
88e6c4a
fixing in progress
afombravo Jan 19, 2025
01132cc
I changed the test data to 2FAST2Q repo data, NOT NF-CORE test data, …
afombravo Jan 19, 2025
c976063
I changed the test data to 2FAST2Q repo data, NOT NF-CORE test data, …
afombravo Jan 19, 2025
3eadb4f
I changed the test data to 2FAST2Q repo data, NOT NF-CORE test data, …
afombravo Jan 19, 2025
1f6b509
I changed the test data to 2FAST2Q repo data, NOT NF-CORE test data, …
afombravo Jan 19, 2025
e32bbb0
testing if it works with just fastq
afombravo Jan 19, 2025
106ae2f
Update modules/nf-core/fast2q/main.nf
afombravo Jan 20, 2025
725bc16
added test data and fixed several output issues
afombravo Jan 20, 2025
f2f64c9
Merge branch 'master' into 2FAST2Q
afombravo Jan 20, 2025
6363e33
fixed the nf.test
afombravo Jan 20, 2025
4cfe166
fixed the nf.test
afombravo Jan 20, 2025
13278d6
fixed the module
afombravo Jan 20, 2025
bc8b153
moved test data up
afombravo Jan 20, 2025
6acdde0
direct link to a test data
afombravo Jan 20, 2025
24e6b89
Merge branch 'master' into 2FAST2Q
afombravo Jan 20, 2025
1e4002c
Merge branch 'master' into 2FAST2Q
afombravo Jan 20, 2025
8c89fea
changed paths to tests
afombravo Jan 20, 2025
c6744ac
Update modules/nf-core/fast2q/tests/nextflow.config
afombravo Jan 22, 2025
1127665
Update modules/nf-core/fast2q/tests/main.nf.test
afombravo Jan 22, 2025
b9a3da1
Update modules/nf-core/fast2q/tests/main.nf.test
afombravo Jan 22, 2025
27df973
Update modules/nf-core/fast2q/tests/main.nf.test
afombravo Jan 22, 2025
b74a003
changed assertions in test
afombravo Jan 23, 2025
82fea39
Merge branch 'master' into 2FAST2Q
afombravo Jan 23, 2025
4949199
removed indentation on main 2fast2q cmd as it was not being recognized
afombravo Jan 23, 2025
7abae13
Merge branch '2FAST2Q' of https://github.com/afombravo/modules into 2…
afombravo Jan 23, 2025
0a9fd26
everything working locally
afombravo Jan 23, 2025
731702a
everything working locally
afombravo Jan 23, 2025
198dd26
everything working locally
afombravo Jan 23, 2025
d50ee04
everything working locally
afombravo Jan 23, 2025
e598c43
everything working locally
afombravo Jan 23, 2025
06bf8d5
everything working locally
afombravo Jan 23, 2025
a6c83c0
Merge branch 'master' into 2FAST2Q
afombravo Jan 24, 2025
ba13f45
enhanced descriptions
afombravo Jan 24, 2025
4cf4026
Merge branch 'master' into 2FAST2Q
afombravo Jan 24, 2025
11f4a9f
updated singularity container
afombravo Jan 24, 2025
f114f57
adding a file deleted by accident
afombravo Jan 24, 2025
b350d66
Merge branch 'master' into 2FAST2Q
afombravo Jan 24, 2025
35ab059
Merge branch 'master' into 2FAST2Q
afombravo Jan 25, 2025
bc794c9
Update module
SPPearce Jan 28, 2025
d4b04b8
Fix linting
SPPearce Jan 29, 2025
6a2be0c
Merge branch 'master' into 2FAST2Q
SPPearce Jan 29, 2025
6c4bd0f
Update meta.yml
afombravo Jan 29, 2025
b573091
Update meta.yml
afombravo Jan 29, 2025
8f85b65
Update meta.yml
afombravo Jan 29, 2025
4823e4e
Merge branch 'master' into 2FAST2Q
afombravo Jan 29, 2025
f791d36
Fix broken test
SPPearce Jan 29, 2025
96c0733
Update main.nf.test.snap
afombravo Jan 30, 2025
0231e7e
Update main.nf.test
afombravo Jan 30, 2025
f75b20b
Merge branch 'master' into 2FAST2Q
afombravo Jan 30, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions modules/nf-core/fast2q/environment.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
channels:
- conda-forge
- bioconda
dependencies:
- bioconda::fast2q=2.7.2
67 changes: 67 additions & 0 deletions modules/nf-core/fast2q/main.nf
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
process FAST2Q {

tag "2FAST2Q"
label 'process_single'

conda "${moduleDir}/environment.yml"
container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
'https://depot.galaxyproject.org/singularity/fast2q:2.7.2--pyh7e72e81_0' :
'biocontainers/fast2q:2.7.2--pyh7e72e81_0' }"

input:
tuple val(meta), path(fastq)
tuple val(meta2), path(library)

output:
tuple val(meta), path("${prefix}.csv") , emit: count_matrix
tuple val(meta), path("${prefix}_stats.csv") , emit: stats
tuple val(meta), path("${prefix}_distribution_plot.png") , emit: distribution_plot
tuple val(meta), path("${prefix}_reads_plot.png") , emit: reads_plot
tuple val(meta), path("${prefix}_reads_plot_percentage.png"), emit: reads_plot_percentage
path "versions.yml", emit: versions

when:
task.ext.when == null || task.ext.when

script:
def args = task.ext.args ?: ''
prefix = task.ext.prefix ?: "${meta.id}"
def input_file = (fastq instanceof Path && fastq.exists()) ? "--s ${fastq}" : ''
def library_file = (library instanceof Path && library.exists()) ? "--g ${library}" : ''

"""
export MPLCONFIGDIR=\$PWD
2fast2q \\
-c \\
--o ./ \\
--fn ${prefix} \\
--cp ${task.cpus} \\
$input_file \\
$library_file \\
$args

mv **/${prefix}* .

cat <<-END_VERSIONS > versions.yml
${task.process}:
2FAST2Q version: \$(2fast2q -v | grep 'Version:' | sed 's/Version: //g')
END_VERSIONS
"""

stub:
prefix = task.ext.prefix ?: "${meta.id}"

"""
touch ${prefix}.csv
touch ${prefix}_stats.csv
touch ${prefix}_distribution_plot.png
touch ${prefix}_reads_plot.png
touch ${prefix}_reads_plot_percentage.png

cat <<-END_VERSIONS > versions.yml
${task.process}:
2FAST2Q version: \$(2fast2q -v | grep 'Version:' | sed 's/Version: //g')
END_VERSIONS
"""

}
98 changes: 98 additions & 0 deletions modules/nf-core/fast2q/meta.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
name: fast2q
afombravo marked this conversation as resolved.
Show resolved Hide resolved
description: A program that counts sequence occurrences in FASTQ files.
keywords:
- CRISPRi
- FASTQ
- genomics
tools:
- 2FAST2Q:
description: |
2FAST2Q is ideal for CRISPRi-Seq, and for extracting and counting any kind of information from reads in the fastq format, such as barcodes in Bar-seq experiments.
2FAST2Q can work with sequence mismatches, Phred-score, and be used to find and extract unknown sequences delimited by known sequences.
2FAST2Q can extract multiple features per read using either fixed positions or delimiting search sequences.
homepage: https://github.com/afombravo/2FAST2Q
doi: 10.7717/peerj.14041
licence: ["GPL-3.0-or-later"]
identifier: ""

input:
- - meta:
type: map
description: |
Groovy Map containing output name.
e.g. [ id:'test']
afombravo marked this conversation as resolved.
Show resolved Hide resolved
- fastq:
type: directory
description: Folder with FASTQ file(s). 2FAST2Q automatically picks up all the FASTQ files inside the provided folder.
pattern: "*.{fastq,gz}"

- - meta2:
type: map
description: |
Groovy Map containing sample information
e.g. [ id:'library_name', multiple_features_per_read:false ]
- library:
type: file
description: .csv library file following the ´Feature_name,sequence´ or ´Feature_name,sequence1:sequence2´ format. See 2FAST2Q instructions for more information.
pattern: "*.csv"

output:
- count_matrix:
- meta:
type: map
description: |
Groovy Map containing output name.
e.g. [ id:'test' ]
- ${prefix}.csv:
type: file
description: |
Count matrix csv file
- stats:
- meta:
type: map
description: |
Groovy Map containing output name.
e.g. [ id:'test' ]
- ${prefix}_stats.csv:
type: file
description: |
File containing all the relevant statistics such as quality passing reads, aligned reads, total reads, and sample run times.
- distribution_plot:
- meta:
type: map
description: |
Groovy Map containing output name.
e.g. [ id:'test' ]
- ${prefix}_distribution_plot.png:
type: file
description: |
Violin plot of the distribution of reads per feature across all samples.
- reads_plot:
- meta:
type: map
description: |
Groovy Map containing output name.
e.g. [ id:'test' ]
- ${prefix}_reads_plot.png:
type: file
description: |
Bar plot with the distribution of reads, in absolute numbers, binned to the different quality metrics indicated in the statistics.csv
- reads_plot_percentage:
- meta:
type: map
description: |
Groovy Map containing output name.
e.g. [ id:'test' ]
- ${prefix}_reads_plot_percentage.png:
type: file
description: |
Bar plot with the distribution of reads, in percentage, binned to the different quality metrics indicated in the statistics.csv
- versions:
- versions.yml:
type: file
description: File containing software versions
pattern: "versions.yml"
authors:
- "@afombravo"
maintainers:
- "@afombravo"
165 changes: 165 additions & 0 deletions modules/nf-core/fast2q/tests/main.nf.test
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
nextflow_process {

name "Test Process 2FAST2Q"
script "../main.nf"
process "FAST2Q"

tag "modules"
tag "modules_nfcore"
tag "fast2q"

afombravo marked this conversation as resolved.
Show resolved Hide resolved
config './nextflow.config'

test("2FAST2Q self-test") {

when {

params {
module_args = '-t'
}

process {
"""
input[0] = [
[ id:'test1' ], // meta map
[]
]
input[1] = [
[ id:'library_name', multiple_features_per_read:false ], // meta map for second input
[]
]
"""
}
}

then {
assertAll(
{ assert process.success },
{ assert snapshot(
process.out.count_matrix,
file(process.out.stats[0][1]).name,
file(process.out.distribution_plot[0][1]).name,
file(process.out.reads_plot[0][1]).name,
file(process.out.reads_plot_percentage[0][1]).name,
process.out.versions, // versions file
path(process.out.versions[0]).yaml // Validate against the versions snapshot
).match()
}
)
}
}

test("Extracting and counting all features at position=0 with default length of 20bp from a FASTQ file (without optional library.csv)") {

when {

params {
module_args = '--mo EC'
}

process {
"""
input[0] = [
[ id:'test1' ], // meta map
file(params.test_data_base + '/data/genomics/mus_musculus/mageck/ERR376998.small.fastq.gz', checkIfExists: true) // FASTQ file
afombravo marked this conversation as resolved.
Show resolved Hide resolved
]
input[1] = [
[ id:'library_name', multiple_features_per_read:false ], // meta map for second input
[]
]
"""
}
}

then {
assertAll(
{ assert process.success }, // Ensure process ran successfully
{ assert snapshot(
process.out.count_matrix,
file(process.out.stats[0][1]).name,
file(process.out.distribution_plot[0][1]).name,
file(process.out.reads_plot[0][1]).name,
file(process.out.reads_plot_percentage[0][1]).name,
process.out.versions, // versions file
path(process.out.versions[0]).yaml // Validate against the versions snapshot
).match()
}
)
}
}

test("Extracting all features at position=0 with default length of 20bp from a FASTQ file, and respective alignment to a file with DNA features (library.csv)") {

when {

params {
module_args = ''
}

process {
"""
input[0] = [
[ id:'test1' ], // meta map
file(params.test_data_base + '/data/genomics/mus_musculus/mageck/ERR376998.small.fastq.gz', checkIfExists: true) // FASTQ file
afombravo marked this conversation as resolved.
Show resolved Hide resolved
]
input[1] = [
[ id:'library_name', multiple_features_per_read:false ], // meta map for second input
file(params.test_data_base + '/data/genomics/mus_musculus/mageck/yusa_library.csv', checkIfExists: true) // library file
afombravo marked this conversation as resolved.
Show resolved Hide resolved
]
"""
}
}

then {
assertAll(
{ assert process.success }, // Ensure process ran successfully
{ assert snapshot(
process.out.count_matrix,
file(process.out.stats[0][1]).name,
file(process.out.distribution_plot[0][1]).name,
file(process.out.reads_plot[0][1]).name,
file(process.out.reads_plot_percentage[0][1]).name,
process.out.versions, // versions file
path(process.out.versions[0]).yaml // Validate against the versions snapshot
).match()
}
)
}
}

test("2FAST2Q self-test - stub") {
options "-stub"
when {

params {
module_args = '-t'
}

process {
"""
input[0] = [
[ id:'test1' ], // meta map
[]
]
input[1] = [
[ id:'library_name', multiple_features_per_read:false ], // meta map for second input
[]
]
"""
}
}

then {
assertAll(
{ assert process.success },
{ assert snapshot(
process.out,
path(process.out.versions[0]).yaml // Validate against the versions snapshot
).match()
}
)
}
}


}
Loading
Loading