Synteny plots are widely used for the comparison of genomic neighbourhoods. Whilst synteny plots are often included as part of larger software suites (e.g. the antiSMASH ClusterBlast module), various low-code, stand-alone tools are now available that allow users to source candidate neighbourhoods and build their own synteny plots. However, a gap remains between:
(i) tools that source these candidate neighbourhoods (e.g. cblaster, which can find hundreds of candidates),
(ii) tools that build the synteny plots (e.g. clinker, which struggles as the number of neighbourhoods exceeds 30-50) and
(iii) the synteny plots themselves, which become much harder to analyse/present as the number of neighbourhoods they include increases.
SyntenyQC is a python app for the curation of neighbourhoods immediately prior to synteny plot creation. SyntenyQC collect supports the systematic definition and annotation of candidate neighbourhoods based on a direct integration to cblaster. SytenyQC sieve offers a flexible method for objectively removing redundant neighbourhoods (sourced using cblaster or any other tool) prior to synteny plot creation. This is in some cases an absolute requirement (e.g. cblaster called via the CAGECAT webserver places a limit of 50 neighbourhoods).
pip install SyntenyQC
Note - SyntenyQC depends on BLAST+, which must be installed by the user (tested with v2.12.0 - but should work with other versions unless there are parameter changes). If this is installed correctly, you should be able to see help messages after typing blastp -h and makeblastdb -h in the command line.
Tests are performed using pytest, but are not distributed with SyntenyQC. To run tests:
- Install pytest
- Clone the
SyntenyQCgithub repository. - Update
path/to/cloned/repository/tests/email.txtwith your email (if left blank, two webscraping tests will fail). - Navigate to the cloned repository via command line (on Windows, type
cd path/to/cloned/repository). - Type
pytestin the command line.
>SyntenyQC -h
usage: SyntenyQC [-h] {collect,sieve} ...
options:
-h, --help show this help message and exit
subcommands:
{collect,sieve} Synteny quality control options
>SyntenyQC collect -h
usage: SyntenyQC collect [-h] -bp -ns -em [-fn] [-sp] [-wg]
Write genbank files corresponding to cblaster neighbourhoods from a specified CSV-format binary file located at
BINARY_PATH. For each cblaster hit accession in the binary file:
1) A record is downloaded from NCBI using the accession. NCBI requires a user EMAIL to search for this record
programatically. If WRITE_GENOMES is specified, this record is written to a local file according to FILENAMES
(see final bulletpoint).
2) A neighbourhood of size NEIGHBOURHOOD_SIZE bp is defined, centered on the cblaster hits defined in the binary
file for the target accession.
3) (If STRICT_SPAN is specified:) If the accession's record is too small to contain a neighbourhood of the
desired size, it is discarded. For example, if an accession record is a 25kb contig and NEIGHBOURHOOD_SIZE
is 50000, the record is discarded.
4) If FILENAMES is "organism", the nighbourhood is written to file called *organism*.gbk. If FILENAMES is
"accession", the neighbourhood is written to *accession*.gbk. Synteny softwares such as clinker can use these
filenames to label synetny plot samples.
Once COLLECT has been run, a new folder with the same name as the binary file will be seen in the directory
that holds the binary file (i.e. the file "path/to/binary/file.txt" will generate the folder "path/to/binary/file").
This folder will have a subdirectory called "neighbourhood", containing all of the neighbourhood genbank files
(i.e. "path/to/binary/file/neighbourhood"). If WRITE_GENOMES is specified, a second direcory ("genome") will also
be present, containing the entire record associated with each cblaster accession (i.e. "path/to/binary/file/genome").
Finally, a log file will be present in the folder "path/to/binary/file", containing all run details.
options:
-h, --help show this help message and exit
-bp, --binary_path
Full filepath to the CSV-format cblaster binary file containing neighbourhoods that should
be extracted
-n, --neighbourhood_size
Size (basepairs) of neighbourhood to be extracted (centered on middle of CBLASTER-defined
neighbourhood)
-em, --email Email - required for NCBI entrez querying
-fn, --filenames
If "organism", all collected files will be named according to organism. If "accession", all
files will be named by NCBI accession. (default:
organism)
-sp, --strict_span
If set, will discard all neighbourhoods that are smaller than neighbourhood_size bp. For
example, if you set a neighbourhood_size of 50000, a 50kb neighbourhood will be extracted
from the NCBI record associateed with each cblaster hit. If the record is too small for this
to be done (i.e. the record is smaller then 50kb) it is discarded.
-wg, --write_genomes
If set, will write entire NCBI record containing a cblaster hit to file (as well as just the
neighbourhood)
>SyntenyQC sieve -h
usage: SyntenyQC sieve [-h] -gf [-ev] [-mi] [-mts] [-mev] -sf
Filter redundant genomic neighbourhoods based on neighbourhood similarity:
- First, an all-vs-all BLASTP is performed with user-specified BLASTP settings and the neighbourhoods in GENBANK_FOLDER.
- Secondly, these are parsed to define reciprocal best hits between every pair of neighbourhoods.
- Thirdly, these reciprocal best hits are used to derive a neighbourhood similarity network. Nodes are neighbourhood
filenames and edges indicate two neighbourhood nodes that have a similarity > SIMILARITY_FILTER.
Similarity = Number of RBHs / Number of proteins in smallest neighbourhood in pair.
- Finally, this network is pruned to remove neighbourhoods that exceed the user's SIMILARITY_FILTER threshold.
Nodes that remain are copied to the newly created folder 'genbank_folder/sieve_results/genbank'.
options:
-h, --help show this help message and exit
-g, --genbank_folder
Full path to folder containing neighbourhood genbank files requiring de-duplication
-ev, --e_value BLASTP evalue threshold. (default: 1e-05)
-mi, --min_percent_identity
BLASTP percent identity threshold. (default: 50)
-mts, --max_target_seqs
BLASTP -max_target_seqs. Maximum number of aligned sequences to keep. (default: 200)
-mev, --min_edge_view
Minimum similarity between two neighbourhoods for an edge to be drawn betweeen them in the RBH
graph. Purely for visualisation of the graph HTML file - has no impact on the graph pruning
results. (default: None)
-sf, --similarity_filter
Similarity threshold above which two neighbourhoods are considered redundant
Data: RBH graph
Result: Nodes from pruned RBH graph
Procedure:
while max(node degrees in RBH graph) > 0:
delete nodes = []
for node in RBH graph:
if node degree == max(node degrees in RBH graph):
delete nodes + node
delete node = random node from delete nodes
RBH graph = RBH graph - delete node
return nodes in RBH graph
Take the actinorhodin BGC from MIBIG, and analyse with cblaster via the command line or CAGECAT web server. This will generate a set of files, one of which will be a 'binary file', stored at folder/with/binary.csv. To show that even stringent searches can generate many hits, all core biosynthetic genes (-r CAC44200.1;CAC44201.1) and >= 11 hits (-mh 11) were required for each hit neighbourhood.
SyntenyQC collect, cblaster must be run with a ',' binary delimeter and no intermediate genes (-bde ',' and no -ig flag)
folder/with/binary.csv
#Note - you should add your email
SyntenyQC collect -bp path/to/BGC0000194_binary.txt -ns 42566 -em [email protected] -fn organism -sp -wg
folder/with/binary/neighbourhood/organism1.gbk, organism2.gbk...organism157.gbk
/genomes /organism1.gbk, organism2.gbk...organism157.gbk ###only if -wg!
/log.txt
From SyntenyQC collect in this example, but any folder with at least one genbank file can be used:
folder/with/binary/neighbourhood/organism1.gbk, ...
SyntenyQC sieve -gf folder/with/binary/neighbourhood -sf 0.7
folder/with/binary/neighbourhood/organism1.gbk, ...
/sieve_results/blastp /results.xml, db.txt, db.pin, ... #call all be deleted
/genbank /organism1.gbk, ...organism38.gbk #use as e.g. clinker input
/visualisations/RBH_graph.html, RBH_histogram.html #see what is being pruned
/log.txt
- Given filenames are purely to show number of files -
neighbourhood/sieve_results/genbank/organism1.gbkis one of the genbanks in theneighbourhoodfolder, but may be different toneighbourhood/organism1.gbk. RBH_graphis an interactive html picture of the similarity graph created bySyntenyQC sieve(before pruning), only showing edges with a similarity >min_edge_view. Edges are black (<similairty_filter) or red (>=similarity_filter).RBH_histogramshows the distribution of edge weights.- Most neighbourhoods that meet the user-defined similarity threshold
-sfwill be removed in a single sieve run. However, thesieve -mtssetting can impact final results. If a protein has homologs in 251 neighboughoods and-mtsis 250, then one of the homologs will be missed by BLASTP, and the host neighbourhood may appear less similar to the neighbourhood with the query. Whilst a high-mtssetting could be used forsieveruns pruning many neighbourhoods, this will generate large blast files that may take up a lot of space and slow down the run. Thus, if users wish to remove the (typically 2-3) redundant neighbourhoods remaining after a singlesievecall, they can run sieve again on the pruned results (syntenyqc sieve -g path/to/sieve_results/genbank -sf 0.7).
Paper: Cameron L M Gilchrist, Thomas J Booth, Bram van Wersch, Liana van Grieken, Marnix H Medema, Yit-Heng Chooi, cblaster: a remote search tool for rapid identification and visualization of homologous gene clusters, Bioinformatics Advances, Volume 1, Issue 1, 2021, vbab016, https://doi.org/10.1093/bioadv/vbab016
Docs: https://cblaster.readthedocs.io/en/latest/
Paper: Cameron L M Gilchrist, Yit-Heng Chooi, clinker & clustermap.js: automatic generation of gene cluster comparison figures, Bioinformatics, Volume 37, Issue 16, August 2021, Pages 2473–2475, https://doi.org/10.1093/bioinformatics/btab007
Docs: https://github.com/gamcil/clinker
Paper: van den Belt, M., Gilchrist, C., Booth, T.J. et al. CAGECAT: The CompArative GEne Cluster Analysis Toolbox for rapid search and visualisation of homologous gene clusters. BMC Bioinformatics 24, 181 (2023). https://doi.org/10.1186/s12859-023-05311-2
Website: https://cagecat.bioinformatics.nl/
Paper (ClusterBlast was introduced in version 1): Medema MH, Blin K, Cimermancic P, de Jager V, Zakrzewski P, Fischbach MA, Weber T, Takano E, Breitling R. antiSMASH: rapid identification, annotation and analysis of secondary metabolite biosynthesis gene clusters in bacterial and fungal genome sequences. Nucleic Acids Res. 2011 Jul;39(Web Server issue):W339-46. doi: 10.1093/nar/gkr466. Epub 2011 Jun 14. PMID: 21672958; PMCID: PMC3125804
Website (latest version): https://antismash.secondarymetabolites.org/#!/start