Low recall and precision using v3.6.0

[LilyWang07]

I ran NanoCaller v3.6.0 on ONT NA24385 data using the default parameters:
NanoCaller --bam ${bam} --ref ${reference} --cpu 15

I also tried specifying the SNP and indel models:
NanoCaller --bam ${bam} --ref ${reference} --seq ont --snp_model ONT-HG002 --indel_model ONT-HG002 --sample HG002

I compared the calling results using hap.py and the hg19 reference. When using both sets of parameters, the evaluated indel recall and precision were ~0.086 and ~.007, respectively. The SNP recall and precision were ~0.44 and ~0.81. Do these low values make sense? If not, please let me know how to fix this. Thanks!

[umahsn]

Hi,

These results are very unexpected. Can you please some more information about the sample and the ground truth used? In particular, what is the coverage and average read length of the sample? What is the ground truth for NA24385 that you are using? Are you also using a high confidence interval for your ground truth?

What is the performance if you evaluate using the method shown here [ONT Case Study.md](https://github.com/WGLab/NanoCaller/blob/master/docs/ONT Case Study.md)? You can replace GRCh38 with GRCh37 in the case study since you are using hg19.

[LilyWang07]

Hi,

Thanks for your quick reply. The sample coverage is ~46x and average read length is 8098. We used the following for the ground truth:

VCF: https://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/release/AshkenazimTrio/HG002_NA24385_son/NISTv4.2.1/GRCh37/HG002_GRCh37_1_22_v4.2.1_benchmark.vcf.gz
High confidence region BED: https://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/release/AshkenazimTrio/HG002_NA24385_son/NISTv4.2.1/GRCh37/HG002_GRCh37_1_22_v4.2.1_benchmark_noinconsistent.bed

I will try the case study method. Please let me know if I can provide any other info.

[umahsn]

Thank you. Is the sequencing data publicly available or would you be able to share? I would love to run NanoCaller on it and see if I can reproduce the issue. Also, can you share the hap.py command you used for evaluation, as well as the alignment commands?