From 40c687d683ab0132b8d113fef110a02dff34b9bb Mon Sep 17 00:00:00 2001
From: Art Rand modkit dmr pair \
-a ${norm_pileup_1}.gz \
@@ -356,8 +360,8 @@
modkit dmr pair \
-a ${norm_pileup_1}.gz \
diff --git a/docs/perf_considerations.html b/docs/perf_considerations.html
index 2558b65..9dd1af8 100644
--- a/docs/perf_considerations.html
+++ b/docs/perf_considerations.html
@@ -196,6 +196,15 @@
--chunk-size can increase parallelism (and decrease run time)
but will consume more memory.
modkit extract
.Transforming reads into a table with modkit extract
can produce large files (especially with long reads).
+Before the data can be written to disk, however, it is enqueued in memory and can potentially create a large memory burden.
+There are a few ways to decrease the amount of memory modkit extract
will use in these cases:
--queue-size
, this decreased the number of batches that will be held in flight.--ignore-index
this will force modkit extract
to run a serial scan of the mod-BAM.--interval-size
, this will decrease the size of the batches.modkit extract
.Transforming reads into a table with modkit extract
can produce large files (especially with long reads).
+Before the data can be written to disk, however, it is enqueued in memory and can potentially create a large memory burden.
+There are a few ways to decrease the amount of memory modkit extract
will use in these cases:
--queue-size
, this decreased the number of batches that will be held in flight.--ignore-index
this will force modkit extract
to run a serial scan of the mod-BAM.--interval-size
, this will decrease the size of the batches.