From 2a103074542e44481fc1d6369c6cad82cae9f456 Mon Sep 17 00:00:00 2001 From: rjust Date: Sun, 3 Nov 2024 11:06:43 -0800 Subject: [PATCH] A few more clarifications in README. --- framework/test/README.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/framework/test/README.md b/framework/test/README.md index 8b2ffe132..0614b4ca2 100644 --- a/framework/test/README.md +++ b/framework/test/README.md @@ -75,11 +75,11 @@ parallel processes): ``` ./jobs_cmd.pl ./test_verify_bugs.sh -A | shuf | parallel -j20 --progress ``` -Reproducing all bugs (20 jobs in parallel) takes ~3h. +Reproducing all bugs (20 jobs in parallel) takes ~90min. -(When upgrading Defects4J it is helpful to drop the `-A` flag at first for -efficiency. After fixing any issues, add the `-A` flag to test for full -reproducibility with all tests.) +When upgrading Defects4J it is helpful to drop the `-A` flag at first for +efficiency. (Reduces runtime to ~50min). After fixing any issues, add the +`-A` flag back in to test for full reproducibility with all tests. ### Code coverage analysis for all bugs (parallel) ``` @@ -99,8 +99,9 @@ Obtaining code-level stats for all bugs (20 jobs in parallel) takes ~45min. ``` Running mutation analysis on all bugs (20 jobs in parallel) takes up to ~24h. -TODO: A few (Closure) bugs have a very long analysis time; we should prioritize -these jobs in the shuffled job list to make sure they run first. +TODO: A few (Closure) jobs have a very long analysis time. (All but a hand full +of jobs finish within 14 hours.) We should prioritize these long jobs in the +shuffled job list to make sure they run early in parallel to many smaller jobs. ### Export command and exported properties ```