rattest QOL update + centroid test (#214)

* ignore test outputs

* Install rattest to install/bin directly.

* Add local / absolute experiment to ratdb path, mainly for testing purposes.

* update documentation

* (attempt to) revive fitcentroid test

* add cformat to test scripts

* remove old files

* add basic ANSI color code for test output

* move template back to workflow root

* clean up fitcentroid code

* add validity guard to fitcentroid test

* update fitcentroid test to remove low energy runs

.github/workflows/rattests-workflow.yml → .github/workflows/rattests-template.yml

@@ -24,7 +24,7 @@ jobs:
       run: |
         source /ratpac-setup/env.sh
         source ../ratpac2/ratpac.sh
-        python3 $RATSHARE/python/rattest.py -t test/full/${{ inputs.rattest-name }}
+        rattest -t test/full/${{ inputs.rattest-name }}
       id: test
       continue-on-error: true
     - name: upload-result

.github/workflows/rattests.yml

@@ -31,6 +31,12 @@ jobs:
 
   acrylic_attenuation:
     needs: build
-    uses: ./.github/workflows/rattests-workflow.yml
+    uses: ./.github/workflows/rattests-template.yml
     with:
       rattest-name: acrylic_attenuation
+
+  fitcentroid:
+    needs: build
+    uses: ./.github/workflows/rattests-template.yml
+    with:
+      rattest-name: fitcentroid

.gitignore

@@ -19,6 +19,7 @@ src/core/Config.hh
 
 *.log
 *.root
+!test/full/*/standard.root
 *.heprep
 .DS_Store
 *.gif
@@ -35,3 +36,7 @@ doc/doxygen
 .pytest_cache*
 ratpac.*sh
 RatpacConfig.cmake
+
+# rattest outputs 
+test/full/*/*.png 
+test/full/*/*.html

CMakeLists.txt

@@ -139,6 +139,9 @@ install(DIRECTORY models/ DESTINATION ${RATSHARE}/models
 install(DIRECTORY python/ DESTINATION ${RATSHARE}/python
   PATTERN "python/*")
 
+# Install rattest executable
+install(PROGRAMS python/rattest.py DESTINATION bin RENAME rattest)
+
 # Install the macro files
 install(DIRECTORY macros/ DESTINATION ${RATSHARE}/macros
   PATTERN "macros/*")

cformat.sh

@@ -11,7 +11,7 @@ fi
 
 echo "Checking formatting..."
 changes=0
-for file in $(find ./src \( -name "*.cc" -o -name "*.hh" -o -name "*.icc" -o -name "*.cpp" -o -name "*.hpp" \)); do
+for file in $(find ./src ./test \( -name "*.cc" -o -name "*.hh" -o -name "*.icc" -o -name "*.cpp" -o -name "*.hpp" -o -name "*.C" \)); do
     retval=$(clang-format -style=file -n -Werror $file)
     if [ $? -eq 1 ]; then
         echo "Formatting $file"

doc/users_guide/rattest.rst

@@ -8,7 +8,7 @@ Rattest is a framework for creating unit and functional tests for RAT. These tes
 
 At minimum, a test consists of a RAT macro and a ROOT macro -- the Monte Carlo and the analysis. New (simplified) geometries, modified RATDB databases, etc. can also be included. When run, these tests are compared to a standard via a KS test, and a web page is created with histograms (standard and current) and KS test results. The standard RAT logs and output ROOT file is also available for analysis.
 
-The existing rattests are included with the standard RAT distribution, in `$RATSHARE/test/`, with the functional tests in `$RATSHARE/test/full/<test-name>`. To run a single test, `cd` to the test directory and simply run `python3 $RATSHARE/python/rattest.py <test-name>` where `<test-name>` corresponds to a folder in `$RATSHARE/test/full`. Rattest will iterate through the directory structure to find the test, run the RAT macro, run the ROOT macro on the output, and generate a report page.
+The existing rattests are included with the standard RAT distribution, in `$RATSHARE/test/`, with the functional tests in `$RATSHARE/test/full/<test-name>`. To run a single test, `cd` to the test directory and simply run `rattest <test-name>` where `<test-name>` corresponds to a folder in `$RATSHARE/test/full`. Rattest will iterate through the directory structure to find the test, run the RAT macro, run the ROOT macro on the output, and generate a report page.
 
 The `rattest.py` script takes the following options::
 
@@ -108,6 +108,11 @@ Keep things as simple as possible, and turn off as many options as possible. The
     
     /run/beamOn 500
 
+You can also create a custom rat "experiment" in the test directory. This experiment can include any custom ratdb tables you want. 
+You can tell rat to use this experiment by adding the line::
+
+    /rat/db/set DETECTOR experiment "cylinder"
+
 5. Write a ROOT macro
 
 The ROOT macro should create a histogram that captures the benchmark you are looking for. It should consist of a single `void` function with the same name as the macro ie `acrylic_attentuation(std::string event_file, std::string outfile)`. `rattest` will automatically fill in the function arguments when it calls the root macro.
@@ -142,8 +147,8 @@ The ROOT macro from `acrylic_attenuation`::
 
 7. Create a standard
 
- From the test directory, run `python3 rattest.py -u [your test name]`. This will create the file `standard.root`, which will be the basis for comparison until the next time you run `rattest` with the `-u` option. Take a look at `results.html` to see how things worked out.
+ From the test directory, run `rattest -u [your test name]`. This will create the file `standard.root`, which will be the basis for comparison until the next time you run `rattest` with the `-u` option. Take a look at `results.html` to see how things worked out.
 
-This is pretty much it. If you run `python3 rattest.py [your test name]` again, you should get a results page (which will open in your default browser unless you specified the `-t` option) with very similar results.
+This is pretty much it. If you run `rattest [your test name]` again, you should get a results page (which will open in your default browser unless you specified the `-t` option) with very similar results.
 
 If you think the test is useful to others, commit it to the RAT repository with svn. Be sure to commit only the `rattest.config`, RAT and ROOT macro, any geometry or RATDB files, and `standard.root`.

python/rattest/__init__.py

@@ -126,7 +126,7 @@ def run(self, regen_mc=False, regen_plots=False, html=None):
         Run a full RAT test, including simulation and the ROOT macro.
         Includes writing results to the HTML file, if option is specified.
         '''
-        print('=' * 5 + ' Run Test: {} '.format(self.name) + '=' * 5)
+        print("\033[1;35m" + '=' * 5 + ' Run Test: {} '.format(self.name) + '=' * 5 + "\033[0m")
         self._do_run(regen_mc=regen_mc, regen_plots=regen_plots)
 
         if html:
@@ -248,7 +248,7 @@ def compare_hists(self, master_fname, current_fname, html=None):
             prob = m_obj.KolmogorovTest(c_obj)
             if m_obj.GetEntries() == 0 and c_obj.GetEntries() == 0:
                 prob = 1
-            print("Comparing {}: {}".format(objname, c_obj.GetTitle()))
+            print("\033[1;34mComparing {}: {}\033[0m".format(objname, c_obj.GetTitle()))
 
             #Require a perfect match if desired, else test against critical_probability
             if self.KS_threshold == 1.0:
@@ -264,7 +264,7 @@ def compare_hists(self, master_fname, current_fname, html=None):
                     success = False
                     overall_success = False
 
-            success_message = "SUCCESS" if (success and overall_success) else "FAILURE"
+            success_message = "\033[1;32mSUCCESS\033[0m" if (success and overall_success) else "\033[1;31mFAILURE\033[0m"
             print("  {}: KS prob = {}".format(success_message, prob))
 
             plotfile = os.path.join(self.testdir, objname + '.png')

src/geo/src/DetectorConstruction.cc

@@ -34,7 +34,15 @@ G4VPhysicalVolume *DetectorConstruction::Construct() {
   // Load experiment RATDB files before doing anything else
   try {
     experiment = ldetector->GetS("experiment");
-    info << "Loading experiment-specific RATDB files for: " << experiment << newline;
+  } catch (DBNotFoundError &e) {
+    info << "No experiment-specific tables loaded." << newline;
+  }
+  info << "Loading experiment-specific RATDB files for: " << experiment << newline;
+  // Attempt to load literal experiments (absolute paths and/or experiments defined in local directory).
+  int result = db->LoadAll(experiment);
+  if (result == 2) {
+    info << "Found experiment files in " << experiment << newline;
+  } else {
     for (auto dir : Rat::ratdb_directories) {
       std::string experimentDirectoryString = dir + "/" + experiment;
       int result = db->LoadAll(experimentDirectoryString);
@@ -43,8 +51,6 @@ G4VPhysicalVolume *DetectorConstruction::Construct() {
         break;
       }
     }
-  } catch (DBNotFoundError &e) {
-    info << "No experiment-specific tables loaded." << newline;
   }
 
   try {