Add benchmarking as test

[sunxd3]

No description provided.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.62%. Comparing base (8ca83f9) to head (32ebc63).

❗ Current head 32ebc63 differs from pull request most recent head a8c33b0. Consider uploading reports for the commit a8c33b0 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #146      +/-   ##
==========================================
- Coverage   80.08%   79.62%   -0.46%     
==========================================
  Files          21       21              
  Lines        1968     1978      +10     
==========================================
- Hits         1576     1575       -1     
- Misses        392      403      +11     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sunxd3]

on my laptop (i7-9750h)

Example Name	Category	Median Time	Minimum Time	Maximum Time	Memory Usage
surgical_realistic	CollectVariables	584.900 μs	555.200 μs	7.641 ms	160.45 KiB
	DataTransformation	680.600 μs	650.800 μs	5.440 ms	171.61 KiB
	NodeFunctions	5.366 ms	5.136 ms	13.870 ms	1.05 MiB
pumps	CollectVariables	435.200 μs	402.700 μs	8.472 ms	122.57 KiB
	DataTransformation	797.900 μs	739.500 μs	6.030 ms	158.81 KiB
	NodeFunctions	4.687 ms	4.436 ms	13.062 ms	846.62 KiB
dogs	CollectVariables	8.987 ms	8.060 ms	15.348 ms	2.64 MiB
	DataTransformation	399.213 ms	389.600 ms	418.577 ms	103.48 MiB
	NodeFunctions	736.163 ms	718.590 ms	757.100 ms	162.84 MiB
magnesium	CollectVariables	4.138 ms	3.799 ms	12.046 ms	1.21 MiB
	DataTransformation	36.673 ms	34.924 ms	56.215 ms	9.24 MiB
	NodeFunctions	61.995 ms	58.299 ms	79.389 ms	14.18 MiB
surgical_simple	CollectVariables	285.100 μs	266.300 μs	5.427 ms	73.97 KiB
	DataTransformation	6.333 μs	5.800 μs	483.033 μs	3.48 KiB
	NodeFunctions	3.141 ms	2.925 ms	9.228 ms	628.22 KiB
salm	CollectVariables	922.000 μs	841.500 μs	5.900 ms	274.93 KiB
	DataTransformation	2.965 ms	2.635 ms	9.307 ms	877.59 KiB
	NodeFunctions	18.084 ms	16.562 ms	33.135 ms	3.84 MiB
stacks	CollectVariables	2.144 ms	1.958 ms	7.070 ms	509.02 KiB
	DataTransformation	47.496 ms	45.505 ms	61.462 ms	9.65 MiB
	NodeFunctions	31.472 ms	29.831 ms	42.235 ms	6.73 MiB
bones	CollectVariables	18.587 ms	16.663 ms	30.589 ms	5.23 MiB
	DataTransformation	303.072 ms	297.214 ms	322.016 ms	69.64 MiB
	NodeFunctions	818.343 ms	807.383 ms	844.248 ms	168.25 MiB
leukfr	CollectVariables	9.424 ms	8.450 ms	22.373 ms	3.24 MiB
	DataTransformation	794.951 ms	777.404 ms	816.670 ms	203.05 MiB
	NodeFunctions	537.267 ms	523.686 ms	545.193 ms	119.75 MiB
lsat	CollectVariables	43.117 ms	39.970 ms	55.252 ms	14.52 MiB
	DataTransformation	1.704 s	1.700 s	1.707 s	394.54 MiB
	NodeFunctions	2.818 s	2.818 s	2.818 s	518.60 MiB
blockers	CollectVariables	898.900 μs	833.600 μs	10.360 ms	298.03 KiB
	DataTransformation	3.946 ms	3.399 ms	15.987 ms	897.06 KiB
	NodeFunctions	24.589 ms	21.946 ms	41.200 ms	4.14 MiB
seeds	CollectVariables	975.450 μs	694.200 μs	8.298 ms	234.99 KiB
	DataTransformation	4.912 ms	4.138 ms	13.730 ms	1.12 MiB
	NodeFunctions	15.211 ms	14.567 ms	78.070 ms	3.71 MiB
equiv	CollectVariables	1.298 ms	1.075 ms	7.380 ms	358.57 KiB
	DataTransformation	11.568 ms	10.832 ms	27.015 ms	3.04 MiB
	NodeFunctions	13.632 ms	12.575 ms	28.464 ms	3.47 MiB
inhalers	CollectVariables	23.230 ms	21.833 ms	41.529 ms	9.30 MiB
	DataTransformation	1.082 s	1.070 s	1.118 s	337.78 MiB
	NodeFunctions	1.485 s	1.479 s	1.527 s	363.42 MiB
rats	CollectVariables	1.595 ms	1.470 ms	8.694 ms	527.41 KiB
	DataTransformation	23.064 ms	21.237 ms	40.015 ms	5.21 MiB
	NodeFunctions	81.859 ms	76.969 ms	98.567 ms	17.37 MiB
mice	CollectVariables	1.343 ms	1.224 ms	5.940 ms	382.84 KiB
	DataTransformation	7.369 ms	6.738 ms	17.053 ms	1.92 MiB
	NodeFunctions	49.244 ms	46.509 ms	56.114 ms	10.99 MiB
leuk	CollectVariables	9.293 ms	8.277 ms	18.569 ms	2.88 MiB
	DataTransformation	679.459 ms	666.365 ms	693.521 ms	179.51 MiB
	NodeFunctions	391.704 ms	380.821 ms	463.007 ms	90.60 MiB
oxford	CollectVariables	2.652 ms	2.412 ms	9.425 ms	1.00 MiB
	DataTransformation	66.211 ms	62.496 ms	85.060 ms	15.00 MiB
	NodeFunctions	244.052 ms	232.052 ms	269.343 ms	48.44 MiB
epil	CollectVariables	3.615 ms	3.340 ms	24.076 ms	1.29 MiB
	DataTransformation	111.638 ms	105.019 ms	125.903 ms	38.36 MiB
	NodeFunctions	331.592 ms	317.156 ms	352.125 ms	85.13 MiB
dyes	CollectVariables	761.400 μs	673.800 μs	9.110 ms	205.51 KiB
	DataTransformation	1.735 ms	1.551 ms	12.006 ms	458.78 KiB
	NodeFunctions	13.115 ms	12.399 ms	27.142 ms	2.78 MiB
kidney	CollectVariables	1.556 ms	1.460 ms	11.480 ms	561.68 KiB
	DataTransformation	13.102 ms	12.188 ms	22.505 ms	4.02 MiB
	NodeFunctions	79.606 ms	74.805 ms	93.232 ms	20.79 MiB

[yebai]

Thanks, @sunxd3 -- these look reasonable. There are a few models which take longer time, e.g. inhalers, lsat.

Do you understand why? In addition, how do these numbers compare to WinBUGS/MultiBUGS?

[sunxd3]

@yebai

Do you understand why?

I have some guesses, but need to look deeper into it.

how do these numbers compare to WinBUGS/MultiBUGS

I don't have a way to measure, but it is quite fast (under a second). I'll need to ask Andrew for a way to do the benchmark.

[yebai]

I don't have a way to measure, but it is quite fast (under a second). I'll need to ask Andrew for a way to do the benchmark.

It sounds good to get some comparison numbers of compiler efficiency per step (e.g., those above) and also log density and Gibbs conditional density evaluation speed.