Add plot for distribution of convergence diagnostics

[aloctavodia]

I don't have a good name for this plot so its code name is plot_nose. The motivation is to visually inspect the values of ESS and R-hat for many variables.

In PyMC-BART we have a plot_convergence function that generates a similar plot, which we used to check the "BART variables" which are always multidimensional with a size at least equal to the observations. The interest is not on any individual variables. This type of plot may be of interest to others outside of PyMC-BART.

azp.plot_convergence_dist(radon, var_names=["za_county"])

I feel tempted to call "rhat_rank" just "rhat". What do you think?

specific diagnostics can be passed as strings.

azp.plot_convergence_dist(radon, var_names=["za_county"], diagnostics=["ess_tail", "ess_median"])

some ess methods accept a prob argument, we can pass it also as part of the string, like a "tuple"

azp.plot_convergence_dist(radon,
              var_names=["za_county"],
              diagnostics=["ess_tail(0.1, 0.9)", "ess_local(0.1, 0.9)", "ess_quantile(0.9)"],
              ref_line=False)

We could also add reference lines for ESS and r-hat following standard recommendations (added). For PyMC-BART, we have an "inflated" R-hat reference value. We use a heuristics to increase the 1.01 value based on the number of variables. Not saying we should use that heuristic, just saying we may want to have something similar.

---

📚 Documentation preview 📚: https://arviz-plots--105.org.readthedocs.build/en/105/

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 66.40000% with 42 lines in your changes missing coverage. Please review.

Project coverage is 84.53%. Comparing base (c2ac9c8) to head (eff896d).

Files with missing lines	Patch %	Lines
src/arviz_plots/backend/plotly/__init__.py	16.66%	10 Missing ⚠️
src/arviz_plots/plots/convergencedistplot.py	87.67%	9 Missing ⚠️
src/arviz_plots/backend/bokeh/__init__.py	27.27%	8 Missing ⚠️
src/arviz_plots/backend/none/__init__.py	50.00%	7 Missing ⚠️
src/arviz_plots/backend/matplotlib/__init__.py	25.00%	6 Missing ⚠️
src/arviz_plots/visuals/__init__.py	66.66%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #105      +/-   ##
==========================================
- Coverage   85.22%   84.53%   -0.69%     
==========================================
  Files          25       26       +1     
  Lines        2876     3000     +124     
==========================================
+ Hits         2451     2536      +85     
- Misses        425      464      +39     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[AlexAndorra]

I love that @aloctavodia , I think it'd be super helpful for most models, because a huge majority is highly dimensional and the classic plots don't allow you to diagnose the parameters quickly

[AlexAndorra]

Anything I can help with to merge this one @aloctavodia ?

[AlexAndorra]

Looks all good @aloctavodia , except for a typo I think. After that we can merge

[AlexAndorra]

LGTM now

[OriolAbril]

For PyMC-BART, we have an "inflated" R-hat reference value. We use a heuristics to increase the 1.01 value based on the number of variables. Not saying we should use that heuristic, just saying we may want to have something similar.

We probably want these two to be arguments to the function.

I feel tempted to call "rhat_rank" just "rhat". What do you think?

I would actually use both (might need a bit extra code to parse diagnostic names though). My thinking/proposed logic. ArviZ defaults are subject to change as new better algorightms/variations of them are developed. rhat would be calling rhat without specifiying any method so the default is used whereas rhat_rank would be rhat(..., method="rank") which currently matches the default but might not match the default in the future. Consequently, the default diagnostics should probably be ess_bulk, ess_tail, rhat. We could do the same for ess, but not sure it is as relevant and in that case I'd leave the default diagnostics to ess_bulk, ess_tail, rhat

EDIT: I saw one of the examples in the docstring was using "rhat" so I already implemented this idea (for rhat only).

[OriolAbril]

Already working example with all backends shown: https://arviz-plots--105.org.readthedocs.build/en/105/gallery/plot_convergence_dist.html (bokeh also has the vertical line but it doesn't extend the axis limits up to the vertical line so ends up hidden, a bit weird but hopefully not super critical)

[aloctavodia]

LGTM, not sure what's the source of the new error.

[OriolAbril]

The old error was pre-commit. I have started adding tests and realized we need to work a bit on the API for the ess methods (test is currently failing but pushed anyway so it is here already)

[aloctavodia]

sorry for the mess!