New R-hat, ESS and quantile-MCSE functions + related plots + rank plots

[avehtari]

Add new R-hat, ESS, quantile-MCSE and new plots described in

• Aki Vehtari, Andrew Gelman, Daniel Simpson, Bob Carpenter, Paul-Christian Bürkner (2019): Rank-normalization, folding, and localization: An improved R-hat for assessing convergence of MCMC. arXiv preprint arXiv:1903.08008.

The reference code in R is available in monitornew.R and in monitorplot.R. I think porting required computation from monitornew.R is easy and porting the plots may require a bit more work, but I'm not expert on Python graphics.

There might be also something useful in the corresponding issues for rstan stan-dev/rstan#617 and bayesplot stan-dev/bayesplot#178

[ColCarroll]

@avehtari thanks to you and collaborators for the nice paper, releasing code at the same time, and for reaching out to help get this all implemented!

Two questions on rank plots:

1. Does it make sense to think about putting error bars on rank plots, like in the simulation based calibration paper? Part of me thinks that if you want to automatically "score" your rank plot you should just use R-hat.
2. Any thoughts on how to set a y-axis for "mean centered" rank plots? I like that they show positives and negatives very clearly, but it is hard to tell scale. The error bars might help with this, I suppose.

Centered Eight Schools:

Non-centered Eight Schools:

[avehtari]

Does it make sense to think about putting error bars on rank plots, like in the simulation based calibration paper?

Yes, but that they are not sufficient as mentioned in SBC paper. I think the benefit of rank plots is the simple interpretation. Additional figure type would be ECDF difference as in Fig 14c in SBC paper. It has a better envelope, but it requires more explanation and thus both are useful to have.

Part of me thinks that if you want to automatically "score" your rank plot you should just use R-hat.

Yes, we recommend Rhat, bulk-ESS, tail-ESS and HMC specific diagnostics for automatic diagnostics and rank plots for helping to see where the problem is or if you want to check just some rank plots.

Any thoughts on how to set a y-axis for "mean centered" rank plots?

I don't like it for this purpose as it's not that familiar for people as histograms and would require additional explanation without containing additional information. I would instead consider adding ECDF difference plot, which has the benefit of being more sensitive to changes in the edges and takes into account dependencies like clustering unlike histogram error bar.