MESMER-X: thoughts on fg_fun_scale

[veni-vidi-vici-dormivi]

TLDR: the current function to optimize for the scale is not optimal for all distributions but it is not worth changing it for now.

For a preliminary first guess for the scale of a distribution, we optimize the scale parameter such that it is close to the absolute deviations of the samples from the location:

mesmer/mesmer/mesmer_x/train_l_distrib_mesmerx.py

Lines 1164 to 1171 in 72d83a5

	def fg_fun_sca(self, x_sca):
	x = np.copy(self.fg_coeffs)
	x[self.fg_ind_sca] = x_sca
	params = self.expr_fit.evaluate_params(x, self.data_pred)
	loc, sca = params["loc"], params["scale"]
	# ^ better to use that one instead of deviation, which is affected by the scale
	dev = np.abs(self.data_targ - loc)
	return np.sum((dev - sca) ** 2)

I wondered if instead of using absolute deviations, we should go for the squared deviations. It turns out that either option works better for specific distributions:

• using squared deviations works better for distributions for whom the scale parameter is closely related to the squared deviations from the location (duh). This is for example the normal distribution where the scale parameter is equal to the variance of the samples (when the number of samples approaches inf). Others include
• using absolute deviations works better for distributions for whom the scale parameter is more closely related to the absolute deviations. This includes the GEV and the Laplace distribution.

I tested it and indeed the first guess after this step improves for a normal distribution when switching to the squared deviations but worsens for the GEV. However, we do another fit for all parameters after this where the first guess is further improved and this worked well for my examples. Thus, I would not change anything for now, I just wanted to write this down to show the tradeoffs we are facing.