Georeferencer: it is not specified how the RMS error value is calculated and why

[agiudiceandrea]

Description

In the Georeferencer documentation it is not specified how the RMS error value is calculated and why.

See https://gis.stackexchange.com/questions/397377/qgis-mean-error-rmse-qgis-georeferencer.
See also: https://issues.qgis.org/issues/2731 / qgis/QGIS#12791 / qgis/QGIS@4d46f19 by @mhugent.

Some hints:

• https://github.com/qgis/QGIS/blob/ffb31addd7f4d62e61dac7e44a23be6b34e402d6/src/app/georeferencer/qgsgeorefmainwindow.cpp#L1785-L1787

  // Calculate the root mean square error, adjusted for degrees of freedom of the transform
  // Caveat: The number of DoFs is assumed to be even (as each control point fixes two degrees of freedom).
  error = std::sqrt( ( sumVxSquare + sumVySquare ) / ( nPointsEnabled - mGeorefTransform.minimumGcpCount() ) );

• https://github.com/qgis/QGIS/blob/ffb31addd7f4d62e61dac7e44a23be6b34e402d6/src/app/georeferencer/qgsgeorefmainwindow.h#L224-L232

    /**
     * Calculates root mean squared error for the currently active
     * ground control points and transform method.
     * Note that the RMSE measure is adjusted for the degrees of freedom of the
     * used polynomial transform.
     * \param error out: the mean error
     * \returns TRUE in case of success
     */
    bool calculateMeanError( double &error ) const;

• https://github.com/qgis/QGIS/blob/ffb31addd7f4d62e61dac7e44a23be6b34e402d6/src/app/georeferencer/qgsgeorefmainwindow.cpp#L2045

    double residualTot = std::sqrt( residual.x() * residual.x() +  residual.y() * residual.y() );

• https://github.com/qgis/QGIS/blob/ffb31addd7f4d62e61dac7e44a23be6b34e402d6/src/app/georeferencer/qgsgcplist.cpp#L162

                   qgsDoubleToString( std::sqrt( pt->residual().x() * pt->residual().x() + pt->residual().y() * pt->residual().y() ) ) );

Page URL: https://docs.qgis.org/testing/en/docs/user_manual/working_with_raster/georeferencer.html#define-the-transformation-settings