Skip to content
This repository has been archived by the owner on Nov 16, 2023. It is now read-only.

Docs for Multiclass Classification Metrics are wrong. #501

Open
antoniovs1029 opened this issue Nov 16, 2020 · 0 comments
Open

Docs for Multiclass Classification Metrics are wrong. #501

antoniovs1029 opened this issue Nov 16, 2020 · 0 comments

Comments

@antoniovs1029
Copy link
Member

The docs on https://docs.microsoft.com/en-us/nimbusml/concepts/metrics#multiclass-classification-metrics are wrong since it states the following:

This corresponds to evaltype=’multiclass’.

[...]

(class N) - Accuracy of class N

Problem is that the (class N) columns that are the output of the evaluation aren't Per class Accuracies but actually Per class Log Loss metrics.

I've stepped into from ML.NET to NimbusML and this was the case.

For instance, ML.NET's MulticlassClassificationMetrics object doesn't hold a PerClassAccuracy member, but it does PerClassLogLoss

https://docs.microsoft.com/en-us/dotnet/api/microsoft.ml.data.multiclassclassificationmetrics?view=ml-dotnet

And in ML.NET's multiclass evaluator there's no code to calculate per class accuracy, only per class log loss:
https://github.com/dotnet/machinelearning/blob/c255ac7c45af7a19a6e5ab1e26fdbe44874f241a/src/Microsoft.ML.Data/Evaluators/MulticlassClassificationEvaluator.cs#L298-L308

So I don't know if maybe in the past ML.NET/NimbusML actually output per class accuracies, but at least now it does seem the docs are wrong and would need to be fixed. Thanks.

@antoniovs1029 antoniovs1029 changed the title Docs of Multiclass Classification Metrics are wrong. Docs for Multiclass Classification Metrics are wrong. Nov 16, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant