Training with 1000 sparse labels

[longlnnm]

I was wondering whether you have tried to trained the model with 1000 sparse labels?
We have tried to trained the model with 750 labels. However, the predictions were not very good. Specifically. it will try to predict True for a constant set of labels, and low confidence for the rest. I would be very grateful if you can perhaps share some of your results with us.

[BartyzalRadek]

I have seen your problem before, it was usually caused by imbalanced classes.
This network is not very good in handling such cases, you can try changing training parameters or take a look at methods add_final_training_ops() and add_evaluation_step() in retrain.py.

[longlnnm]

So basically it means that there are some labels with significantly more training data than others?
Has there been a case where the multilabel neural network trained successfully with 1000 classes?

[BartyzalRadek]

1. Yes, solving the problem of imbalanced classes is usually done by either getting more data or generating more data = apply transformations on samples of classes with smaller number of examples.
2. I am not sure, are you using a public dataset? You need at least hundreds (in this case possibly 1000+) of images for each class and the more classes you have, the more problems you'll get. This is certainly not a trivial task.

[longlnnm]

Thank you,
I will see if I can do something about imbalanced class.
I am not using a public dataset. But it should be the similar to the one from Imagenet challenge.

[acuntex]

Just for understanding:
Since I didn't find any proper definition of imbalanced class, only SEO-optimized crap, i guess a balanced dataset would be something like this in the normal single-label solution:

• 500 apples
• 500 bananas
• 500 peaches
  When you use this dataset, the accuracy should be ok, right?

But how does this apply to the multilabel solution?
Sometimes there are dependencies, e.g. if you have the following labels:

• male
• female
• face

When you have a face, you probably always have female or male.

This would mean, you have:

• 100 male
• 100 female
• 200 face

Would this be balanced or imbalanced? Or would you have to add 100 male and 100 female images without a visible face?

[BartyzalRadek]

The multi-label net actually works with 2^N expanded classes (just made that term up) where N is the number of your classes = the length of the ground_truth vector.

In your example with males, females and faces, you want the net to learn to distinguish between the following 8 expanded classes:

• none
• male
• female
• face
• male + face
• female + face
• male + female
• male + female + face

You should ideally provide approximately the same number of examples from all of the expanded classes that you want to classify, which would be called having a balanced dataset. But that rarely happens, which is one of multiple reasons why you don't see many multi-label nets around, they are just too hard to train.

This project is more of a toy to play with to understand whats going on in multi-label classification, if you want to do something serious you will definitely need to look under the hood to solve the many problems that will inevitably pop up.

[acuntex]

Great explanation, thank you!

So i would need also a image class named "none" which has basically anything else which does not include any of the other classes?

So it's probably easier to create 3 neural networks for each class and to aggregate the results of the positives:
1st:

• male
• not_male

2nd:

• female
• not_female

3rd:

• face
• not_face

[BartyzalRadek]

So i would need also a image class named "none" which has basically anything else which does not include any of the other classes?

No, include only positive examples.

So it's probably easier to create 3 neural networks for each class and to aggregate the results of the positives:

I imagine that in this case it would work quite well, yes :)

[acuntex]

No, include only positive examples.

And this would work? I remember some kind of error message when I tried it a year ago like "multiple classes are needed for classification"

[BartyzalRadek]

I meant to include only images of classes that you want to train.

multiple classes are needed for classification

Of course you need to have more than 1 class, otherwise what would you be training.

[fqeqiq168]

if the data need 8 expanded classes: why not use 8 single-label classes ,instead of 3 classes

none
male
female
face
male + face
female + face
male + female
male + female + face

[BartyzalRadek]

Sure, you could train a single label classifier, but then you would not use the added information of actually having just 3 classes of objects. Ideally we want the network to learn to generalize. Meaning that it learns to recognize males, females and faces and even if it never saw an image with all three together it will correctly output >50% probabilities of all of them for such image.

[fqeqiq168]

@BartyzalRadek Thank you.But i use your network to do multi-label classification, the net must be trainned with images [ male + face,female + face,male + female,male + female + face],if not it can't generalize.for example,i only feed images each only has 1 class of object. then i give a test image wihch has 3 classes of object,the net can't output >50% probabilities of all of them for such image.

[Junhyuk]

@BartyzalRadek
shoud I place the folder for making seperate classes ??

[hbarovertwo]

@leminhlong5194 , did you ever solve this problem? I'm attempting to do this with over 1500 labels possible, however I have a pretty large and labeled dataset. I'm facing the issue of same predictions for each image so my next step is to set the weights for each label manually, hoping that solves the problem.

[civilman628]

i have the same issue. the prediction for the image is wrong.