Nepali Image captioning system

Image dataset used in this project is from FLicker8K dataset.
Used Keras with Tensorflow backend for the code.
The loss value of 0.5 and accuracy of 0.8381 has been achieved which gives good results.
You can check out some examples in notebook file.
Pretrained weights are available HERE

Dependencies

1)glob==0.6
2)h5py==2.8.0
3)keras==1.2.2
4)matplotlib==3.0.2
5)numpy==1.14.6
6)pandas==0.22.0
7)tensorflow=1.x

There are many problem for nepali_captioning:

1)There is equal chance of having a faulty translaton for 40 thousand sentence we are using in nepali captions.
2)There is no trusted and processed globe text file for nepali.
3)As nepali has 36 fundamental latters with different notation(aakar,ukar) prediciton is much more difficult.
4)Nepali grammar is more complex, it comprises with more rules that should be taken care for like aadhar etc
As, this is probably first nepali captioning system. We, tried our best for good prediction.

Examples

References

[1] Oriol Vinyals, Alexander Toshev, Samy Bengio, Dumitru Erhan Show and Tell: A Neural Image Caption Generator
[2] M. Hodosh, P. Young and J. Hockenmaier (2013) "Framing Image Description as a Ranking Task: Data, Models and Evaluation Metrics", Journal of Artificial Intelligence Research, Volume 47, pages 853-899 http://www.jair.org/papers/paper3994.html
[3] CS231n Winter 2016 Lesson 10 Recurrent Neural Networks, Image Captioning and LSTM https://youtu.be/cO0a0QYmFm8

Contact

For any query you can contact here.