Snow?

[amh15]

It would be interesting to see photos in your dataset that feature snow on the ground. I suspect this might defeat the algorithm. A highly relevant application of sky detection is orientation detection in the new(ish) generation of 360 cameras and the footage from these devices often features snow. Perhaps also sand dunes?