CSVs encoded UTF-8 with BOM fail silently

[br8n]

Diana Perpich from the University Library found this issue and was really helpful in showing me how to explain this.

Diana created a CSV to add multiple students to a site with CCM, but the operation never completed. It would pass the error check in the GUI and proceed. Then, a few minutes later, she would receive an email message from CCM explaining it was taking longer than usual. But it would never finish.

I examined her sample file in a text editor and compared it to one I made with the same students and sections and realized Diana's was UTF-8 with BOM -- when you create a CSV file in Excel, it's all too easy to save it as a CSV UTF-8 with Byte Order Mark.

Although there are a lot of StackExchange threads about BOM messing up Python and other programs, the file still passed CCM's error check.

I think CCM should either accept CSVs in UTF-8 with BOM or the error check should catch the BOM and refuse it. Either way we should probably advise users in the documentation to make sure that, if they're using Excel, they're only saving as plain Comma Separate Values format. Something like:

"In Excel you must save as 'Comma Separate Values (.csv)'. The similar and more common 'CSV Comma delimited (.csv)' file format will appear to work but will fail completely.”