Silent Corruption of UT8-Encoded Files missidentified as UTF8-BOM

[MatthiasWinkelmann]

Issue Type: Bug

Opening an UTF-8 encoded file with the following content (3 lines, dashed lines not part of the file):

-----------
({ä üu, öo})
({ä üu, öo})

----------

The file encoding is guessed as "UTF8 (BOM)", even though the first three bytes do not include the BOM sequence. On saving, the first few bytes of the file are changed by prepending the BOM (\uFEFF):

pry> puts Pathname("before").read.dump
"({\u00E4 \u00FCu, \u00F6o})\n({\u00E4 \u00FCu, \u00F6o})\n"

pry> puts Pathname("after").read.dump
"\uFEFF({\u00E4 \u00FCu, \u00F6o})\n({\u00E4 \u00FCu, \u00F6o})\n"
 ^^^^^^^

The resulting file appears identical, but (at least in my case) leads to errors downstream. The Crystal compiler, for example, will complain about an "unknown function 'require' in line 1".

Possibly related to #84495 #84504, tagging @bpasero

VS Code version: Code - Insiders 1.41.0-insider (bf7d03b, 2019-11-15T05:37:27.862Z)
OS version: Darwin x64 18.2.0

System Info

Item	Value
CPUs	Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz (8 x 4009)
GPU Status	2d_canvas: enabled flash_3d: enabled flash_stage3d: enabled flash_stage3d_baseline: enabled gpu_compositing: enabled metal: disabled_off multiple_raster_threads: enabled_on oop_rasterization: unavailable_off protected_video_decode: unavailable_off rasterization: unavailable_off skia_renderer: disabled_off surface_control: disabled_off surface_synchronization: enabled_on video_decode: enabled viz_display_compositor: enabled_on viz_hit_test_surface_layer: disabled_off webgl: enabled webgl2: enabled
Load (avg)	2, 2, 2
Memory (System)	32.00GB (0.14GB free)
Process Argv	--disable-extensions o.cr
Screen Reader	no
VM	0%

Extensions disabled

[bpasero]

Great catch 👍