Support bfloat16 execution

[guillaumekln]

Models that are trained with bfloat16 can have numerical issues when run with float16. See #1074 for example.

We should consider supporting bfloat16 execution which is supported on recent Intel CPUs and NVIDIA GPUs.

[wsxiaoys]

To add another potential related data point of this, I converted SantaCoder ( a float16 trained model) to ctranslate2 for inference, when combined with beam_size = 1:

1. int8 / int8_float16 generate reasonble outputs.
2. float16 generate unusable outputs.

Converted model locates here: https://huggingface.co/TabbyML/SantaCoder-1B/tree/main/ctranslate2

The reported issue: TabbyML/tabby#236

[guillaumekln]

The linked PR is adding support for bfloat16.

Just like other types, you can select it during conversion with --quantization bfloat16 or when loading the model with compute_type="bfloat16". The quantization mode int8_bfloat16 is also supported.

It would be helpful if anyone watching this issue can test the implementation and give feedback. To install the development build:

1. Go to this build page
2. Download the artifact "python-wheels"
3. Extract the archive
4. Install the wheel matching your system and Python version with pip install --force-reinstall <wheel>

Note that a GPU with Compute Capability 8 or greater is required.