master-f685aa4

Fix "'NoneType' object has no attribute 'cast'" error when model is f…

master-25ee75e

Expose n_vocab, n_embed, n_layer to the Python interface (#118)

master-84634c0

Elide logits if the logits pointer parameter is NULL (#107)

* Completely skip calculation of logits if nobody cares

This speeds up sequence mode evaluations by up to 20% if you ingest
a large prompt and then only retrieve the logits at the very end.

Note that you must pass a NULL pointer to the logits parameter in
order to take advantage of this optimization.

* logits_out=NULL documentation

master-ffc085c

Update GGML (#103)

* Update GGML

* Fix linux build

Of course we forgot why we did this, and broke the build again, in
the exact same way, a second time.

* Fix cuBLAS

Properly set the backend and then call ggml_cuda_transform_tensor

* Rename xx to x_prev

probably should slip this in now before we forget it's a thing.

* See how easy updates are now? (update GGML)

master-9cbb9d9

Various improvements (#104)

* Make rwkv_gpu_offload_layers return true only if layers were actually offloaded

* Validate device of tensors

* Offload all layers during test

* Consistently use FP16 and FP32 instead of float16/fp16/F16/etc.

* Use spaces for indentation

* Remove spaces between type name and []

* Add cuBLAS on Windows guide, refactor docs structure

* Insert replacement characters when decoding invalid UTF-8 sequences

* Fix compatibility

* Fix formatting

* Fix copy-pasted tensor validation

master-6b26e0d

Add Python support for sequence mode (#101)

master-5316068

fix static linking for tests and extras, remove unneeded -static flag…

master-15b7c7b

add standalone build option (#99)

* add standalone build option

* Update CMakeLists.txt for more clarity in comment

Co-authored-by: Alex <[email protected]>

* add endofline properly for right formating

---------

Co-authored-by: Alex <[email protected]>

master-c64009e

Fix typo in rwkv.h docs for n_vocab (#96)

World models actually have 65536, not 65535, oops

master-bd65c97

Make sampling with bias numerically stable (#90)

* Update sampling.py

Remove a slow for loop on logit bias. Make the numpy re-softmax operation numerically stable.

* Update sampling.py