Code

Add batch inference support (WIP)#951

Code

Draft

abetlen wants to merge 19 commits intomainabetlen/llama-cpp-python:mainfrom batch-processingabetlen/llama-cpp-python:batch-processingCopy head branch name to clipboard

+150-206

Commits

Commits on Nov 28, 2023

• Clean up llama.py into seperate modules.

  abetlen

  committed
  3c13436

  Copy full SHA for 3c13436

• docs: re-order api reference

  abetlen

  committed
  135b300

  Copy full SHA for 135b300

• Merge branch 'main' into batch-processing

  abetlen

  committed
  b5c33da

  Copy full SHA for b5c33da

• Add sampling context

  abetlen

  committed
  3a1ba77

  Copy full SHA for 3a1ba77

Commits on Nov 29, 2023

• Merge branch 'main' into batch-processing

  abetlen

  committed
  85caba5

  Copy full SHA for 85caba5

• Add vocab utils

  abetlen

  committed
  7ae9a3e

  Copy full SHA for 7ae9a3e

Commits on Nov 30, 2023

• Refactor _create_completion

  abetlen

  committed
  40f2293

  Copy full SHA for 40f2293

Commits on Dec 12, 2023

• Merge branch 'main' into batch-processing

  abetlen

  committed
  4335a9d

  Copy full SHA for 4335a9d

Commits on Dec 18, 2023

• Merge branch 'main' into batch-processing

  abetlen

  committed
  a625412

  Copy full SHA for a625412

• Fix logits are not json serializable

  abetlen

  committed
  fcbd177

  Copy full SHA for fcbd177

Commits on Jan 5, 2024

• Merge branch 'main' into batch-processing

  abetlen

  committed
  b1e9962

  Copy full SHA for b1e9962

• Fix #1038

  abetlen

  committed
  e1cd61e

  Copy full SHA for e1cd61e

Commits on Jan 10, 2024

• Merge branch 'main' into batch-processing

  abetlen

  committed
  456a601

  Copy full SHA for 456a601

• Use sampling context

  abetlen

  committed
  7f4ba48

  Copy full SHA for 7f4ba48

Commits on Jan 12, 2024

• Merge branch 'main' of https://github.com/abetlen/llama-cpp-python into batch-processing

  abetlen

  committed
  7a1c2b5

  Copy full SHA for 7a1c2b5

Commits on Jan 15, 2024

• Merge branch 'main' into batch-processing

  abetlen

  committed
  358593f

  Copy full SHA for 358593f

• Merge branch 'batch-processing' of github.com:abetlen/llama_cpp_python into batch-processing

  abetlen

  committed
  e7ef07d

  Copy full SHA for e7ef07d

Commits on Jan 17, 2024

• Cleanup pyproject

  abetlen

  committed
  6f08021

  Copy full SHA for 6f08021

Commits on Jan 18, 2024

• Merge branch 'main' into batch-processing

  abetlen

  committed
  850416a

  Copy full SHA for 850416a