Add @huggingface/ollama-utils

[ngxson]

With the ollama compatibility layer already been up and running on HF hub for a while now, we want to open-source part of the integration for (1) provide more transparency, and (2) for encouraging the community to contribute.

Therefore, we decided to publish @huggingface/ollama-utils, a package containing tools that power this integration.

For now, the only tool that we provide is chat-template.ts, a tool that takes a parsed GGUF config from @huggingface/gguf as input, then returns the ollama Go template.

chat-template.ts

This module expose one single convertGGUFTemplateToOllama function. It works by trying (by order) these mechanisms:

1. Try to find a matched template already exist on ollama hub. To make this possible, we regularly dump a list of public templates from ollama.com using scripts/generate-automap.ts
2. If no exact match is find, we still check against the list of official templates, but now only match a list of special tokens. These tokens are extracted using regex RE_SPECIAL_TOKEN
3. If we still can't find a match, we search inside CUSTOM_TEMPLATE_MAPPING, a list of hand-picked template mapping
4. If all above still failed, we try parsing jinja template and try converting it to Go template. This works almost all of the time, except if the template is not supported by @huggingface/jinja. See convertJinjaToGoTemplate

[coyotte508]

If you feel up to it you can use tshy instead, like in @hugginface/tasks.

It has more accurate exports / better compatibility with all that includes the package (and as long as you don't have different code in browser/node it should be easy to configure)

[ngxson]

I can't get around the error with this line:

src/chat-template.ts:1:43 - error TS1479: The current file is a CommonJS module whose imports will produce 'require' calls; however, the referenced file is an ECMAScript module and cannot be imported with 'require'. Consider writing a dynamic 'import("@huggingface/jinja")' call instead.
  To convert this file to an ECMAScript module, change its file extension to '.mts' or create a local package.json file with `{ "type": "module" }`.

1 import { Template as JinjaTemplate } from "@huggingface/jinja";
                                            ~~~~~~~~~~~~~~~~~~~~

Do you have any ideas @coyotte508 ?

[coyotte508]

Seems like @huggingface/jinja maybe needs to switch to tshy first to all corner cases (early/late node version, CJS importing MJS, MJS importing CJS, ...) are exported correctly cc @xenova

[xenova]

Strange, the exports should operate correctly

Maybe I need to restructure it like transformers.js does it here.

[xenova]

Do any other hf.js libs use tshy?

[coyotte508]

btw the github action to publish should be added (with optionnally a check to check the @huggingface/jinja is up to date, like @huggingface/hub does with @huggingface/tasks)

[julien-c]

neat!

[Vaibhavs10]

don't know enough typescript to review, but two questions/comments from myside:

1. maybe in a follow-up PR we should add a scraper and something to update the mapping?
2. via comments/ README/ docs we should clarify how users can add their own exception as well

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[Vaibhavs10]

yay! conceptually lgtm!

[Vaibhavs10]

maybe we can do both in the same space?

[ngxson]

I couldn't figure out how to embed my script into gradio space, but will have a look later

(The template debugging space is static btw)