Skip to content

Commit

Permalink
docs: update customization
Browse files Browse the repository at this point in the history
  • Loading branch information
lyie28 committed May 21, 2024
1 parent 21983f0 commit 38ab2aa
Showing 1 changed file with 7 additions and 6 deletions.
13 changes: 7 additions & 6 deletions docs/docs/get-started/customization.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,17 +5,18 @@ Our Web Agents have the following customizable elemnets: LLM, multi-modal LLM (`
These are the attributes of a `Context` object, which can be passed to the Action Engine and World Model.

!!! info "Modifiable elements"
- `llm`: The `LLM` used by the `Action Engine` to translate text instructions into automation code. You can set the `llm` to any `LlamaIndex LLM object`.

- `mm_llm`: The `multi-modal LLM` used by the `World Model` to generate the next instruction to be enacted by the Action Engine based on the current state of the web page. You can set the `mm_llm` argument to any `LlamaIndex multi-modal LLM object`.
- **llm**: The `LLM` used by the `Action Engine` to translate text instructions into automation code. You can set the `llm` to any `LlamaIndex LLM object`.

- `embedding`: The `embedding model` is used by the `retriever` to convert segments of the HTML page of the target website into vectors, capturing semantic meaning. You can set this to any `LlamaIndex Embedding object`.
- **mm_llm**: The `multi-modal LLM` used by the `World Model` to generate the next instruction to be enacted by the Action Engine based on the current state of the web page. You can set the `mm_llm` argument to any `LlamaIndex multi-modal LLM object`.

- `retriever`: The `retriever` is used within the `Action Model` to retrieve the most relevant HTML source code of the webpage to be able to generate the automation code targetting HTML elements. This can be any `LlamaIndex.retriever`.
- **embedding**: The `embedding model` is used by the `retriever` to convert segments of the HTML page of the target website into vectors, capturing semantic meaning. You can set this to any `LlamaIndex Embedding object`.

- `prompt_template`: The prompt template used by the `Action Engine` to query the `LLM` in order to generate automation code for an instruction. You can replace this with your own prompt template as a string.
- **retriever**: The `retriever` is used within the `Action Model` to retrieve the most relevant HTML source code of the webpage to be able to generate the automation code targetting HTML elements. This can be any `LlamaIndex.retriever`.

- `extractor`: The `cleaning function` applied to the automation code generated by the LLM in the `Action Engine`. You can replace this with your own custom method as a callable.
- **prompt_template**: The prompt template used by the `Action Engine` to query the `LLM` in order to generate automation code for an instruction. You can replace this with your own prompt template as a string.

- **extractor**: The `cleaning function` applied to the automation code generated by the LLM in the `Action Engine`. You can replace this with your own custom method as a callable.

These elements are initialized in a `Context` object, which can optionally passed to both the `Action Engine` and `World Model` used by an Agent. If you don't pass them your own Context object, the default OpenaiContext will be used.

Expand Down

0 comments on commit 38ab2aa

Please sign in to comment.