Skip to content

Latest commit

 

History

History
260 lines (158 loc) · 6.98 KB

nemoguardrails.rails.llm.llmrails.md

File metadata and controls

260 lines (158 loc) · 6.98 KB
Raw

module nemoguardrails.rails.llm.llmrails

LLM Rails entry point.

Global Variables

  • explain_info_var
  • streaming_handler_var

class LLMRails

Rails based on a given configuration.

method LLMRails.__init__

__init__(
    config: nemoguardrails.rails.llm.config.RailsConfig,
    llm: Optional[langchain.llms.base.BaseLLM] = None,
    verbose: bool = False
)

Initializes the LLMRails instance.

Args:

  • config: A rails configuration.
  • llm: An optional LLM engine to use.
  • verbose: Whether the logging should be verbose or not.

method LLMRails.explain

explain() → ExplainInfo

Helper function to return the latest ExplainInfo object.

method LLMRails.generate

generate(prompt: Optional[str] = None, messages: Optional[List[dict]] = None)

Synchronous version of generate_async.

method LLMRails.generate_async

generate_async(
    prompt: Optional[str] = None,
    messages: Optional[List[dict]] = None,
    streaming_handler: Optional[nemoguardrails.streaming.StreamingHandler] = None
) → Union[str, dict]

Generate a completion or a next message.

The format for messages is the following:

     [
         {"role": "context", "content": {"user_name": "John"}},
         {"role": "user", "content": "Hello! How are you?"},
         {"role": "assistant", "content": "I am fine, thank you!"},
         {"role": "event", "event": {"type": "UserSilent"}},
         ...
     ]

Args:

  • prompt: The prompt to be used for completion.
  • messages: The history of messages to be used to generate the next message.
  • streaming_handler: If specified, and the config supports streaming, the provided handler will be used for streaming.

Returns: The completion (when a prompt is provided) or the next message.

System messages are not yet supported.

method LLMRails.generate_events

generate_events(events: List[dict]) → List[dict]

Synchronous version of LLMRails.generate_events_async.

method LLMRails.generate_events_async

generate_events_async(events: List[dict]) → List[dict]

Generate the next events based on the provided history.

The format for events is the following:

     [
         {"type": "...", ...},
         ...
     ]

Args:

  • events: The history of events to be used to generate the next events.

Returns: The newly generate event(s).

method LLMRails.register_action

register_action(
    action: <built-in function callable>,
    name: Optional[str] = None
)

Register a custom action for the rails configuration.

method LLMRails.register_action_param

register_action_param(name: str, value: Any)

Registers a custom action parameter.

method LLMRails.register_embedding_search_provider

register_embedding_search_provider(
    name: str,
    cls: Type[nemoguardrails.embeddings.index.EmbeddingsIndex]
) → None

Register a new embedding search provider.

Args:

  • name: The name of the embedding search provider that will be used.
  • cls: The class that will be used to generate and search embedding

method LLMRails.register_filter

register_filter(
    filter_fn: <built-in function callable>,
    name: Optional[str] = None
)

Register a custom filter for the rails configuration.

method LLMRails.register_output_parser

register_output_parser(output_parser: <built-in function callable>, name: str)

Register a custom output parser for the rails configuration.

method LLMRails.register_prompt_context

register_prompt_context(name: str, value_or_fn: Any)

Register a value to be included in the prompt context.

:name: The name of the variable or function that will be used. :value_or_fn: The value or function that will be used to generate the value.

method LLMRails.stream_async

stream_async(
    prompt: Optional[str] = None,
    messages: Optional[List[dict]] = None
) → AsyncIterator[str]

Simplified interface for getting directly the streamed tokens from the LLM.