AI Assistant docs updates for 8.16

docs/en/observability/observability-ai-assistant.asciidoc

@@ -39,6 +39,7 @@ The AI assistant requires the following:
 ** OpenAI `gpt-4`+.
 ** Azure OpenAI Service `gpt-4`(0613) or `gpt-4-32k`(0613) with API version `2023-07-01-preview` or more recent.
 ** AWS Bedrock, specifically the Anthropic Claude models.
+* An {enterprise-search-ref}/server.html[Enterprise Search] server if {enterprise-search-ref}/connectors.html[Search Connectors] are used to populate the knowledge base.
 * The knowledge base requires a 4 GB {ml} node.
 
 [IMPORTANT]
@@ -93,7 +94,7 @@ To set up the AI Assistant:
 any knowledge base articles you created before 8.12 will have to be reindexed or upgraded before they can be used.
 Knowledge base articles created before 8.12 use ELSER v1.
 In 8.12, knowledge base articles must use ELSER v2.
-You can either:
+Options include:
 
 * Clear all old knowledge base articles manually and reindex them.
 * Upgrade all knowledge base articles indexed with ELSER v1 to ELSER v2 using a https://github.com/elastic/elasticsearch-labs/blob/main/notebooks/model-upgrades/upgrading-index-to-use-elser.ipynb[Python script].
@@ -103,13 +104,17 @@ The AI Assistant uses {ml-docs}/ml-nlp-elser.html[ELSER], Elastic's semantic sea
 
 NOTE: Your AI provider may collect telemetry when using the AI Assistant. Contact your AI provider for information on how data is collected.
 
-You can add information to the knowledge base by asking the AI Assistant to remember something while chatting (for example, "remember this for next time"). The assistant will create a summary of the information and add it to the knowledge base.
+External data can be added to the knowledge base through the following methods:
 
-You can also add external data to the knowledge base either in {kib} using the Stack Management UI or using the {es} Index API.
+* The **Knowledge base** UI available at <<obs-ai-settings>> page.
+* Configuring {enterprise-search-ref}/connectors.html[**Search Connectors**] from the <<obs-ai-settings>> page.
+* Manually using **{es} Index and Reindex APIs** (requires advanced skills).
+
+It's also possible to add information to the knowledge base by asking the AI Assistant to remember something while chatting (for example, "remember this for next time"). The assistant will create a summary of the information and add it to the knowledge base.
 
 [discrete]
-[[obs-ai-stack-management]]
-=== Use the UI
+[[obs-ai-kb-ui]]
+=== Use the knowledge base UI
 
 To add external data to the knowledge base in {kib}:
 
@@ -126,16 +131,46 @@ To add external data to the knowledge base in {kib}:
 ----
 {
   "id": "a_unique_human_readable_id",
-  "text": "Contents of item",
+  "text": "Contents of item"
 }
 ----
 
+[discrete]
+[[obs-ai-search-connectors]]
+=== Use Search Connectors
+
+[TIP]
+====
+{enterprise-search-ref}/connectors.html[Search Connectors] differ from the {kibana-ref}/action-types.html[Stack Connectors] configured during the <<obs-ai-set-up, AI Assistant setup>>.
+Search Connectors are only needed when importing external data into the Knowledge base of the AI Assistant, while the Stack Connector to the LLM is required for the AI Assistant to work.
+====
+
+
+{enterprise-search-ref}/connectors.html[Search Connectors] allow to index content from external sources thereby making it available for the AI Assistant. This can greatly improve the relevance of the AI Assistant’s responses.
+These connectors are managed under *Search* -> *Connectors* in {kib}.
+
+[IMPORTANT]
+====
+Search Connectors require an Enterprise Search server. By default the AI Assistant will query the index for every search connector. Change the setting `observability:aiAssistantSearchConnectorIndexPattern` in the <<obs-ai-settings>> page if you want to modify this behavior.
+====
+
+(*TBD*) To use search connectors:
+
+. Open xxx
+. Create and synchronize connector
+. check that data is available for the AI Assistant
+. (pending to determine exact steps or links)
+
 [discrete]
 [[obs-ai-index-api]]
 === Use the {es} Index API
 
+// EEDUGON note: is this section still worthy when we have the option of Search Connectors?
+// in the dest index below we are pointing directly to an ILM managed index that may not be valid. We should point to an existing ILM rollover alias or a data stream (TBD)
+// IMO this is too advanced to appear here. Maybe a different type of guide.
+
 . Ingest external data (GitHub issues, Markdown files, Jira tickets, text files, etc.) into {es} using the {es} {ref}/docs-index_.html[Index API].
-. Reindex your data into the AI Assistant's knowledge base index by completing the following query in *Management* -> *Dev Tools* in {kib}. Update the following fields before reindexing:
+. Reindex the previously indexed data into the AI Assistant's knowledge base index by completing the following query in *Management* -> *Dev Tools* in {kib}. Update the following fields before reindexing:
 ** `InternalDocsIndex` — name of the index where your internal documents are stored.
 ** `text_field` — name of the field containing your internal documents' text.
 ** `timestamp` — name of the timestamp field in your internal documents.
@@ -173,7 +208,7 @@ POST _reindex
 [[obs-ai-interact]]
 == Interact with the AI Assistant
 
-You can chat with the AI Assistant or interact with contextual insights located throughout {observability}.
+Chat with the AI Assistant or interact with contextual insights located throughout {observability}.
 See the following sections for more on interacting with the AI Assistant.
 
 TIP: After every answer the LLM provides, let us know if the answer was helpful.
@@ -183,10 +218,7 @@ Your feedback helps us improve the AI Assistant!
 [[obs-ai-chat]]
 === Chat with the assistant
 
-Click *AI Assistant* in the upper-right corner of any {observability} application to start the chat:
-
-[role="screenshot"]
-image::images/ai-assistant-button.png[Observability AI assistant preview]
+Click *AI Assistant* icon (*TBD*) in the upper-right corner of any {observability} application to start the chat:
 
 This opens the AI Assistant flyout, where you can ask the assistant questions about your instance:
 
@@ -208,7 +240,7 @@ beta::[]
 
 The AI Assistant uses functions to include relevant context in the chat conversation through text, data, and visual components. Both you and the AI Assistant can suggest functions. You can also edit the AI Assistant's function suggestions and inspect function responses.
 
-You can suggest the following functions:
+Main functions:
 
 [horizontal]
 `alerts`:: Get alerts for {observability}.
@@ -250,13 +282,13 @@ Clicking a prompt generates a message specific to that log entry:
 [role="screenshot"]
 image::images/obs-ai-logs.gif[Observability AI assistant example, 75%]
 
-You can continue a conversation from a contextual prompt by clicking *Start chat* to open the AI Assistant chat.
+Continue a conversation from a contextual prompt by clicking *Start chat* to open the AI Assistant chat.
 
 [discrete]
 [[obs-ai-connector]]
 === Add the AI Assistant connector to alerting workflows
 
-You can use the {kibana-ref}/obs-ai-assistant-action-type.html[Observability AI Assistant connector] to add AI-generated insights and custom actions to your alerting workflows.
+Use the {kibana-ref}/obs-ai-assistant-action-type.html[Observability AI Assistant connector] to add AI-generated insights and custom actions to your alerting workflows as follows:
 To do this:
 
 . <<create-alerts-rules,Create (or edit) an alerting rule>> and specify the conditions that must be met for the alert to fire.
@@ -274,7 +306,7 @@ and also include other active alerts that may be related.
 As a last step, you can ask the assistant to trigger an action,
 such as sending the report (or any other message) to a Slack webhook.
 
-NOTE: Currently you can only send messages to Slack, email, Jira, PagerDuty, or a webhook.
+NOTE: Currently only Slack, email, Jira, PagerDuty, or webhook actions are supported.
 Additional actions will be added in the future.
 
 When the alert fires, contextual details about the event—such as when the alert fired,
@@ -307,6 +339,21 @@ The Observability AI Assistant connector is called when the alert fires and when
 
 To learn more about alerting, actions, and connectors, refer to <<create-alerts>>.
 
+[discrete]
+[[obs-ai-settings]]
+== AI Assistant Settings
+
+Access the AI Assistant Settings page:
+
+* From *{stack-manage-app}* -> _Kibana_ *AI Assistants* -> *Elastic AI Assistant for Observability* `Manage Settings` button.
+* From the `More ations` menu inside the AI Assistant window.
+
+The AI Assistant Settings page contains the following tabs:
+
+* *Settings*: Configure the main AI Assistant settings, which are explained directly within the interface.
+* *Knowledge base*: <<obs-ai-kb-ui,manage knowledge base entries>>. 
+* *Search Connectors*: *TBD*
+
 [discrete]
 [[obs-ai-known-issues]]
 == Known issues
@@ -318,5 +365,5 @@ To learn more about alerting, actions, and connectors, refer to <<create-alerts>
 Most LLMs have a set number of tokens they can manage in single a conversation.
 When you reach the token limit, the LLM will throw an error, and Elastic will display a "Token limit reached" error in Kibana.
 The exact number of tokens that the LLM can support depends on the LLM provider and model you're using.
-If you are using an OpenAI connector, you can monitor token usage in **OpenAI Token Usage** dashboard.
+If you are using an OpenAI connector, monitor token utilization in **OpenAI Token Usage** dashboard.
 For more information, refer to the {kibana-ref}/openai-action-type.html#openai-connector-token-dashboard[OpenAI Connector documentation].