feat(collection): add concat (#748)

Because - we need to concat 2 arrays This commit - add concat task in collection - add recipe samples --------- Co-authored-by: Chang, Hui-Tang <[email protected]>
instill-ai · Oct 18, 2024 · 04d1467 · 04d1467
1 parent 952902c
commit 04d1467
Show file tree

Hide file tree

Showing 58 changed files with 744 additions and 14 deletions.
diff --git a/.github/workflows/test-compogen.yml b/.github/workflows/test-compogen.yml
@@ -34,4 +34,3 @@ jobs:
           cd pkg/component/tools/compogen
           go mod tidy
           go test -mod=mod -race -cover ./...
-
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -5,6 +5,7 @@ repos:
       - id: check-yaml
       - id: end-of-file-fixer
         exclude: (?i).*testdata/
+        exclude_types: [svg, mdx]
       - id: trailing-whitespace
         exclude: (?i).*testdata/
   - repo: https://github.com/dnephin/pre-commit-golang

diff --git a/pkg/component/ai/anthropic/v0/README.mdx b/pkg/component/ai/anthropic/v0/README.mdx
@@ -9,14 +9,21 @@ The Anthropic component is an AI component that allows users to connect the AI m
 It can carry out the following tasks:
 - [Text Generation Chat](#text-generation-chat)
 
+
+
 ## Release Stage
 
 `Alpha`
 
+
+
 ## Configuration
 
 The component definition and tasks are defined in the [definition.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/ai/anthropic/v0/config/definition.json) and [tasks.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/ai/anthropic/v0/config/tasks.json) files respectively.
 
+
+
+
 ## Setup
 
 
@@ -122,6 +129,8 @@ The image URL
 | Output Tokens | `output-tokens` | number | The output tokens used by Anthropic |
 </div>
 </details>
+
+
 ## Example Recipes
 
 Recipe for the [List GitHub Repo Issues](https://instill.tech/instill-ai/pipelines/github-demo/playground) pipeline.

diff --git a/pkg/component/ai/cohere/v0/README.mdx b/pkg/component/ai/cohere/v0/README.mdx
@@ -11,14 +11,21 @@ It can carry out the following tasks:
 - [Text Embeddings](#text-embeddings)
 - [Text Reranking](#text-reranking)
 
+
+
 ## Release Stage
 
 `Alpha`
 
+
+
 ## Configuration
 
 The component definition and tasks are defined in the [definition.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/ai/cohere/v0/config/definition.json) and [tasks.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/ai/cohere/v0/config/tasks.json) files respectively.
 
+
+
+
 ## Setup
 
 
@@ -221,3 +228,5 @@ Rerank models sort text inputs by semantic relevance to a specified query. They
 | Search Counts | `search-counts` | number | The search count used by Cohere Models |
 </div>
 </details>
+
+
diff --git a/pkg/component/ai/fireworksai/v0/README.mdx b/pkg/component/ai/fireworksai/v0/README.mdx
@@ -10,14 +10,21 @@ It can carry out the following tasks:
 - [Text Generation Chat](#text-generation-chat)
 - [Text Embeddings](#text-embeddings)
 
+
+
 ## Release Stage
 
 `Alpha`
 
+
+
 ## Configuration
 
 The component definition and tasks are defined in the [definition.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/ai/fireworksai/v0/config/definition.json) and [tasks.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/ai/fireworksai/v0/config/tasks.json) files respectively.
 
+
+
+
 ## Setup
 
 
@@ -163,6 +170,8 @@ An embedding is a list of floating point numbers that captures semantic informat
 | Token Count | `tokens` | number | The token count used by Fireworks AI models |
 </div>
 </details>
+
+
 ## Example Recipes
 
 Recipe for the [Fireworks Chinese Content Writer](https://instill.tech/instill-ai/pipelines/fireworks-chinese-content-writer/playground) pipeline.

diff --git a/pkg/component/ai/groq/v0/README.mdx b/pkg/component/ai/groq/v0/README.mdx
@@ -9,14 +9,21 @@ The Groq component is an AI component that allows users to connect the AI models
 It can carry out the following tasks:
 - [Text Generation Chat](#text-generation-chat)
 
+
+
 ## Release Stage
 
 `Alpha`
 
+
+
 ## Configuration
 
 The component definition and tasks are defined in the [definition.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/ai/groq/v0/config/definition.json) and [tasks.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/ai/groq/v0/config/tasks.json) files respectively.
 
+
+
+
 ## Setup
 
 
@@ -124,6 +131,8 @@ The image URL
 | Output Tokens | `output-tokens` | number | The output tokens generated by GroqCloud OSS models |
 </div>
 </details>
+
+
 ## Example Recipes
 
 Recipe for the [Groq Interview Helper](https://instill.tech/instill-ai/pipelines/groq-interview-helper/playground) pipeline.

diff --git a/pkg/component/ai/huggingface/v0/README.mdx b/pkg/component/ai/huggingface/v0/README.mdx
@@ -25,14 +25,21 @@ It can carry out the following tasks:
 - [Speech Recognition](#speech-recognition)
 - [Audio Classification](#audio-classification)
 
+
+
 ## Release Stage
 
 `Alpha`
 
+
+
 ## Configuration
 
 The component definition and tasks are defined in the [definition.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/ai/huggingface/v0/config/definition.json) and [tasks.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/ai/huggingface/v0/config/tasks.json) files respectively.
 
+
+
+
 ## Setup
 
 
@@ -913,3 +920,5 @@ Audio classification is the task of assigning a label or class to a given audio.
 | Score | `score` | number | A float that represents how likely it is that the audio file belongs to this class. |
 </div>
 </details>
+
+
diff --git a/pkg/component/ai/instill/v0/README.mdx b/pkg/component/ai/instill/v0/README.mdx
@@ -19,16 +19,23 @@ It can carry out the following tasks:
 - [Visual Question Answering](#visual-question-answering)
 - [Chat](#chat)
 
+
+
 ## Release Stage
 
 `Alpha`
 
+
+
 ## Configuration
 
 The component definition and tasks are defined in the [definition.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/ai/instill/v0/config/definition.json) and [tasks.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/ai/instill/v0/config/tasks.json) files respectively.
 
 
 
+
+
+
 ## Supported Tasks
 
 ### Classification
@@ -574,3 +581,5 @@ The image URL
 | :--- | :--- | :--- | :--- |
 | Text | `text` | string | Text |
 </div>
+
+
diff --git a/pkg/component/ai/mistralai/v0/README.mdx b/pkg/component/ai/mistralai/v0/README.mdx
@@ -10,14 +10,21 @@ It can carry out the following tasks:
 - [Text Generation Chat](#text-generation-chat)
 - [Text Embeddings](#text-embeddings)
 
+
+
 ## Release Stage
 
 `Alpha`
 
+
+
 ## Configuration
 
 The component definition and tasks are defined in the [definition.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/ai/mistralai/v0/config/definition.json) and [tasks.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/ai/mistralai/v0/config/tasks.json) files respectively.
 
+
+
+
 ## Setup
 
 
@@ -164,6 +171,8 @@ An embedding is a list of floating point numbers that captures semantic informat
 | Token Count | `tokens` | number | The token count used by Mistral models |
 </div>
 </details>
+
+
 ## Example Recipes
 
 Recipe for the [Short-film Script Writer](https://instill.tech/instill-ai/pipelines/mistral-demo/playground) pipeline.

diff --git a/pkg/component/ai/ollama/v0/.compogen/setup-hosting.mdx b/pkg/component/ai/ollama/v0/.compogen/setup-hosting.mdx
@@ -10,4 +10,4 @@ To set up an Ollama instance on your local machine, follow the instructions belo
     - On Linux and macOS, open the terminal and type `ifconfig`.
     - On Windows, open the command prompt and type `ipconfig`.
 4. Suppose the IP address is `192.168.178.88`, then the Ollama hosting endpoint would be `192.168.178.88:11434`.
-5. Enjoy fast LLM inference on your local machine and integration with 💧 Instill VDP.
+5. Enjoy fast LLM inference on your local machine and integration with 💧 Instill VDP.
diff --git a/pkg/component/ai/ollama/v0/README.mdx b/pkg/component/ai/ollama/v0/README.mdx
@@ -10,14 +10,21 @@ It can carry out the following tasks:
 - [Text Generation Chat](#text-generation-chat)
 - [Text Embeddings](#text-embeddings)
 
+
+
 ## Release Stage
 
 `Alpha`
 
+
+
 ## Configuration
 
 The component definition and tasks are defined in the [definition.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/ai/ollama/v0/config/definition.json) and [tasks.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/ai/ollama/v0/config/tasks.json) files respectively.
 
+
+
+
 ## Setup
 
 
@@ -122,6 +129,7 @@ To set up an Ollama instance on your local machine, follow the instructions belo
     - On Windows, open the command prompt and type `ipconfig`.
 4. Suppose the IP address is `192.168.178.88`, then the Ollama hosting endpoint would be `192.168.178.88:11434`.
 5. Enjoy fast LLM inference on your local machine and integration with 💧 Instill VDP.
+
 ### Text Embeddings
 
 An embedding is a list of floating point numbers that captures semantic information about the text that it represents.
@@ -146,3 +154,5 @@ An embedding is a list of floating point numbers that captures semantic informat
 | :--- | :--- | :--- | :--- |
 | Embedding | `embedding` | array[number] | Embedding of the input text |
 </div>
+
+
diff --git a/pkg/component/ai/openai/v0/README.mdx b/pkg/component/ai/openai/v0/README.mdx
@@ -13,14 +13,21 @@ It can carry out the following tasks:
 - [Text to Speech](#text-to-speech)
 - [Text to Image](#text-to-image)
 
+
+
 ## Release Stage
 
 `Alpha`
 
+
+
 ## Configuration
 
 The component definition and tasks are defined in the [definition.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/ai/openai/v0/config/definition.json) and [tasks.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/ai/openai/v0/config/tasks.json) files respectively.
 
+
+
+
 ## Setup
 
 
@@ -290,6 +297,8 @@ Generate or manipulate images with DALL·E.
 | Revised Prompt | `revised-prompt` | string | Revised prompt |
 </div>
 </details>
+
+
 ## Example Recipes
 
 Recipe for the [PicassoAI: Cubist Creations at Your Command!](https://instill.tech/instill-ai/pipelines/picasso-ai/playground) pipeline.

diff --git a/pkg/component/ai/stabilityai/v0/README.mdx b/pkg/component/ai/stabilityai/v0/README.mdx
@@ -10,14 +10,21 @@ It can carry out the following tasks:
 - [Text to Image](#text-to-image)
 - [Image to Image](#image-to-image)
 
+
+
 ## Release Stage
 
 `Alpha`
 
+
+
 ## Configuration
 
 The component definition and tasks are defined in the [definition.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/ai/stabilityai/v0/config/definition.json) and [tasks.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/ai/stabilityai/v0/config/tasks.json) files respectively.
 
+
+
+
 ## Setup
 
 
@@ -115,3 +122,5 @@ Modify an image based on a text prompt.
 | Images | `images` | array[string] | Generated images |
 | Seeds | `seeds` | array[number] | Seeds of generated images |
 </div>
+
+
diff --git a/pkg/component/ai/universalai/v0/README.mdx b/pkg/component/ai/universalai/v0/README.mdx
@@ -9,14 +9,21 @@ The Universal AI component is an AI component that allows users to connect the A
 It can carry out the following tasks:
 - [Chat](#chat)
 
+
+
 ## Release Stage
 
 `Alpha`
 
+
+
 ## Configuration
 
 The component definition and tasks are defined in the [definition.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/ai/universalai/v0/config/definition.json) and [tasks.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/ai/universalai/v0/config/tasks.json) files respectively.
 
+
+
+
 ## Setup
 
 
@@ -163,3 +170,5 @@ Input parameter
 | Total Tokens | `total-tokens` | integer | Total number of tokens used in the request (prompt + completion). |
 </div>
 </details>
+
+
diff --git a/pkg/component/application/asana/v0/README.mdx b/pkg/component/application/asana/v0/README.mdx
@@ -12,14 +12,21 @@ It can carry out the following tasks:
 - [Crud Project](#crud-project)
 - [Crud Portfolio](#crud-portfolio)
 
+
+
 ## Release Stage
 
 `Alpha`
 
+
+
 ## Configuration
 
 The component definition and tasks are defined in the [definition.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/application/asana/v0/config/definition.json) and [tasks.json](https://github.com/instill-ai/pipeline-backend/blob/main/pkg/component/application/asana/v0/config/tasks.json) files respectively.
 
+
+
+
 ## Setup
 
 
@@ -635,3 +642,5 @@ Create Portfolio
 | User Name | `name` | string | The name of the user. |
 </div>
 </details>
+
+