[FEATURE] More variation for default use cases

[amitgalitz]

Is your feature request related to a problem?

Currently the only way for customers to have more variation on the current templates we offer is through adding another template, or through substituting the given defaults on the domain:

POST /_plugins/_flow_framework/workflow?use_case=semantic_search_with_cohere_embedding
{
    "create_connector.model" : "embed-multilingual-v3.0",
    "text_embedding.field_map.output": "book_embedding",
    "create_index.name": "sparse-book-index"
}

Some users might want to have the ability to give a lot more settings for the create index step of a template or add 2 more unique nested field on their index to test their hybrid queries.

example substitution ready create index step:

{
            "index_name": "${{create_index.name}}",
            "configurations": {
              "settings": {
                "index.knn": true,
                "default_pipeline": "${{create_ingest_pipeline.pipeline_id}}",
                "number_of_shards": "${{create_index.settings.number_of_shards}}",
                "index.search.default_pipeline": "${{create_search_pipeline.pipeline_id}}"
              },
              "mappings": {
                "_doc": {
                  "properties": {
                    "id": {
                      "type": "text"
                    },
                    "${{text_embedding.field_map.output}}": {
                      "type": "knn_vector",
                      "dimension": "${{text_embedding.field_map.output.dimension}}",
                      "method": {
                        "engine": "${{create_index.mappings.method.engine}}",
                        "space_type": "${{create_index.mappings.method.space_type}}",
                        "name": "${{create_index.mappings.method.name}}",
                        "parameters": {}
                      }
                    },
                    "${{text_embedding.field_map.input}}": {
                      "type": "text"
                    }
                  }
                }
              }
            }

users might want to add one more field to this index for example ^

Additionally users have already asked if they can use the hybrid_search use case but without any weights at all. However we currently don't offer substitution for the entire field if its already in the pre-defined template. Users can only overwrite the actual value of the weights field.

What solution would you like?

We should think of a mechanism where it is to substitute in entire steps of a template or for example have additional pre-defined templates where if user's don't specify any substitution we proceed with the template we have today but if they give us their entire index mapping we can just substitute that entire body.

For the request on not having weights at all, a simple solution could be to have two templates for hybrid_search on the backend but only expose one use case to users hybrid_search and if the users don't define any weights in API request then we will now use the template with no weights and vice versa.