diff --git a/docs/change-log/index.md b/docs/change-log/index.md index 3af8a0e20d50..54d0507720c4 100644 --- a/docs/change-log/index.md +++ b/docs/change-log/index.md @@ -1,6 +1,9 @@ (change-log)= # Change log -- [v1.6.4](#v1-6-4-30-june-2024) | [v1.6.3](#v1-6-3-4-june-2024) | [v1.6.2](#v1-6-2-29-march-2024) | [v1.6.1](#v1-6-1-29-february-2024) | [v1.6.0](#v1-6-0-22-february-2024) + + +- [v1.7.0](#v1-7-0-28-october-2024) +- [v1.6.4](#v1-6-4-2-july-2024) | [v1.6.3](#v1-6-3-4-june-2024) | [v1.6.2](#v1-6-2-29-march-2024) | [v1.6.1](#v1-6-1-29-february-2024) | [v1.6.0](#v1-6-0-22-february-2024) - [v1.5.2](#v1-5-2-30-november-2023) | [v1.5.1](#v1-5-1-2-november-2023) | [v1.5.0](#v1-5-0-23-october-2023) - [v1.4.1](#v1-4-1-8-august-2023) | [v1.4.0](#v1-4-0-23-july-2023) - [v1.3.4](#v1-3-4-23-august-2023) | [v1.3.3](#v1-3-3-7-jun-2023) | [v1.3.2](#v1-3-2-4-jun-2023) | [v1.3.1](#v1-3-1-18-may-2023) | [v1.3.0](#v1-3-0-22-march-2023) @@ -11,6 +14,158 @@ - [Limitations](#limitations) - [Deprecations and removed code](#deprecations-and-removed-code) +## v1.7.0 (28 October 2024) + +### Model monitoring + +```{admonition} Note +{ref}`model-monitoring-overview` is in TechPreview status in v1.7.0. + +``` + +| ID |Description | +|-------|----------------------------------------------------------------------------| +|ML-5460|The model monitoring now has a [per-endpoint view](../model-monitoring/index.html#model-and-model-monitoring-endpoints) that presents information data on the monitoring metrics.| +|ML-5851|Model monitoring supports Kafka or V3IO as streaming platforms, and TDEngine or V3IO as TSDB platforms. See [Selecting the streaming and TSDB platforms](../model-monitoring/index.html#selecting-the-streaming-and-tsdb-platforms).| + +### Alerts +| ID |Description | +|-------|----------------------------------------------------------------------------| +|ML-5287|You can now configure {ref}`alerts ` for model monitoring and other possible problem situations. TechPreview. | + + +### Projects +| ID |Description | +|-------|----------------------------------------------------------------------------| +|ML-3874|Node selector can now be [defined in the project spec](../runtimes/configuring-job-resources.html#node-selection).| + + +### Artifacts +| ID |Description | +|-------|----------------------------------------------------------------------------| +|ML-2585|When deleting a artifact, you can also [delete the data of a single artifact](../store/artifacts.html#deleting-artifacts).| + +### Runtimes +| ID |Description | +|-------|----------------------------------------------------------------------------| +|ML-2652|Supports [creating an API gateway using the MLRun SDK](../concepts/nuclio-real-time-functions.html#api-gateway).| +|ML-4601|New {ref}`application runtime ` where you can provide an image (for example, a web-app) that runs as a Kubernetes deployment. | + + +### Workflows +| ID |Description | +|-------|----------------------------------------------------------------------------| +|ML-6885|You can now add run details to the notification by using `{{ runs }}` in the `override_body` section in notifications. See [Configuring notifications for runs](../concepts/notifications.html#configuring-notifications-for-runs).| + +### Functions +| ID |Description | +|-------|----------------------------------------------------------------------------| +|ML-4248|You can now load models from a json file instead of adding all the models to the function spec.| + +### Serving graph +| ID |Description | +|-------|----------------------------------------------------------------------------| +|ML-6015|Storey/Nuclio serving graph: supports [concurrent processing](../serving/model-serving-get-started.html#concurrent-processing), typically used for serving of deep-learning models, where preparation steps and inference can be CPU/GPU heavy, or involving I/O.| + +### Data store +| ID |Description | +|-------|----------------------------------------------------------------------------| +|ML-5726|Add support for [Hadoop/hdfs datastore](../store/datastore.html#hdfs-data-store-profile).| + +### Notifications +| ID |Description | +|-------|----------------------------------------------------------------------------| +|ML-6644|Explicit control of the configuration for the [Pipeline started](../concepts/notifications.html#configuring-notifications-for-pipelines) notification. | + +### Breaking changes +| ID |Description | +|-------|----------------------------------------------------------------------------| +|ML-5741/3206|The new flag `allow_cross_project` ({py:meth}`mlrun.projects.get_or_create_project` and {py:meth}`mlrun.projects.load_project`) gives you options to manage a situation where you call `get_or_create_project` or `load_project` and there is already a `project.yaml` with a different project name. In v1.7 and v1.8 `allow_cross_project` is set to None by default, meaning if there is a mismatch in the name, you get a warning (that the project name is different than the `project.yaml`) and then the flow continues. If `allow_cross_project` is False, an exception is raised. And if `allow_cross_project` is True, the flow continues. (Prior to v1.9 you should set `allow_cross_project` to either True or False; the None option will be deprecated.)| + +### Feature store +| ID |Description | +|-------|----------------------------------------------------------------------------| +|ML-3303|Optimized the parquet read when the partitioning is on fields other than the timestamp.| +|ML-5656|Add [Snowflake as an offline target store](../feature-store/sources-targets.html#snowflake-target)| + +### UI +| ID |Description | +|-------|----------------------------------------------------------------------------| +|ML-4666|The new cross-project view gives a summary of all jobs, workflows, and schedules that ran in the last 24 hours.

cross-project-view


| +|ML-5140|Improved responsiveness for runs and functions.| +|ML-5846|The Filter in the Projects>ML Functions table is now a popup menu.| +|ML-6275|The Projects dashboard now notifies when MLRun isn't reachable.| +|ML-7103/7131|By default the UI only retrieves tagged functions: non-tagged functions are only retrieved if explicitly asked for. By default the Functions page only shows functions that were modified in the last week: added filters for time. The changes an overall improvement in the responsiveness.| + +### Documentation +| ID |Description | +|-------|----------------------------------------------------------------------------| +|ML-6052|New page: {ref}`log-artifacts`.| +|ML-7480|New topic: [Setup for ARM64 (Apple Silicon) Users](../install/remote.html#note-for-arm64-apple-silicon-users)| +|ML-7669|New topic: [Setting the log level](../runtimes/configuring-job-resources.html#setting-the-log-level).| +|NA |New tutorial: {ref}`genai-02-mm-llm`.| +|NA |New page: {ref}`mm-applications`. +|NA |Updated page: {ref}`model-monitoring-des`.| +|NA |Updated {ref}`ecosystem`.| + +### Closed issues +| ID |Description | +|-------|----------------------------------------------------------------------------| +|ML-2223|Can now deploy a function when notebook names contain "." | +|ML-3143/ML-3432|Can now delete remote (Nuclio) functions from the DB with both the SDK and the UI.| +|ML-3680|Function specs that are modified before running the workflow are now saved.| +|ML-3804|A serving step with no class now inherits parameters from the function spec. | +|ML-4442|You can now add monitoring after a model is deployed (`set_tracking()` was not set on the serving function).| +|ML-4636|A local run that was created via `get_or_create_ctx` can be marked as complete using `commit()` method.| +|ML-4846|CE: `V3IO_ACCESS_KEY` is no longer required for Parquet target.| +|ML-4934|Modifying the parameters of a serving-function (for example changing `default_batch_intervals`) that is configured for model-monitoring tracking does not require a specific workflow. | +|ML-5079|You can now update git remote with `project.create_remote()`.| +|ML-5204|UI: The Project settings now provide validation rules on labels.| +|ML-5774|UI: Improved speed of querying for pipelines of specific projects in the Pipelines page.| +|ML-6020|UI: Copy URI in the Datasets main page now copies the same value as in the detailed Dataset page.| +|ML-6065|Fixed serving graphs when working with Kafka.| +|ML-6194|When running a remote workflow, the client side does not need to contain workflow local files.| +|ML-6068|Feature-store Redis-target is now created by default with the project/feature-set in the path.| +|ML-6249|Reduction in the time that feature sets with a large number of entities take to query (and no longer timeout).| +|ML-6253|The `project.build_function()`now passes the `requirements_file parameter` when the engine is not `kfp`.| +|ML-6602/6556|You can now specify a UID when running `artifacts.get_model`.| +|ML-6800|Resolved spiking of CPU and memory of mlrun-api service.| +|ML-6885|Webhook notification now contains run logs when `override_body=True`| +|ML-6991|UI: Now shows KFP pod errors.| +|ML-7103|UI: New option to show untagged functions in the Functions page (disabled by default). Added time filters to the Functions page, by default shows only functions that were modified in the last week.| +|ML-7119|When attempting to create a project without the required permissions, the error now reads: You don’t have permission to create a project.| +|ML-7135|Fixed upgrading to mlrun if the `spec.build` was saved incorrectly.| +|ML-7162|UI: Erroneous "“No data matches…” messages no longer appear.| +|ML-7192|Improved the liveness and readiness probes responsiveness.| +|ML-7202|The run state now indicates if a pod failed on OOM.| +|ML-7226|Exception now raised when using use KFP engine with `pass schedule` argument, which is not supported with the KFP engine.| +|ML-7256|Deleting artifact by UID deletes only the artifact with the given UID.| +|ML-7290|Nuclio functions deployed by MLRun: At the Nuclio service level, node selectors are added by Nuclio. In cases where there is overlap between the node selectors specified at the MLRun service level and those applied by Nuclio, the node selectors from the MLRun side take precedence to prevent any conflicts. See [node selection](../runtimes/configuring-job-resources.html#node-selection).| +|ML-7335|Serving functions now support up to 4500 models and the size of the serving specification to 1 MB.| +|ML-7358|MLRun client now validates SSL certificates by default.| +|ML-7367|Updated OS packages for MLRun UI.| +|ML-7416|An error message is displayed upon workflow failure in the "Monitor workflows" tab.| +|ML-7599|Fixed mlrun-api vulnerabilities.| +|ML-7613|The pipeline SDK output now displays the logs in addition to the graph animation, and it also raises a warning when a client uses notifications with `engine=kfp`.| +|ML-7655|The "filter by label" in the Models page is now case insensitive.| +|ML-7673|Improved performance of `db.list_pipelines()` with project filter in environments with a large number of pipelines.| +|ML-7706|Change `sync_functions` to be "best effort" when scheduling workflows or running remote workflows.| +|ML-7796|New `project.run()` {py:meth}`workflow_runner_node_selector ` parameter that defines the node selector for the workflow runner pod when using a remote engine.| +|ML-7907|Fixed an issue with RabbitMQ event path when using Model serving functions.| + +## v1.6.4 (2 July 2024) + +### UI +| ID |Description | +|----------|---------------------------------------------------------------------------| +|ML-6867|Scalability improvement. The artifacts page (artifacts/datasets/models) now displays a maximum of 1000 items. (Use filters to focus the results.) | + +### Closed issues +| ID |Description | +|----------|---------------------------------------------------------------------------| +|ML-6770 |Resolved MLRun workers restart when running many workflows that produce artifacts.| +|ML-6795 |Can now upgrade to v1.6.4 when cluster has artifacts that do not have a `key`.| + ## v1.6.4 (30 June 2024) ### UI @@ -78,6 +233,7 @@ ### Closed issues | ID |Description | |----------|---------------------------------------------------------------------------| +|ML-4758|Heavy projects can be deleted successfully.| |ML-5808|Fix selecting the project-owner user.| |ML-5907|"Invite New Members" now returns the full list of users when there are 100+ users in system.| |ML-5749, 6037|After the user removes ownership of the currently displayed project, the UI redirects to the Projects page.| @@ -153,6 +309,7 @@ ### Closed issues | ID |Description | |----------|---------------------------------------------------------------------------| +|ML-76|Artifacts submitted from a git based project can now be received (after changing the HEAD git commit of the project between different log_artifact calls) .| |ML-1373|Incorrect service names do not result in stuck pods during ContainerCreating.| |ML-1835|The index record is not duplicated in the datasets metadata. | |ML-3714|Runs that complete successfully do not show errors in Kubeflow. | @@ -160,6 +317,7 @@ |ML-4093|Documentation: Improved description of [handlers](../runtimes/create-and-use-functions.html#using-set-function) and {ref}`functions`. | |ML-4370|Hyper-param and single runs no longer generate artifacts with the same name. | |ML-4563|Local jobs can now be aborted in the UI. | +|ML-4585|The `mlrun/mlrun` image now supports mpijob. | |ML-4613|UI: Fixed the map type hint in the Batch Inference Parameters. | |ML-4642|The UI no longer gets stuck when there is a high number of query results. | |ML-4678|When tagging a specific version of a model using the SDK, it does not clear the tags from the rest of the versions.| @@ -168,6 +326,11 @@ |ML-4608|The artifact `db_key` is now forwarded when registering an artifact.| |ML-4617|Fixed error message when using a feature vector as an input to a job without first calling `get_offline_features` on the vector.| |ML-4714|Logs are not truncated in the MLRun UI logs page for jobs that have a high number of logs or run for over day. | +|ML-4810|Can now rerun a job when the "mlrun/client_version" label has "+" in its value. | +|ML-4821|Very big projects are deleted successfully.| +|ML-4855|MLRun now supports TensorFlow up to 2.15.0.| +|ML-4857|When local runs are aborted in the UI, the actual execution is also aborted.| +|ML-4858|After aborting a job/run from the UI, the logs are populated.| |ML-4922|Preview and Metadata tabs now indicate when there are more columns that are not displayed.| |ML-4967|The **Deploy** button in the **Project > Models** page now creates a new endpoint/serving function.| |ML-4992|Fixed starting a spark job from source archive (using `with_source_archive()`).| @@ -178,6 +341,7 @@ |ML-5089|When trying to delete a running job, an error opens that a running job cannot be deleted and it needs to be aborted first.| |ML-5091|Monitoring does not recreate a deleted run. | | |ML-5146|Resolved OOM issues by reducing the memory footprint when monitoring runs. | +|ML-5175|For Nuclio runtimes, MLRun is now installed together with user requirements, to account for MLRun dependencies. | |ML-5481|You can now use `build_image` using the project source. See the example in [build_image](../projects/run-build-deploy.html#build-image).| |ML-5576|FeatureSet can now ingest data that contains single quotes.| |ML-5746|Labels no longer create partial projects that cannot be deleted.| @@ -351,7 +515,6 @@ See [Deprecations and removed code](#deprecations-and-removed-code). |ML-3733|`mlrun.get_run_db().list_model_endpoints()` returns `list`. Previously, it returned `mlrun.api.schemas.model_endpoints.ModelEndpointList`. | |ML-3773|The aggregation mechanism on Redis databases has improved, but the history of the aggregation (from before the upgrade) is lost, as if there were 0 events during that period. | |ML-4053|Pre-v1.4.0: When logging artifacts during a runtime (regular artifacts, not models (ModelArtifact via context.log_model) or datasets (DatasetArtifact via context.log_dataset)), they were strings in the RunObject outputs property. The strings were the target path to the file logged in the artifact. From v1.4.0, they are the store path of the artifact, and not the target path. (They now appear the same as the store paths for logging models and datasets.) This is breaking behavior only if you use the output of the run object as a parameter to another runtime and not as an input. [View in Git](https://github.com/mlrun/mlrun/pull/3333). | - ``` # Set 2 functions: func1 = project.set_function(...) @@ -906,77 +1069,69 @@ with a drill-down to view the steps and their details. [Tech Preview] ## Open issues -| ID | Description | Workaround | Opened in | -|----------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------| -| ML-76 | In rare cases (after changing the HEAD git commit of the project between different log_artifact calls) artifacts submitted from a git based project can't be received. | NA | v0.5.5 | -| ML-2052 | mlrun service default limits are not applied to the wait-container on pipeline jobs. | NA | v1.0.0 | -| ML-2030 | Need a way to move artifacts from test to production Spark. | To register artifact between different environments, e.g. dev and prod, upload your artifacts to a remote storage, e.g. S3. You can change the project artifact path using MLRun or MLRun UI. `project.artifact_path='s3:`function = mlrun.new_function("serving", image="python:3.9", kind="serving") function.with_commands([ "python -m pip install --upgrade pip", "pip install 'mlrun[complete]' scikit-learn==1.1.2", ])` | v1.3.0 | -| NA | The feature store does not support schema evolution and does not have schema enforcement. | NA | v1.2.1 | -| ML-3526 | Aggregation column order is not always respected (storey engine). | NA | v1.3.0 | -| ML-3626 | The "Save and ingest" option is disabled for a scheduled feature set. | NA | v1.3.0 | -| ML-3627 | The feature store allows ingestion of string type for the timestamp key resulting in errors when trying to query the offline store with time filtration. | Use only timestamp type. | v1.2.1 | -| ML-3636 | `get_online_feature_service` from Redis target returns truncated values. | NA | v1.3.0 | -| ML-3640 | When running a remote function/workflow, the `context` global parameter is not automatically injected. | Use `get_or_create_ctx` | v1.3.0 | -| ML-3646 | MapValues step on Spark ingest: keys of non-string type change to string type, sometime causing failures in graph logic. | NA | v1.2.1 | -| ML-3680 | The function spec does not get updated after running a workflow. | NA | v1.3.0 | -| ML-3804 | A serving step with no class does not inherit parameters from the function spec. | Create a class to forward the parameters. See [Create a single step](../serving/writing-custom-steps.html#create-a-single-step). | v1.3.1 | -| ML-4107 | On scheduled ingestion (storey and pandas engines) from CSV source, ingests all of the source on each schedule iteration. | Use a different engine and/or source. | v1.4.0 | -| ML-4153 | When creating a passthrough feature-set in the UI, with no online target, the feature-set yaml includes a parquet offline target, which is ignored. | NA | v1.4.0 | -| ML-4166 | Project yaml file that is very large cannot be stored. | Do not embed the artifact object in the project yaml. | v1.4.0 | -| ML-4186 | on `get_offline_features` ('local'/pandas engine) with passthrough, a source parquet column of type BOOL has dtype "object" or "bool" in the response | | v1.4.0 | -| ML-4442 | After a model is deployed without applying monitoring (`set_tracking()` was not set on the serving function), monitoring cannot be added. | Delete the existing model endpoint (`mlrun.get_run_db().delete_model_endpoint()`), then redeploy the model. | v1.5.0 | -| ML-4582 | Custom packagers cannot be added to projects created previous to v1.4.0 | NA | v1.6.0 | -| ML-4585 | The `mlrun/mlrun` image does not support mpijob. | Create your own image that includes mpijob. | v1.5.0 | -| ML-4655 | Timestamp entities are allowed for feature store, but format is inconsistent. | NA | v1.5.0 | -| NL-4685 | When using columns with type "float" as feature set entities, they are saved inconsistently to key-value stores by different engines. | Do not use columns with type float as feature set entities. | v1.5.0 | -| ML-4698 | Parameters that are passed to a workflow are limited to 10000 chars. | NA, external Kubeflow limitation. | v1.5.0 | -| ML-4725 | ML functions show as if they are in the "Creating" status, although they were created and used. | NA | v1.4.1 | -| ML-4740 | When running function `batch_inference_v2` from the SDK, the `ingest()` function accepts 3 parameters as Data-item or other types: `dataset`, `model_path` and `model_endpoint_sample_set`. If you provided these parameters as non Data-items and later on you want to rerun this function from the UI, you need to provide these parameters as Data-item. | Prepare suitable Data-item and provide it to the batch-rerun UI. | v1.5.0 | -| ML-4758 | In rare cases, deleting a heavy project is unsuccessful and results in a timeout error message while the project moves to offline state. | Delete again. | v1.5.0 | -| ML-4769 | After deleting a project, data is still present in the Artifacts and Executions of pipelines UI. | NA | v1.4.0 | -| ML-4810 | Cannot rerun a job when the "mlrun/client_version" label has "+" in its value. | Ensure the "mlrun/client_version" label does not include "+". | v1.6.0 | -| ML-4821 | In some cases, deleting a very big project fails with a timeout due to the time required to delete the project resources. | Delete the project again | NA | v1.5.0 | -| ML-4846 | With Docker Compose the V3IO_ACCESS_KEY is required for Parquet target. | replace this line: `feature_set.set_targets(targets=[mlrun.datastore.ParquetTarget()], with_defaults=False)` with a command that specifies the target path for the Parquet target. For example: `feature_set.set_targets(targets=[mlrun.datastore.ParquetTarget(path="/some/path/to/parquet/file")], with_defaults=False)` | v1.5.0 | -| ML-4857 | Local runs can be aborted in the UI, though the actual execution continues. | NA | v1.5.0 | -| ML-4858 | After aborting a job/run from the UI, the logs are empty. | NA | v1.5.0 | -| NL-4881 | Kubeflow pipelines parallelism parameter in dsl.ParallelFor() does not work (external dependency). | NA | v1.4.1 | -| ML-4934 | Modifying the parameters of a serving-function (for example changing `default_batch_intervals`) that is configured for model-monitoring tracking requires a specific workflow. | See [Enable model monitoring](../monitoring/initial-setup-configuration.html#enabling-model-monitoring). | v1.6.0 | -| ML-4942 | The Dask dashboard requires the relevant node ports to be open. | Your infrastructure provider must open the ports manually. If running MLRun locally or CE, make sure to port-forward the port Dask Dashboard uses to ensure it is available externally to the Kubernetes cluster. | v1.5.0 | -| ML-4956 | A function created by SDK is initially in the "initialized" state in the UI and needs to be deployed before running it. | In **Edit**, press **Deploy** | v1.5.1 | -| ML-5079 | Cannot update git remote with `project.create_remote()` | NA | v1.5.1 | -| ML-5175 | For Nuclio runtimes, MLRun must be installed together with user requirements, to account for MLRun dependencies. | Include MLRun in the requirements, for example | v1.6.0 | -| ML-5204 | The **Projects>Settings** does not validate label names. Errors are generated from the back end. | Use [Kubernetes limitations](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set). | v1.6.0 | -| ML-5573 | The default value of feature-set ingest() infer_options is "all" (which includes Preview) and as a result, during ingest, preview is done as well. As a result, if a validator was configured for a feature, each violation causes two messages to be printed. | NA | v1.6.0 | -| ML-5732 | When using an MLRun client previous to v1.6.0, the workflow step status might show completed when it is actually aborted. | Abort the job from the SDK instead of from the UI, or upgrade the client. | 1.6.0 | -| ML-5876 | The maximum length of project name + the longest function name for `project.enable_model_monitoring` is 63 chars. | Keep the name combination at a maximum of 63 chars. | v1.6.0 | + +| ID| Description|Workaround |Opened in | +|--------|----------------------------------------------------------------|-----------------------------------------|-----------| +|ML-2052|mlrun service default limits are not applied to the wait-container on pipeline jobs.|NA|v1.0.0| +|ML-2030|Need a way to move artifacts from test to production Spark. |To register artifact between different environments, e.g. dev and prod, upload your artifacts to a remote storage, e.g. S3. You can change the project artifact path using MLRun or MLRun UI. `project.artifact_path='s3:`function = mlrun.new_function("serving", image="python:3.9", kind="serving") function.with_commands([ "python -m pip install --upgrade pip", "pip install 'mlrun[complete]' scikit-learn==1.1.2", ])`|v1.3.0 | +|NA|The feature store does not support schema evolution and does not have schema enforcement.| NA| v1.2.1 | +|ML-3526|Aggregation column order is not always respected (storey engine).| NA | v1.3.0| +|ML-3626|The "Save and ingest" option is disabled for a scheduled feature set. |NA | v1.3.0| +|ML-3627|The feature store allows ingestion of string type for the timestamp key resulting in errors when trying to query the offline store with time filtration.|Use only timestamp type.| v1.2.1 | +|ML-3636|`get_online_feature_service` from Redis target returns truncated values. | NA | v1.3.0| +|ML-3646|MapValues step on Spark ingest: keys of non-string type change to string type, sometime causing failures in graph logic.| NA | v1.2.1| +|ML-3867|Cannot search for project owners by first name. |NA| v1.4.0| +|ML-6839|Schedules have a minimum interval between two scheduled jobs. By default, a job cannot be scheduled to run more than 2 times in 10 minutes. See {ref}`scheduled-jobs`.| NA | v1.6.3 | +|ML-4107| On scheduled ingestion (storey and pandas engines) from CSV source, ingests all of the source on each schedule iteration. | Use a different engine and/or source. | v1.4.0 | +|ML-4153|When creating a passthrough feature-set in the UI, with no online target, the feature-set yaml includes a parquet offline target, which is ignored.| NA | v1.4.0 | +|ML-4166|Project yaml file that is very large cannot be stored. |Do not embed the artifact object in the project yaml. | v1.4.0 | +|ML-4186|on `get_offline_features` ('local'/pandas engine) with passthrough, a source parquet column of type BOOL has dtype "object" or "bool" in the response | | v1.4.0| +|ML-4582|Custom packagers cannot be added to projects created previous to v1.4.0 | NA | v1.6.0| +|ML-4655|Timestamp entities are allowed for feature store, but format is inconsistent. |NA|v1.5.0| +|NL-4685|When using columns with type "float" as feature set entities, they are saved inconsistently to key-value stores by different engines.| Do not use columns with type float as feature set entities. |v1.5.0 | +|ML-4698|Parameters that are passed to a workflow are limited to 10000 chars.| NA, external Kubeflow limitation. | v1.5.0 | +|ML-4725|ML functions show as if they are in the "Creating" status, although they were created and used.|NA|v1.4.1| +|ML-4740|When running function `batch_inference_v2` from the SDK, the `ingest()` function accepts 3 parameters as Data-item or other types: `dataset`, `model_path` and `model_endpoint_sample_set`. If you provided these parameters as non Data-items and later on you want to rerun this function from the UI, you need to provide these parameters as Data-item.|Prepare suitable Data-item and provide it to the batch-rerun UI.| v1.5.0 | +|ML-4769|After deleting a project, data is still present in the Artifacts and Executions of pipelines UI. | NA | v1.4.0 | +|ML-4881|Kubeflow pipelines parallelism parameter in dsl.ParallelFor() does not work (external dependency). |NA| v1.4.1| +|ML-4942|The Dask dashboard requires the relevant node ports to be open. |Your infrastructure provider must open the ports manually. If running MLRun locally or CE, make sure to port-forward the port Dask Dashboard uses to ensure it is available externally to the Kubernetes cluster. | v1.5.0 | +|ML-4956|A function created by SDK is initially in the "initialized" state in the UI and needs to be deployed before running it. | In **Edit**, press **Deploy** | v1.5.1 | +|ML-5573|The default value of feature-set ingest() infer_options is "all" (which includes Preview) and as a result, during ingest, preview is done as well. As a result, if a validator was configured for a feature, each violation causes two messages to be printed.|NA|v1.6.0| +|ML-5732|When using an MLRun client previous to v1.6.0, the workflow step status might show completed when it is actually aborted.|Abort the job from the SDK instead of from the UI, or upgrade the client. |1.6.0| +|ML-5876|The maximum length of project name + the longest function name for `project.enable_model_monitoring` is 63 chars. |Keep the name combination at a maximum of 63 chars. |v1.6.0| +|ML-7196|The models features statistics `feature_stats` is limited to 16MB. Further limitation to 1MB when using model-monitoring over V3IO-KV will be removed in 1.8.|NA| v1.7.0| +|ML-7568/7915| The SDK does not inform of invalid node selector combinations when running a function, but the pod remains stuck in the Pending state. |See [Preventing and resolving conflicts](../runtimes/configuring-job-resources.html#preventing-and-resolving-conflicts). v1.7.0| +|ML-7571|For executions of Dask runtimes, the UI does not show node-selectors applied to the run. |NA| v1.7.0| +|ML-7746|In some cases, when the pipeline is extremely large it is not displayed in the graph.|NA| v1.7.0| +|ML-7955|The **Owner** field is blank for artifacts that are registered in the UI.|NA | v1.7.0| +|ML-8110|The cross-project summary is not aligned with per project summary: In the Projects page, **Monitoring > Scheduled**, the Jobs and Total counters are incorrect. |NA | v1.7.0| + + ## Limitations | ID |Description |Workaround |Opened in| |---------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------| |ML-1278|Users do not automatically have access rights to the project data of the projects they are members of. | Assign the user access permission for the project folder. | v0.8.0 | |ML-2014|Model deployment returns ResourceNotFoundException (Nuclio error that Service is invalid.) |Verify that all `metadata.labels` values are 63 characters or less. See the [Kubernetes limitation](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set). |v1.0.0 | -|ML-3206|When `get_or_create_project` is called, and there is a project.yaml in the dir, no new project is created (even of the project name is new). The existing project.yaml is loaded instead.|v1.2.1 | |ML-3520|MLRun does not decompress large Kubeflow pipelines. |NA |v1.3.0 | |ML-3731|When trying to identify a failed step in a workflow with `mlrun.get_run_db().list_pipelines('project-name')`, the returned error is `None`. |To see the error, use `mlrun.db.get_pipelines()` instead. | |ML-3743|Setting AWS credentials as project secret cause a build failure on EKS configured with ECR. |When using an ECR as the external container registry, make sure that the project secrets AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY have read/write access to ECR, as described in the [platform documentation](https://www.iguazio.com/docs/latest-release/services/app-services/docker-registry/#create-off-cluster-registry)| |ML-4386|Notifications of local runs aren't persisted. | NA | v1.5.0| |ML-4767|When using mlrun-gpu image, use PyTorch versions up to and including than 2.0.1, but not higher. | You can build your own images with newer CUDA for a later release of PyTorch. | v1.5.0| -|ML-4855|MLRun supports TensorFlow up to 2.13.1. |ML-4907|MLRun Client does not support Win OS. | Use WSL instead. | v1.3.0 | |ML-5274|PySpark 3.2.x cannot always read parquet files written by pyarrow 13 or above. MLRun ingest might fail when `ingest()` is called with engine="spark" and a ParquetSource that points to parquet files that were written by pyarrow 13 or above. |Call `df.to_parquet()` with version="2.4" so that parquet files are backwards compatible.|v1.6.0| |ML-5669|When using mlrun.mlrun image, use PyTorch versions up to and including than 2.0.1, but not higher. See [MLRun runtime images](../runtimes/images.html#mlrun-runtime-images) | You can build your own images with newer CUDA for a later release of PyTorch. |v1.6.0| |ML-5732|When using an MLRun client previous to v1.6.0, the workflow step status might show completed when it is actually aborted. | Upgrade the client to v1.6.0 or higher. | v1.6.0 | +|ML-8115|Deploying a model without monitoring does not create an endpoint. | NA | v1.7.0| ## Deprecations and removed code @@ -987,7 +1142,7 @@ with a drill-down to view the steps and their details. [Tech Preview] | v1.5.0 |ML-4010|Unused artifact types: BokehArtifact, ChartArtifact | | v1.5.0 |ML-4075|Python 3.7 | | v1.5.0 |ML-4366 |MLRun images `mlrun/ml-models` and `mlrun/ml-models-gpu` | -| v1.5.0 |ML-3605|Model Monitoring: Most of the charts and KPIs in Grafana are now based on the data store target instead of the MLRun API. It is recommended to update the model monitoring dashboards since the old dashboards won't be supported. | +| v1.5.0 |ML-3605|Model Monitoring: Most of the charts and KPIs in Grafana are now based on the data store target instead of the MLRun API. It is recommended to update the model monitoring dashboards since the old dashboards are not supported. | | v1.0.0 |NA |MLRun / Nuclio does not support python 3.6. | @@ -995,33 +1150,47 @@ with a drill-down to view the steps and their details. [Tech Preview] | Will be removed|Deprecated|API |Use instead | |---------------|------------|----------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------| -| v1.9.0 |v1.7.0 |Datastore redis:`credentials_prefix` |Datastore profiles| +| v1.9.0 | v1.7.0 |Class: `mlrunn.common.schemas.RunsFormat` |`mlrun.common.formatters.RunFormat` | +| v1.9.0 | v1.7.0 |Class: `mlrunn.common.schemas.ArtifactsFormat` |`mlrun.common.formatters.ArtifactFormat` | +| v1.9.0 | v1.7.0 |Class: `mlrunn.common.schemas.ProjectsFormat` |`mlrun.common.formatters.ProjectFormat` | +| v1.9.0 | v1.7.0 |Class: `mlrunn.common.schemas.PipelinesFormat` |`mlrun.common.formatters.PipelineFormat` | +| v1.9.0 |v1.7.0 |Datastore redis:`credentials_prefix` |Datastore profiles| +| v1.9.0 |v1.7.0 |Parameter: `mlrun.runtimes.nuclio.function.RemoteRuntime.deploy` `auth_info` | NA. Was not used.| +| v1.9.0 |v1.7.0 |Parameter: `mlrun.projects.MlrunProject.list_runs` `state` |`states` | +| v1.9.0 |v1.7.0 |Parameter: `mlrun.db.httpdb.HTTPRunDB.list_runs` `state` |`states` | +| v1.9.0 |v1.7.0 |Class: `mlrun.common.runtimes.constants.RunLabels` |`RunLabels.owner` => `MlrunInternalLabels.owner`

`RunLabels.v3io_user` => `MlrunInternalLabels.v3io_user` | +| v1.9.0 |v1.7.0 |Parameter: `mlrun.runtimes.base.mlrun_op` `rundb` |MLRUN_DBPATH environment variable | +| v1.9.0 |v1.7.0 |Query parameter: GET `/projects/{project}/schedules?labels="label1=val1"` |`label`, which is an array of strings | +| v1.9.0 |v1.7.0 |Query parameter: DELETE `/projects/{project}/artifacts/{key:path}?uid="some-uid"` |`object-uid` | +| v1.9.0 |v1.7.0 |Query parameter: GET `/projects/{project}/artifacts/{key:path}?uid="some-uid"` |`object-uid` | | v1.9.0 |v1.6.3 |`FunctionSpec.clone_target_dir` |`ImageBuilder.source_code_target_dir` | v1.8.0 |v1.6.0 |HTTPDB: `last` parameter of `list_runs` | NA. Was not used.| | v1.8.0 |v1.6.0 |Feature store: `get_offline_features` |`FeatureVector.get_offline_features()`| | v1.8.0 |v1.6.0 |Feature store: `get_online_feature_service` |`FeatureVector.get_online_feature_service()`| | v1.8.0 |v1.6.0 |Feature store: `preview` |`FeatureSet.preview()`| -| v1.8.0 |v1.6.0 |Feature store: `deploy_ingestion_service_v2` |`FeatureSet.deploy_ingestion_service()`| | v1.8.0 |v1.6.0 |Feature store: `ingest` |`FeatureSet.ingest()`| | v1.8.0 |v1.6.0 |Artifacts: `uid` parameter of `store_artifact` | `tree` parameter of `store_artifact` (artifact uid is generated in the backend)| | v1.8.0 |v1.6.0 |Runtimes: `with_requirements` — `requirements` param as a requirements file |`requirements_file` param | -| v1.6.2 |v1.6.0 |`dashboard` parameter of the RemoteRuntime `invoke` |NA. The parameter is ignored. | -| v1.7.0 |v1.5.1 |`skip_deployed` parameter of `MLrunProject.build_image` |NA. The parameter is ignored. | -| v1.7.0 |v1.5.0 |`/files` and `/filestat` |`/projects/{project}/filestat` | -| v1.7.0 |v1.3.0 |`LegacyArtifact` and all legacy artifact types that inherit from it (`LegacyArtifact`, `LegacyDirArtifact`, `LegacyLinkArtifact`, `LegacyPlotArtifact`, `LegacyChartArtifact`, `LegacyTableArtifact`, `LegacyModelArtifact`, `LegacyDatasetArtifact`, `LegacyPlotlyArtifact`, `LegacyBokehArtifact`)|`Artifact` or other artifact classes that inherit from it | - ### Removed APIs | Version|API |Use instead | |---------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------| +| v1.7.0 |Function: `mlrun.utils.helpers.parse_versioned_object_uri` |`mlrun.common.helpers.parse_versioned_object_uri` | +| v1.7.0 |`func_info` |`ast_func_info` | +| v1.7.0 |Class: `MpiRuntimeV1Alpha1` |`MpiRuntimeV1` | +| v1.7.0 |Feature store: `deploy_ingestion_service` |`deploy_ingestion_service_v2`| +| v1.7.0 |`skip_deployed` parameter of `mlrun.projects.project.MlrunProject.build_image` |NA. The parameter was ignored. | +| v1.7.0 |API endpoint GET: `/files` and `/filestat` |`/projects/{project}/filestat` | +| v1.7.0 |`LegacyArtifact` and all legacy artifact types that inherit from it (`LegacyArtifact`, `LegacyDirArtifact`, `LegacyLinkArtifact`, `LegacyPlotArtifact`, `LegacyChartArtifact`, `LegacyTableArtifact`, `LegacyModelArtifact`, `LegacyDatasetArtifact`, `LegacyPlotlyArtifact`, `LegacyBokehArtifact`, `BokehArtifact`, `LegacyArtifact`)|`Artifact` or other artifact classes that inherit from it | +| v1.7.0 |Parameter of mlrun.feature_store.feature_set.FeatureSet.set_targets `default_final_state` |`default_final_step` | +| v1.6.2 |`dashboard` parameter of the RemoteRuntime `invoke` |NA. The parameter was ignored. | | v1.6.0 |`dashboard` parameter of `project.deploy_function`, `RemoteRuntime.deploy`, `RemoteRuntime.get_nuclio_deploy_status`, `ServingRuntime.with_secrets`| NA. The parameter was ignored. | -| v1.6.0 |`MLRunProject.clear_context()` |This method deletes all files and clears the context directory or subpath (if defined). This method can produce unexpected outcomes and is not recommended. | +| v1.6.0 |`MLRunProject.clear_context()` |NA | | v1.6.0 |MLRunProject object legacy parameters |metadata and spec | | v1.6.0 |`BaseRuntime.with_commands` and `KubejobRuntime.build_config` `verify_base_image` param|`prepare_image_for_deploy` | | v1.6.0 |`run_local` |`function.run(local=True)` | | v1.6.0 |CSVSource's `time_fields` parameter |Use `parse_dates` to parse timestamps | -| v1.6.0 |Feature-set `set_targets()`, `default_final_state ` |`default_final_step` | | v1.6.0 |`new_pipe_meta` |`new_pipe_metadata` | | v1.6.0 |ttl param from pipeline |`cleanup_ttl` | | v1.6.0 |objects methods from artifacts list |`to_objects` @@ -1049,8 +1218,9 @@ with a drill-down to view the steps and their details. [Tech Preview] ### Removed CLIs -| Version | CLI | -|---------|------------------------------------------------------------| -| v1.6.0 | deploy `--dashboard (nuclio/deploy)` |No longer supported on client side. Configure using the MLRun API. | +| Version | CLI |Use instead | +|---------|------------------------------------------------------------|------------------------------------------------------------| +| v1.7.0 | `mlrun db` |NA | +| v1.6.0 | deploy `--dashboard (nuclio/deploy)` |No longer supported on client side. Configure using the MLRun API.| | v1.6.0 | project `--overwrite-schedule` |Not relevant. Running a schedule is now an operation. | | v1.5.0 | `--ensure-project` flag of the `mlrun project` CLI command | | \ No newline at end of file diff --git a/docs/concepts/model-monitoring.md b/docs/concepts/model-monitoring.md index 422e0a2fd782..e6f20b06474a 100644 --- a/docs/concepts/model-monitoring.md +++ b/docs/concepts/model-monitoring.md @@ -36,7 +36,6 @@ Model monitoring is currently in Tech Preview status. Migrating from 1.7.0 to la ../model-monitoring/monitoring-models ../model-monitoring/monitoring-models-grafana ../model-monitoring/index -../concepts/alerts ``` **See also** diff --git a/docs/concepts/notifications.md b/docs/concepts/notifications.md index 0ed1da2b8c71..d54899c79c92 100644 --- a/docs/concepts/notifications.md +++ b/docs/concepts/notifications.md @@ -16,15 +16,7 @@ MLRun supports configuring notifications on jobs and scheduled jobs. This sectio ## The notification object -The notification object's schema is: -- `kind`: str - notification kind (slack, git, etc...) -- `when`: list[str] - run states on which to send the notification (completed, error, running) -- `name`: str - notification name -- `message`: str - notification message -- `severity`: str - notification severity (info, warning, error, debug) -- `params`: dict - notification parameters (See definitions in [Notification Kinds](#notification-params-and-secrets)) -- `secret_params`: dict - secret data notification parameters (See definitions in [Notification Params and Secrets](#notification-kinds)) -- `condition`: str - jinja template for a condition that determines whether the notification is sent or not (See [Notification Conditions](#notification-conditions)) +See {py:class}`~module-mlrun.common.schemas.notification`. ## Local vs. remote @@ -33,7 +25,7 @@ Usually, a local run sends locally, and a remote run sends remotely. However, there are several special cases where the notification is sent locally either way. These cases are: - Local or KFP Engine Pipelines: To conserve backwards compatibility, the SDK sends the notifications as it did before adding the run - notifications mechanism. This means you need to watch the pipeline in order for its notifications to be sent. (Remote pipelines act differently. See [Configuring Notifications For Pipelines](#configuring-notifications-for-pipelines For Pipelines for more details.) + notifications mechanism. This means you need to watch the pipeline in order for its notifications to be sent. (Remote pipelines act differently. See [Configuring Notifications For Pipelines](#configuring-notifications-for-pipelines) for more details. - Dask: Dask runs are always local (against a remote Dask cluster), so the notifications are sent locally as well. > **Disclaimer:** Notifications of local runs aren't persisted. @@ -51,34 +43,7 @@ It's essential to utilize `secret_params` exclusively for handling sensitive inf ## Notification kinds -Currently, the supported notification kinds and their params are as follows: - -- `slack`: - - `webhook`: The slack webhook to which to send the notification. -- `git`: - - `token`: The git token to use for the git notification. - - `repo`: The git repo to which to send the notification. - - `issue`: The git issue to which to send the notification. - - `merge_request`: In GitLab (as opposed to GitHub), merge requests and issues are separate entities. - If using merge request, the issue will be ignored, and vice versa. - - `server`: The git server to which to send the notification. - - `gitlab`: (bool) Whether the git server is GitLab or not. -- `webhook`: - - `url`: The webhook url to which to send the notification. - - `method`: The http method to use when sending the notification (GET, POST, PUT, etc...). - - `headers`: (dict) The http headers to send with the notification. - - `override_body`: (dict) The body to send with the notification. If not specified, the body will be a dict with the - `name`, `message`, `severity`, and the `runs` list of the completed runs. You can also add the run's details using: `"override_body": {"message":"Run Completed {{ runs }}"`. - Results would look like - ``` - { - "message": "Run Completed [{'project': 'my-project', 'name': 'my-function', 'host': , 'status': {'state': 'completed', 'results': }}]" - } - ``` - - `verify_ssl`: (bool) Whether SSL certificates are validated during HTTP requests or not, - The default is set to `True`. -- `console` (no params, local only) -- `ipython` (no params, local only) +See {py:class}`~mlrun.common.schemas.notification.NotificationKind`. ## Configuring notifications for runs diff --git a/docs/genai/live-ops/index.md b/docs/genai/live-ops/index.md index 3b51b838149a..f518df86e9ee 100644 --- a/docs/genai/live-ops/index.md +++ b/docs/genai/live-ops/index.md @@ -31,4 +31,4 @@ preventing hallucination: minimizing the risk of LLMs generating factually incor - {ref}`genai-02-mm-llm` - {ref}`realtime-monitor-drift-tutor` - [Build & Deploy Custom (fine-tuned) LLM Models and Applications](https://github.com/mlrun/demo-llm-tuning/tree/main) -- [Large Language Model Monitoring](https://github.com/mlrun/demo-monitoring-and-feedback-loop/blob/main/notebook.ipynb), which includes using an LLM as a judge \ No newline at end of file +- [Large Language Model Monitoring](https://github.com/mlrun/demo-monitoring-and-feedback-loop), which includes using an LLM as a judge \ No newline at end of file diff --git a/docs/model-monitoring/index.md b/docs/model-monitoring/index.md index dd5df8f25520..08284023c3a6 100644 --- a/docs/model-monitoring/index.md +++ b/docs/model-monitoring/index.md @@ -99,7 +99,7 @@ as an MLRun function and create MLRun artifacts, using the built-in class `Evide Projects are used to group functions that use the same model monitoring application. You first need to create a project for a specific application. Then you disable the default app, enable your customer app, and create and run the functions. -The basic flow for classic ML and other models is the same, but the apps and the infer requests are different. See {ref}`realtime-monitor-drift-tutor`. +The basic flow for classic ML and other models is the same, but the apps and the infer requests are different. ## Multi-port predictions diff --git a/docs/runtimes/configuring-job-resources.md b/docs/runtimes/configuring-job-resources.md index b3e88bd32851..f6e9d8b54e5e 100644 --- a/docs/runtimes/configuring-job-resources.md +++ b/docs/runtimes/configuring-job-resources.md @@ -354,6 +354,9 @@ The resulting configuration for the function is: {"region": "us-central1", "zone": "us-east1", "gpu": "true", "arch": "amd64"} ``` +```{admonition} Tip +Be sure that the node selectors you are configuring are compatible with the available nodes in your cluster. Incompatible node selectors are not validated at the project level and could result in scheduling issues when running functions. +``` ### Overriding node selectors You can override and ignore node selectors defined at the project level or service level from the function level @@ -365,13 +368,13 @@ The zone label from the project level is completely removed, and the resulting c ``` {"gpu": "true", "arch": "amd64"} ``` -### Preventing conflicts +### Preventing and resolving conflicts If your function run is stuck with the status `pending`, it's possible that the "specified" node selector does not exist. There are three levels of node selectors in MLRun: function, project, and service. At runtime, the system combines these selectors and applies the resolved configuration to the pod. -How to Investigate: +**How to Investigate:** 1. Check the Configuration Files: Look in the `function.yaml` and `project.yaml` files to see if there are any node selector settings. 2. Review Node Selectors in the UI: Go to **Projects > Jobs and Workflows > Monitor Jobs > Overview > Node Selector**. This shows the node selector that was ultimately defined for the run after combining the function, project, and service settings. @@ -379,7 +382,7 @@ that was ultimately defined for the run after combining the function, project, a that match the specified node selector, the error is displayed here. > For Nuclio functions (Nuclio, serving, and application runtimes) the final resolved node selector is displayed in the Nuclio UI. It is not visible on MLRun function spec since it may be further enriched by Nuclio (See {ref}node-selector-runtimes section for more information). -Resolving Conflicts:
+**Resolving Conflicts**:
If the node selectors from the function, project, or service levels, conflict or result in an impossible combination, you can resolve the issue by specifying the conflicting node selector key with an empty string value on your function. Be cautious with this approach and consult your project admin before making changes to ensure it won’t cause other issues. diff --git a/docs/store/artifacts.md b/docs/store/artifacts.md index 51ad94ed74e0..606ee64d915d 100644 --- a/docs/store/artifacts.md +++ b/docs/store/artifacts.md @@ -139,14 +139,14 @@ Example artifact URLs: Artifacts are comprised of two parts: an artifact object that points to the artifact data; and the artifact data (files). You can delete artifacts from a specific project -and choose what you want to delete. You cannot delete artifacts of type: ModelArtifact, DirArtifact, and DatasetArtifact that has more than one file. +and choose what you want to delete. You cannot delete artifacts of type: ModelArtifact, DirArtifact, or DatasetArtifact that has more than one file. Deleting artifact data is supported for V3IO, Google, Azure, DBFS, Filestore, and S3. The options for {py:meth}`~mlrun.projects.MlrunProject.delete_artifact`: - metadata-only: Delete only the artifact object. The related artifact data remains. - data-optional: Delete the artifact object and the data. If data deletion is unsuccessful, deletes only the object. -- data-force: Delete the artifact object and the data. If data deletion is unsuccessful, does not delete the object either. +- data-force: Delete the artifact object and the data. If data deletion is unsuccessful, the object is also not deleted. For example: ``` diff --git a/docs/tutorials/demos.md b/docs/tutorials/demos.md index 47285d7644f5..29b7f33f4fb9 100644 --- a/docs/tutorials/demos.md +++ b/docs/tutorials/demos.md @@ -7,16 +7,16 @@ These end-to-end demos demonstrate how to use the Iguazio AI platform, MLRun, an |Demo|Description| |-----------------------------------|---------------------------------------------------------------------------------------------| -|[Call center demo](https://github.com/mlrun/demo-call-center)|This demo showcases how to use LLMs to turn audio files, from call center conversations between customers and agents, into valuable data — all in a single workflow orchestrated by MLRun. MLRun automates the entire workflow, auto-scales resources as needed, and automatically logs and ses values between the different workflow steps.| +|[Call center](https://github.com/mlrun/demo-call-center)|This demo showcases how to use LLMs to turn audio files, from call center conversations between customers and agents, into valuable data — all in a single workflow orchestrated by MLRun. MLRun automates the entire workflow, auto-scales resources as needed, and automatically logs and ses values between the different workflow steps.| |[Fine tune an LLM and build a BOT](https://github.com/mlrun/demo-llm-tuning/blob/main)|This demo shows how to fine-tune an LLM and build a chatbot that can answer all your questions about MLRun's MLOps. It starts with a pre-trained model from Hugging Face, fine tunes the model, creates an automated training pipeline, and deploys a serving graph. The serving graph includes post-processing for accuracy of generated text, and filtering for toxicity.| -|[Interactive bot demo using LLMs](https://github.com/mlrun/demo-llm-bot/blob/main/README.md)|This demo showcases the usage of Language Models (LLMs) and MLRun to build an interactive chatbot using your own data for Retrieval Augmented Question Answering. The data will be ingested and indexed into a Vector Database to be queried by an LLM in real-time. The project utilizes MLRun for orchestration/deployment, HuggingFace embeddings for indexing data, Milvus for the vector database, OpenAI's GPT-3.5 model for generating responses, Langchain to retrieve relevant data from the vector store and augment the response from the LLM, and Gradio for building an interactive frontend.| - +|[Interactive bot using LLMs](https://github.com/mlrun/demo-llm-bot/blob/main/README.md)|This demo showcases the usage of Language Models (LLMs) and MLRun to build an interactive chatbot using your own data for Retrieval Augmented Question Answering. The data will be ingested and indexed into a Vector Database to be queried by an LLM in real-time. The project utilizes MLRun for orchestration/deployment, HuggingFace embeddings for indexing data, Milvus for the vector database, OpenAI's GPT-3.5 model for generating responses, Langchain to retrieve relevant data from the vector store and augment the response from the LLM, and Gradio for building an interactive frontend.| +|[LLM monitoring and feedback loop: Banking](https://github.com/mlrun/demo-monitoring-and-feedback-loop/blob/main/README.md)|This demo illustrates how to train, deploy, and monitor, and LLM using an approach described as "LLM as a judge".| ## ML Demos |Demo|Description| |-----------------------------------|---------------------------------------------------------------------------------------------| -|[Mask Detection Demo](https://github.com/mlrun/demo-mask-detection)|This demo contains three notebooks that: Serve the model as a serverless function in an http endpoint; Train and evaluate a model for detecting if an image includes a person who is wearing a mask, by using Tensorflow, Keras, or PyTorch; Write an automatic pipeline where you download a dataset of images, train and evaluate the model, then optimize the model (using ONNX) and serve it.| +|[Mask Detection](https://github.com/mlrun/demo-mask-detection)|This demo contains three notebooks that: Serve the model as a serverless function in an http endpoint; Train and evaluate a model for detecting if an image includes a person who is wearing a mask, by using Tensorflow, Keras, or PyTorch; Write an automatic pipeline where you download a dataset of images, train and evaluate the model, then optimize the model (using ONNX) and serve it.| |[Fraud Prevention (Feature Store)](https://github.com/mlrun/demo-fraud)|This demo shows the usage of MLRun and the feature store. Fraud prevention specifically is a challenge as it requires processing raw transaction and events in real-time and being able to quickly respond and block transactions before they occur. Consider, for example, a case where you would like to evaluate the average transaction amount. When training the model, it is common to take a DataFrame and just calculate the average. However, when dealing with real-time/online scenarios, this average has to be calculated incrementally.| -|[Sagemaker demo](https://github.com/mlrun/demo-sagemaker)|This demo showcases how to build, manage, and deploy ML models using AWS SageMaker and MLRun. It emphasizes the automation of ML workflows from development to production.| +|[Sagemaker](https://github.com/mlrun/demo-sagemaker)|This demo showcases how to build, manage, and deploy ML models using AWS SageMaker and MLRun. It emphasizes the automation of ML workflows from development to production.| |[Building Production Pipelines WIth AzureML and MLRun](https://github.com/mlrun/demos/tree/1.7.x/azureml-demo) |This demo uses the MLRun Feature Store to ingest and prepare data, create an offline feature vector (snapshot) for training, run AzureML AutoML Service as an automated step (function) in MLRun, view and compare the AzureML Models using MLRun tools, Build a real-time serving pipeline, and provide real-time model monitoring. By default, this demo works with the online feature store, which is currently not part of the Open Source MLRun default deployment.| \ No newline at end of file diff --git a/docs/tutorials/genai-02-model-monitor-llm.ipynb b/docs/tutorials/genai-02-model-monitor-llm.ipynb index 82beae73dd77..5e18ba912e3c 100644 --- a/docs/tutorials/genai-02-model-monitor-llm.ipynb +++ b/docs/tutorials/genai-02-model-monitor-llm.ipynb @@ -15,7 +15,7 @@ "- [Add the monitoring-function code](#dd-the-monitoring-function-code)\n", "- [Deploy the model, enable tracking, and deploy the function](#deploy-the-model-enable-tracking-and-deploy-the-function)\n", "\n", - "This tutorial explains how an LLM can be monitored. To see it in action, run the [Large Language Model Monitoring](https://github.com/mlrun/demo-monitoring-and-feedback-loop) demo.\n", + "This tutorial explains how an LLM can be monitored. To see it in action, run the [Large Language Model Monitoring](https://github.com/mlrun/demo-monitoring-and-feedback-loop/blob/main/README.md) demo.\n", "\n", "## Prerequisites" ] @@ -321,7 +321,8 @@ }, { "cell_type": "markdown", - "id": "584083af", + "id": "f713b7d9", + "metadata": {}, "source": [ "Adding the model parameters to the endpoint. This allow the model server class to initialize." @@ -330,7 +331,8 @@ { "cell_type": "code", "execution_count": null, - "id": "e0361bae", + "id": "2ea3139a", + "metadata": {}, "outputs": [], "source": [