-
Notifications
You must be signed in to change notification settings - Fork 0
Release Notes
David Amos edited this page Nov 22, 2024
·
88 revisions
NOTE: Release notes are moving to the RelationalAI website.
- Fixes an issue that caused an some engines to experience an unrecoverable
engine is upgradingerror when an engine upgrade failed to initialize.
-
In version
2024114-2a37e311-1, we changed the instance family for theRELATIONAL_AI_ERP_COMPUTE_POOLprovisioned by new RAI Native App installs toCPU_X64_XSinstead ofCPU_X64_S.In this release, older apps with
CPU_X64_Scompute pools will be automatically migrated toCPU_X64_XS. This reduces the number of Snowflake credits consumed by the application without affecting performance.
- Changes to a data stream’s source object’s columns will now result in the stream being quarantined instead of leaving the stream in a broken state. You may resume the quarantined stream to synchronize the schema changes with models built using the RAI Python API.
- A
transaction metadata not founderror is no longer raised when a transaction is aborted.
- The deprecated
relationalai.app.finalize_install()procedure has been removed. The work that this procedure was responsible for is performed byapp.activate().
You can now configure engines to automatically suspend after a period of inactivity:
-
The api.create_engine() procedure has a new
engine_configparameter that accepts a JSON object, like{'auto_suspend_mins': 60, 'await_storage_vacuum': false}, for configuring how many minutes an engine must be inactive before being suspended and whether or not garbage collection tasks must complete before suspending the engine. Ifengine_configisNULL, the engine is configured withauto_suspend_minsset to0, and the engine will not be automatically suspended. -
The default value for
auto_suspend_minswill change from0to60minutes in a future release. To avoid unexpected behavior, we recommend creating engines with an explicitauto_suspend_minsvalue. Beginning with version0.6.0of the RAI Python API, engines auto-created by the API will be configured withauto_suspend_minset to value of theauto_suspend_minconfiguration key, or60minutes if the key is missing. -
An api.alter_engine_auto_suspend_mins() procedure has been added to alter an existing engine’s
auto_suspend_minsconfiguration parameter. Note that this procedure only works for engines created using the newapi.create_engine()procedure mentioned above. -
Use the new api.resume_engine() procedure to resume a suspended engine. Beginning with version
0.6.0of the RAI Python API, suspended engines will be resumed automatically when Python queries are executed. -
The
api.enginesview, as well as the table returned byapi.get_engine(), now include anAUTO_SUSPEND_MINSandAWAIT_STORAGE_VACUUMcolumns that contain the configured values for each engine. The newSUSPENDS_ATcolumn shows the timestamp of the next scheduled suspension for inactive engines.
- Fixed a bug that periodically resumed the app warehouse while the Native App was deactivated.
- New RAI Native App installations will provision the compute pool named RELATIONAL_AI_ERP_COMPUTE_POOL for the RAI SPCS service as a CPU_X64_XS instance family of nodes. For existing installations, this compute pool was provisioned as a CPU_X64_S instance family and those will be automatically migrated to the new instance family in an upcoming release. This change reduces the number of Snowflake credits consumed by the SPCS service without affecting service operations or performance.
- Application logs have been expanded with more details for improved support.
- The engine upgrade process was improved:
- While a new version of the RelationalAI Native App is being rolled out to a consumer account, it is now possible for transactions to run on engines that are already using the latest version of the app. In previous versions, new transactions were rejected during the upgrade process, even on engines that were already using the latest version.
- You can now create engines on pools that are in the process of being upgraded. These new engines will be created with the latest version.
- Fixed an issue where the
api.enginesview could raise errors when used while an engine is simultaneously being deleted. - Comparisons between zero values now return the correct results. For example,
0.0 = -0.0now returns true and0 = -0also now returns true. Note that0 = 0.0returns false because the types do not match.
- us-east-1: Oct 23, 2024, 4:00 PM PT - 8:00 PM PT
- us-east-2: Oct 23, 2024, 4:00 PM PT - 8:00 PM PT
- us-west-2: Oct 24, 2024, 5:00 AM PT - 9:00 AM PT
- Added support for FixedDecimal and Int128 number types for many trigonometric operators: sin, sinh, asin, asinh, cos, cosh, acos, acosh, tan, tanh, atan, and atanh.
- Added automated cleanup of outdated log files in the app_state.client_log_stage based on a 30 day retention period. This will reduce storage costs and ensure compliance with data retention policies. No customer action is required to enable this feature.
- Fix bug in cdc_status where it would fail if there were multiple entries for the CDC engine returned through
api.enginesdue to incomplete cleanup of the resources associated with older incarnations of the CDC engine. - Fix a bug in delete_engine where it did not fail when dropping engine service was not successful
- The deprecated procedures
app.resume_service()andapp.suspend_service()were removed,app.activate()andapp.deactivate()should be used instead (documentation).
- Fixed an issue that blocked upgrades for accounts that had both the RelationalAI Service running and CDC enabled.
- Ongoing transactions will be marked as aborted when an engine is deleted or under upgrading.
- The application now manages the engine used to run CDC transactions, instead of requiring the user to create an engine and instructing the application to use it. See updated documentation around CDC Stream Management.
- The
relationalai.app.resume_cdc()andrelationalai.app.suspend_cdc()continue to be the way to manage CDC functionality. The CDC engine is not immediately created when CDC is resumed, it will instead be created when there is data to replicate to the RelationalAI App through Data Streams. - Adjust CDC engine size with the
relationalai.app.alter_cdc_engine_size(size string)procedure. The default size is HIGHMEM_X64_S.relationalai.app.cdc_status()has been updated to describe whether CDC is enabled and details about the engine (configured name and size, and current status). - Existing user-created engines assigned to CDC are retained but are no longer used for CDC, and can be deleted/continued to be used for other purposes as appropriate.
- Calling
relationalai.app.setup_cdc()is no longer necessary, and will invokeresume_cdc()instead.setup_cdc()is now deprecated and will be removed in a future release. - CDC starts suspended by default upon installation.
- The CDC engine does not generate RelationalAI surcharges, and is not meant for user submitted transactions.
- The
-
relationalai.app.finalize_upgrade, used in the Native App upgrade process, no longer attempts to resume CDC if it is not enabled.
- The
compute_poolargument is ignored for overloads ofrelationalai.api.create_enginethat have it, and engines will be hosted in application managed pools instead of the provided pool. These overloads will be removed in a future release, and versions ofcreate_enginethat don’t take the compute pool arguments should be used instead. -
relationalai.app.create_service()is deprecated and will be removed in a future release.relationalai.app.activate()should be the procedure to use instead in future instances, and whenrelationalai.app.create_service()is calledrelationalai.app.activate()will be invoked instead.
- Added support for FixedDecimal and Int128 number types for some math operators: sqrt, cbrt, log, and sign.
- It is no longer necessary to call
app.finalize_installafter installing the Native App, this procedure will be fully removed in a later release. -
app.start_servicewas fully removed from the application,app.activateshould be used instead.
- The
transitive_closureof a given node now includes the node itself if there exists a path from the node to itself. For example: 1) thetransitive_closureof a node in a directed cycle now includes the node; 2) thetransitive_closureof a node with a self loop, and no other edges, now includes the node; 3) thetransitive_closureof a node in a directed path graph, with no other edges, does not include the node; and 4) thetransitive_closureof a fully isolated node, i.e. one with no edges, does not include the node.
- The relation
reachable_from, which for a given node provides the set of nodes reachable from that given node, has been added to the graph library. Note that this relation is similar to, but subtly different from,transitive_closure; the key difference is that a given node is always reachable from itself, independent of whether there is a path from the node to the node. - Documentation of the graph library’s
diameter_rangerelation was improved, clarifying thatdiameter_rangereturns the diameter range for the largest (weakly) connected component (instead of infinity) if the graph is disconnected.
- Restored
app.suspend_service()andapp.resume_service()with stub implementations that warn callers of their deprecation. These procedures will be removed in a later release. Therelationalai.app.activate()andrelationalai.app.deactivate()procedures should be used instead of manage the application’s resources (docs). - An example for the graph library’s
adamic_adarrelation has been fixed. - An example for the graph library’s
preferential_attachmentrelation has been fixed.
- The
transitive_closureof a node in a directed graph no longer includes the node itself. - The
preferential_attachmentof a pair of nodes where at least one of them is an isolated node, will return zero. This change means that, now, for every pair of nodes in the graph, we will get a numerical result (as opposed to {} / false previously) -
cosine_similarityover directed graphs was previously based on neighbors (both in- and out-neighbors); it is now based solely on out-neighbors, matching the behavior ofjaccard_similarity. -
weighted_cosine_similaritypreviously produced incorrect results in some cases, and was not consistent withcosine_similarity; it is now consistent withcosine_similarityand produces correct results in all known cases. Results will change in most cases.
- Added the
relationalai.app.activate()andrelationalai.app.deactivate()procedures as a one-call interface to turn application functionality and application compute consumption on and off (docs). This includes the RelationalAI service, engines, CDC functionality, and the underlying Snowflake warehouses and compute pools. - The
relationalai.api.enginesview now has a newversioncolumn describing the engine version for each entry.
- It is no longer possible to create the RelationalAI service using user-created compute pools and warehouses. The
relationalai.app.start_service(...)procedure now ignores all arguments and executesrelationalai.app.create_service()instead. Those manually created resources can be dropped/repurposed as appropriate. - The deprecated
relationalai.app.suspend_serviceandrelationalai.app.resume_serviceprocedures have been removed. - A future release will remove the ability to specifically create/start the RelationalAI SPCS service, removing the
create_service,start_service, anddrop_serviceprocedures from therelationalai.appschema. The newactivate()anddeactivate()procedures should be used instead.
- The RelationalAI service will no longer be unintentionally resumed after being suspended.
-
::graphlib::betweenness_centrality’s node-sampling strategy was changed to improve approximation accuracy. For graphs with fewer than one hundred nodes, results should not change. For most graphs with more than one hundred nodes, results should not change substantially. For some graphs with more than one hundred nodes, results may change substantially, with that change representing a significant improvement in approximation accuracy. -
::graphlib::pagerankpreviously could produce NaNs over graphs with zero-weight edges. This has been fixed; pagerank should no longer produce NaNs over such graphs.
- If engines existed while the RelationalAI service has been dropped, those engines will not be usable for around 30 seconds after the RelationalAI service has fully started up. Submitting a transaction to those engines will yield an error stating that the engine is provisioning.
- The weighted jaccard similarity algorithm (
::graphlib::weighted_jaccard_similarity) no longer produces explicit zero scores; results are now sparse, rather than dense with explicit zeros.
- If engines existed while the RelationalAI service has been dropped, those engines will not be usable for around 30 seconds after the RelationalAI service has fully started up. Submitting a transaction to those engines will yield an error stating that the engine is provisioning.
- The
transitive_closureof a node in an undirected graph no longer includes the node itself. -
is_reachableno longer indicates that a node is reachable from itself in undirected graphs. -
reachable_fromno longer indicates that a node is reachable from itself in undirected graphs. -
degree_centralitynow produces0.0for isolated nodes without self loops, instead ofNaN. -
degree_centralitynow produces1.0for isolated nodes with self loops, instead ofinf. -
jaccard_similarityover directed graphs previously produced scores based on neighbor counts, inconsistent withweighted_jaccard_similarity, which produces scores based on out-neighbor counts over directed graphs.jaccard_similarityover directed graphs now produces scores based on out-neighbor counts, consistent with the corresponding behavior ofweighted_jaccard_similarity.
- This release includes minor updates and improvements to enhance the overall performance and stability of the system.
- Mitigate the engine provision issue caused by a Snowflake release change.
- Action: Customers must upgrade their engines if any HIGHMEM_X64_S size engines are running in their account.
- Please see the note on
graphlib::is_connectedin theFixessection below.
- Fixed a bug that caused Data Streams to be stuck in the DELETING state.
-
graphlib::is_connectedpreviously indicated that empty graphs are disconnected, counter to convention.graphlib::is_connectednow indicates that empty graphs are connected.
- Please see the note on
graphlib::triangle_communityin theFixessection below.
- New procedures & functions were added to help with upgrading all customer engines to the latest version:
-
call relationalai.app.upgrade(): Starts the engines upgrade process in the background. -
select * from relationalai.app.upgrade_status: Returns the current status of the upgrade. -
select * from relationalai.app.upgrade_status_details: Returns the current status of the upgrade per engine. -
call relationalai.app.finalize_upgrade(): Concludes the upgrade.
-
- Enable high throughput disks for select customers.
- Prevent the app’s internal Task from waking up a suspended RAI Service.
-
graphlib::triangle_communitydoes not support directed graphs, but previously could produce non-empty results when given a directed graph. triangle_community now consistently produces empty results when given a directed graph. - The edge list in an example in
graphlib::louvain’s documentation (of a heavy dumbbell embedded in a six-clique) was corrected.
- Engines can now be hosted on resources created and managed by the application, using the
relationalai.api.create_engine(<name>, <size>)procedure. The existing variants of therelationlai.api.create_engine()procedure that take the name of a user-provided compute pool can still be used, but are now deprecated and will be entirely removed in a future release.- To do so, the app now requires the CREATE WAREHOUSE and CREATE COMPUTE POOL privileges. To transition existing installations to the RAI
Service hosted on app-managed resources, run the commands below; this will result in a brief downtime of the app (~2-5 minutes).
grant create compute pool on account to application relationalai; grant create warehouse on account to application relationalai; call relationalai.app.drop_service(); call relationalai.app.create_service(); -- to move each engine to be hosted on app-created resources: -- note name and size call relationalai.api.delete_engine(<name>); call relationalai.api.create_engine(<name>, <size>); - A procedure
relationalai.api.alter_engine_pool_node_limits(<size>, <min>, <max>)has been added to help manage limits on how many of each engine size can be created with the application.
- To do so, the app now requires the CREATE WAREHOUSE and CREATE COMPUTE POOL privileges. To transition existing installations to the RAI
Service hosted on app-managed resources, run the commands below; this will result in a brief downtime of the app (~2-5 minutes).
- A bug has been fixed where certain uncommon type conversions caused a system internal error.
- This release includes minor updates and improvements to enhance the overall performance and stability of the system.
- The RAI Service can now be run with resources created and managed by the application (a X-SMALL warehouse and 1-node CPU_X64_S compute pool) , using the
relationalai.app.create_service()procedure. To do so, the app now requires the CREATE WAREHOUSE and CREATE COMPUTE POOL privileges. The existingrelationlai.app.start_service(<compute pool>, <warehouse>)procedure using user-provided resources can still be used, but it is now a deprecated path and will be entirely removed in a future release.- To transition existing installations to the RAI Service hosted on app-managed resources, run the commands below; this will result in a brief downtime of the app (~2-5 minutes).
grant create compute pool on account to application relationalai;
grant create warehouse on account to application relationalai;
call relationalai.app.drop_service();
call relationalai.app.create_service();
- Lowered the memory and cpu resources for engines of the HIGHMEM_X64_S and HIGHMEM_X64_M sizes based on snowflake recommendations.
- Resolved a vulnerability that could lead to tokens being logged.
- While using
api.execto start a transaction, if the response size exceeds the Snowflake response limit, RAI will return an error directly instead of a Snowflake error. - The abs function now supports Period types (Year, Month, Day, Millisecond, etc.)
- Example: equal(abs[^Day[-5]], ^Day[5])
- Improved the error message for cases where the arity of a vararg native cannot be determined. Previously, this threw an INTERNAL_COMPILER_ERROR. It now throws an UNKNOWN_ARITY error.
- Fixed an issue where the first set of concurrent transactions against a newly created database could fail.
- Deleting the CDC engine using
api.delete_engine(<engine>, true)will now suspend CDC as well. - Added two new columns
last_transaction_idanderrorsto the results ofapi.get_data_streamto improve Data Stream debuggability. - Added a
api.data_stream_errorsview to expose Data Stream errors.
- CoreRel deprecation warnings now have their own code of COREREL_DEPRECATION instead of presenting as a PARSE_PROBLEM.
- Deprecated procedures/functions
api.use_database,api.get_database,api.use_engine, andapi.get_enginehave been removed, along with procedure overloads that relied on these implicit values (api.load_data,api.exec,api.exec_into). Explicit arguments should be used instead. - Deprecated CDC management procedures in the
apischema have been removed. The equivalent versions in theappschema should be used instead.
- Introduced new engine sizes HIGHMEM_X64_S and HIGHMEM_X64_M to depreciate the engine size HighMem|S and HighMem|M.
- Improved retry handling for transactions.
- Abort non-retriable transactions directly.
- Return an error if a transaction occurs during engine provisioning.
- Improved validation during data stream creation:
- Column type validation is now performed when data streams are created. Previously the validation would be done when the data streams ran, leading to a delayed and unobvious failure response.
- Added a check to ensure that the target relation of a data stream is empty during data stream creation.
- Errors due to incorrect table references have been made clearer.
- Data streams now write a unique ID column into the target relation to make it easier and more efficient to retrieve the full set of IDs of the relation.
- The
api.get_engineUDF andapi.enginesview now return the compute pool that each engine service is hosted in. - Better support for data loading natives when used in combination with outline abstractions.
- Fix bug where a relative namespace path that happened to match a fully qualified namespace path would resolve the fully qualified path.
- Improve heuristic for emitting arity mismatch diagnostics on partial applications.
- loose_eq definition now uses ::std::mirror::try_convert so that any custom overloads of the latter will get used in evaluating loose_eq.
- Infomap’s ergodic node visit frequency calculations have changed, improving accuracy and convergence rate. Community assignments may change, nominally to an equivalent or more correct assignment.
- Increased the database name length limit from 63 to 200.
- Excluded engines in pre-pending state in list engines api.
- Exposed engine name in get_transaction and list transactions apis
- When possible, the path that was in the process of being exported is now reported when a runtime exception happens during CSV export
- Performance improvements for label propagation, Louvain, and Infomap.
- Constant fold missing = missing to true
- The mirror metamodel was incorrectly reporting some generated_from dependencies in the eval phase as declarations when they were in fact base relations - this has been corrected
- In some cases, the range was not reported with a runtime exception when it could have been. These ranges are now present.
- Label propagation, Louvain, and Infomap were augmented with convergence acceleration strategies that significantly reduce end-to-end run times. These convergence acceleration strategies also change results in a manner similar in kind and degree to changing the randomization seed. (Where community structure is unambiguous, community assignments are unlikely to change. Where community structure is ambiguous, community assignments may change, nominally to an equivalently correct assignment.)
- Enhanced retry and http return code if the engine is in pending or failed status.
- Two new engine states were introduced: orphaned & gone. Each represents an (inconsistent) state of missing engine information.
- Improved handling of outlining in the presence of untyped variables
- 0.0 = -0.0 now consistently returns true
- Enforce that extension of declarations in other namespaces only works when that relation has already been declared in that other namespace
- Fixed recording of consumption of engines created with the HighMem|S and HighMem|M sizes
-
Improvements to the
::graphlib::directedand::graphlib::undirectedconstructors. These constructors now:- directly accept an edge/weight list, as they did previously, and yield the same result for valid graphs;
- can also accept a relation containing an
:edgerelation, a:noderelation, both, or neither (matching our other parameterized algorithms); - perform basic validation of the provided edge/weight/node relations. The following validations are currently supported:
- node relation is exclusively arity one,
- edge/weight relation is exclusively arity two or exclusively arity three,
- and arity-three edge (i.e. weight) relations are functions i.e. do not contain multiple-weight edges);
- yield false if the constructor arguments do not validate; and
- can provide a report on the validation failure if
:diagnosticsare requested.
-
Enhanced http return codes if the RelationalAI transaction is aborted or canceled.
- Label propagation, Louvain, and Infomap now more gracefully handle edge cases when embedded in recursion.
- If a relation is passed to a reduce function that does not have at least arity 1, this will emit an arity mismatch warning.
- The rel-compat model is no longer included in new databases and when the update libraries procedure is run, it will be removed.
- In the last release we started emitting warnings for any usage of relations in this model. If you didn’t see anything then, you should be unaffected.
- CoreRel deprecation warnings will be emitted for any Rel that does not conform to the CoreRel grammar.
- Please see the cheat sheet for details on CoreRel and how to migrate old Rel to CoreRel. This link is also included in all relevant deprecation warnings.
- Introduced two customer face engine sizes, i.e., HighMem|S and HighMem|M.
- Introduced two more rai transaction states, i.e, Queued and Running.
- Introduced erp profiling endpoints to allow engine profiling in SPCS.
- Enhanced transaction aborted reasons and http return codes.
- Libraries are now versioned. They each have a version relation (e.g. ::std::version) that can be queried, logged, and returned after library updates.
- Improved edge case handling for label propagation, Louvain, and Infomap.
- On-the-fly validation of edge weights for label propagation, Louvain, and Infomap. (NaN weights, +/-Inf weights, and multiple-weight edges now yield empty results and a validation failure report rather than triggering exceptions.)
- Introduced RAI Schema feature. (https://github.com/RelationalAI/rai-sf-app-docs/wiki/Guide-%E2%80%90-Using-the-Application#rai-schemas)
- Added new application roles to allow finer control of privileges. (https://github.com/RelationalAI/rai-sf-app-docs/wiki/Guide-%E2%80%90-Using-the-Application#application-roles)
- Application level CDC management procedures have been moved under the
appschema (setup_cdc,cdc_status,resume_cdc,suspend_cdc). The versions in theapischema will be deprecated in a later release. - Added a variant of
exec_intothat infers column names and types from the Rel query, removing the need to explicitly specify them.
- Power (^) now has the same error-on-overflow semantics as all other arithmetic operations.
- Typeof no longer leaves behind runtime cruft, which could cause erroneous result filtering.
- Previously, the default number of maximum iterations for PageRank was 100, now it is 20.
- To continue to see the old behavior, you can set the maximum number of iterations manually, as follows: def result { pagerank[{(:graph, G); (:max_iter, 100)}] }
- PyRel users should not be affected by this change.
- Previously, the default number of maximum sweeps for Label Propagation was 1000, now it is 20.
- To continue to see the old behavior, you can set the maximum number of iterations manually, as follows: def result { label_propagation[{(:graph, G); (:max_sweeps, 1000)}] }
- PyRel users should not be affected by this change.
- JSON loading will now load names as the ::std::json::Name value type instead of a meta value string.
- Example:
- load_json[{:data, “””{“a”: 1}”””}]
- Previous behavior: (:a, 1)
- New behavior: (::std::json::^Name[“a”], 1)
- Example:
- Usage of relations in the
rel-compatmodel will now emit deprecation warnings. In the next release, we will no longer add therel-compatmodel to new databases and updating libraries will remove it.
- A null character in a CSV header would previously cause the transaction to abort. This is now a user-facing diagnostic.
- The restrictions for namespaces are now better enforced. It is now an error to include a namespace or import declaration inside a module or with/use block.
- Reducing with in an initial value over an empty body would previously produce different values depending on if the empty body was statically known or dynamically known. These now both consistently return false.
- Engine & RAI database resource names are now case-sensitive consistently. Previously, engine names were case-insensitive and database names were case-sensitive on writes but case-insentive on reads.
- pagerank and label propagation briefly directly accepted a graph (in addition to accepting a config relation containing a graph); this ability has been removed.
- As an OOM prevention measure the back-end will now reject very large inputs beyond a predefined threshold, and the back-end will throw an exception if it receives the input larger than this threshold. The default threshold is 10000, and this threshold can be set via the new relconfig parameter :ast_size_exception_threshold, e.g.:
def insert[:relconfig]: {(:ast_size_exception_threshold, 20000)}.
- Engine names are now limited to 50 characters, and a 12-character engine ID has been introduced to engine metadata.
- Improved reporting of errors originating from Snowflake
- Louvain, Infomap, and Label Propagation now support isolated nodes.
- A warning is now raised when the back-end receives an input with a size larger than a predefined threshold. The default threshold value is 1000, and this threshold can be set via the new relconfig parameter :ast_size_warning_threshold, e.g.:
def insert[:relconfig]: {(:ast_size_warning_threshold, 5000)}.
- Concurrency bug causing ERP crashes has been fixed
- Misreporting the engine status when a create database transaction fails has been fixed
- Deletion of an engine during creation was misreported as an internal error
- An issue where Louvain, Infomap, and label propagation would not function under certain conditions when embedded in recursion has been fixed.
- The Snowflake Warehouse supplied to the application must have auto_resume set to true, otherwise the RAI Service will get stuck in retry loops and not be able to perform its functions.
- load_data and CDC will now use RAI’s 128-bit integer type when mapping Snowflake integers to RAI types. Previously Snowflake NUMBER types that didn't actually fit in int64 were mapped to int64 anyways; if int64 behaviour should be preserved, the source Snowflake table should have the relevant columns converted to number(18,0) instead.
- Fixed an issue that kept warehouses from suspending, thus incurring unexpectedly high cost
- Improved performance by using a new metadata layer with memory cache.
- Implemented automatic engine resume functionality when transactions are requested for a suspended engine.
- Added a api.cdc_status procedure to describe the state of CDC’s settings on the application.
- Improved CDC telemetry.
- Louvain now supports several parameters (
sweep_tolerance,level_tolerance,max_sweeps, max_levels, andrng_seed`), its logs are richer, and its cancellation behavior is more consistent and responsive. - Label propagation’s
:diagnosticsnow include termination information and a convergence summary, providing insight into the algorithm’s behavior and facilitating parameter selection and tuning.
- Fixed the issue where engines could not be deleted following a breaking change introduced by Snowflake in a recent release.
- Fixed unexpected errors occurring when transaction requests were made to a pending engine.
- An issue with Louvain’s and Infomap’s behavior on challenging graphs requiring more than two levels for convergence was caught and fixed; convergence and/or solution quality should be noticeably better on large and/or challenging graphs.
-
::graphlib’s graph constructors,undirected_graphanddirected_graph, now accept weighted edge relations containing Int128 weights (which are normalized to Float64s).
- No breaking changes in this release.
- The RAI service and its engines can get suspended due to the underlying Snowflake compute pools getting into a suspended state.
- Increased the engine provision timeout from 20 mins to 30 mins.
- Improved the engine provision error handling.
- No new fixes in this release.
- No breaking changes in this release.
- Deleting a RAI database that is the target of a Data Stream breaks the Data Stream, and prevents new Data Streams from being created on the source table of the original Data Stream.
- The RAI service and its engines can get suspended due to the underlying Snowflake compute pools getting into a suspended state.
- Implemented engine provision timeout feature. If the create_engine call exceeds 20 minutes, the create_engine API call will fail.
- Improved label propagation cancellation responsiveness.
- Restricted deletion of the engine assigned to CDC, unless the new force override is used:
CALL relationalai.api.delete_engine('my_eng', true);
- Improved overall logging, including improvements to infomap logging. (Infomap’s ergodic node visit rate calculation now heartbeats.)
- Filtered out internal operations in api.transactions view.
- Improved CDC batch management by consolidating all pending batches into a single REL query per database.
- Replaced Infomap’s
max_conde.nsationsparameter with amax_levelsparameter, providing the same functionality but more intuitively. To transition from the old parameter to the new parameter, simply replacemax_condensationswithmax_levelsand increment the associated number by one. - Replaced Infomap’s
condensation_toleranceparameter with alevel_toleranceparameter, providing the same functionality more intuitively. To transition from the old parameter to the new parameter, simply replacecondensation_tolerancewithlevel_tolerance.
- Deleting a RAI database that is the target of a Data Stream breaks the Data Stream, and prevents new Data Streams from being created on the source table of the original Data Stream.
- The RAI service and its engines can get suspended due to the underlying Snowflake compute pools getting into a suspended state.
- Improvements were made to make data streams handle changes from the source table/view more quickly.
- The application’s CDC mechanism now groups changes together to form larger transactions for loading data into RAI when possible, which will help in catch-up scenarios where changes from the source object have accrued but have not yet been loaded into RAI.
- Added api.transactions view to list RAI transactions. Queries will be included in a future release.
- Added check to app.suspend_service to prevent accidentally breaking CDC by suspending the central RAI service.
- The engines view now returns created_by, created_on, and updated_on.
- The engine & database views include the user that created the entries ("created_by").
- The database entries are deleted immediately from the api.databases view, i.e. no accumulation of old entries.
- Improved cancellation responsiveness of label propagation, Louvain, and Infomap.
- Improved logging of label propagation (all early returns paths logged, all parameters logged, additional execution checkpoints logged).
- Reduced default number of sweeps label propagation performs to 1000.
- Graph library has been added to the ::graphlib namespace.
- Data streams created on empty tables or views previously did not function, this was fixed.
- Deleting a RAI database that is the target of a Data Stream breaks the Data Stream, and prevents new Data Streams from being created on the source table of the original Data Stream.
- The RAI service and its engines can get suspended due to the underlying Snowflake compute pools getting into a suspended state.
- Fix the issue with database upgrades that were failing when using the
UPDATE_LIBRARIES.
- The storage layer of the engine was changed to be backed by a network attached block storage device that can deliver better performance compared to local disks.
- The graph library API was extended. More specifically,
label_propagationcan now directly accept a graph to operate on in addition to a configuration relation pointing to such a graph. - The get_engine() procedure now returns more columns (version, created_by, created_on, and updated_on) in addition to the existing ones (name, size, status).
- None.
- Deleting a RAI database that is the target of a Data Stream breaks the Data Stream, and prevents new Data Streams from being created on the source table of the original Data Stream.
- The RAI service and its engines can get suspended due to the underlying Snowflake compute pools getting into a suspended state.
- The graph library was extended with new algorithms that can compute:
- the betweenness centrality measure, the local clustering coefficient measure, and the average clustering coefficient measure for the nodes of a graph.
- a range estimate for the diameter of a graph.
- the transitive closure of a graph.
- The query optimizer was improved to avoid generating query plans that make use of cross products in certain cases.
- Memory management under high memory utilization was made more robust.
- None.
- The RAI service and its engines can get suspended due to the underlying Snowflake compute pools getting into a suspended state.
- Starting with this release, it is no longer necessary to recreate databases after an upgrade. The stored procedure
relationalai.api.update_librariesshould be called instead to upgrade their databases. See the guide on RAI application upgrade for more details. - Added the ability to pass input parameters to Rel transactions, allowing users to parameterize their queries and/or prevent having to deal with string interpolation and escaping (reference).
- Added more SQL UDFs and Procedures for managing Rel models, e.g. listing, retrieving model code, deleting models (reference).
- Removed restrictions on the model code passed to
relationalai.api.load_model_code, previously imposed to prevent breaking out of string escaping. - Added the
relationalai.api.cancel_transactionprocedure (reference) to cancel running RAI transactions. - Added the
relationalai.app.get_releaseUDF to identify the version of the deployed code in terms of the versioning described in these release notes. - Standardized file formats when exporting data from Snowflake (
relationalai.api.load_data, or for CDC) to ensure consistent representation of data independent of any Snowflake session settings. - Improved observability around billing and consumption measurement mechanisms.
- Improved label propagation's freezing/thawing strategy to reduce the likelihood of some modes of probabilistic non-convergence. (This change also impacts convergence behavior overall: convergence should require fewer, but commensurately more expensive, sweeps.)
- load_data now produces a relation with tuples of arity 3, to make the relation’s shape consistent with that of CDC. Previously load_data produced a relation of arity 4, with two of the values used together to uniquely identify a row; those two values have been combined into one value. Any Rel logic depending on the Rel relations persisted by load_data should be updated to consume this new form.
- Deleting a RAI database that is the target of a Data Stream breaks the Data Stream, and prevents new Data Streams from being created on the source table of the original Data Stream.
- The RAI service and its engines can get suspended due to the underlying Snowflake compute pools getting into a suspended state.
- Enabled connecting RAI tooling (e.g. VSCode extension and PyRel) to RAI hosted on Snowflake
- Improved observability for graph algorithms (Label Propagation, Infomap and Louvain), including progress logging and termination status. Extend Label Propagation algorithm with more robust handling around some of the possible graph shape-dependent cases of non convergence.
- Deleting a RAI database that is the target of a Data Stream breaks the Data Stream, and prevents new Data Streams from being created on the source table of the original Data Stream
- Documentation moved out of the Streamlit app to this GitHub Wiki, to increase usability and ease of maintenance
-
relationalai.api.exec_intonow returns an object, conveying the rows returned, target table, and RAI transaction ID -
relationalai.api.load_datanow declares that it RETURNS OBJECT explicitly; it always was returning an object but previously was declared RETURNS VARIANT - Enable users to configure and tune graph algorithms parameters, such as maximum number of iterations for Label Propagation and Infomap (please refer to the graph algorithms library documentation for the full list of user facing parameters).
- Stopped logging exception details to prevent un-reviewed error/exception content from being sent automatically to the shared event table
- Fix some attributes in list engines being mixed up
- Ensure transaction cancellation is working as expected for graph algorithms Label Propagation, Infomap and Louvain.
- Changed interface to create data streams to no longer require passing in redundant values
- Existing Data Streams need to be recreated
- User access to Service Functions in the SPCS schema removed; use the corresponding functions/procedures in the API schema instead