Pick upstream mainnet-v.1.36.2

[kodemartin]

Description

This tracks the selective pull of upstream changes up to mainnet-v1.36.2.

Please convert to issue when selecting a task from the list, add the infrastructure label, and assign yourself to the issue.

• chore: improve how diesel is depended on {"side-effects": "minor", "type": "refactoring"}
  sui, cluster-test: remove direct dependency on diesel {"side-effects": "minor", "type": "refactoring"}
• graphql/events: Disable filtering on both event_type and emitting_module {"side-effects": "medium", "type": "optimization"}
• [indexer][graphql] Pruner prunes only epoch-partitioned tables (#19164) {"side-effects": "minor", "type": "fix/refactoring"}
• indexer: rework config and cli arguments (#19162) {"side-effects": "medium", "type": "ui improvement/breaking"}
• [indexer] objects-snapshot-processor refactoring {"side-effects": "minor", "type": "refactoring"}
  * [graphql] Remove usage of legacy objects snapshot processing (#19175)
  * indexer: clean up legacy snapshot codes (#19078)
  * [indexer] Objects Snapshot Processor continuously reads from stream (#19232)
• [indexer] Compatibility check using migration records (#19156) {"side-effects": "minor", "type": "fix/refactoring"}
  * indexer: relax migration check (#19558)
  fix(indexer): Leverage index for singleton query (#19247) {"side-effects": "minor", "type": "fix"}
  fix(indexer): Store JSON-RPC URL in String (#19248) {"side-effects": "minor", "type": "fix", "comment": "we could still use Url and pass the proper string to jsonrpsee"}
  indexer: chunk protocol_config commits {"side-effects": "minor", "type": "optimization/fix"}
• indexer reader: derive dynamic field info #5049
  • indexer: drop df columns and refactoring (#19308)
• pg indexer: split executors in non-colocated setup {"side-effects": "minor", "type": "optimization/fix"}
• indexer/graphql: Minor cleanup on ObjectStatus::WrappedOrDeleted {"side-effects": "minor", "type": "refactoring"}
• indexer: split object version ingestion for perf {"side-effects": "medium", "type": "optimization"}
• indexer: ensure schema always matches what diesel generated #5008
• indexer: index tx_affected_addresses (#19355) {"side-effects": "medium", "type": "feature"}
  * indexer: deprecate TransactionFilter::ToAddress (#19616)
• ingestion: change Worker api to take &CheckpointData {"side-effects": "minor", "type": "refactoring/optimization"}
• indexer json rpc: batch object db reades (#19392) {"side-effects": "minor", "type": "optimization"}
• BackFill features {"side-effects": "major", "type": "feature"}
  * indexer: SQL Backfill command (#19359) {"side-effects": "major", "type": "feature"}
  * Indexer: A few improvements to backfill tool (#19441)
  * Indexer: Add a generic backfill template
  * Indexer: Add ingestion based backfill
  * indexer: backfill tx_affected_objects
  * indexer: tx_affected_objects ingestion-based backfill
  * Indexer: Fix a bug in epochs system state json backfill
  * fix(backfill): chunk up writes to DB
  * chore(indexer): drop redundant curly braces
  * chore(indexer): reduce visibility of crates
  * fix(indexer): fix default db path in SQL ingestion scripts
  * indexer: backfill for tx_affected_addresses
  * graphql: Filter by affected address
  * graphql: unstage affected_addresses
  * graphql: tx_senders -> tx_affected_addresses
• feat(indexer): index tx_affected_objects (#19447) {"side-effects": "medium", "type": "feature"}
  * feat(indexer): Prune tx_affected_objects
  * indexer: support TransactionFilter::AffectedObject
  * indexer: deprecate TransactionFilter::{Input, Output}Object
  * indexer: affected objects include created+wrapped/unwrapped+deleted
  * indexer-reader: remove references to tx_senders
  * indexer: stop indexing tx_senders and tx_recipients
  * graphql: filter transactions by affected object
• Indexer: Add json serialized SuiSystemStateSummary to epochs table (#19428) {"side-effects": "medium", "type": "feature/fix"}
  * [Indexer] Stop reading old system_state in epochs (#19532)([Indexer] Stop reading old system_state in epochs MystenLabs/sui#19532)
• indexer: Adjust transactions queries to the latest db schema {"side-effects": "medium", "type": "refactoring"}. Done in [iota-indexer]: Adapt transactions queries to lack of index on digest column #4586
• Indexer: Break dependency between pruner and epoch watermark {"side-effects": "minor", "type": "optimization/fix"}
• Indexer: Support 2-day retention for objects_history table {"side-effects": "minor", "type": "feature"}
• indexer: add indices to aid pruning {"side-effects": "medium", "type": "optimization"}
• indexer: fix unwrap after wrap bug in a checkpoint batch {"side-effects": "medium", "type": "fix"} #5020
• indexer easy: add sender to events to deprecate senders {"side-effects": "minor", "type": "feature/breaking"}
• Indexer: Do not prune epochs table {"side-effects": "minor", "type": "refactoring"}
• events: Remove unsupported filters {"side-effects": "minor", "type": "refactoring/breaking"}
  * subscriptions: reinstate EventFilter::Any
• indexer(visitor): avoid fully deserializing dynamic field {"side-effects": "medium", "type": "optimization/fix"} (handled by #19518,#19519,#19520)
  * indexer(visitor): avoid fully deserializing dynamic field on write path
  * indexer: no need to deserialize dynamic fields
  * graphql: avoid DynamicField to-and-fro
• segregated indexer: handler trait and migrate snapshot_objects table {"side-effects": "medium", "type": "refactoring"}
• jsonrpc: add instrumentation to read apis {"side-effects": "minor", "type": "feature"}
• Watermarks {"side-effects": "major", "type": "feature"}
  * indexer: Modify PruningOptions to point to a toml file of epochs_to_keep and optional per-table overrides
  * indexer: committer writes upper bounds to watermarks table
  * indexer: rename watermarks table fields to clarify inclusive upper bounds
  * indexer: pruner updates watermarks lower bound
  * indexer: align watermarks table schema in live indexer to alt indexer
  * indexer: committers should read watermark hi directly from table
• indexer: latency metrics for objects version {"side-effects": "medium", "type": "improvement"}
• indexer fix: chunk to avoid PG parameter limit (#19754) {"side-effects": "medium", "type": "fix/critical"}
• data ingestion: introduce reducer functionality {"side-effects": "medium", "type": "feature"}
• indexer ingestion: more eager commit
• indexer: execute objects mutation & deletion in parallel {"side-effects": "medium", "type": "optimization"}
• indexer: exclude deleted/wrapped objects from snapshot {"side-effects": "medium", "type": "refactoring/breaking"}
  * graphql: return live objects from consistent object query
• indexer: Add first_tx_sequence_number to epochs table to decouple from checkpoints table {"side-effects": "medium", "type": "optimization/fix"}
• indexer handler: limit unprocessed buffer size {"side-effects": "minor", "type": "fix/critical"}
• indexer: handle sui safe mode {"side-effects": "medium", "type": "fix"}
• GraphQL: Cache chain identifier & use ticks {"side-effects": "medium", "type": "optimization"}
• GraphQL: fix health check acting up on tests {"side-effects": "minor", "type": "fix"}
  * GraphQL: Fix more tests failing due to not waiting long enough
• GraphQL: Add a mutation payload size {"side-effects": "medium", "type": "optimization"}
• graphql: Remove explicit cleanup_resources {"side-effects": "minor", "type": "refactoring"}
• graphql: Remove references to df_object_id {"side-effects": "minor", "type": "refactoring"}
• graphql: document transaction, event filters {"side-effects": "minor", "type": "docs"}
  * graphql: explain scan limits rationale
  * graphql: Update deprecation notices for fields to be removed
• graphql: tests for staging variant {"side-effects": "medium", "type": "refactoring"}
• graphql: deprecate tx signer filters {"side-effects": "medium", "type": "breaking"}
  * graphql: Remove TransactionBlockFilter.signAddress and AddressTransactionBlockRelationship.SIGN {"side-effects": "minor", "type": "refactoring"}
  * graphql: Remove TransactionBlockFilter.recvAddress and AddressTransactionBlockRelation.RECV
• graphql: flakiness in pruning test {"side-effects": "minor", "type": "fix"}
• graphql: deduplicate configs and clap args {"side-effects": "medium", "type": "refactoring"}
  * graphql: add flag to skip db compatibility
  * graphql: restore --node-rpc-url flag
• graphql: return empty results when provided object lookup keys are empty {"side-effects": "minor", "type": "fix"}
• GraphQL: Rework Event type {"side-effects": "medium", "type": "refactoring/breaking"}
• graphql: add query epochs functionality {"side-effects": "medium", "type": "feature/ui"}
• graphql: Event.transactionBlock {"side-effects": "medium", "type": "ui improvement/breaking"}
• graphql: ObjectOwner::Parent exposed as Owner {"side-effects": "medium", "type": "refactoring"}
• graphql: expose MovePackage as BCS (packageBcs) {"side-effects": "medium", "type": "refactoring"}
• GraphQL: Serialize bcs bytes of TransactionData instead of SenderSignedData {"side-effects": "medium", "type": "refactoring/breaking"}
• data ingestion: disable inotify for macos {"side-effects": "minor", "type": "fix"}
• data ingestion: handle termination in worker pool {"side-effects": "minor", "type": "fix"}
• data ingestion: introduce upper limit for ingestion job {"side-effects": "minor", "type": "feature"}
• data ingestion: Some config changes & cleanups {"side-effects": "medium", "type": "docs", "comment": "the parent PR is actually on the bridge, we are only interested on the doc change"}
• ingestion: reduce memory footprint by wrapping checkpoints in Arcs {"side-effects": "minor", "type": "optimization"}
• data ingestion: kv store: skip items over the DDB limit {"side-effects": "minor", "type": "fix"}
• json-rpc: get transaction block raw effects {"side-effects": "minor", "type": "feature"}
• rpc: reword errors to transaction validator signing {"side-effects": "minor", "type": "fix/docs"}
• jsonrpc: log origin headers for json-rpc {"side-effects": "minor", "type": "feature"}
• json rpc: retry on empty events {"side-effects": "minor", "type": "fix"}
• jsonrpc: populate object cache with input objects {"side-effects": "minor", "type": "feature"}
• jsonrpc: add instrumentation to read apis {"side-effects": "minor", "type": "feature"}
• jsonrpc: add test for sorted coin balances {"side-effects": "minor", "type": "test"}