Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rpc-alt: showDisplay #21333

Open
wants to merge 1 commit into
base: amnn/rpc-config
Choose a base branch
from
Open

rpc-alt: showDisplay #21333

wants to merge 1 commit into from

Conversation

amnn
Copy link
Contributor

@amnn amnn commented Feb 24, 2025

Description

Implement showDisplay for Object responses in rpc-alt. This is based on a new implementation of sui-display which is:

  • Pulled out of the JSON-RPC implementation, so that other uses don't need to take a dependency on a specific RPC implementation.
  • Built on a more traditional lexer/parser stack to make it easier to test each individual component (lexing, parsing, extracting fields from objects). This stack is also more appropriate for Display v2 (which will probably not share the exact codebase, but will be derived from what we have here).
  • Uses the annotated_extractor to perform nested field access on a Move Object without fully deserializing it. This means that Display will work on an object that is otherwise too large for the service to deserialize.

Test plan

New tests:

sui$ cargo nextest run \
  -p sui-display       \
  -p sui-indexer-alt-e2e-tests

Cross-check the responses against the existing implementation by running the new E2E tests on the local network. The two implementations are matched on outputs in the happy path but differ on the error paths (intentionally, to improve clarity).

Check how the new parser behaves on existing Display strings, by fetching them as follows:

postgres=> \COPY sum_displays (object_type, display) TO '/tmp/displays.tsv' WITH DELIMITER E'\t';

Then running the following test code:

#[cfg(test)]
mod tests {
    use super::*;

    use std::{
        fs::File,
        io::{BufRead, BufReader},
    };

    use move_core_types::language_storage::StructTag;
    use sui_types::display::DisplayVersionUpdatedEvent;

    #[test]
    fn test_legacy_displays() {
        let records = File::open("/tmp/displays.tsv").unwrap();
        let reader = BufReader::new(records);

        let mut fails = 0;
        for line in reader.lines() {
            let line = line.unwrap();
            let (typ, fmt) = line.split_once('\t').unwrap();
            let typ = hex::decode(typ.strip_prefix(r#"\\x"#).unwrap()).unwrap();
            let fmt = hex::decode(fmt.strip_prefix(r#"\\x"#).unwrap()).unwrap();

            let typ: StructTag = bcs::from_bytes(&typ).unwrap();
            let fmt: DisplayVersionUpdatedEvent = bcs::from_bytes(&fmt).unwrap();

            if let Err(e) = Format::parse(10, &fmt.fields) {
                println!("{typ} failed on {fmt:#?}\n{e:?}");
                fails += 1;
            }
        }

        assert_eq!(fails, 0);
    }
}

On mainnet as of this commit, there are 142 parsing failures out of 81890 Display formats, all of them legitimate.

Stack


Release notes

Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.

For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.

  • Protocol:
  • Nodes (Validators and Full nodes):
  • gRPC:
  • JSON-RPC:
  • GraphQL:
  • CLI:
  • Rust SDK:

## Description

Implement showDisplay for Object responses in rpc-alt. This is based on
a new implementation of `sui-display` which is:

- Pulled out of the JSON-RPC implementation, so that other uses don't
  need to take a dependency on a specific RPC implementation.
- Built on a more traditional lexer/parser stack to make it easier to
  test each individual component (lexing, parsing, extracting fields
  from objects). This stack is also more appropriate for Display v2
  (which will probably not share the exact codebase, but will be derived
  from what we have here).
- Uses the `annotated_extractor` to perform nested field access on a
  Move Object without fully deserializing it. This means that Display
  will work on an object that is otherwise too large for the service to
  deserialize.

## Test plan

New tests:

```
sui$ cargo nextest run \
  -p sui-display       \
  -p sui-indexer-alt-e2e-tests
```

Cross-check the responses against the existing implementation by running
the new E2E tests on the local network. The two implementations are
matched on outputs in the happy path but differ on the error paths
(intentionally, to improve clarity).

Check how the new parser behaves on existing Display strings, by
fetching them as follows:

```
postgres=> \COPY sum_displays (object_type, display) TO '/tmp/displays.tsv' WITH DELIMITER E'\t';
```

Then running the following test code:

```rust
#[cfg(test)]
mod tests {
    use super::*;

    use std::{
        fs::File,
        io::{BufRead, BufReader},
    };

    use move_core_types::language_storage::StructTag;
    use sui_types::display::DisplayVersionUpdatedEvent;

    #[test]
    fn test_legacy_displays() {
        let records = File::open("/tmp/displays.tsv").unwrap();
        let reader = BufReader::new(records);

        let mut fails = 0;
        for line in reader.lines() {
            let line = line.unwrap();
            let (typ, fmt) = line.split_once('\t').unwrap();
            let typ = hex::decode(typ.strip_prefix(r#"\\x"#).unwrap()).unwrap();
            let fmt = hex::decode(fmt.strip_prefix(r#"\\x"#).unwrap()).unwrap();

            let typ: StructTag = bcs::from_bytes(&typ).unwrap();
            let fmt: DisplayVersionUpdatedEvent = bcs::from_bytes(&fmt).unwrap();

            if let Err(e) = Format::parse(10, &fmt.fields) {
                println!("{typ} failed on {fmt:#?}\n{e:?}");
                fails += 1;
            }
        }

        assert_eq!(fails, 0);
    }
}
```

On mainnet as of this commit, there are 142 parsing failures out of
81890 Display formats, all of them legitimate.
@amnn amnn requested review from damirka, emmazzz, Bridgerz, gegaowp, wlmyng and a team February 24, 2025 13:27
@amnn amnn self-assigned this Feb 24, 2025
Copy link

vercel bot commented Feb 24, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

2 Skipped Deployments
Name Status Preview Comments Updated (UTC)
multisig-toolkit ⬜️ Ignored (Inspect) Feb 24, 2025 1:27pm
sui-kiosk ⬜️ Ignored (Inspect) Feb 24, 2025 1:27pm

@amnn amnn temporarily deployed to sui-typescript-aws-kms-test-env February 24, 2025 13:27 — with GitHub Actions Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant