Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Subgraph composition rework vid wrap3 #5768

Closed
Changes from 1 commit
Commits
Show all changes
135 commits
Select commit Hold shift + click to select a range
3fe298e
graph: Add support for subgraph datasource in manifest
incrypto32 Jul 11, 2024
a45daf3
graph: wrap TriggerFilter with TriggerFilterWrapper
incrypto32 Jul 12, 2024
af3468c
graph,chain: add build_subgraph_block_stream method
incrypto32 Jul 12, 2024
d59c14b
graph,core,chain: use TriggerFilterWrapper in PollingBlockStream
incrypto32 Jul 15, 2024
edc806c
graph: created TriggersAdapterWrapper
incrypto32 Jul 15, 2024
eb41c6b
graph,core,chain: Add a wrapper enum for Triggers to handle subgraph …
incrypto32 Jul 16, 2024
3710cf8
graph, chain: Build subgraph trigger filters in build_filter
incrypto32 Jul 16, 2024
579d886
graph,core: Add subgraph_hosts to RuntimeHostBuilder
incrypto32 Jul 17, 2024
cb4d082
chain, core, graph: Add source_subgraph_stores to indexing inputs
incrypto32 Jul 18, 2024
ff572cf
graph, chain: minor refactoring and formatting
incrypto32 Jul 18, 2024
35b2a54
graph, chain: fix typo in TriggersAdapterWrapper
incrypto32 Jul 18, 2024
fe5d5ba
chain, graph: use TriggerFilterWrapper in scan_triggers
incrypto32 Jul 18, 2024
4c05119
chain/ethereum: implement load_blocks_by_numbers for EthereumAdapter
incrypto32 Jul 24, 2024
d20bafd
graph: refactor BlockWithTriggers impl
incrypto32 Jul 24, 2024
3ab4941
graph, core: Add a new SubgraphFilter struct
incrypto32 Jul 24, 2024
aec82dc
chain/ethereum: Mock implementation of subgraph_triggers for ethereum
incrypto32 Jul 24, 2024
ae7559d
chain,graph : use `chain_head_ptr` method from adapter
incrypto32 Jul 24, 2024
686e7e1
chain, core, graph : use TriggersAdapterWrapper at top level
incrypto32 Jul 25, 2024
3aba98f
chain, graph : move subgraph trigger scanning back to TriggersAdapter…
incrypto32 Jul 25, 2024
9069486
graph: use ReadStore instead of WriteStore for source subgraph stores
incrypto32 Nov 12, 2024
c7ca25a
graph: Better comments
incrypto32 Nov 12, 2024
1dab182
first attempt at reading entities for mutable ones
zorancv Jul 17, 2024
f931d37
remove the key as a parameter
zorancv Jul 18, 2024
a6e4b23
fix regex
zorancv Jul 24, 2024
b7bc168
proper return type
zorancv Jul 30, 2024
5903ee9
support immutable entities
zorancv Aug 2, 2024
57e1463
add test for immutable entities
zorancv Aug 5, 2024
553fb82
store: fix block range query bug when getting entity triggers
incrypto32 Jul 31, 2024
4fc6b3c
fix test
zorancv Aug 5, 2024
441300a
refactor
zorancv Aug 8, 2024
239c6cb
change the range type
zorancv Aug 12, 2024
48accf7
test multiple entities
zorancv Aug 15, 2024
9acd51e
address review
zorancv Nov 13, 2024
e25e9bf
remove get_range from the Queue and Writer
zorancv Nov 15, 2024
0d7e4d5
remove get_range() from WritableStore
zorancv Nov 28, 2024
38c4286
replace ReadSTore with SourceableStore
zorancv Dec 3, 2024
f57b3ba
add block tracking
zorancv Dec 3, 2024
77850be
compiler fix
zorancv Dec 4, 2024
8878caf
cleanup WritableStore
zorancv Dec 5, 2024
70ca609
fetch block from DB
zorancv Dec 6, 2024
0a4a2a6
address PR comments
zorancv Dec 9, 2024
c6e4b36
tests: Add integration test for subgraph datasources
incrypto32 Jul 19, 2024
3ac0c1a
tests: change entity name for integration test
incrypto32 Jul 24, 2024
331ab3f
read entities from database
zorancv Jul 25, 2024
08f17c0
clean up
zorancv Jul 29, 2024
142ce94
simplify
zorancv Jul 29, 2024
8f51340
remove mocks
zorancv Jul 30, 2024
94cbf59
fixes
zorancv Jul 30, 2024
f17fdec
fixes
zorancv Aug 13, 2024
2e96b73
fix
zorancv Dec 5, 2024
3814175
hash removal fix
zorancv Dec 9, 2024
882a111
graph: subgraph_triggers to take block_number hashset
incrypto32 Jul 31, 2024
fc239b8
graph: refactor TriggersAdapterWrapper.scan_triggers
incrypto32 Jul 31, 2024
27d3c1f
graph: fix manifest parsing of source subgraphs
incrypto32 Aug 1, 2024
72a16ca
tests: fix runner tests
incrypto32 Aug 1, 2024
2deaec4
graph: TriggersAdapterWrapper.chain_head_ptr to consider head of sou…
incrypto32 Aug 1, 2024
48b7ffe
chain, core, graph : Refactor subgraph trigger creation
incrypto32 Aug 12, 2024
c4ea03f
graph: refactor TriggersAdapterWrapper.triggers_in_block to not rely …
incrypto32 Aug 12, 2024
58b9231
graph: fix `get_entities_for_range` bug which didn't return all entit…
incrypto32 Aug 16, 2024
60e9d47
tests for multiple entities
incrypto32 Aug 16, 2024
3fb0e72
graph: fix subgraph filter mismatch bug
incrypto32 Aug 22, 2024
c47236c
graph: Avoid unwraps in `blocks_with_subgraph_triggers`
incrypto32 Nov 14, 2024
a60c653
fix futures
zorancv Dec 6, 2024
37bf3cd
rebase fixes for vec
zorancv Dec 9, 2024
5eb3c9a
request all entity types in a single SQL query
zorancv Aug 21, 2024
7561307
add types to the enities
zorancv Aug 23, 2024
b69e99a
query for both lower and upper range
zorancv Aug 28, 2024
b675b16
deduce the types of operation create, modify or delete
zorancv Sep 2, 2024
fae61ee
add vid to the result struct in order to sort it afterwards
zorancv Sep 3, 2024
d0d1bb0
better test
zorancv Sep 4, 2024
5309219
Add casuality region to the SQL querry
zorancv Sep 6, 2024
fbc5313
fix the empty select bug
zorancv Nov 4, 2024
30a5e59
fixes
zorancv Dec 5, 2024
ebfe059
fix return
zorancv Dec 6, 2024
501c8ba
address review
zorancv Dec 9, 2024
4136889
substitute if with match
zorancv Dec 9, 2024
940f4f6
better comment
zorancv Dec 10, 2024
a213d3a
change SQL for the upper bound of immutable entities
zorancv Dec 10, 2024
841bf7d
comments and minor changes
zorancv Dec 12, 2024
59f683e
graph, runtime: Support subgraph entity operation detection in compos…
incrypto32 Sep 9, 2024
522b1e6
tests: udpate subgraph composition integration test cases to handle e…
incrypto32 Sep 9, 2024
51c8b15
chain/ethereum, graph, store: use block cache to load blocks by numbers
incrypto32 Aug 19, 2024
b193c5a
chain/ethereum: Better logs when block is loaded from RPC
incrypto32 Aug 29, 2024
15dca1b
graph, node: Allow rpc clients when using firehose
incrypto32 Sep 2, 2024
df632c2
store: improve comments in `ChainStore.blocks()`
incrypto32 Nov 19, 2024
9461f99
chain/ethereum: remove rpc client from firehose chain clientff
incrypto32 Nov 25, 2024
a20d26f
graph, chain/ethereum: use block ptrs when fetching blocks for compos…
incrypto32 Sep 2, 2024
eb1cb5d
graph: implement deserialize for BlockPtrExt and add tests
incrypto32 Sep 5, 2024
80a0d4b
chain/ethereum: Add block_ptr_batch_size env variable
incrypto32 Sep 5, 2024
9732a46
chain/ethereum: bump block range size for composed subgraphs
incrypto32 Sep 5, 2024
d27793e
chain, graph: Fix block ptr ext parsing bug
incrypto32 Oct 7, 2024
f1bb58c
chain, graph: rename BlockPtrExt to ExtendedBlockPtr
incrypto32 Nov 19, 2024
2610018
chain,graph: use firehose get_block to get block ptrs for subgraph tr…
incrypto32 Nov 13, 2024
e194ad8
graph, chain: make TriggersAdapterTrait methods work with firehose en…
incrypto32 Nov 15, 2024
03ead16
chain/ethereum: Refactor load_block_ptrs_by_numbers
incrypto32 Nov 18, 2024
f432dd0
graph: implement custom deserializer for BlockTime
incrypto32 Nov 18, 2024
5975fea
graph: mock chain store
incrypto32 Nov 22, 2024
7db1319
chain/ethereum: refactor load_block_ptrs_by_numbers and add unit tests
incrypto32 Nov 22, 2024
fef9640
graph: enable ethereum host fns for subgraph datasource
incrypto32 Oct 31, 2024
5768f6c
graph, chain/ethereum: Move declared call related structs to graph crate
incrypto32 Dec 3, 2024
410b740
graph, chain/ethreum: add FindMappingABI trait and use it to refactor…
incrypto32 Dec 3, 2024
7e6510c
graph: Refactor CallArg to include subgraph datasource specific args
incrypto32 Dec 3, 2024
2ff398e
graph, chain/ethereum: Move DeclaredCalls struct to graph crate
incrypto32 Dec 3, 2024
238c64b
graph: Refactor DeclareCall for readability
incrypto32 Dec 3, 2024
13b0c11
graph: Add methods to handle declared calls in subgraph datasource
incrypto32 Dec 3, 2024
84e9b8a
graph, runtime: Add new MappingEntityTrigger struct that takes declar…
incrypto32 Dec 3, 2024
4dfa1af
chain/ethereum: Handle subgraph datasource triggers also in `DecoderH…
incrypto32 Dec 3, 2024
460f33e
store/test-store: Add a test for declared eth calls parsing in manifest
incrypto32 Dec 3, 2024
1379986
renames
zorancv Dec 10, 2024
9658d8c
fix rebase
zorancv Dec 18, 2024
cf13e1f
add vid
zorancv Oct 4, 2024
c44bc15
fixes
zorancv Oct 6, 2024
3a88523
more test fixes
zorancv Oct 8, 2024
548fda1
fix entity_cache tests
zorancv Oct 8, 2024
dd9ce56
fix graphql_query tests
zorancv Oct 9, 2024
29b4679
fix postgress_writable tests
zorancv Oct 10, 2024
be68a68
fix more tests
zorancv Oct 10, 2024
86f6359
fix relational tests
zorancv Oct 11, 2024
f551918
fix graft test
zorancv Oct 11, 2024
abcb442
try to fix graft prune test
zorancv Oct 14, 2024
7b080d8
fix runner test data_source_revert
zorancv Oct 22, 2024
fd5845a
try to remove optional vid fetch
zorancv Oct 25, 2024
a4a0adc
fix
zorancv Oct 28, 2024
5680a8c
add wrapping
zorancv Oct 31, 2024
63a3cb5
cleanup
zorancv Nov 6, 2024
9899b13
cleanup
zorancv Nov 18, 2024
d9e74f1
cleanup
zorancv Nov 18, 2024
fd9f92b
correct vid
zorancv Nov 19, 2024
d8425a1
cleanup
zorancv Nov 19, 2024
d86699e
cleanup
zorancv Nov 19, 2024
faa6e5d
bump the spec version
zorancv Nov 22, 2024
15411a1
simplify
zorancv Dec 17, 2024
62d1e2b
fix tests
zorancv Dec 18, 2024
714c931
addressing review remarks
zorancv Jan 9, 2025
0abd901
check if poi is fine to be generated by us
zorancv Jan 13, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix tests
zorancv committed Jan 3, 2025
commit 62d1e2b874fbd5769cd3e5118ba59657c82efd35
2 changes: 1 addition & 1 deletion graph/src/components/store/entity_cache.rs
Original file line number Diff line number Diff line change
@@ -364,7 +364,7 @@ impl EntityCache {
// check the validate for derived fields
let is_valid = entity.validate(&key).is_ok();

//The next VID is based on a block number and a sequence withing the block
// The next VID is based on a block number and a sequence within the block
let vid = ((block as i64) << 32) + self.vid_seq as i64;
self.vid_seq += 1;
let entity = EntityV::new(entity, vid);
13 changes: 7 additions & 6 deletions store/test-store/tests/postgres/writable.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
use graph::blockchain::block_stream::{EntitySourceOperation, FirehoseCursor};
use graph::data::store::EntityV;
use graph::data::subgraph::schema::DeploymentCreate;
use graph::data::value::Word;
use graph::data_source::CausalityRegion;
@@ -147,7 +148,7 @@ async fn insert_count(
let entity_op = if block != 3 && block != 5 && block != 7 {
EntityOperation::Set {
key: count_key_local(&COUNTER_TYPE, &data.get("id").unwrap().to_string()),
data,
data: EntityV::new(data, block.into()),
}
} else {
EntityOperation::Remove {
@@ -166,7 +167,7 @@ async fn insert_count(
};
let entity_op = EntityOperation::Set {
key: count_key_local(&COUNTER2_TYPE, &data.get("id").unwrap().to_string()),
data,
data: EntityV::new(data, block.into()),
};
ops.push(entity_op);
}
@@ -344,10 +345,10 @@ fn read_range_test() {
r#"(1, [EntitySourceOperation { entity_op: Create, entity_type: EntityType(Counter), entity: Entity { count: Int(2), id: String("1") }, vid: 1 }, EntitySourceOperation { entity_op: Create, entity_type: EntityType(Counter2), entity: Entity { count: Int(2), id: String("1") }, vid: 1 }])"#,
r#"(2, [EntitySourceOperation { entity_op: Modify, entity_type: EntityType(Counter), entity: Entity { count: Int(4), id: String("1") }, vid: 2 }, EntitySourceOperation { entity_op: Create, entity_type: EntityType(Counter2), entity: Entity { count: Int(4), id: String("2") }, vid: 2 }])"#,
r#"(3, [EntitySourceOperation { entity_op: Delete, entity_type: EntityType(Counter), entity: Entity { count: Int(4), id: String("1") }, vid: 2 }, EntitySourceOperation { entity_op: Create, entity_type: EntityType(Counter2), entity: Entity { count: Int(6), id: String("3") }, vid: 3 }])"#,
r#"(4, [EntitySourceOperation { entity_op: Create, entity_type: EntityType(Counter), entity: Entity { count: Int(8), id: String("1") }, vid: 3 }, EntitySourceOperation { entity_op: Create, entity_type: EntityType(Counter2), entity: Entity { count: Int(8), id: String("4") }, vid: 4 }])"#,
r#"(5, [EntitySourceOperation { entity_op: Delete, entity_type: EntityType(Counter), entity: Entity { count: Int(8), id: String("1") }, vid: 3 }, EntitySourceOperation { entity_op: Create, entity_type: EntityType(Counter2), entity: Entity { count: Int(10), id: String("5") }, vid: 5 }])"#,
r#"(6, [EntitySourceOperation { entity_op: Create, entity_type: EntityType(Counter), entity: Entity { count: Int(12), id: String("1") }, vid: 4 }])"#,
r#"(7, [EntitySourceOperation { entity_op: Delete, entity_type: EntityType(Counter), entity: Entity { count: Int(12), id: String("1") }, vid: 4 }])"#,
r#"(4, [EntitySourceOperation { entity_op: Create, entity_type: EntityType(Counter), entity: Entity { count: Int(8), id: String("1") }, vid: 4 }, EntitySourceOperation { entity_op: Create, entity_type: EntityType(Counter2), entity: Entity { count: Int(8), id: String("4") }, vid: 4 }])"#,
r#"(5, [EntitySourceOperation { entity_op: Delete, entity_type: EntityType(Counter), entity: Entity { count: Int(8), id: String("1") }, vid: 4 }, EntitySourceOperation { entity_op: Create, entity_type: EntityType(Counter2), entity: Entity { count: Int(10), id: String("5") }, vid: 5 }])"#,
r#"(6, [EntitySourceOperation { entity_op: Create, entity_type: EntityType(Counter), entity: Entity { count: Int(12), id: String("1") }, vid: 6 }])"#,
r#"(7, [EntitySourceOperation { entity_op: Delete, entity_type: EntityType(Counter), entity: Entity { count: Int(12), id: String("1") }, vid: 6 }])"#,
];
let subgraph_store = store.subgraph_store();
writable.deployment_synced().unwrap();