Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

change(rpc): Adds a TrustedChainSync struct for keeping up with Zebra's non-finalized best chain from a separate process #8596

Merged
merged 37 commits into from
Jul 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
26c0e5d
Adds an init_read_only() fn in zebra-state
arya2 Jun 7, 2024
281668b
moves elasticsearch initialization to `FinalizedState::new_with_debug()`
arya2 Jun 7, 2024
33a3ccf
Updates callers of `FinalizedState::{new, new_with_debug}` to pass a …
arya2 Jun 7, 2024
995811e
Adds a non-finalized read state syncer to zebra-rpc
arya2 Jun 7, 2024
f94bd41
moves, removes, updates, or addresses TODOs
arya2 Jun 7, 2024
d57c50a
reduces disk IO while waiting for the a new chain tip & updates the c…
arya2 Jun 7, 2024
8d246c3
Returns boxed errors from RpcRequestClient methods instead of color_e…
arya2 Jun 7, 2024
5470979
Avoids resetting the non-finalized state when there's an error gettin…
arya2 Jun 7, 2024
0d35fc3
Adds stub for acceptance test(s) and removes outdated TODO
arya2 Jun 7, 2024
0d1f434
adds TODOs for testing
arya2 Jun 7, 2024
dc514b8
Tests that `ChainTipChange` is updated when the non-finalized best ch…
arya2 Jun 11, 2024
302facf
adds a last_chain_tip_hash and uses a FuturesOrdered for getblock req…
arya2 Jun 12, 2024
a1ad80f
Fixes a pre-flush sync issue by using a secondary db instead of a rea…
arya2 Jun 12, 2024
c9319e0
Moves disk IO to blocking tasks
arya2 Jun 13, 2024
9d2d1df
Updates acceptance test to how forks are handled
arya2 Jun 14, 2024
d31526e
Checks synced read state for all of the expected blocks
arya2 Jun 14, 2024
f18344f
checks that there isn't a tip change until the best chain changes
arya2 Jun 14, 2024
112294f
checks for chain tip changes in test
arya2 Jun 14, 2024
ac62bae
Merge branch 'main' into non-final-read-state
arya2 Jun 14, 2024
ce5f7ed
run test without feature
arya2 Jun 14, 2024
e13202d
fixes lint
arya2 Jun 14, 2024
bc37807
Fixes compilation/test issues
arya2 Jun 17, 2024
fd1ab3b
Adds docs / comments, moves HexData out from behind the getblocktempl…
arya2 Jun 17, 2024
737c2e5
Fixes lints
arya2 Jun 17, 2024
d1b25b4
Merge branch 'main' into non-final-read-state
arya2 Jun 17, 2024
28902fa
removes syncer and rpc-syncer features
arya2 Jun 18, 2024
15feed1
Fixes test on Windows, applies suggestions from code review
arya2 Jun 19, 2024
e642876
Merge branch 'main' into non-final-read-state
arya2 Jun 19, 2024
8fb798b
Updates `POLL_DELAY` documentation
arya2 Jun 20, 2024
f0b5ee5
Updates method docs
arya2 Jun 20, 2024
2743051
Fixes a test bug
arya2 Jun 20, 2024
0b875ba
Merge branch 'main' into non-final-read-state
mpguerra Jun 21, 2024
abe0af7
Merge branch 'main' into non-final-read-state
arya2 Jul 2, 2024
8391ae7
use rpc-client feature in zebrad production code
oxarbitrage Jul 8, 2024
d996cc6
use rpc-client feature in zebra-node-services for building zebra-rpc …
oxarbitrage Jul 8, 2024
92db104
Merge branch 'main' into non-final-read-state
mpguerra Jul 8, 2024
0e25311
Merge remote-tracking branch 'origin/main' into non-final-read-state
arya2 Jul 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions zebra-node-services/src/rpc_client.rs
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
//! A client for calling Zebra's JSON-RPC methods.
//!
//! Only used in tests and tools.
//! Used in the rpc sync scanning functionality and in various tests and tools.

use std::net::SocketAddr;

use reqwest::Client;

use color_eyre::{eyre::eyre, Result};
use crate::BoxError;

/// An HTTP client for making JSON-RPC requests.
#[derive(Clone, Debug)]
Expand Down Expand Up @@ -99,21 +99,21 @@ impl RpcRequestClient {
&self,
method: impl AsRef<str>,
params: impl AsRef<str>,
) -> Result<T> {
) -> std::result::Result<T, BoxError> {
Self::json_result_from_response_text(&self.text_from_call(method, params).await?)
}

/// Accepts response text from an RPC call
/// Returns `Ok` with a deserialized `result` value in the expected type, or an error report.
fn json_result_from_response_text<T: serde::de::DeserializeOwned>(
response_text: &str,
) -> Result<T> {
) -> std::result::Result<T, BoxError> {
use jsonrpc_core::Output;

let output: Output = serde_json::from_str(response_text)?;
match output {
Output::Success(success) => Ok(serde_json::from_value(success.result)?),
Output::Failure(failure) => Err(eyre!("RPC call failed with: {failure:?}")),
Output::Failure(failure) => Err(failure.error.into()),
}
}
}
3 changes: 1 addition & 2 deletions zebra-rpc/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ keywords = ["zebra", "zcash"]
categories = ["asynchronous", "cryptography::cryptocurrencies", "encoding", "network-programming"]

[features]
default = []

# Production features that activate extra dependencies, or extra features in dependencies

Expand Down Expand Up @@ -77,7 +76,7 @@ proptest = { version = "1.4.0", optional = true }
zebra-chain = { path = "../zebra-chain", version = "1.0.0-beta.38", features = ["json-conversion"] }
zebra-consensus = { path = "../zebra-consensus", version = "1.0.0-beta.38" }
zebra-network = { path = "../zebra-network", version = "1.0.0-beta.38" }
zebra-node-services = { path = "../zebra-node-services", version = "1.0.0-beta.38" }
zebra-node-services = { path = "../zebra-node-services", version = "1.0.0-beta.38", features = ["rpc-client"] }
zebra-script = { path = "../zebra-script", version = "1.0.0-beta.38" }
zebra-state = { path = "../zebra-state", version = "1.0.0-beta.38" }

Expand Down
1 change: 1 addition & 0 deletions zebra-rpc/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ pub mod constants;
pub mod methods;
pub mod queue;
pub mod server;
pub mod sync;

#[cfg(test)]
mod tests;
36 changes: 34 additions & 2 deletions zebra-rpc/src/methods.rs
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ use crate::{
};

mod errors;
pub mod hex_data;

use errors::{MapServerError, OkOrServerError};

Expand Down Expand Up @@ -171,6 +172,14 @@ pub trait Rpc {
#[rpc(name = "getbestblockhash")]
fn get_best_block_hash(&self) -> Result<GetBlockHash>;

/// Returns the height and hash of the current best blockchain tip block, as a [`GetBlockHeightAndHash`] JSON struct.
arya2 marked this conversation as resolved.
Show resolved Hide resolved
///
/// zcashd reference: none
/// method: post
/// tags: blockchain
#[rpc(name = "getbestblockheightandhash")]
fn get_best_block_height_and_hash(&self) -> Result<GetBlockHeightAndHash>;

/// Returns all transaction ids in the memory pool, as a JSON array.
///
/// zcashd reference: [`getrawmempool`](https://zcash.github.io/rpc/getrawmempool.html)
Expand Down Expand Up @@ -867,15 +876,20 @@ where
.boxed()
}

// TODO: use a generic error constructor (#5548)
fn get_best_block_hash(&self) -> Result<GetBlockHash> {
self.latest_chain_tip
.best_tip_hash()
.map(GetBlockHash)
.ok_or_server_error("No blocks in state")
}

// TODO: use a generic error constructor (#5548)
fn get_best_block_height_and_hash(&self) -> Result<GetBlockHeightAndHash> {
self.latest_chain_tip
.best_tip_height_and_hash()
.map(|(height, hash)| GetBlockHeightAndHash { height, hash })
.ok_or_server_error("No blocks in state")
}

fn get_raw_mempool(&self) -> BoxFuture<Result<Vec<String>>> {
#[cfg(feature = "getblocktemplate-rpcs")]
use zebra_chain::block::MAX_BLOCK_BYTES;
Expand Down Expand Up @@ -1547,6 +1561,24 @@ impl Default for GetBlock {
#[serde(transparent)]
pub struct GetBlockHash(#[serde(with = "hex")] pub block::Hash);

/// Response to a `getbestblockheightandhash` RPC request.
#[derive(Copy, Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct GetBlockHeightAndHash {
/// The best chain tip block height
pub height: block::Height,
/// The best chain tip block hash
pub hash: block::Hash,
}

arya2 marked this conversation as resolved.
Show resolved Hide resolved
impl Default for GetBlockHeightAndHash {
fn default() -> Self {
Self {
height: block::Height::MIN,
hash: block::Hash([0; 32]),
}
}
}

impl Default for GetBlockHash {
fn default() -> Self {
GetBlockHash(block::Hash([0; 32]))
Expand Down
5 changes: 3 additions & 2 deletions zebra-rpc/src/methods/get_block_template_rpcs.rs
Original file line number Diff line number Diff line change
Expand Up @@ -46,15 +46,16 @@ use crate::methods::{
types::{
get_block_template::GetBlockTemplate,
get_mining_info,
hex_data::HexData,
long_poll::LongPollInput,
peer_info::PeerInfo,
submit_block,
subsidy::{BlockSubsidy, FundingStream},
unified_address, validate_address, z_validate_address,
},
},
height_from_signed_int, GetBlockHash, MISSING_BLOCK_ERROR_CODE,
height_from_signed_int,
hex_data::HexData,
GetBlockHash, MISSING_BLOCK_ERROR_CODE,
};

pub mod constants;
Expand Down
1 change: 0 additions & 1 deletion zebra-rpc/src/methods/get_block_template_rpcs/types.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
pub mod default_roots;
pub mod get_block_template;
pub mod get_mining_info;
pub mod hex_data;
pub mod long_poll;
pub mod peer_info;
pub mod submit_block;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
//! Parameter types for the `getblocktemplate` RPC.

use crate::methods::get_block_template_rpcs::types::{hex_data::HexData, long_poll::LongPollId};
use crate::methods::{get_block_template_rpcs::types::long_poll::LongPollId, hex_data::HexData};

/// Defines whether the RPC method should generate a block template or attempt to validate a block proposal.
#[derive(Clone, Debug, serde::Deserialize, serde::Serialize, PartialEq, Eq)]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ pub struct JsonParameters {
/// Response to a `submitblock` RPC request.
///
/// Zebra never returns "duplicate-invalid", because it does not store invalid blocks.
#[derive(Debug, PartialEq, Eq, serde::Serialize)]
#[derive(Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "kebab-case")]
pub enum ErrorResponse {
/// Block was already committed to the non-finalized or finalized state
Expand All @@ -45,7 +45,7 @@ pub enum ErrorResponse {
/// Response to a `submitblock` RPC request.
///
/// Zebra never returns "duplicate-invalid", because it does not store invalid blocks.
#[derive(Debug, PartialEq, Eq, serde::Serialize)]
#[derive(Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
#[serde(untagged)]
pub enum Response {
/// Block was not successfully submitted, return error
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,13 +39,13 @@ use crate::methods::{
get_block_template_rpcs::types::{
get_block_template::{self, GetBlockTemplateRequestMode},
get_mining_info,
hex_data::HexData,
long_poll::{LongPollId, LONG_POLL_ID_LENGTH},
peer_info::PeerInfo,
submit_block,
subsidy::BlockSubsidy,
unified_address, validate_address, z_validate_address,
},
hex_data::HexData,
tests::{snapshot::EXCESSIVE_BLOCK_HEIGHT, utils::fake_history_tree},
GetBlockHash, GetBlockTemplateRpc, GetBlockTemplateRpcImpl,
};
Expand Down
5 changes: 3 additions & 2 deletions zebra-rpc/src/methods/tests/vectors.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1259,8 +1259,9 @@ async fn rpc_getblocktemplate_mining_address(use_p2pkh: bool) {
GET_BLOCK_TEMPLATE_NONCE_RANGE_FIELD,
},
get_block_template::{self, GetBlockTemplateRequestMode},
types::{hex_data::HexData, long_poll::LONG_POLL_ID_LENGTH},
types::long_poll::LONG_POLL_ID_LENGTH,
},
hex_data::HexData,
tests::utils::fake_history_tree,
};

Expand Down Expand Up @@ -1547,7 +1548,7 @@ async fn rpc_submitblock_errors() {
use zebra_chain::chain_sync_status::MockSyncStatus;
use zebra_network::address_book_peers::MockAddressBookPeers;

use crate::methods::get_block_template_rpcs::types::{hex_data::HexData, submit_block};
use crate::methods::{get_block_template_rpcs::types::submit_block, hex_data::HexData};

let _init_guard = zebra_test::init();

Expand Down
Loading
Loading