Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Storage] get_blob_properties for BlobClient #2014

Open
wants to merge 19 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions sdk/storage/.dict.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,7 @@ copyid
deletetype
incrementalcopy
legalhold
missingcontainer
RAGRS
restype
testcontainer
8 changes: 8 additions & 0 deletions sdk/storage/azure_storage_blob/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,19 @@ rust-version.workspace = true
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
async-trait.workspace = true
azure_storage_common.workspace = true
azure_core = { workspace = true, features = ["xml"] }
azure_identity = { workspace = true }
serde = { workspace = true }
time = { workspace = true }
typespec_client_core = { workspace = true, features = ["derive"] }
uuid = { workspace = true }
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved

[lints]
workspace = true

[dev-dependencies]
tokio = { workspace = true, features = ["macros"] }
azure_identity.workspace = true
azure_core_test.workspace = true
65 changes: 65 additions & 0 deletions sdk/storage/azure_storage_blob/src/clients/blob_client.rs
Original file line number Diff line number Diff line change
@@ -1,2 +1,67 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.

use crate::{
models::BlobProperties, pipeline::StorageHeadersPolicy, BlobBlobClientGetPropertiesOptions,
BlobClientOptions, GeneratedBlobClient,
};
use azure_core::{credentials::TokenCredential, BearerTokenCredentialPolicy, Policy, Result};
use std::sync::Arc;

pub struct BlobClient {
pub endpoint: String,
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved
pub container_name: String,
pub blob_name: String,
client: GeneratedBlobClient,
}

impl BlobClient {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In other languages, I thought you can only get a BlobClient from a ContainerClient or something i.e., these aren't creatable. If so, they must be exported from both the crate root (re-exported) and from the clients module in which all clients - creatable or gettable - are exported.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pub fn new(
endpoint: String,
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved
container_name: String,
blob_name: String,
credential: Arc<dyn TokenCredential>,
options: Option<BlobClientOptions>,
) -> Result<Self> {
let mut options = options.unwrap_or_default();

let storage_headers_policy = Arc::new(StorageHeadersPolicy);
options
.client_options
.per_call_policies
.push(storage_headers_policy);

let oauth_token_policy = BearerTokenCredentialPolicy::new(
credential.clone(),
["https://storage.azure.com/.default"],
);
options
.client_options
.per_try_policies
.push(Arc::new(oauth_token_policy) as Arc<dyn Policy>);

let client = GeneratedBlobClient::new(&endpoint, credential, Some(options))?;

Ok(Self {
endpoint,
container_name,
blob_name,
client,
})
}

pub async fn get_blob_properties(
&self,
options: Option<BlobBlobClientGetPropertiesOptions<'_>>,
) -> Result<BlobProperties> {
let response = self
.client
.get_blob_blob_client(self.container_name.clone(), self.blob_name.clone())
.get_properties(options)
.await?;

Ok(BlobProperties::build_from_response_headers(
response.headers(),
))
}
}
2 changes: 2 additions & 0 deletions sdk/storage/azure_storage_blob/src/clients/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,5 @@
mod blob_client;
mod blob_container_client;
mod blob_service_client;

pub use blob_client::BlobClient;
14 changes: 8 additions & 6 deletions sdk/storage/azure_storage_blob/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,15 @@
// Licensed under the MIT License. See License.txt in the project root for license information.
// Code generated by Microsoft (R) Rust Code Generator. DO NOT EDIT.

pub mod clients;
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved
mod generated;
pub mod pipeline;
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved

pub use crate::generated::clients::*;
pub mod models;

pub mod models {
pub use crate::generated::enums::*;
pub use crate::generated::models::*;
}
pub use crate::generated::clients::*;
pub use blob_client::BlobClient as GeneratedBlobClient;
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved

pub use blob_client::BlobClient;
pub use crate::blob_client::BlobClientOptions;
pub use crate::clients::BlobClient;
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved
pub use crate::generated::clients::blob_blob_client::BlobBlobClientGetPropertiesOptions;
52 changes: 52 additions & 0 deletions sdk/storage/azure_storage_blob/src/models/blob_properties.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.

use azure_core::headers::Headers;

/// Properties of an Azure Storage blob.
///
#[derive(Clone, Default, Debug)]
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved
pub struct BlobProperties {
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved
pub access_tier_inferred: String,
pub access_tier: String,
pub blob_type: String,
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved
pub content_length: String,
pub content_md5: String,
pub content_type: String,
pub creation_time: String,
pub etag: String,
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved
pub last_modified: String,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this a string and not a DateTimeOffset which we use for times elsewhere.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrt this comment, is there a currently accepted way to convert from RFC1123 strings (as returned by the service) ex:
creation_time: Some("Sat, 01 Feb 2025 00:17:37 GMT" to a OffsetDateTime object? I know you can provide a custom pattern query to parse, but I assume that if we have OffsetDateTime elsewhere we may have some common utility for this?

pub lease_state: String,
pub lease_status: String,
pub server_encrypted: String,
}

impl BlobProperties {
pub fn build_from_response_headers(response_headers: &Headers) -> BlobProperties {
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved
let mut properties = BlobProperties {
..Default::default()
};
for (key, value) in response_headers.iter() {
match key.as_str() {
"content-length" => properties.content_length = String::from(value.as_str()),
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved
"content-md5" => properties.content_md5 = String::from(value.as_str()),
"content-type" => properties.content_type = String::from(value.as_str()),
"etag" => properties.etag = String::from(value.as_str()),
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved
"last-modified" => properties.last_modified = String::from(value.as_str()),
"x-ms-access-tier-inferred" => {
properties.access_tier_inferred = String::from(value.as_str())
}
"x-ms-access-tier" => properties.access_tier = String::from(value.as_str()),
"x-ms-blob-type" => properties.blob_type = String::from(value.as_str()),
"x-ms-creation-time" => properties.creation_time = String::from(value.as_str()),
"x-ms-lease-state" => properties.lease_state = String::from(value.as_str()),
"x-ms-lease-status" => properties.lease_status = String::from(value.as_str()),
"x-ms-server-encrypted" => {
properties.server_encrypted = String::from(value.as_str())
}
_ => {}
}
}
properties
}
}
9 changes: 9 additions & 0 deletions sdk/storage/azure_storage_blob/src/models/mod.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.

mod blob_properties;

pub use crate::generated::enums::*;
pub use crate::generated::models::*;

pub use blob_properties::BlobProperties;
6 changes: 6 additions & 0 deletions sdk/storage/azure_storage_blob/src/pipeline/mod.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.

mod storage_headers_policy;

pub use storage_headers_policy::StorageHeadersPolicy;
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.

use async_trait::async_trait;
use azure_core::{headers::HeaderName, Context, Policy, PolicyResult, Request};
use std::sync::Arc;
use uuid::Uuid;

#[derive(Debug, Clone)]
pub struct StorageHeadersPolicy;

#[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), async_trait)]
impl Policy for StorageHeadersPolicy {
async fn send(
&self,
ctx: &Context,
request: &mut Request,
next: &[Arc<dyn Policy>],
) -> PolicyResult {
let result = request
.headers()
.get_str(&HeaderName::from("x-ms-client-request-id"));
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved

match result {
Ok(_) => {}
Err(_) => {
let request_id = Uuid::new_v4().to_string();
request.insert_header("x-ms-client-request-id", request_id);
}
}
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved
next[0].send(ctx, request, &next[1..]).await
}
}
67 changes: 67 additions & 0 deletions sdk/storage/azure_storage_blob/tests/test_blob_client.rs
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.

use azure_core_test::recorded;
use azure_identity::DefaultAzureCredentialBuilder;
use azure_storage_blob::{BlobBlobClientGetPropertiesOptions, BlobClient, BlobClientOptions};
use std::{env, error::Error};

#[cfg(test)]
mod tests {
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved

use super::*;

#[recorded::test(live)]
async fn test_get_blob_properties() -> Result<(), Box<dyn Error>> {
// Setup
let storage_account_name = env::var("AZURE_STORAGE_ACCOUNT_NAME")
.expect("Failed to get environment variable: AZURE_STORAGE_ACCOUNT_NAME");
let endpoint = format!("https://{}.blob.core.windows.net/", storage_account_name);
let credential = DefaultAzureCredentialBuilder::default().build()?;

// Act
let blob_client = BlobClient::new(
endpoint,
String::from("testcontainer"),
String::from("test_blob.txt"),
credential,
Some(BlobClientOptions::default()),
)?;
let response = blob_client
.get_blob_properties(Some(BlobBlobClientGetPropertiesOptions::default()))
.await;

// Assert
assert!(response.is_ok());
Ok(())
}

#[recorded::test(live)]
async fn test_get_blob_properties_invalid_container() -> Result<(), Box<dyn Error>> {
// Setup
let storage_account_name = env::var("AZURE_STORAGE_ACCOUNT_NAME")
.expect("Failed to get environment variable: AZURE_STORAGE_ACCOUNT_NAME");
let endpoint = format!("https://{}.blob.core.windows.net/", storage_account_name);
let credential = DefaultAzureCredentialBuilder::default().build()?;

// Act
let blob_client = BlobClient::new(
endpoint,
String::from("missingcontainer"),
String::from("test_blob.txt"),
credential,
Some(BlobClientOptions::default()),
)?;
let response = blob_client
.get_blob_properties(Some(BlobBlobClientGetPropertiesOptions::default()))
.await;

// Assert
assert_eq!(
String::from("HttpResponse(NotFound, \"ContainerNotFound\")"),
response.unwrap_err().kind().to_string()
);

Ok(())
}
}