Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Bedrock Cloud Model provider #21092

Open
wants to merge 70 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 51 commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
264ea71
Added Bedrock UI elements, credential storage and a few other things
5herlocked Oct 28, 2024
c8a650f
Compiles with the appropriate UI fields + saves credentials
5herlocked Oct 28, 2024
db5e220
Starting the transpose to the Bedrock SDK
5herlocked Oct 31, 2024
4edb624
Added Bedrock Error enum that transparently exposes SDK Errors as Bed…
5herlocked Nov 5, 2024
46e817a
Had Q Dev rewrite most of the model enums based on the AWS Bedrock li…
5herlocked Nov 6, 2024
bccd72e
Fixed the names in impl Model
5herlocked Nov 6, 2024
86cf8ac
removed a whole host of unsupported types
5herlocked Nov 15, 2024
36abfe7
seemingly migrated to the new way of managing language models
5herlocked Nov 23, 2024
e00edd6
Removed bedrock from the cloudmodel
5herlocked Nov 25, 2024
3aa5c1c
Fixed a whole host of mismatched types, Arc<dyn HttpClient> will be t…
5herlocked Nov 26, 2024
b621807
Helper functions
5herlocked Nov 29, 2024
47af036
Trying some additional things
5herlocked Dec 2, 2024
9421ed9
Added Nova Models
5herlocked Dec 6, 2024
a1c5e38
continuing work
5herlocked Dec 22, 2024
fb71667
WIP: Rebase commit
5herlocked Jan 13, 2025
df0f600
Migrated to AwsHttpClient to impl Smithy traits in Zed
5herlocked Jan 17, 2025
2c5aa0f
ConverseStreamOutput stabilized, map_to_lm_completion events failing …
5herlocked Jan 19, 2025
0ba0368
Moved AWS components out of the GPUI core components
5herlocked Jan 20, 2025
5db5313
Removed references from RPC collab server and the collab crate itself…
5herlocked Jan 24, 2025
65e4125
Solved the invalid dnshostname issue, basically us-east-1 wasnt' bein…
5herlocked Jan 24, 2025
18aa25e
So yeah, background executor hates me. AWS SDK requires tokio runtime…
5herlocked Jan 24, 2025
f70d3f5
Converting everything into strings at streaming time helped a lot, bu…
5herlocked Jan 24, 2025
9759c15
Cleaned up imports, stopping work to understand how I can lift the to…
5herlocked Jan 28, 2025
c0e28bd
Merge branch 'refs/heads/main' into feature/bedrock-provider
5herlocked Jan 29, 2025
47004a6
Fixed license, pushing for feedback from Gpui_tokio impl
5herlocked Jan 29, 2025
688e64e
Merge branch 'main' into feature/bedrock-provider
5herlocked Feb 1, 2025
0d00909
Merge branch 'refs/heads/main' into feature/bedrock-provider
5herlocked Feb 5, 2025
0d9e0cf
Slow transpose from my personal machine
5herlocked Feb 5, 2025
7b32caa
Merge remote-tracking branch 'refs/remotes/origin/feature/bedrock-pro…
5herlocked Feb 5, 2025
c3f38d6
Pin<Box<dyn Future<>>> is the struggle of this week
5herlocked Feb 6, 2025
d985c6c
We finally have a response from bedrock woohoooo -- file and the othe…
5herlocked Feb 6, 2025
222f24c
bedrock provider can consistently provide responses, some issues stil…
5herlocked Feb 7, 2025
aec9534
updated some models with their cross region inference endpoints
5herlocked Feb 7, 2025
a066979
Cleaned up http_client imports
5herlocked Feb 7, 2025
6a1f061
Merge branch 'main' into pr/21092
maxdeviant Feb 7, 2025
9400392
Update `Cargo.lock`
maxdeviant Feb 7, 2025
69784f4
Sort `Cargo.toml`s
maxdeviant Feb 7, 2025
33a43a2
Add LICENSE files
maxdeviant Feb 7, 2025
dc78b5c
Sort dependencies
maxdeviant Feb 7, 2025
0063bd2
Format
maxdeviant Feb 7, 2025
3fd3f23
Revert unneeded changes
maxdeviant Feb 7, 2025
40c0d26
Fix formatting
maxdeviant Feb 7, 2025
cc1aaef
Remove more lines from diff
maxdeviant Feb 7, 2025
8105f94
Sort `Cargo.toml` for `language`
maxdeviant Feb 7, 2025
7824061
Merge branch 'main' into pr/21092
maxdeviant Feb 7, 2025
ca6e509
Add `aws_http_client` to workspace members
maxdeviant Feb 7, 2025
38092c5
Shadow bindings
maxdeviant Feb 7, 2025
61b684e
Finish `Debug` impl for `AwsHttpConnector`
maxdeviant Feb 7, 2025
fca8e2c
Do some more clean up of `aws_http_client`
maxdeviant Feb 7, 2025
a62a21b
Clean up warnings
maxdeviant Feb 7, 2025
8008b87
Fix Clippy warning
maxdeviant Feb 7, 2025
d29a7a0
Remove another unneeded change from the diff
maxdeviant Feb 7, 2025
de5a9f1
Remove another unneeded change from the diff
maxdeviant Feb 7, 2025
d51b3b7
Tidy imports
maxdeviant Feb 7, 2025
2f55ef9
Removed serde stuff from Bedrock::Request
5herlocked Feb 7, 2025
72f4d43
Cargo machete fixes
5herlocked Feb 7, 2025
dbd0366
Some todos, and trying to understand why i'm hitting throttling excep…
5herlocked Feb 7, 2025
43196e4
Fixed Cargo clippy errors, WIP: bedrock tool use
5herlocked Feb 11, 2025
189a962
Tool use code is written
5herlocked Feb 12, 2025
8b819c9
Fixed issues with stalled event receiver streams
5herlocked Feb 12, 2025
1b344fd
Removed some debug printlns
5herlocked Feb 12, 2025
c07bc6a
Fixed cargo clippy errors
5herlocked Feb 12, 2025
8e544cd
Cargo fmt-d
5herlocked Feb 12, 2025
38f2545
Completely revamped how credentials work to only have one dialog
5herlocked Feb 12, 2025
0c100bf
Tool Use is fully implemented, unverified if functional
5herlocked Feb 12, 2025
85b762e
Merge branch 'zed-industries:main' into feature/bedrock-provider
5herlocked Feb 12, 2025
c379483
Added Bedrock Icon
5herlocked Feb 12, 2025
c122fd3
Updated icon to be cleaner
5herlocked Feb 12, 2025
d6c0305
Fixed the issue with the test -- simply added a gpui_tokio::init to t…
5herlocked Feb 13, 2025
21ad209
Removed unnecessary struct and case in LM Settings
5herlocked Feb 13, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ members = [
"crates/audio",
"crates/auto_update",
"crates/auto_update_ui",
"crates/aws_http_client",
"crates/bedrock",
"crates/breadcrumbs",
"crates/call",
"crates/channel",
Expand Down Expand Up @@ -215,6 +217,8 @@ assistant_tools = { path = "crates/assistant_tools" }
audio = { path = "crates/audio" }
auto_update = { path = "crates/auto_update" }
auto_update_ui = { path = "crates/auto_update_ui" }
aws_http_client = { path = "crates/aws_http_client" }
bedrock = { path = "crates/bedrock" }
breadcrumbs = { path = "crates/breadcrumbs" }
call = { path = "crates/call" }
channel = { path = "crates/channel" }
Expand Down Expand Up @@ -376,6 +380,11 @@ async-trait = "0.1"
async-tungstenite = "0.28"
async-watch = "0.3.1"
async_zip = { version = "0.0.17", features = ["deflate", "deflate64"] }
aws-smithy-types = { version = "1.2.12", features = ["http-body-1-x"] }
aws-smithy-runtime-api = { version = "1.7.3", features = ["http-1x","client"] }
aws-credential-types = { version = "1.2.1", features = ["hardcoded-credentials"] }
aws-config = { version = "1.1.7", features = ["behavior-version-latest"] }
aws-sdk-bedrockruntime = { version = "1.57.0", features = ["behavior-version-latest"] }
base64 = "0.22"
bitflags = "2.6.0"
blade-graphics = { git = "https://github.com/kvark/blade", rev = "b16f5c7bd873c7126f48c82c39e7ae64602ae74f" }
Expand Down
1 change: 1 addition & 0 deletions crates/assistant_settings/src/assistant_settings.rs
Original file line number Diff line number Diff line change
Expand Up @@ -358,6 +358,7 @@ pub struct LanguageModelSelection {
fn providers_schema(_: &mut schemars::gen::SchemaGenerator) -> schemars::schema::Schema {
schemars::schema::SchemaObject {
enum_values: Some(vec![
"bedrock".into(),
"anthropic".into(),
"google".into(),
"lmstudio".into(),
Expand Down
24 changes: 24 additions & 0 deletions crates/aws_http_client/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
[package]
name = "aws_http_client"
version = "0.1.0"
edition = "2021"
publish = false
license = "GPL-3.0-or-later"

[lints]
workspace = true

[lib]
path = "src/aws_http_client.rs"

[features]
default = []
schemars = ["dep:schemars"]

[dependencies]
aws-smithy-runtime-api.workspace = true
aws-smithy-types.workspace = true
futures.workspace = true
http_client.workspace = true
schemars = { workspace = true, optional = true }
tokio = { workspace = true, features = ["rt", "rt-multi-thread"] }
5herlocked marked this conversation as resolved.
Show resolved Hide resolved
1 change: 1 addition & 0 deletions crates/aws_http_client/LICENSE-GPL
118 changes: 118 additions & 0 deletions crates/aws_http_client/src/aws_http_client.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
use std::fmt;
use std::sync::Arc;

use aws_smithy_runtime_api::client::http::{
HttpClient as AwsClient, HttpConnector as AwsConnector,
HttpConnectorFuture as AwsConnectorFuture, HttpConnectorFuture, HttpConnectorSettings,
SharedHttpConnector,
};
use aws_smithy_runtime_api::client::orchestrator::{HttpRequest as AwsHttpRequest, HttpResponse};
use aws_smithy_runtime_api::client::result::ConnectorError;
use aws_smithy_runtime_api::client::runtime_components::RuntimeComponents;
use aws_smithy_runtime_api::http::StatusCode;
use aws_smithy_types::body::SdkBody;
use futures::AsyncReadExt;
use http_client::{AsyncBody, Inner};
use http_client::{HttpClient, Request};
use tokio::runtime::Handle;

struct AwsHttpConnector {
client: Arc<dyn HttpClient>,
handle: Handle,
}

impl std::fmt::Debug for AwsHttpConnector {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> fmt::Result {
f.debug_struct("AwsHttpConnector").finish()
}
}

impl AwsConnector for AwsHttpConnector {
fn call(&self, request: AwsHttpRequest) -> AwsConnectorFuture {
let req = match request.try_into_http1x() {
Ok(req) => req,
Err(err) => {
return HttpConnectorFuture::ready(Err(ConnectorError::other(err.into(), None)))
}
};

let (parts, body) = req.into_parts();

let response = self
.client
.send(Request::from_parts(parts, convert_to_async_body(body)));

let handle = self.handle.clone();

HttpConnectorFuture::new(async move {
let response = match response.await {
Ok(response) => response,
Err(err) => return Err(ConnectorError::other(err.into(), None)),
};
let (parts, body) = response.into_parts();
let body = convert_to_sdk_body(body, handle).await;

Ok(HttpResponse::new(
StatusCode::try_from(parts.status.as_u16()).unwrap(),
body,
))
})
}
}

#[derive(Clone)]
pub struct AwsHttpClient {
client: Arc<dyn HttpClient>,
handler: Handle,
}

impl std::fmt::Debug for AwsHttpClient {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> fmt::Result {
f.debug_struct("AwsHttpClient").finish()
}
}

impl AwsHttpClient {
pub fn new(client: Arc<dyn HttpClient>, handle: Handle) -> Self {
Self {
client,
handler: handle,
}
}
}

impl AwsClient for AwsHttpClient {
fn http_connector(
&self,
_settings: &HttpConnectorSettings,
_components: &RuntimeComponents,
) -> SharedHttpConnector {
SharedHttpConnector::new(AwsHttpConnector {
client: self.client.clone(),
handle: self.handler.clone(),
})
}
}

pub async fn convert_to_sdk_body(body: AsyncBody, handle: Handle) -> SdkBody {
match body.0 {
Inner::Empty => SdkBody::empty(),
Inner::Bytes(bytes) => SdkBody::from(bytes.into_inner()),
Inner::AsyncReader(mut reader) => {
let buffer = handle.spawn(async move {
let mut buffer = Vec::new();
let _ = reader.read_to_end(&mut buffer).await;
buffer
});

SdkBody::from(buffer.await.unwrap_or_default())
}
}
}

pub fn convert_to_async_body(body: SdkBody) -> AsyncBody {
match body.bytes() {
Some(bytes) => AsyncBody::from((*bytes).to_vec()),
None => AsyncBody::empty(),
}
}
28 changes: 28 additions & 0 deletions crates/bedrock/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
[package]
name = "bedrock"
version = "0.1.0"
edition = "2021"
publish = false
license = "GPL-3.0-or-later"

[lints]
workspace = true

[lib]
path = "src/bedrock.rs"

[features]
default = []
schemars = ["dep:schemars"]

[dependencies]
anyhow.workspace = true
aws-config = { workspace = true, features = ["behavior-version-latest"] }
aws-sdk-bedrockruntime = { workspace = true, features = ["behavior-version-latest"] }
futures.workspace = true
gpui_tokio.workspace = true
schemars = { workspace = true, optional = true }
serde.workspace = true
strum.workspace = true
thiserror.workspace = true
tokio = { workspace = true, features = ["rt", "rt-multi-thread"] }
1 change: 1 addition & 0 deletions crates/bedrock/LICENSE-GPL
Loading