Skip to content

Commit

Permalink
feat: A new field response_logprbs is added to message `.google.clo…
Browse files Browse the repository at this point in the history
…ud.aiplatform.v1.GenerationConfig` (#5664)

* feat: add OFF to HarmBlockThreshold

PiperOrigin-RevId: 671827417

Source-Link: googleapis/googleapis@9aad398

Source-Link: googleapis/googleapis-gen@428081a
Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6IjQyODA4MWFmNDk4MWM3YThkY2UwYmRlMDQ1Yzg2YmM5NTc4ZTUwZjIifQ==

* feat: add OFF to HarmBlockThreshold

PiperOrigin-RevId: 671988102

Source-Link: googleapis/googleapis@cb39bdd

Source-Link: googleapis/googleapis-gen@4f92fc4
Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6IjRmOTJmYzRhY2QyYTY5MWFkOGMwZDRiZTk1NDYxMjdkMDAwZDJjMzgifQ==

* feat: add share_point_sources to SharePointSources
feat: add sharepoint_folder_path to SharePointSource
feat: add sharepoint_folder_id to SharePointSource
feat: add drive_name to SharePointSource
feat: add drive_id to SharePointSource
feat: add client_id to SharePointSource
feat: add client_secret to SharePointSource
feat: add tenant_id to SharePointSource
feat: add sharepoint_site_name to SharePointSource
feat: add share_point_sources to ImportRagFilesConfig
feat: add partial_failure_gcs_sink tp ImportRagFilesConfig
feat: add partial_failure_bigquery_sink to ImportRagFilesConfig

PiperOrigin-RevId: 672615314

Source-Link: googleapis/googleapis@ca82297

Source-Link: googleapis/googleapis-gen@bf7ddca
Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6ImJmN2RkY2E2NWEwNzFiZGVhN2JjOGNkODlkYjEzNTcwNDU2MDIwYzAifQ==

* feat: add Pinecone and Vector Search integration for Vertex RAG

PiperOrigin-RevId: 673087899

Source-Link: googleapis/googleapis@afb6b35

Source-Link: googleapis/googleapis-gen@00a4515
Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6IjAwYTQ1MTVhYjQ2NWU5OGQ1NjYyNzA3NTY3NTIwOTYzMWVlNTFmMzkifQ==

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* feat: add file_id to SharePointSource
feat: add share_point_sources to RagFile

PiperOrigin-RevId: 673921028

Source-Link: googleapis/googleapis@9382d74

Source-Link: googleapis/googleapis-gen@7cb8810
Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6IjdjYjg4MTBkMmVhZTVhNjFkNDk3MmUxZTcwYjdjM2VlZmRiMWM5NmIifQ==

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* feat: A new field `property_ordering` is added to message `.google.cloud.aiplatform.v1beta1.Schema`

PiperOrigin-RevId: 674918528

Source-Link: googleapis/googleapis@65306b9

Source-Link: googleapis/googleapis-gen@442fa35
Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6IjQ0MmZhMzVkMTY0MWRlNGZmNGZlZmQ4ZDZlNjcyYWE4OTZhNTEzNTQifQ==

* feat: A new field `property_ordering` is added to message `.google.cloud.aiplatform.v1.Schema`

PiperOrigin-RevId: 675187467

Source-Link: googleapis/googleapis@38d33ad

Source-Link: googleapis/googleapis-gen@95e20a9
Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6Ijk1ZTIwYTk1NDdlMWNiY2IwYTJjZTQ0M2Y2MmQwNWJjMDAxYTc5YmIifQ==

* feat: A new field `generation_config` is added to message `.google.cloud.aiplatform.v1beta1.CountTokensRequest`
feat: A new field `labels` is added to message `.google.cloud.aiplatform.v1beta1.GenerateContentRequest`

PiperOrigin-RevId: 675205129

Source-Link: googleapis/googleapis@fd52b57

Source-Link: googleapis/googleapis-gen@5ce7af1
Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6IjVjZTdhZjFiMjgwOGZhYzEwOGUyNDNhZTMwOWEyMGFjMGE5ODc1MzQifQ==

* fix: set canonical scopes for all LRO calls

PiperOrigin-RevId: 675654692

Source-Link: googleapis/googleapis@afbb0a8

Source-Link: googleapis/googleapis-gen@85160b5
Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6Ijg1MTYwYjVlZDA0MGQzYzViMTFlNmVkZGI5NTdlNzljNjQ4ZDI4ZGEifQ==

* feat: add psc_automation_configs to DeployIndex v1beta1

PiperOrigin-RevId: 675732136

Source-Link: googleapis/googleapis@551cdf4

Source-Link: googleapis/googleapis-gen@e18f938
Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6ImUxOGY5MzhjM2ZkMDc0NGE4YjVkNTc4MWI0MWRiNTMwMmY4ZjMwYjcifQ==

* feat: A new field `generation_config` is added to message `.google.cloud.aiplatform.v1.CountTokensRequest`
feat: A new field `labels` is added to message `.google.cloud.aiplatform.v1.GenerateContentRequest`

PiperOrigin-RevId: 675839175

Source-Link: googleapis/googleapis@49d2887

Source-Link: googleapis/googleapis-gen@e6a2931
Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6ImU2YTI5MzFjNDVjZWVlYTkyMWUyNDU5MDRlZWYxMDc1YzllNDQxODAifQ==

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* feat: Add CIVIC_INTEGRITY category to SafetySettings for prediction service

PiperOrigin-RevId: 676040723

Source-Link: googleapis/googleapis@cf775e2

Source-Link: googleapis/googleapis-gen@17d42e2
Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6IjE3ZDQyZTI2Y2RjYjkwYmIxNGI0ZWZiZjQ3NjFkNzczOWI2YjMwNmIifQ==

* feat: Add CIVIC_INTEGRITY category to SafetySettings for prediction service

PiperOrigin-RevId: 676040944

Source-Link: googleapis/googleapis@2d1f337

Source-Link: googleapis/googleapis-gen@adb1880
Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6ImFkYjE4ODBkNGRhZmM4YjExOWU0ODRjMjUwYjU2OTRhMjUwNDI5NzEifQ==

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* feat: A new field `any_of` is added to message `.google.cloud.aiplatform.v1beta1.Schema`

PiperOrigin-RevId: 676461662

Source-Link: googleapis/googleapis@b86cd1c

Source-Link: googleapis/googleapis-gen@5c6bc1b
Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6IjVjNmJjMWI5YTY5MGU5YThmYjQ3MzFjMjRjMDkwMWI0M2JiMDNiOWQifQ==

* feat: A new field `any_of` is added to message `.google.cloud.aiplatform.v1.Schema`

PiperOrigin-RevId: 676462752

Source-Link: googleapis/googleapis@bd0c988

Source-Link: googleapis/googleapis-gen@3ac8004
Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6IjNhYzgwMDQ1NDU1ODljNjliNGIwYWVlNTFlZmFlNDJlYzViMGYyMmQifQ==

* feat: A new field `response_logprbs` is added to message `.google.cloud.aiplatform.v1beta1.GenerationConfig`
feat: A new field `logprbs` is added to message `.google.cloud.aiplatform.v1beta1.GenerationConfig`
feat: A new field `logprobs_result` is added to message `.google.cloud.aiplatform.v1beta1.Candidate`
feat: A new field `model_version` is added to message `.google.cloud.aiplatform.v1beta1.GenerateContentResponse`

PiperOrigin-RevId: 676506886

Source-Link: googleapis/googleapis@e88eeaa

Source-Link: googleapis/googleapis-gen@35cac65
Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6IjM1Y2FjNjU4OTU0ZDg4MTQ4YTA0ZWMyNjVhMmE4MGY5YTg0ZmE0OTAifQ==

* feat: A new field `response_logprbs` is added to message `.google.cloud.aiplatform.v1.GenerationConfig`
feat: A new field `logprbs` is added to message `.google.cloud.aiplatform.v1.GenerationConfig`
feat: A new field `logprobs_result` is added to message `.google.cloud.aiplatform.v1.Candidate`
feat: A new field `model_version` is added to message `.google.cloud.aiplatform.v1.GenerateContentResponse`

PiperOrigin-RevId: 676556181

Source-Link: googleapis/googleapis@07b069b

Source-Link: googleapis/googleapis-gen@919c469
Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6IjkxOWM0NjkyOWQyZWI2NDI3MTE2ODk1YWU0OTI2ZmQ4NjM2YzdkMmQifQ==

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: sofisl <[email protected]>
  • Loading branch information
3 people authored Sep 20, 2024
1 parent 0d37602 commit 100e106
Show file tree
Hide file tree
Showing 63 changed files with 8,735 additions and 50 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,9 @@ enum HarmCategory {

// The harm category is sexually explicit content.
HARM_CATEGORY_SEXUALLY_EXPLICIT = 4;

// The harm category is civic integrity.
HARM_CATEGORY_CIVIC_INTEGRITY = 5;
}

// The base structured datatype containing multi-part content of a message.
Expand Down Expand Up @@ -197,6 +200,12 @@ message GenerationConfig {
// Optional. Stop sequences.
repeated string stop_sequences = 6 [(google.api.field_behavior) = OPTIONAL];

// Optional. If true, export the logprobs results in response.
optional bool response_logprobs = 18 [(google.api.field_behavior) = OPTIONAL];

// Optional. Logit probabilities.
optional int32 logprobs = 7 [(google.api.field_behavior) = OPTIONAL];

// Optional. Positive penalties.
optional float presence_penalty = 8 [(google.api.field_behavior) = OPTIONAL];

Expand Down Expand Up @@ -247,6 +256,9 @@ message SafetySetting {

// Block none.
BLOCK_NONE = 4;

// Turn off the safety filter.
OFF = 5;
}

// Probability vs severity.
Expand Down Expand Up @@ -412,6 +424,10 @@ message Candidate {
// Output only. Average log probability score of the candidate.
double avg_logprobs = 9 [(google.api.field_behavior) = OUTPUT_ONLY];

// Output only. Log-likelihood scores for the response tokens and top tokens
LogprobsResult logprobs_result = 10
[(google.api.field_behavior) = OUTPUT_ONLY];

// Output only. The reason why the model stopped generating tokens.
// If empty, the model has not stopped generating the tokens.
FinishReason finish_reason = 3 [(google.api.field_behavior) = OUTPUT_ONLY];
Expand All @@ -436,6 +452,34 @@ message Candidate {
[(google.api.field_behavior) = OUTPUT_ONLY];
}

// Logprobs Result
message LogprobsResult {
// Candidate for the logprobs token and score.
message Candidate {
// The candidate’s token string value.
optional string token = 1;

// The candidate’s token id value.
optional int32 token_id = 3;

// The candidate's log probability.
optional float log_probability = 2;
}

// Candidates with top log probabilities at each decoding step.
message TopCandidates {
// Sorted by log probability in descending order.
repeated Candidate candidates = 1;
}

// Length = total number of decoding steps.
repeated TopCandidates top_candidates = 1;

// Length = total number of decoding steps.
// The chosen candidates may or may not be in top_candidates.
repeated Candidate chosen_candidates = 2;
}

// Segment of the content.
message Segment {
// Output only. The index of a Part object within its parent Content object.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,8 @@ enum Type {

// Schema is used to define the format of input/output data. Represents a select
// subset of an [OpenAPI 3.0 schema
// object](https://spec.openapis.org/oas/v3.0.3#schema). More fields may be
// added in the future as needed.
// object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may
// be added in the future as needed.
message Schema {
// Optional. The type of the data.
Type type = 1 [(google.api.field_behavior) = OPTIONAL];
Expand Down Expand Up @@ -98,6 +98,12 @@ message Schema {
// Properties of Type.OBJECT.
map<string, Schema> properties = 3 [(google.api.field_behavior) = OPTIONAL];

// Optional. The order of the properties.
// Not a standard field in open api spec. Only used to support the order of
// the properties.
repeated string property_ordering = 25
[(google.api.field_behavior) = OPTIONAL];

// Optional. Required properties of Type.OBJECT.
repeated string required = 5 [(google.api.field_behavior) = OPTIONAL];

Expand Down Expand Up @@ -128,4 +134,8 @@ message Schema {
// Optional. Example of the object. Will only populated when the object is the
// root.
google.protobuf.Value example = 4 [(google.api.field_behavior) = OPTIONAL];

// Optional. The value should be validated against any (one or more) of the
// subschemas in the list.
repeated Schema any_of = 11 [(google.api.field_behavior) = OPTIONAL];
}
Original file line number Diff line number Diff line change
Expand Up @@ -653,6 +653,11 @@ message CountTokensRequest {
// external systems to perform an action, or set of actions, outside of
// knowledge and scope of the model.
repeated Tool tools = 6 [(google.api.field_behavior) = OPTIONAL];

// Optional. Generation config that the model will use to generate the
// response.
optional GenerationConfig generation_config = 7
[(google.api.field_behavior) = OPTIONAL];
}

// Response message for [PredictionService.CountTokens][].
Expand Down Expand Up @@ -702,6 +707,15 @@ message GenerateContentRequest {
// request.
ToolConfig tool_config = 7 [(google.api.field_behavior) = OPTIONAL];

// Optional. The labels with user-defined metadata for the request. It is used
// for billing and reporting only.
//
// Label keys and values can be no longer than 63 characters
// (Unicode codepoints) and can only contain lowercase letters, numeric
// characters, underscores, and dashes. International characters are allowed.
// Label values are optional. Label keys must start with a letter.
map<string, string> labels = 10 [(google.api.field_behavior) = OPTIONAL];

// Optional. Per request settings for blocking unsafe content.
// Enforced on GenerateContentResponse.candidates.
repeated SafetySetting safety_settings = 3
Expand Down Expand Up @@ -756,12 +770,16 @@ message GenerateContentResponse {
// Number of tokens in the response(s).
int32 candidates_token_count = 2;

// Total token count for prompt and response candidates.
int32 total_token_count = 3;
}

// Output only. Generated candidates.
repeated Candidate candidates = 2 [(google.api.field_behavior) = OUTPUT_ONLY];

// Output only. The model version used to generate the response.
string model_version = 11 [(google.api.field_behavior) = OUTPUT_ONLY];

// Output only. Content filter results for a prompt sent in the request.
// Note: Sent only in the first stream chunk.
// Only happens when no candidates were generated due to content violations.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,9 @@ enum HarmCategory {

// The harm category is sexually explicit content.
HARM_CATEGORY_SEXUALLY_EXPLICIT = 4;

// The harm category is civic integrity.
HARM_CATEGORY_CIVIC_INTEGRITY = 5;
}

// The base structured datatype containing multi-part content of a message.
Expand Down Expand Up @@ -197,6 +200,12 @@ message GenerationConfig {
// Optional. Stop sequences.
repeated string stop_sequences = 6 [(google.api.field_behavior) = OPTIONAL];

// Optional. If true, export the logprobs results in response.
optional bool response_logprobs = 18 [(google.api.field_behavior) = OPTIONAL];

// Optional. Logit probabilities.
optional int32 logprobs = 7 [(google.api.field_behavior) = OPTIONAL];

// Optional. Positive penalties.
optional float presence_penalty = 8 [(google.api.field_behavior) = OPTIONAL];

Expand Down Expand Up @@ -247,6 +256,9 @@ message SafetySetting {

// Block none.
BLOCK_NONE = 4;

// Turn off the safety filter.
OFF = 5;
}

// Probability vs severity.
Expand Down Expand Up @@ -409,6 +421,10 @@ message Candidate {
// Output only. Average log probability score of the candidate.
double avg_logprobs = 9 [(google.api.field_behavior) = OUTPUT_ONLY];

// Output only. Log-likelihood scores for the response tokens and top tokens
LogprobsResult logprobs_result = 10
[(google.api.field_behavior) = OUTPUT_ONLY];

// Output only. The reason why the model stopped generating tokens.
// If empty, the model has not stopped generating the tokens.
FinishReason finish_reason = 3 [(google.api.field_behavior) = OUTPUT_ONLY];
Expand All @@ -433,6 +449,34 @@ message Candidate {
[(google.api.field_behavior) = OUTPUT_ONLY];
}

// Logprobs Result
message LogprobsResult {
// Candidate for the logprobs token and score.
message Candidate {
// The candidate’s token string value.
optional string token = 1;

// The candidate’s token id value.
optional int32 token_id = 3;

// The candidate's log probability.
optional float log_probability = 2;
}

// Candidates with top log probabilities at each decoding step.
message TopCandidates {
// Sorted by log probability in descending order.
repeated Candidate candidates = 1;
}

// Length = total number of decoding steps.
repeated TopCandidates top_candidates = 1;

// Length = total number of decoding steps.
// The chosen candidates may or may not be in top_candidates.
repeated Candidate chosen_candidates = 2;
}

// Segment of the content.
message Segment {
// Output only. The index of a Part object within its parent Content object.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -261,6 +261,12 @@ message DeployedIndex {
//
// Note: we only support up to 5 deployment groups(not including 'default').
string deployment_group = 11 [(google.api.field_behavior) = OPTIONAL];

// Optional. If set for PSC deployed index, PSC connection will be
// automatically created after deployment is done and the endpoint information
// is populated in private_endpoints.psc_automated_endpoints.
repeated PSCAutomationConfig psc_automation_configs = 19
[(google.api.field_behavior) = OPTIONAL];
}

// Used to set up the auth on the DeployedIndex's private endpoint.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -203,3 +203,48 @@ message JiraSource {
repeated JiraQueries jira_queries = 1
[(google.api.field_behavior) = REQUIRED];
}

// The SharePointSources to pass to ImportRagFiles.
message SharePointSources {
// An individual SharePointSource.
message SharePointSource {
// The SharePoint folder source. If not provided, uses "root".
oneof folder_source {
// The path of the SharePoint folder to download from.
string sharepoint_folder_path = 5;

// The ID of the SharePoint folder to download from.
string sharepoint_folder_id = 6;
}

// The SharePoint drive source.
oneof drive_source {
// The name of the drive to download from.
string drive_name = 7;

// The ID of the drive to download from.
string drive_id = 8;
}

// The Application ID for the app registered in Microsoft Azure Portal.
// The application must also be configured with MS Graph permissions
// "Files.ReadAll", "Sites.ReadAll" and BrowserSiteLists.Read.All.
string client_id = 1;

// The application secret for the app registered in Azure.
ApiAuth.ApiKeyConfig client_secret = 2;

// Unique identifier of the Azure Active Directory Instance.
string tenant_id = 3;

// The name of the SharePoint site to download from. This can be the site
// name or the site id.
string sharepoint_site_name = 4;

// Output only. The SharePoint file id. Output only.
string file_id = 9 [(google.api.field_behavior) = OUTPUT_ONLY];
}

// The SharePoint sources.
repeated SharePointSource share_point_sources = 1;
}
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,8 @@ enum Type {

// Schema is used to define the format of input/output data. Represents a select
// subset of an [OpenAPI 3.0 schema
// object](https://spec.openapis.org/oas/v3.0.3#schema). More fields may be
// added in the future as needed.
// object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may
// be added in the future as needed.
message Schema {
// Optional. The type of the data.
Type type = 1 [(google.api.field_behavior) = OPTIONAL];
Expand Down Expand Up @@ -98,6 +98,12 @@ message Schema {
// Properties of Type.OBJECT.
map<string, Schema> properties = 3 [(google.api.field_behavior) = OPTIONAL];

// Optional. The order of the properties.
// Not a standard field in open api spec. Only used to support the order of
// the properties.
repeated string property_ordering = 25
[(google.api.field_behavior) = OPTIONAL];

// Optional. Required properties of Type.OBJECT.
repeated string required = 5 [(google.api.field_behavior) = OPTIONAL];

Expand Down Expand Up @@ -128,4 +134,8 @@ message Schema {
// Optional. Example of the object. Will only populated when the object is the
// root.
google.protobuf.Value example = 4 [(google.api.field_behavior) = OPTIONAL];

// Optional. The value should be validated against any (one or more) of the
// subschemas in the list.
repeated Schema any_of = 11 [(google.api.field_behavior) = OPTIONAL];
}
Loading

0 comments on commit 100e106

Please sign in to comment.