v0.4.1

This is a bug fix release.

What's Changed

• Fix Mill infinite retry if Computation is already failed. (#1238) c32fe55
• Fix the bug in the llv2 that inflates the reach when maximum_frequency = 1. (#1257) 867222c
• Fix ConfirmComputationParticipant checking the status of non-participant Duchies. (#1272) fd8fe88

Full Changelog: v0.4.0...v0.4.1

v0.4.0

See the guide for updating to a new release.

Highlights

• A new preview Reporting system implementation is available which supports v2 of the Reporting API. See the deployment guide.
  • Note that the v2 Reporting system uses a separate database schema, so there is no way to migrate existing data from an existing v1 instance.
  • This is a preview. There may still be API and database changes, meaning that in-place updates may not be supported.
• There is a new Reach-Only Liquid Legions v2 protocol which should result in a ~7x improvement over the existing Reach and Frequency Liquid Legions v2 protocol for REACH Measurements.
  • This requires changes to EDP requisition fulfillment. See the enablement guide.
• Example Terraform configurations have been added for the Kingdom and Duchy, which are referenced from their respective deployment guides.
• Gaussian noise and ACDP composition in PBM are now supported, which should allow 2x the queries for the same privacy budget.
  • See the enablement guide.
• There is a new CLI tool for validating event templates.
• EDP simulators now support synthetic data specifications.
  • See the updated correctness test guide.
• The cross-media-measurement-api dependency has been updated to 0.43.0

Bug Fixes

• Some pods were not having their metrics exported. Fixed by #1140
• Negations in event filters were not being properly handled in the reference Privacy Budget Manager implementation. See #1132
• Computations involving a subset of all Duchies would fail. See #1194
• UpdatePublicKey was not actually updating the certificate. See #1209
• GetDataProvider and GetMeasurementConsumer could return the incorrect certificate. See #1215

Potentially Requiring Action

• Options for methodologies and noise mechanisms are now specified in ProtocolConfig.Direct, with the corresponding selections specified in Measurement.Result. See world-federation-of-advertisers/cross-media-measurement-api#163
  • EDPs must respect these when fulfilling requisitions.
  • EDPs should indicate their choices in Measurement.Result. This will be enforced in the future.
• The event_groups field in RequisitionSpec has been marked as deprecated in favor of a field of the same name inside of the events submessage field. See world-federation-of-advertisers/cross-media-measurement-api#164
  • Halo clients that build RequisitionSpecs will still populate the deprecated field, but the field may be removed in a future release.
• The custom wfa.measurement.api.v2alpha.TimeInterval type in the CMMS public API has been replaced with google.type.Interval. This is a wire-compatible change, but may break compilation of client code. See world-federation-of-advertisers/cross-media-measurement-api#155
• The maximum_frequency field in ProtocolConfig.LiquidLegionsV2 is now deprecated. See world-federation-of-advertisers/cross-media-measurement-api#165
  • This used to be populated from the Kingdom's Llv2ProtocolConfigConfig. It must not be specified there any longer.
• The maximum_frequency_per_user field was dropped from MeasurementSpec.Duration. See world-federation-of-advertisers/cross-media-measurement-api#165
• The maximum_frequency_per_user field in MeasurementSpec.ReachAndFrequency was renamed to maximum_frequency. See world-federation-of-advertisers/cross-media-measurement-api#165
  • This field is now required in CreateMeasurement requests.
• The type of the maximum_watch_duration_per_user field in MeasurementSpec.Duration has been changed from int32 to google.protobuf.Duration. See world-federation-of-advertisers/cross-media-measurement-api#167
• The system API definition was updated to fix AIP conformance issues. See #1074
  • This included a change to the behavior of StreamActiveComputations, which resulted in an incompatible change to the StreamMeasurements method of the Kingdom internal API.
• EDP simulators no longer write to blob storage. See #1112
  • There are now two different EDP simulator implementations with corresponding correctness test implementations. See the updated correctness test guide.
• The Kingdom public and internal API servers have a new required --ro-llv2-protocol-config-config option for the reach-only Liquid Legions v2 protocol.
  • The dev configuration reads this from a ro_llv2_protocol_config_config.textproto file in the certs-and-configs K8s Secret. See the updated deployment guide.
• The v1 Reporting public API server has a new required --measurement-spec-config-file option.
  • The dev configuration reads this from a measurement_spec_config.textproto file in the config-files K8s ConfigMap. See the updated deployment guide.

What's Changed

• Fix image-tag workflow output. by @SanjayVas in #1086
• Use batch DDL for update-vid-model-foreign-keys-and-indexes.sql. by @SanjayVas in #1087
• Adding Hello World React app to the reporting UI by @bdomen-ggl in #1033
• Implement PostgresComputationStatsService by @YuhongWang-Amazon in #1070
• Update common-jvm to 0.59.0. by @SanjayVas in #1084
• Add script to run api-linter. by @SanjayVas in #1073
• Fix api-linter errors in system API. by @SanjayVas in #1074
• Add resource IDs to create requests by @riemanli in #1071
• Add generation of events from SimulatorSyntheticDataSpec. by @tristanvuong2021 in #1083
• Implement Event Groups Service Reporting V2Alpha. by @tristanvuong2021 in #1088
• Create AcdpParamsConverter and ACDP composition method by @iverson52000 in #1062
• Adding base test framework for react by @bdomen-ggl in #1085
• Fix that only allows creation of ModelRollouts with RolloutPeriodStartTime after that of previous ModelRollout by @jojijac0b in #1076
• Replace timeinterval with rollout deploy period by @Marco-Premier in #1091
• Update cross-media-measurement-api to 0.38.0 and common-jvm to 0.60.0. by @SanjayVas in #1092
• Partially Implement Integration Tests for Reporting V2Alpha. by @tristanvuong2021 in #1093
• Replace use of deprecated event_field message option in test templates. by @SanjayVas in #1094
• Refactor panel exchange to use Kustomize by @Marco-Premier in #1090
• Fix SyntheticDataGeneration signature. by @SanjayVas in #1101
• Add the state machine and the encryption methods proto for the reach … by @ple13 in #1096
• Kingdom supports creating ROLLv2 Measurements. by @renjiezh in #1097
• Pass EventGroup to EventQuery. by @SanjayVas in https://github.com/world-federation-of-advertisers/cross-me...

v0.3.0

Highlights

• The panel-exchange-client repository has been merged into this one.
  • Example panel exchange daemon container images are now published to GitHub packages (see #1079)
• Exchange API resources are now covered by retention policy enforcement jobs (see #1027)
• The EDP simulator ignores Requisitions for other MeasurementConsumers (see #1067)
• cross-media-measurement-api has been updated to v0.36.0
• The LLv2 Mill supports multi-threaded operations. The dev and local configurations use 2 threads by default. See #1039

Potentially Requiring Action

• Breaking changes to the public API for AIP conformance.
  • CreateMeasurementRequest now has a required parent field (see world-federation-of-advertisers/cross-media-measurement-api#135)
  • ListEventGroupsRequest now has two patterns for its parent field (see world-federation-of-advertisers/cross-media-measurement-api#148)
    • If the caller is a DataProvider, the parent field must use the dataProviders/{data_provider}/eventGroups/{event_group} pattern. This is the canonical pattern for EventGroup resources.
    • If the caller is a MeasurementConsumer, the parent field must use the measurementConsumers/{measurement_consumer}/eventGroups/{event_group} pattern
  • ListRequisitionsRequest now has two patterns for its parent field (see world-federation-of-advertisers/cross-media-measurement-api#149)
    • If the caller is a DataProvider, the parent field must use the dataProviders/{data_provider}/requisitions/{requisition} pattern. This is the canonical pattern for Requisition resources.
    • If the caller is a MeasurementConsumer, the parent field must use the measurementConsumers/{measurement_consumer}/measurements/{measurement}/requisitions/{requisition} pattern.
• Binary-compatible changes to the public API for AIP conformance.
  • Various fields were renamed. See world-federation-of-advertisers/cross-media-measurement-api#137

What's Changed

• Update lint action to v2. by @SanjayVas in #1008
• Add MeasurementResultSubject Truth extension. by @SanjayVas in #1002
• modelrelease internal api by @Marco-Premier in #988
• Update cross-media-measurement-api to 0.31.0. by @SanjayVas in #1014
• Fix secrets not being available to release workflows. by @SanjayVas in #1013
• adding grpc-gateway to generate bff reporting definitions by @bdomen-ggl in #1006
• Increase the time range for the correctness test. by @SanjayVas in #1015
• Add monitoring service account to all clusters. by @SanjayVas in #1016
• Update dev resource requirements based on observed usage. by @SanjayVas in #1019
• model line public api by @Marco-Premier in #1012
• modelrollout internal api by @Marco-Premier in #1018
• Implement Internal Metrics Service Read Methods for Reporting V2 by @tristanvuong2021 in #1009
• model release public api by @Marco-Premier in #1022
• create model shard spanner service by @Marco-Premier in #1021
• model outage internal api by @Marco-Premier in #1010
• Trim .tar.gz files from Bazel cache before saving. by @SanjayVas in #1020
• Implement exchange deletion for internal spanner implementation by @jcorilla in #1000
• Specify request_id in CreateMeasurement and CreateEventGroup requests. by @SanjayVas in #1023
• sync file with panel-exchange-client repo by @Marco-Premier in #999
• Add Unique indexes on VID models tables by @Marco-Premier in #1025
• Add cronjob configs and classes to enable exchanges retention policy by @jcorilla in #1027
• Check event filter if null or blank. by @riemanli in #1035
• Using CMMS event group key for CMMS event groups. by @riemanli in #1038
• Update cross-media-measurement-api to v0.33.0. by @SanjayVas in #1024
• Put ErrorInfo in Status details instead of trailers. by @SanjayVas in #1043
• Add CLI commands for model suite by @jojijac0b in #1011
• model rollouts public api by @Marco-Premier in #1031
• Fix naming of exchange retention days to live flag by @jcorilla in #1049
• Add model outages public api by @Marco-Premier in #1032
• Add proto definition for simulator synthetic data specification by @jcorilla in #1037
• Implement Internal Measurements Service Reporting V2 by @tristanvuong2021 in #1036
• Add MetricSpecBuilder. by @riemanli in #1041
• Add createReport. by @riemanli in #1004
• Fix event filtration formation in v2alpha metrics service by @riemanli in #1044
• vid distribution cli by @Marco-Premier in #1047
• Add ModelShard public api by @Marco-Premier in #1026
• Sync Maven dependencies with panel-match-client repo. by @SanjayVas in #1052
• Configure metrics in K8s workflows. by @SanjayVas in #1017
• Update common-jvm to 0.56.0. by @SanjayVas in #1054
• Add documentation for kingdom retention policies by @jcorilla in #1050
• Add getReport. by @riemanli in #1029
• Fix Ordering of Responses from Batch Get Methods in Reporting V2 by @tristanvuong2021 in #1056
• Ignore party field in public API GetExchangeRequest. by @SanjayVas in #1028
• Update cross-media-measurement-api to 0.34.0. by @SanjayVas in #1055
• Add type registry and cel env caching. by @tristanvuong2021 in #983
• Add NONE DirectNoiseMechanism. by @iverson52000 in #1040
• Add listReport by @riemanli in #1053
• Fix cross-media-measurement-api repo target to use version. by @SanjayVas in #1058
• Validate creation of event group metadata descriptor by @tristanvuong2021 in #1057
• Implement postgres continuation token service by @YuhongWang-Amazon in #1059
• Divide batch requests to several calls by @riemanli in #1046
• Implement internal reports service report create for reporting v2 by @...

v0.2.0

Highlights

• Added a ReplaceDataProviderRequiredDuchies method to the public API (#966)
  • This allows DataProviders to update the list of which Duchies must participate in Measurements involving that DataProvider.
• Added Terraform modules for CMMS components and the Reporting system (#972, #994, #995)
  • See the cmms root module for an example of how to use these.
• Deployment to the dev environment is now automated.
  • Progress for this release: https://github.com/world-federation-of-advertisers/cross-media-measurement/actions/runs/5016387258

What's Changed

• Update GKE correctness test guide to use GcsCorrectnessTest. by @SanjayVas in #948
• Use new build targets for release workflows. by @SanjayVas in #965
• Enable public API to use new ReplaceDataProviderRequiredDuchies method in internal API by @jojijac0b in #966
• Implement list event group metadata descriptors by @tristanvuong2021 in #967
• Define internal exchange service method for streaming and batch deletion by @jcorilla in #973
• Create Cloud Spanner database schema for the Kingdom VID model distribution related tables by @jojijac0b in #977
• Deploy CMMS to dev environment on release. by @SanjayVas in #978
• Deploy Reporting system to dev environment on release. by @SanjayVas in #981
• model suite internal api by @Marco-Premier in #979
• Fix always-true conditional check in SpannerDataProviderService by @Marco-Premier in #986
• Implement internal spanner stream exchanges method by @jcorilla in #975
• Create protobuf message and service for model shards by @jojijac0b in #980
• Set tags for correctness test targets that run on external clusters. by @SanjayVas in #1001
• Deploy EDP simulators to dev environment on release. by @SanjayVas in #991
• Fix creation of reporting sets with no filters by @tristanvuong2021 in #989
• Rewrite Terraform configuration to be more modular. by @SanjayVas in #972
• Add Terraform config for Reporting system. by @SanjayVas in #994
• Add Terraform config for EDP simulators. by @SanjayVas in #995
• Add createReportingSet. by @riemanli in #985
• Implement internal Metrics Service Create Methods for Reporting V2. by @tristanvuong2021 in #982
• Terraform CMMS as part of update workflow. by @SanjayVas in #996
• Add listReportingSets by @riemanli in #992
• Consolidate GitHub Environment variables. by @SanjayVas in #997
• modelline internal api by @Marco-Premier in #987
• Add CLI command to replace DataProviders duchy list by @jojijac0b in #970
• Introduce Requisition Validators by @uakyol in #990
• Run correctness test as part of update workflow. by @SanjayVas in #998
• Add MultithreadingHelper to refactor multithreaded LLV2 by @renjiezh in #976
• Update Bazel actions to v2. by @SanjayVas in #1003
• model suite public api by @Marco-Premier in #993
• Detail the error types in createMetric and batchCreateMetrics. by @riemanli in #1005

Full Changelog: v0.1.0...v0.2.0

v0.1.0

Initial "minimum viable product" release. The corresponding cross-media-measurement-api release is v0.28.1.

Container Images

Published to https://github.com/orgs/world-federation-of-advertisers/packages?repo_name=cross-media-measurement where the image tag matches the version.

v0.1.0-rc3

Initial "minimum viable product" release candidate.

Changes Requiring Action

• LiquidLegionsV2NoiseConfig has a new noise_mechanism field that must be set in the Kingdom certs-and-configs secret.

Incomplete Features

• Automation of Measurement retention policies

Container Images

Published to https://github.com/orgs/world-federation-of-advertisers/packages?repo_name=cross-media-measurement where the image tag matches the version.

What's Changed

• v2alpha metrics service -- createMetric & batchCreateMetrics by @riemanli in #882
• Add ForwardedStorageCorrectnessTest. by @SanjayVas in #936
• Provide default value for unspecified NoiseMechanism for old Measurements by @renjiezh in #957

Full Changelog: v0.1.0-rc2...v0.1.0-rc3

Initial "MVP" release

Initial "minimum viable product" release candidate.

WARNING

This release has a serious bug that may cause Duchies to crash loop. Do not update to this release. See #956

Changes Requiring Action

• LiquidLegionsV2NoiseConfig has a new noise_mechanism field that must be set in the Kingdom certs-and-configs secret.

Incomplete Features

• Automation of Measurement retention policies

Container Images

Published to https://github.com/orgs/world-federation-of-advertisers/packages where the image tag matches the version number.

What's Changed

• Select Noise Mechanism between Geometric and Gaussian. by @renjiezh in #917
• Add created and updated before fields to stream measurements filter by @jcorilla in #947
• Add etag validation option for internal batch measurement operations by @jcorilla in #933
• Add measurement consumers service to internal reporting v2 by @tristanvuong2021 in #940
• Update page sizes. by @tristanvuong2021 in #928
• Implement logic to update Data Provider duchy list by @jojijac0b in #934
• Fix BigQuery timestamp conversions. by @SanjayVas in #952
• Use date range which exists in test data CSV by @SanjayVas in #954

Full Changelog: v0.1.0-rc1...v0.1.0-rc2

Initial "MVP" release

Initial "minimum viable product" release candidate.

Incomplete Features

• Automation of Measurement retention policies

Container Images

Published to https://github.com/orgs/world-federation-of-advertisers/packages where the image tag matches the version number.