v0.4.0
See the guide for updating to a new release.
Highlights
- A new preview Reporting system implementation is available which supports v2 of the Reporting API. See the deployment guide.
- Note that the v2 Reporting system uses a separate database schema, so there is no way to migrate existing data from an existing v1 instance.
- This is a preview. There may still be API and database changes, meaning that in-place updates may not be supported.
- There is a new Reach-Only Liquid Legions v2 protocol which should result in a ~7x improvement over the existing Reach and Frequency Liquid Legions v2 protocol for
REACH
Measurements.- This requires changes to EDP requisition fulfillment. See the enablement guide.
- Example Terraform configurations have been added for the Kingdom and Duchy, which are referenced from their respective deployment guides.
- Gaussian noise and ACDP composition in PBM are now supported, which should allow 2x the queries for the same privacy budget.
- See the enablement guide.
- There is a new CLI tool for validating event templates.
- EDP simulators now support synthetic data specifications.
- See the updated correctness test guide.
- The cross-media-measurement-api dependency has been updated to 0.43.0
Bug Fixes
- Some pods were not having their metrics exported. Fixed by #1140
- Negations in event filters were not being properly handled in the reference Privacy Budget Manager implementation. See #1132
- Computations involving a subset of all Duchies would fail. See #1194
UpdatePublicKey
was not actually updating the certificate. See #1209GetDataProvider
andGetMeasurementConsumer
could return the incorrect certificate. See #1215
Potentially Requiring Action
- Options for methodologies and noise mechanisms are now specified in
ProtocolConfig.Direct
, with the corresponding selections specified inMeasurement.Result
. See world-federation-of-advertisers/cross-media-measurement-api#163- EDPs must respect these when fulfilling requisitions.
- EDPs should indicate their choices in
Measurement.Result
. This will be enforced in the future.
- The
event_groups
field inRequisitionSpec
has been marked as deprecated in favor of a field of the same name inside of theevents
submessage field. See world-federation-of-advertisers/cross-media-measurement-api#164- Halo clients that build
RequisitionSpec
s will still populate the deprecated field, but the field may be removed in a future release.
- Halo clients that build
- The custom
wfa.measurement.api.v2alpha.TimeInterval
type in the CMMS public API has been replaced withgoogle.type.Interval
. This is a wire-compatible change, but may break compilation of client code. See world-federation-of-advertisers/cross-media-measurement-api#155 - The
maximum_frequency
field inProtocolConfig.LiquidLegionsV2
is now deprecated. See world-federation-of-advertisers/cross-media-measurement-api#165- This used to be populated from the Kingdom's
Llv2ProtocolConfigConfig
. It must not be specified there any longer.
- This used to be populated from the Kingdom's
- The
maximum_frequency_per_user
field was dropped fromMeasurementSpec.Duration
. See world-federation-of-advertisers/cross-media-measurement-api#165 - The
maximum_frequency_per_user
field inMeasurementSpec.ReachAndFrequency
was renamed tomaximum_frequency
. See world-federation-of-advertisers/cross-media-measurement-api#165- This field is now required in
CreateMeasurement
requests.
- This field is now required in
- The type of the
maximum_watch_duration_per_user
field inMeasurementSpec.Duration
has been changed fromint32
togoogle.protobuf.Duration
. See world-federation-of-advertisers/cross-media-measurement-api#167 - The system API definition was updated to fix AIP conformance issues. See #1074
- This included a change to the behavior of
StreamActiveComputations
, which resulted in an incompatible change to theStreamMeasurements
method of the Kingdom internal API.
- This included a change to the behavior of
- EDP simulators no longer write to blob storage. See #1112
- There are now two different EDP simulator implementations with corresponding correctness test implementations. See the updated correctness test guide.
- The Kingdom public and internal API servers have a new required
--ro-llv2-protocol-config-config
option for the reach-only Liquid Legions v2 protocol.- The
dev
configuration reads this from aro_llv2_protocol_config_config.textproto
file in thecerts-and-configs
K8s Secret. See the updated deployment guide.
- The
- The v1 Reporting public API server has a new required
--measurement-spec-config-file
option.- The
dev
configuration reads this from ameasurement_spec_config.textproto
file in theconfig-files
K8s ConfigMap. See the updated deployment guide.
- The
What's Changed
- Fix image-tag workflow output. by @SanjayVas in #1086
- Use batch DDL for update-vid-model-foreign-keys-and-indexes.sql. by @SanjayVas in #1087
- Adding Hello World React app to the reporting UI by @bdomen-ggl in #1033
- Implement PostgresComputationStatsService by @YuhongWang-Amazon in #1070
- Update common-jvm to 0.59.0. by @SanjayVas in #1084
- Add script to run api-linter. by @SanjayVas in #1073
- Fix api-linter errors in system API. by @SanjayVas in #1074
- Add resource IDs to create requests by @riemanli in #1071
- Add generation of events from SimulatorSyntheticDataSpec. by @tristanvuong2021 in #1083
- Implement Event Groups Service Reporting V2Alpha. by @tristanvuong2021 in #1088
- Create AcdpParamsConverter and ACDP composition method by @iverson52000 in #1062
- Adding base test framework for react by @bdomen-ggl in #1085
- Fix that only allows creation of ModelRollouts with RolloutPeriodStartTime after that of previous ModelRollout by @jojijac0b in #1076
- Replace timeinterval with rollout deploy period by @Marco-Premier in #1091
- Update cross-media-measurement-api to 0.38.0 and common-jvm to 0.60.0. by @SanjayVas in #1092
- Partially Implement Integration Tests for Reporting V2Alpha. by @tristanvuong2021 in #1093
- Replace use of deprecated event_field message option in test templates. by @SanjayVas in #1094
- Refactor panel exchange to use Kustomize by @Marco-Premier in #1090
- Fix SyntheticDataGeneration signature. by @SanjayVas in #1101
- Add the state machine and the encryption methods proto for the reach … by @ple13 in #1096
- Kingdom supports creating ROLLv2 Measurements. by @renjiezh in #1097
- Pass EventGroup to EventQuery. by @SanjayVas in #1095
- Fix CEL syntax for Reporting filter conjunctions. by @SanjayVas in #1108
- Drop maxFrequency param from DistributionComparison.of. by @SanjayVas in #1110
- Rename FrontendSimulator to MeasurementConsumerSimulator. by @SanjayVas in #1111
- Add cancel subcommand to measurements in MeasurementSystem CLI. by @SanjayVas in #1113
- Migrate duchy postgres package to deploy/common by @YuhongWang-Amazon in #1103
- Add Deployment Images for Reporting V2 by @tristanvuong2021 in #1099
- Add postgres computation readers by @YuhongWang-Amazon in #1104
- Cancel previous in-progress workflow runs on the same branch. by @SanjayVas in #1117
- Fix K8s dev config error of missing rollv2 textproto. by @renjiezh in #1119
- Don't bother trimming Bazel cache if it's not being saved. by @SanjayVas in #1124
- Add k8s files for reporting v2 deployment by @tristanvuong2021 in #1118
- Add reporting v2 cli tool. by @tristanvuong2021 in #1116
- Update EDP simulator to stop writing plaintext sketches to storage. by @SanjayVas in #1112
- Add postgres computation writers by @YuhongWang-Amazon in #1105
- Create PrivacyBucketAcdpCharges table schema for ACDP composition by @iverson52000 in #1121
- Make negation normal form apply the propagated not statements in the leaf nodes by @uakyol in #1133
- Allow SyntheticGenerator to be used for correctness test on GKE. by @SanjayVas in #1127
- Use SyntheticGenerator instead of BigQuery for K8s test workflow. by @SanjayVas in #1128
- Move UI src and test to experimental folder by @bdomen-ggl in #1139
- adding UI assets by @bdomen-ggl in #1141
- Update cross-media-measurement-api version by @Marco-Premier in #1123
- Duchy Api and System Api support Reach-Only Llv2 protocol. by @renjiezh in #1125
- Read synthetic generator specs from files. by @SanjayVas in #1142
- Drop some metric labels for GCP by @tristanvuong2021 in #1140
- Update PBM InMemoryBackingStore and PostgresBackingStore for ACDP com… by @iverson52000 in #1137
- Add support for protobuf Duration and Timestamp fields in synthetic generator. by @SanjayVas in #1145
- Herald supports reach-only llv2 protocol. by @renjiezh in #1135
- Add postgres computations service by @YuhongWang-Amazon in #1136
- Regenerate expired Reporting TLS cert. by @SanjayVas in #1149
- Implementation of the initialization, setup, and execution phase of the reach-only protocol. by @ple13 in #1129
- Update PrivacyBudgetLedger for ACDP composition by @iverson52000 in #1148
- Add a larger-scale SyntheticEventGroupSpec. by @SanjayVas in #1144
- Remove non-template fields from our test templates. by @SanjayVas in #1150
- Update PrivacyBudgetManager for ACDP composition by @iverson52000 in #1157
- Add in process postgres duchy correctness tests by @YuhongWang-Amazon in #1152
- EdpSimulator fulfills Reach-Only LiquidLegionsV2 requisitions. by @renjiezh in #1146
- Parallelizing the functions that add blinded histogram noise. by @ple13 in #1165
- Update PrivacyQueryMapper for ACDP composition by @iverson52000 in #1163
- Update consent-signaling-client to 0.16.0. by @SanjayVas in #1171
- Compare expected RF results against actual using synthetic data for Reporting V2 integration tests by @tristanvuong2021 in #1147
- Update common-jvm for CommandLineTesting. by @SanjayVas in #1153
- Bdomen react api utils by @bdomen-ggl in #1143
- Update common-jvm for Postgres testing. by @SanjayVas in #1169
- Adding the class ReachOnlyLiquidLegionsV2Mill on the Duchy mill. by @ple13 in #1155
- Update local duchy worker2 to use postgres by @YuhongWang-Amazon in #1160
- Avoid unnecessary joins when updating Requisitions to UNFULFILLED. by @SanjayVas in #1122
- Eliminate compiler warnings in Reach-Only LLV2. by @renjiezh in #1173
- update initial support for population measurement type by @stevenwarejones in #1159
- EDP simulator supports ACDP composition by @iverson52000 in #1170
- Add CLI tool for validating event templates. by @SanjayVas in #1156
- Update GKE Kingdom deployment guide to reference Terraform example. by @SanjayVas in #1161
- Fix gke dev duchy k8s configs by @YuhongWang-Amazon in #1181
- Move various symbols related to integration testing to the correct packages. by @SanjayVas in #1175
- Fix Reporting V1 RF Result by @tristanvuong2021 in #1182
- Add filter to list reports reporting v2 by @tristanvuong2021 in #1177
- Add Reporting V2 Event Group Metadata Descriptors Service. by @tristanvuong2021 in #1162
- Update GKE Duchy deployment guide to reference Terraform example. by @SanjayVas in #1172
- Add data providers service to Reporting V2 by @tristanvuong2021 in #1176
- Stop defaulting imagePullPolicy on K8s Jobs to
always
. by @SanjayVas in #1190 - Measurement statistics calculation by @riemanli in #1080
- Adding List Reports view by @bdomen-ggl in #1174
- Update Benchmark and MeasurementSystem with population results by @stevenwarejones in #1164
- Attempt to address flakiness in integration tests. by @SanjayVas in #1189
- Create Gaussian noise and ACDP composition rollout doc by @iverson52000 in #1185
- Add deserialize subcommand to EncryptionPublicKeys tool. by @SanjayVas in #1191
- Integration test for reach-only llv2 protocol by @renjiezh in #1180
- Circulate direct computation methodology from protocol config to measurement result. by @riemanli in #1154
- Add a guide for updating to a new Halo release. by @SanjayVas in #1196
- Hardcode EventGroup IDs that EDP simulators will fail Requisitions on by @tristanvuong2021 in #1195
- Adding show report skeleton to UI by @bdomen-ggl in #1198
- Store set operation compilation IDs by @riemanli in #1100
- Update cross-media-measurement-api dep for maximum frequency specification change. by @SanjayVas in #1183
- Fix broken jest tests. by @bdomen-ggl in #1204
- Update cross-media-measurement-api for max watch duration. by @SanjayVas in #1192
- Add support for assuming a role when creating an S3 Storage Client by @jonmolle in #1201
- Enable Gaussian noise and ACDP composition by @iverson52000 in #1187
- adding header, cards, and table to Report view by @bdomen-ggl in #1203
- Fix internal UpdatePublicKey to write PublicKeyCertificateId. by @SanjayVas in #1210
- add role_session_name to s3 role by @stevenwarejones in #1213
- Use CMMS testing certs for dev panel match. by @SanjayVas in #1207
- Fix internal GetDataProvider possibly returning incorrect Certificate. by @SanjayVas in #1216
- Simplify Certificate key parsing and auth using sealed interfaces. by @SanjayVas in #1211
- Update cross-media-measurement-api dep for ListCertificates. by @SanjayVas in #1219
- Use participant count instead of gRPC stubs count in LLv2 mills. by @SanjayVas in #1197
- Update kingdom deployment doc for rollv2 protocol config. by @renjiezh in #1178
- Update integration tests to verify direct measurement results by @riemanli in #1212
- Updated measurement retention job to have a flag to control maximum number of measurement to delete by @kungfucraig in #1208
- adding charts to reporting ui by @bdomen-ggl in #1218
- Handle Duchy gRPC errors appropriately w.r.t. retryability. by @SanjayVas in #1224
- Implement AWS duchy runners by @YuhongWang-Amazon in #1193
- Add config for params for Reporting V1. by @tristanvuong2021 in #1202
- Fix internal GetMeasurementConsumer possibly returning incorrect Certificate. by @SanjayVas in #1230
- Wait for build to complete on main branch before running on PR. by @SanjayVas in #1205
- Update common-jvm to 0.65.0. by @SanjayVas in #1225
- Enable reach-only LLv2 protocol in local and dev configurations. by @SanjayVas in #1221
- Fix Reporting workflow to include measurement spec config. by @SanjayVas in #1236
- Incorporate custom direct methodology by @riemanli in #1226
New Contributors
- @ple13 made their first contribution in #1096
- @kungfucraig made their first contribution in #1208
Full Changelog: v0.3.0...v0.4.0