Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Implement TC acknowledgement test cases #37015

Open
wants to merge 25 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
2182924
test: Implement TC acknowledgement test cases 2.5-2.8
swan-amazon Jan 9, 2025
0bf6099
swan-amazon Jan 13, 2025
98a4ec8
test: Implement remaining TC acknowledgement test cases 2.9-2.11
swan-amazon Jan 13, 2025
87deee8
swan-amazon Jan 13, 2025
e9b30ed
swan-amazon Jan 13, 2025
3046d6a
wip
swan-amazon Jan 13, 2025
b1508b1
Merge remote-tracking branch 'origin/master' into pr/swan-amazon/37015
swan-amazon Jan 15, 2025
5c8d3ad
swan-amazon Jan 15, 2025
28565c1
swan-amazon Jan 15, 2025
451e82c
feat(testing): Add PICS guard for Terms & Conditions test steps
swan-amazon Jan 21, 2025
ed7f70f
Added PICS and PIXIT definitions to src/app/tests/suites/certificatio…
swan-amazon Jan 22, 2025
9c51e2b
swan-amazon Jan 22, 2025
3dcbed8
Merge remote-tracking branch 'origin/master' into pr/swan-amazon/37015
swan-amazon Jan 22, 2025
a7af4a5
swan-amazon Jan 22, 2025
be17d16
refactor: Extract commission_devices functionality into reusable methods
swan-amazon Jan 23, 2025
8abdba3
swan-amazon Jan 23, 2025
2ab69be
swan-amazon Jan 23, 2025
d50e50b
Merge branch 'master' into testing/tc-test-plan-scripts-1
swan-amazon Jan 23, 2025
009e813
swan-amazon Jan 23, 2025
a6783f7
Merge remote-tracking branch 'origin/master' into pr/swan-amazon/37015
swan-amazon Jan 23, 2025
e7e9a92
Addressed PR comments
swan-amazon Jan 23, 2025
3a5f399
Merge remote-tracking branch 'origin/master' into pr/swan-amazon/37015
swan-amazon Jan 23, 2025
8f6c6b5
Merge remote-tracking branch 'origin/master' into pr/swan-amazon/37015
swan-amazon Jan 23, 2025
2d8722d
swan-amazon Jan 24, 2025
6cfe31e
Improve test assertions and manual testing steps
swan-amazon Feb 5, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -502,6 +502,7 @@ jobs:
--target linux-x64-fabric-bridge-rpc-ipv6only-no-ble-no-wifi-clang \
--target linux-x64-fabric-sync-ipv6only-no-ble-no-wifi-clang \
--target linux-x64-light-data-model-no-unique-id-ipv6only-no-ble-no-wifi-clang \
--target linux-x64-terms-and-conditions \
--target linux-x64-python-bindings \
build \
--copy-artifacts-to objdir-clone \
Expand All @@ -520,6 +521,7 @@ jobs:
echo "FABRIC_BRIDGE_APP: out/linux-x64-fabric-bridge-rpc-ipv6only-no-ble-no-wifi-clang/fabric-bridge-app" >> /tmp/test_env.yaml
echo "FABRIC_SYNC_APP: out/linux-x64-fabric-sync-ipv6only-no-ble-no-wifi-clang/fabric-sync" >> /tmp/test_env.yaml
echo "LIGHTING_APP_NO_UNIQUE_ID: out/linux-x64-light-data-model-no-unique-id-ipv6only-no-ble-no-wifi-clang/chip-lighting-app" >> /tmp/test_env.yaml
echo "TERMS_AND_CONDITIONS_APP: out/linux-x64-terms-and-conditions/chip-terms-and-conditions-app" >> /tmp/test_env.yaml
echo "TRACE_APP: out/trace_data/app-{SCRIPT_BASE_NAME}" >> /tmp/test_env.yaml
echo "TRACE_TEST_JSON: out/trace_data/test-{SCRIPT_BASE_NAME}" >> /tmp/test_env.yaml
echo "TRACE_TEST_PERFETTO: out/trace_data/test-{SCRIPT_BASE_NAME}" >> /tmp/test_env.yaml
Expand Down
5 changes: 5 additions & 0 deletions integrations/docker/images/chip-cert-bins/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,7 @@ RUN case ${TARGETPLATFORM} in \
--target linux-x64-fabric-admin-rpc-ipv6only \
--target linux-x64-light-data-model-no-unique-id-ipv6only \
--target linux-x64-network-manager-ipv6only \
--target linux-x64-terms-and-conditions \
build \
&& mv out/linux-x64-chip-tool-ipv6only-platform-mdns/chip-tool out/chip-tool \
&& mv out/linux-x64-shell-ipv6only-platform-mdns/chip-shell out/chip-shell \
Expand All @@ -192,6 +193,7 @@ RUN case ${TARGETPLATFORM} in \
&& mv out/linux-x64-fabric-admin-rpc-ipv6only/fabric-admin out/fabric-admin \
&& mv out/linux-x64-light-data-model-no-unique-id-ipv6only/chip-lighting-app out/chip-lighting-data-model-no-unique-id-app \
&& mv out/linux-x64-network-manager-ipv6only/matter-network-manager-app out/matter-network-manager-app \
&& mv out/linux-x64-terms-and-conditions/chip-terms-and-conditions-app out/chip-terms-and-conditions-app \
;; \
"linux/arm64")\
set -x \
Expand Down Expand Up @@ -220,6 +222,7 @@ RUN case ${TARGETPLATFORM} in \
--target linux-arm64-fabric-admin-rpc-ipv6only \
--target linux-arm64-light-data-model-no-unique-id-ipv6only \
--target linux-arm64-network-manager-ipv6only \
--target linux-arm64-terms-and-conditions \
build \
&& mv out/linux-arm64-chip-tool-ipv6only-platform-mdns/chip-tool out/chip-tool \
&& mv out/linux-arm64-shell-ipv6only-platform-mdns/chip-shell out/chip-shell \
Expand All @@ -244,6 +247,7 @@ RUN case ${TARGETPLATFORM} in \
&& mv out/linux-arm64-fabric-admin-rpc-ipv6only/fabric-admin out/fabric-admin \
&& mv out/linux-arm64-light-data-model-no-unique-id-ipv6only/chip-lighting-app out/chip-lighting-data-model-no-unique-id-app \
&& mv out/linux-arm64-network-manager-ipv6only/matter-network-manager-app out/matter-network-manager-app \
&& mv out/linux-arm64-terms-and-conditions/chip-terms-and-conditions-app out/chip-terms-and-conditions-app \
;; \
*) ;; \
esac
Expand Down Expand Up @@ -283,6 +287,7 @@ COPY --from=chip-build-cert-bins /root/connectedhomeip/out/fabric-bridge-app app
COPY --from=chip-build-cert-bins /root/connectedhomeip/out/fabric-admin apps/fabric-admin
COPY --from=chip-build-cert-bins /root/connectedhomeip/out/chip-lighting-data-model-no-unique-id-app apps/chip-lighting-data-model-no-unique-id-app
COPY --from=chip-build-cert-bins /root/connectedhomeip/out/matter-network-manager-app apps/matter-network-manager-app
COPY --from=chip-build-cert-bins /root/connectedhomeip/out/chip-terms-and-conditions-app apps/chip-terms-and-conditions-app

# Create symbolic links for now since this allows users to use existing configurations
# for running just `app-name` instead of `apps/app-name`
Expand Down
4 changes: 2 additions & 2 deletions scripts/build/builders/host.py
Original file line number Diff line number Diff line change
Expand Up @@ -282,8 +282,8 @@ def OutputNames(self):
yield 'water-leak-detector-app'
yield 'water-leak-detector-app.map'
elif self == HostApp.TERMS_AND_CONDITIONS:
yield 'terms-and-conditions-app'
yield 'terms-and-conditions-app.map'
yield 'chip-terms-and-conditions-app'
yield 'chip-terms-and-conditions-app.map'
else:
raise Exception('Unknown app type: %r' % self)

Expand Down
1 change: 1 addition & 0 deletions scripts/tests/local.py
Original file line number Diff line number Diff line change
Expand Up @@ -387,6 +387,7 @@ def as_runner(path):
FABRIC_SYNC_APP: {
as_runner(f'out/{target_prefix}-fabric-sync-no-ble-no-wifi-ipv6only-clang-boringssl/fabric-sync')}
LIGHTING_APP_NO_UNIQUE_ID: {as_runner(f'out/{target_prefix}-light-data-model-no-unique-id-ipv6only-no-ble-no-wifi-clang/chip-lighting-app')}
TERMS_AND_CONDITIONS_APP: {as_runner(f'out/{target_prefix}-terms-and-conditions/chip-terms-and-conditions-app')}
TRACE_APP: out/trace_data/app-{{SCRIPT_BASE_NAME}}
TRACE_TEST_JSON: out/trace_data/test-{{SCRIPT_BASE_NAME}}
TRACE_TEST_PERFETTO: out/trace_data/test-{{SCRIPT_BASE_NAME}}
Expand Down
17 changes: 17 additions & 0 deletions src/app/tests/suites/certification/PICS.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3632,6 +3632,23 @@ PICS:
CommissioningCompleteResponse command?"
id: CGEN.S.C05.Tx

#
# server / features
#
- label:
"Does the device implement the General Commissioning cluster's terms
and conditions feature?"
id: CGEN.S.F00(TC)

- label: "The device's failsafe expiration limit."
id: PIXIT.CGEN.FailsafeExpiryLengthSeconds

- label: "The device's required terms and conditions acknowledgements."
id: PIXIT.CGEN.RequiredTCAcknowledgements

- label: "The device's required minimum terms and conditions revision."
id: PIXIT.CGEN.TCRevision

# General Diagnostics Cluster Test Plan
- label:
"Does the device implement the General Diagnostics cluster as a
Expand Down
9 changes: 8 additions & 1 deletion src/app/tests/suites/certification/ci-pics-values
Original file line number Diff line number Diff line change
Expand Up @@ -729,6 +729,7 @@ SWTCH.C.AO-READ=0
SWTCH.C.AO-WRITE=0

# General Commissioning Cluster
CGEN.C=1
CGEN.S=1
CGEN.S.A0000=1
CGEN.S.A0001=1
Expand All @@ -742,7 +743,13 @@ CGEN.S.C03.Tx=1
CGEN.S.C04.Rsp=1
CGEN.S.C05.Tx=1

CGEN.C=1
#Feature
CGEN.S.F00(TC)=1

#PIXIT
PIXIT.CGEN.FailsafeExpiryLengthSeconds=0
PIXIT.CGEN.RequiredTCAcknowledgements=1
PIXIT.CGEN.TCRevision=1

# LAUNDRY WASHER MODE CLUSTER
LWM.S=1
Expand Down
154 changes: 154 additions & 0 deletions src/python_testing/TC_CGEN_2_10.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
#
# Copyright (c) 2025 Project CHIP Authors
# All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# === BEGIN CI TEST ARGUMENTS ===
# test-runner-runs:
# run1:
# app: ${TERMS_AND_CONDITIONS_APP}
# app-args: >
# --tc-min-required-version 1
# --tc-required-acknowledgements 1
# --custom-flow 2
# --capabilities 6
# script-args:
# --PICS src/app/tests/suites/certification/ci-pics-values
# --in-test-commissioning-method on-network
# --tc-version-to-simulate 1
# --tc-user-response-to-simulate 1
# --qr-code MT:-24J0AFN00KA0648G00
# --trace-to json:log
# factoryreset: True
# quiet: True
# === END CI TEST ARGUMENTS ===

import chip.clusters as Clusters
from chip import ChipDeviceCtrl
from chip.commissioning import ROOT_ENDPOINT_ID
from chip.testing.matter_testing import MatterBaseTest, TestStep, async_test_body, default_matter_test_main
from mobly import asserts


class TC_CGEN_2_10(MatterBaseTest):
def desc_TC_CGEN_2_10(self) -> str:
return "[TC-CGEN-2.10] Verification that required terms can't be unset from TCAcknowledgements with SetTCAcknowledgements [DUT as Server]"

def pics_TC_CGEN_2_10(self) -> list[str]:
""" This function returns a list of PICS for this test case that must be True for the test to be run"""
return ["CGEN.S", "CGEN.S.F00(TC)"]

def steps_TC_CGEN_2_10(self) -> list[TestStep]:
return [
TestStep(0, description="", expectation="", is_commissioning=False),
TestStep(1, "TH reads from the DUT the attribute TCAcceptedVersion. Store the value as acceptedVersion."),
TestStep(2, "TH reads from the DUT the attribute TCAcknowledgements. Store the value as userAcknowledgements."),
TestStep(3, "TH Sends the SetTCAcknowledgements command to the DUT with the fields set as follows:\n* TCVersion: 0\n* TCUserResponse: 65535"),
TestStep(4, "TH reads from the DUT the attribute TCAcceptedVersion."),
TestStep(5, "TH reads from the DUT the attribute TCAcknowledgements."),
TestStep(6, "TH Sends the SetTCAcknowledgements command to the DUT with the fields set as follows:\n* TCVersion: acceptedVersion + 1\n* TCUserResponse: 0"),
TestStep(7, "TH reads from the DUT the attribute TCAcceptedVersion."),
TestStep(8, "TH reads from the DUT the attribute TCAcknowledgements."),
]

@async_test_body
async def test_TC_CGEN_2_10(self):
commissioner: ChipDeviceCtrl.ChipDeviceController = self.default_controller

self.step(0)
if not self.pics_guard(self.check_pics("CGEN.S.F00(TC)")):
self.skip_all_remaining_steps(1)
return

# Step 1: Begin commissioning with PASE and failsafe
commissioner.SetSkipCommissioningComplete(True)
self.matter_test_config.commissioning_method = self.matter_test_config.in_test_commissioning_method
self.matter_test_config.tc_version_to_simulate = None
self.matter_test_config.tc_user_response_to_simulate = None
await self.commission_devices()

# Step 1: Read TCAcceptedVersion
self.step(1)
response = await commissioner.ReadAttribute(nodeid=self.dut_node_id, attributes=[(ROOT_ENDPOINT_ID, Clusters.GeneralCommissioning.Attributes.TCAcceptedVersion)])
accepted_version = response[ROOT_ENDPOINT_ID][Clusters.GeneralCommissioning][Clusters.GeneralCommissioning.Attributes.TCAcceptedVersion]

# Step 2: Read TCAcknowledgements
self.step(2)
response = await commissioner.ReadAttribute(nodeid=self.dut_node_id, attributes=[(ROOT_ENDPOINT_ID, Clusters.GeneralCommissioning.Attributes.TCAcknowledgements)])
user_acknowledgements = response[ROOT_ENDPOINT_ID][Clusters.GeneralCommissioning][Clusters.GeneralCommissioning.Attributes.TCAcknowledgements]

# Step 3: Send SetTCAcknowledgements with invalid version
self.step(3)
response = await commissioner.SendCommand(
nodeid=self.dut_node_id,
endpoint=ROOT_ENDPOINT_ID,
payload=Clusters.GeneralCommissioning.Commands.SetTCAcknowledgements(TCVersion=0, TCUserResponse=65535),
)

# Verify TCMinVersionNotMet error
asserts.assert_equal(
response.errorCode,
Clusters.GeneralCommissioning.Enums.CommissioningErrorEnum.kTCMinVersionNotMet,
"Expected TCMinVersionNotMet error",
)

# Step 4: Verify TCAcceptedVersion unchanged
self.step(4)
response = await commissioner.ReadAttribute(nodeid=self.dut_node_id, attributes=[(ROOT_ENDPOINT_ID, Clusters.GeneralCommissioning.Attributes.TCAcceptedVersion)])
current_version = response[ROOT_ENDPOINT_ID][Clusters.GeneralCommissioning][Clusters.GeneralCommissioning.Attributes.TCAcceptedVersion]
asserts.assert_equal(current_version, accepted_version, "TCAcceptedVersion changed unexpectedly")

# Step 5: Verify TCAcknowledgements unchanged
self.step(5)
response = await commissioner.ReadAttribute(nodeid=self.dut_node_id, attributes=[(ROOT_ENDPOINT_ID, Clusters.GeneralCommissioning.Attributes.TCAcknowledgements)])
current_acknowledgements = response[ROOT_ENDPOINT_ID][Clusters.GeneralCommissioning][Clusters.GeneralCommissioning.Attributes.TCAcknowledgements]
asserts.assert_equal(current_acknowledgements, user_acknowledgements, "TCAcknowledgements changed unexpectedly")

# Step 6: Send SetTCAcknowledgements with invalid response
self.step(6)
response = await commissioner.SendCommand(
nodeid=self.dut_node_id,
endpoint=ROOT_ENDPOINT_ID,
payload=Clusters.GeneralCommissioning.Commands.SetTCAcknowledgements(
TCVersion=accepted_version + 1, TCUserResponse=0
),
)

# Verify RequiredTCNotAccepted error
asserts.assert_equal(
response.errorCode,
Clusters.GeneralCommissioning.Enums.CommissioningErrorEnum.kRequiredTCNotAccepted,
"Expected RequiredTCNotAccepted error",
)

# Step 7: Verify TCAcceptedVersion still unchanged
self.step(7)
response = await commissioner.ReadAttribute(nodeid=self.dut_node_id, attributes=[(ROOT_ENDPOINT_ID, Clusters.GeneralCommissioning.Attributes.TCAcceptedVersion)])
current_version = response[ROOT_ENDPOINT_ID][Clusters.GeneralCommissioning][Clusters.GeneralCommissioning.Attributes.TCAcceptedVersion]
asserts.assert_equal(current_version, accepted_version, "TCAcceptedVersion changed unexpectedly after second attempt")

# Step 8: Verify TCAcknowledgements still unchanged
self.step(8)
response = await commissioner.ReadAttribute(nodeid=self.dut_node_id, attributes=[(ROOT_ENDPOINT_ID, Clusters.GeneralCommissioning.Attributes.TCAcknowledgements)])
current_acknowledgements = response[ROOT_ENDPOINT_ID][Clusters.GeneralCommissioning][Clusters.GeneralCommissioning.Attributes.TCAcknowledgements]
asserts.assert_equal(
current_acknowledgements,
user_acknowledgements,
"TCAcknowledgements changed unexpectedly after second attempt",
)


if __name__ == "__main__":
default_matter_test_main()
Loading
Loading