Skip to content

Commit

Permalink
Add stdf tamer example (#43)
Browse files Browse the repository at this point in the history
- added dependency to use stdf-tamer in tests
- added example for using struct from pypi package (stdf-tamer)
- improved wording in readme
- cleaned white spaces
- added lines to release notes
  • Loading branch information
franzhaas authored Dec 13, 2024
1 parent d941a3f commit 346ea98
Show file tree
Hide file tree
Showing 17 changed files with 214 additions and 67 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,11 @@ jobs:
- name: Install xonsh
run: |
uv tool install xonsh
- name: Run basic tests
run: |
uvx xonsh tasks/baseQC.xsh
- name: Upload Robot Framework log
if: always()
uses: actions/upload-artifact@v3
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/run_radon.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,7 @@ jobs:
- name: Install xonsh
run: |
uv tool install xonsh
- name: Run ruff
run: |
uvx xonsh tasks/run_radon.xsh
3 changes: 1 addition & 2 deletions .github/workflows/run_ruff.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,7 @@ jobs:
- name: Install xonsh
run: |
uv tool install xonsh
- name: Run ruff
run: |
uvx xonsh tasks/run_ruff.xsh
33 changes: 17 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,22 @@
![Build](https://github.com/MarketSquare/robotframework-construct/actions/workflows/main.yml/badge.svg)
![Mutation Testing](https://github.com/MarketSquare/robotframework-construct/actions/workflows/mutations.yml/badge.svg)
![Breakout to C++ example](https://github.com/MarketSquare/robotframework-construct/actions/workflows/pythonBreakout.yml/badge.svg)
![radon maintainabity check](https://github.com/MarketSquare/robotframework-construct/actions/workflows/run_radon.yml/badge.svg)
![radon maintainability check](https://github.com/MarketSquare/robotframework-construct/actions/workflows/run_radon.yml/badge.svg)
![ruff check](https://github.com/MarketSquare/robotframework-construct/actions/workflows/run_ruff.yml/badge.svg)

# robotframework-construct

## I am in a hurry, let's jump-start with an example!
[git](https://github.com/astral-sh/uv) and [uv](https://github.com/astral-sh/uv) need to be preinstalled to run the examples.
[git](https://git-scm.com/) (a version control system) and [uv](https://github.com/astral-sh/uv) (a tool for managing Python virtual environments) need to be preinstalled to run the examples.

```bash
git clone https://github.com/MarketSquare/robotframework-construct.git
cd robotframework-construct
uv sync --extra test --dev
uv run xonsh tasks/baseQC.xsh
uv run xonsh tasks/baseQC.xsh
```

There are examples which require hardware to work. [USB HID](./atests/HIDKeyboard/) and [nfc/nci](./atests/nfc_nci/). For the nci example, STm hardware and firmware is required. For the HID example, a USB Keyboard on a linux machine is sufficient.
Some examples, such as [USB HID](./atests/HIDKeyboard/) and [nfc/nci](./atests/nfc_nci/), require specific hardware to function. For the nci example, STm hardware and firmware is required. For the HID example, a USB Keyboard on a linux machine is sufficient.

## What is robotframework-construct?
robotframework-construct is a [Robot Framework](https://robotframework.org) keyword library powered by [construct](https://construct.readthedocs.io/en/latest/).
Expand All @@ -28,7 +28,7 @@ Aiming for :rocket: speed, :white_check_mark: reliability, and :microscope: visi

Your binary data becomes as accessible as numbers and strings are in Robot Framework.

Checkout the documentation at [robotframework-construct](https://marketsquare.github.io/robotframework-construct/)
Check out the documentation [here](https://marketsquare.github.io/robotframework-construct/)
### Licensing
robotframework-construct is an opensource keyword library licensed under Apache-2.0, leveraging the [construct](https://construct.readthedocs.io/en/latest/) library licensed under MIT.

Expand Down Expand Up @@ -82,40 +82,41 @@ _bson_element = Struct(
})
)
_e_list = GreedyRange(_bson_element)

def _calc_size(this):
return len(_e_list.build(this["elements"]))+5

document = Struct(
document = Struct(
"size" / Rebuild(Int32sl, _calc_size),
"elements" / _e_list,
"EOO" / Const(b"\x00")
)
```

This can be readily and directly derived from the bson specification [bson specification](https://bsonspec.org/spec.html). AI can help do that, efficiently. This is because the mapping between the specification and the declaration is immediate making it easy to supervise the process and verify the result.
This can be readily and directly derived from the [bson specification](https://bsonspec.org/spec.html). AI can assist in this process efficiently. This is because the mapping between the specification and the declaration is a very direct and straightforward task, making it easy to supervise the process and verify the result.

Using AI to generate a parser+generator would result in a larger volume of code to be verified, and the verification is harder.

### Checking and modifying binary data
There are keywords with embedded parameters allowing checking and modifying binary data in a robotframework way
There are keywords with embedded parameters that allow checking and modifying binary data in a Robot Framework way

A checking example
![image](https://github.com/user-attachments/assets/9d01b19d-480a-4393-9cca-1060f3e54712)

and a modifying example
![image](https://github.com/user-attachments/assets/55de01cf-09b5-4ad7-ab46-02aa718dc8db)

note, that this is very natural in the robotframework environment. If multiple elements need to be checked, this needs to be organized in keywords.
**Note:** This is very natural in the Robot Framework environment. If multiple elements need to be checked, these checks should be organized in keywords.

### Observing the binary data
The binary data built and parsed is easily accessible. This helps with trust issues and makes it easier
to read digital analyzer outputs or oscilloscope screens. Also, a name to identify what definition is doing the parsing/generating is provided.
The built and parsed binary data is easily accessible. This helps with trust issues and makes it easier to read digital analyzer or oscilloscope screens. Also, a name to identify what definition is doing the parsing/generating may be provided.

A building example.:
A building example:

![image](https://github.com/user-attachments/assets/9ad060cc-54cd-487e-9cb6-e0798aa53702)


A parsing example.:
A parsing example:

![image](https://github.com/user-attachments/assets/041852dc-ff40-4ade-9d3c-0999c5057cd1)

Expand All @@ -125,11 +126,11 @@ The Structs can be transformed into kaitai. Kaitai is a DSL that can be transfor

Keep in mind that some limitations apply to these transformations.

For reference.: [./tasks/breakoutCpp.xsh] which creates a C++ parser
For reference: [./tasks/breakoutCpp.xsh], which is a script that demonstrates how to transform Construct declarations into a C++ parser using the Kaitai DSL.

## Relationships in the Ecosystem

The number of dependencies is kept low, with no transient dependencies.
The number of dependencies is kept low, with no transitive dependencies.

This is important as it keeps coordination feasible. Construct is well-developed and not expected to change significantly soon. Robot Framework releases major updates annually, but these are well-managed and communicated.

Expand Down Expand Up @@ -177,7 +178,7 @@ Tested examples and acceptance tests using Robot Framework are provided. Unit te

### Mutation Testing

Since this project consists primarily of interface code, it is crucial to catch user errors and produce clear error messages. Mutation testing ensures that all code paths and error messages are tested, supporting efforts to make errors informative.
Since this project primarily consists of interface code, it is crucial to catch user errors and produce clear error messages. Mutation testing ensures that all code paths and error messages are tested, supporting efforts to make errors informative.

## Project To-Do List

Expand Down
4 changes: 2 additions & 2 deletions atests/HIDKeyboard/hid_keyboard.robot
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
*** Settings ***
Documentation This is a simple example for a robot file using robotframework-construct demonstrating accessing real HW over the USB HID interface.
...
... This currently only tested on linux
...
... This currently only tested on linux
Library robotframework_construct
Library Dialogs
Expand Down
2 changes: 1 addition & 1 deletion atests/bson/bson.robot
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ simple bson example using a in memory object including modification
Element 'elements.1.value' in '${returnedDict}' should not be equal to '2'
Element 'elements.1.value' in '${returnedDict}' should not be equal to '4'

simple bson example changint the seperator
simple bson example changint the seperator
${returnedDict}= Parse '${blob}' using construct 'bson_document'
Set element seperator to '->'
Element 'elements->1->value' in '${returnedDict}' should be equal to '1'
Expand Down
6 changes: 3 additions & 3 deletions atests/bson/errormessage_checks.robot
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
*** Settings ***
Documentation This are tests checking for sensible error messages.
...
... This is usefull for quality controll and for trouble shooting. These examples might help to understand the error messages.
Documentation This are tests checking for sensible error messages.
...
... This is usefull for quality controll and for trouble shooting. These examples might help to understand the error messages.
Library bson
Library robotframework_construct
Test Tags mutation_base
Expand Down
2 changes: 1 addition & 1 deletion atests/dns/dns_reflector.robot
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ basic dns request udp
${record1}= Parse '${connection2}' using construct '${dns_payload_udp}'
Write binary data generated from '${exampleRequestRoboconUdp}' using construct '${dns_payload_udp}' to '${connection2}'
${record2}= Parse '${connection1}' using construct '${dns_payload_udp}'

verify that double reflection is not possible with TCP
${PORT1} ${PORT2}= Reflect traffic between ports using 'TCP'
${connection1}= Open TCP connection to server '127.0.0.1' on port '${PORT1}'
Expand Down
40 changes: 20 additions & 20 deletions atests/nfc_nci/nci.py
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@
NCI_RF_CONFIGURATION_RESET=1) # Table 7

NCI_VERSION = Enum(Byte, NCI_VERSION_1_0=0x10,
NCI_VERSION_2_0=0x20) # Table 6
NCI_VERSION_2_0=0x20) # Table 6

CORE_RESET_NTF_STATUS_REASON_CODE = Enum(Byte, UNSPECIFIED=0x00,
CORE_RESET_TRIGGERED=0x01)
Expand All @@ -160,15 +160,15 @@

NFCParameterStruct = Struct(
"SENS_RES_Response" / Bytes(2), # 2 Octets, Defined in [DIGITAL]

"NFCID1_Length" / Enum(
Int8ub,
NOT_AVAILABLE=0x00, # If no NFCID1 parameter is available
LENGTH_4=0x04, # Valid length 4 octets
LENGTH_7=0x07, # Valid length 7 octets
LENGTH_10=0x0A, # Valid length 10 octets
),

"NFCID1" / Switch(
this.NFCID1_Length,
{
Expand All @@ -185,7 +185,7 @@
LENGTH_1=0x01, # Valid length 1 octet
RFU=0xFF # Reserved for Future Use
),

"SEL_RES_Response" / Switch(
this.SEL_RES_Response_Length,
{
Expand All @@ -195,14 +195,14 @@
)
)

NFCC_FEATURES = Struct("Octet_0" / BitStruct("RFU" / BitsInteger(5),
"DiscoverConfigurationmode" / Enum(BitsInteger(2),
NFCC_FEATURES = Struct("Octet_0" / BitStruct("RFU" / BitsInteger(5),
"DiscoverConfigurationmode" / Enum(BitsInteger(2),
SINGLDE_DH=0,
MULTI_NFCEEs=1),
"DiscoverFrequencyConfiguration" / Enum(BitsInteger(1),
IGNORED=0,
RF_DISCOVER_CMD_SUPPORTED=1)),
"Octet_1" / BitStruct("RFU" / BitsInteger(4),
"Octet_1" / BitStruct("RFU" / BitsInteger(4),
"AID_BASED_ROUTING" / BitsInteger(1),
"PROTOCOL_BASED_ROUTING" / BitsInteger(1),
"TECHNOLOGY_BASED_ROUTING" / BitsInteger(1),
Expand Down Expand Up @@ -245,7 +245,7 @@
"MT" / MT,
"PBF" / PBF,
"ConnID" / BitsInteger(4),
"RFU" / Byte,
"RFU" / Byte,
"PayloadLength" / Byte
),
"payload_length" / Int8ub,
Expand Down Expand Up @@ -274,19 +274,19 @@


NFC_RST_CMD= {"header": {"MT": MT.ControlPacketCommand,
"PBF": 0,
"PBF": 0,
"GID": GID.CORE,
"RFU": 0,
"OID": OID_NCI_Core.CORE_RESET},
"RFU": 0,
"OID": OID_NCI_Core.CORE_RESET},
"payload_length": 1,
"payload": {"ResetType": CORE_RESET_CMD.RESET_CONFIGURATION},
"padding": b""}

NFC_INIT_CMD= {"header": {"MT": MT.ControlPacketCommand,
"PBF": 0,
"PBF": 0,
"GID": GID.CORE,
"RFU": 0,
"OID": OID_NCI_Core.CORE_INIT},
"RFU": 0,
"OID": OID_NCI_Core.CORE_INIT},
"payload_length": 2,
"payload": {"ConstValue": b"\x00\x00"},
"padding": b""}
Expand All @@ -300,16 +300,16 @@
"OID": OID_RF_Management.RF_DISCOVER},
"payload_length": 13,
"payload": {"NumInterfaces": 6,
"RF_Discover_Map": [{"RF_Technology_and_Mode":RF_Technology.NFC_A_PASSIVE_POLL_MODE,
"RF_Discover_Map": [{"RF_Technology_and_Mode":RF_Technology.NFC_A_PASSIVE_POLL_MODE,
"RF_Discover_Frequency": 2},
{"RF_Technology_and_Mode":RF_Technology.NFC_B_PASSIVE_POLL_MODE,
{"RF_Technology_and_Mode":RF_Technology.NFC_B_PASSIVE_POLL_MODE,
"RF_Discover_Frequency": 2},
{"RF_Technology_and_Mode":RF_Technology.NFC_F_PASSIVE_POLL_MODE,
{"RF_Technology_and_Mode":RF_Technology.NFC_F_PASSIVE_POLL_MODE,
"RF_Discover_Frequency": 2},
{"RF_Technology_and_Mode":RF_Technology.NFC_15693_PASSIVE_POLL_MODE,
{"RF_Technology_and_Mode":RF_Technology.NFC_15693_PASSIVE_POLL_MODE,
"RF_Discover_Frequency": 2},
{"RF_Technology_and_Mode":RF_Technology.NFC_A_PASSIVE_LISTEN_MODE,
{"RF_Technology_and_Mode":RF_Technology.NFC_A_PASSIVE_LISTEN_MODE,
"RF_Discover_Frequency": 2},
{"RF_Technology_and_Mode":RF_Technology.NFC_F_PASSIVE_LISTEN_MODE,
{"RF_Technology_and_Mode":RF_Technology.NFC_F_PASSIVE_LISTEN_MODE,
"RF_Discover_Frequency": 2},
]}}
6 changes: 3 additions & 3 deletions atests/nfc_nci/nci_interface.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,13 @@ def open_nci_connection_via_uart(self, device: str, baudrate: int):
"""
self._serial_connection = serial.Serial(device, baudrate, timeout=1)
self._serial_connection.reset_input_buffer()
return (open(self._serial_connection.fd, "rb", buffering=0, closefd=False),
return (open(self._serial_connection.fd, "rb", buffering=0, closefd=False),
open(self._serial_connection.fd, "wb", buffering=0, closefd=False),)

def wait_for_data_from_nci(self, timeout: float = 1.0):
"""
Waits for data from the NCI device.
Raises an exception if the NCI connection is not open.
"""
if self._serial_connection and self._serial_connection.is_open:
Expand Down
2 changes: 1 addition & 1 deletion atests/nfc_nci/nfc_basic.robot
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ Actively poll for A cards
Wait For Data From NCI timeout=1
${RESPONSE}= Parse '${NCI_READ}' Using Construct '${NCIControlPacket}'
Wait For Data From NCI timeout=1
${RESPONSE}= Parse '${NCI_READ}' Using Construct '${NCIControlPacket}'
${RESPONSE}= Parse '${NCI_READ}' Using Construct '${NCIControlPacket}'

*** Keywords ***
Receive message from NCI from ${NCI_READ}
Expand Down
6 changes: 3 additions & 3 deletions atests/regmapmockup/errormessage_checks.robot
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
*** Settings ***
Documentation This are tests checking for sensible error messages.
...
... This is usefull for quality controll and for trouble shooting. These examples might help to understand the error messages.
Documentation This are tests checking for sensible error messages.
...
... This is usefull for quality controll and for trouble shooting. These examples might help to understand the error messages.
Library robotframework_construct
Test Setup prepare regmaps
Test Tags mutation_regmap
Expand Down
Loading

0 comments on commit 346ea98

Please sign in to comment.