diff --git a/.dockerignore b/.dockerignore index 0b3b7f7c..5364fdad 100644 --- a/.dockerignore +++ b/.dockerignore @@ -2,8 +2,13 @@ * # Except this... -!ifex -!scripts +!input_filters/ +!models/ +!output_filters/ +!packaging/ +!scripts/ +!tests/ +!transformers/ !setup.py -!tests !requirements.txt +!tox.ini diff --git a/.github/workflows/buildcheck.yml b/.github/workflows/buildcheck.yml index c1b0c7e3..ae0a79ce 100644 --- a/.github/workflows/buildcheck.yml +++ b/.github/workflows/buildcheck.yml @@ -63,8 +63,8 @@ jobs: - name: Run JSON-Schema generator run: | - python ifex/schema/ifex_to_json_schema.py >temp-schema - python ifex/schema/pretty_print_json.py temp-schema >ifex-core-idl-schema.json + python output_filters/schema/ifex_to_json_schema.py >temp-schema + python output_filters/schema/pretty_print_json.py temp-schema >ifex-core-idl-schema.json - name: Clone uservices repository run: | diff --git a/.github/workflows/containers.yml b/.github/workflows/containers.yml index 066332cd..50d75087 100644 --- a/.github/workflows/containers.yml +++ b/.github/workflows/containers.yml @@ -9,23 +9,23 @@ jobs: - name: Build containers run: | - cd docker + cd packaging/docker make build_alpine make build_ubuntu - name: Test Interactive Commands run: | - cd docker + cd packaging/docker variant=alpine make run_interactivity_test variant=ubuntu make run_interactivity_test_pyenv - name: Run unit tests in Ubuntu container run: | - cd docker + cd packaging/docker make run_ubuntu_test - name: Run unit tests in Alpine container run: | - cd docker + cd packaging/docker make run_alpine_test diff --git a/.github/workflows/create-new-release.yml b/.github/workflows/create-new-release.yml index 1f3f00b3..371a7dda 100644 --- a/.github/workflows/create-new-release.yml +++ b/.github/workflows/create-new-release.yml @@ -3,14 +3,17 @@ # Preconditions: # 1) The tag that is determined from the commit message must not already exist # 2) The workflow does not work if pushing an existing tag. -# 3) The precondition ensures this workflow is NOT run if no RELEASE-TAG field. +# 3) The precondition ensures this workflow is NOT run if a RELEASE-TAG is +# not found in the commit message. # # Action steps: -# 1) Commit and add the RELEASE-TAG in message -# 2) Tag locally with the tag -# DO NOT PUSH THE TAG +# 1) Set package version number in setup.py and add RELEASE-TAG: line to the commit message +# 2) Tag locally with the tag name +# DO NOT PUSH THE TAG! * # 3) Push commit (possibly via a pull request) -# 4) When pushed or merge to master, the release process will create the remote tag and the release +# 4) When pushed or merged to master, the release process will create the remote tag and the release +# (Note: Since we use and expect fast-forward merges, "merging" here means fast forwarding +# and therefore the local tag should be on the same commit as the one created remotely.) name: Create release tag and publish JSON Schema @@ -48,8 +51,8 @@ jobs: - name: Run JSON-Schema generator run: | - python ifex/schema/ifex_to_json_schema.py >temp-schema - python ifex/schema/pretty_print_json.py temp-schema >ifex-core-idl-schema.json + python output_filters/schema/ifex_to_json_schema.py >temp-schema + python output_filters/schema/pretty_print_json.py temp-schema >ifex-core-idl-schema.json sed -i 's/TAG-PLACEHOLDER/${{ steps.vars.outputs.TAG }}/' ifex-core-idl-schema.json - name: Create a new release diff --git a/.github/workflows/generate_docs.yml b/.github/workflows/generate_docs.yml index 5766c64c..930ea1f7 100644 --- a/.github/workflows/generate_docs.yml +++ b/.github/workflows/generate_docs.yml @@ -23,7 +23,7 @@ jobs: - name: Generate syntax document from source run: | - python ifex/model/ifex_ast_doc.py >docs/generated-syntax.md + python models/ifex/ifex_ast_doc.py >docs/generated-syntax.md - name: Join docs into one specification run: | diff --git a/.gitmodules b/.gitmodules new file mode 100644 index 00000000..0041de88 --- /dev/null +++ b/.gitmodules @@ -0,0 +1,3 @@ +[submodule "input_filters/franca/pyfranca"] + path = input_filters/franca/pyfranca + url = https://github.com/gunnarx/pyfranca diff --git a/README.md b/README.md index a13b6826..feef2248 100644 --- a/README.md +++ b/README.md @@ -36,7 +36,7 @@ or [tox.ini](./tox.ini)) ### Container use -As an alternative to installation instructions below, all the installations can also be hidden in a container. Refer to the [README in the docker/ directory](./docker/README.md) for running the tools using containers instead. +As an alternative to installation instructions below, all the installations can also be hidden in a container. Refer to the [README in the packaging/docker/ directory](./packaging/docker/README.md) for running the tools using containers instead. ## Installing and use python version(s) with `pyenv` diff --git a/docs/README2.md b/docs/README2.md index f40eb0db..1df41be0 100644 --- a/docs/README2.md +++ b/docs/README2.md @@ -6,14 +6,14 @@ The documentation, [published](https://covesa.github.io/ifex) on GitHub pages, i - The [GitHub actions workflow](/.github/workflows/generate_docs.yml) encodes the build steps. - [markup-markdown](https://github.com/hailiang-wang/markup-markdown) is used to combine several markdown files into one. Some markdown files are static and stored in [[doc-parts]], whereas some are generated from source files. The naming convention for the definition files are `*.m.md` -- [ifex_ast_doc.py](../ifex/model/ifex_ast_doc.py) generates the main syntax documentation directly from the datatypes used in the internal model AST. The language definition for the YAML based IDL represents the internal object model. In the end, both the program behavior _and the documentation of it_ is defined from the python source definition in [ifex_ast.py](../ifex/model/ifex_ast.py) +- [ifex_ast_doc.py](../models/ifex/ifex_ast_doc.py) generates the main syntax documentation directly from the datatypes used in the internal model AST. The language definition for the YAML based IDL represents the internal object model. In the end, both the program behavior _and the documentation of it_ is defined from the python source definition in [ifex_ast.py](../models/ifex/ifex_ast.py) - The script [create-toc.py](create-toc.py) is used to generate a Table of Contents for the document. ## Source files: - [def-specification.stage1.m.md](./def-specification.stage1.m.md). Lists the parts required to produce the main specification. - [def-specification.stage2.m.md](./def-specification.stage2.m.md). To combine the specification with the table-of-contents. -- [ifex_ast.py](../ifex/model/ifex_ast.py) is the python source that is interpreted to produce `generated-syntax.md` which is later included as the Syntax chapter in the specification. +- [ifex_ast.py](../models/ifex/ifex_ast.py) is the python source that is interpreted to produce `generated-syntax.md` which is later included as the Syntax chapter in the specification. - [def-developers-manual.m.md](./def-developers-manual.m.md) lists the parts required to produce the developer documentation. (Work in progress) ## Build sequence diff --git a/docs/generate-types-doc.py b/docs/generate-types-doc.py index b5938bd3..922472f7 100755 --- a/docs/generate-types-doc.py +++ b/docs/generate-types-doc.py @@ -3,7 +3,7 @@ # SPDX-FileCopyrightText: Copyright (c) 2023 Novaspring AB # SPDX-License-Identifier: MPL-2.0 -from ifex.model.ifex_ast import FundamentalTypes +from models.ifex.ifex_ast import FundamentalTypes print("|Name|Description|Min value|Max value|") print("|----|-----------|---------|---------|") diff --git a/docs/static-developer-generators.md b/docs/static-developer-generators.md index 133d24d2..04ebb81b 100644 --- a/docs/static-developer-generators.md +++ b/docs/static-developer-generators.md @@ -39,7 +39,7 @@ A simple generator (with only one template) can be done like this: Unless you need to add more logic, generating one input file with one (or several) templates is basically already available if ifex_generator.py is called as a -main program. You specify the directory name (relative to `/ifex/templates`) +main program. You specify the directory name (relative to `/output_filters/templates`) where the template(s) is stored. Note that `setup.py` "installs" an executable entrypoint `ifexgen` to call the program: @@ -53,7 +53,7 @@ An advanced generator (with several templates) can be done like this: * Import the ifex_generator.py and ifex_parser.py, and TemplateDir modules. * Get the needed Service description file(s) (YAML), for example from command line argument * For each file, get the Abstract Syntax Tree representation by calling `ifex_parser.get_ast_from_yaml_file(service_desc_file)` -* Write templates according to (some, not all) node types. You can call `gen(node)` or `gen(node,