Generator not being called from CMSIS-Build

[LMESTM]

Hi,

Using CMSIS-Buil 0.10.0, with a SW component that relies on a Generator.
The Generator command seems not to be called even though the Generator section is well found because the cbuildgen reports an error when not finding the expected gpdsc file.

(build-cli) cleaning files
(cbuildgen): Build Process Manager 0.10.0 (C) 2020 ARM
M650: Command completed successfully.
(cbuildgen): Build Process Manager 0.10.0 (C) 2020 ARM

*** ERROR M204:
  Path not found: 'C:\[...]....Gen.gpdsc'!

The Generator is defined like below

  <generators>
    <!-- This generator is launched if any component referencing this generator by 'id' is selected and the specified <gpdsc> file does not exist -->
    <generator id="MyTestGen">
      <description>Test</description>
      <exe>
        <command host = "win">myscript.bat</command>
        <!-- $S = Device Family Pack base folder -->
        <argument>$S</argument>
        <!-- D = Device (Dname/Dvariant as configured by environment) -->
        <argument>$D</argument>
        <!-- Project path and project name (as configured by environment) -->
        <argument>$P</argument>
        <argument>#P</argument>
      </exe>
      <!-- workingDir></workingDir --> <!-- path is specified either absolute or relative to PDSC or GPDSC file. If not specified it is the project directory configured by the environment -->
      <gpdsc name="$P\Gen.gpdsc"/>
    </generator>
  </generators>

and the .bat file is not called.

When using the same pack from Keil µVision 5.33, the generator script is being called as expected.

[LMESTM]

@jkrech in case you could help ...

[jkrech]

Sorry for the lack of response.
The current version of the CMSIS-Build system works under the assumption that any "pre-build" steps like code generation and component configuration have taken place before the build is invoked. The challenge for unattended/headless builds is the fact that the generator may not be installed or may require a UI. Certainly an area that needs more work.

[LMESTM]

@jkrech ok thanks for your response. Ok at least I understand this is expected with the current version and not something I missed in my setup - thanks for the ACK. The build is already reporting an error as the GPDSC is missing. Calling the generator may also report an error in case installation are missing. Maybe a hook or error message with the generator to be called could be inserted as a first step. We'll also look at the options on our side.

[fred-r]

Hi @jkrech,

the idea of having a pre-build step is a bit difficult to implement if we use the generator concept.
From the cprj it is easy to determine that a generation step is required.
But then, how to find the required generator in the appropriate pack wihtout re-implementing a pack management ?

Thanks & Regards,
Fred

[jkrech]

Hi @fred-r ,

just to be clear, this is a missing feature (we neither have pre-build, post-build nor generator support in CMSIS-Build today) and it was not my expectation you can easily solve this based on the current version of cmsis-build, however for the purpose of prototyping I believe you could
a) extend the cbuild.sh script with a call of your generator
b) in the generator parse the cprj file for selected components
c) make the component script files with attr="config" so that they are listed in cprj file for the component so you can read the path from where to fetch them (this also gives you the location of the packs). CMSIS-Build uses $CMSIS_PACK_ROOT environment variable and some things probably need to be hardcoded until we have extended the tools.

[fred-r]

Hi @jkrech,

Thanks for the tips.

Point (a) is not so easy because the genrator is part of a pack so I need to find my way to the pack. I do not think we have variables for this except CMSIS_PACK_ROOT but this does not give the pack and its version.

Point (b) => yes, this is what we do in our generator

Point (c) => is it what you suggest to solve point (a) ?

Thanks & Regards,
Fred

[fred-r]

Hi again,

Itried your point (c) as follows:

<component generator="STM32Cube_CodeGen" Cclass="Device" Cgroup="STM32Cube HAL Code Gen" Csub="Generated Code"       Cversion="0.0.1" condition="STM32L5 HAL">
  <description>Temporary component that will store all the resulting generated code</description>
  <files>
      <file category="utility" name=".config/genCode.bat" attr="config"/>
  </files>
</component>

So I declare my generator tool as an utility in the pdsc of the component.
Thanks to this, the .bat is copied locally in the RTE folder but:

1. it does not give me the required variables to invoke it (like uvision does)
2. I face a segmentation fault when calling cbuildgen make:

/__INTERNAL__tools/build-system/scripts/build-cli.sh: line 274: 445408 Segmentation fault cbuildgen make ${OUTPUT_DIR}/${PROJ_NAME}.cprj --toolchain=${TOOLCHAIN} --outdir=${OUTPUT_DIR}/${BUILD_DIRECTORY} --intdir=${OUTPUT_DIR}/${BUILD_DIRECTORY}

If I comment out the new line in the pdsc the segmentation fault disappears.

Thanks & Regards,
Fred

[fred-r]

Follow-up : if I remove the attr="config" then the segmentation fault disappears

<component generator="STM32Cube_CodeGen" Cclass="Device" Cgroup="STM32Cube HAL Code Gen" Csub="Generated Code"       Cversion="0.0.1" condition="STM32L5 HAL">
  <description>Temporary component that will store all the resulting generated code</description>
  <files>
      <file category="utility" name=".config/genCode.bat"/>
  </files>
</component>

but I do not get my batch file locally.

Anyway, I would be more interested in getting the path to the pack itself (this would give me access to my script but also the artefacts it requires).

Thanks & Regards,
Fred

[jkrech]

Hi Fred,
the segmentation fault is bug that we need to investigate, however could you please try to rename your folder .config to something without a leading . Maybe that is the root cause for the segfault?

It is understood that this does not give you the invocation like from uVision out of the box but I was hoping you could work around this somehow until we have this functionality in the tools. Sorry if this wasn't helping.

Regards,

Joachim

[fred-r]

Hi Joachim,

your idea was good, it helped in exploring a possibility, thanks.

For the segmentation fault, I can reproduce it even if I remove the ".".

Besides the "." is not an issue in this other component (but no attr="config"):

<component Cclass="Device" Cgroup="STM32Cube HAL Code Gen" Csub="System Init"       Cversion="0.0.1" condition="Generated Code">
  <description>Code Generation for common System files</description>
  <files>
    <!-- this component will be replaced by the component described in gpdsc from generator -->
    <file category="other" name=".config/ReferenceFiles/Common/stm32_system_template.c"/>
    <file category="other" name=".config/ReferenceFiles/Common/stm32_system_template.h"/>
    <file category="other" name=".config/ReferenceFiles/Common/stm32_target_template.h"/>
    <!-- file category="other" name=".config/stm32l5xx_hal_common_parameters.json" attr="config"/ -->
  </files>
</component>

It might be difficult for you to reproduce it.
I am not sure I can easily share a setup with you (unless you want a Teams session?).

Thanks & Regards,
Fred

[brondani]

Hi Fred,

I was able to reproduce the seg fault and to spot the root cause, thanks for reporting. It happens when generating the audit file, specifically when evaluating "config" files in a "generator" component. This is a bug and will be fixed for the next release.

To get your "hack" working, you could declare and use a dedicated component without the generator attribute, for example removing the generator="STM32Cube_CodeGen" from the component you have described.

Best regards
Daniel

[fred-r]

Hi @brondani and @jkrech,

from this thread I understand two points:

1. CMSIS build needs to evolve to take the generator into account (is this planned ?)
2. A bug fix will be delivered

Shall we keep this ticket open or can we close it assuming that you already track these points ?

Thanks & Regards,
Fred

[brondani]

Hi Fred

Sorry for the delay, I missed the GitHub notification.
Yes you understood it correctly, support for calling generators is planned and the bug fix will be delivered.
Additionally support for pre and post-build steps are also planned (since you mentioned them above).
Let's keep the ticket open until the functionalities are implemented and released.

Thanks
Daniel

[fred-r]

Hi Daniel,

thanks for the follow-up.
Do you already have an idea of the API that will be provided ?

Would it be possible to have an "atomic" API to invoke the generator ?
I would like to be able to invoke code generation without triggering the rest of the build steps.

Thanks & Regards,
Fred

[brondani]

Hi Fred,

My intention would be to create a separated make/cmake target to call the generator and set this target as a build dependency.
In a normal build the generator call and the successive build steps are automatically invoked according to the dependency tree.
As a separated target if needed you could invoke it directly without triggering other build steps.
Does this answer your questions?

Thanks
Daniel