Problems generating Maven and NPM based SBOMs using cdxgen in server mode

[Nowaepon]

Hi,
I'm currently experimenting with cdxgen as I would like to use it in my thesis.

I installed it locally using NPM as described and it works fine and reliably for SBOMs based on Maven and NPM manifests.

Next I tried to run it as a server in a docker compose stack (detailed config at the end of this post).
But now the results of the server are varying.
I'm not sure if I'm just doing something very simple wrong (as I've rarely used docker) or found a bug.

I would appreciate any help as I would love to use cdxgen.
Thanks in advance!

Local cdxgen version: 10.4.3
Docker image version: 10.5.1

Everything I want to achieve is to generate multiple, different sboms by copying the manifest files in the mounted directory, run cdxgen and receive the sbom as json, delete the manifest files and copy the next ones.
I don't provide any source code to cdxgen and I don't integrate cdxgen in a build process.
I only want to generate pre-build SBOMs based on the latest package manager manifests in the project repository.

I tried to generate the SBOM using curl as well as via Python requests.
Both result in the same errors.

In case of NPM it sometimes just tells me that it can't parse the manifest files.
But sometimes, it does work. The likelihood increases, if I create a new directory with the same files in it.
The same directory and files work perfectly well if I run my local cdxgen version.
Does the docker server version do something differently than the local version of cdxgen?
I experienced this error for the first time as I tried to generate the same sbom for a second time.

NPM error:
cdxgen_server | Listening on 0.0.0.0 9090
cdxgen_server | Generating SBOM for /app/scripts/cloneRepos/Projekt/a2testNpm
cdxgen_server | Performing babel-based package usage analysis with source code at /app/scripts/cloneRepos/Projekt/a2testNpm
cdxgen_server | Parsing /app/scripts/cloneRepos/Projekt/a2testNpm/package-lock.json
cdxgen_server | Unable to parse /app/scripts/cloneRepos/Projekt/a2testNpm/package-lock.json without legacy peer dependencies. Retrying ...
cdxgen_server | Unable to parse /app/scripts/cloneRepos/Projekt/a2testNpm/package-lock.json in legacy and non-legacy mode. The resulting SBOM would be incomplete.

For Maven there are two different errors.
When I create a new directory, the SBOM generation usually works reliably.
If I just repeat the SBOM creation afterwards, it works fine, too.
But if I delete the target directory which was created by the first run, I get an error the next time I try to generate the SBOM.
This seems strange to me, because if this behaviour is intended, than I could generate only one single SBOM at all.

Maven error after deleting the target directory
cdxgen_server | Generating SBOM for /app/scripts/cloneRepos/Projekt/MAVENTEST8
cdxgen_server | Executing '/opt/maven/3.9.6/bin/mvn org.cyclonedx:cyclonedx-maven-plugin:2.8.0:makeAggregateBom -DoutputName=bom -DincludeTestScope=true' in /app/scripts/cloneRepos/Projekt/MAVENTEST8
cdxgen_server | Fallback to executing /opt/maven/3.9.6/bin/mvn dependency:tree -DoutputFile=/tmp/cdxmvn-vRsADE/mvn-tree.txt
cdxgen_server | Extracting data from generated bom file /app/scripts/cloneRepos/Projekt/MAVENTEST8/target/bom.json
cdxgen_server | Error: ENOENT: no such file or directory, open '/app/scripts/cloneRepos/Projekt/MAVENTEST8/target/bom.json'
cdxgen_server | at readFileSync (node:fs:448:20)
cdxgen_server | at createJavaBom (file:///opt/cdxgen/index.js:1368:11)
cdxgen_server | at createXBom (file:///opt/cdxgen/index.js:5399:18)
cdxgen_server | at createBom (file:///opt/cdxgen/index.js:5949:20)
cdxgen_server | at file:///opt/cdxgen/server.js:144:28
cdxgen_server | at call (/opt/cdxgen/node_modules/connect/index.js:239:7)
cdxgen_server | at next (/opt/cdxgen/node_modules/connect/index.js:183:5)
cdxgen_server | at next (/opt/cdxgen/node_modules/connect/index.js:161:14)
cdxgen_server | at compression (/opt/cdxgen/node_modules/compression/index.js:220:5)
cdxgen_server | at call (/opt/cdxgen/node_modules/connect/index.js:239:7) {
cdxgen_server | errno: -2,
cdxgen_server | code: 'ENOENT',
cdxgen_server | syscall: 'open',
cdxgen_server | path: '/app/scripts/cloneRepos/Projekt/MAVENTEST8/target/bom.json'
cdxgen_server | }

Another interesting error appears sometimes if I generate an SBOM using curl first and then run a python script to generate the same SBOM once more. The interesting point is, that the files are readable using curl and using cdxgen locally.

Maven error non-readable pom.xml
cdxgen_server | Listening on 0.0.0.0 9090
cdxgen_server | Generating SBOM for /app/scripts/cloneRepos/Projekt/MOVIELENS7
cdxgen_server | Scanning /app/scripts/cloneRepos/Projekt/MOVIELENS7
cdxgen_server | Performing babel-based package usage analysis with source code at /app/scripts/cloneRepos/Projekt/MOVIELENS7
cdxgen_server | Executing '/opt/maven/3.9.6/bin/mvn org.cyclonedx:cyclonedx-maven-plugin:2.8.0:makeAggregateBom -DoutputName=bom -DincludeTestScope=true' in /app/scripts/cloneRepos/Projekt/MOVIELENS7
cdxgen_server | Fallback to executing /opt/maven/3.9.6/bin/mvn dependency:tree -DoutputFile=/tmp/cdxmvn-hIst0l/mvn-tree.txt
cdxgen_server | [INFO] Scanning for projects...
cdxgen_server | [ERROR] [ERROR] Some problems were encountered while processing the POMs:
cdxgen_server | [FATAL] Non-readable POM /app/scripts/cloneRepos/Projekt/MOVIELENS7/pom.xml: /app/scripts/cloneRepos/Projekt/MOVIELENS7/pom.xml (No such file or directory) @
cdxgen_server | @
cdxgen_server | [ERROR] The build could not read 1 project -> [Help 1]
cdxgen_server | [ERROR]
cdxgen_server | [ERROR] The project (/app/scripts/cloneRepos/Projekt/MOVIELENS7/pom.xml) has 1 error
cdxgen_server | [ERROR] Non-readable POM /app/scripts/cloneRepos/Projekt/MOVIELENS7/pom.xml: /app/scripts/cloneRepos/Projekt/MOVIELENS7/pom.xml (No such file or directory)
cdxgen_server | [ERROR]
cdxgen_server | [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
cdxgen_server | [ERROR] Re-run Maven using the -X switch to enable full debug logging.
cdxgen_server | [ERROR]
cdxgen_server | [ERROR] For more information about the errors and possible solutions, please read the following articles:
cdxgen_server | [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
cdxgen_server | Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8
cdxgen_server |
cdxgen_server | The above build errors could be due to:
cdxgen_server |
cdxgen_server | 1. Java version requirement: cdxgen container image bundles Java 21 with maven 3.9 which might be incompatible. Try running cdxgen with the unofficial JDK11-based image ghcr.io/appthreat/cdxgen-java:v10.
cdxgen_server | 2. Private dependencies cannot be downloaded: Check if any additional arguments must be passed to maven and set them via MVN_ARGS environment variable.
cdxgen_server | 3. Check if all required environment variables including any maven profile arguments are passed correctly to this tool.
cdxgen_server |
cdxgen_server | Falling back to parsing pom.xml files. Only direct dependencies would get included!
cdxgen_server | node:fs:448
cdxgen_server | return binding.readFileUtf8(path, stringToFlags(options.flag));
cdxgen_server | ^
cdxgen_server |
cdxgen_server | Error: ENOENT: no such file or directory, open '/app/scripts/cloneRepos/Projekt/MOVIELENS7/pom.xml'
cdxgen_server | at readFileSync (node:fs:448:20)
cdxgen_server | at parsePom (file:///opt/cdxgen/utils.js:1730:19)
cdxgen_server | at createJavaBom (file:///opt/cdxgen/index.js:1337:25)
cdxgen_server | at createMultiXBom (file:///opt/cdxgen/index.js:5026:21)
cdxgen_server | at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
cdxgen_server | at async createBom (file:///opt/cdxgen/index.js:5947:16)
cdxgen_server | at async file:///opt/cdxgen/server.js:144:22 {
cdxgen_server | errno: -2,
cdxgen_server | code: 'ENOENT',
cdxgen_server | syscall: 'open',
cdxgen_server | path: '/app/scripts/cloneRepos/Projekt/MOVIELENS7/pom.xml'
cdxgen_server | }
cdxgen_server |
cdxgen_server | Node.js v22.1.0
cdxgen_server exited with code 1

As I'm receiving errors for NPM and Maven I haven't tried the unofficial cdxgen-java:v10 image yet.
There are no private dependencies in my pom.xml and as the SBOM generation works sometimes, I don't believe that there are missing environment variables.

Docker compose service:
cdxgen_server:
container_name: cdxgen_server
image: ghcr.io/cyclonedx/cdxgen:v10.5.1
command: cdxgen --server --server-host 0.0.0.0 --server-port 9090 --no-recurse
environment:
#FETCH_LICENSE: "true"
#NODE_COMPILE_CACHE: "/tmp/cdxgen-node-cache"
CDXGEN_DEBUG_MODE: "debug"
ports:
- 9090:9090
networks:
- "backend"
volumes:
- ./cloneRepos:/app/scripts/cloneRepos
- /tmp:/tmp

Curl command:
curl "http://127.0.0.1:9090/sbom?path=/app/scripts/cloneRepos/Projekt/MAVENTEST8"

Python script snippet with example values:

request_path = self._build_container_rel_path(data_path)

# For details see: https://cyclonedx.github.io/cdxgen/#/SERVER?id=server-arguments
request_data = {
    "path": "/app/scripts/cloneRepos/Projekt/MAVENTEST8",
    "noBabel": "true", 
    "specVersion": "1.5"
}
response = requests.post(url="http://127.0.0.1:9090/sbom", json=request_data)

if response.status_code == 200:
    # Do something
else:
    # Do something else

[prabhu]

@Nowaepon cdxgen requires the full repo to work properly, so best to pass the git url or a local cloned version. I am already helping a couple of student research projects, so do not have time unfortunately. In general, generating SBOMs requires dedicated time to properly setup and configure the tool and the build environment. The official container image may not work for all applications requiring variants and custom images to be created for specific applications. Wish you all the best!

[Nowaepon]

Thank you for your answer. I was able to fix the problem. It seems my Docker or WSL installation was corrupted. A new installation of both fixed it.

Nevertheless, I have two other small questions @prabhu .
I spotted the multiProject server argument, but found no real documentation about what it actually does (only that it is a boolean). So could you please point me to where I can find this?
Second, if I created SBOMs for multiple python projects, the SBOM of the later projects contains some placeholder dependecies of packages which were only requiered by other projects which I processed before.
I didn't find any environmental or server variable to control this, so do you have an idea how I can fix that?

[prabhu]

multiProject enables recursive mode which is set to true by default in cli mode. Could you share a PR to update the docs?

To generate SBOM for individual projects, you can pass --no-recurse or set CDXGEN_NO_RECURSE=true as env variable. Can you try that option and see if it works?