facing duplicate dependencies issue

[phoenix-aditya]

Hey i am running cdxgen on a number of applications and some of them are failing with the below error.
If possible can someone elaborate as to the cause of the below error?

{
    instancePath: '/dependencies',
    schemaPath: '#/properties/dependencies/uniqueItems',
    keyword: 'uniqueItems',
    params: { i: 1, j: 0 },
    message: 'must NOT have duplicate items (items ## 0 and 1 are identical)'
  }

[prabhu]

@phoenix-aditya, Could you share the dependencies sections 0 and 1? Or is there a way to reproduce this issue?

[phoenix-aditya]

Cant provide the dependencies for this particular case,
Looking into ways to reproduce the issue.
Will update here after some more digging

[phoenix-aditya]

seems the issue is something related to poetry install failing, On that front as far as i know cdxgen used to manually parse poetry lock files right? Should it raise an issue if poetry is not installed?

EDIT: I installed poetry in the image where cdxgen is running, It is still giving the same issue as that poetry install failing

Poetry install has failed. Setup and activate the poetry virtual environment and re-run cdxgen.
[
  {
    instancePath: '/dependencies',
    schemaPath: '#/properties/dependencies/uniqueItems',
    keyword: 'uniqueItems',
    params: { i: 1, j: 0 },
    message: 'must NOT have duplicate items (items ## 0 and 1 are identical)'
  }
]

[cerrussell]

Additionally, please include the version of cdxgen using cdxgen --version, and if you are fetching licenses. Thanks!

[phoenix-aditya]

cdxgen version is 9.7.1,
Can you elaborate more on what you refer to with fetching licenses?

[prabhu]

@phoenix-aditya, do you face this error with 9.8.0 as well? Was the environment variable FETCH_LICENSE=true used to generate the SBoM?

[phoenix-aditya]

Have not tried this with version 9.8.0,
The environment variable was not set to true while generating sbom

[cerrussell]

@phoenix-aditya Regarding the poetry failure, the cause will be one of the following:

1. Poetry is not installed or cannot be located. It is looking for poetry on the system running the scan, not on the image you are scanning. The other option is invoking cdxgen from within a virtual or poetry environment.
2. A base python interpreter cannot be located - I believe poetry looks in the PATH for this.
3. Poetry can locate a base interpreter, but its version is not compatible with the python version specified in pyproject.toml

Can you update to our latest release and add CDXGEN_DEBUG_MODE = debug and run again? This will show you what poetry reports the issue is. This failure probably isn't the reason why you are getting duplicates, but results are most accurate when it is used.

[phoenix-aditya]

I have poetry installed in the host machine
Can you tell me does cdxgen use the POETRY_HOME variable to access poetry?
As through the shell i am able to run poetry commands on host

[cerrussell]

@phoenix-aditya cdxgen doesn't locate poetry, it assumes the poetry command works. Were you able to run with CDXGEN_DEBUG_MODE=debug on our latest release? Did it say it could not locate poetry?

[phoenix-aditya]

I ran it with 9.8.0 with the debug mode set as you stated.
It dosnt really say much regarding poetry or python, Just the following

Poetry install has failed. Setup and activate the poetry virtual environment and re-run cdxgen.
NOTE: Setup and activate a python virtual environment for this project prior to invoking cdxgen to improve SBoM accuracy.
Found 48 python packages at *application_name*

I verified that poetry is installed and accessible through terminal on the host machine

[prabhu]

We may need to implement this ticket to solve this issue. #550

[cerrussell]

@phoenix-aditya Thanks for the info. We'll tag you on #550 when we have changes for you to test.