Composer versioning uses stability modifiers incorrectly

[rarkins]

Discussed in #24989

^{Originally posted by Xerkus October 3, 2023}

How are you running Renovate?

Mend Renovate hosted app on github.com

If you're self-hosting Renovate, tell us what version of Renovate you run.

No response

If you're self-hosting Renovate, select which platform you are using.

None

Was this something which used to work for you, and then stopped?

I am trying to get this working for the first time

Describe the problem

Renovate incorrectly assumes stability flags to be equal to stability suffixes in the version, eg 1.2.3@RC1 to be equal to 1.2.3-rc1

Part of version normalization:

renovate/lib/modules/versioning/composer/index.ts

Lines 46 to 62 in 6f4c389

	function convertStabilityModifier(input: string): string {
	// Handle stability modifiers.
	const versionParts = input.split('@');
	if (versionParts.length === 1) {
	return input;
	}
	// 1.0@beta2 to 1.0-beta.2
	const stability = versionParts[1].replace(
	regEx(/(?:^|\s)(beta|alpha|rc)([1-9][0-9]*)(?: |$)/gi),
	'$1.$2'
	);
	// If there is a stability part, npm semver expects the version
	// to be full
	return padZeroes(versionParts[0]) + '-' + stability;
	}

And as seen in tests:

renovate/lib/modules/versioning/composer/index.spec.ts

Lines 28 to 38 in 0da50b5

	it.each`
	a | b | expected
	${'1.2.0'} | ${'v1.2'} | ${true}
	${'v1.0.0'} | ${'1'} | ${true}
	${'1.0@alpha3'} | ${'1.0.0-alpha.3'} | ${true}
	${'1.0@beta'} | ${'1.0.0-beta'} | ${true}
	${'1.0@rc2'} | ${'1.0.0-rc.2'} | ${true}
	${'1.0.0'} | ${'1.0.0-p1'} | ${false}
	`('equals("$a", "$b") === $expected', ({ a, b, expected }) => {
	expect(semver.equals(a, b)).toBe(expected);
	});

There are three things wrong here:

• stability flags are constraints and never a part of the actual version. Constraint 1.2.3@RC is a range, equivalent to >=1.2.3-rc <1.2.4. Tag datasource with tag 1.2.3@RC will likely produce invalid result if I read the renovate code right.
• in the presence of stability suffix in the version, unless said suffix is stable, composer entirely disregards stability flag. Constraint 1.2.3-alpha@RC is equivalent to 1.2.3@alpha
• stability flag can be one of: stable, RC, alpha, beta or dev with no numerical suffixes allowed. 1.2.3@RC1 is invalid and composer won't parse it.

I discovered this while looking for other non-canonical formats that are supported by composer but not renovate. Eg 1.2.3RC1 or 1.2.3a1. I would like to provide implementation for same version normalization as used by composer and fix this bug as well.

No reproduction needed:

composer require php:~8.2.1@rc1

In VersionParser.php line 521:
                                                                                      
  Could not parse version constraint ~8.2.0@rc1: Invalid version string "~8.2.0@rc1"

Relevant debug logs

No response

Have you created a minimal reproduction repository?

I have explained in the description why a minimal reproduction is impossible

[Xerkus]

Since stability flag is not a valid version but a constraint should I investigate moving this to be handled at composer manager instead?

[rarkins]

I suspect all changes need to be in composer versioning. For example versioning is what determines if a value is a version or a range, or whether a version satisfies a range, etc.

[rarkins]

If you fix and add to the unit tests for composer versioning then you might find that the rest simply works

[Xerkus]

@rarkins I see in tests describe() is used quite sparingly. Is it a convention or merely not provided by original contributors?

Would it be acceptable for me to declare tests as follows

  describe('Only stable|RC|beta|alpha|dev are valid as stability flag', () => {
    it.each`
      version              | expected
      ${'~1.2.3@dev'}      | ${true}
      ${'~1.2.3@dev1'}     | ${false}
      ${'~1.2.3@alpha'}    | ${true}
      ${'~1.2.3@alpha1'}   | ${false}
      ${'~1.2.3@beta'}     | ${true}
      ${'~1.2.3@beta1'}    | ${false}
      ${'~1.2.3@rc'}       | ${true}
      ${'~1.2.3@rc1'}      | ${false}
      ${'~1.2.3@stable'}   | ${true}
      ${'~1.2.3@stable1'}  | ${false}
      ${'~1.2.3@'}         | ${false}
      ${'~1.2.3@patch'}    | ${false}
      ${'~1.2.3@p'}        | ${false}
      ${'~1.2.3-beta@rc'}  | ${true}
      ${'~1.2.3-rc1@beta'} | ${true}
      ${'~1.2.3@RC'}       | ${true}
      ${'~1.2.3@BETA'}     | ${true}
    `('isValid("$version") === $expected', ({ version, expected }) => {
      const res = !!semver.isValid(version);
      expect(res).toBe(expected);
    });
  });

Or should I just fold those into existing test with no extra description?

[rarkins]

You are definitely welcome to improve the amount of documentation for such tests through describe!

[Xerkus]

Should I handle stability flags and normalization for version variations supported by composer as separate PRs?

[rarkins]

Up to you. Generally smaller is better, if it pesky take too much time

[Xerkus]

Digging through Composer source I discovered that I was wrong but in a weirder way.

Stability flags are processed for a root package dependencies. At this level stability flags are applied to repositories - composer analog for datasource - to filter out stabilities lower than minimum stability. This is before remaining versions are passed through constraint matching.
This behavior is consistent with composer documentation:

To allow various stabilities without enforcing them at the constraint level however, you may use stability-flags like @<stability> (e.g. @dev) to let Composer know that a given package can be installed in a different stability than your default minimum-stability setting.

When stability flag or several are present then the lowest flag is used as minimum stability for that package. Root-only option minimum-stability has no effect for that package.
In the absence of stability flag minimum stability for the package is determined by pre-release identifiers or minimum-stability option whichever is lower.

Stability flags also affect parsed constraints but in inconsistent manner. See composer/semver#154
To summarize:

• No effect for exact version, eg 1.2.3@rc
• No effect for tilde range, eg ~1.2.3@rc
• No effect for caret range, eg ^1.2.3@rc
• Not allowed for hyphenated range, eg1.2.0 - 1.3.0
• Allowed but has no effect for wildcard range, eg 1.2.*@rc
• For remaining > >= < <= and <> operators stability flag is processed and appended to a version as pre-release suffix
• No effect when pre-release identifier present that is considered unstable - RC|alpha|beta|dev, eg >1.2.3-alpha@rc
• Broken constraint when stable patch pre-release suffix is present, eg >1.2.3-p1@rc results in > 1.2.3.0-patch1-rc

There is a strange effect that I would need to confirm with composer maintainers as intentional:
Constraint >1.0.0@alpha <2.0.0@rc is parsed as [> 1.0.0.0-alpha < 2.0.0.0-rc] but because minimum stability for a dependency uses lowest stability flag it would consider and allow 2.0.0-alpha version.
However constraint >1.0.0@rc < 2.0.0@rc sets minimum stability to rc and no unstable versions at 2.0.0 would be considered.

Some of this affects renovate and some of it does not. Now I need to figure what is relevant here.