Update conformance tests to see regression

[alancai98]

Relevant Issues

• [Closes/Related To] Issue #XXX

Description

• < A detailed description >

Other Information

• Updated Unreleased Section in CHANGELOG: [YES/NO]

  • < If NO, why? >

• Any backward-incompatible changes? [YES/NO]

  • < If YES, why? >
  • < For this purpose, we define backward-incompatible changes as changes that—when consumed—can potentially result in
    errors for users that are using our public APIs or the entities that have public visibility in our code-base. >

• Any new external dependencies? [YES/NO]

  • < If YES, which ones and why? >
  • < In addition, please also mention any other alternatives you've considered and the reason they've been discarded >

• Do your changes comply with the Contributing Guidelines
  and Code Style Guidelines? [YES/NO]

License Information

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[github-actions]

CROSS-ENGINE-REPORT ❌

	BASE (LEGACY-V0.14.8)	TARGET (EVAL-A4C55CD)	+/-
% Passing	89.67%	94.93%	5.26% ✅
Passing	5287	5616	329 ✅
Failing	609	80	-529 ✅
Ignored	0	220	220 🔶
Total Tests	5896	5916	20 ✅

Testing Details

• Base Commit: v0.14.8
• Base Engine: LEGACY
• Target Commit: a4c55cd
• Target Engine: EVAL

Result Details

• ❌ REGRESSION DETECTED. See Now Failing/Ignored Tests. ❌
• Passing in both: 2622
• Failing in both: 17
• Ignored in both: 0
• PASSING in BASE but now FAILING in TARGET: 20
• PASSING in BASE but now IGNORED in TARGET: 98
• FAILING in BASE but now PASSING in TARGET: 180
• IGNORED in BASE but now PASSING in TARGET: 0

Now FAILING Tests ❌

The complete list can be found in GitHub CI summary, either from Step Summary or in the Artifact.

Now IGNORED Tests ❌

The complete list can be found in GitHub CI summary, either from Step Summary or in the Artifact.

Now Passing Tests

180 test(s) were previously failing in BASE (LEGACY-V0.14.8) but now pass in TARGET (EVAL-A4C55CD). Before merging, confirm they are intended to pass.

The complete list can be found in GitHub CI summary, either from Step Summary or in the Artifact.

[johnedquinn]

See #1720 (comment)

[alancai98]

PR was meant to show new failing conformance tests are not due to #1721 's changes. Closing.