Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
chore(langchain): disable flaky tests (#11511)
There appear to be stability issues with using snapshots and/or LangChain in general. There are failures in the mocked tests that look like: ``` builtins.AssertionError: assert 0 == 1 + where 0 = <MagicMock name='LLMObsSpanWriter().enqueue' id='127482146130048'>.call_count + where <MagicMock name='LLMObsSpanWriter().enqueue' id='127482146130048'> = <MagicMock name='LLMObsSpanWriter()' id='127482147073440'>.enqueue ``` as well as failures with snapshot based tests: ``` builtins.Failed: At request <Request GET /test/session/snapshot >: At snapshot (token='tests.contrib.langchain.test_langchain_community.test_lcel_chain_simple'): - Directory: /go/src/github.com/DataDog/apm-reliability/dd-trace-py/tests/snapshots - CI mode: 1 - Trace File: /go/src/github.com/DataDog/apm-reliability/dd-trace-py/tests/snapshots/tests.contrib.langchain.test_langchain_community.test_lcel_chain_simple.json - Stats File: /go/src/github.com/DataDog/apm-reliability/dd-trace-py/tests/snapshots/tests.contrib.langchain.test_langchain_community.test_lcel_chain_simple_tracestats.json At compare of 1 expected trace(s) to 0 received trace(s): Did not receive expected traces: 'langchain.request' ``` While we investigate a more stable method of testing it makes sense to disable the tests to avoid noise to our neighbours in the library :). DOWN WITH FLAKY TESTS [](https://datadoghq.atlassian.net/browse/MLOB-1920) ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [ ] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)
- Loading branch information