How to test/dev with Node.js v23

[trentm]

the problem

Running tests with Node.js v23 (since v23.6.0) fails:

% npm test

> @opentelemetry/[email protected] test
> nyc mocha test/**/*.test.ts --exclude 'test/platform/browser/**/*.ts'

(node:93409) ExperimentalWarning: Type Stripping is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:93409) [MODULE_TYPELESS_PACKAGE_JSON] Warning: Module type of file:///Users/trentm/tm/opentelemetry-js3/packages/opentelemetry-core/test/baggage/W3CBaggagePropagator.test.ts is not specified and it doesn't parse as CommonJS.
Reparsing as ES module because module syntax was detected. This incurs a performance overhead.
To eliminate this warning, add "type": "module" to /Users/trentm/tm/opentelemetry-js3/packages/opentelemetry-core/package.json.
...

AFAICT, this is incompatible with our usage of ts-node to enable calling mocha directly on the test .ts files.
Generally our tests run mocha test/**/*.test.ts, mocha loads .mocharc.yml which has:

require: 'ts-node/register'

which basically uses require.extensions to tell require() to handle .ts and .tsx files.
My understanding is that the experimental-strip-types handling of .ts files take precedence.

notes

I'd like to see usage of ts-node dropped eventually. require.extensions (the mechanism it uses) has been deprecated since Node.js v0.10.6.

I'm not sure what tsx's mechanism is.

options

option 1: compile tests to .js, run the JS files

Change all our testing to (a) first compile test/... then (b) execute mocha 'build/test/**/*.test.js'.

Roughly that means a diff like this for each package:

diff --git a/packages/opentelemetry-core/package.json b/packages/opentelemetry-core/package.json
index b8c690523..d9bc47622 100644
--- a/packages/opentelemetry-core/package.json
+++ b/packages/opentelemetry-core/package.json
@@ -17,7 +17,8 @@
     "prepublishOnly": "npm run compile",
     "compile": "tsc --build tsconfig.json tsconfig.esm.json tsconfig.esnext.json",
     "clean": "tsc --build --clean tsconfig.json tsconfig.esm.json tsconfig.esnext.json",
-    "test": "nyc mocha test/**/*.test.ts --exclude 'test/platform/browser/**/*.ts'",
+    "pretest": "tsc --build tsconfig.json",
+    "test": "mocha 'build/test/**/*.test.js' --exclude 'build/test/platform/browser/**/*.js'",
     "test:browser": "karma start --single-run",
     "tdd": "npm run tdd:node",
     "tdd:node": "npm run test -- --watch-extensions ts --watch",

However there are wrinkles:

• Compiling the test/**/*.ts files to build/test/... means we are executing in a different directory. References to any non-.ts files in the tests (e.g. fixtures, JSON data files, whatever), means changing those path references to "../../test/...".
• This is mostly straightforward, except that in experimental/packages/opentelemetry-instrumentation/ the two tests npm test and npm run test:browser conflict with each other on attempting to use the recently-added test/common/third-party/node-semver/range-exclude.js files.

#5410 is a draft PR for this option, including showing the currently failing npm run test:browser in experimental/packages/opentelemetry-instrumentation/

option 2: use node --no-experimental-strip-types ...

There is a node option to turn off the type stripping, such that our current ts-node usage works.
Roughly it looks like this diff in each package:

diff --git a/api/package.json b/api/package.json
index 818732c7a..576b9ffc2 100644
--- a/api/package.json
+++ b/api/package.json
@@ -37,7 +37,7 @@
     "lint:fix": "eslint . --ext .ts --fix",
     "lint": "eslint . --ext .ts",
     "test:browser": "karma start --single-run",
-    "test": "nyc mocha 'test/**/*.test.ts'",
+    "test": "nyc mocha --node-option no-experimental-strip-types 'test/**/*.test.ts'",
     "test:webworker": "karma start karma.worker.js --single-run",
     "cycle-check": "dpdm --exit-code circular:1 src/index.ts",
     "version": "node ../scripts/version-update.js",

Or:

-    "test": "nyc mocha 'test/**/*.test.ts'",
+    "test": "NODE_OPTIONS='--no-experimental-strip-types' nyc mocha 'test/**/*.test.ts'",

However, this breaks running with Node.js 18, 20, and older versions of Node.js v22:

% npm test

> @opentelemetry/[email protected] test
> NODE_OPTIONS='--no-experimental-strip-types' nyc mocha 'test/**/*.test.ts'

node: --no-experimental-strip-types is not allowed in NODE_OPTIONS
npm error Lifecycle script `test` failed with error:

We could add that option to NODE_OPTIONS only for Node.js 23 tests in CI.
This is at best a temporary bandaid, because local dev could not be done with Node.js v23 without the developer manually setting NODE_OPTIONS=....

option 3: use tsx

@pichlermarc mentioned an attempt to use tsx but say "some tests are failing".

A starter diff:

diff --git a/.mocharc.yml b/.mocharc.yml
index 45bd4bdc1..f655ed4b5 100644
--- a/.mocharc.yml
+++ b/.mocharc.yml
@@ -1 +1,2 @@
+node-option:
+  - "import=tsx"
diff --git a/package.json b/package.json
index 6e5c8b12d..21cc6715c 100644
--- a/package.json
+++ b/package.json
@@ -141,5 +141,8 @@
     "examples/http",
     "examples/https",
     "examples/esm-http-ts"
-  ]
+  ],
+  "dependencies": {
+    "tsx": "^4.19.2"
+  }
 }

I haven't dug into the test failures here.

[trentm]

Suggestions of other options are welcome.

[trentm]

Current status:

• option 2 has been done (in test: Test Node.js 23 in CI #5416)
• option 1 has a draft PR (chore: add testing with Node.js 23 #5410), but this might not get carried over the line
• Marc mentioned that he will attempt option 3 after the SDK 2.0 milestone has been completed and things have settled down a bit.

[cjihrig]

Regarding option 3 - I believe the failures are related to mocking with tsx. There are a few relevant issues reported against tsx:

• The argument 'methodName' must be a method. Received undefined privatenumber/tsx#400
• The argument 'methodName' must be a method. Received undefined privatenumber/tsx#507
• Tsx ignores type commonjs in package.json privatenumber/tsx#508

Those issues are about the Node test runner's mocking capabilities, but they also apply to sinon.