Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update to work with elasticsearch 8 (maintains support for elasticsearch 7) #488

Draft
wants to merge 8 commits into
base: master
Choose a base branch
from
Draft

Update to work with elasticsearch 8 (maintains support for elasticsearch 7) #488

wants to merge 8 commits into from

Commits on Mar 12, 2024

  1. Update CI to build elastic 8.0.0 and 8.8.0

    @michaelkirk
    michaelkirk committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    44b88d2 View commit details
    Browse the repository at this point in the history

  2. Fix CI: elastic8 startup times out 

    elastic7 seems to start up faster than 8, or maybe it was just random
jitter, but in any case we probably want to avoid spurious timeouts like
this.

e.g.

see elastic7 which started up:
://github.com/michaelkirk-pelias/schema/actions/runs/5272730400/jobs/9535345934

vs. elastic8 which timed out:
https://github.com/michaelkirk-pelias/schema/actions/runs/5272730400/jobs/9535346043
    @michaelkirk
    michaelkirk committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    da99b80 View commit details
    Browse the repository at this point in the history

  3. Fix CI: disable xpack.security for CI 

    With xpack security, elastic8 starts with TLS and user/password auth,
which is unnecessary configuration for CI.
    @michaelkirk
    michaelkirk committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    8dcabb5 View commit details
    Browse the repository at this point in the history

  4. Update elasticsearch package to be compatible with elasticsearch8 

    The previous package was producing an error[^1]. Given that the old npm
package is deprecated[^2], I've switched to the new recommended client.

The new npm package has a peculiar (though well reasoned) versioning
scheme.

The major version of the new npm package denotes the minimum supported
elasticsearch server version.

The package is (apparently) given forward compatibility with newer
version. So in this case, using version 7, means we can talk to
elasticsearch7 and elasticsearch8 (but not elasticsearch6).

This does mean that breaking changes, like dropping node versions, can
occur in a minor release, hence the more restrictive versioning
constraint (~7.17.0 instead of ^7.17.0).

[1] fixes error:
```
--------------
 create index
--------------

/home/runner/work/schema/schema/node_modules/elasticsearch/src/lib/utils.js:303
      throw new TypeError(err);
      ^

TypeError: Invalid apiVersion "8.0", expected a function or one of
_default, 7.6, 7.5, 7.4, 7.3, 7.2, 7.1, 7.0, 6.8, 5.6, 7.7, 7.x, master
    at Object.utils.funcEnum
(/home/runner/work/schema/schema/node_modules/elasticsearch/src/lib/utils.js:303:13)
    at new Client
(/home/runner/work/schema/schema/node_modules/elasticsearch/src/lib/client.js:74:33)
    at Object.<anonymous>
(/home/runner/work/schema/schema/scripts/create_index.js:11:16)
    at Module._compile (node:internal/modules/cjs/loader:1196:14)
    at Object.Module._extensions..js
(node:internal/modules/cjs/loader:1250:10)
    at Module.load (node:internal/modules/cjs/loader:1074:32)
    at Function.Module._load (node:internal/modules/cjs/loader:909:12)
    at Function.executeUserEntryPoint [as runMain]
(node:internal/modules/run_main:81:12)
    at node:internal/main/run_main_module:22:47
Error: Process completed with exit code 1.
```

[2] old package is deprecated
From https://www.npmjs.com/package/elasticsearch
> ⚠️ This client is no longer maintained. We strongly advise you to
migrate to the [new Elasticsearch
client](https://www.elastic.co/blog/new-elasticsearch-javascript-client-released).
    @michaelkirk
    michaelkirk committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    72b5cc9 View commit details
    Browse the repository at this point in the history

  5. adapt to new elasticsearch package 

    - response body moved from `res` to `res.body`
- nodes.info takes *no* arg rather than null before callback
- config.hosts became config.nodes (and changed to string format)
    @michaelkirk
    michaelkirk committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    9a46aef View commit details
    Browse the repository at this point in the history

  6. adapt integration tests to new config 

    We can just use the default here.
    @michaelkirk
    michaelkirk committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    87019dc View commit details
    Browse the repository at this point in the history

  7. Adapt integration suite to new elasticsearch package 

    - On success, elasticsearch errors are now *null* rather than *undefined*,
which broke some tests. Now we just check that `err` is falsy rather than strictly
equal to `undefined`.

- Payload moved from res to res.body

- `statusCode` moved from top level callback arg to `res.statuCode`

- normalized function formatting for tests to `(err, { body })` there
  were a few different styles/indentations.
    @michaelkirk
    michaelkirk committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    9090eba View commit details
    Browse the repository at this point in the history

  8. stdout logger with pelias-elasticsearch 

    The legacy elasticsearch client had an integrated stdout logger.
The new elasticsearch client does not, so instead we use the
pelias-elasticsearch which integrates the modern elasticsearch client
with pelias-logger.
    @michaelkirk
    michaelkirk committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    811b45f View commit details
    Browse the repository at this point in the history