Support for Node.js?

[LoneRifle]

I've briefly fiddled with wa-sqlite in the Node REPL, and as part of a test script I tried running in Node. On both occasions I was greeted with the following:

Uncaught ReferenceError: __dirname is not defined
    at file:///Users/LoneRifle/crashburn/wa-sqlite-s3vfs/node_modules/wa-sqlite/dist/wa-sqlite-async.mjs:11:43

The test script consists of the example adapted from the documentation for SQLiteAPI:

(async function() {
  var SQLite = await import('wa-sqlite')
  var { default: SQLiteESMFactory} = await import('wa-sqlite/dist/wa-sqlite-async.mjs')
  const module = await SQLiteESMFactory()

  const sqlite3 = SQLite.Factory(module)

  const db = await sqlite3.open_v2('myDB')
})()

I understand that this is due to emscripten-core/emscripten#11792. The issue also contains a comment that proposes a workaround.

Would wa-sqlite have better support for Node, perhaps through incorporating appropriate workarounds into libmodule?

[rhashimoto]

I'm not sure what you mean by "libmodule". If you mean the libmodule.js (and .c) in wa-sqlite itself, that is for SQLite module support, i.e. virtual database tables. I don't know if you mistakenly thought that was ES6 module machinery, or if there is some incompatibility in that code with the Emscripten workaround. If the latter can you elaborate?

Just looking at the workaround without trying it, it looks like it can be applied at the application level where the WASM module is loaded, i.e. without modifications to wa-sqlite. However, I don't know if all the Javascript in wa-sqlite is compatible with node, as I generally use constructs as long as they are supported across the evergreen browsers without checking for node support.

I don't really want to do a lot of work to support running in node at the moment, but if you (or anyone) can document how it fails purely due to wa-sqlite (i.e. not in Emscripten-generated code), I would consider making changes if they aren't too burdensome.

[LoneRifle]

Sorry, I guess I should have been more precise! I was indeed referring to libmodule.js, and I mistook it for integrations for ES6.

Afaict, the first three lines of the workaround snippet I pointed to is to be inserted into what would become dist/wa-sqlite.mjs or dist/wa-sqlite-async.mjs, so that the fourth line (the one before // business as usual) can be invoked elsewhere by callers.

If however you prefer for wa-sqlite to not actively support Node.js for now, I can tinker on a bit and figure out how an application can supply __dirname to wa-sqlite. I think the challenge might be that __dirname is meant to be a global scoped to the module, ie, if a CommonJS module dependent defines it for wa-sqlite, it has to find a way to preserve its own.

Would you be happy to accept a PR to the README documenting the workaround to get it to work in Node.js assuming I figure it out?

[rhashimoto]

Afaict, the first three lines of the workaround snippet I pointed to is to be inserted into what would become dist/wa-sqlite.mjs or dist/wa-sqlite-async.mjs, so that the fourth line (the one before // business as usual) can be invoked elsewhere by callers.

That's not how I understand it (which could be wrong). The way it looks to me is that the suggested workaround creates globals for __dirname and require and then imports the unmodified module created by Emscripten. For wa-sqlite I think you would simply swap dist/wa-sqlite.mjs (as-is) for ./my-emscripten-module.js.

I'm generally averse to accepting pull requests. If it's just a matter of documenting what works, adding it to this thread would be good (especially if we can convert it to a question and mark it as the answer) or it could go into the FAQ.

[LoneRifle]

Turns out we were both right in a fashion, so I'm documenting it here for posterity.

I tried both approaches - modifying wa-sqlite.mjs with the workaround and adding the workaround to the calling code - and they both work with some tinkering. The reason why I understood that wa-sqlite had to be modified was because the workaround made reference to import.meta.url, ie, the path of the module.

However, one can use wa-sqlite without modification by using the workaround in client code, simply replacing import.meta.url to the path of the .mjs files relative to the client code. For that approach, the globals have to be set long enough for SQLiteESMFactory() to be called, thereafter they can be reversed. I was reluctant to use this approach though, as it makes the code slightly more brittle, since care has to be taken around the area of code where wa-sqlite is being initialised.

An example of the workaround can be found here.

I'll add an issue for you to vet for inclusion into the FAQ in a bit!

[LoneRifle]

#62 filed for inclusion into the FAQ!