Using tar stream in the browser

[vritant24]

I'm trying to use this library in a browser environment, but I'm running into some issues I'd like help with.

const stream: ReadableStream<Uint8Array> = <get stream> 
await stream.pipeTo(extractor);

this doesn't work since pipeTo expects a WritableStream. If I change the line to:
await stream.pipeTo(extractor as unknown as WritableStream<Uint8Array>);

This fails with the following error:
TypeError: The "transform.writable" property must be an instance of WritableStream. Received an instance of Extract

Is there a recommended way of extracting a tar from a browser ReadableStream?

[mafintosh]

don't know much about browser streams, but if you share some docs on them, mb we can make a wrapper

[vritant24]

https://developer.mozilla.org/en-US/docs/Web/API/Streams_API/Concepts

MDN has the docs for readable streams. unfortunately I'm a little less versed on these as well, but happy to help implement them @mafintosh

[mafintosh]

We just need a streamx -> browser stream general module. prop not too hard :)

[gchartier]

Any update on this? I have the exact same need / problem

[luiscastro193]

+1 on this

[luiscastro193]

In the meantime, I recommend using https://www.npmjs.com/package/readable-web-to-node-stream and https://www.npmjs.com/package/readable-stream-node-to-web

[maxpatiiuk]

readable-web-to-node-stream appears to no longer be maintained, and fails to build in Webpack for me. I was able to use @smessie/readable-web-to-node-stream instead

Example code

Install dependencies:

npm i tar-stream @smessie/readable-web-to-node-stream
npm i -D @types/tar-stream

Content script:

const worker = new Worker('worker.js');

// User picks a file - we convert the file to a stream, and give it to a web worker:
const fileStream = file.stream();
worker.postMessage({ fileStream }, [fileStream]);

Web worker:

import { ReadableWebToNodeStream } from '@smessie/readable-web-to-node-stream';
import type { Readable } from 'node:stream';
import tar from 'tar-stream';

self.addEventListener('message', async (event) => {
  const fileStream: ReadableStream<Uint8Array> = event.data.fileStream;
  const decompressionStream = new DecompressionStream('gzip');
  const decompressedStream = fileStream.pipeThrough(decompressionStream);
  const extract = tar.extract();
  const nodeStream = new ReadableWebToNodeStream(decompressedStream) as unknown as Readable;
  nodeStream.pipe(extract);
  for await (const entry of extract) {
    console.log(entry.header.type, entry.header.name)
    entry.resume();
  }
});

[piranna]

There's Native support for WebStreams un Node.js, we should use them instead of third-party modules.