v.NEXT: Support for async generators (Stream protocol)

[Zearin]

For your next version. (I read that there is a module-based rewrite in the works! 👍 )

---

• iCal feeds can be quite large
• iCal recurrence computation can get quite lengthy

Anything that involves a potentially long read or computation would benefit greatly from the use of async generators. Parsing and computing recurrence rules are the top two candidates that spring to mind.

---

This excellent post on 2ality explains the benefits in far greater detail and clarity than I ever could.

The blog post was also the progenitor of the equivalent section in his book (which presents very similar info, but slightly differently; you may prefer one over the other).

[kewisch]

Would love to use async generators in this project. Do you have a suggestion on what the API could look like specifically for ical.js? If you could base this off #538 it would be great.

[Zearin]

I’m not sure if I’m the best person to ask that question, but I’ll take a quick shot.

From §42.3.3 Example: from chunks to lines:

Node.js streams iterate over chunks (arbitrarily long pieces) of data. The following asynchronous generator converts an async iterable over chunks to an async iterable over lines:

[ example snipped ]

Let’s apply chunksToLines() to an async iterable over chunks (as produced by chunkIterable()):

[ example snipped ]

Now that we have an asynchronous iterable over lines, we can use the solution of a previous exercise, numberLines(), to number those lines:

[ example snipped ]

So what this is doing is reading an arbitrarily long stream, reading chunks as efficiently as possible, but providing those chunks to the consumer in logical units of data.

(And this example does it using small, composable functions. This makes them reusable all over the place!)

My thought is that this technique could be used to read long iCal streams and feed data to the consumer as Event components.

---

WARNING: EXPERTS NEEDED! (that’s not me)

It’s at this point that I know that API design has to balance consumer needs with the needs of the library, so I don’t know if my taking a crack at “what would this look like?” would be useful or not. A more skilled developer could probably do better (or apply async generators to different parts of the library that I’m not even thinking of).

---

That said, here’s an off-the-top-of-my-head attempt:

for await (const vevent of ICAL.stream(fileOrUrl).iterComponents('vevent')  ) {
    let evt = new ICAL.Event(vevent)
    // ... do stuff
}

// or, for batch operations...
let events = Promise.all([ ...ICAL.stream(fileOrUrl).iterComponents('vevent') ])
    .map( vevent => { return new ICAL.Event(vevent) })