Skip to content

Commit

Permalink
fix: fix documentation
Browse files Browse the repository at this point in the history
Many codes of the examples were not compilable.
Fixing it and adding some other unit tests so
some important properties not tested before are
validated
  • Loading branch information
Farenheith committed Nov 9, 2024
1 parent 6c5d29a commit 66fe392
Show file tree
Hide file tree
Showing 9 changed files with 105 additions and 57 deletions.
99 changes: 59 additions & 40 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,9 @@ const result = flattedList
This code looks fluent and easy to read, but it is severe to performance.
That's because each operation do a complete iteration over the array, generating a new one! It can cause serious memory and cpu consumption, so, such practice is not encouraged.
So, to solve this, you can write an equivalent code which will solve everything in with two chained loops. This will give you the best performance possible, but can result in a code harder to maintain.
And that's where **fast-iterable** comes in!
And that's where **fluent-iterable** comes in!

With **fast-iterable**, you can do the same operation with the following code:
With **fluent-iterable**, you can do the same operation with the following code:

``` typescript
const result = fluent(list)
Expand All @@ -68,7 +68,7 @@ const result = fluent(list)
```

Pretty simple, right? With this code, you'll do exactly two chained loops, exactly as the vanilla solution described above!
**fast-iterable** takes advantage of the Iterable and AsyncIterable contracts to achieve this, but it goes beyond. It uses a special library called **augmentative-iterables** that we developed as the core engine of the iterations. This library is focused exclusively in performance, so, with it, we achieved a processing time very close to the vanilla solution above!
**fluent-iterable** takes advantage of the Iterable and AsyncIterable contracts to achieve this, but it goes beyond. It uses a special library called **augmentative-iterables** that we developed as the core engine of the iterations. This library is focused exclusively in performance, so, with it, we achieved a processing time very close to the vanilla solution above!
Comparing to iterations using **for of**, the native way to iterate over iterables of JavaScript, we achieved a result 50% faster!

## Doesn't it what rxjs do?
Expand All @@ -85,20 +85,20 @@ Think of RxJS as Lodash for events.
That's it. Rxjs is focused primarily in event handling.
Over that, some key differences can be pointed out:

* *A previous operation of rxjs doesn't stop when some next operation stops, while with **fast-iterable** it does.*
That's because, with rxjs you can chain multiple operations parallel after one, which makes sense for event handling. With **fast-iterable**, on the other hand, you can only have, normally, a straight line of operations and,f no matter what operation break the iteration, everything stops.
* *With rxjs, a previous operation doesn't wait for a async next operation to end before go to the next step, while with **fast-iterable** it does.*
* *A previous operation of rxjs doesn't stop when some next operation stops, while with **fluent-iterable** it does.*
That's because, with rxjs you can chain multiple operations parallel after one, which makes sense for event handling. With **fluent-iterable**, on the other hand, you can only have, normally, a straight line of operations and,f no matter what operation break the iteration, everything stops.
* *With rxjs, a previous operation doesn't wait for a async next operation to end before go to the next step, while with **fluent-iterable** it does.*
Again, rxjs is focused on events. When dealing with event, you just want to emit them as fast as possible. With a simple iteration, though, you want to make sure that the whole chain of steps is concluded before advancing to the next one.

So, as you see, regardless some similarities, there're some pretty important differences between them and those are libraries focused on quite different problems.

## Usage

**fast-iterable** have some neat operations already implements. If you want to Click here for the [Full API Reference](https://github.com/Codibre/fluent-iterable/blob/master/docs/README.md).
**fluent-iterable** have some neat operations already implements. If you want to Click here for the [Full API Reference](https://github.com/Codibre/fluent-iterable/blob/master/docs/README.md).

### Basics

ECMAScript introduced support for [iterables and generator functions](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Iterators_and_Generators) with version ES6 and their [asynchronous counterparts](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Symbol/asyncIterator) with version ES2018. It has introduced an abstraction over sequential iterators (arrays, maps, generators, etc), enabling us to implement solutions regardless of the actual type of the iterable collection. It is especially powerful when using in tandem with generator functions to avoid storing all items in memory when its avoidable. The API provided by ***fast-iterable*** reads the elements of the underlying iterable only when needed and stops reading elements as soon as the result is determined.
ECMAScript introduced support for [iterables and generator functions](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Iterators_and_Generators) with version ES6 and their [asynchronous counterparts](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Symbol/asyncIterator) with version ES2018. It has introduced an abstraction over sequential iterators (arrays, maps, generators, etc), enabling us to implement solutions regardless of the actual type of the iterable collection. It is especially powerful when using in tandem with generator functions to avoid storing all items in memory when its avoidable. The API provided by ***fluent-iterable*** reads the elements of the underlying iterable only when needed and stops reading elements as soon as the result is determined.

To get started with the fluent API, you need to translate the iterable (can be any object with [Symbol](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Symbol) [iterator](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Symbol/iterator) or [asyncIterator](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Symbol/asyncIterator) defined) into either a **FluentIterable** using **fluent()** or a **FluentAsyncIterable** using **fluentAsync()**.

Expand All @@ -109,7 +109,7 @@ import {
fluentAsync,
FluentIterable,
FluentAsyncIterable,
} from '**fast-iterable**';
} from '@codibre/fluent-iterable';

const iterableOfArray: FluentIterable<number> = fluent([3, 1, 8, 6, 9, 2]);

Expand All @@ -136,7 +136,7 @@ async function* emails(): AsyncIterable<string> {
if (!res.ok) {
break;
}
yield* (await res.json()).data.map((user) => user.email);
yield* (await res.json()).data.map((user: { email: string }) => user.email);
}
}

Expand Down Expand Up @@ -171,11 +171,11 @@ function getNumberOfUsers(iterable: FluentAsyncIterable<ChatMessage>): Promise<n
return getAllUsers(iterable).count();
}

async function getMostActiveUser(iterable: FluentAsyncIterable<ChatMessage>): Promise<string> {
const maxGroup: FluentGroup<ChatMessage> = await iterable
async function getMostActiveUser(iterable: FluentAsyncIterable<ChatMessage>): Promise<string | undefined> {
const maxGroup = await iterable
.group(chatMessage => chatMessage.from) // group the messages by their sender
.max(chatMessage => chatMessage.values.count()); // find one of the groups which has the most messages
return maxGroup.key;
return maxGroup?.key;
}

async function hasUserSentEmptyMessage(iterable: FluentAsyncIterable<ChatMessage>, user: string): Promise<bool> {
Expand All @@ -186,22 +186,42 @@ async function hasUserSentEmptyMessage(iterable: FluentAsyncIterable<ChatMessage
async function createBackupSequential(iterable: FluentAsyncIterable<ChatMessage>): Promise<void> {
await iterable
.execute(chatMessage => console.log(`Backing up message ${chatMessage.id}.`)) // log progress w/o modifying the iterable
.forEachAsync(chatMessage => fetch(BACKUP_URL, { // execute the asynchronous backup operation against all elements one-by-one
.forEach(chatMessage => fetch(BACKUP_URL, { // execute the asynchronous backup operation against all elements one-by-one
method: 'post',
body: JSON.stringify(chatMessage),
headers: { 'Content-Type': 'application/json' },
}));
}

async function createBackupParallel(iterable: FluentAsyncIterable<ChatMessage>): Promise<void> {
const promises = iterable
await iterable
.execute(chatMessage => console.log(`Backing up message ${chatMessage.id}.`)) // log progress w/o modifying the iterable
.map(chatMessage => fetch(BACKUP_URL, { // translate all elements into a promise of their asynchronous backup operation
method: 'post',
body: JSON.stringify(chatMessage),
headers: { 'Content-Type': 'application/json' },
}));
await Promise.all(promises);
.map(chatMessage => {
const result = fetch(BACKUP_URL, { // translate all elements into a promise of their asynchronous backup operation
method: 'post',
body: JSON.stringify(chatMessage),
headers: { 'Content-Type': 'application/json' },
}).then(x => [x]);
return fluentAsync(result);
})
// Joins everything in parallel, generating an AsyncIterable with the results in the order of what yielded first
.flatMerge(
(error) => console.log(error) // This callback will be called whenever some of the fetch calls throws an error
)
.last();
}

function createBackupParallelV2(iterable: FluentAsyncIterable<ChatMessage>): Promise<Response[]> {
return iterable
.execute(chatMessage => console.log(`Backing up message ${chatMessage.id}.`)) // log progress w/o modifying the iterable
.waitAll(chatMessage => {
const result = fetch(BACKUP_URL, { // translate all elements into a promise of their asynchronous backup operation
method: 'post',
body: JSON.stringify(chatMessage),
headers: { 'Content-Type': 'application/json' },
}).then(x => [x]);
return fluentAsync(result);
});
}
```

Expand All @@ -212,7 +232,7 @@ You can see a list of many advanced examples for **fluent** clicking [here!](adv
#### Playing with Fibonacci generator

``` typescript
import { fluent } from '**fast-iterable**';
import { fluent } from '@codibre/fluent-iterable';

function* naiveFibonacci(): Iterable<number> {
yield 0;
Expand Down Expand Up @@ -259,7 +279,7 @@ console.log(
#### Playing with object arrays

``` typescript
import { fluent } from '**fast-iterable**';
import { fluent } from '@codibre/fluent-iterable';

enum Gender {
Male = 'Male',
Expand Down Expand Up @@ -342,7 +362,7 @@ console.log(

``` typescript
import fetch from 'node-fetch';
import { fluentAsync, Pager } from '**fast-iterable**';
import { fluentAsync, Pager } from '@cobidre/fluent-iterable';

interface Data {
id: number;
Expand Down Expand Up @@ -371,7 +391,7 @@ fluentAsync(depaginate(pager))
### Doing an inner join between two iterables:

``` typescript
import { fluent, identity } from '**fast-iterable**';
import { fluent, identity } from '@codibre/fluent-iterable';

const genders = [
{ code: 'm', description: 'male' },
Expand Down Expand Up @@ -420,18 +440,16 @@ fluent(genders)
#### Bonus: How to Scan DynamoDB like a pro

``` typescript
import { DynamoDB } from 'aws-sdk';
import { Key } from 'aws-sdk/clients/dynamodb';
import { depaginate, fluentAsync, Pager } from '**fast-iterable**';
import { DynamoDB, ScanInput, AttributeValue } from '@aws-sdk/client-dynamodb';
import { depaginate, fluentAsync, Pager } from '@codibre/fluent-iterable';

async function *scan<TData>(
input: DynamoDB.DocumentClient.ScanInput
input: ScanInput
): AsyncIterable<TData> {
const ddb = new DynamoDB.DocumentClient(..);
const pager: Pager<TData, Key> = async (token) => {
const ddb = new DynamoDB();
const pager: Pager<TData, Record<string, AttributeValue>> = async (token) => {
const result = await ddb
.scan(input)
.promise();
.scan(input);

return {
nextPageToken: result.LastEvaluatedKey,
Expand All @@ -444,10 +462,10 @@ async function *scan<TData>(

// and use it like this:

const productsParams: DynamoDB.DocumentClient.ScanInput = {
const productsParams: ScanInput = {
TableName : 'ProductTable',
FilterExpression : '#shoeName = :shoeName', // optional
ExpressionAttributeValues : {':shoeName' : 'YeeZys'}, // optional
ExpressionAttributeValues : {':shoeName' : { S: 'YeeZys' } }, // optional
ExpressionAttributeNames: { '#shoeName': 'name' } // optional
};

Expand All @@ -469,17 +487,18 @@ The solution used for this problems was 90% inspired in the [fraxken combine-asy
You can add custom methods to the FluentIterable and FluentAsyncIterable using the *extend* and *extendAsync* utilities. Here is a practical example of how to:

``` TypeScript
declare module '**fast-iterable**' {
import { extendAsync } from '../src';
import { extendAsync } from '../src';

declare module '@codibre/fluent-iterable' {

interface FluentAsyncIterable<T> {
myCustomIterableMethod(): FluentAsyncIterable<T>;
myCustomResolvingMethod(): PromiseLike<number>;
}

extendAsync.use('myCustomIterableMethod', (x) => someOperation(x));
extendAsync.use('myCustomResolvingMethod', (x) => someResolvingOperation(x));
}

extendAsync.use('myCustomIterableMethod', (x) => someOperation(x));
extendAsync.use('myCustomResolvingMethod', (x) => someResolvingOperation(x));
```

Notice that, when you import a code like the above, all the next created FluentAsyncIterable will have the declared methods, so use it with caution!
Expand Down
2 changes: 1 addition & 1 deletion advanced-examples/combining-iterables.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ const categories = [
},
{
parentId: 1,
id: 2
id: 2,
description: 'movies',
},
{
Expand Down
8 changes: 4 additions & 4 deletions advanced-examples/combining-sync-and-async-operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ fluentAsync(dataStream)
})
.map((x) => requestSomeData(x)) // Here, request is an async operation
.map((x) => otherRequest(parse(x))) // Parse is sync, otherRequest async, but that's okay, as long async/await is not used
.group('categoryId'); // A grouping operation is performed by the categoryId value
take(10); //Take just 10 items
.group('categoryId') // A grouping operation is performed by the categoryId value
.take(10); //Take just 10 items
```

The good thing here is that fluent works over the iterable and async iterable mechanism, and in that way, it is possible to chain operations that results in promises, and all the iterable sequence will be respected, instead of generating a lot of promises which will flood node queue.
Expand All @@ -26,8 +26,8 @@ fluent(data)
})
.mapAsync((x) => requestSomeData(x)) // From that point, the FluentIterable is transformed in a FluentAsyncIterable, and you don't need the async suffix any longer
.map((x) => otherRequest(parse(x))) // Parse is sync, otherRequest async, but that's okay, as long async/await is not used
.group('categoryId'); // A grouping operation is performed by the categoryId value
take(10); //Take just 10 items
.group('categoryId') // A grouping operation is performed by the categoryId value
.take(10); //Take just 10 items
```

[Go back](README.md)
2 changes: 1 addition & 1 deletion advanced-examples/iterate-chunks.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ async function doTheWork(request: MyRequest) {
// Also notice that the item type is an array, because we're talking about a strem of chunked data
await fluent<SomeData[]>(getChunkedData(request))
.flatMap() // With flat map, the stream of SomeData[] will unwind and be transformed in a stream of SomeData
.forEach(doEachWork); // You could also just return the iterable without resolve it and del with the data somewhere else, for decoupling purpose
.forEachAsync(doEachWork); // You could also just return the iterable without resolve it and del with the data somewhere else, for decoupling purpose
}
```

Expand Down
4 changes: 2 additions & 2 deletions advanced-examples/nested-flattening-with-back-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ fluent(invoices)
value: x.value
}))
)
)
);
```

Those nested fluent operation can become very messy when your business rule starts to become more complicated. Luckily, fluent offers you a powerful operation called **flatJoin**. With flatJoin, you can automatically obtain the items of each nested level with just one operation, like that:
Expand All @@ -59,7 +59,7 @@ fluent(invoices)
invoiceId: invoice.id,
itemId: items.id,
value: taxes.value
}))
}));
```

Look that the root item of each operation is added to the result in the property named with the symbol **tail**, that you can import from fluent. You can also get the last items from symbol **head**, but in this case it'll be the same value as the property **taxes**.
Expand Down
5 changes: 3 additions & 2 deletions advanced-examples/operators-and-resolvers.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Instead, you need to do as follow:
```ts
const origin = [1, 2, 3];
const evens = fluent(it).filter((x) => x % 2 === 0);
const odds = fluent(it().filter((x) => x % 2 !== 0)
const odds = fluent(it).filter((x) => x % 2 !== 0)
```

That's how iterables works: you can't guarantee a second iteration over the same iterable, and that's needed for some core behaviors of it. Also, an iterable is not guaranteed to support multiples iterations over it. An array supports it, but it's not the rule, it's an exception. Iterables created using the [generator pattern](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Generator) can only be iterated once, for example. Because of this, fluent-iterable don't give support to parallel iterations deriving from the same node and must be used as a straight forward fluent tool.
Expand All @@ -44,7 +44,8 @@ The operations are executed in the chaining order for each iterable item when a
const test = fluent([1, 2, 3])
.repeat(3)
.withIndex()
.flatMap(({ value, idx }) => fluent(value).map((x) => x * (idx + 1)));
.partition(3)
.flatMap((x) => fluent(x).map(({ value, idx }) => value * (idx + 1)));

// Here, the iteration happens
for (const i of test) {
Expand Down
8 changes: 2 additions & 6 deletions src/utils/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -252,9 +252,7 @@ function getAverageStepper() {

return wrapper;
}
function getItemToAssure<
F extends Function | FluentIterable<any> | FluentAsyncIterable<any>,
>(f: F): any {
function getItemToAssure<F extends Function | AnyIterable<any>>(f: F): any {
return typeof f === 'function' && !(f instanceof FluentClass)
? (...args: any[]) => (f as CallableFunction)(...args)
: f;
Expand All @@ -272,9 +270,7 @@ function getItemToAssure<
*
* @param f the function to assure order
*/
function assureOrder<
F extends Function | FluentIterable<any> | FluentAsyncIterable<any>,
>(f: F): F {
function assureOrder<F extends Function | AnyIterable<any>>(f: F): F {
const result = getItemToAssure(f);
result[orderAssured] = 1;
return result;
Expand Down
21 changes: 21 additions & 0 deletions test/unit/fluent-async.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1090,6 +1090,27 @@ describe('fluent async iterable', () => {
expect(resolved).toBe(10);
expect(result).toEqual([1, 2, 3, 4, 5, 6, 7, 8, 9, 10]);
});

it('should resolve promises in parallel', async () => {
let resolved = 0;
let current = 0;

const promise = fluentAsync(fluent(interval(1, 10)).toAsync()).waitAll(
(x) =>
new Promise(async (resolve) => {
await delay(1);
resolved++;
resolve(x + current);
await delay(1);
current = 1;
}),
);

expect(resolved).toBe(0);
const result = await promise;
expect(resolved).toBe(10);
expect(result).toEqual([1, 2, 3, 4, 5, 6, 7, 8, 9, 10]);
});
});

it('should be identifiable as fluent async', () => {
Expand Down
13 changes: 12 additions & 1 deletion test/unit/order-assuring.spec.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { fluent } from '../../src';
import { fluent, o } from '../../src';
import { orderAssured } from '../../src/types-internal';

describe('order assuring', () => {
Expand Down Expand Up @@ -36,6 +36,17 @@ describe('order assuring', () => {

expect(mp[orderAssured]).not.toBeDefined();
});
it('should keep an assured ascending order through filter and takeWhile operations, but not through a mapper when using assurer directly on iterable', () => {
const it: any = fluent(o([1, 2, 3])).filter((x) => x > 1);

expect(it[orderAssured]).toBeDefined();
const tw = it.takeWhile((x: any) => x < 3);

expect(tw[orderAssured]).toBeDefined();
const mp: any = tw.map((x: any) => x * 2);

expect(mp[orderAssured]).not.toBeDefined();
});
it('should keep an assured descending order through filter and takeWhile operations, but not through a mapper', () => {
const it: any = fluent([1, 2, 3])
.od()
Expand Down

0 comments on commit 66fe392

Please sign in to comment.