Better onboarding docs

[dherman]

This is a placeholder for now, until we have a concrete plan.

We should take a fresh look at our onboarding experience, particularly at how we can make a smooth path for newcomers to go through the guides and feel productive as fast as possible.

---

Thoughts about different kinds of documentation for Neon:

1. API docs -- docs.rs is sufficient
2. Recipes -- the examples repo and what we have today on the web site
3. Guidebook -- a more pedagogically laid out tour through the concepts of Neon

This issue is for the third type, the guidebook.

---

Breaking down the work here, it should start with:

• List the main concepts of Neon out
  • handles and memory management
  • contexts
  • some of the key traits like Value and Object
  • concurrency and threads
  • handling native data, i.e. with JsBox
  • some key background concepts that we might link out to other resources on:
    • the JS GC
    • the main thread and event loop
    • key Rust concurrency concepts like Send and Sync
    • key Rust memory management concepts like ownership and borrowing
• Topologically sort the concepts so we can sequence them in pedagogical order
• Pick a book technology (gitbook, etc)
• Create a repo with a book skeleton based on the sequence of concepts

[timetocode]

I'm new to rust, but one particular realm which is unclear to me from following the current examples is how to mix state between rust and node.

There's a pattern across the examples of creating classes and functions followed by exporting them.

cx.export_function('foo', foo) etc

So that (and the equivalent for classes) show me quite well how to write some rust logic, convert my type into something javascript understands, and return it so that I can call the logic from javascript. But that's just logic, and for me it is unclear how to deal with state in general unless the state is entirely in javascript.

For example, I would love to know how to do something like this:

fn add_javascript_object_to_rust_collection(mut cx: FunctionContext, &mut state: State) {
    let object = /* get it from the function context */
    state.objects.push(object); // this is a RUST collection
}

register_module!(mut cx, {
    // example of some state on the rust-side of the application
    let mut state = State {
        objects: Vec::new()
    };
    // seems logical but not valid, this is the hard part
    cx.export_function("add", |cx| add_javascript_object_to_rust_collection(cx, &mut state))?;
    // stuff like this works though
    cx.export_function("foo", foo);
    Ok(())
});

How does one have a function that uses state from js and rust at the same time? Writing the function body of add_javascript_object_to_rust_collection is fairly straight forward after reading the docs. But it's not clear to me how to export it, though maybe it's just because I don't know enough rust.

A few other things that I found myself wondering while trying to learn neon:

• can functions be added to JsObjects the same way a property can be set?
• what exactly is the difference been a rust function that returns a JsResult and a javascript function, some sort of magic seems to happen at export_function which I didn't understand
• how to export a plain object (not a module, nor a class)
• how would threads work?
• potential of fork/childprocess vs threads
• perf implications of passing state in either direction
• examples of i/o and timers with setImmediate/setInterval/setTimeout (unless these simply don't make sense for some reason, and one should be using something else on the rust side..?)

[mainrs]

@timetocode Did you figure out how to do state inside of neon modules? And I would be interested in the thread question as well. Can I just spin up Tokio and spawn some tasks without problems for example?

[kjvalencik]

@sirwindfield With the legacy backend, classes can be used to maintain state. With the N-API backend, JsBox can maintain state.

Both of those use the JavaScript garbage collector to manage lifecycle and the state is passed back and forth across FFI as opaque pointers.

Global state is also an option. For example, for tokio, it might make sense to put the tokio runtime in a global static instead of passing into all Neon functions that need it.

Included below is a simple example of using JsBox for state and tokio to implement an async counter.

use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::Arc;

use neon::prelude::*;
use once_cell::sync::Lazy;
use tokio::runtime::Runtime;

static RUNTIME: Lazy<Runtime> = Lazy::new(|| Runtime::new().unwrap());

struct Counter {
    count: AtomicUsize,
    queue: EventQueue,
}

impl Finalize for Counter {}

impl Counter {
    fn new(queue: EventQueue) -> Arc<Self> {
        Arc::new(Self {
            count: AtomicUsize::new(0),
            queue,
        })
    }

    async fn fetch_add(&self, val: usize, order: Ordering) -> usize {
        // Sleep for demonstration purposes only.
        tokio::time::sleep(std::time::Duration::from_millis(100)).await;
        self.count.fetch_add(val, order)
    }
}

fn counter_new(mut cx: FunctionContext) -> JsResult<JsBox<Arc<Counter>>> {
    let queue = cx.queue();
    let counter = Counter::new(queue);

    Ok(cx.boxed(counter))
}

fn counter_incr(mut cx: FunctionContext) -> JsResult<JsUndefined> {
    let counter = Arc::clone(&&cx.argument::<JsBox<Arc<Counter>>>(0)?);
    let cb = cx.argument::<JsFunction>(1)?.root(&mut cx);

    RUNTIME.spawn(async move {
        let result = counter.fetch_add(1, Ordering::SeqCst).await;

        counter.queue.send(move |mut cx| {
            let cb = cb.into_inner(&mut cx);
            let this = cx.undefined();
            let args = vec![
                cx.undefined().upcast::<JsValue>(),
                cx.number(result as f64).upcast(),
            ];

            cb.call(&mut cx, this, args)?;

            Ok(())
        });
    });

    Ok(cx.undefined())
}

#[neon::main]
fn main(mut cx: ModuleContext) -> NeonResult<()> {
    cx.export_function("counterNew", counter_new)?;
    cx.export_function("counterIncr", counter_incr)?;

    Ok(())
}

"use strict";

const { promisify } = require("util");

const { counterNew, counterIncr } = require("./index.node");

const counterIncrAsync = promisify(counterIncr);

class Counter {
    constructor() {
        this.counter = counterNew();
    }

    async incr() {
        return counterIncrAsync(this.counter);
    }
}

async function run() {
    const counter = new Counter();

    console.log(await counter.incr());
    console.log(await counter.incr());

    const [a, b] = await Promise.all([counter.incr(), counter.incr()]);

    console.log(a, b);
}

run().catch((err) => {
    throw err;
});

There are a couple of interesting things in this code.

• neon does not currently provide Promise. The code uses callbacks and is promisified with glue code in JavaScript.
• neon does not provide classes with the N-API backend. There is a glue code in JavaScript to create a class around a JsBox. All of the Neon/Rust takes the JsBox as the first argument.
• Tokio is initialized with a once_cell::sync::Lazy. There are a few different ways to accomplish this. It's also possible to put it inside the JsBox state. Additionally, an init method could be used to initialize it. Keep in mind that initialization may need to occur in each module context to support WebWorkers.

[dherman]

Update:

• With API docs improvements #942, the API docs are in good shape for 1.0.
• With Revised docs, milestone 1 website#215 and subsequent cleanups, the main guides are in good shape for 1.0.
• With First example, CI and README examples#72 and subsequent additions, the examples repo is updated for modern Neon and in good shape for 1.0.

There's always more to do to improve docs, but since this issue isn't concretely scoped, I'm going to close it for now. We can open future issues for specific future docs goals.