[](https://github.com/finos/perspective/actions/workflows/build.yaml)
@@ -13,8 +17,7 @@
Perspective is an interactive analytics and data visualization component,
which is especially well-suited for large and/or streaming
datasets. Use it to create user-configurable reports, dashboards, notebooks and
-applications, then deploy stand-alone in the browser, or in concert with Python
-and/or [Jupyterlab](https://jupyterlab.readthedocs.io/en/stable/).
+applications.
### Features
@@ -28,35 +31,20 @@ and/or [Jupyterlab](https://jupyterlab.readthedocs.io/en/stable/).
- A framework-agnostic User Interface packaged as a
[Custom Element](https://developer.mozilla.org/en-US/docs/Web/Web_Components/Using_custom_elements),
powered either in-browser via WebAssembly or virtually via WebSocket server
- (Python/Node).
+ (Python/Node/Rust).
- A [JupyterLab](https://jupyter.org/) widget and Python client library, for
interactive data analysis in a notebook, as well as _scalable_ production
- [Voila](https://github.com/voila-dashboards/voila) applications.
+ applications.
### Documentation
- [Project Site](https://perspective.finos.org/)
-- JavaScript (NPM)
- - [`@finos/perspective-viewer`, JavaScript UI API](https://docs.rs/perspective-viewer/latest/perspective_viewer/)
- - [`@finos/perspective`, JavaScript Client/Server API](https://docs.rs/perspective-js/latest/perspective_js/)
- - [`Table` API](https://docs.rs/perspective-js/latest/perspective_js/struct.Table.html)
- - [`View` API](https://docs.rs/perspective-js/latest/perspective_js/struct.View.html)
- - [Installation Guide](https://docs.rs/perspective-js/latest/perspective_js/#installation)
-- Python (PyPI)
- - [`perspective-python`, Python Client/Server API](https://docs.rs/perspective-python/latest/perspective_python/)
- - [`PerspectiveWidget` Jupyter Plugin](https://docs.rs/perspective-python/latest/perspective_python/#perspectivewidget)
- - [`Table` API](https://docs.rs/perspective-python/latest/perspective_python/struct.Table.html)
- - [`View` API](https://docs.rs/perspective-python/latest/perspective_python/struct.View.html)
-- Rust (Crates.io)
- - [`perspective`, Rust API](https://docs.rs/perspective-rs/latest/perspective_rs/)
- - [`perspective-client`, Rust Client API](https://docs.rs/perspective-client/latest/perspective_client/)
- - [`perspective-server`, Rust Server API](https://docs.rs/perspective-server/latest/perspective_server/)
- - [`Table` API](https://docs.rs/perspective-client/latest/perspective_client/struct.Table.html)
- - [`View` API](https://docs.rs/perspective-client/latest/perspective_client/struct.View.html)
-- Appendix
- - [Data Binding](https://docs.rs/perspective-server/latest/perspective_server/)
- - [Expression Columns](https://docs.rs/perspective-client/latest/perspective_client/config/expressions/)
+- [User Guide](https://perspective.finos.org/guide/)
+- [`@finos/perspective`, JavaScript Client API](https://docs.rs/perspective-js/latest/perspective_js/)
+- [`@finos/perspective-viewer`, JavaScript UI API](https://docs.rs/perspective-viewer/latest/perspective_viewer/)
+- [`perspective-python`, Python API](https://docs.rs/perspective-python/latest/perspective_python/)
+- [`perspective`, Rust API](https://docs.rs/perspective-rs/latest/perspective_rs/)
### Examples
diff --git a/cpp/perspective/CMakeLists.txt b/cpp/perspective/CMakeLists.txt
index a2ae272b0c..d636739879 100644
--- a/cpp/perspective/CMakeLists.txt
+++ b/cpp/perspective/CMakeLists.txt
@@ -202,16 +202,16 @@ if(NOT DEFINED PSP_WASM_EXCEPTIONS AND NOT PSP_PYTHON_BUILD)
endif()
if(PSP_WASM_BUILD)
- # ###################
+ ####################
# EMSCRIPTEN BUILD #
- # ###################
+ ####################
set(CMAKE_EXECUTABLE_SUFFIX ".js")
list(APPEND CMAKE_PREFIX_PATH /usr/local)
set(EXTENDED_FLAGS " \
-Wall \
-fcolor-diagnostics \
- ")
+ ")
if(CMAKE_BUILD_TYPE_LOWER STREQUAL debug)
# Pyodide DEBUG block
@@ -232,14 +232,8 @@ if(PSP_WASM_BUILD)
endif()
else()
set(OPT_FLAGS " -O3 -g0 ")
- if(NOT PSP_PYODIDE)
- set(OPT_FLAGS " \
- ${OPT_FLAGS} \
- ")
- endif()
-
if (PSP_WASM_EXCEPTIONS)
- set(OPT_FLAGS "${OPT_FLAGS} -fwasm-exceptions ")
+ set(OPT_FLAGS "${OPT_FLAGS} -fwasm-exceptions -flto --emit-tsd=perspective-server.d.ts ")
endif()
endif()
elseif(PSP_CPP_BUILD OR PSP_PYTHON_BUILD)
@@ -529,7 +523,6 @@ else()
-s DYNAMIC_EXECUTION=0 \
-s POLYFILL=0 \
-s EXPORT_NAME=\"load_perspective\" \
- -s MAXIMUM_MEMORY=16gb \
-s ERROR_ON_UNDEFINED_SYMBOLS=0 \
-s NODEJS_CATCH_EXIT=0 \
-s NODEJS_CATCH_REJECTION=0 \
diff --git a/cpp/perspective/src/cpp/table.cpp b/cpp/perspective/src/cpp/table.cpp
index 5f6801a1ec..a8d340976c 100644
--- a/cpp/perspective/src/cpp/table.cpp
+++ b/cpp/perspective/src/cpp/table.cpp
@@ -905,7 +905,7 @@ Table::update_cols(const std::string_view& data, std::uint32_t port_id) {
}
t_uindex nrows = 0;
- for (const auto& it : document.GetObject()) {
+ for (const auto& it : document.GetObj()) {
if (!it.value.IsArray()) {
PSP_COMPLAIN_AND_ABORT("Malformed column")
}
@@ -940,14 +940,14 @@ Table::update_cols(const std::string_view& data, std::uint32_t port_id) {
auto schema = data_table.get_schema();
- if (is_implicit && !document.GetObject().HasMember("__INDEX__")) {
+ if (is_implicit && !document.GetObj().HasMember("__INDEX__")) {
for (std::uint32_t ii = 0; ii < nrows; ii++) {
psp_pkey_col->set_nth(ii, (m_offset + ii) % m_limit);
}
}
// 3.) Fill table
- for (const auto& column : document.GetObject()) {
+ for (const auto& column : document.GetObj()) {
t_uindex ii = 0;
std::string_view col_name = column.name.GetString();
if (std::string_view{column.name.GetString()} == "__INDEX__") {
@@ -1001,7 +1001,7 @@ Table::from_cols(
t_uindex nrows = 0;
// https://github.com/Tencent/rapidjson/issues/1994
- for (const auto& it : document.GetObject()) {
+ for (const auto& it : document.GetObj()) {
if (!it.value.IsArray()) {
PSP_COMPLAIN_AND_ABORT("Malformed column")
}
@@ -1052,7 +1052,7 @@ Table::from_cols(
const auto& psp_okey_col = data_table.get_column("psp_okey");
// 3.) Fill table
- for (const auto& col : document.GetObject()) {
+ for (const auto& col : document.GetObj()) {
t_uindex ii = 0;
const auto& col_name = col.name.GetString();
LOG_DEBUG(
@@ -1153,7 +1153,7 @@ Table::update_rows(const std::string_view& data, std::uint32_t port_id) {
}
// col_count = m_column_names.size();
- for (const auto& it : row.GetObject()) {
+ for (const auto& it : row.GetObj()) {
std::shared_ptr col;
std::string_view col_name = it.name.GetString();
if (std::string_view{it.name.GetString()} == "__INDEX__") {
@@ -1246,12 +1246,12 @@ Table::from_rows(
[&]() {
for (const auto& row : document.GetArray()) {
- for (const auto& col : row.GetObject()) {
+ for (const auto& col : row.GetObj()) {
columns_seen.insert(col.name.GetString());
}
// https://github.com/Tencent/rapidjson/issues/1994
- for (const auto& col : row.GetObject()) {
+ for (const auto& col : row.GetObj()) {
if (col.name.GetString() == index) {
is_implicit = false;
}
@@ -1307,7 +1307,7 @@ Table::from_rows(
// 3.) Fill table
for (const auto& row : document.GetArray()) {
- for (const auto& it : row.GetObject()) {
+ for (const auto& it : row.GetObj()) {
auto col = data_table.get_column(it.name.GetString());
const auto* col_name = it.name.GetString();
const auto& cell = it.value;
@@ -1399,7 +1399,7 @@ Table::update_ndjson(const std::string_view& data, std::uint32_t port_id) {
psp_pkey_col->set_nth(ii, (ii + m_offset) % m_limit);
}
- for (const auto& it : document.GetObject()) {
+ for (const auto& it : document.GetObj()) {
std::shared_ptr col;
std::string_view col_name = it.name.GetString();
if (std::string_view{it.name.GetString()} == "__INDEX__") {
@@ -1486,12 +1486,12 @@ Table::from_ndjson(
// enhancement we do for regular JSON. For now this only checks the first
// row.
[&]() {
- for (const auto& col : document.GetObject()) {
+ for (const auto& col : document.GetObj()) {
columns_seen.insert(col.name.GetString());
}
// https://github.com/Tencent/rapidjson/issues/1994
- for (const auto& col : document.GetObject()) {
+ for (const auto& col : document.GetObj()) {
if (col.name.GetString() == index) {
is_implicit = false;
}
@@ -1555,7 +1555,7 @@ Table::from_ndjson(
// 3.) Fill table
bool is_finished = false;
while (!is_finished) {
- for (const auto& it : document.GetObject()) {
+ for (const auto& it : document.GetObj()) {
auto col = data_table.get_column(it.name.GetString());
const auto* col_name = it.name.GetString();
const auto& cell = it.value;
diff --git a/docs/book.toml b/docs/book.toml
new file mode 100644
index 0000000000..2e5fc992ad
--- /dev/null
+++ b/docs/book.toml
@@ -0,0 +1,42 @@
+# ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
+# ┃ ██████ ██████ ██████ █ █ █ █ █ █▄ ▀███ █ ┃
+# ┃ ▄▄▄▄▄█ █▄▄▄▄▄ ▄▄▄▄▄█ ▀▀▀▀▀█▀▀▀▀▀ █ ▀▀▀▀▀█ ████████▌▐███ ███▄ ▀█ █ ▀▀▀▀▀ ┃
+# ┃ █▀▀▀▀▀ █▀▀▀▀▀ █▀██▀▀ ▄▄▄▄▄ █ ▄▄▄▄▄█ ▄▄▄▄▄█ ████████▌▐███ █████▄ █ ▄▄▄▄▄ ┃
+# ┃ █ ██████ █ ▀█▄ █ ██████ █ ███▌▐███ ███████▄ █ ┃
+# ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫
+# ┃ Copyright (c) 2017, the Perspective Authors. ┃
+# ┃ ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌ ┃
+# ┃ This file is part of the Perspective library, distributed under the terms ┃
+# ┃ of the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0). ┃
+# ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
+
+[book]
+authors = ["Andrew Stein"]
+language = "en"
+multilingual = false
+src = "md"
+title = "Perspective"
+
+[build]
+build-dir = "static/guide"
+
+[output.html]
+# theme = "my-theme"
+# default-theme = "light"
+# preferred-dark-theme = "navy"
+# smart-punctuation = true
+# mathjax-support = false
+copy-fonts = true
+git-repository-url = "https://github.com/finos/perspective"
+git-repository-icon = "fa-github"
+site-url = "https://perspective.finos.org/guide/"
+additional-css = [
+ "md/perspective.css",
+ "node_modules/@finos/perspective-viewer/dist/css/themes.css",
+]
+# additional-js = []
+# no-section-label = false
+# edit-url-template = "https://github.com/rust-lang/mdBook/edit/master/guide/{path}"
+# site-url = "/guide/"
+# cname = "myproject.rs"
+# input-404 = "not-found.md"
diff --git a/docs/docusaurus.config.js b/docs/docusaurus.config.js
index 5547069fbe..fae23695ec 100644
--- a/docs/docusaurus.config.js
+++ b/docs/docusaurus.config.js
@@ -119,76 +119,20 @@ const config = {
label: "Docs",
items: [
{
- type: "html",
- value: "JavaScript",
- },
- {
- href: "https://docs.rs/perspective-viewer/latest/perspective_viewer/",
- label: "`@finos/perspective-viewer` JavaScript UI API",
- },
- {
- href: "https://docs.rs/perspective-js/latest/perspective_js/",
- label: "`@finos/perspective` JavaScript Client/Server API",
- },
- {
- href: "https://docs.rs/perspective-js/latest/perspective_js/struct.Table.html",
- label: "`Table` API",
- },
- {
- href: "https://docs.rs/perspective-js/latest/perspective_js/struct.View.html",
- label: "`View` API",
- },
- {
- href: "https://docs.rs/perspective-js/latest/perspective_js/#installation",
- label: "Installation Guide",
+ href: "/guide/",
+ label: "User Guide",
},
{
type: "html",
- value: "Python",
- },
- {
- href: "https://docs.rs/perspective-python/latest/perspective_python/",
- label: "`perspective-python` Python Client/Server API",
- },
- {
- href: "https://docs.rs/perspective-python/latest/perspective_python/#perspectivewidget",
- label: "`PerspectiveWidget` Jupyter Plugin",
- },
- {
- href: "https://docs.rs/perspective-python/latest/perspective_python/struct.Table.html",
- label: "`Table` API",
- },
- {
- href: "https://docs.rs/perspective-python/latest/perspective_python/struct.View.html",
- label: "`View` API",
+ value: `@finos/perspective JavaScript API`,
},
{
type: "html",
- value: "Rust",
- },
- {
- href: "https://docs.rs/perspective/latest/perspective/",
- label: "`perspective`, Rust API",
- },
- {
- href: "https://docs.rs/perspective-client/latest/perspective_client/struct.Table.html",
- label: "`Table` API",
- },
- {
- href: "https://docs.rs/perspective-client/latest/perspective_client/struct.View.html",
- label: "`View` API",
+ value: `perspective-python Python API`,
},
{
type: "html",
- value: "Appendix",
- },
- {
- href: "https://docs.rs/perspective-server/latest/perspective_server/",
- label: "Data Binding",
- },
- {
- href: "https://docs.rs/perspective-client/latest/perspective_client/config/expressions/",
- label: "Expression Columns",
+ value: `perspective Rust API`,
},
],
},
diff --git a/docs/md/SUMMARY.md b/docs/md/SUMMARY.md
new file mode 100644
index 0000000000..3a161513cc
--- /dev/null
+++ b/docs/md/SUMMARY.md
@@ -0,0 +1,60 @@
+# Summary
+
+[What is Perspective](./perspective.md)
+
+# Overview
+
+- [Data Architecture](./explanation/architecture.md)
+ - [Client-only](./explanation/architecture/client_only.md)
+ - [Client/Server replicated](./explanation/architecture/client_server.md)
+ - [Server only](./explanation/architecture/server_only.md)
+- [`Table`](./explanation/table.md)
+ - [Construct an empty `Table` from a schema](./explanation/table/constructing_schema.md)
+ - [Schema and column types](./explanation/table/schema.md)
+ - [Loading data](./explanation/table/loading_data.md)
+ - [`index` and `limit` options](./explanation/table/options.md)
+ - [`update()` and `remove()` streaming methods](./explanation/table/update_and_remove.md)
+ - [`clear()` and `replace()` start-over methods](./explanation/table/clear_and_replace.md)
+- [`View`](./explanation/view.md)
+ - [Querying data](./explanation/view/querying.md)
+ - [`group_by`](./explanation/view/config/group_by.md)
+ - [`split_by`](./explanation/view/config/split_by.md)
+ - [`aggregates`](./explanation/view/config/aggregates.md)
+ - [`columns`](./explanation/view/config/columns.md)
+ - [`sort`](./explanation/view/config/sort.md)
+ - [`filter`](./explanation/view/config/filter.md)
+ - [`expressions`](./explanation/view/config/expressions.md)
+ - [Flattening a View into a Table](./explanation/view/config/flattening.md)
+- [JavaScript](./explanation/javascript.md)
+ - [Module Structure](./explanation/javascript_module_structure.md)
+ - [Build options](./explanation/javascript_builds.md)
+- [Python](./explanation/python.md)
+
+# Getting Started
+
+- [Rust](./how_to/rust.md)
+- [JavaScript](./how_to/javascript.md)
+ - [Installation via NPM](./how_to/javascript/installation.md)
+ - [Importing with or without a bundler](./how_to/javascript/importing.md)
+ - [`perspective` data engine library](./how_to/javascript/worker.md)
+ - [Serializing data](./how_to/javascript/serializing.md)
+ - [Cleaning up resources](./how_to/javascript/deleting.md)
+ - [Hosting a `WebSocketServer` in Node.js](./how_to/javascript/nodejs_server.md)
+ - [`perspective-viewer` Custom Element library](./how_to/javascript/viewer.md)
+ - [Theming](./how_to/javascript/theming.md)
+ - [Loading data from a `Table`](./how_to/javascript/loading_data.md)
+ - [Loading data from a virtual `Table`](./how_to/javascript/loading_virtual_data.md)
+ - [Saving and restoring UI state](./how_to/javascript/save_restore.md)
+ - [Listening for events](./how_to/javascript/events.md)
+- [Python](./how_to/python.md)
+ - [Installation](./how_to/python/installation.md)
+ - [Loading data into a `Table`](./how_to/python/table.md)
+ - [Callbacks and events](./how_to/python/callbacks.md)
+ - [Multithreading](./how_to/python/multithreading.md)
+ - [Hosting a WebSocket server](./how_to/python/websocket.md)
+ - [`PerspectiveWidget` for JupyterLab](./how_to/python/jupyterlab.md)
+ - [Tutorial: A `tornado` server with virtual `perspective-viewer`](./tutorials/python/tornado.md)
+
+# API
+
+- [Crate documentation on `docs.rs` ](./api_reference.md)
diff --git a/docs/md/api_reference.md b/docs/md/api_reference.md
new file mode 100644
index 0000000000..c5836cf30b
--- /dev/null
+++ b/docs/md/api_reference.md
@@ -0,0 +1,9 @@
+# API Reference
+
+Perspective's complete API is hosted on `docs.rs`:
+
+- [`perspective-client`](https://docs.rs/perspective-client/latest/perspective_client/index.html)
+ covers `Table` and `View` data engine API methods common for Rust,
+ JavaScript and Python.
+- [`perspective-rs`](https://docs.rs/perspective-client/latest/perspective_client/index.html)
+ adds Rust-specific documentation for the Rust crate entrypoint.
diff --git a/docs/md/explanation.md b/docs/md/explanation.md
new file mode 100644
index 0000000000..b1303b40c8
--- /dev/null
+++ b/docs/md/explanation.md
@@ -0,0 +1 @@
+# Explanation
diff --git a/docs/md/explanation/architecture.md b/docs/md/explanation/architecture.md
new file mode 100644
index 0000000000..2739c61fd4
--- /dev/null
+++ b/docs/md/explanation/architecture.md
@@ -0,0 +1,15 @@
+# Data Architecture
+
+Application developers can choose from
+[Client (WebAssembly)](./architecture/client_only.md),
+[Server (Python/Node)](./architecture/server_only.md) or
+[Client/Server Replicated](./architecture/client_server.md) designs to bind
+data, and a web application can use one or a mix of these designs as needed. By
+serializing to Apache Arrow, tables are duplicated and synchronized across
+runtimes efficiently.
+
+Perspective is a multi-language platform. The examples in this section use
+Python and JavaScript as an example, but the same general principles apply to
+any `Client`/`Server` combination.
+
+
diff --git a/rust/perspective-server/docs/architecture.dot b/docs/md/explanation/architecture/architecture.dot
similarity index 100%
rename from rust/perspective-server/docs/architecture.dot
rename to docs/md/explanation/architecture/architecture.dot
diff --git a/rust/perspective-server/docs/architecture.sub1.dot b/docs/md/explanation/architecture/architecture.sub1.dot
similarity index 100%
rename from rust/perspective-server/docs/architecture.sub1.dot
rename to docs/md/explanation/architecture/architecture.sub1.dot
diff --git a/rust/perspective-server/docs/architecture.sub1.svg b/docs/md/explanation/architecture/architecture.sub1.svg
similarity index 100%
rename from rust/perspective-server/docs/architecture.sub1.svg
rename to docs/md/explanation/architecture/architecture.sub1.svg
diff --git a/rust/perspective-server/docs/architecture.sub2.dot b/docs/md/explanation/architecture/architecture.sub2.dot
similarity index 100%
rename from rust/perspective-server/docs/architecture.sub2.dot
rename to docs/md/explanation/architecture/architecture.sub2.dot
diff --git a/rust/perspective-server/docs/architecture.sub2.svg b/docs/md/explanation/architecture/architecture.sub2.svg
similarity index 100%
rename from rust/perspective-server/docs/architecture.sub2.svg
rename to docs/md/explanation/architecture/architecture.sub2.svg
diff --git a/rust/perspective-server/docs/architecture.sub3.dot b/docs/md/explanation/architecture/architecture.sub3.dot
similarity index 100%
rename from rust/perspective-server/docs/architecture.sub3.dot
rename to docs/md/explanation/architecture/architecture.sub3.dot
diff --git a/rust/perspective-server/docs/architecture.sub3.svg b/docs/md/explanation/architecture/architecture.sub3.svg
similarity index 100%
rename from rust/perspective-server/docs/architecture.sub3.svg
rename to docs/md/explanation/architecture/architecture.sub3.svg
diff --git a/rust/perspective-server/docs/architecture.svg b/docs/md/explanation/architecture/architecture.svg
similarity index 100%
rename from rust/perspective-server/docs/architecture.svg
rename to docs/md/explanation/architecture/architecture.svg
diff --git a/docs/md/explanation/architecture/client_only.md b/docs/md/explanation/architecture/client_only.md
new file mode 100644
index 0000000000..7db35eca0e
--- /dev/null
+++ b/docs/md/explanation/architecture/client_only.md
@@ -0,0 +1,39 @@
+# Client-only
+
+
+
+_For static datasets, datasets provided by the user, and simple server-less and
+read-only web applications._
+
+In this design, Perspective is run as a client Browser WebAssembly library, the
+dataset is downloaded entirely to the client and all calculations and UI
+interactions are performed locally. Interactive performance is very good, using
+WebAssembly engine for near-native runtime plus WebWorker isolation for parallel
+rendering within the browser. Operations like scrolling and creating new views
+are responsive. However, the entire dataset must be downloaded to the client.
+Perspective is not a typical browser component, and datset sizes of 1gb+ in
+Apache Arrow format will load fine with good interactive performance!
+
+Horizontal scaling is a non-issue, since here is no concurrent state to scale,
+and only uses client-side computation via WebAssembly client. Client-only
+perspective can support as many concurrent users as can download the web
+application itself. Once the data is loaded, no server connection is needed and
+all operations occur in the client browser, imparting no additional runtime cost
+on the server beyond initial load. This also means updates and edits are local
+to the browser client and will be lost when the page is refreshed, unless
+otherwise persisted by your application.
+
+As the client-only design starts with creating a client-side Perspective
+`Table`, data can be provided by any standard web service in any Perspective
+compatible format (JSON, CSV or Apache Arrow).
+
+#### Javascript client
+
+```javascript
+const worker = await perspective.worker();
+const table = await worker.table(csv);
+
+const viewer = document.createElement("perspective-viewer");
+document.body.appendChild(viewer);
+await viewer.load(table);
+```
diff --git a/docs/md/explanation/architecture/client_server.md b/docs/md/explanation/architecture/client_server.md
new file mode 100644
index 0000000000..f6df4408eb
--- /dev/null
+++ b/docs/md/explanation/architecture/client_server.md
@@ -0,0 +1,58 @@
+# Client/Server replicated
+
+
+
+_For medium-sized, real-time, synchronized and/or editable data sets with many
+concurrent users._
+
+The dataset is instantiated in-memory with a Python or Node.js Perspective
+server, and web applications create duplicates of these tables in a local
+WebAssembly client in the browser, synchonized efficiently to the server via
+Apache Arrow. This design scales well with additional concurrent users, as
+browsers only need to download the initial data set and subsequent update
+deltas, while operations like scrolling, pivots, sorting, etc. are performed on
+the client.
+
+Python servers can make especially good use of additional threads, as
+Perspective will release the GIL for almost all operations. Interactive
+performance on the client is very good and identical to client-only
+architecture. Updates and edits are seamlessly synchonized across clients via
+their virtual server counterparts using websockets and Apache Arrow.
+
+#### Python and Tornado server
+
+```python
+from perspective import Server, PerspectiveTornadoHandler
+
+server = Server()
+client = server.new_local_client()
+client.table(csv, name="my_table")
+routes = [(
+ r"/websocket",
+ perspective.handlers.tornado.PerspectiveTornadoHandler,
+ {"perspective_server": server},
+)]
+
+app = tornado.web.Application(routes)
+app.listen(8080)
+loop = tornado.ioloop.IOLoop.current()
+loop.start()
+```
+
+#### Javascript client
+
+Perspective's websocket client interfaces with the Python server. then
+_replicates_ the server-side Table.
+
+```javascript
+const websocket = await perspective.websocket("ws://localhost:8080");
+const server_table = await websocket.open_table("my_table");
+const server_view = await server_table.view();
+
+const worker = await perspective.worker();
+const client_table = await worker.table(server_view);
+
+const viewer = document.createElement("perspective-viewer");
+document.body.appendChild(viewer);
+await viewer.load(client_table);
+```
diff --git a/docs/md/explanation/architecture/server_only.md b/docs/md/explanation/architecture/server_only.md
new file mode 100644
index 0000000000..09084c4b70
--- /dev/null
+++ b/docs/md/explanation/architecture/server_only.md
@@ -0,0 +1,33 @@
+# Server-only
+
+
+
+_For extremely large datasets with a small number of concurrent users._
+
+The dataset is instantiated in-memory with a Python or Node.js server, and web
+applications connect virtually. Has very good initial load performance, since no
+data is downloaded. Group-by and other operations will run column-parallel if
+configured.
+
+But interactive performance is poor, as every user interaction must page the
+server to render. Operations like scrolling are not as responsive and can be
+impacted by network latency. Web applications must be "always connected" to the
+server via WebSocket. Disconnecting will prevent any interaction, scrolling,
+etc. of the UI. Does not use WebAssembly.
+
+Each connected browser will impact server performance as long as the connection
+is open, which in turn impacts interactive performance of every client. This
+ultimately limits the horizontal scalabity of this architecture. Since each
+client reads the perspective `Table` virtually, changes like edits and updates
+are automatically reflected to all clients and persist across browser refresh.
+Using the same Python server as the previous design, we can simply skip the
+intermediate WebAssembly `Table` and pass the virtual table directly to `load()`
+
+```javascript
+const websocket = await perspective.websocket("ws://localhost:8080");
+const server_table = await websocket.open_table("my_table");
+
+const viewer = document.createElement("perspective-viewer");
+document.body.appendChild(viewer);
+await viewer.load(server_table);
+```
diff --git a/docs/md/explanation/javascript.md b/docs/md/explanation/javascript.md
new file mode 100644
index 0000000000..ffea687dde
--- /dev/null
+++ b/docs/md/explanation/javascript.md
@@ -0,0 +1,15 @@
+Perspective's JavaScript library offers a configurable UI powered by the same
+fast streaming data engine, just re-compiled to WebAssembly. A simple example
+which loads an [Apache Arrow](https://arrow.apache.org/) and computes a "Group
+By" operation, returning a new Arrow:
+
+```javascript
+import perspective from "@finos/perspective";
+
+const table = await perspective.table(apache_arrow_data);
+const view = await table.view({ group_by: ["CounterParty", "Security"] });
+const arrow = await view.to_arrow();
+```
+
+[More Examples](https://github.com/finos/perspective/tree/master/examples) are
+available on GitHub.
diff --git a/docs/md/explanation/javascript_builds.md b/docs/md/explanation/javascript_builds.md
new file mode 100644
index 0000000000..26efe0ae4c
--- /dev/null
+++ b/docs/md/explanation/javascript_builds.md
@@ -0,0 +1,42 @@
+# JavaScript Builds
+
+Perspective requires the browser to have access to Perspective's `.wasm`
+binaries _in addition_ to the bundled `.js` files, and as a result the build
+process requires a few extra steps. To ease integration, Perspective's NPM
+releases come with multiple prebuilt configurations.
+
+## Browser
+
+### ESM Builds
+
+The recommended builds for production use are packaged as ES Modules and require
+a _bootstrapping_ step in order to acquire the `.wasm` binaries and initialize
+Perspective's JavaScript with them. However, because they have no hard-coded
+dependencies on the `.wasm` paths, they are ideal for use with JavaScript
+bundlers such as ESBuild, Rollup, Vite or Webpack.
+
+### CDN Builds
+
+Perspective's CDN builds are good for non-bundled scenarios, such as importing
+directly from a `
+```
+
+## Node.js builds
+
+The Node.js runtime for the `@finos/perspective` module runs in-process by
+default and does not implement a `child_process` interface. Hence, there is no
+`worker()` method, and the module object itself directly exports the full
+`perspective` API.
+
+```javascript
+const perspective = require("@finos/perspective");
+```
+
+In Node.js, perspective does not run in a WebWorker (as this API does not exist
+in Node.js), so no need to call the `.worker()` factory function - the
+`perspective` library exports the functions directly and run synchronously in
+the main process.
diff --git a/docs/md/how_to/javascript/installation.md b/docs/md/how_to/javascript/installation.md
new file mode 100644
index 0000000000..0083fecc9d
--- /dev/null
+++ b/docs/md/how_to/javascript/installation.md
@@ -0,0 +1,31 @@
+# JavaScript NPM Installation
+
+Perspective releases contain several different builds for use in most
+environments.
+
+## Browser
+
+Perspective's WebAssembly data engine is available via NPM in the same package
+as its Node.js counterpart, `@finos/perspective`. The Perspective Viewer UI
+(which has no Node.js component) must be installed separately:
+
+```bash
+$ npm add @finos/perspective @finos/perspective-viewer
+```
+
+By itself, `@finos/perspective-viewer` does not provide any visualizations, only
+the UI framework. Perspective _Plugins_ provide visualizations and must be
+installed separately. All Plugins are optional - but a ``
+without Plugins would be rather boring!
+
+```bash
+$ npm add @finos/perspective-viewer-d3fc @finos/perspective-viewer-datagrid @finos/perspective-viewer-openlayers
+```
+
+## Node.js
+
+To use Perspective from a Node.js server, simply install via NPM.
+
+```bash
+$ npm add @finos/perspective
+```
diff --git a/docs/md/how_to/javascript/loading_data.md b/docs/md/how_to/javascript/loading_data.md
new file mode 100644
index 0000000000..5a4d76df91
--- /dev/null
+++ b/docs/md/how_to/javascript/loading_data.md
@@ -0,0 +1,38 @@
+# Loading data from a Table
+
+Data can be loaded into `` in the form of a `Table()` or a
+`Promise
` via the `load()` method.
+
+```javascript
+// Create a new worker, then a new table promise on that worker.
+const worker = await perspective.worker();
+const table = await worker.table(data);
+
+// Bind a viewer element to this table.
+await viewer.load(table);
+```
+
+## Sharing a `Table` between multiple ``s
+
+Multiple ``s can share a `table()` by passing the `table()`
+into the `load()` method of each viewer. Each `perspective-viewer` will update
+when the underlying `table()` is updated, but `table.delete()` will fail until
+all `perspective-viewer` instances referencing it are also deleted:
+
+```javascript
+const viewer1 = document.getElementById("viewer1");
+const viewer2 = document.getElementById("viewer2");
+
+// Create a new WebWorker
+const worker = await perspective.worker();
+
+// Create a table in this worker
+const table = await worker.table(data);
+
+// Load the same table in 2 different elements
+await viewer1.load(table);
+await viewer2.load(table);
+
+// Both `viewer1` and `viewer2` will reflect this update
+await table.update([{ x: 5, y: "e", z: true }]);
+```
diff --git a/docs/md/how_to/javascript/loading_virtual_data.md b/docs/md/how_to/javascript/loading_virtual_data.md
new file mode 100644
index 0000000000..da5ec808f4
--- /dev/null
+++ b/docs/md/how_to/javascript/loading_virtual_data.md
@@ -0,0 +1,37 @@
+### Loading data from a virtual `Table`
+
+Loading a virtual (server-only) [`Table`] works just like loading a local/Web
+Worker [`Table`] - just pass the virtual [`Table`] to `viewer.load()`. In the
+browser:
+
+```javascript
+const elem = document.getElementsByTagName("perspective-viewer")[0];
+
+// Bind to the server's worker instead of instantiating a Web Worker.
+const websocket = await perspective.websocket(
+ window.location.origin.replace("http", "ws")
+);
+
+// Bind the viewer to the preloaded data source. `table` and `view` objects
+// live on the server.
+const server_table = await websocket.open_table("table_one");
+await elem.load(server_table);
+```
+
+Alternatively, data can be _cloned_ from a server-side virtual `Table` into a
+client-side WebAssemblt `Table`. The browser clone will be synced via delta
+updates transferred via Apache Arrow IPC format, but local `View`s created will
+be calculated locally on the client browser.
+
+```javascript
+const worker = await perspective.worker();
+const server_view = await server_table.view();
+const client_table = worker.table(server_view);
+await elem.load(client_table);
+```
+
+`` instances bound in this way are otherwise no different
+than ``s which rely on a Web Worker, and can even share a
+host application with Web Worker-bound `table()`s. The same `promise`-based API
+is used to communicate with the server-instantiated `view()`, only in this case
+it is over a websocket.
diff --git a/docs/md/how_to/javascript/nodejs_server.md b/docs/md/how_to/javascript/nodejs_server.md
new file mode 100644
index 0000000000..9a0ce66496
--- /dev/null
+++ b/docs/md/how_to/javascript/nodejs_server.md
@@ -0,0 +1,38 @@
+# Server-only via `WebSocketServer()` and Node.js
+
+For exceptionally large datasets, a `Client` can be bound to a
+`perspective.table()` instance running in Node.js/Python/Rust remotely, rather
+than creating one in a Web Worker and downloading the entire data set. This
+trades off network bandwidth and server resource requirements for a smaller
+browser memory and CPU footprint.
+
+An example in Node.js:
+
+```javascript
+const { WebSocketServer, table } = require("@finos/perspective");
+const fs = require("fs");
+
+// Start a WS/HTTP host on port 8080. The `assets` property allows
+// the `WebSocketServer()` to also serves the file structure rooted in this
+// module's directory.
+const host = new WebSocketServer({ assets: [__dirname], port: 8080 });
+
+// Read an arrow file from the file system and host it as a named table.
+const arr = fs.readFileSync(__dirname + "/superstore.lz4.arrow");
+await table(arr, { name: "table_one" });
+```
+
+... and the [`Client`] implementation in the browser:
+
+```javascript
+const elem = document.getElementsByTagName("perspective-viewer")[0];
+
+// Bind to the server's worker instead of instantiating a Web Worker.
+const websocket = await perspective.websocket(
+ window.location.origin.replace("http", "ws")
+);
+
+// Create a virtual `Table` to the preloaded data source. `table` and `view`
+// objects live on the server.
+const server_table = await websocket.open_table("table_one");
+```
diff --git a/docs/md/how_to/javascript/save_restore.md b/docs/md/how_to/javascript/save_restore.md
new file mode 100644
index 0000000000..32f79fde59
--- /dev/null
+++ b/docs/md/how_to/javascript/save_restore.md
@@ -0,0 +1,99 @@
+# Saving and restoring UI state.
+
+`` is _persistent_, in that its entire state (sans the data
+itself) can be serialized or deserialized. This include all column, filter,
+pivot, expressions, etc. properties, as well as datagrid style settings, config
+panel visibility, and more. This overloaded feature covers a range of use cases:
+
+- Setting a ``'s initial state after a `load()` call.
+- Updating a single or subset of properties, without modifying others.
+- Resetting some or all properties to their data-relative default.
+- Persisting a user's configuration to `localStorage` or a server.
+
+## Serializing and deserializing the viewer state
+
+To retrieve the entire state as a JSON-ready JavaScript object, use the `save()`
+method. `save()` also supports a few other formats such as `"arraybuffer"` and
+`"string"` (base64, not JSON), which you may choose for size at the expense of
+easy migration/manual-editing.
+
+```javascript
+const json_token = await elem.save();
+const string_token = await elem.save("string");
+```
+
+For any format, the serialized token can be restored to any
+`` with a `Table` of identical schema, via the `restore()`
+method. Note that while the data for a token returned from `save()` may differ,
+generally its schema may not, as many other settings depend on column names and
+types.
+
+```javascript
+await elem.restore(json_token);
+await elem.restore(string_token);
+```
+
+As `restore()` dispatches on the token's type, it is important to make sure that
+these types match! A common source of error occurs when passing a
+JSON-stringified token to `restore()`, which will assume base64-encoded msgpack
+when a string token is used.
+
+```javascript
+// This will error!
+await elem.restore(JSON.stringify(json_token));
+```
+
+### Updating individual properties
+
+Using the JSON format, every facet of a ``'s configuration
+can be manipulated from JavaScript using the `restore()` method. The valid
+structure of properties is described via the
+[`ViewerConfig`](https://github.com/finos/perspective/blob/ebced4caa/rust/perspective-viewer/src/ts/viewer.ts#L16)
+and embedded
+[`ViewConfig`](https://github.com/finos/perspective/blob/ebced4caa19435a2a57d4687be7e428a4efc759b/packages/perspective/index.d.ts#L140)
+type declarations, and [`View`](view.md) chapter of the documentation which has
+several interactive examples for each `ViewConfig` property.
+
+```javascript
+// Set the plugin (will also update `columns` to plugin-defaults)
+await elem.restore({ plugin: "X Bar" });
+
+// Update plugin and columns (only draws once)
+await elem.restore({ plugin: "X Bar", columns: ["Sales"] });
+
+// Open the config panel
+await elem.restore({ settings: true });
+
+// Create an expression
+await elem.restore({
+ columns: ['"Sales" + 100'],
+ expressions: { "New Column": '"Sales" + 100' },
+});
+
+// ERROR if the column does not exist in the schema or expressions
+// await elem.restore({columns: ["\"Sales\" + 100"], expressions: {}});
+
+// Add a filter
+await elem.restore({ filter: [["Sales", "<", 100]] });
+
+// Add a sort, don't remove filter
+await elem.restore({ sort: [["Prodit", "desc"]] });
+
+// Reset just filter, preserve sort
+await elem.restore({ filter: undefined });
+
+// Reset all properties to default e.g. after `load()`
+await elem.reset();
+```
+
+Another effective way to quickly create a token for a desired configuration is
+to simply copy the token returned from `save()` after settings the view manually
+in the browser. The JSON format is human-readable and should be quite easy to
+tweak once generated, as `save()` will return even the default settings for all
+properties. You can call `save()` in your application code, or e.g. through the
+Chrome developer console:
+
+```javascript
+// Copy to clipboard
+copy(await document.querySelector("perspective-viewer").save());
+```
diff --git a/docs/md/how_to/javascript/serializing.md b/docs/md/how_to/javascript/serializing.md
new file mode 100644
index 0000000000..7332fa836e
--- /dev/null
+++ b/docs/md/how_to/javascript/serializing.md
@@ -0,0 +1,21 @@
+### Serializing data
+
+The `view()` allows for serialization of data to JavaScript through the
+`to_json()`, `to_ndjson()`, `to_columns()`, `to_csv()`, and `to_arrow()` methods
+(the same data formats supported by the `Client::table` factory function). These
+methods return a `promise` for the calculated data:
+
+```javascript
+const view = await table.view({ group_by: ["State"], columns: ["Sales"] });
+
+// JavaScript Objects
+console.log(await view.to_json());
+console.log(await view.to_columns());
+
+// String
+console.log(await view.to_csv());
+console.log(await view.to_ndjson());
+
+// ArrayBuffer
+console.log(await view.to_arrow());
+```
diff --git a/docs/md/how_to/javascript/theming.md b/docs/md/how_to/javascript/theming.md
new file mode 100644
index 0000000000..b3c1dc4d04
--- /dev/null
+++ b/docs/md/how_to/javascript/theming.md
@@ -0,0 +1,67 @@
+# Theming
+
+Theming is supported in `perspective-viewer` and its accompanying plugins. A
+number of themes come bundled with `perspective-viewer`; you can import any of
+these themes directly into your app, and the `perspective-viewer`s will be
+themed accordingly:
+
+```javascript
+// Themes based on Thought Merchants's Prospective design
+import "@finos/perspective-viewer/dist/css/pro.css";
+import "@finos/perspective-viewer/dist/css/pro-dark.css";
+
+// Other themes
+import "@finos/perspective-viewer/dist/css/solarized.css";
+import "@finos/perspective-viewer/dist/css/solarized-dark.css";
+import "@finos/perspective-viewer/dist/css/monokai.css";
+import "@finos/perspective-viewer/dist/css/vaporwave.css";
+```
+
+Alternatively, you may use `themes.css`, which bundles all default themes
+
+```javascript
+import "@finos/perspective-viewer/dist/css/themes.css";
+```
+
+If you choose not to bundle the themes yourself, they are available through
+[CDN](https://cdn.jsdelivr.net/npm/@finos/perspective-viewer/dist/css/). These
+can be directly linked in your HTML file:
+
+```html
+
+```
+
+Note the `crossorigin="anonymous"` attribute. When including a theme from a
+cross-origin context, this attribute may be required to allow
+`` to detect the theme. If this fails, additional themes are
+added to the `document` after `` init, or for any other
+reason theme auto-detection fails, you may manually inform
+`` of the available theme names with the `.resetThemes()`
+method.
+
+```javascript
+// re-auto-detect themes
+viewer.resetThemes();
+
+// Set available themes explicitly (they still must be imported as CSS!)
+viewer.resetThemes(["Pro Light", "Pro Dark"]);
+```
+
+`` will default to the first loaded theme when initialized.
+You may override this via `.restore()`, or provide an initial theme by setting
+the `theme` attribute:
+
+```html
+
+```
+
+or
+
+```javascript
+const viewer = document.querySelector("perspective-viewer");
+await viewer.restore({ theme: "Pro Dark" });
+```
diff --git a/docs/md/how_to/javascript/viewer.md b/docs/md/how_to/javascript/viewer.md
new file mode 100644
index 0000000000..ef9c86ae5e
--- /dev/null
+++ b/docs/md/how_to/javascript/viewer.md
@@ -0,0 +1,16 @@
+# `` Custom Element library
+
+`` provides a complete graphical UI for configuring the
+`perspective` library and formatting its output to the provided visualization
+plugins.
+
+Once imported and initialized in JavaScript, the `` Web
+Component will be available in any standard HTML on your site. A simple example:
+
+```html
+
+
+```
diff --git a/docs/md/how_to/javascript/worker.md b/docs/md/how_to/javascript/worker.md
new file mode 100644
index 0000000000..dc3d9adbe2
--- /dev/null
+++ b/docs/md/how_to/javascript/worker.md
@@ -0,0 +1,44 @@
+# Accessing the Perspective engine via a `Client` instance
+
+An instance of a `Client` is needed to talk to a Perspective `Server`, of which
+there are a few varieties available in JavaScript.
+
+## Web Worker (Browser)
+
+Perspective's Web Worker client is actually a `Client` and `Server` rolled into
+one. Instantiating this `Client` will also create a _dedicated_ Perspective
+`Server` in a Web Worker process.
+
+To use it, you'll need to instantiate a Web Worker `perspective` engine via the
+`worker()` method. This will create a new Web Worker (browser) and load the
+WebAssembly binary. All calculation and data accumulation will occur in this
+separate process.
+
+```javascript
+const client = await perspective.worker();
+```
+
+The `worker` symbol will expose the full `perspective` API for one managed Web
+Worker process. You are free to create as many as your browser supports, but be
+sure to keep track of the `worker` instances themselves, as you'll need them to
+interact with your data in each instance.
+
+## Websocket (Browser)
+
+Alternatively, with a Perspective server running in Node.js, Python or Rust, you
+can create a _virtual_ `Client` via the `websocket()` method.
+
+```javascript
+const client = perspective.websocket("http://localhost:8080/");
+```
+
+## Node.js
+
+The Node.js runtime for the `@finos/perspective` module runs in-process by
+default and does not implement a `child_process` interface, so no need to call
+the `.worker()` factory function. Instead, the `perspective` library exports the
+functions directly and run synchronously in the main process.
+
+```javascript
+const client = require("@finos/perspective");
+```
diff --git a/docs/md/how_to/python.md b/docs/md/how_to/python.md
new file mode 100644
index 0000000000..4193ef4bec
--- /dev/null
+++ b/docs/md/how_to/python.md
@@ -0,0 +1 @@
+# Python
diff --git a/docs/md/how_to/python/callbacks.md b/docs/md/how_to/python/callbacks.md
new file mode 100644
index 0000000000..cc6b28d12c
--- /dev/null
+++ b/docs/md/how_to/python/callbacks.md
@@ -0,0 +1,34 @@
+# Callbacks and Events
+
+`perspective.Table` allows for `on_update` and `on_delete` callbacks to be
+set—simply call `on_update` or `on_delete` with a reference to a function or a
+lambda without any parameters:
+
+```python
+def update_callback():
+ print("Updated!")
+
+# set the update callback
+on_update_id = view.on_update(update_callback)
+
+
+def delete_callback():
+ print("Deleted!")
+
+# set the delete callback
+on_delete_id = view.on_delete(delete_callback)
+
+# set a lambda as a callback
+view.on_delete(lambda: print("Deleted x2!"))
+```
+
+If the callback is a named reference to a function, it can be removed with
+`remove_update` or `remove_delete`:
+
+```python
+view.remove_update(on_update_id)
+view.remove_delete(on_delete_id)
+```
+
+Callbacks defined with a lambda function cannot be removed, as lambda functions
+have no identifier.
diff --git a/docs/md/how_to/python/installation.md b/docs/md/how_to/python/installation.md
new file mode 100644
index 0000000000..ae2558a9a6
--- /dev/null
+++ b/docs/md/how_to/python/installation.md
@@ -0,0 +1,27 @@
+# Installation
+
+`perspective-python` contains full bindings to the Perspective API, a JupyterLab
+widget, and WebSocket handlers for several webserver libraries that allow you to
+host Perspective using server-side Python.
+
+## PyPI
+
+`perspective-python` can be installed from [PyPI](https://pypi.org) via `pip`:
+
+```bash
+pip install perspective-python
+```
+
+That's it! If JupyterLab is installed in this Python environment, you'll also
+get the `perspective.widget.PerspectiveWidget` class when you import
+`perspective` in a Jupyter Lab kernel.
+
+
diff --git a/docs/md/how_to/python/jupyterlab.md b/docs/md/how_to/python/jupyterlab.md
new file mode 100644
index 0000000000..33f6aea33b
--- /dev/null
+++ b/docs/md/how_to/python/jupyterlab.md
@@ -0,0 +1,61 @@
+# `PerspectiveWidget` for JupyterLab
+
+Building on top of the API provided by `perspective.Table`, the
+`PerspectiveWidget` is a JupyterLab plugin that offers the entire functionality
+of Perspective within the Jupyter environment. It supports the same API
+semantics of ``, along with the additional data types
+supported by `perspective.Table`. `PerspectiveWidget` takes keyword arguments
+for the managed `View`:
+
+```python
+from perspective.widget import PerspectiveWidget
+w = perspective.PerspectiveWidget(
+ data,
+ plugin="X Bar",
+ aggregates={"datetime": "any"},
+ sort=[["date", "desc"]]
+)
+```
+
+## Creating a widget
+
+A widget is created through the `PerspectiveWidget` constructor, which takes as
+its first, required parameter a `perspective.Table`, a dataset, a schema, or
+`None`, which serves as a special value that tells the Widget to defer loading
+any data until later. In maintaining consistency with the Javascript API,
+Widgets cannot be created with empty dictionaries or lists — `None` should be
+used if the intention is to await data for loading later on. A widget can be
+constructed from a dataset:
+
+```python
+from perspective.widget import PerspectiveWidget
+PerspectiveWidget(data, group_by=["date"])
+```
+
+.. or a schema:
+
+```python
+PerspectiveWidget({"a": int, "b": str})
+```
+
+.. or an instance of a `perspective.Table`:
+
+```python
+table = perspective.table(data)
+PerspectiveWidget(table)
+```
+
+## Updating a widget
+
+`PerspectiveWidget` shares a similar API to the `` Custom
+Element, and has similar `save()` and `restore()` methods that
+serialize/deserialize UI state for the widget.
+
+
diff --git a/docs/md/how_to/python/multithreading.md b/docs/md/how_to/python/multithreading.md
new file mode 100644
index 0000000000..5ad1e4d1c4
--- /dev/null
+++ b/docs/md/how_to/python/multithreading.md
@@ -0,0 +1,18 @@
+# Multi-threading
+
+Perspective's server API releases the GIL when called (though it may be retained
+for some portion of the `Client` call to encode RPC messages). It also
+dispatches to an internal thread pool for some operations, enabling better
+parallelism and overall better server performance. However, Perspective's Python
+interface itself will still process queries in a single queue. To enable
+parallel query processing, call `set_loop_callback` with a multi-threaded
+executor such as `concurrent.futures.ThreadPoolExecutor`:
+
+```python
+def perspective_thread():
+ server = perspective.Server()
+ loop = tornado.ioloop.IOLoop()
+ with concurrent.futures.ThreadPoolExecutor() as executor:
+ server.set_loop_callback(loop.run_in_executor, executor)
+ loop.start()
+```
diff --git a/docs/md/how_to/python/table.md b/docs/md/how_to/python/table.md
new file mode 100644
index 0000000000..c15ecbaf93
--- /dev/null
+++ b/docs/md/how_to/python/table.md
@@ -0,0 +1,84 @@
+# Loading data into a Table
+
+A `Table` can be created from a dataset or a schema, the specifics of which are
+[discussed](#loading-data-with-table) in the JavaScript section of the user's
+guide. In Python, however, Perspective supports additional data types that are
+commonly used when processing data:
+
+- `pandas.DataFrame`
+- `polars.DataFrame`
+- `bytes` (encoding an Apache Arrow)
+- `objects` (either extracting a repr or via reference)
+- `str` (encoding as a CSV)
+
+A `Table` is created in a similar fashion to its JavaScript equivalent:
+
+```python
+from datetime import date, datetime
+import numpy as np
+import pandas as pd
+import perspective
+
+data = pd.DataFrame({
+ "int": np.arange(100),
+ "float": [i * 1.5 for i in range(100)],
+ "bool": [True for i in range(100)],
+ "date": [date.today() for i in range(100)],
+ "datetime": [datetime.now() for i in range(100)],
+ "string": [str(i) for i in range(100)]
+})
+
+table = perspective.table(data, index="float")
+```
+
+Likewise, a `View` can be created via the `view()` method:
+
+```python
+view = table.view(group_by=["float"], filter=[["bool", "==", True]])
+column_data = view.to_columns()
+row_data = view.to_json()
+```
+
+## Polars Support
+
+Polars `DataFrame` types work similarly to Apache Arrow input, which Perspective
+uses to interface with Polars.
+
+```python
+df = polars.DataFrame({"a": [1,2,3,4,5]})
+table = perspective.table(df)
+```
+
+## Pandas Support
+
+Perspective's `Table` can be constructed from `pandas.DataFrame` objects.
+Internally, this just uses
+[`pyarrow::from_pandas`](https://arrow.apache.org/docs/python/pandas.html),
+which dictates behavior of this feature including type support.
+
+If the dataframe does not have an index set, an integer-typed column named
+`"index"` is created. If you want to preserve the indexing behavior of the
+dataframe passed into Perspective, simply create the `Table` with
+`index="index"` as a keyword argument. This tells Perspective to once again
+treat the index as a primary key:
+
+```python
+data.set_index("datetime")
+table = perspective.table(data, index="index")
+```
+
+## Time Zone Handling
+
+When parsing `"datetime"` strings, times are assumed _local time_ unless an
+explicit timezone offset is parsed. All `"datetime"` columns (regardless of
+input time zone) are _output_ to the user as `datetime.datetime` objects in
+_local time_ according to the Python runtime.
+
+This behavior is consistent with Perspective's behavior in JavaScript. For more
+details, see this in-depth
+[explanation](https://github.com/finos/perspective/pull/867) of
+`perspective-python` semantics around time zone handling.
+
+```
+
+```
diff --git a/docs/md/how_to/python/websocket.md b/docs/md/how_to/python/websocket.md
new file mode 100644
index 0000000000..bf60ae8a2e
--- /dev/null
+++ b/docs/md/how_to/python/websocket.md
@@ -0,0 +1,129 @@
+# Hosting a WebSocket server
+
+An in-memory `Server` "hosts" all `perspective.Table` and `perspective.View`
+instances created by its connected `Client`s. Hosted tables/views can have their
+methods called from other sources than the Python server, i.e. by a
+`perspective-viewer` running in a JavaScript client over the network,
+interfacing with `perspective-python` through the websocket API.
+
+The server has full control of all hosted `Table` and `View` instances, and can
+call any public API method on hosted instances. This makes it extremely easy to
+stream data to a hosted `Table` using `.update()`:
+
+```python
+server = perspective.Server()
+client = server.new_local_client()
+table = client.table(data, name="data_source")
+
+for i in range(10):
+ # updates continue to propagate automatically
+ table.update(new_data)
+```
+
+The `name` provided is important, as it enables Perspective in JavaScript to
+look up a `Table` and get a handle to it over the network. Otherwise, `name`
+will be assigned randomlu and the `Client` must look this up with
+`CLient.get_hosted_table_names()`
+
+## Client/Server Replicated Mode
+
+Using Tornado and
+[`PerspectiveTornadoHandler`](python.md#perspectivetornadohandler), as well as
+`Perspective`'s JavaScript library, we can set up "distributed" Perspective
+instances that allows multiple browser `perspective-viewer` clients to read from
+a common `perspective-python` server, as in the
+[Tornado Example Project](https://github.com/finos/perspective/tree/master/examples/python-tornado).
+
+This architecture works by maintaining two `Tables`—one on the server, and one
+on the client that mirrors the server's `Table` automatically using `on_update`.
+All updates to the table on the server are automatically applied to each client,
+which makes this architecture a natural fit for streaming dashboards and other
+distributed use-cases. In conjunction with [multithreading](#multi-threading),
+distributed Perspective offers consistently high performance over large numbers
+of clients and large datasets.
+
+_*server.py*_
+
+```python
+from perspective import Server
+from perspective.hadnlers.tornado import PerspectiveTornadoHandler
+
+# Create an instance of Server, and host a Table
+SERVER = Server()
+CLIENT = SERVER.new_local_client()
+
+# The Table is exposed at `localhost:8888/websocket` with the name `data_source`
+client.table(data, name = "data_source")
+
+app = tornado.web.Application([
+ # create a websocket endpoint that the client JavaScript can access
+ (r"/websocket", PerspectiveTornadoHandler, {"perspective_server": SERVER})
+])
+
+# Start the Tornado server
+app.listen(8888)
+loop = tornado.ioloop.IOLoop.current()
+loop.start()
+```
+
+Instead of calling `load(server_table)`, create a `View` using `server_table`
+and pass that into `viewer.load()`. This will automatically register an
+`on_update` callback that synchronizes state between the server and the client.
+
+_*index.html*_
+
+```html
+
+
+
+```
+
+For a more complex example that offers distributed editing of the server
+dataset, see
+[client_server_editing.html](https://github.com/finos/perspective/blob/master/examples/python-tornado/client_server_editing.html).
+
+We also provide examples for Starlette/FastAPI and AIOHTTP:
+
+- [Starlette Example Project](https://github.com/finos/perspective/tree/master/examples/python-starlette).
+- [AIOHTTP Example Project](https://github.com/finos/perspective/tree/master/examples/python-aiohttp).
+
+## Server-only Mode
+
+The server setup is identical to
+[Client/Server Replicated Mode](#client-server-replicated-mode) above, but
+instead of creating a `View`, the client calls `load(server_table)`: In Python,
+use `Server` and `PerspectiveTornadoHandler` to create a websocket server that
+exposes a `Table`. In this example, `table` is a proxy for the `Table` we
+created on the server. All API methods are available on _proxies_, the.g.us
+calling `view()`, `schema()`, `update()` on `table` will pass those operations
+to the Python `Table`, execute the commands, and return the result back to
+Javascript.
+
+```html
+
+```
+
+```javascript
+const websocket = perspective.websocket("ws://localhost:8888/websocket");
+const table = websocket.open_table("data_source");
+document.getElementById("viewer").load(table);
+```
diff --git a/docs/md/how_to/rust.md b/docs/md/how_to/rust.md
new file mode 100644
index 0000000000..837deaea9d
--- /dev/null
+++ b/docs/md/how_to/rust.md
@@ -0,0 +1,28 @@
+# Rust
+
+Install via `cargo`:
+
+```bash
+cargo add perspective
+```
+
+# Example
+
+Initialize a server and client
+
+```rust
+let server = Server::default();
+let client = server.new_local_client();
+```
+
+Load an Arrow
+
+```rust
+let mut file = File::open(std::path::Path::new(ROOT_PATH).join(ARROW_FILE_PATH))?;
+let mut feather = Vec::with_capacity(file.metadata()?.len() as usize);
+file.read_to_end(&mut feather)?;
+let data = UpdateData::Arrow(feather.into());
+let mut options = TableInitOptions::default();
+options.set_name("my_data_source");
+client.table(data.into(), options).await?;
+```
diff --git a/docs/md/javascript.md b/docs/md/javascript.md
new file mode 100644
index 0000000000..52258b5098
--- /dev/null
+++ b/docs/md/javascript.md
@@ -0,0 +1 @@
+# JavaScript
diff --git a/docs/md/perspective.css b/docs/md/perspective.css
new file mode 100644
index 0000000000..610762526e
--- /dev/null
+++ b/docs/md/perspective.css
@@ -0,0 +1,15 @@
+perspective-viewer {
+ height: 500px;
+}
+
+div.javascript:before {
+ content: "JavaScript:";
+}
+
+div.python:before {
+ content: "Python:";
+}
+
+div.rust:before {
+ content: "Rust:";
+}
diff --git a/rust/perspective-js/src/ts/perspective.ts b/docs/md/perspective.js
similarity index 65%
rename from rust/perspective-js/src/ts/perspective.ts
rename to docs/md/perspective.js
index 5789ee682c..4365884f1c 100644
--- a/rust/perspective-js/src/ts/perspective.ts
+++ b/docs/md/perspective.js
@@ -10,38 +10,20 @@
// ┃ of the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0). ┃
// ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
-import * as api from "./browser.ts";
-export type * from "../../dist/pkg/perspective-js.d.ts";
+import "https://cdn.jsdelivr.net/npm/@finos/perspective-viewer/dist/cdn/perspective-viewer.js";
+import "https://cdn.jsdelivr.net/npm/@finos/perspective-viewer-datagrid/dist/cdn/perspective-viewer-datagrid.js";
+import "https://cdn.jsdelivr.net/npm/@finos/perspective-viewer-d3fc/dist/cdn/perspective-viewer-d3fc.js";
+import perspective from "https://cdn.jsdelivr.net/npm/@finos/perspective/dist/cdn/perspective.js";
-import type * as psp from "../../dist/pkg/perspective-js.d.ts";
+const WASM_URL =
+ "https://cdn.jsdelivr.net/npm/superstore-arrow/superstore.lz4.arrow";
-type WasmElement = {
- __wasm_module__: Promise;
-};
+const worker = await perspective.worker();
-export async function compile_perspective() {
- let elem = customElements.get(
- "perspective-viewer"
- ) as unknown as WasmElement;
- if (!elem) {
- console.warn("No `` Custom Element found, waiting");
- await customElements.whenDefined("perspective-viewer");
- elem = customElements.get(
- "perspective-viewer"
- ) as unknown as WasmElement;
- }
+const table = await fetch(WASM_URL)
+ .then((x) => x.arrayBuffer())
+ .then((x) => worker.table(x));
- return elem.__wasm_module__;
+for (const viewer of document.querySelectorAll("perspective-viewer")) {
+ viewer.load(table);
}
-
-export async function websocket(url: string | URL) {
- const wasm_module = compile_perspective();
- return await api.websocket(wasm_module, url);
-}
-
-export async function worker() {
- const wasm_module = compile_perspective();
- return await api.worker(wasm_module);
-}
-
-export default { websocket, worker };
diff --git a/docs/md/perspective.md b/docs/md/perspective.md
new file mode 100644
index 0000000000..676d71c102
--- /dev/null
+++ b/docs/md/perspective.md
@@ -0,0 +1 @@
+{{#include ../../README.md}}
diff --git a/docs/md/tutorials/python/tornado.md b/docs/md/tutorials/python/tornado.md
new file mode 100644
index 0000000000..0cfde7a1ef
--- /dev/null
+++ b/docs/md/tutorials/python/tornado.md
@@ -0,0 +1,109 @@
+# Tutorial: A tornado server in Python
+
+Perspective ships with a pre-built Tornado handler that makes integration with
+`tornado.websockets` extremely easy. This allows you to run an instance of
+`Perspective` on a server using Python, open a websocket to a `Table`, and
+access the `Table` in JavaScript and through ``. All
+instructions sent to the `Table` are processed in Python, which executes the
+commands, and returns its output through the websocket back to Javascript.
+
+### Python setup
+
+Make sure Perspective and Tornado are installed!
+
+```bash
+pip install perspective-python tornado
+```
+
+To use the handler, we need to first have a `Server`, a `Client` and an instance
+of a `Table`:
+
+```python
+import perspective
+
+SERVER = perspective.Server()
+CLIENT = SERVER.new_local_client()
+```
+
+Once the server has been created, create a `Table` instance with a name. The
+name that you host the table under is important — it acts as a unique accessor
+on the JavaScript side, which will look for a Table hosted at the websocket with
+the name you specify.
+
+```python
+TABLE = client.table(data, name="data_source_one")
+```
+
+After the server and table setup is complete, create a websocket endpoint and
+provide it a reference to `PerspectiveTornadoHandler`. You must provide the
+configuration object in the route tuple, and it must contain
+`"perspective_server"`, which is a reference to the `Server` you just created.
+
+```python
+from perspective.handlers.tornado import PerspectiveTornadoHandler
+
+app = tornado.web.Application([
+
+ # ... other handlers ...
+
+ # Create a websocket endpoint that the client JavaScript can access
+ (r"/websocket", PerspectiveTornadoHandler, {"perspective_server": SERVER, "check_origin": True})
+])
+```
+
+Optionally, the configuration object can also include `check_origin`, a boolean
+that determines whether the websocket accepts requests from origins other than
+where the server is hosted. See
+[Tornado docs](https://www.tornadoweb.org/en/stable/websocket.html#tornado.websocket.WebSocketHandler.check_origin)
+for more details.
+
+### JavaScript setup
+
+Once the server is up and running, you can access the Table you just hosted
+using `perspective.websocket` and `open_table()`. First, create a client that
+expects a Perspective server to accept connections at the specified URL:
+
+```javascript
+import "@finos/perspective-viewer";
+import "@finos/perspective-viewer-datagrid";
+import perspective from "@finos/perspective";
+
+const websocket = await perspective.websocket("ws://localhost:8888/websocket");
+```
+
+Next open the `Table` we created on the server by name:
+
+```javascript
+const table = await websocket.open_table("data_source_one");
+```
+
+`table` is a proxy for the `Table` we created on the server. All operations that
+are possible through the JavaScript API are possible on the Python API as well,
+thus calling `view()`, `schema()`, `update()` etc. on `const table` will pass
+those operations to the Python `Table`, execute the commands, and return the
+result back to JavaScript. Similarly, providing this `table` to a
+`` instance will allow virtual rendering:
+
+```javascript
+const viewer = document.createElement("perspective-viewer");
+viewer.style.height = "500px";
+document.body.appendChild(viewer);
+await viewer.load(table);
+```
+
+`perspective.websocket` expects a Websocket URL where it will send instructions.
+When `open_table` is called, the name to a hosted Table is passed through, and a
+request is sent through the socket to fetch the Table. No actual `Table`
+instance is passed inbetween the runtimes; all instructions are proxied through
+websockets.
+
+This provides for great flexibility — while `Perspective.js` is full of
+features, browser WebAssembly runtimes currently have some performance
+restrictions on memory and CPU feature utilization, and the architecture in
+general suffers when the dataset itself is too large to download to the client
+in full.
+
+The Python runtime does not suffer from memory limitations, utilizes Apache
+Arrow internal threadpools for threading and parallel processing, and generates
+architecture optimized code, which currently makes it more suitable as a
+server-side runtime than `node.js`.
diff --git a/docs/package.json b/docs/package.json
index 6c0ba45ccc..4c3141da69 100644
--- a/docs/package.json
+++ b/docs/package.json
@@ -15,32 +15,31 @@
"write-heading-ids": "docusaurus write-heading-ids"
},
"dependencies": {
- "@docusaurus/core": "^3.4.0",
- "@docusaurus/preset-classic": "^3.4.0",
+ "@docusaurus/core": "^3.7.0",
+ "@docusaurus/preset-classic": "^3.7.0",
"@finos/perspective": "workspace:^",
"@finos/perspective-viewer": "workspace:^",
"@finos/perspective-viewer-d3fc": "workspace:^",
"@finos/perspective-viewer-datagrid": "workspace:^",
- "@finos/perspective-webpack-plugin": "workspace:^",
"@mdx-js/react": "^3.0.0",
"blocks": "workspace:^",
"clsx": "^2.0.0",
"mkdirp": "3",
"prism-react-renderer": "^1.3.3",
"puppeteer": "^23",
- "react": "^18.0.0",
- "react-dom": "^18.0.0",
+ "react": "^18",
+ "react-dom": "^18",
"superstore-arrow": "3.0.0"
},
"devDependencies": {
"arraybuffer-loader": "^1.0.2",
- "@docusaurus/module-type-aliases": "^3.4.0",
- "@docusaurus/tsconfig": "^3.4.0",
- "@docusaurus/types": "^3.4.0",
+ "@docusaurus/module-type-aliases": "^3.7.0",
+ "@docusaurus/tsconfig": "^3.7.0",
+ "@docusaurus/types": "^3.7.0",
"typescript": "~5.2.2"
},
"peerDependencies": {
- "@docusaurus/theme-common": "^3.4.0"
+ "@docusaurus/theme-common": "^3.7.0"
},
"browserslist": {
"production": [
diff --git a/docs/plugins/perspective-loader/index.js b/docs/plugins/perspective-loader/index.js
index 745329e450..b34ed7d650 100644
--- a/docs/plugins/perspective-loader/index.js
+++ b/docs/plugins/perspective-loader/index.js
@@ -10,7 +10,7 @@
// ┃ of the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0). ┃
// ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
-const PerspectiveWebpackPlugin = require("@finos/perspective-webpack-plugin");
+const webpack = require("webpack");
module.exports = function (context, options) {
return {
@@ -20,7 +20,10 @@ module.exports = function (context, options) {
config.optimization.minimizer[0].options.minimizer.options.module = true;
}
- config.experiments = config.experiments || {};
+ config.experiments = config.experiments || {
+ asyncWebAssembly: true,
+ };
+
config.experiments.topLevelAwait = true;
config.module.rules.map((x) => {
if (x.test.toString() === "/\\.css$/i") {
@@ -37,7 +40,14 @@ module.exports = function (context, options) {
node: {
__filename: false,
},
- plugins: [new PerspectiveWebpackPlugin({})],
+ plugins: isServer
+ ? [
+ new webpack.NormalModuleReplacementPlugin(
+ /@finos\/perspective/,
+ "@finos/perspective/dist/esm/perspective.js"
+ ),
+ ]
+ : [],
};
},
};
diff --git a/docs/src/components/DocItem/browser.js b/docs/src/components/DocItem/browser.js
index f5993c38a3..8d943cbc15 100644
--- a/docs/src/components/DocItem/browser.js
+++ b/docs/src/components/DocItem/browser.js
@@ -10,8 +10,6 @@
// ┃ of the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0). ┃
// ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
-import { SUPERSTORE_TABLE } from "@site/src/data/superstore.js";
-
export async function main(colorMode) {
await import("@finos/perspective-viewer", { type: "module" });
await import("@finos/perspective-viewer-datagrid");
@@ -21,6 +19,8 @@ export async function main(colorMode) {
"perspective-viewer:not(.nosuperstore)"
);
+ const { SUPERSTORE_TABLE } = await import("@site/src/data/superstore.js");
+
for (const viewer of viewers) {
viewer.load(SUPERSTORE_TABLE);
const token = {
diff --git a/docs/src/components/ExampleGallery/index.js b/docs/src/components/ExampleGallery/index.js
index a951a7e91c..4e880a3c6c 100644
--- a/docs/src/components/ExampleGallery/index.js
+++ b/docs/src/components/ExampleGallery/index.js
@@ -63,7 +63,6 @@ export default function ExampleGallery(props) {
}
function OverlayDemo(props) {
- const { SUPERSTORE_TABLE } = require("@site/src/data/superstore.js");
const ref = useRef();
const dismissCallback = useCallback(
(event) => {
@@ -77,6 +76,10 @@ function OverlayDemo(props) {
const perspectiveRef = useCallback(
(viewer) => {
+ const {
+ SUPERSTORE_TABLE,
+ } = require("@site/src/data/superstore.js");
+
if (viewer !== null) {
viewer.load(SUPERSTORE_TABLE);
viewer.restore({
diff --git a/docs/src/data/worker.js b/docs/src/data/worker.js
index 0c6123ec9f..7fde1a54ae 100644
--- a/docs/src/data/worker.js
+++ b/docs/src/data/worker.js
@@ -10,7 +10,21 @@
// ┃ of the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0). ┃
// ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
-const WORKER = import("@finos/perspective").then((x) => x.worker());
+const WORKER = (async () => {
+ const perspective = await import("@finos/perspective");
+ const perspective_viewer = await import("@finos/perspective-viewer");
+ const wasm = import("@finos/perspective/dist/wasm/perspective-server.wasm");
+ const client_wasm = import(
+ "@finos/perspective-viewer/dist/wasm/perspective-viewer.wasm"
+ );
+
+ await Promise.all([
+ perspective.init_server(wasm),
+ perspective_viewer.init_client(client_wasm),
+ ]);
+
+ return await perspective.worker();
+})();
export function worker() {
return WORKER;
diff --git a/examples/README.md b/examples/README.md
new file mode 100644
index 0000000000..8bb2dc5609
--- /dev/null
+++ b/examples/README.md
@@ -0,0 +1,19 @@
+# Examples
+
+The projects in the directory are self-contained Perspective examples. However,
+they are designed to run within the Perspective source repository tree. This
+makes it easy for Perspective OSS developers to test changes and validate that
+our Examples continue to work, but it also means these examples will most not
+work without properly linking in the Perspective build environment.
+
+In order to _run_ a project in this directory as written:
+
+1. Install and build Perspective from source.
+2. Run the project with `pnpm run start $PROJECT_NAME` from the repository root
+ (_not_ the `/examples` directory).
+
+# Optional
+
+Generally, the changes necessary to make these examples run _without_ the
+Perspective source repository are minor path or metadata corrections. Your
+results may vary.
diff --git a/examples/blocks/README.md b/examples/blocks/README.md
new file mode 100644
index 0000000000..203462c511
--- /dev/null
+++ b/examples/blocks/README.md
@@ -0,0 +1,4 @@
+# Bloc.ks
+
+Examples designed to be hosted on GitHub Pages. See the individual projects in
+the `src/` directory for more info.
diff --git a/examples/blocks/src/citibike/citibike.js b/examples/blocks/src/citibike/citibike.js
index 6f73be549d..73e395603c 100644
--- a/examples/blocks/src/citibike/citibike.js
+++ b/examples/blocks/src/citibike/citibike.js
@@ -85,7 +85,7 @@ async function main() {
}
// Start a recurring asyn call to `get_feed` and update the `table` with the response.
- get_feed("station_status", table.update);
+ get_feed("station_status", table.update.bind(table));
window.workspace.tables.set("citibike", Promise.resolve(table));
const layout = await get_layout();
diff --git a/examples/blocks/src/citibike/index.html b/examples/blocks/src/citibike/index.html
index 8d03e0f15f..ee1c82d697 100644
--- a/examples/blocks/src/citibike/index.html
+++ b/examples/blocks/src/citibike/index.html
@@ -3,20 +3,14 @@
-
diff --git a/examples/blocks/src/covid/index.html b/examples/blocks/src/covid/index.html
index 2c19d0e007..e67fa518b3 100644
--- a/examples/blocks/src/covid/index.html
+++ b/examples/blocks/src/covid/index.html
@@ -2,8 +2,8 @@
-
-
+
+
diff --git a/examples/blocks/src/fractal/index.html b/examples/blocks/src/fractal/index.html
index 5854e8a37c..46e0a1ad10 100644
--- a/examples/blocks/src/fractal/index.html
+++ b/examples/blocks/src/fractal/index.html
@@ -2,8 +2,8 @@
-
-
+
+
diff --git a/examples/blocks/src/market/index.html b/examples/blocks/src/market/index.html
index 96141ef1c4..adb99186db 100644
--- a/examples/blocks/src/market/index.html
+++ b/examples/blocks/src/market/index.html
@@ -2,8 +2,8 @@
-
-
+
+
diff --git a/examples/blocks/src/movies/index.html b/examples/blocks/src/movies/index.html
index 6919cb21e2..e7f45997ae 100644
--- a/examples/blocks/src/movies/index.html
+++ b/examples/blocks/src/movies/index.html
@@ -1,22 +1,17 @@
-
-
+
+
-
-This crate contains the server/engine components of the
-[Perspective](https://perspective.finos.org) data visualization suite. It is
-meant to be used in conjunction with the other crates of this project, e.g.
-`perspective-client` to create client connections to a server.
-
-The [`perspective`] crate provides a convenient frontend for Rust developers,
-including both [`perspective_client`] and [`perspective_server`] as well as
-other convenient integration helpers.
-
-# Data Binding Options
-
-
-
-Application developers can choose from [Client (WebAssembly)](#client-only),
-[Server (Python/Node)](#server-only) or
-[Client/Server Replicated](#clientserver-replicated) designs to bind data, and a
-web application can use one or a mix of these designs as needed. By serializing
-to Apache Arrow, tables are duplicated and synchronized across runtimes
-efficiently.
-
-Perspective is a multi-language platform. The examples in this section use
-Python and JavaScript as an example, but the same general principles apply to
-any `Client`/`Server` combination.
-
-## Client-only
-
-
-
-_For static datasets, datasets provided by the user, and simple server-less and
-read-only web applications._
-
-In this design, Perspective is run as a client Browser WebAssembly library, the
-dataset is downloaded entirely to the client and all calculations and UI
-interactions are performed locally. Interactive performance is very good, using
-WebAssembly engine for near-native runtime plus WebWorker isolation for parallel
-rendering within the browser. Operations like scrolling and creating new views
-are responsive. However, the entire dataset must be downloaded to the client.
-Perspective is not a typical browser component, and datset sizes of 1gb+ in
-Apache Arrow format will load fine with good interactive performance!
-
-Horizontal scaling is a non-issue, since here is no concurrent state to scale,
-and only uses client-side computation via WebAssembly client. Client-only
-perspective can support as many concurrent users as can download the web
-application itself. Once the data is loaded, no server connection is needed and
-all operations occur in the client browser, imparting no additional runtime cost
-on the server beyond initial load. This also means updates and edits are local
-to the browser client and will be lost when the page is refreshed, unless
-otherwise persisted by your application.
-
-As the client-only design starts with creating a client-side Perspective
-`Table`, data can be provided by any standard web service in any Perspective
-compatible format (JSON, CSV or Apache Arrow).
-
-#### Javascript client
-
-```javascript
-const worker = await perspective.worker();
-const table = await worker.table(csv);
-
-const viewer = document.createElement("perspective-viewer");
-document.body.appendChild(viewer);
-await viewer.load(table);
-```
-
-## Client/Server Replicated
-
-
-
-_For medium-sized, real-time, synchronized and/or editable data sets with many
-concurrent users._
-
-The dataset is instantiated in-memory with a Python or Node.js Perspective
-server, and web applications create duplicates of these tables in a local
-WebAssembly client in the browser, synchonized efficiently to the server via
-Apache Arrow. This design scales well with additional concurrent users, as
-browsers only need to download the initial data set and subsequent update
-deltas, while operations like scrolling, pivots, sorting, etc. are performed on
-the client.
-
-Python servers can make especially good use of additional threads, as
-Perspective will release the GIL for almost all operations. Interactive
-performance on the client is very good and identical to client-only
-architecture. Updates and edits are seamlessly synchonized across clients via
-their virtual server counterparts using websockets and Apache Arrow.
-
-#### Python and Tornado server
-
-```python
-from perspective import Server, PerspectiveTornadoHandler
-
-server = Server()
-client = server.new_local_client()
-client.table(csv, name="my_table")
-routes = [(
- r"/websocket",
- perspective.handlers.tornado.PerspectiveTornadoHandler,
- {"perspective_server": server},
-)]
-
-app = tornado.web.Application(routes)
-app.listen(8080)
-loop = tornado.ioloop.IOLoop.current()
-loop.start()
-```
-
-#### Javascript client
-
-Perspective's websocket client interfaces with the Python server. then
-_replicates_ the server-side Table.
-
-```javascript
-const websocket = await perspective.websocket("ws://localhost:8080");
-const server_table = await websocket.open_table("my_table");
-const server_view = await server_table.view();
-
-const worker = await perspective.worker();
-const client_table = await worker.table(server_view);
-
-const viewer = document.createElement("perspective-viewer");
-document.body.appendChild(viewer);
-await viewer.load(client_table);
-```
-
-## Server-only
-
-
-
-_For extremely large datasets with a small number of concurrent users._
-
-The dataset is instantiated in-memory with a Python or Node.js server, and web
-applications connect virtually. Has very good initial load performance, since no
-data is downloaded. Group-by and other operations will run column-parallel if
-configured.
-
-But interactive performance is poor, as every user interaction must page the
-server to render. Operations like scrolling are not as responsive and can be
-impacted by network latency. Web applications must be "always connected" to the
-server via WebSocket. Disconnecting will prevent any interaction, scrolling,
-etc. of the UI. Does not use WebAssembly.
-
-Each connected browser will impact server performance as long as the connection
-is open, which in turn impacts interactive performance of every client. This
-ultimately limits the horizontal scalabity of this architecture. Since each
-client reads the perspective `Table` virtually, changes like edits and updates
-are automatically reflected to all clients and persist across browser refresh.
-Using the same Python server as the previous design, we can simply skip the
-intermediate WebAssembly `Table` and pass the virtual table directly to `load()`
-
-```javascript
-const websocket = await perspective.websocket("ws://localhost:8080");
-const server_table = await websocket.open_table("my_table");
-
-const viewer = document.createElement("perspective-viewer");
-document.body.appendChild(viewer);
-await viewer.load(server_table);
-```
-
-# Feature Flags
-
-The following feature flags are available to enable in your `Cargo.toml`:
-
-- `external-cpp` Set this flag to configure this crate's compile process to
- look for Perspective C++ source code in the environment rather than locally,
- e.g. for when you build this crate in-place in the Perspective repo source
- tree.
diff --git a/rust/perspective-server/src/lib.rs b/rust/perspective-server/src/lib.rs
index 98452d5152..09f9c33921 100644
--- a/rust/perspective-server/src/lib.rs
+++ b/rust/perspective-server/src/lib.rs
@@ -10,8 +10,6 @@
// ┃ of the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0). ┃
// ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
-#![doc = include_str!("../docs/lib_gen.md")]
-
extern crate link_cplusplus;
mod ffi;
diff --git a/rust/perspective-viewer/Cargo.toml b/rust/perspective-viewer/Cargo.toml
index e76fa210b4..4632e65344 100644
--- a/rust/perspective-viewer/Cargo.toml
+++ b/rust/perspective-viewer/Cargo.toml
@@ -46,8 +46,8 @@ anyhow = "1.0.66"
wasm-bindgen-test = "0.3.13"
[dependencies]
-perspective-client = { path = "../perspective-client", version = "3.2.1" }
-perspective-js = { path = "../perspective-js", version = "3.2.1" }
+perspective-client = { version = "3.2.1" }
+perspective-js = { version = "3.2.1" }
# Provides async `Mutex` for locked sections such as `render`
async-lock = "2.5.0"
diff --git a/rust/perspective-viewer/build.js b/rust/perspective-viewer/build.js
index f7f94231ee..2e2576b3b3 100644
--- a/rust/perspective-viewer/build.js
+++ b/rust/perspective-viewer/build.js
@@ -40,35 +40,50 @@ async function build_all() {
// JavaScript
const BUILD = [
+ // WASM assets inlined into a single monolithic `.js` file. No special
+ // loades required, this version of Perspective should be the easiest
+ // to use but also the least performant at load time.
+ // {
+ // 'Import via `
diff --git a/rust/perspective-viewer/test/html/superstore-inline.html b/rust/perspective-viewer/test/html/superstore-inline.html
index 7d9af9d4f9..657b96f1b7 100644
--- a/rust/perspective-viewer/test/html/superstore-inline.html
+++ b/rust/perspective-viewer/test/html/superstore-inline.html
@@ -3,7 +3,7 @@
@@ -24,6 +23,7 @@
+
+
+
diff --git a/rust/perspective-server/docs/architecture.dot b/docs/md/explanation/architecture/architecture.dot
similarity index 100%
rename from rust/perspective-server/docs/architecture.dot
rename to docs/md/explanation/architecture/architecture.dot
diff --git a/rust/perspective-server/docs/architecture.sub1.dot b/docs/md/explanation/architecture/architecture.sub1.dot
similarity index 100%
rename from rust/perspective-server/docs/architecture.sub1.dot
rename to docs/md/explanation/architecture/architecture.sub1.dot
diff --git a/rust/perspective-server/docs/architecture.sub1.svg b/docs/md/explanation/architecture/architecture.sub1.svg
similarity index 100%
rename from rust/perspective-server/docs/architecture.sub1.svg
rename to docs/md/explanation/architecture/architecture.sub1.svg
diff --git a/rust/perspective-server/docs/architecture.sub2.dot b/docs/md/explanation/architecture/architecture.sub2.dot
similarity index 100%
rename from rust/perspective-server/docs/architecture.sub2.dot
rename to docs/md/explanation/architecture/architecture.sub2.dot
diff --git a/rust/perspective-server/docs/architecture.sub2.svg b/docs/md/explanation/architecture/architecture.sub2.svg
similarity index 100%
rename from rust/perspective-server/docs/architecture.sub2.svg
rename to docs/md/explanation/architecture/architecture.sub2.svg
diff --git a/rust/perspective-server/docs/architecture.sub3.dot b/docs/md/explanation/architecture/architecture.sub3.dot
similarity index 100%
rename from rust/perspective-server/docs/architecture.sub3.dot
rename to docs/md/explanation/architecture/architecture.sub3.dot
diff --git a/rust/perspective-server/docs/architecture.sub3.svg b/docs/md/explanation/architecture/architecture.sub3.svg
similarity index 100%
rename from rust/perspective-server/docs/architecture.sub3.svg
rename to docs/md/explanation/architecture/architecture.sub3.svg
diff --git a/rust/perspective-server/docs/architecture.svg b/docs/md/explanation/architecture/architecture.svg
similarity index 100%
rename from rust/perspective-server/docs/architecture.svg
rename to docs/md/explanation/architecture/architecture.svg
diff --git a/docs/md/explanation/architecture/client_only.md b/docs/md/explanation/architecture/client_only.md
new file mode 100644
index 0000000000..7db35eca0e
--- /dev/null
+++ b/docs/md/explanation/architecture/client_only.md
@@ -0,0 +1,39 @@
+# Client-only
+
+
+
+_For static datasets, datasets provided by the user, and simple server-less and
+read-only web applications._
+
+In this design, Perspective is run as a client Browser WebAssembly library, the
+dataset is downloaded entirely to the client and all calculations and UI
+interactions are performed locally. Interactive performance is very good, using
+WebAssembly engine for near-native runtime plus WebWorker isolation for parallel
+rendering within the browser. Operations like scrolling and creating new views
+are responsive. However, the entire dataset must be downloaded to the client.
+Perspective is not a typical browser component, and datset sizes of 1gb+ in
+Apache Arrow format will load fine with good interactive performance!
+
+Horizontal scaling is a non-issue, since here is no concurrent state to scale,
+and only uses client-side computation via WebAssembly client. Client-only
+perspective can support as many concurrent users as can download the web
+application itself. Once the data is loaded, no server connection is needed and
+all operations occur in the client browser, imparting no additional runtime cost
+on the server beyond initial load. This also means updates and edits are local
+to the browser client and will be lost when the page is refreshed, unless
+otherwise persisted by your application.
+
+As the client-only design starts with creating a client-side Perspective
+`Table`, data can be provided by any standard web service in any Perspective
+compatible format (JSON, CSV or Apache Arrow).
+
+#### Javascript client
+
+```javascript
+const worker = await perspective.worker();
+const table = await worker.table(csv);
+
+const viewer = document.createElement("perspective-viewer");
+document.body.appendChild(viewer);
+await viewer.load(table);
+```
diff --git a/docs/md/explanation/architecture/client_server.md b/docs/md/explanation/architecture/client_server.md
new file mode 100644
index 0000000000..f6df4408eb
--- /dev/null
+++ b/docs/md/explanation/architecture/client_server.md
@@ -0,0 +1,58 @@
+# Client/Server replicated
+
+
+
+_For medium-sized, real-time, synchronized and/or editable data sets with many
+concurrent users._
+
+The dataset is instantiated in-memory with a Python or Node.js Perspective
+server, and web applications create duplicates of these tables in a local
+WebAssembly client in the browser, synchonized efficiently to the server via
+Apache Arrow. This design scales well with additional concurrent users, as
+browsers only need to download the initial data set and subsequent update
+deltas, while operations like scrolling, pivots, sorting, etc. are performed on
+the client.
+
+Python servers can make especially good use of additional threads, as
+Perspective will release the GIL for almost all operations. Interactive
+performance on the client is very good and identical to client-only
+architecture. Updates and edits are seamlessly synchonized across clients via
+their virtual server counterparts using websockets and Apache Arrow.
+
+#### Python and Tornado server
+
+```python
+from perspective import Server, PerspectiveTornadoHandler
+
+server = Server()
+client = server.new_local_client()
+client.table(csv, name="my_table")
+routes = [(
+ r"/websocket",
+ perspective.handlers.tornado.PerspectiveTornadoHandler,
+ {"perspective_server": server},
+)]
+
+app = tornado.web.Application(routes)
+app.listen(8080)
+loop = tornado.ioloop.IOLoop.current()
+loop.start()
+```
+
+#### Javascript client
+
+Perspective's websocket client interfaces with the Python server. then
+_replicates_ the server-side Table.
+
+```javascript
+const websocket = await perspective.websocket("ws://localhost:8080");
+const server_table = await websocket.open_table("my_table");
+const server_view = await server_table.view();
+
+const worker = await perspective.worker();
+const client_table = await worker.table(server_view);
+
+const viewer = document.createElement("perspective-viewer");
+document.body.appendChild(viewer);
+await viewer.load(client_table);
+```
diff --git a/docs/md/explanation/architecture/server_only.md b/docs/md/explanation/architecture/server_only.md
new file mode 100644
index 0000000000..09084c4b70
--- /dev/null
+++ b/docs/md/explanation/architecture/server_only.md
@@ -0,0 +1,33 @@
+# Server-only
+
+
+
+_For extremely large datasets with a small number of concurrent users._
+
+The dataset is instantiated in-memory with a Python or Node.js server, and web
+applications connect virtually. Has very good initial load performance, since no
+data is downloaded. Group-by and other operations will run column-parallel if
+configured.
+
+But interactive performance is poor, as every user interaction must page the
+server to render. Operations like scrolling are not as responsive and can be
+impacted by network latency. Web applications must be "always connected" to the
+server via WebSocket. Disconnecting will prevent any interaction, scrolling,
+etc. of the UI. Does not use WebAssembly.
+
+Each connected browser will impact server performance as long as the connection
+is open, which in turn impacts interactive performance of every client. This
+ultimately limits the horizontal scalabity of this architecture. Since each
+client reads the perspective `Table` virtually, changes like edits and updates
+are automatically reflected to all clients and persist across browser refresh.
+Using the same Python server as the previous design, we can simply skip the
+intermediate WebAssembly `Table` and pass the virtual table directly to `load()`
+
+```javascript
+const websocket = await perspective.websocket("ws://localhost:8080");
+const server_table = await websocket.open_table("my_table");
+
+const viewer = document.createElement("perspective-viewer");
+document.body.appendChild(viewer);
+await viewer.load(server_table);
+```
diff --git a/docs/md/explanation/javascript.md b/docs/md/explanation/javascript.md
new file mode 100644
index 0000000000..ffea687dde
--- /dev/null
+++ b/docs/md/explanation/javascript.md
@@ -0,0 +1,15 @@
+Perspective's JavaScript library offers a configurable UI powered by the same
+fast streaming data engine, just re-compiled to WebAssembly. A simple example
+which loads an [Apache Arrow](https://arrow.apache.org/) and computes a "Group
+By" operation, returning a new Arrow:
+
+```javascript
+import perspective from "@finos/perspective";
+
+const table = await perspective.table(apache_arrow_data);
+const view = await table.view({ group_by: ["CounterParty", "Security"] });
+const arrow = await view.to_arrow();
+```
+
+[More Examples](https://github.com/finos/perspective/tree/master/examples) are
+available on GitHub.
diff --git a/docs/md/explanation/javascript_builds.md b/docs/md/explanation/javascript_builds.md
new file mode 100644
index 0000000000..26efe0ae4c
--- /dev/null
+++ b/docs/md/explanation/javascript_builds.md
@@ -0,0 +1,42 @@
+# JavaScript Builds
+
+Perspective requires the browser to have access to Perspective's `.wasm`
+binaries _in addition_ to the bundled `.js` files, and as a result the build
+process requires a few extra steps. To ease integration, Perspective's NPM
+releases come with multiple prebuilt configurations.
+
+## Browser
+
+### ESM Builds
+
+The recommended builds for production use are packaged as ES Modules and require
+a _bootstrapping_ step in order to acquire the `.wasm` binaries and initialize
+Perspective's JavaScript with them. However, because they have no hard-coded
+dependencies on the `.wasm` paths, they are ideal for use with JavaScript
+bundlers such as ESBuild, Rollup, Vite or Webpack.
+
+### CDN Builds
+
+Perspective's CDN builds are good for non-bundled scenarios, such as importing
+directly from a `
+```
+
+## Node.js builds
+
+The Node.js runtime for the `@finos/perspective` module runs in-process by
+default and does not implement a `child_process` interface. Hence, there is no
+`worker()` method, and the module object itself directly exports the full
+`perspective` API.
+
+```javascript
+const perspective = require("@finos/perspective");
+```
+
+In Node.js, perspective does not run in a WebWorker (as this API does not exist
+in Node.js), so no need to call the `.worker()` factory function - the
+`perspective` library exports the functions directly and run synchronously in
+the main process.
diff --git a/docs/md/how_to/javascript/installation.md b/docs/md/how_to/javascript/installation.md
new file mode 100644
index 0000000000..0083fecc9d
--- /dev/null
+++ b/docs/md/how_to/javascript/installation.md
@@ -0,0 +1,31 @@
+# JavaScript NPM Installation
+
+Perspective releases contain several different builds for use in most
+environments.
+
+## Browser
+
+Perspective's WebAssembly data engine is available via NPM in the same package
+as its Node.js counterpart, `@finos/perspective`. The Perspective Viewer UI
+(which has no Node.js component) must be installed separately:
+
+```bash
+$ npm add @finos/perspective @finos/perspective-viewer
+```
+
+By itself, `@finos/perspective-viewer` does not provide any visualizations, only
+the UI framework. Perspective _Plugins_ provide visualizations and must be
+installed separately. All Plugins are optional - but a `
-
-Application developers can choose from [Client (WebAssembly)](#client-only),
-[Server (Python/Node)](#server-only) or
-[Client/Server Replicated](#clientserver-replicated) designs to bind data, and a
-web application can use one or a mix of these designs as needed. By serializing
-to Apache Arrow, tables are duplicated and synchronized across runtimes
-efficiently.
-
-Perspective is a multi-language platform. The examples in this section use
-Python and JavaScript as an example, but the same general principles apply to
-any `Client`/`Server` combination.
-
-## Client-only
-
-