# Perspective
> Perspective is an open-source, WebAssembly-powered data grid, pivot table and charting component for large, real-time and streaming datasets — in the browser, Python, Jupyter, Node.js and Rust.
[](https://github.com/perspective-dev/perspective/actions/workflows/build.yaml)
[](https://www.npmjs.com/package/@perspective-dev/client)
[](https://pypi.python.org/pypi/perspective-python)
[](https://crates.io/crates/perspective)
Perspective is an open-source data grid, pivot table and charting component for
large, real-time and streaming datasets. Build user-configurable reports,
dashboards, notebooks and embedded analytics applications, backed by a
high-performance streaming query engine that runs in-browser via WebAssembly or
server-side in Python, Node.js and Rust — or delegates to a database you
already have.
## Features
- A data-reactive UI packaged as a
[Custom Element](https://developer.mozilla.org/en-US/docs/Web/Web_Components/Using_custom_elements),
with drag-and-drop query and layout configuration. Includes a virtual-scrolling,
editable data grid, WebGL charting engine with 15+ chart types, tile-based
geographic maps, full theme support, and [React](https://react.dev/) bindings.
- A fast, memory-efficient streaming query engine written in C++ and compiled
for [WebAssembly](https://webassembly.org/) (including a 64-bit `memory64`
build for in-browser datasets larger than 4GB, and support for paging
columns to disk via OPFS in the browser or memory-mapped files natively),
[Python](https://www.python.org/) and [Rust](https://www.rust-lang.org/).
Tables update incrementally and views tick in real time, with reactive
joins across tables, a columnar expression language based on
[ExprTK](https://github.com/ArashPartow/exprtk), and read/write/streaming
support for [Apache Arrow](https://arrow.apache.org/), CSV and JSON.
- A symmetric client/server architecture — the same Client API connects to an
engine in-process, in a Web Worker, or remotely over WebSocket, with server
bindings for Python (aiohttp, Starlette, Tornado), Node.js and Rust.
Datasets can be mirrored to the browser for fluid interaction or virtualized
server-side, streaming only what's visible.
- Virtual servers that run Perspective's UI directly on external engines like
[DuckDB](https://duckdb.org/), [ClickHouse](https://clickhouse.com/),
[PostgreSQL](https://www.postgresql.org/) and
[Polars](https://pola.rs/), translating view configurations into native
queries — no ETL or data copy required.
- A [Jupyter](https://jupyter.org/) widget built on
[anywidget](https://anywidget.dev/) and a Python client library for
interactive data analysis in JupyterLab and other notebook environments.
## Documentation
- [Project Site](https://perspective-dev.github.io/)
- [User Guide](https://perspective-dev.github.io/guide/)
- JavaScript API
- [`@perspective-dev/react` React Component](https://perspective-dev.github.io/react/index.html)
- [`@perspective-dev/viewer` Web Component](https://perspective-dev.github.io/viewer/modules/perspective-viewer.html)
- [`@perspective-dev/client` Client (Browser)](https://perspective-dev.github.io/browser/modules/src_ts_perspective.browser.ts.html)
- [`@perspective-dev/client` Client (Node.js)](https://perspective-dev.github.io/node/modules/src_ts_perspective.node.ts.html)
- [`@perspective-dev/client` Clickhouse Virtual Server](https://perspective-dev.github.io/browser/modules/dist_esm_virtual_servers_clickhouse.js.html)
- [`@perspective-dev/client` DuckDB Virtual Server](https://perspective-dev.github.io/browser/modules/dist_esm_virtual_servers_duckdb.js.html)
- Python API
- [`perspective`](https://perspective-dev.github.io/python/index.html)
- [`perspective.widget`](https://perspective-dev.github.io/python/perspective/widget.html)
- [`perspective.handlers.aiohttp`](https://perspective-dev.github.io/python/perspective/handlers/aiohttp.html)
- [`perspective.handlers.starlette`](https://perspective-dev.github.io/python/perspective/handlers/starlette.html)
- [`perspective.handlers.tornado`](https://perspective-dev.github.io/python/perspective/handlers/tornado.html)
- [`perspective.virtual_servers.clickhouse`](https://perspective-dev.github.io/python/perspective/virtual_servers/clickhouse.html)
- [`perspective.virtual_servers.duckdb`](https://perspective-dev.github.io/python/perspective/virtual_servers/duckdb.html)
- [`perspective.virtual_servers.postgres`](https://perspective-dev.github.io/python/perspective/virtual_servers/postgres.html)
- Rust API
- [`perspective`](https://docs.rs/perspective/latest/perspective/)
## Examples
The Perspective project is a member of the
[The OpenJS Foundation](https://openjsf.org/).
Copyright [OpenJS Foundation](https://openjsf.org) and Perspective contributors.
All rights reserved. The [OpenJS Foundation](https://openjsf.org) has registered
trademarks and uses trademarks. For a list of trademarks of the
[OpenJS Foundation](https://openjsf.org), please see our
[Trademark Policy](https://trademark-policy.openjsf.org/) and
[Trademark List](https://trademark-list.openjsf.org/). Trademarks and logos not
indicated on the
[list of OpenJS Foundation trademarks](https://trademark-list.openjsf.org) are
trademarks™ or registered® trademarks of their respective holders. Use of them
does not imply any affiliation with or endorsement by them.
[The OpenJS Foundation](https://openjsf.org/) |
[Terms of Use](https://terms-of-use.openjsf.org/) |
[Privacy Policy](https://privacy-policy.openjsf.org/) |
[Bylaws](https://bylaws.openjsf.org/) |
[Code of Conduct](https://code-of-conduct.openjsf.org) |
[Trademark Policy](https://trademark-policy.openjsf.org/) |
[Trademark List](https://trademark-list.openjsf.org/) |
[Cookie Policy](https://www.linuxfoundation.org/cookies/)
# Real-time dashboards over WebSocket
A real-time dashboard is a set of tables and charts which stay current as the
data behind them changes, without the user reloading. Perspective is built for
this: a [`Table`](../explanation/table.md) accepts streaming
[`update()`](../explanation/table/update_and_remove.md) calls, every
[`View`](../explanation/view.md) over it — grouped, pivoted, filtered or
sorted — is maintained incrementally, and `` repaints only
what changed.
There is no polling and no query re-execution. An update of 50 rows to a 10
million row table costs work proportional to the 50 rows.
## Architecture
1. A server process owns the `Table` and writes to it as new data arrives —
from a message queue, a market data feed, a database change stream, or a
timer.
2. The server exposes that `Table` by name on a WebSocket endpoint.
3. Each browser opens the `Table` by name and loads it into a
``. The user configures their own grouping, filters and
chart type; each browser gets its own `View`.
Perspective offers two ways to split this work between server and browser,
covered in [Data Architecture](../explanation/architecture.md):
- **[Client/server replicated](../explanation/architecture/client_server.md)**
— the browser keeps a synchronized copy of the table in WebAssembly. Queries
run locally, so interaction is instant and the server only ships deltas. Best
when the dataset fits in browser memory.
- **[Server only](../explanation/architecture/server_only.md)** — queries run
on the server and the browser receives only the visible window of rows. Best
for very large tables or thin clients.
## A Python server
```python
import threading
import time
import tornado.ioloop
import tornado.web
from perspective import Server
from perspective.handlers.tornado import PerspectiveTornadoHandler
server = Server()
client = server.new_local_client()
table = client.table(
{"symbol": "string", "price": "float", "time": "datetime"},
name="prices",
)
def feed():
while True:
table.update(next_batch())
time.sleep(0.05)
threading.Thread(target=feed, daemon=True).start()
app = tornado.web.Application([
(r"/websocket", PerspectiveTornadoHandler, {"perspective_server": server}),
])
app.listen(8080)
tornado.ioloop.IOLoop.current().start()
```
Perspective's Python API is thread-safe and releases the GIL, so the feed can
run on its own thread; see [Multithreading](../how_to/python/multithreading.md).
Handlers are also provided for
[Starlette/FastAPI and aiohttp](../how_to/python/websocket.md).
## The browser
```html
```
This is server-only mode. For replicated mode, create a `View` on the server
table and build a local table from it — `worker.table(server_view)` — as shown
in [Hosting a WebSocket server](../how_to/python/websocket.md).
## Keeping a rolling window
For feeds which never end, bound the table. An
[`index`](../explanation/table/options.md) makes updates replace rows by key
(latest price per symbol); a `limit` keeps only the most recent _n_ rows
(a rolling tick history).
## A Node.js server
The same server can be written in Node.js with
[`WebSocketServer`](../how_to/javascript/nodejs_server.md), or in Rust — see the
[`rust-axum` example](https://github.com/perspective-dev/perspective/tree/master/examples/rust-axum).
## See it running
- [Market](https://perspective-dev.github.io/gallery/market-trading-desk.html)
— a simulated order book streaming into a blotter, depth chart and
candlestick chart.
- [`python-tornado-streaming`](https://github.com/perspective-dev/perspective/tree/master/examples/python-tornado-streaming)
— the complete version of the server above.
# Streaming pivot tables
A pivot table groups rows by one set of columns, splits them across another,
and aggregates the cells. A _streaming_ pivot table keeps that result correct
as the underlying rows are inserted, updated and removed — without recomputing
the whole pivot.
In Perspective a pivot is a [`View`](../explanation/view.md) with `group_by`
and `split_by`:
```javascript
const view = await table.view({
group_by: ["Region", "State"],
split_by: ["Category"],
columns: ["Sales", "Profit"],
aggregates: { Sales: "sum", Profit: "avg" },
sort: [["Sales", "desc"]],
});
```
When [`table.update()`](../explanation/table/update_and_remove.md) is called,
the engine applies the delta to only the affected groups and notifies
subscribers:
```javascript
view.on_update(async (updated) => {
const rows = await view.to_json();
}, { mode: "row" });
```
Loaded into ``, the same configuration is an interactive
pivot grid: users drag columns between _Group By_, _Split By_, _Order By_ and
_Where_, expand and collapse row groups, and switch to a chart of the same
pivot.
```javascript
await viewer.load(table);
await viewer.restore({
plugin: "Datagrid",
group_by: ["Region", "State"],
split_by: ["Category"],
columns: ["Sales", "Profit"],
});
```
## What can be pivoted
- **Row pivots** — any number of [`group_by`](../explanation/view/config/grouping_and_pivots.md)
levels, rendered as an expandable tree with subtotals at each level.
- **Column pivots** — any number of `split_by` levels, rendered as grouped
column headers.
- **Aggregates** — sum, count, distinct count, average, weighted mean, median,
min/max, first/last, standard deviation, variance and more, chosen per
column.
- **Computed columns** — [`expressions`](../explanation/view/config/expressions.md)
can be grouped, split, aggregated and filtered like any other column, so
bucketing a datetime by month or binning a number is one expression.
- **Window columns** — [running totals, ranks, lags and rates](../explanation/view/config/windows.md).
- **Joins** — pivot over a [reactive join](../explanation/join.md) of two
streaming tables.
## Where the pivot runs
The same pivot API runs in the browser (WebAssembly), in Node.js, in Python
and in Rust. It can also be delegated to a database: with a
[virtual server](../explanation/virtual_servers.md), a `group_by`/`split_by`
configuration is translated to SQL and executed by DuckDB, ClickHouse or
PostgreSQL.
## Licensing
Row and column pivoting, aggregation, charting of pivots and server-side
virtualization are all part of Perspective's Apache-2.0 open source
distribution. There is no commercial tier.
## Examples
- [Pivot by 2 row levels and 2 column levels](https://perspective-dev.github.io/gallery/feature-05-both-2.html)
- [Superstore workspace](https://perspective-dev.github.io/gallery/superstore-overview.html)
- [All examples](https://perspective-dev.github.io/gallery/index.html)
# Visualizing millions of rows in the browser
Most JavaScript data grids and charting libraries hold rows as JavaScript
objects and lay out one DOM or SVG node per datum, which stalls somewhere
between ten thousand and a few hundred thousand rows. Perspective takes a
different approach at each layer:
- **Columnar engine in WebAssembly.** Data is stored in typed, columnar
buffers inside a C++ query engine compiled to WebAssembly and run in a Web
Worker, off the main thread. Strings are dictionary-encoded. Nothing is
materialized as JavaScript objects unless you ask for it.
- **Apache Arrow in, Apache Arrow out.** A [`Table`](../explanation/table.md)
loads [Arrow](../explanation/table/loading_data.md) directly into those
buffers without per-row parsing, and `View.to_arrow()` exports the same way.
CSV and JSON are also supported.
- **Queries, not rows, cross the boundary.** Grouping, pivoting, filtering and
sorting happen inside the engine. The UI requests only the window of the
result it is about to draw.
- **Virtual-scrolling data grid.** The
[Datagrid](https://www.npmjs.com/package/@perspective-dev/viewer-datagrid)
renders only visible cells, so scrolling a 10 million row grid costs the same
as scrolling a 100 row grid.
- **WebGL charts.** Scatter, heatmap, line and map plugins draw on the GPU,
staying interactive at point counts where SVG and 2D canvas charts do not.
## Loading a large file
```javascript
const worker = await perspective.worker();
const response = await fetch("/data/trips.arrow");
const table = await worker.table(await response.arrayBuffer());
await document.querySelector("perspective-viewer").load(table);
```
Prefer Arrow (optionally LZ4 or ZSTD compressed) over CSV or JSON for large
datasets: it is smaller on the wire, carries its own schema, and skips type
inference.
## More than 4GB: Memory64
32-bit WebAssembly caps the engine's heap at 4GB.
`@perspective-dev/server` also ships a
[Memory64 build](../how_to/javascript/importing.md) which raises the ceiling
to 16GB in browsers which support it. Register both binaries and only the one
the browser selects is downloaded:
```javascript
perspective.init_server({
wasm32: () => fetch(SERVER_WASM),
wasm64: () => fetch(SERVER_WASM64),
});
```
## More than memory: `page_to_disk`
A `Table` normally lives in the engine's memory. Created with
[`page_to_disk`](../explanation/table/options.md#page_to_disk), its columns
are backed by on-disk storage instead — the browser's Origin Private File
System under WebAssembly, memory-mapped files in Python and Rust — and the
engine evicts the coldest columns to it when it is over its
resident memory budget (1 GiB by default in the browser), reading them back
when a query needs them.
```javascript
const table = await worker.table(await response.arrayBuffer(), {
page_to_disk: true,
});
```
Everything else about the `Table` is unchanged, including streaming updates
and the viewer on top of it. Use it for wide tables where users touch a few
columns at a time, or to hold several large tables in one tab; leave it off
for data which fits, since a query over evicted columns pays to read them
back.
## How many rows?
It depends on column count and types more than row count — numeric and
datetime columns are compact, high-cardinality strings are not. Millions of
rows is routine; tens of millions is practical for narrow numeric tables. Free
memory by calling [`delete()`](../how_to/javascript/deleting.md) on views and
tables you no longer need, and consider `page_to_disk` for tables which are
large but only partly in use at any moment.
## When the data does not fit in the browser
Keep it on a server and send the browser only what is visible:
- **[Server-only mode](../explanation/architecture/server_only.md)** — the
same engine, running natively in Python, Node.js or Rust, with the browser
connected over WebSocket.
- **[Virtual servers](../explanation/virtual_servers.md)** — no Perspective
engine at all. View configurations are translated to SQL and run by DuckDB,
ClickHouse, PostgreSQL or Polars, which can be as large as those systems
allow.
In both modes the viewer, its configuration and its saved layouts are
identical to the in-browser case.
## Examples
- [Olympics](https://perspective-dev.github.io/gallery/olympics.html) — 120
years of athlete records loaded from Arrow and pivoted in-browser.
- [NYPD CCRB](https://perspective-dev.github.io/gallery/nypd.html) —
complaint records cross-filtered across grid and heatmaps.
# Trading blotters, order books and market data
Perspective was originally developed at J.P. Morgan and open sourced through
[FINOS](https://www.finos.org/), the Fintech Open Source Foundation, before
joining the [OpenJS Foundation](https://openjsf.org/). Financial market
data is the workload it was designed around: wide tables, high update rates,
keyed replacement of rows, and users who need to re-slice the data themselves
while it is moving.
## The building blocks
**A blotter** is an indexed table in a data grid. With
[`index`](../explanation/table/options.md) set to the order or trade id, each
`update()` replaces that row in place; partial updates (only the changed
fields) are supported, and `remove()` deletes by key.
```javascript
const table = await worker.table(
{ id: "string", symbol: "string", side: "string", price: "float", qty: "integer", status: "string", time: "datetime" },
{ index: "id" },
);
table.update([{ id: "o-1841", status: "filled" }]);
```
**An order book** is a pivot of that same table: `group_by` price,
`split_by` side, `sum` of quantity, filtered to open orders.
```javascript
await viewer.restore({
plugin: "X Bar",
group_by: ["price"],
split_by: ["side"],
columns: ["qty"],
filter: [["status", "==", "open"]],
});
```
**Candlesticks** are a pivot too: `group_by` a time bucket expression, with
`first`, `last`, `high` and `low` aggregates over aliases of the price column,
drawn by the Candlestick or OHLC plugin.
```javascript
await viewer.restore({
plugin: "Candlestick",
group_by: ["bucket(\"time\", 'm')"],
columns: ["open", "close", "high", "low"],
expressions: {
"bucket(\"time\", 'm')": "bucket(\"time\", 'm')",
open: '"price"',
close: '"price"',
high: '"price"',
low: '"price"',
},
aggregates: { open: "first", close: "last", high: "high", low: "low" },
});
```
Because every one of these is a `View` over one streaming `Table`, they stay
mutually consistent tick by tick, and users can change any of them — regroup
by sector, filter to a book, switch the blotter to a heatmap — without code.
## Conditional formatting
The data grid supports per-column number formatting, positive/negative
foreground and background colors, gradients and in-cell bars, all set from the
column settings panel and captured in the saved configuration.
## Deployment shapes
- **Desktop containers and internal web apps** — `` is a
standard Web Component with no framework dependency, and ships
[React bindings](../how_to/javascript/react.md).
- **Python services** — host tables from
[Tornado, FastAPI/Starlette or aiohttp](../how_to/python/websocket.md);
ingest `pandas`, `polars` or `pyarrow` directly.
- **ClickHouse, DuckDB, PostgreSQL, Polars** — put the UI directly over the
tick store with a [virtual server](../explanation/virtual_servers.md).
- **Research notebooks** — the same widget in
[Jupyter](../how_to/python/jupyterlab.md).
## Examples
- [Market](https://perspective-dev.github.io/gallery/market-trading-desk.html)
— blotter, order book chart and candlesticks over one simulated feed.
- [Market — Orders](https://perspective-dev.github.io/gallery/market-order-flow.html)
# Interactive pivot tables and charts in Jupyter
`PerspectiveWidget` puts the full `` UI in a notebook
cell. Pass it a DataFrame and you get a sortable, filterable data grid; drag a
column to _Group By_ and it becomes a pivot table; pick a chart type and it
becomes a bar, line, scatter, heatmap, treemap or map — all without writing
plotting code or re-running the cell.
```bash
pip install "perspective-python[jupyter]"
```
```python
import pandas as pd
from perspective.widget import PerspectiveWidget
df = pd.read_parquet("trips.parquet")
PerspectiveWidget(df)
```
It is built on [anywidget](https://anywidget.dev/), so the same wheel works in
JupyterLab, classic Jupyter Notebook, VS Code notebooks, Google Colab and
marimo, with no separate lab extension to install or version-match.
## Why use it instead of `df.head()` or a plotting library
- **The whole DataFrame, not a preview.** The grid virtual-scrolls, so a
multi-million row frame is browsable, sortable and filterable in place.
- **Exploration without code.** Grouping, pivoting, aggregating, filtering and
charting are drag-and-drop. Computed columns use a built-in
[expression language](../explanation/view/config/expressions.md).
- **Reproducible.** Every choice made in the UI is a keyword argument, so an
exploration can be frozen back into the cell:
```python
PerspectiveWidget(
df,
plugin="Heatmap",
group_by=["pickup_hour"],
split_by=["weekday"],
columns=["fare"],
aggregates={"fare": "avg"},
)
```
- **Live.** Pass a `perspective.Table` instead of a DataFrame and call
`table.update()` from another cell or thread; the widget ticks in real time.
## pandas, polars and pyarrow
`pandas.DataFrame`, `polars.DataFrame`, `pyarrow.Table`, Arrow IPC bytes, CSV
strings, and lists or dicts of Python values are all accepted directly; see
[DataFrame and Arrow compatibility](../how_to/python/table_data.md).
## Where the data lives
By default (`binding_mode="server"`) the data stays in the Python kernel and
the browser is streamed only the window of rows it is displaying, so very
large frames open quickly. For the most fluid interaction on small and medium
data, `binding_mode="client-server"` additionally replicates the table into
the browser's WebAssembly engine:
```python
PerspectiveWidget(df, binding_mode="client-server")
```
For frames which strain the kernel's memory, build the table with
[`page_to_disk`](../explanation/table/options.md#page_to_disk) so its columns
are memory-mapped from disk, and pass the table to the widget:
```python
import perspective
table = perspective.table(df, page_to_disk=True)
PerspectiveWidget(table)
```
See [`PerspectiveWidget` for notebooks](../how_to/python/jupyterlab.md) for the
full widget API.
## From notebook to application
The engine and UI in the notebook are the same ones used in production web
applications. A configuration explored in Jupyter can be saved as JSON and
restored in a `` served from
[Tornado, FastAPI or aiohttp](../how_to/python/websocket.md).
# A pivot and charting UI for DuckDB, ClickHouse and PostgreSQL
If your data already lives in an analytical database, you do not need to load
it into Perspective's engine to explore it. A
[virtual server](../explanation/virtual_servers.md) implements Perspective's
protocol on top of an external engine: when a user drags a column to _Group
By_, adds a filter or scrolls the grid, the resulting `View` configuration is
translated into a native query, executed by the database, and only the visible
window of the result is returned.
The user sees the same `` — data grid, pivot table, WebGL
charts, maps, saved layouts — and the database does the work.
| Engine | Where it runs | Guide |
| --- | --- | --- |
| DuckDB-WASM | In the browser, no server | [JavaScript](../how_to/javascript/virtual_server/duckdb.md) |
| DuckDB | Python server | [Python](../how_to/python/virtual_server/duckdb.md) |
| ClickHouse | Browser or Python server | [JavaScript](../how_to/javascript/virtual_server/clickhouse.md), [Python](../how_to/python/virtual_server/clickhouse.md) |
| PostgreSQL | Python server | [Python](../how_to/python/virtual_server/postgres.md) |
| Polars | Python server | [Python](../how_to/python/virtual_server/polars.md) |
| Anything else | Your code | [Custom virtual servers](../how_to/javascript/virtual_server/custom.md) |
## DuckDB in Python
```python
import duckdb
import tornado.ioloop
import tornado.web
from perspective.handlers.tornado import PerspectiveTornadoHandler
from perspective.virtual_servers.duckdb import DuckDBVirtualServer
conn = duckdb.connect()
conn.execute("CREATE TABLE trips AS SELECT * FROM 'trips/*.parquet'")
app = tornado.web.Application([
(r"/websocket", PerspectiveTornadoHandler, {
"perspective_server": DuckDBVirtualServer(conn),
}),
])
app.listen(8080)
tornado.ioloop.IOLoop.current().start()
```
```javascript
const websocket = await perspective.websocket("ws://localhost:8080/websocket");
const table = await websocket.open_table("trips");
document.querySelector("perspective-viewer").load(table);
```
## DuckDB-WASM, entirely in the browser
With DuckDB-WASM the whole stack — database, query translation and UI — runs
in the browser tab. Because Perspective does not intercept your SQL, DuckDB's
own Parquet, S3 and HTTP readers are available for loading data. See the
[DuckDB-WASM guide](../how_to/javascript/virtual_server/duckdb.md).
DuckDB-WASM can also attach a whole
[DuckLake](https://ducklake.select/) lakehouse over HTTPS; see the
[DuckLake case study](./ducklake.md).
## When to use a virtual server, and when not to
Use a virtual server when the data is larger than memory, already lives in
the database, or must not leave it. Use Perspective's own engine when the data
is _streaming_: its `Table` applies `update()` calls incrementally and pushes
changes to every view, which a request/response SQL engine does not do.
The two can be mixed — one `` workspace can hold panels
backed by different engines.
## Examples
- [`python-duckdb-virtual`](https://github.com/perspective-dev/perspective/tree/master/examples/python-duckdb-virtual)
- [`esbuild-duckdb-virtual`](https://github.com/perspective-dev/perspective/tree/master/examples/esbuild-duckdb-virtual)
- [`python-clickhouse-virtual`](https://github.com/perspective-dev/perspective/tree/master/examples/python-clickhouse-virtual)
- [`python-postgres-virtual`](https://github.com/perspective-dev/perspective/tree/master/examples/python-postgres-virtual)
- [`python-polars-virtual`](https://github.com/perspective-dev/perspective/tree/master/examples/python-polars-virtual)
# Case study: a multi-billion row tick history in a browser tab with DuckLake and DuckDB-WASM
[DuckLake](https://ducklake.select/) is an open lakehouse format which keeps
table data in Parquet files and _all_ metadata — schemas, snapshots, file
lists, statistics — in an ordinary SQL database. With a DuckDB file as that
database, an entire lakehouse catalog is one static file.
This case study puts a self-service analytics UI on a market data lake — a
tick-by-tick trade and price history, partitioned by symbol and trade date —
with **no backend at all**. The figures below are rounded from measurements
against a real _frozen_ (read-only) DuckLake of comparable size and layout:
| | |
| --- | --- |
| Rows | ~3 billion |
| Parquet files | Hundreds of thousands, partitioned by symbol and trade date |
| Data size | Tens of gigabytes, in object storage |
| Catalog | One `.ducklake` file of tens of megabytes, on a static host |
| Servers operated | None |
The browser runs three things:
1. [DuckDB-WASM](https://duckdb.org/docs/current/clients/wasm/overview), with
the `ducklake` extension, as the query engine.
2. Perspective's [DuckDB virtual server](../how_to/javascript/virtual_server/duckdb.md),
which translates `` configurations into DuckDB SQL.
3. ``, as the data grid, pivot table and charts.
```text
static hosting / object storage browser tab
┌─────────────────────────────────┐ ┌──────────────────────────────────┐
│ ticks.ducklake (catalog) │◄──────┤ DuckDB-WASM + ducklake extension │
│ trades/symbol=…/date=…/*.parquet│ HTTPS │ ▲ SQL │
└─────────────────────────────────┘ range │ Perspective DuckDB virtual server│
│ ▲ view config │
│ │
└──────────────────────────────────┘
```
## 1. Attach the lake
```javascript
import perspective from "@perspective-dev/client";
import "@perspective-dev/viewer";
import "@perspective-dev/viewer-datagrid";
import "@perspective-dev/viewer-charts";
import * as duckdb from "@duckdb/duckdb-wasm";
import { DuckDBHandler } from "@perspective-dev/client/dist/esm/virtual_servers/duckdb.js";
const bundle = await duckdb.selectBundle(duckdb.getJsDelivrBundles());
const worker_url = URL.createObjectURL(
new Blob([`importScripts("${bundle.mainWorker}");`], {
type: "text/javascript",
}),
);
const db = new duckdb.AsyncDuckDB(new duckdb.VoidLogger(), new Worker(worker_url));
await db.instantiate(bundle.mainModule, bundle.pthreadWorker);
URL.revokeObjectURL(worker_url);
const conn = await db.connect();
await conn.query(`SET default_null_order=NULLS_FIRST_ON_ASC_LAST_ON_DESC;`);
await conn.query(`
ATTACH 'ducklake:https://data.example.com/ticks.ducklake' AS lake;
`);
```
That is the whole connection. The `ducklake` extension is fetched and loaded
automatically by the `ATTACH`, and the attach takes one to three seconds:
DuckDB reads the catalog by HTTP range request rather than downloading the
whole file. A lake which records absolute `s3://` or `https://` paths for its data
files needs nothing else; one written with relative paths also needs
`DATA_PATH 'https://…/data/'` to say where they now live.
## 2. Pull a slice, then explore it
Every query against the lake is a set of HTTP range requests, so its cost is
set by how much of the table the `WHERE` clause lets DuckDB _skip_. The catalog
holds each file's partition values and column statistics, so pruning
hundreds of thousands of files to the relevant handful happens before any Parquet is touched:
| Query against the lake, in the browser | Rows | Time |
| --- | --- | --- |
| `ATTACH` the lake | — | ~1 s |
| Aggregate one symbol, one day | ~10 thousand | ~1 s |
| `CREATE TABLE … AS SELECT` one symbol, one month | ~300 thousand | ~5 s |
| Count one symbol, two years | ~6 million | ~1.5 min |
| `GROUP BY` over the materialized month | ~300 thousand | ~10 ms |
_Headless Chrome, DuckDB-WASM 1.4.3, one machine on one network, measured
once and rounded; treat these as orders of magnitude._
The last two rows are the design lesson. Interactive pivoting wants
millisecond queries, and a remote scan of millions of rows in hundreds of
small files is not that. So do what an analyst would do: materialize the
slice of interest into the local DuckDB once, and point Perspective at _that_.
```javascript
await conn.query(`
CREATE TABLE trades_slice AS (
SELECT * FROM lake.trades
WHERE symbol IN ('AAPL', 'MSFT')
AND trade_date BETWEEN DATE '2024-01-01' AND DATE '2024-03-31'
);
`);
const handler = new DuckDBHandler(conn);
const client = await perspective.worker(
await perspective.createMessageHandler(handler),
);
const viewer = document.querySelector("perspective-viewer");
await viewer.load(client);
await viewer.restore({
table: "memory.trades_slice",
plugin: "Candlestick",
group_by: ["bucket(\"ts\", 'm')"],
split_by: ["symbol"],
columns: ["open", "close", "high", "low"],
expressions: {
"bucket(\"ts\", 'm')": "bucket(\"ts\", 'm')",
open: '"price"',
close: '"price"',
high: '"price"',
low: '"price"',
},
aggregates: { open: "first", close: "last", high: "high", low: "low" },
});
```
Perspective's DuckDB virtual server discovers tables with `SHOW ALL TABLES`
and names them `.
`, so the slice appears as
`memory.trades_slice` with no registration step. From here the user has the
complete Perspective UI — group, split, filter, sort, expression columns,
every chart type — and each interaction is one local SQL query.
The slice selector — which symbols, which dates — is ordinary application UI
around that one `CREATE TABLE` statement. Several slices can be open as
panels of one `` workspace at once.
### How big can a slice be?
Bigger than "slice" suggests. DuckDB is a columnar, vectorized engine, and
that is still true under WebAssembly. Synthetic tick data — timestamp, symbol,
price, size, side, venue — in DuckDB-WASM, single-threaded, in one browser
tab:
| Local table | Storage | Build | Pivot by symbol × side | 1-minute OHLC for one symbol |
| --- | --- | --- | --- | --- |
| 10 million rows | In memory, 599 MB | 1.7 s | 0.34 s | 0.11 s |
| 50 million rows | OPFS, 408 MiB on disk | 23 s | 0.78 s | 0.25 s |
| 100 million rows | OPFS, ~0.7 GiB on disk | 46 s | 1.5 s | 0.44 s |
_Build time is generating the rows, not fetching them. Synthetic data
compresses better than real ticks; expect real files to be larger._
Two regimes are visible:
- **In memory, plan on tens of millions of rows.** An in-memory DuckDB stores
tables uncompressed — about 60 bytes per tick here — inside 32-bit
WebAssembly's 4 GB address space, of which DuckDB budgets 3.1 GiB. Ten
million rows is comfortable; fifty million of this shape did not fit.
- **On OPFS, plan on a hundred million and up.** Open the database at an
`opfs://` path and tables live in a compressed, persistent file in the
browser's
[Origin Private File System](https://duckdb.org/2026/09/18/opfs-wasm), with
DuckDB paging blocks in and out as needed. The same 50 million rows which
failed in memory took 408 MiB on disk, and 100 million still pivoted in a
second and a half. The slice also survives a reload, so a returning user
does not pay for the fetch twice.
```javascript
await db.open({
path: "opfs://ticks.duckdb",
accessMode: duckdb.DuckDBAccessMode.READ_WRITE,
});
```
OPFS support in DuckDB-WASM is recent; check the
[release notes](https://duckdb.org/2026/09/18/opfs-wasm) for the version to
pin, and `CHECKPOINT` after building a slice you want to keep.
So the practical limit on a slice is not the engine. It is how long the user
will wait for the fetch, which is a property of how well the lake is
partitioned for the question.
## Guard against full scans
DuckDB has no "maximum bytes scanned" setting, and a lakehouse table is only
cheap to query when a filter lets most of it be skipped. In a SQL console an
unfiltered query is something a user has to type. In a pivot UI it is one
drag: `SHOW ALL TABLES` lists `lake.trades` next to the slice, and opening it
and dropping a column on _Group By_ asks the browser to aggregate billions of
rows — tens of gigabytes of downloads, paid for by the reader's connection and
memory and by whoever hosts the bucket.
Design so that cannot happen:
- **Expose slices, never the lake.** Attach the lake on a DuckDB instance the
viewer is not bound to, or do not offer its tables in your table picker;
hand Perspective only the materialized tables.
- **Make the filter mandatory.** Build the `CREATE TABLE … AS SELECT` from
validated inputs (a symbol list, a bounded date range), estimate its size
from the catalog first — `ducklake_data_file` has `record_count` and
`file_size_bytes` per file — and refuse slices over a budget.
- **Bound the damage.** Set DuckDB's `memory_limit` so a runaway query fails
instead of taking the tab down.
- **Mind whose bucket it is.** If the Parquet is someone else's public data,
your application's traffic is their request bill. Do not publish a link
which lets copy-pasted code scan it.
## 3. Time travel as a user feature
Every DuckLake snapshot is queryable, and the catalog says what they are:
```javascript
const snapshots = await conn.query(
`SELECT snapshot_id, snapshot_time, changes FROM lake.snapshots()`,
);
```
Because the virtual server sees DuckDB views as tables, exposing a point in
time to the UI is one statement:
```javascript
await conn.query(`
CREATE OR REPLACE VIEW trades_as_of AS
SELECT * FROM lake.trades AT (VERSION => ${snapshot_id})
WHERE symbol = 'AAPL' AND trade_date = DATE '2024-03-15';
`);
await viewer.restore({ table: "memory.trades_as_of" });
```
For market data this is the correction workflow: put the current slice and an
`AT (VERSION => …)` slice in two panels with the same configuration, and the
user sees a day before and after a vendor's restatement, pivoted however they
like. Alternatively, attach the lake a second time with `SNAPSHOT_VERSION` or
`SNAPSHOT_TIME` to pin a whole catalog.
## Publishing your own
A frozen DuckLake is written by native DuckDB — a nightly job, a notebook,
CI — using a DuckDB file as the catalog:
```sql
INSTALL ducklake;
ATTACH 'ducklake:ticks.ducklake' AS lake (DATA_PATH 's3://my-bucket/ticks/');
CREATE TABLE lake.trades AS
SELECT * FROM read_parquet('raw/trades_*.parquet');
```
Existing Parquet can also be registered in place, without rewriting it.
Upload the catalog file to any static host.
- **CORS and `Range`.** Both the catalog's host and the data's must allow
your origin and honor `Range` requests. This is the most common reason an
attach fails.
- **Partition for the questions users ask.** Pruning is what makes this
interactive; a symbol/date layout is why a one-day query takes a second.
- **Fewer, larger files.** The two-year query above spans hundreds of small
daily files, each costing its own round trips. Compact before publishing.
- **Match versions.** A lake written by a newer DuckLake than the browser's
extension understands will not attach. Pin `@duckdb/duckdb-wasm` and the
writer's DuckDB to compatible releases.
- **The browser is a reader.** Under WebAssembly, `ATTACH`, queries,
`snapshots()`, time travel and even catalog DDL work; data writes to the
lake did not in our testing, and PostgreSQL or MySQL catalogs are out of
reach because browsers have no raw sockets. Write from native DuckDB.
- **Hide the plumbing.** `SHOW ALL TABLES` also lists DuckLake's own metadata
tables (`__ducklake_metadata_.*`), so they appear in the viewer's
table list alongside your data.
- **Private lakes.** DuckDB's `CREATE SECRET (TYPE s3, …)` works in the
browser, but a key in a page is a key in the browser; front a private
bucket with short-lived signed URLs or a proxy.
- **Today's ticks do not belong here.** A lakehouse is request/response. For
the live session use Perspective's own engine, whose `Table` pushes
incremental updates to every view — see
[Trading blotters, order books and market data](./market_data.md). History
from the lake and live panels can share one workspace.
## Related
- [A UI for DuckDB, ClickHouse and PostgreSQL](./database_ui.md)
- [Trading blotters, order books and market data](./market_data.md)
- [DuckDB virtual server (JavaScript)](../how_to/javascript/virtual_server/duckdb.md)
- [Visualizing millions of rows in the browser](./large_datasets.md)
- [`esbuild-duckdb-virtual` example](https://github.com/perspective-dev/perspective/tree/master/examples/esbuild-duckdb-virtual)
# Embedded analytics in a web application
Embedded analytics means giving your application's users a way to explore
_their_ data inside _your_ product — not a static chart you designed, and not a
link out to a separate BI tool. `` is a component built for
that job.
- **A Web Component, not a platform.** It is one Custom Element with no
framework dependency. It works in plain HTML and in React (via
[`@perspective-dev/react`](../how_to/javascript/react.md)), Vue, Svelte and
Angular through standard DOM APIs. There is no server to deploy unless you
want one, and no iframe.
- **Self-service by default.** Users group, pivot, filter, sort, write
computed columns and switch between data grid, charts and maps themselves.
- **State is JSON.** [`save()` and `restore()`](../how_to/javascript/save_restore.md)
round-trip the entire configuration, so "saved views", shareable links and
per-user defaults are a database column, not a feature to build.
- **Multi-panel dashboards.** One element can hold a tabbed, split layout of
many panels with cross-panel global filters, saved and restored with
`saveWorkspace()` and `restoreWorkspace()`.
- **Your brand.** [Themes](../how_to/javascript/theming.md) are CSS custom
properties; several light and dark themes are included.
- **Your data path.** Load data in the browser, replicate it from your server,
virtualize it server-side, or [point it at your database](./database_ui.md).
- **Apache-2.0.** No per-seat or per-deployment licensing, and no feature
tier: pivoting, charts and server-side virtualization are all open source.
## React
```tsx
import * as React from "react";
import perspective from "@perspective-dev/client";
import { PerspectiveViewer } from "@perspective-dev/react";
const worker = await perspective.worker();
const table = worker.table(
fetch("/api/orders.arrow").then((resp) => resp.arrayBuffer()),
);
export function OrdersReport({ saved, onChange }) {
return (
);
}
```
WebAssembly initialization for your bundler is covered in
[Importing with or without a bundler](../how_to/javascript/importing.md); with
Next.js, load the component client-side only (`ssr: false`).
## Plain JavaScript
```javascript
const viewer = document.querySelector("perspective-viewer");
await viewer.load(table);
await viewer.restore(await loadSavedViewFor(user));
viewer.addEventListener("perspective-config-update", async () => {
await persistSavedViewFor(user, await viewer.save());
});
```
## Constraining what users can do
`restore()` sets the starting point; users can change anything from there. To
lock an embedded report down, hide the configuration UI with the `settings`
config field and drive the element only from your own controls. Row-level
security belongs on the server: host a filtered `View`, or a
[virtual server](../explanation/virtual_servers.md) bound to a restricted
database role, rather than relying on a client-side filter.
## An assistant in the box
`` includes an opt-in [LLM agent](./agent.md) which lets
users ask for a view in plain language.
# LLM and agent-driven analytics
`` ships with an embedded LLM agent. A user types "show me
monthly revenue by region as a stacked bar, top five only", and the agent
reads the table's schema, writes the view configuration, authors any computed
columns it needs, picks the chart, and applies it — through the same public
API your own code would use.
It is **opt-in**. The Chat tab stays hidden and no network request is made
until you configure a model:
```javascript
import { providers } from "@perspective-dev/viewer";
const viewer = document.querySelector("perspective-viewer");
viewer.agentConfig({
...providers.anthropic,
apiKey: "sk-ant-...",
});
```
## Why an agent fits Perspective
An LLM is good at translating intent into a small, structured configuration,
and unreliable at arithmetic over data it has to read. Perspective's
configuration is exactly that kind of target: a complete analysis — grouping,
column splits, aggregates, filters, sorts, expressions, chart type — is a few
lines of JSON, and the numbers are computed by the engine, not the model.
- The agent's tools read the table's schema and the viewer's configuration —
none of them read rows, so your data is not sent to the model.
- Every answer is an ordinary, inspectable viewer configuration. The user can
see exactly what was grouped and filtered, adjust it by hand, and save it.
- Because the engine is incremental, an agent-built view over streaming data
keeps updating after the conversation ends.
## Any model, including local ones
The agent speaks the OpenAI chat-completions convention, so it works with
Anthropic, OpenAI, Gemini and OpenRouter endpoints, with local servers such as
[Ollama](https://ollama.com/), [LM Studio](https://lmstudio.ai/), llama.cpp and
vLLM, and with in-page engines such as
[WebLLM](https://github.com/mlc-ai/web-llm) — in which case the data, the
query engine and the model all run inside the browser tab.
## Keys and production use
A key passed to `agentConfig` is a key in the browser. That is fine for local
development and internal tools; for anything shared, point `url` at a proxy
you control and keep the credential on the server. See
[Configuring the LLM agent](../how_to/javascript/agent.md) for the full
connection options.
## Driving Perspective from your own agent
The agent uses no private hooks. `restore()`, `save()`, `Table.schema()` and
`View` are the complete surface, and a view configuration is plain JSON — so
an external agent, a notebook assistant or an MCP tool can produce the same
results by emitting a `ViewerConfig`. The guide is published for that purpose
as Markdown at [`/llms.txt`](https://perspective-dev.github.io/llms.txt) and
[`/llms-full.txt`](https://perspective-dev.github.io/llms-full.txt).
# Data Architecture
Application developers can choose from
[Client (WebAssembly)](./architecture/client_only.md),
[Server (Python/Node)](./architecture/server_only.md) or
[Client/Server Replicated](./architecture/client_server.md) designs to bind
data, and a web application can use one or a mix of these designs as needed. By
serializing to Apache Arrow, tables are duplicated and synchronized across
runtimes efficiently.
Perspective is a multi-language platform. The examples in this section use
Python and JavaScript as an example, but the same general principles apply to
any `Client`/`Server` combination.
# Client-only
_For static datasets, datasets provided by the user, and simple server-less and
read-only web applications._
In this design, Perspective is run as a client Browser WebAssembly library, the
dataset is downloaded entirely to the client and all calculations and UI
interactions are performed locally. Interactive performance is very good, using
WebAssembly engine for near-native runtime plus WebWorker isolation for parallel
rendering within the browser. Operations like scrolling and creating new views
are responsive. However, the entire dataset must be downloaded to the client.
Perspective is not a typical browser component, and datset sizes of 1gb+ in
Apache Arrow format will load fine with good interactive performance!
Horizontal scaling is a non-issue, since here is no concurrent state to scale,
and only uses client-side computation via WebAssembly client. Client-only
perspective can support as many concurrent users as can download the web
application itself. Once the data is loaded, no server connection is needed and
all operations occur in the client browser, imparting no additional runtime cost
on the server beyond initial load. This also means updates and edits are local
to the browser client and will be lost when the page is refreshed, unless
otherwise persisted by your application.
As the client-only design starts with creating a client-side Perspective
`Table`, data can be provided by any standard web service in any Perspective
compatible format (JSON, CSV or Apache Arrow).
## Javascript client
```javascript
const worker = await perspective.worker();
const table = await worker.table(csv);
const viewer = document.createElement("perspective-viewer");
document.body.appendChild(viewer);
await viewer.load(table);
```
# Client/Server replicated
_For medium-sized, real-time, synchronized and/or editable data sets with many
concurrent users._
The dataset is instantiated in-memory with a Python or Node.js Perspective
server, and web applications create duplicates of these tables in a local
WebAssembly client in the browser, synchonized efficiently to the server via
Apache Arrow. This design scales well with additional concurrent users, as
browsers only need to download the initial data set and subsequent update
deltas, while operations like scrolling, pivots, sorting, etc. are performed on
the client.
Python servers can make especially good use of additional threads, as
Perspective will release the GIL for almost all operations. Interactive
performance on the client is very good and identical to client-only
architecture. Updates and edits are seamlessly synchonized across clients via
their virtual server counterparts using websockets and Apache Arrow.
## Python and Tornado server
```python
from perspective import Server, PerspectiveTornadoHandler
server = Server()
client = server.new_local_client()
client.table(csv, name="my_table")
routes = [(
r"/websocket",
perspective.handlers.tornado.PerspectiveTornadoHandler,
{"perspective_server": server},
)]
app = tornado.web.Application(routes)
app.listen(8080)
loop = tornado.ioloop.IOLoop.current()
loop.start()
```
## Javascript client
Perspective's websocket client interfaces with the Python server, then
_replicates_ the server-side Table. When the server-side `Table` has an `index`,
the replica inherits it, and both `update()` and `remove()` on the server are
mirrored in the browser.
```javascript
const websocket = await perspective.websocket("ws://localhost:8080");
const server_table = await websocket.open_table("my_table");
const server_view = await server_table.view();
const worker = await perspective.worker();
const client_table = await worker.table(server_view);
const viewer = document.createElement("perspective-viewer");
document.body.appendChild(viewer);
await viewer.load(client_table);
```
# Server-only
_For extremely large datasets with a small number of concurrent users._
The dataset is instantiated in-memory with a Python or Node.js server, and web
applications connect virtually. Has very good initial load performance, since no
data is downloaded. Group-by and other operations will run column-parallel if
configured.
But interactive performance is poor, as every user interaction must page the
server to render. Operations like scrolling are not as responsive and can be
impacted by network latency. Web applications must be "always connected" to the
server via WebSocket. Disconnecting will prevent any interaction, scrolling,
etc. of the UI. Does not use WebAssembly.
Each connected browser will impact server performance as long as the connection
is open, which in turn impacts interactive performance of every client. This
ultimately limits the horizontal scalabity of this architecture. Since each
client reads the perspective `Table` virtually, changes like edits and updates
are automatically reflected to all clients and persist across browser refresh.
Using the same Python server as the previous design, we can simply skip the
intermediate WebAssembly `Table` and pass the virtual table directly to `load()`
```javascript
const websocket = await perspective.websocket("ws://localhost:8080");
const server_table = await websocket.open_table("my_table");
const viewer = document.createElement("perspective-viewer");
document.body.appendChild(viewer);
await viewer.load(server_table);
```
# Virtual Servers
A Virtual Server allows Perspective to query external data sources (such as
DuckDB or ClickHouse) without loading the entire dataset into Perspective's
built-in data engine. Instead, Perspective translates its query operations
(group by, sort, filter, etc.) into queries the external data source can execute
natively, and only transfers the data needed for the current view.
The Virtual Server API works on any platform that has a Perspective Client —
including JavaScript (both Node.js and the browser via WebAssembly), Python, and
Rust. In the browser, this means a virtual server can front a WASM-based engine
like `@duckdb/duckdb-wasm`, giving `` the ability to query a
database running entirely client-side without loading data into Perspective's
own engine.
This is useful when:
- The dataset is too large to fit in browser memory or a single process.
- Data already lives in a database and you want to avoid duplicating it.
- You want to leverage a database's native query optimizations.
- A WASM build of the data source is available in the browser (e.g.
`@duckdb/duckdb-wasm`) and you want to query it directly.
## How it works
A virtual server implements a handler interface that Perspective calls to
satisfy `Table` and `View` operations. The handler translates Perspective's view
configuration into the external system's query language (typically SQL),
executes the query, and returns the results as columnar data. Because the
handler speaks the standard Perspective Client protocol, it can run anywhere a
Client can — in-process, in a WebWorker, or on a remote server.
```
┌──────────────────────────────────────────────────┐
│ │
└──┬───────────────────────────────────────────────┘
│ ┌──────────────────────────────────────────────────┐
└──►│ Perspective Virtual Server Handler │
└──┬───────────────────────────────────────────────┘
│ ┌──────────────────────────────────────────────────┐
└──►│ External DB (DuckDB, ClickHouse, …). │
└──────────────────────────────────────────────────┘
```
The viewer communicates with the virtual server handler the same way it would
with a regular Perspective server. The handler advertises its capabilities
(which operations it supports) via a _features_ object, and the viewer UI adapts
accordingly — disabling controls for unsupported operations.
## Built-in implementations
Perspective ships with virtual server implementations for:
- **DuckDB** — query DuckDB databases in-browser via WASM
([JavaScript](../how_to/javascript/virtual_server/duckdb.md)) or server-side
([Python](../how_to/python/virtual_server/duckdb.md)).
- **ClickHouse** — query a ClickHouse server from the browser
([JavaScript](../how_to/javascript/virtual_server/clickhouse.md)) or from
Python ([Python](../how_to/python/virtual_server/clickhouse.md)).
- **PostgreSQL** — query a PostgreSQL server (16 or later) from Python
([Python](../how_to/python/virtual_server/postgres.md)).
## Custom implementations
You can implement your own virtual server to connect Perspective to any data
source. See the language-specific guides:
- [JavaScript: Implementing a custom Virtual Server](../how_to/javascript/virtual_server/custom.md)
- [Python: Implementing a custom Virtual Server](../how_to/python/virtual_server/custom.md)
## Features declaration
The `get_features()` / `getFeatures()` method returns an object that tells
Perspective which query operations the virtual server supports. The viewer will
only show controls for supported operations:
| Field | Type | Description |
| ------------- | ------ | ----------------------------------------------------------- |
| `group_by` | `bool` | Whether group-by aggregation is supported |
| `split_by` | `bool` | Whether split-by (pivot) is supported |
| `sort` | `bool` | Whether sorting is supported |
| `expressions` | `bool` | Whether computed expressions are supported |
| `filter_ops` | `dict` | Map of column type to list of supported filter operators |
| `aggregates` | `dict` | Map of column type to list of supported aggregate functions |
| `on_update` | `bool` | Whether update callbacks are supported |
# Table
`Table` is Perspective's columnar data frame, analogous to a Pandas `DataFrame`
or Apache Arrow, supporting append & in-place updates, removal by index, and
update notifications.
A `Table` contains columns, each of which have a unique name, are strongly and
consistently typed, and contains rows of data conforming to the column's type.
Each column in a `Table` must have the same number of rows, though not every row
must contain data; null-values are used to indicate missing values in the
dataset. The schema of a `Table` is _immutable after creation_, which means the
column names and data types cannot be changed after the `Table` has been
created. Columns cannot be added or deleted after creation either, but a `View`
can be used to select an arbitrary set of columns from the `Table`.
# Schema and column types
The mapping of a `Table`'s column names to data types is referred to as a
`schema`. Each column has a unique name and a single data type, one of
- `float`
- `integer`
- `boolean`
- `date`
- `datetime`
- `string`
A `Table` schema is fixed at construction, either by explicitly passing a schema
dictionary to the `Client::table` method, or by passing _data_ to this method
from which the schema is _inferred_ (if CSV or JSON format) or inherited (if
Arrow).
## Arrow type mapping
Perspective's six column types are narrower than Arrow's type system, so Arrow
input is mapped on ingest:
| Arrow type | Perspective type |
| --- | --- |
| `int8`, `int16`, `int32`, `int64`, `uint8`, `uint16`, `uint32`, `uint64` | `integer` |
| `float`, `double` | `float` |
| `decimal`, `decimal128` | `float` |
| `bool` | `boolean` |
| `date32`, `date64` | `date` |
| `timestamp` | `datetime` |
| `time32`, `time64` | `integer` |
| `utf8`, `large_utf8`, `binary`, `dictionary`, `list`, `null` | `string` |
Two mappings are worth calling out:
- Arrow `decimal` columns become `float` — a `DECIMAL` value of `3.14` reads
as `3.14`, not as its unscaled integer representation.
- Arrow `time32`/`time64` (a time-of-day with no date component) becomes
`integer`, not `datetime`. Use a `timestamp` column for a true `datetime`.
Arrow types not listed above — including `decimal256` and the nested types —
are rejected with an error rather than silently coerced.
Arrow input is fully validated before its buffers are read. A malformed IPC
payload — bad offsets, out-of-range dictionary indices, inconsistent chunk
lengths — is rejected with an error rather than producing corrupt data.
## Type inference
When passing CSV or JSON data to the `Client::table` constructor, the type of
each column is inferred automatically. In some cases, the inference algorithm
may not return exactly what you'd like. For example, a column may be interpreted
as a `datetime` when you intended it to be a `string`, or a column may have no
values at all (yet), as it will be updated with values from a real-time data
source later on. In these cases, create a `table()` with a _schema_.
Once the `Table` has been created, further `Table::update` calls will perform
limited type _coercion_ based on the schema. While _coercion_ works similarly to
_inference_, in that input data may be parsed based on the expected column type,
`Table::update` will not _change_ the column's type further. For example, a
number literal `1234` would be _inferred_ as an `"integer"`, but _in the context
of an `Table::update` call on a known `"string"` column_, this will be parsed as
the _string_ `"1234"`.
## `date` and `datetime` inference
Various string representations of `date` and `datetime` format columns can be
_inferred_ as well _coerced_ from strings if they match one of Perspective's
internal known datetime parsing formats, for example
[ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) (which is also the format
Perspective will _output_ these types for CSV).
# Loading data
A `Table` may also be created-or-updated by data in CSV,
[Apache Arrow](https://arrow.apache.org/), JSON row-oriented or JSON
column-oriented formats. In addition to these, `perspective-python` additionally
supports `pyarrow.Table`, `polars.DataFrame` and `pandas.DataFrame` objects
directly. These formats are otherwise identical to the built-in formats and
don't exhibit any additional support or type-awareness; e.g., `pandas.DataFrame`
support is _just_ `pyarrow.Table.from_pandas` piped into Perspective's Arrow
reader.
`Client::table` and `Table::update` perform _coercion_ on their input for all
input formats _except_ Arrow (which comes with its own schema and has no need
for coercion). `"date"` and `"datetime"` column types do not have native JSON
representations, so these column types _cannot_ be inferred from JSON input.
Instead, for columns of these types for JSON input, a `Table` must first be
constructed with a _schema_. Next, call `Table::update` with the JSON input -
Perspective's JSON reader may _coerce_ a `date` or `datetime` from these native
JSON types:
- `integer` as milliseconds-since-epoch.
- `string` as a any of Perspective's built-in date format formats.
- JavaScript `Date` and Python `datetime.date` and `datetime.datetime` are _not_
supported directly. However, in JavaScript `Date` types are automatically
coerced to correct `integer` timestamps by default when converted to JSON.
## Apache Arrow
The most efficient way to load data into Perspective, encoded as
[Apache Arrow IPC format](https://arrow.apache.org/docs/python/ipc.html). In
JavaScript:
```javascript
const resp = await fetch(
"https://cdn.jsdelivr.net/npm/superstore-arrow/superstore.lz4.arrow",
);
const arrow = await resp.arrayBuffer();
```
Apache Arrow input do not support type coercion, preferring Arrow's internal
self-describing schema.
## CSV
Perspective relies on Apache Arrow's CSV parser, and as such uses mostly the
same column-type inference logic as Arrow itself would use for parsing CSV.
## Row Oriented JSON
Row-oriented JSON is in the form of a list of objects. Each object in the list
corresponds to a row in the table. For example:
```json
[
{ "a": 86, "b": false, "c": "words" },
{ "a": 0, "b": true, "c": "" },
{ "a": 12345, "b": false, "c": "here" }
]
```
## Column Oriented JSON
Column-Oriented JSON comes in the form of an object of lists. Each key of the
object is a column name, and each element of the list is the corresponding value
in the row.
```json
{
"a": [86, 0, 12345],
"b": [false, true, false],
"c": ["words", "", "here"]
}
```
## NDJSON
[NDJSON](https://github.com/ndjson/ndjson-spec) (sometimes also referred to as
JSONL) is a streaming-friendly format where each line is a valid JSON object,
separated by newlines. It is commonly used in data streaming and messaging
queues.
```json
{ "a": 86, "b": false, "c": "words" }
{ "a": 0, "b": true, "c": "" }
{ "a": 12345, "b": false, "c": "here" }
```
# Construct a Table
Examples of constructing an empty `Table` from a schema.
Rust:
```rust
let data = TableData::Schema(vec![(" a".to_string(), ColumnType::FLOAT)]);
let options = TableInitOptions::default();
let table = client.table(data.into(), options).await?;
```
## Index and Limit
`limit` cannot be used in conjunction with `index`.
Initializing a `Table` with an `index` tells Perspective to treat a column as
the primary key, allowing in-place updates of rows. Only a single column (of any
type) can be used as an `index`. Indexed `Table` instances allow:
- In-place _updates_ whenever a new row shares an `index` values with an
existing row
- _Partial updates_ when a data batch omits some column.
- _Removes_ to delete a row by `index`.
To create an indexed `Table`, provide the `index` property with a string column
name to be used as an index:
Initializing a `Table` with a `limit` sets the total number of rows the `Table`
is allowed to have. When the `Table` is updated, and the resulting size of the
`Table` would exceed its `limit`, rows that exceed `limit` overwrite the oldest
rows in the `Table`. To create a `Table` with a `limit`, provide the `limit`
property with an integer indicating the maximum rows:
## `page_to_disk`
By default a `Table` keeps its columns in memory. Initializing a `Table` with
`page_to_disk` backs its column data with on-disk storage instead, so the
`Table` can be larger than the memory available to the engine. It is otherwise
an ordinary `Table`: `index`, `update()`, views, aggregates, expressions and
Arrow round-trips all behave as they do in memory, and produce identical
results.
Where the data goes depends on the runtime:
| Runtime | Backing storage |
| --- | --- |
| Browser (WebAssembly, in a Web Worker) | The [Origin Private File System](https://developer.mozilla.org/en-US/docs/Web/API/File_System_API/Origin_private_file_system) (OPFS) |
| Node.js (WebAssembly) | Files under the OS temp directory, via `node:fs` |
| Python and Rust (native) | Memory-mapped files under the OS temp directory |
# `Table::update` and `Table::remove`
Once a `Table` has been created, it can be updated with new data conforming to
the `Table`'s schema. `Table::update` supports the same data formats as
`Client::table`, minus _schema_.
Without an `index` set, calls to `update()` _append_ new data to the end of the
`Table`. Otherwise, Perspective allows
[_partial updates_ (in-place)](#index-and-limit) using the `index` to determine
which rows to update:
Any value on a `Client::table` can be unset using the value `null` in JSON or
Arrow input formats. Values may be unset on construction, as any `null` in the
dataset will be treated as an unset value. `Table::update` calls do not need to
provide _all columns_ in the `Table`'s schema; missing columns will be omitted
from the `Table`'s updated rows.
Rows can also be removed from an indexed `Table`, by calling `Table::remove`
with an array of index values:
```javascript
indexed_table.remove([1, 4]);
```
```python
indexed_table.remove([1, 4])
```
# `Table::clear` and `Table::replace`
Calling `Table::clear` will remove all data from the underlying `Table`. Calling
`Table::replace` with new data will clear the `Table`, and update it with a new
dataset that conforms to Perspective's data types and the existing schema on the
`Table`.
# View
The [`View`] struct is Perspective's query and serialization interface. It
represents a query on the `Table`'s dataset and is always created from an
existing `Table` instance via the [`Table::view`] method.
[`View`]s are immutable with respect to the arguments provided to the
[`Table::view`] method; to change these parameters, you must create a new
[`View`] on the same [`Table`]. However, each [`View`] is _live_ with respect to
the [`Table`]'s data, and will (within a conflation window) update with the
latest state as its parent [`Table`] updates, including incrementally
recalculating all aggregates, pivots, filters, etc. [`View`] query parameters
are composable, in that each parameter works independently _and_ in conjunction
with each other, and there is no limit to the number of pivots, filters, etc.
which can be applied.
The examples in this module are in JavaScript. See perspective docs for the Rust API.
The examples in this module are in Python. See perspective docs for the Rust API.
```rust
let opts = TableInitOptions::default();
let data = TableData::Update(UpdateData::Csv("x,y\n1,2\n3,4".into()));
let table = client.table(data, opts).await?;
let view = table.view(None).await?;
let arrow = view.to_arrow().await?;
view.delete().await?;
```
# Querying data
To query the table, create a [`Table::view`] on the table instance with an
optional configuration object. A [`Table`] can have as many [`View`]s associated
with it as you need - Perspective conserves memory by relying on a single
[`Table`] to power multiple [`View`]s concurrently:
# Grouping and Pivots
## Group By
A group by _groups_ the dataset by the unique values of each column used as a
group by - a close analogue in SQL to the `GROUP BY` statement. The underlying
dataset is aggregated to show the values belonging to each group, and a total
row is calculated for each group, showing the currently selected aggregated
value (e.g. `sum`) of the column. Group by are useful for hierarchies,
categorizing data and attributing values, i.e. showing the number of units sold
based on State and City. In Perspective, group by are represented as an array of
string column names to pivot, are applied in the order provided; For example, a
group by of `["State", "City", "Postal Code"]` shows the values for each Postal
Code, which are grouped by City, which are in turn grouped by State.
### `group_rollup_mode`
The `group_rollup_mode` option controls how the grouped rows themselves render:
- `"rollup"` (the default) - the full hierarchy, with a subtotal row for
every group at every level and a grand total row, each addressable by its
`__ROW_PATH__`.
- `"flat"` - leaf rows only, one row per deepest-level group, with no
subtotal or grand total rows. Useful for chart plugins and exports where
subtotal rows would double-count.
- `"total"` - the grand total row _only_. `"total"` is mutually exclusive
with `group_by` (which is cleared when it is set) - it is the one shape
an empty `group_by` cannot express, since no `group_by` at all yields the
unaggregated dataset.
## Split By
A split by _splits_ the dataset by the unique values of each column used as a
split by. The underlying dataset is not aggregated, and a new column is created
for each unique value of the split by. Each newly created column contains the
parts of the dataset that correspond to the column header, i.e. a `View` that
has `["State"]` as its split by will have a new column for each state. In
Perspective, Split By are represented as an array of string column names to
pivot:
### `split_rollup_mode`
The `split_rollup_mode` option is the `split_by` counterpart to
[`group_rollup_mode`](#group_rollup_mode), controlling whether subtotal
_column groups_ are emitted:
- `"flat"` (the default) - only full-depth split combinations appear as
columns, e.g. `"CA|Sales"`. This is Perspective's historical behavior.
- `"rollup"` - additionally emits a grand-total column per aggregate (named
by the bare column name, e.g. `"Sales"`, aggregating across every split
group) and, when more than one `split_by` column is applied, a subtotal
column per intermediate split group (e.g. `"CA|Sales"` alongside
`"CA|First Class|Sales"`). Total and subtotal columns precede their
groups, in pre-order.
## Aggregates
Aggregates perform a calculation over an entire column, and are displayed when
one or more [Group By](#group-by) are applied to the `View`. Aggregates can be
specified by the user, or Perspective will use the following sensible default
aggregates based on column type:
- "sum" for `integer` and `float` columns
- "count" for all other columns
Perspective provides a selection of aggregate functions that can be applied to
columns in the `View` constructor using a dictionary of column name to aggregate
function name.
```rust
use std::collections::HashMap;
let view = table.view(Some(ViewConfigUpdate {
aggregates: Some(HashMap::from([
("a".into(), "avg".into()),
("b".into(), "distinct count".into()),
])),
..ViewConfigUpdate::default()
})).await?;
```
Every aggregate is described below, grouped by what it computes. Which of them
a given column accepts depends on its type — see
[Availability by column type](#availability-by-column-type).
### Sums and products
| Aggregate | Description | Result type |
| --- | --- | --- |
| `sum` | Total of the group's values | `integer` or `float` |
| `sum not null` | As `sum`, but non-finite (`NaN`) values are skipped rather than poisoning the total | `integer` or `float` |
| `sum abs` | Sum of the absolute values — `Σ abs(v)` | `integer` or `float` |
| `abs sum` | Absolute value of the sum — `abs(Σ v)` | `integer` or `float` |
| `mul` | Product of the group's values | `integer` or `float` |
| `gmv` | Gross market value — leaf rows are a plain `sum`, parent rows sum the _absolute_ subtotal of each immediate child group | `integer` or `float` |
| `pct sum parent` | The group's `sum` as a percentage of its parent row's, `0`–`100`; `100` at the root, and `null` when the parent's sum is `0` | `float` |
| `pct sum total` | The group's `sum` as a percentage of the grand total, `0`–`100` | `float` |
A numeric aggregate widens to the input's numeric class — `integer` columns
accumulate as `integer`, `float` columns as `float`.
### Averages and dispersion
| Aggregate | Description | Result type |
| --- | --- | --- |
| `avg` | Arithmetic mean of the non-null values | `float` |
| `weighted mean` | `Σ(value × weight) / Σ(weight)`, over rows where both the value and the weight are non-null and finite; `null` when the weights sum to `0`. Takes a **weight column** as an argument | `float` |
| `stddev` | Population standard deviation | `float` |
| `var` | Population variance — divides by `N`, not `N - 1` | `float` |
`stddev` and `var` are `null` for a group of fewer than two non-null values.
### Extrema and order statistics
| Aggregate | Description | Result type |
| --- | --- | --- |
| `min`, `max` | Smallest and largest of the group's current values | input type |
| `min by`, `max by` | The value from the row at which a **second column**, supplied as an argument, is smallest or largest | input type |
| `high`, `low` | High and low _water mark_ — the largest and smallest value this `View` has ever observed for the group, which never moves back when rows are updated or removed | input type |
| `high minus low` | `max - min` of the group's current values, i.e. its range. Despite the name this uses `min`/`max`, not the water marks | input type |
| `median`, `q1`, `q3` | The value at the 50%, 25% and 75% position of the group's values; on `float` columns an exact split averages the two adjacent values | input type |
### Positional
| Aggregate | Description | Result type |
| --- | --- | --- |
| `first` | Value from the group's earliest row | input type |
| `last by index` | Value from the group's latest row | input type |
| `last minus first` | `last by index` minus `first` | input type |
| `last` | Value from the group's most recently _updated_ row | input type |
"Earliest" and "latest" are by the `Table`'s `index` column, or by row order
when the `Table` is unindexed. This is not the same as `last`, which tracks
update recency rather than position.
### Cardinality and identity
| Aggregate | Description | Result type |
| --- | --- | --- |
| `count` | Number of rows in the group | `integer` |
| `distinct count` | Number of distinct values in the group | `integer` |
| `unique` | The group's value when every row shares one, otherwise `null` | input type |
| `distinct leaf` | As `unique`, but only on leaf rows — parent rows are blank | input type |
| `dominant` | The most frequent non-null value, i.e. the mode; a tie resolves to whichever value reached the winning count first | input type |
| `any` | The group's first _truthy_ value — any non-null value for `string` columns, the first non-zero for numbers and dates, the first `true` for `boolean` — or `null` if it has none | input type |
| `or` | Identical to `any` | input type |
| `and` | `true` when every value in the group is truthy, else `false` | `boolean` |
| `join` | The group's distinct values, sorted and rendered as a `", "`-delimited string, truncated at 280 characters. Nulls render as `null` | `string` |
### Nulls
Null handling is not uniform, and is usually what makes two similar-looking
aggregates differ:
- `count` counts **rows**, not values — a group of 3 rows whose value is
`null` counts `3`. This is not the same as the `count` [window
aggregate](./windows.md#aggregates), which counts non-null values.
- `distinct count` counts `null` as **one distinct value**, so a group of
`[1, null, null]` counts `2`.
- `sum`, `avg`, `stddev`, `var`, `dominant` and `weighted mean` skip nulls
entirely. `avg` divides by the count of non-null values, so a group of all
nulls is `null` rather than `0`.
- `any` and `or` return the first _truthy_ value, not the first non-null one —
a numeric group of all `0`, or a `boolean` group of all `false`, aggregates
to `null`.
- `join` renders nulls into its output as the literal text `null`.
### Availability by column type
The aggregates a column accepts depend on its type:
**Numeric columns** (`integer`, `float`): `sum`, `abs sum`, `sum abs`,
`sum not null`, `mul`, `gmv`, `any`, `avg`, `mean`, `count`, `distinct count`,
`distinct leaf`, `dominant`, `first`, `last`, `last by index`, `high`, `low`,
`max`, `min`, `min by`, `max by`, `high minus low`, `last minus first`,
`median`, `q1`, `q3`, `pct sum parent`, `pct sum total`, `stddev`, `var`,
`unique`, `weighted mean`.
**String columns**: `count`, `any`, `distinct count`, `distinct leaf`,
`dominant`, `first`, `last`, `last by index`, `join`, `median`, `q1`, `q3`,
`unique`, `min by`, `max by`.
**Date/Datetime columns**: `count`, `any`, `avg`, `distinct count`,
`distinct leaf`, `dominant`, `first`, `last`, `last by index`, `high`, `low`,
`max`, `min`, `median`, `q1`, `q3`, `unique`.
**Boolean columns**: `count`, `any`, `and`, `or`, `distinct count`,
`distinct leaf`, `dominant`, `first`, `last`, `last by index`, `unique`.
avg on a date or
datetime column returns a float — the mean of the
column's underlying numeric representation — not a date.
### Argument-taking aggregates
`weighted mean`, `min by` and `max by` each read a second column, and are
written as a `[name, [argument]]` pair rather than a bare string:
In Rust, Aggregate::from(&str) splits on
" by " to build a MultiAggregate. Single aggregates
whose names contain that substring — "last by index" — must
therefore be constructed as
Aggregate::SingleAggregate("last by index".into()) rather than
"last by index".into(), which silently resolves to
last.
### Aliases
Several aggregates answer to more than one name. Every name below is accepted
anywhere an aggregate is, and each group refers to one function:
| Canonical | Also accepted |
| --- | --- |
| `avg` | `mean` |
| `distinct count` | `distinct`, `distinctcount`, `distinct_count` |
| `first` | `first by index` |
| `last` | `last_value` |
| `high` | `high_water_mark` |
| `low` | `low_water_mark` |
| `pct sum total` | `pct sum grand total`, `pct_sum_grand_total` |
| `var` | `variance` |
| `stddev` | `standard deviation` |
Most multi-word aggregates also answer to a snake_case spelling —
`sum_not_null`, `sum_abs`, `abs_sum`, `weighted_mean`, `distinct_leaf`,
`pct_sum_parent`, `pct_sum_total`, `min_by`, `max_by`. Three do not, and are
only accepted spelled with spaces: `high minus low`, `last minus first` and
`last by index`.
A few names the engine parses are _not implemented_ — `identity`,
`mean by count`, and `div`/`add`, which have no way to receive their operands
from a `ViewConfig`. Naming one is rejected exactly as a misspelled aggregate
is: the `View` fails to construct with an error naming the aggregate and the
column it was given for, and the `Table` is left untouched.
# Selection and Ordering
## Columns
The `columns` property specifies which columns should be included in the
`View`'s output. This allows users to show or hide a specific subset of columns,
as well as control the order in which columns appear to the user. This is
represented in Perspective as an array of string column names:
## Sort
The `sort` property specifies columns on which the query should be sorted,
analogous to `ORDER BY` in SQL. A column can be sorted regardless of its data
type, and sorts can be applied in ascending or descending order. Perspective
represents `sort` as an array of arrays, with the values of each inner array
being a string column name and a string sort direction. When `split_by` are
applied, the additional sort directions `"col asc"` and `"col desc"` will
determine the order of pivot column groups.
The available sort directions are:
| Direction | Description |
|---|---|
| `"asc"` | Ascending order |
| `"desc"` | Descending order |
| `"asc abs"` | Ascending by absolute value |
| `"desc abs"` | Descending by absolute value |
| `"col asc"` | Ascending order for pivot column groups (requires `split_by`) |
| `"col desc"` | Descending order for pivot column groups (requires `split_by`) |
| `"col asc abs"` | Ascending by absolute value for pivot column groups |
| `"col desc abs"` | Descending by absolute value for pivot column groups |
## Filter
The `filter` property specifies columns on which the query can be filtered,
returning rows that pass the specified filter condition. This is analogous to
the `WHERE` clause in SQL. There is no limit on the number of columns where
`filter` is applied, but the resulting dataset is one that passes all the filter
conditions, i.e. the filters are joined with an `AND` condition. The join
condition can be changed to `OR` via the `filter_op` property.
Perspective represents `filter` as an array of arrays, with the values of each
inner array being a string column name, a string filter operator, and a filter
operand in the type of the column:
The available filter operators depend on the column type:
**String columns**: `==`, `!=`, `>`, `>=`, `<`, `<=`, `begins with`,
`not begins with`, `contains`, `not contains`, `ends with`, `not ends with`,
`matches`, `not matches`, `in`, `not in`, `is not null`, `is null`.
The string matching operators (`begins with`, `contains`, `ends with` and
their negations) are case-insensitive, and `matches` / `not matches` are
case-sensitive partial-match [RE2](https://github.com/google/re2) regular
expressions. Null cells match none of these operators, including the negated
forms - filter on `is null` to select them.
**Numeric columns** (`integer`, `float`): `==`, `!=`, `>`, `>=`, `<`, `<=`,
`is not null`, `is null`.
**Boolean columns**: `==`, `is not null`, `is null`.
**Date/Datetime columns**: `==`, `!=`, `>`, `>=`, `<`, `<=`, `is not null`,
`is null`.
# Expressions
The `expressions` property specifies _new_ columns in Perspective that are
created using existing column values or arbitrary scalar values defined within
the expression. In ``, expressions are added using the "New
Column" button in the side panel.
Expressions are strings parsed by Perspective's expression engine (based on
[ExprTK](https://github.com/ArashPartow/exprtk)). Column names are referenced by
wrapping them in double quotes, e.g. `"Sales"`:
## Type Conversion and Coercion
Perspective expressions are typed: every column, literal and function result has
a fixed type, and the validator reports an error before the expression is ever
computed if the types do not fit the operator. To move between types explicitly,
use the conversion functions:
| Function | Description |
| --------------- | ------------------------------------------------------------ |
| `to_string(x)` | Convert any type to string |
| `to_integer(x)` | Convert to integer (null if not parsable) |
| `to_float(x)` | Convert to float (null if not parsable) |
| `to_boolean(x)` | Convert to boolean (truthy/falsy) |
| `integer(x)` | Alias for `to_integer(x)` |
| `float(x)` | Alias for `to_float(x)` |
| `datetime(x)` | Construct a datetime from a POSIX timestamp (ms since epoch) |
| `date(y, m, d)` | Construct a date from year, month, day |
### How coercion works
Numeric types promote to each other. Arithmetic on any mix of `integer` and
`float` operands is computed in floating point and produces a `float`. The
comparison operators compare values across every numeric type: integers are
compared exactly (including signed against unsigned), and as soon as one side is
a `float` both sides are compared as doubles. Numeric literals are `float`, so
`"Quantity" > 3` works on an `integer` column without a cast.
No other implicit coercion exists. `boolean`, `string`, `date` and `datetime`
values can only be compared with values of the same type; comparing a `string`
column to a number, a `boolean` to `1`, or a `date` to a `datetime` is a
validation error that names the operator and both types, for example
`Type Error - cannot compare string and float with '=='`. Similarly, `datetime`
and `date` values are not numeric: to perform arithmetic on them, you must first
convert to a numeric representation, do the math, then convert back.
Boolean contexts cast instead. The condition of `if` and `? :`, and the operands
of `and`, `or`, `not`, `xor`, `nand`, `nor` and `xnor`, accept any type: `null`
is `false`, a `boolean` is its own value, a number is `true` when non-zero, and
a `string` is `true` when non-null. `x == null` and `x != null` test `x` for
null and return `boolean`, the same as `is_null(x)` and `is_not_null(x)`; the
`null` literal is otherwise a value like any null cell: `"x" > 2 ? null : "x"`
yields null in the first case, and `"x" + null` or `"x" < null` are null, exactly
as they would be for a column with a null value.
Internally, `datetime` values are stored as milliseconds since the Unix epoch
(1970-01-01T00:00:00Z). Converting a `datetime` to a `float` yields this
millisecond timestamp, and `datetime()` accepts a millisecond timestamp to
produce a `datetime`.
### Example: offsetting a datetime by 7 days
This expression takes a `"Shipped Date"` column, converts it to its
millisecond-epoch representation, adds 7 days worth of milliseconds (7 ×
24 × 60 × 60 × 1000 = 604800000), and converts the result back
to a `datetime`:
```
// Due Date
datetime(float("Shipped Date") + 604800000)
```
## Operators
Standard arithmetic and comparison operators are supported:
| Operator | Description |
| -------------------------------- | ----------- |
| `+`, `-`, `*`, `/` | Arithmetic |
| `%` | Modulo |
| `==`, `!=`, `<`, `>`, `<=`, `>=` | Comparison |
| `and`, `or`, `not` | Logical |
| `if ... else ...` | Conditional |
## Numeric Functions
ExprTK provides a rich set of built-in numeric functions including `abs`,
`ceil`, `floor`, `round`, `exp`, `log`, `log10`, `sqrt`, `min`, `max`, `pow`,
`clamp`, `iclamp`, `inrange`, and trigonometric functions (`sin`, `cos`, `tan`,
`asin`, `acos`, `atan`).
## String Functions
| Function | Description |
| ------------------------------- | ------------------------------------------------------- |
| `concat(a, b, ...)` | Concatenate strings |
| `upper(s)` | Convert to uppercase |
| `lower(s)` | Convert to lowercase |
| `length(s)` | String length |
| `contains(s, substr)` | Whether `s` contains `substr` |
| `order(col, 'B', 'C', 'A')` | Custom sort order for a string column |
| `match(s, pattern)` | Regex partial match (returns boolean) |
| `match_all(s, pattern)` | Regex full match (returns boolean) |
| `search(s, pattern)` | First capturing group match |
| `indexof(s, pattern)` | Start index of first regex match |
| `substring(s, start, end)` | Substring from `start` (inclusive) to `end` (exclusive) |
| `replace(s, repl, pattern)` | Replace first regex match |
| `replace_all(s, repl, pattern)` | Replace all regex matches |
## Date/Datetime Functions
| Function | Description |
| ------------------------ | ------------------------------------------------------------------------ |
| `today()` | Current date |
| `now()` | Current datetime |
| `date(year, month, day)` | Construct a date |
| `datetime(timestamp_ms)` | Construct a datetime from a POSIX timestamp (ms since epoch) |
| `hour_of_day(dt)` | Hour component (0-23) |
| `day_of_week(dt)` | Day of the week as a string |
| `month_of_year(dt)` | Month of the year as a string |
| `bucket(dt, unit)` | Bucket datetime by unit: `'s'`, `'m'`, `'h'`, `'D'`, `'W'`, `'M'`, `'Y'` |
`bucket` also works on numeric columns: `bucket("Price", 10)` rounds values down
to the nearest multiple of 10.
## Other Functions
| Function | Description |
| ------------------------- | ----------------------------------------------------- |
| `is_null(x)` | Whether the value is null |
| `is_not_null(x)` | Whether the value is not null |
| `percent_of(a, b)` | `a` as a percentage of `b` |
| `inrange(low, val, high)` | Whether `val` is between `low` and `high` (inclusive) |
| `min(a, b, ...)` | Minimum of inputs |
| `max(a, b, ...)` | Maximum of inputs |
| `random()` | Random float between 0.0 and 1.0 |
| `col(name)` | Look up a column by string name at runtime |
| `vlookup(col, key)` | Look up a value in another column by row key |
## See also
Expressions are row-local — each output cell is computed from that row's
values alone. For calculations which span rows, such as moving averages,
cumulative sums or period-over-period differences, see
[Window Columns](./windows.md).
# Window Columns
The `windows` property declares _ordered, partitioned rolling computations_
over the rows of a `Table` — moving averages, cumulative sums,
period-over-period differences — analogous to SQL window functions.
Window Columns are declared per-`View`, keyed by output alias, exactly as
[`expressions`](./expressions.md) are:
Each window produces a new column which may be used anywhere a `Table` column
can — in `columns`, `filter`, `sort`, `group_by`, and so on. An alias must not
collide with a `Table` column, an expression alias, or another window's key.
Window Columns update incrementally as the `Table` updates, including rows
_outside_ an update batch whose window frames were affected by it.
## Spec fields
| Field | Type | Description |
| --- | --- | --- |
| `column` | `string` | The input column — a `Table` column or an expression alias from the same config |
| `aggregate` | `string` | The window function to apply (see below) |
| `partition_by` | `string[]` | Columns whose distinct value tuples partition the rows; omitted partitions the whole `Table` as one group |
| `order_by` | `[string, "asc" \| "desc"]` | The column which orders each partition, and its direction |
| `rows` | `integer` | Frame of the N rows preceding each row, plus the row itself |
| `range` | `number` | Frame of rows whose `order_by` value lies within `range` of each row's |
| `cumulative` | `true` | Frame of all rows from the partition start through each row |
| `offset` | `integer` | Row offset for `lag`/`lead` (default `1`) |
| `alpha` | `number` | Smoothing factor in `(0, 1]` for `ema` |
`rows`, `range` and `cumulative` are **mutually exclusive** — supplying more
than one is an error. `range` requires a numeric or temporal `order_by`.
order_by orders rows within the window
frame only. It does not reorder the View — that is what the
view-level sort
property does.
## Aggregates
| Aggregate | Description | Result type |
| --- | --- | --- |
| `sum`, `avg` | Rolling sum and mean over the frame | `float` |
| `stddev`, `var` | Rolling standard deviation and variance | `float` |
| `count` | Number of non-null values in the frame | `integer` |
| `min`, `max` | Smallest and largest value in the frame | input type |
| `lag`, `lead` | Value `offset` rows behind or ahead | input type |
| `diff` | This row's value minus the value `offset` rows behind | `float` |
| `rate` | Rate of change across the frame | `float` |
| `ema` | Exponential moving average, smoothed by `alpha` | `float` |
`sum`, `avg`, `stddev`, `var`, `diff`, `rate` and `ema` require a numeric
input column.
### Frame compatibility
- `sum`, `avg`, `count`, `min`, `max`, `stddev` and `var` accept any frame.
- `lag`, `lead`, `diff` and `ema` are frame-independent — they are computed
from row offsets rather than a frame.
- **`rate` requires a `range` frame**, and is invalid with `rows` or
`cumulative`.
The first and last window
aggregates are declared in the type definitions but are not yet
implemented by the engine; a View which uses them will be
rejected.
## Examples
### Moving average over a fixed row count
A 10-tick moving average, over the whole table in its natural order:
```json
{
"columns": ["10-tick avg Sales"],
"windows": {
"10-tick avg Sales": {
"column": "Sales",
"aggregate": "avg",
"rows": 10
}
}
}
```
### Moving average over a time range
A 5-second moving average, framing rows by their `Order Date` rather than by
count:
```json
{
"columns": ["5s avg Sales"],
"windows": {
"5s avg Sales": {
"column": "Sales",
"aggregate": "avg",
"order_by": ["Order Date", "asc"],
"range": 5000
}
}
}
```
### Cumulative sum
A running total from the start of each partition:
```json
{
"columns": ["Cumulative Sales"],
"windows": {
"Cumulative Sales": {
"column": "Sales",
"aggregate": "sum",
"order_by": ["Order Date", "asc"],
"cumulative": true
}
}
}
```
### Period-over-period change, per group
`partition_by` restarts the window at each new `Region`, so each region's
first row has no predecessor to difference against:
```json
{
"columns": ["Region", "Sales", "Sales Δ"],
"windows": {
"Sales Δ": {
"column": "Sales",
"aggregate": "diff",
"partition_by": ["Region"],
"order_by": ["Order Date", "asc"]
}
}
}
```
## Support
Window Columns are implemented by Perspective's built-in engine, by the
DuckDB, ClickHouse, PostgreSQL and Polars
[Virtual Servers](../../virtual_servers.md), and by the
`` UI. Virtual Servers advertise support through their
_features_ declaration, so the UI control is hidden for backends which do not
implement it.
# Advanced View Operations
Beyond the standard query configuration, `View` provides additional methods for
interacting with hierarchical results and introspecting data.
## Tree Hierarchy Operations
When a `View` has `group_by` applied, the results form a tree hierarchy.
Perspective provides methods to control which levels of the tree are expanded or
collapsed:
```javascript
const view = await table.view({ group_by: ["Region", "Country", "City"] });
// Collapse the tree at row index 5
await view.collapse(5);
// Expand the tree at row index 5
await view.expand(5);
// Set the expansion depth (0 = fully collapsed, 1 = first level, etc.)
await view.set_depth(1);
```
Using the sync API
```python
view = table.view(group_by=["Region", "Country", "City"])
view.collapse(5)
view.expand(5)
view.set_depth(1)
```
Perspective's built-in engine is lazy — aggregates for
collapsed rows are not recalculated when the underlying `Table` is updated.
Updates are only computed for rows that are currently visible (expanded). When a
collapsed row is later expanded, its aggregates are calculated at that
point.
## Column Range Queries
`View::get_min_max` returns the minimum and maximum values for a given column,
which is useful for setting up scales in custom visualizations:
## Describing a View Config
`Table::describe` validates a complete view config against a table and
reports the schema a `View` built from it would have - without creating one.
`describe` reports a `view_schema` if and only if `Table::view` with the same
config would succeed, and the two schemas are equal. It costs no engine
resources, so it is the right way to check a config before applying it.
```python
verdict = table.describe(
columns=["Sales", "margin"],
group_by=["Region"],
expressions={"margin": '"Profit" / "Sales"'},
aggregates={"margin": "avg"},
)
if "view_schema" in verdict:
...
elif "expression_errors" in verdict:
...
else:
verdict["config_error"]
```
## Expression Validation
`Table::validate_expressions` is a specialization of `Table::describe` over
a config that selects no columns, so only the expressions are checked. It
returns which expressions are valid and their inferred types, plus an
`expression_alias` map echoing the request:
```javascript
const result = await table.validate_expressions({
expr1: '"Sales" + "Profit"',
expr2: "invalid_column + 1",
});
// result.expression_schema contains valid expressions and their types
// result.errors contains invalid expressions and error messages
```
```python
result = table.validate_expressions(['"Sales" + "Profit"', 'invalid + 1'])
```
## View Dimensions
`View::dimensions` returns the number of rows and columns in the current view,
including information about group-by header rows:
When `mode` is set to `"row"`, the callback receives a delta of only the rows
that changed (as Apache Arrow), which is useful for efficiently synchronizing
tables across clients.
## Remove Callbacks
Register a callback to be notified whenever rows are removed from the underlying
`Table` by `remove()`, which requires an `index`. The callback receives the
`port_id` and the removed `index` column values as an Apache Arrow of a single
column named after the index. It fires once per update step, only for rows which
existed before that step; `replace()` reports the keys it does not re-supply,
and `clear()` reports every key:
## Flattening a View into a Table
A [`Table`] can be constructed on a [`Table::view`] instance, which will return
a new [`Table`] based on the [`Table::view`]'s dataset, and all future updates
that affect the [`Table::view`] will be forwarded to the new [`Table`]. This is
particularly useful for implementing a
[Client/Server Replicated](../architecture/client_server.md) design, as it
handles the `View` serialization and `on_update` forwarding for you. This
pattern is available in JavaScript, Python and Rust.
When the source `Table` has an `index`, and the `View` is unpivoted and includes
the index column, the new `Table` inherits that `index` and subscribes to the
source's `on_remove()`, so in-place updates and `remove()` calls on the source
are mirrored rather than appended. A pivoted `View`, or one which omits the
index column, produces an unindexed, append-only `Table`. A `limit` is inherited
the same way. `replace()` and `clear()` on the source are mirrored too.
```rust
let opts = TableInitOptions::default();
let data = TableData::Update(UpdateData::Csv("x,y\n1,2\n3,4".into()));
let table = client.table(data, opts).await?;
let view = table.view(None).await?;
let table2 = client.table(TableData::View(view)).await?;
table.update(data).await?;
```
# Join
`Client::join` creates a read-only `Table` by joining two source tables on a
shared key column. The `left` and `right` arguments can be `Table` objects or
string table names (as returned by `get_hosted_table_names()`). The resulting
table is _reactive_: whenever either source table is updated, the join is
automatically recomputed and any `View` derived from the joined table will
update accordingly.
Joined tables support the full `View` API — you can apply `group_by`,
`split_by`, `sort`, `filter`, `expressions`, and all other `View` operations on
the result, just as you would with any other `Table`.
# Join Types
`Client::join` supports three join types, specified via the `join_type` option.
The default is `"inner"`.
## Inner Join (default)
An inner join includes only rows where the key column exists in _both_ source
tables. Rows from either table that have no match in the other are excluded.
## Left Join
A left join includes all rows from the left table. For left rows that have no
match in the right table, right-side columns are filled with `null`.
## Outer Join
An outer join includes all rows from both tables. Unmatched rows on either side
have their missing columns filled with `null`.
| `join_type` | Left-only rows | Right-only rows |
| ----------- | -------------- | --------------- |
| `"inner"` | excluded | excluded |
| `"left"` | included | excluded |
| `"outer"` | included | included |
# Join Options
## `on` — Join Key Column
The `on` parameter specifies the column name used to match rows between the left
and right tables. This column must exist in the left table and, by default, must
also exist in the right table with the same name and compatible type.
The join key column becomes the index of the resulting table.
## `right_on` — Different Right Key Column
When the join key has a different name in the right table, use `right_on` to
specify the right table's column name. The left table's column name (`on`) is
used in the output schema; the right key column is excluded from the result.
The `on` and `right_on` columns must have compatible types. An error is thrown
if the types do not match.
## `join_type` — Join Type
Controls which rows are included in the result. See
[Join Types](./join_types.md) for details.
| Value | Behavior |
| ----------- | ----------------------------------------------------- |
| `"inner"` | Only rows with matching keys in both tables (default) |
| `"left"` | All left rows; unmatched right columns are `null` |
| `"outer"` | All rows from both tables; unmatched columns are `null` |
## `name` — Table Name
An optional name for the resulting joined table. If omitted, a random name is
generated. This name is used to identify the table in the server's hosted table
registry.
# Reactivity and Constraints
## Reactive Updates
Joined tables are fully reactive. When either source table receives an
`update()`, the join is automatically recomputed and any `View` created from the
joined table will reflect the new data. This includes:
- Updates that modify existing rows in either source table.
- New rows added to either source table that create new matches.
- Chained joins — if a joined table is itself used as input to another join,
updates propagate through the entire chain.
## Duplicate Keys
Like SQL, `join()` produces a cross-product for each matching key value. When
multiple rows in the left table share the same key, each is paired with every
matching row in the right table (and vice versa). The number of output rows for
a given key is `left_count × right_count`.
This behavior depends on whether the source tables are _indexed_:
- **Unindexed tables** (no `index` option) — rows are appended, so duplicate
keys accumulate naturally. Each `update()` appends new rows, which may
introduce additional duplicates.
- **Indexed tables** (`index` set to the join key) — each key appears at most
once per table, so the join produces at most one row per key. Updates replace
existing rows in-place rather than appending.
## Read-Only
Joined tables are read-only. Calling `update()`, `remove()`, `clear()`, or
`replace()` on a joined table will throw an error. Data can only change
indirectly, by updating the source tables.
## Column Name Conflicts
The left and right tables must not have overlapping column names (other than the
join key). If a non-key column name appears in both tables, `join()` throws an
error. Rename columns in your source data or use `View` expressions to avoid
conflicts.
## Source Table Deletion
A source table cannot be deleted while a joined table depends on it. You must
delete the joined table first, then delete the source tables.
# JavaScript Installation and Module Structure
Perspective is designed for flexibility, allowing developers to pick and choose
which modules they need. The main modules are:
- `@perspective-dev/client`
The data engine library, as both a browser ES6 and Node.js module. Provides a
WebAssembly, WebWorker (browser) and Process (node.js) runtime.
- `@perspective-dev/viewer`
A user-configurable visualization widget, bundled as a
[Web Component](https://www.webcomponents.org/introduction). This module
includes the core data engine module as a dependency.
`` by itself only implements a trivial debug renderer, which
prints the currently configured `view()` as a CSV. Plugin modules are packaged
separately and must be imported individually.
- `@perspective-dev/viewer-datagrid`
A custom high-performance data-grid component based on HTML `
`.
- `@perspective-dev/viewer-charts`
A set of charting components base on WebGL.
When imported after `@perspective-dev/viewer`, the plugin modules will register
themselves automatically, and the renderers they export will be available in the
`plugin` dropdown in the `` UI.
## Browser
Perspective's WebAssembly data engine is available via NPM in the same package
as its Node.js counterpart, `@perspective-dev/client`. The Perspective Viewer UI
(which has no Node.js component) must be installed separately:
```bash
$ npm add @perspective-dev/client @perspective-dev/viewer
```
By itself, `@perspective-dev/viewer` does not provide any visualizations, only
the UI framework. Perspective _Plugins_ provide visualizations and must be
installed separately. All Plugins are optional - but a ``
without Plugins would be rather boring!
```bash
$ npm add @perspective-dev/viewer-charts @perspective-dev/viewer-datagrid
```
## Node.js
To use Perspective from a Node.js server, simply install via NPM.
```bash
$ npm add @perspective-dev/client
```
# JavaScript - Importing with or without a bundler
Perspective requires the browser to have access to Perspective's `.wasm`
binaries _in addition_ to the bundled `.js` files, and as a result the build
process requires a few extra steps. Perspective's NPM releases come with
multiple prebuilt configurations.
## ESM builds with a bundler
The recommended builds for production use are packaged as ES Modules and require
a _bootstrapping_ step in order to acquire the `.wasm` binaries and initialize
Perspective's JavaScript with them. Because they have no hard-coded dependencies
on the `.wasm` paths, they are ideal for use with JavaScript bundlers such as
ESBuild, Rollup, Vite or Webpack.
ESM builds must be _bootstrapped_ with their `.wasm` binaries to initialize. The
`wasm` binaries can be found in their respective `dist/wasm` directories.
```javascript
import perspective_viewer from "@perspective-dev/viewer";
import perspective from "@perspective-dev/client";
// TODO These paths must be provided by the bundler!
const SERVER_WASM = ... // "@perspective-dev/server/dist/wasm/perspective-server.wasm"
const CLIENT_WASM = ... // "@perspective-dev/viewer/dist/wasm/perspective-viewer.wasm"
await Promise.all([
perspective.init_server(SERVER_WASM),
perspective_viewer.init_client(CLIENT_WASM),
]);
// Now Perspective API will work!
const worker = await perspective.worker();
const viewer = document.createElement("perspective-viewer");
```
The exact syntax will vary slightly depending on the bundler.
### Memory64 (wasm64)
`@perspective-dev/server` also ships a WebAssembly Memory64 build of the
engine, `dist/wasm/perspective-server.memory64.wasm`, which raises the
engine's heap ceiling from 4GB to 16GB (at some engine performance cost).
`init_server` accepts both binaries at once — register each as a _thunk_ and
only the selected binary is ever downloaded. The wasm64 binary is used
whenever the browser supports Memory64; registering only the wasm32 binary
(as above) opts out.
```javascript
perspective.init_server({
wasm32: () => fetch(SERVER_WASM),
wasm64: () => fetch(SERVER_WASM64),
});
```
### Vite
```javascript
import SERVER_WASM from "@perspective-dev/server/dist/wasm/perspective-server.wasm?url";
import CLIENT_WASM from "@perspective-dev/viewer/dist/wasm/perspective-viewer.wasm?url";
await Promise.all([
perspective.init_server(fetch(SERVER_WASM)),
perspective_viewer.init_client(fetch(CLIENT_WASM)),
]);
```
You'll also need to target `esnext` in your `vite.config.js` in order to run the
`build` step:
```javascript
import { defineConfig } from "vite";
export default defineConfig({
build: {
target: "esnext",
},
});
```
### ESBuild
```javascript
import SERVER_WASM from "@perspective-dev/server/dist/wasm/perspective-server.wasm";
import CLIENT_WASM from "@perspective-dev/viewer/dist/wasm/perspective-viewer.wasm";
await Promise.all([
perspective.init_server(fetch(SERVER_WASM)),
perspective_viewer.init_client(fetch(CLIENT_WASM)),
]);
```
ESBuild config JSON to encode this asset as a `file`:
```javascript
{
// ...
"loader": {
// ...
".wasm": "file"
}
}
```
### Webpack
```javascript
import SERVER_WASM from "@perspective-dev/server/dist/wasm/perspective-server.wasm";
import CLIENT_WASM from "@perspective-dev/viewer/dist/wasm/perspective-viewer.wasm";
await Promise.all([
perspective.init_server(SERVER_WASM),
perspective_viewer.init_client(CLIENT_WASM),
]);
```
Webpack config:
```javascript
{
// ...
module: {
// ...
rules: [
// ...
{
test: /\.wasm$/,
type: "asset/resource"
},
]
},
experiments: {
// ...
asyncWebAssembly: false,
syncWebAssembly: false,
},
}
```
## Inline builds with a bundler
Inline builds are deprecated and will be removed in a
future release.
Perspective's _Inline_ Builds work by _inlining_ WebAssembly binary content as
a base64-encoded string. While inline builds work with most bundlers and _do
not_ require bootstrapping, there is an inherent file-size and boot-performance
penalty. Prefer your bundler's inlining features and Perspective ESM builds
where possible.
```javascript
import "@perspective-dev/viewer/dist/esm/perspective-viewer.inline.js";
import psp from "@perspective-dev/client/dist/esm/perspective.inline.js";
```
## CDN builds
Perspective's CDN builds are good for non-bundled scenarios, such as importing
directly from a `
```
## Node.js builds
The Node.js runtime for the `@perspective-dev/client` module runs in-process by
default and does not implement a `child_process` interface. Hence, there is no
`worker()` method, and the module object itself directly exports the full
`perspective` API.
```javascript
const perspective = require("@perspective-dev/client");
```
In Node.js, perspective does not run in a WebWorker (as this API does not exist
in Node.js), so no need to call the `.worker()` factory function - the
`perspective` library exports the functions directly and run synchronously in
the main process.
# Accessing the Perspective engine via a `Client` instance
An instance of a `Client` is needed to talk to a Perspective `Server`, of which
there are a few varieties available in JavaScript.
## Web Worker (Browser)
Perspective's Web Worker client is actually a `Client` and `Server` rolled into
one. Instantiating this `Client` will also create a _dedicated_ Perspective
`Server` in a Web Worker process.
To use it, you'll need to instantiate a Web Worker `perspective` engine via the
`worker()` method. This will create a new Web Worker (browser) and load the
WebAssembly binary. All calculation and data accumulation will occur in this
separate process.
```javascript
const client = await perspective.worker();
```
The `worker` symbol will expose the full `perspective` API for one managed Web
Worker process. You are free to create as many as your browser supports, but be
sure to keep track of the `worker` instances themselves, as you'll need them to
interact with your data in each instance.
## Websocket (Browser)
Alternatively, with a Perspective server running in Node.js, Python or Rust, you
can create a _virtual_ `Client` via the `websocket()` method.
```javascript
const client = perspective.websocket("http://localhost:8080/");
```
## Node.js
The Node.js runtime for the `@perspective-dev/client` module runs in-process by
default and does not implement a `child_process` interface, so no need to call
the `.worker()` factory function. Instead, the `perspective` library exports the
functions directly and run synchronously in the main process.
```javascript
const client = require("@perspective-dev/client");
```
### Serializing data
The `view()` allows for serialization of data to JavaScript through the
`to_json()`, `to_ndjson()`, `to_columns()`, `to_csv()`, and `to_arrow()` methods
(the same data formats supported by the `Client::table` factory function). These
methods return a `promise` for the calculated data:
```javascript
const view = await table.view({ group_by: ["State"], columns: ["Sales"] });
// JavaScript Objects
console.log(await view.to_json());
console.log(await view.to_columns());
// String
console.log(await view.to_csv());
console.log(await view.to_ndjson());
// ArrayBuffer
console.log(await view.to_arrow());
```
`to_arrow()` writes an uncompressed Arrow IPC stream by default; pass
`compression` to apply LZ4 or ZSTD body compression, which `Client::table` reads
back transparently:
```javascript
const compressed = await view.to_arrow({ compression: "zstd" });
const table2 = await client.table(compressed);
```
# Deleting a `table()` or `view()`
Unlike standard JavaScript objects, Perspective objects such as `table()` and
`view()` store their associated data in the WebAssembly heap. Because of this,
as well as the current lack of a hook into the JavaScript runtime's garbage
collector from WebAssembly, the memory allocated to these Perspective objects
does not automatically get cleaned up when the object falls out of scope.
In order to prevent memory leaks and reclaim the memory associated with a
Perspective `table()` or `view()`, you must call the `delete()` method:
```javascript
await view.delete();
// This method will throw an exception if there are still `view()`s depending
// on this `table()`!
await table.delete();
```
Similarly, `` Custom Elements do not delete the memory
allocated for the UI when they are removed from the DOM.
```javascript
await viewer.delete();
```
# Server-only via `WebSocketServer()` and Node.js
For exceptionally large datasets, a `Client` can be bound to a
`perspective.table()` instance running in Node.js/Python/Rust remotely, rather
than creating one in a Web Worker and downloading the entire data set. This
trades off network bandwidth and server resource requirements for a smaller
browser memory and CPU footprint.
An example in Node.js:
```javascript
const { WebSocketServer, table } = require("@perspective-dev/client");
const fs = require("fs");
// Start a WS/HTTP host on port 8080. The `assets` property allows
// the `WebSocketServer()` to also serves the file structure rooted in this
// module's directory.
const host = new WebSocketServer({ assets: [__dirname], port: 8080 });
// Read an arrow file from the file system and host it as a named table.
const arr = fs.readFileSync(__dirname + "/superstore.lz4.arrow");
await table(arr, { name: "table_one" });
```
... and the [`Client`] implementation in the browser:
```javascript
const elem = document.getElementsByTagName("perspective-viewer")[0];
// Bind to the server's worker instead of instantiating a Web Worker.
const websocket = await perspective.websocket(
window.location.origin.replace("http", "ws"),
);
// Create a virtual `Table` to the preloaded data source. `table` and `view`
// objects live on the server.
const server_table = await websocket.open_table("table_one");
```
# Customizing `perspective.worker()`
`perspective.worker()` creates a `Client` that connects to a Perspective data
engine. By default it spins up a dedicated `Worker` running the built-in
WebAssembly engine, but you can pass an argument to change this behavior:
- A **`Worker`**, **`SharedWorker`**, or **`ServiceWorker`** — runs the
built-in engine in a different worker context.
- A **`MessagePort`** from `createMessageHandler()` — connects to a
[Virtual Server](virtual_server/custom.md) instead of the built-in engine.
## Built-in engine with a custom Worker
Pass a `Worker`, `SharedWorker`, or `ServiceWorker` that loads the worker script
distributed at
`"@perspective-dev/client/dist/cdn/perspective-server.worker.js"`.
`SharedWorker` and `ServiceWorker` have more complicated
behavior compared to a dedicated `Worker`, and will need special consideration
to integrate (or debug).
### Dedicated `Worker`
```javascript
const worker = await perspective.worker(new Worker(url));
```
### `SharedWorker`
```javascript
const worker = await perspective.worker(new SharedWorker(url));
```
### `ServiceWorker`
```javascript
const registration = await navigator.serviceWorker.register(url, {
scope: "", // Your scope here
});
const worker = await perspective.worker(registration.active);
```
## Virtual Server
Instead of the built-in WebAssembly engine, `perspective.worker()` can connect
to a Virtual Server — an adapter that translates Perspective queries into
operations on an external data source such as
[DuckDB](virtual_server/duckdb.md) or
[ClickHouse](virtual_server/clickhouse.md).
Use `perspective.createMessageHandler()` with a `VirtualServerHandler` to create
a `MessagePort`, then pass it to `worker()`:
```javascript
import perspective from "@perspective-dev/client";
const handler = {
/* VirtualServerHandler implementation */
};
const server = perspective.createMessageHandler(handler);
const client = await perspective.worker(server);
const table = await client.open_table("my_table");
```
The returned `Client` works identically to one backed by the built-in engine —
you can pass it to `.load()`, call `open_table()`, etc. The
difference is that queries are fulfilled by your handler rather than the WASM
engine.
For the full `VirtualServerHandler` interface and a worked example, see
[Implementing a custom Virtual Server](virtual_server/custom.md).
# Joining Tables
`perspective.join()` creates a read-only `Table` by joining two source tables on
a shared key column. The result is reactive — it updates automatically when
either source table changes. See [`Join`](../../explanation/join.md) for
conceptual details.
## Basic Inner Join
```javascript
const orders = await perspective.table([
{ id: 1, product_id: 101, qty: 5 },
{ id: 2, product_id: 102, qty: 3 },
{ id: 3, product_id: 101, qty: 7 },
]);
const products = await perspective.table([
{ product_id: 101, name: "Widget" },
{ product_id: 102, name: "Gadget" },
]);
const joined = await perspective.join(orders, products, "product_id");
const view = await joined.view();
const json = await view.to_json();
// [
// { product_id: 101, id: 1, qty: 5, name: "Widget" },
// { product_id: 101, id: 3, qty: 7, name: "Widget" },
// { product_id: 102, id: 2, qty: 3, name: "Gadget" },
// ]
```
## Join Types
Pass `join_type` in the options to select inner, left, or outer join behavior:
```javascript
// Left join: all left rows, nulls for unmatched right columns
const left_joined = await perspective.join(left, right, "id", {
join_type: "left",
});
// Outer join: all rows from both tables
const outer_joined = await perspective.join(left, right, "id", {
join_type: "outer",
});
```
## Reactive Updates
The joined table recomputes automatically when either source table is updated:
```javascript
const left = await perspective.table([{ id: 1, x: 10 }]);
const right = await perspective.table([{ id: 2, y: "b" }]);
const joined = await perspective.join(left, right, "id");
const view = await joined.view();
let json = await view.to_json();
// [] — no matching keys yet
await right.update([{ id: 1, y: "a" }]);
json = await view.to_json();
// [{ id: 1, x: 10, y: "a" }] — new match detected
```
# `` Custom Element library
`` provides a complete graphical UI for configuring the
`perspective` library and formatting its output to the provided visualization
plugins.
Once imported and initialized in JavaScript, the `` Web
Component will be available in any standard HTML on your site. A simple example:
```html
```
`load()` binds the viewer to a `Client`, and `restore()` selects which of that
client's `Table`s to show via the `table` field. Because `load()` alone
selects no table, it does not render — the pair guarantees exactly one atomic
render.
Passing a `Table` directly is still supported as a legacy shorthand,
```javascript
await viewer.load(table);
```
... which is internally equivalent to:
```javascript
await viewer.load(await table.get_client());
await viewer.restore({ table: await table.get_name() });
```
Always give your Table a name.
When name is omitted a random one is assigned, so the
table field in a token from save() will not match
the table after a page reload, and restore() will fail.
## Attributes
`` can be configured via HTML attributes or JavaScript
properties. When set as attributes, the viewer will apply the configuration on
initialization:
```html
```
## UI Features
The viewer provides an interactive side panel with:
- **Column list** - drag and drop columns to configure `group_by`, `split_by`,
`sort`, and `filter` fields.
- **New Column** button - opens an expression editor for creating computed
columns via the [expression language](../../explanation/view/config/expressions.md).
- **Plugin selector** - switch between the visualization plugins registered on
the page. `@perspective-dev/viewer-datagrid` provides `Datagrid`;
`@perspective-dev/viewer-charts` provides `X Bar`, `Y Bar`, `Y Line`,
`Y Scatter`, `Y Area`, `X/Y Scatter`, `X/Y Line`, `Density`, `Treemap`,
`Sunburst`, `Heatmap`, `Candlestick`, `OHLC`, `Map Scatter`, `Map Line` and
`Map Density`.
- **Theme** selector - toggle between available themes.
- **Export** - download the current view as CSV or Arrow.
- **Copy** - copy the current view to the clipboard.
- **Reset** - restore the viewer to its default configuration.
## Methods
A `` hosts one or more _panels_. Methods which address a
single panel take an options-dict with an optional `panel` id, defaulting to
the _active_ panel — e.g. `await viewer.save({ panel: "PANEL_ID_0" })`.
### Binding
| Method | Description |
|---|---|
| `load(client)` | Bind a `Client` (or, legacy, a `Table`) to the viewer |
| `eject(options?)` | Remove a `Client` and dispose every panel bound to it |
| `delete()` | Release the element's resources |
| `getClient(options?)` | Get a bound `Client` |
| `getTable(options?)` | Get a panel's `Table` |
| `getView(options?)` | Get a panel's `View` — `mode` selects `"live"` (default), `"clone"` or `"auto"` |
| `getViewConfig(options?)` | Get a panel's `ViewConfig` |
#### `View` lifecycle
By default `getView()` returns the panel's own _live_ `View`. The viewer owns
it: every config change replaces it, and auto-pause (the element scrolled out
of view, `display: none`, or a backgrounded tab) or `delete()` deletes it. Any
call on a live `View` can therefore fail with "View already deleted", even one
already in flight, and it must never be `delete()`d by the caller. While a
panel is auto-paused or has not rendered yet it has no live `View`, and
`getView()` rejects with `No View for panel ""`; `setAutoPause(false)`
forces one to exist for every panel, at the cost of a live subscription each.
A reference that has to outlive the render lifecycle should not borrow the
viewer's `View` at all. `mode: "clone"` builds a new `View` from the panel's
effective config (the element's global filter included) which is independent
of rendering and pause, and which the caller owns and must `delete()`:
```javascript
for (const panel of viewer.getPanelNames()) {
const view = await viewer.getView({ panel, mode: "clone" });
const arrow = await view.to_arrow();
await view.delete();
}
```
`mode: "auto"` returns the live `View` when one exists and a clone otherwise.
Ownership follows whichever was returned, so use it only where the caller
never deletes the result.
### Configuration
| Method | Description |
|---|---|
| `save(options?)` | Serialize one panel's configuration |
| `restore(config, options?)` | Apply a configuration to one panel |
| `saveWorkspace()` | Serialize the whole element — every panel, plus layout and global filters |
| `restoreWorkspace(config)` | Apply a workspace config update; absent fields are left unchanged |
| `reset(all?, options?)` | Reset configuration (pass `true` to also reset expressions) |
| `resetError()` | Clear the error overlay |
### Panels
| Method | Description |
|---|---|
| `addPanel(config)` | Add a panel, returning its generated id |
| `removePanel(id)` | Remove a panel |
| `getPanelNames()` | List panel ids |
| `getActivePanel()` / `setActivePanel(id)` | Get or set the active panel |
### Output
| Method | Description |
|---|---|
| `export(options?)` | Export a panel — see the export methods below |
| `download(options?)` | Export and download as a file |
| `copy(options?)` | Copy a panel to the clipboard |
`export()`, `download()` and `copy()` all take a `method`, one of `"csv"`,
`"json"`, `"ndjson"`, `"arrow"`, `"arrow-lz4"` or `"arrow-zstd"` — each with
`-all` and `-selected` variants (e.g. `"csv-selected"`, `"arrow-zstd-all"`) —
plus `"html"`, `"json-config"`, and `"plugin"`. The `"arrow-lz4"` and
`"arrow-zstd"` methods write the Arrow IPC stream with LZ4 or ZSTD body
compression.
The `"plugin"` method asks the plugin to render itself, which produces a PNG
for charts and text for the datagrid.
| `getSelection(options?)` / `setSelection(...)` | Get or set the selected region |
| `getEditPort(options?)` | Get a panel's edit port |
| `getRenderStats(options?)` | Get render timing statistics |
### Rendering and chrome
| Method | Description |
|---|---|
| `flush()` | Wait for any pending UI updates to complete |
| `resize(options?)` | Redraw, optionally at a `{dimensions: {width, height}}` size hint |
| `setAutoSize(bool)` / `setAutoPause(bool)` / `setThrottle(ms)` | Render policy |
| `toggleConfig(force?)` | Toggle the settings sidebar |
| `toggleColumnSettings(...)` | Toggle the column settings sidebar |
| `resetThemes(themes?)` | Re-detect or explicitly set available themes |
| `restyleElement()` | Re-read CSS and repaint |
| `getPlugin(name?)` / `getAllPlugins()` | Look up registered plugins |
See [Saving and restoring UI state](./save_restore.md) for the `save`/`restore`
formats and the panel selector, and
[Plugin render limits](./plugin_settings.md) for `getPlugin`.
# Loading data from a Table
Data can be loaded into `` in the form of a `Table()` or a
`Promise
` via the `load()` method.
```javascript
// Create a new worker, then a new table promise on that worker.
const worker = await perspective.worker();
const table = await worker.table(data);
// Bind a viewer element to this table.
await viewer.load(table);
```
## Sharing a `Table` between multiple ``s
Multiple ``s can share a `table()` by passing the `table()`
into the `load()` method of each viewer. Each `perspective-viewer` will update
when the underlying `table()` is updated, but `table.delete()` will fail until
all `perspective-viewer` instances referencing it are also deleted:
```javascript
const viewer1 = document.getElementById("viewer1");
const viewer2 = document.getElementById("viewer2");
// Create a new WebWorker
const worker = await perspective.worker();
// Create a table in this worker
const table = await worker.table(data);
// Load the same table in 2 different elements
await viewer1.load(table);
await viewer2.load(table);
// Both `viewer1` and `viewer2` will reflect this update
await table.update([{ x: 5, y: "e", z: true }]);
```
## Loading from a virtual `Table`
Loading a virtual (server-only) `Table` works just like loading a local/Web
Worker `Table` — just pass the virtual `Table` to `viewer.load()`. In the
browser:
```javascript
const elem = document.getElementsByTagName("perspective-viewer")[0];
// Bind to the server's worker instead of instantiating a Web Worker.
const websocket = await perspective.websocket(
window.location.origin.replace("http", "ws")
);
// Bind the viewer to the preloaded data source. `table` and `view` objects
// live on the server.
const server_table = await websocket.open_table("table_one");
await elem.load(server_table);
```
Alternatively, data can be _cloned_ from a server-side virtual `Table` into a
client-side WebAssembly `Table`. The browser clone will be synced via delta
updates transferred via Apache Arrow IPC format, but local `View`s created will
be calculated locally on the client browser.
```javascript
const worker = await perspective.worker();
const server_view = await server_table.view();
const client_table = worker.table(server_view);
await elem.load(client_table);
```
`` instances bound in this way are otherwise no different
than ``s which rely on a Web Worker, and can even share a
host application with Web Worker-bound `table()`s. The same `promise`-based API
is used to communicate with the server-instantiated `view()`, only in this case
it is over a websocket.
# Theming
Theming is supported in `perspective-viewer` and its accompanying plugins. A
number of themes come bundled with `perspective-viewer`; you can import any of
these themes directly into your app, and the `perspective-viewer`s will be
themed accordingly:
```javascript
// Themes based on Thought Merchants's Prospective design
import "@perspective-dev/viewer/dist/css/pro.css";
import "@perspective-dev/viewer/dist/css/pro-dark.css";
// Other themes
import "@perspective-dev/viewer/dist/css/solarized.css";
import "@perspective-dev/viewer/dist/css/monokai.css";
import "@perspective-dev/viewer/dist/css/vaporwave.css";
// ...
```
Alternatively, you may use `themes.css`, which bundles all default themes
```javascript
import "@perspective-dev/viewer/dist/css/themes.css";
```
If you choose not to bundle the themes yourself, they are available through
[CDN](https://cdn.jsdelivr.net/npm/@perspective-dev/viewer/dist/css/). These can
be directly linked in your HTML file:
```html
```
Note the `crossorigin="anonymous"` attribute. When including a theme from a
cross-origin context, this attribute may be required to allow
`` to detect the theme. If this fails, additional themes are
added to the `document` after `` init, or for any other
reason theme auto-detection fails, you may manually inform
`` of the available theme names with the `.resetThemes()`
method.
```javascript
// re-auto-detect themes
viewer.resetThemes();
// Set available themes explicitly (they still must be imported as CSS!)
viewer.resetThemes(["Pro Light", "Pro Dark"]);
```
`` will default to the first loaded theme when initialized.
You may override this via `.restore()`, or provide an initial theme by setting
the `theme` attribute:
```html
```
or
```javascript
const viewer = document.querySelector("perspective-viewer");
await viewer.restore({ theme: "Pro Dark" });
```
## Custom Themes
The best way to write a new theme is to
[fork and modify an existing theme](https://github.com/perspective-dev/perspective/tree/master/rust/perspective-viewer/src/themes),
which are _just_ collections of regular CSS variables — Perspective's own
themes are plain `.css` files, with no preprocessor involved.
`` is
not "themed" by default and will lack icons and label text in addition to colors
and fonts, so starting from an empty theme forces you to define _every_
theme-able variable to get a functional UI.
### Icons and Translation
UI icons are defined by CSS variables provided by
[`@perspective-dev/viewer/dist/css/icons.css`](https://github.com/perspective-dev/perspective/blob/master/rust/perspective-viewer/src/themes/icons.css).
These variables must be defined for the UI icons to work - there are no default
icons without a theme.
UI text is also defined in CSS variables provided by
[`@perspective-dev/viewer/dist/css/intl.css`](https://github.com/perspective-dev/perspective/blob/master/rust/perspective-viewer/src/themes/intl.css),
and has identical import requirements. Some _example definitions_
(automatically-translated sans-editing) can be found
[`@perspective-dev/viewer/dist/css/intl/` folder](https://github.com/perspective-dev/perspective/tree/master/rust/perspective-viewer/src/themes/intl).
Importing the pre-built `themes.css` stylesheet as well as a custom theme will
define Icons and Translation globally as a side-effect. You can still customize
icons in this mode with rules (of the appropriate specificity), _but_ if you do
not still remember to define these variables yourself, your theme will not work
without the base `themes.css` package available.
# Saving and restoring UI state.
`` is _persistent_, in that its entire state (sans the data
itself) can be serialized or deserialized. This include all column, filter,
pivot, expressions, etc. properties, as well as datagrid style settings, config
panel visibility, and more. This overloaded feature covers a range of use cases:
- Setting a ``'s initial state after a `load()` call.
- Updating a single or subset of properties, without modifying others.
- Resetting some or all properties to their data-relative default.
- Persisting a user's configuration to `localStorage` or a server.
## Serializing and deserializing the viewer state
To retrieve the entire state as a JSON-ready JavaScript object, use the `save()`
method. `save()` also supports a few other formats such as `"arraybuffer"` and
`"string"` (base64, not JSON), which you may choose for size at the expense of
easy migration/manual-editing.
```javascript
const json_token = await elem.save();
const string_token = await elem.save("string");
```
For any format, the serialized token can be restored to any
`` with a `Table` of identical schema, via the `restore()`
method. Note that while the data for a token returned from `save()` may differ,
generally its schema may not, as many other settings depend on column names and
types.
```javascript
await elem.restore(json_token);
await elem.restore(string_token);
```
As `restore()` dispatches on the token's type, it is important to make sure that
these types match! A common source of error occurs when passing a
JSON-stringified token to `restore()`, which will assume base64-encoded msgpack
when a string token is used.
```javascript
// This will error!
await elem.restore(JSON.stringify(json_token));
```
### Updating individual properties
Using the JSON format, every facet of a ``'s configuration
can be manipulated from JavaScript using the `restore()` method. The valid
structure of properties is described via the
[`ViewerConfigUpdate`](https://github.com/perspective-dev/perspective/blob/master/rust/perspective-viewer/src/ts/ts-rs/ViewerConfigUpdate.ts)
and embedded
[`ViewConfigUpdate`](https://github.com/perspective-dev/perspective/blob/master/rust/perspective-js/src/ts/ts-rs/ViewConfigUpdate.ts)
type declarations (both generated from the Rust definitions), and the
[`View`](../../explanation/view.md) chapter of the documentation which has
several examples for each `ViewConfig` property.
```javascript
// Set the plugin (will also update `columns` to plugin-defaults)
await elem.restore({ plugin: "X Bar" });
// Update plugin and columns (only draws once)
await elem.restore({ plugin: "X Bar", columns: ["Sales"] });
// Open the config panel
await elem.restore({ settings: true });
// Create an expression
await elem.restore({
columns: ['"Sales" + 100'],
expressions: { "New Column": '"Sales" + 100' },
});
// ERROR if the column does not exist in the schema or expressions
// await elem.restore({columns: ["\"Sales\" + 100"], expressions: {}});
// Add a filter
await elem.restore({ filter: [["Sales", "<", 100]] });
// Add a sort, don't remove filter
await elem.restore({ sort: [["Prodit", "desc"]] });
// Reset just filter, preserve sort
await elem.restore({ filter: undefined });
// Reset all properties to default e.g. after `load()`
await elem.reset();
```
Another effective way to quickly create a token for a desired configuration is
to simply copy the token returned from `save()` after settings the view manually
in the browser. The JSON format is human-readable and should be quite easy to
tweak once generated, as `save()` will return even the default settings for all
properties. You can call `save()` in your application code, or e.g. through the
Chrome developer console:
```javascript
// Copy to clipboard
copy(await document.querySelector("perspective-viewer").save());
```
## Multi-panel viewers
`save()` and `restore()` operate on a _single_ panel — the _active_ one by
default, or a specific panel via their optional `{ panel }` selector (e.g.
`await elem.save({ panel: "my-panel" })`). If `restore()`'s `panel` names no
existing panel, a new panel is created with that id.
A `` may host multiple panels. To serialize or restore the
_whole element_ — every panel plus the layout and cross-filter state — use
`saveWorkspace()` and `restoreWorkspace()` instead:
```javascript
const workspace_token = await elem.saveWorkspace();
await elem.restoreWorkspace(workspace_token);
```
A `saveWorkspace()` token is a `WorkspaceConfig`
(`{ version, layout, panels, ... }`), not a `ViewerConfig` — passing it to the
single-panel `restore()` will _not_ restore the layout (its `panels`/`layout`
keys are ignored).
Like `restore()`, `restoreWorkspace()` applies an _update_: a field that is
absent leaves that part of the element unchanged, `null` resets it to its
default, and a value replaces it. `panels` is the whole panel set — when
present every existing panel is replaced (`{}` empties the element); when
absent the panels are kept, and `layout`, `active` and `masters` name them by
their existing ids. So a cross-filter can be applied to the current panels
without re-creating them:
```javascript
await elem.restoreWorkspace({ global_filters: [["Region", "==", "West"]] });
await elem.restoreWorkspace({ global_filters: null }); // clear
```
## Colors, palettes and gradients
Per-column color styling lives in a panel's `columns_config`, keyed by column
name, and every color-scale value is a string usable verbatim in CSS:
| Kind | Value |
| -------- | ----------------------------------------------------------------------------------- |
| color | `"#rrggbb"` (`#rgb`, `rgb()` and `rgba()` are accepted on input) |
| palette | `"linear-gradient(to right, #rrggbb, #rrggbb, …)"` — N colors, **no** positions |
| gradient | `"linear-gradient(to right, #rrggbb 0%, #rrggbb 37.5%, …)"` — every stop positioned |
Which reader applies is decided by the style control's kind, never by inspecting
the string. The datagrid's `fg_color`/`bg_color` are read according to the
column's type and its `fg_mode`/`bg_mode`: a gradient for numeric columns, a
color for string and datetime columns in `"color"` mode, and a palette for
string columns in `"series"` mode; the charts' `gradient` is a gradient and
`palette` a palette. A position anywhere in a palette is rejected, while a
gradient may omit positions on input (the CSS implicit-position rules fill them)
and may carry any direction token, which is normalized to `to right`. A mode the
column's type does not accept (say `fg_mode: "series"` on a float column)
rejects the `restore()` call, as does a value that does not parse under the
mode's reader. Values equal to the plugin's default are not serialized.
```javascript
await viewer.restore({
plugin: "Datagrid",
columns_config: {
Profit: {
bg_mode: "gradient",
bg_color: "linear-gradient(to right, #ff0000, #ffffff, #0000ff)",
},
},
});
```
Any of these may instead be a reference to a CSS custom property of the same
kind — `"var(--psp-user--color-)"`, `"var(--psp-user--palette-)"` or
`"var(--psp-user--gradient-)"`. References are resolved when the config is
written, against the element's computed style: the `palette` of the last
`restoreWorkspace()` (below) takes precedence, then any `--psp-user--*` property
a theme or the page defines on the element. An unresolvable reference is dropped
(the plugin's default renders). Panels hold literals from then on — `save()`
always emits literals, and the column style tab always edits a literal.
`saveWorkspace()` emits a **palette**: every color value in use across the
panels is written in `panels` as a `var()` reference, and the top-level
`palette` map (custom property name → value) carries each referenced definition
once. Names are stable — a value keeps the name the last `restoreWorkspace()`
gave it when the values match, reuses a theme entry's name when it matches one
(`--psp-user---1`, `-2`, … are discovered by contiguous numbering), and
otherwise takes a fresh `--psp-user---N`. `restoreWorkspace()` applies
`palette` to the element as inline custom properties (replacing any previously
restored palette) before the panels' references resolve — which also makes it
the way to inject a brand or theme variation for a workspace to draw on.
By default only the values the panels reference are serialized; a restored
palette's unused entries, and values pinned during a session, are in-session
state. Pass `{ full_palette: true }` to emit the element's whole set — in-use
values unioned with the last restored palette and anything pinned since — for a
symmetric round trip:
```javascript
const used_only = await elem.saveWorkspace();
const everything = await elem.saveWorkspace({ full_palette: true });
```
In the column style tab, each color field's **Load** control lists the element's
set (plus theme entries) for every panel and applies a chosen entry's value to
the field; **Pin** — offered while the field holds a value the restored set
lacks — adds that value to the set for the rest of the session.
```javascript
await elem.restoreWorkspace({
palette: {
"--psp-user--gradient-heat":
"linear-gradient(to right, #0366d6, #ff7f0e)",
"--psp-user--palette-brand":
"linear-gradient(to right, #2771a8, #8b86ff, #ff471e)",
},
panels: {
sales: {
table: "superstore",
plugin: "Heatmap",
columns: ["Sales"],
columns_config: {
Sales: { gradient: "var(--psp-user--gradient-heat)" },
},
},
},
});
```
A malformed `palette` entry (a key outside
`--psp-user--{gradient,palette,color}-`, or a value its kind rejects) fails the
whole `restoreWorkspace()` before any panel changes.
# Listening for events
The `` Custom Element fires all the same HTML `Event`s that
standard DOM `HTMLElement` objects fire, in addition to a few custom
`CustomEvent`s which relate to UI updates including those initiaed through user
interaction.
## Update events
Whenever a ``s underlying `table()` is changed via the
`load()` or `update()` methods, a `perspective-view-update` DOM event is fired.
Similarly, `view()` updates instigated either through the Attribute API or
through user interaction will fire a `perspective-config-update` event:
```javascript
elem.addEventListener("perspective-config-update", function (event) {
var config = elem.save();
console.log("The view() config has changed to " + JSON.stringify(config));
});
```
## Click events
Whenever a ``'s grid or chart is clicked, a
`perspective-click` DOM event is fired containing a detail object with
`config`, `column_names`, `row` and `panel`.
The `config` object contains an array of `filters` that can be applied to a
`` through the use of `restore()` updating it to show the
filtered subset of data.
The `column_names` property contains an array of matching columns, the `row`
property returns the associated row data, and `panel` identifies the panel
which fired the event in a multi-panel viewer.
```javascript
elem.addEventListener("perspective-click", function (event) {
const { config, panel } = event.detail;
elem.restore(config, { panel });
});
```
## Selection events
`perspective-select` fires when a plugin's selection changes. Its detail is a
`PerspectiveSelectDetail`, exported from `@perspective-dev/viewer`:
| Field | Type | Description |
| --- | --- | --- |
| `selected` | `boolean` | Whether anything is currently selected |
| `row` | `object` | The associated row data |
| `column_names` | `string[]` | Matching column names |
| `removeConfigs` | `ViewConfigUpdate[]` | Configs whose filters should be _removed_ |
| `insertConfigs` | `ViewConfigUpdate[]` | Configs whose filters should be _applied_ |
| `panel` | `string?` | The originating panel, in a multi-panel viewer |
`removeConfigs` is applied first, then `insertConfigs`. The
`removeFilters` and `insertFilters` getters flatten each to a plain `Filter[]`.
```javascript
import { PerspectiveSelectDetail } from "@perspective-dev/viewer";
elem.addEventListener("perspective-select", function (event) {
const { insertFilters, removeFilters } = event.detail;
console.log("apply", insertFilters, "clear", removeFilters);
});
```
The detail.config field on
perspective-select was replaced by insertConfigs and
removeConfigs. Without an explicit removeConfigs,
a filter on a column outside the source's group_by,
split_by or filter cannot be cleared.
## Global filter events
In a multi-panel viewer, panels toggled to _Master_ contribute filter clauses
to an element-level global filter set, which is applied as a transient overlay
to every _detail_ panel (and never written into their saved configs).
- `perspective-global-filter` fires on a master panel's selection.
- `perspective-global-filter-update` fires whenever the global filter set
changes, with a `Filter[]` detail.
```javascript
elem.addEventListener("perspective-global-filter-update", function (event) {
console.log("Global filters are now", event.detail);
});
```
## Layout events
A multi-panel `` reports changes to its panel _collection_
on two separate channels. They are distinct facts — which panels exist, and
which one is selected — so neither event implies the other.
- `perspective-layout-update` fires when a panel is added to or removed from
the layout. Its `detail.panels` is the placed panel ids in insertion order,
identical to what [`getPanelNames()`](#) returns.
- `perspective-active-panel-update` fires when the active panel changes, with
a `detail.panel` of the new panel's id — or `null` at zero panels.
```javascript
elem.addEventListener("perspective-layout-update", function (event) {
console.log("Panels are now", event.detail.panels);
});
```
Geometry changes — dragging a split divider, reordering tabs — do **not** fire
these events, because they change the layout tree without changing the panel
set. Use `saveWorkspace()` to read the current geometry.
The workspace-layout-update and
workspace-new-view events from the removed
@perspective-dev/workspace package no longer exist.
perspective-layout-update is the closest replacement for the
former; for per-panel config changes use
perspective-config-update.
In the browser, DuckDBHandler resolves
Perspective's WASM module from the registered
<perspective-viewer> custom element, so it cannot be
constructed until that element has been defined. Off-browser, pass the module
explicitly as the second constructor argument.
Perspective never intercepts your SQL — it only discovers what `SHOW ALL
TABLES` reports — so DuckDB's own remote-data features are available directly:
```javascript
await conn.query(`CREATE SECRET (TYPE s3, KEY_ID '...', SECRET '...', REGION 'us-east-1')`);
await conn.query(`CREATE TABLE trades AS SELECT * FROM read_parquet('s3://bucket/trades/*.parquet')`);
```
## Examples
- [Browser DuckDB example](https://github.com/perspective-dev/perspective/tree/master/examples/esbuild-duckdb-virtual)
# ClickHouse Virtual Server
Perspective provides a built-in virtual server for
[ClickHouse](https://clickhouse.com/), allowing `` to query
ClickHouse tables directly from the browser.
For server-side Python usage, see the
[Python ClickHouse guide](../../python/virtual_server/clickhouse.md).
## Installation
```bash
npm install @perspective-dev/client @perspective-dev/viewer @clickhouse/client-web
```
## Usage
Connect to a ClickHouse instance and bind it to a Perspective viewer:
```javascript
import perspective from "@perspective-dev/client";
import "@perspective-dev/viewer";
import { createClient } from "@clickhouse/client-web";
// Connect to ClickHouse
const clickhouseClient = createClient({
url: "http://localhost:8123",
database: "default",
});
// Create a Perspective virtual server backed by ClickHouse
const handler = perspective.ClickhouseHandler(clickhouseClient);
const messageHandler = perspective.createMessageHandler(handler);
// Connect a viewer
const client = await perspective.worker(messageHandler);
const table = await client.open_table("my_table");
document.getElementById("viewer").load(table);
```
## Examples
- [Browser ClickHouse example](https://github.com/perspective-dev/perspective/tree/master/examples/esbuild-clickhouse-virtual)
# Implementing a custom Virtual Server
You can connect Perspective to any data source by implementing the
`VirtualServerHandler` interface and passing it to `createMessageHandler()`.
For background on virtual servers, see the
[Virtual Servers overview](../../../explanation/virtual_servers.md).
## Example
```typescript
import perspective from "@perspective-dev/client";
import type {
VirtualServerHandler,
ColumnType,
TableDescription,
ViewConfig,
ViewWindow,
VirtualDataSlice,
} from "@perspective-dev/client";
const handler = {
async getHostedTables(): Promise {
return ["my_table"];
},
async tableSchema(tableId: string): Promise> {
return { name: "string", price: "float", date: "date" };
},
async tableSize(tableId: string): Promise {
return 1000;
},
async tableMakeView(
tableId: string,
viewId: string,
config: ViewConfig,
): Promise {
// Translate `config` (group_by, sort, filter, etc.) into a query
// against your data source. Store the query keyed by `viewId`
// for later data retrieval.
},
async tableDescribe(
tableId: string,
config: ViewConfig,
): Promise {
// Validate `config` against `tableId` and report the schema the view
// would have, WITHOUT creating it: `{ expression_schema, view_schema }`
// when valid, `{ expression_schema, expression_errors }` when an
// expression is invalid, or `{ config_error }` otherwise.
return { expression_schema: {}, view_schema: { name: "string", price: "float" } };
},
async viewDelete(viewId: string): Promise {
// Clean up resources for this view
},
async viewGetData(
viewId: string,
config: ViewConfig,
schema: Record,
viewport: ViewWindow,
dataSlice: VirtualDataSlice,
): Promise {
// Query your data source using `config` and `viewport` for the
// row/column window. Push columnar results via `dataSlice.setCol()`.
},
getFeatures() {
return {
group_by: true,
sort: true,
filter_ops: {
string: ["==", "!=", "contains", "is null", "is not null"],
float: ["==", "!=", ">", "<", ">=", "<="],
},
aggregates: {
float: ["sum", "avg", "count", "min", "max"],
string: ["count", "any"],
},
};
},
} satisfies VirtualServerHandler;
// Create a message handler and use it like a worker
const messageHandler = perspective.createMessageHandler(handler);
const client = await perspective.worker(messageHandler);
const table = await client.open_table("my_table");
document.getElementById("viewer").load(table);
```
# React Component
We provide a React wrapper to prevent common issues and mistakes associated with
using the perspective-viewer web component in the context of React.
Before trying this example, please take a look at
[how to bootstrap perspective](./importing.md).
## `PerspectiveViewer`
A simple example using the `PerspectiveViewer` component:
```typescript
import React, { useCallback, useEffect, useRef } from "react";
import {
PerspectiveViewer,
} from "@perspective-dev/react";
import perspective from "@perspective-dev/client";
function App() {
const worker = useRef(null);
useEffect(() => {
(async () => {
worker.current = await perspective.worker();
const resp = await fetch("data.arrow");
const arrow = await resp.arrayBuffer();
await worker.current.table(arrow, { name: "my_table" });
})();
}, []);
return (
);
}
```
# What is `perspective-python`
Perspective for Python uses the exact same C++ data engine used by the
[WebAssembly version](https://docs.rs/perspective-js/latest/perspective_js/) and
[Rust version](https://docs.rs/crate/perspective/latest). The library consists
of many of the same abstractions and API as in JavaScript, as well as
Python-specific data loading support for [NumPy](https://numpy.org/),
[Pandas](https://pandas.pydata.org/) (and
[Apache Arrow](https://arrow.apache.org/), as in JavaScript).
Additionally, `perspective-python` provides a session manager suitable for
integration into server systems such as
[Tornado websockets](https://www.tornadoweb.org/en/stable/websocket.html),
[AIOHTTP](https://docs.aiohttp.org/en/stable/web_quickstart.html#websockets), or
[Starlette](https://www.starlette.io/websockets/)/[FastAPI](https://fastapi.tiangolo.com/advanced/websockets/),
which allows fully _virtual_ Perspective tables to be interacted with by
multiple `` in a web browser. You can also interact with a
Perspective table from python clients, and to that end client libraries are
implemented for both Tornado and AIOHTTP.
## Example
A simple example which loads an [Apache Arrow](https://arrow.apache.org/) and
computes a "Group By" operation, returning a new Arrow.
```python
from perspective import Server
client = Server().new_local_client()
table = client.table(arrow_bytes_data)
view = table.view(group_by = ["CounterParty", "Security"])
arrow = view.to_arrow()
```
[More Examples](https://github.com/perspective-dev/perspective/tree/master/examples)
are available on GitHub.
## What's included
The `perspective` module exports several tools:
- `Server` the constructor for a new instance of the Perspective data engine.
- The `perspective.widget` module exports `PerspectiveWidget`, the JupyterLab
widget for interactive visualization in a notebook cell.
- The `perspective.handlers` modules exports web frameworks handlers that
interface with a `perspective-client` in JavaScript.
- `perspective.handlers.tornado.PerspectiveTornadoHandler` for
[Tornado](https://www.tornadoweb.org/)
- `perspective.handlers.starlette.PerspectiveStarletteHandler` for
[Starlette](https://www.starlette.io/) and
[FastAPI](https://fastapi.tiangolo.com)
- `perspective.handlers.aiohttp.PerspectiveAIOHTTPHandler` for
[AIOHTTP](https://docs.aiohttp.org),
### Virtual UI server
As `` or any other Perspective `Client` will only consume
the data necessary to render the current screen (or whatever else was requested
via the API), this runtime mode allows large datasets without the need to copy
them entirely to the Browser, at the expense of network latency on UI
interaction/API calls.
### Notebooks
`PerspectiveWidget` is an [AnyWidget](https://anywidget.dev) that implements
the same API as ``, and runs such a viewer in either
server or client (via WebAssembly) mode.
The widget is bundled entirely inside the `perspective-python` wheel, so
there is no per-host extension to install. It runs identically in
[JupyterLab](https://jupyterlab.readthedocs.io/en/stable/), classic Jupyter
Notebook, VSCode notebooks, Google Colab and Marimo. Install the `jupyter`
extra to pull in `anywidget`:
```bash
pip install "perspective-python[jupyter]"
```
Separately, the _optional_ `@perspective-dev/jupyterlab` package provides
convenient builtin viewers for `csv`, `json`, or `arrow` files in JupyterLab.
With it installed, right-click a file of one of these types and choose the
appropriate `Perspective` option from the context menu.
# Installation
`perspective-python` contains full bindings to the Perspective API, a JupyterLab
widget, and WebSocket handlers for several webserver libraries that allow you to
host Perspective using server-side Python.
## PyPI
`perspective-python` can be installed from [PyPI](https://pypi.org) via `pip`:
```bash
pip install perspective-python
```
That's it! If JupyterLab is installed in this Python environment, you'll also
get the `perspective.widget.PerspectiveWidget` class when you import
`perspective` in a Jupyter Lab kernel.
# Loading data into a Table
A `Table` can be created from a dataset or a schema, the specifics of which are
[discussed](#loading-data-with-table) in the JavaScript section of the user's
guide. In Python, however, Perspective supports additional data types that are
commonly used when processing data:
- `pandas.DataFrame`
- `polars.DataFrame`
- `bytes` (encoding an Apache Arrow)
- `objects` (either extracting a repr or via reference)
- `str` (encoding as a CSV)
A `Table` is created in a similar fashion to its JavaScript equivalent:
```python
from datetime import date, datetime
import numpy as np
import pandas as pd
import perspective
data = pd.DataFrame({
"int": np.arange(100),
"float": [i * 1.5 for i in range(100)],
"bool": [True for i in range(100)],
"date": [date.today() for i in range(100)],
"datetime": [datetime.now() for i in range(100)],
"string": [str(i) for i in range(100)]
})
table = perspective.table(data, index="float")
```
Likewise, a `View` can be created via the `view()` method:
```python
view = table.view(group_by=["float"], filter=[["bool", "==", True]])
column_data = view.to_columns()
row_data = view.to_json()
```
## Polars Support
Polars `DataFrame` types work similarly to Apache Arrow input, which Perspective
uses to interface with Polars.
```python
df = polars.DataFrame({"a": [1,2,3,4,5]})
table = perspective.table(df)
```
## Pandas Support
Perspective's `Table` can be constructed from `pandas.DataFrame` objects.
Internally, this just uses
[`pyarrow::from_pandas`](https://arrow.apache.org/docs/python/pandas.html),
which dictates behavior of this feature including type support.
If the dataframe does not have an index set, an integer-typed column named
`"index"` is created. If you want to preserve the indexing behavior of the
dataframe passed into Perspective, simply create the `Table` with
`index="index"` as a keyword argument. This tells Perspective to once again
treat the index as a primary key:
```python
data.set_index("datetime")
table = perspective.table(data, index="index")
```
## Time Zone Handling
When parsing `"datetime"` strings, times without an explicit timezone offset are
interpreted as _UTC_. Strings with a timezone offset (e.g., `+05:00`) are
converted to UTC. All `"datetime"` values are stored internally as milliseconds
since the Unix epoch, and are _output_ as integer timestamps (milliseconds since
epoch) from methods like `to_columns()` and `to_json()`.
Python `datetime` objects are serialized to strings before parsing. Naive
`datetime` objects (without `tzinfo`) produce strings without timezone
information and are therefore treated as UTC. Timezone-aware `datetime` objects
include their offset in the serialized string, which is used to convert to UTC.
`"date"` values are timezone-agnostic calendar days with no time component.
They are _output_ as integer timestamps at _UTC midnight_ of the calendar day
(equivalent to Arrow `date32` day arithmetic), and integer timestamp _input_ to
a `"date"` column is likewise interpreted as UTC. The host process timezone
never affects `"date"` values — a `Viewer` renders them in UTC, recovering the
stored calendar day exactly. Datetime expression functions such as
`bucket("x", 'D')`, `day_of_week("x")` and `hour_of_day("x")` also compute in
UTC.
# DataFrame and Arrow Compatibility
`perspective-python` accepts a `Table` constructor argument from any of the
common Python columnar data libraries. In all three cases, `perspective.table`
(and `Table.update()`) consume the input directly — there is no need to
serialize to Apache Arrow IPC bytes yourself. However, note is
still the most efficient way to bulk load data into `Table`.
## PyArrow
```python
import pyarrow as pa
import perspective
arrow_table = pa.table({
"int": pa.array([1, 2, 3], type=pa.int64()),
"float": pa.array([1.5, 2.5, 3.5], type=pa.float64()),
"string": pa.array(["a", "b", "c"], type=pa.string()),
})
table = perspective.table(arrow_table)
```
The same applies to `Table.update()`:
```python
table.update(arrow_table)
```
If you have Arrow data already in IPC format (e.g. read from disk, received
over the wire, or produced by another tool), pass the raw `bytes` directly —
both stream and file formats are auto-detected:
```python
with open("data.arrow", "rb") as f:
table = perspective.table(f.read())
```
### Nested columns
Perspective's data model is flat, so Arrow `struct` and `list` columns are
normalized on ingest.
A `struct` column is hoisted into one dotted column per leaf, recursively. A
null parent nulls every descendant leaf:
```python
arrow_table = pa.table({
"id": pa.array([1, 2], type=pa.int64()),
"s": pa.array([{"a": 10}, {"a": 20}], type=pa.struct([("a", pa.int64())])),
})
# Schema is `{"id": "integer", "s.a": "integer"}`
table = perspective.table(arrow_table)
```
Because the flattened names are ordinary columns, a `Table` created from an
explicit schema accepts nested updates with no further configuration:
```python
table = perspective.table({"id": "integer", "s.a": "integer"})
table.update(arrow_table)
```
A `list` column is controlled by the `list_flatten` argument:
- `"zip"` (default) expands a row into one row per list element, repeating
its non-list siblings. An empty or null list yields a single row with a
null in that column, rather than dropping the row. When a row has more than
one list column, their non-empty lengths must match.
- `"cartesian"` expands a row into the product of its list columns' lengths,
with an empty or null list counting as a single null element.
- `"stringify"` encodes each list as a JSON array in a single string column,
leaving the row count unchanged.
```python
arrow_table = pa.table({
"x": pa.array([1, 2], type=pa.int64()),
"y": pa.array([[10, 20], [30]], type=pa.list_(pa.int64())),
})
# `{"x": [1, 1, 2], "y": [10, 20, 30]}`
perspective.table(arrow_table)
# `{"x": [1, 2], "y": ["[10,20]", "[30]"]}`
perspective.table(arrow_table, list_flatten="stringify")
```
## Polars
```python
import polars as pl
import perspective
df = pl.DataFrame({
"a": [1, 2, 3, 4, 5],
"b": ["x", "y", "z", "x", "y"],
})
table = perspective.table(df)
```
Internally, the `DataFrame` is converted to a `pyarrow.Table` before
ingestion, so Polars columns inherit the Arrow type mapping above.
See also Perspective [Virtual Server support for `polars.DataFrame`](./virtual_server/polars.md)
## Pandas
`pandas.DataFrame` is supported via `pyarrow.Table.from_pandas`, which
dictates behavior including type support — see the
[pyarrow pandas docs](https://arrow.apache.org/docs/python/pandas.html) for
details on which pandas dtypes round-trip cleanly.
```python
from datetime import date, datetime
import numpy as np
import pandas as pd
import perspective
data = pd.DataFrame({
"int": np.arange(100),
"float": [i * 1.5 for i in range(100)],
"bool": [True for i in range(100)],
"date": [date.today() for i in range(100)],
"datetime": [datetime.now() for i in range(100)],
"string": [str(i) for i in range(100)],
})
table = perspective.table(data, index="float")
```
# Callbacks and Events
`perspective.Table` allows for `on_update` and `on_delete` callbacks to be
set—simply call `on_update` or `on_delete` with a reference to a function or a
lambda without any parameters:
```python
def update_callback():
print("Updated!")
# set the update callback
on_update_id = view.on_update(update_callback)
def delete_callback():
print("Deleted!")
# set the delete callback
on_delete_id = view.on_delete(delete_callback)
# set a lambda as a callback
view.on_delete(lambda: print("Deleted x2!"))
```
If the callback is a named reference to a function, it can be removed with
`remove_update` or `remove_delete`:
```python
view.remove_update(on_update_id)
view.remove_delete(on_delete_id)
```
Callbacks defined with a lambda function cannot be removed, as lambda functions
have no identifier.
`on_remove` fires when rows are removed from a `View`'s `Table` with an `index`,
and receives the port ID and the removed index values as an Apache Arrow
(`bytes`) of one column named after the index:
```python
def remove_callback(port_id, indices):
print("Removed", client.table(indices).view().to_records())
on_remove_id = view.on_remove(remove_callback)
view.remove_remove(on_remove_id)
```
# Multi-threading
Perspective's API is thread-safe, so methods may be called from different
threads without additional consideration for safety/exclusivity/correctness. All
`perspective.Client` and `perspective.Server` API methods release the GIL, which
can be exploited for parallelism.
Interally, `perspective.Server` also dispatches to a thread pool for some
operations, enabling better parallelism and overall better query performance.
This independent threadpool size can be controlled via
`perspective.set_num_cpus()`, or the `OMP_NUM_THREADS` environment variable.
```python
import perspective
perspective.set_num_cpus(2)
```
## Server handlers
Perspective's server handler implementations each take an optional `executor`
constructor argument, which (when provided) will configure the handler to
process WebSocket `Client` requests on a thread pool.
```python
from concurrent.futures import ThreadPoolExecutor
from tornado.web import Application
from perspective.handlers.tornado import PerspectiveTornadoHandler
from perspective import Server
args = {"perspective_server": Server(), "executor": ThreadPoolExecutor()}
app = Application(
[
(r"/websocket", PerspectiveTornadoHandler, args),
# ...
]
)
```
## `on_poll_request`
`on_poll_request` is an optional keyword argument for `Server()`, which which
can be applied in cases where overlapping `Table.update` calls can be safely
deferred.
When providing a callback function to `on_poll_request`, the `Server` will
invoke your callback when there are updates that need to be flushed, after which
you must _eventually_ call `Server.poll` (or else no updates will be processed).
The exact implementation of `on_poll_request` will depend on the context. A
simple example which batches calls via `threading.Lock`:
```python
lock = threading.Lock()
def on_poll_request(perspective_server):
if lock.acquire(blocking=False):
try:
perspective_server.poll()
finally:
lock.release()
server = Server(on_poll_request=on_poll_request)
```
# Hosting a WebSocket server
An in-memory `Server` "hosts" all `perspective.Table` and `perspective.View`
instances created by its connected `Client`s. Hosted tables/views can have their
methods called from other sources than the Python server, i.e. by a
`perspective-viewer` running in a JavaScript client over the network,
interfacing with `perspective-python` through the websocket API.
The server has full control of all hosted `Table` and `View` instances, and can
call any public API method on hosted instances. This makes it extremely easy to
stream data to a hosted `Table` using `.update()`:
```python
server = perspective.Server()
client = server.new_local_client()
table = client.table(data, name="data_source")
for i in range(10):
# updates continue to propagate automatically
table.update(new_data)
```
The `name` provided is important, as it enables Perspective in JavaScript to
look up a `Table` and get a handle to it over the network. Otherwise, `name`
will be assigned randomly and the `Client` must look this up with
`Client.get_hosted_table_names()`
## Client/Server Replicated Mode
Using Tornado and
[`PerspectiveTornadoHandler`](../../explanation/python.md#whats-included), as well as
`Perspective`'s JavaScript library, we can set up "distributed" Perspective
instances that allows multiple browser `perspective-viewer` clients to read from
a common `perspective-python` server, as in the
[Tornado Example Project](https://github.com/perspective-dev/perspective/tree/master/examples/python-tornado).
This architecture works by maintaining two `Tables`—one on the server, and one
on the client that mirrors the server's `Table` automatically using `on_update`.
All updates to the table on the server are automatically applied to each client,
which makes this architecture a natural fit for streaming dashboards and other
distributed use-cases. In conjunction with [multithreading](#multi-threading),
distributed Perspective offers consistently high performance over large numbers
of clients and large datasets.
_*server.py*_
```python
from perspective import Server
from perspective.handlers.tornado import PerspectiveTornadoHandler
# Create an instance of Server, and host a Table
SERVER = Server()
CLIENT = SERVER.new_local_client()
# The Table is exposed at `localhost:8888/websocket` with the name `data_source`
client.table(data, name = "data_source")
app = tornado.web.Application([
# create a websocket endpoint that the client JavaScript can access
(r"/websocket", PerspectiveTornadoHandler, {"perspective_server": SERVER})
])
# Start the Tornado server
app.listen(8888)
loop = tornado.ioloop.IOLoop.current()
loop.start()
```
Instead of calling `load(server_table)`, create a `View` using `server_table`
and pass that into `viewer.load()`. This will automatically register an
`on_update` callback that synchronizes state between the server and the client.
_*index.html*_
```html
```
For a more complex example that offers distributed editing of the server
dataset, see
[client_server_editing.html](https://github.com/perspective-dev/perspective/blob/master/examples/python-tornado/client_server_editing.html).
We also provide examples for Starlette/FastAPI and AIOHTTP:
- [Starlette Example Project](https://github.com/perspective-dev/perspective/tree/master/examples/python-starlette).
- [AIOHTTP Example Project](https://github.com/perspective-dev/perspective/tree/master/examples/python-aiohttp).
## Server-only Mode
The server setup is identical to
[Client/Server Replicated Mode](#client-server-replicated-mode) above, but
instead of creating a `View`, the client calls `load(server_table)`: In Python,
use `Server` and `PerspectiveTornadoHandler` to create a websocket server that
exposes a `Table`. In this example, `table` is a proxy for the `Table` we
created on the server. All API methods are available on _proxies_, e.g.
calling `view()`, `schema()`, `update()` on `table` will pass those operations
to the Python `Table`, execute the commands, and return the result back to
Javascript.
```html
```
```javascript
const websocket = perspective.websocket("ws://localhost:8888/websocket");
const table = websocket.open_table("data_source");
document.getElementById("viewer").load(table);
```
# Joining Tables
`perspective.join()` creates a read-only `Table` by joining two source tables on
a shared key column. The result is reactive — it updates automatically when
either source table changes. See [`Join`](../../explanation/join.md) for
conceptual details.
## Basic Inner Join
```python
orders = perspective.table([
{"id": 1, "product_id": 101, "qty": 5},
{"id": 2, "product_id": 102, "qty": 3},
{"id": 3, "product_id": 101, "qty": 7},
])
products = perspective.table([
{"product_id": 101, "name": "Widget"},
{"product_id": 102, "name": "Gadget"},
])
joined = perspective.join(orders, products, "product_id")
view = joined.view()
json = view.to_json()
```
## Join Types
Pass `join_type` to select inner, left, or outer join behavior:
```python
# Left join: all left rows, nulls for unmatched right columns
left_joined = perspective.join(left, right, "id", join_type="left")
# Outer join: all rows from both tables
outer_joined = perspective.join(left, right, "id", join_type="outer")
```
## Reactive Updates
The joined table recomputes automatically when either source table is updated:
```python
left = perspective.table([{"id": 1, "x": 10}])
right = perspective.table([{"id": 2, "y": "b"}])
joined = perspective.join(left, right, "id")
view = joined.view()
json = view.to_json()
# [] — no matching keys yet
right.update([{"id": 1, "y": "a"}])
json = view.to_json()
# [{"id": 1, "x": 10, "y": "a"}] — new match detected
```
## Async Client
The async client has the same API:
```python
joined = await client.join(orders, products, "product_id", join_type="left")
```
# `PerspectiveWidget` for notebooks
Building on top of the API provided by `perspective.Table`, the
`PerspectiveWidget` offers the entire functionality of Perspective within a
notebook environment. It supports the same API semantics of
``, along with the additional data types supported by
`perspective.Table`.
## Installation
`PerspectiveWidget` is an [AnyWidget](https://anywidget.dev), shipped as a
prebuilt bundle inside the `perspective-python` wheel. There is no separate
labextension to install or version-match — install the `jupyter` extra, which
adds the `anywidget` dependency:
```bash
pip install "perspective-python[jupyter]"
```
The same wheel works in JupyterLab, classic Jupyter Notebook, VSCode
notebooks, Google Colab and Marimo.
The @perspective-dev/jupyterlab package is
now optional and no longer ships the widget. It provides only the
"Open With → Perspective" file renderers for csv,
json and arrow files in JupyterLab.
## Usage
`PerspectiveWidget` takes keyword arguments for the managed `View`:
```python
from perspective.widget import PerspectiveWidget
w = perspective.PerspectiveWidget(
data,
plugin="X Bar",
aggregates={"datetime": "any"},
sort=[["date", "desc"]]
)
```
## Creating a widget
A widget is created through the `PerspectiveWidget` constructor, which takes as
its first, required parameter a `perspective.Table`, a dataset, a schema, or
`None`, which serves as a special value that tells the Widget to defer loading
any data until later. In maintaining consistency with the Javascript API,
Widgets cannot be created with empty dictionaries or lists — `None` should be
used if the intention is to await data for loading later on. A widget can be
constructed from a dataset:
```python
from perspective.widget import PerspectiveWidget
PerspectiveWidget(data, group_by=["date"])
```
.. or a schema:
```python
PerspectiveWidget({"a": int, "b": str})
```
.. or an instance of a `perspective.Table`:
```python
table = perspective.table(data)
PerspectiveWidget(table)
```
## Updating a widget
`PerspectiveWidget` shares a similar API to the `` Custom
Element, and has similar `save()` and `restore()` methods that
serialize/deserialize UI state for the widget.
## `PerspectiveRenderer`
The optional `@perspective-dev/jupyterlab` package exposes a JS-only
`mimerender-extension`. This lets you view `csv`, `json`, and `arrow` files
directly from the JupyterLab file browser — right-click one of these files and
choose `Open With → Perspective`.
```bash
jupyter labextension install @perspective-dev/jupyterlab
```
This package is independent of `PerspectiveWidget`; install it only if you
want the file renderers.
# Virtual Servers
Perspective's Virtual Server feature lets you connect `` to
external data sources without loading data into Perspective's built-in engine.
Instead, queries are translated and executed natively by the external database.
For a detailed explanation of how virtual servers work, see the
[Virtual Servers](../../explanation/virtual_servers.md) concepts page.
Perspective ships with built-in virtual server implementations for:
- [**DuckDB**](./virtual_server/duckdb.md) — query DuckDB databases using the
`duckdb` Python package.
- [**ClickHouse**](./virtual_server/clickhouse.md) — query a ClickHouse server
using the `clickhouse-connect` Python package.
- [**Polars**](./virtual_server/polars.md) — query in-memory Polars DataFrames
using the `polars` Python package.
- [**PostgreSQL**](./virtual_server/postgres.md) — query a PostgreSQL server
(16 or later) using the `psycopg` Python package.
You can also [**implement your own**](./virtual_server/custom.md) virtual server
to connect Perspective to any data source by subclassing `VirtualServerHandler`.
# DuckDB Virtual Server
Perspective provides a built-in virtual server for
[DuckDB](https://duckdb.org/), allowing `` clients to query
a server-side DuckDB database over WebSocket.
For browser-only usage via DuckDB-WASM, see the
[JavaScript DuckDB guide](../../javascript/virtual_server/duckdb.md).
## Installation
```bash
pip install perspective-python duckdb
```
## Usage
Create a server that exposes a DuckDB database to browser clients:
```python
import duckdb
import tornado.web
import tornado.ioloop
from perspective.virtual_servers.duckdb import DuckDBVirtualServer
from perspective.handlers.tornado import PerspectiveTornadoHandler
# Create DuckDB connection and load data
conn = duckdb.connect()
conn.execute("CREATE TABLE my_table AS SELECT * FROM 'data.parquet'")
# Create virtual server backed by DuckDB
server = DuckDBVirtualServer(conn)
# Serve over WebSocket
app = tornado.web.Application([
(r"/websocket", PerspectiveTornadoHandler, {"perspective_server": server}),
])
app.listen(8080)
tornado.ioloop.IOLoop.current().start()
```
Connect from the browser:
```javascript
const websocket = await perspective.websocket("ws://localhost:8080/websocket");
const table = await websocket.open_table("my_table");
document.getElementById("viewer").load(table);
```
## Window functions
Window columns are DuckDB's own functions, under their DuckDB names — the
advertised name is emitted into the `OVER` clause verbatim.
| | |
| --- | --- |
| Aggregating | `sum` `avg` `count` `min` `max` `product` `median` |
| Deviation / variance | `stddev_samp` `stddev_pop` `var_samp` `var_pop` |
| Navigation | `first_value` `last_value` `nth_value` `lag` `lead` |
| Ranking | `row_number` `rank` `dense_rank` `percent_rank` `cume_dist` `ntile` |
| Perspective's own | `diff` `rate` |
## Examples
- [Python DuckDB example](https://github.com/perspective-dev/perspective/tree/master/examples/python-duckdb-virtual)
# ClickHouse Virtual Server
Perspective provides a built-in virtual server for
[ClickHouse](https://clickhouse.com/), allowing `` clients
to query a ClickHouse server over WebSocket.
For browser-only usage, see the
[JavaScript ClickHouse guide](../../javascript/virtual_server/clickhouse.md).
## Installation
```bash
pip install perspective-python clickhouse-connect
```
## Usage
Create a server that exposes ClickHouse tables to browser clients:
```python
import clickhouse_connect
import tornado.web
import tornado.ioloop
from perspective.virtual_servers.clickhouse import ClickhouseVirtualServer
from perspective.handlers.tornado import PerspectiveTornadoHandler
# Connect to ClickHouse
client = clickhouse_connect.get_client(host="localhost")
# Create virtual server backed by ClickHouse
server = ClickhouseVirtualServer(client)
# Serve over WebSocket
app = tornado.web.Application([
(r"/websocket", PerspectiveTornadoHandler, {"perspective_server": server}),
])
app.listen(8080)
tornado.ioloop.IOLoop.current().start()
```
Connect from the browser:
```javascript
const websocket = await perspective.websocket("ws://localhost:8080/websocket");
const table = await websocket.open_table("my_table");
document.getElementById("viewer").load(table);
```
## Examples
- [Python ClickHouse example](https://github.com/perspective-dev/perspective/tree/master/examples/python-clickhouse-virtual)
# Polars Virtual Server
Perspective provides a built-in virtual server for
[Polars](https://pola.rs/), allowing `` clients to query
in-memory Polars DataFrames over WebSocket.
## Installation
```bash
pip install perspective-python polars
```
## Usage
Create a server that exposes Polars DataFrames to browser clients:
```python
import polars as pl
import tornado.web
import tornado.ioloop
from perspective.virtual_servers.polars import PolarsVirtualServer
from perspective.handlers.tornado import PerspectiveTornadoHandler
# Load data into Polars DataFrames
df = pl.read_parquet("data.parquet")
# Create virtual server backed by Polars (dict of name -> DataFrame)
server = PolarsVirtualServer({"my_table": df})
# Serve over WebSocket
app = tornado.web.Application([
(r"/websocket", PerspectiveTornadoHandler, {"perspective_server": server}),
])
app.listen(8080)
tornado.ioloop.IOLoop.current().start()
```
Connect from the browser:
```javascript
const websocket = await perspective.websocket("ws://localhost:8080/websocket");
const table = await websocket.open_table("my_table");
document.getElementById("viewer").load(table);
```
## Examples
- [Python Polars example](https://github.com/perspective-dev/perspective/tree/master/examples/python-polars-virtual)
# PostgreSQL Virtual Server
Perspective provides a built-in virtual server for
[PostgreSQL](https://www.postgresql.org/), allowing ``
clients to query a PostgreSQL server over WebSocket.
Requires PostgreSQL 16 or later.
## Installation
```bash
pip install perspective-python "psycopg[binary]"
```
## Usage
Create a server that exposes PostgreSQL tables to browser clients:
```python
import tornado.web
import tornado.ioloop
from perspective.virtual_servers.postgres import PostgresVirtualServer
from perspective.handlers.tornado import PerspectiveTornadoHandler
# Create virtual server backed by PostgreSQL. Each browser session opens its
# own connection with this DSN.
server = PostgresVirtualServer("postgresql://user@localhost:5432/mydb")
# Serve over WebSocket
app = tornado.web.Application([
(r"/websocket", PerspectiveTornadoHandler, {"perspective_server": server}),
])
app.listen(8080)
tornado.ioloop.IOLoop.current().start()
```
Connect from the browser (table names are schema-qualified):
```javascript
const websocket = await perspective.websocket("ws://localhost:8080/websocket");
const table = await websocket.open_table("public.my_table");
document.getElementById("viewer").load(table);
```
The server is read-only with respect to your data: each viewer session
materializes its queries as connection-scoped `TEMPORARY VIEW`s, which
PostgreSQL drops automatically when the session disconnects.
## Aggregates
Aggregates are PostgreSQL's own functions, under their PostgreSQL names, and
each column type advertises only the aggregates PostgreSQL defines for it —
for example `bit_and`/`bit_or`/`bit_xor` on integers only, and
`bool_and`/`bool_or`/`every` (rather than `min`/`max`) on booleans.
`any_value` is the default for columns with no explicit aggregate, which is
why PostgreSQL 16 is required.
## Window functions
Window columns are PostgreSQL's own functions, under their PostgreSQL names —
the advertised name is emitted into the `OVER` clause verbatim.
| | |
| -------------------- | ------------------------------------------------------------------- |
| Aggregating | `sum` `avg` `count` `min` `max` |
| Deviation / variance | `stddev_samp` `stddev_pop` `var_samp` `var_pop` |
| Navigation | `first_value` `last_value` `nth_value` `lag` `lead` |
| Ranking | `row_number` `rank` `dense_rank` `percent_rank` `cume_dist` `ntile` |
| Perspective's own | `diff` |
`range` frames require a numeric order key in PostgreSQL, so they are
advertised for numeric column types only.
## Limitations
- **Split by** is not supported — PostgreSQL has no `PIVOT` statement.
- Natural-order (unsorted) window functions are not supported, since
PostgreSQL has no stable row identity; window columns require an explicit
order key.
## Examples
- [Python PostgreSQL example](https://github.com/perspective-dev/perspective/tree/master/examples/python-postgres-virtual)
# Implementing a custom Virtual Server
You can connect Perspective to any data source by subclassing
`VirtualServerHandler`, wrapping it in a `VirtualServer`, and exposing that
via a small _session factory_ object which the WebSocket handlers use to give
each connected client its own session.
For background on virtual servers, see the
[Virtual Servers overview](../../../explanation/virtual_servers.md).
## The handler
`VirtualServerHandler` is imported from `perspective.virtual_servers`. Only
`get_hosted_tables`, `table_schema`, `table_size`, `table_make_view`,
`view_delete` and `view_get_data` are required; the rest have defaults.
```python
from perspective.virtual_servers import VirtualServerHandler
class MyHandler(VirtualServerHandler):
def __init__(self, db):
self.db = db
def get_features(self):
return {
"group_by": True,
"split_by": False,
"sort": True,
"filter_ops": {
"string": ["==", "!=", "contains"],
"float": ["==", "!=", ">", "<"],
},
"aggregates": {
"float": ["sum", "avg", "count"],
"string": ["count"],
},
}
def get_hosted_tables(self):
return ["my_table"]
def table_schema(self, table_name):
return {"name": "string", "price": "float"}
def table_size(self, table_name):
return 1000
def table_make_view(self, table_name, view_name, config):
# Translate `config` (group_by, sort, filter, etc.) into a query
# against your data source. Store the query keyed by `view_name`
# for later data retrieval.
pass
def table_describe(self, table_name, config):
# Validate `config` against `table_name` and report the schema the
# temporary table would have, WITHOUT creating it. Return
# {"expression_schema", "view_schema"} when valid,
# {"expression_schema", "expression_errors"} when an expression is
# invalid, or {"config_error"} otherwise. A model with no cheaper
# answer may `return describe_via_make_view(self, table_name, config)`,
# which builds and drops a real temporary table - at that cost.
pass
def view_delete(self, view_name):
# Clean up resources for this view. The UI does this automatically,
# and can recover if a view dies early.
pass
def view_get_data(self, view_name, config, viewport, data):
# Serialize the rectangular slice `viewport` of the temporary table
# `view_name` into `data`, a push-only `VirtualDataSlice`. Once a
# type has been pushed for a column name it must not change.
pass
```
### Optional methods
| Method | Default | Purpose |
| --- | --- | --- |
| `get_features()` | `columns` only | Which UI controls to enable — see [Features declaration](../../../explanation/virtual_servers.md#features-declaration) |
| `view_schema(view_name, config)` | `table_schema` | Schema of a temporary table, when it differs from its source |
| `view_size(view_name)` | `table_size` | Row count of a temporary table, when it differs from its source |
| `view_get_min_max(view_name, column_name, config)` | unsupported | Column bounds as a `(min, max)` tuple — required for gradient and sparkbar column styles |
## The session factory
The WebSocket handlers call `new_session(callback)` once per connection, so
the object passed as `perspective_server` must provide it. Wrap your handler
in a `perspective.VirtualServer` — which owns the protocol — and return one
session per client:
```python
import perspective
class MyVirtualSession:
def __init__(self, callback, db):
self.session = perspective.VirtualServer(MyHandler(db))
self.callback = callback
def handle_request(self, msg):
self.callback(self.session.handle_request(msg))
class MyVirtualServer:
def __init__(self, db):
self.db = db
def new_session(self, callback):
return MyVirtualSession(callback, self.db)
```
## Serving it
A `MyVirtualServer` instance can then be passed to a Tornado, Starlette or
AIOHTTP handler just like a regular `Server`:
```python
from perspective.handlers.tornado import PerspectiveTornadoHandler
app = tornado.web.Application([
(r"/websocket", PerspectiveTornadoHandler, {
"perspective_server": MyVirtualServer(db),
}),
])
```
The built-in [DuckDB](./duckdb.md), [ClickHouse](./clickhouse.md),
[PostgreSQL](./postgres.md) and [Polars](./polars.md) implementations all
follow exactly this shape and are worth reading as complete references.
# Rust
Install via `cargo`:
```bash
cargo add perspective
```
# Example
Initialize a server and client
```rust
let server = Server::default();
let client = server.new_local_client();
```
Load an Arrow
```rust
let mut file = File::open(std::path::Path::new(ROOT_PATH).join(ARROW_FILE_PATH))?;
let mut feather = Vec::with_capacity(file.metadata()?.len() as usize);
file.read_to_end(&mut feather)?;
let data = UpdateData::Arrow(feather.into());
let mut options = TableInitOptions::default();
options.set_name("my_data_source");
client.table(data.into(), options).await?;
```
# Joining Tables
`Client::join` creates a read-only `Table` by joining two source tables on a
shared key column. The result is reactive — it updates automatically when
either source table changes. See [`Join`](../explanation/join.md) for
conceptual details.
```rust
let orders = client.table(
TableData::Update(UpdateData::JsonRows(
"[{\"id\":1,\"product_id\":101,\"qty\":5},{\"id\":2,\"product_id\":102,\"qty\":3}]".into(),
)),
TableInitOptions::default(),
).await?;
let products = client.table(
TableData::Update(UpdateData::JsonRows(
"[{\"product_id\":101,\"name\":\"Widget\"},{\"product_id\":102,\"name\":\"Gadget\"}]".into(),
)),
TableInitOptions::default(),
).await?;
let joined = client.join(
(&orders).into(),
(&products).into(),
"product_id",
JoinOptions::default(),
).await?;
let view = joined.view(None).await?;
let json = view.to_json().await?;
```
Use `JoinOptions` to configure the join type, table name, or `right_on` column:
```rust
let options = JoinOptions {
join_type: Some(JoinType::Left),
name: Some("orders_with_products".into()),
right_on: None,
};
let joined = client.join(
(&orders).into(),
(&products).into(),
"product_id",
options,
).await?;
```
You can also join by table name strings instead of `Table` references:
```rust
let joined = client.join(
"orders".into(),
"products".into(),
"product_id",
JoinOptions::default(),
).await?;
```
# Perspective with FastAPI and Starlette
`perspective-python` includes a WebSocket handler for
[Starlette](https://www.starlette.io/), and therefore
[FastAPI](https://fastapi.tiangolo.com/). Add one WebSocket route and every
`Table` hosted by your `perspective.Server` is available to
`` clients in the browser.
```bash
pip install "perspective-python[starlette]" fastapi uvicorn
```
```python
import uvicorn
from fastapi import FastAPI, WebSocket
from perspective import Server
from perspective.handlers.starlette import PerspectiveStarletteHandler
server = Server()
client = server.new_local_client()
table = client.table(
{"symbol": "string", "price": "float", "time": "datetime"},
index="symbol",
name="prices",
)
app = FastAPI()
async def websocket_handler(websocket: WebSocket):
handler = PerspectiveStarletteHandler(
perspective_server=server,
websocket=websocket,
)
await handler.run()
app.add_api_websocket_route("/websocket", websocket_handler)
if __name__ == "__main__":
uvicorn.run(app, host="0.0.0.0", port=8080)
```
Anything in your application can now write to the table — a REST endpoint, a
background task, a queue consumer:
```python
@app.post("/ticks")
async def ticks(rows: list[dict]):
table.update(rows)
```
In the browser:
```javascript
const websocket = await perspective.websocket("ws://localhost:8080/websocket");
const table = await websocket.open_table("prices");
await document.querySelector("perspective-viewer").load(table);
```
Every connected viewer updates as `table.update()` is called.
- [Hosting a WebSocket server](../how_to/python/websocket.md) — replicated vs
server-only modes.
- [Multithreading](../how_to/python/multithreading.md) — the `executor`
handler argument and `on_poll_request`.
- [Real-time dashboards over WebSocket](../use_cases/real_time_dashboard.md)
- [`python-starlette` example](https://github.com/perspective-dev/perspective/tree/master/examples/python-starlette)
# Perspective with Next.js, Vue, Svelte and Angular
`` is a standard Web Component, so it works in any
framework which can render a DOM element and call a method on it. React has a
[dedicated wrapper](../how_to/javascript/react.md); elsewhere, use the element
directly.
Three rules apply everywhere:
1. **Initialize WebAssembly once**, before first use, as described in
[Importing with or without a bundler](../how_to/javascript/importing.md).
2. **Client-side only.** Perspective needs Web Workers and WebAssembly, so it
cannot be server-rendered.
3. **`load()` is a method, not an attribute.** Get a reference to the element
and call `viewer.load(table)`; use `viewer.restore(config)` for
configuration.
## Next.js
Load the component with `next/dynamic` and `ssr: false`, so Perspective is
only imported in the browser:
```tsx
import dynamic from "next/dynamic";
const Report = dynamic(() => import("../components/Report"), { ssr: false });
```
`components/Report.tsx` then uses
[`@perspective-dev/react`](../how_to/javascript/react.md) as normal.
## Vue
Tell the template compiler that `perspective-viewer` is a custom element:
```javascript
// vite.config.js
vue({
template: {
compilerOptions: {
isCustomElement: (tag) => tag.startsWith("perspective-"),
},
},
});
```
```html
```
## Svelte
```html
```
## Angular
Add `CUSTOM_ELEMENTS_SCHEMA` to the component or module, and reach the element
with `@ViewChild`:
```typescript
@Component({
selector: "app-report",
template: ``,
schemas: [CUSTOM_ELEMENTS_SCHEMA],
})
export class ReportComponent implements AfterViewInit {
@ViewChild("viewer") viewer!: ElementRef;
async ngAfterViewInit() {
await this.viewer.nativeElement.load(table);
}
}
```
## Cleaning up
When the component unmounts, call `viewer.delete()`, and `delete()` any
`View` and `Table` you created, in that order. See
[Cleaning up resources](../how_to/javascript/deleting.md).
# Perspective with Kafka and other message queues
Perspective has no queue-specific connector because it does not need one: a
[`Table`](../explanation/table.md) is updated by calling `update()`, so any
consumer loop is an integration. The pattern is the same for Kafka, Redpanda,
NATS, RabbitMQ, Redis streams or a WebSocket feed.
1. Create a `Table` from a schema, on a `perspective.Server`.
2. Consume messages, batch them, and call `table.update(batch)`.
3. Host the server on a WebSocket so browsers can open the table.
## Python
```python
import json
import threading
import tornado.ioloop
import tornado.web
from confluent_kafka import Consumer
from perspective import Server
from perspective.handlers.tornado import PerspectiveTornadoHandler
server = Server()
client = server.new_local_client()
table = client.table(
{"order_id": "string", "symbol": "string", "qty": "integer", "price": "float", "ts": "datetime"},
index="order_id",
name="orders",
)
def consume():
consumer = Consumer({"bootstrap.servers": "localhost:9092", "group.id": "perspective"})
consumer.subscribe(["orders"])
while True:
messages = consumer.consume(num_messages=500, timeout=0.1)
rows = [json.loads(m.value()) for m in messages if m.error() is None]
if rows:
table.update(rows)
threading.Thread(target=consume, daemon=True).start()
app = tornado.web.Application([
(r"/websocket", PerspectiveTornadoHandler, {"perspective_server": server}),
])
app.listen(8080)
tornado.ioloop.IOLoop.current().start()
```
`confluent_kafka` is used here for illustration; nothing in Perspective depends
on it.
## Design notes
- **Batch.** One `update()` of 500 rows is much cheaper than 500 updates of
one row. Consume in small time windows, as above.
- **Pick `index` or `limit`.** A topic is unbounded; a browser is not. Use
[`index`](../explanation/table/options.md) when messages are upserts to
entities (orders, positions, devices), and `limit` when they are events and
you want the most recent _n_.
- **Threads are fine.** Perspective's Python API is thread-safe and releases
the GIL. See [Multithreading](../how_to/python/multithreading.md).
- **Arrow if you have it.** If your messages are already Arrow record batches,
pass the bytes straight to `update()`.
The browser side is identical to any other server-hosted table; see
[Real-time dashboards over WebSocket](../use_cases/real_time_dashboard.md).
# Tutorial: A tornado server in Python
Perspective ships with a pre-built Tornado handler that makes integration with
`tornado.websockets` extremely easy. This allows you to run an instance of
`Perspective` on a server using Python, open a websocket to a `Table`, and
access the `Table` in JavaScript and through ``. All
instructions sent to the `Table` are processed in Python, which executes the
commands, and returns its output through the websocket back to Javascript.
### Python setup
Make sure Perspective and Tornado are installed!
```bash
pip install perspective-python tornado
```
To use the handler, we need to first have a `Server`, a `Client` and an instance
of a `Table`:
```python
import perspective
SERVER = perspective.Server()
CLIENT = SERVER.new_local_client()
```
Once the server has been created, create a `Table` instance with a name. The
name that you host the table under is important — it acts as a unique accessor
on the JavaScript side, which will look for a Table hosted at the websocket with
the name you specify.
```python
TABLE = client.table(data, name="data_source_one")
```
After the server and table setup is complete, create a websocket endpoint and
provide it a reference to `PerspectiveTornadoHandler`. You must provide the
configuration object in the route tuple, and it must contain
`"perspective_server"`, which is a reference to the `Server` you just created.
```python
from perspective.handlers.tornado import PerspectiveTornadoHandler
app = tornado.web.Application([
# ... other handlers ...
# Create a websocket endpoint that the client JavaScript can access
(r"/websocket", PerspectiveTornadoHandler, {"perspective_server": SERVER, "check_origin": True})
])
```
Optionally, the configuration object can also include `check_origin`, a boolean
that determines whether the websocket accepts requests from origins other than
where the server is hosted. See
[Tornado docs](https://www.tornadoweb.org/en/stable/websocket.html#tornado.websocket.WebSocketHandler.check_origin)
for more details.
### JavaScript setup
Once the server is up and running, you can access the Table you just hosted
using `perspective.websocket` and `open_table()`. First, create a client that
expects a Perspective server to accept connections at the specified URL:
```javascript
import "@perspective-dev/viewer";
import "@perspective-dev/viewer-datagrid";
import perspective from "@perspective-dev/client";
const websocket = await perspective.websocket("ws://localhost:8888/websocket");
```
Next open the `Table` we created on the server by name:
```javascript
const table = await websocket.open_table("data_source_one");
```
`table` is a proxy for the `Table` we created on the server. All operations that
are possible through the JavaScript API are possible on the Python API as well,
thus calling `view()`, `schema()`, `update()` etc. on `const table` will pass
those operations to the Python `Table`, execute the commands, and return the
result back to JavaScript. Similarly, providing this `table` to a
`` instance will allow virtual rendering:
```javascript
const viewer = document.createElement("perspective-viewer");
viewer.style.height = "500px";
document.body.appendChild(viewer);
await viewer.load(table);
```
`perspective.websocket` expects a Websocket URL where it will send instructions.
When `open_table` is called, the name to a hosted Table is passed through, and a
request is sent through the socket to fetch the Table. No actual `Table`
instance is passed inbetween the runtimes; all instructions are proxied through
websockets.
This provides for great flexibility — while `Perspective.js` is full of
features, browser WebAssembly runtimes currently have some performance
restrictions on memory and CPU feature utilization, and the architecture in
general suffers when the dataset itself is too large to download to the client
in full.
The Python runtime does not suffer from memory limitations, utilizes Apache
Arrow internal threadpools for threading and parallel processing, and generates
architecture optimized code, which currently makes it more suitable as a
server-side runtime than `node.js`.
# API Reference
Perspective's complete API is hosted on `docs.rs`:
- Python API
- [`perspective`](https://perspective-dev.github.io/python/index.html)
- [`perspective.widget`](https://perspective-dev.github.io/python/perspective/widget.html)
- [`perspective.handlers.aiohttp`](https://perspective-dev.github.io/python/perspective/handlers/aiohttp.html)
- [`perspective.handlers.starlette`](https://perspective-dev.github.io/python/perspective/handlers/starlette.html)
- [`perspective.handlers.tornado`](https://perspective-dev.github.io/python/perspective/handlers/tornado.html)
- JavaScript API
- [`@perspective-dev/client` Browser](https://perspective-dev.github.io/browser/modules/src_ts_perspective.browser.ts.html)
- [`@perspective-dev/client` Node.js](https://perspective-dev.github.io/node/modules/src_ts_perspective.node.ts.html)
- [`@perspective-dev/viewer`](https://perspective-dev.github.io/viewer/modules/perspective-viewer.html)
- [`@perspective-dev/react`](https://perspective-dev.github.io/react/index.html)
- Rust API
- [`perspective`](https://docs.rs/perspective/latest/perspective/)
- [`perspective-client`](https://docs.rs/perspective-client/latest/perspective_client/)
- [`perspective-server`](https://docs.rs/perspective-server/latest/perspective_server/)
- [`perspective-python`](https://docs.rs/perspective-python/latest/perspective_python/)
- [`perspective-js`](https://docs.rs/perspective-js/latest/perspective_js/)
- [`perspective-viewer`](https://docs.rs/perspective-viewer/latest/perspective_viewer/)
# Benchmarks
Perspective's performance is tracked by a benchmark suite which lives in the
repository. CI runs it on every tagged release and attaches the raw results to
that
[GitHub release](https://github.com/perspective-dev/perspective/releases/latest)
as Apache Arrow files — and the results are, naturally, explored in
Perspective.
## Results
Open the published results for the latest release, live in your browser:
| Build | Dashboard | By release | Raw data |
| --- | --- | --- | --- |
| JavaScript (WebAssembly engine under Node.js) | [Benchmarks — JavaScript](https://perspective-dev.github.io/gallery/benchmarks-js.html) | [History](https://perspective-dev.github.io/gallery/benchmarks-js-history.html) | [`benchmark-js.arrow`](https://github.com/perspective-dev/perspective/releases/latest/download/benchmark-js.arrow) |
| Python (native engine, driven over WebSocket) | [Benchmarks — Python](https://perspective-dev.github.io/gallery/benchmarks-python.html) | [History](https://perspective-dev.github.io/gallery/benchmarks-python-history.html) | [`benchmark-python.arrow`](https://github.com/perspective-dev/perspective/releases/latest/download/benchmark-python.arrow) |
Each file holds one row per timed iteration:
| Column | Meaning |
| --- | --- |
| `benchmark` | The case, e.g. `.view({group_by})` |
| `version` | The Perspective release the case ran against |
| `version_idx` | Release order; `0` is the build being released |
| `real_time`, `cpu_time`, `user_time`, `system_time` | Microseconds |
| `outlier` | `true` when the iteration falls outside 1.5 × the interquartile range of its case |
The dashboards are ordinary Perspective views over that file — mean
`real_time` in milliseconds, excluding outliers, grouped by `benchmark` and
`version` — so they can be re-pivoted, filtered to one case, or switched to
another chart type in place.
**Environment.** Results are produced by GitHub Actions on an `ubuntu-22.04`
x86_64 hosted runner with Node.js 22 and Python 3.11, over the Superstore
sample dataset. Hosted runners are shared, modest machines: read these numbers
as a release-over-release trend on constant hardware, not as the ceiling for
your own.
## What is measured
The cross-platform suite (`tools/bench/cross_platform_suite.mjs`) defines the
cases, which are written against the `Client` API and so can be pointed at any
build of the engine:
| Area | Cases |
| --- | --- |
| Table construction | `table(arrow)`, `table(csv)`, `table(json)`, `table(columns)`, and `table(arrow, {limit})` |
| Streaming | `table.update(arrow)`, and `table.update(arrow)` with window columns active |
| Queries | `view()`, `view({group_by})`, `view({group_by, aggregates: "median"})`, `view({expressions})`, `view({windows})` |
| Joins | `join()` |
| Serialization | `to_arrow()`, `to_csv()`, `to_columns()`, `to_json()` |
A separate suite (`charts_suite.mjs`) measures chart rendering in a real
browser.
Each case is run repeatedly against the current build _and_ against
previously published releases, so every number can be read relative to the
versions before it. Results are written as Apache Arrow files under
`tools/bench/dist/`.
## Running it
From a built checkout of the
[repository](https://github.com/perspective-dev/perspective):
```bash
cd tools/bench
pnpm run bench_js
pnpm run bench_python
pnpm run bench_charts
```
## Reading results sensibly
- **Arrow is the fast path.** Loading Arrow avoids parsing and type inference;
CSV and JSON construction times measure the parser as much as the engine.
- **Column types matter more than row count.** Numeric and datetime columns
are fixed-width; string columns are dictionary-encoded and cost more to
build and to group.
- **Updates scale with the delta.** The cost of `update()` on a table with
active views is driven by the size of the update and the number of groups it
touches, not by the size of the table.
- **WebAssembly is single-threaded per worker; native is not.** The Python,
Node.js and Rust builds use a thread pool (`perspective.set_num_cpus()`), so
native numbers are typically better than in-browser numbers for the same
case.
- **Rendering is separate from querying.** The data grid draws only the cells
in view, so a grid over ten million rows renders in the same time as one over
ten thousand; the query is what scales.
For guidance on sizing, see
[Visualizing millions of rows in the browser](./use_cases/large_datasets.md).
# Glossary
Short definitions of the terms used throughout this guide.
## Data grid
A scrollable, sortable table of rows and columns rendered in a user interface.
Perspective's data grid, the
[Datagrid plugin](https://www.npmjs.com/package/@perspective-dev/viewer-datagrid),
is _virtualized_: it renders only the visible cells, so its cost is independent
of the number of rows.
## Pivot table
A table which groups rows by one or more columns (row pivots), optionally
splits them across the distinct values of other columns (column pivots), and
shows an aggregate in each cell. In Perspective, row pivots are `group_by` and
column pivots are `split_by`. See
[Grouping and Pivots](./explanation/view/config/grouping_and_pivots.md).
## Streaming pivot table
A pivot table whose groups and aggregates are updated incrementally as rows
are inserted, updated or removed, rather than recomputed. See
[Streaming pivot tables](./use_cases/streaming_pivot_table.md).
## `Table`
Perspective's columnar, typed data store. A [`Table`](./explanation/table.md)
is created from a schema or a dataset (Apache Arrow, CSV, JSON, or a DataFrame
in Python), and modified with `update()`, `remove()`, `clear()` and
`replace()`.
## `View`
A continuous query over a `Table`: a combination of `group_by`, `split_by`,
`columns`, `aggregates`, `filter`, `sort` and `expressions`. A
[`View`](./explanation/view.md) stays current as its `Table` changes and
notifies `on_update` subscribers.
## `group_by`
The columns whose distinct values become the rows of a pivot. Multiple levels
form an expandable tree with subtotals.
## `split_by`
The columns whose distinct values become the column headers of a pivot.
## Aggregate
The function which reduces a group's values to one cell — `sum`, `avg`,
`count`, `distinct count`, `median`, `weighted mean`, `first`, `last` and
others. Chosen per column.
## Expression column
A computed column defined in Perspective's
[expression language](./explanation/view/config/expressions.md), based on
ExprTK. Expressions are evaluated column-wise inside the engine and can be
grouped, filtered, sorted and aggregated like stored columns.
## Index
A `Table` option naming a primary key column. Updates to an indexed table
replace the row with the matching key (and may be partial); without an index,
updates append. See [`index` and `limit`](./explanation/table/options.md).
## Limit
A `Table` option which keeps only the most recent _n_ rows, for rolling
windows over unbounded streams.
## `page_to_disk`
A `Table` option which backs the table's columns with on-disk storage — the
Origin Private File System in the browser, memory-mapped files natively — so
a `Table` can exceed the engine's memory. See
[`page_to_disk`](./explanation/table/options.md#page_to_disk).
## ``
The Web Component (Custom Element) providing Perspective's user interface:
configuration panel, plugins, themes, multi-panel layout and save/restore.
## Plugin
A visualization hosted by `` — the Datagrid, or one of
the WebGL charts (bar, line, area, scatter, heatmap, treemap, sunburst,
candlestick, OHLC, maps).
## Client, Server
A `Server` owns tables and executes queries. A `Client` is a handle to a
`Server`, whether that server is in the same process, in a Web Worker, or
across a WebSocket. The API is the same in every case.
## Client-only mode
The engine runs in the browser as WebAssembly in a Web Worker; no server is
involved. See [Client-only](./explanation/architecture/client_only.md).
## Client/server replicated mode
A server owns the authoritative `Table`; each browser keeps a synchronized
copy and queries it locally. See
[Client/Server replicated](./explanation/architecture/client_server.md).
## Server-only mode
Queries run on the server and the browser receives only the rows it is
displaying. See [Server only](./explanation/architecture/server_only.md).
## Virtual server
An implementation of Perspective's protocol over an external query engine,
such as DuckDB, ClickHouse, PostgreSQL or Polars, which translates view
configurations into that engine's native queries. See
[Virtual Servers](./explanation/virtual_servers.md).
## Virtual scrolling
Rendering only the rows and columns currently in the viewport, and fetching
more as the user scrolls.
## Apache Arrow
A language-independent columnar memory format. It is Perspective's preferred
interchange format: it loads without parsing and preserves types.
## WebAssembly
A portable binary instruction format which runs at near-native speed in
browsers. Perspective's C++ query engine and Rust UI are both compiled to
WebAssembly.
## Memory64
A WebAssembly extension for 64-bit memory addressing. Perspective ships an
optional Memory64 engine build which raises the in-browser heap limit from
4GB to 16GB.
# FAQ
## Installation
### Python installation fails on Windows
Python wheels are published for supported Python versions and platforms. On
Windows, ensure you have a compatible Python version and architecture. Install
with:
```bash
pip install perspective-python
```
If you encounter C++ binding errors or link errors, make sure you are using a
supported Python version and that your `pip` is up to date. Pre-built wheels
eliminate the need for a C++ compiler in most cases.
### Python `import perspective` fails with `ImportError` or undefined symbol
This typically happens when the C++ shared library (`libpsp.so`) cannot be found
or was built against a different Python version. Ensure your Python version
matches the installed wheel. On Linux, verify that required system libraries are
present. If you see errors about `libpsp.so` or undefined symbols, try
reinstalling in a clean virtual environment.
### Python installation fails on macOS
On Apple Silicon (M1/M2/M3), make sure you are using a native ARM Python build,
not one running under Rosetta. The published wheels include `aarch64` variants
for supported platforms.
### How do I install Perspective in a Docker container?
Perspective's Python wheels are built against `manylinux_2_28` containers (see
[`.github/workflows/build.yaml`](../../.github/workflows/build.yaml)), so they
are compatible with most Linux distributions based on glibc 2.28+ (e.g., Debian
10+, Ubuntu 20.04+, RHEL 8+). Use a compatible base image:
```dockerfile
FROM python:3.12-slim
RUN pip install perspective-python
```
Alpine Linux uses musl instead of glibc and is **not** compatible with the
published wheels.
## JavaScript Bundling
### How do I use Perspective with Vite, Webpack, or esbuild?
Perspective no longer exports bundler plugins. Instead, you must manually
bootstrap the WASM binaries using your bundler's asset handling. See
[Importing with or without a bundler](./how_to/javascript/importing.md) for
complete examples for Vite, Webpack, esbuild, CDN, and inline builds.
## Framework Integration
### How do I use Perspective with React?
Perspective provides a dedicated
[React component](./how_to/javascript/react.md). You must also still initialize
Perspective's WebAssembly as per your bundler — see
[Importing with or without a bundler](./how_to/javascript/importing.md).
### How do I use Perspective with Next.js?
Perspective relies on Web Workers and WASM, which require client-side rendering.
Use dynamic imports with `ssr: false` in Next.js to load Perspective components
only on the client.
### How do I use Perspective with Vue.js/Angular/etc?
As a standard Web Component, `` works in most JavaScript web
frameworks directly via standard HTML/DOM APIs, but does not have dedicated
integration libraries for these frameworks.
## Expressions
### How do I create computed/expression columns?
Use the [`expressions`](./explanation/view/config/expressions.md) config option
in your `View` to define new columns with ExprTK syntax, which must then be
_used_ somewhere else in your config (like `columns`) to actually be visible &
calculated. In ``, expression columns can be created from
the UI column sidebar by clicking the "New Column" button.
### Can I reference one expression column from another?
No, you must duplicate calculations that are shared between expression columns.
### Can I do date arithmetic in expressions?
Yes, but they must be converted to `float` values first (`integer` is an `i32`
which is too small). See
[Expressions](./explanation/view/config/expressions.md).
### Can I do rolling sums or cumulative calculations?
Yes — use [Window Columns](./explanation/view/config/windows.md), the
`windows` property of a `View` config. These are ordered, partitioned rolling
computations analogous to SQL window functions, declared per-`View` like
expression columns:
```javascript
const view = await table.view({
columns: ["Cumulative Sales"],
windows: {
"Cumulative Sales": {
column: "Sales",
aggregate: "sum",
order_by: ["Order Date", "asc"],
cumulative: true,
},
},
});
```
Window Columns are supported by Perspective's built-in engine, by the DuckDB,
ClickHouse, PostgreSQL and Polars
[Virtual Servers](./explanation/virtual_servers.md), and
by the `` UI. They update incrementally as the `Table`
updates.
## Filters
### Can I compose filters with OR logic?
Perspective
[filters](./explanation/view/config/selection_and_ordering.md#filter) are
composed with AND logic by default. As an alternative, you can use
[expression columns](./explanation/view/config/expressions.md) to create a
boolean column that encodes your OR logic (or any arbitrary multi-column
predicate), then filter on that column:
```javascript
const view = await table.view({
expressions: {
or_filter:
"if (\"State\" == 'Texas') true; else if (\"State\" == 'California') true; else false",
},
filter: [["or_filter", "==", true]],
});
```
### How do I update filters programmatically?
Set the [`filter`](./explanation/view/config/selection_and_ordering.md#filter)
property on a `View` config, or use the ``
[`.restore()`](./how_to/javascript/save_restore.md) method to update filters at
runtime.
### Does date filtering support ranges?
Date columns can be
[filtered](./explanation/view/config/selection_and_ordering.md#filter) with
comparison operators (`>`, `<`, `>=`, `<=`) to achieve range-based filtering.
Apply two filters on the same date column for a range.
## Notebooks
### `PerspectiveWidget` is not loading
`PerspectiveWidget` is an [AnyWidget](https://anywidget.dev), shipped entirely
inside the `perspective-python` wheel. There is no separate JupyterLab
extension to install or version-match for the widget, so
`jupyter labextension list` is not where to look.
Install the `jupyter` extra, which pulls in `anywidget`:
```bash
pip install "perspective-python[jupyter]"
```
Then restart the kernel — and, for JupyterLab, reload the browser page. See
the [`PerspectiveWidget` guide](./how_to/python/jupyterlab.md).
### Does `PerspectiveWidget` work outside JupyterLab?
Yes. Because the widget is an AnyWidget bundled into the wheel rather than a
JupyterLab labextension, it runs in any AnyWidget-compatible host —
JupyterLab, classic Jupyter Notebook, **VSCode notebooks**, Google Colab and
Marimo — with no per-host install step.
The separate `@perspective-dev/jupyterlab` package is now _optional_ and
provides only the "Open With → Perspective" file renderers for `csv`, `json`
and `arrow` files in JupyterLab.
## Memory and Performance
### Perspective has a memory leak
Maybe, but please review the
[Cleaning up resources](./how_to/javascript/deleting.md) docs carefully before
opening an Issue reporting it (and of course review
[`CONTRIBUTING.md`](https://github.com/perspective-dev/perspective/blob/master/CONTRIBUTING.md)
before opening _any_ Issue). Ensure you call `.delete()` on Views, Tables, and
`` instances when they are no longer needed, in reverse
dependency order.
### How many rows can Perspective's built-in engine handle?
Perspective is designed for large datasets and can handle millions of rows
depending on the number of columns and available memory. Performance also
significantly depends on column types (`"string"` being slower and larger than
other types due to dictionary interning).
For tables larger than the engine's memory, create the `Table` with
[`page_to_disk`](./explanation/table/options.md#page_to_disk), which pages
columns out to disk (OPFS in the browser). For datasets which should not be
loaded at all, see [Virtual Servers](./explanation/virtual_servers.md).
### How do I control threading in `perspective-python`?
The Python library uses a thread pool internally. For advanced threading
control, consult the
[multithreading documentation](./how_to/python/multithreading.md).
## Theming and Styling
### How do I enable dark theme?
Import `themes.css` (see [Theming](./how_to/javascript/theming.md)) and set the
theme via `restore()`:
```javascript
await viewer.restore({ theme: "Pro Dark" });
```
Or import just the dark theme directly:
`import "@perspective-dev/viewer/dist/css/pro-dark.css";`
### Can I create a custom cell renderer for the datagrid?
The datagrid plugin supports custom styling via
[`column_config`](https://perspective-dev.github.io/viewer/types/src_ts_ts-rs_ColumnConfigValues.ts.ColumnConfigValues.html)
and CSS custom properties, but custom cell renderers require building a custom
plugin.
### How do I customize chart colors?
Chart colors can be customized via
[CSS custom properties](./how_to/javascript/theming.md#custom-themes) on the
`` element.
## Streaming and Real-Time Updates
### How do I stream data into a Perspective table?
Use [`table.update()`](./explanation/table/update_and_remove.md) to push new
data incrementally. For [indexed](./explanation/table/options.md) tables,
updates with matching index values will replace existing rows.
### `table.update()` raises "No Running Event Loop"
Perspective 3+ is now threadsafe by default and no longer requires special loop
integration.
### How do I listen for data updates?
Use `view.on_update()` to register a callback that fires when the underlying
table data changes. See [Listening for events](./how_to/javascript/events.md)
and [Advanced View Operations](./explanation/view/advanced.md#update-callbacks).
## Server Architecture
### What is the difference between Client-only, Client/Server, and Server-only modes?
- **Client-only**: The Perspective engine runs entirely in the browser via WASM.
Best for small to medium datasets.
- **Client/Server (replicated)**: Data is hosted on a server and replicated to
the client. The client has a full copy and performs queries locally.
- **Server-only**: All queries are executed on the server. The client only
renders results. Best for very large datasets.
See [Data Architecture](./explanation/architecture.md) for detailed explanations
of each mode.
### Is the WebSocket Perspective `Server` safe to expose to untrusted clients?
No. The WebSocket `Server` is not a security boundary. Every connected `Client`
is treated as the author of the queries it submits, and is permitted to create
and delete `Table`/`View` resources, author arbitrary
[expression columns](./explanation/view/config/expressions.md), and — for
[Virtual Server](./explanation/virtual_servers.md) backends like DuckDB,
ClickHouse or PostgreSQL — author SQL fragments executed under the configured
database role. The bundled WebSocket adapters
(`tornado.py`/`aiohttp.py`/`starlette.py`/`WebSocketServer`) are reference
integrations and do not authenticate, authorize, or enforce origin policy.
WebSocket Deployments that need per-user isolation must put an authenticating
proxy in front of the `Server`, run a least-privileged database role for any
`Virtual Server` backend, and/or isolate users into separate `Server`
instances. See [`SECURITY.md`](../../SECURITY.md) for the full threat model
and deployment guidance.
Obviously, none of this applies to WASM DBs like Perspective and DuckDB.
### Does Perspective sanitize SQL `Virtual Server`s?
No, by design. [Virtual Server](./explanation/virtual_servers.md) backends
interpolate client-supplied `view_id`, `table_id`, `column_name`, expression
strings, and filter operators directly into SQL templates without
parameterization or whitelist validation. The `Client` is the author of the
queries — there is no privilege boundary inside the engine for sanitization
to enforce. If your deployment needs to restrict the SQL surface area exposed
to a `Client`, the supported boundary is the database role the `Virtual Server`
is configured with (read-only etc), or better complete isolation via WASM
backend.
### How do I set up WebSocket authentication?
The [`WebSocketServer`](./how_to/javascript/nodejs_server.md) does not include
built-in authentication. Implement authentication at the transport layer (e.g.,
via middleware in your HTTP server) before the WebSocket upgrade. For more
complex needs, `WebSocketServer` is a simple example server based on the
`node:http` module which can serve as a starting point for a custom server.
### Can I bind Perspective to a database?
Perspective supports [Virtual Servers](./explanation/virtual_servers.md) that
proxy queries to external data sources, with built-in implementations for e.g.
[DuckDB](./how_to/javascript/virtual_server/duckdb.md).
## Aggregation
### Can I apply multiple aggregates to the same column?
Yes, by creating a duplicate/alias for your column via
[`expressions`](./explanation/view/config/expressions.md):
```javascript
await viewer.restore({
columns: ["Sales", "Sales 2"],
expresions: { "Sales 2": '"Sales"' },
aggregate: {
Sales: "sum",
"Sales 2": "avg",
},
});
```
### Can I compute a ratio between aggregated columns?
Use [expression columns](./explanation/view/config/expressions.md) on an
aggregated View to compute ratios. Define an expression that divides one column
by another.
## Data Loading and Arrow
### How do I load Apache Arrow data into Perspective?
Perspective natively accepts
[Apache Arrow format](./explanation/table/loading_data.md). Pass an
`ArrayBuffer` containing Arrow IPC data directly to `table()` or
`table.update()`.
### What data formats does Perspective accept?
Perspective accepts (see [Loading data](./explanation/table/loading_data.md)):
- **JavaScript**: JSON (row-oriented or column-oriented objects), CSV strings,
Apache Arrow `ArrayBuffer`
- **Python**: `dict`, `list`, `pandas.DataFrame`, `pyarrow.Table`, CSV strings,
Apache Arrow bytes
### CSV update fails but CSV creation works
When updating a table created with a schema, ensure the CSV column names and
types match the schema exactly. Mismatched column names or types will cause
update failures.
## Export
### Can I export the viewer to HTML, PNG or PDF?
HTML export is available via `viewer.export({ method: "html" })`. For an
image, use `{ method: "plugin" }`, which asks the plugin to render itself —
this produces a PNG for chart plugins (and text for the datagrid). For PDF,
render the viewer and use browser or headless browser screenshot capabilities.
### Can I export data to Excel?
Perspective does not have built-in Excel export. Export data via
`view.to_csv()`, `view.to_json()`, or `view.to_arrow()` (see
[Serializing data](./how_to/javascript/serializing.md)) and convert to Excel
using a library like `xlsx` (JavaScript) or `openpyxl` (Python).
### How do I copy data from a cell or row?
Use one of the `-selected` export methods, which operate on the current
selection. To place it on the clipboard:
```javascript
await viewer.copy({ method: "csv-selected" });
```
... or to get it as a string, `await viewer.export({ method: "csv-selected" })`.
`json-selected` and `arrow-selected` are also available.
## Table Operations
### `table.remove()` does not update the viewer
The [`remove()`](./explanation/table/update_and_remove.md) method requires an
[indexed](./explanation/table/options.md) table. Ensure your table was created
with an `index` option, and pass the index values to remove.
## Viewer Configuration
### How do I save and restore the viewer state?
Use
[`viewer.save()` and `viewer.restore()`](./how_to/javascript/save_restore.md) to
serialize and deserialize the full viewer configuration.
### Can I hide the configuration panel?
The settings panel can be toggled programmatically via
`await viewer.restore({ settings: false })`.
### Can I collapse row groups by default?
Row group can be closed imperatively via
[`view.set_depth()`](./explanation/view/advanced.md). Expansion state is not
persisted or configurable via the `save`/`restore` API currently.
## Internationalization
### Can I change the UI language?
Perspective's UI text is defined via CSS variables, which can be customized per
theme. See the
[Icons and Translation](./how_to/javascript/theming.md#icons-and-translation)
section of the theming guide for details.
## Rust
### How do I build Perspective from Rust?
See the [Getting Started](./how_to/rust.md) guide for Rust. The Rust crate wraps
the C++ engine and requires a C++ toolchain. You need `cmake` installed and on
your path to build the engine.
## Miscellaneous
### Can I use Perspective without ``?
Yes. The `perspective` library (data engine) can be used independently for
server-side data processing without any UI. Use
[`table()` and `view()`](./how_to/javascript/worker.md) directly to query data.
### Can I use Perspective in Pyodide?
Yes. Perspective publishes Emscripten wheels to PyPI under
[PEP 783](https://peps.python.org/pep-0783/), so `perspective-python` can be
installed by Pyodide's own package resolution — there is no need to download
and host a wheel yourself.
Emscripten wheels are ABI-tied to a specific Emscripten version, and thus to
the Pyodide versions built against it. If resolution fails, check that your
Pyodide version matches a published wheel tag.
### How do I handle row selection events?
Listen for
[`perspective-click` and `perspective-select`](./how_to/javascript/events.md)
events on the `` element.
# Client-only
_For static datasets, datasets provided by the user, and simple server-less and
read-only web applications._
In this design, Perspective is run as a client Browser WebAssembly library, the
dataset is downloaded entirely to the client and all calculations and UI
interactions are performed locally. Interactive performance is very good, using
WebAssembly engine for near-native runtime plus WebWorker isolation for parallel
rendering within the browser. Operations like scrolling and creating new views
are responsive. However, the entire dataset must be downloaded to the client.
Perspective is not a typical browser component, and datset sizes of 1gb+ in
Apache Arrow format will load fine with good interactive performance!
Horizontal scaling is a non-issue, since here is no concurrent state to scale,
and only uses client-side computation via WebAssembly client. Client-only
perspective can support as many concurrent users as can download the web
application itself. Once the data is loaded, no server connection is needed and
all operations occur in the client browser, imparting no additional runtime cost
on the server beyond initial load. This also means updates and edits are local
to the browser client and will be lost when the page is refreshed, unless
otherwise persisted by your application.
As the client-only design starts with creating a client-side Perspective
`Table`, data can be provided by any standard web service in any Perspective
compatible format (JSON, CSV or Apache Arrow).
## Javascript client
```javascript
const worker = await perspective.worker();
const table = await worker.table(csv);
const viewer = document.createElement("perspective-viewer");
document.body.appendChild(viewer);
await viewer.load(table);
```
# Client/Server replicated
_For medium-sized, real-time, synchronized and/or editable data sets with many
concurrent users._
The dataset is instantiated in-memory with a Python or Node.js Perspective
server, and web applications create duplicates of these tables in a local
WebAssembly client in the browser, synchonized efficiently to the server via
Apache Arrow. This design scales well with additional concurrent users, as
browsers only need to download the initial data set and subsequent update
deltas, while operations like scrolling, pivots, sorting, etc. are performed on
the client.
Python servers can make especially good use of additional threads, as
Perspective will release the GIL for almost all operations. Interactive
performance on the client is very good and identical to client-only
architecture. Updates and edits are seamlessly synchonized across clients via
their virtual server counterparts using websockets and Apache Arrow.
## Python and Tornado server
```python
from perspective import Server, PerspectiveTornadoHandler
server = Server()
client = server.new_local_client()
client.table(csv, name="my_table")
routes = [(
r"/websocket",
perspective.handlers.tornado.PerspectiveTornadoHandler,
{"perspective_server": server},
)]
app = tornado.web.Application(routes)
app.listen(8080)
loop = tornado.ioloop.IOLoop.current()
loop.start()
```
## Javascript client
Perspective's websocket client interfaces with the Python server, then
_replicates_ the server-side Table. When the server-side `Table` has an `index`,
the replica inherits it, and both `update()` and `remove()` on the server are
mirrored in the browser.
```javascript
const websocket = await perspective.websocket("ws://localhost:8080");
const server_table = await websocket.open_table("my_table");
const server_view = await server_table.view();
const worker = await perspective.worker();
const client_table = await worker.table(server_view);
const viewer = document.createElement("perspective-viewer");
document.body.appendChild(viewer);
await viewer.load(client_table);
```
# Server-only
_For extremely large datasets with a small number of concurrent users._
The dataset is instantiated in-memory with a Python or Node.js server, and web
applications connect virtually. Has very good initial load performance, since no
data is downloaded. Group-by and other operations will run column-parallel if
configured.
But interactive performance is poor, as every user interaction must page the
server to render. Operations like scrolling are not as responsive and can be
impacted by network latency. Web applications must be "always connected" to the
server via WebSocket. Disconnecting will prevent any interaction, scrolling,
etc. of the UI. Does not use WebAssembly.
Each connected browser will impact server performance as long as the connection
is open, which in turn impacts interactive performance of every client. This
ultimately limits the horizontal scalabity of this architecture. Since each
client reads the perspective `Table` virtually, changes like edits and updates
are automatically reflected to all clients and persist across browser refresh.
Using the same Python server as the previous design, we can simply skip the
intermediate WebAssembly `Table` and pass the virtual table directly to `load()`
```javascript
const websocket = await perspective.websocket("ws://localhost:8080");
const server_table = await websocket.open_table("my_table");
const viewer = document.createElement("perspective-viewer");
document.body.appendChild(viewer);
await viewer.load(server_table);
```
# Virtual Servers
A Virtual Server allows Perspective to query external data sources (such as
DuckDB or ClickHouse) without loading the entire dataset into Perspective's
built-in data engine. Instead, Perspective translates its query operations
(group by, sort, filter, etc.) into queries the external data source can execute
natively, and only transfers the data needed for the current view.
The Virtual Server API works on any platform that has a Perspective Client —
including JavaScript (both Node.js and the browser via WebAssembly), Python, and
Rust. In the browser, this means a virtual server can front a WASM-based engine
like `@duckdb/duckdb-wasm`, giving `