chore: import upstream snapshot with attribution

This commit is contained in:
wehub-resource-sync
2026-07-13 12:25:07 +08:00
commit a26e856398
1681 changed files with 296950 additions and 0 deletions
+34
View File
@@ -0,0 +1,34 @@
# Callbacks and Events
`perspective.Table` allows for `on_update` and `on_delete` callbacks to be
set—simply call `on_update` or `on_delete` with a reference to a function or a
lambda without any parameters:
```python
def update_callback():
print("Updated!")
# set the update callback
on_update_id = view.on_update(update_callback)
def delete_callback():
print("Deleted!")
# set the delete callback
on_delete_id = view.on_delete(delete_callback)
# set a lambda as a callback
view.on_delete(lambda: print("Deleted x2!"))
```
If the callback is a named reference to a function, it can be removed with
`remove_update` or `remove_delete`:
```python
view.remove_update(on_update_id)
view.remove_delete(on_delete_id)
```
Callbacks defined with a lambda function cannot be removed, as lambda functions
have no identifier.
+27
View File
@@ -0,0 +1,27 @@
# Installation
`perspective-python` contains full bindings to the Perspective API, a JupyterLab
widget, and WebSocket handlers for several webserver libraries that allow you to
host Perspective using server-side Python.
## PyPI
`perspective-python` can be installed from [PyPI](https://pypi.org) via `pip`:
```bash
pip install perspective-python
```
That's it! If JupyterLab is installed in this Python environment, you'll also
get the `perspective.widget.PerspectiveWidget` class when you import
`perspective` in a Jupyter Lab kernel.
<!--
### Anaconda
`perspective-python` can also be installed for [Anaconda](https://anaconda.org/)
via [Conda Forge](https://conda-forge.org)
```bash
conda install -c conda-forge perspective
``` -->
+64
View File
@@ -0,0 +1,64 @@
# Joining Tables
`perspective.join()` creates a read-only `Table` by joining two source tables on
a shared key column. The result is reactive — it updates automatically when
either source table changes. See [`Join`](../../explanation/join.md) for
conceptual details.
## Basic Inner Join
```python
orders = perspective.table([
{"id": 1, "product_id": 101, "qty": 5},
{"id": 2, "product_id": 102, "qty": 3},
{"id": 3, "product_id": 101, "qty": 7},
])
products = perspective.table([
{"product_id": 101, "name": "Widget"},
{"product_id": 102, "name": "Gadget"},
])
joined = perspective.join(orders, products, "product_id")
view = joined.view()
json = view.to_json()
```
## Join Types
Pass `join_type` to select inner, left, or outer join behavior:
```python
# Left join: all left rows, nulls for unmatched right columns
left_joined = perspective.join(left, right, "id", join_type="left")
# Outer join: all rows from both tables
outer_joined = perspective.join(left, right, "id", join_type="outer")
```
## Reactive Updates
The joined table recomputes automatically when either source table is updated:
```python
left = perspective.table([{"id": 1, "x": 10}])
right = perspective.table([{"id": 2, "y": "b"}])
joined = perspective.join(left, right, "id")
view = joined.view()
json = view.to_json()
# [] — no matching keys yet
right.update([{"id": 1, "y": "a"}])
json = view.to_json()
# [{"id": 1, "x": 10, "y": "a"}] — new match detected
```
## Async Client
The async client has the same API:
```python
joined = await client.join(orders, products, "product_id", join_type="left")
```
+61
View File
@@ -0,0 +1,61 @@
# `PerspectiveWidget` for JupyterLab
Building on top of the API provided by `perspective.Table`, the
`PerspectiveWidget` is a JupyterLab plugin that offers the entire functionality
of Perspective within the Jupyter environment. It supports the same API
semantics of `<perspective-viewer>`, along with the additional data types
supported by `perspective.Table`. `PerspectiveWidget` takes keyword arguments
for the managed `View`:
```python
from perspective.widget import PerspectiveWidget
w = perspective.PerspectiveWidget(
data,
plugin="X Bar",
aggregates={"datetime": "any"},
sort=[["date", "desc"]]
)
```
## Creating a widget
A widget is created through the `PerspectiveWidget` constructor, which takes as
its first, required parameter a `perspective.Table`, a dataset, a schema, or
`None`, which serves as a special value that tells the Widget to defer loading
any data until later. In maintaining consistency with the Javascript API,
Widgets cannot be created with empty dictionaries or lists — `None` should be
used if the intention is to await data for loading later on. A widget can be
constructed from a dataset:
```python
from perspective.widget import PerspectiveWidget
PerspectiveWidget(data, group_by=["date"])
```
.. or a schema:
```python
PerspectiveWidget({"a": int, "b": str})
```
.. or an instance of a `perspective.Table`:
```python
table = perspective.table(data)
PerspectiveWidget(table)
```
## Updating a widget
`PerspectiveWidget` shares a similar API to the `<perspective-viewer>` Custom
Element, and has similar `save()` and `restore()` methods that
serialize/deserialize UI state for the widget.
<!--
## `PerspectiveRenderer`
Perspective also exposes a JS-only `mimerender-extension`. This lets you view
`csv`, `json`, and `arrow` files directly from the file browser. You can see
this by right clicking one of these files and `Open With->CSVPerspective` (or
`JSONPerspective` or `ArrowPerspective`). Perspective will also install itself
as the default handler for opening `.arrow` files. -->
+67
View File
@@ -0,0 +1,67 @@
# Multi-threading
Perspective's API is thread-safe, so methods may be called from different
threads without additional consideration for safety/exclusivity/correctness. All
`perspective.Client` and `perspective.Server` API methods release the GIL, which
can be exploited for parallelism.
Interally, `perspective.Server` also dispatches to a thread pool for some
operations, enabling better parallelism and overall better query performance.
This independent threadpool size can be controlled via
`perspective.set_num_cpus()`, or the `OMP_NUM_THREADS` environment variable.
```python
import perspective
perspective.set_num_cpus(2)
```
## Server handlers
Perspective's server handler implementations each take an optional `executor`
constructor argument, which (when provided) will configure the handler to
process WebSocket `Client` requests on a thread pool.
```python
from concurrent.futures import ThreadPoolExecutor
from tornado.web import Application
from perspective.handlers.tornado import PerspectiveTornadoHandler
from perspective import Server
args = {"perspective_server": Server(), "executor": ThreadPoolExecutor()}
app = Application(
[
(r"/websocket", PerspectiveTornadoHandler, args),
# ...
]
)
```
## `on_poll_request`
`on_poll_request` is an optional keyword argument for `Server()`, which which
can be applied in cases where overlapping `Table.update` calls can be safely
deferred.
When providing a callback function to `on_poll_request`, the `Server` will
invoke your callback when there are updates that need to be flushed, after which
you must _eventually_ call `Server.poll` (or else no updates will be processed).
The exact implementation of `on_poll_request` will depend on the context. A
simple example which batches calls via `threading.Lock`:
```python
lock = threading.Lock()
def on_poll_request(perspective_server):
if lock.acquire(blocking=False):
try:
perspective_server.poll()
finally:
lock.release()
server = Server(on_poll_request=on_poll_request)
```
+90
View File
@@ -0,0 +1,90 @@
# Loading data into a Table
A `Table` can be created from a dataset or a schema, the specifics of which are
[discussed](#loading-data-with-table) in the JavaScript section of the user's
guide. In Python, however, Perspective supports additional data types that are
commonly used when processing data:
- `pandas.DataFrame`
- `polars.DataFrame`
- `bytes` (encoding an Apache Arrow)
- `objects` (either extracting a repr or via reference)
- `str` (encoding as a CSV)
A `Table` is created in a similar fashion to its JavaScript equivalent:
```python
from datetime import date, datetime
import numpy as np
import pandas as pd
import perspective
data = pd.DataFrame({
"int": np.arange(100),
"float": [i * 1.5 for i in range(100)],
"bool": [True for i in range(100)],
"date": [date.today() for i in range(100)],
"datetime": [datetime.now() for i in range(100)],
"string": [str(i) for i in range(100)]
})
table = perspective.table(data, index="float")
```
Likewise, a `View` can be created via the `view()` method:
```python
view = table.view(group_by=["float"], filter=[["bool", "==", True]])
column_data = view.to_columns()
row_data = view.to_json()
```
## Polars Support
Polars `DataFrame` types work similarly to Apache Arrow input, which Perspective
uses to interface with Polars.
```python
df = polars.DataFrame({"a": [1,2,3,4,5]})
table = perspective.table(df)
```
## Pandas Support
Perspective's `Table` can be constructed from `pandas.DataFrame` objects.
Internally, this just uses
[`pyarrow::from_pandas`](https://arrow.apache.org/docs/python/pandas.html),
which dictates behavior of this feature including type support.
If the dataframe does not have an index set, an integer-typed column named
`"index"` is created. If you want to preserve the indexing behavior of the
dataframe passed into Perspective, simply create the `Table` with
`index="index"` as a keyword argument. This tells Perspective to once again
treat the index as a primary key:
```python
data.set_index("datetime")
table = perspective.table(data, index="index")
```
## Time Zone Handling
When parsing `"datetime"` strings, times without an explicit timezone offset are
interpreted as _UTC_. Strings with a timezone offset (e.g., `+05:00`) are
converted to UTC. All `"datetime"` values are stored internally as milliseconds
since the Unix epoch, and are _output_ as integer timestamps (milliseconds since
epoch) from methods like `to_columns()` and `to_json()`.
Python `datetime` objects are serialized to strings before parsing. Naive
`datetime` objects (without `tzinfo`) produce strings without timezone
information and are therefore treated as UTC. Timezone-aware `datetime` objects
include their offset in the serialized string, which is used to convert to UTC.
`"date"` values are timezone-agnostic calendar days with no time component.
They are _output_ as integer timestamps at _UTC midnight_ of the calendar day
(equivalent to Arrow `date32` day arithmetic), and integer timestamp _input_ to
a `"date"` column is likewise interpreted as UTC. The host process timezone
never affects `"date"` values — a `Viewer` renders them in UTC, recovering the
stored calendar day exactly. Datetime expression functions such as
`bucket("x", 'D')`, `day_of_week("x")` and `hour_of_day("x")` also compute in
UTC.
+81
View File
@@ -0,0 +1,81 @@
# DataFrame and Arrow Compatibility
`perspective-python` accepts a `Table` constructor argument from any of the
common Python columnar data libraries. In all three cases, `perspective.table`
(and `Table.update()`) consume the input directly — there is no need to
serialize to Apache Arrow IPC bytes yourself. However, note is
still the most efficient way to bulk load data into `Table`.
## PyArrow
```python
import pyarrow as pa
import perspective
arrow_table = pa.table({
"int": pa.array([1, 2, 3], type=pa.int64()),
"float": pa.array([1.5, 2.5, 3.5], type=pa.float64()),
"string": pa.array(["a", "b", "c"], type=pa.string()),
})
table = perspective.table(arrow_table)
```
The same applies to `Table.update()`:
```python
table.update(arrow_table)
```
If you have Arrow data already in IPC format (e.g. read from disk, received
over the wire, or produced by another tool), pass the raw `bytes` directly —
both stream and file formats are auto-detected:
```python
with open("data.arrow", "rb") as f:
table = perspective.table(f.read())
```
## Polars
```python
import polars as pl
import perspective
df = pl.DataFrame({
"a": [1, 2, 3, 4, 5],
"b": ["x", "y", "z", "x", "y"],
})
table = perspective.table(df)
```
Internally, the `DataFrame` is converted to a `pyarrow.Table` before
ingestion, so Polars columns inherit the Arrow type mapping above.
See also Perspective [Virtual Server support for `polars.DataFrame`](./virtual_server/polars.md)
## Pandas
`pandas.DataFrame` is supported via `pyarrow.Table.from_pandas`, which
dictates behavior including type support — see the
[pyarrow pandas docs](https://arrow.apache.org/docs/python/pandas.html) for
details on which pandas dtypes round-trip cleanly.
```python
from datetime import date, datetime
import numpy as np
import pandas as pd
import perspective
data = pd.DataFrame({
"int": np.arange(100),
"float": [i * 1.5 for i in range(100)],
"bool": [True for i in range(100)],
"date": [date.today() for i in range(100)],
"datetime": [datetime.now() for i in range(100)],
"string": [str(i) for i in range(100)],
})
table = perspective.table(data, index="float")
```
+20
View File
@@ -0,0 +1,20 @@
# Virtual Servers
Perspective's Virtual Server feature lets you connect `<perspective-viewer>` to
external data sources without loading data into Perspective's built-in engine.
Instead, queries are translated and executed natively by the external database.
For a detailed explanation of how virtual servers work, see the
[Virtual Servers](../../explanation/virtual_servers.md) concepts page.
Perspective ships with built-in virtual server implementations for:
- [**DuckDB**](./virtual_server/duckdb.md) — query DuckDB databases using the
`duckdb` Python package.
- [**ClickHouse**](./virtual_server/clickhouse.md) — query a ClickHouse server
using the `clickhouse-connect` Python package.
- [**Polars**](./virtual_server/polars.md) — query in-memory Polars DataFrames
using the `polars` Python package.
You can also [**implement your own**](./virtual_server/custom.md) virtual server
to connect Perspective to any data source by subclassing `VirtualServerHandler`.
@@ -0,0 +1,52 @@
# ClickHouse Virtual Server
Perspective provides a built-in virtual server for
[ClickHouse](https://clickhouse.com/), allowing `<perspective-viewer>` clients
to query a ClickHouse server over WebSocket.
For browser-only usage, see the
[JavaScript ClickHouse guide](../../javascript/virtual_server/clickhouse.md).
## Installation
```bash
pip install perspective-python clickhouse-connect
```
## Usage
Create a server that exposes ClickHouse tables to browser clients:
```python
import clickhouse_connect
import tornado.web
import tornado.ioloop
from perspective import ClickhouseVirtualServer
from perspective.handlers.tornado import PerspectiveTornadoHandler
# Connect to ClickHouse
client = clickhouse_connect.get_client(host="localhost")
# Create virtual server backed by ClickHouse
server = ClickhouseVirtualServer(client)
# Serve over WebSocket
app = tornado.web.Application([
(r"/websocket", PerspectiveTornadoHandler, {"perspective_server": server}),
])
app.listen(8080)
tornado.ioloop.IOLoop.current().start()
```
Connect from the browser:
```javascript
const websocket = await perspective.websocket("ws://localhost:8080/websocket");
const table = await websocket.open_table("my_table");
document.getElementById("viewer").load(table);
```
## Examples
- [Python ClickHouse example](https://github.com/perspective-dev/perspective/tree/master/examples/python-clickhouse-virtual)
@@ -0,0 +1,64 @@
# Implementing a custom Virtual Server
You can connect Perspective to any data source by subclassing
`VirtualServerHandler` and wrapping it with `VirtualServer`.
For background on virtual servers, see the
[Virtual Servers overview](../../../explanation/virtual_servers.md).
## Example
```python
from perspective import VirtualServerHandler, VirtualServer
class MyModel(VirtualServerHandler):
def get_features(self):
return {
"group_by": True,
"split_by": False,
"sort": True,
"filter_ops": {
"string": ["==", "!=", "contains"],
"float": ["==", "!=", ">", "<"],
},
"aggregates": {
"float": ["sum", "avg", "count"],
"string": ["count"],
},
}
def get_hosted_tables(self):
return ["my_table"]
def table_schema(self, table_name):
return {"name": "string", "price": "float"}
def table_size(self, table_name):
return 1000
def table_make_view(self, table_name, view_id, config):
# Translate `config` (group_by, sort, filter, etc.) into a
# query against your data source. Store the query keyed by
# `view_id` for later data retrieval.
pass
def view_delete(self, view_id):
# Clean up resources for this view
pass
def view_get_data(self, view_id, start_row, end_row, start_col, end_col, ctx):
# Execute the stored query with the given row/column window.
# Push results via `ctx`.
pass
```
The `VirtualServer` instance can then be passed to a Tornado, Starlette, or
AIOHTTP handler just like a regular `Server`:
```python
from perspective.handlers.tornado import PerspectiveTornadoHandler
app = tornado.web.Application([
(r"/websocket", PerspectiveTornadoHandler, {"perspective_server": VirtualServer(MyModel)}),
])
```
@@ -0,0 +1,53 @@
# DuckDB Virtual Server
Perspective provides a built-in virtual server for
[DuckDB](https://duckdb.org/), allowing `<perspective-viewer>` clients to query
a server-side DuckDB database over WebSocket.
For browser-only usage via DuckDB-WASM, see the
[JavaScript DuckDB guide](../../javascript/virtual_server/duckdb.md).
## Installation
```bash
pip install perspective-python duckdb
```
## Usage
Create a server that exposes a DuckDB database to browser clients:
```python
import duckdb
import tornado.web
import tornado.ioloop
from perspective import DuckDBVirtualServer
from perspective.handlers.tornado import PerspectiveTornadoHandler
# Create DuckDB connection and load data
conn = duckdb.connect()
conn.execute("CREATE TABLE my_table AS SELECT * FROM 'data.parquet'")
# Create virtual server backed by DuckDB
server = DuckDBVirtualServer(conn)
# Serve over WebSocket
app = tornado.web.Application([
(r"/websocket", PerspectiveTornadoHandler, {"perspective_server": server}),
])
app.listen(8080)
tornado.ioloop.IOLoop.current().start()
```
Connect from the browser:
```javascript
const websocket = await perspective.websocket("ws://localhost:8080/websocket");
const table = await websocket.open_table("my_table");
document.getElementById("viewer").load(table);
```
## Examples
- [Python DuckDB example](https://github.com/perspective-dev/perspective/tree/master/examples/python-duckdb-virtual)
@@ -0,0 +1,49 @@
# Polars Virtual Server
Perspective provides a built-in virtual server for
[Polars](https://pola.rs/), allowing `<perspective-viewer>` clients to query
in-memory Polars DataFrames over WebSocket.
## Installation
```bash
pip install perspective-python polars
```
## Usage
Create a server that exposes Polars DataFrames to browser clients:
```python
import polars as pl
import tornado.web
import tornado.ioloop
from perspective.virtual_servers.polars import PolarsVirtualServer
from perspective.handlers.tornado import PerspectiveTornadoHandler
# Load data into Polars DataFrames
df = pl.read_parquet("data.parquet")
# Create virtual server backed by Polars (dict of name -> DataFrame)
server = PolarsVirtualServer({"my_table": df})
# Serve over WebSocket
app = tornado.web.Application([
(r"/websocket", PerspectiveTornadoHandler, {"perspective_server": server}),
])
app.listen(8080)
tornado.ioloop.IOLoop.current().start()
```
Connect from the browser:
```javascript
const websocket = await perspective.websocket("ws://localhost:8080/websocket");
const table = await websocket.open_table("my_table");
document.getElementById("viewer").load(table);
```
## Examples
- [Python Polars example](https://github.com/perspective-dev/perspective/tree/master/examples/python-polars-virtual)
+129
View File
@@ -0,0 +1,129 @@
# Hosting a WebSocket server
An in-memory `Server` "hosts" all `perspective.Table` and `perspective.View`
instances created by its connected `Client`s. Hosted tables/views can have their
methods called from other sources than the Python server, i.e. by a
`perspective-viewer` running in a JavaScript client over the network,
interfacing with `perspective-python` through the websocket API.
The server has full control of all hosted `Table` and `View` instances, and can
call any public API method on hosted instances. This makes it extremely easy to
stream data to a hosted `Table` using `.update()`:
```python
server = perspective.Server()
client = server.new_local_client()
table = client.table(data, name="data_source")
for i in range(10):
# updates continue to propagate automatically
table.update(new_data)
```
The `name` provided is important, as it enables Perspective in JavaScript to
look up a `Table` and get a handle to it over the network. Otherwise, `name`
will be assigned randomly and the `Client` must look this up with
`Client.get_hosted_table_names()`
## Client/Server Replicated Mode
Using Tornado and
[`PerspectiveTornadoHandler`](python.md#perspectivetornadohandler), as well as
`Perspective`'s JavaScript library, we can set up "distributed" Perspective
instances that allows multiple browser `perspective-viewer` clients to read from
a common `perspective-python` server, as in the
[Tornado Example Project](https://github.com/perspective-dev/perspective/tree/master/examples/python-tornado).
This architecture works by maintaining two `Tables`—one on the server, and one
on the client that mirrors the server's `Table` automatically using `on_update`.
All updates to the table on the server are automatically applied to each client,
which makes this architecture a natural fit for streaming dashboards and other
distributed use-cases. In conjunction with [multithreading](#multi-threading),
distributed Perspective offers consistently high performance over large numbers
of clients and large datasets.
_*server.py*_
```python
from perspective import Server
from perspective.handlers.tornado import PerspectiveTornadoHandler
# Create an instance of Server, and host a Table
SERVER = Server()
CLIENT = SERVER.new_local_client()
# The Table is exposed at `localhost:8888/websocket` with the name `data_source`
client.table(data, name = "data_source")
app = tornado.web.Application([
# create a websocket endpoint that the client JavaScript can access
(r"/websocket", PerspectiveTornadoHandler, {"perspective_server": SERVER})
])
# Start the Tornado server
app.listen(8888)
loop = tornado.ioloop.IOLoop.current()
loop.start()
```
Instead of calling `load(server_table)`, create a `View` using `server_table`
and pass that into `viewer.load()`. This will automatically register an
`on_update` callback that synchronizes state between the server and the client.
_*index.html*_
```html
<perspective-viewer id="viewer" editable></perspective-viewer>
<script type="module">
// Create a client that expects a Perspective server
// to accept connections at the specified URL.
const websocket = await perspective.websocket(
"ws://localhost:8888/websocket",
);
// Get a handle to the Table on the server
const server_table = await websocket.open_table("data_source_one");
// Create a new view
const server_view = await table.view();
// Create a Table on the client using `perspective.worker()`
const worker = await perspective.worker();
const client_table = await worker.table(view);
// Load the client table in the `<perspective-viewer>`.
document.getElementById("viewer").load(client_table);
</script>
```
For a more complex example that offers distributed editing of the server
dataset, see
[client_server_editing.html](https://github.com/perspective-dev/perspective/blob/master/examples/python-tornado/client_server_editing.html).
We also provide examples for Starlette/FastAPI and AIOHTTP:
- [Starlette Example Project](https://github.com/perspective-dev/perspective/tree/master/examples/python-starlette).
- [AIOHTTP Example Project](https://github.com/perspective-dev/perspective/tree/master/examples/python-aiohttp).
## Server-only Mode
The server setup is identical to
[Client/Server Replicated Mode](#client-server-replicated-mode) above, but
instead of creating a `View`, the client calls `load(server_table)`: In Python,
use `Server` and `PerspectiveTornadoHandler` to create a websocket server that
exposes a `Table`. In this example, `table` is a proxy for the `Table` we
created on the server. All API methods are available on _proxies_, e.g.
calling `view()`, `schema()`, `update()` on `table` will pass those operations
to the Python `Table`, execute the commands, and return the result back to
Javascript.
```html
<perspective-viewer id="viewer" editable></perspective-viewer>
```
```javascript
const websocket = perspective.websocket("ws://localhost:8888/websocket");
const table = websocket.open_table("data_source");
document.getElementById("viewer").load(table);
```