chore: import upstream snapshot with attribution
This commit is contained in:
@@ -0,0 +1,34 @@
|
||||
# Callbacks and Events
|
||||
|
||||
`perspective.Table` allows for `on_update` and `on_delete` callbacks to be
|
||||
set—simply call `on_update` or `on_delete` with a reference to a function or a
|
||||
lambda without any parameters:
|
||||
|
||||
```python
|
||||
def update_callback():
|
||||
print("Updated!")
|
||||
|
||||
# set the update callback
|
||||
on_update_id = view.on_update(update_callback)
|
||||
|
||||
|
||||
def delete_callback():
|
||||
print("Deleted!")
|
||||
|
||||
# set the delete callback
|
||||
on_delete_id = view.on_delete(delete_callback)
|
||||
|
||||
# set a lambda as a callback
|
||||
view.on_delete(lambda: print("Deleted x2!"))
|
||||
```
|
||||
|
||||
If the callback is a named reference to a function, it can be removed with
|
||||
`remove_update` or `remove_delete`:
|
||||
|
||||
```python
|
||||
view.remove_update(on_update_id)
|
||||
view.remove_delete(on_delete_id)
|
||||
```
|
||||
|
||||
Callbacks defined with a lambda function cannot be removed, as lambda functions
|
||||
have no identifier.
|
||||
@@ -0,0 +1,27 @@
|
||||
# Installation
|
||||
|
||||
`perspective-python` contains full bindings to the Perspective API, a JupyterLab
|
||||
widget, and WebSocket handlers for several webserver libraries that allow you to
|
||||
host Perspective using server-side Python.
|
||||
|
||||
## PyPI
|
||||
|
||||
`perspective-python` can be installed from [PyPI](https://pypi.org) via `pip`:
|
||||
|
||||
```bash
|
||||
pip install perspective-python
|
||||
```
|
||||
|
||||
That's it! If JupyterLab is installed in this Python environment, you'll also
|
||||
get the `perspective.widget.PerspectiveWidget` class when you import
|
||||
`perspective` in a Jupyter Lab kernel.
|
||||
|
||||
<!--
|
||||
### Anaconda
|
||||
|
||||
`perspective-python` can also be installed for [Anaconda](https://anaconda.org/)
|
||||
via [Conda Forge](https://conda-forge.org)
|
||||
|
||||
```bash
|
||||
conda install -c conda-forge perspective
|
||||
``` -->
|
||||
@@ -0,0 +1,64 @@
|
||||
# Joining Tables
|
||||
|
||||
`perspective.join()` creates a read-only `Table` by joining two source tables on
|
||||
a shared key column. The result is reactive — it updates automatically when
|
||||
either source table changes. See [`Join`](../../explanation/join.md) for
|
||||
conceptual details.
|
||||
|
||||
## Basic Inner Join
|
||||
|
||||
```python
|
||||
orders = perspective.table([
|
||||
{"id": 1, "product_id": 101, "qty": 5},
|
||||
{"id": 2, "product_id": 102, "qty": 3},
|
||||
{"id": 3, "product_id": 101, "qty": 7},
|
||||
])
|
||||
|
||||
products = perspective.table([
|
||||
{"product_id": 101, "name": "Widget"},
|
||||
{"product_id": 102, "name": "Gadget"},
|
||||
])
|
||||
|
||||
joined = perspective.join(orders, products, "product_id")
|
||||
view = joined.view()
|
||||
json = view.to_json()
|
||||
```
|
||||
|
||||
## Join Types
|
||||
|
||||
Pass `join_type` to select inner, left, or outer join behavior:
|
||||
|
||||
```python
|
||||
# Left join: all left rows, nulls for unmatched right columns
|
||||
left_joined = perspective.join(left, right, "id", join_type="left")
|
||||
|
||||
# Outer join: all rows from both tables
|
||||
outer_joined = perspective.join(left, right, "id", join_type="outer")
|
||||
```
|
||||
|
||||
## Reactive Updates
|
||||
|
||||
The joined table recomputes automatically when either source table is updated:
|
||||
|
||||
```python
|
||||
left = perspective.table([{"id": 1, "x": 10}])
|
||||
right = perspective.table([{"id": 2, "y": "b"}])
|
||||
|
||||
joined = perspective.join(left, right, "id")
|
||||
view = joined.view()
|
||||
|
||||
json = view.to_json()
|
||||
# [] — no matching keys yet
|
||||
|
||||
right.update([{"id": 1, "y": "a"}])
|
||||
json = view.to_json()
|
||||
# [{"id": 1, "x": 10, "y": "a"}] — new match detected
|
||||
```
|
||||
|
||||
## Async Client
|
||||
|
||||
The async client has the same API:
|
||||
|
||||
```python
|
||||
joined = await client.join(orders, products, "product_id", join_type="left")
|
||||
```
|
||||
@@ -0,0 +1,61 @@
|
||||
# `PerspectiveWidget` for JupyterLab
|
||||
|
||||
Building on top of the API provided by `perspective.Table`, the
|
||||
`PerspectiveWidget` is a JupyterLab plugin that offers the entire functionality
|
||||
of Perspective within the Jupyter environment. It supports the same API
|
||||
semantics of `<perspective-viewer>`, along with the additional data types
|
||||
supported by `perspective.Table`. `PerspectiveWidget` takes keyword arguments
|
||||
for the managed `View`:
|
||||
|
||||
```python
|
||||
from perspective.widget import PerspectiveWidget
|
||||
w = perspective.PerspectiveWidget(
|
||||
data,
|
||||
plugin="X Bar",
|
||||
aggregates={"datetime": "any"},
|
||||
sort=[["date", "desc"]]
|
||||
)
|
||||
```
|
||||
|
||||
## Creating a widget
|
||||
|
||||
A widget is created through the `PerspectiveWidget` constructor, which takes as
|
||||
its first, required parameter a `perspective.Table`, a dataset, a schema, or
|
||||
`None`, which serves as a special value that tells the Widget to defer loading
|
||||
any data until later. In maintaining consistency with the Javascript API,
|
||||
Widgets cannot be created with empty dictionaries or lists — `None` should be
|
||||
used if the intention is to await data for loading later on. A widget can be
|
||||
constructed from a dataset:
|
||||
|
||||
```python
|
||||
from perspective.widget import PerspectiveWidget
|
||||
PerspectiveWidget(data, group_by=["date"])
|
||||
```
|
||||
|
||||
.. or a schema:
|
||||
|
||||
```python
|
||||
PerspectiveWidget({"a": int, "b": str})
|
||||
```
|
||||
|
||||
.. or an instance of a `perspective.Table`:
|
||||
|
||||
```python
|
||||
table = perspective.table(data)
|
||||
PerspectiveWidget(table)
|
||||
```
|
||||
|
||||
## Updating a widget
|
||||
|
||||
`PerspectiveWidget` shares a similar API to the `<perspective-viewer>` Custom
|
||||
Element, and has similar `save()` and `restore()` methods that
|
||||
serialize/deserialize UI state for the widget.
|
||||
|
||||
<!--
|
||||
## `PerspectiveRenderer`
|
||||
|
||||
Perspective also exposes a JS-only `mimerender-extension`. This lets you view
|
||||
`csv`, `json`, and `arrow` files directly from the file browser. You can see
|
||||
this by right clicking one of these files and `Open With->CSVPerspective` (or
|
||||
`JSONPerspective` or `ArrowPerspective`). Perspective will also install itself
|
||||
as the default handler for opening `.arrow` files. -->
|
||||
@@ -0,0 +1,67 @@
|
||||
# Multi-threading
|
||||
|
||||
Perspective's API is thread-safe, so methods may be called from different
|
||||
threads without additional consideration for safety/exclusivity/correctness. All
|
||||
`perspective.Client` and `perspective.Server` API methods release the GIL, which
|
||||
can be exploited for parallelism.
|
||||
|
||||
Interally, `perspective.Server` also dispatches to a thread pool for some
|
||||
operations, enabling better parallelism and overall better query performance.
|
||||
This independent threadpool size can be controlled via
|
||||
`perspective.set_num_cpus()`, or the `OMP_NUM_THREADS` environment variable.
|
||||
|
||||
```python
|
||||
import perspective
|
||||
|
||||
perspective.set_num_cpus(2)
|
||||
```
|
||||
|
||||
## Server handlers
|
||||
|
||||
Perspective's server handler implementations each take an optional `executor`
|
||||
constructor argument, which (when provided) will configure the handler to
|
||||
process WebSocket `Client` requests on a thread pool.
|
||||
|
||||
```python
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from tornado.web import Application
|
||||
from perspective.handlers.tornado import PerspectiveTornadoHandler
|
||||
from perspective import Server
|
||||
|
||||
args = {"perspective_server": Server(), "executor": ThreadPoolExecutor()}
|
||||
|
||||
app = Application(
|
||||
[
|
||||
(r"/websocket", PerspectiveTornadoHandler, args),
|
||||
|
||||
# ...
|
||||
|
||||
]
|
||||
)
|
||||
```
|
||||
|
||||
## `on_poll_request`
|
||||
|
||||
`on_poll_request` is an optional keyword argument for `Server()`, which which
|
||||
can be applied in cases where overlapping `Table.update` calls can be safely
|
||||
deferred.
|
||||
|
||||
When providing a callback function to `on_poll_request`, the `Server` will
|
||||
invoke your callback when there are updates that need to be flushed, after which
|
||||
you must _eventually_ call `Server.poll` (or else no updates will be processed).
|
||||
|
||||
The exact implementation of `on_poll_request` will depend on the context. A
|
||||
simple example which batches calls via `threading.Lock`:
|
||||
|
||||
```python
|
||||
lock = threading.Lock()
|
||||
|
||||
def on_poll_request(perspective_server):
|
||||
if lock.acquire(blocking=False):
|
||||
try:
|
||||
perspective_server.poll()
|
||||
finally:
|
||||
lock.release()
|
||||
|
||||
server = Server(on_poll_request=on_poll_request)
|
||||
```
|
||||
@@ -0,0 +1,90 @@
|
||||
# Loading data into a Table
|
||||
|
||||
A `Table` can be created from a dataset or a schema, the specifics of which are
|
||||
[discussed](#loading-data-with-table) in the JavaScript section of the user's
|
||||
guide. In Python, however, Perspective supports additional data types that are
|
||||
commonly used when processing data:
|
||||
|
||||
- `pandas.DataFrame`
|
||||
- `polars.DataFrame`
|
||||
- `bytes` (encoding an Apache Arrow)
|
||||
- `objects` (either extracting a repr or via reference)
|
||||
- `str` (encoding as a CSV)
|
||||
|
||||
A `Table` is created in a similar fashion to its JavaScript equivalent:
|
||||
|
||||
```python
|
||||
from datetime import date, datetime
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import perspective
|
||||
|
||||
data = pd.DataFrame({
|
||||
"int": np.arange(100),
|
||||
"float": [i * 1.5 for i in range(100)],
|
||||
"bool": [True for i in range(100)],
|
||||
"date": [date.today() for i in range(100)],
|
||||
"datetime": [datetime.now() for i in range(100)],
|
||||
"string": [str(i) for i in range(100)]
|
||||
})
|
||||
|
||||
table = perspective.table(data, index="float")
|
||||
```
|
||||
|
||||
Likewise, a `View` can be created via the `view()` method:
|
||||
|
||||
```python
|
||||
view = table.view(group_by=["float"], filter=[["bool", "==", True]])
|
||||
column_data = view.to_columns()
|
||||
row_data = view.to_json()
|
||||
```
|
||||
|
||||
## Polars Support
|
||||
|
||||
Polars `DataFrame` types work similarly to Apache Arrow input, which Perspective
|
||||
uses to interface with Polars.
|
||||
|
||||
```python
|
||||
df = polars.DataFrame({"a": [1,2,3,4,5]})
|
||||
table = perspective.table(df)
|
||||
```
|
||||
|
||||
## Pandas Support
|
||||
|
||||
Perspective's `Table` can be constructed from `pandas.DataFrame` objects.
|
||||
Internally, this just uses
|
||||
[`pyarrow::from_pandas`](https://arrow.apache.org/docs/python/pandas.html),
|
||||
which dictates behavior of this feature including type support.
|
||||
|
||||
If the dataframe does not have an index set, an integer-typed column named
|
||||
`"index"` is created. If you want to preserve the indexing behavior of the
|
||||
dataframe passed into Perspective, simply create the `Table` with
|
||||
`index="index"` as a keyword argument. This tells Perspective to once again
|
||||
treat the index as a primary key:
|
||||
|
||||
```python
|
||||
data.set_index("datetime")
|
||||
table = perspective.table(data, index="index")
|
||||
```
|
||||
|
||||
## Time Zone Handling
|
||||
|
||||
When parsing `"datetime"` strings, times without an explicit timezone offset are
|
||||
interpreted as _UTC_. Strings with a timezone offset (e.g., `+05:00`) are
|
||||
converted to UTC. All `"datetime"` values are stored internally as milliseconds
|
||||
since the Unix epoch, and are _output_ as integer timestamps (milliseconds since
|
||||
epoch) from methods like `to_columns()` and `to_json()`.
|
||||
|
||||
Python `datetime` objects are serialized to strings before parsing. Naive
|
||||
`datetime` objects (without `tzinfo`) produce strings without timezone
|
||||
information and are therefore treated as UTC. Timezone-aware `datetime` objects
|
||||
include their offset in the serialized string, which is used to convert to UTC.
|
||||
|
||||
`"date"` values are timezone-agnostic calendar days with no time component.
|
||||
They are _output_ as integer timestamps at _UTC midnight_ of the calendar day
|
||||
(equivalent to Arrow `date32` day arithmetic), and integer timestamp _input_ to
|
||||
a `"date"` column is likewise interpreted as UTC. The host process timezone
|
||||
never affects `"date"` values — a `Viewer` renders them in UTC, recovering the
|
||||
stored calendar day exactly. Datetime expression functions such as
|
||||
`bucket("x", 'D')`, `day_of_week("x")` and `hour_of_day("x")` also compute in
|
||||
UTC.
|
||||
@@ -0,0 +1,81 @@
|
||||
# DataFrame and Arrow Compatibility
|
||||
|
||||
`perspective-python` accepts a `Table` constructor argument from any of the
|
||||
common Python columnar data libraries. In all three cases, `perspective.table`
|
||||
(and `Table.update()`) consume the input directly — there is no need to
|
||||
serialize to Apache Arrow IPC bytes yourself. However, note is
|
||||
still the most efficient way to bulk load data into `Table`.
|
||||
|
||||
## PyArrow
|
||||
|
||||
```python
|
||||
import pyarrow as pa
|
||||
import perspective
|
||||
|
||||
arrow_table = pa.table({
|
||||
"int": pa.array([1, 2, 3], type=pa.int64()),
|
||||
"float": pa.array([1.5, 2.5, 3.5], type=pa.float64()),
|
||||
"string": pa.array(["a", "b", "c"], type=pa.string()),
|
||||
})
|
||||
|
||||
table = perspective.table(arrow_table)
|
||||
```
|
||||
|
||||
The same applies to `Table.update()`:
|
||||
|
||||
```python
|
||||
table.update(arrow_table)
|
||||
```
|
||||
|
||||
If you have Arrow data already in IPC format (e.g. read from disk, received
|
||||
over the wire, or produced by another tool), pass the raw `bytes` directly —
|
||||
both stream and file formats are auto-detected:
|
||||
|
||||
```python
|
||||
with open("data.arrow", "rb") as f:
|
||||
table = perspective.table(f.read())
|
||||
```
|
||||
|
||||
## Polars
|
||||
|
||||
```python
|
||||
import polars as pl
|
||||
import perspective
|
||||
|
||||
df = pl.DataFrame({
|
||||
"a": [1, 2, 3, 4, 5],
|
||||
"b": ["x", "y", "z", "x", "y"],
|
||||
})
|
||||
|
||||
table = perspective.table(df)
|
||||
```
|
||||
|
||||
Internally, the `DataFrame` is converted to a `pyarrow.Table` before
|
||||
ingestion, so Polars columns inherit the Arrow type mapping above.
|
||||
|
||||
See also Perspective [Virtual Server support for `polars.DataFrame`](./virtual_server/polars.md)
|
||||
|
||||
## Pandas
|
||||
|
||||
`pandas.DataFrame` is supported via `pyarrow.Table.from_pandas`, which
|
||||
dictates behavior including type support — see the
|
||||
[pyarrow pandas docs](https://arrow.apache.org/docs/python/pandas.html) for
|
||||
details on which pandas dtypes round-trip cleanly.
|
||||
|
||||
```python
|
||||
from datetime import date, datetime
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import perspective
|
||||
|
||||
data = pd.DataFrame({
|
||||
"int": np.arange(100),
|
||||
"float": [i * 1.5 for i in range(100)],
|
||||
"bool": [True for i in range(100)],
|
||||
"date": [date.today() for i in range(100)],
|
||||
"datetime": [datetime.now() for i in range(100)],
|
||||
"string": [str(i) for i in range(100)],
|
||||
})
|
||||
|
||||
table = perspective.table(data, index="float")
|
||||
```
|
||||
@@ -0,0 +1,20 @@
|
||||
# Virtual Servers
|
||||
|
||||
Perspective's Virtual Server feature lets you connect `<perspective-viewer>` to
|
||||
external data sources without loading data into Perspective's built-in engine.
|
||||
Instead, queries are translated and executed natively by the external database.
|
||||
|
||||
For a detailed explanation of how virtual servers work, see the
|
||||
[Virtual Servers](../../explanation/virtual_servers.md) concepts page.
|
||||
|
||||
Perspective ships with built-in virtual server implementations for:
|
||||
|
||||
- [**DuckDB**](./virtual_server/duckdb.md) — query DuckDB databases using the
|
||||
`duckdb` Python package.
|
||||
- [**ClickHouse**](./virtual_server/clickhouse.md) — query a ClickHouse server
|
||||
using the `clickhouse-connect` Python package.
|
||||
- [**Polars**](./virtual_server/polars.md) — query in-memory Polars DataFrames
|
||||
using the `polars` Python package.
|
||||
|
||||
You can also [**implement your own**](./virtual_server/custom.md) virtual server
|
||||
to connect Perspective to any data source by subclassing `VirtualServerHandler`.
|
||||
@@ -0,0 +1,52 @@
|
||||
# ClickHouse Virtual Server
|
||||
|
||||
Perspective provides a built-in virtual server for
|
||||
[ClickHouse](https://clickhouse.com/), allowing `<perspective-viewer>` clients
|
||||
to query a ClickHouse server over WebSocket.
|
||||
|
||||
For browser-only usage, see the
|
||||
[JavaScript ClickHouse guide](../../javascript/virtual_server/clickhouse.md).
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install perspective-python clickhouse-connect
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
Create a server that exposes ClickHouse tables to browser clients:
|
||||
|
||||
```python
|
||||
import clickhouse_connect
|
||||
import tornado.web
|
||||
import tornado.ioloop
|
||||
from perspective import ClickhouseVirtualServer
|
||||
from perspective.handlers.tornado import PerspectiveTornadoHandler
|
||||
|
||||
# Connect to ClickHouse
|
||||
client = clickhouse_connect.get_client(host="localhost")
|
||||
|
||||
# Create virtual server backed by ClickHouse
|
||||
server = ClickhouseVirtualServer(client)
|
||||
|
||||
# Serve over WebSocket
|
||||
app = tornado.web.Application([
|
||||
(r"/websocket", PerspectiveTornadoHandler, {"perspective_server": server}),
|
||||
])
|
||||
|
||||
app.listen(8080)
|
||||
tornado.ioloop.IOLoop.current().start()
|
||||
```
|
||||
|
||||
Connect from the browser:
|
||||
|
||||
```javascript
|
||||
const websocket = await perspective.websocket("ws://localhost:8080/websocket");
|
||||
const table = await websocket.open_table("my_table");
|
||||
document.getElementById("viewer").load(table);
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
- [Python ClickHouse example](https://github.com/perspective-dev/perspective/tree/master/examples/python-clickhouse-virtual)
|
||||
@@ -0,0 +1,64 @@
|
||||
# Implementing a custom Virtual Server
|
||||
|
||||
You can connect Perspective to any data source by subclassing
|
||||
`VirtualServerHandler` and wrapping it with `VirtualServer`.
|
||||
|
||||
For background on virtual servers, see the
|
||||
[Virtual Servers overview](../../../explanation/virtual_servers.md).
|
||||
|
||||
## Example
|
||||
|
||||
```python
|
||||
from perspective import VirtualServerHandler, VirtualServer
|
||||
|
||||
class MyModel(VirtualServerHandler):
|
||||
def get_features(self):
|
||||
return {
|
||||
"group_by": True,
|
||||
"split_by": False,
|
||||
"sort": True,
|
||||
"filter_ops": {
|
||||
"string": ["==", "!=", "contains"],
|
||||
"float": ["==", "!=", ">", "<"],
|
||||
},
|
||||
"aggregates": {
|
||||
"float": ["sum", "avg", "count"],
|
||||
"string": ["count"],
|
||||
},
|
||||
}
|
||||
|
||||
def get_hosted_tables(self):
|
||||
return ["my_table"]
|
||||
|
||||
def table_schema(self, table_name):
|
||||
return {"name": "string", "price": "float"}
|
||||
|
||||
def table_size(self, table_name):
|
||||
return 1000
|
||||
|
||||
def table_make_view(self, table_name, view_id, config):
|
||||
# Translate `config` (group_by, sort, filter, etc.) into a
|
||||
# query against your data source. Store the query keyed by
|
||||
# `view_id` for later data retrieval.
|
||||
pass
|
||||
|
||||
def view_delete(self, view_id):
|
||||
# Clean up resources for this view
|
||||
pass
|
||||
|
||||
def view_get_data(self, view_id, start_row, end_row, start_col, end_col, ctx):
|
||||
# Execute the stored query with the given row/column window.
|
||||
# Push results via `ctx`.
|
||||
pass
|
||||
```
|
||||
|
||||
The `VirtualServer` instance can then be passed to a Tornado, Starlette, or
|
||||
AIOHTTP handler just like a regular `Server`:
|
||||
|
||||
```python
|
||||
from perspective.handlers.tornado import PerspectiveTornadoHandler
|
||||
|
||||
app = tornado.web.Application([
|
||||
(r"/websocket", PerspectiveTornadoHandler, {"perspective_server": VirtualServer(MyModel)}),
|
||||
])
|
||||
```
|
||||
@@ -0,0 +1,53 @@
|
||||
# DuckDB Virtual Server
|
||||
|
||||
Perspective provides a built-in virtual server for
|
||||
[DuckDB](https://duckdb.org/), allowing `<perspective-viewer>` clients to query
|
||||
a server-side DuckDB database over WebSocket.
|
||||
|
||||
For browser-only usage via DuckDB-WASM, see the
|
||||
[JavaScript DuckDB guide](../../javascript/virtual_server/duckdb.md).
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install perspective-python duckdb
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
Create a server that exposes a DuckDB database to browser clients:
|
||||
|
||||
```python
|
||||
import duckdb
|
||||
import tornado.web
|
||||
import tornado.ioloop
|
||||
from perspective import DuckDBVirtualServer
|
||||
from perspective.handlers.tornado import PerspectiveTornadoHandler
|
||||
|
||||
# Create DuckDB connection and load data
|
||||
conn = duckdb.connect()
|
||||
conn.execute("CREATE TABLE my_table AS SELECT * FROM 'data.parquet'")
|
||||
|
||||
# Create virtual server backed by DuckDB
|
||||
server = DuckDBVirtualServer(conn)
|
||||
|
||||
# Serve over WebSocket
|
||||
app = tornado.web.Application([
|
||||
(r"/websocket", PerspectiveTornadoHandler, {"perspective_server": server}),
|
||||
])
|
||||
|
||||
app.listen(8080)
|
||||
tornado.ioloop.IOLoop.current().start()
|
||||
```
|
||||
|
||||
Connect from the browser:
|
||||
|
||||
```javascript
|
||||
const websocket = await perspective.websocket("ws://localhost:8080/websocket");
|
||||
const table = await websocket.open_table("my_table");
|
||||
document.getElementById("viewer").load(table);
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
- [Python DuckDB example](https://github.com/perspective-dev/perspective/tree/master/examples/python-duckdb-virtual)
|
||||
@@ -0,0 +1,49 @@
|
||||
# Polars Virtual Server
|
||||
|
||||
Perspective provides a built-in virtual server for
|
||||
[Polars](https://pola.rs/), allowing `<perspective-viewer>` clients to query
|
||||
in-memory Polars DataFrames over WebSocket.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install perspective-python polars
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
Create a server that exposes Polars DataFrames to browser clients:
|
||||
|
||||
```python
|
||||
import polars as pl
|
||||
import tornado.web
|
||||
import tornado.ioloop
|
||||
from perspective.virtual_servers.polars import PolarsVirtualServer
|
||||
from perspective.handlers.tornado import PerspectiveTornadoHandler
|
||||
|
||||
# Load data into Polars DataFrames
|
||||
df = pl.read_parquet("data.parquet")
|
||||
|
||||
# Create virtual server backed by Polars (dict of name -> DataFrame)
|
||||
server = PolarsVirtualServer({"my_table": df})
|
||||
|
||||
# Serve over WebSocket
|
||||
app = tornado.web.Application([
|
||||
(r"/websocket", PerspectiveTornadoHandler, {"perspective_server": server}),
|
||||
])
|
||||
|
||||
app.listen(8080)
|
||||
tornado.ioloop.IOLoop.current().start()
|
||||
```
|
||||
|
||||
Connect from the browser:
|
||||
|
||||
```javascript
|
||||
const websocket = await perspective.websocket("ws://localhost:8080/websocket");
|
||||
const table = await websocket.open_table("my_table");
|
||||
document.getElementById("viewer").load(table);
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
- [Python Polars example](https://github.com/perspective-dev/perspective/tree/master/examples/python-polars-virtual)
|
||||
@@ -0,0 +1,129 @@
|
||||
# Hosting a WebSocket server
|
||||
|
||||
An in-memory `Server` "hosts" all `perspective.Table` and `perspective.View`
|
||||
instances created by its connected `Client`s. Hosted tables/views can have their
|
||||
methods called from other sources than the Python server, i.e. by a
|
||||
`perspective-viewer` running in a JavaScript client over the network,
|
||||
interfacing with `perspective-python` through the websocket API.
|
||||
|
||||
The server has full control of all hosted `Table` and `View` instances, and can
|
||||
call any public API method on hosted instances. This makes it extremely easy to
|
||||
stream data to a hosted `Table` using `.update()`:
|
||||
|
||||
```python
|
||||
server = perspective.Server()
|
||||
client = server.new_local_client()
|
||||
table = client.table(data, name="data_source")
|
||||
|
||||
for i in range(10):
|
||||
# updates continue to propagate automatically
|
||||
table.update(new_data)
|
||||
```
|
||||
|
||||
The `name` provided is important, as it enables Perspective in JavaScript to
|
||||
look up a `Table` and get a handle to it over the network. Otherwise, `name`
|
||||
will be assigned randomly and the `Client` must look this up with
|
||||
`Client.get_hosted_table_names()`
|
||||
|
||||
## Client/Server Replicated Mode
|
||||
|
||||
Using Tornado and
|
||||
[`PerspectiveTornadoHandler`](python.md#perspectivetornadohandler), as well as
|
||||
`Perspective`'s JavaScript library, we can set up "distributed" Perspective
|
||||
instances that allows multiple browser `perspective-viewer` clients to read from
|
||||
a common `perspective-python` server, as in the
|
||||
[Tornado Example Project](https://github.com/perspective-dev/perspective/tree/master/examples/python-tornado).
|
||||
|
||||
This architecture works by maintaining two `Tables`—one on the server, and one
|
||||
on the client that mirrors the server's `Table` automatically using `on_update`.
|
||||
All updates to the table on the server are automatically applied to each client,
|
||||
which makes this architecture a natural fit for streaming dashboards and other
|
||||
distributed use-cases. In conjunction with [multithreading](#multi-threading),
|
||||
distributed Perspective offers consistently high performance over large numbers
|
||||
of clients and large datasets.
|
||||
|
||||
_*server.py*_
|
||||
|
||||
```python
|
||||
from perspective import Server
|
||||
from perspective.handlers.tornado import PerspectiveTornadoHandler
|
||||
|
||||
# Create an instance of Server, and host a Table
|
||||
SERVER = Server()
|
||||
CLIENT = SERVER.new_local_client()
|
||||
|
||||
# The Table is exposed at `localhost:8888/websocket` with the name `data_source`
|
||||
client.table(data, name = "data_source")
|
||||
|
||||
app = tornado.web.Application([
|
||||
# create a websocket endpoint that the client JavaScript can access
|
||||
(r"/websocket", PerspectiveTornadoHandler, {"perspective_server": SERVER})
|
||||
])
|
||||
|
||||
# Start the Tornado server
|
||||
app.listen(8888)
|
||||
loop = tornado.ioloop.IOLoop.current()
|
||||
loop.start()
|
||||
```
|
||||
|
||||
Instead of calling `load(server_table)`, create a `View` using `server_table`
|
||||
and pass that into `viewer.load()`. This will automatically register an
|
||||
`on_update` callback that synchronizes state between the server and the client.
|
||||
|
||||
_*index.html*_
|
||||
|
||||
```html
|
||||
<perspective-viewer id="viewer" editable></perspective-viewer>
|
||||
|
||||
<script type="module">
|
||||
// Create a client that expects a Perspective server
|
||||
// to accept connections at the specified URL.
|
||||
const websocket = await perspective.websocket(
|
||||
"ws://localhost:8888/websocket",
|
||||
);
|
||||
|
||||
// Get a handle to the Table on the server
|
||||
const server_table = await websocket.open_table("data_source_one");
|
||||
|
||||
// Create a new view
|
||||
const server_view = await table.view();
|
||||
|
||||
// Create a Table on the client using `perspective.worker()`
|
||||
const worker = await perspective.worker();
|
||||
const client_table = await worker.table(view);
|
||||
|
||||
// Load the client table in the `<perspective-viewer>`.
|
||||
document.getElementById("viewer").load(client_table);
|
||||
</script>
|
||||
```
|
||||
|
||||
For a more complex example that offers distributed editing of the server
|
||||
dataset, see
|
||||
[client_server_editing.html](https://github.com/perspective-dev/perspective/blob/master/examples/python-tornado/client_server_editing.html).
|
||||
|
||||
We also provide examples for Starlette/FastAPI and AIOHTTP:
|
||||
|
||||
- [Starlette Example Project](https://github.com/perspective-dev/perspective/tree/master/examples/python-starlette).
|
||||
- [AIOHTTP Example Project](https://github.com/perspective-dev/perspective/tree/master/examples/python-aiohttp).
|
||||
|
||||
## Server-only Mode
|
||||
|
||||
The server setup is identical to
|
||||
[Client/Server Replicated Mode](#client-server-replicated-mode) above, but
|
||||
instead of creating a `View`, the client calls `load(server_table)`: In Python,
|
||||
use `Server` and `PerspectiveTornadoHandler` to create a websocket server that
|
||||
exposes a `Table`. In this example, `table` is a proxy for the `Table` we
|
||||
created on the server. All API methods are available on _proxies_, e.g.
|
||||
calling `view()`, `schema()`, `update()` on `table` will pass those operations
|
||||
to the Python `Table`, execute the commands, and return the result back to
|
||||
Javascript.
|
||||
|
||||
```html
|
||||
<perspective-viewer id="viewer" editable></perspective-viewer>
|
||||
```
|
||||
|
||||
```javascript
|
||||
const websocket = perspective.websocket("ws://localhost:8888/websocket");
|
||||
const table = websocket.open_table("data_source");
|
||||
document.getElementById("viewer").load(table);
|
||||
```
|
||||
Reference in New Issue
Block a user