wehub-resource-sync
12 KiB
The Python language bindings for Perspective, a high performance data-visualization and analytics component for the web browser.
perspective docs for the Rust API.
A simple example which loads an Apache Arrow and computes a "Group By" operation, returning a new Arrow.
from perspective import Server
client = Server().new_local_client()
table = client.table(arrow_bytes_data)
view = table.view(group_by = ["CounterParty", "Security"])
arrow = view.to_arrow()
Perspective for Python uses the exact same C++ data engine used by the WebAssembly version and Rust version. The library consists of many of the same abstractions and API as in JavaScript, as well as Python-specific data loading support for NumPy, Pandas (and Apache Arrow, as in JavaScript).
Additionally, perspective-python provides a session manager suitable for
integration into server systems such as
Tornado websockets,
AIOHTTP, or
Starlette/FastAPI,
which allows fully virtual Perspective tables to be interacted with by
multiple <perspective-viewer> in a web browser. You can also interact with a
Perspective table from python clients, and to that end client libraries are
implemented for both Tornado and AIOHTTP.
As <perspective-viewer> will only consume the data necessary to render the
current screen, this runtime mode allows ludicrously-sized datasets with
instant-load after they've been manifest on the server (at the expense of
network latency on UI interaction).
The included PerspectiveWidget allows running such a viewer in
JupyterLab in either server or
client (via WebAssembly) mode, and the included PerspectiveTornadoHandler
makes it simple to extend a Tornado server with virtual Perspective support.
The perspective module exports several tools:
Serverthe constructor for a new isntance of the Perspective data engine.- The
perspective.widgetmodule exportsPerspectiveWidget, the JupyterLab widget for interactive visualization in a notebook cell. - The
perspective.handlersmodules exports web frameworks handlers that interface with aperspective-clientin JavaScript.
This user's guide provides an overview of the most common ways to use
Perspective in Python: the Table API, the JupyterLab widget, and the Tornado
handler.
More Examples are available on GitHub.
Installation
perspective-python contains full bindings to the Perspective API, a JupyterLab
widget, and a WebSocket handlers for several webserver libraries that allow you
to host Perspective using server-side Python.
PyPI
perspective-python can be installed from PyPI via pip:
pip install perspective-python
Table
A Table can be created from a dataset or a schema, the specifics of which are
discussed in the JavaScript section of the user's
guide. In Python, however, Perspective supports additional data types that are
commonly used when processing data:
pandas.DataFramepolars.DataFramebytes(encoding an Apache Arrow)objects(either extracting a repr or via reference)str(encoding as a CSV)
A Table is created in a similar fashion to its JavaScript equivalent:
from datetime import date, datetime
import numpy as np
import pandas as pd
import perspective
data = pd.DataFrame({
"int": np.arange(100),
"float": [i * 1.5 for i in range(100)],
"bool": [True for i in range(100)],
"date": [date.today() for i in range(100)],
"datetime": [datetime.now() for i in range(100)],
"string": [str(i) for i in range(100)]
})
table = perspective.table(data, index="float")
Likewise, a View can be created via the view() method:
view = table.view(group_by=["float"], filter=[["bool", "==", True]])
column_data = view.to_columns()
row_data = view.to_json()
Pandas and Polars Support
Perspective's Table can be constructed from pandas.DataFrame and
polars.DataFrame objects. Internally, this just uses
pyarrow::from_pandas,
which dictates behavior of this feature including type support.
If the dataframe does not have an index set, an integer-typed column named
"index" is created. If you want to preserve the indexing behavior of the
dataframe passed into Perspective, simply create the Table with
index="index" as a keyword argument. This tells Perspective to once again
treat the index as a primary key:
data.set_index("datetime")
table = perspective.table(data, index="index")
Time Zone Handling
When parsing "datetime" strings, times without an explicit timezone offset are
interpreted as UTC; strings with an offset are converted to UTC. All
"datetime" values are stored internally as milliseconds since the Unix epoch
and are output as integer timestamps from methods like to_columns() and
to_json(). "date" values are timezone-agnostic calendar days, output as
integer timestamps at UTC midnight of the calendar day. The host process
timezone never affects engine results; localization is the display layer's
responsibility.
Callbacks and Events
perspective.Table allows for on_update and on_delete callbacks to be
set—simply call on_update or on_delete with a reference to a function or a
lambda without any parameters:
def update_callback():
print("Updated!")
# set the update callback
on_update_id = view.on_update(update_callback)
def delete_callback():
print("Deleted!")
# set the delete callback
on_delete_id = view.on_delete(delete_callback)
# set a lambda as a callback
view.on_delete(lambda: print("Deleted x2!"))
If the callback is a named reference to a function, it can be removed with
remove_update or remove_delete:
view.remove_update(on_update_id)
view.remove_delete(on_delete_id)
Callbacks defined with a lambda function cannot be removed, as lambda functions have no identifier.
Hosting Table and View instances
Server "hosts" all perspective.Table and perspective.View instances
created by its connected Clients. Hosted tables/views can have their methods
called from other sources than the Python server, i.e. by a perspective-viewer
running in a JavaScript client over the network, interfacing with
perspective-python through the websocket API.
The server has full control of all hosted Table and View instances, and can
call any public API method on hosted instances. This makes it extremely easy to
stream data to a hosted Table using .update():
server = perspective.Server()
client = server.new_local_client()
table = client.table(data, name="data_source")
for i in range(10):
# updates continue to propagate automatically
table.update(new_data)
The name provided is important, as it enables Perspective in JavaScript to
look up a Table and get a handle to it over the network. Otherwise, name
will be assigned randomlu and the Client must look this up with
CLient.get_hosted_table_names()
Client/Server Replicated Mode
Using Tornado and
PerspectiveTornadoHandler, as well as
Perspective's JavaScript library, we can set up "distributed" Perspective
instances that allows multiple browser perspective-viewer clients to read from
a common perspective-python server, as in the
Tornado Example Project.
This architecture works by maintaining two Tables—one on the server, and one
on the client that mirrors the server's Table automatically using on_update.
All updates to the table on the server are automatically applied to each client,
which makes this architecture a natural fit for streaming dashboards and other
distributed use-cases. In conjunction with multithreading,
distributed Perspective offers consistently high performance over large numbers
of clients and large datasets.
server.py
from perspective import Server
from perspective.hadnlers.tornado import PerspectiveTornadoHandler
# Create an instance of Server, and host a Table
SERVER = Server()
CLIENT = SERVER.new_local_client()
# The Table is exposed at `localhost:8888/websocket` with the name `data_source`
client.table(data, name = "data_source")
app = tornado.web.Application([
# create a websocket endpoint that the client JavaScript can access
(r"/websocket", PerspectiveTornadoHandler, {"perspective_server": SERVER})
])
# Start the Tornado server
app.listen(8888)
loop = tornado.ioloop.IOLoop.current()
loop.start()
Instead of calling load(server_table), create a View using server_table
and pass that into viewer.load(). This will automatically register an
on_update callback that synchronizes state between the server and the client.
index.html
<perspective-viewer id="viewer" editable></perspective-viewer>
<script type="module">
// Create a client that expects a Perspective server
// to accept connections at the specified URL.
const websocket = await perspective.websocket(
"ws://localhost:8888/websocket",
);
// Get a handle to the Table on the server
const server_table = await websocket.open_table("data_source_one");
// Create a new view
const server_view = await table.view();
// Create a Table on the client using `perspective.worker()`
const worker = await perspective.worker();
const client_table = await worker.table(view);
// Load the client table in the `<perspective-viewer>`.
document.getElementById("viewer").load(client_table);
</script>
For a more complex example that offers distributed editing of the server dataset, see client_server_editing.html.
We also provide examples for Starlette/FastAPI and AIOHTTP:
Server-only Mode
The server setup is identical to Distributed Mode above,
but instead of creating a view, the client calls load(server_table): In
Python, use Server and PerspectiveTornadoHandler to create a websocket
server that exposes a Table. In this example, table is a proxy for the
Table we created on the server. All API methods are available on proxies,
the.g.us calling view(), schema(), update() on table will pass those
operations to the Python Table, execute the commands, and return the result
back to Javascript.
<perspective-viewer id="viewer" editable></perspective-viewer>
const websocket = perspective.websocket("ws://localhost:8888/websocket");
const table = websocket.open_table("data_source");
document.getElementById("viewer").load(table);