chore: import upstream snapshot with attribution

This commit is contained in:
wehub-resource-sync
2026-07-13 12:25:07 +08:00
commit a26e856398
1681 changed files with 296950 additions and 0 deletions
+231
View File
@@ -0,0 +1,231 @@
# Advanced View Operations
Beyond the standard query configuration, `View` provides additional methods for
interacting with hierarchical results and introspecting data.
## Tree Hierarchy Operations
When a `View` has `group_by` applied, the results form a tree hierarchy.
Perspective provides methods to control which levels of the tree are expanded or
collapsed:
<div class="javascript">
```javascript
const view = await table.view({ group_by: ["Region", "Country", "City"] });
// Collapse the tree at row index 5
await view.collapse(5);
// Expand the tree at row index 5
await view.expand(5);
// Set the expansion depth (0 = fully collapsed, 1 = first level, etc.)
await view.set_depth(1);
```
</div>
<div class="python">
Using the sync API
```python
view = table.view(group_by=["Region", "Country", "City"])
view.collapse(5)
view.expand(5)
view.set_depth(1)
```
</div>
<div class="rust">
```rust
let view = table.view(Some(ViewConfigUpdate {
group_by: Some(vec!["Region".into(), "Country".into(), "City".into()]),
..ViewConfigUpdate::default()
})).await?;
view.collapse(5).await?;
view.expand(5).await?;
view.set_depth(1).await?;
```
</div>
<span class="warning">Perspective's built-in engine is lazy — aggregates for
collapsed rows are not recalculated when the underlying `Table` is updated.
Updates are only computed for rows that are currently visible (expanded). When a
collapsed row is later expanded, its aggregates are calculated at that
point.</span>
## Column Range Queries
`View::get_min_max` returns the minimum and maximum values for a given column,
which is useful for setting up scales in custom visualizations:
<div class="javascript">
```javascript
const [min, max] = await view.get_min_max("Sales");
```
</div>
<div class="python">
```python
min_val, max_val = view.get_min_max("Sales")
```
</div>
## Expression Validation
Before creating a `View` with expressions, you can validate them against the
table's schema using `Table::validate_expressions`. This returns information
about which expressions are valid and their inferred types:
<div class="javascript">
```javascript
const result = await table.validate_expressions({
expr1: '"Sales" + "Profit"',
expr2: "invalid_column + 1",
});
// result.expression_schema contains valid expressions and their types
// result.errors contains invalid expressions and error messages
```
</div>
<div class="python">
```python
result = table.validate_expressions(['"Sales" + "Profit"', 'invalid + 1'])
```
</div>
## View Dimensions
`View::dimensions` returns the number of rows and columns in the current view,
including information about group-by header rows:
<div class="javascript">
```javascript
const dims = await view.dimensions();
// { num_view_rows, num_view_columns, num_table_rows, num_table_columns, ... }
```
</div>
<div class="python">
```python
dims = view.dimensions()
```
</div>
## View Configuration Introspection
`View::get_config` returns the full configuration used to create the view:
<div class="javascript">
```javascript
const config = await view.get_config();
// { group_by: [...], split_by: [...], sort: [...], filter: [...], ... }
```
</div>
<div class="python">
```python
config = view.get_config()
```
</div>
## Update Callbacks
Register a callback to be notified whenever the underlying `Table` is updated
and the `View` has been recalculated:
<div class="javascript">
```javascript
view.on_update(
(updated) => {
console.log("View updated", updated.port_id);
},
{ mode: "row" },
);
// Later, remove the callback
view.remove_update(callback);
```
</div>
<div class="python">
```python
def on_update(port_id, delta):
print("View updated", port_id)
view.on_update(on_update, mode="row")
view.remove_update(on_update)
```
</div>
When `mode` is set to `"row"`, the callback receives a delta of only the rows
that changed (as Apache Arrow), which is useful for efficiently synchronizing
tables across clients.
## Flattening a View into a Table
In Javascript, a [`Table`] can be constructed on a [`Table::view`] instance,
which will return a new [`Table`] based on the [`Table::view`]'s dataset, and
all future updates that affect the [`Table::view`] will be forwarded to the new
[`Table`]. This is particularly useful for implementing a
[Client/Server Replicated](server.md#clientserver-replicated) design, by
serializing the `View` to an arrow and setting up an `on_update` callback.
<div class="javascript">
```javascript
const worker1 = perspective.worker();
const table = await worker.table(data);
const view = await table.view({ filter: [["State", "==", "Texas"]] });
const table2 = await worker.table(view);
table.update([{ State: "Texas", City: "Austin" }]);
```
</div>
<div class="python">
```python
table = perspective.Table(data);
view = table.view(filter=[["State", "==", "Texas"]])
table2 = perspective.Table(view.to_arrow());
def updater(port, delta):
table2.update(delta)
view.on_update(updater, mode="Row")
table.update([{"State": "Texas", "City": "Austin"}])
```
</div>
<div class="rust">
```rust
let opts = TableInitOptions::default();
let data = TableData::Update(UpdateData::Csv("x,y\n1,2\n3,4".into()));
let table = client.table(data, opts).await?;
let view = table.view(None).await?;
let table2 = client.table(TableData::View(view)).await?;
table.update(data).await?;
```
</div>
@@ -0,0 +1,149 @@
# Expressions
The `expressions` property specifies _new_ columns in Perspective that are
created using existing column values or arbitrary scalar values defined within
the expression. In `<perspective-viewer>`, expressions are added using the "New
Column" button in the side panel.
Expressions are strings parsed by Perspective's expression engine (based on
[ExprTK](https://github.com/ArashPartow/exprtk)). Column names are referenced by
wrapping them in double quotes, e.g. `"Sales"`:
<div class="javascript">
```javascript
const view = await table.view({
expressions: {
"Profit Ratio": '"Profit" / "Sales"',
},
});
```
</div>
<div class="python">
```python
view = table.view(expressions={'Profit Ratio': '"Profit" / "Sales"'})
```
</div>
<div class="rust">
```rust
let view = table.view(Some(ViewConfigUpdate {
expressions: Some(Expressions([
("Profit Ratio", "\"Profit\" / \"Sales\"".into())
].into_iter().collect())),
..ViewConfigUpdate::default()
})).await?;
```
</div>
## Type Conversion and Coercion
Perspective expressions are strongly typed — each column and literal has a fixed
type, and most operators require matching types on both sides. To work across
types, use the conversion functions:
| Function | Description |
| --------------- | ------------------------------------------------------------ |
| `to_string(x)` | Convert any type to string |
| `to_integer(x)` | Convert to integer (null if not parsable) |
| `to_float(x)` | Convert to float (null if not parsable) |
| `to_boolean(x)` | Convert to boolean (truthy/falsy) |
| `integer(x)` | Alias for `to_integer(x)` |
| `float(x)` | Alias for `to_float(x)` |
| `datetime(x)` | Construct a datetime from a POSIX timestamp (ms since epoch) |
| `date(y, m, d)` | Construct a date from year, month, day |
### How coercion works
Perspective does not implicitly coerce types. For example, you cannot directly
add an `integer` to a `float` — you must cast one side explicitly. Similarly,
`datetime` and `date` values are not numeric: to perform arithmetic on them, you
must first convert to a numeric representation, do the math, then convert back.
Internally, `datetime` values are stored as milliseconds since the Unix epoch
(1970-01-01T00:00:00Z). Converting a `datetime` to a `float` yields this
millisecond timestamp, and `datetime()` accepts a millisecond timestamp to
produce a `datetime`.
### Example: offsetting a datetime by 7 days
This expression takes a `"Shipped Date"` column, converts it to its
millisecond-epoch representation, adds 7 days worth of milliseconds (7 &times;
24 &times; 60 &times; 60 &times; 1000 = 604800000), and converts the result back
to a `datetime`:
```
// Due Date
datetime(float("Shipped Date") + 604800000)
```
## Operators
Standard arithmetic and comparison operators are supported:
| Operator | Description |
| -------------------------------- | ----------- |
| `+`, `-`, `*`, `/` | Arithmetic |
| `%` | Modulo |
| `==`, `!=`, `<`, `>`, `<=`, `>=` | Comparison |
| `and`, `or`, `not` | Logical |
| `if ... else ...` | Conditional |
## Numeric Functions
ExprTK provides a rich set of built-in numeric functions including `abs`,
`ceil`, `floor`, `round`, `exp`, `log`, `log10`, `sqrt`, `min`, `max`, `pow`,
`clamp`, `iclamp`, `inrange`, and trigonometric functions (`sin`, `cos`, `tan`,
`asin`, `acos`, `atan`).
## String Functions
| Function | Description |
| ------------------------------- | ------------------------------------------------------- |
| `concat(a, b, ...)` | Concatenate strings |
| `upper(s)` | Convert to uppercase |
| `lower(s)` | Convert to lowercase |
| `length(s)` | String length |
| `contains(s, substr)` | Whether `s` contains `substr` |
| `order(col, 'B', 'C', 'A')` | Custom sort order for a string column |
| `match(s, pattern)` | Regex partial match (returns boolean) |
| `match_all(s, pattern)` | Regex full match (returns boolean) |
| `search(s, pattern)` | First capturing group match |
| `indexof(s, pattern)` | Start index of first regex match |
| `substring(s, start, end)` | Substring from `start` (inclusive) to `end` (exclusive) |
| `replace(s, repl, pattern)` | Replace first regex match |
| `replace_all(s, repl, pattern)` | Replace all regex matches |
## Date/Datetime Functions
| Function | Description |
| ------------------------ | ------------------------------------------------------------------------ |
| `today()` | Current date |
| `now()` | Current datetime |
| `date(year, month, day)` | Construct a date |
| `datetime(timestamp_ms)` | Construct a datetime from a POSIX timestamp (ms since epoch) |
| `hour_of_day(dt)` | Hour component (0-23) |
| `day_of_week(dt)` | Day of the week as a string |
| `month_of_year(dt)` | Month of the year as a string |
| `bucket(dt, unit)` | Bucket datetime by unit: `'s'`, `'m'`, `'h'`, `'D'`, `'W'`, `'M'`, `'Y'` |
`bucket` also works on numeric columns: `bucket("Price", 10)` rounds values down
to the nearest multiple of 10.
## Other Functions
| Function | Description |
| ------------------------- | ----------------------------------------------------- |
| `is_null(x)` | Whether the value is null |
| `is_not_null(x)` | Whether the value is not null |
| `percent_of(a, b)` | `a` as a percentage of `b` |
| `inrange(low, val, high)` | Whether `val` is between `low` and `high` (inclusive) |
| `min(a, b, ...)` | Minimum of inputs |
| `max(a, b, ...)` | Maximum of inputs |
| `random()` | Random float between 0.0 and 1.0 |
| `col(name)` | Look up a column by string name at runtime |
| `vlookup(col, key)` | Look up a value in another column by row key |
@@ -0,0 +1,147 @@
# Grouping and Pivots
## Group By
A group by _groups_ the dataset by the unique values of each column used as a
group by - a close analogue in SQL to the `GROUP BY` statement. The underlying
dataset is aggregated to show the values belonging to each group, and a total
row is calculated for each group, showing the currently selected aggregated
value (e.g. `sum`) of the column. Group by are useful for hierarchies,
categorizing data and attributing values, i.e. showing the number of units sold
based on State and City. In Perspective, group by are represented as an array of
string column names to pivot, are applied in the order provided; For example, a
group by of `["State", "City", "Postal Code"]` shows the values for each Postal
Code, which are grouped by City, which are in turn grouped by State.
<div class="javascript">
```javascript
const view = await table.view({ group_by: ["a", "c"] });
```
</div>
<div class="python">
```python
view = table.view(group_by=["a", "c"])
```
</div>
<div class="rust">
```rust
let view = table.view(Some(ViewConfigUpdate {
group_by: Some(vec!["a".into(), "c".into()]),
..ViewConfigUpdate::default()
})).await?;
```
</div>
## Split By
A split by _splits_ the dataset by the unique values of each column used as a
split by. The underlying dataset is not aggregated, and a new column is created
for each unique value of the split by. Each newly created column contains the
parts of the dataset that correspond to the column header, i.e. a `View` that
has `["State"]` as its split by will have a new column for each state. In
Perspective, Split By are represented as an array of string column names to
pivot:
<div class="javascript">
```javascript
const view = await table.view({ split_by: ["a", "c"] });
```
</div>
<div class="python">
```python
view = table.view(split_by=["a", "c"])
```
</div>
<div class="rust">
```rust
let view = table.view(Some(ViewConfigUpdate {
split_by: Some(vec!["a".into(), "c".into()]),
..ViewConfigUpdate::default()
})).await?;
```
</div>
## Aggregates
Aggregates perform a calculation over an entire column, and are displayed when
one or more [Group By](#group-by) are applied to the `View`. Aggregates can be
specified by the user, or Perspective will use the following sensible default
aggregates based on column type:
- "sum" for `integer` and `float` columns
- "count" for all other columns
Perspective provides a selection of aggregate functions that can be applied to
columns in the `View` constructor using a dictionary of column name to aggregate
function name.
<div class="javascript">
```javascript
const view = await table.view({
aggregates: {
a: "avg",
b: "distinct count",
},
});
```
</div>
<div class="python">
```python
view = table.view(
aggregates={
"a": "avg",
"b": "distinct count"
}
)
```
</div>
<div class="rust">
```rust
use std::collections::HashMap;
let view = table.view(Some(ViewConfigUpdate {
aggregates: Some(HashMap::from([
("a".into(), "avg".into()),
("b".into(), "distinct count".into()),
])),
..ViewConfigUpdate::default()
})).await?;
```
</div>
The available aggregate functions depend on the column type:
**Numeric columns** (`integer`, `float`): `sum`, `abs sum`, `sum abs`,
`sum not null`, `any`, `avg`, `mean`, `count`, `distinct count`, `dominant`,
`first`, `last`, `last by index`, `high`, `low`, `max`, `min`,
`high minus low`, `last minus first`, `median`, `q1`, `q3`,
`pct sum parent`, `pct sum total`, `stddev`, `var`, `unique`,
`weighted mean`, `min by`, `max by`.
**String columns**: `count`, `any`, `distinct count`, `dominant`, `first`,
`last`, `last by index`, `join`, `median`, `q1`, `q3`, `unique`, `min by`,
`max by`.
**Date/Datetime columns**: `count`, `any`, `avg`, `distinct count`, `dominant`,
`first`, `last`, `last by index`, `high`, `low`, `max`, `min`, `median`,
`q1`, `q3`, `unique`.
**Boolean columns**: `count`, `any`, `distinct count`, `dominant`, `first`,
`last`, `last by index`, `unique`.
@@ -0,0 +1,138 @@
# Selection and Ordering
## Columns
The `columns` property specifies which columns should be included in the
`View`'s output. This allows users to show or hide a specific subset of columns,
as well as control the order in which columns appear to the user. This is
represented in Perspective as an array of string column names:
<div class="javascript">
```javascript
const view = await table.view({
columns: ["a"],
});
```
</div>
<div class="python">
```python
view = table.view(columns=["a"])
```
</div>
<div class="rust">
```rust
let view = table.view(Some(ViewConfigUpdate {
columns: Some(vec![Some("a".into())]),
..ViewConfigUpdate::default()
})).await?;
```
</div>
## Sort
The `sort` property specifies columns on which the query should be sorted,
analogous to `ORDER BY` in SQL. A column can be sorted regardless of its data
type, and sorts can be applied in ascending or descending order. Perspective
represents `sort` as an array of arrays, with the values of each inner array
being a string column name and a string sort direction. When `split_by` are
applied, the additional sort directions `"col asc"` and `"col desc"` will
determine the order of pivot column groups.
<div class="javascript">
```javascript
const view = await table.view({
sort: [["a", "asc"]],
});
```
</div>
<div class="python">
```python
view = table.view(sort=[["a", "asc"]])
```
</div>
<div class="rust">
```rust
let view = table.view(Some(ViewConfigUpdate {
sort: Some(vec![Sort("a".into(), SortDir::Asc)]),
..ViewConfigUpdate::default()
})).await?;
```
</div>
The available sort directions are:
| Direction | Description |
|---|---|
| `"asc"` | Ascending order |
| `"desc"` | Descending order |
| `"asc abs"` | Ascending by absolute value |
| `"desc abs"` | Descending by absolute value |
| `"col asc"` | Ascending order for pivot column groups (requires `split_by`) |
| `"col desc"` | Descending order for pivot column groups (requires `split_by`) |
| `"col asc abs"` | Ascending by absolute value for pivot column groups |
| `"col desc abs"` | Descending by absolute value for pivot column groups |
## Filter
The `filter` property specifies columns on which the query can be filtered,
returning rows that pass the specified filter condition. This is analogous to
the `WHERE` clause in SQL. There is no limit on the number of columns where
`filter` is applied, but the resulting dataset is one that passes all the filter
conditions, i.e. the filters are joined with an `AND` condition. The join
condition can be changed to `OR` via the `filter_op` property.
Perspective represents `filter` as an array of arrays, with the values of each
inner array being a string column name, a string filter operator, and a filter
operand in the type of the column:
<div class="javascript">
```javascript
const view = await table.view({
filter: [["a", "<", 100]],
});
```
</div>
<div class="python">
```python
view = table.view(filter=[["a", "<", 100]])
```
</div>
<div class="rust">
```rust
let view = table.view(Some(ViewConfigUpdate {
filter: Some(vec![Filter::new("a", "<", FilterTerm::Scalar(Scalar::Float(100.0)))]),
..ViewConfigUpdate::default()
})).await?;
```
</div>
The available filter operators depend on the column type:
**String columns**: `==`, `!=`, `>`, `>=`, `<`, `<=`, `begins with`,
`contains`, `ends with`, `in`, `not in`, `is not null`, `is null`.
**Numeric columns** (`integer`, `float`): `==`, `!=`, `>`, `>=`, `<`, `<=`,
`is not null`, `is null`.
**Boolean columns**: `==`, `is not null`, `is null`.
**Date/Datetime columns**: `==`, `!=`, `>`, `>=`, `<`, `<=`, `is not null`,
`is null`.
+50
View File
@@ -0,0 +1,50 @@
# Querying data
To query the table, create a [`Table::view`] on the table instance with an
optional configuration object. A [`Table`] can have as many [`View`]s associated
with it as you need - Perspective conserves memory by relying on a single
[`Table`] to power multiple [`View`]s concurrently:
<div class="javascript">
```javascript
const view = await table.view({
columns: ["Sales"],
aggregates: { Sales: "sum" },
group_by: ["Region", "Country"],
filter: [["Category", "in", ["Furniture", "Technology"]]],
});
```
</div>
<div class="python">
```python
view = table.view(
columns=["Sales"],
aggregates={"Sales": "sum"},
group_by=["Region", "Country"],
filter=[["Category", "in", ["Furniture", "Technology"]]]
)
```
</div>
<div class="rust">
```rust
use crate::config::*;
let view = table
.view(Some(ViewConfigUpdate {
columns: Some(vec![Some("Sales".into())]),
aggregates: Some(HashMap::from_iter(vec![("Sales".into(), "sum".into())])),
group_by: Some(vec!["Region".into(), "Country".into()]),
filter: Some(vec![Filter::new("Category", "in", &[
"Furniture",
"Technology",
])]),
..ViewConfigUpdate::default()
}))
.await?;
```
</div>