chore: import upstream snapshot with attribution

This commit is contained in:
wehub-resource-sync
2026-07-13 12:23:53 +08:00
commit bf6f0825b2
1681 changed files with 296950 additions and 0 deletions
@@ -0,0 +1,23 @@
# `Table::clear` and `Table::replace`
Calling `Table::clear` will remove all data from the underlying `Table`. Calling
`Table::replace` with new data will clear the `Table`, and update it with a new
dataset that conforms to Perspective's data types and the existing schema on the
`Table`.
<div class="javascript">
```javascript
table.clear();
table.replace(json);
```
</div>
<div class="python">
```python
table.clear()
table.replace(df)
```
</div>
@@ -0,0 +1,47 @@
# Construct a Table
Examples of constructing an empty `Table` from a schema.
<div class="javascript">
JavaScript:
```javascript
var schema = {
x: "integer",
y: "string",
z: "boolean",
};
const table2 = await worker.table(schema);
```
</div>
<div class="python">
Python:
```python
from datetime import date, datetime
schema = {
"x": "integer",
"y": "string",
"z": "boolean",
}
table2 = perspective.table(schema)
```
</div>
<div class="rust">
Rust:
```rust
let data = TableData::Schema(vec![(" a".to_string(), ColumnType::FLOAT)]);
let options = TableInitOptions::default();
let table = client.table(data.into(), options).await?;
```
</div>
+87
View File
@@ -0,0 +1,87 @@
# Loading data
A `Table` may also be created-or-updated by data in CSV,
[Apache Arrow](https://arrow.apache.org/), JSON row-oriented or JSON
column-oriented formats. In addition to these, `perspective-python` additionally
supports `pyarrow.Table`, `polars.DataFrame` and `pandas.DataFrame` objects
directly. These formats are otherwise identical to the built-in formats and
don't exhibit any additional support or type-awareness; e.g., `pandas.DataFrame`
support is _just_ `pyarrow.Table.from_pandas` piped into Perspective's Arrow
reader.
`Client::table` and `Table::update` perform _coercion_ on their input for all
input formats _except_ Arrow (which comes with its own schema and has no need
for coercion). `"date"` and `"datetime"` column types do not have native JSON
representations, so these column types _cannot_ be inferred from JSON input.
Instead, for columns of these types for JSON input, a `Table` must first be
constructed with a _schema_. Next, call `Table::update` with the JSON input -
Perspective's JSON reader may _coerce_ a `date` or `datetime` from these native
JSON types:
- `integer` as milliseconds-since-epoch.
- `string` as a any of Perspective's built-in date format formats.
- JavaScript `Date` and Python `datetime.date` and `datetime.datetime` are _not_
supported directly. However, in JavaScript `Date` types are automatically
coerced to correct `integer` timestamps by default when converted to JSON.
## Apache Arrow
The most efficient way to load data into Perspective, encoded as
[Apache Arrow IPC format](https://arrow.apache.org/docs/python/ipc.html). In
JavaScript:
```javascript
const resp = await fetch(
"https://cdn.jsdelivr.net/npm/superstore-arrow/superstore.lz4.arrow",
);
const arrow = await resp.arrayBuffer();
```
Apache Arrow input do not support type coercion, preferring Arrow's internal
self-describing schema.
## CSV
Perspective relies on Apache Arrow's CSV parser, and as such uses mostly the
same column-type inference logic as Arrow itself would use for parsing CSV.
## Row Oriented JSON
Row-oriented JSON is in the form of a list of objects. Each object in the list
corresponds to a row in the table. For example:
```json
[
{ "a": 86, "b": false, "c": "words" },
{ "a": 0, "b": true, "c": "" },
{ "a": 12345, "b": false, "c": "here" }
]
```
## Column Oriented JSON
Column-Oriented JSON comes in the form of an object of lists. Each key of the
object is a column name, and each element of the list is the corresponding value
in the row.
```json
{
"a": [86, 0, 12345],
"b": [false, true, false],
"c": ["words", "", "here"]
}
```
## NDJSON
[NDJSON](https://github.com/ndjson/ndjson-spec) (sometimes also referred to as
JSONL) is a streaming-friendly format where each line is a valid JSON object,
separated by newlines. It is commonly used in data streaming and messaging
queues.
```json
{ "a": 86, "b": false, "c": "words" }
{ "a": 0, "b": true, "c": "" }
{ "a": 12345, "b": false, "c": "here" }
```
+59
View File
@@ -0,0 +1,59 @@
## Index and Limit
<div class="warning">`limit` cannot be used in conjunction with `index`.</div>
Initializing a `Table` with an `index` tells Perspective to treat a column as
the primary key, allowing in-place updates of rows. Only a single column (of any
type) can be used as an `index`. Indexed `Table` instances allow:
- In-place _updates_ whenever a new row shares an `index` values with an
existing row
- _Partial updates_ when a data batch omits some column.
- _Removes_ to delete a row by `index`.
To create an indexed `Table`, provide the `index` property with a string column
name to be used as an index:
<div class="javascript">
JavaScript:
```javascript
const indexed_table = await perspective.table(data, { index: "a" });
```
</div>
<div class="python">
Python
```python
indexed_table = perspective.Table(data, index="a");
```
</div>
Initializing a `Table` with a `limit` sets the total number of rows the `Table`
is allowed to have. When the `Table` is updated, and the resulting size of the
`Table` would exceed its `limit`, rows that exceed `limit` overwrite the oldest
rows in the `Table`. To create a `Table` with a `limit`, provide the `limit`
property with an integer indicating the maximum rows:
<div class="javascript">
JavaScript:
```javascript
const limit_table = await perspective.table(data, { limit: 1000 });
```
</div>
<div class="python">
Python:
```python
limit_table = perspective.Table(data, limit=1000);
```
</div>
+41
View File
@@ -0,0 +1,41 @@
# Schema and column types
The mapping of a `Table`'s column names to data types is referred to as a
`schema`. Each column has a unique name and a single data type, one of
- `float`
- `integer`
- `boolean`
- `date`
- `datetime`
- `string`
A `Table` schema is fixed at construction, either by explicitly passing a schema
dictionary to the `Client::table` method, or by passing _data_ to this method
from which the schema is _inferred_ (if CSV or JSON format) or inherited (if
Arrow).
## Type inference
When passing CSV or JSON data to the `Client::table` constructor, the type of
each column is inferred automatically. In some cases, the inference algorithm
may not return exactly what you'd like. For example, a column may be interpreted
as a `datetime` when you intended it to be a `string`, or a column may have no
values at all (yet), as it will be updated with values from a real-time data
source later on. In these cases, create a `table()` with a _schema_.
Once the `Table` has been created, further `Table::update` calls will perform
limited type _coercion_ based on the schema. While _coercion_ works similarly to
_inference_, in that input data may be parsed based on the expected column type,
`Table::update` will not _change_ the column's type further. For example, a
number literal `1234` would be _inferred_ as an `"integer"`, but _in the context
of an `Table::update` call on a known `"string"` column_, this will be parsed as
the _string_ `"1234"`.
## `date` and `datetime` inference
Various string representations of `date` and `datetime` format columns can be
_inferred_ as well _coerced_ from strings if they match one of Perspective's
internal known datetime parsing formats, for example
[ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) (which is also the format
Perspective will _output_ these types for CSV).
@@ -0,0 +1,88 @@
# `Table::update` and `Table::remove`
Once a `Table` has been created, it can be updated with new data conforming to
the `Table`'s schema. `Table::update` supports the same data formats as
`Client::table`, minus _schema_.
<div class="javascript">
```javascript
const schema = {
a: "integer",
b: "float",
};
const table = await perspective.table(schema);
table.update(new_data);
```
</div>
<div class="python">
```python
schema = {"a": "integer", "b": "float"}
table = perspective.Table(schema)
table.update(new_data)
```
</div>
Without an `index` set, calls to `update()` _append_ new data to the end of the
`Table`. Otherwise, Perspective allows
[_partial updates_ (in-place)](#index-and-limit) using the `index` to determine
which rows to update:
<div class="javascript">
```javascript
indexed_table.update({ id: [1, 4], name: ["x", "y"] });
```
</div>
<div class="python">
```python
indexed_table.update({"id": [1, 4], "name": ["x", "y"]})
```
</div>
Any value on a `Client::table` can be unset using the value `null` in JSON or
Arrow input formats. Values may be unset on construction, as any `null` in the
dataset will be treated as an unset value. `Table::update` calls do not need to
provide _all columns_ in the `Table`'s schema; missing columns will be omitted
from the `Table`'s updated rows.
<div class="javascript">
```javascript
table.update([{ x: 3, y: null }]); // `z` missing
```
</div>
<div class="python">
```python
table.update([{"x": 3, "y": None}]) # `z` missing
```
</div>
Rows can also be removed from an indexed `Table`, by calling `Table::remove`
with an array of index values:
<div class="javascript">
```javascript
indexed_table.remove([1, 4]);
```
</div>
<div class="python">
```python
indexed_table.remove([1, 4])
```
</div>