chore: import upstream snapshot with attribution

2026-07-13 12:23:53 +08:00
commit bf6f0825b2
1681 changed files with 296950 additions and 0 deletions
@@ -0,0 +1,87 @@
+# Loading data
+
+A `Table` may also be created-or-updated by data in CSV,
+[Apache Arrow](https://arrow.apache.org/), JSON row-oriented or JSON
+column-oriented formats. In addition to these, `perspective-python` additionally
+supports `pyarrow.Table`, `polars.DataFrame` and `pandas.DataFrame` objects
+directly. These formats are otherwise identical to the built-in formats and
+don't exhibit any additional support or type-awareness; e.g., `pandas.DataFrame`
+support is _just_ `pyarrow.Table.from_pandas` piped into Perspective's Arrow
+reader.
+
+`Client::table` and `Table::update` perform _coercion_ on their input for all
+input formats _except_ Arrow (which comes with its own schema and has no need
+for coercion). `"date"` and `"datetime"` column types do not have native JSON
+representations, so these column types _cannot_ be inferred from JSON input.
+Instead, for columns of these types for JSON input, a `Table` must first be
+constructed with a _schema_. Next, call `Table::update` with the JSON input -
+Perspective's JSON reader may _coerce_ a `date` or `datetime` from these native
+JSON types:
+
+- `integer` as milliseconds-since-epoch.
+- `string` as a any of Perspective's built-in date format formats.
+- JavaScript `Date` and Python `datetime.date` and `datetime.datetime` are _not_
+  supported directly. However, in JavaScript `Date` types are automatically
+  coerced to correct `integer` timestamps by default when converted to JSON.
+
+## Apache Arrow
+
+The most efficient way to load data into Perspective, encoded as
+[Apache Arrow IPC format](https://arrow.apache.org/docs/python/ipc.html). In
+JavaScript:
+
+```javascript
+const resp = await fetch(
+    "https://cdn.jsdelivr.net/npm/superstore-arrow/superstore.lz4.arrow",
+);
+
+const arrow = await resp.arrayBuffer();
+```
+
+Apache Arrow input do not support type coercion, preferring Arrow's internal
+self-describing schema.
+
+## CSV
+
+Perspective relies on Apache Arrow's CSV parser, and as such uses mostly the
+same column-type inference logic as Arrow itself would use for parsing CSV.
+
+## Row Oriented JSON
+
+Row-oriented JSON is in the form of a list of objects. Each object in the list
+corresponds to a row in the table. For example:
+
+```json
+[
+    { "a": 86, "b": false, "c": "words" },
+    { "a": 0, "b": true, "c": "" },
+    { "a": 12345, "b": false, "c": "here" }
+]
+```
+
+## Column Oriented JSON
+
+Column-Oriented JSON comes in the form of an object of lists. Each key of the
+object is a column name, and each element of the list is the corresponding value
+in the row.
+
+```json
+{
+    "a": [86, 0, 12345],
+    "b": [false, true, false],
+    "c": ["words", "", "here"]
+}
+```
+
+## NDJSON
+
+[NDJSON](https://github.com/ndjson/ndjson-spec) (sometimes also referred to as
+JSONL) is a streaming-friendly format where each line is a valid JSON object,
+separated by newlines. It is commonly used in data streaming and messaging
+queues.
+
+```json
+{ "a": 86, "b": false, "c": "words" }
+{ "a": 0, "b": true, "c": "" }
+{ "a": 12345, "b": false, "c": "here" }
+```