Files
wehub-resource-sync ee3943b5b1
CI / test (deps-minimum, windows-latest, unit) (push) Blocked by required conditions
CI / test_py314 (deps-latest, ubuntu-latest, unit) (push) Blocked by required conditions
CI / test_py314 (deps-latest, windows-latest, unit) (push) Blocked by required conditions
CI / test_py314_future (deps-latest, ubuntu-latest, unit) (push) Blocked by required conditions
CI / test_py314_future (deps-latest, windows-latest, unit) (push) Blocked by required conditions
Secret Leaks / trufflehog (push) Waiting to run
CI / check_code_quality (push) Waiting to run
CI / test (deps-latest, ubuntu-latest, integration) (push) Blocked by required conditions
CI / test (deps-latest, ubuntu-latest, unit) (push) Blocked by required conditions
CI / test (deps-latest, windows-latest, integration) (push) Blocked by required conditions
CI / test (deps-latest, windows-latest, unit) (push) Blocked by required conditions
CI / test (deps-minimum, ubuntu-latest, integration) (push) Blocked by required conditions
CI / test (deps-minimum, ubuntu-latest, unit) (push) Blocked by required conditions
CI / test (deps-minimum, windows-latest, integration) (push) Blocked by required conditions
Build documentation / build (push) Failing after 0s
chore: import upstream snapshot with attribution
2026-07-13 13:24:32 +08:00

121 lines
2.6 KiB
Plaintext

# Loading methods
Methods for listing and loading datasets:
## Datasets
[[autodoc]] datasets.load_dataset
[[autodoc]] datasets.load_from_disk
[[autodoc]] datasets.load_dataset_builder
[[autodoc]] datasets.get_dataset_config_names
[[autodoc]] datasets.get_dataset_infos
[[autodoc]] datasets.get_dataset_split_names
## From files
Configurations used to load data files.
They are used when loading local files or a dataset repository:
- local files: `load_dataset("parquet", data_dir="path/to/data/dir")`
- dataset repository: `load_dataset("allenai/c4")`
You can pass arguments to `load_dataset` to configure data loading.
For example you can specify the `sep` parameter to define the [`~datasets.packaged_modules.csv.CsvConfig`] that is used to load the data:
```python
load_dataset("csv", data_dir="path/to/data/dir", sep="\t")
```
### Text
[[autodoc]] datasets.packaged_modules.text.TextConfig
[[autodoc]] datasets.packaged_modules.text.Text
### CSV
[[autodoc]] datasets.packaged_modules.csv.CsvConfig
[[autodoc]] datasets.packaged_modules.csv.Csv
### JSON
[[autodoc]] datasets.packaged_modules.json.JsonConfig
[[autodoc]] datasets.packaged_modules.json.Json
### XML
[[autodoc]] datasets.packaged_modules.xml.XmlConfig
[[autodoc]] datasets.packaged_modules.xml.Xml
### Parquet
[[autodoc]] datasets.packaged_modules.parquet.ParquetConfig
[[autodoc]] datasets.packaged_modules.parquet.Parquet
### Arrow
[[autodoc]] datasets.packaged_modules.arrow.ArrowConfig
[[autodoc]] datasets.packaged_modules.arrow.Arrow
### SQL
[[autodoc]] datasets.packaged_modules.sql.SqlConfig
[[autodoc]] datasets.packaged_modules.sql.Sql
### Images
[[autodoc]] datasets.packaged_modules.imagefolder.ImageFolderConfig
[[autodoc]] datasets.packaged_modules.imagefolder.ImageFolder
### Audio
[[autodoc]] datasets.packaged_modules.audiofolder.AudioFolderConfig
[[autodoc]] datasets.packaged_modules.audiofolder.AudioFolder
### Videos
[[autodoc]] datasets.packaged_modules.videofolder.VideoFolderConfig
[[autodoc]] datasets.packaged_modules.videofolder.VideoFolder
### HDF5
[[autodoc]] datasets.packaged_modules.hdf5.HDF5Config
[[autodoc]] datasets.packaged_modules.hdf5.HDF5
### TsFile
[[autodoc]] datasets.packaged_modules.tsfile.TsFileConfig
[[autodoc]] datasets.packaged_modules.tsfile.TsFile
### Pdf
[[autodoc]] datasets.packaged_modules.pdffolder.PdfFolderConfig
[[autodoc]] datasets.packaged_modules.pdffolder.PdfFolder
### Nifti
[[autodoc]] datasets.packaged_modules.niftifolder.NiftiFolderConfig
[[autodoc]] datasets.packaged_modules.niftifolder.NiftiFolder
### WebDataset
[[autodoc]] datasets.packaged_modules.webdataset.WebDataset