wehub-resource-sync
c56bef871b
Sync docs with Docusaurus / sync (push) Waiting to run
Tests / Check if changed (push) Waiting to run
Tests / format (push) Blocked by required conditions
Tests / check-imports (push) Blocked by required conditions
Tests / Unit / macos-latest (push) Blocked by required conditions
Tests / Unit / ubuntu-latest (push) Blocked by required conditions
Tests / Unit / windows-latest (push) Blocked by required conditions
Tests / mypy (push) Blocked by required conditions
Tests / Integration / ubuntu-latest (push) Blocked by required conditions
Tests / Integration / macos-latest (push) Blocked by required conditions
Tests / Integration / windows-latest (push) Blocked by required conditions
Tests / notify-slack-on-failure (push) Blocked by required conditions
Tests / Mark tests as completed (push) Blocked by required conditions
Docker image release / Build base image (push) Waiting to run
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
18 lines
1.2 KiB
Plaintext
18 lines
1.2 KiB
Plaintext
---
|
||
title: "Fetchers"
|
||
id: fetchers
|
||
slug: "/fetchers"
|
||
description: "Fetchers retrieve content from external sources – URLs, web crawls, or cloud storage such as SharePoint and Google Drive – so you can use it as data for your pipelines."
|
||
---
|
||
|
||
# Fetchers
|
||
|
||
Fetchers retrieve content from external sources – URLs, web crawls, or cloud storage such as SharePoint and Google Drive – so you can use it as data for your pipelines.
|
||
|
||
| Component | Description |
|
||
| --- | --- |
|
||
| [FirecrawlCrawler](fetchers/firecrawlcrawler.mdx) | Crawls websites with Firecrawl, following links to discover subpages, and returns them as Documents. |
|
||
| [GoogleDriveFetcher](fetchers/googledrivefetcher.mdx) | Fetches the full content of Google Drive files via the Drive API v3 and returns it as ByteStreams. |
|
||
| [LinkContentFetcher](fetchers/linkcontentfetcher.mdx) | Fetches the contents of the URLs you give it so you can use them as data for your pipelines. |
|
||
| [MSSharePointFetcher](fetchers/mssharepointfetcher.mdx) | Fetches the full content of Microsoft SharePoint and OneDrive items via the Microsoft Graph API and returns it as ByteStreams. |
|