wehub-resource-sync
231 lines
10 KiB
Plaintext
231 lines
10 KiB
Plaintext
---
|
|
id: docker-images
|
|
title: Running in Docker
|
|
description: Example Docker images to run your crawlers
|
|
---
|
|
|
|
import ApiLink from '@site/src/components/ApiLink';
|
|
|
|
import Tabs from '@theme/Tabs';
|
|
import TabItem from '@theme/TabItem';
|
|
import CodeBlock from '@theme/CodeBlock';
|
|
|
|
import jsNodeDocker from '!!raw-loader!./docker_node_js.txt';
|
|
import tsNodeDocker from '!!raw-loader!./docker_node_ts.txt';
|
|
import jsBrowserDocker from '!!raw-loader!./docker_browser_js.txt';
|
|
import tsBrowserDocker from '!!raw-loader!./docker_browser_ts.txt';
|
|
|
|
Running headless browsers in Docker requires a lot of setup to do it right. But there's no need to worry about that, because we already created base images that you can freely use. We use them every day on the [Apify Platform](../deployment/apify-platform).
|
|
|
|
All images can be found in their [GitHub repo](https://github.com/apify/apify-actor-docker) and in our [DockerHub](https://hub.docker.com/orgs/apify).
|
|
|
|
## Overview
|
|
|
|
Browsers are pretty big, so we try to provide a wide variety of images to suit the specific needs. Here's a full list of our Docker images.
|
|
|
|
- [`apify/actor-node`](#actor-node)
|
|
- [`apify/actor-node-puppeteer-chrome`](#actor-node-puppeteer-chrome)
|
|
- [`apify/actor-node-playwright`](#actor-node-playwright)
|
|
- [`apify/actor-node-playwright-chrome`](#actor-node-playwright-chrome)
|
|
- [`apify/actor-node-playwright-firefox`](#actor-node-playwright-firefox)
|
|
- [`apify/actor-node-playwright-webkit`](#actor-node-playwright-webkit)
|
|
|
|
## Versioning
|
|
|
|
Each image is tagged with up to 2 version tags, depending on the type of the image. One for Node.js version and second for pre-installed web automation library version. If you use the image name without a version tag, you'll always get the latest available version.
|
|
|
|
> We recommend always using at least the Node.js version tag in production Dockerfiles. It will ensure that a future update of Node.js will not break our automations.
|
|
|
|
### Node.js versioning
|
|
|
|
Our images are built with multiple Node.js versions to ensure backwards compatibility. Currently, Node.js **versions 20, 22, and 24 are supported** (legacy versions still exist, see DockerHub). To select the preferred version, use the appropriate number as the image tag.
|
|
|
|
```dockerfile
|
|
# Use Node.js 24
|
|
FROM apify/actor-node:24
|
|
```
|
|
|
|
### Automation library versioning
|
|
|
|
Images that include a pre-installed automation library, which means all images that include `puppeteer` or `playwright` in their name, are also tagged with the pre-installed version of the library. For example, `apify/actor-node-puppeteer-chrome:24-24.43.1` comes with Node.js 24 and Puppeteer v24.43.1. If you try to install a different version of Puppeteer into this image, you may run into compatibility issues, because the Chromium version bundled with `puppeteer` will not match the version of Chromium that's pre-installed.
|
|
|
|
Similarly `apify/actor-node-playwright-firefox:24-1.60.0` runs on Node.js 24 and is pre-installed with the Firefox version that comes with v1.60.0.
|
|
|
|
Installing `apify/actor-node-puppeteer-chrome` (without a tag) will install the latest available version of Node.js and `puppeteer`.
|
|
|
|
### Pre-release tags
|
|
|
|
We also build pre-release versions of the images to test the changes we make. Those are typically denoted by a `beta` suffix, but it can vary depending on our needs. If you need to try a pre-release version, you can do it like this:
|
|
|
|
```dockerfile
|
|
# Without library version.
|
|
FROM apify/actor-node:24-beta
|
|
```
|
|
|
|
```dockerfile
|
|
# With library version.
|
|
FROM apify/actor-node-playwright-chrome:24-1.60.0-beta
|
|
```
|
|
|
|
## Best practices
|
|
|
|
For production crawlers, we recommend pinning both the Node.js version **and** the automation library version in your Dockerfile tag. This ensures reproducible builds and prevents unexpected behavior when new versions are released.
|
|
|
|
### Recommended approach: Pin both versions
|
|
|
|
Match the automation library version in your `package.json` with the version in your Docker image tag:
|
|
|
|
```dockerfile
|
|
FROM apify/actor-node-playwright-chrome:24-1.60.0
|
|
```
|
|
|
|
```json
|
|
{
|
|
"dependencies": {
|
|
"crawlee": "^3.0.0",
|
|
"playwright": "1.60.0"
|
|
}
|
|
}
|
|
```
|
|
|
|
:::warning Why version matching matters
|
|
|
|
If you pin the Docker image to `24-1.60.0` but install a different Playwright version via `package.json`, you may encounter browser compatibility issues. The browsers pre-installed in the image are specifically built for that Playwright version.
|
|
|
|
:::
|
|
|
|
### Alternative approach: Using asterisk `*`
|
|
|
|
You can also use asterisk `*` as the automation library version in your `package.json`:
|
|
|
|
```dockerfile
|
|
FROM apify/actor-node-playwright-chrome:24
|
|
```
|
|
|
|
```json
|
|
{
|
|
"dependencies": {
|
|
"crawlee": "^3.0.0",
|
|
"playwright": "*"
|
|
}
|
|
}
|
|
```
|
|
|
|
This makes sure the pre-installed version of Puppeteer or Playwright is not re-installed on build. However, this approach is less predictable because you'll get whatever version was latest when the Docker image was built.
|
|
|
|
## Finding available tags
|
|
|
|
To see all available tags for each image, you can visit Docker Hub directly:
|
|
|
|
- [apify/actor-node](https://hub.docker.com/r/apify/actor-node/tags)
|
|
- [apify/actor-node-puppeteer-chrome](https://hub.docker.com/r/apify/actor-node-puppeteer-chrome/tags)
|
|
- [apify/actor-node-playwright](https://hub.docker.com/r/apify/actor-node-playwright/tags)
|
|
- [apify/actor-node-playwright-chrome](https://hub.docker.com/r/apify/actor-node-playwright-chrome/tags)
|
|
- [apify/actor-node-playwright-firefox](https://hub.docker.com/r/apify/actor-node-playwright-firefox/tags)
|
|
- [apify/actor-node-playwright-webkit](https://hub.docker.com/r/apify/actor-node-playwright-webkit/tags)
|
|
|
|
You can also query available tags programmatically:
|
|
|
|
```bash
|
|
curl -s "https://registry.hub.docker.com/v2/repositories/apify/actor-node-playwright-chrome/tags?page_size=50" | jq '.results[].name'
|
|
```
|
|
|
|
### Warning about image size
|
|
|
|
Browsers are huge. If you don't need them all in your image, it's better to use a smaller image with only the one browser you need.
|
|
|
|
You should also be careful when installing new dependencies. Nothing prevents you from installing Playwright into the`actor-node-puppeteer-chrome` image, but the resulting image will be about 3 times larger and extremely slow to download and build.
|
|
|
|
When you use only what you need, you'll be rewarded with reasonable build and start times.
|
|
|
|
## Apify Docker Images
|
|
|
|
### actor-node
|
|
|
|
This is the smallest image we have based on Alpine Linux. It does not include any browsers, and it's therefore best used with <ApiLink to="cheerio-crawler/class/CheerioCrawler">`CheerioCrawler`</ApiLink>. It benefits from lightning fast builds and container startups.
|
|
|
|
​<ApiLink to="puppeteer-crawler/class/PuppeteerCrawler">`PuppeteerCrawler`</ApiLink>, <ApiLink to="playwright-crawler/class/PlaywrightCrawler">`PlaywrightCrawler`</ApiLink> and other browser based features will **NOT** work with this image.
|
|
|
|
```dockerfile
|
|
FROM apify/actor-node:24
|
|
```
|
|
|
|
### actor-node-puppeteer-chrome
|
|
|
|
This image includes Puppeteer (Chromium) and the Chrome browser. It can be used with <ApiLink to="cheerio-crawler/class/CheerioCrawler">`CheerioCrawler`</ApiLink> and <ApiLink to="puppeteer-crawler/class/PuppeteerCrawler">`PuppeteerCrawler`</ApiLink>, but **NOT** with <ApiLink to="playwright-crawler/class/PlaywrightCrawler">`PlaywrightCrawler`</ApiLink>.
|
|
|
|
The image supports XVFB by default, so you can run both `headless` and `headful` browsers with it.
|
|
|
|
```dockerfile
|
|
FROM apify/actor-node-puppeteer-chrome:24
|
|
```
|
|
|
|
### actor-node-playwright
|
|
|
|
A very large and slow image that can run all Playwright browsers: Chromium, Chrome, Firefox, WebKit. Everything is installed. If you need to develop or test with multiple browsers, this is the image to choose, but in most cases, it's better to use the specialized images below.
|
|
|
|
```dockerfile
|
|
FROM apify/actor-node-playwright:24
|
|
```
|
|
|
|
### actor-node-playwright-chrome
|
|
|
|
Similar to [`actor-node-puppeteer-chrome`](#actor-node-puppeteer-chrome), but for Playwright. You can run <ApiLink to="cheerio-crawler/class/CheerioCrawler">`CheerioCrawler`</ApiLink> and <ApiLink to="playwright-crawler/class/PlaywrightCrawler">`PlaywrightCrawler`</ApiLink>, but **NOT** <ApiLink to="puppeteer-crawler/class/PuppeteerCrawler">`PuppeteerCrawler`</ApiLink>.
|
|
|
|
It uses the [`PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD`](https://playwright.dev/docs/api/environment-variables/) environment variable to block installation of more browsers into the image to keep it small. If you want more browsers, either use the [`actor-node-playwright`](#actor-node-playwright) image override this env var.
|
|
|
|
The image supports XVFB by default, so we can run both `headless` and `headful` browsers with it.
|
|
|
|
```dockerfile
|
|
FROM apify/actor-node-playwright-chrome:24
|
|
```
|
|
|
|
### actor-node-playwright-firefox
|
|
|
|
Same idea as [`actor-node-playwright-chrome`](#actor-node-playwright-chrome), but with Firefox
|
|
pre-installed.
|
|
|
|
```dockerfile
|
|
FROM apify/actor-node-playwright-firefox:24
|
|
```
|
|
|
|
### actor-node-playwright-webkit
|
|
|
|
Same idea as [`actor-node-playwright-chrome`](#actor-node-playwright-chrome), but with WebKit
|
|
pre-installed.
|
|
|
|
```dockerfile
|
|
FROM apify/actor-node-playwright-webkit:24
|
|
```
|
|
|
|
## Example Dockerfile
|
|
|
|
To use the above images, it's necessary to have a [`Dockerfile`](https://docs.docker.com/engine/reference/builder/). You can either use this example, or bootstrap your projects with the [Crawlee CLI](../introduction/setting-up) which automatically adds the correct Dockerfile into our project folder.
|
|
|
|
<Tabs>
|
|
<TabItem value="node+js" label="Node+JavaScript">
|
|
<CodeBlock language="dockerfile">
|
|
{jsNodeDocker}
|
|
</CodeBlock>
|
|
</TabItem>
|
|
<TabItem value="node+ts" label="Node+TypeScript">
|
|
<CodeBlock language="dockerfile">
|
|
{tsNodeDocker}
|
|
</CodeBlock>
|
|
</TabItem>
|
|
<TabItem value="browser+js" label="Browser+JavaScript">
|
|
This example is for Playwright. If you want to use Puppeteer, simply replace <b>playwright</b>
|
|
with <b>puppeteer</b> in the <code>FROM</code> declaration.
|
|
<CodeBlock language="dockerfile">
|
|
{jsBrowserDocker}
|
|
</CodeBlock>
|
|
</TabItem>
|
|
<TabItem value="browser+ts" label="Browser+TypeScript">
|
|
This example is for Playwright. If you want to use Puppeteer, simply replace <b>playwright</b>
|
|
with <b>puppeteer</b> in both <code>FROM</code> declarations.
|
|
<CodeBlock language="dockerfile">
|
|
{tsBrowserDocker}
|
|
</CodeBlock>
|
|
</TabItem>
|
|
</Tabs>
|