wehub-resource-sync
adf0d17497
publish / version_or_publish (push) Has been cancelled
storybook-build / changes (push) Has been cancelled
storybook-build / :storybook-build (push) Has been cancelled
Sync Gradio Skills to Hugging Face / sync-skills (push) Has been cancelled
functional / changes (push) Has been cancelled
functional / build-frontend (push) Has been cancelled
functional / functional-test-SSR=false (push) Has been cancelled
functional / functional-reload (push) Has been cancelled
js / changes (push) Has been cancelled
js / js-test (push) Has been cancelled
docs-build / changes (push) Has been cancelled
docs-build / docs-build (push) Has been cancelled
docs-build / website-build (push) Has been cancelled
functional / functional-test-SSR=true (push) Has been cancelled
hygiene / hygiene-test (push) Has been cancelled
python / changes (push) Has been cancelled
python / build (push) Has been cancelled
python / test-ubuntu-latest-flaky (push) Has been cancelled
python / test-ubuntu-latest-not-flaky (push) Has been cancelled
python / test-windows-latest-flaky (push) Has been cancelled
python / test-windows-latest-not-flaky (push) Has been cancelled
18 lines
1.3 KiB
Markdown
18 lines
1.3 KiB
Markdown
# Load Test
|
|
|
|
This folder contains the content to load test gradio apps, and check the ability to handle multiple connections simultaneously.
|
|
|
|
|
|
## Setup
|
|
|
|
For a proper "production" environment load test, you can run gradio behind an nginx config. Install nginx and add `nginx.conf` to `/etc/nginx/conf.d/*.conf` on the machine running gradio. (You may want to have a machine running gradio that is separate from the machine running the load test, to include the effect of network latency).
|
|
|
|
`gradio.py` contains a simple gradio chat app that streams 500 tokens at a rate of 100 tokens/sec. This app is compatible with gradio 3.x as well.
|
|
|
|
`simple.py` contains a simple fastapi that streams 500 tokens at a rate of 100 tokens/sec, with both a WS and SSE endpoint. This does not use gradio. The purpose of this file is to compare the performance of streaming raw websockets and SSE, vs with gradio overhead.
|
|
|
|
`workers.py` contains a fastapi that uses queues and worker threads to stream 500 tokens at a rate of 100 tokens/sec, with both a WS and SSE endpoint. The purpose of this file is to compare the performance of streaming websockets and SSE with the implementation of gradio but without all the overhead.
|
|
|
|
|
|
`load.ipynb` supports running load tests on `chat.py` with gradio 3.x and 4.0, as well as on `app.py`. Simply configure the URL to point to where the app is running.
|