wehub-resource-sync
64 lines
2.2 KiB
Markdown
64 lines
2.2 KiB
Markdown
# Testing
|
|
|
|
## Prereqs
|
|
|
|
The tests rely on pytest and tabulate
|
|
|
|
```bash
|
|
pip install pytest tabulate
|
|
```
|
|
|
|
To run tests specific to a particular database or vector database requires that the resources are up and running.
|
|
|
|
Some tests may require API Keys to particular cloud model providers. If you would like to test that provider, then
|
|
you can either edit the test code to include your own, or update ```~/set-env.sh``` and run the following to get
|
|
the environment variables loaded in your shell:
|
|
|
|
```bash
|
|
source ~/set-env.sh
|
|
```
|
|
|
|
### Running all tests
|
|
|
|
```bash
|
|
python3 ./run-tests.py
|
|
```
|
|
|
|
Note: We do not currently recommend running all tests concurrently - we are updating our automation and test scope
|
|
to improve this capability going forward.
|
|
|
|
WARNING: Initiating this script will "clean" the environment before running *_all tests_*:
|
|
- Uninstall any llmware modules locally installed
|
|
- Install the llmware module from this repo (simulating a user installing from pypi)
|
|
- Completely remove $HOME/llmware_data
|
|
- Reset Mongo and Milvus (drop all collections) -> not currently updated to drop Postgres and SQLITE DBs or other Vector DBs
|
|
- Run ./models/test_all_generative_models.py which will pull down many GBs of model files to the test systems.
|
|
|
|
|
|
### Only running tests from a specific folder or file
|
|
|
|
This is our recommended testing approach, unless a PR has significant cross-module changes, and even in such cases
|
|
it is usually more effective to run 2-3 individual tests and supplement with manual testing.
|
|
|
|
Note: please note that we are in the process of updating our test automation coverage suite. We would welcome
|
|
contributions from the community to continue to improve the automation and coverage scope of these tests.
|
|
|
|
In addition to the formal tests, we would recommend using recent examples as well as good proxies, especially in
|
|
targeted areas of the code base.
|
|
|
|
```bash
|
|
python3 ./run-tests.py library
|
|
```
|
|
|
|
```bash
|
|
python3 ./run-tests.py models/test_all_generative_models.py
|
|
```
|
|
|
|
### Including printed output while test is running
|
|
|
|
By default pytest only shows output if tests fail. If you want see printed output for all tests add the _-s_ switch
|
|
|
|
```bash
|
|
python3 ./run-tests.py library -s
|
|
```
|