wehub-resource-sync
73 lines
2.2 KiB
Markdown
73 lines
2.2 KiB
Markdown
# Adding `examples` unit tests to `pytest` test suite
|
|
|
|
Two types of test runs for code in `examples` directory are supported:
|
|
|
|
- Examples run by `mlflow run`
|
|
- Examples run by another command, such as the `python` interpreter
|
|
|
|
Each of these types of runs are implemented using `@pytest.mark.parametrize` decorator. Adding a new
|
|
example to test involves updating the decorator list as described below.
|
|
|
|
For purpose of discussion, `new_example_dir` designates the
|
|
directory the example code is found, i.e., it is located in `examples/new_example_dir`.
|
|
|
|
## Examples that utilize `mlflow run` construct
|
|
|
|
The `@pytest.mark.mark.parametrize` decorator for `def test_mlflow_run_example(directory, params):`
|
|
is updated.
|
|
|
|
If the example is executed by `cd examples/new_example_dir && mlflow run . -P param1=99 -P param2=3`, then
|
|
this `tuple` is added to the decorator list
|
|
|
|
```
|
|
("new_example_dir", ["-P", "param1=123", "-P", "param2=99"])
|
|
```
|
|
|
|
as shown below
|
|
|
|
```
|
|
@pytest.mark.parametrize(("directory", "params"), [
|
|
("sklearn_elasticnet_wine", ["-P", "alpha=0.5"]),
|
|
(os.path.join("sklearn_elasticnet_diabetes", "linux"), []),
|
|
("new_example_dir", ["-P", "param1=123", "-P", "param2=99"]),
|
|
])
|
|
def test_mlflow_run_example(directory, params):
|
|
```
|
|
|
|
The `tuple` for an example requiring no parameters is simply:
|
|
|
|
```
|
|
("new_example_dir", []),
|
|
```
|
|
|
|
## Examples that are executed with another command
|
|
|
|
For an example that is not run by `mlflow run`, the list in
|
|
`@pytest.mark.parametrize` decorator for `test_command_example(tmpdir, directory, command):` is updated.
|
|
|
|
Examples invoked by `cd examples/new_example_dir && python train.py` require this tuple added
|
|
to the decorator's list
|
|
|
|
```
|
|
("new_example_dir", ["python", "train.py"]),
|
|
```
|
|
|
|
as shown below
|
|
|
|
```
|
|
@pytest.mark.parametrize(("directory", "command"), [
|
|
('sklearn_logistic_regression', ['python', 'train.py']),
|
|
('h2o', ['python', 'random_forest.py']),
|
|
('quickstart', ['python', 'mlflow_tracking.py']),
|
|
("new_example_dir", ["python", "train.py"]),
|
|
])
|
|
def test_command_example(tmpdir, directory, command):
|
|
```
|
|
|
|
If the example requires arguments to run, i.e., `python train.py arg1 arg2`, then the
|
|
tuple would look like this
|
|
|
|
```
|
|
('new_example_dir', ['python', 'train.py', 'arg1', 'arg2'])
|
|
```
|