# How to run the KubeRay autoscaling test This page provides suggestions on running the test `test_autoscaling_e2e` locally. You might want to do this if your PR is breaking this test in CI and you want to debug why. Running the test must happen in stages: 1. Tear down any running `kind` cluster 2. Remove the existing ray docker image that will be deployed to the cluster 3. Build a new docker image containing the local ray repository 4. Create a new `kind` cluster 5. Load the docker image into the cluster 6. Set up kuberay 7. Run the test To help with this, there is a `Dockerfile` and a `rune2e.sh` bash script which together run these things for you. ## Test requirements 1. Ensure `kind` and `kustomize` are both installed 2. Run `ray/autoscaler/kuberay/init-config.sh` to clone `ray-project/kuberay`, which contains config files needed to set up kuberay. 3. Finally, make sure that the `Dockerfile` is using the same python version as what you're using to run the test. By default, this dockerfile is built using the `rayproject/ray:nightly-py310` build. 4. Modify `EXAMPLE_CLUSTER_PATH` in `test_autoscaling_e2e.py`. Now you're ready to run the test. ## Running the test Run `./rune2e.sh` to run the test. The test itself does not tear down resources on failure; you can - examine a Ray cluster from a failed test (`kubectl get pods`, `kubectl get pod`, `kubectl get raycluster`) - view all logs (`kubectl logs `) or just logs associated with the autoscaler (`kubectl logs -c autoscaler`) - delete the Ray cluster (`kubectl delete raycluster -A`) - rerun the test without tearing the operator down (`RAY_IMAGE=/: python test_autoscaling_e2e.py`) - tear down the operator when you're done `python setup/teardown_kuberay.py` - copy files from a pod to your filesystem (`kubectl cp :/path/to/file /target/path/in/local/filesystem`) - access a bash prompt inside the pod (`kubectl exec -it bash`)