ludwig-ai--ludwig/examples/serve

Ludwig Model Serve Example

This example shows Ludwig's http model serving capability, which is able to load a pre-trained Ludwig model and respond to REST APIs for predictions. A simple client program illustrates how to invoke the REST API to retrieve predictions for provided input features. The two REST APIs covered by this example:

REST API	Description
/predict	Single record prediction
/batch_predict	Prediction for batch of records

Preparatory Steps

• Run the simple_model_training.py example in examples/titanic. This should result the following file structures:

examples/
    titantic/
        results/
            simple_experiment_simple_model/
                model/
                description.json
                training_statistics.json

Run Model Server Example

• Open two terminal windows
• In first terminal window:
  • Ensure current working directory is examples/serve
  • Start ludwig model server with the titanic trained model. The following command uses the default host address (0.0.0.0) and port number (8000).

ludwig serve --model_path ../titanic/results/simple_experiment_simple_model/model

Sample start up messages for ludwig model server

███████████████████████
█ █ █ █  ▜█ █ █ █ █   █
█ █ █ █ █ █ █ █ █ █ ███
█ █   █ █ █ █ █ █ █ ▌ █
█ █████ █ █ █ █ █ █ █ █
█     █  ▟█     █ █   █
███████████████████████
ludwig v0.3 - Serve

INFO:     Started server process [4429]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

• In the second terminal window:
  • Ensure current working director is examples/serve
  • Run the sample client program

python client_program.py

Output should look like this

retrieved 1309 records for predictions
single record for prediction:
 {'PassengerId': 1, 'Survived': 0.0, 'Pclass': 3, 'Name': 'Braund, Mr. Owen Harris', 'Sex': 'male', 'Age': 22.0, 'SibSp': 1, 'Parch': 0, 'Ticket': 'A/5 21171', 'Fare': 7.25, 'Cabin': nan, 'Embarked': 'S', 'split': 0}

invoking REST API /predict for single record...

Received 1 predictions
Sample predictions:
   Survived_predictions  Survived_probabilities_False  Survived_probabilities_True  Survived_probability
0                 False                      0.906132                     0.093868              0.906132

invoking REST API /batch_predict for entire dataframe...

Received 1309 predictions
Sample predictions:
   Survived_predictions  Survived_probabilities_False  Survived_probabilities_True  Survived_probability
0                 False                      0.906132                     0.093868              0.906132
1                  True                      0.165714                     0.834286              0.834286
2                  True                      0.441169                     0.558831              0.558831
3                  True                      0.228311                     0.771689              0.771689
4                 False                      0.878072                     0.121928              0.878072```