Grace Platform – Developing & Deploying Models with FastAPI

User Guide

1. Preparing Your Code

Place your model code in the Grace work directory:

• Either through the activated workspace.
• Or by using a remote workspace connection.

Add code files:

• Write code directly in the editor.
• Or clone a Git repository into the workspace.

2. Required Files for Deployment

To deploy a model using FastAPI, you’ll typically need:

• model.pkl → The serialized trained model (Pickle file).
• model.py → The FastAPI script defining endpoints.
• requirements.txt → Python dependencies.
• (Optional) Training script(s) that generate the pickle file.

3. Writing the model.py Script

Follow FastAPI standards:

• Import and use APIRouter for routing.
• Define an endpoint, e.g.:

@router.post("/predict")
def predict(input: InputSchema):
    # Load model & return prediction
  

Load and interact with your model inside the endpoint function.

Using Grace Deployment Config

Grace allows adding custom variables at deployment. Access these via:

import os, json

config_path = os.environ["GRACE_DEPLOYMENT_CONFIG_PATH"]
with open(config_path) as f:
    config = json.load(f)
  

Example use cases:

• Choose a specific model version.
• Dynamically configure endpoints.
• Modify runtime behavior.

4. Registering the Model in Grace

1. Go to API → Model Info.
2. Click Add New.
3. Fill in the wizard:
   • Name (e.g., testAPI).
   • Version (required).
   • Other fields (optional).
   • Toggle Host model on Grace.
   • Select:
     • Model type → FastAPI.
     • Source folder (e.g., ~/testAPI).
     • Base image → match your training environment (e.g., Python 3.11).
4. Click Create.

5. Creating & Deploying a Snapshot

1. After creation, check Snapshots.
2. Status will show Pending until ready.
3. Once ready, go to Deployment → Deploy.
4. Select your prepared snapshot (e.g., testAPI).
5. Configure:
   • User token (must be created beforehand).
   • CPU type & resources.
   • (Optional) JSON variables for runtime customization:
     • Add and validate JSON.
     • Remove if not needed.
6. Click Deploy.

6. Using the Deployed Model

1. Once deployed, copy the model URL.
2. In your workspace:
   • Update your client script with the model URL.
   • Provide the Grace client secret for authentication (from Project Description → Tokens).
   • Example: CPH_team token.
3. Run your script:
   • Authenticate using the service token.
   • Interact with the deployed model (e.g., send predictions).

7. Summary

• Develop models inside Grace.
• Use FastAPI to expose prediction endpoints.
• Deploy and host models as scalable APIs.
• Securely interact with deployed models via tokens.