API Tutorial: Calling the GRACE LLM API with Python

Overview

This guide provides step-by-step instructions for developers to interact with the GRACE LLM API using Python's requests library. The API is OpenAI-compatible, making it easy to integrate with existing tools and libraries that support the OpenAI API format.

---

API Endpoints Overview

The GRACE LLM API exposes several endpoints, following an OpenAI-compatible format:

Endpoint	Method	Description
/chat or /v1/chat	POST	Send a chat-style conversation to a model and receive a response.
/completions or /v1/completions	POST	Send a prompt to a model and receive a text completion.
/fim/completions	POST	Special completions endpoint for Codex-style code generation.
/models	GET	List all available models and their IDs.
/embeddings	POST	Generate vector embeddings for a given text input.
/	GET	Simple echo endpoint to check if the server is running.

Note: This tutorial focuses on:

• GET /models — to list available models
• POST /chat/completions — to send a chat message and get a response

Coming soon:

• Image generation endpoints — for creating images from text prompts, similar to OpenAI’s images/generations API.
• Audio endpoints — for speech-to-text and text-to-speech, similar to OpenAI’s audio/transcriptions and audio/speech APIs.

These will also follow the OpenAI-compatible format for seamless integration.

---

OpenAI-Compatible API Format

The GRACE LLM API uses the OpenAI-compatible REST API format for endpoints such as:

• GET /models — list available models
• POST /chat/completions — send chat messages and receive responses
• POST /embeddings — generate vector embeddings for text

Why this matters

• Reuse existing OpenAI client libraries — simply change the base_url and api_key to point to the GRACE API.
• Integrate with third-party tools that already support OpenAI (e.g., LangChain, LlamaIndex, AutoGPT) without major code changes.
• Use IDE extensions (like VS Code AI assistants) that expect OpenAI’s API format — just configure them to use your endpoint and token.
• Leverage existing SDKs in multiple languages (Python, JavaScript, Go, etc.) without writing custom wrappers.
• Reduce onboarding time for developers — copy-paste existing OpenAI examples and swap in your credentials.

In short: if a tool or library works with OpenAI’s API, it will likely work with the GRACE LLM API with minimal or no changes.

---

Step 1: Setup

1. Install the requests library if you haven't already:

   pip install requests

2. Set up your base URL and authentication headers:

   import requests
   import json
   
   # Base URL of your API deployment
   base_url = "ENDPOINT_API_URL_ON_GRACE"
   
   # Replace with your actual Bearer token
   headers = {
       "Authorization": "Bearer PROJECT_SERVICE_TOKEN",
       "Content-Type": "application/json"
   }
   

---

Step 2: List Available Models

1. Send a GET request to the /models endpoint:

   url = f"{base_url}models"
   response = requests.get(url, headers=headers)
   if response.status_code == 200:
       models = response.json()
       print(json.dumps(models, indent=2))
   else:
       print(f"Error: {response.status_code} - {response.text}")
   

---

Step 3: Chat Completion Request

1. Pick a model ID from the list of available models.
2. Send a POST request to the /chat/completions endpoint:

   # Pick a model ID from the list above
   model_id = "MODEL_ID_FROM_MODEL_ENDPOINT"
   url = f"{base_url}chat/completions"
   data = {
       "model": model_id,
       "messages": [
           {"role": "user", "content": "Hi! My name is John Snow?"}
       ]
   }
   response = requests.post(url, headers=headers, json=data)
   if response.status_code == 200:
       completion = response.json()
       print(json.dumps(completion, indent=2))
   else:
       print(f"Error: {response.status_code} - {response.text}")
   

---

Step 4: Embedding Request

1. Send a POST request to the /embeddings endpoint:

   url = f"{base_url}embeddings"
   data = {
       "input": "Hello, world!",
       "model": embedding_model_id # e.g. text-embedding-3-large from azure openai
   }
   response = requests.post(url, headers=headers, json=data)
   if response.status_code == 200:
       embeddings = response.json()
       print(json.dumps(embeddings, indent=2))
   else:
       print(f"Error: {response.status_code} - {response.text}")
   

Example Output

When calling /models and then sending a chat completion request to each model with /chat/completions, you might see output like:

Request successful!
{
  "data": [
    { "id": "2021.ai_mistral", "object": "model" },
    { "id": "2021.ai_chatgpt_gpt-oss-120b", "object": "model" },
    { "id": "2021.ai_chatgpt_4o", "object": "model" },
    { "id": "2021.ai_chatgpt_gpt-4.1", "object": "model" },
    { "id": "2021.ai_chatgpt_gpt-5-chat", "object": "model" },
    { "id": "2021.ai_chatgpt_gpt-5", "object": "model" },
    { "id": "2021.ai_chatgpt_o4_mini", "object": "model" },
    { "id": "2021.ai_chatgpt_o3_mini", "object": "model" },
    { "id": "2021.ai_claude_opus_4_thinking", "object": "model" },
    { "id": "2021.ai_claude_opus_4", "object": "model" },
    { "id": "2021.ai_claude_4_sonnet_thinking", "object": "model" },
    { "id": "2021.ai_claude_4_sonnet", "object": "model" },
    { "id": "2021.ai_claude_3_7_sonnet_thinking", "object": "model" },
    { "id": "2021.ai_claude_3_7_sonnet", "object": "model" }
  ]
}
___________
{'id': '2021.ai_mistral', 'object': 'model'}
Hello John Snow! It's nice to meet you. Is it John Snow like the character from Game of Thrones? I'm here to help with any questions or problems you might have, so feel free to ask me anything!
___________
{'id': '2021.ai_chatgpt_gpt-oss-120b', 'object': 'model'}
Hey there! Great to meet you, John Snow (or should I say Jon Snow? 😉). How can I help you today?
...
___________
{'id': '2021.ai_chatgpt_gpt-5', 'object': 'model'}
Hi! Nice to meet you. Should I call you John, John Snow, or something else you prefer? I also noticed the question mark—were you introducing yourself, or did you want to ask something about John/Jon Snow? How can I help today?
  

GRACE_LLM_api_calling.html Displaying GRACE_LLM_api_calling.html.