Integrating Continue VS Code Extension with GRACE API Models

Overview

This guide provides step-by-step instructions for integrating the Continue code assistant extension in Visual Studio Code with GRACE API models. By following these steps, you can enhance your development workflow with advanced AI-powered features, including conversational coding assistance, code refactoring, semantic search, and more.

---

1. Download and Install the Continue Extension

1. Open Visual Studio Code
   • Launch VS Code on your system.
2. Access the Extensions Panel
   • Use Ctrl+Shift+X (Windows/Linux) or Cmd+Shift+X (macOS) to open the Extensions panel.
3. Install the Continue Extension
   • Search for "Continue" in the marketplace.
   • Locate the official Continue extension and click Install.
4. Initial Configuration
   • After installation, launch the extension.
   • Select the Local Assistant option when prompted. This prepares your environment for custom YAML configuration.

2. Configure the Continue Extension for GRACE API

1. Locate or Create the Configuration File
   • Find the config.yaml file in the Continue configuration directory:
     • macOS/Linux: ~/.continue/config.yaml
     • Windows: %USERPROFILE%\.continue\config.yaml
   • If the file does not exist, create it manually in the specified directory.
2. Edit the Configuration File
   • Insert the following template, replacing placeholder values with your GRACE API details:

     name: <ASSISTANT-NAME>
     version: 0.0.1
     schema: v1
     models:
       - name: <MODEL-DISPLAY-NAME>
         provider: <PROVIDER-TYPE>
         model: <GRACE-MODEL-NAME>
         apiBase: <GRACE-API-BASE-URL>
         apiKey: <GRACE-API-KEY>
         defaultCompletionOptions:
           temperature: <MODEL-TEMPERATURE>  # Example: 0.5
           maxTokens: <MODEL-MAX-TOKENS>     # Example: 2000
         roles:
           - chat
           - edit
     # Additional models can be added here...
     context:
       - provider: <CONTEXT-PROVIDER>
       # Additional context providers...
     

3. Model Roles and Capabilities

Model roles define the specific functionalities available for each configured model in Continue. Assign roles based on your workflow needs:

• chat: Conversational interactions and general coding assistance (e.g., code explanations, debugging, discussions).
• edit: Code transformations and refactoring (e.g., restructuring code, applying design patterns).
• apply: Precise code modifications and diff applications (e.g., targeted changes, bug fixes).
• autocomplete: Real-time code suggestions as you type (e.g., productivity enhancement).
• embed: Generates embeddings for semantic code understanding (e.g., code similarity analysis, semantic search).
• rerank: Improves search result relevance and ordering (e.g., optimizing code search workflows).

4. Operating Modes

• Chat Mode (Recommended)
  • Secure, conversational environment for code discussions and learning.
  • No risk of codebase modification.
  • Ideal for code reviews, architectural planning, and conceptual discussions.
• Plan Mode
  • Read-only exploration and analysis of your codebase.
  • Safe for codebase familiarization, impact analysis, and audits.
• Agent Mode
  • Full tool access for automated codebase modifications via natural language.
  • Enables file creation, refactoring, and command execution.
  • To enable Agent/Plan modes, add tool_use capability:

    models:
      - name: My Model
        provider: openai
        model: gpt-4
        capabilities:
          - tool_use      # Enables Agent/Plan mode functionality
    

  • Note: GRACE currently has limited support for these advanced capabilities.

5. Context Providers

Context providers supply relevant information to the AI model, enhancing its responses based on your project state and environment.

• code: References functions, classes, and symbols in your project.
• docs: Integrates indexed documentation and knowledge bases.
• diff: Includes recent git changes from your current branch.
• terminal: References recent terminal commands and outputs.
• problems: Includes IDE errors, warnings, and diagnostics.
• folder: References code from specific directories.
• codebase: Enables project-wide semantic and keyword search.

How to use: In Continue, use the @ symbol to access a dropdown of available context providers.

Example context configuration:

context:
  - provider: code
  - provider: docs
  - provider: diff
  - provider: terminal
  - provider: problems
  - provider: folder
  - provider: codebase

Refer to the official Continue documentation for a complete list of context providers and configuration options.

6. Model Configuration Options

• Default Completion Options
  • contextLength: Maximum context window size (in tokens). Use higher values for complex tasks.
  • maxTokens: Maximum tokens generated per response. Adjust for desired response length.
  • temperature: Controls randomness/creativity (0.0 = deterministic, 1.0 = creative).
  • topP: Nucleus sampling for diverse token selection (recommended: 0.9–0.95).
  • topK: Limits to top K likely tokens for more predictable outputs.
  • stop: Array of strings to terminate generation at specific points.

Example model configuration:

models:
  - name: Creative-Assistant
    provider: anthropic
    model: claude-3-5-sonnet-latest
    defaultCompletionOptions:
      temperature: 0.9
      maxTokens: 2000
      topP: 0.95
      stop: ["---END---"]
      contextLength: 128000

7. Complete Configuration Template Example

name: grace_chat
version: 0.0.1
schema: v1
models:
  - name: 2021.ai Chat
    provider: mistral
    model: 2021.ai_chat
    apiBase: ${GRACE-MODEL-URL}
    apiKey: ${GRACE-SERVICE-TOKEN}
    defaultCompletionOptions:
      temperature: 0.5
      maxTokens: 2000
    roles:
      - chat
      - edit
  - name: 2021.ai Auto Completion
    provider: openai
    model: 2021.ai_auto_completion
    apiBase: ${GRACE-MODEL-URL}
    apiKey: ${GRACE-SERVICE-TOKEN}
    roles:
      - autocomplete
  - name: 2021.ai Embed
    provider: mistral
    model: mistral-embed
    apiBase: ${GRACE-MODEL-URL}
    apiKey: ${GRACE-SERVICE-TOKEN}
    roles:
      - embed
context:
  - provider: code
  - provider: docs
  - provider: diff
  - provider: terminal
  - provider: problems
  - provider: folder
  - provider: codebase

---