Use an inference snap from your favorite IDE

Inference snaps provide an API that can be integrated with other software. This guide explains how to integrate an inference snap with your favorite IDE. It assumes that the snap has already been installed and configured.

Install Continue

Continue enables integration of locally running models with the IDE.

VS CodeJetBrains IDEs

Install the Continue extension by searching for Continue - open-source AI code assistant in the VS Code Extensions view.

Configure Continue

Open the configuration file config.yaml. The config.yaml reference page describes the possible locations of this file. It’s usually located at ~/.continue/config.yaml.

Note

If you find a config.json file instead of the YAML file, refer to this YAML migration guide.

VS CodeJetBrains IDEs

You can also open it from the extension as follows:

1. In the Activity Bar, select the Continue logo

2. Click on the Select model drop down menu

3. In the new window, click the configuration icon (⚙️) to open config.yaml

Find the models list in the YAML file and add:

  - name: Qwen-VL
    provider: openai
    apiBase: http://localhost:8326/v3
    model: Qwen2.5-VL-3B-Instruct-ov-int4
    roles:
      - chat
      - edit

The values above are examples based on the qwen-vl inference snap. Update name, model, and apiBase to match your specific snap and its configuration. To identify the correct apiBase and model name, check out this guide on using the snap via its OpenAI API.

For additional configuration options, visit the Continue reference page.

Use the inference snap with Continue

Once the model is configured, it can be selected from the Select Model drop down at the bottom of the Continue chat box. Any requests in the chat box will be sent to the selected model.

Tip

Explore the Continue documentation to learn how to use it for coding, chat and more.