Bring your own model to GitLab Duo Self

Published on: September 8, 2026 17 min read Learn how to connect GitLab Duo Self-Hosted to GPT, Claude, Llama, and Mistral models hosted in Microsoft Foundry, and how to choose between them. For many organizations, the question about AI coding tools isn't whether they help, but where the code goes. Teams under data sovereignty, residency, or regulatory constraints need to know which network handles their source code before they can adopt anything. GitLab Duo Self-Hosted answers that by letting administrators connect GitLab Duo features to models running on infrastructure they choose, with control over hosting, region, network path, and credentials. In this tutorial, we'll connect GitLab Duo Self-Hosted to models hosted in Microsoft Foundry. Foundry is worth treating as a general-purpose serving platform rather than an OpenAI endpoint: Its catalog spans OpenAI GPT, Anthropic Claude, Meta Llama, and Mistral, and the families GitLab supports overlap heavily with what Foundry offers. That matters because GitLab lets you assign a model per feature. You can run one family for agentic work, a code-specialized model for Code Suggestions, and a smaller model where request volume dominates, all deployed and billed through one Azure subscription. The setup steps are the same regardless of family. Only the model you deploy, the deployment name you reference, and the Model family you select in GitLab change. Microsoft's catalog moves quickly and will contain releases newer than GitLab's supported-model matrix. Catalog availability alone does not establish GitLab Duo support, so always check both vendors' documentation before choosing a model. GitLab Duo Self-Hosted lets you connect GitLab Duo to models deployed on-premises or through a supported cloud provider. Microsoft Foundry is useful when your organization already operates in Azure and wants to manage model deployments, access, networking, and consumption there. Key benefits include: The GitLab Duo Self-Hosted solution consists of three core components: A single AI Gateway serves every model you configure, and GitLab decides which deployment to call based on the feature that made the request: flowchart LR subgraph self["Your infrastructure"] GL["Self-Managed<br>GitLab"] GW["AI Gateway<br>(port 5052)"] end subgraph azure["Your Azure tenant"] subgraph foundry["Microsoft Foundry"] M1["duo-chat<br>Agentic Chat, Agent Platform"] M2["duo-code-gen<br>Code generation"] M3["duo-code-completion<br>Code completion"] end end GL -- HTTPS --&gt; GW GW --&gt; M1 GW --&gt; M2 GW --&gt; M3 Note: You can use another serving platform if you are running on-premises or using another cloud provider. This shape of deployment is what makes GitLab Duo workable for teams with data sovereignty obligations, so it's worth being precise about where data travels. In a fully self-hosted configuration, GitLab's documentation states that inference data — code inputs, model prompts, and model responses — does not leave your network. Requests go from your GitLab instance to your AI Gateway to your Foundry deployment, all within infrastructure you control. GitLab also does not capture which model or model provider you use. What does leave, on an online license, is billing metadata: an instance ID, a de-identified user ID, a call count, and a timestamp. On an offline license, your instance doesn't connect to GitLab's billing components at all. Two conditions attach to that: Choosing a model means satisfying two independent constraints: GitLab must support it, and Foundry must offer it. Neither implies the other. GitLab rates each supported model against four capability areas: code completion, code generation, GitLab Duo Agentic Chat, and GitLab Duo Agent Platform. These families appear in both GitLab's supported-model table and the Foundry catalog: | Family | How Foundry offers it | What to know | |---|---|---| | GPT | Sold by Azure | Deepest overlap with GitLab's table; simplest path. Includes general-purpose, coding-optimized, and smaller low-latency variants | | Claude | From partners, via Azure Marketplace | Strong agentic ratings; extra Marketplace prerequisites and narrower region coverage | | Llama | Sold by Azure and from partners | Ratings vary sharply by model size and feature | | Mistral | From partners | Includes Codestral, which is code-specialized | Specific version numbers move quickly on both sides, so this post deliberately avoids naming a "best" model. Instead, pick using these rules: Always confirm current ratings in GitLab's supported models page before you commit, and treat any model named in this post as an example rather than a recommendation. Before we begin, you'll need: Note: If you aren't a GitLab customer yet, you can start a free trial of GitLab Ultimate. 1. Deploy one or more models in Microsoft Foundry Open the Foundry portal, select your project, and deploy your chosen models from the catalog. Model availability varies by region, cloud, and deployment type, and some subscriptions require an approved quota increase. Before you deploy, confirm your model is offered in the region and deployment type you intend to use. Microsoft publishes this in Region availability for Foundry Models sold by Azure, with separate tabs for standard, provisioned, and batch. Coverage is uneven: a model can be broadly available as Global Standard yet offered in only a handful of regions as Regional Provisioned Managed. Partner models are more constrained than Azure OpenAI models. Claude, for example, is concentrated in a small number of regions, and deploying it requires accepting Marketplace terms. Verify both region and subscription eligibility before committing to a family. Give each deployment a name you can trace later. GitLab references the deployment name rather than the catalog model name, so a vague name is hard to audit. Naming by role rather than by model version, such as duo-chat or duo-code-completion , has a practical advantage: when you upgrade to a newer model, you can point the deployment at it without editing the model identifier in GitLab. Naming by version, such as duo-gpt-5-2 , is more explicit but means reconfiguring GitLab on every model change. If you're starting out, deploy a single model and expand once it works end to end. After deployment, record these values for each deployment: Use a secret-management process approved by your organization. Do not commit the API key to a repository. 2. Install the AI Gateway The AI Gateway routes requests between GitLab and the selected model endpoint. Install it with Docker or the Helm chart by following the current AI Gateway installation guide. Avoid copying an old image tag or API-version example from another tutorial. The AI Gateway and GitLab versions should remain compatible, and GitLab's installation guide provides the current image and required settings. The Foundry endpoint and API key are entered when you add each model to GitLab in step 4; they do not need to be embedded in the example Docker command. One gateway serves every model you configure, so you don't need a separate gateway per deployment or per model family. 3. Configure GitLab to access the AI Gateway Now that the AI gateway is running, configure your GitLab instance to use it: https://ai-gateway.example.com:5052 ).For production deployments, use TLS and restrict network access to the gateway. If the gateway uses a private IP address or internal hostname, add it to GitLab's outbound-request allowlist. 4. Add each deployment to GitLab Repeat this procedure once per Foundry deployment you want GitLab to use. Agentic Chat on Microsoft Foundry .azure/YOUR-DEPLOYMENT-NAME , using the exact deployment name from Foundry. For example, azure/duo-chat .The prefix describes how a model is served, not who built it. Foundry can expose non-OpenAI models through the Azure OpenAI endpoint, in which case azure/ still applies. If you deploy a partner model that you reach through a different Foundry endpoint, confirm the correct prefix in GitLab's configuration documentation rather than assuming. A mismatched prefix produces the "Model not found" error described below. 5. Assign models to GitLab Duo features This is where the breadth of the catalog pays off, because features do not have to share one model or even one family. One reasonable split: Treat that split as a starting hypothesis rather than a tuned configuration. Begin with a single model across the features you plan to enable so you have a clean quality, latency, and cost baseline, then introduce a second model only where your own measurements justify it. To ensure that your GitLab Duo Self-Hosted implementation with Microsoft Foundry is working correctly, perform these verification steps: 1. Run the health check After running the health check of your model to be sure that it's up and running, return to the GitLab Duo section from the Admin page and click on Run health check. This will verify if: If the health check reports issues, refer to the troubleshooting guide for common errors. 2. Scaffold a FastAPI service with code generation Code generation is a good first test because it exercises the whole path end to end and produces an unmistakable result. It also sends more context than code completion, so a misconfigured endpoint or an undersized quota shows up immediately. Code Suggestions has two distinct behaviors, and knowing which one you're triggering matters when you interpret the result: To scaffold a FastAPI service: main.py . Code generation is more accurate when the file has fewer than five lines, so an empty file is the ideal starting point. # Create a FastAPI service with a health check endpoint and CRUD endpoints for # a "tasks" resource backed by an in-memory list. Use Pydantic models for # request and response bodies, and return appropriate HTTP status codes. Naming the framework explicitly is what makes this work. GitLab's guidance for code generation is to state the outcome, stay specific but concise, and name the library or framework you want. A vaguer comment such as # web service gives the model far less to work with. Two things to expect. Code generation output is capped at roughly 2048 tokens, so you'll get a solid scaffold rather than a finished application. And because these models are non-deterministic, the same comment won't produce identical code twice, which is normal rather than a sign of misconfiguration. If you assigned different models to different features, test them separately so a failure points at one deployment. Confirming code generation also confirms that the model you mapped to Code generation in Step 5 is the one actually serving the request. 3. Check AI Gateway logs Review the AI gateway logs to see requests being routed to your Microsoft Foundry deployments: In your terminal, run: docker logs gitlab-ai-gateway --tail 100 -f You should see log entries indicating successful requests to the configured model endpoint. Avoid enabling prompt logging unless your organization's data-handling policy permits it. Here are the next steps to take. The GitLab team actively tests each model's performance for each feature and provides tier ranking of model's performance and suitability: Do not infer compatibility from the model name or from its presence in the Foundry catalog. Use GitLab's models and hardware requirements page as the source of truth for GitLab's current ratings. Cost optimization strategies: While this guide focuses on Microsoft Foundry integration, GitLab Duo Self-Hosted supports multiple deployment options: Microsoft Foundry offers unique advantages for organizations already invested in the Azure ecosystem: These capabilities can support a compliance program, but they do not make a GitLab Duo deployment compliant by themselves. Validate the complete architecture, data flows, logging, retention, contracts, and operational controls against your requirements. GitLab Duo Self-Hosted with Microsoft Foundry lets organizations serve GitLab Duo features from Azure-hosted models. Because Foundry's catalog spans GPT, Claude, Llama, and Mistral, and GitLab supports models from each, one Azure subscription can cover the families you need. Start with one broadly capable model, then assign different models to different features as your measurements justify. Model names in this post are examples, not recommendations. Both catalogs change frequently, so treat the two vendor matrices as the source of truth at the time you deploy. The durable lesson is to treat model selection as a compatibility decision, not a catalog-shopping exercise. Check GitLab's support matrix, confirm availability and lifecycle status in Microsoft Foundry, deploy the models, and then validate them with your own workloads before broad rollout. Enjoyed reading this blog post or have questions or feedback? Share your thoughts by creating a new topic in the GitLab community forum. Share your feedbackStart building faster today See what your team can do with the intelligent orchestration platform for DevSecOps.