Django + LLM

How to Add OpenAI or Claude to a Django App: A Production Pattern

2026-11-09 · by Talha Jaleel

Quick answer

Calling an LLM from Django takes five lines, but surviving real users takes a provider-agnostic service layer that wraps OpenAI and Claude behind one function, streaming for perceived speed, background jobs for long tasks, caching and per-call cost logging, and security basics like server-side keys and per-user rate limits.

How to add OpenAI or Claude to a Django app: a production pattern

Calling an LLM from Django takes five lines. Doing it in a way that survives real users (slow responses, rate limits, rising costs, provider outages) takes a bit more structure.

This is the pattern I use when adding OpenAI or Anthropic's Claude to Django apps for clients. It is provider-agnostic, so you can switch models without touching your views. If you are still choosing a framework, see [Django vs FastAPI for AI applications](/blog/django-vs-fastapi-for-ai-applications) first.

What we're building

A service layer that wraps both providers behind one function
A streaming view so users see output as it is generated
Background jobs for long-running tasks
Caching and usage logging to control cost
Security basics that most tutorials skip

Step 1: Install and configure

Install the official SDKs (OpenAI Python SDK, Anthropic Python SDK):

pip install openai anthropic

Both SDKs read their API keys from environment variables (OPENAI_API_KEY, ANTHROPIC_API_KEY). Never put keys in settings.py or your repo.

# settings.py
import os

LLM_PROVIDER = os.environ.get("LLM_PROVIDER", "anthropic")  # "openai" or "anthropic"
LLM_MODEL = os.environ["LLM_MODEL"]  # set to a current model name for your provider

Reading the model name from the environment means upgrading models is a config change, not a deploy.

Step 2: Build a provider-agnostic service layer

Do not call the SDKs directly from views. Put every LLM call behind one module. This gives you one place to add retries, logging, caching, and provider switching. The reasoning is covered in more depth in my production LLM integration guide.

# ai/services.py
from dataclasses import dataclass
from functools import lru_cache

from django.conf import settings


@dataclass
class LLMResult:
    text: str
    input_tokens: int
    output_tokens: int


@lru_cache
def _openai_client():
    from openai import OpenAI
    return OpenAI(timeout=30, max_retries=2)


@lru_cache
def _anthropic_client():
    from anthropic import Anthropic
    return Anthropic(timeout=30, max_retries=2)


def complete(system: str, user: str, max_tokens: int = 1024) -> LLMResult:
    if settings.LLM_PROVIDER == "openai":
        resp = _openai_client().chat.completions.create(
            model=settings.LLM_MODEL,
            max_completion_tokens=max_tokens,
            messages=[
                {"role": "system", "content": system},
                {"role": "user", "content": user},
            ],
        )
        return LLMResult(
            text=resp.choices[0].message.content or "",
            input_tokens=resp.usage.prompt_tokens,
            output_tokens=resp.usage.completion_tokens,
        )

    resp = _anthropic_client().messages.create(
        model=settings.LLM_MODEL,
        max_tokens=max_tokens,
        system=system,
        messages=[{"role": "user", "content": user}],
    )
    return LLMResult(
        text="".join(block.text for block in resp.content if block.type == "text"),
        input_tokens=resp.usage.input_tokens,
        output_tokens=resp.usage.output_tokens,
    )

A few deliberate choices here: timeouts and retries are set on the client, because LLM calls can hang and a 30-second timeout keeps a slow provider from tying up your workers; clients are created once and reused, not rebuilt on every request; and token counts are returned so you can log cost (Step 5).

Step 3: Stream responses to the user

LLM responses can take several seconds. Streaming shows text as it is generated, which feels dramatically faster.

# ai/services.py (continued)
def stream(system: str, user: str, max_tokens: int = 1024):
    if settings.LLM_PROVIDER == "openai":
        events = _openai_client().chat.completions.create(
            model=settings.LLM_MODEL,
            max_completion_tokens=max_tokens,
            messages=[
                {"role": "system", "content": system},
                {"role": "user", "content": user},
            ],
            stream=True,
        )
        for chunk in events:
            if chunk.choices and chunk.choices[0].delta.content:
                yield chunk.choices[0].delta.content
        return

    with _anthropic_client().messages.stream(
        model=settings.LLM_MODEL,
        max_tokens=max_tokens,
        system=system,
        messages=[{"role": "user", "content": user}],
    ) as events:
        for text in events.text_stream:
            yield text

# ai/views.py
from django.contrib.auth.decorators import login_required
from django.http import StreamingHttpResponse
from django.views.decorators.http import require_POST

from .services import stream

SYSTEM_PROMPT = "You are a helpful assistant for our product. Answer concisely."


@login_required
@require_POST
def ask(request):
    question = request.POST.get("question", "")[:2000]
    response = StreamingHttpResponse(
        stream(SYSTEM_PROMPT, question),
        content_type="text/plain; charset=utf-8",
    )
    response["X-Accel-Buffering"] = "no"  # stop nginx from buffering the stream
    response["Cache-Control"] = "no-cache"
    return response

Two deployment notes: proxies buffer by default, so the X-Accel-Buffering header handles nginx (check your host's docs if you are behind another proxy or CDN); and under ASGI, Django prefers async iterators for streaming responses. The sync generator above works but Django will warn; for high-traffic ASGI deployments, use the SDKs' async clients with an async generator.

Step 4: Move long tasks to background jobs

Summarizing a 50-page document or processing a batch of records should not block a web request. Use a task queue like Celery:

# ai/tasks.py
from celery import shared_task

from .models import Document
from .services import complete


@shared_task(bind=True, autoretry_for=(Exception,), retry_backoff=True, max_retries=3)
def summarize_document(self, document_id: int):
    doc = Document.objects.get(pk=document_id)
    result = complete(
        system="Summarize the document in 5 bullet points for a busy executive.",
        user=doc.text[:100_000],
    )
    doc.summary = result.text
    doc.save(update_fields=["summary"])

The view enqueues the job (summarize_document.delay(doc.id)) and the frontend polls or listens for completion. In production, narrow autoretry_for to the SDKs' rate-limit and connection errors so you do not retry bugs in your own code.

Step 5: Cache repeated calls and log every call's cost

LLM costs creep up quietly. Two cheap habits prevent most surprises. Cache identical requests with Django's caching framework:

# ai/services.py (continued)
import hashlib

from django.core.cache import cache


def cached_complete(system: str, user: str, ttl: int = 60 * 60 * 24) -> LLMResult:
    raw = f"{settings.LLM_PROVIDER}:{settings.LLM_MODEL}:{system}:{user}"
    key = "llm:" + hashlib.sha256(raw.encode()).hexdigest()
    hit = cache.get(key)
    if hit is not None:
        return hit
    result = complete(system, user)
    cache.set(key, result, ttl)
    return result

Only cache where the same input should give the same answer (classification, extraction, FAQ-style questions), and never across users when the prompt contains private data. Then log usage per call so you can see cost by feature and by user:

# ai/models.py
from django.conf import settings
from django.db import models


class LLMCall(models.Model):
    user = models.ForeignKey(settings.AUTH_USER_MODEL, null=True, on_delete=models.SET_NULL)
    feature = models.CharField(max_length=64)
    model = models.CharField(max_length=128)
    input_tokens = models.PositiveIntegerField()
    output_tokens = models.PositiveIntegerField()
    created_at = models.DateTimeField(auto_now_add=True)

With this table you can answer "which feature costs the most?" in one query. For more ways to bring the bill down, see cutting LLM API costs in production.

Step 6: Security basics

Keys stay server-side. Never call LLM APIs from the browser with your key.
Rate-limit per user. One script hammering your endpoint can run up a large bill fast.
Cap input size, as the view above does, to control cost and abuse.
Treat user input as untrusted. Prompt injection is real: do not give the model access to tools or data the current user should not reach.
Keep tenants separate. If your prompts include customer data, make sure one user's data can never end up in another user's context or cache.

Next steps

This pattern covers calling a model with your own prompt. Most real products eventually need the model to answer from your own data, which is RAG, and it is where accuracy problems usually start. If you are planning one, start with a RAG proof of concept on a small slice of real data.

Frequently Asked Questions

Can I use OpenAI and Claude in the same Django app?

Yes. With a service layer like the one above, the provider is a setting. Some teams route different features to different providers based on cost or quality.

Should LLM calls go in Django views?

Short calls can, as long as they go through a service layer. Long-running work belongs in a background job so it does not block web workers.

Is Django good for AI applications?

Yes, especially when the AI feature lives inside a larger product with users, permissions, and an admin. For AI-only microservices with heavy async traffic, FastAPI is often a better fit. I compare them in detail in Django vs FastAPI for AI applications.

How do I stream ChatGPT or Claude responses in Django?

Use StreamingHttpResponse with a generator that yields text chunks from the SDK's streaming API, and disable proxy buffering.

Sources

Talha Jaleel

I build custom operations software and local and AI-search systems for home service contractors. My most recent build was a multi-division field operations platform and local SEO program for a home services contractor in Colorado. If you want to talk through your operation, get in touch.

Building something like this?

I build LLM features into Django, FastAPI, and Next.js products for US startups.If you're scoping a project, I can tell you what it would take for your setup in a quick call.

Book a 30-min scoping call See the case study See services