AI Infrastructure

In one line: AI is now a stack layer. Pick a model provider (Claude / GPT / Gemini), wrap it with the Vercel AI SDK (for streaming), store embeddings in pgvector, and watch your costs with Langfuse or Helicone.

In plain English

Every modern app in 2026 has at least one AI feature — chat assistance, summarization, semantic search, content generation. The AI infrastructure layer is the set of tools that makes those features practical: who you call for the model, the SDK that handles streaming and tool calls, the database that stores embeddings, and the dashboards that track cost and latency.

For deep coverage of patterns for AI features, see Chapter 9: AI Integration.

Model APIs

Provider	Strengths
Anthropic Claude	Strong reasoning, longer context, excellent for coding.
OpenAI GPT	Largest ecosystem, broad capabilities.
Google Gemini	Strong on multimodal, good pricing.
Cohere, Mistral, Together AI	Open-weight model hosting.

You'll often use multiple providers in production — different models for different tasks, with fallbacks.

SDKs

• Vercel AI SDK — Dominant TypeScript abstraction; streaming chat is trivial.
• LangChain.js / LlamaIndex — More complex agentic workflows.

// AI SDK 5 — streaming chat in a few lines:
import { streamText, convertToModelMessages, type UIMessage } from 'ai';
import { anthropic } from '@ai-sdk/anthropic';

const messages: UIMessage[] = [/* chat history from the request */];

const result = streamText({
  model: anthropic('claude-opus-4-7'),
  messages: convertToModelMessages(messages),
});

// Stream to the browser as typed UIMessage parts:
return result.toUIMessageStreamResponse();

In English: streamText calls the Anthropic API and returns a streaming result object — not a finished string. toUIMessageStreamResponse() wraps that stream in an HTTP response using Server-Sent Events so the browser sees tokens land one at a time. That "typewriter" effect every ChatGPT-style chat UI has is just this one-liner under the hood. (The full client/server pattern — and what changed from the older toDataStreamResponse() / handleSubmit API — is in Chapter 8 → Streaming Chat.)

Version stamp: AI SDK 5/6 — as of mid-2026

The stable floor is AI SDK 5 (typed UIMessage/ModelMessage, transport-based useChat). AI SDK 6 (Dec 2025) adds a first-class Agent abstraction and a stable @ai-sdk/mcp package for MCP — reach for it once a feature grows past a single prompt into a tool-using agent.

Vector databases

(Also covered in Databases.) pgvector is the popular 2026 choice — a Postgres extension, so you don't need a separate database.

Embeddings

Models that turn text/images/audio into vectors (lists of numbers representing meaning):

• OpenAI — text-embedding-3-small / large
• Voyage AI — High-quality.
• Cohere — Multilingual.

Streaming

Server-Sent Events (SSE) is the standard for streaming LLM responses to the browser. The Vercel AI SDK handles this for you; under the hood it uses SSE.

AI observability

Tool	Notes
Langfuse	Open-source, comprehensive.
Helicone	Simple proxy that adds observability.
LangSmith	LangChain's own.
Braintrust	Eval-focused (testing prompts the way you'd test code).

Highlight: AI cost discipline starts on day one

The biggest 2026 surprise for new AI feature developers: cost. A single user with a long conversation can rack up dollars in API spend if you're not careful. Habits to adopt early:

• Cache aggressively. Use prompt caching when the provider supports it (Anthropic and OpenAI both do).
• Use cheaper models for cheaper tasks. Don't call Claude Opus to classify a single sentence — use Haiku.
• Track per-request cost. Helicone, Langfuse, or your own metric collection.
• Set monthly spend limits on the provider dashboard before you ship.

One viral tweet about your app + no spend limit = an awful Monday morning.

Common mistakes

Where people commonly trip up

• Shipping a chat feature with no spend cap. One viral post + an unbounded conversation loop + Opus on every turn = a five-figure bill before lunch. Set hard monthly limits in the provider dashboard before you launch, not after.
• Routing every task through the biggest model. Classifying a sentence, rewriting a title, extracting JSON — these don't need Opus or GPT-5. Use Haiku / Mini / Flash tier models for cheap deterministic work; reserve the heavy models for genuine reasoning.
• Calling your provider key from the browser. Any key shipped to the client is public — anyone can pull it out of DevTools and burn your account. Always proxy through your backend; never put OPENAI_API_KEY in a NEXT_PUBLIC_* env var.
• Skipping prompt caching when the provider supports it. Anthropic and OpenAI both let you mark long, stable prefixes as cached — system prompts, retrieved context, conversation history — and bill the cached portion at a fraction of the cost. If you ignore this, you're paying full price to re-encode the same tokens every turn.
• Building RAG with a separate vector DB on day one. pgvector inside the Postgres you already operate is the right starting point. Pinecone/Qdrant/Weaviate add a service, a sync problem, and a bill. Move to one only when pgvector hits a measured wall.
• Treating embeddings from one model as compatible with another. OpenAI's text-embedding-3-large lives in a different vector space than Voyage's or Cohere's. Switching embedding models means re-embedding your entire corpus — pick deliberately, store which model produced each vector, and plan migrations.

Page checkpoint

Checkpoint Quiz

Did AI infrastructure stick?

Required

What's next

→ Continue to Hosting Platforms — where your code actually runs.