Section 3

Application Layer

From "Which model?" to "Can it do real work?"

INPUT LAYER DATA AND MODEL LAYER APPLICATION LAYER RAG Agents Tools / Skills / Standards Agentic Frameworks OUTPUT LAYER CHALLENGES
Application

What Changed in the Application Layer

Delegation Replaced Answers

Success shifted from “good responses” to “completed outcomes.”

RAG Became Infrastructure

Hybrid retrieval, reranking, and structure-aware pipelines replaced naive chunking.

Agent Types Diverged

Deep research, ambient automation, computer-use, and coding became distinct surfaces.

Standards Consolidated

MCP + A2A shifted into open governance; fragmentation started to recede.

RAG

Flashback: What RAG Is

RAG = Retrieve → Augment → Generate

  • Retrieve: pull the most relevant chunks from your knowledge base
  • Augment: inject those chunks into the model’s context
  • Generate: answer using retrieved evidence (ideally with citations)

Naive RAG meant one-shot retrieval and hope. It breaks on synthesis, drift, and noisy chunks.

Naive RAG pipeline

Source: Google Cloud

RAG

Then context windows increased—and people assumed RAG was over.

If you can fit a whole corpus into context, why retrieve at all? That was the belief. Reality was messier: cost, freshness, permissions, and noise didn’t disappear.

RAG

RAG didn't die. Naive RAG did.

Long context is a bigger desk. RAG is still choosing the right papers to put on it—and doing so under real-world constraints.

RAG

Why Retrieval Stayed Relevant

  • Cost control. Huge context windows are expensive. Retrieval lets you pay only for what you need.
  • Freshness. If data changes daily, you don't want to keep repacking massive context. Fetch what's current.
  • Access control. "Put it all in the prompt" breaks down when different users have different permissions.
  • Auditability. Retrieval makes it easier to show what sources were used and why.
  • Long‑context reality: Databricks finds performance often peaks, then degrades as context grows—effective context is shorter than the max window.
Long context performance curve

Source: Databricks Research Brief

RAG

RAG Grew Up: Structure Beats Chunks

What it is

1
Extract entities
2
Build graph
3
Summarize layers
4
Query top-down

Best for: policies, incident timelines, architecture tradeoffs

Why it works: captures relationships before retrieval, not after

When chunks win: narrow fact lookups with high precision

2025 trend

GraphRAG-style pipelines became shippable OSS and moved from research to production for synthesis-heavy questions.

GraphRAG pipeline

Source: Essential GraphRAG (Manning)

RAG

Retrieval Became Agentic

What it is

Plan

Rewrite into focused sub-queries

Retrieve

Parallel search across text + vectors

Fuse

Rerank and synthesize grounded context

2025 trend

Agentic retrieval shipped as platform features, with “retrieval reasoning effort” knobs and built-in semantic ranking.

Agentic retrieval architecture

Source: Microsoft Azure AI Search

RAG

Multimodal/PDF RAG: Parsing Became the Work

What it is

1
Parse layout
2
Handle images
3
Index by structure

Image → Text

Caption/OCR and index as text for retrieval

Image → Vector

Embed with multimodal models for direct search

2025 trend

Hosted file search + parsers became standard. Extraction quality became the dominant bottleneck.

Parsing Stack

Layout: sections, tables, headings, footnotes

Images: OCR + captioning + diagram text

Chunking: structure-aware splits for clean retrieval

RAG

RAG Moved Into Platforms

What it is

Managed RAG Engines

Managed vector DB + retrieval strategies (KNN/ANN) with tunable index parameters.

Hosted File Search

Vector stores that auto‑parse/chunk/embed, with query rewrite + keyword/semantic search and reranking.

Warehouse‑Native RAG

Hybrid retrieval + semantic reranking built into governed data platforms.

Platform primitives now include: vector storage, retrieval strategy, and retrieval controls

Governance pressure: keep retrieval near data, reuse platform security and access controls

2025 trend

Buy vs build shifted: teams start with managed RAG engines, hosted file search, or warehouse‑native search—and customize only where needed.

Examples

Vertex AI RAG Engine: managed vector storage, chunking, and retrieval strategies

OpenAI File Search: auto parsing/chunking + keyword/semantic search + reranking

Snowflake Cortex Search: hybrid retrieval with semantic reranking built in

RAG

Hybrid + Reranking Is the Baseline

What it is

1
BM25 + Vector
2
RRF Fusion
3
Semantic Rerank

Why hybrid

Keyword hits + semantic similarity raises recall on real queries

Why rerank

Second‑stage ranking improves precision on the short list

Knobs that matter: text recall window (maxTextRecallSize), RRF fusion, and reranker on/off

Where it shows up: hybrid queries fuse with RRF, then semantic rankers rerank top results

2025 trend

Hybrid + rerank shipped as defaults across platforms; retrieval quality became tunable engineering, not guesswork.

Where it’s baked in

Azure AI Search: RRF fusion for hybrid results + semantic reranker on top

Amazon Bedrock KB: reranker models can be applied during retrieval

Snowflake Cortex Search: hybrid retrieval + semantic reranking by default

Agents

2025 Was the Year of Agents

Research Agents

Multi‑step analysis that produces auditable reports and citations.

Computer‑Use Agents

Browser + UI automation when APIs don’t exist.

Coding Agents

IDE, terminal, and PR surfaces for real engineering work.

Workflow Agents

Ops/support/app‑building workflows with reviewable outputs.

The signal: agents shipped across categories, not just in one standout demo.

Agents

2025’s T‑Shape: Wide Wins + A Few Deep Wins

Wide (Shallow) Wins

  • Bounded workflows with review gates
  • Customer support, IT/service desk, internal ops
  • Value came from speed + coverage, not autonomy

Deep (Vertical) Wins

  • Auditable deliverables (citations, PRs, logs)
  • Deep Research‑style work where “good enough” still helps
  • Fewer domains, much higher ROI when it hits

Reality check: many projects stalled when ROI and reliability weren’t clear.

Agents

2025 was the year agent work split into distinct categories.

"Agent" stopped meaning "LLM that can call a tool" and started meaning "a system that can complete work across many steps, over time, with integration, and with guardrails."

Agents

Deep Agents: Long-Horizon Work

Deep agents handle tasks that take minutes to hours, with many steps, context management, and delegation.

Methodology: Plan → Delegate → Verify

  • Plan: break goals into verifiable subtasks
  • Delegate: route work to subagents or tools
  • Verify: check outputs before shipping
  • State: persist artifacts, not just chat history

Deep research products:

  • OpenAI Deep Research
  • Anthropic Research system (sub‑agents)
  • ChatGPT agent mode (research + action in one flow)
Anthropic research system with sub-agents

Source: Anthropic Research

Agents

Ambient & Background Agents: Always-On Automation

Deep agents proved long‑horizon work. But most production volume shifted to ambient/background agents that act on events.

They respond to:

  • Event streams, logs, monitoring alerts
  • Tickets breaching SLA, churn signals spiking
  • Build failures, incident starts, contract renewals

Core ingredients:

  • Triggers: event streams, schedules, webhooks
  • Policies: what it can do automatically vs. what needs approval
  • Memory of ongoing state: what's already handled

Background mode: async delegation that returns reviewable artifacts (PRs, reports, tickets).

Why they work: bounded actions + review gates keep autonomy safe.

Where they win

IT ops triage • Security alert routing • SLA management • Compliance checks

Agents

Computer Use: UI Control When No API Exists

Agents that operate the real surface area people use: browsers and SaaS UIs.

OpenAI Operator → ChatGPT agent mode

  • Uses screenshots to “see” and virtual mouse/keyboard to act
  • Books reservations, fills forms, places orders
  • Bridges research and action in one workflow

Anthropic Computer Use

  • Developer‑facing tool for UI automation
  • Useful when no reliable API exists

The risk

UI brittleness, broad access requirements, irreversible actions. Require approvals for anything permanent.

Agents

The best agents ask questions before they act.

A quiet 2025 shift: spec‑clarification became a built‑in step. Lovable shipped a “questions tool” because most agent mistakes start with missing requirements.

Agents

Multi-Agent: When It Helps, When It Hurts

When It Helps

  • Parallel research: Multiple agents gather evidence simultaneously, then consolidate
  • Role separation: Planner, executor, reviewer as distinct agents
  • Tool specialization: Different agents with different permissions or domains
  • Parallel attempts: Multiple solutions, choose the best

When It Hurts

  • Non-determinism: Outcomes vary more with multiple agents
  • Coordination overhead: Agents disagree or duplicate work
  • Error amplification: One agent's wrong assumption spreads
  • Cost: You pay for parallel runs
Agents

Coding agents became the first place many teams experienced real agents.

Why? Software work has the perfect control surface: repos, tests, CI, and pull requests. Every step is visible. Humans steer via PR review. Claude Code, Cursor, and Copilot made this real.

Agents

Three Surfaces for Coding Agents

IDE Agents

Multi-step work inside your editor. Finds files, edits across modules, runs tests, fixes and retries. Cursor’s agent-first IDE pushed this surface forward.

Repo & PR Agents

Work through issues and pull requests in CI. Produce PRs, logs, and reviewable commits. GitHub Copilot coding agent made this a mainstream workflow.

Terminal Agents

Agentic coding from the command line. Delegates substantial engineering tasks from the terminal—Claude Code made this feel native.

Agents

Coding Agents: From Snippets to Full Tasks

  • Then: tab completion and small snippet help
  • Now: multi-file edits, tests, and CI-aware workflows
  • Shift: from “assist in-editor” to “deliver reviewable work”
  • Surface: PRs + agent workspaces became the control plane

2025 was the year coding agents moved from suggestions to execution.

Coding agents evolution timeline (part 1) Coding agents evolution timeline (part 2)

Source: internal 2025 timeline

Agents

The Foundation: Tools Become Standard (Late 2024 → Early 2025)

Key beats

  • MCP made tool connectivity a standard (\"USB‑C for tools\")
  • Agent = model + tool protocol + runtime permissions
  • Integrations shifted from bespoke plugins to reusable wiring
  • Product signal: MCP servers + registries became a real ecosystem

Once tools became standard, the rest of the stack could scale.

Nov 2024 Feb 2025

Tool Hub

GitHub • Jira • DB • Slack

Client Surfaces

IDE • Terminal • Web

Standard tool wiring

Agents

From Chat to Execution Loops (Feb → Jun 2025)

Key beats

  • Claude Code: terminal‑native agent workflow (edit, run tests, commit)
  • Codex CLI + cloud tasks: delegated work with logs + patches
  • Dev loop shifts to: plan → edit → run → verify → commit

You stop copying snippets; you start handing off tasks.

$ run tests

✓ 48 passed

$ edit module

✓ patch ready

$ git status

Plan

↓

Implement

↓

Test

↓

Review

Agents

Control + Quality at Speed (Jul → Sep 2025)

Key beats

  • Cursor: To‑dos + queues made long tasks steerable
  • Bugbot / agent review: scaled PR quality checks
  • Session resume: longer context + memory reduced “agent forgot”

Steerability

Queue next task
Review intermediate plan
Resume with memory
TODO: refactor auth flow
TODO: add retry tests

PR Guardrails

+ if (!isValid) return err
+ await saveRecord()
// agent review: missing retry path
// add unit coverage for edge case
logic security tests

Session Resume

Checkpoint: auth refactor
Context pack compressed
Resume work in new session

Context snapshot · 12 files · 3 decisions

Agents

Scale & Reuse (Oct → Dec 2025)

Key beats

  • Workflow‑native agents fit real team processes
  • Parallel agents work in isolated worktrees
  • Skills/plugins package reusable competence

Organizations can standardize behavior instead of re‑prompting.

Issue → @agent → sandbox run → PR → CI → review → merge

Release PR

Skills pack

Migration

Skills pack

Test Fixer

Skills pack

Parallel agents fan‑out → isolated worktrees

Agents

Methodologies for Building with Coding Agents

Spec‑Driven Development

Write the spec first, then plan tasks, then implement. Keeps agent work aligned to intent.

Common flow: specify → plan → tasks → implement.

Research → Plan → Implement

Separate exploration from execution, then lock a plan before code changes.

Human leverage points: review research + plan before commit.

Verification‑First Loops

Run/verify cycles keep agents honest—reproduce, patch, re‑run tests, report.

Plan → edit → run → verify.

Human leverage review points for coding agents Human leverage for compressing context and checkpoints

Source: Humanlayer ACE

Agents

Why Enterprise Agents Still Fail

Skills

Skills: Packaged Expertise for Agents

Agents execute.

They reason, call tools, handle multi-step workflows. But agents are general-purpose—they don't inherently know your domain.

Skills equip.

Folders of instructions, scripts, and resources that agents load on demand. BigQuery queries. NDA review procedures. PDF extraction. Domain expertise, packaged and portable.

Agent + Skills + Virtual Machine

Agent configuration includes equipped skills and MCP servers. Skills live as directories in the agent's file system—loaded when relevant. Source: Anthropic

Frameworks

Frameworks stopped being libraries. They became runtimes.

Agents in 2025 aren’t just prompts and tools—they run inside systems built for state, control, and recovery.

Frameworks

What Changed in Agentic Frameworks

Explicit Workflows

Event-driven graphs with named steps and typed state replaced ad-hoc loops. The flow is visible and debuggable.

Durable State + Human Gates

Checkpoint, pause, and resume became native. Humans can approve, redirect, or recover long-running tasks.

Scalable Tool Use

Tool search + programmatic calling made orchestration explicit, cheaper, and more reliable than prompt-only routing.

Multi-Agent: Supported, Not Default

Coordination primitives matured, but teams learned to use multi-agent as an advanced pattern, not the default.

Tracing + Evals by Default

Observability and quality scoring moved into the runtime so agents can be monitored and graded.

Standards

2025 was the year everyone realized fragmentation would kill adoption.

Two different interoperability problems emerged. How do AI apps talk to tools and data sources? How do agents coordinate with other agents? The industry started racing toward standard rails.

Standards

Two Standards, Two Problems

MCP: Tool Connectivity

A universal connector contract so AI apps can talk to tools and data sources consistently.

  • Standardized tool descriptions
  • Any agent can understand any MCP-compatible tool
  • Reduces integration cost over time

A2A: Agent-to-Agent

A protocol for agents coordinating with other agents across systems.

  • Different layer than MCP
  • Enables multi-vendor agent collaboration
  • Critical for complex enterprise workflows
Standards

The Protocol Wars: From Fragmentation to Open Governance

Standards

Standards reduce vendor lock-in. Standards reduce integration cost. Open governance wins.

The move to Linux Foundation wasn't just about technology. It was about trust. Enterprises adopt standards they can rely on outlasting any single vendor's strategy.

Takeaways

Application Layer: Key Takeaways