Lessons from Trillion Token Deployments at Fortune 500s — Alessandro Cappelli, Adaptive ML
Summary
Alessandro Cappelli, co-founder and Chief Customer Officer at Adaptive ML, presents the case for reinforcement learning (RL) as the only industrially viable path to deploying LLMs in Fortune 500 production environments. Drawing on deployments with AT&T, Manulife, and CCS (a medical supply company), Cappelli argues that the widely cited 95% GenAI pilot failure rate stems not from a deployment or prompt engineering gap but from a fundamental feedback integration problem.
Cappelli's central thesis is what he calls "the myth of the last mile." Most teams treat the MVP demo as the hard part and assume a short final mile to production. In reality, the journey from MVP to production — and sustained improvement beyond it — is the true marathon. Instruction fine-tuning and proprietary models both fall short here: fine-tuning requires creating new datasets every iteration cycle, and proprietary models offer only system prompt adjustments with no systematic way to track or resolve defect classes. RL, by contrast, mathematically integrates feedback signals from multiple sources — LLM judges, direct KPIs, and business outcome metrics — enabling continuous model lifecycle acceleration.
A concrete example from AT&T illustrates the tokenomics argument: AT&T summarizes every customer-agent transcript at a cost of millions of dollars annually. RL enables training a substantially smaller model to match the performance of a large frontier model (e.g., ChatGPT or Claude Sonnet), yielding lower serving costs and, critically for speech-to-speech customer support systems, latency under one-third of a second — a hard constraint that large models cannot satisfy. Models in the 10B-parameter family (Gemma 4, Mistral, Qwen) become viable for production when RL-tuned rather than simply prompted or instruction-fine-tuned.
For agentic workloads, Cappelli argues the RL advantage widens further. Agents consume 10x or more tokens compared to simple summarization tasks, making tokenomics even more critical. Agents also have direct access to live databases, raising the error-tolerance bar significantly. RL was originally designed for robotic agents operating in environments, so it maps naturally to tool-using LLM agents. Adaptive ML supports two scenarios: plugging a trainable model into an existing agentic workflow (as with Manulife's settled agent architecture), or constructing a mock environment with simulated tools and a mock user (itself an LLM) when no live environment exists.
On the data scarcity problem — particularly acute for agents, since no web-scraped dataset of tool-using LLM trajectories exists — Cappelli notes that RL training itself generates a synthetic data pipeline as a byproduct. The reward function labels which trajectories are good, enabling rejection sampling to bootstrap an initial training dataset. Existing proprietary data (e.g., real customer call transcripts) can train the mock user to simulate realistic, sometimes distressed callers, as in the CCS medical supply case where callers may be panicked.
Regarding human-in-the-loop, Cappelli reframes the RLHF narrative: rather than expensive annotation campaigns, humans define rubrics and system prompts for LLM judges, an exercise taking hours rather than weeks. CCS maximizes containment rate (calls resolved end-to-end by the model) as a direct scalar reward. Early in deployment, a large prompted model (e.g., Qwen 235B) acts as the judge; as production feedback accumulates into thousands of labeled examples, a dedicated reward model is trained to scale the signal.
The PPO algorithm requires orchestrating four LLMs simultaneously, and newer algorithms like GSPO add further complexity. Adaptive Engine abstracts this with pre-built training recipes so enterprises configure rubrics and business outcomes without implementing RL from scratch. Cappelli closes by positioning the Adaptive Engine as a holistic RL Ops platform covering observation, training, and serving in a unified lifecycle.
Key Takeaways
- •The hard part of GenAI is not building an MVP but continuously improving the model post-launch — RL is the only systematic mechanism for integrating production defects and business metrics into model updates.
- •Proprietary models and instruction fine-tuning both lack a principled feedback loop; switching system prompts to fix one defect class typically introduces others, with no mathematical way to track the tradeoff.
- •RL enables training smaller models (10B-class) to match large frontier model performance, which is often the only way to satisfy hard latency constraints (under 333ms) and reduce serving costs at scale.
- •For agents, the RL advantage amplifies: agentic workloads use 10x more tokens, have near-zero tolerance for errors affecting live databases, and map directly to the environments RL was originally designed for.
- •If no real agentic environment exists, mock it: simulate tools and use an LLM as the mock user, trained on real proprietary transcripts to exhibit realistic caller behavior including distressed or repetitive users.
- •RL training on an environment generates a synthetic trajectory dataset as a byproduct via rejection sampling — solving the agent training data scarcity problem without requiring a pre-existing labeled dataset.
- •Replace expensive annotation campaigns with rubric-definition sessions (hours, not weeks): humans define LLM judge system prompts; as production data accumulates, train dedicated reward models to scale the signal.
- •Direct scalar business KPIs (e.g., call containment rate, code execution success) make the strongest reward signals and should be preferred over open-ended LLM judge rubrics wherever measurable.
- •Early-stage deployment can use a large prompted model (e.g., Qwen 235B) as the judge; after thousands of production-labeled examples accumulate, distill into a trained reward model for cost-effective scaling.
Transcript
Hello everyone. I am Alessandro Cappelli. I'm co-founder and chief customer officer at Adaptive ML. At Adaptive ML we build an RL ops platform as in reinforcement learning operation that allows large enterprises like AT&T, Manulife, CCS to build, evaluate, and serve in production their own specialized large large language models. I'm I'm here to show you how reinforcement learning RL is not just any other algorithm for post-training, but is an an algorithm that at its core will bring models to p...
Speakers
Topics
Related Sessions
