Applied Researcher Brief | AI Engineer Europe 2026 — ConferenceDigest
Applied Researcher Brief
For: Applied researchers working on RL, training, and model internals
Applied Researcher Intelligence Brief
AI Engineer Europe 2026
*Prepared for: Applied researchers working on RL, training, and model internals*
Executive Summary
AI Engineer Europe 2026 surfaced a cluster of findings with direct relevance to applied researchers: the scaling consensus is fracturing, post-training techniques are now the primary lever for capability improvement at every scale, and the boundary between training-time and inference-time compute is dissolving. Sara Hooker of Adaption Labs delivered the sharpest challenge to orthodoxy, arguing that 95% of weights can be pruned post-training, that doubling model size captures only a long tail of rare artefacts at high cost, and that her AutoScientist system — which self-improves training co-optimisation across 30+ models — already outperforms Adaption's own human research staff on cross-architecture tasks and can train a frontier model in two days. This is not a fringe position: it resonated with complementary evidence from Maxime Labonne (Liquid AI), Cassidy Hardin (Google DeepMind), and Stephen Batifol (Black Forest Labs).
On the RL side, the conference confirmed that Reinforcement Learning with Verifiable Rewards (RLVR) and Group Relative Policy Optimisation (GRPO) have become the de-facto post-training stack, displacing expensive PPO and curated SFT-only pipelines. Stefano Fiorucci provided the most detailed public walkthrough of GRPO training mechanics, including curriculum design, stratified sampling, and the critical role of batch size (below 256 causes collapse). Maxime Labonne at Liquid AI extended this with a rigorous account of doom-loop detection and mitigation in small reasoning models — a failure mode that afflicts over 50% of rollouts in naive scaled-down large models and drops to near zero with combined on-policy DPO and RLVR. These practical details are rarely in papers.
Architecturally, two trends deserve sustained attention. First, Gemma 4's interleaved local/global attention with differentiated GQA by layer type and Per-Layer Embeddings for on-device deployment represent a coherent response to the memory-bandwidth bottleneck that limits small-model deployment — Cassidy Hardin's deep dive made the design tradeoffs unusually legible. Second, Black Forest Labs' SelfFlow paper introduced a self-supervised dual-model (student/teacher EMA) architecture that replaces external encoder dependencies (DINOv2/DINOv3) with joint generation-and-representation learning, outperforming flow-matching baselines across audio, images, and video simultaneously, and enabling robotic action prediction as an emergent capability. This is among the most architecturally novel findings from the conference.
For applied researchers, the overarching signal is that production-relevant model work has shifted dramatically toward post-training, inference-time efficiency, and systematic evaluation — with scaling laws for pre-training being actively challenged. The most productive intersection for this audience is the space between small-model RL training (LFM2.5, GRPO mechanics), architecture search for bandwidth-constrained inference, and the emerging benchmark infrastructure (Kaggle Standardised Agent Exams, Terminal Bench) that is beginning to provide rigorous evaluation signal for agentic capabilities.
Key Findings
Finding 1: The Post-Brute-Force Scaling Era Is Arriving Faster Than Expected
Sara Hooker (Adaption Labs) provided the conference's most forceful challenge to scaling orthodoxy in a summary session at AI Engineer Singapore Day 2. Her core claims: small models can outperform much larger ones; 95% of weights can be pruned post-training without catastrophic capability loss; and doubling model size captures only the long tail of rare artefacts at disproportionate compute cost. Crucially, these are not theoretical positions — Adaption's AutoScientist system self-improves training co-optimisation across more than 30 models on Together AI infrastructure, trains a frontier model in two days, and has demonstrably outperformed Adaption's own human research staff on cross-architecture tasks. The system has processed 27 million data points covering 242 languages in four weeks.
This aligns with the AIE Europe keynote summary where Andrew Wilson (Anthropic) traced the improvement in Claude Code's minimum harness score at 50% task completion rising from roughly 1 hour with Claude Opus 3.7 to 12 hours with Opus 4.6 — attributing this not to parameter scaling but to post-training improvements (context self-awareness, server-side compaction, specific agentic capability work). The pattern across multiple labs is the same: marginal returns from raw scale are declining; returns from targeted post-training, RL, and architecture-aware inference optimisation are growing.
For applied researchers, the implication is that the empirical frontier has moved. The most actionable work is not pre-training larger models from scratch but rather understanding which post-training interventions have the largest per-compute return, how pruning interacts with RL fine-tuning, and what evaluation frameworks can detect capability gaps created by overfitting on dense but unrepresentative benchmarks.
Finding 2: RLVR + GRPO Has Become the Production RL Stack — With Concrete Engineering Pitfalls
Stefano Fiorucci (Deepset/Haystack) delivered the conference's most technically detailed walkthrough of building RLVR environments using the Verifiers library (Prime Intellect), training a Liquid AI LFM-2 small model to play Tic-Tac-Toe as a benchmark task. Several findings are directly transferable to research practice.
On algorithm selection: GRPO has displaced PPO as the default choice for language model RL at small-to-medium scale because it avoids the high cost of curated SFT data (as demonstrated by DeepSeek R1). The CISPO optimizer is presented as a further improvement over vanilla GRPO. On training stability: batch size below 256 caused training collapse in this environment — a practical finding that does not appear in most RLVR papers. On curriculum design: opponent skill curricula using minimax with configurable random-move probability (20–70%) and stratified sampling across difficulty levels were necessary for stable training. On seed management: per-turn deterministic seeds derived from example seed and board state are needed to reduce noise across GRPO group rollouts. On cold-start: starting from a non-reasoning instruct model outperformed starting from a reasoning model when GPU budget is limited, because reasoning models produce long thinking traces that hit truncation limits.
The result after full training: LFM-2 dominated random opponents and drew 85% of the time against an optimal minimax opponent — outperforming GPT-4o-mini (the teacher model used for SFT warm-up data) on optimal play. Fiorucci explicitly cited Ilya Sutskever's NeurIPS 2024 comment that pre-training is no longer sufficient, and DeepSeek R1's open-sourcing of RLVR methodology, as the motivation for this work. The Verifiers library supports MCPEnvironment and StatefulToolEnv classes that extend the same RL loop to tool-using agent environments — opening the RLVR paradigm to multi-step agentic tasks.
Finding 3: Doom Loops in Small Reasoning Models Are a Systematic Failure Mode With Specific Mitigations
Maxime Labonne, Head of Pre-training at Liquid AI, presented the most rigorous account of training small language models (350M–1.2B parameter range) at the conference. His central finding for the research community: doom loops — infinite token repetition sequences during generation — are not a random failure but a systematic one that disproportionately affects small reasoning models on hard tasks. Liquid measured a 15–16% doom-loop rate in their LFM2.5 1.2B Thinking model after pre-training, compared to over 50% in Qwen 3.5 0.8B (characterised as a naively scaled-down large model).
Two mitigations were detailed. First, on-policy preference alignment via a pipeline that generates five diverse temperature-sampled rollouts plus one greedy rollout per training example, submits all six to an LLM jury, and trains the model to prefer non-looping responses via DPO. Second, RLVR with n-gram repetition penalties and temperature sampling. Combining both stages reduced the doom-loop ratio to near zero. Labonne also reported training LFM2.5 on 28 trillion pre-training tokens for a 350M parameter model — far beyond Chinchilla-optimal — citing Roberts et al. on test-time scaling laws showing performance continues improving with additional tokens even at smallest scales.
The LFM2 architecture itself represents a departure: on-device hardware profiling on real silicon (AMD Ryzen Max Plus 395, Samsung Galaxy S25 Ultra) to identify optimal operators, resulting in a gated short convolution block that outperforms sliding window attention, gated Delta Net, and GQA alternatives on memory and latency while maintaining GPU throughput at high concurrency. Labonne critiqued large embedding layers in distilled small models — 63% of Gemma 3 270M parameters are embeddings — as a specific inefficiency introduced by distillation from large-vocabulary teacher models, and a problem Liquid explicitly avoids.
Cassidy Hardin (Google DeepMind researcher) delivered a technical deep dive into Gemma 4 architecture that contained several findings relevant to researchers working on efficient model design. The key architectural innovation for on-device deployment is Per-Layer Embeddings (PLE): each of the model's layers (35 for E2B, 42 for E4B) has its own dedicated embedding table stored in flash memory rather than VRAM. Each PLE table uses a 256-dimensional compact embedding projected up to full model size at runtime. The result: the E2B has 5.1 billion representational parameters but only 2.3 billion effective parameters — i.e., the VRAM footprint is that of a 2.3B model while representational depth approaches a 5.1B model.
The attention architecture uses interleaved local/global attention at a 5:1 ratio (local layers use 512-token sliding window; global layers attend all preceding tokens) with differentiated Grouped Query Attention by layer type: local layers group 2 queries per KV head, global layers group 8 queries per KV head with KV head dimension doubled to 512 (versus 256 for local). The 26B MoE model uses 128 total experts with 8 activated per forward pass plus one always-active shared expert at 3x the size of individual experts — yielding only 3.8–3.9B active parameters during inference.
On vision: the shift from Gemma 3's pan-and-scan (splitting variable-ratio images into multiple fixed squares) to Gemma 4's variable-aspect-ratio native encoding (16x16-pixel patches with explicit spatial positional encodings, pooled 3x3 into soft tokens) is architecturally significant. The soft-token budget is configurable across five options, with higher budgets (560 or 1120 tokens) recommended for OCR and spatial tasks. The 31B dense model ranks third on LM Arena global leaderboard while outperforming models 20x its size — the clearest empirical case at the conference for parameter efficiency over raw scale.
Finding 5: SelfFlow — Self-Supervised Dual-Model Training Breaks the External Encoder Dependency
Stephen Batifol (Black Forest Labs) presented SelfFlow, a paper released approximately six weeks before the conference, as the most architecturally novel model training contribution at AI Engineer Europe 2026. The problem SelfFlow addresses: standard diffusion model training relies on external encoder models (DINOv2/DINOv3) to provide representation alignment — without which convergence is 70x slower. But this dependency introduces a scaling ceiling, modality specialisation constraints, and objective misalignment. Most strikingly, upgrading from DINOv2 to DINOv3 actually degraded generative model performance, illustrating the fragility.
SelfFlow replaces the external encoder with a self-supervised dual-model architecture: a student model receives high-noise inputs and learns both generation and representation jointly, while a teacher model (a stabilised exponential moving average of the student) receives low-noise inputs. The student minimises both generation loss and representation loss simultaneously — eliminating the separate alignment stage entirely. Research models trained with SelfFlow outperformed flow-matching baselines across all three modalities (audio, images, video) while converging faster and avoiding the plateau observed in baseline training.
The most striking finding is cross-modal emergence: the same model trained on images can generate video, audio, and predict robotic actions. A robot manipulation task demo showed the SelfFlow model correctly picking up and moving an object while the baseline produced erratic arm movements. This positions SelfFlow as directly relevant to researchers working on world models, robotics policy learning, and multimodal pre-training — not just image generation. BFL explicitly stated their roadmap targets world models as the substrate for training autonomous agents and scaling self-driving systems.
Finding 6: Benchmark Infrastructure Is Being Rebuilt From the Ground Up — With Direct Implications for Evaluation Methodology
Nicholas Kang and Michael Aaron (Google DeepMind / Kaggle) delivered one of the conference's most practically useful talks for researchers: a systematic account of why current AI benchmarks are broken and what Kaggle is building to fix them. The core finding: on SWE-Bench Pro, the harness configuration shifts performance by 22 percentage points — more than the difference between frontier models. A competing lab reran a Kaggle-hosted benchmark with their own API compaction settings and published substantially better results that were neither fraudulent nor informative. This harness-versus-model ambiguity is the central methodological crisis.
Kaggle's four new infrastructure products are directly relevant: the Standardised Agent Exams (one-line prompt, instant leaderboard score, 500+ submissions in the first week with no active promotion); Game Arena (PvP benchmarking via Bradley-Terry pairwise comparison, requiring 400,000 poker hands for statistical significance, revealing distinct model personalities — Grok goes all-in frequently; newer generation models are risk-averse and perform worse at poker); the community Benchmarks Platform (assertion-based tasks with LLM-as-judge or rule-based scoring); and hackathons with the Google DeepMind AGI team focused on five of ten cognitive faculties from a recently published AGI measurement paper.
The most actionable finding for researchers: approximately 30,000 AI researchers are creating evals for a world of 30 million technical practitioners, producing coverage gaps that deepen cognitive jaggedness (superhuman performance in some domains paired with mediocre performance in others). The solution is not more centralised benchmark creation but democratised, open-source contribution from domain experts — the wastewater treatment engineer from Turkey who built a novel safety benchmark after incidents in his country is the model for what's missing.
Finding 7: Inference-Time Architecture Is Being Redesigned Around Disaggregation and Heterogeneous Hardware
Sarah Chieng (Cerebras) and Adrian Bertagnoli (Callosum) provided complementary perspectives on the inference architecture transition underway. Chieng identified the key hardware shift: memory movement accounts for 50–80% of inference latency on NVIDIA GPUs with off-chip HBM, creating the memory wall. The response is disaggregated inference — splitting compute-bound prefill from memory-bound decode onto separate, purpose-optimised hardware. The Cerebras/AWS partnership (Cerebras wafer for decode's bandwidth-intensive operations, AWS Trainium for prefill's compute needs) and NVIDIA's $20 billion Groq acquisition are cited as macro validation. At the model level, Mixture of Experts architectures, Router-Weighted Expert Activation Pruning (REAP) to eliminate unused experts, and KV cache reuse combine to deliver 1,200 tokens per second from Codex Spark — a 20x improvement over the 50–150 token/second plateau maintained across Claude, Gemini, and GPT families for roughly two years.
Bertagnoli extended this to the theoretical case for heterogeneous intelligence: a mathematical production function argument shows that homogeneous systems can only optimise for a single peak of skill demand, while heterogeneous multi-agent systems match complex, multi-peaked demand functions. Empirically, Callosum's system routing the MIT recursive language model architecture across different chips and models on Cerebras ran 7x cheaper and 5x faster than GPT-5.2 on the Ulong benchmark ($0.54 vs $3.75, ~400 seconds vs ~2,000 seconds). On SambaNova: 12x cost reduction and 3x speed improvement. The implication for researchers: architecture decisions that were previously constrained to single-chip, single-model assumptions now need to account for heterogeneous routing as a first-class design variable.
Finding 8: Post-Training Preference Alignment Mechanics — LLM-as-Jury and On-Policy Data Generation Are the Operative Patterns
Several sessions converged on a common post-training methodology that is worth noting explicitly for researchers. The operative pattern across Liquid AI (LFM2.5), Zed (Zeta 2 edit prediction), and the Anthropic harness work is: generate diverse rollouts at temperature, submit to an LLM jury or verifiable reward function, use on-policy data as training signal (DPO or RLVR), and iterate. The key differences from naive fine-tuning: data is generated on-policy from the model being trained (not from a static corpus), the reward signal is verifiable (correct answers, won games, non-looping outputs) rather than proxy metrics, and the feedback loop is tight enough to catch specific failure modes like doom loops and reversal behaviors.
For diffusion and generative models, Sander Dieleman (Google DeepMind) confirmed that classifier-free guidance remains the single most impactful technique — and that post-training preference tuning via RLHF or DPO is now a standard lever for diffusion model behavior, with the guidance mechanism allowing models to punch above their parameter weight. He also clarified that introducing advanced conditioning signals (reference images, camera motion, event timing) in post-training rather than pre-training is the current best practice, with three architectural strategies for injecting conditioning into transformer-based denoisers (extra tokens, broadcast signals, cross-attention variants).
Ben Kunkle (Zed) provided a concrete case study: Zeta 2 (edit prediction model) uses a settled-data filtering approach — waiting for a 10-second editing pause to capture final ground truth — combined with student-model self-distillation (running 50 student rollouts per example rather than 1 million frontier-model calls) to filter for hard, non-obvious predictions at the middle of the distance distribution. The reversal ratio (how often the model predicts the inverse of the user's last edit) is used as a regression diagnostic rather than a standard NLP metric.
Strategic Implications
The empirical frontier has moved from pre-training scale to post-training precision. The sessions from Adaption Labs, Liquid AI, and Zed all point in the same direction: the highest-leverage applied research is now in understanding which post-training interventions have the best per-compute return, how to design RL environments with reliable verifiable rewards, and how to measure capability improvements with benchmarks that can distinguish model quality from harness configuration. Researchers who are still primarily oriented toward pre-training scale should actively reorient toward this space.
Small-model RL is an underexplored frontier with outsized practical impact. The Liquid AI and Verifiers/Deepset sessions demonstrate that RLVR at 350M–1.2B scale produces qualitatively different behaviour than naive fine-tuning or distillation. The failure modes (doom loops, training collapse from small batch size, curriculum sensitivity) are specific and addressable, but they require experimental infrastructure and engineering discipline that the research community has not fully standardised. There is a significant opportunity to contribute rigorous study of these phenomena.
Architecture search needs to be grounded in real hardware profiling. Liquid AI's explicit use of on-device hardware profiling to select operators is a methodological shift worth noting. Standard architecture comparisons run on server GPUs with ample memory bandwidth produce different rankings than comparisons run on mobile NPUs or Apple Silicon. As on-device deployment becomes the target for a growing fraction of use cases, researchers whose architecture choices are informed only by data-center benchmarks are making increasingly context-specific claims without realising it.
Benchmark methodology is in crisis — and this is a research opportunity. The Kaggle/DeepMind session documented a 22-percentage-point performance swing from harness configuration alone on SWE-Bench Pro. This is not a small-scale anomaly: it is a systemic problem that undermines virtually all current comparative evaluation. Applied researchers who can contribute rigorous work on benchmark design, harness standardisation, and evaluation methodology will have outsized impact at a moment when the field badly needs it.
SelfFlow and joint multimodal training represent a paradigm shift in generative model architecture. Black Forest Labs' elimination of the external encoder dependency is not just an engineering optimisation — it removes a fundamental bottleneck on what modalities can be trained jointly and how representations transfer across domains. Researchers working on world models, robotics policy learning, or multimodal pre-training should engage with this paper directly and consider its implications for their own training pipelines.
Action Items
Read the SelfFlow paper from Black Forest Labs immediately. The conference presentation was a summary; the full paper contains the loss function design, ablations, and empirical results needed to evaluate the claim that joint generation-and-representation learning outperforms external encoder alignment across all three modalities. Assess whether the student/teacher EMA architecture is applicable to your domain.
Audit your RL training pipeline against the Verifiers library and LFM2.5 findings. Specifically: check batch size (below 256 is a failure mode for GRPO), evaluate whether you have stratified sampling across curriculum difficulty, implement per-rollout deterministic seeds, and measure doom-loop rates in your reasoning model. The Verifiers library (Prime Intellect) provides a standardised RL environment API that is worth adopting as a baseline.
Run your evaluations on Terminal Bench and Kaggle Standardised Agent Exams. These are the emerging community benchmarks for agentic capabilities. Terminal Bench (two tools: send keystrokes to tmux, read output) has revealed that minimal harnesses outperform richer harnesses for most model families — a finding directly relevant to researchers building evaluation infrastructure. SAE provides a one-line prompt, instant leaderboard score, and growing community engagement.
Profile your architecture on target deployment hardware, not just data-center GPUs. Liquid AI's on-device profiling methodology is directly reproducible using publicly available hardware. If your architecture choices affect on-device deployment (mobile NPU, Apple Silicon, Raspberry Pi), reproduce your architecture comparisons in those environments. The rankings can differ substantially from server-GPU results.
Engage with the Kaggle AGI measurement hackathon. The Google DeepMind AGI team is running a hackathon focused on five cognitive faculties from a recently published AGI measurement paper. This is a direct opportunity to contribute to evaluation methodology at a moment when the field's measurement infrastructure is being rebuilt — and to build relationships with the researchers constructing the next generation of benchmarks.
Examine the Per-Layer Embeddings (PLE) architecture from Gemma 4. The technique of separating representational depth (flash-resident embedding tables) from active inference parameters (VRAM-resident weights) is a novel solution to a general problem in resource-constrained deployment. The design is fully documented in Cassidy Hardin's conference presentation and the associated technical report — consider whether PLE applies to your own architecture targets.
Sessions to Watch
"Everything I Learned Training Frontier Small Models" — Maxime Labonne, Liquid AI: The most technically dense and directly applicable session for RL/training researchers. Covers doom-loop detection and mitigation, on-policy DPO with LLM jury, RLVR at 1B scale, and hardware-profiled architecture selection. Rare combination of theory and production engineering.
"Let LLMs Wander: Engineering RL Environments" — Stefano Fiorucci, Deepset: Full walkthrough of GRPO training mechanics with the Verifiers library, including curriculum design, batch size failure modes, and stratified sampling — practical engineering details absent from most RLVR papers.
"Building Generative Image and Video Models at Scale" — Sander Dieleman, Google DeepMind: Authoritative account of latent diffusion training at scale covering U-Net vs. transformer architectures, classifier-free guidance, consistency model distillation, JAX/Pjit sharding for TPUs, and post-training preference tuning. The spectral autoregression framing of diffusion is worth the watch alone.
"FLUX, Open Research, and the Future of Visual AI" — Stephen Batifol, Black Forest Labs: SelfFlow paper summary, student/teacher EMA dual-model architecture for joint generation-and-representation learning, cross-modal emergence (image, video, audio, robotics from a single model), and the road to world models.
"Gemma 4 Deep Dive" — Cassidy Hardin, Google DeepMind: Detailed architectural analysis of interleaved local/global attention, differentiated GQA, 26B MoE design (128 experts, 8 active, 1 shared), and Per-Layer Embeddings for flash-memory-resident representational depth on mobile hardware.
"Agentic Evaluations at Scale, For Everybody" — Nicholas Kang and Michael Aaron, Google DeepMind / Kaggle: Documents the benchmark infrastructure crisis (22-point harness swing on SWE-Bench Pro) and the four Kaggle products being built to fix it — essential context for any researcher whose work is evaluated on public leaderboards.
"AIE Singapore Day 2" (Sara Hooker / Adaption Labs segment): Hooker's challenge to scaling consensus — 95% pruning, AutoScientist self-improving training, 30+ model co-optimisation in two days — is the conference's most provocative claim about where training research is heading. Evaluate the evidence directly.
"How Transformers Finally Ate Vision" — Isaac Robinson, Roboflow: Rigorous architectural history from CNNs through Swin, ConvNeXt, Hiera, DINOv2/v3 to SAM — with the decisive insight that MAE pretraining's ViT-exclusivity is the source of transformer's moat, and FlashAttention eliminated the O(n^4) penalty. Directly relevant to researchers working on vision architectures or pretraining methodology.