Google Research Blog

Google — Invalid Date

Frontier AI systems have the potential to transform biology research and drug discovery by intelligently navigating vast combinatorial search spaces (e.g., a single 100-amino-acid protein spans at least 20¹⁰⁰ possible sequences) calling for dynamic multi-candidate exploration. Our long-term research vision is that AI agents will not only ideate and analyze, but also run lab experiments to test their hypotheses, learn from the results, and self-improve to solve complex problems — all driven by human scientific agency.

Realizing this vision requires solving two intertwined challenges: First, frontier models still do not reason reliably about 3-dimensional geometry and physical state — which poses a problem in the automation of the specialized liquid-handling robots that move reagents, track volumes, and execute molecular biology protocols in a wet lab. Second, while agentic hypothesis generation and in-silico design are advancing rapidly, needing repeated cycles of real-world feedback to validate the proposals and navigate vast search spaces, real-world wet-lab experiments have not kept pace, making experimental readouts more of a bottleneck than ever.

In this post we share three research projects highlighting our progress on bridging the gap between in-silico AI exploration and real-world wet-lab experimentation:

Enabling AI agents to perform safe, resource efficient experimentation grounded in physical measurements by using the VERID framework.
Demonstrating closed-loop agentic scientific discoveries on our prototypes for aptamer and de-novo protein design.

Grounding agents in physical reality

First, we present the VERified Infrastructure for Discovery Laboratories (VERID) framework to bridge the gap between the high-level reasoning of AI agents and the complex, error-prone physical reality of a wet lab. It is a semantic digital twin of the laboratory workcell, i.e., a live, stateful 3D volumetric entity tree that tracks everything from robotic arm geometry to individual well volumes, reagent identities, and pipette tip inventory.

The safety of a command depends on the live physical context as well as the code. An agent can generate syntactically valid code that is physically impossible (e.g., lowering an 8-channel pipette into a narrow trough that is too short), logistically invalid (e.g., aspirating from a depleted well), or scientifically compromised by unmodeled fluidic effects.

Through VERID, an AI agent plans, simulates, and executes liquid-handling protocols on a liquid handling robot, combining synchronized multi-camera views with live state tracking to refine experiments across cycles.

Built on top of the open-source PyLabRobot framework, VERID provides four core capabilities:

Live 3D state tracking: Allow agents to check deck layout, well volumes, and available tips before acting.
Fast virtual dry runs: Simulate protocols in seconds to catch collisions or empty wells before touching reagents.
Hardware safety checks: Block invalid commands before they reach the robot.
Synchronized video and action logs: Capture a complete record of every run for research rigor, reproducibility, and debugging.

In our report, we show these capabilities in action through case studies. First, synchronized video and execution traces caught uneven dye stratification caused by dispensing too close to the well floor, allowing the agent to refine its mixing strategy. In the same campaign, the persistent digital twin state allowed the agent to recover mid-run from a plate-reader failure without re-pipetting. Second, when the simulator rejected an 8-channel aspiration that physically wouldn't fit in a narrow trough, the agent replanned on the fly and designed a targeted factorial diagnostic isolating a subtle bias caused by residual tip-wall carryover that replication alone would have missed. Finally, in a closed-loop self-play campaign, an agent iteratively designed and ran robot experiments to reconstruct blinded dye mixtures from overhead camera images, identifying its own optical and fluidic artifacts across runs to improve accuracy more than fivefold.

Reproducible execution and full trace logs — capturing procedural variables like dwell times, evaporation, and plate layout — lay the groundwork for agents to eventually be able to separate biological signals from experimental noise as they explore and refine many candidates in parallel across rounds.

Learning to learn with aptamers

Agents need real experimental readouts to refine their hypotheses as computational models do not capture the full picture of what happens in a wet-lab reaction and cannot replace empirical validation. In “Aptamers as a sandbox for lab-in-the-loop agentic molecular discovery”, we use protein-binding DNA aptamers to prototype the lab in a closed loop with AI agents over 3 design-test-iteration cycles. Aptamer discovery is a type of experiment common in research efforts for clinical biomarker discovery. A complete aptamer design-test-analyze cycle takes just 7–10 days, as DNA can be chemically synthesized in pooled libraries and read out directly by sequencing. In each round, agents design candidate sequences, which are screened against target proteins on magnetic beads to isolate functional binders, and then sequenced to return rich quantitative data for the next cycle.

Agents iteratively design, synthesize, select, and sequence aptamers in a closed loop.

Across three rounds on two protein targets with varying difficulty levels (myoglobin and α-thrombin), AI-driven rational and de novo design raised confirmed on-target hit rates as can be seen in the image below (to 33.5% and 42.1% for myoglobin and α-thrombin, respectively), though each target required identifying and overcoming a different wet-lab failure mode.

On α-thrombin, Round 1 yielded 0% binders because fixed primer overhangs disrupted folding in 48% of sequences; once the agents introduced clamped flanking stems in Round 2 and diversified in Round 3, hit rates climbed to 26.0% and then 42.1% at 98.5% specificity.

On myoglobin, as agents prioritized raw binding affinity in Round 2, they heavily enriched G-quadruplex motifs (63% of designs). While raw pull-down increased, the wet-lab data showed that most of these G-quadruplexes were non-specific "sticky" binders adhering to off-target proteins and bare magnetic beads, causing both target specificity (from 75.0% to 19.1%) and true, bead-corrected hit rate (from 10.2% to 3.7%) to plunge. Recognizing this artifact from the Round 2 readout, the agents autonomously revised their design rules for Round 3, capping promiscuous motifs, subtracting bead binders, and pivoting to non-G4 structural scaffolds to recover specificity (72.4%) and surge ninefold in on-target hit rate (33.5%).

Across three rounds, agents learn from wet-lab readouts to recover target specificity and improve confirmed de novo hit rates on myoglobin (left) and α-thrombin (right).

In the paper, we also examine how agents reason like scientists, analyzing data and suggesting tweaks to wet-lab protocols, but also sometimes think atypically, exploring sequence spaces that structural biologists might not.

Accelerating de novo protein binder design

Closing the loop for proteins is more complex. Unlike aptamers, which are chemically synthesized, proteins must be expressed biologically, so each loop adds cellular or cell-free selection, expression, and folding steps. Together with a larger sequence space, this creates bottlenecks on both the computational and wet-lab sides.

In “Agent-guided epitope excision enables economical de novo binder design”, we applied our workflow to fibroblast activation protein alpha (FAP), a large, challenging receptor overexpressed in solid tumors. In just a few experimental runs, the top designs yielded binders with single-digit nanomolar affinity, a level of tight, drug-like binding typically achieved only after extensive rounds of laboratory optimization.

Surgical epitope excision accelerates structure prediction 8.6x (left), enabling a human–agent hybrid pipeline to nearly double the in-silico hit rate (top right) and discover single-digit nanomolar FAP binders in the wet lab (bottom right).

Computationally, de novo binder design requires generating candidate backbones, designing sequences to fit them, and re-folding the resulting complexes to predict binding. That final re-folding step consumes the vast majority of GPU compute, as it scales quadratically with target length. By surgically excising just the local binding pocket during folding, our pipeline accelerated each cycle more than eightfold without losing structural accuracy. This turns decisions about where to bind and how much surrounding structure to include into tactical levers that LLM agents are well-suited to optimize.

In a head-to-head comparison at equal compute, human–agent collaboration, where a scientist sets the high-level strategy and an agent drives tactical execution, nearly doubled the in-silico hit rate of a scientist working alone while uncovering hits across a much broader range of target sites. To scale wet-lab screening alongside our computational pipeline, we operationalized yeast surface display with fluorescence-activated cell sorting (FACS), a workflow that allows us to screen large libraries of engineered protein variants in parallel by fluorescence.

Our next step is to weave these threads together, combining VERID’s fully traceable automation with closed-loop learning for protein discovery to enable agents to directly navigate complex, rugged biological search spaces in the wet lab.

Safety

In line with Google’s Frontier Safety Framework and our approach to bioresilience, we build safeguards across both the computational and wet-lab sides of our workflows:

Safe testbeds and screening: We deliberately selected well-controlled, non-infectious biological systems without gain-of-function risk, as testbeds to evaluate agent behavior and catch drift early. Every designed sequence goes through biosecurity screening.
Simulation before execution: Through VERID, agents never send unverified code directly to the liquid handling robots. Every protocol is dry-run in simulation and validated against the digital twin before touching reagents.
Human oversight: Humans directly authorize and review designed sequences, verify wet-lab protocols, and supervise robot runs.

As AI agents take on more of the design-test cycle, new safety and reliability challenges emerge. Moving toward an autonomous wet lab requires solving these open problems with opportunities for human agency and oversight: ensuring agents optimize for true target binding rather than unintended experimental shortcuts, verifying multi-step wet-lab protocols, and continuously monitoring agent reasoning and execution traces.

From grounding agents in physical lab robotics, to running fast closed-loop aptamer cycles where agents learn to explore sequence space, to discovering single-digit nanomolar protein binders, these efforts form the building blocks for human-directed autonomous discovery in the life sciences. We're excited by this progress and look forward to sharing what comes next.

Acknowledgements

This research is led by Annalisa Pawlosky, alongside Pooja Rao and Petar Sirkovic. We would like to thank our co-authors and collaborators across the foundational papers supporting this work: Zach Cutts, John Coller, Rachel Chen, Mathias Bellaiche, Hua Tang, Ryder Easterlin, and Dina Zverinski (aptamer discovery); Jun Kim, Artiom Myaskovsky, Oscar Donosa, Jamie Rogers, and Lawrence Chan (protein design); and Yves Delley, John Coller, Zach Cutts, Alexander Daryin, Mathias Bellaiche, Jun Kim, Oscar Donosa, Srinidhi Seshadri, Tanner Clary, and Sandy Klemm, with additional technical and experimental contributions from Xinyi Deng, Sanath Ramachandran and Jiawu Bi (VERID automation).

We are deeply grateful to Reuven Sayag, Avinatan Hassidim, Ronit Levavi Morad, Yossi Matias, and James Manyika for their leadership guidance and to Matthias Heiler, Burak Gokturk, Bill Jia and Pushmeet Kohli for their support throughout this research. We also thank Shekoofeh Azizi, Andrew Carroll, Philip Q. Nelson, Yishay Mansour, Dan Popovici, and Riccardo Pisoni for their reviews and insightful feedback, Amanie Brik for overall contributions to the effort, and Udi Graff for visual design and graphics for this post.

Google — Invalid Date

Generative AI has accelerated many technical fields. It is now common for AI to play some role in writing scientific papers and technical documents, running code, designing experiments, and drafting proposals. However, accelerating content creation has created a review challenge: the human capacity for verification has remained largely fixed amid this rapid increase in AI-generated content. Reviewing complex documents like long scientific manuscripts or complex legal agreements requires specialized expertise and advanced subject knowledge. This imbalance creates a validation bottleneck, where information is being generated faster than it can be rigorously reviewed.

Today, we’re excited to announce the public release of Deep Review, an agentic AI system engineered to provide rigorous feedback and technical verification on complex documents. Initially designed for computer science academic paper analysis, the system’s capabilities have expanded to review documents across many domains, such as legal and finance. It can be used as both an iterative improvement tool during the drafting process and a tool for the scientific process verification, claims, and evaluation during the review process.

Deep Review represents the evolution of our Paper Assistant Tool (PAT) — an experimental initiative developed to help researchers stress-test manuscripts prior to conference submission. Over the past year, we partnered with four flagship computer science conferences (STOC 2026, ICML 2026, NeurIPS 2026, and ICLR 2027) to provide private, pre-submission feedback to authors of more than 16,000 manuscripts who opted in. Further details on PAT and these pilots can be found in our manuscript, Towards Automating Scientific Review with Google’s Paper Assistant Tool.

With this release, Deep Review is now accessible worldwide:

Tiered access and scalable API: Every user receives two complimentary document reviews per week. Users with high-volume workflows can connect their Gemini API key for higher usage limits and programmatic integration via the Deep Review API.
Format flexibility: Deep Review will initially support PDFs and plain text as input formats, with support for multi-file inputs coming soon.
Privacy and data policy: Documents uploaded to Deep Review are never used to train, fine-tune, or improve Google models. Uploaded documents are stored for one year for the sole purpose of display to the user in the Deep Review UI.

Deep Review accelerates technical validation

One of the most significant examples of the validation bottleneck is in academic peer review. This can be seen in the uptick of submissions to academic publication venues, especially venues for AI research itself, which have seen a dramatic rise. This year, there were over 74,000 submissions to three major AI conferences (ICLR, NeurIPS, and ICML) up from approximately 45,000 in 2025, 32,000 in 2024, and 23,000 in 2023. Each of these submissions requires multiple high-quality reviews to ensure the integrity of the peer review process, which can place a larger burden on human experts.

Increase in submissions to NeurIPS, ICML, and ICLR between 2023 and 2026.

Deep Review aims to help alleviate this expanding burden on human reviewers across technical fields. By providing detailed, objective evaluations of documents, Deep Review checks logical consistency and mathematical statements, suggests improvements, grounds facts, and compares with existing work. It addresses some of the most cognitively burdensome facets of review, while leaving higher-level judgements to humans.

A proven track record for scientific review

Deep Review has been extensively tested and deployed under its experimental name, the Paper Assistant Tool (PAT), in four pilot partnerships with the flagship computer science conferences STOC 2026, ICML 2026, NeurIPS 2026, and ICLR 2027. During these pilots, authors were able to opt-in to receive feedback and analysis on their paper via PAT in the weeks before the submission deadline. The goal was to provide authors with actionable feedback to help improve the quality of papers in submission. This feedback was completely decoupled from the official conference review process. PAT did not collect any of the data for training, and papers were deleted from Google systems promptly after each program concluded.

Author reactions to the tool have been extremely positive, and reflect continued improvements to both underlying models and the PAT harness. With the addition of Gemini 4 Argon support in Deep Review, the underlying capabilities of the tool have further increased beyond the older iterations of PAT described below.

Statistics and author feedback from the pilot programs of PAT at machine learning and theoretical computer science conferences.

How Deep Review works

Single-prompt LLM calls often fail when evaluating long, complex documents due to exhausting their token reasoning budget, overlooking long-context cross-section dependencies, or accepting nuanced claims without fully scrutinizing them. Deep Review solves these challenges through a four-stage agentic pipeline:

Document segmentation: The document is first ingested and autonomously decomposed by a segmenter agent into logical segments (e.g., Introduction, experiments, methodology, etc.). These segments can be overlapping, and must cover all the content of the input document.
Adaptive budgeting: Not all sections require equal cognitive depth. The pipeline inspects information density and dynamically allocates a compute budget, along with a specialized inference scaling pipeline for each section. For instance, sections with dense mathematical proofs may receive more compute than an introduction or a related work section in an academic paper.
Deep parallel verification: Specialized reasoning agents scrutinize each segment in parallel. Each agent has access to the full document context to track cross-references, but focuses its reasoning power specifically on the task of reviewing its specific target section. This ensures both that reviewing tasks are manageable for each review swarm, and that parallel swarms are properly coordinated.
Global synthesis and agentic grounding: A dedicated synthesis agent consolidates the segment-level reports, and does final consistency checks between sections, rerunning logical checks if required. The agent further performs external grounding checks and tool calls as a final preventative measure against hallucinations.

System architecture showing the four-stage pipeline.

Universal verification and review: Beyond research papers

While Deep Review originated in academic paper reviews, long-context review is generalizable: reasoning mechanisms used to check whether a mathematical proof holds up in a research paper can also help surface logical gaps and inconsistencies in other types of documents, where claims must be backed by evidence. Therefore, Deep Review can be used to bring structured critique to a broader range of everyday workflows.

Users can now apply Deep Review to analyze complex documents across many domains. For instance, it can be run on:

Design documents: Catch unstated architectural assumptions, spot edge cases, and suggest concrete improvements.
Presentations: Ensure slide transitions flow logically and bulleted takeaways match visual charts.
Legal and financial documents: Check internal consistency across clauses and data tables, flag ambiguous terms or downstream consequences, and ground claims against supporting evidence.

Deep Review meets Antigravity

We’re also introducing the Teamwork Deep Document Review Pattern, a document review implementation pattern inspired by Deep Review built into Antigravity using the Teamwork multi-agent orchestration framework. This pattern is designed systematically to evaluate research papers, manuscript drafts, and technical proposals using a multi-agent hierarchy with self-correction and segment-based rubrics. Antigravity users can invoke this pattern using the /teamwork-preview command.

Making thorough, high-quality technical reviews widely accessible

Comprehensive pre-submission critique in computer science has generally depended on access to extensive review networks — like senior faculty, well-resourced academic laboratories, or extensive peer advisory networks. Providing reliable, automated logical verification can help many academics stress-test their assertions, identify avoidable flaws, refine and strengthen their documents based on verified foundations, and present their work in a review process with more confidence.

Get started

We invite researchers, engineers, and creators everywhere to explore Deep Review:

Try Deep Review: Visit the Google Research Portal to review your first documents.
Integrate with the API: Access programmatic verification in your continuous integration and drafting workflows using your Gemini API key.
Read the research: Read the preprint to explore the underlying methodology and pilot evaluations.

Conclusion and looking ahead

Given the utility it demonstrated during our pilot programs with academic conferences, we believe that tools like Deep Review will play a significant role across the peer review process in the coming years by empowering both authors and reviewers.

Moving forward, we are continuing to expand the platform’s capabilities. With the rollout of Gemini 4 Argon support, Deep Review’s underlying capabilities further improve, delivering even higher quality feedback and analysis to authors. Additionally, with multi-file support coming soon, users will also be able to run these same consistency checks across entire sets of related documents, spreadsheets, and exhibits simultaneously. We hope these advancements will make high-quality technical reviews more widely accessible for knowledge workers everywhere.

Acknowledgments

Deep Review is a collaborative effort across Google Research, Google DeepMind, and Google Cloud. We thank our core contributors and research collaborators — Santiago Balseiro, Taylan Cemgil, Tommaso D'Orsi, Kirk Draheim, Lindi Emoungu, Ayse Erduran, Palash Goyal, Caroline Hartnell, Shobana Kannan, Amy Keeling, Siyi Kou, Bridget Lee, Ian Li, Anoushka Mazumdar, Taylor Montgomery, Mouad Naciri, Uche Okonkwo, Mihir Parmar, Anthony Phalen, Rory Pilgrim, Sarah Presotto Post, Rob Tierney, Drew Tyler, Vishnu V., Vahab Mirrokni, Laura Vardoulakis, Johannes Welbl, David Woodruff, Yunhan Xu, Jinsung Yoon — as well as our executive sponsors Katherine Chou, Corinna Cortes, Burak Gokturk, Vahab Mirrokni, Tomas Pfister, Yossi Matias and the Program Chairs of STOC, ICML, NeurIPS, and ICLR for their partnership.

Google — Invalid Date

When we introduced Google Earth AI, we shared a suite of geospatial foundation models — spanning remote sensing, weather, flood hydrology, and population dynamics — alongside an initial prototype showing how a Gemini-powered agent could connect those models to answer cross-modal questions.

Since then, we have evolved a Geospatial Reasoning agent from an early concept into a much more capable, experimental system built for multi-step, geospatial reasoning across planet-scale data and autonomous training of custom predictive models. Today, we are sharing three updates:

A new technical report on geospatial reasoning: We present systematic evaluations across four benchmarks, including GeoKTools and GeoExpertQueries, two new benchmarks we developed and are releasing for researchers to test their own geospatial reasoning agents. Our results quantify the value of Earth AI’s specialized tools and planning over out-of-the-box LLMs.
Insights from real-world partner testing: Over the past few months, we invited a small group of social impact and commercial partner organizations — including the World Health Organization (WHO) Regional Office for Africa, People in Need, GiveDirectly, the Milken Institute and WPP — to test the agent in support of their own real-world workflows, such as disease outbreak investigations, disaster preparedness, and clean energy screening.
Expanding early access on Google Earth and Google Cloud: We are expanding experimental access for both multi-step geospatial analysis and for planetary prediction. We’re offering early access to an experimental deep research update for Ask Google Earth. We’re also providing early access to an agent-to-agent API for developers building custom workflows, including integration recipes for connecting to Gemini.

What’s new in Geospatial Reasoning

General-purpose LLMs and standard coding tools struggle with real-world spatial problems because planetary datasets are massive, with specialized map and satellite formats that require careful multi-step planning to combine accurately. To tackle complex spatial workflows in practice, our agent brings together four key capabilities:

Connected to Earth AI foundation models and planet-scale data

The agent connects Gemini to the capabilities of Earth AI’s foundation models — including WeatherNext, Flood Forecasting, the Population Dynamics Foundation Model (PDFM), AlphaEarth Foundations and Remote Sensing Foundations — alongside information from Google Maps Platform (e.g., place information, routing, air quality, rooftop solar potential), Google Earth Engine, Data Commons and Google Dataset Search.

Modular skills, upfront planning and a dedicated spatial workspace

Earth AI capabilities are packaged as modular skills that the agent calls only when a user question needs them. Before running complex operations, the agent builds a step-by-step analytical plan, selects appropriate fallback datasets when local coverage is sparse, and processes large map layers in a dedicated workspace. This approach avoids overwhelming the agent context with massive amounts of spatial data, while preserving accuracy and tracking data attribution. Every response shows the agent’s plan and the data sources used so experts can check assumptions or substitute datasets.

Bring your own data

Organizations rarely rely on public data alone. Users can now input their own datasets via the Geospatial Reasoning agent API as CSV spreadsheets, custom map boundaries (GeoJSON), or cloud tables. Users can then ask the agent to join their private locations with Earth AI’s global models and catalogs.

From multi-step reasoning to autonomous Planetary Prediction

Beyond answering complex analytical questions, the agent also powers our recently announced Planetary Prediction Engine. Given a user’s local outcome data, such as regional disease rates, crop yields, or grid outages, the agent automatically selects relevant population and environmental signals, uses our Population Dynamics Insights and the annual Satellite Embedding dataset (Google products built on PDFM and AlphaEarth Foundations, respectively) and trains validated machine learning models for spatial prediction, super-resolution downscaling, and forecasting. In our recent public health research, combining these population and satellite signals improved predictions of health outcomes by 12% over demographic baselines and enabled mapping at 8X finer spatial resolution.

Our Geospatial Reasoning agent responds to a user’s question by first planning a multistep workflow, then selecting key Earth AI tools, datasets and models that are relevant to the query, and finally executing each stage of the plan in a workspace outside the context window. The answer incorporates multiple Earth AI capabilities into a text, tabular and map-based report.

Evaluating a Geospatial Reasoning agent

To measure the impact of domain-specific tools and upfront planning on geospatial reasoning, our technical report compares four agent configurations across Gemini 3.8 Flash and Gemini 3.1 Pro: (1) a standalone LLM, (2) the LLM with basic web search and maps grounding tools, (3) the LLM with Earth AI tools (such as Remote Sensing Foundations and WeatherNext 3) but without upfront planning, and (4) the full Geospatial Reasoning agent which begins every query with a dedicated planning stage that reasons over available capabilities to assess what it can and cannot do and sets out the stages of retrieval and processing needed to reach an answer.

We evaluate four agent configurations, from the base Gemini model, to agents progressively augmented with skills and orchestration that are optimized for geospatial reasoning.

The evaluation spans four benchmarks, covering expert-level open-ended analysis, multi-tool orchestration, and community-developed geospatial problems with deterministic answers. CBGB and UnivEARTH are two externally developed benchmarks that test geospatial code generation and question answering respectively. To test agents on challenging tasks that go beyond a single question or use case, we developed two new benchmarks. GeoExpertQueries introduces complex, real-world queries with expert-authored golden workflows that capture analysis best practices in domains like crisis resilience, site selection and public health. GeoKTools contains templated queries with verifiable answers, each requiring correct selection and use of 1 to 4 distinct geospatial tools. Together, this suite provides substantially broader coverage than prior evaluations of geospatial reasoning systems. To accelerate research and development of geospatial agents, we release the development splits of both our new benchmarks in full.

Example queries from our new GeoKTools (left) and GeoExpertQueries (right) benchmarks. GeoKTools tests an agent’s ability to select and use specific geospatial tools and datasets while GeoExpertQueries evaluates reasoning on open-ended, multi-step workflows.

Results and key takeaways

Two key takeaways stand out from our evaluation: First, domain-specific skills are essential for spatial reasoning. The agent with only basic tools (web search, maps grounding and code execution) struggles on quantitative spatial tasks. Equipping the agent with Earth AI skills drives a significant jump across GeoExpertQueries, GeoKTools, and CBGB. The only exception is UnivEARTH, where even a standalone LLM scores 93.6%, likely because the source NASA articles and answers are available on the public web.

Second, upfront planning unlocks open-ended, real-world workflows. While Earth AI skills alone are enough for narrower tasks like those in CBGB and GeoKTools, tackling open-ended problems in GeoExpertQueries requires upfront planning to keep the agent on track across multi-step analyses and prevent it from making unreliable assumptions or taking shortcuts like using web-search instead of spatial tools.

The table below shows scores across all four benchmarks using Gemini 3.8 Flash:

Scores are 0–100, averaged over three runs. Each benchmark is scored differently, so compare within a row, not across rows.

The technical report also discusses limitations. GeoExpertQueries scores come from an autorater that compares the agent’s performance to expert workflows that define the steps an analysis should take, not the correct final answer. We therefore consider the agent’s output as a strong first draft for experts to review and build on, not a final answer.

Earth AI in action: Early testing with partners

To evaluate how this research translates to real-world challenges, we gave a small group of partners early access to test the experimental agent on their toughest problems. Across public health, disaster response, and infrastructure planning, below is an overview of how they are putting Earth AI into action:

World Health Organization (WHO AFRO) & DRC’s INRB: At the start of the 2026 Ebola outbreak in the Democratic Republic of the Congo (DRC), WHO AFRO’s Emergency Preparedness and Response Hub in Dakar used the Geospatial Reasoning agent to map remote mining corridors where the exposure risk and human mobility are high by cross-referencing forest loss (from satellite images), mining sites (from maps and web data), and population data. In minutes, the team pinpointed 48 exposed settlements and located more than 45,500 at-risk people — a process that normally would have taken weeks. This allowed local responders to proactively deploy mobile laboratories and coordinate border surveillance. Meanwhile, the Epidemic Modeling and Intelligence Unit (UMIE) at the DRC’s National Institute for Biomedical Research (INRB) used the Planetary Prediction Engine to estimate the risk of Ebola spreading into uninfected zones. By combining mobility flows with historical case trends and Earth AI models and datasets, these weekly insights give coordinators critical planning time before cases arrive.
- “Mobile populations like miners move faster than traditional paper surveillance can track, creating dangerous blind spots. Earth AI fills those gaps with satellite and mobility signals—moving us from reactively chasing a virus to actively forecasting where it will strike next.” Dr Etien Koua, HIS Program Area Manager & Hub Coordinator for WHO AFRO Emergency Preparedness and Response (watch the video)

People in Need: Geospatial reasoning helped combine natural disaster forecasts with population and imagery information, shifting analysts’ focus from data processing to providing insights:
- "The experimental Geospatial Reasoning agent helped us analyze 14 historical cyclone paths to identify high-exposure villages in minutes rather than days of manual GIS work, turning prepared complex data into immediate insights and freeing up critical time for life-saving anticipatory action." - Alonso Gonzales, GIS and Remote Sensing Coordinator

GiveDirectly: Teams uploaded custom map boundaries to rapidly calculate population exposure by wind-speed zones during Tropical Cyclone Maila and benchmark tornado damage in Mississippi, cutting crisis scoping from days to minutes inside a single workspace.

WPP: Used geospatial reasoning to connect Population Dynamics Insights with their proprietary data. Their case study demonstrates how Earth AI insights enhanced the performance of geospatial marketing campaigns, while preserving the privacy of their data.

Milken Institute: Pre-development teams used the agent to combine their own lists of 1,500+ candidate buildings with Google Maps Platform’s Solar API and census data in order to screen locations for building community resilience hubs, taking into account rooftop solar potential, mobility data, nearby transit hubs, climate risks, and federal investment zone eligibility.

Watch the film

Link to Youtube Video

Early testing with these partners helped our evaluation and expansion of the agent’s geospatial skills, driving our recent additions of road-network routing, support for larger user-uploaded datasets, and better discovery of authoritative local data sources.

Expanding early access: Where you apply to test it

Today, we are expanding access across our Earth AI research capabilities to gather more feedback on these experimental geospatial reasoning capabilities. We are making these available to a limited group of early users:

Ask Google Earth: preview expanded deep analysis capabilities directly inside Ask Google Earth. This provides an interactive chat experience where you can ask complex spatial questions in natural language, analyze data — including your own — in Google Earth projects, and explore visual results directly on the map.
Geospatial Reasoning Agent API: For developer teams that want a customizable, do-it-yourself integration to power custom workflows, we offer an experimental headless Agent-to-Agent(A2A) service. Early partners can connect the Geospatial Reasoning agent to their own custom agents and data pipelines. We provide step-by-step developer recipes for teams who want to connect the service to their Gemini for business environment.

Our Planetary Prediction Engine capabilities are also available for early preview — accessible both through an interactive interface for rapid, plain-language modeling and via the Agent API for custom workflows.

To express interest in getting access to these research capabilities, please complete our interest form. To be among the first to try out our early stage projects, you can also sign up for our Early Access Pool.

Google — Invalid Date

We are living through a golden age of research and scientific discovery. For the scientists and researchers at Google Research, the opportunity extends beyond pioneering breakthroughs. We’re also building and sharing robust tools that empower the global ecosystem. AI acts as an amplifier of human ingenuity, creating the conditions for curiosity and rigorous inquiry to flourish.

Today, at our Research@ event in Mountain View, we are demonstrating how we put never-before-possible insights and capabilities into the hands of doctors, scientists, developers, public health officials, and others.

Here are 10 highlights:

1. Google Earth AI enables planetary intelligence for organizations and society

Addressing global challenges requires accurate, real-world planetary intelligence. With Google Earth AI, it becomes possible to forecast public health threats, accelerate clean energy investments, provide businesses with foresight, and more. Today, we’re expanding early access and inviting partners to explore three new ways to access Google Earth AI’s new, experimental capabilities:

Conducting deeper geospatial analysis as part of an early access experimental feature in Google Earth.
Generating risk assessments and forecasting different scenarios via our planetary prediction engine.
Integrating our Geospatial Reasoning agent skills directly into custom workflows via an experimental agent-to-agent API, provides developers with programmatic access to core features like satellite imagery parsing, routing and demographic embeddings.

We’re also announcing the integration of Google Dataset Search for querying from more than 100 million public datasets, and TimesFM for planetary nowcasting and forecasting iterations. All these capabilities are powered by over 70 geospatial tools including planetary foundation models like Remote Sensing Foundation Model (RSFM) and Population Dynamics Foundation Model (PDFM). They are validated by empirical benchmarks detailed in today’s Earth AI Technical Report. They enable developers, health leaders and enterprise teams to automate complex geospatial workflows.

To further support social impact efforts, Google.org is providing $10 million in funding to organizations working on crisis resilience and public health. This includes $2 million to The Milken Institute to unlock infrastructure bottlenecks for affordable housing, clean energy, broadband, safe water systems, and other essential infrastructure.

Google Earth AI’s geospatial foundation models are being used to help predict disease outbreaks and enable public health teams to take proactive, preventive action.

2. Multi-agent orchestration helps scientists build on existing knowledge

Existing literature offers a great source of knowledge for driving new scientific discoveries. We built a multi-agent orchestrator, enabling scientists to reproduce previous methods and analyses and then apply them in new settings to tackle important research challenges. As an example, this helped researchers apply a new methodology for solving inverse design problems in fluid mechanics to the design of urinary catheters. They discovered catheter designs that, in simulation, were better at preventing bacterial infections.

This complements our existing tools to accelerate scientific workflows. Empirical Research Assistance enables scientists to generate and test thousands of code variations in parallel, and has helped researchers produce results from biology to cosmology to economics. Co-Scientist, built in collaboration with Google DeepMind, is a multi-agent tool that can generate novel hypotheses and has already helped partners, like researchers at Stanford University, uncover repurposed medicines to fight liver fibrosis.

3. Deep Review helps authors improve the clarity and accuracy of technical documents

Thorough peer review is a core part of the scientific method. We created Deep Review, an agentic system that provides rigorous verification across complex documents. This is particularly important as AI is increasingly being used to help experts draft technical documents. Many Google researchers and external authors have already used an early version of this system to review over 16,000 research publications submitted to top-tier computer science conferences. It helps with the heavy cognitive lifting by checking logical consistency and grounding facts, while scientists maintain authority over creative direction. Today, we’re expanding access to Deep Review to the general public to help people verify a variety of technical documents from legal agreements to financial reports.

System architecture showing the four-stage pipeline.

4. Next-gen Edge AI chip enables manufacturers to build the wearables of tomorrow

With the rise of wearable devices, hardware manufacturers need the tools to develop efficient edge applications to meet the high expectations of customers for all-day performance. Earlier this year, we launched the first Coralboard from Synaptics, using Google’s Coral NPU accelerator core. It enables AI and ML engineers to rapidly prototype and build devices. Next week, we’ll be revealing our co-designed, next-gen, low-power Edge AI chip, created by Synaptics using our open source IP. The chip delivers greater efficiency and performance for design engineers building the next generation of smart glasses and watches, connected cameras, streamers, and industrial vision and voice systems.

The new low-power Edge AI chip from Synaptics, as displayed next to a coin for its relative size.

5. TimesFM and TabFM transform how businesses work with numbers

While language models excel with text, business and operational decisions also rely on numbers. TimesFM and TabFM are foundational models engineered specifically to reason over raw numerical distributions — time-series streams (data that changes over time) and structured tabular data (like spreadsheets). Both models deliver accurate predictions out of the box with zero training. They rank at the top of leading public benchmarks in their fields and are available to all Cloud BigQuery users. Today, we are announcing our upcoming integration of these models into Gemini for business. Soon, business owners, analysts, operation leads or coordinators can forecast trends like product sales or hospital bed capacity, based on complex operational tables and millions of rows of data. Sensitive data stays safely within an organization’s perimeter, and people get the answers they need in seconds.

TimesFM-3's multivariate forecast helps to anticipate sales lift for a business on planned promotion days in the future.

6. MedGemma enables innovators to build and deploy health AI applications

MedGemma, published last week in Nature Medicine, has been downloaded more than 10 million times by more than 65,000 unique innovators. It serves as an open foundation for developers building specialized applications. Developers have adapted it to meet the needs of their local communities, from enabling faster triage in overcrowded clinics to screening for eye disease and cervical cancer.

As developers fine-tune MedGemma for more specialities and care settings, we’re pushing the frontier of open health AI. Our latest research advances MedGemma with tool use, so it can draw on available health records and other applications clinicians rely on. We’re exploring new modalities like mammography, ultrasound and electrocardiograms. Early signs point to substantial gains in capabilities, such as answering clinical questions and interpreting radiology scans, skin images and pathology slides. With these advances, we hope to expand what innovators can build, bringing effective healthcare to more communities around the world.

Watch the film

Link to Youtube Video

7. AMIE study advances the frontier of how conversational AI can support patients and clinicians

With our research on AMIE, we’re exploring the use of AI in clinical reasoning and conversations, to understand how it could empower clinicians and strengthen the patient-clinician relationship. Our research this year in Nature advanced AMIE’s capabilities beyond initial diagnosis toward long-term disease management, and in Nature Medicine, we showed how it can reason across multimodal data like medical images and lab results.

Our new study in The Lancet offers a promising look into how its capabilities could come to life in real-world clinical settings. At Beth Israel Deaconess Medical Center, 98 patients safely consulted AMIE ahead of urgent care visits, and clinicians reported that the AI summaries helped them prepare in 75% of cases. We’re also advancing AMIE’s multimodal capabilities towards real-time video consultations and have embarked on a nationwide randomized study of AI in real-world virtual care settings in partnership with Included Health.

AMIE is part of a rigorous research journey going back to our landmark MedPalm paper, and it’s still early days.

Watch the film

Link to Youtube Video

8. Groundsource makes it possible for researchers to predict natural disasters

AI can help us predict disasters with greater accuracy and lead time, giving communities a better chance to prepare and save lives. We’ve been making progress towards our crisis resilience north star: a world where no one is surprised by a natural disaster.

Take flood forecasting. Our global hydrologic model, published in Nature, allows us to provide reliable river forecasts up to 7 days in advance covering more than 2 billion people in over 150 countries for the most significant and impactful riverine floods. Yet just one year ago, predicting flash floods still seemed out of reach. Groundsource is our recent AI-powered methodology that turns publicly available information about crises into high-quality historical datasets, allowing researchers to build more accurate predictive models.

We first applied Groundsource with Gemini to analyze flood-related news reports and created a structured dataset with over 2.6 million flood events. This dataset helped us overcome a lack of ground-truth data for flash floods and train a new model that can predict the likelihood of flash flooding in urban areas up to 24 hours in advance. The dataset is now part of the Google Earth AI collection and we've made it openly available to help scientists conduct further analysis and research.

A global map showing the density of flood events in Groundsource. Red dots indicate floods from GDACS.

9. Learning interactives give teachers the ability to create custom, guided simulations

With generative UI, we can go beyond static outputs to dynamically generate functional, interactive software. Our research experiment explores this in the domain of education. We fine-tune generative UI with pedagogical guardrails to enable teachers to create custom, guided STEM simulations tailored to their curriculum. The sample library features over 30 AI-generated learning interactives, all vetted by teachers, for middle and high school physics, chemistry, biology, and math. Schools can sign up to the Google for Education Pilot Program to share feedback. This will help us improve the simulations and develop more for public use, making practice far more interactive and engaging.

10. Play with Putty facilitates collaborative, real-time prototyping for teams

Play with Putty allows teams to build and prototype together in a shared environment without breaking their creative flow. It demonstrates what’s possible as AI models become more efficient and capable of generating entire interfaces from natural language prompts. People can interact directly with any part of the interface, while multi-player editing supports seamless, real-time collaboration. We first designed Play with Putty to empower our researchers and teams across Google to prototype together — for example, exploring new solutions and visualizations with Google Earth AI. We’ve since opened up the waitlist on Google Labs to bring the joy of human collaboration to teams using AI everywhere.

These 10 updates highlight recent progress from our extensive research portfolio across many domains. Technology succeeds when we amplify human ingenuity. In this golden age of research and scientific discovery, the opportunity is greater than ever — to pioneer breakthroughs and share tools that empower others, in collaboration with many. We’re excited about the journey ahead.

Google — Invalid Date

Judgment is what sets senior professionals apart from their junior peers. Building judgment requires thousands of hours learning on the job, usually through tedious but formative engagement with relatively routine work, often under guided supervision by experienced workers. But AI is already changing how on-the-job learning works. On the one hand, empirical studies in radiology, business problem-solving, job-seeker writing, and legal education show that when AI tools are thoughtfully embedded into training workflows, less-experienced workers can improve their independent performance.

On the other hand, recent experiments with software engineers, management consultants, high school students, and clinicians demonstrate that while AI serves as a temporary exoskeleton that boosts immediate output, those gains often fail to persist once the tool is removed. Understanding how AI can erode or enhance expertise requires three elements rarely found together: (1) sustained AI usage embedded in ordinary professional work over months rather than hours, (2) a credible evaluation of unassisted judgment when AI is unavailable, and (3) blinded grading by domain experts.

In “Does AI Assistance Enhance or Erode Expertise? Evidence from a Three-Month Field Experiment in Patent Drafting”, published by the National Bureau of Economic Research, we describe a field experiment used to capture changes in both short-term productivity and longer-term skill building that occur as a result of AI usage in the highly expertise-intensive profession of intellectual property law. In our experiment, we randomized access to a then-unreleased Google Labs AI patent writing assistant tool, now part of Gemini Notebook, among 133 lawyers at eleven intellectual property law firms that are engaged in regular, non-exclusive business with Google. Two-thirds of lawyers at each firm (“treatment” group) were given early access to the tool, while the remainder ("control" group) were given some training on how to use AI but were withheld from using the tool until after the three-month study period.

Better work with AI

After 10 days of AI assistance, we sent all participating lawyers a packet of inventor materials for a hypothetical invention and asked them to draft a patent. We did the same after 90 days, with a different set of simulated inventor materials. We saw the expected AI boost in performance on the drafting tasks at both time periods. At 10 days, AI tool access raised drafting scores (graded on a rubric with Likert scales for five different aspects of quality by independent legal experts) by 0.34 standard deviations, equivalent to a 10-point climb in percentile ranking among control group scores; by 90 days, this improvement increased to 0.38 SD, implying an 11-percentile point increase. These improvements occurred with remarkable consistency across all quality dimensions. Tellingly, better performance came from lower frequency of poor scores and higher frequency of good scores, and no change in the frequency of excellent scores. Formally, scores among lawyers allocated AI access moved from the bottom to the lower-middle and middle quintiles, but there were no noticeable changes in the number of exceptional scores. This pattern was particularly clear for junior lawyers, whose improvements in quality were also matched by significant time savings (18 minutes faster than the control group’s average speed of 124 minutes on the 10-day task, for example).

But patent lawyers don’t just draft patents; they also review each other’s work for critical mistakes (sometimes called “patent profanity”) that can make patents unenforceable in court — for instance, by overclaiming the novelty or scope of an invention. We therefore added another task at 90 days, which required subjects to redline (to manually mark up and correct) an existing hypothetical patent that contained many errors, both substantive and stylistic. This test task mirrors the kind of professional judgment that more senior lawyers routinely use and for which they most often do not have the luxury of relying on AI. Critically, while treatment group lawyers were encouraged to use the unreleased AI tool or any other AI tool on the drafting tasks, no one was allowed to use AI for the redlining task, making their scores on this task a measure of how well their skills had developed. For all tasks, drafting as well as redlining, we worked with third party patent professionals to score submissions on five dimensions of quality: (1) enforceability, (2) accuracy, (3) strategic ambiguity (the tactical scoping of claims), (4) completeness, and (5) clarity.

Estimated impact of AI access on patent drafting and redlining quality. Results are reported for all lawyers (black squares), junior lawyers under 7 years of experience (blue triangles), and senior lawyers with 7+ years of experience (orange diamonds). All estimates account for differences across law firms and adjust for slight variations in how many ratings each lawyer received.

When AI is removed

When the AI assistant was removed for the 90-day redlining task, the leveling effect vanished. Lawyers given AI tool access outperformed controls by 0.32 SD on this unassisted task (a 9-point equivalent climb in percentile rankings), but this effect was driven entirely by senior lawyers, who outperformed control subjects by 0.45 SD (a 13-point climb), while juniors showed no discernible improvement. Instead, scores of juniors bifurcated: more very low scores, fewer mediocre ones, more good ones, and no more excellent ones.

What do these results mean for learning? Seniors who had been given AI tool access clearly harvested significant value in terms of their professional judgment from their use of the tool over the previous three months. But the junior-level picture is more nuanced. Some scores improved, hinting at AI’s ability to leapfrog learning. Other scores spread to the lower rungs of the distribution, showing that in some scenarios AI can help juniors deliver adequate work, but their higher machine-assisted performance does not translate into faster acquisition of the expertise that makes senior professionals distinctly valuable.

Distributions of the average redlining score for treated (orange) and control (blue) subjects by experience for all lawyers, junior lawyers under 7 years of experience, and senior lawyers with 7+ years of experience. Observations are individual ratings averaged over all dimensions of quality.

Examining qualitative editing patterns provides some insight into how professionals of different experience levels engage with generative AI tools. Across both treatment and control groups, junior submissions followed rigid formalism: juniors worked sequentially from top to bottom, frequently exhausting their time copy-editing low-stakes introductory sections before reaching the main patent claims. Juniors also focused on surface-level synonym swapping rather than commercial scope improvement. Even when juniors spotted serious flaws, they often left descriptive comments diagnosing the defect rather than executing the redline to fix it. Because this "diagnose-without-execute" posture was equally common among unassisted control juniors, it represents a baseline junior deficit that three months of relying on AI to execute rewrites did not remediate.

Senior lawyers, by contrast, benefited consistently from AI exposure. Treated seniors spent longer on redlining than juniors, bypassing low-stakes prose to rebuild the claims from scratch, strip out language that could inadvertently narrow legal rights or limit protective scope, and pair many edits with explicit legal doctrines. In follow-up interviews, seniors described treating AI output not as a finished product, but as a "logic auditor". It weakened attachment to existing prose and forced them to articulate the why and how of structural edits, activating and sharpening their foundational expertise.

Further directions

Improving performance and building durable professional judgment are different goals. For junior professionals in particular, they may be in tension with one another. Foundational expertise may be the missing link that allows AI-assisted repetition to translate into seasoned professional judgment over the course of a career. Such judgment will likely continue to be important even as model capabilities advance: lawyers will continue to use their judgment in their interactions with judges, clients and colleagues. They will not be able to rely on an AI tool to assist them in all settings. A critical challenge of this AI moment is ensuring that tools designed to produce better work today do not prevent junior professionals from becoming the experts we need tomorrow.

Studying professionals in their working environments is a challenge for researchers. Our experiment includes a limited sample of patent lawyers, a reflection of the high hourly billing rate of top-tier professionals in the field. We only observe them over three months, a small window compared with the years they would need to develop durable expertise. Moreover, the rapid advance in both model capabilities and baseline AI familiarity (as well as AI-enabled product penetration in the legal domain) over the past year might affect our results if we re-ran the trial now. These limitations give rise to opportunities for further research, some of which we hope to pursue in the months ahead. Nevertheless, a clear takeaway from our work is that understanding the effect of AI assistance on professional skill requires tests that separate the effect of using tools from the effect of unassisted user performance.

Acknowledgements

A huge thank you to co-authors Josh Martin, Zanna Iscenko, Scott Strand, David Pearl and Melissa Ferere; to James Manyika, Ruth Porat, Kent Walker, Josh Woodward, Anu Madgavkar, Daniel Rock, and Fabien Curto Millet for their guidance and support; to Tom Shane, Mauricio Carneiro, Johnny Jacobs, Victor Lavrenko, Yinghao Sun, Samuel Yang, Daniel Nishi, Eric Wang, Kevin Li, Hao Zheng and Franz Och of Google Labs for partnering with us on the experiment; to Kiera Russell for product access and trusted tester support; to Steve Gong and Taylor Montgomery for legal guidance and recruitment support; and to our Comms, Marketing, and Research Blog partners including Kerry McHugh, Zoe Ortiz, Emily Short, Joseph Mack, Esmeralda Cardenas and Leanne Trujillo for driving the launch.

Google — Invalid Date

Public health decisions rely heavily on timely, granular data. For health conditions, such as cardiovascular disease or postpartum depression, that data shows us where to focus resources and support. For acute disease outbreaks, such as dengue or cholera, it can help inform urgent operational protocols and resource allocation. However, conventional epidemiological surveillance is often hindered by limitations in data: multi-year reporting lags, data siloed by rigid geopolitical boundaries, and data sparsity. Even without these limitations, traditional modeling approaches require extensive task-specific data collection and custom data engineering pipelines that are difficult to deploy during rapid outbreaks or in resource-constrained settings.

To address these systemic bottlenecks, we introduce a new paradigm in public health leveraging planetary geospatial foundation models. Using Google Earth AI’s Population Dynamics Foundation Model (PDFM) as a proof-of-concept, we demonstrate how self-supervised, pre-trained representations of "place" can be integrated directly into existing health sciences and epidemiological workflows as plug-and-play inputs — enhancing the statistical and machine learning (ML) models epidemiologists already use, rather than building new pipelines from scratch. PDFM compresses privacy-preserving search trends, human mobility, built-environment density, and environmental determinants into location embeddings. Without requiring task-specific fine-tuning, these off-the-shelf location embeddings matched or improved on conventional inputs across a wide variety of disease domains, geographic settings, and epidemiological tasks.

What is PDFM?

Part of Google Earth AI — our suite of geospatial models connecting satellite imagery, weather, anonymous search trends, human mobility, and other population dynamics — PDFM uses self-supervised learning to synthesize the following diverse, privacy-preserving signals into compact, versatile embeddings that serve as “fingerprints” for locations refreshed at a monthly cadence:

Aggregated search trends: Search frequencies of topics and resources that have garnered community-level interest
Built environment and mobility: Local density and busyness of places such as pharmacies, clinics, and parks.
Environmental determinants: High-resolution weather and air quality metrics and statistics.

Rather than requiring researchers to collect and process these raw data streams themselves, PDFM embeddings can be easily plugged into existing ML workflows to provide ready-to-use geospatial context.

Prior work showed that these embeddings are task-agnostic. Because these everyday signals capture the underlying social, behavioral, and environmental determinants of health, the same embeddings showed strong performance on filling gaps in a wide variety of CDC health metrics.

However, establishing a new paradigm for global health carries a higher burden of proof. To meet this standard, we set out to demonstrate the value of PDFM embeddings across a wider variety of health challenges, and in different environments across the globe.

Overview of Google Earth AI’s PDFM applied to a wide range of global health challenges.

Validation matrix at a glance

To achieve this, our global health research partners facilitated independent evaluations across five distinct public health challenges — spanning diverse epidemiological tasks, resource settings, and disease types:

Case study & domain	Partner institution	Epidemiological task	Geographic scope	Key impact & quantitative results
MMR vaccination (immunization)	Mount Sinai Health System; Boston Children’s Hospital	Cross-border extrapolation	US–Canada Border (146 border counties)	+36% relative gain in explained variance (0.159 0.216, statistically significant) by capturing cross-border mobility and information spillovers.
Cardiovascular disease (noncommunicable disease)	Department of Population Health, NYU Grossman School of Medicine	Mortality nowcasting & interpolation	Contiguous United States (3,091 counties)	Comparable accuracy to census-based models for nowcasting (mean absolute error 18.7 vs 19.1 deaths per county; RMSE 46.00 vs 57.69; differences not statistically significant), making PDFM a timely stand-in when census data are outdated or unavailable.
Dengue outbreaks (vector-borne disease)	University of Oxford and Tecnológico de Monterrey	Short-horizon probabilistic forecasting	Mexico (~2,450 municipalities)	Statistically significant 1-month forecast gain ( Δ Weighted Interval Score = -0.0051, lower is better) via TimesFM integration, concentrated in active transmission hotspots. Accuracy improved in up to 72% of active transmission municipalities.
Postpartum depression (mental health)	Institute on Human Development and Disability, University of Washington	Individual risk prediction	United States (CDC PRAMS) (332,970 respondents)	Transferable gain in risk prediction (AUC +0.0020 on a 0.62 baseline in seen states; +0.0038 in unseen states). Encoded area poverty (R² = 0.45). Recovers about 15% of the predictive signal of income and insurance records.
Cholera emergence (water-borne disease)	WHO AFRO	Outbreak onset prediction	Democratic Republic of the Congo (403 health zones, 89 weeks)	Statistically significant relative gains: +9.7% in Area Under the Precision-Recall Curve at 4 weeks; +18.1% Precision@5 (share of the model's five highest-risk zones each week that actually went on to have an outbreak) at 8 weeks, and +19.3% Precision@5 in endemic health zones.

Capturing cross-border dynamics in immunization

Our evaluations show that while epidemiological models are constrained by sovereign borders, the outcomes they track are not. For example, in the 146 U.S. counties situated within 150 km of the Canadian border, domestic-only models often struggle to predict local vaccine uptake.

By supplementing U.S. county embeddings with Canadian Forward Sortation Area embeddings, researchers at the Mount Sinai Health System and Boston Children’s Hospital developed models that captured cross-border behavioral and mobility spillovers. To help public health officials pinpoint communities at risk of measles outbreaks, the researchers used this added cross-border context to increase the share of variation in MMR vaccination coverage explained by the models from 16% to 22% (a 36% increase) and refine coverage estimates by at least 3 percentage points for 4.7 million border residents, revealing local patterns that domestic-only models may miss.

Reducing reporting lags in chronic disease surveillance

Cardiovascular disease (CVD) claims over 916,000 lives annually in the U.S. However, unsuppressed official county-level mortality data from the National Vital Statistics System (NVSS) typically lags by 1–2 years, and American Community Survey (ACS) covariates reflect conditions up to 2–3 years in the past. PDFM is available in more places and has much less of a lag, allowing for the ability to improve our capacity to model disease.

Our partners at NYU Grossman School of Medicine tested whether PDFM could stand in for these census-based inputs when estimating current-year CVD deaths across roughly 3,100 U.S. counties. They found no statistically significant differences between PDFM and traditional census data on nowcasting, with PDFM being much fresher and available in far more places:

Comparable to census data: Models using PDFM predicted 2023 county-level CVD deaths about as accurately as models using ACS demographic and socioeconomic data (an average error of 18.7 vs. 19.1 deaths per county when nowcasting) while cutting large county outlier errors (RMSE) by 20% (57.7 → 46.0). The differences were not statistically significant, suggesting that PDFM can serve as a substitute for census inputs.
More timely and more widely available: ACS covariates pool five years of survey data and are released up to a year after collection. The PDFM embeddings used in this study were built from a single month of data. PDFM is also available in 17 countries, while ACS covers only the U.S.

These results suggest that PDFM can enable health departments to guide prevention resources using current conditions rather than waiting on multi-year survey cycles.

Sharpening forecasts of active outbreak hotspots

Time is especially of the essence when addressing outbreaks of vector-borne diseases like dengue. In partnership with public health researchers at the University of Oxford and Tecnológico de Monterrey, we coupled PDFM with TimesFM 2.0 (our open time-series foundation model) to forecast dengue case counts across ~2,450 Mexican municipalities between 2020 and 2025.

The model’s greatest performance gains occurred when predicting one month ahead, improving forecast accuracy in up to 72% of active dengue transmission municipalities, yielding total error reductions 3.4X larger than degradations. This performance occurred precisely where timely vector control and clinical staffing decisions matter most, especially considering the demonstrated utility of short term forecasts.

Testing geographic boundaries in clinical screening

Screening for maternal mental health conditions like postpartum depression (PPD) typically relies on clinical intake information, which rarely captures the broader community conditions that shape a mother’s risk and access to care. To test whether geospatial foundation models can help bridge this gap, researchers at the University of Washington evaluated PDFM embeddings across 332,970 participants in the CDC PRAMS national survey.

In addition to PDFM capturing community-level socioeconomic conditions (R2=0.45), adding the embeddings provided a consistent, statistically significant boost to predicting which mothers were at risk (AUC +0.0020 in seen states; +0.0038 in unseen states). Crucially, this signal held up in states the model had never seen during training—acting as complementary local context that recovers about 15% of the predictive signal of a mother’s own income and insurance records when those details are unavailable.

This transferable context made the biggest difference in how screening resources are directed in new states. In simulations where a health system can follow up with the 20% highest-risk mothers, adding PDFM helped reach 5,640 more rural mothers with postpartum depression each year. Alternatively, in systems aiming to catch 80% of all cases, PDFM cut 17,723 false alarms annually—showing how geospatial embeddings can help health systems either broaden rural outreach or improve follow-up efficiency.

Extending lead times in prospective cholera emergence

Early warning for waterborne epidemics like cholera is critical for prepositioning oral cholera vaccines and clean water supplies. Fortunately, outbreaks are rare: in any given week, fewer than 1 in 100 of the country's 403 health zones sees one begin. Unfortunately, this very same rarity makes them hard to anticipate.

To address a use case defined by the World Health Organization Regional Office for Africa using national surveillance data from the Democratic Republic of the Congo’s Integrated Disease Surveillance and Response (IDSR) reporting, we tested whether a lightweight, low-resource version of PDFM (adapted for regions with sparse internet connectivity) could help forecast cholera hotspots.

The benefit depended on how far ahead we looked. One or two weeks out, recent case counts told most of the story and PDFM did not significantly enhance the accuracy. Four to eight weeks out, when there is still time to move supplies, it helped produce:

Sharper shortlists: Response teams work from a short list of high-risk zones, so each week we checked how many of the model's top five picks went on to have an outbreak. Eight weeks ahead, PDFM raised this from 1.78 to 2.10 correct picks per week, an 18% improvement.
Bigger gains where cholera is endemic: In the 15 zones that had reported cholera in at least half of all weeks, Precision@5 of eight-week shortlists improved from 0.3333 to 0.3975, a relative gain of 19%.

These findings demonstrate that while short-term tracking can rely on recent clinical data, foundation model embeddings capture underlying environmental, connectivity, and population determinants that supplement historical data, allowing for proactive planning one to two months in advance.

Looking ahead

Geospatial foundation models enable moving from reactive, localized modeling to proactive and time-sensitive health intelligence at planetary scale. Current limitations, such as static snapshots, are driving active research into temporally dynamic embeddings and geographic transfer learning for under-connected regions.

By integrating planetary contextual data into existing public health workflows, we can help take steps to improve how healthcare resources, interventions, and outbreak alerts reach the communities that need them most. Read the paper for more details.

PDFM embeddings are commercially available in Preview as Population Dynamics Insights, a geospatial embeddings dataset from Google Maps Platform. Academics and public health researchers can also request no-cost access for select, non-operational research use cases.

Google — Invalid Date

We're rapidly transitioning to a computing landscape defined by highly general, increasingly autonomous agents. Driven by large language models (LLMs) that can dynamically generate plans and invoke external tools, these systems offer the potential for AI to seamlessly handle complex, multi-step tasks on our behalf. However, realizing this potential requires solving a key challenge: enabling agent capability while ensuring that agents act appropriately.

Today, we share a comprehensive new workshop report, "Open and Emergent Problems in Agentic Privacy and Security: A Contextual Angle," the result of a collaborative effort bringing together more than 50 academic and industry leaders from numerous institutions. We met at the Google Contextual Agent Privacy and Security (CAPS) Workshop, held in late 2025 in New York City. In this report, you’ll find a breakdown of foundational privacy and security challenges that autonomous agents face today, needing coordinated defenses at the system, model, user, and ecosystem levels.

Agentic trade-offs

The core challenge of agentic AI is that, for an agent to be useful, it may need to have access to personal data and the ability to take consequential actions across a broad range of contexts. However, this access together with agents’ behavioral flexibility requires meaningfully different approaches from those used in traditional software. Unlike traditional deterministic software, agents differ in three critical dimensions that lead to challenges the research community must address together:

Unstructured interfaces and input ambiguity: Agents process instructions in formats like natural language and images, which makes it difficult to define "expected behavior" or protect against adversarial manipulations like prompt injection.
Probabilistic control flows: Generative planning leads to probabilistic execution paths that traditional testing methodologies can’t easily secure.
Autonomy and delegation: As agents handle longer tasks and delegate sub-tasks to other agents, traditional user oversight becomes less effective and likely insufficient, risking "confirmation fatigue".

Agentic AI privacy and security challenges.

The contextual lens

Given that useful agents may need to share data and complete tasks, we believe that the future of trustworthy agents depends on their ability to reason about, understand, and be constrained by the social norms and appropriateness of their actions— for the specific context in which they operate.

But how do we teach an agent to understand "context" and appropriateness? Our report is grounded in the theory of Contextual Integrity (CI), which defines privacy not just as secrecy or control, but as "appropriate information flow" according to established and justifiable social norms. A context-specific informational norm is defined by its actors (who is sending and receiving information about whom), the types of information (specific categories of information, like medical or financial records), and transmission principles (the rules governing the flow, like confidentiality or reciprocity). For example, you might be willing to share your gift shopping list with a virtual shopping assistant, but not your family and friends. Our report extends and generalizes contextual integrity for information sharing to contextual security, that is the appropriateness of agent actions. By anchoring agent privacy and security in CI, we explore how we can design systems that evaluate whether an action is socially and contextually appropriate before executing it.

Model of a general AI agent interacting with other users, tools, and agents across contexts.

Operationalizing Contextual Integrity

For the first time since the development of the theory of Contextual Integrity, LLMs provide an opportunity to create machine readable policy that is truly context dependent. There has historically been a semantic gap between high-level contextual norms and low-level system permissions. For example, “protect my data while organizing my travel for a conference” requires specific instructions on what user information is appropriate to share when booking flights, applying for visas, and communicating with organizers.

Traditional methods like manual permissions or expert-written policies cannot scale to a future with AI agents autonomously performing multiple complex, long-running tasks. As language models begin to understand CI, we argue that this gap can be bridged. Our report advocates for complementing model and user interaction advances with a contextual policy engine that forms part of a supervisor layer to monitor and enforce the appropriateness of actions. This policy engine includes a dynamic policy generation loop that can operate in real time (illustrated below) to tailor policies to the user request and open-ended, dynamic contexts, including new tools and capabilities that might be discovered at runtime. This allows the system to evaluate whether a requested data flow is appropriate before any information leaves the user's workspace.

An illustrative example of how a contextual policy engine architecture could be instantiated within an agentic system.

Multi-layered approach to agentic privacy and security

Working towards contextual policy engines together with necessary advances at the model- and user-level results in a multi-layered approach. Our report outlines opportunities for innovation across the entire stack:

System-level sandboxing: The execution environment can impose additional guardrails to limit the impact of agent or model failures and facilitate detection and response. Traditional operating systems sandbox applications by statically limiting access, say to files or networks. For agents, we explore dynamically limiting agentic capabilities, establishing agent identity, and authorizing or revoking access to data based on changes in context.
Model-level reasoning: An agent performing complex tasks has to judge the appropriateness of its actions, such as sharing user data. Future research should enable models to disambiguate under-specified user prompts and reason about appropriateness under changing contextual norms.
User-centric controls: The traditional “Notice and Choice” framework assumes that individuals can make fine-grained decisions about actions affecting personal data and relies on an assumption of foreseeability. However, autonomous agents’ behavior is inherently probabilistic, high volume, and generative. We explore shifting interfaces towards dynamic, contextual, and personalized control mechanisms consistent with user mental models.
Multi-agent interactions: So that AI agents can safely collaborate with each other on complex tasks we need to develop effective guardrails to prevent the agents from colluding and violating contextual norms.
Ecosystem governance: We explore challenges in governance mechanisms to arrive at a set of contextual norms that are collected and shared across agents. Furthermore, as norms can differ across domains and entities we need mechanisms that allow us to resolve conflicts, negotiate norm changes, and verify compliance.

Measuring privacy and security dynamically

Finally, we propose the development of new approaches to safety evaluations that more directly apply to highly autonomous, multi-agent systems. Our report highlights the need for standardized, multi-agent benchmarks — dynamic "Agent Gym" environments where researchers can safely simulate complex, cascading interactions during extended periods. With these open-source sandboxes, we can establish a robust, shared privacy, security, and safety baseline across academia and industry.

A call to action

This is an ambitious project, but a safe, secure, and trustworthy agentic ecosystem is too large a task for any single discipline, organization or sector. Calling for new ideas and unprecedented collaborations, the report lays out foundational opportunities; it is a call to action for the broader research community across academia, government, civil society, and industry to develop the contextual foundations necessary for a safe, secure, and privacy-respecting ecosystem. We invite you to read the technical report.

Acknowledgements

We would like to thank Lillian Tsai for serving as co-primary author and Sarah de Haas for coordinating the workshop, report writing, and blogpost. Kassem Fawaz, Stefan Mellem, Helen Nissenbaum, and Nina Taft contributed to conceptualization, writing across multiple sections, and guided the writing process. Sebastian Benthall, Madiha Zahrah Choksi, Seliem El-Sayed, Ferdinando Fioretto, Adrià Gascón, Ashish Hooda, Ivan Petrov, Francesco Pinto, Octavian Suciu, Trishita Tiwari, Harold Triedman, Ren Yi, and Wen Zhang served as section leads. We are grateful for the contribution by the over 50 researchers, academic partners, and industry leaders who co-authored the report or participated in workshop discussions. This work was supported by Daniel Ramage, Corinna Cortes, Yossi Matias, Pushmeet Kohli, Hank Levy, Raluca Ada Popa, Amanda Walker, Brendan McMahan, and James Manyika. Eugene Bagdasarian is also an Assistant Professor at University of Massachusetts Amherst.

Google — Invalid Date

In 2017, Google introduced Federated Learning (FL) a machine learning technique that trains models across decentralized, private data. It has been used to power everyday helpful features, including next-word prediction and Smart Compose on Gboard, reply suggestions in Google Messages, and Smart Text Selection in Android.

Our FL systems development is guided by four essential privacy principles: (1) data minimization, (2) data anonymization, (3) transparency and control, and (4) verifiability and auditability. Years of research development on anonymization have led to strong differential privacy (DP) guarantees for production models through algorithms like matrix factorization DP-FTRL (MF-DP-FTRL) and distributed differential privacy coupled with Secure Aggregation. In 2025, we introduced an evolved definition of FL centered on these four principles:

Federated learning (FL) is a machine learning setting where multiple entities (clients) collaborate in solving a machine learning problem, under the coordination of a service provider. A complete FL system should enable clients to maintain full control over their data, the set of workloads allowed to access their data, and the anonymization properties of those workloads. FL systems should provide appropriate transparency and control to the users whose data is managed by FL clients.

In “Toward provably private learning from federated data”, we announce the next generation of our FL system, which leverages Trusted Execution Environments (TEEs) to provide fully verifiable and auditable data anonymization guarantees. Logic that runs in TEEs is remotely attestable (third parties can verify the logic that is being executed), and it also gains confidentiality (its internal state cannot be observed) and integrity (the logic cannot be disrupted), subject to current-generation TEE limitations. Our new TEE-based FL system builds on these properties which TEEs offer at the level of a single machine to form a fully verifiable end-to-end FL system that achieves stronger privacy guarantees and improved accuracy. Gboard has already adopted the new system and is benefiting from substantially faster compute times than our previous FL system.

Building a verifiably private FL system out of individual TEEs

Our TEE-based FL system builds on techniques developed in our earlier work on confidential federated analytics and provably private insights.

The system coordinates four core operational concepts:

Data upload: Client devices locally encrypt training examples and upload them. The devices pre-authorize an access policy, which is the set of TEE computations that will be allowed to process this data, and these computations only release anonymized results. Clients require access policies to be published to a public transparency log.
KMS and policy verification: The Key Management System (KMS), which consists of a cluster of TEEs implementing the RAFT consensus protocol, only gives decryption keys to server-side TEE workloads that match the computations in the access policy.
Workload execution: A data processing TEE executes a Python program that implements a training loop. This "root" TEE delegates parallelizable subtasks to a cluster of worker TEEs. Distributed logic is expressed using Federated Language, an open-source, framework-agnostic orchestration language derived from TensorFlow Federated, which powered our earlier FL system. The training loop periodically releases anonymized model weights to the data analyst.
Fault-tolerant recovery: The Python program saves a KMS-encrypted recovery state at the end of the training round. This can be used to recover from intermittent root or worker failures, without leaking any additional privacy-sensitive information.

This diagram shows both the flow of data through the system as well as the steps needed to validate the privacy guarantees of workloads running on the system.

For more details on the TEE-based FL system design please see our whitepaper, Toward provably private learning from federated data.

How TEE-based FL strengthens privacy

In earlier FL systems, device data was uploaded for the purpose of immediate aggregation, but there was no way for external observers to verify that the data was never logged or inspected. Later, Secure Aggregation allowed uploads to be protected cryptographically, but was not compatible with state-of-the-art central DP guarantees. Our new TEE-based system represents the next milestone in our ongoing effort to completely remove the need to trust the server operator.

In our new TEE-based FL system, only metrics and differentially private model weights are visible to workload operators. Encrypted training data collected from devices can only be decrypted and processed within TEEs running Python training programs represented in the access policies, and only for a limited amount of time after upload.

Public transparency log and reproducible builds

Devices participating in our new TEE-based FL system know the full set of server workloads that may access data they’re uploading. The access policies representing these potential future server workloads are published to Rekor, a public transparency log, and external auditors are able to observe these logs to track the full set of server-side workloads that devices could potentially be participating in.

The KMS and data processing binaries used in our FL system can be reproducibly built from open source code published in the Confidential Federated Compute Github repository.

Verifiable execution with dynamic sideloading

In our earlier FL systems, the logic running on the server could be verified neither by devices nor auditors, and thus we needed to be trusted to correctly add random noise to gradient sums to provide differential privacy.

In our new TEE-based FL system, the access policies that are published to Rekor directly describe the Python program that expresses the FL training logic. To protect proprietary model architectures and data preprocessing logic while preserving auditability, our data processing TEEs support sideloading serialized information into the Python program at runtime. As long as all privacy-relevant logic remains hardcoded in the Python program, this sideloading functionality allows logic that must remain proprietary to run in the TEE while still providing strong externally verifiable privacy guarantees (see below for a discussion on side-channel observations).

How Gboard is using TEE-based FL

Gboard has deployed this TEE-based FL system to launch English and Japanese next word prediction models with stronger privacy guarantees and improved accuracy. These improvements can be attributed to several aspects of the new system design.

Mitigating diurnal availability constraints

By collecting all device uploads before running the server-side training workload, we no longer have to worry about diurnal variations in device availability impacting training progress. At server-side execution time, we can dynamically calculate the optimal device participation schedule within the program, and can use it to tune other DP parameters.

This graph shows the stronger privacy guarantees and/or smaller noise multipliers that become possible with the new TEE-based system by optimizing device participation schedules. These curves were derived from training an English next word prediction model for 5000 rounds with cohorts of 6500 devices on both systems.

Training speedups

In the past, training these FL models could take 1-2 months each, with progress limited by device availability, on-device compute, and competition across multiple training workloads for the same set of device resources. With the new TEE-based system, bottlenecks have been moved to the server, and computation parallelization across many machines allows us to achieve significant speedups in training time, currently only limited by TEE resource availability.

What’s next?

In our new TEE-based FL system, computation of client gradients is shifted to the server, lifting limitations related to on-device compute resources that were present in earlier systems. This paves the way for training increasingly larger models using FL techniques. Integrating TEEs with accelerators will play an important role in such use cases.

The system we have described above is capable of executing not just FL training workloads in a verifiable manner, but also arbitrary workloads that can be expressed using Python. We are experimenting with running other types of workloads on this infrastructure, such as synthetic data generation workloads. Another area of exploration: using the data processing TEEs that execute arbitrary Python in combination with our other data processing TEEs that specialize in functionality such as LLM inference.

This work is a step toward rigorous proof that server side processing preserves individual privacy. With external verifiers able to inspect exactly what code we run at Google, we are able to offer strong assurances that data is processed server-side exactly as described. We expect future TEE hardware, along with ongoing research into mitigating side-channel observations, to offer deeper protections for dynamically loaded workloads against malicious server-side attacks. We anticipate that systems like ours may one day come with full proofs of correctness of the software implementations of the DP algorithms and system components.

Acknowledgements

The authors would like to thank the collaborators who contributed to the infrastructure design and implementation: Arun Ganesh, Brendan McMahan, Brett McLarnon, Chunxiang (Jake) Zheng, Emily Glanz, Maya Spivak, Michael Reneer, Nova Fallen, Stefan Dierauf, Suxin Guo, Timon Van Overveldt, Yu Xiao, Zachary Charles, and Zachary Garrett. We also thank close partners who supported the Gboard integration: Haicheng Sun, Heng Su, Jianpeng Hou, Liyang Jiang, Noriyuki Takahashi, Wenzhi Mao, Xiaojuan Fang, Yanxiang Zhang, Yingjie Liu, Yuanbo Zhang, and Yun Wang. This work was supported by Corinna Cortes, Shumin Zhai, and Yossi Matias. We additionally thank Maysam Moussalem for feedback on the writing of this post.

Google — Invalid Date

The rapid advancement of text-to-image AI models, such as Nano Banana, Stable Diffusion and Flux, has fundamentally transformed creative design, allowing anyone to synthesize photorealistic, high-fidelity images from textual descriptions. However, steering these massive models to meet precise user intent, downstream goals, or strict visual constraints remains a delicate and unpredictable balancing act. For example, imagine prompting a model for "a lizard wearing sunglasses". The model might generate a realistic lizard that's not wearing sunglasses. Alternatively, forcing the model to include the sunglasses might distort the lizard's face, ruining the image quality.

Existing methodologies that guide or fine-tune image generation are very disconnected. On the one hand, developers use inference-time techniques (e.g., classifier-free diffusion guidance) to adjust the text prompt’s influence and guide the image generation process on the fly. On the other hand, they rely on heavy fine-tuning using parameter-efficient adapters like LoRA, reward-weighted regressions, or policy gradients to alter a model's behavior.

Because these tools have historically been treated as distinct and unrelated fixes, the field has lacked a single, principled mathematical language to unify, analyze, and optimize how we control generative models. This fragmented approach often forces engineers to rely on guesswork when balancing user preference alignment against image quality.

To solve this balancing act, we present the Diffusion Controller framework. Instead of treating image generation as a rigid sequence of isolated steps, Diffusion Controller reframes the entire denoising process as a smooth, continuous control problem. Our results show that Diffusion Controller’s lightweight add-on network outperformed the industry standard for matching human preferences. Moreover, its fully unlocked version (i.e., the fine-tuned model with "white-box" or unrestricted access to alter internal model weights) achieved a 90% win rate over the baseline model.

The Diffusion Controller framework

The Diffusion Controller framework treats the image generation process (where an AI model starts with random noise and gradually refines it into a clear picture) as a smoothly controlled journey. Imagine the base pre-trained model as a massive, powerful motorcycle; rebuilding its core engine to change how it drives is inefficient and risky. Instead, the Diffusion Controller acts as a lightweight steering damper attached to it while the main model (the motorcycle) remains completely frozen and safely untouched.

Rather than guessing how to guide the generation at each step, the Diffusion Controller’s core mechanism (the steering damper) dynamically adjusts the generation trajectory as the image is created. It smoothly recalibrates the model’s standard, default behavior, giving more weight to directions that maximize a user-defined target (such as achieving an artistic style or contextual alignment).

Side-by-side visual matrix comparing base pre-trained models, LoRA, and Diffusion Controller outputs across complex prompts.

By mathematically optimizing these shifts in the generation trajectory with feedback, the Diffusion Controller framework strikes a balance, successfully steering the model’s generation process toward new user preferences while fully preserving the base model’s crisp visual image quality and underlying stability. Returning to the lizard example above, this means the steering damper steers the model to ensure the sunglasses are included, but the penalty guardrail kicks in to prevent the system from distorting the lizard's natural scales and proportions to make it happen.

From theory to practice

To turn this theoretical control problem into a practical tool, we bridged the gap between abstract and possibly intractable equations into two efficient fine-tuning methods based entirely on a final reward score:

Method 1: Policy gradient and PPO (the steady optimizer): This approach iteratively steers the model through calculated, incremental adjustments. It includes a built-in "clipping rule" that acts like a speed limiter, preventing the system from making massive, erratic changes in the models behavior to ensure training remains smooth and stable.
Method 2: Reward-weighted loss (the shortcut): This functions as a direct optimization path. It heavily rewards generation processes that yield high-quality results, providing a mathematical guarantee that the model will reliably learn to generate the exact types of images users intend.

The "steering damper" network

The Diffusion Controller framework operates like a steering damper, which solves a massive real-world business problem. Usually, to change how a model behaves, you need "white-box" access to dig into its core engine and change its internal settings. However, the world’s best image-generation models are often corporate secrets (referred to as black boxes or gray boxes).

The steering damper network relies on the perfect image-steering instruction, a combination of the base model’s knowledge plus a small correction. It observes the image as it is being cleared of static, and injects precise, microscopic steering corrections. This allows engineers to perfectly control and customize even tightly locked, closed-source models without ever touching the underlying code.

High-level schematic illustrating how the frozen backbone passes the intermediate reverse mean to the Diffusion Controller Side Network to calculate the combined score.

Experiments and results

We evaluated the Diffusion Controller framework’s capabilities using a Stable Diffusion v1.4 backbone across three fine-tuning regimes: supervised fine-tuning (SFT), reward-weighted loss (RWL), and PPO. Performance was measured using the standardized Human Preference Score (HPS-v2) to determine how well generated images aligned with user prompts and aesthetic choices.

We have implemented four network structures under the Diffusion Controller framework:

Diffusion Controller: our steering damper network for grey-box access with the intermediate reverse mean as an input and the proposed side adapter stream.
Diffusion Controller-Naive: a naive design of steering damper network with neither the intermediate reverse mean nor the side adapter stream.
Diffusion Controller-J: our white-box solution by jointly training both the steering damper network and the base model.
Diffusion Controller-S: another white-box solution by separately training the steering damper network and the base model.

HPS-v2 win rate against the pre-trained model on the HPS-v2 test prompt set for all setups (each at its reported checkpoint): SFT (top), RWL (middle), and PPO (bottom).

The results demonstrated that Diffusion Controller excels in both fully accessible "white-box" environments and highly restrictive "gray-box" environments, consistently outperforming its corresponding baselines and achieving a better quality-efficiency trade-off.

In the SFT and RWL tracks, the gray-box Diffusion Controller steering damper network outperformed LoRA — the state-of-the-art parameter-efficient, white-box approach — in HPPS-v2 win rates. This is a significant milestone, considering Diffusion Controller accomplished this while manipulating significantly fewer internal model layers than LoRA. Furthermore, in comprehensive human evaluation panels, Diffusion Controller recorded the best subjective quality and prompt-matching results across complex, multi-attribute test prompts.

End-of-training win-rate vs. baselines. Each subplot reports three paired comparisons: (gray-box) Diffusion Controller vs. Diffusion Controller-Naive; (white-box) Diffusion Controller-J vs. LoRA; (white-box) Diffusion Controller-S vs. LoRA. (a-c): HPSv2 win rates for SFT/RWL/PPO with orange error bars showing standard deviation; (d): human-evaluated win rate for PPO.

A core feature of the framework is its flexibility at runtime. By adjusting a single inference-time guidance strength parameter, users can dynamically dial up or down the intensity of the control constraints. This allows for smooth, granular adjustment of prompt alignment on the fly without breaking baseline image stability or causing the visual distortions typical of older guidance methods.

Conclusion

By combining a mathematical control with an image generation model, Diffusion Controller bridges the gap between pure mathematics and modern creative tools. Instead of relying on a patchwork of guesswork and quick fixes to tune a model, it provides a single, mathematically sound system for steering image generation. Best of all, it offers a practical, lightweight "steering damper" blueprint that works perfectly even on access-restricted, closed-source models.

Looking ahead, this unified framework opens up exciting new paths for research. Because the control layer is completely separated from the model's core engine, future developers can use Diffusion Controller for much more than just matching text prompts. Immediate next steps include using the framework to build advanced personalization tools, developing robust safety mechanisms to help mitigate harmful content generation, and adapting the steering damper network to control complex, next-generation video models.

Acknowledgements

We would like to thank our co-authors and collaborators from Google Research, Google DeepMind, and academia for their contributions to this work.

Google — Invalid Date

Recent advancements in video diffusion demonstrate remarkable high-fidelity generation with models that can render realistic scenes in seconds. However, while diffusion models generate high-fidelity video clips, transforming them into coherent long storytelling engines remains challenging.

Most existing agentic pipelines automate this process via chained modules but suffer from semantic drift (subtle shifts in character attire or scenery across shots) and cascading failures (e.g., an upstream asset artifact corrupting downstream video synthesis) due to independent, handcrafted prompting. Because early errors propagate and break long-horizon consistency, the process often requires exhaustive manual intervention. From a structural perspective, this reflects the classical credit assignment problem, as terminal failures are difficult to trace back to specific prompts. Furthermore, existing methods suffer from feature drift, where entities and environments gradually change unintentionally, or content collapse, where narratives fail to progress meaningfully.

Today, we introduce our research on an AI video co-director, a unified, multi-agent framework that explicitly plans visual continuity in multi-shot narratives. Built as an orchestration layer on top of Gemini and Veo, this framework natively inherits safety mechanisms like SynthID watermarking. By treating long-form generation as a global optimization and world-state tracking problem, we have developed a suite of frameworks — Co-Director (to appear at COLM 2026), CANVAS (to appear at EMNLP 2026), A²RD, and VQQA —that translate high-level human creative specification into execution by automating repetitive orchestration tasks, from multi-model prompting and shot chaining to closed-loop visual refinement.

We designed these frameworks to act as responsive creative partners that abstract away the burdens of maintaining visual continuity, freeing users to concentrate on the art of storytelling. This architecture decouples creative synthesis from consistency by modeling quality as a test-time objective. Across comprehensive evaluations, our framework demonstrates substantial gains in multi-shot narrative consistency and character persistence, successfully generating minutes-long videos while mitigating visual drift and pipeline error propagation.

How it works

To solve the multi-faceted problem of long-horizon video, we broke the research down into four foundational pillars, each addressing a specific bottleneck in the generative pipeline.

1. Orchestrating creative intent with AI video co-director

To ensure semantic coherence across an entire video, we present AI video co-director, a hierarchical multi-agent framework formalizing video storytelling as a global optimization problem. Rather than relying on rigid, linear prompt chains, we introduce hierarchical parameterization: a multi-armed bandit (MAB) globally identifies promising creative directions.

This formalizes the creative process as a search for the optimal balance between exploration of novel narrative strategies with the exploitation of effective creative configurations. The system samples abstract creative trajectories — such as combining an informational strategy with a vignette narrative mode and a specific aesthetic archetype — and dynamically injects these into the system prompts of sub-agents. This top-down steering guarantees that the entire pipeline operates under a unified vision. Because our AI video co-director framework operates as an orchestration layer, it achieves this vision by feeding these structured prompts directly into the foundational Gemini and Veo models (though its model-agnostic architecture allows it to sit on top of any foundation generative model). This architecture ensures that all generated images, video, and audio inherently carry native safety protections, including SynthID watermarking. For production, additional safety classifiers can be applied across the final video to safeguard against unintended contextual interactions between individually safe clips.

The pipeline executes global optimization through two interconnected loops: strategic steering and multi-stage production. First, the Orchestrator Agent evaluates the inputs using a MAB algorithm to select a creative configuration across three dimensions: (1) Creative Strategy (intent), (2) Narrative Mode (story structure), and (3) Aesthetic Archetype (visual tone and cinematography). This configuration drives the production hierarchy. The Pre-Production Agent synthesizes a brief scene-by-scene storyline and visual assets into a unified storyboard. The Production Agent then translates this storyboard into concrete audiovisual media using specialized sub-agents: the Keyframe Agent anchors character and scene visuals, the Video Agent adds motion, and the Audio Agent layers in matching voiceover and score.

Finally, a multimodal LLM (MLLM) Judge critiques the compiled cut across the three designated dimensions, feeding a factored reward signal back to the MAB to iteratively refine and optimize choices across successive generation loops.

AI video co-director multi-agent pipeline. Top: The Orchestrator Agent utilizes MAB to navigate a factored creative action space. Bottom: The Pre-Production Agent synthesizes a creative brief, storyline, and visual assets to establish a consistent storyboard, which the Production Agent translates into synchronized keyframes, video clips, and audio. An MLLM Judge evaluates the final video to generate a factored reward signal which is propagated back to the MAB to iteratively refine the creative configuration over optimization loops.

2. Visual storyboarding with CANVAS

Even with a unified script, generating long sequential shots often leads to character drift and unstable environments. To address this, we introduce Continuity-Aware Narratives via Visual Agentic Storyboarding (CANVAS), a multi-agent framework that explicitly plans visual continuity in multi-shot narratives.

CANVAS enforces coherence by maintaining structured representations of characters, locations, and object states as the narrative evolves. Built as an orchestration layer on top of Gemini, it relies on a persistent visual memory. By retrieving visual anchors from memory or initializing new ones when needed, CANVAS ensures smooth transitions within the same setting. This explicit world-state modeling ensures that characters retain their identity and environments preserve their spatial structure when revisited.

To see these in action, the figure below showcases a multi-shot museum heist sequence, comparing CANVAS against representative baselines: direct generation using the underlying base model (Gemini-3.1-Pro) and an alternative multi-agent framework (AutoStudio). The prompts at the bottom of the figure highlight recurring elements — e.g., the thief, the exhibit hall, and the gemstone — that must maintain identical visual traits. Notice how each method handles consecutive transitions (e.g., the character’s clothing) versus non-consecutive transitions (e.g., when the camera returns to the main hall after a detour). Generation with Gemini-3.1-Pro alone exhibits prop inconsistency (the artifact changes) and background drift (the room layout shifts), showing the limits of unguided generation. AutoStudio also degrades across cuts, resulting in character drift (the thief’s cap disappears) and background inconsistency. In contrast, CANVAS's persistent visual memory ensures that characters, spatial geometry, and object states remain perfectly coherent across the entire narrative.

Visual storyboard comparison on a museum heist sequence from HardContinuityBench, contrasting CANVAS against AutoStudio and Gemini-3.1-Pro baselines.

3. Scaling temporal dynamics with A²RD

To translate storyboards into actual minutes-long video, we developed A²RD, an agentic autoregressive video generation architecture. A²RD features segment-by-segment generation augmented with a multimodal video memory that tracks segment contexts and dynamics.

For each segment, it operates in a retrieve-synthesize-refine-update loop. A critical part of this loop is how the agent adaptively determines the segment generation mode. It smoothly switches between extrapolation — to allow for natural narrative progression — and interpolation, which anchors segments to existing entities and environments. This effectively balances the need for the story to move forward with the necessity of maintaining the physical reality of the scene.

To test this system, the ten-minute movie below showcases a long-form generation, which demands consistent narrative progression across minutes-long temporal gaps. The video highlights how the system dynamically shifts between its two operational modes: using extrapolation to push the plot into new narrative beats, and interpolation to anchor returning characters and environments to their original designs. While standard video generators suffer from severe visual decay — where characters mutate and locations morph — A²RD continuously queries its multimodal video memory to maintain character identity, costume details, and structural geometry from the opening shot to the final frame.

Watch the film

Link to Youtube Video

Long-form video demonstrating A²RD's ability to maintain strict visual consistency and narrative progression over a continuous ten-minute duration.

4. Closed-loop refinement with VQQA

Finally, we needed a way for the system to autonomously identify and fix visual artifacts. Existing test-time optimization methods are typically either computationally expensive or require white-box access to model internals. To address this, we developed Video Quality Question Answering (VQQA), a unified, multi-agent framework generalizable across diverse input modalities and video generation tasks.

VQQA dynamically generates visual questions tailored to the specific prompt and uses the resulting Vision-Language Model (VLM) critiques as semantic gradients (provides natural language directional feedback to guide iterative refinement, analogous to numerical gradients in backpropagation). This replaces traditional, passive evaluation metrics with human-interpretable, actionable feedback. The system can then execute a highly efficient, iterative feedback loop (where the model generates a video, evaluates it via visual questions, and refines the text prompt based on the critique) via a natural language interface. To prevent semantic drift during this refinement, VQQA employs a Global Selection mechanism: rather than blindly taking the final iteration's output, a global VLM rater evaluates every video generated across the optimization trajectory against the original, unedited prompt. The system then selects the highest-scoring candidate, ensuring localized corrections do not compromise the broader context.

In practice, VQQA operates as a black-box prompt optimizer rather than a pixel-level editor. Instead of modifying pixels directly, it iteratively refines the text prompt to correct high-level compositional defects like attribute binding errors or inconsistent character attributes. This updated prompt guides the generator to sample a new path in its latent space. In the examples below, VQQA does not mask or paint over the original frames; rather, it resolves the model’s material binding struggle by rendering a realistic mylar balloon texture onto a strict cuboid geometry, and fixes a chaotic mid-performance instrument change by keeping the violinist and pianist consistently anchored to their respective instruments across cuts.

Resolving attribute binding errors. Prompt: "A cuboid balloon drifts past a circular window, sunlight streaming through the glass panes." While the vanilla generation (left) fails to capture the properties of a balloon by rendering a rigid, textureless cube, VQQA (right) successfully enforces the intended cuboid shape while maintaining the texture and material properties of an inflated balloon. By leveraging iterative visual feedback, the optimized prompt guides the generator to render a distinct, box-shaped mylar balloon with defined edges and seams drifting smoothly past the sunlit window.

Resolving temporal inconsistency. Prompt: "Violinist and pianist captivate with harmonious duet performance." The vanilla generation (left) suffers from a severe continuity break across shot transitions, showing a female pianist and a male violinist initially but suddenly depicting the woman playing the violin after the cut. VQQA (right) maintains strict semantic consistency throughout the performance, keeping the female pianist and male violinist anchored continuously to their respective instruments.

Key results & benchmarks

To rigorously evaluate our frameworks, we developed three specialized benchmarks designed to mirror the challenges of professional video production.

GenAD-Bench: Uses a human-in-the-loop pipeline that pairs Gemini 3 Pro with specialized image models to construct 50 fictional brands containing four distinct products each. Across 400 unique scenarios, it tests exact marketing constraints without relying on copyright conflicts or training priors.
HardContinuityBench: Designed to evaluate spatial and environmental continuity. Generated using intricate GPT-5.2 multi-shot narrative storyboards, this benchmark challenges models with massive gaps between scene reappearances, frequent costume and accessory changes for actors, and complex state changes for interactive props.
LVBench-C: Addresses long-horizon temporal dynamics. This dataset contains 120 text-only scenarios grouped into evolving character states, changing object properties, and progressive environmental revelations, maintaining a strict gap rule where critical visual assets must disappear for at least 10 segments before returning with realistic, narrative-driven changes.

Our evaluations demonstrate measurable performance improvements over existing video generation architectures. By navigating the creative strategy search space, AI video co-director achieves a peak quality score of 81.4 on GenAD-Bench and enhanced story consistency on ViStoryBench. CANVAS leverages structured visual memory to mitigate scene drift, yielding significant continuity gains across scene reappearances on ST-Bench and HardContinuityBench. For long-duration temporal dynamics, A²RD minimizes layout drift to improve character and environment consistency over continuous multi-minute runs on VBench-Long and LVBench-C. Finally, VQQA's closed-loop prompt optimization resolves physical and compositional inconsistencies, delivering notable absolute quality gains across T2V-CompBench, VBench2, and VBench-I2V . Please consult the individual papers for comprehensive details on our model architectures, training configurations, and baseline evaluations.

Future directions

These frameworks represent a foundational step toward unlocking coherent, long-horizon visual storytelling for creators. As we continue to refine these agentic architectures, we are exploring how to integrate human-in-the-loop workflows. Our ultimate goal is not to replace human storytelling but to empower creators by abstracting away the tedious complexities of temporal consistency and world-state tracking, ensuring they remain the control of creative direction and narrative design.

Acknowledgements

This work was made possible by the dedicated efforts of our broader research teams across Google. We would like to thank Andrew Pan, Brett Slatkin, Burak Gokturk, Carina Claassen, Daniel Vlasic, Do Xuan Long, Ishani Mondal, Jasmine Leon, Jingyun Liu, Joe Timmons, Jordan Boyd-Graber, Khanh G. LeViet, Kuang Su, Long T Le, Mihir Parmar, Min-Yen Kan, Nathan Hodson, Nick Losier, Palash Goyal, Rhyard Zhu, Sebastian Ko, Scott Penberthy, Yan Xu, Yang Li, Ye Jin, Zack Chomyn, and Tomas Pfister for their invaluable contributions to this suite of research.

Google — Invalid Date

How can a Dutch poffert arrive at your door, 450 miles (700 km) away, the very next day? It’s thanks to careful logistics optimization — especially the middle-mile segment. This part of the journey covers the longest distance, represents a huge portion of the overall costs, and most importantly dictates whether your poffert arrives fresh or stale.

Logistics research has historically focused on the first mile (moving goods from producers to initial consolidation points) and the last mile (delivering to the consumer). Both stages are typically modeled as variants of the vehicle routing problem (VRP). However, the middle mile, which handles the bulk movement of goods between distribution centers at a regional or continental scale, has received significantly less attention in operational research despite representing a sizable portion of total logistics expenditure. Academic progress in middle-mile optimization has been hindered by a lack of public, high-quality data. Indeed, most logistics companies treat their network topologies and demand volumes as highly sensitive proprietary information.

Middle-mile logistics has many applications in the supply chain. These range from moving goods from factories to consumers in e-commerce and retailers in city centers, to carrying the right parts from individual plants and central storage to car manufacturers and shops. It also includes time-sensitive movements, like transporting temperature-controlled pharmaceuticals between storage facilities and hospitals.

Middle-mile logistics bridges the gap between first and last mile.

To address the lack of standardized data for this domain, in “A Novel Instance Generator for Simulating Middle-Mile Logistics Networks”, we introduce MilleMiglia, a C++ instance generator designed to create realistic benchmarks for middle-mile delivery problems. This work serves as a foundational building block to enable future research results. In this post, we explore the unique constraints of the middle mile and how MilleMiglia successfully captures them to generate realistic, privacy-preserving data. The source code and documentation are available on GitHub.

The logistics spectrum: First, last and middle mile

The distinction between first-, middle- and last-mile logistics lies in the journey of an individual shipment. Throughout this journey, the primary operational goal is to efficiently use a fleet of vehicles to visit multiple locations. Consider the example of a manufacturer that sells goods on a typical online marketplace to reach individual consumers.

In first- and last-mile logistics, a specific shipment remains in a single vehicle from its origin (the factory in the first mile, the distribution center in the last mile) to its destination (the distribution center in the first mile, the customer in the last mile). These VRPs involve optimizing a fleet of several vehicles over a limited time span, usually a single day. The optimization challenge is essentially one of assignment and sequencing: determining which vehicle handles which set of shipments, and in what order.

In our example, the first mile corresponds to the collection of the items that have been sold by the manufacturer (e.g., pofferts) while the last mile covers the final delivery to the consumers (some of them being quite hungry!). In both cases, a single truck transports goods to or from the regional distribution center. However, if the manufacturer and the consumer are in different regions, middle-mile logistics bridge the gap between far-away distribution centers. For instance, goods from a manufacturer in Groningen (Netherlands) would first move to the regional distribution center in Utrecht, travel to another center in Paris (France) before being delivered to a consumer in Versailles.

In contrast to the first and last mile, the middle mile functions as a relay race. A single shipment may be transported by several different vehicles across a continental network before reaching its final destination, maybe a week after departing. At intermediate distribution centers, the shipment may be unloaded, sorted by destination, and consolidated with other freight before being loaded onto the next vehicle. This creates a complex synchronization problem: the shipment must arrive at a distribution center within a specific time window to catch its scheduled outgoing truck. If it misses its scheduled connection, it will have to sit at the distribution center until the next cycle, leading to significant delays.

In our example, once the manufacturer’s goods arrive in the Utrecht regional center, they are loaded onto the first truck for Antwerp (Belgium) to arrive the same day. Because the most immediate truck to Paris is full, and let’s say the customer opted for standard shipping, the goods take the second truck the following day from Antwerp to Paris. The parcel arrives in Paris on the night of the second day, where it enters the last-mile network for the final delivery to the customer the next day.

Life of a shipment: from the manufacturer in Groningen (Netherlands) to the customer in Versailles (France), a poffert makes most of its journey in the middle-mile network of the freight forwarder.

Mathematical modeling and solver limitations

The mathematical structure of middle-mile delivery differs from the standard VRP in several key ways.

In a traditional VRP, such as those solved by open-source tools like OR-Tools or specialized APIs like Google Maps Platform Route Optimization (GMPRO), the goal is typically to optimize tours for a fleet. The focus is on vehicle routing and sequencing of stops to meet tight customer deadlines. Unlike last-mile delivery, middle-mile logistics has the added flexibility of moving between trucks. We model this added dimension as a multi-commodity flow problem on a space-time graph. In these models:

Nodes: Represent a specific distribution center at a specific time interval.
Arcs: Represent vehicle movements over time, or a shipment being held at a distribution center (storage/sorting by destination).

Hard constraints

While many academic VRPs are defined with few constraints, middle-mile operational constraints are difficult to relax without distorting the structure of the operational problem at hand:

Fixed schedules: Vehicles typically operate on fixed timetables that must be respected.
Distribution center throughput: Distribution centers have physical limits on how much volume can be sorted or cross-docked within a given hour.
Synchronization: The arrival of one vehicle is the prerequisite for the departure of shipments on a different vehicle.

Because of these dependencies, existing VRP solvers cannot apply to the middle mile. The problem requires a sequence of intermediate distribution centers and assignments across multiple vehicles, often over a multi-day time horizon.

MilleMiglia: Generating realistic benchmarks

Data-driven distributions

MilleMiglia uses a variety of statistical distributions to ensure that the synthetic networks look like actual distribution networks without revealing any private information:

Spatial distribution: Distribution centers are placed using gravity models or spatial clustering to reflect real-world population and industrial density.
Demand: Shipments are generated with origin-destination pairs, following realistic volume and weight distributions.
Rotations: The generator creates structured vehicle schedules rather than arbitrary connections between nodes, linking either two major distribution centers or a major distribution center and its neighboring, smaller-scale distribution centers.

The distributions interpolate between publicly available information from industrial actors and privately disclosed data.

Performance and scale

MilleMiglia is written in C++. It uses Protocol Buffers for data serialization, so that the data in its diversity can be stored in a single file for each instance. Thus, the generated instances are compact and can be easily consumed by solvers written in different programming languages.

Unlike VRP instances, with many variants such as the CVRP (with capacities), VRPTW (with time windows), or PDPTW (pickup and delivery with time windows) to capture diverse operational requirements, the structure of our middle-mile data format embeds all interesting constraints in the same file format: fixed vehicle schedules, distribution-center throughput limits, and complex synchronization prerequisites are all fundamental elements of the problem structure.

The intent is to provide the community with a range of instances:

Small instances: Equivalent to academic "toy" problems for testing exact algorithms.
Industrial instances: Large-scale, continent-wide problems. These problems require advanced heuristics or metaheuristics to find good solutions.
Any size in-between, with instances of medium size and/or hardness.

The generator also enables learning scenarios, as it can create huge data sets to train ML algorithms.

Collaborative research and future solvers

MilleMiglia is the first step toward a standardized benchmarking suite for middle-mile logistics, similar to what CVRPLIB (Capacitated Vehicle Routing Problem Library) provides for the VRP community.

This project comes from an ongoing collaboration between Google and academic partners at UniBrescia and ENPC Paris. Beyond instance generation, we are currently working on a specialized solver and API designed specifically for middle-mile operational problems. This solver aims to leverage the unique structure of middle-mile flows.

By open-sourcing our instance generator, we hope to encourage the broader research community to focus on the operational challenges of the middle mile, leading to more robust and efficient global supply chains. We hope to start a challenge on middle-mile problems to increase the interest from academics and industrial solver developers in this underlooked-but-in-need-of-optimization venue. Anyone interested in the field can start by looking at a sample instance hosted in the GitHub repo.

Acknowledgements

This research was primarily conducted by Aymane Lotfi during his Student Researcher tenure at Google and by Matteo Petris (now at ENPC Paris), as part of an ongoing collaboration. Thanks to Thibaut Cuvelier and Bruno De Backer for their contributions to this work. Special thanks to Claudia Archetti (now at UniBrescia) for her leadership and support.

Google — Invalid Date

Digital educational tools have transformed how students around the world access information, from online textbooks to video libraries. Yet, for all the remarkable leaps in technology and accessibility, digital learning can often feel like a passive experience. Interactive, engaging, multimodal forms of practice that can encourage students to think for themselves and work through solutions have great potential for learning but remain largely out of reach. They are expensive to create, limited in number, and often require a lot more effort from the teacher. We wanted to see if AI could help close this gap.

Today, we’re sharing our latest research which pushes the frontiers of interactive learning. Our new research experiment allows educators to create custom, interactive, and guided educational simulations. These learning interactives are tailored to the teacher’s objectives and curriculum, and are generated dynamically, leveraging a novel application of generative user interfaces (GenUI) that we’ve optimized for learning.

Having received initial positive teacher feedback from a trusted tester pool, we’re also releasing a sample library of over 30 learning interactives in English for STEM subjects including physics, chemistry, biology, and math with a focus on middle and high school. These are all generated by AI and reviewed by teachers. Schools using Google Workspace for Education can sign up to provide feedback to improve learning interactives through the Google for Education Pilot Program. This pilot is an early step toward developing more learning interactives for public use.

The library is available for public use and includes learning interactives on a variety of biology, chemistry, computer science, earth science, mathematics and physics topics.

The case for active learning

Learning is not a spectator sport. From the work of John Dewey, a foundational education theorist, who argued back in 1916 that we should “give the pupils something to do” to that of Jean Piaget, the influential psychologist whose pioneering work showed how learners construct knowledge, it is well established that students learn better through active engagement. Modern cognitive research, such as the ICAP framework, affirms that interactive behaviors consistently yield deeper schema construction and long-term retention than passive listening or reading. In short, students learn by doing. When students actively experiment, test hypotheses, and solve problems, they build a much more complete mental model.

Active learning is one of the key learning science principles that we optimize for in our research. It is fundamental to LearnLM, Google’s family of generative AI models fine-tuned for education released in 2024, and was explored in a 2025 Learn Your Way research experiment that reimagines the classic textbook with generative AI. Building on this earlier research, we set out to explore how the latest advances in generative models could be used to further transform content, helping teachers create digital learning that is much more active and engaging.

Adapting generative UI for education

To make this possible, we turned to generative UI, an active area of research whereby AI models dynamically construct user interfaces rather than requiring those interfaces to be coded in advance.

We explored how to optimize generative interfaces for deeper educational journeys as opposed to quick interactions. By using carefully guided instructional design and pedagogical guardrails, we want to empower teachers to create their own interactive environments — tailored to their curriculum and adapted to their contextual inputs.

Instructional design

We first sought to determine what good, interactive learning experiences look like. We drew on established learning science to define a number of key pedagogical principles, aligning with those behind the development of LearnLM:

Aligning with a curriculum and teacher-approved learning objectives (e.g., for earth science, comparing how varying degrees of cloud cover influence local temperature and predicting how wind speed and direction affect weather patterns).
Promoting active, inquiry-based learning, which requires both motivation and guidance to be effective.
Ensuring each simulation is factuality accurate.

These principles come to life in our game-based learning design. To encourage motivation, each learning interactive features a series of progressively difficult challenges, based on the learning objectives (e.g., in the earth science example mentioned above, the first level focuses on the temperature, before progressing to harder challenges about rapid warming and storms). This is combined with a suite of scaffolded hints, instructions and feedback (e.g., directing the learner to the relevant formula or explaining a specific term) to provide each individual learner with the support they need to complete each level.

We define generation requirements to include:

Careful articulation of learning objectives: By design, the educator is in the lead and suggests the topic they want to focus on. We then generate a set of precise and coherent learning objectives. These are modifiable and can be tailored to suit the curriculum goals. They must be approved by the teacher and they serve as the basis for the generation of the learning interactives.
Structured game levels: We build upon elements of game-based learning and break each complex topic into levels with clear goals aligned with the learning objectives. The students explore the topic through a series of progressively difficult challenges (see the progression of levels at the top of the visual below). This promotes active experimentation and sustains learner motivation by pairing deliberate practice with calibrated challenge, fostering a growing sense of competence as students gain proficiency in increasingly complex concepts.
AI generated scaffolding: In order to effectively guide learners through the levels, we generate a suite of scaffolded guidance. Shown below, this includes an introduction to prime a student’s prior knowledge, a toolbox with relevant formulas and theories, multiple levels of hints, tailored feedback reflecting on why a specific response is working or not working, and worked solutions to strengthen comprehension after the student’s own exploration. These were all tested and iterated upon with teachers and students. By providing real-time, context-aware guidance, rather than simply revealing the answers, we encourage critical thinking and help students figure out the solutions for themselves.

The learning interactives interface features hints, feedback and a toolbox to help guide students through the various levels.

Iterative generation with pedagogical guardrails

To ensure quality control, we built self-correcting loops into the generation process — meaning that it is an iterative process, driven by a number of pedagogical guardrails. While this increases the time required to generate the final learning interactives, the aggressive reinforcement loop ensures closer adherence to quality criteria. These criteria include pedagogy (e.g., are the levels correctly covering the learning objectives and becoming progressively harder?), the mechanics (e.g., do the buttons work? Can this level be solved?), and visual aspects (e.g., are there redundant objects on the interface that could be distracting?). Within the self correcting loops there are auto evaluation processes that are agentic in nature (e.g., a solvability evaluation opens a Chrome instance and interacts with the simulation as if it were a user.) The goal is not just to test the validity of a specific solution but also to try adversarial actions such as taking knobs to extreme values. The self-correcting loop repeats until the generated outcome meets all of the required criteria.

Flow chart depicting the different validation stages built into the creation of a learning interactive.

Generated by AI, vetted by teachers

Throughout our research, a core guiding principle has been that technology should be in service of educators and their goals. The teacher is at the heart of any classroom and is best placed to understand not only which AI-driven simulations would engage their students, but also when and where they fit into the curriculum. All learning interactives released in the library and available today were vetted and approved by teachers. These include topics from school curriculums such as Kepler's Laws of Planetary Motion, Data Visualization and Projectile Motion.

In addition, a collection of learning interactives was evaluated by STEM teachers in the UK. Results show that overall rating is good or excellent with physics and chemistry being the most amenable to simulation creation. Full details and results are available in our tech report.

Average ratings of pedagogical criteria by expert teachers for 40 STEM learning interactives.

We also conducted an initial study with 12 teachers in the US. Each of these teachers requested three different custom interactives, which were generated for their specific classroom needs. The feedback was highly positive with an average teacher rating of 8 out of 10 on the interactives’ quality. Teachers highlighted how dynamic generation solves a long-standing classroom challenge: the inability to differentiate instruction using static, off-the-shelf simulations. As one high school science teacher explained, “If I was teaching and I could type this in [for any curriculum topic] and then a simulation would [be generated], that would be amazing... I've never been able to differentiate any of the simulations because it's just, you get what you get“.

Educators also noted how closely the generated design elements aligned with their instructional goals: “That's why this was exciting to actually craft and build something that aligns perfectly with instructional goals and learning objectives” (middle school science teacher). They also praised the built-in-student scaffolding, noting that the tiered hints and worked solutions model the kinds of step-by-step guidance they provide when supporting students individually, and that the level progressions corresponded well to authentic assessment and practice questions.

Our next steps with teachers

As we expand the library, there is still much to learn and improve, and we will do so in collaboration with classroom teachers.

In collaboration with Google for Education, we will pilot learning interactives in schools and classrooms around the world. Schools can sign up to join an upcoming pilot through the Google for Education Pilot Program, giving their teachers the opportunity to request simulations for any custom STEM concept tailored to their curriculum, learning goals, and grade level. The newly generated learning interactives will be sent to the teacher who requested them for review. Only after teacher validation and approval can new learning interactives be added to our library and available for public use.

In addition, we will be conducting UX research and field studies to evaluate learning gains and student engagement when using learning interactives in classrooms.

By optimizing generative technologies for learning, we come closer to a future where learning practice is more active, effective, and tailored for every moment. We thank teachers for their partnership with this ongoing research and look forward to building learning interactives that can benefit students around the world.

Acknowledgements

Shout out to all those who have contributed to this work: Alex Moy, Alisa Kovshov, Anisha Choudhury, Anna Iurchenko, Ayça Cakmakli, Ayelet Shasha Evron, Brit Mennuti, Diana Akrong, Femi Olanubi, Ian Li, Ido Lerer, Julia Wilkowski, Lidan Hackmon, Michal Gordon, Nir Kerem, Preeti Singh, Rena Levitt, Rotem Yulzary, Sarah Smith, Shlomi Ben Shimon, Sophie Allweis, Tracey Lee-Joe, Tzvika Stein, Yaniv Carmel, Yishay Mor, and Yuri Lev. Special thanks to our executive champions: Niv Efron, Avinatan Hassidim, Maureen Heymans, Amy Keeling, Katherine Chou, Ronit Levavi Morad, Yossi Matias, Chris Phillips and Ben Gomes.