Why Scaling AI Agents Breaks Generalization in New Worlds

TL;DR: A study of 120 configurations reveals that while larger models improve performance in familiar games like Minecraft, they hinder adaptation to new worlds due to over-specialization. This scaling paradox highlights a critical limit in current imitation learning for generalist AI agents.

Key facts

• A study of 120 configurations by miHoYo and Tsinghua University reveals that scaling AI agents improves performance in familiar games but hinders adaptation to new virtual worlds.
• Researchers used continued pre-training (CPT) on Qwen3.5 vision-language models, testing sizes ranging from 0.8 billion to 27 billion parameters across various game environments.
• Increasing training data volume consistently improved ‘in-corpus’ performance in seen games, but diversity mattered more than sheer volume for transferring skills to unseen worlds.
• A balanced mix strategy (S3) using 27% Minecraft data and the rest from other games outperformed single-world training by teaching broader interface skills like movement and menu interaction.
• Larger models (up to 27B parameters) were more data-efficient but generalized worse to distant worlds when trained on single-world data due to a two-phase learning process focusing too heavily on specific mechanics.
• Agents predicted the next 200ms of keyboard/mouse actions in six 33ms steps, performing competitively with specialized Minecraft bots and frontier AI models in real-time settings.
• The open-source release includes inference code and bfloat16 model weights on Hugging Face via the GameScaling GitHub repository, though training code is excluded.

The Scaling Paradox in Virtual Worlds

A study of 120 training configurations reveals a critical limitation in how AI agents learn to navigate virtual environments: while scaling up data and model size improves performance in familiar games, it often hinders the ability to adapt to entirely new worlds [1]. This finding challenges the assumption that simply adding more diverse gameplay footage will automatically create generalist AI agents capable of mastering any 3D environment.

The research, conducted by teams from miHoYo, Tsinghua University, and several other major institutions, investigates whether the scaling laws observed in physical robotics—where more data leads to better performance—apply to video games [1]. Unlike the physical world, where all objects obey the same laws of physics, virtual worlds vary wildly in their visual styles, control schemes, and objectives [7]. The team used continued pre-training (CPT) on Qwen3.5 vision-language models to test this hypothesis across model sizes ranging from 0.8 billion to 27 billion parameters [4].

Data Volume vs. Diversity

The study found a nuanced relationship between the type of data and the resulting capabilities of the agent [1]. Increasing the volume of training data consistently improved performance in games the agent had already seen, known as “in-corpus” performance [5]. However, when it came to transferring skills to related but unseen worlds, diversity mattered more than sheer volume [6].

Interestingly, the researchers identified a specific mixture strategy that worked best. While one configuration (S1) was trained exclusively on Minecraft gameplay, another (S3) used a balanced mix of 27% Minecraft data and the rest from other games [4]. This diverse approach helped the agent learn broader interface skills—such as how to move, jump, or interact with menus—that are common across many games [1].

The Model Size Trade-off

Perhaps the most counterintuitive finding concerns model size. Larger models (up to 27 billion parameters) were more data-efficient, meaning they learned faster from less information [5]. However, when trained on single-world data, these larger models generalized worse to distant worlds than their smaller counterparts [6].

The authors describe this as a two-phase learning process. In the early “interface phase,” the agent learns universal controls that benefit cross-world performance [1]. In the later “world phase,” the agent focuses heavily on specific mechanics of the training game, which narrows its ability to adapt elsewhere [7]. This suggests that current imitation-learning methods may be too focused on replicating specific behaviors rather than understanding underlying principles.

Real-Time Action Prediction

The models were tested on their ability to predict the next 200 milliseconds of keyboard and mouse actions, broken down into six steps of 33 milliseconds each plus mouse movement [4]. In this real-time setting, the agents performed competitively with specialized Minecraft bots and frontier AI models [5]. They excelled in environments where quick reactions were required, demonstrating that CPT can produce highly capable specialists.

Implications for Generalist Agents

Despite these successes, the study concludes that significant gaps remain when agents face “distant worlds”—games with vastly different visuals or control schemes [1]. The authors argue that continued pre-training alone is insufficient for building truly generalist embodied agents [7]. They suggest that richer training stages beyond simple imitation are necessary to bridge this gap, though they do not specify exactly what those stages should be [3].

The open-source release includes inference code and model weights in bfloat16 format on Hugging Face, but notably excludes the training code [4]. This transparency allows other researchers to test these agents in new environments, potentially uncovering more about why current scaling strategies hit a wall when moving from familiar to unknown virtual worlds.

Accessing the Research

The full paper and model weights are available through the GameScaling GitHub repository and Hugging Face [4]. The study provides a detailed empirical scaling law relating validation loss to data volume, diversity, and model size, offering a roadmap for future experiments in synthetic data generation [1]. As AI developers look to use games as training grounds for real-world robotics, these findings highlight the need to balance specialization with generalization.

Sources

1. Scaling in Games: Continued Pre-Training For Embodied Agents In Diverse Virtual Worlds (arxiv.org) — 2026-10-20
2. Scaling in Games: Continued Pre-Training For Embodied Agents In Diverse Virtual Worlds (arxiv.org) — 2026-07-31
3. GitHub - THUSI-Lab/GameScaling: Scaling in Games: Continued Pre-Training for Embodied Agents in Diverse Virtual Worlds. Weights on Hugging Face. (github.com) — 2026-10-10
4. Paper page - Scaling in Games: Continued Pre-Training For Embodied Agents In Diverse Virtual Worlds (huggingface.co) — 2026-10-22
5. Scaling in Games: Continued Pre-Training for Embodied Agents in Diverse Virtual Worlds (www.alphaxiv.org) — 2026-10-20
6. Scaling in Games: Continued Pre-Training for Embodied Agents in Diverse Virtual Worlds (gamescaling.github.io) — 2026-10-20