META TURNS EMPLOYEE WORK INTO AI TRAINING DATA

Meta is beginning to collect mouse movements, clicks, keystrokes, and occasional screen snapshots from U.S.-based employees’ work computers as part of a new internal effort to train AI agents on real workplace behavior.

- 

The program, called the Model Capability Initiative, is designed to give Meta’s models examples of how people actually use computers during ordinary office tasks, from navigating dropdown menus to using keyboard shortcuts and moving between work applications. Reuters reported the internal rollout through memos seen by the outlet, and Meta spokesperson Andy Stone confirmed that the data is intended for AI training rather than employee evaluation. The initiative reflects one of the central challenges in the race to build autonomous AI agents. Large language models can write, summarize, code, and reason through text, but they still struggle with many basic interface actions that human workers perform without thinking. Clicking through menus, selecting the right field, switching tabs, copying information from one internal tool to another, and choosing the correct keyboard shortcut all require a kind of embodied software fluency that is hard to learn from text alone. Meta now wants to capture that behavior directly from employees’ work machines.

WHY META WANTS REAL COMPUTER BEHAVIOR

Meta's stated reason is straightforward: AI agents need better examples of how people use computers. According to internal memos reported by Reuters, the company believes its models still struggle with routine desktop actions such as selecting items from dropdown menus and using keyboard shortcuts. Those are not glamorous AI milestones, but they are essential for agents expected to complete real work inside software environments.