Brain waves and borrowed muscles: the unusual data farms training tomorrow's robots

Key points

• Encord, a data-tooling company based in San Leandro, California, is collecting brain-wave readings from human workers to build richer training data for robots.
• The brain-wave sensors come from Zander Labs, a German neuroscience startup, and the project is currently a trial run to test whether the data actually improves robot performance.
• Encord's head of robot learning estimates it would take a dataset roughly five times the size of YouTube's entire video library to meaningfully advance physical AI.
• Generating physical training data costs real money, unlike the near-free text scraping that built today's chatbot AI models.
• A growing number of startups now treat training-data creation as a business in its own right, not just a research problem.

Andrew Ceja is pulling wooden blocks from a Jenga tower. That part sounds simple. The headset he is wearing does not.

Ceja works at Encord, a company that builds tools for training artificial intelligence models. His job title is "pilot," which is Encord's word for a robotic trainer, a person whose movements and decisions become lessons for robot software. The headset on his head records what he sees. It also reads his brain waves.

Why would anyone track a worker's brain waves?

The idea is to give robot software richer clues about what a task actually feels like from the inside. A standard video of someone pulling a block from a tower tells a robot what the hand did. Brain-wave data from the same moment can tell a model when the worker felt uncertain, made a mental error, or was caught off guard.

The headset was built by Zander Labs, a German neuroscience startup. Lucas Gehrke, a Zander neuroscientist overseeing the project, says the level of brain activity at any moment during a task can help model builders decide when their software needs to work hardest. Encord calls this effort a trial run: build an initial brain-wave-tagged dataset, test it on real robotics models, and see whether it moves the needle before scaling up.

What is the actual problem these companies are trying to solve?

Robot AI is stuck on a data shortage. The chatbot AI you know from tools like ChatGPT was trained on effectively the whole internet, billions of pages of text gathered at almost no cost. Physical AI, the kind that teaches a robot arm to pour coffee or unplug a cable, cannot do the same thing. You cannot scrape a warehouse off the web.

Vineeth Velmurugan, Encord's head of robot learning and a former member of OpenAI's robot lab, puts the gap in stark terms. He estimates reaching a real breakthrough in physical AI would take a dataset roughly five times the size of YouTube's entire video library. "The data simply does not exist," he told TechCrunch AI.

To build it, Encord runs a facility in San Leandro where workers use leader-follower rigs, paired robotic arms where one arm is controlled by a human and the other copies every movement, to generate footage of tasks like pouring coffee and stacking poker chips. Storage racks hold fake flowers, plastic vegetables, bags of cables, and kitty-litter trays: props for teaching robot arms the kind of household and warehouse work people want automated.

Encord also uses muscle-reading sensors strapped to workers' forearms. Video of hands manipulating objects often misses part of the hand. The forearm sensors read the electrical signals muscles produce and let Encord build a three-dimensional picture of where the hand is at every moment.

Each video clip gets a written label describing exactly what is happening, for example "right hand tightens bolt." Velmurugan says that kind of detailed annotation is worth a hundred times more than untagged footage for training specific skills, and costs only twenty times more to produce. On paper, that is a good trade. In practice, it still costs real money, which is the point where the comparison between robot AI and chatbot AI breaks down entirely.

What does this mean for ordinary people?

Nothing alarming today. But the workers in these facilities, such as Ceja and his colleague Sofia Infante, are doing something that did not exist as a job five years ago: manufacturing the raw material that robot software learns from. As that software improves, the robots it trains will handle more of the physical tasks currently done by people in warehouses, data centres, and eventually homes.

For now, watch for the gap between what robots can do in a demo and what they can do reliably in a real building. That gap is a data problem, and companies like Encord are the ones trying to close it.