Brain waves and borrowed muscles: the unusual data farms training tomorrow's robots

Key points

• Encord, a data-tooling company based in San Leandro, California, is collecting brain-wave readings from human workers to build richer training data for robots.
• The brain-wave sensors come from Zander Labs, a German neuroscience startup, and the project is currently a trial run to test whether the data actually improves robot performance.
• Encord's head of robot learning estimates it would take a dataset roughly five times the size of YouTube's entire video library to meaningfully advance physical AI.
• Generating physical training data costs real money, unlike the near-free text scraping that built today's chatbot models.
• A growing number of startups now treat training-data creation as a business in its own right, not just a research problem.

Andrew Ceja is pulling wooden blocks from a Jenga tower. The headset he's wearing does not make that simple.

Ceja works at Encord, a company that builds tools for training AI models. His job title is "pilot," Encord's word for a robotic trainer: a person whose movements and decisions become lessons for robot software. The headset records what he sees. It also reads his brain waves.

Why would anyone track a worker's brain waves?

The idea is to give robot software richer clues about what a task actually feels like from the inside. A standard video of someone pulling a block from a tower tells a robot what the hand did. Brain-wave data from the same moment can flag when the worker felt uncertain, made a mental error, or was caught off guard.

The headset was built by Zander Labs, a German neuroscience startup. Lucas Gehrke, a Zander neuroscientist overseeing the project, says the level of brain activity at any moment during a task can help model builders decide when their software needs to work hardest. Encord calls this a trial run: build an initial brain-wave-tagged dataset, test it on real robotics models, see whether it moves the needle before scaling up.

What is the actual problem these companies are trying to solve?

Robot AI is stuck on a data shortage. The chatbot AI behind tools like ChatGPT was trained on effectively the whole internet, billions of pages of text gathered at almost no cost. Physical AI, the kind that teaches a robot arm to pour coffee or unplug a cable, can't do the same thing. You can't scrape a warehouse off the web.

Vineeth Velmurugan, Encord's head of robot learning and a former member of OpenAI's robot lab, puts the gap plainly. He estimates reaching a real breakthrough in physical AI would take a dataset roughly five times the size of YouTube's entire video library. "The data simply does not exist," he told TechCrunch AI.

To build it, Encord runs a facility in San Leandro where workers use leader-follower rigs, paired robotic arms where a human controls one and the other copies every movement, to generate footage of tasks like pouring coffee and stacking poker chips. Storage racks hold fake flowers, plastic vegetables, bags of cables to feed robot training on household and warehouse work.

Encord also fits workers with muscle-reading sensors on their forearms. Video of hands manipulating objects often misses part of the hand; the forearm sensors read electrical signals muscles produce and let Encord build a three-dimensional picture of hand position at every moment.

Each video clip gets a written label: "right hand tightens bolt," for instance. Velmurugan says that kind of detailed annotation is worth a hundred times more than untagged footage for training specific skills, and costs only twenty times more to produce. Good trade on paper. In practice it still costs real money, which is exactly where the comparison between robot AI and chatbot AI breaks down.

We've been watching this data-scarcity problem build across 103 robotics stories in the last 30 days, and the consistent finding is that hardware and model architecture are ahead of the data needed to drive them.

Should you worry?

Not today. But Ceja and his colleague Sofia Infante are doing something that didn't exist as a job a few years ago: manufacturing the raw material that robot software learns from. As that software improves, the robots it trains will take on more of the physical work done by people in warehouses, data centres or homes.

The honest judgement from this beat: the brain-wave angle is genuinely interesting and probably underhyped, but the bigger story is economic. Building physical training data at scale costs orders of magnitude more than scraping text ever did. Whoever solves that cost problem, not just the technical one, will have the real use over where physical AI goes next.