Introduction
A man wears a headset wired to track his brain activity while pulling wooden blocks from a wobbling Jenga tower. That is not a lab experiment for fun. It is one small piece of a much bigger puzzle in robotics right now. Could brain wave data for physical AI be the missing ingredient that finally makes humanoid robots useful in the real world?
That question is exactly what a company called Encord is trying to answer inside a warehouse in San Leandro, California. Robotics companies have spent years chasing bigger models and faster chips. However, Encord’s bet is different. The company believes the real barrier holding back physical AI is not the model at all. It is the lack of real-world data to train that model on.
This story matters for anyone watching the race toward smarter humanoid robots and automated warehouses. It shows that the next major leap in physical AI might not come from a lab breakthrough. Instead, it could come from something as personal as reading human brain signals while people go about ordinary physical tasks.
The Real Bottleneck in Physical AI Isn’t the Model
For years, most people assumed better robots would simply come from better AI models. Encord builds data tooling used to train AI models, and its leadership sees the problem differently.
Vineeth Velmurugan, Encord’s head of robot learning, says the real constraint on humanoid and warehouse robotics is not model architecture. Instead, it is the scarcity of real-world physical training data. Velmurugan previously worked at OpenAI’s robot lab and at Berkshire Grey, a warehouse automation firm, so he has watched this problem unfold from several angles.
Encord did not start out chasing this mission. The company was originally founded to help machine-vision businesses annotate data and evaluate their models. As robotics clients began applying end-to-end learning to manipulation tasks, though, a gap became obvious.
“The data simply does not exist,” Velmurugan said.
That single line explains why Encord shifted from managing data to manufacturing it from scratch.
Why Video Alone Can’t Solve the Physical AI Data Gap
Self-driving car companies have spent over a decade collecting physical-world data on their own, and even they struggle to scale that work. Training robots from video helps to a point. Video, though, lacks the fidelity of real physical interaction. As a result, robotics companies are hitting a wall similar to the one self-driving car makers faced years earlier.
Velmurugan estimates it would take a data set roughly five times the size of YouTube’s entire video library to meaningfully break through this barrier. That number alone explains why data generation has become a business of its own, rather than a side project inside a research lab.
Inside Encord’s San Leandro Robot Data Facility
Encord’s warehouse in San Leandro is ground zero for this experimentation. The company employs pilots, its term for the human trainers who generate the physical movements robots later learn from.
Two main data sources currently power the robotics industry. The first is egocentric video, meaning footage recorded from a worker’s own point of view, often paired with extra camera angles and sensor readings. The second comes from robots operated remotely by humans.
Encord uses both approaches. The company gathers egocentric data from several factories worldwide. Meanwhile, it uses the San Leandro site to test new data types and to build focused skill data sets for fine-tuning specific robot behaviors.
During a visit to the facility, pilots were seen using leader-follower rigs. These rigs pair two robotic arms together. One arm is controlled directly by a human, and the second arm mirrors its movements in real time. Pilots used these rigs for tasks such as pouring coffee from a pot into mugs, which turns out to be surprisingly messy, and stacking poker chips.
“Every humanoid company has asked us for these pieces,” Velmurugan said.
The facility’s storage racks hold everyday household items, including fake flowers in vases, books, plastic vegetables, kitty litter trays and scoops, and bundles of wires. These ordinary objects form the raw material for teaching robots to handle common household chores.
The Ethernet Cable Test Shows Robots’ Dexterity Gap
One task on display revealed just how far robotic dexterity still needs to go. Pilot Sofia Infante used robotic arms to plug and unplug ethernet cables from the back of a server, a task data center operators would love to automate.
Taking a turn at the controls made the challenge clear. Robotic pincers are far less dexterous than human fingers. They also lack the degrees of freedom that human hands and arms take for granted. This gap is exactly what better training data hopes to close over time.
Why Zander Labs Is Bringing Brain Waves Into Robot Training
This brings the story back to the headset. The brain wave sensor that pilot Andrew Ceja wore while dismantling the Jenga tower was built by Zander Labs, a German neuroscience startup. Zander’s core idea is that measuring brain activity can help reveal mental states such as error, intent, and surprise. In theory, that data could produce a more useful training set for AI models.
Encord’s collaboration with Zander is currently a trial run rather than a full rollout. The plan involves building an initial brain wave-tagged data set, running it through customer robotics models, and evaluating whether it genuinely improves performance before scaling anything further.
Lucas Gehrke, a Zander neuroscientist supervising the project, explained that the level of brain activity recorded during a task offers useful signals. Specifically, it can help model builders figure out when a task needs their highest-effort AI models versus when a lighter response will do.
Velmurugan describes this brain wave experiment for physical AI as the “bleeding edge” of solving the robotics data bottleneck. The approach is not proven at scale yet. Still, it shows how far companies will go to find an edge in physical AI training.
Muscle Signal Sensors Add Another Layer of Robot Training Data
Brain waves are not the only new data modality Encord is testing. The company is also developing a system that uses sensors strapped to the forearm to detect electrical signals in muscles.
Standard video of human hands manipulating objects often fails to capture the entire hand and wrist. Velmurugan hopes the arm sensors can help build a 3D picture of hand position at any given moment. That would create a more complete and useful data set for training robotics models.
The Real Cost of Manufacturing Physical AI Training Data
Money is one of the most important parts of this story. Large language model companies built their systems by scraping text from the internet, including sources like Stack Overflow, at close to zero cost. Physical AI does not get that same advantage, and the rising cost of AI compute and chips only adds to the pressure.
Encord annotates its data with detailed physical descriptions of what happens in each video clip, such as “right hand tightens bolt.” These descriptions help LLM-based models understand the physical world with much more precision.
Velmurugan estimates that this kind of dense annotation is worth about 100 times as much as unannotated, or “junky,” ego data when training models for specific tasks. However, it also costs roughly 20 times more to produce. On paper, that trade-off still favors quality data. In practice, though, generating physical training data remains expensive, and that changes the entire economics of building physical AI systems compared with building chatbots.
This is the fundamental limit of comparing physical AI to large language models. Text can be scraped freely. Physical interaction data, on the other hand, has to be manufactured one careful task at a time, a cost dynamic playing out across the industry as AI spending keeps outpacing cash flow at major tech companies.
What Brain Wave Data Could Mean for the Future of Robotics
Encord’s position across the industry gives it a unique vantage point. Working with many robotics companies at once, rather than serving just one client, lets Encord spot which data techniques are gaining traction industry-wide before any single company would notice on its own. Velmurugan says this visibility is helping the field make real progress, even though the underlying data shortage remains unsolved.
Meanwhile, the people doing this hands-on work are building an entirely new kind of career, one of many signs that AI is reshaping jobs rather than simply eliminating them. Both Sofia Infante and Andrew Ceja previously worked at Scale, another AI data annotation company, before joining Encord’s team of roughly a dozen pilots.
Ceja’s path shows how varied this emerging workforce really is. He previously worked at a waste management company, where his interest in technology led him to maintain a robotic trash sorter. Now he spends his days solving physical training tasks for robots instead. As he put it while the Jenga tower toppled in front of him, “It’s something new every day!”
Conclusion
The push to train smarter robots has run into a wall that better algorithms alone cannot fix. Physical AI needs physical data, and that data has to come from real human movement rather than scraped internet text.
Encord’s experiments with brain wave sensors, muscle signal tracking, and leader-follower robotic rigs show how creative companies are getting in order to solve this problem. Whether brain wave data for physical AI becomes a standard training method or stays a niche experiment remains to be seen. For now, it signals that the next major advance in robotics might depend on understanding the human body and mind just as much as it depends on bigger AI models.
As robotics companies race toward more capable humanoid and warehouse robots, expect data generation to remain one of the industry’s biggest bottlenecks. At the same time, it stands as one of its biggest business opportunities.
FAQs
1. What is brain wave data for physical AI?
Brain wave data for physical AI refers to recordings of human brain activity, including signals tied to error, intent, and surprise, captured while a person performs a physical task. Companies like Encord are testing whether this data can help robots learn to perform similar tasks.
2. Why is Encord using brain wave headsets to train robots?
Encord tests brain wave headsets built by Zander Labs to see whether measuring mental states during a task can create more useful training data than video alone. The goal is to help AI models understand when a task needs more effort or attention.
3. What is the biggest challenge in training physical AI models today?
The biggest challenge is a shortage of real-world physical training data. Text data for chatbots can be scraped from the internet at low cost. Physical interaction data, however, must be manually generated, which makes it far more expensive.
4. How much physical training data is needed to advance robotics?
Encord’s head of robot learning, Vineeth Velmurugan, estimates that it may take a data set roughly five times the size of YouTube’s entire video corpus to meaningfully improve physical AI models.
5. What other new data collection methods is Encord developing besides brain waves?
Alongside brain wave sensors, Encord is developing forearm sensors that detect electrical muscle signals. These sensors aim to build a more complete 3D picture of hand movement than standard video alone can capture.
