The key to improving robot swarm technology may lie within video games

That’s according to a research team from the University at Buffalo Artificial Intelligence Institute, which received a $316,000 federal grant to study the decisions people make — as well as biometric information such as their brain waves and eye movements — while gaming.

Researchers will use this data to build artificial intelligence they believe can improve coordination among teams of autonomous air and ground robots.

“The idea is to eventually scale up to 250 aerial and ground robots, working in highly complex situations. For example, there may be a sudden loss of visibility due to smoke during an emergency. The robots need to be able to effectively communicate and adapt to challenges like that,”

says the grant’s principal investigator, Souma Chowdhury, PhD, assistant professor of mechanical and aerospace engineering in the School of Engineering and Applied Sciences.

Co-investigators include David Doermann, PhD, director of the UB’s Artificial Intelligence Institute and SUNY Empire Innovation Professor of computer science and engineering; Eshan Esfahani, PhD, associate professor of mechanical and aerospace engineering; and Karthik Dantu, PhD, assistant professor of computer science and engineering.

Swarm robotics research is inspired by many sources; ant colonies and schooling fish are examples. But the potential to improve AI systems by learning from humans is enormous, says Chowdhury, a member of UB’s Sustainable Manufacturing and Advanced Robotic Technologies (SMART) Community of Excellence.

The study, which is funded by the Defense Advanced Research Projects Agency, will center on real-time strategy games. These are time-based (as opposed to turn-based) and involve using resources to build units and defeat opponents. Examples include StarCraft, Stellaris and Company of Heroes.

Students will play a basic real-time strategy game developed by the research team. In addition to recording the decisions the gamers make, researchers will track their eye movements with high-speed cameras and their brain activity through electroencephalograms

Students wears a device that measure the electrical activity in his brain
while he plays video games during experiments for the project.

Inside UB’s SMART Motion Capture Lab, Credit: Douglas Levere, University at Buffalo.

“We don’t want the AI system just to mimic human behavior; we want it to form a deeper understanding of what motivates human actions. That’s what will lead to more advanced AI,”

Chowdhury says.

Eventually, the team will integrate and evaluate artificial intelligence it develops into more sophisticated virtual environments developed by DARPA’s partner organizations.

“This project is one example of how machine intelligence systems can address complex, large-scale heterogeneous planning tasks, and how the University at Buffalo Artificial Intelligence Institute is tackling fundamental issues at the forefront of AI,”

says Doermann.