Robots

The technology behind self-driving cars has been racing ahead – and as long as they are cruising along familiar streets, seeing familiar sights, they do very well.

But the University of Toronto’s Florian Shkurti says that when driverless vehicles encounter something unexpected, all that progress can come screeching to a halt.

He offers the example of a self-driving car that is following a large truck on a winter road.

“There’s a wind gust – and now the snow is coming at you, so you can’t see anything,” says Shkurti, an assistant professor in the department of mathematical and computational sciences at U of T Mississauga who runs the Robot Vision and Learning (RVL) lab. “And suppose your LIDAR (light detection and ranging system) misperceives the snow as an array of objects, so it thinks there are a million small objects coming at the car.”

Shkurti’s research extends far beyond self-driving cars to autonomous systems in general. How do they learn? How we can make them learn better? How they can successfully navigate complex environments at the service of humans? That includes making sure that robots can handle so-called “edge cases,” like the snowy truck example – cases where the robot “comes across a rare scenario, for which there’s little or no training data.

“Then you have to either collect more data, or you have to accept that there will be these rare events that your perception system won’t recognize,” Shkurti says.

Simulation is an important training tool. Self-driving cars, for example, can be trained on simulated roads and highways before they’re let loose on actual city streets. But scalability remains a challenge. If an autonomous system has to be specially trained for every possible scenario it might encounter, progress will be haltingly slow; there will be no way to take what’s been learned from one scenario and scale it up so that the system can handle more general cases.

In an ideal world, Shkurti says, a robot could learn similar to the way a human would.

Take, for example, robots that help scientists collect data underwater – an effort Shkurti has been involved with for several years. A human diver “has to collect data manually, one data point at a time, one location at a time,” Shkurti says. “It’s painstaking work; it’s not scalable.”

An autonomous robot, on the other hand, could take over the data collection process if it’s capable of maneuvering underwater and equipped with a camera and other sensors. “If the robot could understand what it’s doing – if it has a model of what the scientist thinks is important to pay attention to in a particular environment – then the robot could collect data on behalf of the scientist.”

Such an approach has many benefits, according to Shkurti: It’s much cheaper to deploy additional robots than to train more scientists; and it frees up the scientist to look after higher-level tasks. “The scientist can give the robot some hints as to where to collect the data – but then the robot can take care of the rest,” he says.

Shkurti, who did his undergraduate studies at U of T before earning his PhD in computer science at McGill in computer science, was hired by U of T in 2018. He recently received a Connaught New Researcher Award for a project titled “Robotics and Machine Learning in the Wild: New Directions in Automated Environmental Monitoring.”

Hey says that while everything about computer science fascinates him, the field of robotics holds special appeal.

“Robotics lets you play in different ‘playgrounds,’ like control, perception, and machine learning,” he says. “It allows you to examine these different fields, and I really valued that – and I still value it.”

As for the lofty philosophical questions that sometimes crop up when people talk about advanced computer systems – such as whether machines could learn to “think” – Shkurti prefers to stay focused on the science. Machines can reason, he says, and they can try to act optimally as they strive to achieve their goals.

“If that’s thinking, then they’re doing it,” he says. “But I don’t spend very much time worrying about ‘consciousness.’ I have enough other things to worry about.”

If robots need to persuade humans, they shouldn't resemble authority figures: U of T study

Christopher.Sorensen — Fri, 24 Sep 2021 20:01:43 +0000

If robots need to persuade humans, they shouldn't resemble authority figures: U of T study

Christopher.Sorensen Fri, 09/24/2021 - 16:01

Cutline

U of T Engineering researchers found that Pepper, a humanoid robot, was less persuasive when it was presented to study participants as an authority figure than when it was presented as a peer-helper (photo by Liz Do)

Tyler Irving

Topic

Breaking Research

CIFAR

Faculty of Applied Science & Engineering

Graduate Students

Mechanical & Industrial Engineering

Research & Innovation

Robotics

Robots

In the future, socially interactive robots could help seniors age in place or assist residents of long-term care facilities with daily activities. But will people actually accept advice or instructions from a machine?

That depends on how the robot in question behaves, according to a University of Toronto study.

“When robots present themselves as human-like social agents, we tend to play along with that sense of humanity and treat them much like we would a person,” says Shane Saunderson, a PhD candidate in the department of mechancial and industrial engineering in the Faculty of Applied Science & Engineering who is lead author of a paper published recently in Science Robotics.

“But even simple tasks – like asking someone to take their medication – have a lot of social depth to them. If we want to put robots in those situations, we need to better understand the psychology of robot-human interactions.”

Saunderson says there’s no magic bullet when it comes to persuasion – even among humans. But one key concept is authority, which can be further divided into two types: formal authority and real authority.

“Formal authority comes from your role. If someone is your boss, your teacher or your parent, they have a certain amount of formal authority,” he explains. “Real authority has to do with the control of decisions – often for entities such as financial rewards or punishments.”

To simulate these concepts, Saunderson set up an experiment where a humanoid robot named Pepper was used to help 32 volunteer test subjects complete a series of simple tasks, such as memorizing and recalling items in a sequence.

For some participants, Pepper was presented as a formal authority figure: It took on the role of researcher and was the only “person” the subjects interacted with. For others, Saunderson was presented as the researcher, while Pepper helped the subjects complete the tasks.

Each participant ran through a set of three tasks twice. First, Pepper offered financial rewards for correct answers to simulate positive real authority. Next, Pepper offered financial penalties for incorrect answers, simulating negative real authority.

Generally, Pepper was less persuasive when it was presented as an authority figure than when it was presented as a peer-helper. Saunderson says the result might stem from a question of legitimacy.

“Social robots are not commonplace today, and in North America at least, people lack both relationships and a sense of shared identity with robots,” he says. “It might be hard for them to come to see them as a legitimate authority.”

Another possibility is that people might disobey an authoritative robot because they feel threatened by it. Saunderson notes that the aversion to being persuaded by a robot acting authoritatively seemed to be particularly strong among male participants, who have been shown in previous studies to be more defiant to authority figures than females and may perceive an authoritative robot as a threat to their status or autonomy.

“A robot’s social behaviours are critical to acceptance, use and trust in this type of distributive technology – by society as a whole,” says Goldie Nejat, a professor of mechanical engineering
who is Saunderson’s supervisor and the other co-author on the paper.

Nejat holds the Canada Research Chair in Robots for Society and is a member of U of T’s Robotics Institute. She and Saunderson conducted the work with support from AGE-WELL, a national network dedicated to the creation of technologies and services that benefit older adults and caregivers, as well as CIFAR.

“This ground-breaking research provides an understanding of how persuasive robots should be developed and deployed in everyday life, and how they should behave to help different demographics, including our vulnerable populations such as older adults,” she says.

Saunderson says the big take-away for designers of social robots is to position them as collaborative and peer-oriented, rather than dominant and authoritative.

“Our research suggests that robots face additional barriers to successful persuasion than the ones that humans face,” he says. “If they are to take on these new roles in our society, their designers will have to be mindful of that and find ways to create positive experiences through their behaviour.”

U of T's Angela Schoellig named to MIT's list of Innovators Under 35

ullahnor — Wed, 16 Aug 2017 14:38:09 +0000

U of T's Angela Schoellig named to MIT's list of Innovators Under 35

ullahnor Wed, 08/16/2017 - 10:38

Cutline

Professor Angela Schoellig demonstrates a drone that can land on water to take environmental samples (photo by Tyler Irving)

Tyler Irving

Author legacy

Tyler Irving

Topic

Breaking Research

Faculty of Applied Science & Engineering

Subheadline

Engineering researcher recognized by MIT Technology Review

Assistant Professor Angela Schoellig of the University of Toronto Institute for Aerospace Studies (UTIAS) has been named one of the world’s top Innovators Under 35 by MIT Technology Review.

The Faculty of Applied Science & Engineering's Schoellig works on control theory and applying machine learning to drones, autonomous vehicles and other robots.

“It’s an honour,” says Schoellig of the publication's recognition. “To be named among this group of luminaries comes as a wonderful surprise. It’s also great for my students and postdoctoral researchers, because it’s really about their work as well.”

As a child, Schoellig was captivated by mathematics. During her master’s degree, she studied mathematical models that could describe everything from the chemical reactions in a living cell to the movements of birds. When she learned that mathematical algorithms could also be used to program robots, she was hooked.

“I really wanted to do something where you directly see the impact and the result,” she says.

As a PhD student at ETH Zurich, where she worked under the supervision of robotics pioneer and U of T engineering alumnus Raffaello D’Andrea, Schoellig and her colleagues worked on software that could enable flying robots to execute a triple flip in the air, as well as other acrobatics. But she quickly discovered that the algorithms alone weren’t enough.

“The model that we had would not provide enough information for the robot to do the task,” she says. “You need to use the data from previous runs to improve the task execution, which gets you closer to how humans learn things.”

In other words, she needed to apply machine learning to robots.

Machine learning, a form of artificial intelligence, has become a common part of our lives – it’s what enables smartphones to recognize voice commands and computer programs to recognize faces in photos. But teaching a robot is a very different challenge from teaching a computer.

“If a computer doesn’t recognize a face, that’s not a big deal,” says Schoellig. “But if a robot makes a mistake, it could crash. While you can train a computer on millions of photos, getting that amount of data for a robot is very expensive and difficult.”

One of Schoellig’s biggest challenges is designing algorithms that are flexible enough to enable robots to experiment but rigid enough to ensure that they will be safe while they are learning the new task.

Another major challenge Schoellig works on is handling dynamic environments, where conditions change over time. One of her current projects aims to enable autonomous drones to make deliveries to remote locations, such as communities in Canada’s north. These drones would need to adapt to changing wind speeds and light conditions that make navigation difficult.

Schoellig also has projects in the mining and environmental monitoring sectors, such as developing a drone that can land on water and take samples to track pollution levels. She is even looking at the possibility of using drones to deliver automated external defibrillators (AEDs) to treat patients suffering cardiac arrest.

For Schoellig, smarter robots have the potential to make our lives better and easier.

“We hear a lot about how robots will replace humans, but that’s not how I see it,” she says. “Humans have always built tools to help them to advance, from a simple hammer to a computer. We couldn’t predict in the 1970s how computers would change our society. I think we are at a similar point with robotics, and I’m excited to see what creative ideas will emerge.”

U of T computer scientist develops programming for effective prosthetics and agile robots

ullahnor — Mon, 31 Jul 2017 18:23:42 +0000

U of T computer scientist develops programming for effective prosthetics and agile robots

ullahnor Mon, 07/31/2017 - 14:23

Cutline

A U.S. veteran tries out a prosthetic arm. A computer scientist from U of T as well as researchers from MIT and Adobe have developed computational methods for improved prosthetic body parts (Photo by Timothy A. Clary/AFP/Getty Images)

Nina Haikara

Author legacy

Nina Haikara

Topic

Scientists designing agile robots and efficient prosthetics can soon expect a new tool in their toolbox.

A University of Toronto computer scientist, researchers from Massachusetts Institute of Technology (MIT) and Adobe Research have developed a set of computational methods that will help automate the design of mechanisms that propel movement.

“The goal is to bring the physical rules of virtual reality much closer to those of actual reality,” says David I. W. Levin, an assistant professor and researcher in the Dynamic Graphics Project lab at U of T’s department of computer science.

Levin researches computational physics, a field which develops methods for a wide range of applications from biomechanical models of muscle to mechanical analysis of 3D printed objects.

“Computers are great at designing things, but they need to be able to predict the real-world behaviour of whatever it is they are designing,” says Levin. “That’s where computer simulation comes in.”

The researchers will present methods such as Dynamics-Aware Coarsening (DAC), which speeds-up simulations, and Boundary Balanced Impact (BBI), which models the impact behaviour of flexible objects. Together, these two methods simulate objects performing real-world flips, throws and jumps at rates 70 times faster than current state-of-the-art methods, while simultaneously improving the accuracy of such computer simulations.

Levin and lead author, MIT PhD student Desai Chen, as well as MIT Professor Wojciech Matusik and Adobe researcher Danny M. Kaufman will present their study this week at the world's largest conference and exhibition in computer graphics and interactive techniques.

Prior to joining U of T, Levin was an associate research scientist at Disney Research, which takes computer graphics and animation outside of the studio, applying it to a real-world problem.

“Dynamics is crucial for understanding how humans and animals move and for building everything from better car tires to more efficient prosthetics,” he says. “That’s what is so exciting about DAC and BBI – they give us a new, powerful, broadly applicable computational tool.”

The work was supported in part by the National Science Foundation (NSF), and Levin's research has also received funding from the Connaught New Researcher Award.

Helping drones find their way home: U of T researchers, government, industry collaborate on visual navigation

ullahnor — Thu, 27 Apr 2017 16:45:47 +0000

Helping drones find their way home: U of T researchers, government, industry collaborate on visual navigation

ullahnor Thu, 04/27/2017 - 12:45

Cutline

U of T's Angela Schoellig and Tim Barfoot are part of a new collaboration with Drone Delivery Canada and Defence Research and Development Canada that will help drones navigate using digital cameras (photo by Roberta Baker)

Tyler Irving

Author legacy

Tyler Irving

Topic

City & Culture

Robots

Faculty of Applied Science & Engineering

Collaboration

Research

Subheadline

"Think of it as Hansel and Gretel with bread crumbs"

A new research partnership between academia, industry and government will enable drones – unmanned aerial vehicles (UAVs) – to safely find the way back to the launch point, even if communication or GPS navigation systems fail.

The technology could greatly expand the use of drones, enabling them to be used over much larger distances in everything from delivery of goods to military reconnaissance. The partnership, which hopes to develop a navigation system that relies on digital photos taken by the drones, is a collaboration between U of T's Faculty of Applied Science & Engineering, Drone Delivery Canada and Defence Research and Development Canada (DRDC).

“If drones are going to move beyond their current applications, they will need to be able to deal with all kinds of challenging conditions without help from ground-based operators,” says Assistant Professor Angela Schoellig. “Visual navigation provides a way forward, and we are excited to apply our expertise in this field to develop a solution.”

University of Toronto Institute for Aerospace Studies' Schoellig is leading the team, which also includes Professor Tim Barfoot.

Currently, GPS technology allows drones to navigate without the help of a ground crew. But if this system gets disrupted by bad weather or malfunctioning equipment – and if the drone is beyond the reach of radio communication – the drone could lose control and crash, posing a safety hazard, not to mention the financial cost of lost equipment and cargo.

Federal regulations require that UAVs remain within visual range of their operators. The idea is that if there is a problem, the operator will notice and step in with manual controls to avert a crash.

But in the future, companies like Drone Delivery Canada envision using drones over larger distances, where maintaining visual contact will be impossible.

“In Canada, there are hundreds of remote communities that lack infrastructure, specifically roads, which makes access to goods both very limited and very expensive,” says Tony Di Benedetto, CEO of Drone Delivery Canada. “We are looking at utilizing drones as a logistics vehicle to help our fellow Canadians in these communities.”

Schoellig and her team are proposing to develop the navigation system which relies on digital photos taken by the drones as they fly. If communications or GPS are disrupted, the drone could use those images to find its way back to launch point.

“Think of it as Hansel and Gretel with breadcrumbs,” says Barfoot. “We’re tracing our path on the outbound flight, and then if something goes wrong, we can follow the trail back to where we started.”

Though the concept may seem simple, the execution is not. The system needs to be able to recognize the same scene in different lighting and weather conditions and to account for jostling due to wind. It also needs to be self-contained, executable on a computer small enough to fit on the drone without access to an external network and consume very little power.

The project is funded by a DND/NSERC Research Partnership Grant, with approximately half of the funding contributed by the two external partners, Drone Delivery Canada and DRDC. In total, the partnership will provide more than $450,000 over four years, after which time the team aims to have the first prototypes ready to fly.

“Creating algorithms that can adapt to the very complex conditions of the real world is very challenging, but the potential gains are enormous,” says Schoellig. “It would open up a whole new range of possible applications, not only for our partners but for the entire industry.”

Di Benedetto agrees.

“We see this as a critical component for the evolution of drone technology in Canada,” he says. “We’re doing something completely new.”

GM Canada president: “The tree of artificial intelligence was planted in Toronto”

ullahnor — Wed, 16 Nov 2016 20:00:52 +0000

GM Canada president: “The tree of artificial intelligence was planted in Toronto”

ullahnor Wed, 11/16/2016 - 15:00

Cutline

Steve Carlisle, head of General Motors Canada, looks at autonomous robots built by U of T's Institute for Aerospace Studies (photos by Roberta Baker)

Jennifer Robinson

Author legacy

Jennifer Robinson

Topic

Global Lens

Institute for Aerospace Studies

Faculty of Applied Science & Engineering

Artificial Intelligence

Subheadline

Steve Carlisle scouts U of T talent, talks partnerships

The automotive industry is going through a period of profound disruption – the most since the car replaced the horse – but Canadian universities like the University of Toronto have what's needed to help automakers leapfrog ahead of the competition, says the head of GM Canada.

That’s what drew Steve Carlisle, president and managing director of General Motors Canada, and his team on Tuesday to U of T's downtown Toronto campus where they met some of the University’s world-leading researchers involved in artificial intelligence, deep learning, robotics and systems control.

The frenzied race to build the autonomous vehicle has brought big new tech players into the automotive field like Google and Tesla and everyone is searching for the next big thing and the talent – like U of T’s computer science and engineering students – to make it happen.

“I tell them in Detroit that the tree of artificial intelligence was planted in Toronto,” Carlisle said, referring to the pioneering work of U of T's Geoffrey Hinton, an emeritus professor of computer science who holds the title of University Professor, the highest distinction U of T bestows on its faculty. Hinton now splits his time between the University and Google.

“And boy, wouldn’t it be great to do some of that work here in Toronto,” he added.

U of T President Meric Gertler (left) greets GM Canada head Steve Carlisle (right) during his visit to U of T's downtown Toronto campus

Canada is well positioned to lead the world thanks to this research prowess and its manufacturing innovation but time is of the essence, Carlisle said. And competition is fierce with places like Silicon Valley and Israel.

No one – yet – has put together all the pieces of how autonomous vehicles will work best from consumer safety to affordable price points, he explained. That’s why companies like GM Canada are open to partnering with leading academic institutions like U of T because “collaborations will help us get there faster,” Carlisle said.

That collaborative spirit is one that U of T has already embraced with a number of strategic research agreements with industry, said Professor David Sinton, vice-dean of research at the Faculty of Applied Science & Engineering.

Along with being a global leader in a number of critical fields, U of T is an important anchor that is leveraging its talent and research excellence with partners like GM to help grow the Ontario economy.

The university is “building a culture of entrepreneurship in our students” with nine campus-led accelerators that support startups to transform great ideas into products, said Vivek Goel, U of T’s vice-president of research and innovation.

For example, the Rotman School of Management’s Creative Destruction Lab is home to 50 artificial intelligence companies – considered one of the greatest concentrations of AI companies in any program in the world.

As part of his visit here, Carlisle met with Professor Tim Barfoot of U of T's Institute for Aerospace Studies (UTIAS) and Canada Research Chair in autonomous space robotics, who provided a demonstration of one of the institute’s autonomous robots on campus.

GM head Steve Carlisle (left) takes a closer look at an autonomous robot with Professor Tim Barfoot (right)

The yellow, golf cart-sized robot, developed with the support of industry partner Clearpath Robotics, is equipped with advanced sensory equipment. For the demonstration, the vehicle used only vision from a camera to determine its movements along the twists and turns of a pathway beside University College.

Carlisle also visited with the Blue Sky Solar Racing Team and met with Raquel Urtasun, associate professor in U of T’s department of computer science and Canada Research Chair in machine learning and computer vision, who gave the GM team a sneak peek at the latest advancements in using deep-learning techniques to improve the ability of autonomous vehicles to interpret the landscape around them.

To demonstrate its commitment to embracing and leading this new future for the automotive industry, GM Canada has announced plans to expand its engineering base in Ontario to reach a total of approximately 1,000 positions over the next few years. The company is also opening the Automotive Software Development Centre in Markham and plans to develop an Urban Mobility Campus with a focus on electric vehicles in southeastern Toronto.

This "new" GM Canada is “not your parents’ auto industry anymore,” Carlisle told students at a recruitment talk. Today’s GM is heavily focused on the “development of new connected, autonomous and shared vehicles and mobility systems.”

“Our goal is to disrupt ourselves,” he said.

The new areas that the automotive industry of the 21st century needs to excel in are mobile connectivity, data analytics, artificial intelligence, advanced battery technology, sensors, software development and cyber security, Carlisle explained.

“After visiting many of our top Canadian universities – including right here at the University of Toronto – I can tell you we have the strengths right in our backyard,” he said.

“In fact, Ontario now produces more qualified science, technology, engineering and mathematics professionals (STEM) than California. Yet, ask yourself, who has the reputation for innovation? That just doesn’t seem quite right. That seems like something we should spend some time on squaring up.”

The wisdom of crowds: U of T PhD student builds "swarm intelligence" robots

ullahnor — Mon, 14 Nov 2016 16:33:52 +0000

The wisdom of crowds: U of T PhD student builds "swarm intelligence" robots

ullahnor Mon, 11/14/2016 - 11:33

Cutline

Engineering PhD student Justin Kim has built nine robots not much larger than a thumb that exhibit “swarm intelligence” (photo by John Guatto)

Scott Anderson

Author legacy

Scott Anderson

Topic

Global Lens

Robots

Faculty of Applied Science & Engineering

Artificial Intelligence

Robotics

Subheadline

These robots could one day help rescue earthquake survivors or explore other planets

A single ant or termite is not very intelligent, but in large groups they can perform complex and useful behaviours – like protecting each other or securing food – that make the entire colony seem smart.

This, in a nutshell, is the kind of intelligence that Justin Kim, a PhD student in engineering, is trying to bring to tiny robots he’s been building over the past seven months in a basement lab of U of T’s mining building.

The robots are not much bigger than a human thumb – 32 mm tall to be exact. Their brain is a 32-bit processor, and they move and turn using two tiny motors. With the help of infrared signals, they can locate their fellow robots and interact with them in particular ways.

To demonstrate, Kim powers up nine of the robots and spreads them several inches apart on what looks like an air-hockey table. He explains that each robot uses an infrared sensor to determine if there is another robot nearby and if so, move toward it.

If not, the robot will pause and wait until another robot passes by. Over the next minute, the robots silently twist and inch closer to each other, slowly clumping together.

Kim emphasizes that he doesn’t program the robots to exhibit the desired collective behaviour. Rather, he gives each one a few simple rules.

For example, to get the robots to form a chain, Kim programs them to “follow a nearby robot if you see one” and “not collide into other robots.”

“They would have no idea that they were supposed to form a chain,” he says. “But that’s the collective behaviour that emerges.”

Although “swarm intelligence” – a branch of AI – is still in its early stages, Kim envisions plenty of potential applications like rescue operations during earthquakes.

“Most of the time human rescuers can’t get through the rubble,” he says. “The robots could be equipped with cameras and a light source that would wirelessly provide video to human supervisors monitoring the swarm.”

Justin Kim talks about his swarming millirobots

In space, such robots could be used to explore other planets or asteroids. Shrunk small enough, “micro-robots” deployed inside the human body à la Fantastic Voyage could be used to attack cancerous tumours, or clean plaque from blood vessel walls.

Worldwide, many researchers are working on various aspects of swarming robots, but Kim says his creations are more modular, meaning that other scientists can easily switch out “off-the-shelf” components without having to rebuild the entire robot.

Kim, who joined the U of T Robotics Association as a first-year engineering student, says his immediate plans are to continue with his thesis under the direction of U of T's Engineering Professor Beno Benhabib and Associate Professor Goldie Nejat – and to keep learning. “It’s an interdisciplinary field so it’s exciting to see what other researchers think of when they build a robot,” he says. “Everything is new and exciting.”

Scott Anderson is the editor of U of T Magazine, where this story originally appeared; read more stories in U of T Magazine