Now, student-developed software that works with the camera technology promises to help conservationists better understand how to protect future generations of the toads so that they can continue to thrive in their natural habitat in Tanzania. Next year, two more students will pick up the project.

From Africa to the Bronx, and Back Again

Discovered in 1996, the Kihansi spray toad lived in a five-acre microhabitat created by the spray of waterfalls in the Kihansi Gorge, which came under threat with the construction of a nearby dam that dramatically changed the habitat and decreased the size of the mist zone. The species was last seen in the wild in 2005 and was declared extinct-in-the-wild by 2009 by the International Union for Conservation of Nature, likely due to the environmental changes and the emergence of a deadly fungus.

As the toad population declined, a partnership between the Bronx Zoo and the Tanzanian government, and the World Bank facilitated the collection of 499 spray toads to be brought to the Bronx to initiate an off-site conservation program. Custom microhabitats replicating their home in Tanzania were built in bio-secure facilities at the Bronx Zoo and later at the Toledo Zoo where they successfully bred the toads in the hopes of reintroducing them to the wild.

Back in Tanzania, the government managed the Lower Kihansi Environment Management Project to create a gravity-fed misting system. The project resuscitated the toads’ habitat and in 2010 the first 200 toads were returned to Tanzania to a breeding facility at University of Dar es Salaam. The first of several reintroductions to the gorge occurred in 2013, making them the first amphibian species to be reintroduced after being declared extinct in the wild.

Enter Fordham

The Fordham piece of the project began about five years ago when Kelly Cunningham, FCRH ’14, worked with James MacDonall, Ph.D., professor emeritus of psychology, to study the pecking behavior of pigeons. At the time, contact switches and touch-screen sensors were the state of the art for recording pigeons pecking at a target as part of psychological learning experiments, but a disadvantage of that simple mechanism is that when the pigeons’ beaks began to hurt, they stopped pecking at the switches. Further complicating things was the fact that this technology missed when pigeons were distracted or facing the wrong way, said Lyons.

As a computer scientist under the tutelage of Lyons, Cunningham worked in Fordham’s Computer Vision Lab to institute the use of the Microsoft Kinect sensor for the study. The Kinect is a motion-sensing input system initially developed for Xbox. Its cameras presented a flexible and inexpensive image-based approach to solving the tired-beak problem, as well as a way to observe behaviors beyond pecking.

Lyons and Cunningham wrote a paper published in a Psychometric Society journal in 2014 on their findings, which caught the eye of Avi Shuter at the Wildlife Conservation Society’s Bronx Zoo. Shuter is the Senior Wild Animal Keeper in the zoo’s Department of Herpetology.

He was researching the behavior of the Kihansi spray toad, and he thought the technology might be helpful in the zoo’s efforts to better understand the animal. He reached out to Lyons, who in turn put Armando Califano, FCRH ’17, GSAS ’19, on the case.

Taking the Toads to Task

With the help of an undergraduate research grant from Fordham College at Rose Hill, Califano refined the tracking system developed by Cunningham, shifting the camera from Microsoft Kinect to the Intel RealSense, which had more accurate depth perception. But Califano could only take the project so far before entering graduate school, and the experiment was put on hold.  That’s when Philip Bal, FCRH ’19, came into the picture.

In Bal’s junior year he decided to shift his focus from biology to computer science—making him a perfect candidate to pick up the project.  Over the past year and a half, Bal wrote new software that would use the camera to track the toads and generate behavior analytics, ultimately by distinguishing toads from other moving and stationary elements in their tanks. With Lyons overseeing the computer technology and Shuter overseeing the biology, Bal was able to further develop software that responded to the needs of zookeepers.

A Tiny Target

The average size of the toad is no more than an inch, at most. The tanks that they are kept in are the typical fish tank size, about two feet wide, two feet deep, and about three feet high. The camera sits an inch and a half away from the glass. Researchers choose a subsection of the tank to focus on, just a few dozen cubed inches along the bottom or the top. A focal length is established to determine how deep into the tank the camera will take measurements. The camera has two lenses: one that’s recording color, and infrared that records movement.

“We have to do a whole bunch of calculations, try to figure what’s actually a toad moving and eliminate the noise, like moving leaves,” said Bal. “The first thing we do to track toads is to match them to a particular movement.”

Providing a More Accurate Picture

The group gathered approximately two days of footage that took up four and a half terabytes of stored data. Up until then, previous behavioral studies relied on direct observations of toads by scientists at predetermined time intervals. Those projects were an important start, but this new technology and software will give researchers a more complete view of toad behavior, said Shuter.

“Previous studies almost didn’t see any toads hopping,” said Shuter, who worked with the Fordham students and Lyons. “This can be a pretty shy species of toad that hides or stays still when you walk by. A lot of their behavioral repertoire also seems to be made up of split-second movements, like quick calls or hops. So, that’s part of the reason why I thought that a system where a computer could catch all that would give us a more accurate idea of what’s going on.”

One of the things that distinguish the Fordham research from other studies on these toads was that the technology and software were new.

“This is from the ground up; it didn’t exist before,” said Shuter, adding that as a result, the project is more complex than previous studies. “I’m amazed that it has only taken this long to get to where we’ve gotten since it’s totally from scratch.”

Bal said that the project taught him quite a bit about programming.

“I learned what I was capable of, I created thousands of lines of code I never thought I would be able to write,” he said. “This is one of my favorite things to talk about, my passion project.”

Shuter said that when the zoo first recovered the toads in the year 2000, the focus at the time was to build up a colony in captivity that could be relied upon in the event that the wild population continued to decline. The zoo was able to bring the number of toads to almost 2,000 toads.

“The struggle back then was to make more, make more, make more, and we didn’t publish research about their natural history or biology, aside from what would keep them alive, healthy, and breeding in zoos,” said Shuter. “Now, we’re a little bit calmer and things are going well in Tanzania, and we have a good handle on how to keep them alive. So now, we’re starting to look more into, ‘what’s their behavior like?’”

At a recent meeting at the zoo, Bal presented some interactions he observed in the data, including “meetings” of toads, characterized by a certain distance between the toads and the amount of time spent together.

Shuter plans to continue observing these interactions, and also plans to examine fighting behaviors and look to tell them apart from mating—also referred to as amplexing.

And he may get some help from the next cohort of Lyons’ students.

“These guys might end up doing some track analysis for that,” said Lyons, gesturing to two younger students in the lab. “That’s great! We might be able to distinguish fights from amplexing.”