Merlin: Online bird identification with human learning and machine learning

Background/Question/Methods

Each year, 6 million people visit the Cornell Lab of Ornithology’s All About Birds website, many of them attempting to identify birds. Google Analytics data show that these visitors use the search box as an ID tool, typing descriptors such as “small bird with black stripe,” “ID of sparrow with red crown,” or “black bird with pointy head.” However, search engines are not built as ID tools and can return incomplete or misleading results. Existing online ID tools typically use a key-based approach in which users select traits such as color, size, or habitat. The computer matches these descriptors with expert data to identify the species. The challenge is that mismatches between descriptors in the database and those chosen by beginners can lead to false identifications. Using artificial intelligence, we are developing Merlin, an online identification tool that 1) handles “noisy” data by using probabilities rather than dichotomous keys; 2) dynamically asks a question and uses the response to inform the next question; 3) uses citizen-science data from eBird.org to assess the likelihood of encountering each species based on date and location; and 4) taps into a database with expert descriptors and continuously accumulating user-contributed descriptors so Merlin gets “smarter” the more people use it. We will design Merlin to help identify the most common North American birds, with links to more information. We will assess whether this tool helps people learn the names and habits of birds, and enhances identification skills.

Results/Conclusions

In development, we launched two activities that asked people to view photographs of birds and answer questions similar to those that Merlin will ask to help people identify birds. Players have completed more than 250,000 rounds in six months to “teach” Merlin how people perceive and describe birds. Preliminary results suggest that although these activities were designed to help a machine learn, people are learning too, by focusing on and describing birds in an organized way. When people learn the names of birds and other wildlife they encounter, it can open the door to finding more information, enhancing the understanding of biodiversity. If successful, the techniques used to develop Merlin can be adapted for other taxa, providing a new generation of online identification tools to aid the 87% of Americans who say they use the Internet as a research tool.