The AI model is trained on images from Conservation International's Tropical Ecology and Monitoring (TEAM) Network, Snapshot Serengeti, Caltech Camera Traps, North American Camera Trap Images, WWF and One Tam, which include 614 species from around the world.

We are adding to this core training dataset frequently on an ongoing basis with data from Wildlife Insights core members: Conservation International, Smithsonian Institution, North Carolina Museum of Natural Sciences, Wildlife Conservation Society, ZSL, and WWF. We have also trained using openly available datasets on lila.science.

Our training data contains images labeled with WI taxonomy including Class > Order > Family > Genus > Species. Species, which is the most granular level, is used as a class label to train a multi-class image classifier.

We are using a deep convolutional neural net for multi-class classification using Google’s open source TensorFlow framework to train an AI model to identify animal species in camera trap images. 

Like humans, AI models generally get better at recognizing and identifying animals if they can look at hundreds or thousands of diverse images of that particular species. If you have camera trap data with many images of popular species or even a few images of rare species, we encourage you to contact us to get trusted tester access to Wildlife Insights, so that you can more easily manage and identify your camera trap images and contribute to the accuracy of Wildlife Insights AI models. 

During training, our AI models learn to recognize the unique characteristics of different species, such as patterns, textures, colors, etc. Using integrated gradients, we can visualize which parts of the image are most important to the model when making a species prediction. See examples of images and the associated integrated gradients visualizations below.

Convolutional neural networks are a widely successful AI paradigm for computer vision models. At a high level, the model takes an image as a 2D input (array of pixels in single channel or RGB channels) and runs mathematical operations in a set of steps. Each step is referred to as a layer. There are some peculiar layer types used in CNNs for images, like convolution and pooling. Multiple such layers are collated together to form a deep convolutional neural network.

Some models have been trained using large amounts of generic image data that can be re-purposed by tuning them to specific problem (like species identification in camera trap images). We start with one such model called Inception-V4 and fine-tune the model for species classification in camera trap images using labeled data from Wildlife Insights.

Why fine-tune from a pre-trained model?

Fine-tuning is done to adapt the model to characteristics of camera trap images e.g. blurring, low lighting, etc. There are many common characteristics learned by the pre-trained model, like detecting edges of objects or identifying patterns like stripes and spots. These generic visual features are useful for identifying species in camera trap images, and by fine-tuning from a model that already has these capabilities, we are able to quickly leverage those features for our species classification task.

How is the model evaluated?

We believe it is important to evaluate our models in a method that is similar to how they will be used. We want to ensure that models will work well for new users, uploading data from camera locations unseen during training. In order to evaluate how well the model does on unseen data, we hold out the images from some of the camera locations in our dataset to serve as an unseen “test set”. We bin all of our dataset camera locations into 10x10meter lat/long grids, and then select a random set of these grid cells to serve as our test set. This ensures that we do not train and evaluate on similar images (e.g. same background), which may lead to incorrectly high accuracy numbers.

Users are able to upload images containing humans to Wildlife Insights, and they will be classified by AI models in the uploading process. However, images of humans will not be made public, nor are they downloadable. Some wildlife researchers are interested in studying human-wildlife interactions. In order to facilitate this, metadata for images containing humans (that does not contain any personally identifiable information) will be available to the public for download (please refer to our Terms of Use for more information).

Across the 614 species that the models have been trained on classes like blue duiker, African elephant, southern pig-tailed macaque or a small antelope called Suni have between 80% and 98.6% probability of being correctly predicted by the AI models. 

Categorizing species in camera trap data can be very challenging, even for humans.  Data quality can play a huge part in our ability to correctly classify an image, and even human experts struggle with images that are poorly illuminated, blurry, or where the animal is very small, hidden behind vegetation, or far away.  There are also many sets of species that are easily confused, like bobcats and lynx. Images that have low data quality or contain easily-confused species are harder for both humans and AI.  That said, AI improves when given many, diverse examples of a given species.  

This is where you can help!  You can correct any mis-labeled species using the Wildlife Insights interface and improve the model accuracy for that species class. In addition, by adding your data to Wildlife Insights, you can help provide sufficient examples of each species to our AI systems so that the species can be accurately identified in the future. By uploading data from your unique camera traps you are not only improving species accuracy.  Each individual camera trap has a set of biases, such as the background, perspective on the animals, and lighting conditions. Your data is also increasing the camera diversity of our dataset, and in turn improving the robustness of our AI models to varied camera conditions.

Once you’ve uploaded your camera trap images to Wildlife Insights, you will see the per-image classification confidence displayed alongside the image in the Identify section of Wildlife Insights. 

You can also look up individual species to see if that species has examples in our current training dataset, and what the model performance is on that class. 

Search for a species of your interest. Let’s take an example of mule deer. If there were total 100 images of mule deer in the data you uploaded, we will be able to identify 81 of those as mule deer. This is the recall for the class. If we predict 100 images as mule deer, about 93 of them are likely to be actually mule deer and the other 7 are likely something else. That is the precision for the class.

If you see “Needs More Data” in the metrics, this means our AI model is not able to predict a single class with confidence above a fixed threshold (threshold is tuned manually). This may happen due to a low number of images of this species in training, due to less diversity in the images (e.g. all from same camera location, similar background etc.), or just because the characteristics to identify the species is shared between many species which confuses the model. You can help improve our model accuracy by contributing additional data to Wildlife Insights for your species of interest. As users upload more images - from different regions, more diverse species etc., our AI models will get better at recognizing more species.

If you do not see your species of interest listed it means that we currently have no examples of that species in our dataset. This is all the more reason to contribute, so we can continue to grow the number of supported species in Wildlife Insights.

Learn more about assessing classification accuracy for AI models in general.