This AI knows where you took which photo



summary
Summary

Danger or opportunity? The ability to pinpoint the location of an image has enormous potential in both directions. The PIGEON and PIGEOTTO image geolocation systems are more accurate than all previous technologies – and than humans.

A group of Stanford students demonstrated their PIGEON and PIGEOTTO AI models, which can predict exactly where a photo was taken. This ability opens up new possibilities, but also raises concerns that it could be misused for surveillance or stalking.

Example screenshots of PIGEON in the game GeoGuessr. The AI finds the location to an accuracy of 11 kilometers, a human opponent only to 788 kilometers. | Image: Haas et al.

PIGEON beats GeoGuessr legend

PIGEON was trained on more than 500,000 images from Google Street View. The system requires a panorama of four images as input and was able to guess the correct country 92 percent of the time in test runs. In more than 40 percent of the cases, it found the location with an accuracy of about 25 miles (about 40 km).

Image: Haas et al.

PIGEON is inspired by the online phenomenon GeoGuessr, in which players are virtually teleported to random Street View environments and then have to locate them as accurately as possible on a map.

Ad

Ad

In six consecutive matches, PIGEON even beat community legend Rainbolt, who is considered one of the best GeoGuessr players in the world. In the overall GeoGuessr ranking, PIGEON is in the top 0.01 percent.

PIGEOTTO sets new standards

PIGEON’s progress inspired the developers to create another model. The second model, called PIGEOTTO, was trained on four million images from Flickr and Wikipedia and requires only an image as input instead of a Street View panorama, making it an even more powerful tool for image localization.

In benchmarks with existing systems of this type, PIGEOTTO performed best in class, reducing the median deviation by a factor of two to five.

Sample images from the 2016 MediaEval dataset used to train PIGEOTTO. | Image: Haas et al.

The researchers stress that they are interested in potentially useful applications, not in stoking fears of misuse of the technology. It could help home users find old snapshots. In the scientific field, it could be used to identify invasive plant species.

With PIGEON and PIGEOTTO, the scientists hope to spark important discussions in this area. “To our knowledge, this is the first state-of-the-art image geolocalization paper in the last five years that is not funded by military contracts,” they conclude.

Recommendation

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top