What Makes Paris Look Like Paris?-Machine Learning Department - Carnegie Mellon University

What Makes Paris Look Like Paris?

Carl Doersch, a Ph.D. student in machine learning, and his CMU colleagues developed visual data mining software that can detect sometimes subtle features, such as street signs, streetlamps and balcony railings, that give cities a distinctive look. It was a hit at SIGGRAPH in 2012 and is on the cover of December's Communications of the ACM. Read the research article and watch the accompanying video. In a technical perspective, Cornell University's Noah Snavely describes automatically detecting a place's visual signature as "magical."

So, what makes Paris look like Paris? We argued that the "look and feel" of a city rests not so much on the few famous landmarks (e.g., the Eiffel Tower), but largely on a set of stylistic elements, the visual minutiae of daily urban life. We proposed a method that can automatically find a subset of such visual elements from a large dataset offered by Google Street View, and demonstrated some promising applications. This work is but a first step toward our ultimate goal of providing stylistic narratives to explore the diverse visual geographies of our world. Currently, the method is limited to discovering only local elements (image patches), so a logical next step would be trying to capture larger structures, both urban (e.g., facades), as well as natural (e.g., fields, rivers). Finally, the proposed algorithm is not limited to geographic data.