The Unreasonable Effectiveness of Data

Big data sets are changing the way we approach the world around us and the culture of inquiry itself say Jacob Vanderplas, but even more problematic may be the unintended consequence that some of the most promising researchers find no place for themselves in the academic community:
Photo by flickr user  j_cadmus

In 1960, the physicist Eugene Wigner published his famous essay, The Unreasonable Effectiveness of Mathematics in the Natural Sciences. It expounds on the surprising extent to which abstract mathematical concepts seem to hold validity in contexts far beyond those in which they were developed. After all, who would have guessed that Riemann’s 19th century studies in non-Euclidean geometry would form the basis of Einstein’s rethinking of gravitation, or that a codification of the rotation groups of abstract solids might eventually lead physicists to successfully predict the existence of the Higgs Boson?


Echoing this, in 2009 Google researchers Alon Halevy, Peter Norvig, and Fernando Pereira penned an article under the title The Unreasonable Effectiveness of Data. In it, they describe the surprising insight that given enough data, often the choice of mathematical model stops being as important — that particularly for their task of automated language translation, “simple models and a lot of data trump more elaborate models based on less data.”

Read more…

Write a Comment

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>