Originally posted on VentureBeat:
Kaggle, a San Francisco-based startup that hosts data science competitions, has uncovered some disconcerting insights about human behavior in its two-year run. At times, its founders have been surprised by the accuracy of an algorithm, and the competitions continue to evoke controversy.
In short, data can be dangerous. I caught up with the company’s founder and CEO, Anthony Goldbloom, to find out more about recent data-driven discoveries that have rocked the boat.
1) “The Essay Scoring Competition”
Sponsor: Hewlett Foundation / Prize: $100,000
Goal: To get the computer to give an essay the same score a human grader would.
The idea was that by analyzing spelling and punctuation, as well as sentence structure, an algorithm could give an essay a reliable score, perhaps even more consistent than a human grader.
Martin O’Leary, a glacier scientist at the University of Michigan, was one of hundreds of competitors from around the…
View original 417 more words