AP receives Knight Foundation grant of $475,000 to help scale mountains of documents

NEW YORK – The Associated Press will lead the design and development of an open source analytical tool to help journalists find stories in massive amounts of data, thanks to a grant from the John S. and James L. Knight Foundation.

The $475,000 grant, announced today at the MIT-Knight Civic Media Conference in Cambridge, Mass., was awarded as part of the foundation’s Knight News Challenge, an international contest to fund digital news experiments that use technology to inform and engage communities.

So-called document dumps, from tens to hundreds of thousands of pages, “are becoming increasingly common, whether the result of freedom of information requests, government transparency initiatives, or leaks,” the AP said in its grant proposal. “We want to build a tool to answer the question, ‘what’s in there?’"

The AP team, which looks to go beyond searching and indexing, will create “an open-source, production-quality visual analytics system designed specifically for journalistic understanding and discovery within large sets of unstructured or semi-structured text documents, and distributed with comprehensive training materials,” the AP’s proposal said.

“We are delighted to have won the Knight Foundation competition with an idea that will help the entire industry,” said AP Senior Vice President and Executive Editor Kathleen Carroll. “AP is a leader in Freedom of Information Act requests -- our journalists file more than 1,000 a year, unearthing stacks of paper and large databases. A tool that helps make sense of the material in those caches will help journalists and citizens alike better understand the world"

AP Interactive Technology Editor Jonathan Stray, who will lead the AP team, said: “This tool will help journalists find stories in large amounts of data by cleaning it up, illustrating patterns and creating data visualizations. Overview, as we dubbed the tool, will create maps that display relationships among topics, people, places and dates. The goal is an interactive system where the computers do the visualization, while a human guides the exploration.”

“The availability of large amounts of data is both a challenge and opportunity for news organizations,” said John S. Bracken, director of digital media at Knight Foundation. “We see this as a timely investment in an august news organization with a strong team led by one of the nation’s leading data-journalists.”

The Knight funding, which will be used over two years, will enhance AP’s long-standing commitment to developing innovative solutions to meet the digital challenges faced by its member news organizations across the country and around the world.

About the AP
The Associated Press is the essential global news network, delivering fast, unbiased news from every corner of the world to all media platforms and formats. Founded in 1846, AP today is the largest and most trusted source of independent news and information. On any given day, more than half the world’s population sees news from AP.

About the Knight Foundation
The Knight Foundation supports transformational ideas that promote quality journalism, advance media innovation, engage communities and foster the arts. We believe that democracy thrives when people and communities are informed and engaged.