Integrating data has always been a challenge. The information management community has made great progress in tackling this challenge, both on the theory and the practice. But in the last ten years, the world has changed dramatically. New platforms, devices and applications have made huge volumes of heterogeneous data available at speeds never contemplated before, while the quality of the available data has if anything degraded. Unstructured and semi-structured formats and no-sql data stores undercut the old reliable tools of schema, forcing applications to deal with data at the instance level. Deep expertise in the data and domain, in the tools and systems for integration and analysis, in mathematics, computer science, and business are needed to discover insights from data, but rarely are all of these skills found in a single individual or even team. Meanwhile, the availability of all these data has raised expectations for rapid breakthroughs in many sciences, for quick solutions to business problems, and for ever more sophisticated applications that combine and analyze information to solve our daily needs. In the Accelerated Discovery Lab, we support data scientists working with a broad range of data as they try to find the insights to solve problems of business or societal importance and I will describe the environment we are creating, the advances in the field that enable it, and the challenges that remain.

Laura Haas is an IBM Fellow and Director of IBM Research’s Accelerated Discovery Lab, which is creating a plug-and-play environment to facilitate deriving insight from data. The environment will meet dual goals: (1) to enable research in and improvements to the tools and systems that facilitate discovery, and (2) to enable the business person or domain expert who uses the environment to focus on their investigations, alleviating the systems and data challenges to speed discovery.