Visualizing Big Data

The wonderful Liz Grumbach, project manager for ARC (http://www.ar-c.org),

and I (Laura Mandell) are here at North Carolina State University where we just received a tour of the most amazing new library, the James B. Hunt, Jr. Library: Markus Wust (markus_wust@ncsu.edu) was our guide as well as big data muse along with Matthew Davis. Markus and Matt, former graduate fellow of the IDHMC and current CLIR Postdoctoral Fellow at NCSU, created a visualization of search returns from the ARC catalog, containing over 1.5 million metadata records that link to digital objects.

When you search that many items, you tend to get search returns numbering in the 20,000s — not a list you can really read through, unless you have a few years:

So Markus and Matt made us the most amazing visualization tool usable on their giant touch screen (one among many of the amazing visualization resources available at the Hunt Library):

Next, Jon Saklofske of Acadia University demonstrated his NewRadial tool for further searching, collecting, and annotating:

These tools will be usable at Texas A&M, with the ARC Catalog AND the 45 million pages that we will adding to it after completion next year of the eMOP Project:

Thanks to the most valiant efforts of Phil Galanter, Nancy Sumpter, and Bryan Jackson,

we are building a data visualization screen at the IDHMC:

With any luck, we will be able to come up with additional funds to add touch-screen capacity. Our wildest dreams include this: you search through 45 million pages of texts published on all topics between 1473 and 1800. Using Matt and Markus’s tool, you visualize categories of returns. You hit one of categories with your finger, and a message comes up: “View in NewRadial?” You say, yes. You are suddenly looking at big text images, all arranged according to your searches. Your students, 7 or 8 of them, are in the room with you. They have handhelds, and they have been searching the 45 million pages using NewRadial, too. They have texts and searches on their screens. Using the new feature added to touch technology by Andruid Kerne, each of the students touch their handhelds to the screen and, with the swipe of a finger upward, transfer the searches and texts on their screens into iFrames on the big screen. Now we are all talking and comparing—walking around, pointing, pulling images next to each other—about our cultural heritage, about the human record centuries old. It will be the newest thing!