“We hope to create a platform that will allow scholars and the general public to access data across museums through a simple and visually appealing online interface,” said Laura Auricchio, Ph.D., dean of Fordham College at Lincoln Center, a co-principal investigator for the project.

Several representatives from major museums and libraries, including the Metropolitan Museum of Art, the Museum of Modern Art, and the Library of Congress, were present at an October project workshop at Fordham. Joining them were scholars from Fordham, Harvard University, MIT, the New School, Sciences Po of Paris, and University of Potsdam in Germany. The group has been collaborating continually to produce a final report for the NEH in March, after which they’ll seek additional funding for the project.

Connecting Museums and Their Data

Auricchio said that the project is similar to how museums are connected in the physical realm through the exchange of traveling works of art, but instead of art they would be exchanging research data, or metadata, spawned by their collections. Auricchio distinguished the two data sets by using museum “tombstones” as an example. Tombstones are the placards one sees beside a painting in a museum. The metadata would be the boldfaced information found at the top of the placard: the name of the artists, the years the artist lived, the name of the work of art, and the medium. The research data would be the paragraph below the metadata, which would include more nuanced and detailed information about the painting: its history, influences, and place within art history. Also included in the research data would be essays from exhibition catalogs.

“Only a fraction of a museum’s holdings are photographed for catalogs, the rest is represented through this research data and metadata,” she said.

This new platform would help foster “a new kind of knowledge production for scholars, artists, curators, educators, and an interested public,” she said.

Anne Luther, Ph.D., a co-principal investigator on the project, said that one of the primary challenges is that museums publish their data in silos, and even within institutions the internal databases don’t necessarily follow the same protocol. Luther, along with Auricchio, brought the NEH-funded project to Fordham.

“A museum may have one database system they are using, but from department to department they are using it differently,” Luther said at the October workshop. “The goal is to make this data available as a public good, but at the moment they’re [the data]  not speaking to each other.”

The challenge in dealing with large institutions is that the computer science protocols have already been established, in many cases over the course of years. Luther said there have been long-standing efforts that try to connect museum data internationally, but projects that have tried to impose new standards and new protocols have failed.

“We’re not trying to bring new standards to describing metadata, but rather we want to build, on one side, a protocol that would allow us to connect them,” she said. “We want to allow for the diversity of metadata on object descriptions within the museums to remain the same. We’re not asking the museum to rewrite. We’ll fish that out.”

Speaking the Same Language

Of course, “fishing” for common phases that describe a period, or a work of art, is also one of the great challenges for the project.

Sarah Schwettmann, a graduate student at Massachusetts Institute of Technology’s Center for Brains, Minds, and Machines, said a protocol layer that aligns metadata from museums’ digital collections could be the best route.  She noted that with machine learning, which is akin to artificial intelligence, there are increasingly more tools that allow computer scientists to work with and analyze metadata. She said the resulting platform needn’t be a simple search engine or website, but could be something more.

“We could build a protocol that actually asks, ‘Can we compare how different museums talk about items in their collection?’” Schwettmann said at the workshop. “This interface would allow one to interoperate specific terms and cultural language that the various museums have developed over time. This is important because each museum develops bodies of scholarship that are specific to that institution.”

“We want a protocol layer that points back to how individual museums talk about their objects and allows users to interact with and see the diversity in terminology,” she said.

One-Stop Research

Matthew Battles, associate director, metaLAB at Harvard University, noted that today art historians will often need to travel from several galleries, museums, and archives in order to gather the strands of a story about a particular artist, particular genre, and particular period.

“We want to facilitate the research activity of a scholar who wants to tell those stories across an institutional context so that rather than spending five years visiting 25 institutions, they could have access to the data of those various institutions in one place,” he said.

He noted that while diverse institutions feature objects from similar periods in history, they may interpret that history differently. As an example, he noted that all institutions agree there was a Byzantine era, though not all agree on a start date or end date. Where one researcher might want to have a numerically specific date, another might be interested in how various institutions have defined Byzantine.

He said that rather than proposing yet one more system to bring all of the museum systems into alignment, which hasn’t worked anyway, it would be better to provide a “roadmap” of how you can bring the various data into agreement or, if one chooses, eliminate the distinctions.

Battles said the NEH seed money—known as a discovery grant—was key, since the resulting research would be a public good that could impact the way stories are told at exhibitions, in elementary school classrooms, and in higher education, all of which would be “more richly informed by a broader array of resources.”

Guo provided some examples by describing the work of his lab. The lab is organized in several sectors, starting with the Hub, which deals with machine learning and data management and analysis. The Hub anchors other labs, including those focused on data economy, business analytics, data assimilation, computational privacy, social and cultural analytics, and behavior.

Reading the Brain’s Reactions

Often the lab’s work combines two of these sectors. For example, Guo said, the machine methodology of the Hub can overlap with behavior analytics. Together, the two could hold the potential to explain the effects of advertising on a variety of individuals by distinguishing brain reactions based on sex, race, age, economic bracket, and even lifestyle. Subjects cab be wired to an EEG machine to measure brain activity while another machine measures eye movement.

“We want to look at the correlation between the two signals; one is measured on the brain wave and the other is on the eye movement and we translate one signal to the other,” said Guo.

The data can not only measure interest in a particular ad, but also precisely what part of a commercial the subject was looking at when that interest peaked.

Potential Heath Care Benefits

It’s also easy to imagine health care applications, Guo said. Doctors can compare their patients’ EEG data against the profiles of thousands of participants, while including specific details beyond just the eye movements, such as a person’s gait and other muscle movements.

When the data sets are compared, anomalies readily reveal themselves. This information can have practical diagnostic applications for everything from Parkinson’s disease to sports medicine and neuroprosthetics.

A View into Dreams

This meshing of data, together with a person’s body movements, can literally reveal to scientists what is going on in your head.

“If I point, that involves a sequence of activities and those activities constitute a language, a sentence, a grammar,” Guo said. “So, what you can do is align those sentences with a brain activity.”

Then, from viewing maps of brain activity, scientists can tell which type of motion or gesture a person might be carrying out in their thoughts or dreams.

“The implication is that at night when you’re asleep, from inside our machine I can pretty much know what your dream is all about,” he said. “That’s pretty scary, but that’s the research.”

With such a level of intimacy exposed, Guo said scientists must maintain strict ethical standards and subjects must be fully aware of the implications before agreeing to participate in a study.

For more on big data and artificial intelligence, watch the entire lecture below.

Who he is: Director of Advancement Technologies and Business Analytics, Advancement Services

What he does: He is modernizing information systems and making business process improvements for  the Office of Development and University Relations.

How long at Fordham: Two years

When Rocco Pugliese’s family moved to the United States from Bari, Italy, he didn’t speak a word of English. His story, like that of so many New York City immigrants, is one of hard work and innovation.

“We moved here when I was 7-years-old and I was thrust into first grade,” he said. “I had to learn English through total immersion, and it was a struggle.”

The struggle paid off, however, with adaptation becoming a touchstone throughout his career, said Pugliese. From the moment the family settled in Bensonhurst, Brooklyn, his encounters with unfamiliar environments spurred an interest in new places and people, and new opportunities. Over the course of his career, he has worked in the private, government, and nonprofit sectors, observing and learning along the way.

A product of New York City public education, he received his bachelor’s in business administration from Baruch College at the City University of New York. There he studied international marketing management at night while working for a shipping company by day. He delved into business theory in his classes, while gaining practical business knowledge on the job.

“I always had a full-time workload, so I had to be very deliberate about balancing work, studies, and my social life,” he said.

After spending several years acclimating to the corporate world, Pugliese began to navigate a way to “add value” to organizations by combining technological know-how with an understanding of how large organizations operated. And while he was educated in business management, his tech knowledge was self-taught.

“My dad bought me a PC with a 5¼ inch floppy and a green-screen DOS, so that’s what I grew up with,” he said. “I’m not necessarily a ‘tech head,’ but necessity is the mother of invention. Solving business problems using technology is a fun challenge.”

Pugliese worked for startups at the height of the dot com era. There he developed digital tools specific to the companies’ needs. He said that he left the industry just as the market was becoming volatile and companies were beginning to go bust. It was then that he went to work in higher education, at Pace University’s World Trade Institute based in the World Trade Center. He was on his way to work the morning the towers were attacked.

“We relocated, but the institute just wasn’t the same,” he said. “The enrollment never came back, since most of our students were international tenants from the towers.”

Pugliese tried his hand in city government at the Department of Transportation before coming to Fordham.

“The attraction here is that people are very thoughtful about carrying out the mission,” he said.

At Fordham, he said he feels that people appreciate not just what he brings to the table, but the talent that his colleagues in Advancement Services bring as well. He noted that the team, which is part of the Office of Development and University Relations, records more than 20,000 gifts a year—anything from $10 to $10 million or more. Each gift represents a personal affirmation of support from valued alumni, friends, and students, so it’s more than just data, he said.

Nevertheless, Fordham’s vast community requires big data analytics to keep track of the thousands of transactions and interactions.

“Our senior management knows the questions they want answered, so that they can best serve all our constituents across the country and around the world, and fully understand the impact of their gifts and involvement with the University,” he said.

However, if the computer and information science field focuses too narrowly on producing the next best gadget rather than improving technology already in use, the cost could be our safety and wellbeing, cautioned Marios Polycarpou, PhD at Fordham’s 2015 Clavius Distinguished Lecture.

In an April 17 talk on “Intelligent Big-Data Monitoring of Critical Infrastructure Systems,” Polycarpou stressed preempting catastrophic technological failure by focusing on making our smart devices smarter.

This is especially important for technology that underlies our critical infrastructure systems (CIS)—such as power and energy systems, telecommunication networks, transportation systems, and water networks.

“These systems are critical for everyday life and well-being, and people expect them to always be available. But the problem is that they do fail. And when they fail, the consequences are tremendous,” said Polycarpou, a professor of electrical and computer engineering and the director of the KIOS Research Center for Intelligent Systems and Networks at the University of Cyprus.

To prevent natural disasters, equipment failures, or malicious attacks leading to catastrophe, he said, “we need to design smarter infrastructure networks. This means designing smarter software to handle any faulty hardware.”

Smart technology is the next logical step following the “sensor revolution” of the 2000s, Polycarpou said. The introduction of sensors gave technological devices physical capabilities akin to the traditional human senses—balance, pressure, and temperature as well as sight, hearing, smell, and touch.

Now that these gadgets can sense their environments, they need the ability to process the information they gather and make decisions in response—which is the essence of smart technology.

“These devices have sensors that provide information and a brain that processes that information,” Polycarpou said. “They’re not just passive devices that make tasks easier for us, but have intelligent software to … make decisions.”

The growth in global population (Polycarpou cited that in 2000 there were 18 megacities around the world, whereas there will be an estimated 30 megacities by 2020 and 60 by 2050) makes CIS intelligence even more important, as most infrastructure networks are intertwined.

“If there’s an earthquake and everyone tries to use their phones, the communication system could break down,” he said. “There’s a lot of interdependence. When something goes wrong, it can propagate through the other infrastructures.”

The annual Clavius lecture and the Clavius Distinguished Professorship of Science, which is held by D. Frank Hsu, PhD, honors 16th-century mathematician Christopher Clavius, SJ, who helped develop the Gregorian calendar and was an early advocate of Galileo’s heliocentric model of the universe.

On a humid night in July, students crammed into a small fifth-floor classroom at Fordham College at Lincoln Center to hear a talk delivered by Herbert S. Chase, M.D., professor of clinical medicine in the Biomedical Informatics Department of Columbia University and resident scholar at Fordham’s Center for Digital Transformation.

Though the lecture was part of a Graduate School of Business Administration (GBA) course on big data, professors from the Graduate School of Arts and Sciences (GSAS) joined students from the Graduate School of Social Service (GSS).

“That’s what struck me,” said Dr. Chase. “In that room there were these people with such diverse interests.”

Even as scholars debate the very definition of big data, most agree that big data will encompass almost every discipline and that coordination will be key to harnessing its power. For his part, Dr. Chase defines big data as “a humanly incomprehensible amount of information that only a machine with sophisticated algorithms can understand.”

Dr. Chase is not a computer scientist; he is a kidney specialist. His role is essentially that of translator between medical specialists and computer scientists. For big data to produce results, he said, the two disciplines must work in tandem.

“It’s the classic team effort: the computer scientists need to tell the researchers what’s possible, and the experts know the questions,” said Dr. Chase. “And that’s what is happening in every discipline. There’s the content expert and the computer expert.”

For many in computer science, big data is just a new term for an old concept.

“Big Data has been there for a long time, but the phrase has become popular because of the explosion in data collection,” said Frank Hsu, Ph.D., the Clavius Distinguished Professor of Science and professor of computer and information science. “It has been branded by the business sectors, just like the Internet. The Internet has been around since the 1970s, but in the 1990s when the WWW was introduced, everyone said, ‘Oh this is so useful! The Internet!’”

Gary Weiss, Ph.D., an associate professor of computer and information science, traced the etymology of the term to “data mining,” which he said has been popular for the past 10 to 15 years. It was preceded by “machine learning.”

“The other term that is becoming very popular is ‘data science,’” said Weiss, who joins Hsu and other Arts and Sciences faculty to teach Fordham courses on data mining, bioinformatics, and information fusion.

But the question as to what constitutes “big” is nearly as subjective as the definition of big data. Does big data require billions of records? Weiss said that in theory a classic case would be a multinational corporation like Wal-Mart analyzing sales records that include billions of transactions for millions of people. But it could also be argued that big data includes relatively modest health study focusing on a couple hundred people.

For example, through Fordham’s Wireless Sensor Data Mining Lab, Weiss has developed a mobile healthcare application called Actitracker. The app collects large amounts of data about the users’ activity via an accelerometer embedded in their smartphone. Weiss said such “mobile health” applications (yet another fresh tech term to add to the digital lexicon) represent a huge growth area for big data.

“From a single subject we’re collecting data every 50 milliseconds, which is 20 times per second, so you can see how that can add up over 10 hours,” said Weiss—especially when multiplied over the apps’ 147 current users.

The Actitracker system reports 2,535 hours of data. That’s 182.5 million records, or data points, gathered from 147 people. Just one user using the app for 12 hours a day for 30 days generates 26 million records, or 104 million pieces of information.

“Sure there’s the health data, but people are applying these techniques to any kind of data,” he said. “It certainly relates to business. In astronomy a telescope is going to generate terabytes of data. Then there’s the digital humanities.”

“R.P.” Raghupathi, director of GBA’s Business Analytics program and Fordham’s Center for Digital Transformation “Big data was just lazy data that was just sitting there, but now that we have the technology to analyze the data, all sorts of issues are emerging, such as the privacy issues, security issues, as well as governance and the ownership.”

Though his courses are situated squarely in GBA, “R.P.” Raghupathi, Ph.D., director of GBA’s Business Analytics program and Fordham’s Center for Digital Transformation, spent time discussing under-tapped areas of big data in the humanities, such as video, music, text, and audio.

In addition to the course in big data analytics, GBA has developed a host of programming to address market needs, including master’s programs in business analytics and marketing intelligence. All analytics courses are full this semester. A new program in applied statistics and decision-making is awaiting state approval.

Though Raghupathi is enthusiastic about preparing students for big data’s potential, he does have concerns.

“Big data was just lazy data that was just sitting there, but now that we have the technology to analyze the data, all sorts of issues are emerging, such as the privacy issues, security issues, as well as governance and the ownership,” he said. “Who owns this data?”
Raghupathi said that ethics and related issues, such as privacy concerns, are woven into every course at GBA.

Joel Reidenberg, Ph.D., the Stanley D. and Nikki Waxberg Chair of Law, is doing some revealing big data research on the question of ownership and privacy.

Reidenberg directs Fordham Law’s Center on Law and Information Policy (CLIP), which has zeroed in on the use of personal information gathered within big data.

“Big data is a catchphrase that is poorly understood by the general public, and most of it is taking place behind the scenes,” he said. “It involves the large-scale collection of personal information that can be used for predictive modeling of behavior, planning, detection, and surveillance.”

Reidenberg’s recent research through CLIP has centered on education and children’s privacy. As the federal government has encouraged or forced states to set up databases reporting children’s progress, detailed information—ranging from a child’s weight to a bad report for cursing—could become a permanent part of a child’s records.

CLIP is looking into how public schools are outsourcing storage of student information to the digital cloud, which could contain everything from a student’s seventh-grade PowerPoint presentation to his 12th-grade SAT scores.

Entire cities are contracting with data analytic companies that can, in turn, sell municipal information to yet another party, he said. The companies’ business models sometimes include little or no charge for services because they make up their costs by data mining and then reselling information.

Reidenberg said it is quite clear that school districts have difficulty understanding what they’re doing, let alone being able to protect a student’s privacy.

“Does the data get deleted or archived when the kid leaves the school, or does the kid’s seventh-grade blog post pop up when he or she applies for college or a job?” said Reidenberg. “That detailed personal data is part of what’s being crunched in big data and there are questions on the ethicacy of collecting that.”

Raghupathi noted that privacy concerns about identifying patterns in big data were further exacerbated after the National Security Administration mined phone records data. The NSA’s antiterrorism strategy certainly raised public awareness about data mining, but not in a good way, said Raghupathi. He expressed concern that the fallout from controversies could overshadow progress.

“The technology is there for us to use it for good purposes,” he said. “It is very important to resolve these legal and social policy issues before the public perception about big data gets distorted.”