The Morgue – Old Problem, New Solution (Artificial Intelligence)

The Morgue – Old Problem, New Solution (Artificial Intelligence)

What did nuclear anxiety look like in the 1950s? How did a man without a college degree save a mission to space from failure? Did African American men play a role in cleaning up a fire the White House’s West Wing that sparked in Theodore Roosevelt’s abandoned paper archive and interrupted then-president Herbert Hoover’s Christmas Eve Dinner?

The answers to these questions, and an infinite number of others we don’t even know to ask, are buried in The Morgue, the paper-based news archive of the The New York Times. In its heyday, it was manned nearly 24/7 by a staff of 30. It’s 300 tons of clippings and photos dating back to the 1890s. “If it went into the new Times building, all the floors would pancake. It couldn’t stand the weight,” says the lone remaining archivist, Jeff Roth, who tends to this antiquated collection, located several stories underground.

I was introduced to The Morgue while watching the documentary Obit, about obituary writers at The New York Times who dip into the folders to add historical color to the recently deceased. And it struck a chord, because in The Morgue I saw the same problem faced by my clients: digging through a paralyzing amount of data to find a morsel of insight that will make the difference.

For The Times, one such morsel was a picture of a two-year-old Pete Seeger, American folk singer, filed in his musician father’s clippings folder from the 1920s. “I always kept saying, ‘hey you should look at that picture because no one’s going to have it,’ and no one did have it because we paid $10 for it in 1921” says Roth, who happened on the photo before the musician’s death in 2014. The photo ran with Seeger’s obituary. “Once it ran in the paper and on the website the whole world see is, it changes the story,” he says. “It changes the perspective.”

“You always have somebody check to see if there’s something in The Morgue,” says obit writer Paul Vitello. “The fear of missing something, it’s a defining aesthetic.”

Ah, missing something. It’s not hard to imagine missing something when your archive is comprised largely of fragile, century-old newspaper that’s as likely to disintegrate into dust as it is to reveal a juicy lost insight. But are our modern databases really that much better?

We’re buried in data as voluminous as The Morgue, and most of us have only the most rudimentary search functions, getting keyword hits. We draw data from limited sources, search only text, and it’s still up to the Jeff Roth or human on the other end to make connections.

Isn’t it?

Actually, no.

My job at NuSoft is unleashing a new kind of search (and rescue) intelligence to uncover more information about customers or products, which helps companies save or make a lot of money. I’m like an archivist mad scientist, running amok in digitized corporate versions of The Morgue. If held in physical copy, most would be as voluminous. Held digitally, they aren’t that much more useful, devoid of useful search parameters, rife with duplicate and often contradictory records, and subject to every user’s wildly different approach to organization. (I once had a colleague who saved every file as a sequential number and kept a handwritten binder to index the content. I mean, what??)

Getting a person, or even a team of people, to fix enterprise data is unimaginable. This is a computer task, for sure. And it’s doable, with a combination of algorithms and machine learning to uncover tasty data morsels.

Algorithms: Here’s the truth: most AI just reuses algorithms originally invented in the 1940s. They are so old they may well find a fat file about themselves in that old NYT archive, if text ever gets digitized. Like every one of our competitors, we started with those classic algorithms. But we’ve infused different capabilities as well, like the ability to hunt down data that’s hiding in odd places on your corporate intranet, in visual records like presentations and photos, and in synonymous or duplicate-named records. Algorithms go deeper into data to capture more information.

Machine learning: With more data comes more problems. We need the best information to surface first or we just waste time. To do that, we teach the machine. It’s possible to teach the search to better understand what it’s seeing, and even make connections, just like Roth did in looking for a relative to find a famous musician. Machine Learning is like having an infinite army of Roth’s, unleashed in an archive and turning up the best information.

If you’re like most of my clients, you don’t think you have a problem because the idea of a solution with this much power is inconceivable. But I’d like to prove the problem with the solution: Call NuSoft today and we’ll schedule a demo using your own data. We can show you a better way to get your information out of The Morgue and utilize to drive targeted customer messages that work.

Steve Smith
Steve Smith

Data and Artificial Intelligence Practice Lead