Thursday, November 3, 2011

Man or Machine?

By C.J. West, author of Addicted To Love

Last week in the seventh game of the world series an umpire made a mistake that could have turned the course of the most important baseball game of the year. The count was three balls, two strikes when Yadier Molina took his final pitch. The umpire called it a ball. Molina was awarded first and since the bases were loaded, the Cardinals scored a critical run late in the game.

It is well known in baseball circles that umpires tend to call strikes a certain way, meaning their calls might not be consistent with the actual strike zone according to the rules. Pitchers and hitters know this and try to figure out where the strike zone is for that night.

Why does this happen? Where the umpire stands behind the catcher can affect the way he sees balls cross the plate. The umpire also knows when a $10 million player is at bat against a rookie pitcher and vice versa. In a fair world, these things wouldn’t come into play. A strike would be a strike. But umpires are only human.

Before the explosion of electronic technology, umpires were the best way to assure a fair game. Unfortunately for human umpires, we now have instant replay and technologies like the Amica Pitch Zone that can immediately tell us whether the umpire made the right call or not. Television announcers regularly highlight umpire mistakes and some think it is only a matter of time before machines replace the sharp-eyed men behind the plate.

This got me thinking about machines replacing judges of all sorts. Clearly baseball would be fairer if we allowed machines to call balls and strikes. But can machines help us choose books?

I had an interesting experience about two years ago with a recommendation I received from Amazon. The company’s computer algorithm recommended Freakonomics by Steven D. Levitt and Stephen J. Dubner. This was really strange because the only books I’d ever purchased from Amazon were mysteries, thrillers and some computer software books. So how did a computer at Amazon get the crazy idea to recommend a book about consumer economics?

What was more surprising to me is that I loved Freakonomics. I started thinking about how exciting it would be if a computer could judge books as impartially as the Amica Pitch Zone and make recommendations that you would absolutely love?

If you’d like to read a discussion about how the Amazon algorithm works, you can start here

I’m going to speculate that Amazon is doing what is called data mining. They have lots and lots of data about which books customers buy. They know which of these the customers reviewed favorably and which they did not. When we shop at Amazon they also have lots of information about us and can compare us to other shoppers with similar tastes and suggest books they liked that we haven’t read yet. It seems pretty simple, though the technology underneath is certainly complex.

Amazon collects the input of thousands of human reviewers. Their open platform for reviews has its own set of problems, but let’s put those problems aside for now and take this one step further.

Imagine a search tool that actually looks inside books the way Google scans the web for the best page when we search for a toaster oven, swim fins, or a digital camera. It may seem far-fetched, but couldn’t a program analyze a writer’s use of language, settings, and subject matter?

Sometimes I wonder if we really know what it is about a book that makes it unputdownable. Certainly a computer program can’t quantify that, but maybe if the right metrics were combined with reviewer data, those recommendations might help us find hit after hit (for our own personal taste). Since so many books are available in electronic format, the content is readily available to a scanning tool.

We’ve seen the decline of reviews in print media and an explosion of book bloggers. Will the next big thing in books be a computer program that can scan millions of books and point out the next great thing?

6 comments:

  1. Fascinating idea that a software program could analyze manuscripts for market success. Amazon's algorithms aren't a pure form of this because they push books that are already doing well, creating even more success. But who is going to develop such an idea once self-publishing becomes the norm?

    And for the record, I loved Freakonomics too.

    ReplyDelete
  2. Lots of food for thought here, CJ! I find it kind of helpful that Amazon recommends books I might like, based on what I've already purchased from them (or searched, I guess). But those are mainly just categories, like "thrillers", "fiction writing", "photography", etc., as well as individual authors or series I've purchased. And I guess similar authors...?

    A computer program that can scan the content of millions of books for me? We're all busy people, so maybe that would be a great thing -- as long as we didn't rely solely on the computer!

    Then again, there's always the worry that "Big Brother" already knows too much about each of us and our preferences!

    ReplyDelete
  3. I don't pay any attention to what Amazon recommends, mostly because I use Amazon to purchase gifts for friends and family, so their recommendations always include things I'd never buy for myself. Also, I often find it unusual that Amazon tells me I've recently looked at things that I either looked at once several months ago, or have never looked at, ever. So the algorithms may help others, but I'm going to continue finding new authors the old fashioned way - by meeting them (face-to-face or on the Internet).

    ReplyDelete
  4. LJ,

    When I wrote this blog I was thinking about how book marketing favors the big fish and how a computer program could reach down and find great books written by unknown writers. No person can read every book written, but a program could.

    Someday we'll perfect a system for reader reviews so that great books do rise. I wonder what form that will take.

    CJ

    ReplyDelete
  5. Jodie & Gayle,

    I've gotten some great recommendations from Amazon. It is a little spooky that big brother is watching every click and figuring out not only what we'll buy, but how much we'll pay.

    So far though, they seem to be opening markets for the little guys and driving prices down. Two things I'm really in favor of.

    CJ

    ReplyDelete
  6. I thought it interesting when an author friend of mine announced that Amazon had come up with a book she might enjoy . . . one of hers!

    Every time we buy something on a credit card, or other electronic means, data gets sent somewhere. Even at the grocery store. Sometimes good, sometimes full of you-know-what. Like Gayle, I'll research names of books for any number of reasons, and may not actually have any interest in reading them. And Amazon, when it comes to book research, if far more efficient than Google (who has its own set of algorithms).

    If there were some way to separate the wheat from the chaff of books, that would be interesting. My my wheat and chaff are all mixta-uppa, and so any value to me would be lost.

    Interesting post!

    ReplyDelete

Note: Only a member of this blog may post a comment.