Tag Archives: Metadata

How much does metadata cost?

by

It is very hard to find numbers on what it really costs for metadata, but here is one from a Netflix job posting. $6 per movie for “original, descriptive movie and TV episode synopses.”

Here are links to a Hacking Netflix blog posting (likely to remain a valid URL) and to the Netflix job posting (guaranteed to succumb to link rot as soon as the opening
is filled).

The only other published numbers I’ve found are similar, $6.20 to $14.67 per jazz CD depending on the detail in 2003 at the Public Library of Cincinnati and Hamilton County. They were given a collection of 6200 jazz records and were estimating what that gift would cost them. See the article How Much Will It Cost? Making Informed Policy Choices Using Cataloging Standards.

The Netflix numbers are probably closer for an ecommerce or search application. Still, the close agreement in the numbers makes it pretty safe to say “less than $10 per document”.

Remember that the metadata must be updated when the document changes. Maybe “$10 per document per year” is a better number. HP was spending about that much to manage the HP-UX spec (man pages) about ten years ago. That covered all activities, not just metadata.

The Netflix job posting is for six openings, each with a six week duration. That sounds like a lot of work, but if I assume each writer does three synopses per hour (seems very fast for finished work), that is still only 4300 movies. Metadata is very, very expensive.

I have a couple of other stories without dollars, but still instructive.

One publishing company needed to digitize their back content and planned to start a division in the Philippines with 3000 employees to get it done. They found a different way.

I was consulting with a telecom company, and the CEO asked for metadata on every page in their intranet. They had 4M documents.

One final note, since I work for Netflix. All of the Netflix info here is derived from the job posting. No insider information was required or is included in this post.

Advertisements

Everything is Miscellaneous (not!)

by

I checked out David Weinberger’s Everything is Miscellaneous from the local library with every intention of reading the whole book, but after 150 pages, I just can’t waste any more of my time on it. It is just too sloppy — badly researched and the conclusions are flimsy.

One of my father’s frequent sayings is “Do your homework.” This doesn’t mean to finish your spelling paper, it means you should research your subject and not go in half-prepared. Weinberger has not done his homework and his book shows it. He repeatedly starts with bad assumptions, so the book crumbles into a pile of anecdote and opinion.

I find it especially embarrassing to finish something and find out that I’ve done a halfway job of reinventing a known solution, so I try to follow Dad’s advice and do my homework. I’ve been working in search for the past ten years and I’ve done a lot of homework. In multiple places in this book, Weinberger just gets it wrong, and wrong in places where it shoots down his argument.

Let’s take Chapter 3, where he criticizes how libraries organize information. Libraries have been beating on this problem for centuries and they have something that works. It isn’t always pretty, but it works. Weinberger takes the Dewey Decimal Classification (DDC) as representative of all library organization. It isn’t. DDC looks like a classification scheme, but it is really used to put books on shelves in some useful order, not to fully describe their topics. The goal is to get the WWI books near the WWII books, not to decide whether war is history, politics, or technology.

Then he makes fun of Library of Congress Classification (LCC) for putting the the Balkans at the same level as Africa. He manages to make three ignorant mistakes in one example. First, LCC is enumerative, so it doesn’t have a strong hierarchy. It is designed so it is easy to add space at the end, perhaps because there is no end to knowledge. Second, book classification schemes are shaped by when they were designed and by the universe of books. There are lots of books about the Balkans, probably as many as there were about Africa when the scheme was designed. Of course, he makes the same mistake as he made with DDC, confusing a locating scheme with a subject classification scheme.

He certainly should have looked at the Library of Congress Subject Headings (LCSH). These have the cross-referenced graph structure that he wants DDC and LCC to have. They are updated weekly, extensible, and on-line.

Weinberger picks The Little House Cookbook to demonstrate that Amazon’s info on the book is superior to a library “card catalog”. He shows the Amazon categories below, then uses a customer’s list to find additional books about and by Laura Ingalls Wilder.

Children’s Books>Authors & Illustrators, A-Z>Williams, Garth
Children’s Books>History & Historical Fiction>United States>1800s
Children’s Books>Sports & Activities>Cooking

Now let’s look up the record in WorldCat. Amazingly, it is already tagged with Laura Ingalls Wilder as one of the subjects. We don’t need user lists to do that, just good categories and catalogers (both of which are expensive). Aside from the quaint word “cookery”, this list of categories seems more useful than the ones at Amazon:

Cookery, American — History — Juvenile literature.
Literary cookbooks.
Wilder, Laura Ingalls.
Cookery, American — History.
Frontier and pioneer life.

This is his crowning example in Chapter 3, which he uses to show that Amazon is better than his misunderstanding of DDC. But Amazon isn’t better in this example, so the whole thing falls flat.

If an author makes these kind of mistakes in a book about organizing information, I can’t trust him. Library technology has been continually developed since at least Callimachus at Alexandria. If you care about information, you need to grok library technology and its true strengths and weaknesses, not tell us why Melvil Dewey spelled his name oddly.

I recommend you don’t waste your time on this book. I gave up when I was spending most of my time picking apart his examples and not learning anything in the process. Disagreeing with someone who has done their homework is invigorating, but this was just red pencil time.

Instead, choose one of these books. I found each one of these increased the depth and breadth of my thinking. You might not agree with the authors, but you’ll get a good workout doing it.

By the way, Clay Shirky is just as sloppy. His Ontology is Overrated makes the same mistakes. What a mess.

Variant Spellings of “Guns N’ Roses”

by

Paul Lamere posts different spellings of “Guns N’ Roses” and “Tchaikovsky” from ID3 tags. Here are the top twelve for “Guns N’ Roses”:

Guns N Roses
Guns and Roses
guns ‘n’ roses
Guns ‘N Roses
Guns & Roses
Guns’N’Roses
Guns N’Roses
Guns’N Roses
Guns´n Roses
Guns N´ Roses
Guns -N- Roses
GNR

You’ll need to go to the posting for the “Пётр Ильич Чайковский” variants.

Cataloger or Director of Metadata?

by

Marc Siry posts about Job Descriptions from the Future: Director of Metadata, but it sure sounds like a cataloging librarian to me. Here are the key parts of the job description:

  • Fashion metadata requirements for content partnerships
  • Determine metadata needs for editorial programming interfaces
  • Develop a schema that supports product capabilities for discovery and reporting
  • build out the CMS and data entry methods to support the schema
  • plan for future expansions and revisions of the schema as the business evolves

A 1200 word post on managing metadata with no mention of librarians. It looks like NBC Universal is going to be reinventing a lot of stuff. To reword Isaac Newton’s famous comment, this is standing on the toes of giants, not their shoulders.