Metrics, Metrics, Metrics: Time-To-Expertise(TTE)

Metrics are very important.  Without clear metrics, it’s difficult to assess where we’re at, how we’re doing, and how things are progressing.

Metrics change our behavior and decisions; “you manage what you measure, and you measure what you manage.”  Consequently, it’s important what metrics you choose to pay attention to. If you choose the wrong metrics, you won’t get the desired outcome.  Choose the wrong metrics to assess a prospective hire, and you’ll hire the wrong person.  Gauge an employees performance by the wrong metrics, and you’re optimizing for the wrong thing.

In my view, we often use the wrong metrics, so I like to think about new types of metrics and what outcomes they might lead to.  These metrics may be infeasible, but it’s an interesting thought experiment that could lead to something valuable.

Here’s one: Time To Expertise (TTE)

Imagine you’re interviewing someone.  You ask them, “what’s something you know nothing about?”  They answer.  You then say,”Go learn about this topic.  Come back when you’ve become an expert on it.”

You then have them record the hours they spent on the subject each day.

So what does such a metric tell you?  First, this information would tell you a lot about their learning and working habits.  It may also give you some indicator on how fast they learn.

But what I find more interesting is that it tells you how much they know what they don’t know. I think this is important to understand about someone. People that know what they don’t know, claim to know fewer things, but when they have something to say on a topic, you better listen up.

Having someone carry this out would probably be impractical, but even just seeing how they respond would provide valuable information. If it’s a relatively narrow topic, if they say, “okay, I’ll be back in 3 days.”  They probably have little understanding of the time it takes to gain deep knowledge on a topic, or don’t really know what deep, meaningful knowledge is.

If they’re clearly an intelligent person, and they say, “well, I’ll see you in a year,” it doesn’t mean they’re slow learners or aren’t very ambitious.  They just know there is a lot to know about something and don’t consider themselves an expert unless they really know something.  Understanding this disposition is important.

-Kevin
3.18.2010