It’s an understatement scale, I guess, because even as my approval increases, the terminology is…less than exuberant, as in these examples:

• That’s okay.  (Barely acceptable.)
• That’s not bad.
• Not half bad.  (Well above average.)
• Not bad at all.
• That’s all right. (At least one Michelin star.)
• Pretty good.  (At least two.)

There’s a theoretical maximum, “really good.”  It’s like absolute zero, only warmer; you don’t find it much in nature.

I asked my children whether they’d ever heard me apply these terms.  They couldn’t say, because it’s hard to talk when you’re convulsed in laughter.

The purpose of a scale is twofold: measuring and evaluating.  Measuring is a comparison with some standard: you’re this tall (in inches, in cubits, in stacked-up poker chips).  You typed 268 characters in 3 minutes and made 4 errors.

Evaluating is forming a judgment, usually by means of a further comparison.  You’re tall for a 14-year-old boy.  You meet the minimum speed required for this job.

Thanks to Stephen Downes’s OLDaily, I came across Clarence Fisher’s connecting assessment.  It’s a rubric he created for middle schoolers “to help students think about the connections and global understandings they are establishing.”

He doesn’t plan to assign grades based on where students are–this is a conversation starter, he says.  To me, it’s a way to say to the student, “This is how it might look if you’re at a beginner level of skill.  This is more-than-beginner.  This is how it looks if you’re accomplished.”

Fisher offered another rubric in an earlier post–one to help grade student blog posts.

What I like about these is that Fisher shares what he’s come up with for a particular situation.  He even provides Google doc versions (blogging rubric, connecting rubric) in case someone wants to use them as starting points.

Pretty good, I’d say.

“Approval scale” image adapted from this CC-licensed photo by mag3737 / Tom Magliery
(images are his; cartoon balloons are mine).

Similar bans, the article claims, exist at William and Mary, the University of Virginia, and other big-name schools.

It’s been years for me since college, so my own notions are just notions.  That rarely stops me from musing.

• That law lecture occurred in a room with a hundred students.  Ipso facto, it seems to me, the average student didn’t get to say ten words.  Not that you have to say something to rework, reconsider, connect what’s new to what’s known–but talking about new material is at least as helpful as writing notes on paper.
• It’s not as if a room without laptops is a room without distractions (or a room that suddenly has interesting lecture).  As a U-Va professor says, “If students don’t want to pay attention, the laptop is the least of your problems.”
• One comment added to the Post story reminds us of all the people who doodled, crossword-puzzled, or just read the sports pages while safely and quietly lodged toward the back of the lecture hall.

I don’t mean to seem one-sided.  No matter how cool your keyboard, even ten people in a room going clickety clickety clickety can be distracting–just as Worlds of Warcraft can be when it’s on the screen of the person in front of you during Conflict in Nineteenth-Century East Asia.

Stepping completely outside things I know about: maybe the tried-and-true formal education approach isn’t always ideal.  A law professor in a lecture hall might not be so impartial about his methods as to concede their shortcomings.  Is a lecture to 100 people an optimal way to achieve whatever the goal is for “democracy and coercion?”

Maybe not–because formal systems like law school have a built-in time and exposure constraint, culminating in not just the law degree but the bar exam.

Mostly I think the question hinges on specifics: what’s the purpose of this (presumably in-person) class?  Why is it in-person?  Am I as the professor dispensing knowledge (the Font of Wisdom approach)?  Am I encouraging people to explore issues, grapple with implications, bring in things from the outside?

Consider the approach of another Georgetown law professor (who does allow laptops).  He told his class that Chief Justice John Roberts was stepping down from the Supreme Court.

That was untrue, as the professor knew–but the news flew out.  It seems the real point of the lesson was: credibility. (Much more on this story at Above the Law, including a follow-up.)

CC-licensed image of lecture notes by Kevin Lawver.

You never have the right of way.
You can only yield the right of way.

Recalling this precept got me thinking about driver education / driver training, and that got me thinking about how people have very different readings for “training,” “education,” and “learning.”

Learning to drive is a good example of a complex skill (the kind van Merrienboër and Kirschner grappled with in Ten Steps to Complex Learning).  We tend to think we know what the outcome of the education or training will be: a good driver.

But what’s that?

On the formal side, it’s really about passing requirements.  If you’re an adult who moves to Maryland, for instance,  you have to:

• Pass a vision test
• Have had an out-of-state license within the past year (no suspensions)

And if your out-of-state license expired a year ago, you have to take “the knowledge and skills tests,” which I take to mean a road test and what was once known as a written test.

I’ve been driving for more than 40 years and have had licenses in four states, but I don’t recall taking more than one road test.  Not that I’m eager to do so, but you do get the impression that if you pass it once, still drive, and haven’t lost your license, you’re doing okay.

Some of what vM&K would call constituent skills for driving are recurrent ones–how to start the car, how to stop, how to steer, how to recognize signals and respond to them.  But there are many non-recurrent skills (things we do differently in each situation).  The other day I was exiting a strip-mall parking lot, wanting to turn right onto the highway.  An oncoming car on that highway had its right turn signal on.

Did that mean he’d be turning into the lot I was exiting?  How could I tell?  How could I help a novice driver figure that out?

My old instructor’s advice about right of way was a kind schema or mental model, like the two-second rule–one way to help new learners acquire cognitive strategies.

In writing about this, I’m seeing more clearly that there’s also an overlap of stakeholders: the general public (represented by the state) wants the roads to be safe; new drivers want to be able to drive; parents want their children to drive safely.

They might not even agree on the outcome.  Is it “status as skilled driver” or simply “holder of a driver’s license?” Is “skilled” the same as “safe?”

(I can answer that one: no.  Just take a drive through heavy traffic with someone who prides himself on what a skillful driver he is.)

Maryland’s Motor Vehicle Adminstration publishes a skills log and practice guide “to help the new drives gain valuable experience in operating a motor vehicle in a variety of conditions and highway environments.”  Maryland now requires 60 hours of supervised driving prior to taking the tests, with 10 of those hours at night.  The parent, guardian, or mentor of the new driver must sign a statement attesting to this, in addition to the 6 hours of behind-the-wheel instruction in the mandatory driver education course.

I like the guide (other than the mid-60s bureaucratic tone of the writing).  A “planning guide” (on the right; click for a larger version) summarizes skills; individual sections amplify them with descriptions, examples, and checklists.

Because of the state’s interest in having competent drivers, it makes sense for the state to have created this.  Is 60 hours the right amount?  Are these skills the right skills?  Will parents or guardians follow the guide, or simply certify that they had?

I can’t say–and, frankly, neither can you.  This is a complex skill; there’s no one right answer.  I think you can make a case that most of the skills in the guide are basic ones for a competent driver.  At the same time, no test is going to guarantee that a new driver, or even an experienced one, will never have an accident.  (I’d settle at times for “will not talk on the phone while driving.”)

I found myself connecting what he says to the world of work; I’ll keep that to myself till after you hear Chris. (Note: You can’t see all Lehmann’s slides, so I’ve posted them below the video clip.)

I actually stopped the video a few times to scribble stuff down.

“Good data costs a lot more than we want to spend.”

That’s true for schools, and it’s also true in the world of work. There’s a lot of lip service paid to Kirkpatrick’s levels and to ROI, but in reality, we can’t afford to assess everything at Level IV, and if we’re doing a full ROI assessment on whether to devote a day and a half of our own time to learning some new technology, we’re going to end up getting to spend more time with our families.

I absolutely believe in the value of data — it’s the requirement for performance improvement — but as I listened to Chris Lehmann, I realized that ofter we are in great shape if we have good enough data. Claude Lineberry (as energetic a guy as Lehmann) hammered in the point that businesses don’t do control groups. Some data, carefully chosen, is a hell of a lot better than no data, which is what many people run with all the time.

“Tests and quizzes as dipsticks…”

When I get gas for my car, I always get a fill-up; I calculate the mileage and record it in a booklet I keep in the glove compartment. This is a kind of dipstick — it’s one stream of data that I can glance at, and if I see a variation from my car’s typical performance, then I go looking for more data and for causes.

Lehmann is pushing back from treating tests as goals. As he talked, I thought of the painful annual corporate ritual, the performance review. More than once in my career, I was asked to create a list of what I’d done so my boss could “update” my goals. In other words, I was backing from accomplishments to goals.

Which, I suppose, is better than being slammed for not doing stuff people forgot about nine months ago. The platonic ideal, where you and your manager (or, you poor schmo, your “leader”) regularly look at what you’re doing, what you’re getting done, and what needs to get done — I don’t know how often that happens, but when it does, it’s the dipstick model in action.

“You want to see what kids have learned, give them a project.”

As Lehmann points out, we adults learn when we’re trying to solve something, which means we’re trying to achieve a result. A depressing amount of corporate “learning” involves passive reception: listening to presentations, clicking through page-turners, reading documents. Nothing happens, which means there probably aren’t any new neural connections forming and few old ones getting stronger.

Working on a specific outcome probably leaves gaps in your learning. You can hear someone saying, “Okay, great, you got the web page menus to work entirely with CSS — but you don’t know how to do A, B, and C.” There are two assessments there: was the point to get the menus working, or to do A, B, and C?

I content that much of the time, getting X accomplished is the way to go. If afterward, you feel you don’t have the right result, then you go back and redefine X. I have seen perfectly harmless people subjected to a one-hour lecture on the step-by-step telephone switch, only to learn afterward that their telephone-company employer did not actually own any step-by-step switches; the last one had been replaced more than 10 years before, by computers.

But it was “good for them” to learn about the switches.

In a TED talk in February, 2007, he discussed why we don’t understand as much as we think. Points that stood out for me:

• We look for evidence to support our mental models.
• Some people are all too ready to supply that evidence.
• Early mental models are extremely persistent.
• We collude: we design tests so people pass them.

At one point, Drori says that kids learn by messing around with everyday objects, things that are “bodged and stuffed.” “Bodge” was new to me, but “quick and dirty” seems a good American English counterpart. Urban Dictionary gives this example:

Bodge (verb)… to repair hastily and without care of durability or aesthetics or perfection. Popularized in British television show “Scrapheap Challenge,” known as “Junkyard Wars” in the US, and by producer Cathy Rodgers.
“Your task is to bodge together a hovercraft from nothing but twisted metal, scrapped cars, and other assorted bits of rubbish!”

But he can speak for himself:

(Here’s a direct link to Drori’s talk on the TED site.)