Ruth and Richard on worked examples

I’m reading e-Learning and the Science of Instruction by Ruth Colvin Clark and Richard E. Mayer.  I’ve admired Clark for years; she energetically and effectively applies research to the problem of learning at work.

One strategy they recommend for elearning (and that you’ll find applies in other situations) is the use of worked examples.

A worked example is a step-by-step demonstration of how to perform a task or solve a problem.

That means that in some cases, a worked example can look a lot like a job aid.  Especially for procedural tasks (those you perform the same way each time), worked examples are natural ways to show specifically how to accomplish some task.

Clark and Mayer offer four guidelines:

  • Replace some practice problems with worked examples.
  • Apply good practice when using text, audio, and graphics in worked examples.
  • Provide diverse, job-realistic worked examples to help build mental models.
  • Train learners to self-explain as they use worked examples.

Practice: less can be more

Remember homework?  It’s an attempt to strengthen the use of procedure skills.  Clark and Mayer cite research (as they do throughout the book) to suggest that you can save learning time by replacing some practice with worked examples.

“One [caveat] is that worked examples are only effective if the learner studies them.”  So design some worked examples as completion problems: partly-worked examples that the learner finishes.

Other approaches: make the worked example interactive — like, say, a widget that allows the learner to change one or more factors and see the result.

The authors point out that worked examples seem to benefit novices more than they do people already skilled in a topic.

The media can work

I heard more than an echo of Ten Steps to Complex Learning. (That’s no coincidence; the book cites research by Ten Steps co-author J. J. G  van Merrienboër.)  Clark and Mayer advocate applying sound principles for media use when you create worked examples.  For instance:

  • Integrate text with graphics; don’t restrict text to a caption at the edge.
  • Use audio to expand on visuals; don’t use it to narrate text on the screen.
  • Personalize.  Use conversational tone.  Use virtual agents (like a coach who addresses the learner).

Act like work

It’s almost depressing to think this point needs stressing.  When you create worked examples, make sure they involve realistic tasks that people face on the job.  (All the more reason to involve typical performers in the design, if you ask me.)

And vary the examples.  That’s more than changing the names; change the structure of the example.  Doing so helps you approximate the range of problems that show up on the job, where not everyone comes in asking the same thing.

…When teaching tasks that require judgment and problem-solving–tasks known as far transfer–more than one example will be needed…

Thre is no one right method for performing these tasks, since each job situation will be different.  Solving these far-transfer tasks, whether in highly structured domains such as programming…or in more ill-defined areas such as sales…requires more flexible knowledge in long-term memory.

Interestingly, worked examples help to lower extraneous cognitive load (the mental burden imposed by the course design).  A variety of examples adds to the intrinsic cognitive load, which can improve learning.

The idea is that the learner works at figuring out what the different examples have in common, and thus builds up her own mental model for the skills in question.

Do-it-yourself explaining

“Successful learners can explain worked examples to themselves, and their explanations focus on the principles behind the examples.”

So Clark and Mayer suggest that a virtual coach can demonstrate how to work through a worked example.  In other words, the worked example is an example of explaining a worked example.  From the text:

  • (Onscreen text in a quality-control unit)
    Take 4 sequential widgets off the line every hour for 24 hours.  These are your subgroups.
  • (Jim, the onscreen virtual coach, in audio:)
    First, I notice that the subgroups are selected on a regular basis–four in a row, every how.

So what?

Here’s what I think is worthwhile about the use of worked examples (and about the book generally):

  • It’s based on research, not someone’s preferred way to present.
  • It works for both procedural and non-procedural skills.
  • It suggests that design does, in fact, matter, so that even an advocate of informal learner can benefit by applying the principles to things meant to foster that learning

7 thoughts on “Ruth and Richard on worked examples

  1. Dave, great stuff. I’m inclined to add in some lessons from Alan Schoenfeld (as described in Cognitive Apprenticeship).

    For one, he found that having the ‘expert’ narration not only discuss the steps, but the rationale behind the decision to take approach X vs Y valuable.

    Secondarily, he recommended having the experts make mistakes, backtrack, and repair. This had both a cognitive and affective/conative role. The former is that it unpacks and elaborates the self-monitoring and using the structure to provide away to remediate. The latter is showing that experts make mistakes too and support learner recognition that you don’t necessarily get it right the first time.

    As always, a focus on the good stuff, thanks!

  2. Clark, I think Ruth and Richard’s book is worth at least two $1200 seminars on e-learning.

    What you’re describing is part of a reality that the “informal learning” label obscures. You don’t need to have that expert in a training classroom, of course, but you do need to have her — you need to analyze the skills, enlist her cooperation, plan logical sequences. It’s not all just going to happen because you need it to.

    And, absolutely, the articulation of problem-solving and of correction is essential. I especially like the point you highlight: experts don’t always get it right. It could well be that openness to being wrong is how they continue to be experts.

  3. Again, great post.

    Another thing to add might be ESOL’s Task-based Language Learning approach [wikipedia link]

    Although it often got diluted into another variant of project learning, the original idea was to have learners complete tasks after seeing/hearing them modeled by native speakers. It never really caught on as a mainstream approach because – in the distant past, like 10 years ago – it was difficult to find/make recordings of native speakers.

    The idea of Cognitive Apprenticeship is a new one to me. But it really fits in with my current Education as a Craft obsession – thanks. I can’t help thinking that we’ve lost the art of having apprentices. Partly because, ironically, we call things Worked Examples when they might be better framed as stories?

    Worked Examples (and that killer idea of lowering extraneous cognitive load) are a neat additional strut to my learning schema.

  4. You’re probably familiar with sites like ChinesePod (and its linguistic siblings) or LiveMocha, which try to incorporate more native speaking and less formal study of grammar and vocabulary lists.

    I’d quibble a bit with the label “story” for a worked example. The worked example is (or should be) built deliberately to support the skill in question. My worry is that “story” is a bit loose; like “objective,” it’s a conceptual hand-me-down that people slip onto nearly anything.

    Your context may be different, of course. An expert at one client used “war story” to mean “detailed example that makes concrete the thing we’re talking about.” To me, “war story” is one of those interminable tangents: “We did something like that back in 89, or maybe 88. The plant manager in Columbus — I think his name was O’Brien, or maybe it was Kowalski — anyway, he was having some trouble with inventory control. Now, the inventory supervisor was this guy from Oregon…”

    The Clark/Mayer book is excellent, clearly written, and packed with references to research. You might want to get a copy.

  5. e-Learning and the Science of Instruction is a great book. I turn to it often as a reference. Clark makes an interesting suggestion about having experts not just demonstrate problem solving, but also demonstrate (correcting of) mistakes during problem solving.

    Ruth Clark has another very useful and interesting book called Building Expertise: Cognitive Methods for Training and Performance Improvement (2008), also published by Pfeiffer. Much of the content will feel very familiar to someone who has read e-Learning, but there is some additional research that she expounds upon and a slightly different slant to the book overall: expertise and performance improvement, instead of e-learning.

  6. Ruth Clark’s put a lot of effort into applying cognitive psychological principles to real-life learning, Matt (as you know). Building Expertise is also on my learning shelf. I’m glad to see I had the sense to cite it here a time or two.

    (Added a little later:)

    From another post here that mentioned Building Expertise, here’s classically-trained violinist Dave Greenberg, talking about learning something like Cape Breton Island fiddling:

    If you’re coming at it from age 30 from Mississippi or wherever, you’ve missed thirty of the most important years, especially that first ten, not listening and being immersed, in a natural way, to this music. So it’s not part of your first language….

    You can’t just say, I’m going to listen to recordings and I’m going to figure out how they’re doing the rhythms. First you have to say, I’m at a huge disadvantage here because I’m learning something like Sanskrit. You just can’t say this is all fiddle tunes and it’s all basically the same. You’re never going to get it that way….

    That’s why most people do believe you have to have it in the blood and be brought up there — because that’s the only natural way of doing it.

  7. I’ve ordered the book and am looking forward to it.

    You’re right, of course, about ‘story’ being a conceptual hand-me-down. But it’s also a conceptual pick-me-up :)

Leave a Reply

Your email address will not be published. Required fields are marked *