The main elements in the Minimal Manual test–a task-centric approach to training people in using computer software–were lean documentation, guided exploration, and realistic exercises. So the first document that learners created was a letter. In earlier, off-the-shelf training, the first task had been typing a description of word processing, “something unlikely to be typed at work except by a document processing training designer.”

This sort of meta-exercise is very common, and I think almost always counterproductive. Just as with Amtrak’s training trains that (as I said here) didn’t go over real routes, trivial tasks distract, frustrate, or confuse learners. They don’t take you anyplace you wanted to go.

Not that the practice exercise needs to look exactly like what someone does at his so-called real job; the task simply needs to be believable in terms of the work that someone wants to get done.

Into the pool

After creating the Minimal Manual, Carroll’s team created the Typing Pool test.  They hired participants from a temp agency and put them in a simulated office environment, complete with partitions, ringing phones, and office equipment. These people were experienced office workers with little prior computer knowledge. (Remember, this was in the 1980s; computer skills were comparatively rare. And Carroll was testing ways to train people to use computer applications.)

(Click to enlarge.)

Each group of two or three participants was given either the Minimal Manual (MM) or the systems style instruction manual (SM). Participants read and follow the training exercises in their manuals and periodically received performance tasks, each related to particular training topics. (You can see the task list by enlarging the image on the right.)

Some topics were beyond the scope of either the MM or the SM; interested participants could use an additional self instruction manual or any document in the system reference library.

After finishing the required portion of training material, participants took the relevant performance test. They were allowed to use any of the training material, the reference library.  They could even call a simulated help line. This last resource had an expert on the system who was familiar with the help line concept but unaware of the goals of the study.

So what happened?  Carroll provides a great deal of detail; I’ll summarize what seem to me to be the most important points.

Minimal learning was faster learning.

In all, the MM participants used 40% less learning time then the SM participants — 10 hours versus 16.4. (“Learning time” refers to time spent with either the MM or SM materials, not including time spent on the performance tasks.) This was true both for the basic tasks (1 through 3 on the list) and the advanced wants.

In addition, the MM group completed 2.7 times as many subtasks, as the SM group. One reason was that some SM participants ran out of time and were unable to try some of the advanced tasks. Even for those tasks that both groups completed, the MM group outperformed by 50%.

We were particularly satisfied with the result that the MM learners continued to outperform their SM counterparts for relatively advanced topics that both groups studied in the common manual. This indicates that MM is not merely Wiccan dirty for getting started… Rather, we find MM better then SM in every significant sense and with no apparent trade-offs. The Minimal Manual seem to help participants learn how to learn.

In the second study, more analytical while more limited in scope, similar results were found. In this study, Carol’s group also compared learning by the book (LBB) with learning by doing (LWD).  The LBB group were given training manuals and assigned portions to work with. After a set period of learning, they were given performance tasks. This cycle was repeated three times. The LWD learners received the first task at the start of the experiment, as they completed each task, they received the next one. There was also an SM by-the-book group and an SM learn-by-doing group.

So there are two ways to look at the study: MM versus SM as with previous study, and LWD versus LBB for each of those formats. To make that clear, both sets of LWD learners received at the start both the training materials and the relevant performance test to complete; both sets of LBB learners had a fixed amount of time to work with the training materials (which included practice) before receiving the performance tests.

Among the things that happened:

• MM learners completed 58% more subtasks than SM learners did.
• LWD learners completed 52% more subtasks than LBB learners did.
• MM learners were twice as fast to start the system up as SM learners.
• MM learners made fewer errors overall, and tended to recover from them faster.

Mistakes were made.

One outcome was the sort of thing that makes management unhappy and training departments uneasy: the average participant made a lot of errors and spent a lot of time dealing with them.  Carroll and his colleagues observed 6,885 errors and classified them into 40 categories.)

Five error types seemed particularly important–along the accounted for over 46 percent of the errors; all were at least 50 percent more frequent than the sixth most frequent error…

The first three of these were errors that the MM design specifically targeted.  They were imprtant errors: learners spent an average of 36 minutes recovering from the direct consequences of these three errors, or 25 percent of the average total amount of error recovery time [which was 145 minutes or nearly half the total time].

The MM learners man significantly fewer errors for each of the top three categories–in some cases nearly 50% less often.

This to me is an intriguing, tricky finding. A high rate of errors that includes persistence and success can indicate learning, though I wonder whether the participants found this frustrating or simply an unusual way to learn. I’m imagining variables like time between error and resolution, or number of tries before success. Do I as a learner feel like I’m making progress, or do I feel as though I can’t make any headway?

The LWD participants (both those on MM and on SM) had a higher rate for completing tasks and a higher overall comprehension test score than their by-the-book counterparts. So perhaps there’s evidence for the sense of progress.

Was that so hard?

Following the trial, Carroll’s team asked the participants to imagine a 10-week course in office skills.  How long would they allow for learning to use the word processing system that they’d been working with.  The SM people thought it would need 50% of that time; the MM people, 20%.

Slicing these subjective opinions differently, the LBB (learn-by-book) group estimated less time than the LWD (learn-while-doing) group. In fact, LBB/MM estimated 80 hours while LWD/MM estimated 165.

What this seems to say is that in general the MM seemed to help people feel that word processing would be easier to learn compared with SM, but also that LWD would require more time than LBB.

♦  ♦  ♦

The post you’re reading and its predecessor are based on a single chapter in The Nurnberg Funnel–and not the entire chapter.  Subsequent work he discusses supports the main design choices:

• Present real tasks that learners already understand and are motivated to work on.
• Get them started on those tasks quickly.
• Encourage them to rely on their own reasoning and improvisation.
• Reduce “the instructional verbiage they must passively read.”
• Facilitate “coordination of attention” — working back and forth between the system and the training materials.
• Organize materials to support skipping around.

I can see–in fact, I have seen–groups of people who’d resist this approach to learning.  And I don’t only mean stodgy training departments; sometimes the participants in training have a very clear picture of what “training” looks like, what “learning” feels like, and spending half their time making errors doesn’t fit easily into those pictures.

That’s an issue for organizations to address–focusing on what it really means to learn in the context of work.  And it’s an issue for those whose responsibilities include supporting that learning. Instructional designers, subject-matter experts, and their clients aren’t always eager to admit that explanation-laden, application-thin sheep-dip is ineffective and even counterproductive.

CC-licensed image: toy train photo by Ryan Ruppe.

(The term dates to at least 15th-century Germany, and I suspect the notion of funneling or otherwise stuffing knowledge into someone is a few months older than that.)

The Nurnberg Funnel is humorous as well, in a slightly drier way. John M. Carroll’s 1990 book, subtitled Designing Minimalist Instruction for Practical Computer Skill, describes efforts to help people learn to use computers and software.  In 1981, Carroll and his colleagues analyzed problems that people had learning then-new technology like the IBM Displaywriter and the Apple Lisa.

In one extended experiment, Carroll and his colleagues had volunteers work with the Lisa, its owners guide, and the documentation for LisaProject.  The goal was to find out what interested but untrained users actually did with these materials.

Mostly what they did was struggle.

On average, the learners took three times the half hour estimated by Apple and enthusiastic trade journals–just to complete the online tutorial. “Two [learners] who routinely spent more than half of their work time using computers… failed to get to our LisaProject learning task at all.”

Carroll calls into question what he refers to as the systematic or systems approach to user training. To him this means “a fine-grained decomposition of target skills” used to derive an instructional sequence: you practice the simple stuff before you go on to more complex tasks they contribute to.

Carroll believes that “the systems approach to instructional design has nothing in common with general systems theory.” What’s worse is that in the workplace, the highly structured step-by-step approach just doesn’t work.

If only people would cooperate!  But they don’t.

The problem is not that people cannot follow simple steps; it is that they do not… People are situated in a world more real to them than a series of steps… People are always already trying things out, thinking things through, trying to relate what they already know to what is going on…

In a word, they are too busy learning to make much use of the instruction.

(that emphasis is Carroll’s, not mine — DF)

After further experiments, Carroll and his colleagues created what they called the Minimal Manual.  Earlier they’d made up a deck of large cards “intended to suggest goals and activities” for learners, and useful as quick-reference during self-chosen activity. In chapter 6 of The Nurnberg Funnel, he describes the next stage–a self-instruction manual designed on the same minimalist model.

Training on real tasks

The Minimal Manual used titles like “Typing Something” or “Printing Something on Paper” rather than suboptimal, system-centric ones in the original Displaywriter materials.  Carroll’s materials also eliminated material that was not task oriented–like the entire chapter entitled ”Using Display Information While Viewing a Document.”

At the same time, the experiment included essential material not well covered in the original document.  It was easy for learners to accidentally add blank lines but difficult for them to get rid of them.  The Minimal Manual turned this into a goal-focused task that made sense to the learner: “Deleting Blank Lines.” While not catchy, that title’s a big improvement on “how to remove a carrier return control character.”

Getting started fast

In the Minimal Manual the learner switches on the system and begins the hands-on portion of instruction after four pages of introduction.  In the systems-style instruction manual, hands-on training begins after 28 pages of instruction.

Learners created their first document only seven pages into the Minimal Manual…. In the commercial manual, the creation of a first document was delayed until page 70.

Carroll shows several ways in which the comprehensive systems-style manual bogs down, overloads the learner, and gets in the way of doing anything that seems like real work.  I can remember endless how-to-use-your-computer courses that spent 45 minutes on file structure and hierarchy before the target audience had ever created a document that needed to be saved.  This is like studying the house numbering scheme for a city before learning how to get to your new job.

Reasoning and improvising

The Minimal Manual approach included “On Your Own” work projects–for example, make up a document and compose the text yourself.  Then try inserting, deleting, and replacing text.

Some explanation is always necessary, but the minimalist approach kept that to… a minimum.  ”The Displaywriter stores blank lines as carrier return characters.”  That’s it.  You don’t really have to know what a carrier return character is–what’s important to you as a user is (a) it’s what creates blank lines, and (b) if you delete it, you delete the blank line.

In general, this approach introduced a procedure only once.  The three-page chapter “Printing Something on Paper” was the only place that printing was explained.  Elsewhere, exercises simply told the learner to print.  If he wasn’t sure how, he’d have to go back to that chapter.

In part, the team chose this approach because of the endless and often fruitless searching that learners had done in earlier trials, losing themselves in thickets of manuals and documents.  The fewer pages you have and the clearer their titles, the easier it is to find what you’re looking for.

Here’s the entire explanation for the cursor control keys:

Moving the cursor

The four cursor-movement keys have arrows on them (they are located on the right of the keyboard).

Press the ↓ cursor key several times and watch the cursor move down the screen.

The ↑, ←, and → keys work analogously.  Try them and see.

If you move the cursor all the way to the bottom of the screen, or all the way to the right, the display “shifts” so that you can see more of your document.  By moving the cursor all the way up and to the left, you can bring the document back to where it started.

Connecting the training to the system

Carroll’s subhead here is actually “Coordinating System and Training,” but I wanted to be more direct.  His team deliberately used indirect references in order to encourage learners to pay attention to the system they were learning.  In those long-ago days, for example, computers had two floppy-disk drives.  The Minimal Manual didn’t tell learners which drive to put a diskette in.  ”We left it to the learner to consult the system prompts.”

Supporting error recognition and recovery

As with other parts of the experiment, Carroll and his colleagues used error information from previous testing to guide the support provided by the Minimal Manual.  Multi-key combinations (hold down one key while pressing another) baffled many learners, especially when the labels on the keys were meaningless to them: (“press BKSP, then CODE + CANCL”).  And then there was this:

A complication of the Code coordination error is that the recovery for pressing Cancel without holding the Code key is pressing Cancel while holding the Code key.

Good thing we never see anything like that any more, huh?

Exploiting prior knowledge

It’s easy to forget how confusing word processing can be–at least till you try learning some new application for which you have very little background.  (I’ve taken a stab at learning JavaScript, and I can see that’s probably not the basis of my next career.)  The Minimal Manual strove to counter the relentless, technocratic, system-centric thinking in the original.  ”The impersonal term ‘the system’ was replaced by the proper name…the Displaywriter.”

I can hear IT people I’ve worked with sniffing “so what?”  I’ve actually had a programmer say to me, of a useful but very complicated tool, “If they can’t understand this, they don’t deserve it.”

One particularly useful approach: document names.  Back when most white-collar work did not involve computers, people created paper documents all the time, but rarely thought of documents as requiring a name.  (What’s the name of a letter?  What’s the name of a memo?) So the bland instruction “Name your document” seems like one more small technical obstacle in the way of getting something useful done.

Carroll’s team had learned that naming created lots of problems for learners, and so found a way to ease learning of this unfamiliar concept.

In the terminology of the Displaywriter you will be “creating a document” — that is, typing a brief letter.  You will first name the document, as you might make up a name for a baby before it is actually born.  Then you will assign the document to a work diskette — this is where the document will be stored by the Displaywriter.  And then, finally, you will type the document at the keyboard, and see the text appear on the screen.

It might still feel odd to have to name a document, but the baby analogy brings the idea a bit closer to what the average person already knows.

  ♦  ♦  ♦

There’s a great deal more in chapter 6 that I’ll have to return to in another post.  I wanted to share what’s here, though, because I think it’s extremely relevant to the future of learning at work.

That omnipresent quotation from a movie puppet often exasperates me.

Of course there’s try–in fact, it’s the effort involve in genuinely trying that’s essential.  Otherwise, no Jedi training and not much need for a master; Yoda could just take a seat behind Statler and Waldorf.

Trying and succeeding leads to conclusions that may or may not be correct–sometimes they’re simplistic, sometimes they’re downright erroneous.  Trying and falling short, in an environment where such trying is encouraged, can lead to analysis, to greater awareness of the available steps, inputs, and tools, and to improved performance.

The bigger lesson, I am more and more convinced, is that comprehensive systems training is a myth.  People might spend extended time in formal classes, or labor their way through highly structured text or tutorials, but most of the time they’re looking for how to accomplish something that seems valuable to them.  Just tell me how to get these images posted.  Let me create a series of blog posts that have automatic navigation.  How can I search this mass of data to find things that are X, Y, and Z, but not Q?

As I put it in a different context (vendor-managed inventory), I don’t want to know about standard deviation.  I want to know whether the grocery warehouse computer’s going to order more mayonnaise–and how to tell it not to, if that’s what I think is best.

In no way am I saying that analysis doesn’t matter.  It matters a lot–witness the skillful observation and analysis of user testing that led Carroll and his associates to the Minimal Manual.  That for them was a starting point–they examined data from their testing to gain further insight and to guide decisions about supporting learning.

First he talks about the world of neuroscience and perception, into which he’s often invited as a speaker.  And he makes the point that when it comes to experimenting with human perception, neuroscientists are amateurs compared with magicians.

I recall his partner Penn Gillette saying once that they were not magicians.  They were tricksters, swindlers.  His point was that nothing in their act was magical.  They’re not exempt from the laws of physics. Instead, as magicians have done for thousands of years, they rely on trickery, on quirks of perceptions.

It’s well worth reading the original (link in the first paragraph, above) to enjoy Teller’s style and to take in the details he provides for points like these:

• Exploit pattern recognition.  Our brains constantly seek patterns, especially when there isn’t one.  That’s why the night sky has constellations, but an evenly spaced series of dots seems to have no pattern at all.
• Distract with laughter. What Teller’s really talking about here is a kind of cognitive overload–if you’re watching the performance and laughing at the comedy, you’re likelier to miss some small detail.  I think the same thing applies when a training exercise is sufficiently engrossing–people don’t care as much about elegant presentation and high-end graphics if the exercises feels like interesting, useful work.
• Nothing fools you better than the lie you tell yourself.  Here, he’s talking about allowing the audience (or the learner) to reach their own conclusions, make their own judgments, even if as the “designer” he knows these will be erroneous.  For a magic act, that means the audience is all the more mystified by the effect–thus, success.  When it comes to learning, the learner is comparing a conclusion she arrived at with new data that conflicts with that conclusion.  That, gentle reader, is where the learning starts.

He goes on; you don’t need me to repeat it here.  I found the article engaging enough that I wanted to see more, and came across a 2008 article in Nature Reviews – Neuroscience.   In Attention and awareness in stage magic: turning tricks into research, Teller and several coauthors study magic tricks so that “neuroscientists can learn powerful methods to manipulate attention and awareness in the lab.”

If you’re doubtful, take a look at this demonstration by one of the coauthors, pickpocket Apollo Robbins.

I think it’s worth the 16 minutes.  Watch carefully during the first two-thirds, when (I’m not giving away much here) Robbins actually picks the pockets of a volunteer who’s pretty sure that’s what’s going to happen.  You’ll find the subsequent explanation all the more compelling.

“If I’m here (standing alongside the mark), and I want to split his attention… I’ll bring my chin up into his personal space. His head will whip up to my face, and he won’t focus on that movement (of my hands).”

• Is a job aid mandatory? If not, does speed or rate on the job prohibit the use of a a job aid?
• Do the characteristics of the task tell you that a job aid makes sense?

If they do, you might feel ready to leap right into design.  But in the real world, people don’t just perform a task; they work within a complex environment.  So the third part of your decision is to ask if any obstacles in that environment will hamper the use of a job aid.

You could ask these question in either order, but physical barriers are sometimes easier to address than social ones.

Often people have to work in settings where a job aid might be a hindrance or even a danger.  Someone repairing high-tension electrical lines, for example.  Or someone assembling or disassembling freight trains at a classification yard:

You don’t need to watch this video about humping railroad cars, but as the narrator points out around the 4:00 mark, in the distant past a worker would have to ride each car as gravity moved it down a manmade hill (the hump), applying the brake by hand if the car was moving faster than about 4 mph. It would have been impossible to give the brakeman a job aid for slowing the car, so his training (formal or otherwise) would have required lots of practice and feedback about judging speed.  And possible trial and error.

Texas highway map, 1936

Rather than develop impractical job aids for aspects of this set of tasks, modern railroads rely on computers to perform many of them.  For example, radar monitors the speed of cars more accurately than a person could, and trackside retarders act to moderate that speed.

Remember, the goal is not to use job aids; the goal is to produce better on-the-job results.  Sometimes you can do that by assigning difficult or repetitive tasks to machinery and automation.

In many cases, though, you can overcome physical obstacles to the use of a job aid  by changing its form.  No law requires a job aid to be on an 8 1/2 by 11 inch laminated piece of paper. Nor on the formerly ubiquitous, multifolded paper of a highway map.

A road map can support different kinds of tasks.  You can use it at a table to plan where you’re going to go, to learn about the routes.  No barriers to such use.  But for a person who’s driving alone, a paper road map is at best a sub-optimal support.  It’s hard to use the map while trying to drive through an unfamiliar area.

Deep in the heart of Oslo

Real-time support for the driver now includes geosynchronous satellites, wireless technology, a constantly updated computer display–and a voice.

That voice is transformative: it’s a job aid you don’t have to read. Because the GPS gives timely, audible directions, there’s no need to take your eyes off the road and decipher the screen.

Other examples of overcoming physical barriers: attach the job aid to equipment. Use visual cues, like a change of color as movement or adjustment gets closer to specification.  Combine audio with voice-response technology (“If the relay is intact, say ‘okay.’ If the relay is damaged, say ‘damaged.’”)

But he had to look it up!

Overcoming physical barriers is one thing.  Overcoming social barriers is…a whole bunch of things. Your job aid will fail if the intended performer won’t use it.

Popular culture places a great value on appearing to know things.  When someone turns to an external reference, we sometimes have an irrational feeling that she doesn’t know what she’s doing–and that she should.  In part, I think we’re mistaking retention of isolated facts with deep knowledge, and we think (reasonably enough) that deep knowledge is good.

At its worst, though, this becomes the workplace equivalent of Trivial Pursuit. A railroading example might be someone who can tell you not only the train numbers but the locomotive numbers that ran on a certain line decades ago–but who can’t issue you a ticket in a prompt, accurate, courteous manner.

The performer herself may be the person believing that performance guided by a job aid is somehow inferior.  Coworkers may hold it, putting pressure on the individual.  Even clients or other stakeholders may prefer not to see the performer using a job aid.

Maybe there’s a way around this bias.  The job aid could be embedded in a tool or application, such that the performer is merely applying one feature.  That’s essentially what a software wizard does.  Watch me turn this data into a chart–I just choose what I want as I go along.

(And doesn’t “choose what I want” sound much more on top of things than “look stuff up?”)

For a injection gun used for immunizations in third-world settings, healthcare workers occasionally had to make adjustments to clear jams and similar equipment glitches.  Some senior workers did not want to seem to need outside help to maintain their equipment, but couldn’t retain all the steps.  (Remember in Part 2?  Number of steps in task, complexity of steps?)  So the clearing instructions were attached to the equipment in such a way that the worker could follow the job aid while clearing the gun.

♦ ♦ ♦

The considerations here aren’t meant as either exhaustive or exclusive.  They are, however, important stops to make, a kind of reality check before you hit the on-ramp to job aid design.  The reason for building a job aid is to guide performance on the job while reducing the need for memorization, in order to achieve a worthwhile result.  If the performer can’t use it because of physical obstacles, or won’t use it because of social ones, the result will be… no result.

CC-licensed photos:
1936 Texas highway map by Justin Cozart.
Norwegian GPS by Stig Andersen.
1879 Michigan Central RR timetable from the David Rumsey Map Collection.

(LSS Financial Counseling Service is part of the work of Lutheran Social Service of Minnesota.)

Who uses this job aid?

Most likely someone trying to learn the added financial burden of buying on credit.  (See additional thoughts from the group that created it, later in this post.)

What is the task supported?

I would say “awareness” or even “empowerment.”  The goal is to help someone understand the additional cost of purchasing on credit.  I filled in the numbers you see in this example.  The result says to me that “spreading out” credit payments for my holiday buying makes those purchases nearly 10% more expensive than I’d thought.

Notice that it doesn’t render judgment (“$68.72 extra?  Are you nuts?!?”).  The job aid simplifies the process so I can more readily see and understand the impact of buying on credit.  I’m free to make my own decisions about what to do next.

More about the Scrooge-O-Meter

LSS Financial Counseling Service wants consumers to know that they can turn to a national network of nonprofit financial counseling and debt management (FCS is a member of that network).  The page with the Scrooge-O-Meter offers a toll-free number, online counseling, a newsletter, and other resources.

Darryl Dahlheimer, program director of LSS Financial Counseling Service, was kind enough agree to its appearing here and also to provide these details:

There are many tools to help consumers calculate credit card repayment, but here are three reasons we like this one:

1. It sets a playful tone, to overcome the shame/intimidation of finances for so many who feel “dumb about money” but want to learn.
2. It helps make the true cost of using credit visible.  Plug in an example of buying that $500 iPad at a major store on their 21% interest credit card and then paying only the $15 minimum each month. You will pay a whopping $757 and take over four years to pay off.
3. Conversely, it allows you to see the tangible benefits of paying more than minimums.