Learning

What would you do with a brain if you had one? (Dorothy Gale, to the Scarecrow)

Jan 312014
 

I recently came across a link to this infographic by Julian Hansen.

Infographic by Julian Hansen

I don’t see most infographics as a job aid. They usually aren’t intended to guide you through a task, and don’t usually serve well as reference job aids (my term for information that’s been organized for quick reference). I don’t think this would serve as a true job aid for most non-designers–it’s really busy, and the criss-crossing paths could easily confuse someone.

As this Fontfeed article states, though, that wasn’t really Hansen’s goal.

 Instead of simply browsing through type specimens, Julian wondered if he could come up with something more rational, a systematic approach [to choosing typefaces]. His project took the form of a flowchart on a poster. Studying different type finders made him come to the conclusion that selecting type really could be a matter of taste…. This made Julian decide that his poster should not only be useful, but also be light-hearted and make fun of stereotypes. This made him throw in options like “is it an Italian restaurant?” for instance. His ultimate goal was to show that typefaces convey a whole lot of meaning that “ordinary” people just don’t see.

Assuming that’s true, I see the chart as one way to demonstrate understanding: here’s what I think about fonts and when to use them. This is part of what I think Jane Bozarth means when she says, “We learn by doing, and by telling what we’re doing, and by watching others do things, and by showing others how we did something.”

Personally, I’m not much info fonts.

That’s not the point, though. Work like Hansen’s has the potential to trigger further interest in people.  For example, after reading his chart and the Fontfeed article, I happened to see a tweet by @MizMinh linking to an article on The Next Web:

The Science Behind Fonts (and How They Make You Feel)

Personally, all my working out loud lately has been done on site, in my new job. I’m not unhappy about that; I’m working on an engaging project and I have collaborative colleagues. But I’ve been neglecting other avenues, and this post is one effort to overcome that neglect.

Feb 182013
 

In the Learning Creative Learning online course, one suggested activity this week was to read Gears of My Childhood, Seymour Papert’s essay on how playing with gears as a very young child has influenced his life, and to share with others in the course a similar reflection based on your own experience.

I’ve enjoyed reading many of these. People talk about skateboards, about a box of dress-up clothes, about a “typewriter” with 12 keys (constructed from an egg carton, a paper-towel tube, and similar highly engineered materials).

One woman wrote about a box of watercolor paints her mother got for her:

…which she said were the best watercolors on the market at that time. I felt so professional! I made many paintings with them, including huge ones… The little watercolor pans are incredibly visually appealing to me and have a particular paint smell that I still find irresistible. I love the case, the way it snaps, the way the brushes fit elegantly in the isle between the rows of pans, and the way the palette comes out and attaches to the box to create huge mixing space.

She captured me with that snap. To me the word, the sound perfectly captures a way in which childhood memories are stored so deeply. We’re attending (without necessarily focusing deliberately) on so many parts of the experience and interpreting them in ways that make sense to us.

So the snap of the box is a central part of how she remembers and relives her paintbox experience. She is now a teacher of visual and media arts. In her comments, she says:

I recommend that my students go touch all the sketchbooks in the art store and buy the one that feels the best to hold. For many it helps establish a different relationship with the work and be a lot more productive. I think this concept also applies to the physical spaces in which we live and work.

Immediately I thought of an artifact from long ago — a repair manual I bought in college to help maintain my 1963 VW Beetle. I’ve written before about How to Keep Your Volkswagen Alive as an outstanding job aid.

The words about touch, though, reminded me of chapter 3, “How to Buy a Volkswagen.” The chapter is 10 pages long, including an 18-step “pre-purchase procedure” that starts by telling you what tools to bring along.

It’s crammed with practical information intended to help the novice make a better decision about a used car:

[Start the car, and with the engine idling]…put your hands over the tail pipes, quickly because they’ll soon be hot, and feel the pressure. Feel the pulses; they should be even…

Then hold your hands about four to five inches away, letting the exhaust pass over them. The pulses should be even and about the same temperature   If they are not, the engine needs or will soon need a valve job.

 Prior to that pre-purchase procedure, Muir has advice on things to do before you even put the key in the ignition. These are paraphrases:

  • Walk around and look at the car. Does it sag and look beat? Do the doors open and close well?
  • Put your foot on the brake; it should stop three inches or more from the floor.
  • Push the clutch pedal with your hand till it’s hard to push. Let it up and see how much free play there is. More than two inches: the clutch is suspect.

He goes on with a short paragraph about the upholstery (as an indicator of overall treatment), the engine (it’s air cooled – dirt is a bad sign), play in the front wheel.

And then:

Now sit back and look at it again. Does it stand up with pride? Does it feel good to you? Would you like to be its friend? Use your other senses. Sit in the driver’s seat and scrunch your butt around. Hold the wheel and close your eyes and FEEL!

…Get away from the car and the owner or salesman to let your mind and feelings go over the car and the idea of the car. What has its karma been? can you live with the car? Walk around or find a quiet place, assume the good old lotus and let the car be the thing. At this point some revelation will come to you and you will either be gently guided away from that scene…

It is important that you neither run the motor or ride in the car until this preliminary scene has run its course. It also puts the owner-salesman up the wall because he has no idea of what you are doing and will be more pliable when the hard dealing time comes.

I was never quite that touchy-feely, not even when I bought my original copy of this guide from the Whole Earth Catalog back in 1968 or so. But I think Muir did a great job of situating the pragmatic, procedural parts of VW ownership and maintenance within the context of the reader situating the car into his life.

Mar 092012
 

(This is a continuation of a previous post based on John M. Carroll’s The Nurnberg Funnel)

The main elements in the Minimal Manual test–a task-centric approach to training people in using computer software–were lean documentation, guided exploration, and realistic exercises. So the first document that learners created was a letter. In earlier, off-the-shelf training, the first task had been typing a description of word processing, “something unlikely to be typed at work except by a document processing training designer.”

You call this training?This sort of meta-exercise is very common, and I think almost always counterproductive. Just as with Amtrak’s training trains that (as I said here) didn’t go over real routes, trivial tasks distract, frustrate, or confuse learners. They don’t take you anyplace you wanted to go.

Not that the practice exercise needs to look exactly like what someone does at his so-called real job; the task simply needs to be believable in terms of the work that someone wants to get done.

Into the pool

After creating the Minimal Manual, Carroll’s team created the Typing Pool test.  They hired participants from a temp agency and put them in a simulated office environment, complete with partitions, ringing phones, and office equipment. These people were experienced office workers with little prior computer knowledge. (Remember, this was in the 1980s; computer skills were comparatively rare. And Carroll was testing ways to train people to use computer applications.)

Tasks for the Typing Pool test

(Click to enlarge.)

Each group of two or three participants was given either the Minimal Manual (MM) or the systems style instruction manual (SM). Participants read and follow the training exercises in their manuals and periodically received performance tasks, each related to particular training topics. (You can see the task list by enlarging the image on the right.)

Some topics were beyond the scope of either the MM or the SM; interested participants could use an additional self instruction manual or any document in the system reference library.

After finishing the required portion of training material, participants took the relevant performance test. They were allowed to use any of the training material, the reference library.  They could even call a simulated help line. This last resource had an expert on the system who was familiar with the help line concept but unaware of the goals of the study.

So what happened?  Carroll provides a great deal of detail; I’ll summarize what seem to me to be the most important points.

Minimal learning was faster learning.

In all, the MM participants used 40% less learning time then the SM participants — 10 hours versus 16.4. (“Learning time” refers to time spent with either the MM or SM materials, not including time spent on the performance tasks.) This was true both for the basic tasks (1 through 3 on the list) and the advanced wants.

In addition, the MM group completed 2.7 times as many subtasks, as the SM group. One reason was that some SM participants ran out of time and were unable to try some of the advanced tasks. Even for those tasks that both groups completed, the MM group outperformed by 50%.

We were particularly satisfied with the result that the MM learners continued to outperform their SM counterparts for relatively advanced topics that both groups studied in the common manual. This indicates that MM is not merely Wiccan dirty for getting started… Rather, we find MM better then SM in every significant sense and with no apparent trade-offs. The Minimal Manual seem to help participants learn how to learn.

In the second study, more analytical while more limited in scope, similar results were found. In this study, Carol’s group also compared learning by the book (LBB) with learning by doing (LWD).  The LBB group were given training manuals and assigned portions to work with. After a set period of learning, they were given performance tasks. This cycle was repeated three times. The LWD learners received the first task at the start of the experiment, as they completed each task, they received the next one. There was also an SM by-the-book group and an SM learn-by-doing group.

So there are two ways to look at the study: MM versus SM as with previous study, and LWD versus LBB for each of those formats. To make that clear, both sets of LWD learners received at the start both the training materials and the relevant performance test to complete; both sets of LBB learners had a fixed amount of time to work with the training materials (which included practice) before receiving the performance tests.

Among the things that happened:

  • MM learners completed 58% more subtasks than SM learners did.
  • LWD learners completed 52% more subtasks than LBB learners did.
  • MM learners were twice as fast to start the system up as SM learners.
  • MM learners made fewer errors overall, and tended to recover from them faster.

Mistakes were made.

One outcome was the sort of thing that makes management unhappy and training departments uneasy: the average participant made a lot of errors and spent a lot of time dealing with them.  Carroll and his colleagues observed 6,885 errors and classified them into 40 categories.)

Five error types seemed particularly important–along the accounted for over 46 percent of the errors; all were at least 50 percent more frequent than the sixth most frequent error…

The first three of these were errors that the MM design specifically targeted.  They were imprtant errors: learners spent an average of 36 minutes recovering from the direct consequences of these three errors, or 25 percent of the average total amount of error recovery time [which was 145 minutes or nearly half the total time].

The MM learners man significantly fewer errors for each of the top three categories–in some cases nearly 50% less often.

This to me is an intriguing, tricky finding. A high rate of errors that includes persistence and success can indicate learning, though I wonder whether the participants found this frustrating or simply an unusual way to learn. I’m imagining variables like time between error and resolution, or number of tries before success. Do I as a learner feel like I’m making progress, or do I feel as though I can’t make any headway?

The LWD participants (both those on MM and on SM) had a higher rate for completing tasks and a higher overall comprehension test score than their by-the-book counterparts. So perhaps there’s evidence for the sense of progress.

Was that so hard?

Following the trial, Carroll’s team asked the participants to imagine a 10-week course in office skills.  How long would they allow for learning to use the word processing system that they’d been working with.  The SM people thought it would need 50% of that time; the MM people, 20%.

Slicing these subjective opinions differently, the LBB (learn-by-book) group estimated less time than the LWD (learn-while-doing) group. In fact, LBB/MM estimated 80 hours while LWD/MM estimated 165.

What this seems to say is that in general the MM seemed to help people feel that word processing would be easier to learn compared with SM, but also that LWD would require more time than LBB.

♦  ♦  ♦

The post you’re reading and its predecessor are based on a single chapter in The Nurnberg Funnel–and not the entire chapter.  Subsequent work he discusses supports the main design choices:

  • Present real tasks that learners already understand and are motivated to work on.
  • Get them started on those tasks quickly.
  • Encourage them to rely on their own reasoning and improvisation.
  • Reduce “the instructional verbiage they must passively read.”
  • Facilitate “coordination of attention” — working back and forth between the system and the training materials.
  • Organize materials to support skipping around.

I can see–in fact, I have seen–groups of people who’d resist this approach to learning.  And I don’t only mean stodgy training departments; sometimes the participants in training have a very clear picture of what “training” looks like, what “learning” feels like, and spending half their time making errors doesn’t fit easily into those pictures.

That’s an issue for organizations to address–focusing on what it really means to learn in the context of work.  And it’s an issue for those whose responsibilities include supporting that learning. Instructional designers, subject-matter experts, and their clients aren’t always eager to admit that explanation-laden, application-thin sheep-dip is ineffective and even counterproductive.

CC-licensed image: toy train photo by Ryan Ruppe.

Mar 022012
 

Are we having funnel yet?The Nuremberg Funnel, according to Wikipedia, is a humorous expression for a kind of teaching and learning.  It implies knowledge simply flowing effortlessly into your brain as you encounter it–or else a teacher cramming stuff in the mind of a dullard.

(The term dates to at least 15th-century Germany, and I suspect the notion of funneling or otherwise stuffing knowledge into someone is a few months older than that.)

The Nurnberg Funnel is humorous as well, in a slightly drier way. John M. Carroll’s 1990 book, subtitled Designing Minimalist Instruction for Practical Computer Skill, describes efforts to help people learn to use computers and software.  In 1981, Carroll and his colleagues analyzed problems that people had learning then-new technology like the IBM Displaywriter and the Apple Lisa.

Minimal sense

In one extended experiment, Carroll and his colleagues had volunteers work with the Lisa, its owners guide, and the documentation for LisaProject.  The goal was to find out what interested but untrained users actually did with these materials.

Mostly what they did was struggle.

On average, the learners took three times the half hour estimated by Apple and enthusiastic trade journals–just to complete the online tutorial. “Two [learners] who routinely spent more than half of their work time using computers… failed to get to our LisaProject learning task at all.”

Carroll calls into question what he refers to as the systematic or systems approach to user training. To him this means “a fine-grained decomposition of target skills” used to derive an instructional sequence: you practice the simple stuff before you go on to more complex tasks they contribute to.

Carroll believes that “the systems approach to instructional design has nothing in common with general systems theory.” What’s worse is that in the workplace, the highly structured step-by-step approach just doesn’t work.

If only people would cooperate!  But they don’t.

The problem is not that people cannot follow simple steps; it is that they do not… People are situated in a world more real to them than a series of steps… People are always already trying things out, thinking things through, trying to relate what they already know to what is going on…

In a word, they are too busy learning to make much use of the instruction.

(that emphasis is Carroll’s, not mine — DF)

After further experiments, Carroll and his colleagues created what they called the Minimal Manual.  Earlier they’d made up a deck of large cards “intended to suggest goals and activities” for learners, and useful as quick-reference during self-chosen activity. In chapter 6 of The Nurnberg Funnel, he describes the next stage–a self-instruction manual designed on the same minimalist model.

Training on real tasks

The Minimal Manual used titles like “Typing Something” or “Printing Something on Paper” rather than suboptimal, system-centric ones in the original Displaywriter materials.  Carroll’s materials also eliminated material that was not task oriented–like the entire chapter entitled “Using Display Information While Viewing a Document.”

At the same time, the experiment included essential material not well covered in the original document.  It was easy for learners to accidentally add blank lines but difficult for them to get rid of them.  The Minimal Manual turned this into a goal-focused task that made sense to the learner: “Deleting Blank Lines.” While not catchy, that title’s a big improvement on “how to remove a carrier return control character.”

Getting started fast

In the Minimal Manual the learner switches on the system and begins the hands-on portion of instruction after four pages of introduction.  In the systems-style instruction manual, hands-on training begins after 28 pages of instruction.

Learners created their first document only seven pages into the Minimal Manual…. In the commercial manual, the creation of a first document was delayed until page 70.

Carroll shows several ways in which the comprehensive systems-style manual bogs down, overloads the learner, and gets in the way of doing anything that seems like real work.  I can remember endless how-to-use-your-computer courses that spent 45 minutes on file structure and hierarchy before the target audience had ever created a document that needed to be saved.  This is like studying the house numbering scheme for a city before learning how to get to your new job.

Reasoning and improvising

The Minimal Manual approach included “On Your Own” work projects–for example, make up a document and compose the text yourself.  Then try inserting, deleting, and replacing text.

Some explanation is always necessary, but the minimalist approach kept that to… a minimum.  “The Displaywriter stores blank lines as carrier return characters.”  That’s it.  You don’t really have to know what a carrier return character is–what’s important to you as a user is (a) it’s what creates blank lines, and (b) if you delete it, you delete the blank line.

In general, this approach introduced a procedure only once.  The three-page chapter “Printing Something on Paper” was the only place that printing was explained.  Elsewhere, exercises simply told the learner to print.  If he wasn’t sure how, he’d have to go back to that chapter.

In part, the team chose this approach because of the endless and often fruitless searching that learners had done in earlier trials, losing themselves in thickets of manuals and documents.  The fewer pages you have and the clearer their titles, the easier it is to find what you’re looking for.

Here’s the entire explanation for the cursor control keys:

Moving the cursor

The four cursor-movement keys have arrows on them (they are located on the right of the keyboard).

Press the ↓ cursor key several times and watch the cursor move down the screen.

The ↑, ←, and → keys work analogously.  Try them and see.

If you move the cursor all the way to the bottom of the screen, or all the way to the right, the display “shifts” so that you can see more of your document.  By moving the cursor all the way up and to the left, you can bring the document back to where it started.

Connecting the training to the system

Carroll’s subhead here is actually “Coordinating System and Training,” but I wanted to be more direct.  His team deliberately used indirect references in order to encourage learners to pay attention to the system they were learning.  In those long-ago days, for example, computers had two floppy-disk drives.  The Minimal Manual didn’t tell learners which drive to put a diskette in.  “We left it to the learner to consult the system prompts.”

Supporting error recognition and recovery

As with other parts of the experiment, Carroll and his colleagues used error information from previous testing to guide the support provided by the Minimal Manual.  Multi-key combinations (hold down one key while pressing another) baffled many learners, especially when the labels on the keys were meaningless to them: (“press BKSP, then CODE + CANCL”).  And then there was this:

A complication of the Code coordination error is that the recovery for pressing Cancel without holding the Code key is pressing Cancel while holding the Code key.

Good thing we never see anything like that any more, huh?

Exploiting prior knowledge

It’s easy to forget how confusing word processing can be–at least till you try learning some new application for which you have very little background.  (I’ve taken a stab at learning JavaScript, and I can see that’s probably not the basis of my next career.)  The Minimal Manual strove to counter the relentless, technocratic, system-centric thinking in the original.  “The impersonal term ‘the system’ was replaced by the proper name…the Displaywriter.”

I can hear IT people I’ve worked with sniffing “so what?”  I’ve actually had a programmer say to me, of a useful but very complicated tool, “If they can’t understand this, they don’t deserve it.”

One particularly useful approach: document names.  Back when most white-collar work did not involve computers, people created paper documents all the time, but rarely thought of documents as requiring a name.  (What’s the name of a letter?  What’s the name of a memo?) So the bland instruction “Name your document” seems like one more small technical obstacle in the way of getting something useful done.

Carroll’s team had learned that naming created lots of problems for learners, and so found a way to ease learning of this unfamiliar concept.

In the terminology of the Displaywriter you will be “creating a document” — that is, typing a brief letter.  You will first name the document, as you might make up a name for a baby before it is actually born.  Then you will assign the document to a work diskette — this is where the document will be stored by the Displaywriter.  And then, finally, you will type the document at the keyboard, and see the text appear on the screen.

It might still feel odd to have to name a document, but the baby analogy brings the idea a bit closer to what the average person already knows.

  ♦  ♦  ♦

There’s a great deal more in chapter 6 that I’ll have to return to in another post.  I wanted to share what’s here, though, because I think it’s extremely relevant to the future of learning at work.

That omnipresent quotation from a movie puppet often exasperates me.

Of course there’s try–in fact, it’s the effort involve in genuinely trying that’s essential.  Otherwise, no Jedi training and not much need for a master; Yoda could just take a seat behind Statler and Waldorf.

Trying and succeeding leads to conclusions that may or may not be correct–sometimes they’re simplistic, sometimes they’re downright erroneous.  Trying and falling short, in an environment where such trying is encouraged, can lead to analysis, to greater awareness of the available steps, inputs, and tools, and to improved performance.

The bigger lesson, I am more and more convinced, is that comprehensive systems training is a myth.  People might spend extended time in formal classes, or labor their way through highly structured text or tutorials, but most of the time they’re looking for how to accomplish something that seems valuable to them.  Just tell me how to get these images posted.  Let me create a series of blog posts that have automatic navigation.  How can I search this mass of data to find things that are X, Y, and Z, but not Q?

As I put it in a different context (vendor-managed inventory), I don’t want to know about standard deviation.  I want to know whether the grocery warehouse computer’s going to order more mayonnaise–and how to tell it not to, if that’s what I think is best.

In no way am I saying that analysis doesn’t matter.  It matters a lot–witness the skillful observation and analysis of user testing that led Carroll and his associates to the Minimal Manual.  That for them was a starting point–they examined data from their testing to gain further insight and to guide decisions about supporting learning.

(I wrote a follow-up to this post:
Minimal Training: a Plunge into the Typing Pool

Jan 042012
 

When I read about the Organize Series plugin for WordPress (a focus of Monday’s post), I thought, “This could do it.”

No I didn’t.  I don’t know about you, but I rarely think to myself in complete sentences.  Phrasing like this is how we capsulize a more complex experience.  What I believe was going on at the time was something like this: I had a situation I wanted to change (the way I used to manage a series of posts here on my blog no longer worked). And the Organize Series plugin at first glance looked like it could accomplish at least two things:

  • Provide automatic navigation between posts in a series (so I wouldn’t have to hard-wire the links).
  • Display a list of all the posts in a given series (for me to use as a summary or as a table of contents for the series).

If I’d thought about it longer, I might have articulated another goal: have some way to list all the different series I have.  But I’m not usually that strategic.  Still, what I came up with (provide navigation, display a list) acted as my critical-to-quality elements.  CTQs were widely used at GE when I worked there; I use that acronym partly tongue-in-cheek and partly to highlight informal criteria.

So, I put Organize Series to work, and within 10 minutes I had automatic next/previous navigation for posts in a series, along with an indication that they were part of a series:

No, this isn't the entire post.

(You can click the image to see the entire post.)

When I was still considering whether to use the plugin, I said to my wife, “Wouldn’t it be great to know how to write a plugin?”  On reflection, I realize this statement was another capsulization–a series of them, nested inside each other.  “Know how to write a plugin” really means:

  • “Know how to write a plugin” really means “write a plugin that works….”
  • Which in turn means “write one that produces results…”
  • Which means “write one that people use to accomplish things that matter to them.”

To me, this is an important distinction for workplace learning: You can learn on your own for your personal satisfaction, and if you’re satisfied, then that’s a sufficient result.  In the workplace, though, you’re part of a larger group (even if that group is you and one individual client), and so the result has to matter within that context.

What’s this got to do with my plugin tinkering?

Think of it as my own workplace learning.  At this point, I was still some distance from my (loosely articulated) end state.  I hadn’t moved much toward my other CTQ of displaying a list of all the posts in a series. In fact, I didn’t yet grasp all the options in the plugin, let alone know how to make them work in a way useful to me.

I only put this here to scare you a little.

About 5% of the info from the plugin's page of options

But…In my first 15 minutes with the plugin, I’d achieved a result that I found valuable.  That left me more willing to experiment–which, put another way, says I was somewhat more willing to spend time trying to achieve the next valuable result.

To me, this is a core principle for any type of workplace learning: formal or informal, face-to-face or virtual.  I need to be able to accomplish something that looks to me like real work–produce something that I see has having on-the-job value.  And I need to do that sooner rather than later, which is why twenty minutes on introductions, half an hour on expectations for this workshop, and twenty minutes on learning objectives will invariably drive me to teeth-clenching frustration. Or to eating more of those lowest-bid-hotel pastries.

One of the unexpected outcomes of achieving an initial on-the-job goal is that you end up better able to visualize other goals.  In a sense, learning leads to new problems (or opportunites) because you’re better at grasping the current situation and at visualizing different ones.

In the course of my experimenting with the Organize Series plugin, I did find at least one way to display a list of all the posts in a series.  I can make a box like this appear alongside the title for each post:

Example of a 'series post list box' - a box listing posts in a series

The posts in my most complex series

You can click that image if you’d like to see the first post in the series, though I’ve turned this “series post list box” feature off for now, until I learn how to control the way it displays.  Having managed to produce it, though, I’ve picked up several more goals for myself.  I was about to write “learning goals,” but I want to stress that they’re all tied to accomplishment.

  • I want to learn how to use code that’s part of the plugin to, for example, display a list of posts like the last example where and when I want it.
  • I want to find out how to modify the plugin’s template (the tool it uses to display the full text of all the posts in a series).
  • I may even want to learn how to modify the PHP or CSS code to make things happen.

That last is quite a goal for someone who doesn’t really know how to program.  But my various experiments to date, and especially the things I see as successes, have taught me that I can learn to successfully modify small bits of PHP code and achieve relatively high-value results.

So I’m accomplishing what looks like real work to me.