If you follow this blog, you haven’t had much to follow lately.  I’m still grappling with site-hacking problems.  Their persistence leaves me frustrated and far, far from gruntled.

Two busy, productive couple of days working with a great client have re-energized me, so I’m back in business.  And, as a bonus, we’re halfway through the Ten Steps.

Step 5 in Ten Steps to Complex Learning is “analyze cognitive structures.”  As the diagram depicts, this step falls under the “supportive information” component of the overall learning blueprint.

Van Merriënboer and Kirschner point out that when someone encounters a task that’s different from what he’s seen before, procedural knowledge isn’t enough.  He needs strategic knowledge–a way to figure out how to do what he haven’t done.

I’ve already talked about SAPs (strategic approaches to problem-solving).  If you’re designing learning, these may already exist in job descriptions, instructional material, and other stored knowledge.  If not, you need to identify and make such SAPs explicit, which explains the three sections of this chapter:

• Specify SAPs.
• Analyzing intuitive cognitive strategies.
• Using SAPs in design decisions.

(Time out for a quibble, and not about vM&K’s apparent determination to avoid parallel lists.  I really dislike their use of “intuitive” here.  What they mean, as you’ll see in a bit, are the homegrown strategics that newcomers bring with them to a new domain.  In other words, those existing strategies aren’t intuitive in the sense of ingenious;  they’re naive.)

SAP: a plan with a goal–and rules of thumb

A strategic approach to problem-solving has a goal, and likely subgoals as well, related to solving problems in a particular domain.  The SAP also has heuristics to help reach that goal.  In fact, those are key to SAPs.

…SAPs are always heuristic….though they may help a learner to solve problems in the task domain, their application does not guarantee the solution of the problem.

Although they are less powerful than algorithmic procedures, power is exchanged for flexibility, because SAPs may be helpful in many more different problem situations than a solution algorithm.

In identifying the cognitive strategies that go into SAPs, you will:

• Assemble material to develop problem-solving guidance (like process worksheets)
• Help refine classes of learning tasks (in other words, identify a sequence of simpler to more complex cognitive strategies)
• Start the development of a key part of the supportive information discussed under Step 4.

If you’ve read any of the other posts in this series, you know how vM&K will go about identifying these strategies.  They’ll interview and observe expert performers performing the actual tasks, starting with relatively simpler ones and progressing to more difficult ones.

Sometimes the problem-solving process is mostly linear or sequential.  The much-maligned ADDIE model for instructional design (analyze, design, develop, implement, evaluate) fits that category.  If you were working with expert designers, you could identify simpler and more complex goals in each of those stages, and from that develop simpler and more complex SAPs.

vM&K use “solving thermodynamics problems” as an example of a non-linear set of phases to use in a particular domain.  Another example might be the CRAP principles for designing text and graphics (contrast, repetition, alignment, and proximity) that Robin Williams explains in The Non-Designer’s Design Book.

Grasping rules of thumb
The general form of a heuristic is, “If you want X, then do Y.”  Rules of thumb are midway between domain-specific algorithms (like, say, how to calculate the number of shingles needed for a roof) and full-blown problem-solving methods in a domain.

The image at the right (which you can click to enlarge) is an example of rules of thumb for managing conversations on a blog.

vM&K offer this advice:

• Provide learners only with rules of thumb they don’t yet know.
• Limit the rules to those necessary for performing the most important tasks.
• Write them clearly.  (That is, “formulate rules-of-thumb in a readily understandable way…and use the imperative to make clear that they are directions for desired actions.” )
• Find a balance so the text is as specific as possible but applies to all the situations where the rules make sense.

Analyzing intuitive strategies

You see that the first step is to analyze the cognitive strategies used by experts, since these are built up and refined over time as the experts grapple with a range of problems in the domain.  What you end up with is how the task should be performed.

If you observe the target learners addressing the same tasks, you find out how they are performed.   These are the approaches that non-experts figure out on their own–that they intuit, hence the lable “intuitive strategies.”  Often, though, these naive approaches don’t work all that well.

For examples, novices often take what the authors call a top-down, depth-first approach.  In programming, that might mean breaking an overall problem into a group of subproblems, and then attacking each subproblem in depth: writing highly detailed code for one section at a time.

As a result, the novices can lose track of relationships between sections, and they’ll often spent a lot of time linking things together after the fact.

Experts in contrast take a top-down, breadth first approach.  After identifying the subproblems, they’ll continue to expand each subproblem before starting extensive coding.

An instructional design example might be breaking a large course up into what seems like logical sections.  Less-experienced designers tunnel down into the separate sections, writing them in detail.  More experience designers (ahem) will break those subsections down but will tend to think of them in outline or summary mode, making it easier to move up and down levels and see the larger picture.

In a client project I’ve been working on, this means we haven’t yet finalized the overall sequence for what will be a two-and-a-half-day workshop, but we have what I think of as lots of recipes for areas of concentration and for activities.  We sketch enough detail so understand them and look at the design for other course elements.  It’s much easier to refine and shift at this stage.

So we won’t end up with, say, a freestanding section on ethics (which most of the time is like a long sermon on a warm Sunday).  Instead, we plan an activity where participants handle a typical contract problem, then they’ll get questioned by the client about business-ethics issues, embedded (as in real life) in what looks like real work.

Mind the gap

Not only do gaps exist between the rules of thumbs that novices use and those that experts use, but the expert heuristics may be counter-intuitive to the newcomer.  (Maybe this is where “intuitive strategy” came from?  Not that I’m obsessing about the term.)

Think of a beginning driver: her naive model for steering is “keep the car straight at all times,” resulting in constant adjustments of the wheel.  The skilled driver’s heuristic might be something like “aim 20 feet ahead of the car,” providing more reaction time and seeming to ignore the distance directly in front.  But the latter is the more effective strategy.

So what?  If you find that novice learners come to complex learning with naive strategies in place, these will conflict with more carefully worked-out strategies.

SAPs and design strategy

If you’ve developed strategic approaches to problem solving, you’ve got a head start on a number of design decisions.

• You can design problem-solving guidance, like process worksheets or performance constraints.
• You can present rules of thumb as statements (“if you want X, do Y”) or as guiding questions (“what might lead you to develop a job aid for this task?”).
• You can develop or refine task classes based on a sequence of SAPs.

You may need to provide extra guidance, especially to prevent the unproductive or even mistaken results that would come from the naive strategies.  Performance constraints can come into play.  The authors have an example in which researchers have to frame an effective database query before they can enter it into the database–to provide practice in preparing effective queries.

Similarly, the use of task classes (groups of problems at a similar level of difficulty) provide extra opportunity for the novice learner to work with the new strategies–although, as vM&K point out, some intuitive strategies are highly resistant to change.

Modeling examples can help (seeing a skilled performer in action and hearing her comment specifically on her strategies).  In addition, less use of inductive approaches and more guided discovery will allow learners to interconnect parts of the new skills, and to integrate the new strategies into their repertory.

In a final triple play:

• You can give an SAP directly to a learner (it’s supportive information, remember, which means it’s helpful for the non-recurrent tasks).
• SAPs can help you find or design modeling examples.
• SAPs can support cognitive feedback–for example, have learners compare their own process with a given SAP.

This was the shortest Ten Steps chapter to date, so the next post in the series will move on to Step 6, Analyze Mental Models.

CC-licenses project-status photo by brylyn.
CC-licensed rules for blog conversations by choconancy1.
CC-licensed audio-mixer photo by magnifynet.

“What we know determines what we see, and not the other way around.”  That’s how van Merrienboër and Kirschner start Step 6 (analyze mental models) in Ten Steps to Complex Learning. Two people hiking the mountains in France, they suggest, see different places: the geologist sees rock formations; the cyclist sees gear ratios.

As the diagram shows, Step 6 fits with other elements that make up the supportive information component of the Ten Steps model.

(You can click the diagram to enlarge it.)

From a learning perspective, supportive information underpins those skills that you apply differently in different situations.

vM&K see three different types of mental models: conceptual ones that answer the question, “What is this?”, causal ones that answer, “How does this work?”, and structural once that answer, “How is this organized?”

This chapter of the book parallels the previous one:

• Specify domain models
• Analyze intuitive (meaning “naive”) models
• Using domain models to make design decisions

Models: good news, bad news

A mental model is a description of how the world of some domain is organized.  You often hear “mental map” as a synonym, because a map shows layout and relationships.

You can find mental models in documents; you can also develop them by talking with expert performers who can describe the models they use as they work.

vM&K suggest that such discussions start with simpler classes of tasks and work up to more complex ones.  At the same time, they lean away from the common belief that the more you know about a domain, the better able you are to solve problems in it.

In a sense, everything is related to everything, and thus the analyst can build seemingly endless networks of interrelated pieces of knowledge.  Therefore, new relationships must not be added if a competent task performer cannot clearly explain why newly associated facts or concepts improve her or his performance.

Getting the concept

Conceptual models explain a domain by various features.  You can classify computers by their processors, or by their age, or by manufacturer, or by color (though most of the time, color’s not a particularly helpful organizer).

• In kind-of relationships, the model explains that both “chair” and “table” are kinds of furniture.  Kind-of relationships lead naturally to taxonomies, which are hierarchical models, like the biological kingdom, phylum, class, order, and so on.
• Similarly, part-of relationships show that one concept is contained within a larger one.  A leg is part of a chair; a keyboard is part of a laptop computer.

A hierarchical model like a taxonomy isn’t the only possible conceptual model.  Concept maps and semantic networks also show relationships; the latter is more carefully definite with explicit labels for relationships, as in the example on the left (from Wikipedia).

These relationships are fairly obvious, but vM&K offer some other types of relationships that would be useful in analyzing and in teaching mental models.

For example, experiential relationships connect a concept to a familiar, concrete example: the alternator on a car is like a generator (the device that cars used before alternators).  Similarly, analogical relationships aid understanding by comparing a new concept to one that’s familiar although outside the domain in question–for example, the theme used by a WordPress blog is like a set of clothes: changing the theme changes the appearance of the blog, but not the content (the posts).

Other types of relationships:

• Prerequisites: if you understand the concept of division, it’s easier to learn the concept of prime number.
• Location-in-time: the relation in time between two concepts in the domain (pilot program relative to implementation)
• Location-in-space: in a scientific article, the results section appears after the method section.
• Natural-process: evaporation and condensation have a relationship, but not a causal one (evaporation doesn’t cause condensation, and vice-versa)

Structure matters

Another major type of mental model is the structural model, which explains how elements of the model, related by time or space, fit together.  For example, the life-stages of an organism are a time-based mental model.  vM&K mention the location-in-space models of chess experts, who are able to grasp the relationships of chess pieces.

I’ve read elsewhere that when chess pieces are arrayed randomly, expert players aren’t any better able than novices to remember the positions, probably because the relationships seem arbitrary to the experts.

Often there are interrelated sets of structural models–this is complex learning we’re talking about.  You can think of these as the 50,000-foot version, the 20,000-foot version, the treetop version, and so on, as the focus becomes more and more specific.  I could have a model for writing a nonfiction book; within that, a model for a typical chapter; within that, models for constructing sections, paragraphs, and even sentences.

Causal models: domains at work

Causal models concentrate on cause-and-effect or natural processes.  They form principles that allow learners to explain how something works, or why it isn’t working.

Molecular biologist Bonnie Bessler and her colleagues discovered a form of communication in bacteria that helps explain how single-cell organisms as a group can engage in behavior beyond their individual capabilities.  You can hear her describe this quorum-sensing model in a February 2009 TED talk.

vM&K acknowledge that it may be helpful to focus on only one type of model at first, depending on the nature of the domain.  For example, areas like mechanical engineering, architecture, and instructional design are heavily analytic, and so structural models make a good starting place.

Intuition: not necessarily a good thing

As in Step 5  (analyze cognitive strategies), newcomers to a field often bring with them their own mental models.  These “intuitive or naive mental models are often fragmented, inexact, and incomplete,” vM&K argue.  Novices are usually unaware of the underlying relationships between elements, and frequently carry unexamined misconceptions.

One example is the notion that the internet is a centralized system, with all computers eventually connected to some vast, central server.   This hierarchical model is easy to imagine, but gets in the way of someone trying to understand how the internet actually works.

A favorite example of mine is the Microsoft Word document template; another is the cascading style sheet.  Both are used to control appearance and layout, either of a Word document or a web page.  The underlying concept, not always grasped by newcomers, is that there is no necessary relationship between the text on a page and the appearance of that text (size, color, alignment, etc.).

To the newcomer, a document is a document.  Only when that naive model is extended does the newcomer understand that the width, spacing, font, height, weight, and so on can be controlled without any regard to the actual words.

Models and design decisions

As with cognitive strategies, the designer develops mental models only if there aren’t pre-existing ones.  With models in hand, the developer is able to carry out essential design tasks.

For example, a progression of mental models helps to refine the task classes that are a central component of the Ten Steps.  The simplest domain model enables learners to perform the simplest class of learning tasks.

vM&K give a three-level progression for electronics troubleshooting as an example of this relationship between models and task classes:

• In the zero-order model, the model has basic principles of electricity.  The tasks might involve a problem like “given this arrangement, will the light bulb be on, or off?”
• The first-order model adds principles related to changes–”Is there an increase in voltage when the resistance is lowered?”
• The quantitative model moves on to more complex laws of electricity (e.g., Ohm’s law) with questions like “what is the voltage across points X and Y in this circuit?”

Because mental models are supportive information, you can present them explicitly to the learner (e.g., to introduce or explain the domain).  You can also use them to help identify relevant examples and case studies.  Finally, you can use the model as a form of cognitive feedback, allowing the learner to compare his work with a given model.

Intuitive mental models–those homegrown structures that novices may bring to a new field–can be especially resistant to change.  So it might be worth investigating whether that’s true of novices in the field you’re dealing with.  You may need to begin with assumptions in those naive models and gradually work on elaborating new information.

Mental models versus cognitive strategies

The Ten Steps sees a reciprocal relationship between these two types of supportive information.  The main difference is in how they’re used.  Cognitive strategies focus more on the task performer’s actions, which mental models focus more on the domain itself.  going back to the map analogy I used earlier, you can use the map to plan a route from A to B (the cognitive strategy), and you can use the same map to understand the layout of a geographic area (the mental model).

CC-licensed images: mental map photo by michale.
Bird-chess photo by striatic.
Sailing image by A Siegal.

Ten Steps to Complex Learning says that complex skills have aspects that you apply differently for each task (the non-recurrent skills) and aspects that you apply in the same way each time you use them (recurrent skills). Heck, even I’ve said that a time or two in this series.  I bring it up again because Step 7, design procedural information, marks the move into the third of the four components in van Merriënboer and Kirschner’s model: the procedural information component into which those recurrent skills fit.

(You can click the image to see the complete chart with all four components and all ten steps.) vM&K see three types of procedural information:

• Just-in-time (JIT) display of information (prerequisites, rules, procedures)
• Demonstrations of applying the information
• Corrective feedback about errors

The heart of this chapter is that when you’re dealing with recurrent skills like formatting a document or processing a mortgage or digitizing video, you want to help the learner acquire these skills–compile the knowledge, in the book’s terminology.  As he does so, he creates his own specific cognitive rules, and then can perform without the need for a cognitive schema (which would be needed for a non-recurrent skill). Take reading: English has a complex grammar and vocabulary, so learning to read it is a complex skill.  That skill includes many recurrent aspects.  As you read this paragraph, you’re not sounding out the individual syllables; you’re scarcely aware of individual letters unless your attention is drawn to them.  That’s because you’ve automated rules for recognizing a word and relating it to those around it. Sections in this chapter of the Ten Steps:

• Providing JIT displays
• Giving examples of JIT information (okay, so the book says “exemplifying”)
• Presentation strategies
• Corrective feedback
• Procedural information in the training blueprint

Just the steps, just in time One definition of a job aid is something that tells you what to do and when to do it, in order to reduce the need to memorize.  JIT display of information is like that. For a given set of tasks (meaning, ones with similar difficulty), the procedural information is likely the same, and so you’d provide that information for the first task.  You might gradual reduce or fade the amount of procedural information as the learner continued with additional tasks. The two forms of JIT information correspond to the next two steps in the book: the cognitive rules that you follow (Step 8), and the prerequisite information necessary to apply those rules (Step 9).

As I read this chapter, it occurred to me that a lot of organizational training often tries to treat non-recurrent tasks as recurrent, and vice-versa.

Take the supposed soft skill of interviewing a job candidate.  Some aspects of that skill are recurrent: reviewing the requirements for the job, reading the candidate’s resume, arranging a time and place for the interview, making a checklist of items to verify, using both open and closed questions.

So in “how to interview” training, you could provide a checklist to help prepare and conduct an interview.

The actual conversation, though, is always going to be non-recurrent: you’re not interviewing the same person over and over.  Generic training on “interview skills” can’t just give you a checklist of ten questions and five phases.  As noted earlier, for non-recurrent skills, there’s no single right answer.

What makes for successful just-in-time display of procedural information?

First, a modular structure with small steps.  You want the information to have a clear goal that the performer can easily accomplish: “How to change page orientation” is vM&K’s example.  The small size and straightforward goal minimize cognitive load.  Prerequisite knowledge (e.g., definitions of landscape and portrait) can appear as callouts or hyperlinks. And when it comes to procedural information, nice-to-know isn’t.

Next, a focus on entry-level users.  A key difference between a procedure and a systematic approach to problem-solving is that procedures are virtually mechanical (vM&K say “algorithmic”): if you follow these steps, you get this result.

Third, you want to avoid splitting the learner’s attention.  That means minimizing any requirement for the learner to shift back and forth between the actual task and the supporting information.

Here’s an example of those two items being closely joined (in this case, how to test the resistance of conductors in wiring):

(This diagram and some other examples of the split attention effect appear in section 5.8 of  this article by Dr. Graham Cooper of Australia, “Research into Cognitive Load Theory and Instructional Design at UNSW.”)

Sometimes you can’t help splitting attention–a person might need to consult a procedure separately from the immediate task.  You can’t easily assemble machinery while holding the manual, and you can’t always label the parts with the steps.

One approach is to put parts of the task into the reference (through diagrams and illustrations); another one, just on the horizon, is to adapt the environment–e.g., a heads-up display that makes the steps visible through a special pair of glasses.

JIT: know it when you see it

One way to present just-in-time information is through demonstrations, like the “show me” options in help systems that demonstrate how to, say, create a chart in Excel.

vM&K strongly urge using relevant, whole tasks for such examples.  Let’s say the specific procedure is getting a set of PowerPoint slides to start with a specifed number.  One demonstration could have a person who’s created two files and wants to start numbering one set whether the other left off.  The larger context is someone who wants two files (Part 1 and Part 2), and the specific example is getting Part 2 to start numbering at, say, 45.

That places the renumbering skill in a concrete setting.  A later example could have someone wanting to start numbering after the title slide (so the title doesn’t have number, and so the second slide is numbered 1).  The same procedure as the other example, but a different demonstration with a different real-life context.

A second way to deliver JIT information is through what the Ten Steps call instances, meaning examples.  Here you might provide the learner with an example (an instance) of concepts or principles of the skills being learned.  For a researcher, you might provide descriptions of different types of databases.  Again, you’d want to do so in a context that relates to the task being learned–databases relevant to science if the research is in a scientific setting.

Finally, you may need to provide a range of demonstrations or instances.  In vM&K’s example of changing page orientation, you might change from portrait to landscape, and vice-versa.  You might change the layout of a multi-page document.  And you might change the layout of one section in a multipage document, and then change back afterward.

That’s enough for one post.  Next time: presentation strategies for procedural information (some of which surprised me), corrective feedback, and probably a few remarks about knowledge compilation.

The previous two posts in this series about Ten Steps to Complex Learning were the first ones dealing with procedural information.   Step 8, analyze cognitive rules, continues that focus.

(You can click the image to see a chart with all four components and all ten steps.)

In the Ten Steps, cognitive rules and procedures are how you carry out recurrent tasks in a correct way.  “Recurrent task,” to authors van Merriënboer and Kirschner, means one you perform the same way every time.

x = 23 – ( 7 * 14 ) / ( 81 – ( 6 / 2) ) + 9

The procedural part of solving that math problem involves a set of rules that for the most part ignore the numbers in the equation.  In other words, procedural rules are algorithmic, not heuristic; they’re a set of steps and not a handful of rules of thumb.

vM&K says that cognitive rules are “strong methods.”  If you follow the rules, you get the right result.  Perform multiplication or division before addition or subtraction; perform operations inside a parenthesis before those outside the parens.

Strong methods tend to have limited flexibility; they apply in very specific circumstances.  Rules for arithmetic don’t do you much good in spelling.

Spelling out the rules

As with supportive information (for non-recurrent tasks), you can often find procedural information in existing documents, training, and other forms.  If not, you’ve got to do the spadework because of what your analysis can provide:

• It highlights prerequisite knowledge essential for carrying out the rules.  (Step 9 will deal with analyzing prerequisites.)
• Specifying if-then rules gives you the building blocks for designing the procedural information your learner will work with.
• Your analysis also becomes the foundation for part-task practice (the subject of Step 10) to build automaticity in the use of procedural information.

Working with existing documents and interviewing expert practitioners can take two main forms. 

Rule-based analysis applies when there isn’t a specific order in which to perform the steps.  vM&K have an extended example about rules for efficiently stacking a set of different sized buckets; it’s similar to rules you’d use for solving the Towers of Hanoi puzzle.

You use an information-processing analysis when order does matter, such as following a fault tree when troubleshooting.  To me, this is a good way to highlight the differences between algorithmic and heuristic approaches.  Many experts have so automated the rule-based stuff that it seems intuitive to them.  Still, when you’re dealing with entry-level behavior, vM&K’s step-by-step approach  enables novices to achieve the desired result in any situation to which the procedure applies.

Note that it’s important to validate your analysis.  That involves at least two dimensions, I think: having exemplary performers review the resulting rules and procedures, and having typical learners try them out in context.

How specific is a rule?  (How long is a ladder?)

Step 8 says that you shouldn’t get too granular in spelling out the procedural rules.  You don’t want to say, “Retrieve X from your long-term memory.”  On the other hand, vM&K say it’s easy to overestimate the prior knowledge of your learners.  All the more reason for quick prototyping, I’d say, if you’re actually going to revise the material based on the tryouts.

Specificity is one of the things that distinguishes procedural tools from supportive ones.  A flowchart or job aid for reporting airplane flight status (arrived, departed, delayed, ontime, early, late) could be extremely specific; one for dealing with difficult customers would have to be more general.

(And while using the customer-service job aid wouldn’t guarantee correct performance in every situation, using the flight-status job aid would.)

Errors and malrules

From experts and existing procedures, you can build the correct rules.  It’s useful (to say the least) to know common errors and misunderstandings (or “malrules” ); these become important resources for learning design.

Errors, in this point of view, result from things that are difficult to apply, easily left out, and so on.  Double-clicking a mouse is “obvious” to an experienced computer user, but the novice has to master the time between the first and second click.  (Well, I did double click, but nothing happened…)

A malrule, in contrast, is an erroneous procedure.  It could be the result of what vM&K call intuitive models, as discussed earlier in this series. It seems logical to shut down the computer by switching the power off; that’s what you do with other electronic devices.

Now what?  Cognitive rules and design decisions

Cognitive rules focus entirely on how you deal with the recurrent aspects of tasks.  You need to examine the rules and the target audience to determine prerequisite knowledge.  Knowing the order of math operations isn’t sufficient if you can’t multiply.  Knowing that Firefox is your browser won’t get you far if you don’t know how to launch it.  (The next post in the series will go further into prerequisite knowledge.)

“In a psychological sense,” the Ten Steps says, “recurrent constituent skills are analyzed as if they were automatic psychological processes.”

Thats because by design you want to encourage compilation (the building of specific rules by the individual) and, often, automaticity.

So you explicitly present the rules or procedures just in time, along with the task, following the guidelines mentioned with Rule 7 (previous two posts in this series).  That’s where the well-designed displays come in.  They don’t fiddle around with nice-to-know stuff; they don’t needlessly split attention.

You can also use the rules to design effective demonstrations of the skills in question, and to develop corrective feedback.  “Wrong!” isn’t usually good feedback, though it’s important to make clear that the response is incorrect.

Errors and malrules–easy to uncover through tryouts–offer potential for design.  You can focus learners’ attention on rules or steps that are prone to error.  (“This is tricky, and you may get it wrong the first few times.”)  Displays of procedural information can include error-recovery steps.  And providing many demonstrations of steps with a high chance of error means that learners have more opportunity to perceive.

Next time: Step 9, analyze prerequisite knowledge–what you need to know in order to perform recurrent aspects of a complex task.

CC-licensed images: Pythagorean-theory image by tobo;
Towers of Hanoi image by chuyanyatyuet;
computer-competition practice photo by iBjorn.