In my Building Job Aids workshop (presented last Tuesday at DevLearn 2015), participants analyze multiple case studies, applying techniques and using job aids to, well, build job aids. Among the skills they practice are the ability to choose the right type of job aid for a task, and the ability to use that type effectively.

There’s a lot of thinking and writing: I make an effort to avoid explaining much before an exercise. Instead, there’s a minimal introduction, with a lot of what would have been explained turned into a print resource to be consulted as needed.

One potential downside is that especially an hour or so after lunch, thinking and writing are conducive to dozing off.

At the same time, my assumption was that participants would want and need additional practice on relevant examples. How could I give someone the chance to assess different job aids and rate their effectiveness? Did she think the samples would produce the desired result? How did they align with ideas in our workshop?

The challenge wasn’t so much finding the examples as structuring the evaluation. The tradeoffs I saw (or believe now that I saw):

• Time constraints
• Relevance
• My desire for multiple elements in a rating system
• My desire for a simple, overall total

Then the format presented itself in three words:

I liked this title so much, I was determined to use it. But I’ve learned not to be literal about this kind of borrowing. What  makes Jeopardy!-style games in training a dumb idea (even a counterproductive one) so often is not (necessarily) Jeopardy! itself. It’s the mismatch between the content and a format best suited to recalling isolated facts.

Some characteristics of dog shows that I thought suited my goals: I had widely different types of job aids, like the different dog breeds. I had limited time, which at least for me was like the dog-judging segment where the trainer fast-walks the dog in a set pattern before the judge. Plus judging.

That’s where I had the most trouble.  How to get multiple points, an overall total per judge, and a logistically sane process? I started with a three item scale, rating each job aid on its fit (is this a good job aid for this kind of task?), its function (is it likely to produce the desired result?), and its format (how does it stack up against the job aid guidelines in the workshop).

I could score each of those from 1 to 3, with an extra point thrown in for personal preference. No matter how I squinted, though, it looked like way too much math.

Then I remembered the Apgar score – a quick assessment of a baby at birth. Five qualities like heart rate or respiration are each assigned a score of 0, 1, or 2. The total describes the baby’s physical condition on a scale of 0 – 10.

So I came up with a five-point scale for Best In Show:

• Aptness: how well the job aid fit the task and the setting
• Payoff: how likely it’ll achieve the desired result
• Guidelines: how it fit with guidelines in general and for its particular type of job aid
• Appearance: overall effectiveness of the design
• Response: the judge’s own reaction to the job aid.

As you can see, each item had a line for its score, with a box on top for the total.

In the interest of time, I limited myself to six competitors. This was the score sheet:

Off to the show

I was pretty sure I’d have a decent internet connection. I made a slide with links to my six examples. I explained the scoring, distributed the ballots, and showed each competitor for 30 – 60 seconds, with some contextual commentary as needed.

If I’d had a large group, my plan was for each person to fold the completed ballot between the six boxes, so as to tear it into six individual sheets. I’d have had one person total the ballots for competitor A, one for B, and so on. My workshop group was small enough that I could divide a sheet of flipchart paper in six as Voting Headquarters. It was little trouble for me write down scores by candidate and then total them.

How it went

Best in Show was a success, both as a change of pace and as an exercise in judging job aids. It also broadened exposure: half the competitors were new; the other half had been seen only briefly, as examples, earlier in the day.

An unexpected plus: everyone could see all the individual totals. One job aid received solid 10s except from one person who rated it a 7. Another participant said to her, “I want to know why you rated it a 7.” The question was not a challenge but rather genuine interest in how another person applied the principles of the workshop.

Aftermath

I’m really pleased this went as well as it did. I’m thinking of ways to make it work better (one participant was confused by my instructions and rated on a scale of 1 to 3 rather than zero to 2).

And if I have more time, I’ll have a follow-on exercise: Raise the Runt. The idea would be to see which job aid scored the lowest, and then talk about why and about how to improve it.

 

In a previous post, I showed a fictionalized example of an actual guide for assembling a piece of industrial equipment. These instructions weren’t particularly well done, and I set myself the task of making some improvements. I also wanted to explain what I did, and why I thought it was an improvement.

The original, written on seventeen sheets in an Excel workbook, had more than 50 steps. I didn’t fictionalize them all, and I haven’t tried to revise them all. I’ve done two pages (that cover what 5 of the original pages did). I think these revisions are representative. (You can click each image to open a full-sized version in another window.)

By the way: I don’t have any documentation other than what’s in the original widget vat instructions. In some places, I’ve made an educated guess about details that aren’t clear; in other cases I made things up. I’ll point these out along the way.

On to the revision!

Why my version looks this way

We’re guiding a task that’s mainly a procedure: a sequence of steps that follow one another logically, with relatively little decision-making.

I see this as a job aid for lots of reasons, including the large number of steps. A number of my revisions are part of what any good job aid should include.

A clear title

The original read “Main Assembly Instructions,” which is ambiguous at best, unless Quasimodo Corporation only assembles one thing.

Here I’m pretending that there are several models of widget vats that differ mainly in size. I’ve put the model numbers into the title so the assemblers can tell easily if these are the instructions they want.

First things first: safety and prerequisites

Every page of the original version had a box listing the same three pieces of safety equipment–as if the developer thought the assemblers might take off their safety glasses between pages 11 and 12.

I’m also pretending that the “submittal drawing” spells out things like how many fasteners you need of each type. If that were not the case, I’d have to find out from my client whether (for example) the fasteners were stored at the assembly point, and if so how the workers could get more when they needed.

Emphasizing what’s important

The original version didn’t make much effective use of contrast, alignment, or spacing, so it’s much harder to figure out what’s important. That’s one reason I’m a fan of language like “before you begin.” In my revision, that stands on its own as the header for a number of items that we want the worker to do before starting the assembly.

And rather than the original’s unfortunate use of ALL CAPS, I recommend using capitals, boldface, or similar techniques in specific, limited circumstances. One of those circumstances is to highlight words like DO NOT, EXCEPT, DANGER, and so on.

Don’t confuse people with detail

Although the original used over 50 photos, in my revision I’ve hardly used any. In part that’s because I didn’t have good replacements for the photos–but also because many of the original photos fail to contribute useful guidance.

One problem with the original images–and with photos in general–is that they often provide too much detail, making it hard to grasp what’s important in a particular step. Or they ignore context, also making it hard to grasp what’s important.

Take a look at these three examples on one page of the original (click to enlarge):

Drawbacks to these:

• Left: Why do I need to see a picture of the parts cart? The instructions say it’s important to push rather than pull the cart–but they don’t say if there’s a specific end I should push from. If there is, show it. If there isn’t, spell that out.
• Center: you might be able to figure out, after studying this picture, that you’re viewing the assembly area from the side–but you can’t tell whether the top of the assembled widget will be on the left or the right.
• Right: the instructions tell you:
  • CHOCK ONE (1) WHEEL ON EACH END OF EACH VAT ASSEMBLY FIXTURE (FOUR (4) CHOCKS TOTAL).
  • In my first revision, I’d missed the “four chocks total” part, which implies that neither the all-caps approach nor the FOUR (4) text/numeral approach helped.

Although I didn’t have much to work with, I have a couple of examples of images I think would work better.

The top picture in “image detail” shows a closeup helpful for one assembly step: the flanges should face up, and they should face out from each other.

(Yes, I made up the term “bleem flange.”)

The lower picture is a line drawing rather than a photo. Line drawings are a great way to eliminate unnecessary detail. I’ve teried to reduce details to the essentials: you’re fastening two (more-or-less) L-shaped pieces together, and you fasten through holes that alternate large/small/large/small.

(I’ve made the assumption here that the assembler should alternate directions when fastening these things: one fastener from one side, the next from the opposite side. If it were important to do that alternating while fastening, I’d add a box to spell that out. If you can install all the fasteners from one side, and then from the other, I’d spell that out. )

Some  problems in the original version stem from the decision to always use three photos per page in the original. That’s a lot like deciding that you need to use one semicolon per paragraph: not wrong, exactly, but needlessly specific.

What I’d do instead:

• Leave out the cart photo. My assumption: the assemblers know what the parts cart looks like. If they don’t, they shouldn’t be assembling widget vats.
• Possibly include a close-up of the portion of the cart that I’m supposed to push, assuming there’s some sort of handle or push plate.
• Use a bird’s-eye-view line drawing to show the layout of the assembly area.
  • In the text for this step in my revision, I made up some floor guidelines to show “top” and “bottom” of the assembled widget.
  • I also made up guidelines on the frame to show where to place some of the initial pieces.
• Clarify where to place the wheel chocks.
  • Chock one wheel of each assembly fixture.
    • Written this way, it doesn’t matter how many fixtures you have, so you don’t need the FOUR (4) business so beloved by people writing procurement specs.
  • Place the chock so it’s closer to the center of the fixture than the wheel it’s touching. (If this is a best practice; I have no idea.)

Trying to build a useful set of performance guides like this can be a tool for analysis tool as well. You can’t create a successful guide without knowing what the inputs are, what the process is, what the outputs are, and what the standards for success are.

How many assembly fixtures do I need? On the framework, which side represents the top of the assembled piece? Can I install 20 fasteners in one direction, all in a row, and then 20 in the other? And what the hell is a “submittal drawing,” let alone the cryptic “BOM?”

What’s more, you can identify items that don’t need to be in the guide. Frankly, I think it’s…perhaps less than helpful to list “vat part cart” as an assembly aid. That’s like saying, “In order to build your IKEA Poang chair, buy the Poang chair and bring it home.”

The changes here, and the reasons behind them, apply to most performance guides like this. In my next widget vat post, I’ll show some revisions I’d make that are specific to procedural or step-by-step job aids like these instructions.

Last week, I had a Twitter conversation with Jane Bozarth and David Glow. All three of us, I think, started by having fun with the, um, suboptimal material Jane had been given to start with. I briefly described a spectacular example of a wrongheaded job aid, but we moved pretty quickly to the idea that mockery (however well deserved) wasn’t enough.

So rather than post this fictionalized version as a candidate for the Worst Job Aid Ever, I thought I’d try doing two things: talk about why this attempt works so poorly, and suggest some things I’d do differently if invited. I’m grateful to David and Jane for encouraging me in this.

It’ll be a bit wordy, so  I’ll break it into two posts. This is the first.

Here are two pages taken from a set of assembly instructions for a piece of industrial equipment. I’ve edited the pages, but only to conceal the source. The layout is exactly the same as the original. (You can click the images to see them full size in a new window.)

 (By the way, when I said the layout is exactly the same as the original, I also meant that fifteen of the seventeen pages have an identical layout. Each page has space for three dropped-in photos; each has three borderless boxes of instructions [centered vertically, with text almost always in ALL CAPS]; each has the six boxes you see for safety, tools, quality, and so on.)

You may have noticed that the assembly instructions were created in Excel. I confess that I’ve kept this unique document for selfish reasons: it’s one of the most bizarre attempts at guiding performance that I’ve ever seen.

I’m not here to talk about bizarre–at least not today. I’m here to talk about why these instructors fail so badly.

What’s not working, and why

It’s true that never before nor since have I seen an assembly guide written in a spreadsheet, but that’s more a symptom than the underlying problem. A more important question: what’s Quasimodo’s problem? In other words, what are they trying to get done?

I’d say their goal is to have assembled widget vats that pass inspection and meet cost guidelines.

I’ve covered a lot of territory with “pass inspection,” but the second-last sheet makes reference to a number of Quasimodo standards. (Click this image to open it in a new window.)

Although I never saw the actual equipment, it’s clear from photos that the finished product is about the length and width of a roomy parking space. Two or three people assembly it in a factory, from start to finish (meaning, not on an assembly line–the parts come to the assemblers). Assembly involves over 50 steps, several of which are variations of “repeat steps 1-5 for all four edges.”

So what?

Well, mostly these steps are sequences of actions with few decisions:

1. Position center end wall panel CAD on assembly fixture.
   • Use two people or a jib crane to lift panels.
   • Check submittal for end wall connection and bottom connection orientation.
2. Fasten center end wall panel CAD to floor plane using 5/16 x 3/4 LG tappers.
3. Fasten center end wall panel CAD to floor support channel ZCA using 3/8 x 1-1/4 LG screw.
4. And so on and so forth…

I imagine these instructions were printed, pages slipped into sheet protectors and stored in a ring binder at the assembly area. They’re like a recipe from an industrial cookbook. So the fact they were created in Excel, while non-typical, is less relevant than the barriers created by their overall layout and especially by their approach to guiding behavior.

What else do we know about assembling the widget vat?

• Workers need safety equipment like hearing protection and safety glasses.
• Parts of the task involve specialized equipment (like that jib crane).
• Some ways of working are more efficient or more effective than others (“start at the center and work your way out to the ends”).
• Detail often matters (as in the note above to check the submittal, a kind of specification for one specific assembly job).

And why doesn’t this attempt work well?

To shoot the biggest fish in the barrel: Excel isn’t a word processor. It’s not a publishing tool (unless you’re publishing numbers and charts, or else tables of data). Creating this guide in a spreadsheet needlessly complicates the task of updating and revising — and even searching.

This isn’t even well-done document publication via Excel. Here’s a portion of page 1-5 (the second image above).  Note that the photo includes two callouts labeled CA while the accompanying text refers to panels CAA, CAB, and CAF. If you had the Excel file, you could enlarge the two CA callouts in the picture, and then you’d see that one of them actually reads CAA while the other reads CAB.

So Doctor Spreadsheet may not have been a proofreading whiz.

But who cares? The reason Excel is a poor choice is that nothing calls for Excel. There isn’t a single calculation in the entire document. You might as well have produced this in PowerPoint. Or taken photos of alphabet magnets you arranged on your fridge.

From a graphics standpoint, the lockstep layout assigns equal weight to two areas (task photos and procedural steps), and the same total weight to six blocks, one of which (quality) reads “n/a” on all but one of the 15 pages.  Most of the time, a third of the space on a page is sitting around doing nothing. Nothing except confining the actual performance steps to their all-cap prison.

From a job-aid standpoint:

• Before you begin information (like equipment and parts to have) should appear before the steps in the procedure.
• Visuals, when necessary, should appear next to the step they illustrate.
• Information that’s not needed on a page should not appear and shouldn’t have a reserved parking space that does nothing but delineate whether the information might show up on a subsequent page.
• Steps should be clearer delineated, not crammed into a trio of one-size-fits-all holding pens:

The vertical centering manages to complicate reading even more, a remarkable feat for such a small amount of text. Another complication: in this example, step 5 says to repeat steps 1 through 5–a good recipe for an endless loop. I think the assemblers would figure out what the designer meant, but it’s no thanks to the designer.

So, as it exists, the QAC assembly instructions are hard to update, hard to read (from a graphics standpoint), and hard to follow (from a getting-your-work-done standpoint). It doesn’t seem like they’d easily get Quasimodo to the goal of assembled widget vats that passed inspection at a cost acceptable to the company — at least not until the workforce managed to build enough of these things to not need the instructions.

In my next post, I’ll show some possible revisions and talk about why I think they’d help. But I don’t have to have all the fun: add your comments. Ask your questions–if I’m able to, I’ll answer them. Let’s see if we can get to IPI sign-off a bit faster. You know, so we can excel.