Dave's Whiteboard

The New York Times reports on research suggesting that if you really want to learn, you should take a test.  Pam Belluck’s article cites work by Jeffrey D. Karpicke and Janell R. Blunt recently published in ScienceExpress (linked article is on Scribd).

The researchers looked at “elaborative studying” (in this case, working from a text to create your own concept map) and “retrieval practice”–writing a freeform essay after reading the material.  In the latter case, you’re writing without the material; hence, you’re retrieving information from memory.

Here’s the researchers’ abstract:

Educators rely heavily on learning activities that encourage elaborative studying, while activities that require students to practice retrieving and reconstructing knowledge are used less frequently.

Here, we show that practicing retrieval produces greater gains in meaningful learning than elaborative studying with concept mapping.

The advantage of retrieval practice generalized across texts identical to those commonly found in science education. The advantage of retrieval practice was observed with test questions that assessed comprehension and required students to make inferences. The advantage of retrieval practice occurred even when the criterial test involved creating concept maps. Our findings support the theory that retrieval practice enhances learning by retrieval-specific mechanisms rather than by elaborative study processes. Retrieval practice is an effective tool to promote conceptual learning about science.

This is is sort of thing that’ll end up on the evening news: “Researcher Says Take Tests, Don’t Study.”  The reality is more nuanced, of course.

As Karpiche and Blunt say, “It is beyond question that activities that promote effective encoding, known as elaborative study tasks, are important for learning.”  What they were questioning, in part, is the notion that retrieval of information is “neutral and uninfluential” in the learning process.

Because each act of retrieval changes memory, the act of reconstructing knowledge must be considered essential to the process of learning.

I’m sorry that most reports about this study use the word “test,” one of those terms (like “training”) that’s a kind of conceptual rent-a-truck; people load them up with all sorts of meaning.

I know I tend to.  And despite knowing better, when I hear “test,” I have a hard time not picturing the multiple-guess, factoid-shackled artifact that so often is labeled as a knowledge nugget.

In the world of learning at work, we don’t always consider that “test” can refer to something other than a mid-semester quiz.   This, despite the fact that the workplace is full of other, more robust examples of testing.

Like load tests on a server.  Stress tests for a product.  Market testing for a new product (or for a media campaign).  Engineering testing aimed at continuous improvement in a process.

Even if you’re aiming at (allegedly) objective assessment, you can shoot for more than recall of discrete bits of information.  So in Karpicke and Blunt’s research, the final testing  involved both verbatim questions (for “conceptual knowledge stated directly in the text”) and inference questions that required the learner to relate different points in the original content.

It’s interesting that participants in the student couldn’t predict whether their retrieval practice would help them learn:

Students predicted that repeated studying would produce the best long-term retention and that practicing retrieval would produce the worst retention, even though the opposite was true.

One version of the study, as part of the “final test,” had students create a concept map.  Once again, students who engaged in retrieval practice produced better concept maps (by which I assume “more accurate ones”) than did the students whose study included creating concept maps in the first place.

CC-licensed images:
ASVAB scores by Krista Kennedy.
Test-box photo by Dave Blaisdale.

On LinkedIn’s Learning, Education, and Training Professionals group, two months ago, a member kicked off a discussion with this question:

Increasingly, we are finding that people bring their phones, computers and Blackberrys to class expecting that it will be OK to use them. How are you dealing with this issue?

As of this morning, there are 83 contributions to the discussing.  Although I’ve disagreed strongly with some of the opinions and suggestions, I’ve come to see this question as yet another example of a complex problem–in other words, one without a single, correct solution.

Here’s my paraphrase of what several participants said.  To minimize my biases, I chose every 8th comment.  Well, I left out one, which happened to be my own.  (Just coincidence that it feel into the every-eighth sequence.)

• I display a slide with logistics (breaks, fire exits, etc.) that asks people to turn off phones or at least put them on vibrate.
• I show a humorous YouTube video and say this is what I did with the last phone that rang during my presentation. I make everyone take out their phone and turn them off in front of everyone.  I include a 20-30 minute break several times a day.
• Ask the class to set the rules.  You are there to learn.  If people were on vacation instead of training, why would they check email?  They can do that during lunch.
• Sometimes people are using BlackBerries and other devices to take notes.
• Lately I don’t even mention phones.  I trust adults to act like adults.  I do like (another person’s) suggestion of asking people to turn them on to integrate outside information.
• Set your phone to ring 3 minutes into the session.  Pretend to talk with the president of the company, who wants to know if everyone’s turned their phones off. Exception: if you expect the president to call, or if someone’s seriously ill. I also believe people are adults who must make their own decision.
• There’s no right or wrong answer.  Some teaching strategies are still focused on a society that no longer exists.  Use appropriate technology at the appropriate time.
• Go with the flow.  I can get irritated if a phone rings, but if the class is good and people are engaged, they’ll take their own responsibility.
• I like letting the learners decide how to deal with device interruptions.

I don’t do much formal instruction any more, by which I mean acting as the primary source (and predominant voice) in a scheduled learning event.  I’ve done quite a bit of that, but over time found that people seemed to learn best when I talked less and they did more.

Yes, when people are new to a topic, they generally need some grounding and some concepts.  Most of my experience is has not been with people new to the organization and the industry, however.  That means they tend to need less “before we begin” than a lot of instructors (and instructional designers) seem to think.  Even for a topic as information-dense as Amtrak’s reservation system, I found that a lean approach (less talking, more doing) suited the goal of having people able to use the system.

The LinkedIn discussion does provide a glimpse at the many ways that people working in this field view cell phones, PDAs (does anyone say PDA any more?), and smartphones.  (Almost none of the comments address computers as such.) I see a kind of clustering around “they’re here to learn (from me),” and a smaller one around “I’m here to help them learn.”

My own phone, like my computer, is as basic a tool as pen and paper.  Yes, I take paper notes, but when I have the choice, I take electronic ones so I can tag, search, re-use, copy, paste–all of which are tougher to do with PowerPoint handouts or handwritten notes.

I don’t want someone else telling me how to capture or retrieve information.   If they say things that I find condescending or just plain silly (“enter the world of civilized people,”  “phones are an interruption to learning”), I’ll get the message–though it may not be the one intended.

CC-licensed images:
Retro phone photo by Robert Bonnin.
Classroom sign photo by Ben+Sam.

The third week of PLENK2010 is ending.  When I decided to sign up for this online course (“Personal Learning Environments, Networks, and Knowledge”) I didn’t have a goal much more solid than “find out more.”  I assumed the “more” would involve PLEs and PLNs.

Finding out more about knowledge wasn’t that high on my list.  Like a vegan who’s confessing his weakness for Twinkies, I feel vaguely uneasy saying this, but most high-abstraction discussion just makes me sleepy.

That probably explains a stretch of about five days when I didn’t do anything PLENK-related; I even closed the Tweetdeck column I had for the #plenk2010 hashtag.  Everyone and his cousin Bernie drew diagrams of their networks, and a daunting number of discussion posts probed the nuances of PLE versus PLN versus VLE… truly, I expected to see RSVP versus SPQR.

That’s okay.  I don’t have to like, much less use, a term like MOOC (Massive Open Online Course).  I see the intention as an opportunity that’s participatory, that has a focus but not a required sequence, and from which I select what I choose to select.

This post is just me, flipping through things that stand out after three weeks.

In how to take this course, the PLENK facilitators said the purpose isn’t to use a bunch of tools; it’s to practice using those tools to select content, rework it, create something new, and share–particularly with others in the course.

I already had this blog and a Delicious account.  I take Twitter so much for granted that I was surprised the PLENK profile form didn’t have a specific field for it.

Putting the “dis” in “discuss”

I find the Moodle-based discussion kludgy, from a user standpoint.  For one thing, each week’s discussion list gets sorted by most-recent-addition, which means the order constantly changes.  But there’s nothing to tell me if I’ve already read the posts in a given discussion.  Since there are 36 discussions in Week 2 and 12 in Week 3, I’m unlikely to remember on my own that I’ve seen everything in the thread with 34 posts but not everything in the one that has 31.

The same way that you make less coffee if you have a finicky coffeemaker,  I find the recurring annoyance of navigating the discussions has probably conditioned me to open them less often.  I have to nag myself to make an effort, because I’ve benefited from what other people have shared.

Net worth

Lots of people have made lots of diagrams of their PLEs and PLNs.  I’ve skimmed some, though after a while they all kind of look the same.  I realize at some level they are the same, because the diagrams show relationships that matter to those who created them.

Me?  I read Dave Cormier’s thought that the difference between PLE and PLN is mostly semantic, and thought, “Works for me.”

Yeah, yeah, semantics matter, but in this case not very much to me.

About the best thought I can draw for myself is the value of reflecting on where and how I learn, especially in my professional life.  Some of this–exchanges with my peers, reading what experts (however defined) have written–is of such longstanding, it’s kind of like the way I write.  I don’t spend a lot of time thinking about my use of passive verbs, or having pronouns agree with their antecedents, because I’m at pretty freakin’ high strength for those particular skills.

I do find myself thinking more along the lines of “so what?”  What (if anything) do I do with what I take in?  I don’t want to be some kind of conceptual holding tank, or a guy with 10 gallons of Knowledge stored in the basement next to the water heater.

Follow disgruntle

I’ve used this phrase before.  It’s my way of telling myself to pay attention to annoying things I want to dismiss.  (Note: when you come with your friend to evangelize me, I’m still going to pretend I’m not home.)

So, for example, I like to make fun of words like “affordances,” which at times seems like the Learning Business’s latest synonym for “general wonderfulness.”  Deep down, I knew all along that probably wasn’t the full picture; I just couldn’t drag myself to find what it was.  But in a side conversation with yet another online colleague (YAOC), I got some useful, cognitive-psychology explanation.

I’m still going to make fun of “affordances,” but not as often.

Since I didn’t make a map of my PLE, though, I’m feeling as though I have to at least try making a concept map.  Of what, I’m not sure.  Why?  Mostly to put myself through the exercise of trying to do it–to see whether I end up with a result that seems of higher value than expected.

I understand what George Siemens meant when he said mindmaps have a center but concept maps don’t.  I have to do one, I suppose, to grasp the implications of how concept maps “communicate relatedness and reasons for relatedness.”

Seeking Ellumination

I missed the first week’s Elluminate broadcast because I couldn’t get it to work.  It took till nearly the second week’s broadcast to discover the problem was on my end (my so-called security software wouldn’t let the session open).

These are handled about as well as you can handle a high-tech conference call.  The session moderators have done well, I think, and the participants share enough of a focus that the level of silliness or randomness (in the “backchannel” of the chat window) is about what you’d get in an in-person workshop.

What I didn’t expect to happen is that I’ve done some post-session followup each time with individuals who spoke (or, more accurately, chatted) up during the session.  As with Twitter discussions, what someone says in an Elluminate session can move me to make further connections.

* * *

Hmmm… I’m kind of a wordy blogger anyway.  Skimming what I have here, I don’t have much sense of a conclusion.  In part I wrote this from the notes I kept (in an Evernote document) over the past couple of weeks.  There’s more, equally ill-formed, so I’m just going to stop now and let things percolate (or incubate) for a bit.

* * *

(Added afterward)

I forgot to mention that my survey (asking about the backgrounds of people in the course) did confirm an impression I had.  Only 12% of respondents say they work in corporate / for-profit areas, while 39% work in academia.

Another 6% say they work at the Vatican, which is a good reminder of the value of informal surveys.

If you saw Shakespeare in Love, you may remember an early scene in which Philip Henslowe, the producer, is warned by moneylenders that when people don’t pay their debts, their boots catch fire.  (The real-life Henslowe kept a diary–actually an account book listing payments and other data–that’s a prime source for information about the Elizabethan theater.)  Eventually Henslowe convinces the money guys to back Will Shakespeare’s new play, Romeo and Ethel, the Pirate’s Daughter.

Geoffrey Rush as Philip Henslowe

In this first week’s experience of PLENK 2010 (the online course about personal learning environments), I kept hearing Henslowe and the moneylender discuss how a play comes to be.

Henslowe: Mr. Fennyman, allow me to explain about the theatre business. The natural condition is one of insurmountable obstacles on the road to imminent disaster.

Fennyman: So what do we do?

Henslowe: Nothing. Strangely enough, it all turns out well.

Fennyman: How?

Henslowe: I don’t know. It’s a mystery.

By no means am I implying that PLENK is on the road to imminent disaster.  Or better, it’s a road company, in at least two senses:

It’s a work in progress.  What goes in, what happens, and especially what comes out can’t be known.  Like the road company for a play, it takes place in multiple locations.  (See the Google map started by Heli Nurmi, with only some of the 1,000+ registrants.)

It’s a group of people.  They’ve met in this virtual space for their own reasons, much like an earlier group:

At nyght were come into that hostelrye
Wel nyne and twenty in a compaignye
Of sondry folk, by aventure yfalle
In felaweshipe, and pilgrimes were they alle,
That toward Caunterbury wolden ryde.

By nightfall, into those lodgings had come
Nine-and-twenty people in a company
Of sundry folk, by chance fallen
Into fellowship, and they were all pilgrims
Wanting to ride to Canterbury.

As with Chaucer’s pilgrims, each person in PLENK showed up at the virtual Tabard Inn because of his own reasons: curiosity, a desire for focus, challenges to address.  And each one will have a story to tell.

More than one story, I think.  Harry Bailey, the host, urged that compaignye to each tell two stories on the way to Canterbury, and two on the return.  He wanted the travelers to enjoy the two-day to Canterbury (an early suggestion that the journey could be the reward).

PLENK’s company isn’t like Chaucer’s; one of our commonalities is that we’ve got different destinations (if in fact we’ve figured out where we want to go).

So the “road” that the company travels isn’t a specific route.  It’s more like the Oregon Trail or the Silk Road: a general direction with multiple paths.

For my own part, I’ve read a sheaf of blog posts and discussion posts from participants this week, along with some of the resources contained in PLENK’s daily feed.  These are the stories that the pilgrims tell–not fictional ones, told on the way to Canterbury, but sense-making ones, told on the way to understanding.

I’ve found people trying to make sense of PLNs and PLEs in contexts like high school teaching, graduate education, personal growth, and (thank goodness) learning on the job.

Not all the sense they’re making makes sense to me, but it’s not supposed to, any more than every presentation at a conference or every course in the catalog is supposed to.  Really, I’m still feeling my way along, but I’m not too uncomfortable with that.

PLENK facilitator Rita Kop wrote about information abundance and economy of attention the other day.  She mentioned John Hagel‘s thoughts on attention as an increasingly scarce resource.  My quick take on what that means: the more inputs available to you, the less you can afford to, well, pay attention to all of them–because you’ve only got so much attention to spread around before you hit cognitive homeopathy.

Kop was trying to work out concerns of some PLENK participants and wondering about whether there’s a good match between “learner needs and educator support.”  I couldn’t say, but included this in my comment at her post:

For some people, plopping into PLENK is like an American suddenly teleporting to London. Or maybe Amsterdam, where enough people speak English that he’s mostly disconcerted by all that Dutch on signs.

For some, though, it’s like being teleported to Riga or Mumbai, with a lot more “foreignness” — an abundance of unfamiliar information. When it comes to economy of attention, they feel like their account is overdrawn.

Speaking of which, if attention’s an account, then time is the wallet you keep the card in, and I have to watch how often I get that wallet out.

I’ve been neglecting my cognitive tools.  Admittedly, they don’t need the shot of WD-40 that I use on the garden clippers.  But I’m not taking care of the tools I rely on every day.

My grandfather would disapprove.  He was a craftsman with serious technology (locomotives) and practical technique (carpentry).

I’m sure Jack D (as everyone called him) had plenty of tools, but probably not too many. He’d consider the likely benefit against the cost. His skill meant he could achieve superior results with adequate means.

As for the tools he did have, he had them ready to use.  Blades were honed, sawdust was cleared, dirt was wiped away.  Pegs, racks, drawers, tins were chosen and rearranged to support effective use.

Earlier today, I came across a colleague’s question related to RSS feeds.  To help answer his question, I opened my (often neglected) NetVibes feed reader.  And it’s a mess.

Messy in part because it’s easy to add feeds, so I’ve added lots.  I don’t always step back and thing about what value I get from a particular feed, though.

I do sometimes cluster them.  NetVibes has tabs, so I group learning stuff under one, science stuff under another, “not work” stuff under a third.

I see this tendency to collect things without much reflection in my Delicious tags (513 of them) as well.  And in my Evernote notebooks.

Adding is just collecting.  Grouping is a potentially helpful advance.  It take more time to pause and consider what you’ve got, what you get, what you think about that, and what you want to do differently.

Which reinforced the need for (and the value of) organizing what I’ve got.

If I charged for these posts, I’d call it curating, one of those highfalutin words I enjoy satirizing, like  affordances.

The concept is apt, though; the medieval Latin word curator, related to “care,” meant an overseer, manager, or guardian.  I can’t resist adding that in the Middle Ages, according to the Online Etymology Dictionary, this meant the care of “minors, lunatics, etc.”

Care is much more than amassing.  In fact, care sometimes includes pruning: cutting back and discarding things that aren’t useful, things that can even impede productive growth.  “Productive,” naturally, is up to you and the results you have in mind.

Or, up to me.  So I’ve got some chunkifying to do.  It’s not always copy and paste, you know.  Sometimes select and delete has a big payoff as well.  If you’re going to keep the saw, then make sure it’s sharp.  But every so often, ask if that’s a saw worth the keeping.

CC-licensed image:
Coffee cup and clippers by Pollyalida.