Data: not the plural of “anecdote”

I like stories–hearing them, telling them.  A Gaelic proverb says a good tale’s no worse for being told twice (Cha mhisde sgeul mhath aithris da uair).

A story can be an example, or it can be an anecdote.  Trouble looms when we generalize from anecdotes–individual instances, rather than a significant accumulation of data.

Really, everyone does learn better when you have an LMS.

One of the points Jamais Cascio mentioned was science.  To me, the essence of science involved observing, musing, predicting, checking…and trying again.  As Isaac Asimov said, the key phrase is not “Eureka!” but “That’s funny…”  We notice something askew and start wondering about it.

When it comes to learning, and I think especially when it comes to training, we tend to lean more on anecdote rather than data.  Will Thalheimer’s worked diligently to dispel myths about how much we learn through various means, but the legends will likely outlast him, me, and perhaps the nation.

If you work in areas related to learning, what do you do that’s based on data?  What do you along the lines of observing your work in action: checking to see whether the things you anticipated turned out as expected?  For me, there are few things more compelling that someone sharing what they tried, why they tried it, and what happened–or didn’t.

Ruth Colvin Clark has for years put learning principles to work.  You can sometimes use her insights to be diplomatic, highlighting “cognitive load” rather than saying “there’s too much stuff here for anyone to pay attention to.”

Similarly, Clark Quinn regularly stresses the need to pay attention to how people work (and to try using what people have learned in the past).

All these things are instances of the great principle: the more we learn, the better we see what we don’t know.

The fuel on which science runs is ignorance. Science is like a hungry furnace that must be fed logs from the forests of ignorance that surrounds us. In the process, the clearing we call knowledge expands, but the more it expands, the longer its perimeter and the more ignorance comes into view. A true scientist is bored by knowledge – it is the assault on ignorance that motivates him.

Matt Ridley, in Genome

Feathery anecdote photo by * Cati Kaoe *.

3 thoughts on “Data: not the plural of “anecdote”

  1. Kia ora Dave

    The most useful set of data I ever gathered of elearning was to do with engagement – a cornerstone for elearning success.

    Over a period of two years I studiously gathered every bit of data I could about my students online.

    I kept in regular contact with all my students. Most of this was done by telephone and email and through contact by web-notices.

    I also sent out what I called bulletins in those days, which was a weekly emailed letter, blind carbon-copied to all my students.

    Early in my data gathering I noticed that there was a bump in the general return of completed assignments from my students a day or so following the bulletin being sent out. On the occasional weeks when I hadn’t send the bulletin that bump didn’t occur, and this is what drew my attention to the pattern – the ‘that’s funny’ bit that Asimov spoke of.

    The reason I didn’t find such a correlation between communication and return of completed assignments at other times was simply because of the more or less random way that these occurred. The data was less ordered and so the occurrence of a response associated with a communication from me to the student went unnoticed as the pattern was difficult to recognise.

    When I examined this random data carefully, however, the same responses were to be found and the pattern was clearly identifiable. Of course, I had become aware of what to look for by then, and was able to sort my data accordingly, so that the connection between nudge and response could be more easily compared.

    These observations changed my pattern of communication with students. Instead of sending one summary weekly bulletin, I send two or more briefer catch-ups to all students. The result was a more even engagement of all students, some of the results of which are summarised in one of my posts last year.

    Little did I know at that time what elearning was all about. But the importance to engagement of frequent teacher-learner contact had a major influence on my teaching practice and the way I communicated with students, following these studies.

    I found that the friendly nudge was especially effective with learners who had a tendency to be a bit reluctant. These learners tended to move their behavioural position from being diffident engagers to a more regular pattern of engagement.

    Catchya later
    from Middle-earth

  2. Ken, thanks for the terrific example. I think there are few questions more useful than “what makes you think that?” asked in an open way.

    Claude Lineberry, once president of ISPI, said that most corporations aren’t interested in formal studies with control groups. At the same time, they do tend to be open to “we looked at results from 75 learners.”

  3. Kia ora Dave

    Of course, control results from 75 learners is far easier to say than conducted. I was lucky, but at the same time frustrated. I was only permitted 15 – 20 students at any one level (I was studying two levels. Ideally for a study such as the ones I did, a group greater than 20 is (more) statistically significant.

    BUT, being a qualified scientist, my reporting used the words ‘suggests that’ rather than ‘shows that’. A small group of results that suggests a pattern is very useful for pointing the way to further study, and is valid research reporting and is also scientific.

    On that note, the anecdote is perfectly acceptible for this too, provided one takes care to say that ‘this instance suggests that this could be a good area for further study’. After all, the anecdote is what leads the good researcher to make further observations while looking for evidence of a pattern.

    Having said that, Goethe warned us that “if we go looking for evidence to support our claim, we are likely to find it.” Looking for truth rather than evidence to support a claim, is were he was coming from. He was a true scientist. Perhaps police investigators could take a leaf out of Goethe’s book on this one :-)

    Catchya later
    from Middle-earth

Comments are closed.