Year: 2012

This is a guest post by Peter Green of the Better Time Management blog. It’s a great resource, and he’s a terrific thinker who deserves our gratitude for the research he’s done.

Introduction
A trawl of the research on the effectiveness of time management training finds a dearth of empirical evidence. Research that does exist is contradictory and inconclusive. On the other hand, research I co-authored with Denise Skinner published in 2005 based on 19 courses found an overall 20% median improvement in time management skills. To see why the findings of our research contrast with earlier studies, I believe we need to look at training content and research methods.

Training Content
Covey and colleagues have usefully categorised four levels of time management training. They conclude that earlier approaches with their emphasis on efficiency and organisation have failed to help people match what they consider to be important with how they actually spend their time. They describe a fourth generation of time management which includes the best features of previous thinking but also focuses on helping people spend more time on what is really important at both work and home. So if some of the earlier research was based on ‘old school’ time management training, this could have contributed to their mainly negative conclusions. The training in our courses on the other hand was based on the first edition of my book Managing Time: Loving Every Minute and fits Covey’s description of ‘fourth generation’ content.

Research Methods
Major studies published between 1982 and 1996 raise questions on sample size, evaluation, measurement tools and data collection. This helped to shape the design of our own project.
For instance on sample size, some studies were as low as single figures and some purely involved students who arguably are not representative of typical time management training attendees. By contrast ours covered 134 respondents or 53% of people who attended the 19 courses, which were for ten organisations plus others represented on open courses. They covered a wide range of industry sectors and job functions thereby giving a more representative sample.

Evaluation of time management training is universally accepted as being difficult in terms of what to measure, when and how. Readers may be familiar with Kirkpatrick’s four level evaluation model where 1 is reaction to the training, 2 is what has been learnt, 3 is about behaviour changes due to the training and 4 is about organisational benefits directly attributable to the training. Level 4 is notoriously difficult to measure simply and with certainty so we opted for a level 3 study; what changes in time management behaviours were evident in the workplace sometime after the training?

Measurement of time management has proved challenging. A variety of questionnaires have been used in previous research but all were prescriptive and unable to accommodate varying training needs. Furthermore, we felt that their measurements were too detailed. For example Macan, whose work has been mentioned on this site, used a questionnaire with constructs such as ‘I set short-term goals for what I want to accomplish in a few days or weeks’ and ‘I make a list of things to do each day and check off each task as it is accomplished’. The problem with this approach is that in order to cover all the aspects of the subject it is often desirable to have multiple definitions of the same concept. Applying this to time management would result in a huge number of operational definitions, making the instrument unwieldy and impractical. Macan attempted to overcome this by simplifying the subject into three broad areas, raising concerns on oversimplification.
So rather than measure at the operational level, we opted for the concept level. Using a Key Skills Questionnaire, participants were asked to rate themselves against the key principles covered by the time management training. These included the ability to plan, prioritise and schedule each day, spend more time on the important rather than the urgent, manage interruptions, reduce forgetfulness, to say ‘no’ in a non-career-threatening way, procrastinate less, decide what is personally important, improve the work/life balance and influence the organisation’s time culture. The instrument was designed to be quick and easy to complete, comprehensive in its coverage and able to be adapted to slightly varying training needs. For instance managing interruptions was not relevant to five sales teams but achieving monthly objectives was. Seven to fifteen measures were therefore used according to the specific needs and duration of each course.

On data collection, most surveys offered either quantitative or qualitative data but few combined the two. We sought to do both by supporting our quantitative scores with qualitative feedback from participants’ managers.

Findings
At the end of each course, participants were invited, on a voluntary basis, to complete their scores on a 1 (low) to 7 (high) rating and again some months after the training. Of the 134 self-reported ‘before and after’ scores, 75 were triangulated through feedback with participants’ managers. 46% agreed and 49% tended to agree with the scores. The remaining 5% who disagreed were equally split between concerns on under and over scoring. Managers were also given the opportunity to provide examples of behaviour changes to support the quantitative findings. This was further supported by two in-depth face-to-face interviews with the managers of two especially high and low scoring courses.

With this methodology, all 19 courses showed a percentage performance gain. The lowest-scoring course averaged an 11% improvement, the highest 48%. Individual improvements varied widely with 89% recording some improvement, 7% no change and 4% showing worse scores. We believe that this is consistent with other research on non-training factors affecting training outcomes. These include personal, psychological and organisational issues, some of which have also been discussed on this website.

In summary, the quantitative and qualitative evidence collected from this survey does suggests that in contrast to many earlier studies, whilst time management training is affected by organisational context and personal motivation, ‘fourth generation training’ does have a positive impact for the majority of participants.

Sources
Green, P. (2004). Managing time: Loving every minute (2nd ed.). Cookham, UK: Chartered Institute of Marketing.
Green, P., & Skinner, D. (2005). Does time management training work? An evaluation. International Journal of Training and Development, 9(2), 124–139.
Contact
Peter Green: [email protected] and www.bettertimemanagement.co.uk
Denise Skinner: [email protected] and http://wwwm.coventry.ac.uk/researchnet/cucv/Pages/Profile.aspx?profileID=527

 

I have created a new page on this website that shows the support that I have found in academic research for the principles that are used here at 2Time Labs. It’s a page that I intend to update whenever I find new sources that speak to these principles (either positively or negatively.)

Click here -Research Supporting Time Management 2.0

I have been catching up on my time management research by looking at some of the few published articles in the field. In some cases, I am re-reading papers that I first saw several years ago with a more informed, mature view than I had in the past.

That’s what I like to think anyway!

A few things are jumping out that never hit me before, that makes me yearn for the establishment of a “School of Time Management” at good institution.

1. Where are the Engineers?

The articles that I am looking at are published from departments of psychology in the main, and are written with a great deal of concern for the individual’s opinion about their time management skills. While I am sympathetic to this point of view, I am doubtful that, by itself, it makes much of a difference. Your opinion about your skills might be interesting to know, but it’s unlikely to produce a breakthrough in performance that can be generalized to all professionals.

It might be my bias as an engineer, as there is something impersonal about someone managing time demands, from my perspective. It’s a bit like shoving parts down an assembly line. Your opinion about the capacity of the line might be interesting, but there is something empirical happening on the line, and it cannot be ignored when we are trying to improve the way the line is balanced. I simply can’t find this kind of research anywhere as it pertains to time management. The engineers have left the building.

2. Everyone is a Time Management User

I noticed this underlying idea popping up in the research: “some people use time management and some don’t.” The researchers then attempt to make a comparison between the two groups in order to show that time management is useful, or not.

This mental model strikes me as fundamentally flawed.

Instead, my research tells me that every functioning adult engages in time management behaviors. If someone can understand what it means to “show up at 2:30pm” then they are engaging in these behaviors, even if they never actually make the appointment.

What varies are the levels of skill, and here at 2Time Labs we have distinguished these skills into Belt levels: white, yellow, orange and green. But it’s a mistake to assume that working professionals don’t do anything. Instead, I observe them acting in ways that are perfectly consistent with their established habit patterns developed over many years, even when they are unaware of them.

This mistake leads to strange outcomes in the research as the comparisons end up being invalid.

3. Training Makes (Almost) No Difference

This finding is probably based on the problems with the two points listed above. If you are not measuring the right things (opinions vs. capacity) and you are comparing apples to apples without knowing it, then it’s not likely that the research will yield too many answers that are actually useful.

As we’d say in business, the “so what” is sadly missing (in the research.)

This doesn’t help the trainers, writers, coaches, professional organizers, etc. who are looking for something that they can use to help people deal with real everyday problems. The research shows correlations between time management training/skills and feel-good constructs like “job satisfaction” but there is little research on the impact of time management on individual capacity or job performance, and the little that exists is mixed.

Obviously, something does make a real difference in the minds of people who are in this field, but it’s too bad that so very little research that’s completed actually sheds any light on what that might be.

And to think… this skill is one that every single working professional in the world uses!

Sources: A Review of the Time Management Literature by Claessens, van Eerde and Rutte.