Assessing the impact of elearning: a statistical approach to learner satisfaction.
A review of “A model for assessing the impact of e-learning systems on employees’satisfaction” Computers in Human Behavior 53 (2015) 475–485
This is a quite fascinating piece of research undertaken in Iran by Nima Jafari Navimipour and Batool Zareie (Young Researchers and Elite Club) at Tabriz Branch, Islamic Azad University, Tabriz, Iran.
It is great to see and even better to share research from differing perspectives into the use of elearning in the workplace, and we were very impressed with this research.
This research, conducted in a telecoms business, has provided us with both a statistical model and a fascinating insight into what makes a satisfied e-learner.
It is perhaps dangerous to see the list in a simple manner, and we are sure that L & D professionals all over the world will already recognise the indicators that will deliver more or less success (or indeed failure) of an e-learning project in general and within their own organisation.
However, with some blog license we put forward in rank order what the research told us:
So if you are looking to ensure high levels of employee satisfaction these should be the order of your priorities:
1. Motivation
Ensure your learners are motivated to learn using e-learning
…this means incentives, rewards and recognition, including financial benefits from career progression
2. Quality Content
Ensure your e-learning content is of the required quality
…this means good well designed e-learning, flexibility of access and social sharing and collaboration
3. Skills & Attitude
Ensure your learners have the right skills and attitude to learn online
…this means the right skills to use online learning environments, with the self confidence
4. The LMS
Ensure your learning platform (LMS) works
…this means an LMS with support, with an engaging user interface (UI) that is reliable and available
All these components are of course needed for an online learning project, but this research ranks statistically what the learners themselves believe to be the most important, leading to satisfied learners.
Do we sometimes place too much priority on the ‘e‘ in eLearning, focusing on the technology disproportionately to the higher importance of learner motivation and skills, and quality content?
More Detail on the Research
The model is built based on a comprehensive literature review to produce a series of indicators of measurement against which employee satisfaction with e-learning can be measured which we paraphrase in more detail below:
The indicators
There are 4 high level indicators, each made up of a series of subsets:
- The indicators to measure the educational technology:
- Learning system quality
- Availability of the system
- Knowledge use of the system
- Technical support system
- User friendliness
By this we see measuring the LMS, its performance, accessibility and support. We have long seen learner support as a key enabler at the commencement of any e-learning project roll out.
- The indicators to measure the educational content (e-learning):
- Content quality
- Content quality of learning
- Spatial and temporal flexibility
- Effectiveness of content
- Working group
Learning Light undertakes specific e-learning course content evaluations against a set of metrics for quality using our e-LCET approach so were very interested in this research indeed as we put high store on the quality of e-learning content.
The accessibility at a time and location of the learners choosing (spatial and temporal flexibility in the terminology of this study) is an important factor – (not strictly pure play “mobile learning” as this term was not referenced), effectiveness is a key indicator…how well did I improve my actual performance is key and the Working group refers to opportunities of collaboration amongst learners to share and collaborate – social learning by any other name…perhaps.
- The indicators to measure the innovation and motivation:
- Organizational promotion
- Internal knowledge to promote
- Financial motives
The importance to the individual is related to the importance given by the organisation to the e-learning and the recognition (and reward mechanism) for completion is important as well as the “internal motivation” – that is the individuals intrinsic desire to learn new things. The financial motives again refer to the individuals desire to progress in their career and earn more money in this model, not the opportunity for the organisation to save money!
- Indicators to measure the attitude of the learner:
- Internet use skills
- Personal experience
- Self – confidence and anxiety
This indicator set focuses on learner attitude towards technology, skills and experience (not previous bad experiences ) and levels of self- confidence.
We would argue that there is considerable overlap with motivation of the learner and their attitude and skills. Here we can clearly see the quality of the e-learning content and the design – particularly the User Interface of the LMS are important enablers of success and will be key to overcoming anxiety.
The Research Methodology
Using these indicators a questionnaire was developed by the researchers that were deployed to 148 employees who were randomly selected from a sample of 240 people. 135 questionnaires were returned with 128 being deemed suitable for analysis, so a reasonably robust sample size was analysed from an Iranian telecommunications company, giving a statistically significant set of results as the paper makes clear in some detail.
The research highlights some interesting findings:
The most important factor in rank order of effect in the model was motivation (scoring 9.33 T-test results) of the learners this was the most important variable….
Should we surprised when too often we hear of e-learning deployments achieving only 25% utilisation amongst workforces?
The second most important factor (which scored 9.24 in the T-test very close to motivation) is educational content….
We should not be surprised that engaging and effective e-learning that is well designed is so important to e-learners, but in our view not enough effort it put into evaluating content in an objective manner when it is being procured or indeed increasingly when being curated.
Thirdly there is attitude (scoring 7.02) and skills and experience of the learner
Again, no surprises that if the learner lacks skills and experience in using technology for learning there will be issues.
Fourthly there is the educational technology – the LMS (scoring 5.63)
Surprised…well maybe but…. we would argue that a well – designed LMS that is configured to the organisations needs will disappear into the background and be less of an issue to the learner once they are familiar and confident in its usage and the quality and experience of the e- learning content will come to the fore in the learners mind.
The researchers are quick to point out that the research was limited to one organisation and (as ever with researchers) more comparable research should be undertaken to achieve greater understanding, and the scores should be treated in context of the statistical analysis…they all have a measurable effect on e-learning satisfaction scores amongst learners.
However, while the ranking is interesting the factors of importance listed above provide a very powerful model on which to build successful deployments that recognise the full set of measures from a learners satisfaction perspective and not from the all too often perspective of an LMS alone.
The full paper is available to purchase from Elsevier:
http://www.sciencedirect.com/science/article/pii/S0747563215300376
Contribute to Further Research – Quick Survey
There is another opportunity for individuals and organisations to contribute to research being undertaken, by completing a short survey, which builds on some of this work, being undertaken by Jalal Sarabadani MCP, MCSA, Researcher, Corporate English Language Instructor and Coach.
Please fill in this brief online questionnaire for Jalal – it should take just a few minutes
We will be able to share his findings with you when the work is complete.
Thank you in advance and we look forward to his next report!
