In my latest post If You Build It, Will They Come?, I shared how ExcelinEd created an open online learning environment to meet the various needs of our learners. We launched these courses five months ago, and today we have a greater understanding of who our learners are and their outcomes to date. As the courses remain open for enrollment and we don’t have a specific requirement for when people complete the courses, we evaluate data on a weekly basis.
Massive Open Online Courses (MOOCs) have a varied level of engagement based on the objective of the participant. Some individuals register for a course just to browse the content, others drop in to learn about one specific topic and then leave, while others are actively engaged with the intent to complete. That is one of the contributing factors to less than 10 percent completion rates for higher education courses offered through EdX and Coursera. [i]
Each type of engagement is valuable because every participant is unique in their interests and what they hope to achieve from the course. Therefore, to only measure course completions would not tell the entire story. This has created a challenging puzzle for us to solve. How do we take the data and align it with these various participant goals? How can we measure success beyond course completions?
To help us with measuring and visualizing these different users, we decided to break the overall registrants into four subsets: participants who had just signed up but have taken no action, observers, active participants and course completers. This allows us to focus on engaging participants at different steps in the engagement funnel, as well as compare the actions of the participants to the intentions they indicated on the pre-course survey. For example, of those who indicated on the pre-course survey that they intended to complete the course, 51 percent did so.
We also looked at data such as participant demographics, content views and rating, if participants would recommend the courses to a friend, time actually spent to complete the courses, and future course topics participants would like to see. Here some examples of that data:
- We have registrants from all 50 states and D.C. Of those, participants representing 40 states and D.C. have completed a course.
- We have participants from 70 countries.
- 64 percent of course completers spent 2-6 hours to complete a course.
- 96 percent of course completers said they were motivated to complete the course.
- 92 percent of course completers would recommend the course to a friend.
We are continuously use this valuable insight to explore how we can motivate our learners to complete the courses, who we are actually reaching and how we can design future courses to increase engagement and completions. If you are designing a professional development offering of your own, allow the goals to help shape desired outcomes and define the metrics upfront in order to set expectations for all stakeholders involved.
Sign up for EdPolicy Leaders Online today.
Want to hear what real-life policymakers and education leaders think of EdPolicy Leaders Online? Check out these posts:
- Rhode Island Ed Leader: What I Learned from EdPolicy Leaders Online
- EdPolicy Leaders Online: Meet Lauren Heiger
- EdPolicy Leaders Online: Meet Shelly Larson
- EdPolicy Leaders Online: Meet Tony Shivers
[i] Forbes, MOOCs—Completion is Not Important, 2014; Virtual Canuck, MOOCs and Distance Education Institutions, 2013; HarvardX, Learner Intention Recasts “Low” MOOC Completion Rates, 2014; The Chronicle of Higher Education, Coursea Takes a Nuanced View of MOOC Dropout Rates, 2013; http://blog.classesandcareers.com/education/2013/07/10/infographic-how-many-students-are-completing-free-online-courses/; http://mfeldstein.com/combining-mooc-student-patterns-graphic-stanford-analysis/
About the author
Sarah Bishop-Root serves as the MOOC Program Director (EdPolicy Leaders Online) at the Foundation for Excellence in Education Prior to joining ExcelinEd, Sarah Bishop-Root spent four years at Blackboard, focusing on educational technology adoption and emerging trends in education. As Senior Manager of Client Engagement for Blackboard, she developed and executed a MOOC and Open Education strategy. In this role she worked closely with institution decision makers and educators globally and contributed to the development and launch of Blackboard’s MOOC platform.