Abstract
This study examines the research methods, topics, and trends of empirical MOOC research to gain a comprehensive understanding of the MOOC phenomenon through reviewing 541 empirical MOOCs research published from 2009 to 2019. The results indicate that: (1) the majority of studies adopted quantitative research methods followed by mixed research methods and qualitative research methods, (2) the most frequently adopted data collection method was survey, followed by platform database and interviews, (3) almost half of the studies used at least two data collection methods such as survey and interview, (4) the majority of researchers adopted descriptive statistics for data analysis, followed by inferential statistics and content analysis, (5) the research topics primarily focused on students, followed by design-focused, context and impact-focused, and instructor-focused. Among these student-focused topics, learner retention, learning experience, social learning, and engagement were the most mentioned, and (6) the affiliations of the first authors of the MOOC studies were mainly from the U.S. followed by China and Spain. Implications and future research were discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bali, M. (2014). MOOC pedagogy: Gleaning good practice from existing MOOCs. MERLOT Journal of Online Learning and Teaching, 10(1), 44–56. Retrieved from https://oerknowledgecloud.org/sites/oerknowledgecloud.org/files/bali_0314.pdf
Bulfin, S., Pangrazio, L., & Selwyn, N. (2014). Making ‘MOOCs’: The construction of a new digital higher education within news media discourse. The International Review of Research in Open and Distributed Learning, 15(5). Retrieved from https://www.irrodl.org/index.php/irrodl/article/view/1856/3072
Carver, L., & Harrison, L. M. (2013). MOOCs and democratic education. Liberal Education, 99(4), 20. Retrieved from https://aacu.org/liberaleducation/2013/fall/carver-harrison.
Chuang, I., & Ho, A. D. (2016). HarvardX and MITx: Four years of open online courses--Fall 2012-Summer 2016. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2889436
Coffrin, C., Corrin, L., de Barba, P., & Kennedy, G. (2014). Visualizing patterns of student engagement and performance in MOOCs. In M. Pistilli, J. Willis, D. Koch, K. Arnold, S. Teasley, & A. Pardo (Eds.), Proceedings of the fourth international conference on learning analytics and knowledge—LAK '14 (pp. 83–92). New York: ACM Press. Doi: 10.1145/2567574.2567586.
Cooper, H. (1988). The structure of knowledge synthesis: A taxonomy of literature reviews. Knowledge in Society, 1, 104–126.
Creed-Dikeogu, G., & Clark, C. (2013). Are you MOOC-ing yet? A review for academic libraries. Kansas Library Association College and University Libraries Section Proceedings, 3(1), 9–13.
Creswell, J. W., & Plano-Clark, V. L. (2017). Designing and conducting mixed methods research (3rd ed.). Thousand Oaks, CA: Sage.
Deng, R., & Benckendorff, P. (2017). A contemporary review of research methods adopted to understand students’ and instructors’ use of massive open online courses (MOOCs). International Journal of Information and Education Technology. https://doi.org/10.18178/ijiet.2017.7.8.939.
Downes, S. (2008). Places to go: Connectivism & connective knowledge. Innovate. Journal of Online Education, 5(1), 1–6. Retrieved from https://nsuworks.nova.edu/cgi/viewcontent.cgi?article=1037&context=innovate
Ebben, M., & Murphy, J. S. (2014). Unpacking MOOC scholarly discourse: A review of nascent MOOC scholarship. Learning, Media and Technology, 39(3), 328–345. https://doi.org/10.1080/17439884.2013.878352.
Fini, A. (2009). The technological dimension of a massive open online course The case of the CCK08 course tools. The International Review of Research in Open and Distributed Learning, 10(5), 1–26. https://doi.org/10.19173/irrodl.v10i5.643.
Gašević, D., Kovanović, V., Joksimović, S., & Siemens, G. (2014). Where is research on massive open online courses headed? A data analysis of the MOOC research initiative. The International Review of Research in Open and Distributed Learning, 15(5), Retrieved from https://www.irrodl.org/index.php/irrodl/article/view/1954/3099.
Guo, S., Zhang, G., & Guo, Y. (2016). Social network analysis of 50 years of international collaboration in the research of educational technology. Journal of Educational Computing Research, 53(4), 499–518. https://doi.org/10.1177/0735633115611114.
Gupta, R., & Sambyal, N. (2013). An understanding approach towards MOOCs. International Journal of Emerging Technology and Advanced Engineering, 3(6), 312–315.
Hew, K. F., & Cheung, W. S. (2014). Students’ and instructors’ use of massive open online courses (MOOCs): Motivations and challenges. Educational Research Review, 12, 45–58. https://doi.org/10.1016/j.edurev.2014.05.001.
Hollands, F., & Kazi, A. (2019, June 3). MOOC-based credentials: What’s the value to the learner? EDUCAUSE Review. Retrieved from https://er.educause.edu/articles/2019/6/mooc-based-alternative-credentials-whats-the-value-for-the-learner
Jordan, K. (2014). Initial trends in enrolment and completion of massive open online courses. The International Review of Research in Open and Distributed Learning. https://doi.org/10.19173/irrodl.v15i1.1651.
Jacobs, A. J. (2013). Two cheers for Web U. The New York Times, 162(56113), 1–7. Retrieved from https://www.nytimes.com/2013/04/21/opinion/sunday/grading-the-mooc-university.html
Jona, K., & Naidu, S. (2014). MOOCs: Emerging research. Distance Education, 35(2), 141–144. https://doi.org/10.1080/01587919.2014.928970.
Kellogg, S. (2013). Online learning: How to make a MOOC. Nature, 499(7458), 369–371.
Kellogg, S., Booth, S., & Oliver, K. (2014). A social network perspective on peer supported learning in MOOCs for educators. The International Review of Research in Open and Distributed Learning. https://doi.org/10.19173/irrodl.v15i5.1852.
Kemp, S. (2019). Digital 2019: Global Internet use accelerates. We Are Social. Retrieved from https://wearesocial.com/blog/2019/01/digital-2019-global-internet-use-accelerates
Kennedy, J. (2014). Characteristics of massive open online courses (MOOCs): A research review, 2009–2012. Journal of Interactive Online Learning, 13(1), 1–16.
Kop, R. (2011). The challenges to connectivist learning on open online networks: Learning experiences during a massive open online course. The International Review of Research in Open and Distributed Learning, 12(3), 19–38.
Koutropoulos, A., Gallagher, M. S., Abajian, S. C., de Waard, I., Hogue, R. J., Keskin, N. Ö., & Rodriguez, C. O. (2012). Emotive vocabulary in MOOCs: Context & participant retention. European Journal of Open, Distance and E-Learning, 15(1). Retrieved from https://www.eurodl.org/?p=archives&year=2012&halfyear=1&article=507
Lee, D., Watson, S. L., & Watson, W. R. (2019). Systematic literature review on self-regulated learning in massive open online courses. Australasian Journal of Educational Technology. https://doi.org/10.14742/ajet.3749.
Liyanagunawardena, T. R., Adams, A. A., & Williams, S. A. (2013). MOOCs: A systematic study of the published literature 2008–2012. The International Review of Research in Open and Distributed Learning, 14(3), 202–227. Retrieved from https://www.irrodl.org/index.php/irrodl/article/view/1455/2531
McAuley, A., Stewart, B., Siemens, G., & Cormier, D. (2010). The MOOC model for digital practice (CC attribution) Retrieved from
McKenzie, L. (2018, October 12). EdX: From MicroMasters to online master’s degrees. Inside Higher Ed. Retrieved from https://www.insidehighered.com/news/2018/10/12/edx-launches-nine-low-cost-online-degrees
Moody, J. (2018, October 14). How MOOC-based master’s degrees may transform higher ed, Forbes. Retrieved from https://www.forbes.com/sites/joshmoody/2018/10/14/how-mooc-based-masters-degrees-may-transform-higher-ed/#8e88e634efc2
Mota, R., & Scott, D. (2014). Education for innovation and independent learning. Oxford: Elsevier.
O’Brien, T. L. (2012). Change in academic co-authorship, 1953–2003. Science, Technology, & Human Values, 37(3), 210–234. https://doi.org/10.1177/0162243911406744.
Okoli, C. (2015). A guide to conducting a standalone systematic literature review. Communications of the Association for Information Systems, 37, 879–910.
Pappano, L. (2012). The year of the MOOC. The New York Times, 2(12), 2012. Retrieved from https://edinaschools.org/cms/lib07/MN01909547/Centricity/Domain/272/The%2520Year%2520of%2520the%2520MOOC%2520NY%2520Times.
Park, Y., Jung, I., & Reeves, T. C. (2015). Learning from MOOCs: A qualitative case study from the learners' perspectives. Educational Media International, 52(2), 72–87. https://doi.org/10.1080/09523987.2015.1053286.
Paton, R. M., Fluck, A. E., & Scanlan, J. D. (2018). Engagement and retention in VET MOOCs and online courses: A systematic review of literature from 2013 to 2017. Computers & Education, 125, 191–201. https://doi.org/10.1016/j.compedu.2018.06.013.
Pérez-Sanagustín M., Hernández-Correa J., Gelmi C., Hilliger I., Rodriguez M.F. (2016) Does taking a MOOC as a complement for remedial courses have an effect on my learning outcomes? A pilot study on calculus. In K. Verbert, M. Sharples, T. Klobučar (Eds.), Adaptive and adaptable learning. EC-TEL 2016. Lecture Notes in Computer Science, vol 9891. Springer, Cham.
Pickard, L. (2019, March 4). 35+ legit master’s degrees you can now earn completely online. Class Central. https://www.class-central.com/report/mooc-based-masters-degree/
Raffaghelli, J. E., Cucchiara, S., & Persico, D. (2015). Methodological approaches in MOOC research: Retracing the myth of Proteus. British Journal of Educational Technology, 46(3), 488–509.
Reich, J. (2015). Rebooting MOOC research. Science, 347(6217), 34–35. https://doi.org/10.1126/science.1261627.
Ross, J., Sinclair, C., Knox, J., Bayne, S., & Macleod, H. (2014). Teacher experiences and academic identity: The missing components of MOOC pedagogy. Journal of Online Learning and Teaching, 10(1), 57.
Sanchez-Gordon, S., & Luján-Mora, S. (2018). Research challenges in accessible MOOCs: a systematic literature review 2008–2016. Universal Access in the Information Society, 17(4), 775–789. https://doi.org/10.1007/s10209-017-0531-2.
Schaffhauser, D. (2018, September 12). Coursera's CEO on the Evolving Meaning of 'MOOC.’ Campus Technology. Retrieved from https://campustechnology.com/articles/2018/09/12/courseras-ceo-on-the-evolving-meaning-of-mooc.aspx
Schroeder. R. (2019, March 6). The maturing MOOC. Inside Higher Ed. Retrieved from https://www.insidehighered.com/digital-learning/blogs/online-trending-now/maturing-mooc
Shah, D. (2014, December 27). Online courses raise their game: A review of MOOC stats and trends in 2014. Retrieved from https://www.classcentral.com/report/moocs-stats-and-trends-2014/
Shah, D. (2016a, December 22). 6 Biggest MOOC Trends of 2016. Class Central. Retrieved from https://www.classcentral.com/report/biggest-mooc-trends-2016/
Shah, D. (2016b, December 25). By the numbers: MOOCs in 2016. Class Central. Retrieved from https://www.class-central.com/report/mooc-stats-2016/
Shah, D. (2018a, January 17). Six tiers of monetization. Class Central. Retrieved from https://www.classcentral.com/report/six-tiers-mooc-monetization/
Shah, D. (2018b, May 21). The second wave of MOOC hype Is here, and it’s online degrees. EdSurge. Retrieved from https://www.edsurge.com/news/2018-05-21-the-second-wave-of-mooc-hype-is-here-and-it-s-online-degrees
Shah, D. (2019a, January 6). Year of MOOC-based degrees: A review of MOOC stats and trends in 2018. Class Central. Retrieved from https://www.class-central.com/report/moocs-stats-and-trends-2018/
Shah, D. (2019b, June 22). Coursera’s monetization journey: From 0 to $100+ million in revenue. Class Central. Retrieved from https://www.classcentral.com/report/coursera-monetization-revenues/
Skrypnyk, O., & Joksimović, S., Kovanović, V., Gašević, D., & Dawson, S., (2015). Roles of course facilitators, learners, and technology in the flow of information of a cMOOC. The International Review of Research in Open and Distributed Learning. https://doi.org/10.19173/irrodl.v16i3.2170.
Soffer, T., & Cohen, A. (2015). Implementation of Tel Aviv University MOOCs in academic curriculum: A pilot study. The International Review of Research in Open and Distributed Learning, 16(1), 80–97.
Tashakkori, A., & Teddlie, C. (2003). Handbook on mixed methods in the behavioral and social sciences. Thousand Oaks, CA: Sage.
The Chronicle of Higher Education Alamanac. (2017). The Almanac of Higher Education 2017–18. Washington, DC: The Chronicle of Higher Education. Retrieved from https://www.chronicle.com/article/Cumulative-Growth-in-Number-of/240707?cid=cp135.
Utah State University Library (2020), Why use peer-reviewed articles? Retrieved from https://library2.usu.edu/howto/why-use-peer-reviewed-articles
van de Oudeweetering, K., & Agirdag, O. (2018). MOOCS as accelerators of social mobility? a systematic review. Journal of Educational Technology & Society, 21(1), 1–11.
Veletsianos, G., & Shepherdson, P. (2015). Who studies MOOCs? Interdisciplinarity in MOOC research and its changes over time. The International Review of Research in Open and Distributed Learning, 16(3). Retrieved from https://www.irrodl.org/index.php/irrodl/article/view/2202/3348
Veletsianos, G., & Shepherdson, P. (2016). A systematic analysis and synthesis of the empirical MOOC literature published in 2013–2015. The International Review of Research in Open and Distributed Learning, 17(2). Retrieved from https://www.irrodl.org/index.php/irrodl/article/view/2448/3655
Watson, S. L., Watson, W. R., Yu, J. H., Alamri, H., & Mueller, C. (2017). Learner profiles of attitudinal learning in a MOOC: An explanatory sequential mixed methods study. Computers & Education, 114, 274–285. https://doi.org/10.1016/j.compedu.2017.07.005.
Yuan, L., & Powell, S. (2013). MOOCs and open education: Implications for higher education. Bolton: University of Bolton, Centre for Educational Technology, Interoperability and Standards & JISC. Retrieved from https://publications.cetis.ac.uk/wp-content/uploads/2013/03/MOOCs-and-Open-Education.pdf
Zhang, J., Perris, K., Zheng, Q., & Chen, L. (2015). Public response to “the MOOC movement” in China: Examining the time series of microblogging. The International Review of Research in Open and Distributed Learning. https://doi.org/10.19173/irrodl.v16i5.2244.
Zhang, K., Bonk, C. J., Reeves, T. C., & Reynolds, T. H. (Eds.). (2020). MOOCs and open education in the Global South: Challenges, successes, and opportunities. NY: Routledge. https://doi.org/10.4324/9780429398919.
Zhang, Y. (2013). Benefiting from MOOC. In World conference on educational multimedia. Hypermedia and Telecommunications, 2013(1), pp. 1372–1377.
Zhu, M., Sari, A., & Lee, M. M. (2018). A systematic review of research methods and topics of the empirical MOOC literature (2014–2016). The Internet and Higher Education, 37, 31–39. https://doi.org/10.1016/j.iheduc.2018.01.002.
Acknowledgements
The authors wish to express their immense gratitude to Curt Bonk, Professor of Instructional Systems Technology (IST) at Indiana University (IU), for his most valuable feedback and insightful editorial suggestions of this paper.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhu, M., Sari, A.R. & Lee, M.M. A comprehensive systematic review of MOOC research: Research techniques, topics, and trends from 2009 to 2019. Education Tech Research Dev 68, 1685–1710 (2020). https://doi.org/10.1007/s11423-020-09798-x
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11423-020-09798-x