Sustainable development in education – automating curriculum assessment
Abstract
The perpetual need for developing a sustainable economic environment places the education policies at the foundation of social adaptability. Creating and maintaining curriculum content that meets the demands of a continuously changing society, and the challenges that such a rapid evolution put on the labour market, is one of the top priorities for any education system and institution involved in education at any level. This paper proposes a cognitive computing solution for assessing, in a programmatic manner, large corpora of curriculum content created by teachers from lower secondary education environment for Informatics instruction in Romanian schools. The result of this initiative at the national level is corpora of curricular content that must be evaluated to verify the degree to which the material meets the requirements of the national curriculum. We addressed this crucial yet tedious process by designing and implementing a solution for automating curriculum assessment through cognitive computing. The paper outlines a sustainable framework to evaluate curriculum content in an automated fashion, and for providing critical feedback timely to both content creators, and to policy makers responsible for creating economically viable and future adaptable education strategies.
First published online 05 July 2021
Keyword : education for sustainable development, curriculum content assessment automation, cognitive computing, cluster analysis, GIS technologies
How to Cite
Vințe, C., Smeureanu, I., Dârdală, M., & Reveiu, A. (2021). Sustainable development in education – automating curriculum assessment. Technological and Economic Development of Economy, 27(5), 1159-1185. https://doi.org/10.3846/tede.2021.15018
This work is licensed under a Creative Commons Attribution 4.0 International License.
References
Barendsen, E., & Steenvoorden, T. (2016). Analyzing conceptual content of international informatics curricula for secondary education. In A. Brodnik, & F. Tort (Eds.), Lecture notes in computer science (LNCS): Vol. 9973. Informatics in schools: Improvement of informatics knowledge and perception (pp. 14–27). Springer. https://doi.org/10.1007/978-3-319-46747-4_2
Barr, V., & Stephenson, C. (2011). Bringing computational thinking to K-12: What is involved and what is the role of the computer science education community? ACM Inroads, 2(1), 48–54. https://doi.org/10.1145/1929887.1929905
Barroso, R. M. R., Ferreira, F. A. F., Meidute-Kavaliauskiene, I., Banaitiene, N., Falcao, P. F., & Rosa, A. A. (2019). Analyzing the determinants of e-commerce in small and medium-sized enterprises: A cognition-driven framework. Technological and Economic Development of Economy, 25(3), 496–518. https://doi.org/10.3846/tede.2019.9386
British Department for Education. (2013). Computing programmes of study: Key stages 1 and 2. National curriculum in England. https://www.gov.uk/government/publications/national-curriculum-in-england-computing-programmes-of-study
Caspersen, M. E., Gal-Ezer, J., McGettrick, A., & Nardelli, E. (2018). Informatics for All: The strategy. ACM Europe & Informatics Europe. https://doi.org/10.1145/3185594
Chang, X., & Li, J. (2019). Business performance prediction in location-based social commerce. Expert Systems with Applications, 126, 112–123. https://doi.org/10.1016/j.eswa.2019.01.086
Cid-López, A., Hornos, M. J., Carrasco-Gónzález, R. A., & Herrera-Viedma, E. (2018). Prioritization of the launch of ICT products and services through linguistic multi-criteria decision-making. Technological and Economic Development of Economy, 24(3), 1231–1257. https://doi.org/10.3846/tede.2018.1423
Danenas, P., Skersys, T., & Butleris, R. (2020). Natural language processing-enhanced extraction of SBVR business vocabularies and business rules from UML use case diagrams. Data & Knowledge Engineering, 128, 101822. https://doi.org/10.1016/j.datak.2020.101822
Dârdală, M., Furtună, T. F., & Ioniță, C. (2019). Design and implementation of a software component for geospatial data visualization in Excel. In Proceedings of the IE 2019 International Conference (pp. 293–298). https://doi.org/10.12948/ie2019.04.22
Diana, N., Eagle, M., Stamper, J., & Koedinger, K. R. (2017). Teaching informal logical fallacy identification with a cognitive tutor. In International Conference on Artificial Intelligence in Education. Springer, Cham.
Diethelm, I., & Schaumburg, M. (2016). IT2 school development of teaching materials for CS through design thinking. In A. Brodnik, & F. Tort (Eds.), Lecture notes in computer science (LNCS): Vol. 9973. Informatics in schools: Improvement of informatics knowledge and perception (pp. 193–198). Springer. https://doi.org/10.1007/978-3-319-46747-4_16
Dowek, G. (2016, October 13–15). Elements to define a coherent curriculum for the K12 education: The example of France. In Informatics in schools improvement of informatics knowledge and perception. Proceedings of the 9th International Conference on Informatics in Schools: Situation, Evolution, and Perspectives, ISSEP 2016. Münster, Germany. Springer.
Ertmer, P. A., & Ottenbreit-Leftwich, A. T. (2010). Teacher technology change: How knowledge, confidence, beliefs, and culture intersect. Journal of Research on Technology in Education, 42(3), 255–284. https://doi.org/10.1080/15391523.2010.10782551
Fethi, I. A., & Lowther, D. L. (2010). Factors affecting technology integration in K-12 classrooms: A path model. Educational Technology Research and Development, 58(2), 137–154. https://doi.org/10.1007/s11423-009-9132-y
Flanagan, M. (2017). ArcGIS maps for office: Your maps in Excel and PowerPoint. http://proceedings.esri.com/library/userconf/fed17/papers/fed_08.pdf
Guerra, V., Kuhnt, B., & Blöchliger, I. (2012). Informatics at school – Worldwide. An international exploratory study about informatics as a subject at different school levels. University of Zurich, Switzerland. https://www.researchgate.net/publication/275031370_Informatics_at_school_-_worldwide_An_international_exploratory_study_about_informatics_as_a_subject_at_different_school_levels
Hernández-Lara, A. B., Perera-Lluna, A., & Serradell-López, E. (2019). Applying learning analytics to students’ interaction in business simulation games. The usefulness of learning analytics to know what students really learn. Computers in Human Behavior, 92, 600–612. https://doi.org/10.1016/j. chb.2018.03.001
Hurwitz, J., Kaufman, M., & Bowles, A. (2015). Cognitive computing and big data analytics. John Wiley & Sons. https://www.wiley.com/en-us/Cognitive+Computing+and+Big+Data+Analytics-p-9781118896624
Iandoli, L., Ponsiglione, C., & Zollo, G. (2010). Modelling networked cognition: A socio-computational approach. Economic Computation and Economic Cybernetics Studies and Research, 4(44), 121–141. https://www.researchgate.net/publication/292491734_Modelling_networked_cognition_A_sociocomputational_approach
Informatica în școli. (2020). Ghid pentru profesorii de clasa a V-a. www.informaticainscoli.ro
Informatics education: Europe cannot afford to miss the boat. (2013). Report of the joint Informatics Europe & ACM Europe Working Group on Informatics Education. April 2013.
International Business Machines. (2018a). IBM Cloud API Docs. Java Discovery. https://cloud.ibm.com/apidocs/discovery?code=java
International Business Machines. (2018b). IBM Cloud Docs. Watson Knowledge Studio. https://console.bluemix.net/docs/services/watson-knowledge-studio/typesystem.html#typesystem
International Business Machines. (2018c). IBM Watson Discovery API. Java Class Discovery. https://watsondeveloper-cloud.github.io/java-sdk/docs/master/com/ibm/watson/discovery/v1/Discovery.html
International Business Machines. (2020). IBM Watson Products and Services. https://www.ibm.com/watson/products-services/
Lee, J. (2013). Cross-disciplinary knowledge: desperate call from business enterprises in coming smart working era. Technological and Economic Development of Economy, 19(Supp1), S285–S303. https://doi.org/10.3846/20294913.2013.880080
Lytras, M., Visvizi, A., Damiani, E., & Mthkour, H. (2019). The cognitive computing turn in education: Prospects and application. Computers in Human Behavior, 92, 446–449. https://doi.org/10.1016/j.chb.2018.11.011
McCalla, G. (2000). The fragmentation of culture, learning, teaching and technology: Implications for the artificial intelligence in education research agenda in 2010. International Journal of Artificial Intelligence in Education, 11(2), 177–196.
Micheuz, P. (2006). Is it Computer Literacy, IT, ICT or Informatics? What is going on in Austria’s compulsory schools in the context of educational standards? http://sedici.unlp.edu.ar/bitstream/handle/10915/24357/Documento_completo.pdf%3Fsequence%3D1
Modha, D. S., Ananthanarayanan, R., Esser, S. K., Ndirango, A., Sherbondy, A. J., & Singh, R. (2015). Cognitive Computing, Communications of the ACM, 54(8), 62–71. https://dl.acm.org/doi/10.1145/1978542.1978559
Neti, C. (2018, June 26). How Watson Education, Scholastic and Edmodo are using AI to close the learning gap. https://www.ibm.com/blogs/watson/2018/06/using-ai-to-close-learning-gap/
Paes de Faria, A. C. C., Ferreira, F. A. F., Dias, P. J. V. L., & Cipi, A. (2020). A constructivist model of bank branch front-office employee evaluation: An FCM-SD-based approach. Technological and Economic Development of Economy, 26(1), 213–239. https://doi.org/10.3846/tede.2020.11883
Pereira, C., & Tikhonenko, S. (2017). Informatics education in Europe: Institutions, degrees, students, positions, salaries. Key data 2011–2016 (An Informatics Europe Report). Informatics Europe.
Serban, D., Pelau, C., & Dinca, V. M. (2019). Panel data analysis for measuring the impact of e-skills on the ecological behavior of individuals. Economic Computation and Economic Cybernetics Studies and Research, 53(1), 57–74. https://doi.org/10.24818/18423264/53.1.19.04
Sysło, M. M., & Kwiatkowska, A. B. (2015). Introducing a new computer science curriculum for all school levels in Poland. In A. Brodnik & J. Vahrenhold (Eds.), Lecture notes in computer science: Vol. 9378. ISSEP 2015: Informatics in Schools. Curricula, Competences, and Competitions (pp. 141–154). Springer. https://doi.org/10.1007/978-3-319-25396-1_13
Thurai, A. (2017). What can cognitive computing do for you? https://www.ibm.com/blogs/cloud-computing/2017/08/cognitive-computing-watson/
Webb, M., Davis, N., Bell, T., Katz, Y. J., Reynolds, N., Chambers, D. P., & Sysło, M. M. (2017). Computer science in K-12 school curricula of the 2lst century: Why, what and when? Education and Information Technologies, 22, 445–468. https://doi.org/10.1007/s10639-016-9493-x
Wenger, E. (1987). Artificial intelligence and tutoring systems: Computational and cognitive approaches to the communication of knowledge. Morgan Kaufmann Publishers. https://dl.acm.org/doi/10.5555/42185
Wilson, M. (2018). Why cloud AI is a solid bet. https://www.ibm.com/blogs/cloud-computing/2018/03/cloud-ai-solid-bet/
World Economic Forum. (2016). The future of jobs employment, skills and workforce strategy for the fourth industrial revolution. Executive Summary. http://reports.weforum.org/future-of-jobs-2016/chapter-1-the-future-of-jobs-and-skills/
Barr, V., & Stephenson, C. (2011). Bringing computational thinking to K-12: What is involved and what is the role of the computer science education community? ACM Inroads, 2(1), 48–54. https://doi.org/10.1145/1929887.1929905
Barroso, R. M. R., Ferreira, F. A. F., Meidute-Kavaliauskiene, I., Banaitiene, N., Falcao, P. F., & Rosa, A. A. (2019). Analyzing the determinants of e-commerce in small and medium-sized enterprises: A cognition-driven framework. Technological and Economic Development of Economy, 25(3), 496–518. https://doi.org/10.3846/tede.2019.9386
British Department for Education. (2013). Computing programmes of study: Key stages 1 and 2. National curriculum in England. https://www.gov.uk/government/publications/national-curriculum-in-england-computing-programmes-of-study
Caspersen, M. E., Gal-Ezer, J., McGettrick, A., & Nardelli, E. (2018). Informatics for All: The strategy. ACM Europe & Informatics Europe. https://doi.org/10.1145/3185594
Chang, X., & Li, J. (2019). Business performance prediction in location-based social commerce. Expert Systems with Applications, 126, 112–123. https://doi.org/10.1016/j.eswa.2019.01.086
Cid-López, A., Hornos, M. J., Carrasco-Gónzález, R. A., & Herrera-Viedma, E. (2018). Prioritization of the launch of ICT products and services through linguistic multi-criteria decision-making. Technological and Economic Development of Economy, 24(3), 1231–1257. https://doi.org/10.3846/tede.2018.1423
Danenas, P., Skersys, T., & Butleris, R. (2020). Natural language processing-enhanced extraction of SBVR business vocabularies and business rules from UML use case diagrams. Data & Knowledge Engineering, 128, 101822. https://doi.org/10.1016/j.datak.2020.101822
Dârdală, M., Furtună, T. F., & Ioniță, C. (2019). Design and implementation of a software component for geospatial data visualization in Excel. In Proceedings of the IE 2019 International Conference (pp. 293–298). https://doi.org/10.12948/ie2019.04.22
Diana, N., Eagle, M., Stamper, J., & Koedinger, K. R. (2017). Teaching informal logical fallacy identification with a cognitive tutor. In International Conference on Artificial Intelligence in Education. Springer, Cham.
Diethelm, I., & Schaumburg, M. (2016). IT2 school development of teaching materials for CS through design thinking. In A. Brodnik, & F. Tort (Eds.), Lecture notes in computer science (LNCS): Vol. 9973. Informatics in schools: Improvement of informatics knowledge and perception (pp. 193–198). Springer. https://doi.org/10.1007/978-3-319-46747-4_16
Dowek, G. (2016, October 13–15). Elements to define a coherent curriculum for the K12 education: The example of France. In Informatics in schools improvement of informatics knowledge and perception. Proceedings of the 9th International Conference on Informatics in Schools: Situation, Evolution, and Perspectives, ISSEP 2016. Münster, Germany. Springer.
Ertmer, P. A., & Ottenbreit-Leftwich, A. T. (2010). Teacher technology change: How knowledge, confidence, beliefs, and culture intersect. Journal of Research on Technology in Education, 42(3), 255–284. https://doi.org/10.1080/15391523.2010.10782551
Fethi, I. A., & Lowther, D. L. (2010). Factors affecting technology integration in K-12 classrooms: A path model. Educational Technology Research and Development, 58(2), 137–154. https://doi.org/10.1007/s11423-009-9132-y
Flanagan, M. (2017). ArcGIS maps for office: Your maps in Excel and PowerPoint. http://proceedings.esri.com/library/userconf/fed17/papers/fed_08.pdf
Guerra, V., Kuhnt, B., & Blöchliger, I. (2012). Informatics at school – Worldwide. An international exploratory study about informatics as a subject at different school levels. University of Zurich, Switzerland. https://www.researchgate.net/publication/275031370_Informatics_at_school_-_worldwide_An_international_exploratory_study_about_informatics_as_a_subject_at_different_school_levels
Hernández-Lara, A. B., Perera-Lluna, A., & Serradell-López, E. (2019). Applying learning analytics to students’ interaction in business simulation games. The usefulness of learning analytics to know what students really learn. Computers in Human Behavior, 92, 600–612. https://doi.org/10.1016/j. chb.2018.03.001
Hurwitz, J., Kaufman, M., & Bowles, A. (2015). Cognitive computing and big data analytics. John Wiley & Sons. https://www.wiley.com/en-us/Cognitive+Computing+and+Big+Data+Analytics-p-9781118896624
Iandoli, L., Ponsiglione, C., & Zollo, G. (2010). Modelling networked cognition: A socio-computational approach. Economic Computation and Economic Cybernetics Studies and Research, 4(44), 121–141. https://www.researchgate.net/publication/292491734_Modelling_networked_cognition_A_sociocomputational_approach
Informatica în școli. (2020). Ghid pentru profesorii de clasa a V-a. www.informaticainscoli.ro
Informatics education: Europe cannot afford to miss the boat. (2013). Report of the joint Informatics Europe & ACM Europe Working Group on Informatics Education. April 2013.
International Business Machines. (2018a). IBM Cloud API Docs. Java Discovery. https://cloud.ibm.com/apidocs/discovery?code=java
International Business Machines. (2018b). IBM Cloud Docs. Watson Knowledge Studio. https://console.bluemix.net/docs/services/watson-knowledge-studio/typesystem.html#typesystem
International Business Machines. (2018c). IBM Watson Discovery API. Java Class Discovery. https://watsondeveloper-cloud.github.io/java-sdk/docs/master/com/ibm/watson/discovery/v1/Discovery.html
International Business Machines. (2020). IBM Watson Products and Services. https://www.ibm.com/watson/products-services/
Lee, J. (2013). Cross-disciplinary knowledge: desperate call from business enterprises in coming smart working era. Technological and Economic Development of Economy, 19(Supp1), S285–S303. https://doi.org/10.3846/20294913.2013.880080
Lytras, M., Visvizi, A., Damiani, E., & Mthkour, H. (2019). The cognitive computing turn in education: Prospects and application. Computers in Human Behavior, 92, 446–449. https://doi.org/10.1016/j.chb.2018.11.011
McCalla, G. (2000). The fragmentation of culture, learning, teaching and technology: Implications for the artificial intelligence in education research agenda in 2010. International Journal of Artificial Intelligence in Education, 11(2), 177–196.
Micheuz, P. (2006). Is it Computer Literacy, IT, ICT or Informatics? What is going on in Austria’s compulsory schools in the context of educational standards? http://sedici.unlp.edu.ar/bitstream/handle/10915/24357/Documento_completo.pdf%3Fsequence%3D1
Modha, D. S., Ananthanarayanan, R., Esser, S. K., Ndirango, A., Sherbondy, A. J., & Singh, R. (2015). Cognitive Computing, Communications of the ACM, 54(8), 62–71. https://dl.acm.org/doi/10.1145/1978542.1978559
Neti, C. (2018, June 26). How Watson Education, Scholastic and Edmodo are using AI to close the learning gap. https://www.ibm.com/blogs/watson/2018/06/using-ai-to-close-learning-gap/
Paes de Faria, A. C. C., Ferreira, F. A. F., Dias, P. J. V. L., & Cipi, A. (2020). A constructivist model of bank branch front-office employee evaluation: An FCM-SD-based approach. Technological and Economic Development of Economy, 26(1), 213–239. https://doi.org/10.3846/tede.2020.11883
Pereira, C., & Tikhonenko, S. (2017). Informatics education in Europe: Institutions, degrees, students, positions, salaries. Key data 2011–2016 (An Informatics Europe Report). Informatics Europe.
Serban, D., Pelau, C., & Dinca, V. M. (2019). Panel data analysis for measuring the impact of e-skills on the ecological behavior of individuals. Economic Computation and Economic Cybernetics Studies and Research, 53(1), 57–74. https://doi.org/10.24818/18423264/53.1.19.04
Sysło, M. M., & Kwiatkowska, A. B. (2015). Introducing a new computer science curriculum for all school levels in Poland. In A. Brodnik & J. Vahrenhold (Eds.), Lecture notes in computer science: Vol. 9378. ISSEP 2015: Informatics in Schools. Curricula, Competences, and Competitions (pp. 141–154). Springer. https://doi.org/10.1007/978-3-319-25396-1_13
Thurai, A. (2017). What can cognitive computing do for you? https://www.ibm.com/blogs/cloud-computing/2017/08/cognitive-computing-watson/
Webb, M., Davis, N., Bell, T., Katz, Y. J., Reynolds, N., Chambers, D. P., & Sysło, M. M. (2017). Computer science in K-12 school curricula of the 2lst century: Why, what and when? Education and Information Technologies, 22, 445–468. https://doi.org/10.1007/s10639-016-9493-x
Wenger, E. (1987). Artificial intelligence and tutoring systems: Computational and cognitive approaches to the communication of knowledge. Morgan Kaufmann Publishers. https://dl.acm.org/doi/10.5555/42185
Wilson, M. (2018). Why cloud AI is a solid bet. https://www.ibm.com/blogs/cloud-computing/2018/03/cloud-ai-solid-bet/
World Economic Forum. (2016). The future of jobs employment, skills and workforce strategy for the fourth industrial revolution. Executive Summary. http://reports.weforum.org/future-of-jobs-2016/chapter-1-the-future-of-jobs-and-skills/