Climbing the STAIRs: Assessing students’ social scientific reasoning skills
Keywords:
social science education, formative assessment, assessment design, social scientific reasoning, causal analysis, civic educationAbstract
Highlights:
- Assessing complex skills in secondary school teaching practice is considered challenging
- We developed items (STAIRs) to formatively assess students’ social scientific reasoning
- STAIRs were validated by experts, teachers, think-aloud interviews, and test administration
- STAIRs elicited students’ reasoning about social problems in three proficiency levels
- The design principles may be applied by teachers in the development of assessment items
Purpose: Assessing complex skills is considered important but challenging. This study focused on developing assessment items to evaluate secondary social science students’ proficiency in the subskill of causal analysis.
Design/methodology/approach: Based on a conceptual framework of social scientific reasoning, we designed formative assessment items known as STAIRs (Social science Teaching Assessment Items of Reasoning). The STAIRs were validated in three focus groups: two groups of assessment experts (N = 7 and N = 3) and one group of social science teachers (N = 10). Additionally, think-aloud interviews were conducted with eight social science students. The quality of the STAIRs was evaluated by administering the items to 338 social science students in 21 Dutch social science classes.
Findings: The results showed that it is possible to distinguish between the three performance levels in students’ reasoning using the STAIRs.
Practical implications: The design principles for the STAIRs may aid teachers in developing additional assessment items.
References
Abrami, P. C., Bernard, R. M., Borokhovski, E., Wade, A., Surkes, M. A., Tamim, R., & Zhang, D. (2008). Instructional interventions affecting critical thinking skills and dispositions: A stage 1 meta-analysis. Review of Educational Research, 78(4), 1102–1134. https://doi.org/10.3102/0034654308326084
Amani, J., Kitta, S., Kapinga, O. S., & Mbilinyi, C. (2021). Secondary school teachers’ knowledge on procedures for constructing quality classroom tests in Tanzania. Üniversitepark Bülten, 10(1), 40–54. https://dx.doi.org/10.22521/unibulletin.2021.101.3
Amin, M. E. K., Nørgaard, L. S., Cavaco, A. M., Witry, M. J., Hillman, L., Cernasev, A., & Desselle, S. P. (2020). Establishing trustworthiness and authenticity in qualitative pharmacy research. Research in Social and Administrative Pharmacy, 16(10), 1472–1482. https://doi.org/10.1016/j.sapharm.2020.02.005
Bijsterbosch, H. (2018). Professional development of geography teachers with regard to summative assessment practices [Doctoral dissertation, Utrecht University]. Utrecht University Reposito-ry. https://dspace.library.uu.nl/handle/1874/364154
Bransford, J. D., Brown, A. L., & Cocking, R. R. (Eds.). (2000). How people learn: Brain, mind, experi-ence, and school. National Academy Press.
Breakstone, J. (2014). Try, try, try again: The process of designing new history assessments. The-ory & Research in Social Education, 42(4), 453–485. https://doi.org/10.1080/00933104.2014.965860
Breakstone, J., Smith, M., & Wineburg, S. (2013). Beyond the bubble in history/social studies as-sessments. Phi Delta Kappan, 94(5), 53–57. https://doi.org/10.1177/003172171309400512
Brookhart, S. M. (2010). How to assess higher-order thinking skills in your classroom. Ascd.
Brookhart, S. M., & Durkin, D. T. (2003). Classroom assessment, student motivation, and achievement in high school social studies classes. Applied Measurement in Education, 16(1), 27–54. https://psycnet.apa.org/doi/10.1207/S15324818AME1601_2
Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educa-tional Researcher, 18(1), 32–42. https://doi.org/10.3102/0013189X018001032
Campbell, C. (2013). Research on teacher competency in classroom assessment. In J. H. McMillan (Ed.), SAGE handbook of research on classroom assessment (pp. 71–84). SAGE. https://doi.org/10.4135/9781452218649.n5
College van Toetsen en Examens. (2019). Syllabus Maatschappijwetenschappen VWO. Centraal Examen 2022. Versie 2, juli 2020 [Syllabus Social Sciences Education. Central exam. Version 2, July, 2020]. College van Toetsen en Examens.
Cooper, A., Klinger, D. A., & McAdie, P. (2017). What do teachers need? An exploration of evi-dence-informed practice for classroom assessment in Ontario. Educational Research, 59(2), 190–208. https://doi.org/10.1080/00131881.2017.1310392
Curry, K., & Smith, D. (2017). Assessment practices in social studies classrooms: Results from a longitudinal survey. Social Studies Research and Practice, 12(2), 168–181. https://doi.org/10.1108/SSRP-04-2017-0015
Douglas, M., Wilson, J., & Ennis, S. (2012). Multiple-choice question tests: A convenient, flexible and effective learning tool? A case study. Innovations in Education and Teaching Internation-al, 49(2), 111–121. https://doi.org/10.1080/14703297.2012.677596
Ercikan, K., & Seixas, P. C. (Eds.). (2015). New directions in assessing historical thinking. Routledge. https://doi.org/10.4324/9781315779539
Ercikan, K., Seixas, P., Kaliski, P., & Huff, K. (2016). Assessment of history learning. In H. Braun (Ed.), Meeting the challenges of measurement in an era of accountability (pp. 236–267). Routledge. http://dx.doi.org/10.4324/9780203781302-12
Guba, E. G. (1981). Criteria for assessing the trustworthiness of naturalistic inquiries. ECTJ, 29(2), 75–91. https://doi.org/10.1007/BF02766777
Hayes, A. F., & Krippendorff, K. (2007). Answering the call for a standard reliability measure for coding data. Communication Methods and Measures, 1(1), 77–89. https://doi.org/10.1080/19312450709336664
Jansson, T. (2023). Civics teachers’ assessment practices in Swedish upper secondary schools. A qualitative study. JSSE - Journal of Social Science Education, 22(2). https://doi.org/10.11576/jsse-5938
Jonassen, D.H., & Ionas, I.G. (2008). Designing effective supports for causal reasoning. Education-al Technology Research and Development, 56, 287-308. https://doi.org/10.1007/s11423-006-9021-6
Klijnstra, T., Stoel, G. L., Ruijs, G. J., Savenije, G. M., & van Boxtel, C. A. M. (2023). Toward a frame-work for assessing the quality of students’ social scientific reasoning. Theory & Research in So-cial Education, 51(2), 173–200. https://doi.org/10.1080/00933104.2022.2132894
Krippendorff, K. (2004). Content analysis: An introduction to its methodology (2nd ed.). SAGE. https://doi.org/10.4135/9781071878781
Lee, C. D., White, G., & Dong, D. (2021). Educating for civic reasoning and discourse. National Acad-emy of Education.
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. In. SAGE. http://dx.doi.org/10.1016/0147-1767(85)90062-8
Liu, Q., Wald, N., Daskon, C., & Harland, T. (2023). Multiple-choice questions (MCQs) for higher-order cognition: Perspectives of university teachers. Innovations in Education and Teaching In-ternational, 61(4), 802–814. https://doi.org/10.1080/14703297.2023.2222715
Löfström, J., & Ouakrim-Soivio, N. (2022), Politics and ethics of civic and citizenship education curricula in Denmark, Finland, Iceland, Norway and Sweden. In R. Desjardins & S. Wiksten (Eds.), Handbook of civic engagement and education (pp. 182–190). Edward Elgar. https://doi.org/10.4337/9781800376953.00025
Löfström, J., Rosenlund, D., & Weber, B. (2023). Assessment and national exams in social studies and social sciences. JSSE - Journal of Social Science Education, 22(2). https://doi.org/10.11576/jsse-6594
Lombardi, D., Sibley, B., & Carroll, K. (2013). What’s the alternative? Using model-evidence link diagrams to weigh alternative models in argumentation. The Science Teacher, 80(5), 36–41. http://dx.doi.org/10.2505/4/tst13_080_05_50
Maddox, L. E., & Saye, J. W. (2017). Using hybrid assessments to develop civic competency in his-tory. The Social Studies, 108(2), 55–71. https://doi.org/10.1080/00377996.2017.1283288
Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13–23. https://doi.org/10.3102/0013189X023002013
Mills, C. W. (2000). The sociological imagination. Oxford University Press. (Original work pub-lished in 1959)
Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2003). A brief introduction to evidence‐centered de-sign. ETS Research Report Series, 2003(1), i–29. https://doi.org/10.1002/j.2333-8504.2003.tb01908.x
Moss, P. A. (2013). Validity in action: Lessons from studies of data use. Journal of Educational Measurement, 50(1), 91–98. https://doi.org/10.1111/jedm.12003
Newmann, F., Carmichael, D., & King, M. (2016). Authentic intellectual work. Corwin. https://doi.org/10.4135/9781506322308
Olgers, T., Meijs, L., & Dogterom, K. (2021). Een korte geschiedenis van maatschappijleer [A brief history of social science education]. In R. van den Boorn (Ed.), Handboek vakdidactiek Maatschappijleer (pp. 131–167). Landelijk Expertisecentrum Mens- en Maatschappijvakken.
Pellegrino, J. W., Chudowsky, N., & Glaser, R. (2001). Knowing what students know: The science and design of educational assessment. National Academy Press. https://doi.org/10.17226/10019
Reich, G. A. (2009). Testing historical knowledge: Standards, multiple-choice questions and stu-dent reasoning. Theory and Research in Social Education, 37, 325–360. https://doi.org/10.1080/00933104.2009.10473401
Rosenberg, S., Ward, D., & Chilton, S. (1988). Political reasoning and cognition: A Piagetian view. Duke University Press. https://doi.org/10.1215/9780822381525
Ruijs, G. L., & Klijnstra, T. (2017, February 2). Denkvaardigheden bij maatschappijwetenschappen [Thinking skills in social science education]. Paper presented at National Conference Renewed Dutch Social Science Program, Utrecht, The Netherlands.
Ruijs, G. L., & Klijnstra, T. (2021). Hogere denkvaardigheden: denkgereedschap voor maatschappijleer [Higher thinking skills: thinking tools for social science education]. In R. van den Boorn (Ed.), Handboek vakdidactiek Maatschappijleer (pp. 169–205). Landelijke Expertise-centrum Mens- en Maatschappijvakken.
Sandahl, J. (2015). Preparing for citizenship: The value of second-order concepts in social science education. JSSE - Journal of Social Science Education, 14(1), 18–29. https://doi.org/10.4119/jsse-732
Sandahl, J. (2020). Opening up the echo chamber: Perspective taking in social science education. Acta Didactica Norden, 14(4), 24-sider. https://doi.org/10.5617/adno.8350
Schmeiser, C. B., & Welch, C. J. (2006). Test development. Educational measurement, 4, 307–353. https://doi.org/10.1016/j.pse.2014.11.006
Schraw, G., & Robinson, D. H. (Eds.). (2011). Assessment of higher order thinking skills. Infor-mation Age Publishing.
Seixas, P., Morton, T., Colyer, J., & Fornazzari, S. (2013). The big six: Historical thinking concepts. Nelson Education.
Sluijsmans, L. (2013). Evaluatie van het vernieuwde examenprogramma maatschappijwetenschap-pen voor havo: Pilot 2010-2013 [Evaluation of the renewed examination program for social sciences for pre-university secondary education: pilot 2010-2013]. SLO. https://www.slo.nl/@4250/evaluatie-vernieuwde-0/
Sluijsmans, L. (2014). Evaluatie van het vernieuwde examenprogramma maatschappijweten-schappen voor vwo: pilot 2010-2014 [Evaluation of the renewed examination program for social sciences for pre-university education: pilot 2010-2014]. SLO. https://www.slo.nl/@4248/evaluatie-vernieuwde/
Smith, M., & Breakstone, J. (2015). History assessments of thinking: An investigation of cognitive validity. In K. Ercikan & Peter Seixas (Eds.), New directions in assessing historical thinking (pp. 233–245). Routledge.
Stitzlein, S. M. (2021). Defining and implementing civic reasoning and discourse: Philosophical and moral foundations for research and practice. In C. D. Lee, G. White, & D. Dong (Eds.), Edu-cating for civic reasoning and discourse (pp. 23–52). National Academy of Education. https://doi.org/10.31094/2021/2
Stoel, G. L., van Drie, J. P., & van Boxtel, C. A. (2015). Teaching towards historical expertise: Devel-oping a pedagogy for fostering causal reasoning in history. Journal of Curriculum Studies, 47(1), 49–76. https://doi.org/10.1080/00220272.2014.968212
Ultee, W. C., Arts, W. A., & Flap, H. D. (2003). Sociologie: Vragen, uitspraken, bevindingen [Sociology: questions, statements, findings] (3rd ed.). Wolters-Noordhoff.
Van Berkel, H., Bax, A., & Joosten-ten Brinke, D. (2017). Toetsen in het hoger onderwijs [Tests in higher education] (4th ed.). Bohn Stafleu van Loghum. https://doi.org/10.1007/978-90-368-1679-3.
Van Boxtel, C., Hemker, A., Klijnstra, T., & Ruijs, G. (2017). Toetsen van denkvaardigheden en conceptuele kennis bij maatschappijwetenschappen [Assessing thinking skills and conceptual knowledge in social science education]. Landelijk Expertisecentrum Mens- en Maatschappijvakken. https://shorturl.at/9uVd5
Van Boxtel, C., & van Drie, J. (2018). Historical reasoning: Conceptualizations and educational applications. In S. A. Metzger & L. M. Harris (Eds.), The Wiley international handbook of history teaching and learning (pp. 149–176). John Wiley & Sons. https://doi.org/10.1002/9781119100812.ch6
Van Tubergen, F. (2020). Introduction to sociology. Routledge. https://doi.org/10.4324/9781351134958
Wiliam, D., & Leahy, S. (2015). Embedding formative assessment: Practical techniques for k-12 classrooms. Learning Sciences International.
Wineburg, S. (2004). Crazy for history. Journal of American History, 90, 1401–1414. https://doi.org/10.2307/3660360
Woerdman, E. (2013). Politiek en politicologie [Politics and political science]. Noordhoff.
Young, K. M., & Leinhardt, G. (1998). Writing from primary documents: A way of knowing in his-tory. Written communication, 15(1), 25–68. https://doi.org/10.1177/0741088398015001002
Zegota, S., Becker, T., Hagmayer, Y., & Raupach, T. (2022). Using item response theory to appraise key feature examinations for clinical reasoning. Medical Teacher, 44(11), 1253¬–1259. https://doi.org/10.1080/0142159X.2022.2077716
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 JSSE - Journal of Social Science Education
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
