Climbing the STAIRs: Assessing students’ social scientific reasoning skills

Authors

Keywords:

social science education, formative assessment, assessment design, social scientific reasoning, causal analysis, civic education

Abstract

Highlights:

- Assessing complex skills in secondary school teaching practice is considered challenging

- We developed items (STAIRs) to formatively assess students’ social scientific reasoning

- STAIRs were validated by experts, teachers, think-aloud interviews, and test administration

- STAIRs elicited students’ reasoning about social problems in three proficiency levels

- The design principles may be applied by teachers in the development of assessment items

 

Purpose: Assessing complex skills is considered important but challenging. This study focused on developing assessment items to evaluate secondary social science students’ proficiency in the subskill of causal analysis.

Design/methodology/approach: Based on a conceptual framework of social scientific reasoning, we designed formative assessment items known as STAIRs (Social science Teaching Assessment Items of Reasoning). The STAIRs were validated in three focus groups: two groups of assessment experts (N = 7 and N = 3) and one group of social science teachers (N = 10). Additionally, think-aloud interviews were conducted with eight social science students. The quality of the STAIRs was evaluated by administering the items to 338 social science students in 21 Dutch social science classes.

Findings: The results showed that it is possible to distinguish between the three performance levels in students’ reasoning using the STAIRs.

Practical implications: The design principles for the STAIRs may aid teachers in developing additional assessment items.

References

Abrami, P. C., Bernard, R. M., Borokhovski, E., Wade, A., Surkes, M. A., Tamim, R., & Zhang, D. (2008). Instructional interventions affecting critical thinking skills and dispositions: A stage 1 meta-analysis. Review of Educational Research, 78(4), 1102–1134. https://doi.org/10.3102/0034654308326084

Amani, J., Kitta, S., Kapinga, O. S., & Mbilinyi, C. (2021). Secondary school teachers’ knowledge on procedures for constructing quality classroom tests in Tanzania. Üniversitepark Bülten, 10(1), 40–54. https://dx.doi.org/10.22521/unibulletin.2021.101.3

Amin, M. E. K., Nørgaard, L. S., Cavaco, A. M., Witry, M. J., Hillman, L., Cernasev, A., & Desselle, S. P. (2020). Establishing trustworthiness and authenticity in qualitative pharmacy research. Research in Social and Administrative Pharmacy, 16(10), 1472–1482. https://doi.org/10.1016/j.sapharm.2020.02.005

Bijsterbosch, H. (2018). Professional development of geography teachers with regard to summative assessment practices [Doctoral dissertation, Utrecht University]. Utrecht University Reposito-ry. https://dspace.library.uu.nl/handle/1874/364154

Bransford, J. D., Brown, A. L., & Cocking, R. R. (Eds.). (2000). How people learn: Brain, mind, experi-ence, and school. National Academy Press.

Breakstone, J. (2014). Try, try, try again: The process of designing new history assessments. The-ory & Research in Social Education, 42(4), 453–485. https://doi.org/10.1080/00933104.2014.965860

Breakstone, J., Smith, M., & Wineburg, S. (2013). Beyond the bubble in history/social studies as-sessments. Phi Delta Kappan, 94(5), 53–57. https://doi.org/10.1177/003172171309400512

Brookhart, S. M. (2010). How to assess higher-order thinking skills in your classroom. Ascd.

Brookhart, S. M., & Durkin, D. T. (2003). Classroom assessment, student motivation, and achievement in high school social studies classes. Applied Measurement in Education, 16(1), 27–54. https://psycnet.apa.org/doi/10.1207/S15324818AME1601_2

Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educa-tional Researcher, 18(1), 32–42. https://doi.org/10.3102/0013189X018001032

Campbell, C. (2013). Research on teacher competency in classroom assessment. In J. H. McMillan (Ed.), SAGE handbook of research on classroom assessment (pp. 71–84). SAGE. https://doi.org/10.4135/9781452218649.n5

College van Toetsen en Examens. (2019). Syllabus Maatschappijwetenschappen VWO. Centraal Examen 2022. Versie 2, juli 2020 [Syllabus Social Sciences Education. Central exam. Version 2, July, 2020]. College van Toetsen en Examens.

Cooper, A., Klinger, D. A., & McAdie, P. (2017). What do teachers need? An exploration of evi-dence-informed practice for classroom assessment in Ontario. Educational Research, 59(2), 190–208. https://doi.org/10.1080/00131881.2017.1310392

Curry, K., & Smith, D. (2017). Assessment practices in social studies classrooms: Results from a longitudinal survey. Social Studies Research and Practice, 12(2), 168–181. https://doi.org/10.1108/SSRP-04-2017-0015

Douglas, M., Wilson, J., & Ennis, S. (2012). Multiple-choice question tests: A convenient, flexible and effective learning tool? A case study. Innovations in Education and Teaching Internation-al, 49(2), 111–121. https://doi.org/10.1080/14703297.2012.677596

Ercikan, K., & Seixas, P. C. (Eds.). (2015). New directions in assessing historical thinking. Routledge. https://doi.org/10.4324/9781315779539

Ercikan, K., Seixas, P., Kaliski, P., & Huff, K. (2016). Assessment of history learning. In H. Braun (Ed.), Meeting the challenges of measurement in an era of accountability (pp. 236–267). Routledge. http://dx.doi.org/10.4324/9780203781302-12

Guba, E. G. (1981). Criteria for assessing the trustworthiness of naturalistic inquiries. ECTJ, 29(2), 75–91. https://doi.org/10.1007/BF02766777

Hayes, A. F., & Krippendorff, K. (2007). Answering the call for a standard reliability measure for coding data. Communication Methods and Measures, 1(1), 77–89. https://doi.org/10.1080/19312450709336664

Jansson, T. (2023). Civics teachers’ assessment practices in Swedish upper secondary schools. A qualitative study. JSSE - Journal of Social Science Education, 22(2). https://doi.org/10.11576/jsse-5938

Jonassen, D.H., & Ionas, I.G. (2008). Designing effective supports for causal reasoning. Education-al Technology Research and Development, 56, 287-308. https://doi.org/10.1007/s11423-006-9021-6

Klijnstra, T., Stoel, G. L., Ruijs, G. J., Savenije, G. M., & van Boxtel, C. A. M. (2023). Toward a frame-work for assessing the quality of students’ social scientific reasoning. Theory & Research in So-cial Education, 51(2), 173–200. https://doi.org/10.1080/00933104.2022.2132894

Krippendorff, K. (2004). Content analysis: An introduction to its methodology (2nd ed.). SAGE. https://doi.org/10.4135/9781071878781

Lee, C. D., White, G., & Dong, D. (2021). Educating for civic reasoning and discourse. National Acad-emy of Education.

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. In. SAGE. http://dx.doi.org/10.1016/0147-1767(85)90062-8

Liu, Q., Wald, N., Daskon, C., & Harland, T. (2023). Multiple-choice questions (MCQs) for higher-order cognition: Perspectives of university teachers. Innovations in Education and Teaching In-ternational, 61(4), 802–814. https://doi.org/10.1080/14703297.2023.2222715

Löfström, J., & Ouakrim-Soivio, N. (2022), Politics and ethics of civic and citizenship education curricula in Denmark, Finland, Iceland, Norway and Sweden. In R. Desjardins & S. Wiksten (Eds.), Handbook of civic engagement and education (pp. 182–190). Edward Elgar. https://doi.org/10.4337/9781800376953.00025

Löfström, J., Rosenlund, D., & Weber, B. (2023). Assessment and national exams in social studies and social sciences. JSSE - Journal of Social Science Education, 22(2). https://doi.org/10.11576/jsse-6594

Lombardi, D., Sibley, B., & Carroll, K. (2013). What’s the alternative? Using model-evidence link diagrams to weigh alternative models in argumentation. The Science Teacher, 80(5), 36–41. http://dx.doi.org/10.2505/4/tst13_080_05_50

Maddox, L. E., & Saye, J. W. (2017). Using hybrid assessments to develop civic competency in his-tory. The Social Studies, 108(2), 55–71. https://doi.org/10.1080/00377996.2017.1283288

Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13–23. https://doi.org/10.3102/0013189X023002013

Mills, C. W. (2000). The sociological imagination. Oxford University Press. (Original work pub-lished in 1959)

Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2003). A brief introduction to evidence‐centered de-sign. ETS Research Report Series, 2003(1), i–29. https://doi.org/10.1002/j.2333-8504.2003.tb01908.x

Moss, P. A. (2013). Validity in action: Lessons from studies of data use. Journal of Educational Measurement, 50(1), 91–98. https://doi.org/10.1111/jedm.12003

Newmann, F., Carmichael, D., & King, M. (2016). Authentic intellectual work. Corwin. https://doi.org/10.4135/9781506322308

Olgers, T., Meijs, L., & Dogterom, K. (2021). Een korte geschiedenis van maatschappijleer [A brief history of social science education]. In R. van den Boorn (Ed.), Handboek vakdidactiek Maatschappijleer (pp. 131–167). Landelijk Expertisecentrum Mens- en Maatschappijvakken.

Pellegrino, J. W., Chudowsky, N., & Glaser, R. (2001). Knowing what students know: The science and design of educational assessment. National Academy Press. https://doi.org/10.17226/10019

Reich, G. A. (2009). Testing historical knowledge: Standards, multiple-choice questions and stu-dent reasoning. Theory and Research in Social Education, 37, 325–360. https://doi.org/10.1080/00933104.2009.10473401

Rosenberg, S., Ward, D., & Chilton, S. (1988). Political reasoning and cognition: A Piagetian view. Duke University Press. https://doi.org/10.1215/9780822381525

Ruijs, G. L., & Klijnstra, T. (2017, February 2). Denkvaardigheden bij maatschappijwetenschappen [Thinking skills in social science education]. Paper presented at National Conference Renewed Dutch Social Science Program, Utrecht, The Netherlands.

Ruijs, G. L., & Klijnstra, T. (2021). Hogere denkvaardigheden: denkgereedschap voor maatschappijleer [Higher thinking skills: thinking tools for social science education]. In R. van den Boorn (Ed.), Handboek vakdidactiek Maatschappijleer (pp. 169–205). Landelijke Expertise-centrum Mens- en Maatschappijvakken.

Sandahl, J. (2015). Preparing for citizenship: The value of second-order concepts in social science education. JSSE - Journal of Social Science Education, 14(1), 18–29. https://doi.org/10.4119/jsse-732

Sandahl, J. (2020). Opening up the echo chamber: Perspective taking in social science education. Acta Didactica Norden, 14(4), 24-sider. https://doi.org/10.5617/adno.8350

Schmeiser, C. B., & Welch, C. J. (2006). Test development. Educational measurement, 4, 307–353. https://doi.org/10.1016/j.pse.2014.11.006

Schraw, G., & Robinson, D. H. (Eds.). (2011). Assessment of higher order thinking skills. Infor-mation Age Publishing.

Seixas, P., Morton, T., Colyer, J., & Fornazzari, S. (2013). The big six: Historical thinking concepts. Nelson Education.

Sluijsmans, L. (2013). Evaluatie van het vernieuwde examenprogramma maatschappijwetenschap-pen voor havo: Pilot 2010-2013 [Evaluation of the renewed examination program for social sciences for pre-university secondary education: pilot 2010-2013]. SLO. https://www.slo.nl/@4250/evaluatie-vernieuwde-0/

Sluijsmans, L. (2014). Evaluatie van het vernieuwde examenprogramma maatschappijweten-schappen voor vwo: pilot 2010-2014 [Evaluation of the renewed examination program for social sciences for pre-university education: pilot 2010-2014]. SLO. https://www.slo.nl/@4248/evaluatie-vernieuwde/

Smith, M., & Breakstone, J. (2015). History assessments of thinking: An investigation of cognitive validity. In K. Ercikan & Peter Seixas (Eds.), New directions in assessing historical thinking (pp. 233–245). Routledge.

Stitzlein, S. M. (2021). Defining and implementing civic reasoning and discourse: Philosophical and moral foundations for research and practice. In C. D. Lee, G. White, & D. Dong (Eds.), Edu-cating for civic reasoning and discourse (pp. 23–52). National Academy of Education. https://doi.org/10.31094/2021/2

Stoel, G. L., van Drie, J. P., & van Boxtel, C. A. (2015). Teaching towards historical expertise: Devel-oping a pedagogy for fostering causal reasoning in history. Journal of Curriculum Studies, 47(1), 49–76. https://doi.org/10.1080/00220272.2014.968212

Ultee, W. C., Arts, W. A., & Flap, H. D. (2003). Sociologie: Vragen, uitspraken, bevindingen [Sociology: questions, statements, findings] (3rd ed.). Wolters-Noordhoff.

Van Berkel, H., Bax, A., & Joosten-ten Brinke, D. (2017). Toetsen in het hoger onderwijs [Tests in higher education] (4th ed.). Bohn Stafleu van Loghum. https://doi.org/10.1007/978-90-368-1679-3.

Van Boxtel, C., Hemker, A., Klijnstra, T., & Ruijs, G. (2017). Toetsen van denkvaardigheden en conceptuele kennis bij maatschappijwetenschappen [Assessing thinking skills and conceptual knowledge in social science education]. Landelijk Expertisecentrum Mens- en Maatschappijvakken. https://shorturl.at/9uVd5

Van Boxtel, C., & van Drie, J. (2018). Historical reasoning: Conceptualizations and educational applications. In S. A. Metzger & L. M. Harris (Eds.), The Wiley international handbook of history teaching and learning (pp. 149–176). John Wiley & Sons. https://doi.org/10.1002/9781119100812.ch6

Van Tubergen, F. (2020). Introduction to sociology. Routledge. https://doi.org/10.4324/9781351134958

Wiliam, D., & Leahy, S. (2015). Embedding formative assessment: Practical techniques for k-12 classrooms. Learning Sciences International.

Wineburg, S. (2004). Crazy for history. Journal of American History, 90, 1401–1414. https://doi.org/10.2307/3660360

Woerdman, E. (2013). Politiek en politicologie [Politics and political science]. Noordhoff.

Young, K. M., & Leinhardt, G. (1998). Writing from primary documents: A way of knowing in his-tory. Written communication, 15(1), 25–68. https://doi.org/10.1177/0741088398015001002

Zegota, S., Becker, T., Hagmayer, Y., & Raupach, T. (2022). Using item response theory to appraise key feature examinations for clinical reasoning. Medical Teacher, 44(11), 1253¬–1259. https://doi.org/10.1080/0142159X.2022.2077716

Downloads

Metrics
Views/Downloads
  • Abstract
    35
  • PDF
    40
Further information

Published

2025-07-03

How to Cite

Klijnstra, T., Savenije, G., Huijskes, C., & van Boxtel, C. (2025). Climbing the STAIRs: Assessing students’ social scientific reasoning skills. JSSE - Journal of Social Science Education, 24(2). Retrieved from https://www.jsse.org/index.php/jsse/article/view/7938

Issue

Section

Article - Open Call