Video

Video by Mike Tate

Oral Comprehensive Exams: Lessons on the Assessment of Student Achievement of Program Learning Outcomes at the U.S. Army War College

Air University Public Affairs

Dec. 7, 2023 | 14:28

Christopher Hickey, Ph.D., Brett Weigle, Ed.D., & Matthew Woessner, Ph.D.

Strategic guidance, including The Joint Chiefs of Staff Vision and Guidance for Professional Military Education and Talent Management and Department of Defense Instruction 1322.35, Military Education, has emphasized the importance of high-quality assessment of student learning and program effectiveness. Establishing an appropriate process and criteria for measuring adult educational success has long been a critical challenge for educators (Harnett and Willingham, 1980). Research suggests that comprehensive examinations have been helpful for assessment in at least one military education institution (Collins, Welch, and Martin, 2020). The U.S. Army War College resident education program administers an individual oral comprehensive exam as one of several means to assess student achievement of six Program Learning Outcomes (PLOs).

The presentation will first review the PLOs, how the examination seeks to assess their achievement, and critical lessons learned from the exam’s evolution intended to implement best practices in assessment, including authentic assessment (Wiggins 1989, 1998). It will then discuss findings from regression analysis and a review of student and faculty surveys to address four questions. First, what is the relationship between course grades and comprehensive exam grades? Second, what is the relative strength of the six PLOs in influencing comprehensive exam grades? Third, do the exam grades provide evidence that students in specialized programs experience better outcomes in achieving the PLOs? Fourth, is there evidence of changes in overall outcomes related to changes in the curriculum? Initial work suggests that course grades and comp exam grades are correlated, that the strategic communication PLO has the most substantial impact on comprehensive exam grades, and that there is evidence of greater achievement of program learning outcomes in some specialized programs. The findings may be limited by the degree of subjectivity inherent in grading. The lessons and findings will be helpful to institutions considering options to improve assessment practices.