2007 Reaffirmation Teams
3.5.1 - 2007 Reaffirmation Teams
The institution identifies college-level competencies within the general education component core and provides evidence that graduates have attained those competencies.
The University of Texas at Dallas (UT Dallas) has identified college-level competencies within the general education core curriculum and annually collects evidence that graduates have attained those competencies. The content of the core curriculum has been continually developed and refined since 1998, as have the methods used to evaluate the attainment of the component competencies.Brief History of General Education at UT Dallas
Prior to 1990, UT Dallas was not authorized to offer lower-division undergraduate courses. All students transferring to UT Dallas were required to have completed 60 semester credit hours (SCH) that included general education breadth courses. When UT Dallas began admitting freshman and sophomore students in 1990, a new set of general education courses (45 hours) addressing nine component areas was developed. In 1998, the Texas Legislature passed Senate Bill 148  which mandated that the Texas Higher Education Coordinating Board (THECB) develop new state-wide rules governing general education core curriculum at all state-funded institutions of higher education. Chapter 4 of the THECB rules  mandated that all institutions require at least 36 hours of course work in the following areas: six hours of communication, three hours of math, six hours of natural sciences, three hours of fine arts, three hours of humanities, six hours of history, six hours of political science, and three hours of social/behavioral science. The THECB also required that institutions specify six to 12 additional hours of course work from these eight component areas to create a core curriculum of at least 42 hours but no greater than 48 hours.
The state mandate precipitated an in-depth review and overhaul of the UT Dallas core curriculum beginning in October 1998. The Academic Senate appointed an ad hoc committee of faculty that met extensively over the course of the year. The committee gathered input from the general faculty, administrators, and state officials and wrote a report  that made two sets of recommendations: a) a policy document  that created a standing Committee on the Core Curriculum (CCC)  and a set of procedures for defining the components of the core curriculum and for reviewing/approving courses to satisfy each component, and b) a definition of UT Dallas’ 42-hour core curriculum and a list of courses that had been reviewed and approved to satisfy each of the eight components . The Academic Senate and the provost approved the policy document and the course list which were subsequently forwarded to and approved by the THECB. The course approval process required that courses be approved by the standing CCC on a two-year basis, which coincided with UT Dallas’ biennial catalog update. The CCC review process drew on application information provided by program or department heads and required that a course syllabus be supplied as part of the application . These procedures continue today.
In keeping with the guidelines articulated in THECB’s Core Curriculum: Assumptions and Defining Characteristics , the aims of UT Dallas’ core curriculum were to ensure that graduates: a) communicate effectively in clear and correct prose in a style appropriate to the subject, occasion, and audience; b) be able to apply basic mathematical tools in the solution of real-world problems; c) understand and evaluate relationships in the natural sciences, and to understand the basis for building and testing theories; d) appreciate the human condition and human cultures, especially in relation to behaviors, ideas, and values expressed in works of human imagination and thought, and e) understand how social and behavioral scientists discover, describe, and explain the behaviors and interactions among individuals, groups, institutions, events, and ideas.
To achieve these learning objectives, all students graduating from UT Dallas are required to enroll in courses in eight core component areas : a) six hours of Rhetoric and Composition, b) six hours of Mathematics and Quantitative Analysis, c) nine hours of Natural Science, d) three hours of Humanities, e) three hours of Fine Arts, f) six hours of U.S. History, g) six hours of Texas and U.S. Government, and h) three hours of Social/Behavioral Science. In the cases of introductory Rhetoric and Texas/U.S. Government, all students are required to complete the same courses. For all other components, students select courses from a limited menu of courses that have been reviewed and approved by the CCC in consultation with the faculty and Academic Senate. By allowing students a choice, the core curriculum cultivated the growth of a healthy range of individual viewpoints on science, culture, and society.Evaluation of the Core Curriculum, 1999-2005
Prior to 2006, instructors and program heads were asked to monitor student progress and performance in courses and to make improvements to curricula where they deemed it appropriate. The CCC also monitored the core curriculum and issued annual reports to the Academic Senate; the initial and most recent reports are provided as examples  . No formal process of assessing student attainment of learning objectives at the level of individual courses existed (which was fully implemented in 2006 as a result of the 1999-2005 core curriculum evaluation report discussed below ); however, UT Dallas did undertake several evaluations of student levels of mastery in math, writing, and critical thinking.
Mathematics: In fall 2001, a committee of faculty and administrators was formed to evaluate student performance in college algebra and statistics courses. Students graduating from UT Dallas are expected at a minimum to master the formal principles of college-level algebra and one advanced field of mathematics beyond college algebra, the minimum being inferential statistics. The ad hoc committee developed specific benchmark assessment items (based on the THECB’s Exemplary Educational Objectives ) that were administered to a random sample of students (N = 75) who had taken all their mathematics courses at UT Dallas. The committee concluded in their 2003 report that “the results from our spring 2003 mathematics evaluation of liberal arts and social and behavioral science students seems a satisfactory indication of these students’ mastery of the broad educational objectives in mathematics set down by the UT System .”
In the spring of 2006, a Math Focus Group committee was formed to evaluate student performance in introductory calculus courses. The committee included faculty and administrators from the Math, Engineering/Computer Science, Management, and Natural Science programs. The committee was charged with evaluating the rates of performance in calculus courses and devising a plan for improving student learning of calculus. The committee gathered considerable data and compared trends within UT Dallas to those at other major universities . The committee concluded that a) the rates of students earning grades of “D,” “F,” or “W” (the DFW rate) from courses for first-time freshmen was comparable to other tier-one schools (roughly 25%), but b) since UT Dallas enrolls more transfer students, DFW rates for all students taking calculus courses were higher than comparison universities (roughly 38% at UT Dallas compared to 33% at UT Austin). In addition, student performance in UT Dallas’ pre-calculus courses was not a good predictor of subsequent performance in calculus and therefore did not appear to provide adequate preparation for success in calculus. The committee made several recommendations for improving screening with the SAT II mathematics tests (including increasing the threshold score for testing out of a class) and more closely monitoring and prescribing mandatory student attendance . The material from this report and other work by the Math Focus group serves as part of the foundational data for UT Dallas’ Quality Enhancement Plan, “Gateways to Excellence in Math and Science” (GEMS) .
Writing: In spring 2002 and spring 2004, the CCC evaluated student writing in the context of Rhetoric 1302, which is required of all UT Dallas undergraduate students . Writing samples (N =34) were gathered from 10% of students enrolled in the course both semesters. A scoring rubric was developed that examined five areas of competency: rhetorical knowledge; critical thinking, reading, and writing; collaboration; research and processes, and technology. Overall, performance met expectations (65% rated four or five on a five-point scale) but indicated that there was room for improvement. The director of the rhetoric program identified a number of improvements that could be made in course curriculum and resources, including enhanced teaching assistant training, more diverse writing assignments, and maintaining restricted class sizes .
The new director of the rhetoric program issued a follow-up report in January of 2007 . The report indicated that a number of the suggested improvements from the 2003 report had been implemented with some success. The rhetoric program currently uses AT6 to assess the communications components of the core curriculum. The rhetoric program has enhanced instructor training, graded feedback to students, and assessment of program effectiveness.
Critical Thinking: Although no single component area of the core curriculum is exclusively devoted to the development of critical thinking, the enhancement of critical thinking is included as a student learning objective, explicit or implicit, for some components of the core curriculum. For the past three years, UT Dallas has participated in the Collegiate Learning Assessment (CLA) study conducted by the Council for Aid to Education. The CLA is a standardized test purported to assess students’ performance on tasks that require them to think critically, reason analytically, solve realistic problems, and write clearly. The results have shed some light on students’ general levels of critical thinking. Samples of 93 freshmen and 68 seniors at UT Dallas took part in the 2004-05 assessment  and samples of 81 freshmen and 76 seniors took part in the 2005-06 assessment . Overall, the results reveal that, compared to national norms, UT Dallas freshmen and seniors achieve very high levels of critical thinking. By their senior year, roughly 75% of UT Dallas students scored above the national 80th percentile in critical thinking. While the report also revealed that students gained less in critical thinking between freshmen and senior years than statistically expected when SAT scores were taken into consideration, this finding may be due to the lack of a longitudinal sample at UT Dallas.Report to the THECB on the Core Curriculum, 2004
THECB Rule 4.30  requires that every five years Texas colleges and universities submit a report describing efforts to evaluate their core curriculum. UT Dallas submitted its first report in fall 2004 ; the next report is due in 2009. The 2004 report described how the components of the UT Dallas core curriculum comply with Rule 4.28 and how specific courses in the core curriculum address the THECB’s Exemplary Learning Objectives. The report also summarized the findings of the assessment activities described above and concluded that the available findings indicate that the UT Dallas core curriculum was in compliance with THECB mandates. The report goes on to outline plans for how the core curriculum would be assessed in the future.Move to Course-specific Assessment System, 2005-present
The THECB 2004 report called for faculty curriculum committees to define “educational goals and objectives for their curriculum” and for the university to “hire technical staff to assist in translating these decisions into assessment procedures and methods” that would be implemented in 2006 . Thus, beginning in fall 2005, the standing CCC worked closely with the UT Dallas Center for Effective Learning and Teaching (CELT) to develop a comprehensive and regularized system of annually evaluating and improving the core curriculum at the level of individual courses. The goals were to: a) articulate clear learning objectives for each component of the core curriculum, b) have instructors identify course-specific learning objectives that address the core curriculum learning objectives, c) have instructors gather multiple types of embedded benchmark assessments to evaluate student success in meeting learning objectives, and d) have instructors submit to the CCC each semester assessment reports wherein they describe assessment methods and results, along with future plans for improving courses. The CCC’s intent was to use course assessment reports in the annual evaluation of the effectiveness of core curriculum and to formulate actions to be taken to improve it. More generally, the broader goal was to create a culture of assessment and an infrastructure that supports continuous cycles of assessment-analysis-improvement-assessment and so on extending indefinitely into the future.
Core Component Learning Objectives: In fall 2005, the CCC carefully reviewed THECB’s Exemplary Learning Objectives  for clarity and compatibility with UT Dallas’ unique mission . Based on this review, clearer and more measurable student learning objectives were created for each of the eight component areas of the core curriculum . For each component, three or four student learning objectives were specified. These objectives serve the dual purpose of guiding course learning activities and specifying the foci of assessments needed to evaluate student success in achieving the objectives.
Alignment of Core Curriculum Objective and Course-Specific Objectives: In spring 2006, instructors of all core curriculum courses were required to submit lists of student learning objectives for their courses. CELT provided instruction in how to formulate learning objectives in terms of Bloom’s taxonomy , and then CELT reviewed and provided instructors with feedback regarding the learning objectives. These lists of learning objectives were incorporated into course syllabi starting summer 2006 and were used by the CCC to evaluate whether course-specific objectives were aligned with core curriculum learning objectives. This process was repeated in fall 2006 and spring 2007 and will continue in the future.
Course Assessment Planning: Beginning in spring 2006, CELT undertook a major initiative to educate instructors in how to develop effective methods of assessing student attainment of core course learning objectives. Workbook manuals  and step-by-step instructions  were developed and made available to instructors. Numerous meetings, workshops, and individual consultations were held with instructors. The goal was for instructors to create prior to the semester’s start course assessment plans that would guide assessment activities during the term. Plan documents took the form of a table with each row representing a specific learning objective. For each objective, separate columns asked for information about: a) statement of the learning objectives, b) specific assessment methods, c) criterion of success, and d) planned timeline of assessment . During summer 2006, 73% of the instructors teaching core courses submitted course assessment plans; in fall 2006, 85% submitted plans; and in spring 2007, 91% submitted plans. Starting in fall 2006, UT Dallas released a new online assessment tool (then AT4 which has since been updated to AT6) that allowed instructors to use a web-based interface to enter Course Assessment Plans and Reports . This web-based system includes a number of features that facilitate the submission, tracking, reviewing, approval, and archiving of plans/reports .
Course Assessment Reporting: At the end of each semester instructors compile their assessment results for each learning objective in order to determine the number of students who met, partially met, or did not achieve the core curriculum’s learning objectives. These results are entered into an online core course assessment report, along with: a) copies of the actual assessments used, b) a discussion analyzing the meaning of the results, and c) proposed future actions to be taken to improve student achievement of learning objectives . The CCC reviews all reports and provides corrective feedback where necessary .
Analyzing Results and Implementing Improvement: The main goal of the initial summer 2006 assessment was to pilot test the newly devised procedures for assessment planning, data gathering, and report submission. The CCC carefully reviewed all the course assessment reports submitted by instructors and identified a number of common difficulties. There was apparent confusion in the wording of the core curriculum learning objectives for the science, writing, and social/behavioral science component areas, and therefore the CCC revised the wording of these objectives. The CCC also noted that some instructors had not gathered assessments to address every core curriculum learning objective and that some instructors relied on one method only of assessing objectives. The CCC subsequently revised its instructions to instructors to address these shortcomings and provided relevant corrective feedback to instructors during the review-revision process for the fall 2006 and spring 2007 semester.
The fall 2006 assessments yielded the first meaningful evaluation of the core curriculum using the course-specific assessment system. The CCC aggregated results across courses for each of the eight components of the core curriculum for the fall 2006 semester. The table “Fall 2006: Summary of Number of Fall 2006 Course Sections Meeting Success Criteria”  summarizes the number of courses that met, partially met, and did not meet the success criteria that had been specified. For reporting purposes, the CCC adopted fairly conservative criteria for classifying courses. To be classified as “met” criteria, 100% of a course’s outcome assessments (i.e. rows) must have met or surpassed success criteria. To be classified as “partially met,” 50% to 99% of a course’s outcome assessments must have met or surpassed success criteria. Courses that met success criteria for less than 50% of assessed outcomes were classified as “not met.” When averaged across component areas, 53% of courses had met 100% of course objectives, 29% of courses had met 50% to 99% of course objectives, and 17% of courses had met less than 50% of course objectives. Overall, 82% of courses had met or partially met their learning objectives. The CCC viewed the results as demonstrating that UT Dallas graduates attain core curriculum competencies and that there remain areas in which the core curriculum could be improved.Annual Core Curriculum Assessment Report
Each year in March the CCC produces for the Academic Senate and the provost an annual core curriculum assessment report covering the previous calendar year. The annual assessment report draws together information from the course assessment reports, course grades, course syllabi, student evaluation ratings, and other assessments such as the CLA. The purpose of the annual assessment report is to provide feedback to instructors and administrators about the past effectiveness of core curriculum and as the basis for proposed initiatives to improve the core curriculum. The annual core curriculum assessment report for 2006  was distributed to all instructors teaching courses in the core curriculum. In an effort to foster the broadest possible faculty input and participation, separate Core Component Committees were formed for each of the eight components of the core, made up of instructors who teach courses in the component. Committees were charged with reviewing and discussing the individual course assessment reports and the CCC’s annual assessment report. The committees then prepared “report cards” on their core components . The report cards included considerations of: a) instructor cooperation in assessment, b) success in achieving each learning objective, c) whether learning objectives need to be revised, d) whether assessment methods need to be revised, e) the appropriateness of instructors’ action plans, and f) general consideration of how the core component might be improved. These report cards are forwarded to the CCC where they are reviewed, incorporated into the annual assessment report, and used in formulating future initiatives by the CCC .
The annual core curriculum assessment report for 2006 proposed two main initiatives aimed at future improvements in general education. They both called for the CCC to conduct “hearings” to explore ways to improve foundation-level math and writing instruction at UT Dallas. Faculty, administrators, and students will be invited to participate in these open hearings. The goal is to formulate specific recommendations that can be implemented if possible starting in fall 2007.
The results of the course assessment reports for the math component revealed lower-than-hoped-for levels of meeting learning objectives (70%). In addition, past analyses of grades awarded in calculus courses have indicated that 30% to 40% of students attempting these courses earn grades of “D,” “F,” or “W.” Because an important portion of UT Dallas’ mission  is to graduate science, engineering, and business professionals, it is imperative that UT Dallas have effective mechanisms that lead to a greater numbers of students achieving proficiency in calculus.
The CCC noted that the course assessment reports showed a positive assessment of student writing proficiency (90% successful). The committee felt that these results ran counter to often-voiced complaints by instructors that many students lack basic writing skills. In addition, based on assessment reports and course syllabi, the CCC noted considerable unevenness across introductory and advanced writing courses in terms of: a) the amount of writing required, b) the extent and type of feedback given by instructors, and c) the likely validity of assessments. This discovery led the CCC to conclude that ways to increase the quantity and quality of writing instruction across UT Dallas should be explored. Based on the recommendations from the hearings, the CCC will work with the provost, undergraduate dean, unit heads, and instructors to make sure that approved recommendations are implemented.Student Perceptions of Their General Education
Interestingly, the National Survey of Student Engagement (NSSE) data regarding general education mirrors some of the CCC’s findings. For the 2003-2006 four-year period, about 70% of both freshmen and seniors indicate (with answers of “Quite a Bit” and “Very much”) that they believe they are “acquiring a broad general education”. Their own perception of their writing skills averages in the mid-50’s, but their critical thinking ratings average in the 70’s as do the responses for seniors regarding quantitative skills . This information, coupled with the CCC’s concern for increasing the effectiveness of instruction in writing and mathematics, provides a clear course of action for the CCC in the coming year.Transfer Students’ Core Curriculum Competencies
Students who transfer into UT Dallas often transfer courses from other colleges or universities that count toward completion of the UT Dallas core curriculum requirements. Specifically, as discussed in the response to Principle 2.7.3 , THECB Rule 4.28 (sections c-f)  requires that core curriculum courses be completely transferable among all Texas state institutions, either separately or as an entire block, and that students completing the entire core curriculum at one institution shall not be required to take additional core curriculum courses at any state institution to which they transfer. In other words, UT Dallas is required to accept these courses as fulfilling UT Dallas’ core curriculum. Similarly, courses of students transferring from non-Texas state institutions are evaluated for equivalency against a comprehensive list of Texas Common Course Number titles and catalog descriptions that has been reviewed and approved by the CCC. To be certain that transfer students-and, for that matter, all UT Dallas students-attain general education competencies and actually use them in their course of study, each undergraduate degree program has aligned its program objectives with core curriculum competencies and has, in turn, devised measures to ensure those competencies. This alignment is charted in AT6 via a series of check-boxes, and the alignment is charted in the related items column beside the description of the program objective. A sample review segment is supplied in the supporting documents for each of UT Dallas’ seven schools       .