2007 Reaffirmation Teams :: 4.1 - 2007 Reaffirmation Teams

2007 Reaffirmation Teams

4.1 - 2007 Reaffirmation Teams

The institution evaluates success with respect to student achievement, including as appropriate, consideration of course completion, state licensing examinations, and job placement rates.

Compliance Judgment

Compliance

Narrative

The University of Texas at Dallas (UT Dallas) engages in ongoing assessment of its students and programs. In so doing, UT Dallas considers a variety of relevant measures to evaluate not only program achievement but also student achievement. The university reviews course completion rates and graduation rates as a part of this evaluation, but the university also considers licensing examination results and job placement results in this review process. Because an institution’s mission statement is a crucial element in providing a basis for evaluation, UT Dallas views such measures within the context of its recently revised mission statement [1]. The evaluation of student achievement with course completion rates, licensing examinations results, and job placement rates provides evidence that UT Dallas is meeting the commitments outlined in its mission statement to produce “engaged graduates, prepared for life, work, and leadership in a constantly changing world [1].”

The assessment and evaluation of student achievement plays a central role in local discussions of the university’s success in addressing the mission, the stated goals and objectives, and the assessment of program content as stated in the response to Principle 2.7.2 [2]. Such assessment includes a number of factors, including periodic academic program reviews as discussed in the response to Principle 3.3.1.1 [3]. Annual program reviews are equally important and include a number of factors program faculty members determine important in measuring their stated program goals; a full discussion of annual program reviews, using UT Dallas’ web-based assessment tool, AT6, is available in the responses to Principle 3.3.1.1 [3]. While programs, departments, and schools may focus on more course-based or program-based measures, the university also reviews course completion rates, state licensing examinations (where appropriate), and job placement rates as additional measures of student (and program) achievement. In its simplest form, the mission of UT Dallas indicates that the university will produce engaged graduates prepared for the future that awaits them and will develop and maintain academic and research programs that will benefit not only those graduates but also the citizens of the state at large.

UT Dallas Policy Memorandum 94-III.24.63, Academic Program Review [4], outlines the basic information to be reviewed within the program review: “Typically the team will consider the undergraduate and graduate curricula and programs of instruction (including student learning outcomes), the student demand for these programs, the scholarly activity of the unit’s faculty, the unit’s facilities, the national stature and impact of the unit’s programs, the quality of its students, the market for its graduates, the level of support for the unit, the effectiveness of the unit’s leadership, and the effectiveness of the unit in furthering the university’s Affirmative Action/Equal Opportunity goals.” That the team usually considers quality of its students and market for its graduates, course completion rates, licensing examinations (where appropriate), and placement rates naturally fits within the scope of such a review. The cycle for Academic Program Review is updated annually [5].

In 2002, The University of Texas System (UT System) Board of Regents proposed a more comprehensive approach to accountability and performance review for UT System institutions [6]. This required report also serves as a response to Governor Rick Perry’s Executive Order RP 31 [7], which mandates an accountability system “to determine the effectiveness and quality of the education students receive … and to evaluate the institutions’ use of state resources.” Within this context, an annual publication of targeted measures is produced with input from each of the institutions and governing board [8] [9]. The information for UT Dallas provides a more global view of the university’s progress; work by the deans and the various school faculties provides more detailed information for informed decision making at the local level.

Apart from the cyclical program review process, the dean of graduate studies also monitors information about graduate students across the disciplines such as the progress of graduate students [10] and graduation data [11]. An accreditation review of the Master of Public Affairs program in 2002 [12] occurred outside the school’s program review cycle, but the report yielded a list of program graduates’ placements for the previous four years [13]. In a similar vein, the dean of undergraduate education monitors placement of targeted populations and publishes, in concert with the university registrar, course completion rates and grade reports [14], especially for gateway courses. Of particular interest at UT Dallas for the past two years has been the ongoing grade distribution study for math gateway courses [15], particularly in relation to UT Dallas’ proposed Quality Enhancement Plan, Gateways to Excellence in Math and Science [16]. On an annual basis, academic programs conduct an assessment of the program to address primarily curricular concerns, although some elements related to course completion rates may enter into the discussion [17] [18] [19] [20] [21]. Course completion rates, particularly among the general education (core curriculum) courses after a review by the Core Curriculum Committee [22], have been used to re-assign faculty course load, to assess which courses should be added and dropped from the core curriculum, and to determine which courses needed to be redesigned [23]. Additional information about the core curriculum can be found in the response to Principle 3.5.1 [24].

Licensing Examinations

The specific issue of licensing examinations arises in only two areas at UT Dallas: Teacher Education/Certification and Communication Sciences and Disorders (Audiology). These figures are reported annually in The University of Texas System Board of Regents Accountability and Performance Report, the most recent version produced for 2006-07 [9]. Within this report, UT Dallas reports a 100% pass rate for the Teacher Education/Certification examination and has done so for the past three years [25]. This report also displays the increase in the number of Teacher Education program completions over a ten-year period, with a net increase of nearly 50% [26]. The same document, while not addressing licensure results, shows that the audiology program at UT Dallas continues to score a U.S. News and World Report ranking of fifth in the nation [27].

According to the dean of graduate studies at UT Dallas, students graduating with the M.S. in communication disorders or the doctorate in audiology are required to pass a national examination, the PRAXIS, in speech-language pathology or audiology in order to be clinically certified by the America Speech-Language-Hearing Association (ASHA) [28]. This national examination is designed and administered by Educational Testing Service (ETS). Passing the PRAXIS is also a requirement for licensure to practice in Texas and most other states. Pass rates for graduates in the communication disorders program were 88% (81 students) in 2003-04, 93% (70 students) in 2004-05, and 97% (91 students) in 2005-06. Pass rates for Au.D. students were 100% for all three years (eight students per year). Pass rates on the PRAXIS provide a standardized external measure of student knowledge in these fields. Summary PRAXIS results separated by content area are also reported to the program annually by ETS. These data may be used to identify specific strengths and weaknesses, by content area, in the test performance of students in the program.

Course Completions

University officials focus more on course completion rates (or more accurately course non-completion- or drop/fail/withdraw-rates) for specific programs, particularly core curriculum or general education courses, and more particularly the gateway courses in mathematics and the sciences. Most often viewed as the barrier courses for some students, such courses’ completion rates provide the impetus for section scheduling and faculty assignment, as well as potential course entrance requirements, among the associate deans in undergraduate programs [29]. Student achievement in these math and science gateway courses remains a critical factor in a student’s success in an undergraduate academic program in the sciences, mathematics, and technology. Given the increasing pressure to improve persistence to graduation rates for undergraduate students, identifying ways to improve student achievement in these courses is a high priority [30] [31]. Additionally, the registrar makes available to the schools a grade distribution report each term [32]. Within the School of Engineering and Computer Sciences (ECS), the assistant dean for assessment uses grade distribution in concert with course evaluation data to provide a summary view of the progress of both program faculty and students alike.

The undergraduate dean has tracked completion rates for calculus courses in an effort to provide a better scoring rubric for determining placement in the correct first course; this resulted from early reviews that suggested placement testing would improve calculus sequence progression. The results have shown that the placement test, required of entering freshmen, has improved actual completion rates for entering freshmen [33]. The recent studies, however, focus more on curricular or other changes to provide better progression for continuing and transfer undergraduate students. One study in particular included mandatory (and checked) attendance as another factor for consideration and showed that mandatory attendance does yield a higher pass rate [34].

Job Placement Rates

The UT Dallas Career Center publishes a graduate follow-up survey, and much of that information becomes a part of the UT System Accountability and Performance Report [9], as does information provided by the Office of Strategic Planning and Analysis from each of the schools. This survey also provides useful information about employers, salaries, job titles, and schools attended. One of the striking results is that over 50% of the surveyed undergraduates and 60% of the surveyed graduates entered the full-time work force immediately after graduation, thus often contributing to the state’s economy. The large number of graduates from the School of Management (SOM) has a dramatic effect on the results. Even with the influence of the large enrollments in SOM and ECS, 20% of the undergraduates report continuing their education and 11% of the graduates report continuing their education [35]. For example, the Career Center report for 2005-06 graduates provides information about placement rates for approximately 21% of the graduates for the academic year [36] [37] [38]. As approximately 80% of the total graduating class (as surveyed) either enters the work force or continues in academic endeavors, UT Dallas graduates appear to be engaged at a high level. This phenomena is especially striking among pre-health and pre-law students, where in 2005 61% of the pre-health students were placed in medical schools (in comparison to the national average of 47%) and 79% of pre-law students were placed in law schools (in comparison to the national average of 71%) [39].

Recent surveys from the schools suggest that graduates find success in employment and/or continued educational programs. The Department of Mathematical Sciences, for example, reports the employment status of 21 of its Ph.D. recipients between 1998 and 2006, with no fewer than 12 of the graduates having entered into tenure-track faculty positions across the country at such institutions as Michigan State, University of Louisville, DePaul University, and University of North Carolina-Charlotte [40]. The graduates stand as evidence that the School of Natural Sciences and Mathematics addresses two of the three “commitments” within the mission of UT Dallas by producing graduates engaged in leadership and research in the changing world of academia. Similarly, the School of Behavioral and Brain Sciences indicates that 53% of its doctoral graduates between 2001 and 2004 entered into postdoctoral or other research positions, 20% entered into tenure-track faculty positions, and another 20% entered into clinical practice or administration in a related field [41]. These graduates also address UT Dallas’ first two commitments but also address the third commitment of benefiting the lives of Texas citizens.

The dean of graduate studies provides entering student data to the graduate associate deans to facilitate their tracking of students through graduation or other exit points. The dean indicates that of the 33 Ph.D. graduates of the Department of Molecular and Cell Biology between 1999 and 2006, 22 entered postdoctoral appointments in either academic or medical settings. Within the geosciences graduate program between 1995 and 2006, the majority of the program graduates (62%) entered private industry settings, while another 20% entered either governmental or academic settings.

Other Performance Measures

As a part of UT System, UT Dallas annually reports the characteristics of its entering freshman class, a group of talented students as evidenced by an average Scholastic Aptitude Test (SAT) score of over 1245 out of 1600 [9], the class’s retention rate to sophomore year of 82.5% [9], as well as a six-year graduation and persistence rate of 77% [9]. Such metrics help to define where UT Dallas’ students begin their academic careers, but other measurements also provide valuable information about UT Dallas’ students and their achievement in relation to the university’s mission.

The response to Principle 2.5 [42] includes information about course completion rates, use of the National Survey of Student Engagement (NSSE) data to monitor student satisfaction on a variety of topics, use of the Collegiate Learning Assessment (CLA) to compare entry and exit performance and the potential of value-added measures, and the entry and achievement data of underrepresented student populations as a measure of the university’s commitment to producing engaged graduates that more fully represent the Texas population than in years past.

The specific use of course completion rates, licensing examinations, and job placement rates indicates that UT Dallas graduates fulfill a significant part of the university mission in that they are engaged and contributing to their academic disciplines through further study or to society through their involvement in the economy as employees across numerous disciplines. Many UT Dallas doctoral graduates pursue an academic career and contribute to their fields of research through academic appointments in major research universities.

Supporting Documents

Footnote Document
[1]UT Dallas Mission Statement
PDF Document, 2 Pages, 11.70 KB (statement1017)
[2]Principle 2.7.2 - Program Content (u208)
Link to UT Dallas 2007-ccr Compliance Certification Report
[3]Principle 3.3.1.1 - Assessment of Educational Programs (u320)
Link to UT Dallas 2007-ccr Compliance Certification Report
[4]Policy Memorandum 94-III.24-63 Academic Program Review
PDF Document, 5 Pages, 91.44 KB (policy1114)
[5]Program Reviews Schedule
PDF Document, 1 Page, 10.92 KB (report1208)
[6]UT System Accountability and Performance Report Creation and Executive Order RP31
PDF Document, 2 Pages, 29.89 KB (report1196)
[7]Texas Governor Executive Order RP31 re: accountability of higher education systems and institutions - dated 20040122
PDF Document, 1 Page, 10.71 KB (policy1117)
[8]UT System Accountability and Performance Report 2005-2006 Complete Report
PDF Document, 432 Pages, 2.34 MB (report1197)
[9]UT System Accountability and Performance Report 2006-07
PDF Document, 416 Pages, 1.97 MB (report1158)
[10]Annual Doctoral Student Progress Report
PDF Document, 1 Page, 13.18 KB (form1036)
[11]Diagram Ph.D.s Awarded by UT Dallas - The Office of Graduate Studies - Web Page
PDF Document, 1 Page, 21.21 KB (diagram1107)
[12]2002 NASPAA Self Study Report - Masters of Public Affairs - Report
PDF Document, 51 Pages, 2.59 MB (report1047)
[13]Survey re: Student Services and job placement for MPA students
PDF Document, 4 Pages, 144.58 KB (survey1011)
[14]Memo from Undergraduate Dean - December 20,2006 - Undergraduate Grades
PDF Document, 10 Pages, 593.36 KB (memo1079)
[15]Math Report 06u Grade Distribution Tables
PDF Document, 11 Pages, 26.40 KB (table1009)
[16]Principle 2.12 - QEP (u216)
Link to UT Dallas 2007-ccr Compliance Certification Report
[17]Faculty Meeting Agenda and Notes, Program Assessment in Public Administration - 16 February 2007
PDF Document, 8 Pages, 37.13 KB (minutes1017)
[18]Meeting minutes for SPAU, COMD, and CSD programs - 2007-02-16
PDF Document, 2 Pages, 17.12 KB (minutes1016)
[19]Meeting Minutes Chemistry Faculty March 6,2007
PDF Document, 16 Pages, 203.78 KB (minutes1080)
[20]Meeting Minutes Science/Mathematics Education February 19,2007
PDF Document, 2 Pages, 22.80 KB (minutes1081)
[21]Meeting Minutes Psychological Sciences- PhD Program February 14, 2007
PDF Document, 2 Pages, 13.41 KB (minutes1082)
[22]POLICY MEMORANDUM 95-III.21-66 - Committee on the Core Curriculum Charge
PDF Document, 2 Pages, 65.22 KB (policy1060)
[23]2006 Annual Assessment Report Core Curriculum Committee
PDF Document, 80 Pages, 3.15 MB (report1328)
[24]Principle 3.5.1 - General Ed Competencies (u337)
Link to UT Dallas 2007-ccr Compliance Certification Report
[25]Teacher Certification Initial Pass Rates by Ethnicity at U. T. Academic Institutions, 2001-2005
PDF Document, 1 Page, 26.62 KB (table1035)
[26]Number of Initially Certified Teachers Produced by UT System Institutions, UT System, and the State of Texas 1996-2006
PDF Document, 1 Page, 30.79 KB (table1034)
[27]UT Dallas USNWR rankings of graduate programs
PDF Document, 1 Page, 26.26 KB (survey1010)
[28]PRAXIS Exam
PDF Document, 3 Pages, 25.41 KB (memo1021)
[29]Core Course Statistics 06f
PDF Document, 18 Pages, 30.15 KB (report1135)
[30]UT Dallas Compact with the UT System, FY2007 through FY2008 detailing mission, goals, and objectives
PDF Document, 18 Pages, 125.50 KB (plan1035)
[31]Work Plan for D.E. Daniel, UT Dallas, June 1, 2006 - May 31, 2007 - dated 20060911
PDF Document, 4 Pages, 847.86 KB (plan1039)
[32]Grade distribution for Spring 2007
PDF Document, 62 Pages, 287.62 KB (table1032)
[33]Math Report 06u Placement and Performance in Mathematics Gateway Classes
PDF Document, 8 Pages, 238.31 KB (report1136)
[34]Math 2417 Calculus I Follow-up and Attendance Study Report Fall 2006
PDF Document, 11 Pages, 158.28 KB (report1096)
[35]UT Dallas Career Center Graduate Surveys 2003-2005
PDF Document, 5 Pages, 29.50 KB (survey1004)
[36]UT Dallas Graduate Follow-up Survey - Class of 2005-06 Reported Job Titles - Compiled by the UT Dallas Career Center
PDF Document, 3 Pages, 10.77 KB (survey1006)
[37]UT Dallas Graduate Follow-up Survey - Class of 2005-06 Reported Employers - Compiled by the UT Dallas Career Center
PDF Document, 5 Pages, 12.82 KB (survey1007)
[38]UT Dallas Graduate Follow-up Survey - Class of 2005-06 Survey Responses - Compiled by the UT Dallas Career Center
PDF Document, 2 Pages, 12.50 KB (survey1008)
[39]Memo from Dean of Undergraduate Education - July 9, 2007 - Pre-Law and Pre-Health Data
PDF Document, 8 Pages, 259.86 KB (memo1080)
[40]Employment Status of PhD Math Sci Grad 1998-2006
PDF Document, 1 Page, 10.64 KB (survey1003)
[41]UT Dallas School of Behavioral and Brain Sciences Program Graduates 2001-2004
PDF Document, 1 Page, 8.90 KB (survey1009)
[42]Principle 2.5 - Institutional Effectiveness (u205)
Link to UT Dallas 2007-ccr Compliance Certification Report
Toggle Sidebar
(null clip target)