Evaluation Forum 2006

Geoff Scott
Pro Vice-Chancellor (Quality), University of Western Sydney, Australia
Listening to the student voice: lessons for engaging and retaining students at University

Dr Geoff Scott's specific areas of research and writing are in strategic change management in post-secondary and higher education; quality tracking and improvement in universities; identifying what engages students in productive learning; program innovation and evaluation; assessment of professional capability; studies of effective leadership and the equity implications of the increased use of I.T. in education. He is the author of a wide range of refereed journal articles and his book Change matters: making a difference in education and training is widely used in a number of countries.

Currently he is director of the national CEQuery project which created and is now using a new tool for analyzing more than 150,000 comments written by Australian graduates on the national higher education course experience questionnaire. He is coordinator of a series of studies of successful graduates in a wide range of professions and has just completed a study of what distinguishes effective school principals.

He is a member of advisory groups to the Federal Minister for Education on quality assurance for Australia's offshore higher education programs and the new $250 million national learning and teaching performance fund, along with being a member of the national organizing committee for the Australian Universities Quality Forum.

This session will focus on the key findings which have emerged from the recently completed national CEQuery research project. In particular it will identify a set of key quality checkpoints for engaging students in productive learning and retaining them, and how CEQuery can be used in conjunction with other tracking systems and data to identify improvement priorities and to identify solutions.

A second theme will identify how we can apply the key lessons from 25 years' research on effective change implementation in universities to ensuring that the CEQuery tool and its findings are acted upon to improve the total student experience in higher education.

See Scott, G (2006): Accessing the Student Voice: Using CEQuery to identify what retains students and promotes engagement in productive learning in Australian higher education

Robert Carmichael
Audit Director, AUQA
Learning about learning and teaching from AUQA audit findings

For ten years Rob was the Head of the Office for Quality Education at Swinburne University of Technology. In this capacity he was responsible for the initial development, ongoing operation and continual improvement of the Swinburne Quality Management System. Rob also had responsibility for the higher education Subject Evaluation System and the University's annual teaching, research, and service excellence awards program. For years he also ran a quality improvement projects program, and was the institutional contact officer for the annual Quality Assurance & Improvement Plan, CUTSD programs, and the CQAHE quality assurance program. Prior to this Rob worked in the VET sector as a curriculum development specialist, as a teacher, and as an academic manager. He is also a formally qualified Quality Assessor, a trained auditor, a member of the AAIR, and a founding member of the Quality in Education and Training Network in Victoria.

Since joining AUQA in 2002, Rob has been appointed Audit Director for the following audits (year of audit in brackets): University of Canberra ('03); Southern Cross University ('03); ACT Accreditation & Recognition Council ('03); James Cook University and the NSW Department of Education and Training ('04); and the University of Tasmania and the Melbourne College of Divinity ('05). In 2004, Rob chaired of the Joint Steering Group for AUQF2004 in Adelaide, and in 2005 he also represented AUQA on the Working Group established by DEST for the auditing of non-self accrediting Higher Education Providers.

The Australian Universities Quality Agency (AUQA) was established with responsibility for auditing Australian universities against their stated missions and goals using a "fitness for purpose" approach based on a self-review portfolio and site visits. Audit reports include commendations for good practice and recommendations for improvement, and more recently affirmations which are a subset of recommendations. To-date all but one of the universities has now been audited.

This paper builds-on and extends work done by Barbara de la Harpe of RMIT and Alex Radloff of CQU in their paper 'Learning from AUQA Commendations' delivered at HERDSA 2006. Radloff and de la Harpe focused on the AUQA audit commendations that related to learning and teaching in order to gain insight into what universities identified as their areas of strength and which were validated by the audit process. They categorised commendations from 24 audit reports using a qualitative approach, to identify six categories of good practice and used these to identify insights for learning and teaching and to suggest how the sector can build on these to further enhance good practice. The author of this paper applied the same methodology to AUQA audit recommendations and concludes the paper by drawing out some issues of relevance for institutional evaluation practitioners. For consistency, the same 24 audit reports were used for this part of the study.

The author expressly acknowledges and thanks Alex Radloff and Barbara de la Harpe for their permission to use their research data and information in this paper.

Invited speakers