How should we assess and report student generic attributes?

University graduates are expected to have attributes such as critical thinking, ethical practice and communication skills and their university experiences are intended to contribute significantly to the development of these attributes. A review of the literature reveals that it is an international trend to describe these attributes explicitly and direct resources towards measuring their achievement. Australian universities recently have been swept up in this international trend.

Although there is general agreement on the areas of generic attributes which are desirable, universities have generally reserved the right to develop their own specific descriptions and many have done so. However, much less has been achieved in incorporating generic attributes into curriculum and even less progress has been made in developing means to assess whether graduates have acquired these attributes. There is also a lack of knowledge about how employers use evidence of generic attributes and the extent to which they desire reports on the generic attributes of university graduates.

This paper raises several questions which should be considered when describing generic attributes, deciding how to incorporate them into curriculum, and developing assessment and reporting processes for student acquisition of generic attributes.

Introduction

• the increasingly popular perspective that education is a lifelong process;
• a greater focus on the direct relationship between education and training and the employment of graduates; and
• the development of outcome measures of education and training which in universities has arisen out of the quality improvement movement and its focus on measurable outcomes as a means of judging the efficacy of the education process.

Common generic attributes

Clanchy & Ballard (1993) argue that universities should be able to certify that graduates have mastered a body of knowledge and have acquired certain generic attitudes and the following three distinctive generic skills; thinking (reasoning), research (including methods of inquiry and management of information) and communication (oral and written). Although many universities have published list of generic attributes, Clanchy & Ballard found that nowhere has this resulted in any serious curriculum revision.

Employer views on generic qualities

There is widespread use of generic attributes, explicitly and implicitly, in the recruitment and promotion of employees; Employers use their own terms to describe generic attributes, and these overlap considerably with externally identified generic attributes; and Firms would like to align position descriptions with generic attributes.

Five essential questions relating to assessing and reporting generic attributes

Question 1: How to describe each generic attribute?

Communicating ideas and information: the capacity to communicate effectively with others using the range of spoken, written, graphic and other non-verbal means of expression (Mayer Key Competencies).

Communication: Make connections that create meaning between yourself and your audience. Learn to speak, read, write and listen effectively, using graphics, electronic media, computers and quantified data (University of Canberra).

Graduates will be able to communicate effectively in professional practice and community activity. Some indicators include demonstrate oral, written, mathematical and visual literacies as appropriate to the discipline or professional area, display sensitivity to their audience in organising and presenting ideas, and communicate appropriately with professional colleagues and the public (Murdoch University draft attribute).

Question 2: How to incorporate generic attributes into curriculum?

Based on experience in the VET sector, two general approaches to the incorporation of generic skills into curriculum have developed, adjunct and integrated.

An adjunct approach separates aspects of generic attributes from technical aspects of the course. Separation can occur at a number of levels within the curriculum:

• In the development of discrete learning outcomes or objectives within the curriculum;
• Distinct modules or subjects which focus on the generic attributes aspects of the course; and
• In the design of assessment events which explicitly focus on the assessment of generic attributes.

In an integrated approach, the generic attributes become an integral part of the course philosophy, are expressed in course descriptions, course objectives or course outcomes and opportunities for their development are identified explicitly in the curriculum. Supports of this approach argue that the explicit identification of generic attributes facilitates their development by increasing the opportunity for the delivery, assessment and reporting of the skills.

Important opportunities for the development of generic attributes occur in the selection of delivery methods. Teaching contexts can provide an explicit focus on the development of generic attributes, thus providing students with opportunities to develop them. The development of generic attributes will be promoted where there is the opportunity for students to practice these attributes within learning activities.

Delivery methods which have been used with some success include:

• Group learning activities which enable students to practice generic attributes while focussing on a discipline related issue;
• Activities which focus on generic attributes - for example integrative classroom/workplace investigations and/or problem solving exercises;
• Teaching and learning materials which include a focus on the development of generic attributes integrated with the development of discipline specific knowledge or skills.

Question 3: Can each generic attribute be measured?

The question of whether a generic quality can be measured is intimately linked to the reasons why the assessment is being done and for whom it is being done. There are at least three reasons linked to specific stakeholders in the assessment process which are important to consider. First, assessment might be done to provide individual students with evidence of whether they have the necessary attributes to contribute to society and to continue to learn throughout their lives. Second, assessment of generic attributes could provide students and employers with evidence that the student has the necessary skills, knowledge, and values to gain employment. Third, assessment could measure the extent to which universities contribute to a student's generic attributes and produce the graduates they aim to produce. There is further discussion under Question 4 about the relationship between assessing attributes and the users of that assessment.

Question 3a: If a generic attribute can be measured, how should it be assessed?

Evidence from survey conducted by McCurry (1996) and Cummings & Ho (1996) indicate that employers believe a single mark, as provided for academic subjects, is not appropriate as evidence of generic attributes. Instead, they would like to have available one or more of the following; evidence of the student's level of generic attribute against some criterion, a description of what constitutes this level of performance, and examples of the student's work which demonstrate this attribute.

Research in the secondary school and VET sectors (McCurry, 1996; NIEF, 1996) advocates the use of a portfolio approach incorporating assessment by several teachers, self-assessment by the student and examples of work produced by the student which demonstrates each attribute.

There is some concern that a portfolio approach may be better suited to secondary schools where teachers have a more personal, pastoral care role, than universities where the lecturer-student relationship is more formal. It may be that lecturers would not have sufficient experience of individual students to make judgements about generic skills. There is at present very little empirical research on the appropriateness of different approaches to assessing generic attributes.

Question 4: How should the assessment result be reported?

Universities, in turn, may want the same reports as for students and employers but also seek evidence which is comparable across disciplines and provides information about the quality of course of study.

McCurry (1996) compares the strengths and weaknesses of several approaches to assessment and reporting and concludes that it is possible to develop an assessment framework which meets the criteria of good assessment, addresses employers requirements for reporting, uses a subject based approach, with staff as assessors, and can be monitored by central agencies.

Two other criteria which might be applied to the assessment of generic attributes are its useability, that is, how well and to what extent the assessment results will be used by the individual, future employers, and society, and its credibility. Clearly these are related because if the results have low credibility their level of use is likely to be diminished.

Summary

References

Clanchy, J. & Ballard, B. (1993). Generic Skills in the Context of Higher Education. A discussion paper on aspects of 'Generic Skills' as they affect teaching and learning within ANU. Study Skills Centre.

Cummings, R. & Ho, R. (1996). Survey of Key Competencies in Selected Industries in Western Australia. Perth: Murdoch University.

Cummings, R., Ho, R. & Bunic, S. (1997). Assessing and Reporting of Student Generic Qualities in Higher Education, Paper presented at the HERDSA Conference, Adelaide, SA.

Goddard, D. & Ferguson, K. (1996a). Small Business Owners, their Employment Practices and Key Competencies. Department of Employment, Education, Training and Youth Affairs and Business in the Community, Ltd.

Goddard, D. & Ferguson, K. (1996b). The Southern Province Enterprise Network Project: Enterprise - Go For It. Department of Employment, Education, Training and Youth Affairs and Business in the Community, Ltd.

Golding, B., Marginson, S. & Pascoe, R. (May 1996). Changing Context, Moving Skills: Generic skills in the context of credit transfer and the recognition of prior learning, Canberra: Australian Government Publishing Service.

Hill, P. W. (undated). Key Competencies: Which Way Forward? Melbourne: National Industry and Education Forum.

Mayer, E. (Chair) (1992). Putting general education to work: The Key Competencies. The Australian Education Council and Ministers for Vocational Education and Training.

McCurry, D. (1996). Approaches to Assessing the Key Competencies. Melbourne: Australian Council for Educational Research.

McCurry, D. and Bryce, J. (1997). The School Based Key Competencies Levels Assessment Project, Final Report. Melbourne: Australian Council for Educational Research.

National Industry and Education Forum (1996). Business, Industry, Key Competencies and Portfolios. Melbourne: NIEF.

Secondary Education Authority (1996). Key Competencies Project Report. Perth: SEA.

Schwartz, S., Piper, D. W. & Crook, N. (1993). Defining, Assessing, and Improving the Outcomes of Higher Education: An Annotated Bibliography. Brisbane: The University of Queensland.

Werner, M. C. (1994). Australian Key Competencies in an International Perspective. South Australia: National Centre for Vocational Education Research Ltd.