Student peer assessment in tertiary education: Promise, perils and practice

This paper first discusses some of the theoretical pedagogical issues related to the use of student peer assessment in university teaching and canvasses the suggested strengths and weaknesses of such practice. It then outlines the author's experience with introducing peer assessment as part of the formal assessment in a law unit including observations of the apparent effect on grades achieved and student evaluations of the subject.

Introduction

My interest in putting peer assessment into practice in my own teaching environment has been increasing as a result of being exposed to these influences. In 1993 I incorporated some peer evaluation into a law unit taught at Murdoch University and then again in a different form in the same unit in 1995. This paper is a brief account of those experiences, focussing on some of the dilemmas I have encountered, and beginning with the theoretical structure upon which my practice has been based.

The Promise

There is a growing awareness that students should be encouraged to think of their university degree as merely a milepost in a life of learning rather than as a terminal destination in their intellectual development. In order to become effective life-long learners students must acquire some proficiency in answering such questions as: "What do I know?" What do I need to know?" What must I do to gain that knowledge?" It is thought that this sort of insight can be improved by practising both self and peer assessment. As one writer puts it:

... life-long learning requires that individuals are able not only to work independently but also to assess their own performance and progress. Involvement in the assessment process would hopefully heighten our awareness and knowledge of the student approach to learning and enable students to make rational and objective judgements about their own strengths, weaknesses and range of skills [5].

One school of thought has focussed on the importance of "metacognition" in higher learning, particularly in relation to problem solving skills. Metacognition has been described as involving "knowledge and control of one's own thinking" [6]. The manifestation of metacognition lies in the ability to plan, monitor and evaluate in the course of problem solving which entails a kind of dialogue with oneself. In the words of two mathematics teachers, students "need to develop such metacognitive skills of 'inner speech' in order to become effective [problem solving] modellers" [7]. One way of stimulating this ability is to allow students to experience dialogue with their peers concerning the evaluation of their work as a model of what an individual student may also do within herself. Here again it seems there is a conception of learning as gaining the ability to be flexible and adaptive in one's thought rather than placing reliance on a store of static knowledge.

Other educators have emphasised the need for tertiary level students to take responsibility for their own learning, a clear criticism of the "transmission model" of teaching and learning where full responsibility relies on the instructor to ensure that prescribed information is given and received. One prominent writer has pointed out the inconsistency between many traditional practices of higher education and the ideals of independence, thoughtfulness and critical analysis which are also espoused [8]. Assessment practices have in the past ignored the possibility of a student role in the process, thereby perpetuating an intellectual dependency and perhaps lack of self-confidence on the part of students. One way to remedy this is to give students the responsibility to participate in the assessment of their own and fellow students' work. Here, learning is again conceived of as a process of active engagement rather than passive reception.

Finally, it has been pointed out that peer reviewing skills are an essential component of expertise in professional, business and academic life. As two educators have noted:

Throughout their working lives, students will need to assess the quality of the work of their subordinates, their peers, their superiors and, realistically, themselves. Hence, building confidence through experience must take place at some stage. As part of "education for life" this can also assist in the essential task of allowing students to become self-learners, a measure of the quality of the educational programme they are undertaking [9].

However, the desirability of good assessment skills is not confined to professional fields. In order to succeed in academia one must learn the unique criteria which govern the valuation of work within a disciplinary tradition. One way of becoming familiar with those standards is to practise applying them. In a sense, most university education is concerned with introducing students to the standards by which relevant and valuable contributions to disciplinary knowledge are identified. Why shouldn't they be given the opportunity to try applying those standards themselves at some point in their tertiary education? While most students will not become academics and thus inevitably involved in peer review processes, all students may benefit from engagement with the criteria of academic excellence by assessing the work of others.

Before moving on to consider the possible perils of peer assessment another important but more practical advantage of that process should be mentioned. Whether or not the results of peer assessment are counted toward the grades awarded, student peer involvement in reviewing and commenting on student work can help to reduce the workload of the teacher. Particularly in those subjects where multiple items of student work are set, the contribution of students to giving feedback (and possibly marks) to each other can help to reduce the time demanded of the instructor to perform these tasks.

These then are the conceptual arguments for the promise of peer assessment. But there is also some evidence from teaching practice to support the view that self and peer assessment promises benefits for tertiary learning. Two educators have noted that:

The indications are that there is a direct relationship between assessment skills and the quality of work which a student produces. More excitingly, there is evidence that the skills can be exploited directly with a consequential improvement in student performance [12].

The Perils

So far as potential negative impact on student learning is concerned we might conveniently divide the difficulties into those that have to do primarily with peer assessment as a component in formative evaluation of students and those that have more to do with such assessment used for summative purposes.

Doubts about the place of peer assessment in formative evaluation of students relate to the question whether they are able to give each other valuable feedback. Here, the potential problems may concern both the content of such communications and their tone in so far as they may have an adverse effect on interpersonal relations and possibly academic self-confidence. As one writing teacher has noted, exposing one's work to peers can seem a risky proposition but peers can act as motivators in addition to being astute critics if a spirit of cooperation and not competition is fostered [16].

The constructiveness of feedback received from peers from an academic standpoint will depend on the application of appropriate criteria by students in making their evaluations. Several authors have pointed to the difficulty students may have in identifying relevant grounds for criticism of each others' work [17]. Students have taken up this point in their comments on peer assessment as in the following reported comments: "Students will need to be told what points to look for when assessing other people's work."; "Students will not usually have the same level of understanding of subject matter compared to the teacher" [18]. Other educators however have pointed to the social factors involved in gaining understanding of the accepted criteria and standards of judgment of work within a discipline. For them learning these is a process of acculturation in which peer assessment may play a positive role:

Peer and self assessment provide a mechanism for the acculturation of the student. ...Peer assessment facilitates and encourages the generation of consensual objectivity by requiring students to discuss, challenge and justify their subjective constructs. The teacher mediates and guides this process of legitimization by selectively approving some constructs above others [19].

The key question arising out of the use of student peer assessment in summative evaluation is the issue of the validity of marks awarded. Typically, this has been framed in terms of the degree of correlation between marks students award themselves and their peers and the marks instructors would give for the same work. A closely related problem concerns the use of different criteria for assessment by students and teachers.

To the extent that tertiary education performs an accrediting function, particularly in professional fields, the issue of the validity of student-awarded marks if they are to count towards the academic requirements for a degree is a crucial one. For this reason, and also perhaps because the question appears suitable for quantitative analysis, there have been many published studies of student marking. In 1989 two researchers surveyed the literature on student self-assessment in an attempt to determine if there were any clear trends [22]. They found research which seemed to indicate both under and over marking by students compared with teachers and problems with the methodology of much of the published work. Since that review was performed there have been further reports of inquiries into the validity of student self and peer assessment, and the results should probably continue to be described as inconclusive [23].

In addition to the question of students' level of knowledge as it may affect the accuracy of their judgements of student work there is also the issue of the impact of personality and related interpersonal factors. Researchers have been alive to the possibility that such personal factors may affect the validity of student-awarded marks and some have reported attempts to avoid this problem, for example, by ensuring anonymous marking. On the other hand one teacher has concluded that a group which is well-known to each other and relatively cohesive offers the right climate for reliable and valid peer assessment [24].

Finally, in the area of summative evaluation by students there is the question of using appropriate teacher-sanctioned criteria for marking. The literature shows that many teachers have attempted to involve students in formulating the criteria for evaluation as part of an approach to sharing responsibility with them. It seems most instructors have been pleased with the degree of "fit" between student-generated criteria and their own. However, there is some indication that students may tend to weight "effort", the apparent amount of work undertaken as distinguished from the intrinsic value of the product of that labour, more heavily than would teachers [25].

Clearly, there are some formidable problems and unresolved issues connected with the use of student peer assessment whether for formative or summative evaluation in the tertiary setting. Nevertheless, for me the promise of peer assessment appeared to be worth risking the perils and I have introduced it into my law teaching. What follows is a description of my practice and some of the dilemmas I have encountered in the course of that experience.

The Practice

Peer assessment can of course be structured in a wide variety of ways and the literature records many permutations. It has been noted that one of the difficulties with the research in this area is frequent failure to adequately specify the details of the assessment which is being examined [26] Therefore, although my use of peer assessment in teaching law was not intended as a research project, I will use the format for reporting on my experience which has been recommended by leading investigators [27], as follows:

What was being assessed?
Each student was required to prepare six individual documents throughout the year in a format specified by me. These ranged from business letters of a legal nature to contracts and deeds. Half of the class were designated as "originators" for each assignment. The originators prepared their document first and each of them then sent it to a paired student (the "responders") who replied by sending their own document back, incorporating desired amendments to the originators' draft. It was these documents which were self and peer assessed by the students. Documents and assessments were exchanged by students using electronic mail with their student identification numbers functioning as addresses, so that anonymity of the author was the rule.

What were the participant characteristics?
The class of 84 students were mostly in their last or penultimate year of law study and included both school leavers and mature students of both sexes. A range of ability as indicated by previous grades was present in the class although it is reasonable to suggest that due to the existence of quotas for entry to law school the average level of ability amongst law students is likely to be higher than amongst the university student population as a whole. The students' experience with self or peer assessment prior to taking this subject is unknown.

What subject matter, type and level of content was being assessed?
The documents assessed were those types frequently prepared by practicing solicitors. I expected them to show a high level of understanding of the relevant legal concepts and principles and the governing legislative framework. A factual situation was presented to the class as the basis on which the documents were to be prepared as a simulation of actual practice. Although the legal knowledge required to produce the documents was largely taught in prior subjects I was aware that these students did not have much if any practice in preparing such documents.

What was the time period?
The subject was a full-year unit taught over 26 weeks with weekly classes.

What opportunities did it provide for practice and feedback?
All self and peer assessments of the 6 required documents were included in the formal assessment of the unit; no practice assessments were provided for. Students in the class were surveyed by the university by means of a written questionnaire immediately before the end of the year which provided feedback to the instructor. In several classes examples of student assessments were made available to all and discussed by me as feedback to the students.

Did the marks count towards formal assessment?
Yes, the mark for each of the 6 documents prepared by each student counted as 7.5% of their mark in the subject (total of 45%). These marks were arrived at by averaging the self and peer assessment marks awarded for each document, with a proviso that the instructor retained the discretion to modify averages which fell above 80% or below 50%, by up to 10 marks, based on a review of the document.

What criteria were used?
Criteria for assessment were generated by class discussion based upon guidelines set out in a document prepared by the Curtin University of Technology School of Design entitled "Assessment Explained". The result of that class discussion was a document embodying the agreed criteria which is attached (Appendix A). Each student was required to complete a form containing the self-assessment of their own work which accompanied the document they sent to their paired student, and to prepare a similar form for the peer-assessment of the document received from their paired student. The forms for self and peer assessment were prepared by me based upon recommendations made by members of the same School of Design [28]. The peer assessment form is also attached (Appendix B) for reference.

What marking scale was used?
As shown in the document containing the assessment criteria (Appendix A) percentage marks were tied to grades according to the usual scale used by Murdoch University with which the students were familiar.

Observations?
For those students attaining average marks of over 80% my review of their documents usually resulted in the maximum reduction by me of the mark (up to 10%) although a trend was noticed to less reduction being made as the year progressed.

Very quickly it seemed that a majority of students adopted the practice of regularly giving self and peer assessment marks in the range 75-79%.

Very few marks of less than 50% were awarded most low marks were the result of penalties being imposed for lateness in submitting a document.

The majority of students tended to merely quote, parrot-like, portions of the assessment criteria in their self and peer assessment forms with little attempt to relate the criteria to specific aspects or portions of the document assessed.

Most peer evaluations echoed closely the mark given through self-assessment as if there were a sense of reciprocity at work. In some more isolated instances retaliation for low marks seemed to occur.

My impression was that the students did not like the self and peer assessment process. Comments made in the open ended section of the student evaluations are indicative: "The idea of marking the work of peers was a good one, but unsuccessful. 'Normal' students want to do as well as they can get away with. Once one person abuses the system so will all others." Self and peer assessment contributed largely to the result that a majority of the class received final marks for the unit in the range 70-75%.

Appendix A

Criteria and Standards For Assessment of Individual and Group Projects in L369 Legal Practice and Documentation 1995

1. Present a documented solution to the client's problem or instructions which is effective (enforceable), comprehensive and shows evidence of dealing successfully with issues or opportunities which do not obviously appear on the surface of the issues presented; the work submitted demonstrates in depth analysis and treatment of legal concepts and frameworks.

2. Show a clear and complete understanding of the nature of the problem or instructions together with a recognition of the wider social context within which a solution must operate.

3. Take the intended audience into account by producing a document which is appropriate in style, content and language.

4. Present documentation which in its organisation, coherence, internal consistency and readability is of a professional standard.

5. Demonstrate critical awareness of the role of style and form of a document in establishing legal validity and meaning.

6. Show a high level of ability to locate, assemble, analyse and present salient information in an innovative and creative way.

7. Produce relevant information that meets professional standards acceptable in the wider community.

8. Show a high level of flexibility in finding a solution to suit the client's interests; adapt standard clauses to better suit the fact situation presented.

9. Demonstrate clear and eloquent verbal skills.

10. Research and reference all sources accurately where required.

11. Present documentation which is highly accurate and contains an optimum level of detail.

12. Produce well structured, logical, cohesive and coherent documents which are easy to follow, flow well and have no internal inconsistencies.

1. Present an enforceable documented solution to a problem or set of instructions which deals successfully with all major issues and opportunities and which demonstrates a clear analysis of all relevant legal concepts and frameworks.

2. Show clear and comprehensive understanding of the nature of the problem or instructions and the legal framework within which it will occur; all elements of the solution are effective and coherent.

3. Present documentation which is readable, well-organised and coherent at a standard expected of highly achieving law students.

4. Show comprehension of the influence of form and structure of documentation in conveying legally relevant information.

5. Demonstrate skill in organising and presenting information elegantly in accordance with established standards and methods.

6. Produce information that is legally sound but not necessarily at a professional standard.

7. Make no use of irrelevant standard clauses and demonstrate some flexibility in tailoring a solution to the client's needs.

8. Have good verbal skills with some room for improvement.

9. Research and reference most sources accurately where required.

10. Present documentation which is generally accurate with an optimum level of detail.

11. Produce documents which avoid inconsistencies, are well structured and relatively easy to understand.

1. Present an adequate documented solution to the problem or instructions which may be based upon reliable and appropriate precedents; demonstrate understanding of and ability to make use of the key legal concepts and frameworks.

2. Show an adequate understanding of the problem or instructions and the major elements of the legal framework which govern the solution.

3. Demonstrate an ability to adequately organise a document at the level of an average law student.

4. Show an awareness of the importance of form and style and the influence it may have on the effectiveness and interpretation of documents.

5. Demonstrate some skill in organising and presenting information but with less clarity and elegance than those whose work is of a Distinction standard.

6. Make some use of irrelevant standard clauses and show a low level of flexibility in arriving at a solution.

7. Have adequate verbal skills but are not convincing.

8. Research and reference some sources accurately where required.

9. Present documentation which is moderately accurate with a reasonable level of detail.

10. Produce documents which are readable but have some significant internal inconsistencies which do not jeopardise their enforceability.

1. Present a document which accomplishes the client's main aims and objectives but which leaves other subsidiary ones unfulfilled; students shows an incomplete understanding of the relevant concepts and legal frameworks.

2. Show limited or incomplete understanding of the problem or instructions although the key elements are grasped and adequately resolved.

3. Show an awareness of the need to organise a document to make it logical, consistent and coherent but fail in some aspects to do so.

4. Show little comprehension of the issues of form and style as they bear on legal documentation.

5. Demonstrate some basic skills in locating and presenting information.

6. Frequently use irrelevant standard clauses and show no flexibility in arriving at a solution.

7. Do not communicate clearly.

8. Research and reference some sources where required, but not accurately.

9. Present documentation with some inaccuracies either lacking in necessary detail or with superfluous unnecessary detail.

10. Produce documents which are hard to read because of their poor organisation and internal inconsistencies but which are nevertheless enforceable.

1. Present documentation which is unenforceable or which is ineffective in dealing with the main elements of the problem or instructions to which it responds; if they were practicing lawyers it is likely they would be considered to have been negligent in the performance of the task.

2. Show serious lack of understanding of the problem or instructions with the result that major elements are ignored or dealt with ineffectively.

3. Fail to take the intended audience into account and therefore produce documents which are inappropriate (sometimes to the extent of unintelligibility) in style, content and language.

4. Show serious lack of appreciation for the necessity to plan and organise a document to make it logical, coherent, and internally consistent.

5. Are unaware of issues of form and style as they relate to the production of legal documentation.

6. Demonstrate lack or underdevelopment of skills in locating and presenting information.

7. Produce information that is not legally sound.

8. Rely on standard clauses which do not provide an effective and enforceable solution having regard to the client's interests.

9. Have poor verbal skills and as a result fail to communicate effectively, thereby endangering the enforceability of documentation.

10. Do not research or reference sources.

11. Present documentation which is rendered unenforceable due to inaccuracies and which incorporates superfluous detail or omits necessary detail.

12. Produce documents which are hard to follow and which are unenforceable due to internal inconsistencies or ambiguities.

Appendix B

L369 Legal Practice And Documentation 1995

Peer-Assessment Form

Your Student Number:

Student Number of author of work assessed:

List three aspects of the project you consider have been covered particularly well having regard to the project instructions and agreed assessment criteria:

1.

2.

3.

1.

2.

3.

%

Footnotes

1. See for instance, Burke, R., "Some preliminary data on the use of self-evaluation and peer ratings in assigning university course grades", (1969) The Journal of Educational Research, vol 62, July-August, 444. [back]
2. See Boud, David, and Falchikov, Nancy, "Quantitative studies of student self-assessment in higher education: a critical analysis of findings", (1989) Higher Education, vol 18, 529; and Rawson, Shirley and Tyree, Alan L, "Self and peer assessment in legal education", (1989) Legal Education Review, vol 1, 135. [back]
3. See Kelmar, John, "Peer assessment: a study of graduate students", in Latchem, Colin, and Herrmann, Allan, eds, Higher Education Teaching and Learning: The Challenge, Perth: Curtin University of Technology, 1992, 83. [back]
4. One example is the paper by Denise Kirkpatrick and Richard Fuller, "The challenge of peer assessment", in Summers, Laurie, ed, Quality in Teaching and Learning: A Focus on Learning, Perth: Edith Cowan University, 1995, 146. [back]
5. Stefani, Lorraine A J, "Comparison of collaborative self, peer and tutor assessment in a biochemistry practical", (1992) Biochemical Education, vol 20(3), 148. [back]
6. Brown, A, "Metacognition and other mechanisms", in Weinert, F E, and Kluwe, R H, eds, Metacognition, Motivation and Understanding, Hillsdale, NJ: Lawrence Erlbaum, 1987, 65-116, as referred to in Tanner, Howard and Jones, Sonia, "Using peer and self-assessment to develop modelling skills with students aged 11 to 16: a socio-constructive view", (1994) Educational Studies in Mathematics, vol 27, 413, at 415. [back]
7. Tanner and Jones, supra note 6, 415. [back]
8. Boud, D, "Assessment and the promotion of academic values", (1990) Studies in Higher Education, vol 15(1), 101-111, as referred to in Rushton, Christopher, Ramsey, Phillip and Rada, Roy, "Peer assessment in a collaborative hypermedia environment: a case study", (1993) Journal of Computer-Based Instruction, vol 20(3), 75. [back]
9. Oldfield, Keith A and Macalpine, Mark K, "Peer and self-assessment at tertiary level - an experiential report", (1995) Assessment & Evaluation in Higher Education, vol 20(1), 125. [back]
10. See for instance Boud, D J and Tyree, A L, "Self and peer assessment in professional education: a preliminary study in law", (1979) Journal of the Society of Public Teachers of Law, vol 15(1) 65. [back]
11. See Schon, Donald A, Educating the Reflective Practitioner, San Francisco: Jossey-Bass, 1987. [back]
12. Rawson, Shirley and Tyree, Alan L, "Self and peer assessment in legal education", (1989) Legal Education Review, vol 1, 135, 137, citing Boud, D and Lublin, J, Self Assessment in Professional Education: A Report to the Commonwealth Education Research and Development Committee, Kensington: University of New South Wales Tertiary Education Research Centre, 1983. [back]
13. ee Rawson and Tyree, "Self and peer assessment in legal education", supra note 10 and Rushton, Ramsey and Rada, "Peer assessment in a collaborative hypermedia environment: a case study", supra note 8. S[back]
14. Rawson and Tyree, "Self and peer assessment in legal education", supra note 10, 141. [back]
15. Rushton, Ramsey and Rada, "Peer assessment in a collaborative hypermedia environment: a case study", supra note 8, 79. [back]
16. See Reilly, Kathleen C, "Expanding audiences: breaking the circle of assessment", (1995) The Clearing House, vol 68(4), 240. [back]
17. See, for instance, Fulcher, Alison and Moss, Richard, "Teaching students to give constructive feedback", paper presented at APLEC '92, Sydney, November 1992 and Rushton, Ramsey and Rada, "Peer assessment in a collaborative hypermedia environment: a case study", supra note 8. [back]
18. Rushton, Ramsey and Rada, "Peer assessment in a collaborative hypermedia environment: a case study", supra note 8, 76. [back]
19. Tanner and Jones, "Using peer and self-assessment to develop modelling skills with students aged 11 to 16: a socio-constructive view", supra note 6, 426.back]
20. Fulcher, Alison and Moss, Richard, "Teaching students to give constructive feedback", supra note 14, 2-3. For a useful guide to the use of peer feedback see Boud, David, Implementing Student Self Assessment, HERDSA Green Guide No. 5, Campbelltown: Higher Education Research and Development Society of Australasia, 1991, Appendix "Giving and Receiving Feedback: A Guide to the Use of Peers in Self Assessment". [back]
21. Id. [back]
22. Boud, David and Falchikov, Nancy, "Quantitative studies of student self-assessment in higher education: a critical analysis of findings", (1989) Higher Education, vol 18, 529. [back]
23. See for instance, Stefani, Lorraine J, "Comparison of collaborative self, peer and tutor assessment in a biochemistry practical", (1992) Biochemical Education, vol 20(3), 148, Rushton, Ramsey and Rada, "Peer assessment in a collaborative hypermedia environment: a case study", supra note 8, and Oldfield, Keith A and Macalpine, Mark K, "Peer and self-assessment at tertiary level - an experiential report", (1995) Assessment & Evaluation in Higher Education, vol 20(1), 125. [back]
24. See Hall, Jean, "Small group peer assessment of the human computer interface: a case study", in Summers, Laurie, ed, Quality in Teaching and Learning: A Focus on Learning, Perth: Edith Cowan University, 1995, 111. [back]
25. See Stover, Robert V, "The impact of self-grading on performance and evaluation in a constitutional law course" (1976) Teaching Political Science, vol 3(3), 303, Kelmar, John, "Peer assessment: a study of graduate students", supra note 3, and Boud, David, Implementing Student Self Assessment, HERDSA Green Guide No. 5, Campbelltown: Higher Education Research and Development Society of Australasia, 1991, 15. [back]
26. See Boud, David and Falchikov, Nancy, "Quantitative studies of student self-assessment in higher education: a critical analysis of findings", supra note 19. [back]
27. Ibid, 544-545. [back]
28. See Shaw, Jacqueline, et al, "Feedback sheets as a form of assessment to support learning", in Summers, Laurie, ed, Quality in Teaching and Learning: A Focus on Learning, Perth: Edith Cowan University, 1995, 234. [back]