UCLES: A questionnaire for evaluating portfolio cultures in postgraduate teaching

As a university teacher educator working in the field of science and mathematics education I am concerned with reforming the epistemology that underpins postgraduate university teaching. In this paper, I present an account of my current attempts to create a constructivist learning environment in one of my postgraduate classes, and describe a newly-developed questionnaire (UCLES) that I am using to evaluate the efficacy of my teaching reforms.

Introduction

Little wonder then that new postgraduate students bring with them historically-grounded expectations for more of the same -- more lecturing, more examinations, more absorption of expert knowledge, more external control of learning. In this teaching and learning culture of consumption, the quality of the learning 'product' (i.e. students' knowledge) is judged by criteria established entirely by others (i.e. academic 'quality contollers'). Little opportunity exists for students to develop skills of judging the quality of their own learning, and less opportunity exists for students to learn how to generate criteria of quality.

As a university teacher, it is my concern that unless we reform the prevailing transmissionist epistemology of university teaching, our students will remain trapped in an unhealthy culture of uncritical, unreflective and reproductive thinking that is intellectually and emotionally disempowering. For me, the starting point for epistemological reform is my own postgraduate teaching in which I must model exemplary teaching practice to my postgraduate students who are, themselves, professional school teachers. It is no exaggeration to say that, left to their own devices, most of these mature-age students would readily re-adopt the passive and impoverished learning roles of their undergraduate years. Such is the power of enculturation and habituation.

Constructivism: A referent for teaching reform

The press for basing my postgraduate teaching on a constructivist epistemology has arisen from a number of sources. Fifteen years of concerted research by science and mathematics educators worldwide has resulted in constructivism achieving an international status as the recognised alternative epistemology for curriculum reform of school science and mathematics (Tobin, 1993). During this period, constructivism also has come to serve as a powerful referent for framing questions that research addresses and ways that research is conducted. In the field of education, it has made research much more accesssible as the metaphor of research as learning has been widely adopted by teachers (Denzin & Lincoln, 1994).

My own research has shown, however, how difficult it can be for well-intentioned and well-supported school teachers to create constructivist learning environments based only on 'cold' rational considerations of how to reform their teaching methods (Taylor, 1992, 1994). It seems to me that if teachers are to understand the rich implications of constructivism as a referent for the reform of teaching then they must be given opportunities to experience first-hand what it means to be a learner in a classroom environment framed by a constructivist epistemology. I am motivated, therefore, to provide opportunities in my own postgraduate classes for teachers to reflect on their own 'hot' experiences of their struggles to become empowered learners.

Principles of constructivist teaching

1. Knowledge is a transformative growth process shaped by the learner's sense of purpose, rather than a product to be absorbed from external sources (Reflexivity, Relevance, Management).

2. The teacher is a crafter and facilitator of knowledge growth, rather than a disseminator, and modifies and adapts learning activities, rather than adheres rigidly to a prescribed curriculum (Accountability).

3. Students interactively construct their knowledge in social and cultural contexts (Negotiation).

4. The curriculum goals are concerned with how and why we know what we claim to know, and with knowledge growth and explanation (Reflexivity).

Organisation and structure of course

In the second session, students assemble in self-selected special interest groups (or SIGs) for the purpose of examining in detail a particular issue related to the overall theme of curriculum. Initially, I assist groups to form by suggesting a range of topics and then I 'kick start' each group by providing preliminary readings. Currently, the range of SIGs includes: Ethics, Technology, Culture, and Assessment. During the ensuing weeks, I impose on groups the requirement to submit in writing their emerging personal and negotiated group learning goals. During my visits to each SIG, I listen carefully, respond to questions, and stimulate thinking about the connection between the work of the SIG and the epistemological framework that is the subject of the first session. Each SIG is required to submit a report which is to be included in each student's individual portfolio for the purpose of formal course assessment.

Portfolio assessment

A portfolio is a coherently organised collection of work completed during the semester and should provide evidence of: (1) your process of learning, especially changes in your understanding that result from critical self-reflective thinking; and (2) your understanding of selected key issues in the domain of curriculum that reflect your own learning goals. My assessment of your portfolio will be guided by your self-assessment report, and will be based on the following four criteria.

• Sound evidence of your learning during the course.
• Sound evidence of your understanding of substantive issues related to the domain of curriculum.
• Well-organised and coherent portfolio that is clearly focused and relevant to your professional interests.
• Critically insightful self-evaluation report, in narrative form, that serves as an advance organiser for reading your portfolio.

The teacher researcher

I have decided, therefore, to generate data that are useful for evaluating my attempts to transform the epistemology of my teaching of the Curriculum course. My experience as an interpretive researcher (Erickson, 1986) focusses my attention on the need for multiple sources of data. For this study, I draw on (1) students' portfolios, (2) student interviews, (3) my own reflections that I record in a journal at the end of each class, and (4) a questionnaire designed to obtain measures of students' perceptions of key dimensions of the classroom learning environment (UCLES).

The University Constructivist Learning Environment Survey

The UCLES comprises 30 statements arranged in five scales each of which focuses on a key aspect of a constructivist learning environment. The five scales are termed 'Relevance', 'Reflexivity', Accountability', ' Management', and 'Negotiation'. Table 1 presents scale descriptions and a sample item of each scale. The UCLES has a 5-point Likert-type frequency response scale which comprises the categories: almost always (5 points), often (4), sometimes (3) seldom (2), and almost never (1). Therefore, the maximum possible mean score of each 6-item scale is 30 and the minimum possible scale mean score is 6. The UCLES is available in two forms.

• Student Preferred Form. This form of the questionnaire is useful for determining the extent to which students' prefer to adopt the role of a learner in a constructivist classroom environment. A teacher can use this form to (1) assess students' initial conceptual readiness prior to introducing a portfolio culture and (2) gauge the extent to which students have come to accept the practices of a portfolio culture towards the end of the course.

• Student Perceived Form. This form asks students about their current learning roles and is useful for determining the extent to which students perceive the classroom environment to be engaging them in constructivist learning activities. A teacher can use this form to obtain insights into the efficacy of a newly-established portfolio culture by gauging students' perceptions toward the end of the course.

Conclusion

References

Duschl, R. A and Gitomar, D. H. (1993, April). Emergent Questions About Portfolio Assessment. Paper presented as part of the symposium 'The Intellectual Activity of Science Teaching' at the annual meeting of the American Educational Research Association, Atlanta, GA.

Erickson, F. (1986). Qualitative Methods In Research On Teaching. In Wittrock M. (ed.), Handbook of Research On Teaching (3rd. ed.). New York, NY: Macmillan.

Fraser, B. J. (1989). Twenty Years On Classroom Environment Work: Progress and Prospect. Journal of Curriculum Studies, 21, 307-327.

Grundy, S. (1987). Curriculum: Product or Praxis? East Sussex, UK: The Falmer Press.

Kemmis, S. and McTaggert, R. (eds.), (1988). The Action Research Planner (3rd ed.). Geelong: Deakin University Press.

Taylor, P. C. (1992). An Interpretive Study Of The Role Of Teacher Beliefs In The Implementation Of Constructivist Theory In A Secondary Mathematics Classroom. Unpublished doctoral thesis, Perth: Curtin University.

Taylor, P. C. (1994). Mythmaking And Mythbreaking In The Mathematics Classroom. SMEC, Curtin University of Technology, Perth, Australia.

Taylor, P. C., Fraser, B. J. and White, L. R. (1994, April). CLES: An Instrument For Monitoring The Development Of Constructivist Learning Environments. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.

Taylor, P. C., Dawson, V. and Fraser, B. J. (1995, April). Classroom Learning Environments Under Transformation: A Constructivist Perspective. Paper to be presented at the annual meeting of the American Educational Research Association, San Fransisco, CA.

Tobin, K. (ed.) (1993). The Practice of Constructivism in Science Education. Washington, DC: Association for the Advancement of Science (AAAS) Press.

Von Glasersfeld, E. (1993). Questions and Answers about Radical Constructivism. In Tobin, K. (ed.), The Practice of Constructivism in Science Education. Washington, DC: Association for the Advancement of Science (AAAS) Press.