Modelling and feedback: Providing constructive guidance through a web medium

With the increasing use of the world wide web as an instructional medium, there are still many challenges awaiting educators. In particular, we need to explore how we can provide feedback and guidance for those learning more complex principles and applications. This paper explores some of the strategies which may be used to provide feedback and further models of good practice to learners. The methods are currently being developed and tested using a database searching task, and illustrate the potential for capturing learner response sets and using these to structure the learning cues provided online. The paper identifies some changes to instructional design methods which may need to be integrated into course planning if educators are serious about quality online instruction.

The use of the world wide web as a medium has created many possibilities for training and education. Increasingly, tertiary educational programs are being mounted as web based instructional processes, with the claim that students can acquire similar levels of understanding and expertise as that provided through campus instruction. This form of instruction has many cost benefits, and enables more flexible access for students. It offers a cost effective and creative means of providing guidance to learners as and when they require it. With its capacity to provide visual and textual stimuli, and a menu based instructional set of options, it offers both flexibility and the potential for individualised instruction. However, the argument that this leads to similar educational outcomes assumes that web based instruction can emulate the better facets of face to face instruction.

Whilst it is true that learners may obtain a strong understanding of the content and view examples of good practice through web based learning, a great challenge lies in creating opportunities to practice those competencies and gain targeted feedback. Where the knowledge and understanding to be gained relates to simple, predictable and controllable tasks, forms may be created to provide immediate response on the selected answer. These enable learning to be guided and structured around the correct responses of the student. In this context, it is relatively manageable to provide opportunities for quick cues as to correct understanding. However, not all learning is simple.

At tertiary levels, there is an expectation that students will acquire deep and complex learning in highly specialised and advanced areas. These may not offer the same easily controlled learning environment. Whilst opportunities to interact with the instructor may be constructed through bulletin boards, online tutorials and similar real time interactions, they are time and labour intensive for instructors, and may not be available at the opportune moment for those seeking this guidance. As instructors through a technological medium, we must work toward developing some alternative strategies for providing such guidance and coaching at point of need for complex learning acquisition. Web based instruction may be the future of tertiary education - as many predict. In order to be so, it needs to embrace the some understanding of learning principles across all levels of learning. The provision of feedback and guidance at point of need is essential to this process. This paper outlines a current attempt to provide web based instruction for those learning a complex task, using both the capture of response sets, and the development of opportunities to construct benchmarks of performance and self feedback. It also identifies other issues which still need to be addressed.

Feedback and modelling complex behaviours

Learners seek feedback from a range of sources when learning new processes. Their own understanding of the task serves to inform their personal judgement as to the suitability of their actions. In addition, they seek to gain information from the context in which they operate, and from expert sources, such as an instructor. The lack of contextual clues in an Internet learning environment reduces the power of this source of feedback. Instead, the learner becomes increasingly reliant on direct support from the instructional screen. Whilst this can be generated for simple tasks, it is less easily built into web learning environments when the learning performance tasks are complex.

There is a growing understanding of the ways in which feedback operates, and can be constructed. Kluger and De Nisi (1996) argue that successful feedback strongly focuses on the task, and directs the learner's attention to this. This assists the learner in exploring naturally occurring performance information from the task, and enables the gaining of better insights as to the task and its composition. As tasks become more complex, it becomes more difficult to directly interpret sufficient information about the corrective strategies, and the steps involved may become more extensive and convoluted (Payne, Bettman and Johnson, 1993; Wood, 1986). Feedback relating to the required behaviours, strategies or expected outcomes are less readily provided and obtained. At the same time, learners need more extensive information about their performance and their achievement of the skills which should have been demonstrated. Feedback that provides information on the effectiveness of particular strategies, and their reflection of the learning principles needs to be provided. This feedback emphasises a focus on the task and its completion using the appropriate strategies and behaviours which have been taught (eg. Balzer, Doherty & O'Connor, 1989; Earley, Northcraft, Lee & Lituchy, 1990; Locke & Latham, 1990). Information is provided on both the functional validity of the strategy, and the important features of the task. Both help to reorientate the learner toward a better understanding of the key skills which must be acquired and applied (Benson & Onkal, 1992; Schunk & Rice, 1991). This is particularly powerful if it is further linked to the initial instructional model which was taught to the learner, and if that model is reinforced during the review process.

Providing modelling and feedback opportunities within an information search task

However, the training, while effective, needs to be reinforced by additional feedback and modelling following the learner's own experiential processes. The complexity of the process necessitates a lengthy process of learning and reconsolidation to build the search skills to higher levels of expertise. After three hours, for example, most learners still demonstrate numerous errors and inconsistent search practices (Debowski, 2001). Further, they tend to revert to simpler, less demanding search processes unless the initial models of search are reinforced and reviewed regularly (Wood, Debowski and Goodman, 2001). This has led to further efforts to identify ways in which feedback and modelling can be used to assist the learner in an online environment, where the direct intervention of the instructor may be removed.

An ideal support mechanism for novice learners of processes is to receive feedback on their performance as they work through the task (Goodman, 1998). This can be challenging, since the instructor needs to be able to predict likely wrong answers and right answers. This advanced level of feedback and guidance requires pre-planning and testing, so that a knowledge of learner responses can be accrued. This is one of the strategies currently being tested, with data sets of learner strategies and keywords being collected for a number of test tasks. A data set of correct and incorrect keywords has so far been developed, for example, using expert searches and the searches of sixty participants. The range of items identified by the participants provides a sample of real practice, as well as the normal best practice, to facilitate in the preparation of cued examples which might prompt the learner. By seeking an expansive set of both error practice and best practice, there is a stronger likelihood of guiding the learner into better search choices. This form of guidance and feedback enables better guidance of the learner during each step of the process, but is time consuming and requires pre-planning in order to sample learner behaviours.

A second feedback strategy being tested is to encourage learners to self evaluate their own performance. To incorporate this approach, it is necessary to consider the key criteria which should be demonstrated by the learner, and to ensure these are well-introduced during the instructional process. For example, in the search processes used in the afore-mentioned study, participants are taught to consider how their search reflects the depth of search and sequencing of the search steps which are undertaken by experts, and which they are encouraged to apply. The use of the same evaluation structure by the participant learners reinforces the initial teaching strategy, and also encourages stronger levels of self awareness. This is a cost effective means of building stronger strategy awareness, and can ensure that the learner re-orientates back to the core strategy which should be reflected. Thus, this is a relatively easy way to reinforce the initial instructional process, and to also provide a feedback mechanism which is available at all times. It seeks to empower and inform the learner.

This second strategy is more powerful if it is integrated with a third process: providing a model of best practice against which the learner may contrast his or her efforts on particular performance tasks. This is increasingly important as the complexity of the process increases. The personal feedback evaluation is made much richer through the comparison with this benchmark, since it serves to highlight areas of further improvement which could be undertaken, and also illustrates the particular stages which may need refinement. In the case of database searching, it also enables a better sense of search outcomes, as the learner may then also review the number of records retrieved and compare it with an expert benchmark (Benson & Onkal, 1992; Ilgen, 1971). This type of guidance also encourages a stronger focus on the task and the information which may be gained from analysing the performance on the task.

These three feedback mechanisms have been designed to provide disintermediated feedback to the learner, despite the absence of the instructor. They enable the capture of typical search behaviours, and the use of this information to provide formatted responses, self managed feedback tools and modelled feedback templates for the learner. While ideally suited to the online learning environment, they also make eminently good sense in normal instructional contexts also.

These strategies are still being tested, but they do offer promise for the future as we move toward online instruction. As educators in this medium, we need to consider the challenges facing those learning through web based training. They should be provided with effective support and feedback mechanisms to enable effective learning transfer. This is particularly critical when the instructional task relates to complex learning which requires multiple competencies to be developed.

We are faced with many challenges in this globalised educational environment. If education via a technological interface is to be an effective and quality alternative, we need to address a number of questions. For example, how can the learner be provided with examples of high quality outcomes, to create some standards of performance? Second, how can learners be encouraged to use these sources of guidance in a self managed learning environment?. Third, how can the instructor identify the types of errors and problems which are likely to occur? Fourth, how might these problems be generalised and developed into feedback models for learners?

As educators, we are responsible for seeking the best forms of supporting learners - particularly through complex learning processes. This form of creating a feedback system may be one option which could work for some forms of web based instruction.

References

Benson, P. G., & Onkal, D. (1992). The effects of feedback and training on the performance of probability forecasters. International Journal of Forecasting, 8(4), 559-573.

Debowski, S. (2001). Wrong Way! Go Back! An exploration of novice search behaviours while conducting an information search. The Electronic Library (in press).

Debowski, S. J., Wood, R. E. & Bandura, A. (2001). Impact of guided mastery and enactive exploration on self-regulatory mechanisms and knowledge construction through electronic inquiry. Journal of Applied Psychology (in press).

Earley, P. C., Northcraft, G. B., Lee, C. & Lituchy, T. R. (1990). Impact of process and outcome feedback on the relation of goal setting to task performance. Academy of Management Journal, 33(1), 87-105.

Goodman, J. S. (1998). The interactive effects of task and external feedback on practice performance and learning. Organizational Behavior and Human Decision Processes, 76, 223-252.

Ilgen, D. R. (1971). Satisfaction with performance as a function of the initial level of expected performance and the deviation from expectations. Organizational Behavior and Human Performance, 6(May), 345-361.

Kluger, A. N. & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119, 2.

Locke, E. A. & Latham, G. P. (1990). A Theory of Goal Setting and Task Performance. Englewood Cliffs, New Jersey: Prentice-Hall.

Payne, J. W., Bettman, J. R., & Johnson, E. J. (1993). The Adaptive Decision Maker. Cambridge: Cambridge University Press.

Schunk, D. H., & Rice, J. M. (1991). Final strategy goals and progress feedback during reading comprehension instruction. Journal of Reading Behavior, 23(3), 351-364.

Wood, R. E. (1986). Task complexity: Definition of the construct. Organizational Behavior and Human Decision Processes, 37, 60-82.

Wood, R. E., Debowski, S. & Goodman, J. (2001). Feedback effects on electronic search. Manuscript submitted for publication.

Wood, R. E., George-Falvy, J., & Debowski, S. (2001). Motivation and information search on complex tasks. In Erez, M., Kleinbeck, U., & Thierry, H. (Eds), Work Motivation in the Context of a Globalizing Economy. Hillsdale, NJ: Lawrence Erlbaum Associates: 27-49.