The remote delivery of cognitive science

The vast distances between metropolitan centers within the country create unique difficulties for Australian universities. In particular, numerically small disciplines such as cognitive science are struggling to remain competitive in the face of small and isolated local student populations. Flexible delivery of teaching may provide a solution to this problem. We report the results of a trial in simultaneous internet and video conferencing delivery of an introductory cognitive science laboratory. Students found the delivery to be nearly as effective as a locally run session.

Introduction

Factors in flexible delivery of cognitive science

Synchronous, one to one audio delivery over short band radio, for example, has been the foundation of the Australian "school of the air" for over fifty years (although it has recently been augmented by email). University education has different needs, however. The student populations we are concerned with have access to computing centers, and expect a high level of instructor interaction. Electronic interaction in teaching can range from simple email contact, through to studio produced, satellite delivered, multimedia programs (Colbert, Voglimacci & Finkelstein, 1995). Audiovisual (A-V) systems on their own, with high bandwidth distribution, have been in use for upwards of 10 years, and appear to be well used (Kirstein & Beckwith, 1991, Pugh, Parchman & Simpson, 1992) and effective in delivering lectures (Whittington, 1987). Such systems are however very expensive in terms of initial equipment purchase, transmission costs, and personnel requirements. A different set of limitations is associated with exclusive use of information technology (IT) delivery, for example through simple asynchronous email systems. Although IT can be an effective aid to teaching (Hacker & Sova, in press), it cannot substitute for the interactivity of a live tutor.

For distance learning to be effective, one obvious constraint is that all contacted sites must share a similar level of communication technology (Colbert, et al., 1995). Additionally, the choice of a medium for instruction delivery is partly determined by the nature of the material being communicated. In the case of cognitive science, the multi-disciplinary nature of the discipline has produced a variety of teaching methodologies. For example, in philosophy, the emphasis is on discussion and debate. In computer science, the emphasis is on problem solving, while in psychology it is on observation and experimentation. A similar diversity applies to the learning styles among the students likely to choose cognitive science. By implication, for flexible delivery of cognitive science to be successful, it needs to combine a variety of delivery options. As a first step toward assessing the potential of flexible delivery for cognitive science in Australia, we conducted a trial coast to coast hookup that involved simultaneous video and internet link.

A flexible delivery trial in cognitive science

We anticipated that immediate feedback would be important, particularly so in the tutorial component. Our experience in the teaching of computer simulation classes suggested that students commonly encounter interface difficulties or minor conceptual problems, which if not dealt with swiftly hamper the learning process and increase frustration. Consequently, we favored a synchronous communication mode. In addition, the BrainWave simulator provides a highly visual interface, much like a drawing program. We suspected that much of this advantage would be lost if we did not employ a visual channel. Thus, the lecture plus tutorial format involved a two way communication structure that was simultaneously one to many and one to one.

One partial solution to the problem of the excessive cost of A-V systems is to use a narrowband format (e.g., 128 kbit ISDN). This makes delivery affordable, albeit at the expense of imperfect live image quality. To maximise teaching effectiveness, it is important to match the live image of the tutor with high quality visual aids (e.g., overhead slides). Video conferencing (VC) systems typically include a document camera, which requires manual switching by the tutor to show overheads or other visual aids. A large proportion of the discomfort in using the VC system arises from the difficulties associated with use of the camera. We avoid this problem by using Internet compliant software to allow a tutor in one location to control a projected computer display in a distant location. The simultaneous display of computer based text and graphics with live video represents a highly effective delivery option previously available only with the aid of a professional production director (Colbert, et al., 1995). To meet our requirements we chose a combination of standard ISDN video conferencing and internet linked computers (using off the shelf Microsoft NetMeeting). Both video conferencing and the linked computers are synchronous forms of communication. Both provide visual channels and the video conferencing also provides an auditory channel. The linked computers are primarily useful in the one to many situation of the lecture (providing clear lecture slides and the ability to demonstrate the BrainWave software), while the video conferencing unit could be used for one to many and one to one communication. By employing multiple communication systems, we expected to be able to maximise the effectiveness of our distance delivery (Fellers & Moon, 1995).

Method

Equipment. The UWA cognitive science teaching laboratory has a mixture of SGI workstations and Pentium PC's running Windows 95. The BrainWave simulator is a Java applet, so both platforms were available for the trial. Communication was simultaneous over the Internet, via Microsoft's NetMeeting, which allows real time remote control of a PC, and by video conference, via a 128 kbit ISDN connection. The narrowband format ISDN allowed undistorted audio, but only moderate quality video communication.

The trial was delivered from a video conferencing studio in Queensland to the teaching laboratory in Western Australia, where the video image was displayed on a video conferencing system, and the computer image was projected onto a large screen. Students therefore had access to two live images, one that displayed the lecturer (the third author, S.D.) in Queensland and one that displayed his computer interactions.

Procedure. Communication occurred in two modes, passive and interactive. Initially, S.D. delivered a brief lecture, using the internet connection to supply high quality visuals (PowerPoint slides to accompany lecture) alongside the lower quality live video. The passive mode did allow for limited interactivity, via questions from the audience in Western Australia.

Following the lecture, students followed a web based tutorial on their own, with local tutors handling some of the students' questions, but directing others to S.D. via the video conferencing system. The parallel NetMeeting connection, which gave S.D. remote access to students' PC screens, was used to illustrate questions as necessary.

Results and evaluation

Difficulties were experienced with the video conferencing system due to the periodic breakup of a synthesised ISDN line. Thus, the system disconnected several times during the first laboratory, requiring a redial to reconnect. Changes in interstate traffic made the connection more stable after the first hour.

The second day followed a similar pattern, although disconnects were less frequent. The internet connection was vital in maintaining continuity of contact. Remote and local sites used the NetMeeting chat facility to provide updates on the technical problems.

Response by academics. The trial was attended by two or three academics on each of the two days. The remote lecture was delivered well, with both lecturer and students developing a tolerance to the periodic disconnects. Students had to be encouraged to speak clearly in response to lecturer questions but rapidly learned to do so. Academics on both ends viewed the delivery as effective, due in large part to the mixture of media (internet and video). Owing to the limited temporal resolution of the video connection, the high quality computer visuals delivered by the internet were particularly important. In fact, both the lectures themselves and the students' responses appeared very similar to local, live deliveries.

Response by students. Students' perceptions of the experimental session were collected twice. First, students were asked to write down their reactions to the delivery in an assignment collected one week following the sessions. This was followed by an end of term questionnaire, administered together with standard teaching evaluations, in which students were asked a range of questions about the lecture and tutorial phases of the session, along with some overall impressions. The final questionnaire contained only rating scales, while the assignment contained only free form comments.

Free form comments were very positive, consistent with the impressions of the academics involved. Students knew that the session was a special event, and so the source of their satisfaction is of course open to question. The majority of students viewed the exercise as a "success", and most found it "exciting".

However, the results of the rating scale data taken at the end of the semester (some 7 weeks later) were more neutral (see Table 1). The questionnaire asked students to compare the experimental session to the average standard laboratory in the course. The difference in the interactivity, level of understanding achieved, comfort with the situation, perceived success of the exercise, and responsiveness of the remote instructor were measured for the lecture and tutorial portion separately. Overall satisfaction with the laboratory session as a whole and the amount of engagement the students felt toward the session were also measured, along with the students' preference for the remote vs. live laboratory. Finally, the questionnaire asked students to indicate the proportion of their feelings about the laboratory that were associated with the novelty of the situation.

Table 1: Student perceptions of remote delivery laboratory. Lecture and tutorial sections were measured separately, and general aspects of remote delivery were measured overall. Ratings ranged from 1 (negative) to 5 (positive). N=19

Students were still largely satisfied with the session, although they preferred a live laboratory. In particular, students generally agreed that the tutorial portion was below the average for the course on all measures except comfort. The lecture was less interactive, and less comfortable for the students, compared with the norm for the course. Since the lecture occurred before the tutorial component commenced, the difference in comfort may indicate an effect of experience. Students felt that more than half of their impressions of the session were based on its novelty.

Discussion

We used a relatively expensive delivery method, in which an ISDN line carried video and audio from the lecturer to the students, and back. Because of the low bandwidth on the line, the frame rate was relatively low, although audio quality was very high. There were only very brief lags between the sites, making two way audio communication possible. Accordingly, students found the interactivity of the presentation to be only marginally, if at all, below that of a live laboratory.

Students prefer their tuition to be delivered in the style they are most familiar with (Casey, 1998). For the current generation of students, this means the presence of a live instructor, able to respond quickly to their words and gestures. Our novelty results confirm this: students were less satisfied with the novel laboratory session than with the standard ones. However, undergraduate students also are used to high quality visual aids. The web, even with its relatively low bandwidth, is ideal for carrying such visuals, in this case via a projected screen image. The visuals used in both portions of the remote delivery session were very familiar to students. The PowerPoint overheads were displayed just as they would be in a live lecture, although the remote lecturer had to use the mouse, rather than a pointer, in gesturing to visual elements. The BrainWave web site was familiar to students, as they had visited it during previous laboratories. Nonetheless, having to rely on the video based instructor more than a live one caused some residual dissatisfaction in the students when evaluating the tutorial component.

Finally students expect to be able to show their instructor their screen when asking questions about their exercises. While our trial included live local tutors, the NetMeeting software did allow remote access to student screens, at least for students on compatible machines.

The video conferencing system, while less expensive than a studio produced video, was still quite costly. Students in their homes would not have access to this technology, and most computer laboratories are not equipped to handle video. The student's internet connection is thus, in principle, available as a channel for the video link. However, such links are not necessarily interactive. Streaming video can be produced cheaply, but digitisation introduces substantial lags. Interactive video conversations are not yet possible from the average desktop. While graduate students may be relatively insensitive to changes in channel type (Rao, 1995), our results suggest that undergraduate students are sensitive to the degree of interaction available with their instructor, even when expensive technologies are used to support their interaction. Our trial was, in this respect, a success, as the perceived quality of the teaching did not diminish significantly. In other respects, however, we take our trial to be a caution. The pace of change in internet technologies is rapid, compared with the rate at which we can change our teaching materials. Thus, instruction over the internet is best modeled on existing instruction, with internet delivery only added as technology makes it convenient. The delivery of instruction using existing off the shelf technology was not as good as the local delivery of instruction, at least in terms of student satisfaction.

References

Catchpole, M. (1986). An instructors guide to producing and hosting a live, interactive telecourse. Distance Education, 7, 129-142.

Colbert, M., Voglimacci, C., & Finkelstein, A. (1995). Live, audio-visual communication systems for distance learning: experience, heuristics and ISDN. Behaviour & Information Technology, 14, 267-288.

Denley, I, Hedman, L., Hill, B., Whitefield, A. Papadopoulous, K., Clarke, A., & Hine, N. (1993). Usability principles for service design. In P. Byerley, P. Barnard & J. May (Eds.), Computers, communications and usability (pp 291-342). Amsterdam: North-Holland.

Fellers, J. W., & Moon, D. K. (1995). Distance education in the future: Exploring the application of distributed group support systems. Group Decision and Negotiation, 4, 273-286.

Hacker, R. & Sova, B. (in press). Flexible learning in a partnership context for beginning teachers. Journal of the Association for Learning Technology.

Hansford, B. & Baker, R. (1990). Evaluation of a cross-campus interactive video teaching trial. Distance Education, 11, 287-307.

Hiltz, S. R. (1992). Constructing and evaluating a virtual classroom. In M. Lea (Ed.), Contexts of computer-mediated communication. Harvester Wheatsheaf, NY.

Kirstein, P. & Beckwith, R. (1991). Experiences with the University of London interactive video education network. Electronics and Communication Journal, February, 4-12.

Mason , R. & Kaye, T. (1989). Mindweave: Communication, computers and distance education. Pergamon.

Nielsen, J. (1994). Heuristic evaluation. In J. Nielsen & R. Mack (Eds.), Usability inspection methods (pp 25-64). Wiley.

Pugh, H., Parchman, S., & Simpson, H. (1992). Video telecommunications for distance education: A field survey of systems in the U.S. public education, industry and the military. Distance Education, 13, 43-64.

Rao, V. S. (1995). Effects of teleconferencing technologies: An exploration of comprehension, feedback, satisfaction and role-related differences. Group Decision and Negotiation, 4, 251-272.

Stillings, N. (1993). Undergraduate education in cognitive science: Current status and future prospects. Report of a Planning Workshop for the National Science Foundation.

Whittington, N. (1987). Is instructional television educationally effective? A research review. The American Journal of Distance Learning, 1, 47-47.