|
| Teaching and Learning Forum 2004 [ Proceedings Contents ] |
Donald C. McDermid and Anita A. Bose
Edith Cowan University
The paper discusses the results of an action research study whose aim was to improve the quality of teaching systems concepts by using a software program that actually executes the models in order to re-inforce the concepts being learnt. The conclusions reached were that using such software in a structured learning environment did indeed improve the quality of teaching and had an impact on the ability of students to model problems more systemically. Further, lessons were learnt in the mechanics of how to structure laboratory exercises to support this goal.
While much of the teaching that goes on in IT courses is necessarily techno-centric focussing on the software and hardware technologies that underpin the knowledge economy, it has become widely accepted within the profession that there needs to be a balance between the techno-focus and the context or environment in which IT is undertaken (ACS Certification Program, 2003). This latter is usually referred to as the people or organisational context. Many coursework masters degrees in IT, for example those at the School of Computer and Information Science at Edith Cowan University, offer a number of units in this socio-organisational domain to complement the more technically oriented units and thus provide a more well-rounded preparation for working in the IT industry. Such units cover professionalism, ethics, legal issues, service level management, strategic planning and computer consultancy. The unit discussed in this paper is entitled 'software and systems management'. Its brief is to introduce students to a range of methods and concepts that empower an IT professional with respect to facilitating solutions, analysing problems, creating business cases and so on. A major part of this unit is the teaching of systems concepts as a means of modelling the underlying features of many situations that commonly occur in practice. It is important that students work in as 'authentic' a manner as possible.
As an example of the kind of problem discussed and modelled, please refer to Figure 1. Here, a typical organisational problem is depicted involving late orders, expediting orders and order delivery. The diagram should be read as follows. An increase in the number of late orders leads to a need to expedite more orders. Expediting more orders will increase the rate of order delivery. Increasing the rate of order delivery will reduce the number of late orders and thus have a balancing effect on the system. In Figure 1 the symbol 's' indicates 'same' or more formally an increase in X leads to an increase in Y and the symbol 'o' stands for opposite which means that an increase in X leads to a decrease in Y. The symbol B in the figure indicates that the system depicted is in balance. Or at least that is the theory. Figure 2 indicates that an unexpected side effect has occurred because expediting more orders has meant that there has been more disruption to production and so there has been an increase in missed production. Overall this means that the initial solution proposed (Figure 1) has actually had the opposite effect. In fact it has produced a system that will steadily get worse, because as more deliveries are missed, there will be more late orders and thus (as many systems find in practice) more pressure to expedite more orders and so on and so on. The 'R' in the lower part of Figure 2 stands for re-inforcing which means that the system will spiral out of control.
Figure 1: Naive scenario of a typical organisational problem situation
Figure 2: Realistic scenario of a typical organisational problem situation
adapted from Maani and Cavana (2000)
While this kind of modelling is an effective tool that provides significant enlightenment to students and provokes much discussion and debate in class, it doesn't go far enough or deeper in terms of providing understanding of the unique dynamics of any particular situation. For example, from a management perspective it would be valuable to know if it were possible to increase the rate of expediting orders to a certain level before production levels are significantly affected.
Clearly in order to accomplish this, it is necessary to have a quantitative model of the relationship between each of the above components of the 'system' and to be able to execute a running model in a similar way as we execute spreadsheets to answer 'what if' type questions such as the one above. Ithink software and its educational partner Stella (High Performance Systems Inc, n.d.) offer the opportunity for students to explore such problems quantitatively and therefore to obtain a much more intimate understanding of the nature (dynamics) of such situations. It is contended that this juxtaposition between theory and practice (or conceptual model and executable model) is a precursor to deep learning and provides valuable insight into the nature of the domain being studied.
Figure 3 is an example of a model produced by a student using ithink as part of an assignment. While it is outside the scope of this paper to introduce the detailed notation used in this diagram, the reader should note the following
Figure 3: An example of a solution produced by a student using the ithink software
The remainder of this paper describes a research project using action research in which an honours student was supervised in conducting a study to extend the Systems and Software Management unit to incorporate an additional component that, through laboratory exercises, coached postgraduate students in the use of the ithink software and attempted to measure the impact of this experience on their learning. The remaining sections are thus entitled research method, results and reflection.
... educative, problem-focused, context specific and future-orientated method for conducting qualitative research, involving individuals as members of social groups whilst aiming at improvement and involvement. It involves a change intervention and a cyclic process in which research, action and evaluation are interlinked. Action research has been founded upon a research relationship in which those involved are participants in the change process. Hart and Bond (1995, pp.256)
It should be apparent from the foregoing discussion that the scope of this action research project was socio-technical i.e. not just involving the technical or functional aspects of the computer software and hardware but also the social and organisational context in which the teaching took place. This context included at one end how the delivered unit actually fitted into the routine day-to-day operational environment of the students through to how systems concepts made sense to students in general.
The main research question was to ascertain the extent to which using the ithink software in a structured learning environment of laboratory exercises enabled students to create better mental models and solutions to complex problems, thus improving the academic achievement of students. A related minor aim of the research was to ascertain the number and nature of laboratory exercises that best supported the academic achievement of students. The research team included supervisor and research student as principal researchers and student group as participants.
The study was divided into five main phases for the purpose of supervising the student and took approximately one year to complete.
Figure 4: Operationalised research flowchart after Checkland (1991)
The following were used to conduct the research and collect the data.
'Thinking' was the label for those comments made by respondents that refer to some positive cognitive aspect relating to the laboratory exercises directly or the connection between the theoretical content and the laboratory material. Since the main research question concerned intellectual development and modelling, it was re-assuring to receive a high frequency of positive comments in this regard.
'Applying' refers to the emphasis placed on applying the ideas i.e. enabling the conceptual discussion to be taken through to a point where they can be implemented and simulated on a computer. Also, under this theme were comments that related to students finding that they could bring in outside knowledge and apply it to the problems given.
'Meeting study objectives' was the next most frequent theme. This was very much an holistic comment relating to the degree of satisfaction respondents felt about how this component supported and re-inforced their whole of unit study.
'Structuring of the course' was a label used to capture all suggestions for change or improvement with regard to mostly the laboratory sessions but also the unit itself. Details of these are discussed shortly.
'Solving complex problems' refers principally to statements about students' confidence in their new-found ability to embrace and solve complex problems of similar nature to the ones introduced in this unit.
'Interaction' was the last of the six main themes identified and refers to the feelings of involvement and participation felt by students throughout this whole research process. Yes, the Hawthorne effect is alive and well!
The fact that the students were framing the unit in the above terms suggests that they were conceptualising the benefits of the unit at a high level of abstraction. Rather than seeing the unit as 'easy' or 'hard work', the themes from the content analysis demonstrate a degree of intellectualising regarding the unit by the students which is consistent with the hypothesised research questions stated at the beginning of the paper.
Overall performance in the unit was better that previously both in assignments and examination with a greater number of high achievers. Low achievers were breaking up problems into constituent parts better than their previous counterparts and while they were not particularly able in synthesising solutions this represented still a milestone for them in their intellectual development. In general most students were able to discuss their problems openly and how they were overcome and this represented a shift from the previous cohort in terms of how they framed their discussion (previously discussion was more 'matter of fact' reporting usually only the basic logic of the model). Another noticeable difference was the frequency in which students brought in outside knowledge to bear on a problem or solution. For example, one student brought in his knowledge of 'S curves', a standard manpower planning technique common in project management. Perhaps the most remarkable area was high achievers. Of course every class has them, so that in itself is not remarkable. However, it was clear that a number of students were working ahead of the laboratory sessions or picking up techniques that were not part of the planned syllabus. For example, a number of students presented in their assignment a control panel (Figure 5), something which had not been covered up till that time. A control panel is a much more user-friendly and often more powerful means of allowing a user explore 'what if' type questions. In Figure 5, the user can run and reset the model at the touch of a button and move the sliders thus allowing different values and assumptions to be run through the model. These high achievers were exploring this effectively independently.
Figure 5: An example of a control panel submitted by a student
In terms of operating the laboratory sessions, a lot of useful experience was accumulated. Students preferred not to start a brand new problem each week. They preferred to build on previous weeks. That said, it was found prudent to start each new exercise with a revision or even test of previous accomplishments and to provide a starting model (i.e. file) that represented where students should have reached previously (otherwise the weaker students would fall behind). Another lesson was in terms of the manner in which material should be introduced. Fairly quickly we discovered it was wise to start each session by introducing only basics of a concept and get the students to practise that before covering more advanced aspects of a concept.
They say two heads are better than one. It certainly was the case that students brought forth very practical suggestions for improvement which frankly were not identified at the outset. Suggestions included worked demonstrations of aspects of ithink in addition to the traditional 'laboratory exercise'. It was also suggested that feedback and discussion sessions be held the following week in class from the previous week's laboratory exercise.
Bose A. (2003). Using action research to design a systems thinking module. Unpublished honours thesis, Edith Cowan University, Perth, Western Australia.
Checkland, P. (1991). From framework through experience to learning: The essential nature of action research. In Information Systems Research: Contemporary Approaches and Emergent Traditions. North-Holland: Amsterdam, pp. 397-403.
Hart, E. and Bond, M. (1995). Action Research for Health and Social Care: A Guide to Practice. Buckingham, Open University Press.
High Performance Systems Inc. (n.d). Systems thinking. [viewed 5 Nov 2003 at http://www.hps-inc.com/MoreAboutSytemsThinking.htm, verified 19 Jun 2004 at http://www.iseesystems.com/(mjp534vyd54d50mczvlnu0nd)/Community/STArticles/SystemsThinking.aspx
Maani K and Cavana M. (2000). Systems thinking and modelling: Understanding change and complexity. Pearson Education: New Zealand.
Mayo, E. (1933). The human problems of an industrial civilization. New York: Macmillan. Ch.3.
Miles, M. B., and Huberman, A. M. (1994). Qualitative Data Analysis. Thousand Oaks, California: Sage.
| Authors: Donald C McDermid and Anita A Bose School of Computer and Information Science Edith Cowan University, Joondalup, Perth, Western Australia Email: d.mcdermid@ecu.edu.au Please cite as: McDermid, D. C. and Bose, A. A. (2004). Using action research to embed systems thinking concepts in postgraduate IT teaching. In Seeking Educational Excellence. Proceedings of the 13th Annual Teaching Learning Forum, 9-10 February 2004. Perth: Murdoch University. http://lsn.curtin.edu.au/tlf/tlf2004/mcdermid.html |