|
| Teaching and Learning Forum 2002 [ Proceedings Contents ] |
Barbara de la Harpe
Faculty of Education, Languages and Community Services, RMIT University
Carmela Briguglio
CBS Communications Skills Centre, Curtin University
Employers, professional bodies, instructors and students themselves recognise the need for graduates to leave university with a good understanding of both disciplinary content as well as a range of highly developed professional skills. Many universities have responded to this need by encouraging the development of such skills in their programs of study. Curtin Business School has implemented the Professional Skills Project that aims to integrate the teaching and assessing of professional skills into the units of the Bachelor of Commerce degree program.As part of this initiative, the first author successfully won a grant to implement a project that focussed on supporting students' development of their presentation and written communication skills in his third year management unit. These skills were specifically selected based on the lecturer's previous experiences of teaching the unit which showed that while students in CBS are given opportunities to develop their of professional skills throughout the course, there seemed to be a significant variation in students' skill levels. The variation in skill development was particularly apparent in the assessment of group presentations and written assignments in semester one 2001. Thus, opportunities for students to develop these skills were integrated into the unit and data on students' perceptions of their skill development were obtained by them completing a questionnaire at the beginning and end of the unit.
In this paper, we outline how the skills were taught and assessed, present the data on the changes in students' perceptions of their skill levels, and discuss the implications for teaching and assessing presentation and written communication skills in the context of the discipline.
Universities are increasingly focussing on the need for developing core skills or competencies in their graduates as articulated by professional associations and industry (Rosenman, 1996). This is done against the backdrop of rapid change and high levels of competition in the labour market. By developing and integrating transferable skills or competencies universities hope to increase the employability of their graduates (Murphy, 1997). The dominant skill deficiencies of graduates as perceived by employers in the context of the Australian labour market include inter alia communication skills, interpersonal skills and problem solving and the understanding of business practice (ACNeilson, 2000).
Research to identify skills sought in new graduates by Australian employers lists academic achievement, time management, written business communication, oral communication, interpersonal, teamworking, problem solving and comprehension of business processes as the skills most often sought by employers (ACNeilson, 2000). These skills are sought in all graduates entering industry and Government Departments.
Many universities have responded to the need to produce graduates with the skills sought by employers by encouraging the development of such skills in their programs of study. As part of Curtin's goals and strategic plan for teaching and learning, which is to produce graduates who are well equipped for careers in their chosen fields (CBS(a), 2001), Curtin Business School (CBS) has implemented the Professional Skills Project (PSP) that aims to integrate the teaching and assessing of professional skills into the units of the Bachelor of Commerce degree program. The PSP aims to enhance both the professional skills and employability of CBS graduates (CBS(b), 2001). In the assessment of appropriate skills for CBS graduates by a cross discipline Task Force, the following five key professional skills were identified; communication (written, presenting and speaking out), teamwork, decision making, and computer and information literacy (CBS(a), 2000).
The approach towards teaching professional skills is based on a developmental process over the 3 years of students' undergraduate study (CBS(b), 2000). In line with the literature on skill development, it is deemed more effective to teach, apply and assess skills in the context of the relevant disciplines (Rosenman, 1996). According to Murphy (1997), while some skills are common to most disciplines, the focus and mode of development and the context for transfer might differ across the disciplines. In addition, Rosenman (1996) points out that although core skills should be agreed upon across the university or faculty, the implementation thereof should be done within individual subjects within departments. In line with this suggestion, CBS invited staff during March 2000 to express interest in undertaking a Teaching and Learning Mini Project to foster discipline based skill development and assessment strategies (CBS(c), 2001). The aim of these projects is to establish models and case studies for other staff and at the same time to build a core group of staff who will be 'champions' and 'ambassadors' for skill development within CBS (CBS(a), 2001).
Against this background and the lecturer's previous experiences of teaching the unit, these skills were specifically selected. The experience inter alia showed that while students in CBS are given opportunities to develop their professional skills throughout the course, there seemed to be a significant variation in students' skill levels. The variation in skill development was particularly apparent in the assessment of group presentations and written assignments in semester one 2001. The variation in skill levels of students can possibly be attributed to the fact that a large number of students are not primarily English speaking and the limited exposure to communicating in English despite them being third year students in Australia. Students are able to do a number of units in their undergraduate studies with partners off shore (Malaysia, Singapore, Hong Kong) and transfer to Australia to finish the outstanding number of units.
During the course of the 13-week unit, students, working in self selected groups are required to submit a written assignment of about 3000 words on a topic relevant to the unit. In addition, groups are expected to make two presentations on the assignment during the semester. Part of the CBS Professionals Skill project is that skills should be assessed (CBS(c), 2000) and in this unit the presentations and written assignment represent 40% of the final mark for the unit. Of this, about 60% is for the skills and 40% is for academic content.
Although students were also involved in developing their teamwork skills through working in groups, no support was given in their activities and interaction in teams. The teamwork activity can, therefore, be interpreted as a comparison activity in terms of the impact of providing or not providing support on skill development.
The questionnaire was developed in such a way that it served three purposes. Firstly, to collect data on the perceptions of students at the beginning of the semester (week one). Secondly, to collect data on the perceptions of students after the presentations (between week 7 and 13) and at the end of the semester (week 13). Thirdly, to indicate to students the actual criteria that would be used to assess the skills throughout the course.
All students attending lectures in International Management 375 on the Bentley campus were requested to complete the self-assessment. In week one there were 131 attendants while in week 13 there were 92 attendants. Presentation assessments were administered immediately after individuals had made presentations and a total of 116 assessments were collected.
After the data were collected in week one, a consultant from the Communication Skills Centre visited all classes and presented a 90 minute workshop on making presentations. The first part consisted of class discussion and input of ideas into what makes a good presentation. Discussion included ideas relating to principles of good communication as well as technical aspects such as appropriate use of overhead transparencies or PowerPoint. Students were then placed into groups and given a topic on which to prepare a brief 5 to 10 minute team presentation. Each group then presented using overhead transparencies developed in class. After the presentation, class members and the Consultant made friendly suggestions for improvement where appropriate.
Between weeks three and seven all groups had to make their initial presentation. Feedback based on the skills as identified in the questionnaire was given to each group of presenters. This implied that during these weeks students were exposed to at least two presentations per session. In addition, students were asked to assess the final presentations of their peers, thus giving them the opportunity to assess the skills of others.
The written assignment was to be submitted to the CBS Communications Skills Centre before being assessed by the lecturer. Students made appointments to meet with consultants at the Centre and submitted their draft assignment the day before the scheduled meeting. Consultants read assignments, underlined errors and unclear expression and wrote comments and suggestions for improvement on a sheet developed for this purpose, which had to be submitted to their lecturer with the final draft of their assignment. Group members then met with a consultant and discussed improvements for their assignment. It should be noted that assignments were not edited or corrected by Centre staff. Rather it was stressed to students that they were responsible for making necessary improvements and corrections, and Centre staff were there to provide advice on how this might be done. The perceptions of teamwork were collected to serve as a control item in the sense that no support was provided to students to prepare them to address problems with team interactions. Students had a free choice of team members in week one and had to complete both presentations and the written assignment as a team.
As mentioned, the data on students' perceptions of their teamwork skills were collected to serve as comparison data in the sense that no support was provided to students to prepare them for or to address any problems that they may have had with team interactions. As stated previously, students, in week one, had free choice in the selection of team members and had to complete both presentations and the written assignment as part of that team.
Pre and post data gathered were analysed in terms of the demographics and responses to the different skills (presenting, writing and teamwork). For the data on the skills effect sizes were calculated. Effect sizes were based on the calculation of the mean of the responses to each question and the change between the pre- and post-semester perceptions. The change was also interpreted in terms of its educational significance.
| Demographics | n | % | ||
| Gender | Male | Female | Male | Female |
| 55 | 76 | 42 | 58 | |
| Language background |
English | Non-English | English | Non-English |
| 51 | 80 | 39 | 61 | |
| Permanent residence |
Australia | Other | Australia | Other |
| 45 | 86 | 34 | 66 | |
| Age | Under 21 | 21 and over | Under 21 | 21 and over |
| 41 | 90 | 31 | 69 | |
| Units completed | n | % | ||||
| < 10 | 10 to 20 | > 20 | < 10 | 10 to 20 | > 20 | |
| Total | 11 | 72 | 43 | 8 | 55 | 33 |
| At Bentley | 59 | 37 | 30 | 45 | 28 | 23 |
| Skill implementation | n | % | ||||
| Yes | No | Yes | No | |||
| Does multiple reinforcement improve professional skills | 91 | 1 | 99 | 1 | ||
| Professional skills are developed in different ways by different units | 88 | 4 | 96 | 4 | ||
| Professional skills add value to my qualification | 92 | 0 | 100 | 0 | ||
| The amount of exposure to skill development is | Too little | Just right | Too much | Too little | Just right | Too much |
| 12 | 74 | 6 | 13 | 80 | 7 | |
Most students were near the end of their course, with 55% having completed between 10 and 20 units and 33% having completed more than 20 units of the 24 required to graduate. Incidentally, 45% of the students had completed less than 10 units at the Bentley campus in Australia.
Questions on the value and the exposure to professional skills development were included to determine whether the lack of coordination and integration across units, programs and schools within CBS was experienced negatively by students. Analysis of the data showed that the response to these questions was overwhelming positive and in favour of skill development. For example, as indicated in Table 1, all students indicated that professional skills added value to their qualification, while 99% agreed that multiple reinforcement improved their skill levels. Notwithstanding the lack of coordination in the teaching of skills, 96% of students indicated that they experienced the same professional skills being taught in different ways by different units. The level of exposure to skill development in the different units was assessed as just right by 80% of students. While 13% of students indicated that they would prefer more support for skill development in their units, only 7% indicated that they would prefer less exposure. The data suggests that the future inclusion of professional skills in units should be supported.
Data analysis of the student responses to the questionnaires administered at the beginning and the end of the semester are presented in Table 2.
| Item No. | Item description | Means | ES | N | ||||
| Pre | Post | change | g | sig | Pre | Post | ||
| 1.1 | I was comfortable presenting in English | 3.87 | 4.16 | 0.28 | 0.32 | * | 131 | 116 |
| 1.2 | I was confident when talking in front of an audience | 3.18 | 3.73 | 0.56 | 0.68 | ** | 131 | 116 |
| 1.3 | I was able to use aids effectively to improve my presentation | 3.66 | 3.90 | 0.24 | 0.33 | * | 131 | 116 |
| 1.4 | I was comfortable speaking in a loud and clear voice | 3.50 | 4.02 | 0.51 | 0.65 | ** | 131 | 116 |
| 1.5 | I was able to effectively make eye contact with the audience | 3.47 | 3.90 | 0.43 | 0.49 | * | 131 | 116 |
| 1.6 | I found it easy to structure my presentation | 3.44 | 3.85 | 0.41 | 0.55 | ** | 131 | 116 |
| 1.7 | I was able to present without having to read from notes | 3.04 | 3.61 | 0.57 | 0.62 | ** | 131 | 116 |
| 1.8 | I found it easy to stick to the time allocated | 3.33 | 3.86 | 0.53 | 0.63 | ** | 131 | 116 |
| 1.9 | I could keep the audience interested | 2.99 | 3.50 | 0.51 | 0.64 | ** | 131 | 116 |
| 1.10 | I felt comfortable when asked questions by the audience | 3.09 | 3.76 | 0.67 | 0.72 | ** | 131 | 116 |
| 1.11 | Overall, I was a good presenter | 3.19 | 3.79 | 0.60 | 0.82 | *** | 131 | 116 |
| Presentation Skill (average items: 1.1 to 1.11) | 3.34 | 3.83 | 0.48 | 0.54 | ** | |||
| 2.1 | I find it easy to express myself in English | 3.80 | 4.04 | 0.24 | 0.26 | * | 131 | 92 |
| 2.2 | I have no problems with grammar and spelling | 3.46 | 3.62 | 0.16 | 0.16 | 131 | 92 | |
| 2.3 | I find it easy to write in an academic style | 3.37 | 3.65 | 0.28 | 0.34 | * | 131 | 92 |
| 2.4 | I can construct an informative introduction | 3.44 | 3.71 | 0.26 | 0.35 | * | 131 | 92 |
| 2.5 | I am comfortable in writing a comprehensive summary | 3.35 | 3.62 | 0.27 | 0.37 | * | 131 | 92 |
| 2.6 | I am able to develop a suitable structure for the content | 3.40 | 3.85 | 0.45 | 0.65 | ** | 131 | 92 |
| 2.7 | I know how to present a report professionally | 3.31 | 3.76 | 0.45 | 0.53 | ** | 131 | 92 |
| 2.8 | I find it easy to locate relevant resources | 3.33 | 3.74 | 0.41 | 0.56 | ** | 131 | 92 |
| 2.9 | I know how to reference when using the exact words of the author | 3.63 | 4.00 | 0.37 | 0.44 | * | 131 | 92 |
| 2.10 | I know how to reference when paraphrasing the words of the author | 3.61 | 3.95 | 0.33 | 0.39 | * | 131 | 92 |
| 2.11 | I know exactly how to construct a reference list | 3.81 | 4.15 | 0.34 | 0.42 | * | 131 | 92 |
| 2.12 | Overall, I believe I am a good writer | 3.28 | 3.66 | 0.38 | 0.55 | ** | 131 | 92 |
| Writing Skill (average items: 2.1 to 2.12) | 3.48 | 3.81 | 0.33 | 0.40 | * | |||
| 3.1 | I find it easy to work in a team using English as the language of communication | 4.09 | 4.10 | 0.01 | 0.01 | 131 | 92 | |
| 3.2 | I find it easy to stick to the preset time frames | 3.73 | 3.80 | 0.08 | 0.12 | 131 | 92 | |
| 3.3 | I try to make an equal contribution as a team member | 4.27 | 4.21 | -0.06 | -0.09 | 131 | 92 | |
| 3.4 | I am sensitive to needs of members from other cultures | 3.74 | 3.90 | 0.16 | 0.19 | 131 | 92 | |
| 3.5 | I realise that not all members show the same commitment and sense of responsibility | 3.93 | 3.99 | 0.06 | 0.06 | 131 | 92 | |
| 3.6 | I prefer being the team leader | 2.97 | 3.28 | 0.31 | 0.34 | * | 131 | 92 |
| 3.7 | I am able to accommodate different attitudes and approaches | 3.71 | 3.75 | 0.04 | 0.06 | 131 | 92 | |
| 3.8 | I am willing to learn a lot from other team members | 4.30 | 4.12 | -0.18 | -0.27 | * | 131 | 92 |
| 3.9 | I enjoy working in a team | 3.95 | 3.62 | -0.33 | -0.37 | * | 131 | 92 |
| 3.10 | Overall, I believe I am a good team worker | 3.92 | 4.10 | 0.18 | 0.28 | * | 131 | 92 |
| Team Working Skill (average items: 3.1 to 3.10) | 3.86 | 3.89 | 0.03 | 0.03 | ||||
| Note: Large effect size = *** (g>0.8), Medium effect size = ** (g>0.5), Small effect size = * (g>0.2) | ||||||||
As shown in Table 2, of all the skills, the greatest improvement in students' perceptions of their skill levels was for the presentation skill (g=0.54) Data suggests that the greatest change in this skill was due to students indicating that they believed that they were better presenters at the end of the semester that at the beginning of the semester (g=0.82). Amongst presentation skills there were six other aspects that showed medium effect size improvements (g>0.5) and there was no aspect that had no effect size improvement. The aspects of the presentation skill that students rated themselves highly for in the pre-semester assessment were also rated highly in the post-semester assessment.
The improvement in writing skills was less significant than for the presentation skill, the overall effect size of the improvement of writing skills was 0.4. There were only four skills where students indicated to have improved more than 0.5. Although the aspects of the writing skill rated the highest pre-semester were still rated the highest post-semester there was a change in the skills perceived to be worst pre-semester compared to post-semester. In the pre-semester assessment the skill rated the lowest was the overall perception to be a good writer while in the post-semester assessment grammar, spelling and the writing of a comprehensive summary were the worst skills.
The experiences of students in terms of teamwork skills were significantly different. Apart from the fact that there were no significant changes in pre-semester and post-semester assessment for 7 of the 10 skills, there were two skills that have deteriorated. Students indicated a small effect size (g.0.2) deterioration in their willingness to learn a lot from other team members and their enjoyment of working in a team. Overall, the change in students' perceptions of their teamwork skills indicated no significant change.
Data in this paper suggests that the more students were exposed and supported in the development of professional skills, the more confident students became in these skills or ...the more they perceived that their skills had improved/developed. The continuous exposure of students to presentations, both of themselves and peers throughout the unit seems to have contributed to the most significant improvement in the self assessed skills. Writing skills were supported by inputs from the CBS communications skills centre and subsequently also showed an improvement. The group of skills where no support was provided, namely teamwork, showed no improvement from students and even indicated that some of these skills were worse after the semester than they were perceived at the beginning of the semester.
Data also suggested that the random inclusion of professional skill development, as currently applied in CBS units, is appreciated by students. Also, the actual development of skills seem to be positively influenced by providing support and exposure to the criteria used in the assessment of the skills.
CBS (2000a). Skills Curtin Business School has chosen. [verified 20 Dec 2001] http://www.cbs.curtin.edu.au/staff/PSP/CBS_skills.cfm
CBS (2000b). Employers and students. [verified 20 Dec 2001] http://www.cbs.curtin.edu.au/staff/PSP/what_emp.cfm
CBS (2000c). Assessing professional skills. [verified 20 Dec 2001] http://www.cbs.curtin.edu.au/staff/PSP/Assess.cfm
CBS (2001a). Professional skills project: Mini Projects background. [verified 20 Dec 2001] http://www.cbs.curtin.edu.au/staff/PSP/background.cfm
CBS (2001b). CBS professional skills project. [verified 20 Dec 2001] http://www.cbs.curtin.edu.au/staff/PSP/Page_3.cfm
CBS (2001c). Professional skills project: Mini projects. [verified 20 Dec 2001] http://www.cbs.curtin.edu.au/staff/PSP/mini_projects.cfm
Ingleton, S. (1996). Changing literacy: Changing teaching and learning. Paper for the First National Conference on Tertiary Literacy, Victoria University of Technology, March 15-16.
McInnes, C., James, R & Hartley, R. 2000. Trends in the First Year Experience in Australian Universities. Department of Education, Training and Youth Affairs, Canberra.
Murphy, E. (1997). From university to work: Developing transferable skills. In Pospisil, R. and Willcoxson, L. (Eds), Learning Through Teaching, p239-243. Proceedings of the 6th Annual Teaching Learning Forum, Murdoch University, February 1997. Perth: Murdoch University. http://cleo.murdoch.edu.au/asu/pubs/tlf/tlf97/murp239.html
Promnitz, J & Germain, C. (1996). Student Support Services and Academic Outcomes: Achieving Positive Outcomes. Department of Employment, Education Training and Youth Affairs, Canberra. [verified 20 Dec 2001] http://www.dest.gov.au/archive/highered/eippubs/student/eip96-10.htm
Rosenman, L. (1996). The Broadening of University Education: An Analysis of Entry Restructuring and Curriculum Change Options. Department of Employment, Education, Training and Youth Affairs, Canberra. [verified 20 Dec 2001] http://www.dest.gov.au/archive/highered/eippubs/eip9612.htm
| Contact person: Werner Soontiens, Lecturer, Management, Curtin University of Technology. soontiew@cbs.curtin.edu.au
Please cite as: Soontiens, W., de la Harpe, B. and Briguglio, C. (2002). Assessing professional skill development at a third year level. In Focusing on the Student. Proceedings of the 11th Annual Teaching Learning Forum, 5-6 February 2002. Perth: Edith Cowan University. http://lsn.curtin.edu.au/tlf/tlf2002/soontiens.html |