Cognitive learning theory and the development of information literacy

A current focus in teaching and learning at the University of Canberra is the importance of developing two generic competencies to equip students for the workforce: cognitive skills, including problem solving and analysis; and information literacy.

In 1997 the researcher conducted an experimental study with 255 first year undergraduate students. The aim of that study was to determine the efficacy of a short teaching module based on learning theory paradigms in developing students' ability to retrieve information from electronic databases.

Teaching strategies emphasised the development of multiple concepts of the information retrieval process, the acquisition of realistic expectations of electronic database capabilities, and the modelling of a problem-solving heuristic.

Results suggested that a conceptual approach to teaching information retrieval increased subjects' likelihood of adopting the search methods employed by expert searchers.

Introduction

A small number of studies has suggested that information retrieval requires conceptual knowledge and problem-solving ability, as well as skills training in the mechanics of database operation. As Balaraman (1991) observed, a majority of researchers has found an interaction between some form of conceptual training and the task performed. However, although there is a strong theoretical basis for using a concept-based approach in the teaching of more effective information retrieval strategies, there is little empirical evidence of their efficacy (Oberman, 1991).

The researcher combined elements of three cognitive learning theories - transforming mental models, cognitive flexibility theory, and situated cognition - to design the teaching module used in this study. These theories suggest that learning will be improved through the teaching of new information grounded in existing knowledge; the modelling of procedures; the development of complex constructs through analogy and concept-based principles; and an emphasis on the web-like nature of knowledge.

The purpose of the teaching module was to develop a conceptual knowledge of, and to demonstrate a problem-solving heuristic for, use by novice end-users in information retrieval. Penhale & Taylor (1986, in Jacobson & Jacobson, 1993), observed that expert searchers display more citations, review more search sets, enter more search terms, adjust their search strategies more often, and change databases more often, than do novices. The teaching module was designed to encourage the development of some of these skills, such as using more search terms and adjusting search strategies more often, in novice end-users.

Methodology

Research design

Variables

Limitations

Firstly, a large sample size was chosen to enable control of intervening variables such as age, gender, and faculty of study. Secondly, the experiment was administered early in first semester, to a first-year cohort, in order to reduce the impact of knowledge of electronic databases. Thirdly, the study followed a pre-test, post-test, experimental/control group design, in order to ensure as accurate a measure as possible of any influence of the independent variable.

Hypotheses

Hypothesis 1a	There is no difference between experimental and control groups on pre-test (Survey 1, Section 2) scores. If this hypothesis is accepted, the conclusion that the experimental and control groups are equivalent in terms of their existing knowledge of electronic databases can be drawn.
Hypothesis 1b	There is no difference between experimental and control groups on post-test (Survey 2, Section 2) scores. If this hypothesis is rejected, it may be concluded that the teaching module influenced the development of electronic database knowledge, as intervening variables were controlled.
Hypothesis 2	There is no difference in search outcome (information retrieval assignment) scores depending on type of instruction. If this hypothesis is rejected, it may be possible to conclude that type of instruction influenced information retrieval assignment performance.
Hypothesis  3	There is no difference on information retrieval scores depending on problem-solving ability as measured by Survey 1, Section 3. If this hypothesis is rejected, it may be that problem-solving ability and information retrieval scores are related.

Results and discussion

With regard to education, 71.1% of subjects had completed Year 12 as their highest level of academic achievement. Eighteen per cent had completed a TAFE course, and 5% held a degree. The remaining subjects held an overseas Year 12 equivalent qualification.

Thirty three per cent of subjects had never used an electronic database before; another 26% had used them "once or twice"; 24% had used them only several times a year.

Hypotheses 1a and 1b

These results support a decision to accept hypothesis 1a, and to reject hypothesis 1b, leading to the conclusion that the teaching module may have assisted subjects to acquire electronic database knowledge.

Hypothesis 2

A Ratings Sheet was designed to enable the Information Retrieval Assignment data to be coded and analysed. Variables included: number of concepts identified; number of databases accessed; self-evaluation of search success; number of synonyms used; use of truncation; number of search strategy reformulations; use of Boolean operators; overall suitability of search strategies; and overall search success.

Results suggested that there were significant differences between experimental and control groups on a number of variables for each of the three research questions that comprised the Information Retrieval Assignment. Significant results are summarised in Table 1 below.

Variable	Experimental group (%)	Control group (%)	Statistical test
Topic 1
No. of concepts	18	12	z= -2.0898; p=.0366
Self-evaluation of search success "quite useful"/ "very useful"	19	50	Pearson chi-square 13.388; df 2; p=.00124
Topic 2
No. of concepts	78	54	z = -3.3337; p=.0009
"and/or" Boolean operators	13.2	6.7	Pearson chi-square 6.46643; df2; p=.03943
Topic 3
No. inappropriate concepts	64	48	t = 2.15; df 161; p=.033
"and/or" Boolean operators	20.3	5.4	Pearson chi-square 29.85; df 3; p=.000

For research topic 1, the experimental group used significantly more concepts than did the control group. They also used significantly more truncation, either correct or incorrect (Pearson chi-square value 15.418; df 3; p = .00149). Seventy-five per cent of control subjects used no truncation at all, compared with 58% of the experimental group.

For self-evaluation of search success on topic 1, the control group rated their search success higher than did the experimental group. This finding provides some support for the phenomenon of the "false positive" reported by Applegate (1993), which describes the tendency of novice end-users to report satisfaction with the results of poor searches. For topic 1, there was greater agreement between researcher ratings and experimental group self-ratings, than between researcher ratings and control group self-ratings. It is possible that experimental group self-ratings were more realistic than control group ratings because the teaching module had emphasised the importance of evaluating the relevance of sources located.

For research topic 2, the experimental group again used significantly more concepts than did the control group. Again, they used significantly more truncation, either correct or incorrect (Pearson chi-square value 15.806; df 3; p = .00124). Sixty per cent of the control group made no attempt to use truncation, compared with 45% of the experimental group. The experimental group used significantly more compound ("and/or") Boolean operators. This topic was rated by the researcher as the most straightforward of the three search topics.

For research topic 3, the experimental group used significantly more inappropriate concepts than did the control group, which may have reflected the emphasis of the experimental module on the importance of identifying as many terms as possible to use in a search. The experimental group used significantly more databases in their searches (z = -2.3554; p = .0185). As with topic 2, the experimental group used significantly more compound Boolean operators, compared with the control group.

Results regarding search strategy formulation as measured by use of truncation, number of concepts, and use of compound Boolean operators, support a decision to reject the null hypothesis, and lead to a tentative conclusion that the teaching module assisted subjects to develop appropriate search strategies.

Hypothesis 3

Log cross ratio analysis indicated that participants who scored above the mean on problem-solving questions were more likely to use more Boolean "and/or" operators, to use more truncation, and to design more successful search strategies, than participants who scored below the mean on problem-solving questions. Good problem-solvers were 90% more likely to score above the mean on electronic database knowledge than participants who scored below the mean on problem solving.

Further, significant correlations were found between problem-solving ability and number of synonyms (r = .2153; p = .026), number of concepts (r = .2132; p = .020), and number of reformulations (r = .1872; p = .056; Spearman's rank order correlation p = .046) on topic 2. There were significant correlations between problem-solving ability and number of concepts (r = .2071; p = .207) and number of reformulations (r = .2320; p = .028) on topic 3.

These results suggested that hypothesis 3 should be rejected, and supported a tentative conclusion that good problem solvers were more likely to formulate appropriate search strategies than poor problem solvers.

Conclusions

Previous research (Borgman, 1986) has suggested the phenomenon of the "permanent novice" in search behaviour, as end-users do not use electronic databases with sufficient frequency to develop expertise. The researcher's results to date, however, suggest that it is not the "novice" status of the searcher per se, but their problem-solving ability, which affects the formulation of good search strategies. This finding is important, as domain-specific problem-solving heuristics can be learned. Other correlates appear to be knowledge of electronic databases, and level of education.

Concept-based teaching was effective in assisting novice end-users to develop a realistic understanding of the capabilities of electronic databases. Modelling of a problem-solving heuristic aided the design of search strategies. However, it should be kept in mind that the experimental treatment was of short duration. The researcher is currently conducting further research to determine whether similar results will be achieved with a series of concept-based modules.

References

Balaraman, K. (1991). End user studies in CD-ROM environment. Proceedings of the ASIS Annual Meeting, 28, 283-294.

Borgman, C. (1986). Why are online catalogs hard to use? Lessons learned from information retrieval studies. Journal of the American Society for Information Science, 37(6), 387-400.

Bruce, C. (1998). The phenomenon of information literacy. Higher Education Research & Development, 17(1), 25-43.

Halpern, D.F. (1998). Teaching critical thinking for transfer across domains. American Psychologist, 53(4), 449-455.

Jacobson, F. & Jacobson, M. (1993). Representative cognitive learning theories and BI: A case study of end user searching. Research Strategies, 11, 124-137.

Julien, H. (1998). User education in New Zealand tertiary libraries: An international comparison. Journal of Academic Librarianship, 24(4), 304-313.

Oberman, C. (1991). Avoiding the cereal syndrome, or critical thinking in the electronic environment. Library Trends, 39, 189-202.

Yerbury, H. & Parker, J. (1998). Novice searchers' use of familiar structures in searching bibliographic information retrieval systems. Journal of Information Science, 24(4), 207-214.