A D R I A N

K J Ow E T T

Did the market force Subject Review? A case study

published or not. The documents may come

L'archive ouverte

Introduction

Subject Review (SR) is the process of Quality Assurance (QA) in Higher Education (HE) in place in England and Wales, and the first cycle occurred between 1998and 2000(QAA, 1997)). Although this particular QA exercise has been short lived, in a single country, it is part of a process that has affected HE in most developed countries for at least a decade (e. g. Crebbin, 1997). Many authors consider that QA programmes in the UK were born from the political environment of the 1980s that enforced efficiency gains across the whole of the public sector (Pollitt, 1987). However this underestimates the international imperative to develop QA active learning in higher education procedures in HE. Despite there being no evidence of a causal relationship between HE provision and national development, the OECD have suggested that increased access to HE and competence in the population are vital to global competitiveness (Woodhouse, 1996). Over the last 40 years there has certainly been increased participation in HE, as exemplified by the expansion in the UK from 6.5 per cent of school leavers in 1963 to 37 per cent in 2002 (DfES, 2002). Unfortunately, governments have become concerned that the expansion has been achieved at the expense of quality, which might in turn adversely affect economic health. Ironically through much of this period of expansion, public funding of HE has declined; for instance by 30% per student in England between 1991and 1996(Forth, 1996).

The publication of the Jarrett Report in 1985 (Jarrett, 1985) was a significant landmark for QA in English HE as it linked future development of HE with the introduction of Performance Indicators (PIs). PIs are a managerial construct used to compare a process both across time and between institutions (Pollitt, 1987). Such tools are used to measure an organization's effectiveness and efficiency, central tenets of the Conservative Government's policy (Sizer, 1988) at that time. Good PIs will demonstrate an organization's success and therefore are also a means by which consumers can make informed choices about where to obtain services (Pollitt, 1987). In this way HE is exposed to market forces. In 1988 the Quality Assurance Agency (QAA), charged with QA in England and Wales, decided to measure the teaching system rather than teaching itself (described by Elton, 1988). The PI approach chosen and refined for SR considers that quality HE can be delivered by academic staff becoming managers of learning (Brennan et al., 1997) working towards a set of defined institutional objectives. Notably, this stance separates the educational process from its outcomes, in keeping with a political will, at that time, to insist that academic standards are a matter for HE institutions (Forth, 1996).

This study seeks to explore the views of a cohort of academic staff regarding the external QA process of SR. The department studied embodies key aspects behind the current QA imperative: its non-clinical courses have seen a dramatic increase in numbers from around 30 entrants per annum in 1990 to 120 in 1998; prospective students have a choice of similar courses at surrounding pre-1992 and post-1992 (former polytechnic) universities; and the university prides itself on being a prestigious institution providing a high quality education. Additionally, in this context of increasing student load the staff group also teach basic medical sciences to gradually expanding cohorts of pre-clinical dental and medical students. This paper provides a detailed insight of how a particular staff cohort believes that SR might influence HE practice and quality and affect the position of their Department in the market.

Method

Sample and interviewing

The Head of Department of the Department of Biomedical Science gave the author permission to interview staff; 15 of 21 academic staff agreed to participate and were interviewed between September and November 1998. The participants included a range of academic staff including the Head and Deputy Head of Department, established readers/lecturers and newly appointed lecturers on probationary contracts. The non-participants were all established staff; two refused to be interviewed, the other four failed to reply to written and email messages. To discover both the form and reasons for staff perceptions an 'active interview' (Holstein and Gubrium, 1995) was adopted to build up an image of the respondent during the session. The interviews were conducted as an open discussion of the issues of SR and how they impinge on the department. An interview guide (see Figure 1) based on the aims of the study and the issues in the SR handbook (QAA, 1997) was referred to during the interviews to centre the discussion on key topics. Nonetheless, the respondents' ideas were the focus and so the path of each interview was unique.

Records and analysis

With the consent of respondents, all interviews were tape recorded and detailed written notes taken. Interviews were no longer than 90 minutes and tapes were transcribed within a week. The notes and transcripts were read line by line and, in a way derived from Grounded Theory (Glaser, 1967), each new theme became a category and each related phrase was fragmented and indexed into a cell of a word processor table. All of the interview data were collated into this table finally containing 1181 phrases sieved into 134 preliminary categories. The categories within the table were reviewed and compared with one another and those with similar themes coalesced. This exercise was repeated twice, reducing the data to 36 categories. Relevant data were collated into seven categories core to the study and the remainder, where discussion had strayed beyond the study, discarded.

Results

Twenty-two hours of interview yielded a considerable quantity of data. Data analysis discovered links and contradictions between respondents in three key areas:

• the origin of Subject Review • the process of Subject Review • external impact of Subject Review.

The origin of Subject Review

All respondents agreed that SR was 'a good thing' but had diverse objections to it. They described how SR arose through the ethos of market forces. Some staff had an intrinsic belief in the market's efficacy:

My own preference [for funding] is outright market forces. You end up with the right result.

whilst others described a social structure which forces HE to follow the political environment derived from Thatcherism of the 1980s: The six aspects of provision

A C T I V E L E A R N I N G I N H I G H E R E D U C A T I O N 6(

BMS and in general

Figure 1 The interview guide including a checklist to use with a respondent immediately prior to the interview and key headings to focus the session

There is now an ethos of everything, not just teaching and research, being counted and assessed so we accept.

This passive acceptance of managerialism, voiced by several staff, was in contrast to statements of how academics prefer to work autonomously:

The independence is, I suspect, one reason why people of a naturally rebellious nature gravitate towards university work.

Respondents reconciled these views by acknowledging their responsibility to society through looking at HE from the perspective of either parents or taxpayers:

There is always a need for people to be accountable . . . if I wasn't involved in academia I would want an objective system to provide information on whether I was spending money appropriately.

A new, young staff member suggested that he thought this was due to being socialized during the 1980s, although the issue of accountability was raised across the age spectrum. Certainly there were many comments that HE had been too introverted and independent for too long. The concept of stakeholders in HE can collate ideas about to whom HE is accountable and what constitutes appropriate funding.

The role of stakeholders

Students are the obvious customers of HE but there was not a consensus of how staff considered students to view this role, even in relation to the increase in numbers. In one case it was suggested that the increase in student numbers had resulted in them being less keen:

You have lots of students coming through with a fair proportion of them not really knowing what they are doing here, not overly interested in the subject matter.

whereas another respondent located failure within the department itself: . . . our problems come from the way we teach them. We have too many to do it effectively in the way we do it.

Another view put forward in several ways was that the expectations of staff and students are not congruent: and that attempts to align expectations resulted in personal criticism as students may not recognize good teaching and staff are unclear of the department's role in society.

It was stated that the university was established to serve the local community but there was disagreement as to what this entailed. This lack of clarity was exemplified by attitudes to the role of employers in shaping courses. A simple view of the market suggests that the department will be successful if staff:

. . . include what we think is appropriate and we will be judged in terms of the people doing the course by employers.

whereas other staff directly opposed this view:

It is a course for your own personal development. This is one of the privileges of being able to do a university course. I defend that, I'm opposed to the idea that courses should have a prescribed outcome and that you enrol at 18 to fill a role in later life.

Furthermore, it was suggested that very few employers directly fund HE and therefore have little right to influence courses directly. As the vast majority of HE is funded by central government, staff noted that it is the most powerful stakeholder, even though money comes from taxpayers. This powerful axis was considered to have a poor impression of HE institutions and staff:

. . . we are not understood by the rest of society. We think that they think little of us . . . One respondent suggested that in part this resulted from HE staff having a 'free ride' in the 1960s, but many others explained that the poor impression arose from ignorance of the diverse tasks undertaken by a highly competent staff: Public perception would be amazed that universities are appointing people with the experience they have for the salaries. We are not here for the money but the level of work and responsibility is above that perceived.

Either way, staff felt that HE was in a position where 'we couldn't prove we were good' to the diverse stakeholder groups resulting in the imposition of SR.

The process of Subject Review

The majority of staff stated that they did not fully understand the process of SR despite having seen the QAA's documentation. Nonetheless they were well aware that SR examines the process of Management by Objectives (MBO) of teaching rather than the teaching itself. A high proportion of staff accepted that assessment of the difference between intentions and achievements and 'do we have the means to do it?' are good indicators of a department's QA. Importantly a number of respondents voiced concerns that a department could, in theory, falsify its entire management cycle, or at the very least construct an impression of reality:

Assessors are not really seeing what's going on. They are seeing what the department has been preparing for 18 months to show it More positively, most staff believed that their department had made a genuine attempt to involve all staff and that the generation of the requisite paperwork had frequently been associated with procedural improvements: . . . staff are striving to get a good score, but in doing so they are improving quality by updating practice and starting to document what was often taking place already.

However a few staff suggested exactly the converse:

It feels a bit like anything political, that it becomes a game to make the paper whilst carrying on with what they were doing before.

and they all reported that the process was excessively burdensome and probably distracting already over-stretched staff from other duties.

External impact of Subject Review

Subject Review appeared to have caused staff to acknowledge a need to change to respond to the climate of measurable excellence. They described how opinions on the quality of HE in the past were based largely upon hearsay and that SR may provide an opportunity to say something objective about quality in a given department. Many staff believed that a positive outcome of SR, and to a large degree independent of its findings, is the increased profile of teaching in HE and the significance of it within the work of academics:

Importantly it increases the status of teaching within universities and makes all staff, including the VC, realize that this is an intrinsic part of our activity and it should be good . . . Notably several staff commented that SR had forced the vice-chancellor (VC) to take a greater interest in teaching to maintain the university's reputation and that a poor outcome in SR may lead to closure of affected departments, establishing the idea that '. . . there should be some consequence of the outcomes'.

SR and the market

Staff were well aware that the QAA are answerable to the public and committed to publishing their findings. This was seen to be beneficial as it would be a means for HE to demonstrate its accountability to stakeholders and thus strengthen its position in society:

At the end of the day we have to have a mandate from the taxpayer to allow us to do what we do. If we can demonstrate that we are fully accountable and that our product is an important one, then that mandate will get stronger and thus define our role. This could potentially be a positive outcome.

The published reports were frequently discussed as a potential factor affecting student recruitment to the department. A good report was suggested to be a 'good marketing tool' as informed schools and parents would use such reports when advising students, especially as many parents will have adopted the 'measure it' ethos. Most staff explained that they believed the reports would be simplified into 'league tables' by newspapers:

What's this got to do with teaching quality? . Such tables were seen to be an element that would contribute to a holistic impression of an institution more than the full report on a specific department. The image of a university was seen to arise from an impression of quality, geographic factors such as access and local attractions and the preferences of friends and family. Many respondents thought that SR may play only a minor part in this: I think, even now, that geography is probably more important to entrants than quality. For example, [university name] is at best mediocre, but is very popular as it is near major conurbations.

Staff also explained that they realized they had prejudices concerning the quality of HE institutions and suspected that they, and the families of prospective students, would have difficulty in changing their opinions on the basis of SR alone:

It will need several generations of graduates for the written evidence to overcome past prejudice.

Nonetheless, most respondents considered that prospective students and their advisers would use SR to an extent. It would thereby influence recruitment to some degree and thus have secondary financial consequences. They acknowledged that QAA and HEFCE had not linked future HE funding to QAA findings but suspected: . . . that the VC will use it for internal funding distribution.

A few respondents discussed how a poor result may show that a

A C T I V E L E A R N I N G I N H I G H E R E D U C A T I O N 6(1)
department warranted additional support but doubted that this would occur; likewise they did not anticipate a reward for a good result:

We get high marks or low marks and suffer the horrible consequences either way. I'm cynical . . . we can't win.

Collectively, these data show that the respondents held a range of often contradictory views.

Discussion

Qualitative analysis of these data revealed a spectrum of attitudes to SR. All staff considered that the political imposition of market forces upon HE had led to QA procedures but there was a range of views of the desirability of this. Respondents were able to describe the managerial rationale for the QA process but there was a consensus that it may be possible to achieve a high SR score without delivering what staff believed is a quality education. Although SR may well increase the 'marketability' of the department, staff described how its impact was likely to be minor and that the most significant outcome may be an increased status of teaching within HE institutions. This sample of staff, like others before them, are generally supportive of the principle of SR although dubious of the precise method (Sanders, 1994). Acceptance for SR arose from an acknowledgement that the academic profession had failed to justify itself to other HE stakeholders so that assessment was overdue (Howarth, 1995). By encouraging staff to view themselves as stakeholders in HE as academics, taxpayers and parents they outlined how they felt some personal desire to demonstrate their skills and provide external accountability. However there were concerns that SR was probably not the optimum tool.

Prior to the introduction of teaching quality assessment it had been suggested that institutions would not comply with imposed QA systems (Husbands, 1992) and this scenario was discussed with respondents. It was widely suggested that the converse was more likely and there were frequent references to pressures from 'the institution' being eager to prove itself in the HE marketplace. This aspect was clearly prominent in respondents' minds as they had well developed arguments to explain the degree to which SR would affect recruitment based on their views of society's acceptance of the 'evaluative state' (Brennan et al., 1997). Thus respondents who suggested that the public had little faith in the use of PIs in public service would be more concerned by the particular circumstances of a department. For example, staff explained how the department benefited from its location and this probably strengthened its position in the market and enabled it to attract an excess of UCAS applicants. This confirms the view of Tight (2000) that in certain institutions geographical factors are more important than high SR scores. The uncertainty over the impact of SR data on student intake was no different to the views on the impact of SR on other stakeholders. Moreover there was little agreement on the power that such stakeholders should wield:

We are trying to decide on behalf of the nation what skills may be useful.

This statement encapsulates a current problem within HE as the proportion of graduates in the workforce increases. Staff recognized that in the past they were providing a course designed to train a few career scientists whereas now the course is for a large cohort of students with diverse aspirations and different career opportunities to their predecessors. In particular this was suggested to conflict with the department's intention to provide research-led teaching, much of which may be inappropriate to the student cohort. This situation illustrates a potential weakness in SR in that the target objectives are set by the HE providers, and therefore the outcomes of the educational processes are distinct for each provider. Quality is sometimes described as fitness for purpose (Juran, 1979), but if that purpose is illdefined by society, even if a given department fulfils its objectives, how can the educational quality be assessed? This is probably why the majority of PIs used in HE relate to the more readily quantified efficiency in training rather than quality of learning (Ball and Halwahi, 1987).

The discussion of these arguments by respondents revealed a thorough understanding of how the MBO process of SR would be applied to the department. Many respondents presented opinions on the limitations of the application of PIs to HE congruent with those discussed by Ball and Halwahi (1987), implying that comments denying an understanding of SR were illfounded. In a similar fashion, staff were concerned by the absence of clear outcomes of the SR process at all levels from government to department. Such an environment encouraged rumours and cynicism and probably added to the belief that SR is largely a paper exercise. It may even serve to perpetuate the belief that teaching is considered less important than research when the Research Assessment Exercise (RAE) is a vital factor in providing funding for research. The use of a numeric scale to grade the assessment was also seen as evidence that an unwritten aim of SR may be to reduce funding by closing poorly ranked departments.

A stated aim of SR was to improve quality (QAA, 1997) but the respondents were doubtful that it would do this effectively. It was suggested that by producing the documentary evidence of the department's MBO systems these systems would themselves be improved, leading to improved QA. The most significant impact was suggested to be ensuring that staff examine their practice, leading to a change in normality. Naturally many staff believed that the HE provision in their department and elsewhere was already of a high standard and suggested that in itself SR was wasteful and drawing staff away from other duties. One respondent correctly predicted that the QA process of 2000 would be unique and not repeated. The aims and objectives of the 2002-2005 Institutional Audit (IA) QA process are comparable to those of SR (QAA, 2002). There are, however, two key differences.

First, the new process takes place at an institutional level and assumes that they will have in place detailed audits of their teaching processes in each discipline. Second, the use of Subject Benchmarks provides a framework for attainment. Audit of the institution rather than every subject area should reduce the cost of IA to both QAA and HE institutions. During SR there were 5802 opportunities for unsatisfactory grades to be awarded but only seven were given (Underwood, 2000), implying that SR was an expensive way to identify a minimal amount of poor provision. The IA process retains the MBO approach but accepts many respondents' view that SR was an excessive burden on individual teaching departments. Review at the institutional level occurred a decade ago in Australia so as to encourage holistic changes to planning and management (Harman, 1996). Several respondents suggested that as some staff had been trained as QAA assessors there might be an opportunity for internal assessment and dissemination of good practice in the future. To an extent this has occurred with the requirement that disciplines must demonstrate internal 'audit trails'. This also encourages maintenance of systems developed by departments for SR, which if adequate, negate the need for visits (Elton, 1988). Moreover, determining the degree of inspection required based on past performance is congruent with other governmental QA processes, for example Ministry of Defence procurement (Alderman, 1996b). Respondents had described that they believed post-1992 universities would have better QA systems but that pre-1992 universities may provide better teaching. IA avoids the pitfalls of determining the level of inspection by maturity (Alderman, 1996a) by reviewing the institution's QA system just eight weeks before the visitation. Respondents considered that the published reviews of departmental SR would support the image of an institution and this possible confusion for stakeholders has been removed by the publication of institutional reports.

Second, by including reference to Subject Benchmark Statements the new process has included an element of defining frames of reference for a course, but avoided prescription of course outcome. This starts to address the 'purpose' aspect of quality by establishing what is understood to be required by stakeholders of a particular HE course. A key aspect in which IA retains features of SR is recruitment of auditors. Respondents discussed what made a good assessor for SR (data not shown) and the principal factor was that they should be academic peers if the process was to be credible to staff within the assessed unit. This may reduce the credibility of the process to other stakeholders but is essential for professional quality improvement (Elton, 1988).

These interviews were undertaken before the department's SR and by their own admission staff were relatively ignorant of the detail of SR. However, their involvement in the QA processes is evident from their insight into the underpinning of SR and that the new proposal incorporates many of their concerns. One, now retired, respondent summed it up well:

Quality wasn't low and won't necessarily improve but the key impact is that we will have systems which allow us to monitor and correct if elements fall down.

By identifying the views and beliefs of 'frontline' academics this case study has shown that they accept a need to be accountable to other stakeholders in HE but they suspect the emergence of QA to be driven by political dogma rather than a genuine desire by the administration to improve quality. Staff weighed up their attitudes as academics, taxpayers and parents and predicted that SR would provide a small benefit to the department in the student marketplace. Despite their self-proclaimed ignorance of QA processes they correctly identified limitations of the SR process and predicted features of the new system introduced for 2002/03. This suggests that the respondents had engaged thoroughly with the need for QA and to this extent SR was successful in this department. It is impossible to assume that this case study reflects the views of staff in all departments of HE institutions. The department studied is in a pre-1992 university with an institutional objective to provide research-led teaching. Respondents and those in other studies (Alderman, 1994) have suggested that post-1992 universities had well established QA procedures and would find SR simple. However the converse has been found to be true in both England and Australia (Harman, 1996) as well-funded established institutes tended to gain higher scores (Alderman, 1994;1996a).

Although research and teaching are both learning processes they are assessed through separate PI processes in England. Unfortunately the products of the two (i.e. papers and graduates) are the focus of assessment rather than the underlying learning by both staff and students. This dichotomy may cause institutions to adopt managerial processes for audit rather than quality enhancement (Rowland, 2000). It may be, therefore, that excellence in preparation for the RAE confers an advantage when approaching SR. General theories on the impact of SR, or other QA processes, may be drawn if the study were extended through 'triangulation' -a process analogous to finding one's position by reference to landmarks (Strauss and Corbin, 1990). This requires further sampling to enrich the data, possibly by undertaking a similar study in another department or by obtaining data from other stakeholder groups. Nonetheless, the findings are largely in accordance with those of other studies. In surveying the impact of TQA, which immediately preceded SR, Underwood (2000) summarized that the major benefit had been to improve departments' self-criticality and QA housekeeping processes. Students' impression of quality education has been reported to centre on the staff/student relationship and the flexibility and effectiveness with which staff facilitate learning (Hill et al., 2003). It is unlikely that SR assessed this at all nor that IA will, as it is difficult to define the objectives of dynamic teaching (Ball and Halwahi, 1987). These QA systems are designed to assess if HE and its staff are competent performers and cannot identify the flexible experts (Dreyfus and Dreyfus, 1980) sought by the most questioning students.

  your input been requested? 2. Are your views / work taken seriously? 3. Do you want to be more / less involved? it a 'good thing'?

  . . . we have a problem with the types and expectations of students not necessarily meshing with what we as a [research-led] university expect of them .

  .

  .

1) 76 Subject Review Interview Checklist At outset

	1. Thank participant
	2. Explain that I want their opinions about the areas they feel important and that session therefore unstructured
	3. Ask permission to record and note that they can stop if required
	4. I'll take some notes to help records
	Things to try to cover
	2. Are the criteria valid?
	3. Is it done appropriately?
	4. Will it improve quality?
	5. Are the reports useful to the public?
	6. Why have we agreed to do it? rebel?
	The rationale of subject review
	1. Value for money
	2. Encourage quality
	3. Public information
	The process of review
	1. The review is by (self-) selected peers
	2. Self assessment against Aims and Objectives
	3. Review visit Ð are you apprehensive?
	4. Publication of dept. & nationwide reports

  . . 'League tables' is what we are taking about . . . it's the bottom line isn't it. To produce reports that A is better than B. Well I can understand this.

J O W E T T : D I D T H E M A R K E T F O R C E S U B J E C T R E V I E W ?

Acknowledgments

I am grateful to Professor P. W. Andrews and the staff of the Department of Biomedical Science for their cooperation in this study. I also thank Professor P. G. Robinson for his constructive criticism of drafts of this paper.

Biographical note

A D R I A N J OW E T T is a clinical lecturer in the School of Dentistry at the University of Sheffield, having previously lectured and researched in Anatomy and Cell Biology in the Department of Biomedical Science for nine years. Address: Department of Adult Dental Care, School of Clinical Dentistry, University of Sheffield, Sheffield S10 2TA, UK. [email: a.k.jowett@shef.ac.uk]


Editorial

LY N N E P. B A L DW I N Brunel University, UK

The sharing and discussion of ideas, both face to face and via technology such as email, is a communicative task which forms an essential part of the learning process, whether in the context of higher education or in other areas of our lives more generally. Oral, face-to-face interaction and involvement with others concerns listening and speaking, and these are not aspects of communication which we need to be taught unless, of course, we are born with some kind of physical attribute which makes this necessary or desirable. We learn how to listen and speak via the informal context of the environment of our home and family. In contrast, the skills of reading and writing have to be taught and are, in our culture, normally practised and learned in the formal context of the classroom. Our skills of comprehension (listening and reading) and those of production (speaking and writing) are, however, not fixed, immutable; we can learn how to improve these. Helping our learners to improve their skills of communication in the context of the higher education classroom is not without its challenges, however. As the six articles which comprise this issue attest, both we, as educators, and those that we teach have some learning to do if we are to foster the kind of interactive learning community which we purport to wish to develop in higher education.

In the study reported by Esat Alpay in the first article, entitled 'Group Dynamic Processes in Email Groups', discussion and decision-making via email are explored against the backdrop of theories concerning social facilitation and loafing. The applicability of psychodynamic and interactionbased models is discussed and it is argued that these may be insightful when looking at interaction via email. How we, as facilitators, might effectively promote email groups and how, practically speaking, we might do so in the classroom (virtual or otherwise) is set out. Interactivity in the classroom or, rather, the conventional lecture hall, is the focus of the second article, entitled 'Learning in Lectures: Do "Interactive Windows" Help?' by Mark Huxham. The current trend or argument in higher education is that we need to make our conventional classroom/lecture room more interactive in some way but, as the author posits, there is little evidence to support either its acceptability to our learners or whether there is any justification active learning in higher education for so doing. The study described in this article involved over 500 respondents and results provide insights into the influence or otherwise of 'interactive windows' on recall and learning.

In the third article, entitled 'Tête à Tête: Reading Groups and Peer Learning' by Sara-Jane Finlay and Guy Faulkner, discussion and communication is looked at in the context of how we might encourage learners to help themselves in terms of engaging with the subject matter and developing skills of criticality. The study reported here, in which reading groups of three to five students were formed, suggests that learners not only had opportunities for sharing their understanding but that so doing improved their time management; they reported that their workload was reduced. Personal contact and interaction is a strand in the fourth article, by Nick Zepke and Linda Leach, entitled 'Integration and Adaptation: Approaches to the Student Retention and Achievement Puzzle'. The authors note that a recurring theme of how we might best engage with our learners and, in so doing, reduce the number who drop out or under-achieve is ensuring that they have regular and meaningful contact with us, their teachers. How we might (better) foster a sense of community in our own institutions is explored.

Interaction with our learners in order to find out their experiences of, and engagement with, our modules and/or us as educators, is the subject of the fifth article, by Mark N. K. Saunders and Christine S. Williams. In 'From Evaluation towards an Agenda for Quality Improvement: The Development and Application of the Template Process', the authors discuss how we might better understand the view of our learners and in so doing improve the quality of service we provide. As they rightly comment, seeing learners as 'customers' paying for a 'service' has provoked a great deal of discussion and, not surprisingly, many a heated argument in the climate of higher education today. Nevertheless, regardless of your view on this, it is certainly the case that we are these days required to be more accountable for what we provide and how we do so. In the sixth and final article, 'Did the Market Force Subject Review? A Case Study', Adrian Jowett brings us face to face with such accountability, and how good we are. Or, rather, how difficult it is to measure performance. Subject review is said to be a process which in some way assures the quality in HE in England and Wales. Quite what 'quality' we are 'assuring' is the subject of much debate in the UK, and the study described in this article makes a much-welcomed contribution to this.

As is customary, this issue concludes with several reviews of books. The review section of the Members' Resource Area (MRA) on the website provides further book reviews which, for reasons of space, could not be included in this or previous issues of the Journal.


E S At

A L Pay

Group dynamic processes in email groups

Keywords: groups, group dynamics, interaction model, psychodynamic model

A B S T R AC T Discussion is given on the relevance of group dynamic processes in promoting decision-making in email discussion groups. General theories on social facilitation and social loafing are considered in the context of email groups, as well as the applicability of psychodynamic and interaction-based models. It is argued that such theories may indeed provide insight into email group interactions, but that communication limitations may severely hinder the effectiveness, and possibly the natural evolution, of email-based groups. Based on the various theoretical perspectives on group dynamics, some general recommendations are provided on promoting effective email groups, which include the set-up of communication and decision protocols, the cogent use of a group facilitator, and where possible, the supplementary use of face-to-face interactions.

Introduction

With the advent of information and communication technologies, there is increasing dependence on group discussion and decision-making in which email, rather than face-to-face, interactions are involved. The benefits of such electronic groups include: (i) logistical convenience, particularly if the group members are based in disparate locations such as in distance-learning programmes, (ii) a communication method in which it is possible for all participants to express opinions or ideas, and (iii) the automatic documentation of member responses for later contemplation, evaluation or reflection. However, email communication may lead to potential problems in group discussions owing to, for example, incoherent dialogue from one member to the next, and ambiguities in the interpretation of written material, particularly in the absence of any immediate and direct clarification of issues. Although much consideration has been given to group processes in educational contexts (see e.g. Jaques, 2001), this has been mainly confined to conventional face-to-face interactions. An understanding of such group dynamics has led to considerable insights into issues of group formation and cohesion, leadership and decision-making. The question then arises as to whether or not an understanding of group dynamics is equally applicable to those groups established around electronic communication media.

In the following sections, specific consideration will be given to the value of an understanding of group dynamics in promoting effective discussion and decision-making in email groups. A comparison of the nature of email and face-to-face groups will first be presented. Some key theories in group dynamics will then be considered, and the potential relevance and implications of these theories to email groups considered.

The nature of the group

For both face-to-face and email discussion groups, in which a group decision or outcome is sought, the following qualities are expected to exist: (i) group members share common aims, goals or objectives, (ii) group members openly perceive themselves as being a group, and (iii) group members expect to interact to reach an effective decision. Even with such a general description of a group, distinct differences between email and faceto-face groups may arise, such as the effective clarification of shared goals, motivations of members towards the group situation, and the nature and frequency of interactions. In particular, it may not always be possible to coordinate email groups such that members are interacting with each other at the same time, as is implicit of face-to-face interactions. In other words, the feedback mechanisms of the email groups are likely to be less effective, or at least delayed, which may ultimately lead to the poor evaluation of ideas owing to limitations in criticism and debate, or affirmation and support. Even if feedback delay issues in email groups could be overcome, the expressiveness of a member through a written medium may be rather different than through verbal and non-verbal means. It may be unlikely, for example, that group members fully gauge the conviction for an argument, or the emphasis of a particular aspect of an argument, in the absence of verbal (e.g. tone, pauses, or inflections in the voice) and non-verbal (e.g. facial expressions, body posture) cues. An absence of such cues may also limit the extent to which members become emotionally involved, or socially motivated, in the group task. This is turn could influence the level of information processing, in that members may become less likely to be deeply engrossed in the discussions in the absence of such emotional involvement.

Given the shared qualities of the groups mentioned above, and the fact that effective group work is often needed to reach a decision which is mutually supported by all group members, then one may expect group dynamic processes to be relevant to email groups. Indeed, some models of group dynamics do implicitly account for environmental, communicatory and socio-emotional factors (see discussions below), which could principally account for the differences between email and face-to-face groups. Thus, at least, an understanding of group dynamics is expected to give insight on why elements such as delayed feedback, or a lack of human reciprocity, may hinder the decision-making process. Subsequently, the application of group dynamics theories may help in the set-up of compensatory methods for supporting potential pitfalls in email groups. A somewhat negative perspective upon the relative effectiveness of email discussion groups is being implied here, and this in part is due to personal experiences in a study carried out at the Institute of Education, University of London (unpublished work). In particular, problems in communication flow and full member involvement were apparent with the email groups, and decisions could not be reached after three weeks of email interactions, whereas the groups were able to do so after approximately 15 minutes of face-to-face interactions. Similar observations were made by Barile and Durso (2002) on computer-mediated communication in collaborative writing. However, it is important to note that other group circumstances, task objectives and any pre-existing relationships may lead to highly effective work or outcomes in email groups, particularly if group involvement is strongly in the personal interest of the members (see e. g. Huws et al., 2001).

Group dynamic processes

In any face-to-face situation, social influences arise whereby attitudes and behaviours are affected by the presence, or implied presence, of others. Social norms may then arise which help to define the group, and in some cases influence the compliance or even conformance of members, as demonstrated by the classic experiments of Asch (1956) andMilgram (1974). In the context of discussion groups, such influences may help to maintain group unity and focus, as well as define implicit protocols for interaction. By definition, group decision-making needs to involve a level of conformance. Some research evidence suggests that for cognitive tasks, conformance is better achieved with an optimum group size of approximately three to five individuals (Stang, 1976). This is possibly related to the level of interaction and involvement of individual members, as well as to an individual's perception that their opinion or beliefs are being objectively validated in the presence of a few other group members. Conformity is greatly reduced if group unanimity on opinions, beliefs or intermediate decisions in a discussion cannot be established (Allen, 1975). In the case of email groups, poor continuity in discussions, which do not follow from the previous responses, questions or concerns, or in which there is poor acknowledgement of a raised issue by all group members, could yield the perception of a lack of unanimity. This could lead to a breakdown in conformity or convergence towards a decision. Likewise, the nature of feedback, with a lack of verbal and non-verbal cues, could seriously challenge an individual's affirmation of ideas.

The effect of the group on the individual may also be of either a facilitatory or inhibitory nature. Social facilitation could arise, for example, through drives which are instinctively aroused by the presence of others (Zajonc, 1965), through apprehensions of evaluation by others (Cottrell, 1972), or through a greater self-awareness and thus a concern to present the ideal-self (Carver and Scheier, 1981) or the best possible impression (Bond, 1982); see the discussions of Hogg and Vaughan (1998). Such possible antecedents of social facilitation could of course be of different significance under email and face-to-face group situations. For example, an individual may be less self-aware in an email interaction, and feel less apprehension due to any unexpected possibility of evaluation, and thus perhaps be less socially motivated towards involvement in the discussions. Social loafing, on the other hand, describes the situation when there is a reduction in an individual's effort when working in a group rather than working alone. In other words, social loafing is the reduction in the commitment, enthusiasm and motivation for a task in a group situation. Reasons for loafing may again have an evaluation-apprehension explanation, but in which excess apprehension is debilitating, rather than stimulating, to personal motivation. In other circumstances, loafing may arise from a perception of output equity, i.e. the belief that everyone else in the group is loafing, so one loafs as well. In a similar way, there may be a perception of low group standards, and thus a reduction in personal standards. However, loafing may be alleviated under circumstances in which there is: (i) personal accountability and identification, i.e. role differentiation in which an individual perceives a specific role in the group, such as co-ordinator, evaluator or technical expert, (ii) genuine personal interest in the task, and thus intrinsic or curiosity-driven motivation, and (iii) frequent inter-group comparisons as an objective evaluation of group standards. Furthermore, factors such as personal or social attraction (i.e. a liking of other group members because of common group membership), mutual positive support, and perceived inter-personal dependence (i.e. a need for other members to achieve personal goals), may lead to improved group cohesion. This could in turn produce a lower likelihood of loafing or indeed a higher likelihood of social facilitation.

In the context of email discussion groups, in which limited communication cues may exist, aspects of output equity, group standards, or reciprocity on personal and social attraction, may be difficult to gauge. Of course, it is possible that some group members in this circumstance could idealise the social situation and overestimate the standards and efforts of others. Nevertheless, the mutual support which sometimes arises in groups is unlikely to be as influential or convincing in an email group. For example, the need for the emotional support of a group member may not be recognized in the absence of certain visual, non-verbal cues, and thus possibly poor empathic accuracy under these circumstances (Ickes, 1997). In other words, different levels of social facilitation of the group members may not be effectively gauged, unconsciously or otherwise. This in turn could prevent compensatory efforts through, for example, emotional support and tension release, or indeed, the raising of group standards through the high motivation of certain members. Even if the need for such support or compensation is recognized, the effectiveness of its delivery through a written medium may be questionable. This then leads to issues of member roles in effective group work, and the possible psychodynamic aspects of member interactions; see discussions later in the text.

Consideration is also needed on the rules that relate individual opinions to the final group decision. Various social decision schemes may exist, such as the unanimity rule, majority wins rule and truth wins rule. As stated by Hogg and Vaughan (1998: 305): decisions can sometimes be quite accurately predicted from the pre-discussion distribution of opinions in the group and from the decision-making rule that prevails in the group at that time.

The prevalent rule will, of course, be situation dependent, and may also be established through the social norms of the group. Thus poor frequency, continuity and coherence of communications may lead to vague and inconsistent perceptions of how a decision is to be adopted. Furthermore, such communication problems may influence the quality of brainstorming and information recall (i.e. group memory) in addressing and evaluating the issues pertaining to the task. However, interferences and distractions from other group members in a face-to-face situation could also lead to lower creativity and productivity in idea generation, i.e. a process referred to as production blocking (Paulus et al., 1993). Indeed, Stroebe andDiehl (1994) suggest that production blocking may in fact be the main obstacle in idea generation in groups, and recommend the use of email-based brainstorming sessions. However, such an approach assumes that the effective initiation of a well-defined brainstorming session, and the subsequent collation of, and action on, ideas may be achieved. Separate ancillary tasks such as brainstorming also necessitate a certain degree of group organization and coordination, which may be harder to implement in an email situation.

Much attention has been given to the application of psychodynamic theories to group situations (see e.g. Jaques, 2001;Morgan and Thomas, 1996). The basis of such approaches evolves from Bion's view that a group operates at two levels: the work group and the basic assumption group (Bion, 1961). The former involves carrying out specific tasks, and defines the 'overt and conscious purpose for the group's existence'. The second level involves the basic assumption group for the 'unconscious concern with intra-group tensions, relationships and emotions' (Morgan and Thomas, 1996). Unconscious processes may, of course, be present in some of the occurrences mentioned above, such as personal and social attraction, and production blocking. However, Bion extends these ideas to the group as a whole, i.e. the group as an entity upon which unconscious forces exist for the preservation, security and prosperity of the group. Bion has explicitly categorized the unconscious drives as those for: (i) obtaining security and protection, e.g. from a group leader, (ii) preserving itself through group equivalents of the fight-or-flight responses, e.g. by creating a scapegoat (fight) or distracting itself with harmless or irrelevant issues (flight), and (iii) maintaining hope within the group through mutual support and bonding. Such group forces may arise from individual defences, which through social identity and introjection, lead to equivalent group defences, e.g. denial, cynicism, intellectualization and out-group competition. Many group defences may arise from the individuals' primitive, infantile defences of splitting and projection (see e.g. Morgan and Thomas, 1996). Splitting involves the compartmentalization of the negative and positive aspects of oneself, others and experiences, such that the individual or group may either adopt highly negative or highly favourable images as a coping mechanism. Projection involves ascribing individual beliefs, values, emotive states or other subjective processes onto another individual or group, and thus is a means of not having to deal with these processes in oneself. Collective projection and splitting are then believed to significantly influence the actions, behaviours and decisions of the group itself. Such group defence mechanisms are not necessarily undesired, in that they may help the cohesion and identity of the group, and may serve in the interest of self-esteem of individual group members.

Are, then, such psychodynamic forces relevant to email groups? How prominent is the basic assumption group under situations where any triggers of unconscious forces are limited to written responses? The cues for association and transference of primitive unconscious forces are expected to be rather complex, in which, for example, emotive, situational, and environmental factors may exist. It is possible that such forces between members may not be fully resolved owing to communicatory limitations, and thus an effective basic assumption group may not emerge. Likewise, the process of projective identification, in which a member adopts those beliefs or states being projected onto him/her, may be hindered, thus possibly influencing the nature of the relationships, and even role identity, within the group. Psychodynamic theories, therefore, may help to explain aspects of group cohesion, identity, defences and role differentiation, and subsequently differences in the nature of email and face-to-face groups. Psychodynamic processes do not cease to exist in the absence of face-toface interactions, but the degree and nature of unconscious, intra-psychic interactions are expected to be different.

The issue of roles in groups has also been considered by Bales (1970). Here, the emphasis has been on the interpersonal behaviours between group members rather than unconscious processes. Such interactions are postulated to fall into categories describing socio-emotional and task related aspects. Within these categories, specific bi-polar behaviours are considered, such as giving friendship/being unfriendly, giving information/seeking information, providing opinion or evaluation/asking questions. Problems in the group, such as communication, control, tension release and decisionmaking, are then deemed to arise when there is poor accommodation of, or synergy between, the behaviours of members. Functional groups are believed to evolve through progression through the task communication, evaluation and decision-making phases. Interestingly though, overall behavioural profiles for various discussion groups are indicated by Bales to be relatively consistent, although individual profiles may vary widely. This suggests that people may adopt different roles in different groups, but that there may exist a natural tendency towards groups with a similar overall behavioural profile. Furthermore, Bales' work indicates the importance of both task and the socio-emotional aspects of effective group work. What implications, then, does this have on email groups? Perhaps without clear verbal and non-verbal behavioural indicators, the effective interactions which overcome potential problems of communication, evaluation, control, tension, group integration and, subsequently, decision-making cannot be achieved. Likewise, the role of individuals could again be vaguely defined, leading to process inefficiencies, and indeed to a lack of group maintenance roles. However, if there is a human tendency towards certain behavioural profiles in groups, and if this profile cannot be perceived or gauged by members (as may be the case in some email interactions), then what influence will this have on motivation for group work? The interaction model of Bales may provide insight into some of the potential pitfalls of non-face-toface groups, but may not fully explain how roles arise in those groups which are effective nevertheless.

Given the various theoretical perspectives on group dynamic processes, the following general recommendations for promoting effective emailbased group work are suggested:

1. The explicit identification of group goals, and individual responsibilities and accountability at the onset of the group discussions, as well as the use of explicit communication and decision protocols. Protocols may promote group discussions through effective mechanisms for clarification, evaluation and question-response actions, as well as encourage group members to interact over matters which may be causing personal tension. 2. The use of a facilitator, i.e. an electronic tutor, for monitoring and supporting group maintenance roles, such as the employment of communication protocols, and highlighting both social and task reflexivity within the group. Such a facilitator could also convey inter-group comparisons, and thus provide the group with an objective means for self-evaluation. Tutors could employ sociograms to identify communication networks, cliques and attraction or antagonism between members within the group. 3. The use of real-time chat-room style interactions during a specified period of the day so as to promote free or casual discussion, and immediate feedback, progression, elaboration or query on member comments. 4. Where possible, it would be beneficial to supplement email discussion groups with conventional face-to-face meetings, which may help to further promote group identity, task objectives and role differentiation, help in the resolution of any communication issues and address unconscious group issues In a similar way, it may be helpful to establish the group through face-to-face pre-meetings before moving to an emailbased communication medium.

Conclusions

Where people interact in discussions in order to come to some mutual decision, it is argued that general facilitating, inhibiting and decisionmaking processes, individual psychodynamic processes, and even perhaps human inclinations towards a natural group behaviour profile, are in operation irrespective of the communication medium present. Indeed, in the above discussions, group theories have been used to explain some of the potential problems of email groups. The absence of certain communication methods, such as non-verbal cues, as well delayed or incoherent discussions, could severely hinder the quality of interactions, and subsequently the task and social-emotional related qualities of the group. An understanding of group dynamic theories could, however, help in the facilitation, organization and support of such groups. For example, communication protocols could be defined, task and support responsibilities explicitly identified, and tutors made aware of some of the special problems of email groups. However, some implications here are that any natural evolution of a group (see e.g. Tuckman (1965) model of group formation) can be mimicked through careful facilitation and explicit member awareness of the group processes, and that effective member roles can be explicitly identified rather than implicitly emerging from the social interactions.

Further research is needed in support of such propositions.


M A R K H U X H A M

Learning in lectures Do 'interactive windows' help?

Keywords: buzz groups, evaluation, interaction, lectures, recall

A B S T R AC T Many educational development resources recommend making conventional lectures more interactive. However, there is little firm evidence supporting either the acceptability (to students) or efficacy of doing so. This research examined the use of short 'interactive windows' (discussions and problem-solving exercises) in first year evolution lectures delivered to between 73 and 126 students over five years. Semi-structured evaluations of the teaching, involving more than 500 responses, identified the interactive nature of the lectures as the single most popular feature of the sessions. The division of the year class into two separate groups allowed the opportunistic testing of how interactive windows influenced learning about discrete problems within each lecture. Two short problem-solving or discussion sessions were devised for each lecture; one of these sessions was taught interactively to the first student group, the second was taught interactively to the second group. Comparing test scores achieved in questions addressing these paired problems showed strong evidence for a generally weak, positive influence of the interactive windows on recall and learning.

Introduction

Lecturing is under attack. Abundant evidence supports the conclusion that lectures are a poor way of stimulating thought and of changing attitudes (see extensive review in Bligh, 1998: 269-89). Even the relatively trivial educational goal of imparting information is not achieved much better through lectures than through other common teaching methods, such as seminars (Bligh, 1998). Lectures are often unpopular with students, especially those in advanced years of study, who demonstrate their feelings by not attending them. Maloney and Lally (1998), for example, recorded an absentee rate of 40 percent among third-year students, and Sander et al. (2000) found formal lectures were ranked amongst the least favoured active learning in higher education teaching methods by their sample of psychology, medical and business studies students. Whilst research into the relationship between attendance at lectures and academic achievement usually reports a positive correlation, the relationship is often surprisingly weak and may be better explained by confounding factors such as motivation, rather than by any direct educational value of strict and regular attendance (Hammen and Kelland, 1994;St. Clair, 1999).

Given this evidence, it is not surprising that criticism of the traditional, didactic lecture as a teaching method is virtually ubiquitous in modern educational development literature. Challenging the dominance of lectures is a staple in most induction courses for new academics. Despite all this, 'lecture' slots still fill university timetables, and it seems that the simple transmission of information remains the predominant mode of teaching within these slots (Lammers and Murphy, 2002;Van Dijk et al., 2001). The prevalence of lectures probably reflects the formidable forces of economic efficiency, institutional inertia and personal habit. Lectures are thus likely to remain a major part of traditional Higher Education for the foreseeable future, regardless of the arguments against them. Given this, a plethora of sources are at hand to advise the lecturer on how to make the best of a bad (or at least sub-optimal) job, and to improve the learning experience within the lecture. Most of these emphasize the importance of limiting the time spent simply imparting information, and of encouraging thinking through interactive and reflective exercises. For example, Bligh (1998: 65) declares:

It behoves lecturers to lecture less, to convince students of the intellectual aims of their courses, and to create opportunities, in lessons and outside, in which thinking can flourish.

Such opportunities for the flourishing of thinking often consist of short 'interactive windows', opened within the lecture to allow the fresh air of discussion and thought into the somnolent atmosphere. A frequently recommended tool is that of the small 'buzz group', in which students are asked to spend a few minutes discussing a question or solving a problem with a few of their peers. This flexible method can be applied in most teaching situations. It is appropriate even within large lectures in tiered lecture halls, where less structured forms of interaction, such as asking questions, may be difficult (Alien et al., 1999). It is the method used in the current study (although I will use the term 'interactive window' to mean any relatively short activity, within a traditional lecture, designed to encourage interaction and stimulate thought).

Whilst common sense and limited empirical evidence support the idea that opening these 'windows' should enhance learning, there are also arguments for keeping them shut. These include:

• Loss of teaching time -Lammers and Murphy (2002) found that lecturers who used interactive techniques had more periods within their classes when no one was involved in a learning activity, perhaps because of delays and difficulties in shifting between teaching styles. • Reduction in content -introducing interaction is likely to reduce the total amount of factual content that can be covered in a session, perhaps dramatically. Murray and Brightman (1996) recommend 'the holiday rule': 'for a successful holiday, halve your packing and double your spending money; for successful interactive lectures, halve your material and double your time'. This may be a particularly important consideration for some groups of students. For example, Merrill (2001) found that obtaining concise information for essays and examinations was a key objective of attendance at lectures for part-time students with little study time. • Reduction in the accuracy of transmission -asking students to work on problems and derive their own solutions may increase the risk of them leaving a lecture with incorrect solutions. • Student resistance and perception -some studies (e.g. Van Dijk et al., 2001) report a preference amongst students for 'traditional' lectures over those with interactive elements. Lake (2001) found that student perceptions of course and lecture effectiveness were lower for interactive compared with lecture sessions, even though student performance was higher in the former. • Loss of control -teaching methods that reduce a lecturer's control over a class, such as using peer discussions, may be avoided by teachers (Van Djik et al., 1999). This could imply a lack of confidence on the part of the teacher, which could undermine his or her efficiency were it to become clear to the students.

There are thus potential disadvantages in making lectures more interactive. These, along with the documented resistance of some academics to changing lecturing style (e.g. Van Djik et al., 1999) and the considerable institutional and personal inertia that may have to be overcome to do so, illustrate the need for more research on whether interactive windows really help learning. Research also needs to establish which of the numerous possible interactive tools is most effective. It is not sufficient for educational developers to encourage the opening of interactive windows on the basis of common sense or educational theory alone; empirical evidence is needed.

So research needs to answer (a) do interactive windows help and (b) which types, if any, are most effective? Answering the first of these is, in principle, straightforward; answering the second requires much more thought. To address the first question, studies need to compare the learning of students in lectures with and without interactive windows. However, there are considerable logistical and statistical barriers to overcome in achieving this. Because of the many influences on learning in lectures, large sample sizes are likely to be needed to detect any effect. Note that in this case, the lectures, not the students, will usually be the replicates; because students within a given class are not independent of each other, it will generally be incorrect to regard their individual efforts as single replicates. This implies that many lectures must be given in both treatment and control groups. A single individual is unlikely to be able to give all these lectures him or herself. So different teachers may be involved, introducing further sources of variation into the putative study and therefore implying even greater replication. Answering the second question involves all these difficulties, and includes additional complications introduced by the many possible techniques and the many ways in which they can be used. Given these challenges, along with the notorious tendency of students to behave as autonomous and unpredictable human beings (rather than compliant research subjects), it is perhaps not surprising that much remains to be discovered about interaction in lectures.

The present study has less ambitious goals than those discussed above. Its purposes are twofold. First, to explore what support there is for interactive windows amongst students and second, to examine whether there is evidence for interactive windows enhancing learning about specific problems and topics. As a relatively small and opportunistic study, it does not address the question of whether interaction can enhance learning in lecture sessions as a whole.

Methods

Interactive windows were introduced into a first-year lecture series on evolution in 1998, and have been used every year since (giving five years in total). The module contains one lecture a week, for 11 weeks, and is delivered to a student group which has ranged in size from 73 to 126, and in age from 17 to over 40. Each lecture contains a minimum of two interactive windows. These consist of short problem-solving exercises or discussion points. 'Buzz groups' of two or three students are given three to six minutes to discuss the problem; the correct answers are explained in class immediately after the interactive session. Two typical examples of problems are given below:

Human genes can be transferred into bacteria, and the bacteria will successfully 'read' the genes and produce the protein that is coded by them. What does this tell us about the genetic code, and how does this provide evidence for evolution?

The coefficient of relatedness between mother and offspring is 0.5. What is its value between grandmother and granddaughter?

Evaluation

Lectures are evaluated by using a quick, semi-structured, qualitative method after the fourth lecture (this timing allows students to become accustomed to my teaching style, but also gives time to adjust the style and content in the light of feedback received). All students are asked to write anonymous, short answers to the following three questions: Testing the effects of interactive windows Most of the people studying this module are full-time students, but a group of approximately 15-20 each year are part-time students, studying one day a week on placement from industry. From 1998-2000, the two groups were taught together. In 2001, a change in timetabling separated the two groups. In 2001 and 2002, I therefore delivered each lecture twice, on different days of the same week, to the two different student groups. This made the assessment of the effects of interactive windows statistically possible. Two interactive windows in each lecture (excluding the first, introductory lecture) were refined, as far as possible, to be of the same length, difficulty and type (e.g. discussion point or numerical problemsolving exercise). One window in each lecture was randomly assigned to full-time students, and one to part-timers. During lectures, each student group was asked to participate in the window assigned to them. The window selected for the other group was 'left closed'; that is, the question was posed on an overhead, and I explained its solution in a traditional, didactic manner without any student involvement. Hence each student group was assigned ten 'open' windows (one per lecture), each paired with a 'closed' one taught in the same lecture (see Figure 1).

Attentiveness varies during lectures, with items at the beginning and end most likely to be recalled (Holen and Oaster, 1976). It was therefore important in this study to control for the effects of timing. It was also necessary to avoid any bias created by one student group having been randomly assigned easier interactive windows. To avoid these confounders, windows were made as comparable as possible and were given, where possible, at roughly the same point in the lecture. They were also swapped between groups in 2002, with the 'open' windows assigned to part-timers in 2001 given to full-timers in 2002 and vice-versa.

Recall and understanding were tested in three ways. First, a quick, anonymous multiple-choice test was given at the end of the sixth lecture in both years, with no prior warning given to the students. This consisted of short questions, each one specific to a window opened in either full or part-time sessions. Second, the examination at the end of the module (three months after the first interactive windows were introduced) includes seven compulsory short-answer questions. As far as possible (without, for example, repeating questions from a previous year or missing important learning outcomes) these were made relevant to a particular interactive window, and the appropriate paired question (referring to the other topic covered by a window in the relevant lecture) was included. Third, the final examination also includes two elective essay questions on evolution. When students attempted these, their answers were examined for the inclusion of all possible examples and principles covered by the windows. Two marks were given for each example given which was in the correct context and was correctly understood; one mark was given for an example without full explanation or evidence that it was fully understood. All marks were summed for each possible example within the full-time and part-time groups, giving two frequency distributions. Although group identity was known when marking the class tests, all examination questions were marked blind, without knowledge of which group a student belonged to. Statistical analyses were performed using chi-squared tests. For each type of question (the multiple choice class tests, short answer and essay exam questions) frequency distributions of correct answers for full and part-time groups were compared. In each case, 2 ϫ 2 contingency tables were constructed using student type (full or part-time) as the rows, and a question along with its relevant pair as the columns; cells consisted of the summed total for all correct answers. Where more than one correct answer was given in a test (as was the case, for example, in the essay questions) all marks were summed into their relevant categories, to produce 2 ϫ 2 tables; this avoided any problems with low sample size in any individual chisquare bin. For example, suppose that in an essay question the part-time group scored a summed total of 20 marks by referring to topics and examples covered by the interactive windows that were 'opened' for them during the semester, whilst they scored a total of 15 referring to topics covered by 'closed' windows. Suppose also that the full-time group scored summed totals of 80 for their 'open' topics (those that were 'closed' for the part-timers) and 60 for their closed topics (the 'open' ones for the parttimers) in the same question. The resulting contingency table would be as shown in Table 1.

A C T I V E L E A R N I N G I N H I G H E R E D U C A T I O N 6(
Because chi-squared tests depend on proportions, results of these analyses are not confounded by any differences in ability between the two student groups. A total of 12 tests were performed; in this analysis, individual students are not regarded as replicates. Rather, each different window examined within a given year is equivalent to a repeat experiment on the same groups of students.

Results

Evaluation

A total of 515 responses were collected over the five years. Answers to the question 'what do you like about these lectures?' were coded into nine categories (see Figure 2); chosen to best fit the responses received. The category 'interaction' was used for comments such as: I like the way you set problems and walk around the class.

Getting the students involved in discussion makes it more interesting. 'Interaction' was the most frequently reported positive aspect of the lectures, followed by 'explanation' and 'interesting content' (Figure 2). Only one student in the five years mentioned interaction in response to the question 'what do you not like about these lectures'?

H U X H A M : L E A R N I N G I N L E C T U R E S 23

Effectiveness of interactive windows

The answers of 120 full-time students were compared to those of 18 parttimers in 2001. Equivalent class sizes in 2002 were 64 and 14.

Results of the chi-squared tests are given in Table 2, which also shows whether the expected trend was found in each individual test. That is, regardless of the statistical significance of the test, was there a higher proportion of correct answers in the expected categories? Figure 3 shows an example distribution.

None of the tests in 2001 was significant (although answers to essay question 1 approached significance at the 5% level). However, all of them showed the expected trend. Two comparisons in 2002 gave significant results, with five of the six tests showing the expected trend.

A one-tailed binomial test, for goodness-of-fit to a binomial distribution

A C T I V E L E A R N I N G I N H I G H E R E D U C A T I O N 6(1)
24 assuming that there is an equal probability for each comparison to show the expected or the unexpected trend, gave a significant result (P = 0.0032).

Discussion

The current study posed two questions: are interactive windows popular with students, and can they enhance recall and understanding? The results support a positive response to both of these. Semi-structured student evaluations identified 'interaction' as the single most highly-rated feature of the lectures. Student answers to class tests and examination questions showed a significant trend towards enhanced performance in topics covered during interactive windows. Entwhistle et al. (2000), in their review of literature on student evaluation, identify 'the three Es' as being those factors most favoured by students as important for good teaching: enthusiasm, empathy and explanation. All of these categories appear in my feedback, but were mentioned less frequently than interaction. Of course, the current study provides no grounds for comparing between teachers or for evaluating 'good teaching' in general. Students were asked only to identify what features they liked In each case, d.f. = 1. P values are given, along with whether any differences, regardless of significance, were in the expected direction.

H U X H A M : L E A R N I N G I N L E C T U R E S 25
about the evolution lectures; had the teaching been different (for example, had it been more or less enthusiastic) then the ranking of categories may also have differed. However, it is clear that the students considered the interactive windows to have been a positive aspect of the teaching. This is important in itself, since students do not always welcome interaction.

Although some studies of interaction concur with the present one in finding that students prefer more interactive methods (Goldfinch 1996;Sander et al., 2000;Tam et al., 1993), others found a negative response.

For example Van Dijk et al. (1998) report students preferring traditional to 'activating' lectures. The possible reasons for this difference are many, and could include the different type of student body (engineering students) and different expectations on the part of students and staff. Lake ( 2001)

A C T I V E L E A R N I N G I N H I G H E R E D U C A T I O N 6(1)
26

Figure 3 Example distribution of correct responses to a paired comparison Note. 'Part time window' refers to the question taught interactively to the part-timers. This is paired with a topic covered in the same lecture, but taught interactively to the full-time group. In this case, although both groups of students performed better answering the topic covered interactively in the part-time lecture, the proportion of correct answers (as a percentage of all the correct answers they gave in both questions) given by part-timers to this question was greater. Thus, although the difference was not significant, the trend was in the expected direction. compared performance and perceptions in two groups of physical therapy students given either all traditional lectures or fewer lectures supplemented by hour-long discussion groups. Despite achieving higher performances in tests, the group exposed to the more interactive teaching method believed that they had learned less, and rated their teacher less highly, compared with the group taught by lectures alone. Goodwin et al. (1991) also found that students with 50 percent discussion groups felt they had learned less compared with students in lecture-only courses, despite their comparable performance. It is possible that student resistance to interactive methods may be related to age and life experience, with mature students more confident in discussion and more likely to appreciate interactive teaching (Merrill, 2001). Lake ( 2001) suggests that courses early in a curriculum should limit the interactive teaching to 25 percent, and steadily increase this percentage in later years, in order to overcome the problem of students perceiving this form of teaching as less effective. Similarly, Murray and Brightman (1996) suggest that interactive teaching is likely to be more successful in later years, after a firmer grounding in a subject has been achieved. This is an argument for combining interaction with traditional methods in the same sessions in earlier years, as was done in this study. By incorporating windows into traditional lectures, no distinctions are made between 'lectures' and 'discussions' and student perceptions of what constitutes expected university teaching may be more easily met. Evaluations from both part-time and full-time students in 2001 and 2002 (when their responses were separate) rated interaction as the most important element, despite the suggestion that mature students might differ from school leavers in their perceptions of interactive methods (Merrill, 2001).

Student evaluations, on their own, do not provide sufficient grounds for changing teaching practice. The phenomenon of 'seduction' (deliberately courting good evaluation, regardless of teaching efficiency) is well recognized (see Bligh, 1998: 181), and what students want may not be what is pedagogically best. So evidence that a teaching practice works to enhance learning is essential, especially when it is likely to involve some disadvantages, such as the loss of time that may be associated with interactive windows. The contrasts found here, between responses to topics taught using interactive windows and traditional didactic methods, provide strong evidence of a generally weak, positive effect. Most of the P values shown in Table 1 are far from significant, reflecting the often small differences found between the groups. But the chances of finding so many differences in the expected direction, in the absence of any real effect of interaction, are less than one in 300.

There is evidence that interactive windows had different effects on performance in short answer and essay type questions in the examination; significant differences were found in essay questions in year 2, and the differences approached significance in this kind of question in year 1. This may suggest a rather subtle effect of interaction on recall, one which could be labelled a 'Palomar Moment'. In 'Mr Palomar' by Italo Calvino, the eponymous hero is a connoisseur of cheese. Despite an encyclopaedic knowledge of, and gastronome's enthusiasm for, obscure and delicious cheeses, he inadvertently requests the blandest mass-market product when put under pressure and forced to choose quickly, because this product has been most reinforced by advertising and comes quickest to mind. It is possible that, when given a choice, students will recall the example or topic reinforced by interactive study. In contrast, when given no choice, as in the short answer questions, the effects of interactive windows may be less, since the student has either revised and understood the topic or has not.

The most significant contrast occurred in the 2002 class test. Stronger effects of interactive windows might be expected in the class tests, since they were conducted in the sixth lecture, a maximum of five weeks (and a minimum of half an hour) since teaching the topics covered in the tests. In contrast, the examination was sat three months after the start of the module. Previous work (Hollingsworth, 1995) has reported a rapid decline in the efficacy of interactive methods in enhancing recall, hence it might be expected that the examination questions would reveal a smaller difference between groups. However, the non-significant result in the 2001 class test suggests a more complex picture than one of simple decline in recall with time, although this may well be a factor.

The present study addresses the hypothesis that interactive windows of the type used here can enhance learning of those topics discussed during the windows. It cannot test hypotheses about the effects of interaction on learning in a lecture as a whole. This is because there were no control lectures without interaction, and there are good reasons to believe that interactive windows may influence learning even after they are shut. For example, Bligh (1998: 57) shows that asking questions in class can affect the heart rates (and hence presumably the attention) even of students to whom the questions are not addressed, and that the effect can last for some time after the question is asked. Similarly, Ruhl and Suritsky (1995) demonstrate significant enhancement in student note-taking and recall as a result of inserting three, two-minute pauses into a lecture. No learning activities were prescribed during these pauses -they were effectively rests. They affected the recall of the lecture as a whole, rather than any specific topics within it.

It is therefore quite possible that the interactive windows used in the present study affected the learning of topics covered in traditional ways. This would have reduced any contrasts found between the paired interactive and non-interactive topics, and the conclusion that interaction can nevertheless have positive affects is therefore a conservative one.

Previous studies on the effects of interaction in lectures have given mixed results. Van Dijk et al. (2001) studied learning in three groups of students. One was given a 90-minute lecture taught using an electronic Interactive Voting System (IVS). The second listened to the same lecture, with IVS and also with peer instruction (similar to my interactive windows). The third group acted as a control, with no interaction. They recorded a significant reduction in learning in the IVS group, compared with the control; this result serves to reinforce the importance of conducting empirical research, rather than assuming interaction -high technology or otherwise -will enhance learning. There was a small, non-significant improvement in test scores in the combined IVS/peer instruction group. Tam et al. (1993) used interactive handouts, and reported no improvement in learning compared with a control group. In contrast, Hollingsworth (1995) found a significant effect of short interactive discussions on performance in a test completed immediately after the lecture, but not in a test ten days after. These studies used comparisons between only one, or very few, sets of experimental and control lectures. If the effects of interaction are rather weak and variable, as in the present study, then significant differences are unlikely to emerge over short time periods, and mixed results are to be expected.

In conclusion, the present study found that interactive windows opened in traditional lectures were popular with students. The high rating for interaction was consistent in all years and was shown in both full-time and parttime classes, despite previous suggestions that these different groups of students may differ in their responses to interactive methods. Interactive windows had a generally small, but measurable, effect on recall and understanding. There are reasons to believe that these conclusions are conservative (because interaction may enhance learning in the lecture as a whole, and not only in the topics covered interactively, and because my sample sizes -in particular for the part-time group -were small). Many of the potential disadvantages of interactive methods, such as overcoming student expectations and loss of time and content, are mitigated by incorporating small interactive sessions into traditional lectures, rather than replacing lectures entirely. Hence the costs of incorporating interaction in this way are likely to be small. Given this, along with the enthusiastic response of students to the interactive windows in this study, these results support the inclusion of interactive windows in lectures.

  (a) what do you like about these lectures? (b) what do you dislike about these lectures? (c) what would you like to see changed?

Figure 2

 2 Figure 2 Ranking of coded evaluation responses to the question 'what do you like about these lectures?' as percentages of all responses (N = 515)

  Single lecture, given in the same week to two separate groups of students, containing two 'windows'

			1)
	Part-time group	Full-time group
	Window 1	Window 2	Window 1	Window 2
	'open'	'closed'	'closed'	'open'
	Figure 1 Research design
	22

Note. Each lecture was repeated twice, with a different window 'opened' for each of the two groups.

Table 1 Hypothetical example showing the construction of a contingency table to test the efficacy of open windows

 1

		Questions 'opened'	Questions 'opened'
		for full-timers	for part-timers
	Full time	80	60
	Part time	15

Table 2 Results of 12 chi-squared comparisons conducted on marks from class tests, short answer (no choice) examination questions and elective essay examination questions

 2

	Test	P value	Expected trend?
	2001 results
	Class test	0.330	√
	Exam (short answer 1)	0.900	√
	Exam (short answer 2)	0.180	√
	Exam (short answer 3)	0.850	√
	Exam (essay answer 1)	0.060	√
	Exam (essay answer 2)	0.500	√
	2002 results
	Class test	0.009	√
	Exam (short answer 1)	0.900	√
	Exam (short answer 2)	0.800	√
	Exam (short answer 3)	0.150	×
	Exam (essay answer 1)	0.026	√
	Exam (essay answer 2)	0.137	√

Note.

Acknowledgements

This work was possible only because of the goodwill of the hundreds of students who participated. The manuscript was improved after helpful comments from Shirley Earl, Judy Goldfinch and two anonymous referees.

Biographical note

M A R K H U X H A M is senior lecturer and teaching fellow in ecology at Napier University. He is interested in finding simple ways of enhancing student learning and the student experience within the confines of mass higher education. He specializes in marine ecology and ecosystem function, teaches on undergraduate and postgraduate courses and really believes the cliché that good research should enhance good teaching, and vice versa. Address: School of Life Sciences, Napier University, Edinburgh EH10 5DT, UK. [email: m.huxham@napier.ac.uk]


published or not. The documents may come

This is an excellent textbook and a valuable addition to the growing field of 'how to' books providing advice and guidance for practitioners and students wishing to undertake fieldwork of an ethnographic nature. Gray's familiarity with the methodological terrain is clearly apparent and her experiences as both researcher and tutor enable her to address the sort of problems that students and researchers are likely to encounter in setting up their own fieldwork projects for either undergraduate or postgraduate research. This book can also serve as an effective resource for lecturers delivering courses or modules concerned with research methods across a range of disciplines. Media, sociology and cultural studies practitioners, in particular, will find this book invaluable.

The book is divided into two sections; Part One (Ethnographic Methods) consists of four chapters which provide a coherent and succinct overview of ethnography and its place within cultural studies as a research method. Core issues and debates are neatly addressed along with a critique of ethnographic methods. Key texts and some classic studies are signposted throughout and recommended for further reading.

Issues concerning methodology, reflexivity, epistemology, validity, ethics and the politics of research -all vital to an understanding of ethnography and empirical research more generally -are addressed in an easily digestible and refreshingly jargon-free manner. Part Two (The Research Process) contains six chapters that build on the themes and concerns of the first section but provide in-depth and practical advice on how to actually undertake research and the various stages of the research process from design and implementation through to writing up.

Engaging and easily readable throughout, this book works on a number of levels in the sense that both lecturers and students will find material of value in it. Whether you are a student or researcher planning to undertake fieldwork in the near future, or a lecturer involved in delivering courses or modules on research methods and methodology, this book deserves to be high on your reading list.

Book reviews

Research Practice for Cultural Studies: Ethnographic Methods and Lived Cultures by Ann Gray. London: SAGE, 2003. ISBN 0-7619-5175-X. 208 pp. £11.99.