The Researcher Librarian Partnership: building a culture of research

Increasingly, the library and information science (LIS) practitioner is being challenged to incorporate research into the context of their professional work. This paper reports on the Researcher-Librarian Partnership , a research-mentoring programme that was initiated by the International Federation of Library Associations and Institutions. Six new LIS practitioners within their first seven years of professional practice took part in the programme. Each was partnered with an experienced LIS researcher who provided mentoring and support. During the 12-month programme the new professionals designed and implemented a research project on a topic of interest. This paper outlines the details of the programme providing observations on how research mentoring can be a powerful

Introduction
An on-going criticism of the library and information science (LIS) profession is the lack of rigorous research (see Powell, Baker and Mika, 2002). In today's everchanging information rich and technologically driven world it has become more important than ever that library professionals embrace a culture of research and scholarly enquiry. The current economic environment also requires librarians to assess and evaluate their services to articulate the value of libraries to the community and funding agencies. This is necessary to compete for limited external and internal resources. Librarians must be capable of assessing services and systems based on how and why their clients and communities use them, why they do not use them, and why they choose other services instead; yet, LIS education and training are not offering sufficient courses in user-centred assessment and evaluation (Connaway, 2014).
Research skill and knowledge are the essential tool kit for ensuring that libraries and librarians continue to effectively and efficiently meet the evolving needs of the clients and communities they serve. Research must be promoted as a valuable activity for all library professionals. There is an obligation on the part of library educators, employers and professional associations to ensure that practitioners have the necessary skill and knowledge to conduct research effectively. As the international body for the library sector, the International Federation of Library Associations and Institutions (IFLA) has a vital role to play in leading the research charge in the profession. This paper reports on a research-mentoring programme that was implemented by the IFLA Library Theory and Research Section. A research-mentoring programme provides the opportunity for a library research culture to develop among those who are new to the profession. It provides encouragement, assistance and expertise in the research process for the new professionals or protégés taking part. It also enables the more experienced professionals or mentors to acquire leadership skills and to become proactive in their profession. A research-mentoring programme will help to ease the transition from the practice-orientated librarian to the research-orientated librarian. This paper begins by first exploring the key literature relevant to the role of research in the professional practice of LIS professionals and how mentoring can be a powerful tool through which to engender a culture of research in a profession. The paper will then outline the mentoring programme that influenced the recommendations for the future potential offerings of the programme. The value of research in the LIS profession has been well-discussed within the literature. In 2001 the Centre for Information Research was commissioned by the Chartered Institute of Library and Information Professionals (CILIP) to conduct an investigation into the research landscape for library and information science (LIS). This concluded that the value of research in the LIS discipline could be experienced on both the professional and personal levels. At the professional level, research can inform practice, assist in the future planning of the profession, and raise the profile of the discipline, and the library and information service. At the personal level, research can "broaden horizons and offer individuals development opportunities" (McNicol and Nankivell, 2001, 77). A similar sentiment was voiced by Powell, Baker and Mika (2002) when they stated: research by LIS practitioners is needed to create new knowledge and thereby contribute to the growth of LIS as a profession or discipline. (Powell, Baker and Mika, 2002) They noted the benefits of research as contributing to: the improvement of decision making, problem solving, critical and analytical thinking; the development of more critical consumers of the research literature and librarians who are better equipped to provide user-centred information services; the opportunity for career advancement; and the enhancement of staff morale and librarians' stature. Juznic and Urbanija (2003) take the discussion one step further by arguing that: If research is absent, non existent or even scarce, there is no profession, but only an occupation grounded in techniques, routine and common sense. (Juznic and Urbanija, 2003, 325) Similarly, Harvey (2002, xiii) argued that "research and professional practice are inextricably linked" and as such "research skills are a prerequisite [italics added] for those who want to work successfully in information environments". He postulates that research skills are an "essential set of tools which enable information workers to become information professionals" (Harvey, 2002, xiii). According to Harvey (2002): The work of information professionals is being transformed. The information services we offer, the information products we develop and sell, the information systems we design and implement, are undergoing rapid change. So, too, is the society in which we operate. We have a continual need to determine what is happening, how it is changing, how it will affect our places of work, how it will alter the services we offer. Change and its ramifications is the most important reason why research is necessary, and why it is here to stay. If you don't know something about the tools of research and about how to use these tools, then you cannot be an effective information professional. (Harvey, 2002, xii) The challenges for undertaking research also have been well-articulated in the LIS literature. Funding, time, experience, support and access to research have all been noted as obstacles to LIS professionals conducting research (Koufogiannakis and Crumley, 2006;Berg, Jacobs and Cornwell, 2013). Inadequate funding has been identified as one of the major obstacles preventing LIS professionals conducting research. LIS professionals lack the range of opportunities to obtain funds to contribute to the implementation of research projects. When funding is available it is often small and does not fully cover the actual costs of undertaking the research. In addition, practitioners may not be comfortable with conducting research and consequently "experience", or lack of it, can also be a major obstacle for librarians undertaking research projects. Many LIS professionals did not receive research training in their formal LIS studies or received only limited training, and access to the opportunity to gain practical research experience in their work environment is not readily available. Koufogiannakis and Crumley (2006) note: Without keen interest and motivation, research is easily left on the bottom of the priority list and without adequate knowledge of research methods and a lack of experience, it can be difficult to even know where to start. (Koufogiannakis and Crumley, 2006, 334) Some LIS professionals lack access to peer reviewed journals and databases. Not being able to regularly read the current literature will limit the extent to which LIS professionals are exposed to new ideas and are able to develop an awareness of the research process. Many librarians have neither the time in their workday to conduct research nor the support of their employer to undertake research during work time (Berg, Jacobs and Dayna, 2013). As noted by Koufogiannakis and Crumley (2006): without a supportive employer who allows work time to be developed to professional development including research it is challenging to incorporate research into daily practice. (Koufogiannakis and Crumley, 2006, 334) The LIS community must find ways to stimulate greater interest and respect for research (McNicol and Nankivell, 2001) and must find ways to overcome the obstacles that keep practitioners from conducting research (Koufogiannakis and Crumley, 2006, 337 Hallam and Partridge (2006) also observed that the biggest challenge to having the LIS profession engage in research was to encourage stakeholderseducators, individual professionals, employers and the professional associationsto play a role in working collaboratively to develop a research culture that should pervade the profession. This paper aims to contribute to meeting the challenge, by describing a formal LIS research-mentoring programme that was initiated by a professional association and which involved the support of LIS educators and LIS professionals.

The role of research mentoring
Mentoring relationships and their benefits have long been discussed in the human resources field, and more recently in the LIS arena. According to Holmes, Hodgson, Simari and Nishimura (2010), the mentoring relationship can be described as: a series of complex interactions between 2 individuals who have as their primary purpose the growth of the mentee, although this process often results in the professional growth of both parties. (Holmes, Hodgson, Simari and Nishimura, 2010, 336) Kram (1985) defines a mentor as a person with more experience and knowledge than the protégé / mentee, who is committed to providing career guidance and advice to allow the protégé to enhance career prospects.
Much has been written about the favourable personal, social and career outcomes of participating in a mentoring relationship. For example, an exhaustive metaanalysis of mentoring relationship research from 1985 to 2006 found that mentoring was significantly related to a myriad of positive outcomes, including behaviours, attitudes, health, interpersonal relationships, motivations, and careers (Eby, Allen, Evans, Ng and DuBois, 2008). Protégés are more likely to display increased levels of satisfaction in their work and higher job performance than those without mentors (Lumpkin, 2011).
One way that mentoring may enact its benefits is through socialisation and networking. According to Holmes et al. (2010) mentoring should assist the development of personal learning networks. The authors describe these networks as lifelong resources for continued career progression and personal and professional growth. Lumpkin (2011) echoes this sentiment, and argues that mentors can facilitate networking for their protégés. As such, mentoring may be particularly useful for individuals new to a job or profession, who have not had the time or experience to build professional networks. Gibb (1999) reported that participation in a mentoring relationship was found to improve performance, provide support and improve the socialisation of new professionals. The mentoring relationship also has the potential to provide an enriching and supportive environment to new LIS professionals (Freedman, 2009). For example, Hallam and Newton-Smith (2006) described a mentoring programme that aimed to help LIS students transition to their first professional position. The mentoring programme intended to support the students' learning needs and socialise them into the LIS profession. The authors found that protégés reported improved job application skills, stronger professional networks and developed stronger career plans after participating in the programme.
At its root, mentoring allows the transferral of relevant knowledge from the mentor to the protégé (Freedman, 2009;Holmes et al., 2010). Indeed, mentors have been described as "role models" (Kostovich, Saban and Collins, 2010) and "advisors" (Farmer, Stockham and Trussell, 2009), highlighting that protégés witness and learn from the behaviours and attitudes displayed by their mentors.
However, the benefits of the mentoring relationship do not seem to be limited to the protégé; the relationship provides benefits to both parties. This includes the benefits of having a "role model", as Hallam and Newton-Smith (2006) found. The authors reported that protégés, as expected, appreciated having a professional role model in the form of their mentor and that they learned from their mentor's personal experience. Likewise, mentors reported that they also benefited from learning from their protégé's personal experience. It appears that mentoring is a two-way street, with gains for both parties. Freedman (2009) argues that mentors gain a personal satisfaction from helping the less-experienced individual and renew their enthusiasm and commitment to their profession. Additionally, Hallam and Newton-Smith (2006) reported that mentors felt an increased level of job satisfaction. It has also been suggested that mentoring programmes highlight the importance of lifelong learning for both mentors and protégés (Hallam and Newton-Smith, 2006).
Farmer, Stockham, and Trussell (2009) describe a formalised mentoring programme at the Kansas State Libraries that had been in place for over 20 years. The main aim of the programme was to guide junior (pre-tenured) librarians through the promotion and tenure process at the University Library. The authors concluded that this mentoring programme met the needs of the new employees and effectively imparted salient information and advice in a coordinated, on-going process. Mentors self-reported to have acted as advisors, advocates and resources to the protégés. Moreover, protégés described their mentors as accessible, approachable, supportive, offering constructive criticism and acting with professional integrity.
An important aspect of professional development, promotion and tenure (where applicable) in the LIS field is conducting research. Research skills and methods can work to ensure that the LIS profession remains current and relevant during changing times (Thorpe, Partridge and Edwards, 2008). As noted earlier there are many obstacles that prevent practising librarians from engaging in research (Koufogiannakis and Crumley, 2006); lack of experience has emerged as one of the more important obstacles to be addressed. Koufogiannakis and Crumley (2006) suggest that many practising librarians are not comfortable conducting research as they have not received the necessary training during their education and may not have had the opportunity for practical research experience in their work environment.
Recent research has highlighted the benefits of research mentoring to increasing research confidence, experience and output, particularly in the health and medicine fields. For example, authors have asserted that mentoring is fundamentally important in academic medical research (Bettmann, 2009;Blixen, Papp, Hull Rudick and Bramstedt, 2007;Keyser et al., 2008;Kostovich, Saban and Collins, 2010). Moreover, Lumpkin (2009) has suggested that researchmentoring programmes also help to ease the transition of University faculty into new roles.
The mentor-protégé research process enacted through formal research mentoring generally includes both direct communication about relevant research issues, as well as indirect observation of the mentor's behaviour and practice over a period of time (Keyser et al., 2008). Thus, the protégé is able to ask questions regarding the research process, check his / her personal understanding and model appropriate research behaviours. For the LIS professional this would be a valuable opportunity to experience the research process, possibly for the first time.
Kostovich, Saban and Collins (2010) state that the most important function of the research mentor is that of a teacher. Although new professionals may have learnt the theories behind qualitative and quantitative research design, they still need to learn the practicalities and skills involved in actually conducting research. The research mentor acts as both a consultant and advisor and: can provide the knowledge learned from personal experience, since the research mentor has been down this road at an earlier time. (Kostovich, Saban and Collins, 2010, 283) Thus, the mentor's ability to act as a "role model" once again can provide the protégé with the opportunity to witness and learn from the behaviours and attitudes displayed by their mentors. By being exposed to a more experienced researcher, the protégé can hone individual research skills and confidence, which is of particular importance in the LIS field, as new professionals often lack research experience and confidence, as previously discussed. Keyser et al. (2008) describe the roles and responsibilities of the mentor and protégé in the research mentoring relationship. The authors suggest that the role of the mentor is to support the protégé's personal and professional development through strengthening their academic competency, their knowledge of and adherence to responsible conduct of research and by providing support and encouragement. In contrast, as with traditional mentoring relationships, the role of the protégé is to commit to the relationship and take personal ownership for the quality of the relationship. Over and above the benefits of mentoring described above, the research mentoring relationship has been found to help protégés develop strong professional networks, prepare manuscripts for submission for review and publication, apply for research grants and participate in professional research-related meetings (Keyser et al., 2008).
The positive effects of research mentoring may work, at least in part, through increased self-efficacy of the protégé. Research self-efficacy refers to beliefs about one's ability to carry out and complete the tasks associated with conducting research (Bishop andBieschke, 1998, as cited in Love, Bahner, Jones andNilsson, 2007). Love, Bahner, Jones and Nilsson (2007) report that gaining experience in conducting research is a vital aspect of the development of research self-efficacy. The authors' survey data indicated that faculty support and mentoring were the most important contributors to satisfactory individual research experiences (Love, Bahner, Jones and Nilsson, 2007). Thus, being exposed to a supportive research mentoring environment and having the opportunity to learn from a mentor appears to increase an individual's positive beliefs regarding the ability to carry out research, which in turn should increase the total research commitment and output.
In the LIS spectrum, the benefits of mentoring have predominantly been explored with regards to career development, career transitioning and succession planning in light of the changing face of the library workforce (e.g., Freedman, 2009;Mosley, 2005;Murphy, 2008). Although Farmer, Stockham and Trussell (2009) reported that both LIS mentors and protégés listed positive research-related outcomes of their mentoring relationship, including presentations at conferences, conducting research and publishing, there is little published information regarding research mentoring in the LIS profession. Such information would add value to the LIS industry, as research skills and methods have been identified as an increasingly important part of the LIS profession (for example, Thorpe, Partridge and Edwards, 2008). This paper aims to contribute in this regard, by describing a formal LIS research-mentoring programme, from inception to evaluation.

Project aim and objectives
The project was funded by the International Federation of Library Associations and Institutions (IFLA) and administered by the Library Theory and Research (LTR) Section. The two-year project aimed to encourage and support the development of research skills in the library profession, with a particular focus on new professionals. The specific objectives were to: 1. establish a formal research-mentoring programme; 2. develop practical strategies and recommendations on how to develop an ongoing sustainable research programme.

The mentoring programme
Planning of the programme commenced in February 2010 with a call for applications made in June. Twenty-two applications (including a CV, personal statement, and research topic outline) were received and reviewed by the project team. Six new LIS professionals within their first seven years of professional practice took part in the mentoring programme. Each new professional was partnered with an experienced library researcher or professional who provided mentoring and support. Together the protégés and mentors represented eight of the world's countries, Canada, Fiji, Germany, Italy, Jamaica, Nigeria, United States of America, and Trinidad and Tobago. In allocating mentors the following criteria were used, aiming to ensure that the mentor was: 1. located geographically close to the protégé (e.g. same city, same state or same country); 2. an experienced researcher in the topic and / or method that the protégé was interested in exploring; 3. interested, committed and available to take part in the full life of the mentoring programme.
No mentors had a previous relationship with the protégé to which they were partnered. To avoid any conflict of interest and to ensure a safe and open learning environment mentors and protégés were not based at the same institution (e.g. colleagues working together).
The protégés and mentors participated in a twelve-month mentoring programme from October 2010 to September 2011. During the programme each protégé was required to design and undertake a small research project in an area of interest to them. The mentor and protégé were introduced virtually (i.e. via email) and were encouraged to meet with each other (e.g. either in person or online using a tool such as Skype). At the commencement of the mentoring partnership, the protégé and mentor were required to compile and submit a mentoring agreement which outlined the agreed goals and objectives for the life of the partnership as well as strategies for communicating and dealing with problems that arise. The protégé also was encouraged to keep a learning journal during the project, a template for this was provided (Appendix A in the Supplementary File). It was originally intended that all 12 mentors and protégés would take part in a monthly online meeting (e.g. via technology such as Skype). The meetings would provide a vehicle to build a community of practice for all those taking part in the programme. Unfortunately the meetings could not be conducted as planned because of issues with technology, the busy schedules of the mentors and protégés, and challenges with locating a suitable time in light of the different time zones. However, during the programme regular contact with the mentors and protégés was made via email. This contact ensured those involved in the programme had someone to approach for support and / or advice. It also helped to keep the programme on track with reminders regarding due dates for milestones and other activities.
The protégé was required to submit two deliverables during the mentoring programme; a research proposal (see Appendix B) due approximately three months after commencing and a final report (see Appendix C) due at the end of the programme. A template for both deliverables was provided. The research proposal was peer reviewed by one international expert, and the project supervisory team reviewed the final report. In both cases feedback was provided to the protégé. All documentation used during the project is available upon request. A Moodle site was used to provide access to all the templates and other key resources relevant to building a successful mentoring relationship and in designing and undertaking a research study (e.g. recommended readings, how to guides). The site was dynamic; a basic site was established at the commencement of the programme with developments made progressively during the life of the programme. All mentors and protégés were encouraged to add to the site and / or to identify content they would like sourced and included into the site. The mentors and protégés were notified (via email) and encouraged to engage when new content was added to the site.
The mentoring programme was designed to recognise the individual needs of the six protégés, with each progressing well in light of their own circumstances. Four protégés completed their projects producing final reports, one protégé completed her project but did not submit a final report and one protégé completed her research proposal. The protégés each received a certificate of participation. The two best research projects, in the judgement of the project supervisor team, were invited to present their research results at the 78 th IFLA General Conference and Assembly in Helsinki, Finland. Links to these two papers (Chiessi, 2012, andNilsen, 2012) can be found in this paper's list of references.

Project evaluation
Evaluation of the programme took place from October to December 2011. All protégés and mentors were invited to provide feedback on the mentoring programme via an online questionnaire (see Appendix D). A total of 6 (3 mentors, 3 protégés) completed the questionnaire, providing a response rate of 50%. A brief summary of the key results follows.
Most of the respondents (4) had been involved in a mentoring programme in the past. The reasons respondents gave for participating in the Partnership could generally either be classified as Learning; to have practical experience of the research process (3 responses) or Teaching; to assist the profession by providing expertise to new researchers (3 responses), in line with what would be expected to be the driving forces behind being a protégé and mentor, respectively.
The hopes and expectations individuals reported in participating in the Partnership all seemed in line with the programme's goals: To see and accompany a practical trial of an impact survey.
To publish the research I carried out and have a working relationship with my Mentor.
I had hoped to learn, make friends and gain insight to research and to actually publish my research.
I was hoping to help someone develop their research skills and advance their career.
To benefit from the experience and guidance of my mentor in order to learn how to deal with my research topic and how to deal with research in general.
To offer guidance where it was needed.
However, only 50% of respondents reported that their expectations in participating in the programme had been met. Qualitative comments from respondents indicated that at least 2 of the 3 individuals who did not report that their expectations had been met did so because their research had not yet been published. Perhaps more support related to publishing research could be provided in futurethis was one aspect of the research cycle that was largely left untouched by the programme support staff.
E-mail was the most frequent form of contact between mentors and protégés (4 responses), while in-person meetings were the most common form of contact for the remaining 2 respondents. Mentors met with their protégés monthly (84%) or several times a month (16%) and qualitative responses from survey participants indicated that the frequency of meetings varied depending on the stage of the project and their respective availabilities, as would be expected. However, the amount of contact may not have been enough: although 50% of respondents indicated that the contact was "just right", 50% reported that they would have preferred more contact.
Feedback about the Moodle site was mixed. Four participants reported that they visited the site and 2 of these participants reported that they found the site "helpful" while 2 reported that they found the site "neither helpful nor unhelpful". This indicates that there may be some room for growth with providing a support network for future participants. This was confirmed by comments from participants.
Only 2 of the 6 respondents reported that they agreed or strongly agreed with the statement "my expectations of my mentor/mentee were met" (3 neither agreed nor disagreed and 1 respondent disagreed). The expectations reported by participants included: "that she would find her own way in this very new and rather difficult topic", "I expected him to guide me all through my study", "That they would provide me with cues about how much guidance was needed and on what specific aspects of the project they most needed assistance". More clearly outlining the expectations of the mentors and protégés at the outset of the project may have been useful, as it is unclear whether both parties were aware of the expectations that their mentor/protégé had of them. Although the Moodle site included resources and readings about mentoring, 3 participants indicated that they would have appreciated more guidance on the mentor-protégé relationship. Perhaps some additional or more practical information would have been useful, such as a mentoring contract where mentors and protégés formally agree on goals and expectations.
Although the Moodle site provided space for mentors and protégés to share ideas (private and public discussion forums) 2 respondents indicated that they would have liked a space such as this. This suggests that either participants were not aware of the Moodle website or that the spaces provided were not suitable to their needs. As one respondent writes: I think that having 6 persons from 5 different parts of the world was a great opportunity: different research topics, different experiences (in life and profession). But I know almost nothing about the others and about their projects, and I think it was a lost occasion. Obviously every one of us was very busy with their everyday life and job AND the research, but maybe it would have been possible to share a little more of our experiences. Perhaps the sharing should be "run from above", by someone in charge of the partnership.
The Moodle spaces were largely left to the participants to use as they saw fit. For example, everyone was invited to introduce themselves and share time management / organisational tools and tips but only a few participants took this opportunity. Perhaps in the future the sharing needs to be more organised, for example reminder e-mails that include a small bio of each participant and a small blurb about the research project.
All mentors and protégés agreed that the Partnership was either "effective" or "very effective" in assisting protégés to develop research skills. One mentor commented, "the project focused the mentee to think deeply about research and to ask questions". The most rewarding part of the Partnership seemed to come from the research process itself: I welcomed the chance to encourage someone who wanted to do research in the field.

I was (and still am) really interested in my research's topic and somehow I made the research I wanted to read.
[the most rewarding part of participating in the programme was] when I concluded my research work.
Thus, it seems clear that the Partnership brought the importance of conducting research to the forefront of the participant's minds.
Most pleasingly, 100% of participants reported that they would be involved in a research-mentoring programme again and all respondents indicated that they would recommend the IFLA Researcher-Librarian Partnership to other potential mentors and protégés.

Recommendations
In conclusion, a number of recommendations on the role of and process for delivering research mentoring programmes are offered.
Recommendation 1: Based on the feedback from participants, it is suggested that the IFLA Researcher-Librarian Partnership should be offered again in the future.

Recommendation 2:
It is suggested that the Moodle site be retained but additional support resources be added and the site be appropriately publicised, to ensure all participants are aware of the resources available. Particular emphasis should be placed on the mentor-protégé relationship (for example, adding practical guidance such as a goal setting template) and information about how to publish research.

Recommendation 3:
To foster a meaningful and productive mentoring relationship regular communication and engagement between the mentor and protégé throughout the life of the programme should be encouraged and supported. For example, an online orientation at the commencement of the programme would allow participants to meet and to establish a rapport; and during the life of the project participants should make use of technologies such as Twitter, Skype and Instant Messaging to nurture an active relationship.

Recommendation 4:
To ensure overall quality and sustainability of the programme suitable resourcing must be provided (e.g. a project officer responsible for developing and maintaining the Moodle site, for fostering a collegial environment).

Discussion and Conclusions
This paper has provided an overview of a research mentoring programme initiated by the International Federation of Library and Information Associations and Institutions (IFLA). The Researcher Librarian Partnership demonstrates how LIS practitioners, educators and professional associations can work together to support the development of research skills and knowledge in the profession.
The Partnership was a one-time two-year initiative that was undertaken to explore what was possible, and to consider how mentoring could play a part in building a culture of research in the LIS profession. The programme reveals how mentoring can help foster a research culture. Though this programme current LIS practitioners had the opportunity to develop their "experience" (Koufogiannakis and Crumley, 2006) with research. The programme provided a supportive and non-threatening environment in which the protégés could develop their skill, knowledge and confidence. By learning how to critically engage with, and conduct research, LIS practitioners will be better able to serve the individuals and communities they support.
The program came to a close at the end of 2011. Since this time a significant number of LIS professionals have contacted the association wanting to know if the programme would be offered again. Clearly there is both interest and a need for research mentoring in the LIS profession. The challenge is to find a way to continue this type of initiative in a sustainable and meaningful way.
One option which could be considered is to enlist publishers of LIS research to provide support for an annual research mentoring programme modelled on the IFLA LTR Section's project. Through this annual programme aspiring researchers would be invited to submit proposals to the LTR by a specified date each year, with volunteer members of the Section acting as reviewers of the projects and putting out a call for researchers from the IFLA LTR membership. In addition, the members of national and regional associations with LIS research interests could volunteer to work with the aspiring researchers on the project proposals selected as the most viable. This effort might be supported by a rotating group of LIS research publishers and research organisations coordinated by the IFLA LTR Section. It is hoped that the success of the project described in the paper would stimulate volunteers to step forward.
In terms of practical implications this project has shown that there is a need for mentoring of early career LIS professionals. The project has also shown one way that this need can be met. It now is up to the LIS profession to accept the responsibility to ensure that the mentoring process of early LIS career professionals be continued. It is suggested that international and national professional groups such as IFLA, the iSchools Caucus and the various national and regional LIS and information-orientated education programmes should be encouraged to participate in the future planning of a programme to mentor LIS and information researchers.


The Effects of Plyometric and Resistance Training on Explosive Power and Strength of Young Male Volleyball Players

This paper examines the effects of plyometric and resistance training on the explosive power and the amount of young male volleyball players’ strength in Guilan province. The participants include 45 volleyball players, 15-22 years old, which were randomly divided into three groups: two training groups and one control group. Among the two training groups, one was engaged in plyometric and the other in resistance exercise training (six weeks, two times per week).The control group was doing its common volleyball training. Variables under measurement consist of strength and explosive power. We found that after a six weeks training, there were significant differences between the average of explosive power and strength in both groups of plyometric and resistance training in comparison to control group (P≤0.05). The researcher observed that there was a significant difference in the average of explosive power and strength within both plyometric and weight groups from pre to post-test (p≤0.05). The amount of improvement in resistance training group was more than plyometric group.

INTRODUCTION
Volleyball is one of the most active and exciting sport. Nowadays, it has got a special place among all other ones. In this sport, the presence of aerobic and anaerobic energy and also unpredicted fluctuations of biologic needs are completely observable (1). In the modern world, athletic experts are attempting to utilize new scientific methods to prepare professional athletes at different national and international competitions. Therefore, utilization of the best appropriate methods of training is the most important basis which must be performed by the experienced in professional fields. Nowadays, there is a wide range of scientific research on athletes, competitions and physical training. In this regard, one of the most effective ways in relation to the improvement in strength and resistance increase, speed and athletes' physical energy is plyometric training (2).
One of the other long standing ways used so far is weightlifting training. Historically, these exercises were meant as the amount of physical strength. But, nowadays, people at different ages, use resistance trainings to increase their physical performance, strength and speed, prevent injuries and keep their skeletal function safety (2). Sankey, Jones, and Bampouras (2008) have conducted a research on the effect of two plyometric training programs with different intensity on vertical jump performance among high school athletes. They concluded that 6-week plyometric training cause to improve in vertical jump performance (3). Brown et al. (2007) examined the effect of two plyometric training with weightlifting programs on aesthetic jumping ability among men at university. There was no any different between plyometric and weightlifting groups in jumping performance (4). Jothi, Vinu, and Eleckuvan (2010) have examined the effects of a 7-week-combined resistance and plyometric program on the improvement of selected biomotor abilities. The findings revealed that feet's strength (%4.13), resistance power (%11.81), speed (%4) and in anaerobic strength (%7.53) improved after the training program. Therefore, combined resistance and plyometric training programs had statistically significant effects on the improvement of mentioned variables (5).
More efficient programs can be designed for athletes who possess physiological capabilities. Therefore, utilizing some appropriate strategies by the experts and trainers will bring desirable consequences. Similarly, the time and the cost spent for the preparation of each athlete will be decreased. Some trainer and also researchers believe that young people may be injured due to plyometric or resistance exercise. They think that growth sheets at the end of long bones may be injured and adolescents height affected by this injury. In fact resistance exercise is the same as double-edged sword. On one hand, it can promote athletic performance, and on the other, it can stop skeletal-muscle adaptation and bring injuries at growing phase in teenagers. In this way, it can result in the early and bitter separation between teenager and sport. Since weight training at this age necessitates special care and attention, and there are very few researches in this regard, this paper examines the effects of plyometric and resistance trainings with weight on explosive power of young Guilan province volleyball players.

MATERIALS AND METHODS
This study was a semi-empirical and applied research and a pretest-posttest plan with two experimental groups was used.
Subjects. forty five young male volleyball players (16-18 yrs BMI: 21.75 Downloaded from aassjournal.com at 13:43 +0430 on Friday July 5th 2019 kg/m 2 ), who participated in province matches in 1391, were studied. They were randomly divided to three equal (n=15) Plyometric, resistance and control groups.
A written consent was obtained from all subjects after explanation of all phases of the test. All health parameters were considered and the participants were allowed dropping out of the research whenever they want. All tests and trainings sessions were conducted at the standard clubs in Guilan. Explosive power and isotonic strength were measured by sargent and squat tests (1RM) respectively, before and after exercise programs in three groups.
Training program. Exercise training protocols included: resistance and plyometric were conducted for six weeks, two sessions per week and 90 min (warm-up, main exercise, volleyball special training and cool down) per session. Control group was under previous exercise training. Plyometric program contained 14 various elementary and mid-level jumps (6). Resistance training included 5 stationary manner exercises with 60% one repetition maximum (1RM), 12-15 repetition and 2-3 set.
Statistical Analysis. Descriptive statistics were used in order to describe the variables and compare the average difference in the participants' pretest and posttest. To examine the normality of the data distribution Kolmogorov-Smirnov test was used. To compare the changes in variables between three groups one way ANOVA was used. The level of significance was p≤0.05.

RESULTS
Explosive power and strength of two experimental groups increased significantly from pre to post test (p≤0.05) but there were no any significant changes in control group during the same time (p˃0.05) (Figure 1and 2).    As it can see in Table 1 there were significant differences in explosive power and strength changes between groups. In fact post hoc test showed that the increase of explosive power and strength in two experimental groups were significantly deference to compare with control group (p= 0.001).

DISCUSSION
It was evident that after six weeks plyometric and resistance training, young volleyball players have had great improvement in their skeletal strength and explosive power. In comparison to different training methods, the most improvement was observed in resistance training group and then in plyometric group.
Bosco believes that the increase in explosive power is the result of two factors: adaptation in bio motor organs and the improvement in skeletal functions to store motor energy in elastic elements. In other words, the role of plyometric training is feasibility in neural-skeletal system which results in the fast transference from external constriction to internal one (7). Therefore, increase in the strength as a result of plyometric training, may be because of the increase in the size of muscle fibers. Adaptation in the production of skeletal power is in close association with the increase in muscle fibers. Plyometric training programs cause to significant increase at the first and second type of skeletal muscle fibers (8).
Perhaps the increase in explosive power after plyometric trainings in this study, was due to neural adaptations such as: preparation of more biomotor ability, excitation of motor units with more frequency, selective utilization of motor units in adapted muscles, concurrent constriction of some muscles, the change in the speed of external and internal constriction phase and the increase in skeletal resistance and strength.
The results of this study are in line with the results of some other researches. The participants' explosive power after six-week resistance training has revealed significant increase. Weight Training can increase skeletal-muscle cross section and increases the adaptation of motor units. Although hypertrophy plays a role in the first level of resistance training, neural compatibility is of more importance (9,10). Resistance training can increase body's ability up to 50%; and then activates neural system and prepares the skeletal muscles (11). Previous studies show that the performance of resistance trainings is necessary to increase skeletal strength and speed; because it results in motor excitation and increases reflective potentials (12,13). It seems that performing weight trainings with 50 to 85% intensity, results in resistance increase. Also, the participants were asked to perform the actions fast. In this way, the transference from internal phase to external phase takes less time. On the other hand, since strength is a combination of resistance and speed, with performing weight trainings, improvement in skeletal strength is evident. This study is in line with other researchers' findings.
The results of this study shows that resistance and plyometric trainings have significant effect on the strength of young volleyball payers 'feet. Also, it was found that resistance training group has got better results than plyometric group; the findings are in line with Vissing et al. (2008) (14), and in contrast with Brown et al. (2007) and MacDonald et al. (2013) (4,15). Vissing et al. (2008) examined the effects of resistance and plyometric trainings on skeletal resistance. They found that with a 12-week-training, the strength was significantly increased. This increase may be due to increase motor unit utilization. Strength among those participants, who haven't experienced resistance exercise, was most increase, because of the improvement in motor skills and the harmony in their motor units (14).
Therefore, it seems that neural factors, especially at the first few weeks are one of the most important reasons in the participants' strength improvement (10).
One of the most important parameters of resistance in association with plyometric trainings is the flexibility of skeletal muscles; i.e., its ability increase in tension. The variation range of flexibility is directly related to the ability of tissue for resistance against power and its return to its first mode when removing the exposed pressure. This is the stretched function that is utilized in plyometric training (15,16).
Selected weight training in this study, provided the development in the participants' explosive power. These findings are in line with Singh (2012) (16).
Main finding of this study is that light to moderate plyometric and stationary manner resistance training on 60% 1RM can be helpful for improving in explosive power and strength. These are most important variables for volleyball skills. It seems that exercise trainings like those implicated in this study are safe and effective methods for improvement of young volleyball players' motor skills.

CONCLUSION
The research finding shows that the participants' explosive power and their skeletal resistance will have significant increase with the help of plyometric and resistance trainings. In comparison between two experimental groups, there have been significant differences, in which resistance training group have shown more improvement.


Shallow Theory of English Learning Strategies and Autonomous Learning

When university students are required to acquire an even higher English language ability, it is evident that a compound teaching mode of teacher-facilitated, student-centred classroom work combined with computer-assisted spare-time autonomous learning will replace the traditional single and teacher-centred instruction. On the basis of defining the characteristics of English learning strategies on autonomous learning, this paper is to verify through an empirical research the impact of English language learning strategies on autonomous learning. Accordingly, students are expected to rely on the teaching ability of learning strategies for individualized and task-oriented learning strategies so that eventually effective autonomous learning is generated.

learning strategies can be either dominant external behavior, or a recessive psychological activity; learning strategies can be training and learning.

Classification of Learning Strategies
At present, the learning strategy mainly has four kinds of classification methods: Based on the cognitive theory framework and information processing theory of Anderson (1998), the learning strategy is classified into met cognitive strategies, cognitive strategies, and social/effective strategies. Cohen (1998) divided language learning strategies into learning strategies and tactics on the basis of the purpose of using strategies. Ox-ford (1990) thought language learning are indirect and direct strategies based on the relationship between learning strategies and language materials. WEN Qiu-fang (1993) in China both attached the relationship between learning strategy and learning process, and emphasized that learning strategies should be divided into management strategies and language learning strategies on the basis of the relationship between learning strategies and learning materials. Among them, the classification method of WEN Qiu-fang is the product of referring, summarizing, and supplying; it adds some Chinese characteristics, and has a great guiding significance for Chinese students' learning strategies under Chinese's environment (Brown, 2002).

Significance of Learning Strategies
WEN qiu-fang (1993) thought differences in English learning strategy have a decisive impact on performance when other things are equal. She also pointed out that any properly learning strategies are more likely to make the learners learning success. Maclntyre and Noels (1996) thought that learning strategy is helpful in understanding the process of learning. At the same time, learning strategy can also reduce confusion and anxiety for scholars, which will keep their learning enthusiasm and motivation. Besides, they also thought learning strategy will improve students' learning attitude, and enhance their learning motivation. In some people's view, met cognitive strategy will make learners get a comprehensive understanding of the learning process, have the ability of foreseeing and monitoring error, and eventually complete the reflection, evaluation, and construction of knowledge. Language starts from and meets the needs of society, so language learning should also be in the real society, and such learners will realize information exchange in communication and interaction. There is not only the phenomenon of information unequal, but also the lack of the language form when using the target language to express some content for foreign language learners. At this point, the communicator can complement each other with the aid of social communication strategy, improve their language form and content, and complete the communicative tasks by circuitous type when interlingua is still not perfect.

Definition of Autonomous Learning
For the definition of autonomous learning, linguists and education experts always failed to reach a consensus. But this does not mean chaos, just shows the richness and personalization of autonomous learning research. Such differences are not bad, but good; it shows the wide attention of the importance of independent learning itself, and indicates that the research and results in the future will be more comprehensive. Synthesized, combined with the particularity of foreign language learning, foreign language autonomous learning can be summarized as: the results of the unifying of external environment and internal environment harmonious.
External environment consists of teachers, teaching facilities, and corpora; it has a promoting effect on internal environment, and it is the precondition to realize autonomous learning and material basis. Internal environment contains the learners' attitude and ability (Wennden, 1992, pp. 27, 432-441, 121, 163).

Characteristics of Autonomous Learning
College English autonomous learning has the following characteristics: (1) The ability of using learning strategies effectively. Research shows that in the process of college English learning, students have unfamiliar feeling naturally and fuzzy understanding in final results, which makes the activation and train of the ability of college English learning strategy important particularly. Their first need is self-management ability, namely, "a high degree of self-discipline, enough self-organizing ability and rigorous planning", so having and effectively using met cognitive strategy is the key for them to achieve learner autonomy.
At the same time, it is the first link to cultivate students' autonomous learning ability of English.
(2) The external environment is very important. First of all, in the college English autonomous learning, teachers must take the role of "organizer", "the instructor", and "helper" advocated by constructivism learning.
Autonomy does not necessarily mean full independence. Secondly, modern teaching facilities are another important factor to realize the effective autonomous learning. A blackboard and a piece of chalk in the traditional single teaching mode do no good to develop cross-cultural communication ability of studying English, and cannot finish the request on time. And multimedia technology introduced to campus makes it possible to learn English autonomous learning in the technical environment. Finally, the diversification and three-dimensional corpus are the direct content sources for students to learn the target language culture, and is also the centre of autonomous learning for many students (Dornyei & Csizer, 1998, pp. 203-29).
(3) The learning process of interaction. Language communication is interactive, both interpersonal interaction and man-machine interaction. Therefore, autonomous learning in college English is not a simple personal learning for students; it must be born in the interaction between students and others. Autonomous learners should not only have an independent spirit, but also have the spirit of cooperation, deepen the understanding of their selves in the cooperation, and solve their own problems (Guay, Ratelle, & Channel, 2008, pp. 233-240).

Learning Strategies and Autonomous Learning of New Students
In this paper, we make a research on English learning strategies and autonomous learning making 80 freshmen in non-English major of grade 2013 as the research object. These new students are not strangers for learning strategies and autonomous learning. It can be found in the questionnaire that, students' previous learning experience and strategies of natural acquisition tend to be at work in "scene strategy" and "interactive learning".
There are more than 80% freshmen being using and planning to use met cognitive strategy, only the "adjust" indicators have not reached 80%, which shows that the self-management for freshmen in autonomous learning already has a certain foundation, as long as being provided opportunities, they must be able to successfully do a good job in autonomous learning. It also reminds us to pay attention to some links, such as "self-assessment" and "self-adjustment". There is a classmate who said in an interview: I was able to set learning goals, also can separate the long-term and short-term planning, optionally using some basic learning strategies, and consciously to monitor their own learning process, but I don't know how to evaluate the effectiveness of the learning process and result of significant, the occurrence of unexpected situation and tend not to adjust itself when setbacks. (personal communication, September 21, 2005) Freshmen have basic cognitive strategy awareness; they call it "learning method". The use of "memory strategy", "lexical grammar input", "translation strategy", and "recitation strategy" is respectively l00%, 100%, 94.5%, and 91%. But the information also shows that, most of the methods of freshmen's cognitive strategies are the most basic, traditional, and even may be time-consuming, which almost can only make students learn the target language in grammar translation. The ignorance rate of freshmen in more advanced and interactive learning strategies is quite high, which shows the senior learning strategies need activating and training, also suggests the guidance of middle school English teachers in the usual teaching method. There is a classmate who said: I learned from others how to learn English better, but my time is limited, and can only be centred on teachers and teaching materials, memorize the words and contents teacher said, as well as the follow final exam and quiz. (Spratt & Chan, 1979/1981 Freshmen's English learning motivation is still dominated by instrumental; doing hard in learning English just for passing the exam is as many as 98%. The test drive not only determines the learning content and learning style, and also leads the student to ignore or be too busy to engage in independent learning. Learning the use of good foreign language environment in a real language context by understanding the target language culture is far from forming. Therefore, the understanding of the concept and significance of autonomous learning in freshmen is very shallow. At the same time, they have no ability to determine language requirements and choose corpora; teachers and textbooks are their most dependent external environment.
In addition, the study also found that variables related to autonomous learning have also changed when learning strategies change. Students must understand that the college English autonomous learning is not studying by themselves, they must cultivate communicative competence in the interaction of real situations.
Learning motivation by instrumental to synthesizing type enhances the sustainability of English autonomous learning. With the students' ability in learning language being stronger, the possibility of autonomous learning will further increase. The improvement of the ability of meta-cognitive strategy in autonomous learning makes students learn self-management. At the same time, students will also learn to arrange specific learning tasks in the frame of the macro process, and achieve the goal of autonomous learning in the process of setting goals, making plans, monitoring and evaluating gain and loss. Learning to use cognitive strategies properly when implementing specific learning tasks will enrich the ways of autonomous learning, at the same time, it will further strengthen the confidence of the students' in autonomous learning. Promoting and realizing the college English autonomous learning by the media strategy is a new important means in the environment of new technology. To make the autonomous learning not restricted by the space-time, students should use the Internet and campus online to embody the corpus of real context, and use the activity and learning system specifically for autonomous learning.
In fact, the improvement of college English autonomous learning ability should mainly express for: (1) timely and flexible use of various positive interactive communicative learning strategies, (2) reducing the dependence on teachers, and (3) doing more self-service learning by means of media resources and teaching materials. A science boy in the university entrance exam said: After trying to use learning strategies, I consolidate the classroom learning experience, learn more new knowledge, and I learned how to do autonomous learning. Besides, I can solve many problems without the teacher's help, and I have been able to determine what to learn and how to learn. (personal communication, December, 2013)

Conclusion
Although learning strategy is only a part of the autonomous learning ability, the formation of the ability of learning strategy makes it possible to autonomous learning. The consciousness of using learning strategies is stronger, the process of autonomous learning will be more complete, and the effect will be better. There is no doubt that learning strategy has a significant effect on autonomous learning, but it is not simple, its mature speed and quality will be effected by traditional education, examination system, and the external environment of independent study. At the same time, we must realize that learning strategy is a process, not an end, and cultivating autonomous learning ability is the final purpose, so that learning strategy training in the final analysis is to internalize the independent learning ability. In the state of improving the ability of autonomous learning, in addition to using the methods above, we also can try to put autonomous learning contents into regular examination or four, six levels of tests, play the drive function of test, and guide the autonomous learning ability of students. All in all, the successful implementation of "curriculum requirements" in new college English needs the support of students' autonomous learning ability, and the general language education workers, especially the college English teachers, need to find ways to create and provide a platform for students to learn theory and practice, and try their best to bring up a lifelong learner.


The Flipped Classroom: A Modality for Mixed Asynchronous and Synchronous Learning in a Residency Program

Introduction A “flipped classroom” educational model exchanges the traditional format of a classroom lecture and homework problem set. We piloted two flipped classroom sessions in our emergency medicine (EM) residency didactic schedule. We aimed to learn about resident and faculty impressions of the sessions, in order to develop them as a regular component of our residency curriculum. Methods We evaluated residents’ impression of the asynchronous video component and synchronous classroom component using four Likert items. We used open-ended questions to inquire about resident and faculty impressions of the advantages and disadvantages of the format. Results For the Likert items evaluating the video lectures, 33/35 residents (94%, 95% CI 80%–99%) responded that the video lecture added to their knowledge about the topic, and 33/35 residents felt that watching the video was a valuable use of their time. For items evaluating the flipped classroom format, 36/38 residents (95%, 95% CI 82%–99%) preferred the format to a traditional lecture on the topic, and 38/38 residents (100%, 95% CI 89%–100%) felt that the small group session was effective in helping them learn about the topic. Most residents preferred to see the format monthly in our curriculum and chose an ideal group size of 5.5 (first session) and 7 (second session). Residents cited the interactivity of the sessions and access to experts as advantages of the format. Faculty felt the ability to assess residents’ understanding of concepts and provide feedback were advantages. Conclusion Our flipped classroom model was positively received by EM residents. Residents preferred a small group size and favored frequent use of the format in our curriculum. The flipped classroom represents one modality that programs may use to incorporate a mixture of asynchronous and interactive synchronous learning and provide additional opportunities to evaluate residents.

INTRODUCTION
Didactic conference scheduling presents a challenge to the leadership of residency programs in all specialties. Programs that have switched to shift-based schedules to accommodate duty-hour requirements are finding that residents' attendance at conference is impacted. 1 Emergency medicine (EM), with its inherent shift-based schedule, is no stranger to this dilemma. Not only does shift work impact EM residents' ability to attend conference, but EM faculty are challenged with finding a balance between preparing for and delivering didactics and managing their own clinical shift schedules. Residents at our institution have commented that they would like greater participation by faculty in didactic sessions. At the same time, the implementation of the Accreditation Council for Graduate Medical Education (ACGME) milestones project in the United States has increased the importance of opportunities for face-toface evaluation of a resident's medical knowledge.
Over recent years, the methods and information sources that Young et al. The Flipped Classroom residents use to learn have changed. The availability of online material in the EM community has exploded, with residents increasingly making use of blogs, podcasts, and social media for learning. 2 An entire movement, dubbed FOAM (free openaccess "meducation") has emerged in the online EM community. 3 The FOAM concept promotes freely available, "sophisticated, cutting-edge learning resources that enable clinicians and students to update their knowledge and improve their understanding in a fun, motivating and time efficient way." The increase in online medical education material parallels one that has occurred in secondary education. The Khan academy is a non-profit organization whose website offers thousands of video lectures on multiple subjects. 4,5 Secondary educators have used these and their own videos to create an alternative to traditional didactics that has been called a "flipped classroom." 6 A flipped classroom exchanges the usual process of delivering content synchronously in the form of a classroom lecture and then assigning a problem set for students to complete at home. The didactic is viewed at home asynchronously, usually in the form of a video lecture, and students use classroom time to work on the problem set in a setting where they can receive help and clarification from an expert. The concept has also been described in undergraduate, 7,8 as well as graduate education. 9 Advantages of the asynchronous video component include the ability of the student to pause, rewind, and fast forward the content to process information at their own pace. 10,11 Suggested benefits of the in-class session include the ability of the educator to guide the application of knowledge. 10,11 The need for alternatives to traditional didactics in EM has been acknowledged. The Council of Emergency Medicine Residency Directors (CORD) Academic Assembly Conference Alternative Workshop published recommendations in 2008 that included the integration of asynchronous learning activities (also called "individualized interactive instruction") and promoted more flexibility in didactics. 12 The Residency Review Committee (RRC) for EM allows for residents to use individualized interactive instruction for up to 20% of the planned educational experiences or didactics. 13 However, the best method to incorporate asynchronous learning remains unclear. 14 Early efforts, including sending residents tasks and receiving responses over e-mail, 15 journal article discussion boards, 16 institutional faculty-developed online video lectures 17 and modules, 18 and the adaptation of pre-existing modules, 19 have had mixed results.
We trialed a flipped classroom format in our didactic schedule as a potential means to incorporate 1) an asynchronous component that could allow for more schedule flexibility and appeal to a millennial audience and 2) a synchronous, interactive component with the potential to increase residents' interactions with faculty during our didactic sessions. Our aim was to learn about resident and faculty impressions of the sessions in order to develop them as a regular component of our formal curriculum. We present a description and evaluation of two pilot sessions below. To our knowledge, this is the first comprehensive description of a flipped classroom model in the setting of a residency training program.

Study Design
We undertook an evaluation of two flipped classroom sessions developed for EM residents. We created a mixedmethods quantitative and qualitative questionnaire to evaluate the flipped classroom sessions. The questionnaire included a fixed-response (quantitative) component with questions meant to evaluate specific aspects of the format, as well as an openended (qualitative) component aimed at exploring participant perspectives in a more in-depth manner. This is known as a convergent design and offers the benefit of observing trends while obtaining more detailed responses (and thus a potentially more complete understanding of a phenomenon) from a small sample size. 20 Responses for quantitative items took the form of either yes/no, fill in the blank, multiple choice, or five-point likert-type items, which were chosen for familiarity. The study was presented to our institutional review board and designated as exempt from review.

Study Setting and Population
Our EM residency is a three-year ACGME accredited program that consists of 13 residents per year, for a total of 39 residents. Our educational conferences are five-hour weekly sessions. Traditionally, this time has been filled by 50-minute lectures, separated by 10-minute breaks.

Study Protocol
The first flipped classroom session took place in December 2012. The topic was syncope and the session was scheduled for two hours. We chose a freely available online lecture about syncope created by Andy Neil and hosted on his website. 21 One week prior to the session, residents were emailed a link to the video with instructions to watch it prior to conference. A reminder email was also sent out on the day prior to the session.
To facilitate the moderated discussion, we created a worksheet of 22 questions on the topic of syncope. The worksheet was a combination of open-ended, multiple-choice, and matching questions. These included questions from EM board review sources such as the Physician's Evaluation and Educational Review in Emergency Medicine (PEER) series of question books, 22 the Council of Residency Directors in Emergency Medicine (CORD-EM) question bank, 23 as well as some written by residency leadership. Questions were selected to reinforce and expand on the topics presented in the video. Question stems were augmented with images of electrocardiograms (ECGs) and ultrasound stills. A corresponding key, with expanded answers and explanations for each question on the worksheet, was distributed in advance to the three faculty members moderating the small groups. The question set was not distributed to residents in advance.
On the day of the session, question sheets were distributed The Flipped Classroom Young et al.
to the residents in attendance and the group was divided into three small groups. The three faculty moderators each led a small group discussion. Other faculty members in attendance were distributed among the groups. The small groups met at large tables in the same room. Moderators led the residents through the question set. Residents were sequentially asked to each answer a question, taking turns in an "around the table" format until every question had been discussed. The discussion points were at the discretion of the faculty member, although potential "teaching points" were suggested in the answer key. After the session was complete, residents were asked to complete an anonymous evaluation form before leaving the room. The evaluation form had no identifiers other than postgraduate year, and residents placed their own evaluations in a collection box to ensure anonymity.
For the second session in February 2013, we chose the topic of pediatric gastrointestinal presentations. Again, we chose a freely available online video on the topic. 24 Residents were similarly emailed a link one week before the classroom session, with a reminder email the day prior to the session.
In response to concerns from residents in the first session about the additional time required to view the video outside of conference, we altered our plan for the second session. Residents were told that they would be excused from the last hour of conference if they attested that they had viewed the hour-long video prior to conference. They would be given the option to view the video in conference for the last hour of the day if they did not watch the video outside of conference. For this session, two members of the faculty of pediatric EM and a senior pediatric EM fellow were recruited to lead the groups. Again, a 22-item worksheet was developed, along with an answer key. Both were distributed to the faculty moderators in advance. Questions on the worksheet included images of radiographs and key physical findings.
The second session was run like the first, with several small changes. This time, the two-hour session started three hours before the end of conference. When the session was complete, residents were again asked to complete the same evaluation prior to leaving the room. Residents who signed that they had viewed the online video outside of conference were excused. For the last hour of conference, the online video was projected for residents who did not watch the video outside of conference.
To obtain the faculty perspective of the flipped classroom sessions, we surveyed the faculty participants for the two sessions. Faculty were asked to comment on advantages and disadvantages of the flipped classroom format.

Outcome Measures
Our primary outcome was the residents' impression of the value of the session, as measured by the proportion of positive responses ("Agree" or "Strongly Agree") to two Likert items evaluating the assigned video lectures and two Likert items evaluating the flipped classroom format. Our secondary outcomes included the proportion of residents who viewed the video prior to conference, how the residents accessed it, residents' opinion of the ideal group size, their preference for frequency of the format in our curriculum, and the qualitative impressions of the residents and faculty regarding the advantages and weaknesses of the flipped classroom format.

Data Analysis
We compiled survey data using Excel for Mac 2011 version 14 (Microsoft Corporation, Redmond, WA). Proportions, medians and interquartile ranges were calculated and histograms were generated using Stata SE 12.1 (Statacorp, College Station, TX). We calculated 95% confidence intervals for proportions with Stata using the modified Wald technique.

RESULTS
Sixteen residents signed in for the first session, of which 14 (86%) submitted surveys. The most common year of training reported was first year (6/12, 50%). For the second session, 31 residents signed the attendance sheet, and 27 (87%) surveys were collected. The classes were more equally represented for this session, which consisted of eight first-year residents (31%), 10 second year residents (38%), and eight third-year residents (31%). Two survey respondents did not report year of training for the first session, and one did not report year of training for the second session. For the second session, three residents stayed for the last hour of conference to view the video because they had not watched it outside of conference.
The figure displays distributions for Likert item responses to questions regarding residents' perceptions of the video as well as the small group format. For the Likert items evaluating the video lectures, 33/35 residents (94%, 95% confidence interval 80%-99%) responded that the video lecture added to their knowledge about the topic, and 33/35 residents felt that watching the video was a valuable use of their time. For the items evaluating the flipped classroom format, 36/38 residents (95%, 95% confidence interval 82%-99%) preferred the format to a traditional lecture on the topic, and 38/38 residents (100%, 95% confidence interval 89%-100%) felt that the small group session was effective in helping them learn about the topic. Most residents preferred to see the format monthly in our curriculum and chose an ideal group size of 5.5 (first session) and 7 (second session) (Table 1a and Table 1b).
Resident responses to open-ended questions regarding advantages and disadvantages of the flipped classroom are presented in Table 2. Faculty responses to the same openended questions are presented in Table 3.

DISCUSSION
We observed a high rate of positive responses from residents to both components of our flipped classroom model. Themes for the perceived advantages of the model in the residents' qualitative evaluations included interactivity, active participation, access to experts, and retention of material. Faculty noted that the sessions       offered the ability to assess residents' critical thinking and provide feedback. Themes for disadvantages perceived by residents included the increased time necessary to prepare for conference. These comments came from the first session, and we responded by allowing for conference "credit" if the video had been viewed asynchronously before the day of conference. We felt that this was a sustainable practice due to the RRC-EM allowance for 20% asynchronous learning, and seemed to help with the perception that additional work was being assigned. Themes for disadvantages perceived by residents in the second session centered on noise levels. We saw improved attendance at the second session, which was likely due to the first session occurring during the holidays. The improved attendance likely contributed to the increased noise level, as we held the conference in the same single room. We suspect that comments regarding lack of participation and the need to make the session "turn based" were due to faculty not adhering to our suggested protocol.
The shift in the way in which residents accessed the video from the first to second session probably reflects our decision to provide all incoming residents with iPad tablet computers, starting with our first-year class. The larger proportion of first-year residents at the first session probably accounted for the greater use of tablet computers to access the video for that session.
Our residents' perception of the ideal group size was small. A small group size would likely help with the issue of noise level that was commented on in the second, betterattended session. It would also likely accentuate the decrease in anxiety that was cited as an advantage, and increase breadth of resident participation, which was cited as a disadvantage.
To accommodate the group sizes that our residents preferred our full residency complement would need to be split into 5-7 small groups. Our residents felt that the ideal frequency of implementation of the flipped classroom in our curriculum was monthly, which would entail the development of 36 unique in order to create a non-repeating three-year curriculum. We were expecting a lower preferred frequency due to the newness of the format, and felt this was a positive endorsement for the format.
The RRC-EM has the following requirements for individualized interactive instruction: the program director must monitor resident participation; there must be an evaluation component; there must be faculty oversight; and the activity must be monitored for effectiveness. 13 The model as we have incorporated it provides the ability to monitor resident participation in the synchronous classroom session. Faculty provide oversight by selecting the content, creating the worksheet, and leading the small group discussions. We plan to monitor the format for effectiveness through evaluations of the sessions and by monitoring residents' performance on in-service exams after its incorporation into our curriculum.
We feel that faculty exposure to residents' patterns of critical thinking during the small group sessions is a strength of the format. When errors in critical thinking are found, residents can be given the opportunity to relearn critical concepts with faculty guidance. We anticipate that this will provide us with a robust platform for faculty to evaluate residents' medical knowledge that will complement evaluations done on clinical shifts. Our faculty commented that senior-level conversations during the sessions facilitated the more novice learners. The classroom small group sessions may allow residents with greater mastery of a topic to take more of a leadership and teaching role in discussions. This may offer faculty a potentially elusive opportunity to evaluate upper-level milestones.
From our experience with these two sessions, the best topics to choose for an EM flipped classroom are those that are broad and complex enough to sustain in-conference discussion, as well as those that lend themselves well to visual stimuli such as ECGs, clinical photos and radiographic images. We have also found that a rate limiting step can be finding a high quality video on such a topic, and suggest that as an early criterion. If the schedule permits (ours did not), it may be beneficial to schedule the "optional" hour in which the video is shown as the first hour of conference, so that those who have not watched the video can come at the usual conference start time and those who have not can come at the start of the flipped classroom session. This option would theoretically allow for more residents to have seen the video prior to the synchronous session.

LIMITATIONS
We report on a small sample size of residents from one institution, which limits the generalizability of our results. We evaluated an additional pilot session in an effort to increase our sample size. While this allowed us to obtain evaluations from a larger pool of residents, repeated measurements also likely biased our results toward the perspectives of residents who are more diligent with conference attendance. While familiar to respondents and easily quantifiable, Likert-type items limit dimensionality of responses, which likely did not capture the full breadth of resident attitudes. We hoped to mitigate this with the addition of open-ended questions. Our pilot sessions did not look at performance indicators or patient care outcomes to evaluate the success of the flipped classroom format. This would be an important future step that could be accomplished by a pre-and post-test comparing the flipped classroom to a traditional lecture format.

Future directions
For future sessions, we plan to place the small groups in separate rooms to address issues with the noise level generated by discussion. We plan to recruit more faculty to lead groups for future sessions in order to decrease group size. We are excited about the possibility of re-using the sessions in the future, allowing a broader base of faculty to participate in didactics without onerous preparation. As we proceed with the implementation of the ACGME milestones, we plan to incorporate faculty evaluations of residents' performance in the sessions as a means to provide data to our clinical competency committee regarding residents' progress. We are currently developing an online format for the flipped classroom worksheet that will allow for embedded video and audio to augment the problem set.

CONCLUSION
Our flipped classroom model was positively received by EM residents. Residents preferred a small group size and favored frequent use of the format in our curriculum. The flipped classroom represents one modality that programs may use to incorporate a mixture of asynchronous and interactive synchronous learning and provide additional opportunities to evaluate residents.