project curve, part six: collaborative instruction portfolios.

My last post described our increasingly fantastic rolling library project (check out a recent v-day Maker Break cart excursion and a alendar developed using the free version of LibCal). This post focuses on something considerably more stationary: collaborative teaching portfolios, one area discussed in a recent SCILWorks 2012 presentation given by myself and my two Instruction Services comrades, Science Librarian Sean Stone and Instruction Librarian Natalie Tagge.

The theme of the conference was strategies for “The Ubiquitous One-Shot,” so our aim was to explore curriculum mapping, rubrics, and portfolios, three interrelated efforts at the Claremont Colleges Library that are being used to better understand and target the collective teaching and outreach efforts of our community of librarians, which are (is as so often the case) frequently packaged in one (or two)-shot course-related workshops.

First, preso details and slide deck:

Methods Behind the (One-Shot) Madness: Enhancing Instruction through Mapping, Portfolios, and Rubrics

In-class techniques and e-learning tips can undeniably improve one-shot sessions, but collaborative management and outcomes-driven assessment of an overall instruction program is key to increasing instructor effectiveness and student learning in even the briefest of interactions. This presentation explores three pilot initiatives designed to build a structured and strategic framework behind “traditional” undergraduate instruction at the Claremont Colleges Library: instructor portfolios, curriculum mapping, and information literacy rubrics. Instructor portfolios collect teaching materials, student work, and coordinated evaluations in an integrative attempt to holistically assess one-shot library instruction from the perspective of faculty, students and librarians. Curriculum and knowledge mapping is a way of visualizing the path a learner takes through a discipline, department, or degree, as well as an engaging planning and brainstorming tool. Rubrics help define specific student outcomes as well as provide a quantifiable tool for assessment of information literacy skills.  When applied in tandem, these approaches can be used to gather powerful insight into the learner experience and create the impetus for more collaborative, creative, and lasting library instruction.

Our core message was that no one-shot is an island. Library instruction should be aware of and integrated within a wider learning context at the institutional and organizational levels, a process that requires planning, coordination, and reflection outside of the virtual or physical classroom (and preferably among colleagues). By taking steps to investigate and strategize in respect to a broader learning culture, even the briefest teaching interactions can be better organized in advance and have more impact during and after the moment of instruction.

portfolios as vortex abatement

Instructor portfolios provide a planning and evaluation platform for all of these temporal phases. Portfolio assessment has been applied in student learning and instructional development contexts for many years. A constructivist approach to achieving what Natalie described in our presentation as “holistic assessment”, portfolios are a collection of elements or assignments that, compiled over time, provide a fuller picture of a given learning experience. At the process’ completion, specific elements or the portfolio in its entirety can be reviewed and revised, thus creating a sense of progress and continuity as well as a coherent, intentional body of output.

Portfolios are often an individual exercise, but can also be used to excellent effect within a community of practice. For some time I have compiled simple personal portfolios of my own teaching efforts (e.g., a virtual folder and sometimes supplementary tangible file of course-related materials and assignments, planning docs, etc. I use with a class, presentation, or training). This is a productive process, but I have always been irked by the fact that so many useful templates and teaching materials are relegated to what is in effect an instructional vortex: e.g., my own machine or in clouds only I can access. I am also regularly vexed by the fact that, while I appreciate being able to see and learn from the strategies and approaches of my colleagues, they are similarly squirreled away in the deep space doldrums of digital and physical filing cabinets.

Our local project augmented the positive aspects of personal portfolios by using them as the basis of a shared, programmatic attempt to understand and assess a first-year library instruction program. This provided a structured basis for assessment and communication as well as reducing the all-too-common invisibility of individual compilations of instructional materials.

id1 portfolio project

This effort involved ten teaching librarians at CCL, who worked collaboratively over Fall 2011 to create virtual teaching portfolios for each of the 29 sections of our first-year library instruction program at Pomona College (known as Critical Inquiry: ID1). With no set syllabus or definitive subject area, ID1 courses range across disciplines, vary according to assigned faculty, and require a great deal of faculty/librarian communication and customization of course content and teaching strategies. In the past, librarians taught their respective ID1 sections largely independently and employed no common methods of evaluation, which, while fostering highly customized strategies and content, also resulted in a distributed teaching experience with minimal internal contact or collaboration.

Based on these strengths and constraints, I initially conceived the portfolio project to bring CCL teaching librarians together in a forum that would achieve three objectives:

1 To create collective librarian, student, and faculty assessment strategies that could be used in diverse teaching scenarios,

2 produce a robust archive of course-related and library instructional materials, and

3 foster collaborative ownership of the focus and outcomes of ID1 library involvement while also preserving individualized teaching strategies.

A longer-term goal was to test whether instructor portfolios might be scaled to other first-year and disciplinary teaching initiatives across the seven Claremont Colleges.

community (of practice) building

Instead of imposing the work these objectives required in a top-down manner (bad move for a new instruction coordinator), the hope was that a collegial process would identify points of common interest and generate buy-in and skill-sharing among participating librarians, thus facilitating group and individual instruction outcomes while teaching myself and several other CCL newcomers about the needs and priorities of our colleagues and the programs we work with.

The portfolio-building process was simple. We:

a) co-developed common assessment strategies for students, faculty, and librarians during a pre-instruction retreat, then deployed each evaluation at different points during the semester,

b) collected and uploaded course-related materials over the course of the term for each of our class sections (between 1-6 per librarian), and

c) met in a post-instruction retreat to reflect on our experience and evaluations in order to assess the project and inform future iterations.

Structurally, the local learning management system (Sakai) provided the basis for portfolio management, an enduring authenticated space involving little to no learning curve on the part of librarians.

This initiative revolved around each individual librarian or teaching pair conducting specific tasks at different points during the semester, such as uploading assignments and sending faculty surveys. Therefore, a portfolio checklist that laid out tasks and timing (the bright idea of Natalie Tagge) plus a few email reminders over the term aligned efforts and managed the weight of additional work that may otherwise have made the process unscalable.

collective assessment

The process of meeting in retreat to vet and revise student, faculty, and librarian assessment instruments was among the most valuable aspects of this process. In a situation where each librarian(s) communicates closely with a faculty member to customize instruction, common assessment strategies can be particularly difficult to achieve. In the case of the portfolio, I drafted initial student (end of class) and faculty (post-class and end-of-term) surveys and developed a group critique activity for our first portfolio retreat wherein three small groups critiqued and suggested revisions for each questionnaire, which significantly changed (and vastly improved) the assessments.

Student and faculty surveys were created in Google Docs and collectively administered so that responses fed into a common and transparent data pool, but were also subdivided for each class portfolio so that individual instructors could view their own evaluations separate from the group. Survey returns varied based on delivery method, with sizeable participation from students but much lower rates among faculty (revisions that will address this issue in subsequent portfolio projects is to develop an in-class instrument for faculty and adjust timing of the final assessment). For purposes of self-evaluation librarians completed and uploaded a three-question reflection shortly after their last in-class interaction with students, a strategy which prompted them to consider a positive and negative aspect of the learning interaction, as well as to identify an area for future improvement.

Students completed post-instruction evals in-class (typically embdedded as tabs in course guides) at a 59% rate of participation, while faculty post-instruction and end of term surveys (20% and 7%, respectively) were delivered to faculty via email. We also sent faculty end-of-term requests for anonymous examples of poor, average, and excellent student work. The feedback received from students was extremely valuable both on an aggregate and individual instructor level, providing concrete input into strengths and improvement areas (see below image for an excerpt of our evaluations). Faculty (post-instruction and end of term) and student survey instruments are available online.

Although student work examples were submitted by only one faculty member, discussing these anonymized papers (which included the instructor’s comments and grades) during our post-instruction retreat was among the most valuable exercises of the project. Our instruction so frequently catches students mid-stream that library educators aren’t often able to view authentic course output, effectively cutting us off from the very outcomes we hope to achieve. It is an eye-opening experience to understand the level and quality of student writing and research skills, and to gain insight into faculty evaluation priorities.

archiving instruction

An important deliverable of this project is that it creates an lasting, annualized archive of the cumulative efforts, learning objects, and outcomes related to a given instruction program on both the course and aggregate level. This strategy addresses previously mentioned preparation/assessment vortex that is created when teaching librarians collect and archive their work solely in inaccessible spaces – huge amounts of valuable syllabi, assignment prompts, handouts, exercises, slide decks, and assessments that, if organized and shared collegially, could inform stronger institutional communication and practice and vastly reduce duplication of effort.

Portfolio projects of this nature also streamline group efforts and produce ready programmatic evidence of instructional effectiveness and outcomes, crucial to the processes of accreditation, review, and value demonstration so many academic libraries are engaged in. The approach can be easily scaled to smaller and larger programs and participant groups, and can utilize many other learning/project management tools such as Google Apps or an institutional intranet – anything that provides a secure but accessible location to house instructional materials and evaluations.

aligning efforts

Perhaps most importantly, compiling and designing this project as a group fostered a transformative and project-based instructor development environment which was, to quote Jack Mezirow, “more inclusive, discriminating, self-reflective, and integrative of experience.”(1) At our post-retreat, participating librarians characterized the portfolio project as a valuable exercise that increased the sense of community and collective effort in an intensive teaching scenario, additional work that they found well worth the effort. We assessed involved learners and stakeholders in a voluntary and structured manner, empowering participating librarians with the ability to contribute views and suggestions we well as learn from their peers. In other words, our portfolio project combated the all-to-common satellite/black hole effect in academic library instructional design and assessment while  facilitating shared learning among a community of practitioners.

A significant number of revisions offered by the group will be included in the next iteration of this and other portfolio projects, including the aforementioned revised faculty evaluation strategies, collaborative rubrics of librarian content and teaching competencies that would align our messages while preserving teaching autonomy, as well as longer pre- and post-retreats.

Stay tuned for the next project curve post, topic ytbd.

(1) Mezirow, Jack. Transformative Learning: Theory to Practice. New Directions for Adult and Continuing Education, No. 74, Summer 1997. p 5.