ECAR Study of Undergraduate Students and IT ‘08

The ECAR Study of Undergraduate Students and IT 2008 was released today.  You can link to the key findings here, but the full findings are also freely available online.

This year’s study includes some information literacy questions based on the ACRL Information Literacy Competency Standards for Higher Education.  At first glance (I haven’t read the full report yet), these look like student self-report of selected items related to searching web-based sources (freely available and licensed databases).  Here’s what they found:

• 79.5% rate themselves highly for their ability to “use the Internet effectively and efficiently to search for information,” with almost half rating themselves as “very skilled” and another third rating themselves as “experts.”
• About half of respondents also say they are “very skilled” or “expert” when it comes to “evaluating the reliability and credibility of online sources of information” or “understanding the ethical and legal issues surrounding the access and use of digital information.”

I’m looking forward to examining this section more closely, but if anyone else gets to it first I’d be very interested to hear your thoughts.

This year’s report also includes a special focus on social networking sites.  One of the PIs for this study, Judy Caruso, is right here at Madison and I heard her talk about ECAR last year at a brownbag.

E-Portfolios and the UW System “Second Transcript”

I have been representing the library on a group that is exploring the concept of e-Portfolios.  This is a broad-based, campus group chaired by Wren Singer of the Center for the First Year Experience and Argyle Wade from the Office of the Dean of Students.  We submitted a proposal for TEL (Technology Enhanced Learning Funding) but eventually came to agreement that TEL was not a good match for our project because we wanted to start a broad exploratory process rather than create a product (yet).  This morning, we met with Aaron Brower and Mo Noonan Bischof to talk about next steps.  We agreed that we would embark on a more entrepreneurial process, where different groups (e.g. Leadership Certificate) would pilot reflective/portfolio approaches and our group would try to facilitate some convergence to identify what we would like to do as a campus.

At the meeting, I also became aware of a system-level “co-curricular transcript” or “second transcript” project.   To quote the Chronicle of Higher Education (subscription required or find the article in our databases): “The University of Wisconsin system might consider giving students two transcripts to send to potential employers [....]  The first would show classes and grades, and the second would describe the student’s personal achievements during college, including leadership experience and volunteer service.”  The group will meet for the first time tomorrow, and while commenters on the Chronicle article raised some serious concerns, it will be interesting to see how this fits with the e-Portfolio concept.

Why am I interested in all this?  A portfolio where students collect examples of their work provides a great opportunity for direct assessment — what evidence can we look at to see if students are information literate?  It also provides a nice bridge between reflective, individual learning and normative approaches like learning outcomes and standards.   (Lisa Janicke Hinchliffe talked about this normative idea at LOEX, and I’m finding it really useful in approaching less  structured projects with an open mind.  There’s more to learning than learning outcomes).

wordle – making meaning of arcane reports

wordle.net … I write a lot of arcane reports, and find this visual presentation of the text really interesting. Maybe we should do this instead of executive summaries… This is a twenty page report about assessment of student learning written for the Mellon Library/Faculty Fellowship of Undergraduate Research.

Understanding Students & Faculty: Susan Gibbons talk at CUWL Conference

Today at the statewide CUWL meeting and conference , Susan Gibbons, Dean and Vice-Provost at the University of Rochester River Campus Libraries, discussed their user research program. They are a small residential campus but an ARL member. The library’s work with anthroplogist Nancy Fried Foster is the subject of a recent monograph (free for download from ACRL) and a lot of positive press.

The presentation began with a discussion of characteristics of the “Net Generation.” As a group, these students are relatively more sheltered than previous generations. For example, they regularly seek advice from their parents on their schoolwork and have absorbed Mr. Rogers’s “you are special” message. They expect services to be customized, personalized, one-stop shopping, and in the “Mommy model” of service. These students feel pressured to succeed; 21% of those aged 15-17 have been diagnosed with some type of emotional disorder. They want to know the rules, are team-oriented, and want to collaborate in person and online. They tend to ask peers, not experts, for information. They are multitaskers, and value speed over accuracy in their work.

The next topic was techniques we can use to understand our students…

UW Alumni Profiles Project

Thought this excerpt from a press release about 2008 Alumni profiles was interesting…  http://apa.wisc.edu/Alumni/alumni_profiles.html

“Alumni Profiles answer some broad questions about where undergraduate alumni live, if they are employed or enrolled in educational programs, and how much they value their UW-Madison education.   Alumni Profiles are available for schools and colleges and for undergraduate majors.”

Among UW-Madison alumni who graduated within the past 10 years: (a few highlights)

• 88% are employed full-time (81%) or part-time (7%).  Nearly half of alumni have provided detailed employer and job title information
• 80% of those employed say that the skills they developed at UW-Madison in problem solving, written and verbal communication, and other general skills are related or highly related to their current position
• 59% of those employed say that their current position is related or highly related to the major of their UW-Madison degree

Voluntary System of Accountability Report

I learned last week that UW-Madison is participating in the Voluntary System of Accountability and created a College Portrait of Undergraduate Education.

In November, 2007, the Chronicle of Higher Education published an article on the system (subscription, also in Lexis-Nexis and other online databases): “In one of the most sweeping responses yet to calls for accountability in higher education, a public-university association has adopted a template, called the College Portrait, that will allow institutions to share with outsiders online data about such matters as students’ academic progress.  Use of the portrait will be voluntary, but its approval this month by the Board of Directors of the National Association of State Universities and Land-Grant Colleges marked the beginning of a formal effort by the association to encourage institutions to use it.”

The short portrait includes data on the campus, as well as data related to student learning outcomes from three standardized instruments.  Perhaps more significantly, it includes data on how UW Madison intends to measure student learning outcomes locally.   For those who have been following the ongoing struggle over accountability measures in higher ed, this represents institutions’ push to be proactive in defining and controlling their own measures of student learning, rather than having something imposed from the outside.   Here at UW, the effort to supplement long, scholarly reports on student learning outcomes with shorter “nuggets” to share with the public is a very important campus effort.

Breakfast Meeting on Essential Learning Outcomes – history in the making?

This morning, I was a facilitator at a meeting on essential learning outcomes sponsored by Vice-Provost for Teaching and Learning Aaron Brower, the University General Education Committee, the Offices of the Dean of Students, the University Assessment Council, Academic Planning and Analysis, Orientation and New Student Programs, L&S, and L&S Academic Affairs. Phew! I serve on UGEC and the Assessment Council. The purpose was to invite selected faculty who teach large Gen Ed courses and courses that serve large numbers of new students to discuss the question: what do we hope our students will learn which transcends content (essential learning outcomes).  We addressed the question in a few different ways.

Aaron Brower opened the session by saying that this was the first time faculty had been asked to address this question, so history in the making. Jolanda Vanderwal Taylor talked a bit about LEAP and then we addressed the question in two table groupings: one across disciplines and a second with people in the same subject area (STEM, quantitative reasoning, communication, humanities, or social sciences).

At my first table, there was a real feeling of shared responsibility for developing the skills that help students succeed: everything from learning to take notes to how to analyze readings. There was also a sentiment that large GE courses can get students engaged, but cannot hope to equip them with the methodologies of the discipline they are taught in. Instead, they should enable students to approach a subject (science, political science) critically over the course of their lifetimes, and also understand the “limits of a discipline.” The second table I facilitated was the people who manage and teach courses that meet the Communication requirements. I have frequent conversations with this group, and we already have a framework of shared learning outcomes. The discussants felt that other faculty should understand that these courses do have their own “content,” and should not become the place to teach all baseline skills. We also talked about intentionality; how we share learning outcomes with students.    While information literacy was not discussed explicitly, most of the conversation was about “learning to learn,” so directly related to our work in this area.

It was a brilliant group of people, so facilitation was a very easy job. I agreed with Aaron Brower that it was a(n) historic moment — working together to make our goals for student learning explicit. I look forward to sharing the notes and handouts when they are posted. I asked for permission last week but it was felt that they were “not ready for prime time.” I think we can begin to examine them in meetings to inform our own planning, but it’s not a good idea to post them in this public forum until they are made available publicly.

Informal Assessments

I received an e-copy of the College of Engineering’s Teaching and Learning Insights newsletter and it is really nice. I know that Wendt is working on getting some kind of information literacy project featured here in the future, which would be really great.

Each issue has a feature on assessment. So far their Informal Assessment Series includes clickers and quizzes. Sharing strategies for classroom assessment is a project our Learning Assessment Working Group is engaged in right now. It’s interesting that while our teaching contexts are frequently different (one-shot/semester-long, hands-on/large lecture, librarian/faculty) the techniques are similar.

In fact, we’re all using to Cross and Angelo’s Classroom Assessment Techniques as a starting point. This book is the source of the minute paper, muddiest concept, and other handy techniques you’ve probably encountered before.  A copy of this is available in my office if anyone would like to look, or I’m sure you could find it in the library!

Mellon Assessment Report

It’s up!!!! I spent several years working as the Assessment Consultant for the Mellon Library/Faculty Fellowship for Undergraduate Research The goals of the project were to redesign courses and assignments to incorporate research-based learning. My activities included facilitating sessions on assessment in a summer faculty institute, working with teams of support staff to implement redesigned courses (often very large courses), and working with faculty to develop strategies for assessing student learning. You can read more about my work on the project and what we learned in the Assessment of Student Learning Report.