Cognitive Apprenticeship

I just read a fabulous article on Cognitive Apprenticeship by Allan Collins, Ann Holum, John Seely Brown .  It's available online here: http://www.21learn.org/site/archive/cognitive-apprenticeship-making-thinking-visible/1.  Their theory is that the model from which we learn most naturally is as apprentices and that it is possible to re-design school learning so that even cognitive-based tasks are learned/taught in an apprenticeship model.

"But in school, teachers are working with a curriculum centered around reading, writing, science, math, history, etc. that is, in large part, divorced from what students and most adults do in their lives. In cognitive apprenticeship, then, the challenge is to situate the abstract tasks of the school curriculum in contexts that make sense to students."

A difficulty is that in a traditional apprenticeship the tasks are well-defined and focused on a discrete output.

"The tasks in schooling, however, demand that students be able to transfer what they learn. In cognitive apprenticeship, the challenge is to present a wide range of tasks, varying from systematic to diverse, and to encourage students to reflect on and articulate the elements that are common across tasks. As teachers present the targeted skills to students, they can increasingly vary the contexts in which those skills are useful. The goal is to help students generalize the skill, to learn when the skill is or is not applicable, and to transfer the skill independently when faced with novel situations."

They go on to suggest a series of teaching methods that help students see the applicability of the content to a variety of tasks and that help them transfer what they have learned in one context to other contexts.

"Teaching methods should be designed to give students the opportunity to observe, engage in, and invent or discover expert strategies in context. Such an approach will enable students to see how these strategies combine with their factual and conceptual knowledge and how they use a variety of resources in the social and physical environment. The six teaching methods advocated here fall roughly into three groups: the first three (modeling, coaching, and scaffolding) are the core of cognitive apprenticeship, designed to help students acquire an integrated set of skills through processes of observation and guided practice. The next two (articulation and reflection) are methods designed to help students both to focus their observations of expert problem solving and to gain conscious access to (and control of) their own problem-solving strategies. The final method (exploration) is aimed at encouraging learner autonomy, not only in carrying out expert problem-solving processes but also in defining or formulating the problems to be solved."

Go read this one!

Lisa

(Photo credits:  Sorcerer's hat from Loren Javier, http://www.flickr.com/photos/lorenjavier/5382572927/ , Apprentice from Okinawa Soba, http://www.flickr.com/photos/24443965@N08/2585609947/)

How do you picture your learners?

Great post:  http://growchangelearn.blogspot.com/2011/03/picturing-learning.html

Lisa

When is Multiple Choice a good Choice?

I'm co-developing a workshop on the BB testing tool.  My role is to add the pedagogy to the technology.  While the tool has more to it than multiple choice questions I want to start there -  when is multiple choice a good choice?  

Step 1 for faculty:  Re-read the learning outcomes for your course.  Every course needs to ensure the learning outcomes are covered in the material.  They don't all have to be assessed, but many of them should be.

• Decision Point 1:  Which outcomes need to be assessed?
• Decision Point 2:  Which outcomes require a graded assessment and which are more appropriate in a self-assessment model?

Now you have a list of the outcomes you want to assess.  It's a good idea at this point to think about time, both yours and your learner's.  I am unable to give up the discussion area - this is the place in the course where instructors facilitate the material and interact with learners most directly.  It's critical that you leave enough time in your schedule to actively manage and participate in discussions.  (Note also that discussions do require learners to write.  Discussions can be casual or formal; they can require straight-forward questions or simple comments or they can require essays and complex analysis.)

So, some of your time will be spent in the discussions.  Some will be spent grading.  Requiring essays or significant projects without allowing adequate time for the provision of extensive feedback is not a good idea, so keep that in mind as you design your assessments.   And some outcomes can be assessed with an automatically-graded multiple choice test.

When to use MC:

1. When your goal is to assess straight-forward vocabulary and recall-type requirements/learning outcomes.  Traditional stem/3 or 5 answers multiple questions can give your learners a chance to practice recalling facts and vocabulary questions, particularly for introductory, freshman-level courses.  These types of questions can be productively used both in a graded-assessment and in non-graded, self-assessment scenarios.  Self-assessment quizzes can encourage learners to read the text and other materials.
2. When asking learners to do scenario analysis.  Set the stage with a description of a scenario, then ask a series of MC questions about the scenario.
3. When learners are going to be working in groups.  It is possible to use question pools such that each learner in the group receives a different set of questions.  By the time a group has worked through each permutation of the exam they will understand the material.  Develop question pools by replacing key numbers or phrases in each question with different numbers of phrases.
4. When it is to the benefit of learners to take a quiz or exam more than once.  See question pools above.  This is especially valuable when learners have significant test anxiety or when they might not read at the level required by the quiz or test.
5. When it's appropriate to have a variety of assessment techniques in a course.  Again, particularly in courses where learners need a lot of encouragement or may be more likely to experience test/school anxiety it's a good idea to include as many self-assessment activities as possible.  The automatic grading feature of MC quizzes make them an invaluable tool for self-assessment.

Writing good MC questions:

From Faculty Focus, Creating Better Multiple-Choice Tests for Online Courses, by Patti Shank, PhD, CPT, May 2009

• Provide clear directions. Group questions with the same directions together.
• Include as much of the question as possible in the stem, and reduce wordiness of alternatives.
• Include words in the stem that would otherwise be repeated in each of the alternatives.
• Make sure language is precise, clear, and unambiguous. Include qualifiers as needed, but don’t add unnecessary information or irrelevant sources of difficulty.
• Avoid highly technical language or jargon unless technical knowledge and jargon are part of the assessment.
• Avoid negatives and these words: always, often, frequently, never, none, rarely, and infrequently. When a negative is used, it should be CAPITALIZED, underlined, or bolded to call attention to it.
• Don’t use double negatives or double-barreled questions (asking two things in one question). 

Other examples of what to do or not to do available here:  http://web.utk.edu/~mccay/apdm/mchoice/mc_b.htm.   

Photo Credit: http://www.flickr.com/photos/maistora/3014414972/

ARE we "Academically Adrift?"

The recently published, somewhat controversial and apparently eye-opening book, Academically Adrift: Limited Learning on College Campuses, has already been featured in a number of blog and newsletter articles. Research for the book was supported by various well-known organizations, including the Lumina and Ford foundations, and the Carnegie Corporation.

This opening salvo, "Colleges and Universities, for all the benefits they bring, accomplish far less for their students than they should,"  a quote by Derek Box, former president of Harvard University, apparently sets the tone for "Adrift." At issue, at least on the surface, is a wide-spread conviction that undergraduate students leave college with less-than-stellar (or perhaps non-existent) critical and analytical thinking skills.

On the other hand, the book acknowledges early on that students surveyed in made comments such as, "I hate classes with a lot of reading that is tested on. Any class where a teacher is just gonna give us notes and a worksheet or something like that is better. . .Whereas, if I'm expected to read. . .and then write a three page essay on it. . .I'll probably do worse on that test because I probably wouldn't have read the book. . .I rarely actually do reading assignments or stuff like that, which is a mistake I'm sure, but it saves me a lot of time."

That being the case, does the fault lie with colleges, or with earlier educational organizations? And even if we believe students should come to college with better skills, what should/can we do to ensure that those students leave us with the skills and knowledge they need?


I've just started reading the book, and I'm sure as an educator I'll be by turn appalled and defensive, but it should be an interesting read based on the attention it is receiving. Stay tuned!

Thinking about little class details

Little things I am thinking about-

1) How many times can students take an exam?  My math classes are set to 2.  I just changed econ to 2.  Both classes use pooled questions even for the short answer, so students won't take the same exam the second time.  My students are scattered around for most of my online classes, so I don't think they are working together at all.  My thoughts on this might change if I were teaching at a traditional college with resident students. 

2)  What's the best schedule?  I like my math classes -- the unit exams are due on Thursday evening.  I can grade on Friday and everyone seems to take the weekend off.  I think that is nice for online students and faculty who seem to always be "on".  I might move the deadline up to 5:00 so I am around for any last minute troubleshooting.  

My econ classes have exams due on Sunday night -- I did that so they had the weekend, but since I tend to not be very accessible on Saturday and Sunday morning I'm not sure that's a great time.  Makes for a potentially stressful weekend.

3) Student workload:  I have heard from students in several classes lately about workload.  After listening to them I think it's not so much the amount of work as what work.  So, for example, I rarely hear from the PHI students who have to write 7 major essays and participate in discussions.  I think that is because the philosophy faculty have really worked on the essay topics and the students see exactly why they are doing what they are doing.  They also get lots of good feedback on their work.  When I get complaints it seems to be more because the students see an assignment as "busy work" and possibly the faculty do also because they aren't giving a lot of feedback on it.  If students perceive the instructor as working as hard as they are then they don't complain. :^)

Faculty Teaching Tips from Erin Beaver

These are great tips!

• Have students discuss what it takes to be successful in an online class at the start of the online class.  I like to do this the first week and then compile their answers into a slide show. 
• Take a survey at the start of the semester to find out who is in his/her first online class.  Then, I call those students personally after the first few weeks to check in.
• At the start of the semester, I always create a copy of the current semester schedule and re-save it with a new file name.  Then, as I move through the semester, I make adjustments to the schedule, add in notes to myself, etc., as I think about ways to improve the organization/design/etc. of the course. 
• Use a variety of modalities: video/audio, text, images, etc.
• Be consistent in language/terminology.  If I call a given document the “Style & Tone Handout,” I make sure that I do that throughout the shell and in notes to the class.

Lisa

Instructional Design Feed Bundle

This is a Google feed bundle for instructional designers posted in Twitter by Jadekaz, a designer from WI.  It's a great resource!  http://www.google.com/reader/bundle/user%2F16979012530158715558%2Fbundle%2FInstructional%20and%20Performance%20Technology

I love the article on Salman Khan on flipping the classroom.  I'm going to team teach a course for the remainder of the semester, me in the BB shell and someone else in the classroom, so that one was particularly relevant.  I also liked the post on juicy feedback - I'm looking at ore self-assessments and that makes a lot of sense to me.

Lisa

Where Do They Go?

It's a phenomenon that I have yet to comprehend. . .the sometimes gradual, sometimes quick, disappearance of students from online classes. You know the ones: they never say anything, they never ask for help, and they never let you know why they are leaving. They just disappear.

If we could actually identify the day that they left, we might be able to draw some conclusions. Was it right before (or after) a difficult assignment? Right after financial aid funds are disbursed?

I don't have the answers, and the faculty who teach for me are concerned that they have so few students, particularly in the 10-week session, who are participating. I tell them that "all they can do is all they can do," and our best efforts likely will not markedly alter this outcome. Why do they leave? What happens to them later on? Do they face repercussions for having just disappeared? One would think so, if they are receiving financial aid.

But where do they go?

Are We--Or Should We Be--Teaching Higher Order Thinking?

An article in today's Faculty Focus (http://www.facultyfocus.com/articles/teaching-professor-blog/the-facts-on-higher-order-thinking/) by our friend in distance education, Maryellen Weimer, poses the question, "Is there empirical evidence to support the frequent criticism that introductory courses are fact filled with little content that challenges higher order thinking?" Apparently someone studied 50 biology faculty and their courses and discovered that, "93% of 9,713 test questions were rated at level 1 or 2 of Bloom's taxonomy." In addition, "69% of the syllabi goals for those courses were at those same two levels."

My first reaction was, "Now wait a minute. Isn't that what those introductory courses are for?" And then I continued to read the article. Two questions in the article have me now pondering my courses: "Must students know the facts before they can think at higher levels? ". . .is knowing the facts all that's needed to think at higher levels?"

In accounting, for example, students do need to know a number of "facts" before they can successfully navigate higher-order tasks. If they don't know the definition of an asset or a liability, they cannot analyze the liquidity of a company to predict the company's ability to pay off its debts. I'm sure, however, that there are also a number of "facts" that are presented in my introductory accounting courses that student do indeed, as the article notes, forget as soon as the exam is over. . .and may never need to think about again.

I imagine that most disciplines are similar to mine: some facts do need to be presented and learned, and some could possibly never be introduced and never missed. Also, I imagine I could do more to foster higher-order thinking skills in my students instead of assuming that they just magically learned how to analyze, synthesize, and evaluate information somewhere else!

Now where DID I put those flashcards?

One-on-One Coaching Does Help

From The Chronicle:

A Close-Up Look at Student Coaching

March 9, 2011, 10:35 pm

By Eric Hoover

In recent years, various two- and four-year colleges have incorporated personalized coaches into their retention strategies. But how well does “success” coaching work?
A new independent study <by Eric Bettinger and Rachel Baker> finds that students who received coaching were considerably more likely to stay enrolled—and to graduate—than students who did not get such help. Moreover, the study found that one-on-one coaching was a cost-effective strategy......

The full report is available online at http://www.nber.org/papers/w16881. 

I'd like to see some research on how one-on-one coaching affects course completion rates for students enrolled in developmental math and English courses.

Lisa

Community College Blog from the Chronicle

The Chronicle has a new 2-year track blog.  http://chronicle.com/blogs/onhiring/category/the-two-year-track

Lisa

Balance

I never thought I would find myself at a place where I am encouraging instructors to add multiple choice and other automatically graded assessments to their classes.  Weird.  That comes from a discussion on balance and prioritization though.   I believe in the concept of opportunity cost (remember your first semester macroeconomics class).  When you choose to do something with your limited resources, in this case time, you are also choosing not to do something else with that time.  Because our time as an instructor each day is limited we need to prioritize our activities.  

I strongly believe (and the literature seems to support that belief) that active, teacher-led discussions help learners stay involved in a course and help them successfully complete courses.  But facilitating good discussions takes at least some time almost every day.  Responding to learners, gently leading the discussion through the course content takes though and effort.  

Written assignments are required to be a part of all guaranteed transfer courses in Colorado and most faculty seem to agree that they are an important part of most courses.  Assessment design is critical here also -- carefully thought out, scaffolded assignments get better results from learners than simply assigning a 5-10 page research paper.  And of course good feedback every step of the way is critical to the success of an assignment - again fairy time-consuming.  

This does leave room for assignments with automated feedback, especially in the non-graded, self-assessment area of the course, but also in the graded area.  Not everything learners are expected to end a course with involves critical thinking.  In many introductory courses learning the vocabulary is also important and one of the important outcomes.  That can be assessed well with a multiple choice exam with automated feedback, leaving the instructor with more time to focus on the discussions and written assessments.

I'm suggesting instructors think about the learning outcomes and how they are best addressed.  Sometimes that is better than drowning yourself if written assignments requiring significant grading time.

Lisa

(Photo credit: Photo taken in Oakley, CA by Mark. N. Woodcopf, Flickr, http://www.flickr.com/photos/cityofoakley/4625427080/)

I feel old. . .

. . .after reading this post: "http://janeknight.typepad.com/pick/2011/03/are-we-wired-for-mobile-learning-infographic.html". This post apparently appeared on Voxy Blog. The point is that "digital natives" are wired for mobile learning, and the post points out Mark Prensky's assertion that digital natives struggle with our current education system because they grew up playing video games and using other forms of technology.

Did you know that (according to this post), 93% of Americans age 12-29 are online? That 93% of adults age 18-29 and 75 of teenagers age 12-28 use cellphones? And that the the overhead project was first used in a classroom in 1930? I remember creating stacks of overhead slides to use in class. I carefully separate them with separator sheets, kept them in notebooks, wrote on them with markers that I could erase with a soggy paper towel. Sigh.

I remember taking an educational technology course in my undergraduate teaching program. One of the "high tech" techniques we learned about was to create multi-color "ditto masters" to run handouts for classes. Oh my. I also remember how those "copies" from the ditto machine felt and smelled. I miss that smell. Scary, isn't it?

Education and educational technology have come a long way. . .or have they? Are we any better off? Ah, that's for another day, another post.

More on OER--Like Toothbrushes?

From the "Open Content" blog (http://www.opencontent.org/blog/): An analogy comparing OER to toothbrushes?

Remember the campaign by one company (I think it was Oral B) to distribute thousands of toothbrushes? The blog author in a February 28 post likens the distribution of these toothbrushes to the distribution of OER, saying that just because they are distributed does not mean that they are being used. . .and that there may not be data to support the continued financing (or even the validity) of that distribution. He makes some valid points, and it's an interesting post, bringing to mind the line from Costner's Field of Dreams: "If you build it, he will come." (My all-time favorite movie, by the way.)

It made me think, though, about ALL the "bells and whistles" that we incorporate into teaching/learning, particularly in the online arena. We work hard to incorporate interactive learning tools, to provide opportunities for students to engage with each other and the instructor, to streamline the navigation process, and to develop and deliver information that we believe is critical to student success in our courses. I'm generally confident that at least some students do derive benefit from all this hard work!

However, when several students in a row ask a question about something covered in one of those meticulously prepared documents or complain about something not working right when, if only they had READ the instructions/watched the video/completed the practice work, they would not have encountered that roadblock, I sometimes become cynical and wonder why I spent all that time!

Is there value in all these materials and activities? Is there data to support the work we put into this, or would the students who succeed in our courses succeed regardless of the "extras" that we provide? Ah yes, that's what data analytics is all about, right?

Ok, I know that there IS data to support a great deal of what we are doing in our courses, and I believe that we're on the road to gathering even more data, but I also believe that there is a lesson to be learned. The fact that we make learning activities available isn't enough. We can provide students with a plethora of engaging, interactive learning tools. . . but if we don't direct them in how and when to use them, they are likely to be either overwhelmed by the sheer number of available activities (as in some of the publisher-prepared sites), or underwhelmed by the usefulness of the activities.

Timeliness is next to Godliness

It's not even quite mid-semester yet and I'm beginning to hear from frustrated students.  They've emailed their instructors 3,  *3*, times and haven't heard back.  They've called and left messages and haven't received a return call.  Days have gone by with no response - and of course now the assignment they have a problem with is due tomorrow.  

I always assume some exaggeration in these claims, really I do, but honestly a week is a long time in the life of a student.   Students are busy.  They have jobs, they have spouses, they have children.  They have things to do and commitments beyond school. They have specific times set aside to manage their homework and study and if they can't do it then they may not have another time to do it.  And we, as faculty with equally busy lives, need to be respectful of that.

That means you need to respond asap.  Even if the response is that you don't have an answer yet and that you will have to respond again later you still need to let them know that much right away.  Your student needs to know you received the message and that you are looking into it. The sooner you can give them a response the easier it is for them to rearrange their complex lives to meet the classroom requirements.  So don't delay.  Call or email right now!

Lisa

(Photo Credit:  sirspacepilot on Flickr, http://www.flickr.com/photos/sirspacepilot/5144846100/)

Bullet Points on Designing Assessments

If I were going to give faculty 5 basic tips on designing an assessment plan for a course they would be 

• Read the Assessment info on the CCCOnline Faculty Wiki (http://at.ccconline.org/faculty/wiki/Teaching_Resources_-_Assessment_Design)
• Consider:  What do you want to assess and why? Assessments should be aligned with the course learning outcomes.
• Not all assessments have to be graded - self-assessments are useful for learners also.
• Build flexibility into your assessment plan.  No learner should fail a course because of a single missed assessment.
• Assessments should not be added on the fly -- you need a plan for the entire course before the semester begins. Adult learners need consistency in due dates and the ability to organize and plan.

Lisa 
(Photo credit:  William & Mary Law Library Marshall-Wythe School of Law)