I usually don’t participate in social media, but since I worked with Grant, and since my and others’ research on authentic intellectual work since then launched further studies and professional development to implement the framework in US schools and abroad I’d like to respond to Grant Wiggin's blog on authentic assessment.
Origins of Concern for Authentic Assessment
Education reformers in the1980’s voiced many criticisms of common testing procedures (multiple choice, short answer, true-false). Most of the critiques boiled down to the objection that, because they are designed mainly for convenience of scoring, and in the case of standardized tests, in order to achieve a statistically normal distribution (where half the population must score below the mean) such tests measured only meaningless, trivial, contrived forms of mastery. Such tests failed to indicate students’ ability to master complex intellectual tasks, or to explain solutions to important problems, both within and beyond academic disciplines.
Beyond the nature of tests, other procedures were criticized. Grading was as unfair and invalid. As secret enterprise in which the teacher’s criteria were usually not available to the student or the public, a grade gave no indication of what kind of mastery had been demonstrated. Grading on the curve, rather than by demonstration of mastery, like standardized testing, denied half of the students the opportunity to succeed. Teachers making only “final” judgments about the quality of student work, without prior feedback and an opportunity for students to improve, had no instructional value.
Doug Archbald and I (Beyond standardized testing: Assessing authentic academic achievement in secondary schools. National Association of Secondary Principals, Reston, VA, 1988) summarized these concerns and presented examples of assessments that addressed many of them. We began by noting that traditional assessments, especially standardized tests, communicate very little about the quality or substance of students’ intellectual accomplishments. As mentioned above, the type of accomplishment measured is usually considered trivial, meaningless and contrived by students and adult authorities as well.
We realized before suggesting new assessments that addressed this criticism, we first needed to identify criteria that would identify worthwhile, significant, and meaningful intellectual accomplishment. But instead of beginning with abstract notions of intellectual quality, we considered examples of intellectual accomplishment by successful adults in diverse endeavors (e.g. scientists, historians, jurists, literary and artistic critics, journalists, physicians, designers, skilled technicians and trades people). From these examples, we tried to deduce characteristics that differentiated their intellectual work from most of the work students do in school. We proposed three main criteria: disciplined inquiry, integration of knowledge, and value beyond evaluation (production of discourse, things and performances that have utilitarian or aesthetic value apart from determining the competence of the learner). We considered achievements that met these three criteria “authentic” in the sense that they represented the kind of intellectual work done by adults that would be considered worthwhile, significant, and meaningful.
From this perspective there is no particular technique or set of techniques that constitute an authentic assessment. Instead, an authentic assessment can be any task or activity that requires, or makes demands, for authentic intellectual work.
Research and Development on AIW Since the 1980’s
Since the 1980’s I, along with many colleagues at the University of Wisconsin and beyond, published many research reports on instruction and assessment that promotes authentic intellectual work (while the research in Chicago that Grant mentioned was among the most significant, several other studies also offer important evidence on the power of AIW), and on professional development to help teachers teach toward AIW. Grant Wiggins cited one of the most recent publications. It is Newmann, King and Carmichael (2007), Authentic Instruction and Assessment: Common Standards for Rigor and Relevance in Teaching Academic Subjects. Des Moines, IA: Iowa Department of Education, available at http://centerforaiw.com/sites/centerforaiw.com/files/Authentic-Instruction-Assessment-BlueBook.pdf.
a) refines the criteria for authentic intellectual work into the following categories: Construction of Knowledge (organizing, interpreting, evaluating, or synthesizing prior knowledge, rather than retrieving or reproducing it); Disciplined Inquiry (having a prior knowledge base, in-depth conceptual understanding, elaborated communication), and Value Beyond School (solving problems that have utilitarian, aesthetic, or personal value beyond demonstrating success with school tasks);
b) summarizes research showing strong positive relationships between teachers who promote AIW and student achievement on measures of authentic intellectual work and on standardized tests as well (the most recent and comprehensive review of literature related to the AIW framework can be found in Saye, J. and Social Studies Inquiry Research Collaborative, 2013. Authentic pedagogy: Its presence in social studies classrooms and relationship to student performance on state-mandated tests. Theory and Research in Social Education, 41:1, 89-132);
c) provides examples of student work, teachers’ lessons and assessments that meet the criteria to varying degrees;
d) outlines a professional development program to implement the AIW framework.
Since 2007 the Center (www.centerforaiw.com) has worked with the Iowa Department of Education and districts elsewhere in more than 100 schools and 3000 teachers to implement the AIW framework. We are in the process of revising the 2007 publication to refine the presentation of the AIW framework and report in more detail on professional development to sustain such work. Contact the Center for more information.
Comparing Our Work to Wiggins
Grant Wiggins’ comments here that ‘authentic tests are representative challenges within a discipline” are consistent with our criterion of disciplined inquiry, but this represents only one of our three criteria. To be sure, it could be argued that representative challenges within a discipline must necessarily involve construction of knowledge. In this sense, it seems that Wiggins embraces at least two of our three criteria. Construction of knowledge, however, may not always represent a challenge within a discipline, as when teachers ask students to organize information in new ways not relevant to a specific discipline (“Write a story that illustrates possible problems in always getting what you wish for”).
We agree with Wiggins that “hands-on” or “real world” activities do not necessarily demand authentic intellectual work. Our third criterion “value beyond school” requires only that the intellectual challenge has utilitarian or aesthetic meaning beyond demonstration of competence demanded in school. Students in a classroom could be given dimensions of a bedroom, along with information about paint such as square feet covered per gallon and price, then asked to calculate the cost of painting the room. The activity occurred in school and did not involve “hands-on” activity, but the problem occurs in the world beyond school and solving it has utilitarian meaning, beyond showing the teacher that the student knows certain mathematical definitions and applications.
The main difference that I notice between our work and Grant’s is that our framework is more concise and, we think more useful to teachers. We focus exclusively on the nature of authentic intellectual work by adults wherever it occurs, and articulate just three main criteria for recognizing it. We deliberately do not suggest techniques for assessing it –only that whatever techniques are used make intellectual demands that call for construction of knowledge, through the use of disciplined inquiry, to produce discourse, products, or performances that have value beyond school. In contrast, Grant proposed an “unwieldy” (his words) set of about 27 criteria, dealing with multiple issues, that “might bear on authentic assessment.” Still, many educators have valued Grant’s work.
Though we chose different paths to authentic assessment, I respect his contribution and believe the conceptual compatibility between our perspectives is on solid ground.
It is January and -31 from where I’m writing. For most Americans, the season after New Year’s and before Valentine’s day is all about the Super Bowl, The Oscars and the Grammys. But in the land of AIW, we know better—this is the season for Mid-Year Institutes. These regional collaboration days are an important opportunity for practitioners of AIW reform from different schools to meet together and exchange ideas, score and “talk AIW.”
This tradition started in 2008 when the nine original pilot schools in Iowa Department of Education’s high school reform project. Originally, the Mid-Year Institute was a two-day event, with the first day dedicated to scoring artifacts from the field and the second day set up as a conference, with workshops targeting specific issues or practices in the field. Over time, the tradition has been condensed into a one-day event and expanded to several locations with distinct regional twists. Some Institutes include a keynote to kick off the day, while others are all about the workshops, and move participants through 90-minute blocks of time. The only common elements that remain include scoring with people form the same discipline who are not from your regular AIW team, (and preferably not from your school) and exchanging of ideas on how to make our practice better using the Framework for AIW and the scoring standards.
This AIW Mid-Year tradition is remarkable and part of why some regions retain vibrancy and freshness in the reform. Yet, sometimes the evaluations don’t reflect how amazing the event truly is. In short, some of us (yes, I would put myself at the top of this list) need reminders about which small stuff to sweat and what to let pass. So as I prepare for my own break out sessions and select the sessions I’ll be attending as a participant here is my own list of reminders:
Reminders as a Participant
Ask for what you need. When I’m in a session and the presenters are talking “at the audience” too long, it’s ok to politely ask for some time to discuss the ideas being presented with a partner or in small groups. I’ve found that more often than not, the presenters wrote that in their workshop lesson plan, but find themselves talking too long because they are nervous. A gentle nudge towards group discussion can be a welcome reminder to an anxious presenter.
- Be kind and prepare to disagree. AIW is at its best when cognitive dissonance flourishes. That said, there are times when passion becomes stubbornness; when a strong point of view turns a good thinker into a jerk. I find the best antidote comes from the three features that define substantive conversation, especially the second feature: “conversation involves the sharing of ideas....(and)...is not completely scripted or controlled by one party,” 1
- See the glass as half-full. Lastly, I try to always remember that we are all learners, including the presenter. Even if the workshop was less than I had hoped for, the evaluation is a place to give constructive feedback—not to scare the presenter away from ever doing a workshop again.
Notes to Self as a Presenter
- Don’t talk too much. I try to always remember that People are there to learn and the best learning on these days comes from the power of the collective, not from anyone person (including me). My job is to frame the learning objectives and create the space for others to learn from each other. My anchor question for this is Who is constructing the knowledge?
- Everyone wants to walk away with something of value. While this may be different for different people, I need to make sure that participants have a chance to think about what to take back to their classroom or school. This should be unique for each person, but common to the theme or topic of the workshop. My anchor question for this is What’s the Value Beyond the Workshop?
- Cite, acknowledge and thank. This is always tricky because in education we teach our students how to cite, yet as teachers we customarily “beg, borrow, and steal,” whatever we can. It’s a strange double standard. Having spent more of my own professional career in K-12 teaching than in academia, I struggle with this issue myself. Recently I’ve been thinking about it a lot more, especially when I read a recent blog by a famous educator who linked the what we in the AIW world fondly call the bluebook, without citing the source or even referencing its content correctly. As a relative novice in the world of citation, I passed this particular situation along to my more famous co-authors, Fred Newmann and Bruce King, to handle the response. As for myself, here are some resources and questions I’m currently looking at to expand my own understanding of how to do a better job.
- Who set me up for success in this workshop?Was there a teacher’s story, a video clip someone gave me, an idea offered that created breakthrough thinking for me as I put the workshop together? If there was, I can use the “hat tip” approach, common among journalists to acknowledge the source of a scoop, story or idea for their article. 2
- Did I modify, adapt, or recycle someone else’s stuff? This is always tricky. But here is the best rule to live by. If the person who originally created what I’m using sat in my workshop and would recognize it as theirs, I should have a footnote or acknowledgment thanking that person, or at least acknowledging the origins of the idea. The best source I’ve found for thinking about this are outlined in the general guidelines for sources and citations at Dathmouth 3. It’s a practical and understandable way to think the intellectual sharing of ideas and I highly recommend reading it.
- Is what I’m talking about “common knowledge?” The best example of this for an AIW conference or workshop is the language in the AIW framework. It is not necessary to cite when you use key words, such as “substantive conversation," "disciplined inquiry," or "elaborated communication.” We all know they are stadnards in the AIW framework. However, when we move outside our AIW family and do general presentations or publications, properly citing the work is important.
1 Newmann, F.M., King M.B. & Carmichael, D.L. (2009) Teaching for Authentic intellectual work: Standards and Scoring Criteria for Teachers’ Tasks, Student Performance, and Instruction. Minneapolis, MN: Tasora Books, (80).
2 The Wall Stree Jounral example of a hat tip is available at http://blogs.wsj.com/health/2008/04/21/univ-of-cincinatti-psychiatrist-under-more-scrutiny-over-funding/?mod=WSJBlog?mod=relevancy and came from a wikipedia defining hat-tipping at http://en.wikipedia.org/wiki/Hat_tip
Winter break in Minnesota has moved to a new level as we rung in 2014. No only is it colder than its been in decades, the break from school has been longer due to mandated cancellations by the governor, a right granted to him by the Minnesota legislature. So cloistered in with my family I 've had ample time to knit, bake, read, think, and write. I even took a stab at whimsically posting random thoughts on Facebook, Twitter and LinkedIn, none of which come naturally to me.
On one such posting I mentioned that I was thinking about my essential question for 2014. A fellow AIW and good friend picked up on my post and asked if I would share. So in the spirit of sharing, which AIW work is all about, here is a stab at elaborated communication about this eccentric past time of mine. Enjoy!
Every year I craft an essential question for myself that I explore through fiction, nonfiction, film, essay, and conversations with people. I started about ten years ago after I finished my PhD. I was sick and tired of reading for academic knowledge and couldn't bring myself to read for pleasure. Truthfully, I had grown up surrounded in a family of readers and never quite understood their penchant for staying inside curled up with a book when you could go out exploring!
But with two small children in tow and a non-native English-speaking husband, I felt obligated to at least pretend I liked reading for pleasure. As I got into the myriad of fables and storybooks I began to notice something about my own interest level: it increased the more connected the books were to each other. I wondered if I could trick myself into reading for pleasure more by changing my approach.
Could my love for ideas translate into an approach for reading?
The answer turned out to be YES! I discovered that if I could connect what I read to a theme, with an essential question guiding my choices, a new world opened up. To be honest, it has been amazing. My big line to people is that I'm in a book club for one, so I have to only be accountable to myself.
The truth is, I LOVE it and have developed a true passion for learning through my questions. Not only do I read essays, novels, and non-fiction, but I’ve also been able to fold in my passion for watching movies and listening to NPR. The net result is that I average investigating hundreds of artifacts each year, and just for my own personal pleasure.
If you look at the list below you'll see that for a long time, the questions seem to go back and forth between heavy and light themes. More recently, I’ve gotten more esoteric. But the cool part is that each year, the next question emerges from the previous year's readings and I never know what it’s going to be.
Here is what I have explored from most recent back to the beginning:
- 2013—How does knowing or not knowing your Purpose impact life? How do you if it's divine?
- 2012—What is power? Who has it?
- 2011—(last six months) What is honor? How do you get it? How do you keep it?
- 2011—(first six months) Random recommended readings!
- 2010—Which ideas change us and why?
- 2009—What was the common man's experience in Great Britain from 1066 to the Victorian era?
- 2008—What's the back-story to the British monarchy, especially in from 1500 to 1700?
- 2007—Who is the greatest female protagonist? Why?
- 2006—What is the inter-relationship among the religious, spiritual, metaphysical and historical schools of thought?
- 2005—How do people experience a magical world?
- 2004—What does it mean to read for fun?
- 2003—Finished my PhD
I’m still deciding on 2014's question. My three contenders are:
- Why does place matter?
- Where does courage hide and is it so fickle?
- “Good Vibrations?” What is the science behind the Beach boy’s famed lyric?
But to be honest I'm not sure. And please don’t ask for the reading list. My only regret has been my greatest indulgence: NOT keeping good records of what I read, heard or saw.
Happy New Year!
Closet reader for life
Recently, I have been focusing on how to create essential questions. These five key points are so useful!. If you want some spubject specific examples, click on Onhand School’s Essential Questions Guide.
Here are the five specific features of a quality essential question identified in the guide:
- Core-focused: The learning objective poses the question. It is the essence of what students should examine and know in a course of study. The same question can be re-asked throughout a main subject (for example, Math), but with increasing levels of sophistication.
- Inquiry-based: The question is open-ended and resists an obvious simple or single right answer. It precludes a creative choice that transforms the search for knowledge.
- Reinforce Thinking Skills: Requires students to draw upon content knowledge, personal experience, and other information they have gathered to construct their own answers. It causes students to search for an answer using critical thinking (ultimately using Bloom’s higher order thinking).
- Interdisciplinary: They usually lend themselves to multidisciplinary investigations, requiring for example, that students apply the skills and perspectives of math and language arts to social studies or science.
- Engaging: Should be created to provoke and sustain student interest. Engaging questions are thought provoking, likely to produce interesting student questions, and take into consideration diverse interests and learning styles.
Superstar AIW Coach Katy Evenson sent this blog our way. It's a fun activity for a learner of any age. Here is the prompt:
Recently my leadership coach presented me with a challenge: write about what you have learned in your years of experience as a school leader that you bring to the new position you have begun this year. The task sparked my imagination as I remembered the young educator I was thirteen years ago when I began my first principalship and sixteen years ago when I began my first school administrative position. What is it I believed then, I wondered, and what is it I believe now?
Check out how the blogger responded. What about you?
The 2012-2013 academic year marked the beginning of district-wide AIW implementation in the Monticello Community School District. A recurring question throughout the district has been how to construct tasks with high levels of value beyond school. At the elementary level a fourth grade science teacher and a language arts teacher teamed up to create a task that would meet this challenge. They never imagined this task would end up influencing audiences outside the community of Monticello.
The teachers identified the concepts of systems and producing text by narrowing them into a persuasive writing project about human body systems. The students had to construct knowledge about human body systems and how they are affected by healthy and unhealthy choices. The teachers facilitated students as they took the role of being scientists needing to support their findings with their writing. Through elaborated communication, students created final products to educate and persuade younger students and the public of the importance of healthy lifestyles. Their products were displayed in health service related businesses throughout Monticello and surrounding communities. Some made it as far away as the University of Iowa Hospital and Clinics.
Students used concepts from science and language arts in a practical way to influence audiences out of their classroom. This was one example in which teachers found a way to meet the challenge of implementing value beyond school with their students.
Just thought I'd let you know I had an ah hah! moment today. The kids are reading a historical fiction novel then writing a newspaper focusing on one particular battle of the Civil War from the novel. I was afraid of it turning into a "fluff" project because they would probably just rewrite whatever articles they happened to read, until I started thinking what I really wanted out of them relating to AIW. Rather than a simple report they should collect facts, interpret what happened based on the facts collected, make an analysis of the two sides (what advantages/disadvantages), make an evaluation as to why one side was able to win (draw a conclusion), then synthesize (create an article about the battle that is completely new) an article.
Check out Sarah Brown Wessling's Mid-Year keynote from the 2013 Iowa AIW Mid-Year Institute, sponsored by Heartland Area Education Agency (AEA 11). This event was hosted by Gilbert Community School District in February 2013.
Recently, Greg Anrig, an important voice in American public policy mentioned our AIW work in Iowa in his blog, Tearing Down Classroom Walls. I posted the response below and would love to hear from any teachers and administrators at AIW schools. Does this account jibe with your experience? Please don’t hesitate to post a comment here or on Anrig’s blog, or even on our Center FaceBook site. Let’s use social media to share more perspectives on how to transform American schools!
This blog is a great start for a larger conversation about how American education evolves in the 21st century. As you've pointed out, the work in Iowa included teachers working with one another to analyze the quality of their assignments and instruction against the standards for Authentic Intellectual Work (AIW) in teams with 4 to 6 members. While we did not require schools to create interdisciplinary teams, we did find that the departmental silos typical in high school settings dissolved more quickly and staff from all disciplines came to know, understand, and respect one another's work. This shift seems to help the learning climate.
For us, the key ingredient to impacting student learning comes through de-privatizing the classroom regardless of an AIW team's membership. Teachers involved in AIW reform come to see the responsbility for change lies with them and that they are up to the challenge of improving the quality of what they offer students. They collaborate about their pedagogy and revise lessons and assignments rather than point out student shortcomings. Subsequently, student engagement increases along with their performance in school.
When administrators engage in the reform work right along with their teachers there is another striking impact on climate.. The vertical hierarchy's intensity appears to soften. Trust is born when everyone repeatedly shares ineffective lessons and works on the revisions together. When students are successful, the team celebrates. The shared mission of transforming students' lives generates energy and fosters civic discourse, intellectual risk taking, and yes, higher standardized test scores. Ironically, for the schools truly involved in AIW, high test scores become footnotes to the far loftier goal of fostering young scholarship and civic responsibility for tomorrow's world.
When working with teachers new to AIW, there is often a moment when a few eager teachers get overwhelmed by the thought of revising their entire curriculum. At that point, most coaches refer to a quote on the bottom of page 72 in the “Blue book,” Authentic Instruction and Assessment: Common Standards for Rigor and Relevance in Teaching Academic Subjects, where teachers are encouraged to move the percentage of high scoring lessons from 10-15% up to say 50%; nowhere do we advocate 100%.
The truth is that there are times when it is not only appropriate, but also necessary for students to practice what they’ve learned. For both the student and the teacher, the key is knowing when you are doing it, why you are doing it, and how long is long enough! Although I can’t supply a simple answer, I can offer a personal anecdote, where as the student, I was given just the right “drill” time.
In fall of 2010 I decided I would join my husband at doing a triathlon instead of just watching him. My first step was to join the Master’s Swim class at the YWCA. It seemed logical, so I showed up, excited to share my goals with the rest of the swimmers as we each introduced ourselves. Then into the pool we plunged to whip out our first 50 yards.
I’m not sure what I looked like—all I know is that the coach (Beth) very kindly pulled me aside and suggested I not return until, in short, I learned how to swim. She suggested I start with the basic stroke class that met on Sunday afternoons. I’m sure I nodded in agreement, bewildered and quite out of breath. Honestly, I was shocked and my pride was a little hurt, but I really had no alternative except to swallow that pride and begin a long journey to relearn how to swim.
Fast forward to this past fall 2012. I had in fact learned a ton and actually successfully completed an Olympic distance triathlon in August, which required I swim just under a mile before moving on to the other events. This had gone pretty well—mainly because I neither drowned nor quit the race. So I was again full of confidence as I showed up to give Master’s Swim class another try. This time I could swim 100 yards in under 2 minutes; yet, alas, again the coach pulled me aside. “You are doing really well,” Beth said, “but your right elbow is dropping and your six-beat kick is going to wear you out before you even get to biking and running. Why don’t you take some lessons?”
The second time of being redirected out of the “gifted” class back to a scaffolded and individualized learning opportunity did not prick my pride nearly as much the second time. I dropped out of Masters Swim and met with Beth privately. She was not only encouraging, but I could see she was really looking at the big picture of my swimming and felt confident that she could devise a series of drills that would break me of my motor-like kick and replace it with the slow, imperceptible swish-swish of a two-beat kick. I was psyched!
That is until I tried it. It felt more like I was immobile in the water waiting only to sink like a stone to the bottom of the pool. In fact, I couldn’t even do the drills without sheer panic. I tried a plastic noodle to keep my hips afloat, a pull buoy to prevent my legs from automatically flailing, and even a flotation belt old ladies use in water aerobics. Each time I needed them less. My only goal was to do the drills she gave me, week after week. There was no HOT needed on my part, just practice. And so I committed to this drudgery. She told me I could come back to Master’s Swim class when I could hold eight 100-yard sets at two minutes without reverting back to a six-beat kick. Miraculously, I finally achieved this my final swim of 2012.
So, you have probably guessed right — I did go back to Master’s Swim class, last Thursday. Not only did I stay, but I also led my lane in our workout: 1600 yards, 4 (4x100) at 2:30. Of course, I was in the slow lane, but who cares. It was amazing. You might think I credit my success to my own perseverance. Sure, that helped, but in truth, I credit it to my teacher. Beth’s knowledge of swimming and her willingness to break down what was wrong with my technique and dole out small chunks of learning for me to practice over short periods of time combined with her faith that I could do it is where the credit lies. In truth, my willingness to stick with it came from believing her when she said I would be faster and more efficient in the end. She was right!
The lesson I took away was a deep reminder that when remove away the thinking and give students drills, they better be short-lived, to the point, and yield a gain so sweet only the student knows!