Tuesday, March 18, 2014

JWA at RNF and CCCC in Indianapolis!

JWA will be at the Research Network Forum and CCCC in Indianapolis, March 19-22, 2014.

JWA will be at the Editors' Roundtable discussion on Wednesday, March 19, 2014 from 1:15-2:30 pm.

 If you would like to talk to someone from JWA about a potential project, you can reach Peggy O'Neill at poneill1 [at] loyola [dot] edu or you can contact Jessica Nastal-Dema at jlnastal [at] uwm [dot] edu.

See you there!

Wednesday, February 5, 2014

Review of _Building Writing Center Assessments that Matter_ by Ellen Schendel and William Macauley

Review of Building Writing Center Assessments that Matter by Ellen Schendel and William Macauley (2012). Utah State University Press.

ISBN 978-0-87421-816-9, paper $28.95; ISBN 978-0-87421-834-3 e-book $22.95

By Marc Scott, Shawnee State University

     Ellen Schendel and William Macauley’s 2012 book, Building Writing Center Assessments that Matter (Building), is a co-authored text featuring an introduction and coda by both authors, three chapters authored by Macauley, three by Schendel, a brief interchapter by Neal Lerner, and an afterward by Brian Huot and Nicole Caswell. Much of Building explores how important writing assessment scholarship can apply to writing center program assessment, and often uses specific examples from the authors’ experiences directing writing centers. Schendel and Macauley’s goal in writing Building is to provide Writing Center Directors (WCDs) new to program assessment with a text that speaks specifically to the unique needs and opportunities of writing center work. While the text is geared toward assisting WCDs navigate program assessment, Building also provides assessment scholars and practitioners with important ideas and concepts for program assessment, including how to frame assessment and how to think through methodological options.
     Those wishing to develop a culture of assessment at their institution can learn much from Schendel and Macauley’s text. Throughout Building, the authors use tutoring and writing processes as metaphors for assessment work. Just as writers gain invaluable insights by sharing their work with other writers, sharing assessment projects and data with peers only benefits writing assessment. Furthermore, in Writing Center scholarship and practice, tutors strive to help a writer establish a healthy writing process rather than just proofread or edit a text. When applied to writing assessment, a similar emphasis on process over product might help instructors and students engage in assessment as a reciprocal and recursive form of inquiry that improves the writer holistically, rather than a linear process with one correct approach for each context (p. xix). In addition, the assessment process—much like the writing process—benefits from careful attention to exigency, context, purpose, and audience. Using the recursion of writing processes and the context-sensitive nature of tutoring as metaphors for assessment may provide an accessible concept for colleagues reluctant to embrace assessment.
     Writing assessment practitioners can also benefit from Building’s discussion of assessment methodologies. Schendel describes how Writing Center Directors should work to connect a program assessment’s methodology with each specific project’s purpose, audience, and available data. In fact, Schendel provides a useful chart that describes different forms of data a WCD might collect and explains how the data might be collected and who might collaborate in such efforts (pp. 127-131). The design of a writing assessment—be it a placement exam, a portfolio program, or a classroom assessment technique—should take the assessment’s context and purpose into account at each stage of the process, not just in analyzing results.  Rather, a writing assessment should be sensitive to the context of the student and classroom. Neal Lerner’s brief interchapter helps WCDs understand how qualitative and quantitative assessment methodologies might impact assessment projects in writing centers, and his thoughts can also help persuade those reluctant to assess. He argues against “maintaining the status quo” and operating on only a “felt sense” of the work done in Writing Centers (p. 113). Classroom teachers and WPAs might also feel like they “know” their classrooms, but unless they can provide evidence through assessment for what they know, their claims will fail to persuade important stakeholders.

     Building, while effectively tailored to the needs of WCDs, provides assessment scholars and practitioners with useful metaphors for discussing assessment and a thoughtful discussion of assessment methodologies. The bulk of the text provides important information for those interested in programmatic assessment, but it does so by thoughtfully weaving together assessment scholarship in a way relevant to writing centers.

Friday, January 31, 2014

JWA at WRAB 2014 in Paris, France

The Journal of Writing Assessment will be at the upcoming Writing Research Across Borders conference in Paris, France, February 19-22, 2014.

If you will be there and would like to talk with Diane Kelly-Riley, co-editor of JWA, please email her at dianek [at] uidaho [dot] edu.  We welcome WRAB presenters to adapt their writing assessment focused presentations for publication consideration to the Journal of Writing Assessment.  Presenters can find JWA submission information here.

Monday, September 2, 2013

Review of _Teaching the New Writing: Technology, Change, and Assessment in the 21st- Century Classroom_, by Anne Herrington, Kevin Hodgson, and Charles Moran, Editors.

Review of Teaching the New Writing: Technology, Change, and Assessment in the 21st- Century Classroom, by Anne Herrington, Kevin Hodgson, and Charles Moran, Editors.

Herrington, A., Hodgson, K., & Moran, C. (Eds.) (2009) Teaching the New Writing:  Technology, Change and Assessment in the 21st Century Classroom.  New York:  Teachers College Press.

By Susan Garza, Texas A&M University-Corpus Christi

Although this text was published in 2009, in 2013 as I write this review for the Journal of Writing Assessment, the use of technology continues to grow in the field of writing, and assessment seems on its way to becoming the overriding focus at all levels of education. The use of machine-scoring for writing assessment and the debate surrounding this issue is just one example of the current issues in assessment. Herrington and Moran begin the book by pointing out that while technology has propelled us forward with teaching and learning possibilities, we are being driven backwards as the drive for machine scoring unfortunately seems to be working up steam. In the last chapter, all three editors reflect on the other chapters in the book stating that their goal for the collection was to see “teachers working creatively with an expanded sense of what writing is becoming as it has accommodated emerging technologies,” and to develop an “expand[ed] sense of what the word writing might include in this new century” (198). Each chapter does have a section on assessment, although most are short and several read as if they were tacked on without much development. However, many useful rubrics/checklists that the teachers have developed in their attempts to align their activities with the curriculum standards within which they operate are included. This book could be used for a brief introduction for teachers beginning to learn about assessing writing, but it provides mostly assignment specific evaluation examples.

The remaining chapters were written by teachers at all levels from various part of the country and are divided into three sections. Most of the information is explanatory -- here's how to do an activity -- with an inspirational tone -- here’s how my students were changed by the experience.
Elementary and Middle School – In this section we learn about an elementary writing workshop for struggling readers that emphasizes the social nature of writing by having the students create web pages about topics of their choice (vampires); a fourth grade collaborative writing project where students create podcasts in order to gain a better understanding of revision; and sixth graders who create math- and science-based digital picture books to help students actively create information that ordinarily they would just read about.

Secondary Grades – In these examples, all from high school classes, we are shown how students participate in blogging for a New Journalism/Technology course by writing and sharing about their chosen topics; how students interpret poetry in the form of a video as part of the Poetry Fusion project; how seniors display their research paper findings (encompassing information from across the curriculum) in such formats as i-movies and Web pages; and how blogging and podcasting are used in a speech class.

College Years – In this final grouping, we see science writing made stronger using graphics and story boarding; students using Web 2.0 tools to create multimodal documents; and literacy narratives presented through hybrid essays using text and images.

The book is an easy read for teachers looking for innovative activities. One thing the book does well is illustrate how rich the experiences of using technology to teach writing can be. The teachers mentioned over and over that their roles change to becoming more of a coach or mentor than someone who presents information just for testing purposes. All of the chapter authors point to some way that the writing experience is deepened and made more meaningful through the use of technology, including:
• better understanding of process—making choices/revising/planning/delivering
• increased interest and ownership of writing
• more writing overall
• real world applications
• more social and interactive experiences.

In the final chapter, the editors situate the assessment discussion in the book under two categories: “Classroom Based Assessment,” where the need is for new criteria that relate more to composing with technology, which results in criteria from print culture being adapted and teachers recognizing the role of creativity in the use of a tool of choice; and “State Curriculum Standards and Standardized Assessments,” where the teachers were able to apply curriculum frameworks, but had more difficulty relating to what standardized testing can and seeks to measure. Authors Reed and Hicks, referring to work by Lankshear & Knobel (2006), provide one example of the difficulties faced: “Much of what we see in the enactment of this curriculum (as a result of the tests) does not engender the kind of changed mindset that a new literacies perspective requires: one that is characterized by openness, collaboration, collective intelligence, distributed authority, and social relations” (p. 136). In their chapter, Frost, Myatt, and Smith also talked about having to deal with “departmental outcomes that still refer to the number of pages students should write.” They felt a sense of “discomfort” when thinking about how to assess differently for “projects that couldn’t be word-count quantified” (p. 182). Many of the authors discuss how the use of technology is seen as an add-on in most standards lists, but the new literacies that students engage in, or assessment frameworks for these new literacies, aren’t included.

This collection presents a good starting point for understanding the need for new and different assessment as we experience the different writing experiences afforded with ever advancing technology possibilities.

Wednesday, April 24, 2013

Part III: Review of Norbert Elliot's and Les Perelman's (Eds.) _Writing Assessment in the 21st Century: Essays in Honor of Edward M. White_

Part III:  Review of Norbert Elliot's and Les Perelman's (Eds.)  Writing Assessment in the 21st Century:  Essays in Honor of Edward M. White

Elliot, N., & Perelman, L. (Eds.) (2012).  Writing Assessment in the 21st Century:  Essays in Honor of Edward M. White.  New York, NY:  Hampton Press. 

By Jessica Nastal, University of Wisconsin-Milwaukee

This is the third review in a series of five about Writing Assessment in the 21st Century:  Essays in Honor of Edward M. White, edited by Norbert Elliot and Les Perelman.  The collection is a “testament to White’s ability to work across disciplinary boundaries” as it includes contributions from the writing studies (including the National Writing Project, writing centers, classroom instruction, and writing programs) and educational measurement communities (p. 2).  It is also a snapshot – or a series of snapshots, since it is over 500 pages – of contemporary interests in and concerns about writing assessment; an update on Writing Assessment: Politics, Policies,Practices (1996), edited by White, William Lutz, and Sandra Kamusikiri.

Each chapter in Part III, “Consequence in Contemporary Writing Assessment:  Impact as Arbiter,” drives toward the last sentence of the last chapter in the section, written by Liz Hamp-Lyons:  “You cannot build a sturdy house with only one brick” (p.395).  Elliot and Perelman highlight the section’s dedication to the question of agency, in Edward M. White’s words as the “rediscovery of the functioning human being behind the text” (qtd. p. 371).  I also see the authors in Part III as demonstrating their dedication to understanding the variety of methods and interpretations and social consequences of writing assessment. 

Elbow pauses in his “Good Enough Evaluation” and writes, “I seem to be on the brink of saying what any good postmodern theorist would say: there is no such thing as fairness; let’s stop pretending we can have it or even try for it” (p. 305).  He doesn’t cross that brink, of course, and the writers in this section discuss how writing assessment in the twenty-first century might strive for building sturdy houses with many bricks of various shapes and sizes.  

In Chapter 17, Peter Elbow urges teachers and administrators of writing to consider “good enough evaluation,” not as a way to get us off the hook of careful evaluation, but as a way to rediscover the human being both writing and reading the text.  In the spirit of White’s practical and realistic forty-year approach, Elbow reminds us that the “value of writing is necessarily value for readers”; and yes, this even means teachers of writing (p. 310). He concludes by explaining that using such evaluation could result in evaluation sessions with “no pretense at ‘training’ or ‘calibrating’ [readers] to make them ignore their own values” (p. 321). 

Elliot and Perelman have set up another interesting contrast in Part III:  While many readers will agree with Elbow (how can we not?!), we might have some questions about how this good enough evaluation works in practice, which Doug Baldwin helps to highlight.  How is it that the results become “more trustworthy” through this process (p. 319)?  What makes Directed Self Placement the “most elegant and easy” alternative to placement testing (p. 317; Royer and Gilles discuss the public and private implications of DSP in Chapter 20)?  What impact would multidimensional grading grids, instead of GPAs, have on reading student transcripts (pp. 316-317)?  Baldwin helps to ask how we can ensure the “technical quality” of Elbow’s ideal – though non-standardized – evaluations (p. 327). 

For Baldwin, fairness, a concept authors of this section are dedicated to, “refers to assessment procedures that measure the same thing for all test-takers regardless of their membership in an identified subgroup” (p. 328).  He uses the chapter to expose instances that might display “face fairness” – allowing students to choose their prompt, use a computer, or use a dictionary – but that might reveal deeper unfairness for students.  Baldwin’s conclusion provides guidance for those of us concerned about the state of writing and writing assessment in the twenty first century, our diverse populations of students, and our “concerns about superimposing one culture’s definition of ‘good writing’ onto another culture’” (p. 336).

Asao B. Inoue and Mya Poe (Chapter 19), Gita DasBender (Chapter 21), and Liz Hamp-Lyons (Chapter 22) continue probing questions of agency, fairness, and local contexts.  The “generation 1.5” students DasBender worked with were confident in their literacy skills, identified as being highly motivated, and expressed satisfaction with their writing courses.  On the surface, it seemed like the mainstream writing courses served them well; however, instructors believed students “struggled to succeed” in them (p. 376).  DasBender observed, “generation 1.5 students’ self-perceptions as reflected in their DSP literacy profile…is at odds with” the abilities they demonstrate in mainstream writing courses (p. 383). 

This conflict seems representative of some of the concerns about contemporary writing assessment in action.  What are programs to do when they employ theoretically sound, fair policies designed to enable student participation and responsibility (“asking them where they fit,” in Royer and Gilles’ words) but that seem to fail in the eyes of instructors or administrators?  DasBender, Elbow, Baldwin, Inoue, Poe, Royer, Gilles, and Hamp-Lyons remind us that while Writing Assessment in the 21st Century does much to situate writing assessment and Ed White’s role within it, we have more work to do on behalf of all our students – which Part IV:  “Toward a Valid Future” alludes to.

Monday, April 22, 2013

National Council of Teachers of English Position Statement on Machine Scoring

NCTE just released a statement about the use of automated essay scoring (AES) in writing assessment. The statement explains why AES shouldn’t be used for evaluating student writing, offers some alternatives, and includes an annotated bibliography of research of machine scoring of student writing. The bibliography is based on the JWA bibliography compiled by Haswell, Donnelly, Hester, O'Neill and Schendel, published in 2012.

What do you think of NCTE’s statement on machine scoring? How can it be useful? Does it go far enough? Is it solidly grounded in research? Let us know what you think.

Monday, March 4, 2013

JWA at RNF in Las Vegas!

Will you be attending the Research Network Forum prior to the 2013 CCCC Annual Convention?

If so, stop by the Journal of Writing Assessment's table during the Editor's Roundtable session from 1:15-2:15 PM on Wednesday, March 13, 2013!  For more information about the RNF, visit their website.

We would love to talk with you about potential projects!

See you there!