A policymaker's primer on education research: How to understand, evaluate and use it. Retrieved April 25, 2007 from http://www.ecs.org/html/educationIssues/Research/primer/analyzingflowchart.asp.

Summary: The Analyzing Research Flowchart is an online tool that asks users a series of questions related to analyzing various components of a research study. The flowchart begins with questions to clarify the purpose and type of research conducted and then takes users through the process of data analysis covering topics such as rival explanations, applicability, bias, and peer review. Users can print the questions and their related responses.

Description: Tool

Capacities: Analysis, Professional Development, Skills and Knowledge, Tools

Type: Website

Allal, L., & Lopez, L. M. (2005). Formative assessment of learning: A review of publications in French. In Organisation for Economic Co-operation and Development (Ed.), Formative assessment: Improving learning in secondary classrooms. Centre for Educational Research and Innovation.

Summary: This literature review, based on publications about formative assessment by researchers in France, and in the French-speaking regions of Belgium, Canada, and Switzerland, draws on more than 100 articles published in the major French-language assessment journals, books, and related conferences. The authors focus on formative assessment of student learning in elementary and secondary schools, but also consider developments in teacher training and higher education that have influenced the conceptualization and practice of formative assessment in classrooms. The authors argue that the impact of French-language publications on the expansion of the conception of formative assessment is tempered by the lack of experimental and empirical work in this area.

Description: Guidance

Capacities: Multiple Measures

Type: Website

American Association of School Administrators. (2002). Using data to improve schools: What's working. Arlington, VA: KSA-Plus Communications. Retrieved December 1, 2006 from http://aasa.files.cms-plus.com/PDFs/Publications/UsingDataToImproveSchools.pdf

Summary: This is an easy-to-read guide to using data to drive school improvement. Education leaders and their staff can use this tool to build a districtwide culture of data-driven inquiry, and learn strategies to engage their communities in data-driven decision-making.

Description: Guidance

Capacities: Communication, Culture of Inquiry, Expectations for Data Use, Multi-level Data Use, Reciprocal Feedback, Stakeholder Collaboration

Type: Website

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

Summary: This guide addresses the professional and technical issues of test development and their use in education and psychology. It addresses legal and measurement trends affecting validity; testing individuals with disabilities or different linguistic backgrounds; and uses of new and old tests.

Description: Knowledge

Capacities: Data Quality Assurance, Multiple Measures, Policies, Standards

Type: Book

American School Board Journal. Retrieved April 8, 2006 from http://www.asbj.com

Summary: Website offers readers practical advice on data driven decision-making management and policy concerns.

Description: Tool

Capacities:

Type: Website

Anderson, J., Goertz, M. E., Goldwasser, M., Hovde, K., Massell, D., Mueller, J. A., et al. (2006). SchoolNet: A case study of implementation in three schools. Philadelphia, PA: Consortium for Policy Research in Education.

Summary:

Description:

Capacities:

Type: Website

Anderson, S., Fowler, D., & Klein, S., et al. (2005). Judging student achievement: Why getting the right data matters. Washington, DC: MPR/NCEA. Retrieved December 7, 2006 from http://www.dataqualitycampaign.org/files/Tools-Judging_Student_Achievement.pdf

Summary: This policy brief stresses the importance of using good data to develop data information management systems and make judgments about student and school performance. The authors discuss the benefits of longitudinal data and provide examples of how such data can be used to assess student academic growth and proficiency over time, monitor student mobility, retention and attrition, examine prior achievement for student subgroups, and predict future student achievement. The article concludes with information on the key elements of a comprehensive statewide data system.

Description: Guidance

Capacities: Data Integrity, Data Quality Assurance, Data Systems, Professional Development, Quality of Assessment Data, Skills and Knowledge

Type: Website

Armstrong, J., & Anthes, K. (2001). Identifying the factors, conditions and policies that support schools' use of data for decision-making and school improvement. Denver: Education Commission of the States. Retrieved April 8, 2006 from http://www.ecs.org/clearinghouse/30/69/3069.htm

Summary: This article briefly identifies the factors, conditions, and policies that support data-driven decision-making for school improvement. Drawing on a series of school and district interviews conducted in five states, the authors address key findings at the state, district, and school level. They provide preliminary recommendations for state policymakers, including: redesigning state accountability systems with the idea of distributing accountability throughout the system and providing ongoing assessments; creating "item banks" for districts to use for interim assessments; and having states provide districts with data-analysis software.

Description: Guidance

Capacities: Culture of Inquiry, Multi-level Data Use, Multiple Measures

Type: Website

Armstrong, J., Anthes, K. (2001). How data can help. American School Board Journal 188(11), 38-41.

Summary: This article highlights the key practices and attributes of six school districts in five different states that effectively used data to improve teaching and learning. All of the districts served low-income, transient and low-achievement populations of students and all were able to substantially improve student achievement through the use of data. Essential factors across these districts included: strong leadership; a supportive districtwide culture for using data; a strong service orientation toward principals and teachers; partnerships with universities, businesses and nonprofit organizations; mechanisms for supporting and training personnel to use data; close monitoring of students' performance on academic standards; and a well-defined, data-drive school improvement process.

Description: Knowledge

Capacities: Decision-Making, Leadership, Professional Development, Skills and Knowledge, Standards

Type: PDF

Assessment in Education: Principles, Policy and Practice (1998, March). Special issue on assessment and classroom learning 5(1).

Summary: This special issue of "Assessment in Education: Principles, Policy and Practice", focuses on assessment practices and classroom learning. This special issue covers various topics related to assessment strategies in the classroom including student self-assessment, formative assessment practices, the role of summative assessment, a deductive approach to assessment and classroom learning, and theory related to formative assessment. All articles are written by experts in the field. (Publisher did not grant permission to post articles. Link to publisher's website is provided.)

Description: Guidance

Capacities: Balanced Assess't System, Multiple Measures

Type: Website

Association for Supervision and Curriculum Development. (2003, February). Using data to improve student achievement. Educational Leadership, 60(5). Retrieved April 9, 2006 from http://www.ascd.org/portal/site/ascd/menuitem.459dee008f99653fb85516f762108a0c/

Summary: This issue addresses how teachers and schools use data to make instructional changes and to assess the effectiveness of school programs. Article topics include classroom assessments to improve student learning, a reader's guide to scientifically based research, how to conduct data collection and analysis, the effects of testing on student motivation and learning, and data warehousing and information management systems.

Description: Guidance

Capacities: Action for Learning, Analysis, Balanced Assess't System, Culture of Inquiry, Data Integrity, Data Quality Assurance, Data Systems, Decision-Making, Expectations for Data Use, Infrastructure, Leadership, Policies, Priorities, Professional Expertise, Quality of Assessment Data, Reporting, Skills and Knowledge, Standards, Tools, Transformation Plan

Type: Website

Bagley, S.S. (2008). High school students' perceptions of narrative evaluations as summative assessment. American Secondary Education, 36(3), 15-32.

Summary: This study examines students' and teachers' ideas and perceptions of narrative evaluations compared to summative assessments. Data were collected at a high school in Southern California that exclusively relies on narrative evaluations (i.e., detailed written feedback on assignments) rather than letter grades or marks. The researcher collected survey data from 9th-12th grade students (N = 115), and conducted individual interviews with a subset of students and teachers to determine what they liked most and least about narrative evaluations. They also asked students to discuss whether receiving the narrative evaluations on assignments was more or less stressful than receiving letter grades. Results indicated that narrative evaluations were more stressful to students because of the intensity of detail provided, teacher subjectivity, and the daunting task of future revision after receiving such detailed feedback. Students, however, also reported that narrative evaluations were more useful than letter grades because they provided clear guidelines for improving their work. Teachers expressed that while providing narrative feedback was very time-consuming, it also gave them an opportunity to provide personalized feedback to students. Bagley concludes that there are pros and cons to using narrative evaluations as summative assessments in high schools, yet these detailed evaluations ultimately become an important source of information to help students improve performance in school. (Permission to post article is pending.)

Description: Research

Capacities: Balanced Assess't System, Reciprocal Feedback

Type: Website

Bailey, A.L., & Heritage, M. (2008). Formative assessment for literacy grades K-6: Building reading and academic language skills across the curriculum. Thousand Oaks, CA: Corwin Press.

Summary:

Description: Knowledge

Capacities: Capacity Building, Culture of Collaboration, Culture of Inquiry, Multiple Measures, Professional Development

Type: Book

Baker, E.L, & Linn, R.L. (2002). Validity issues for accountabilty systems. (CSE Tech. Rep. No. 585). Los Angeles, CA: Center for the Study of Evaluation and National Center for Student Testing. Retrieved October 27, 2008 from http://www.cse.ucla.edu/products/summary.asp?report=585

Summary: In this report, the authors analyze validity issues that arise in the context of educational accountability practices. First, the authors review the theory of action in educational research and the conditions necessary for research to improve practice (e.g., results are accurate and interpreted correctly, selected action is effectively implemented, etc.). Second, they present the ways in which testing and assessment relate to accountability and discuss the importance of validity based on the purpose of a given test and population being tested. Third, the authors present numerous strategies, guided by the Standards for Educational and Psychological Testing, to improve educational accountability systems. The authors make suggestions about validity and testing standards, appropriate use of testing in diverse student populations, stakes associated with testing for teachers and students, effective ways to report results, and evaluation to assess the effectiveness of accountability programs.

Description: Guidance

Capacities: Data Integrity, Expectations for Data Use, Professional Expertise

Type: Website

Bandura, A., Schunk, D.H. (1981). Cultivating competence, self-efficacy, and in- trinsic interest through proximal self-motivation. Journal of Personality and Social Psychology, 41(3), 586-598.

Summary: In this article, the authors test the hypothesis that self-motivation through proximal goal-setting is an effective way to develop competencies, self-perceptions of efficacy, and inherent interest. The researchers matched 40 elementary school-aged children with deficits and disinterest in mathematics to learning under various conditions of goal-setting. Results from this study support the benefits of proximal goal-setting: increases in self-directed learning, efficacy, and attention to topics that were initially of little interest, mastery of subject knowledge, and accurate self-knowledge of abilities. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Multiple Measures

Type: Website

Bangert-Drowns, R., Kulik, J., & Morgan, M. (1991). The instructional effect of feedback in test-like events. Review of Educational Research, 61, 213-238.

Summary: In this meta-analysis, the authors examine how the nature of feedback, the timing of feedback, and the type of instruction associated with feedback impact student achievement. Among other findings, the authors conclude that feedback does not always increase student achievement, immediate feedback and feedback that is more than "correct-incorrect" is associated with larger gains, and feedback is most effective at increasing achievement when students are mindful of the purpose and process of feedback on learning. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Multiple Measures

Type: Website

Barnes, F. D. (2004).Inquiry and action: Making school improvement part of daily practice. Annenberg Institute for School Reform, Brown University.

Summary: A comprehensive school-improvement guide with supplemental worksheet templates, this resource provides an overview of school improvement and a framework for using a cycle of inquiry and action, as well as descriptions of how schools have put a self-study cycle into practice by collecting, analyzing, and acting on information.

Description: Tool

Capacities: Analysis, Decision-Making, Multi-level Data Use, Multiple Measures, Professional Development, Skills and Knowledge, Tools

Type: PDF

Bell, B. & Cowie, B. (2000). The characteristics of formative assessment in science education. New York, NY: John Wiley.

Summary:

Description:

Capacities: Multiple Measures

Type: Book

Bell, B. & Cowie, B. (2001). The characteristics of formative assessment in science education. Science Education, 85, 536-553. Retrieved on April 30, 2008 from: http://www.d.umn.edu/~bmunson/Courses/Educ5560/readings/BellSciEd01-85-536.pdf

Summary: This article presents the findings from a two-year research study exploring the formative assessment practices of science teachers in New Zealand. The authors begin by discussing three trends in education that have drawn attention to the significance of formative assessment: continuous summative assessment, multiple purposes for assessment and teaching, and assessment for conceptual development. Next, the authors describe the process of formative assessment in the classroom, emphasizing the importance of student-teacher interactions and reciprocal feedback. This is followed by a description of the nine key characteristics of formative assessment (e.g., responsiveness, sources of evidence, contextualized nature of the process, etc.) identified by teachers involved in the study. The researchers conclude that student disclosure is essential to the effectiveness of formative assessment and maintain that making formative assessment strategies explicit to teachers wanting to improve practice is a valuable process. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Guidance

Capacities: Multiple Measures

Type: Website

Bennett, N., Wise, C., Woods, P., & Harvey, J. (2003). Distributed leadership (full report). Oxford, UK: National College for School Leadership.

Summary: This study provides an overview of findings from the distributed leadership literature, highlighting the key factors that must be considered in order to obtain a clear, evidence-based understanding of the concept and considering the implications for professional development and future research. The authors identify conceptual elements and variable features of distributed leadership in order to generate a clearer understanding of the concept. They also offer ideas for leadership preparation and the ongoing professional development of school leaders. Lastly, they identify three areas for future research on distributed leadership, noting the importance of identifying, documenting and analyzing those leadership practices that help create and maintain distributed leadership. Detailed discussions of each book and article reviewed for the study can be found in the appendix.

Description: Knowledge

Capacities: Leadership, Professional Development

Type: Website

Bernhardt, V. (1999). The school portfolio: A comprehensive framework for school improvement (2nd ed.). Larchmont, NY: Eye on Education.

Summary: Bernhardt explores the improvement of school performance through the use of school portfolios, that is, a self-assessment tool that exhibits a school’s goals, progress, achievements, and vision for future improvement. Chapter topics include the school portfolio, criteria for assessing school improvement, leadership, student achievement, quality planning, professional development, partnership development, continuous improvement and evaluation, and building individual school portfolios. Bernhardt uses examples of schools that have successfully implemented school portfolios to demonstrate how to develop a portfolio tailored to a school’s continuous improvement efforts. Appendices include various tools including improvement continuums, sample staff-developed rubrics, an outline of the continuous improvement process, and sample questionnaires.

Description: Guidance

Capacities: Professional Development, Tools

Type: Book

Bernhardt, V. (2006). Using data to improve student learning in school districts. Larchmont, NY: Eye on Education, Inc.

Summary: The purpose of this book is to help both teachers and administrators better understand and make use of data collected by their schools including student achievement test scores and other qualitative/ quantitative measures. The author highlights why data use is so important in an era of high-stakes testing and accountability, what data should be collected, the basic steps in carrying out data analysis, examples of what analyzed data should look like, and how to use these data to inform a continuous improvement planning model. Examples of these methods are used to illustrate the process of data use and analysis within the context of a particular school district. The book also includes an accompanying CD providing other examples of data use within school districts and the tools to implement these strategies in a variety of school settings. The book is designed for all education professionals interested in using data to improve curriculum and instruction.

Description: Guidance

Capacities: Analysis, Culture of Inquiry, Multi-level Data Use

Type: Book

Bernhardt, V. L. (2003). No schools left behind. Educational Leadership, 60(5), pp. 26-30.

Summary: This article provides information on the kinds of data that are important for continuous school improvement and how educators can best access and analyze these data. The author discusses the four types of data that are useful for assessing school effectiveness; how to organize data through data snapshots and intersecting data categories; and the technical support needed in order to effectively and accurately manage and analyze data.

Description: Guidance

Capacities: Analysis, Data Integrity, Data Systems, Professional Development, Quality of Assessment Data

Type: Website

Bernhardt, V. L. (2003). Using data to improve student learning in elementary schools. Larchmont, NY: Eye on Education, Inc.

Summary: This book provides information on how to gather, analyze, and use information to improve schools. The author presents practical tools to help educators make better decisions based on data and includes examples on data collection from schools K-12.

Description: Guidance

Capacities: Professional Development, Skills and Knowledge, Tools

Type: Book

Bernhardt, V. L. (2004). Data analysis for comprehensive schoolwide improvement. Larchmont, NY: Eye on Education.

Summary: Manual details how schools can use data to enhance the school improvement process.

Description: Tool

Capacities: Culture of Inquiry, Reporting, Stakeholder Collaboration

Type: Book

Bernhardt, V. L. (2007). Translating data into information to improve teaching and learning. Larchmont, NY: Eye on Education.

Summary: The purpose of this book is to help education professionals evaluate data elements and tools needed to support data-driven decision making at all levels. The author includes examples to demonstrate how actual school districts use data to make decisions about strategies to promote student educational gains. Topics include (but are not limited to) data discovery and mapping, selecting data tools, creating a culture to use data, reporting and using results, and managing the data warehouse. This book is designed for school and district administrators and teacher leaders who want to use data to improve teaching and learning, and university professors who teach future educators at the undergraduate and graduate levels.

Description: Guidance

Capacities: Decision-Making, Multi-level Data Use

Type: Book

Black, P. & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan 80(2), 139-144.

Summary: The authors argue that formative assessment is an essential component of classroom work and that its development can raise standards of student achievement. (Publisher granted permission to post abstract only. Link to article abstract is provided.)

Description: Knowledge

Capacities:

Type: Website

Black, P. (1999). Assessment, learning theories and testing systems. In P. Murphy (Ed.), Learners, Learning and Assessment. London: Open University Press.

Summary:

Description: Knowledge

Capacities: Multiple Measures

Type:

Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy and Practice, 5(1), 7-74.

Summary: In this review of the literature on formative assessment, the authors consider the impact of classroom formative assessment practices on student learning, the role of students in self-assessment, and teacher practice and the formative strategies used by teachers. Through a detailed and theoretical analysis of the nature of feedback, the authors provide a basis for discussion on the development of models for formative assessment and ways to improve teacher practice. (Publisher granted permission to post abstract only. Link to article abstract is provided.)

Description: Knowledge

Capacities: Multiple Measures

Type: Website

Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2003). Assessment for learning: Putting it into practice. New York, NY: Open University Press.

Summary: This book explores the benefits of assessment for learning and demonstrates how teachers and researchers can work together to implement formative assessment strategies to improve teaching and learning. Findings presented in this book are from the experiences of 36 teachers involved in two-year project designed to improve formative assessment practices in the classroom. The authors begin by explaining the larger project and review research evidence demonstrating the benefits of assessment for learning. In subsequent chapters, they describe the assessment practices that teachers found most effective as well as challenges and frustrations that arose during implementation. The authors include practical examples to demonstrate how to use these strategies and explain why these strategies are effective, while their study results provide further evidence that formative assessment practices lead to significant improvements in student achievement. This book is designed for teachers, administrators, or anyone else who hopes to raise standards in education by including assessment for learning in the classroom.

Description: Guidance

Capacities: Action for Learning, Culture of Collaboration, Multiple Measures, Professional Development

Type: Book

Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2004). Working inside the black box: Assessment for learning in the classroom. Phi Delta Kappan, 86(1), 9-21.

Summary: In this follow-up to their "Inside the Black-Box," the authors report on their efforts to gather evidence about how to improve classroom formative assessment. In order to carry out this work, the authors conducted a study of the implementation of formative assessment concepts in practice with 24 secondary school mathematics and science teachers in England. Through the use of improved questioning techniques, feedback focusing on how to improve rather than grading, involving students in peer-assessment and self-assessment, and using summative assessments in a formative way, teachers found that the motivation and attitudes of their students improved, and the students achieved higher scores on tests than their peers in the same schools. Changes brought about by improved assessment for learning also prompted teachers to increase their knowledge and use of learning theory and to develop assessment strategies appropriate for their subject matter. The authors conclude with suggestions for teachers to enhance student learning, including, rethinking lesson planning and redefining their roles and expectations. (Publisher granted permission to post abstract only. Link to article abstract is provided.)

Description: Knowledge

Capacities: Culture of Inquiry, Multiple Measures, Professional Development, Professional Expertise

Type: Website

Black, P., McCormick, R., James, M., & Pedder, D. (2006, June). Learning how to learn and assessment for learning: A theoretical inquiry. Research Papers in Education, 21(2), 119-132.

Summary: The purpose of this article is to examine the concept of learning how to learn and explore how assessment for learning (AFL), learning to learn (L2L) and learning how to learn (LHTL) are conceptually related. The authors begin by defining LHTL, distinguishing it from related or similar terms. Second, the authors explore how teachers and schools might give greater attention to students' ability to LHTL, using empirical data to support the notion that student understanding of learning practices leads to improvements in student learning and performance. Third, the authors explore the challenges associated with assessing LHTL, discussing theoretical and practical problems of determining whether students have learned how to learn. The authors conclude that in supporting student's ability to LHTL, educators should focus on practices that increase student autonomy in learning. (Publisher granted permission to post abstract only. Link to article abstract is provided.)

Description: Knowledge

Capacities: Action for Learning, Multiple Measures

Type: Website

Blank, R. K. & DEC Project Team (2004). Data on enacted curriculum study: Summary of Findings. Retrieved April 8, 2006 from http://www.ccsso.org/content/pdfs/DECStudy.pdf

Summary: This study considers the Data on Enacted Curriculum (DEC) model for improving math and science instruction in urban school districts and middle schools as a means to design professional development programs to help teachers analyze and use data to improve curriculum content and teaching practices.

Description: Knowledge

Capacities: Culture of Inquiry, Professional Development, Skills and Knowledge

Type: Website

Boston, C. (2002). The concept of formative assessment. Practical Assessment, Research & Evaluation, 8(9). Retrieved May 15, 2008 from http://pareonline.net/getvn.asp?v=8&n=9

Summary: This paper explores formative assessment practices and reviews related research findings. The author begins by explaining the concept of formative assessment, defined as "diagnostic use of assessment to provide feedback to teachers and students over the course of instruction" (p. 1). Next, the author describes the purpose and benefits of using formative assessment practices using key findings from related research. More specifically, the author presents evidence that formative assessment practices are particularly beneficial to students who struggle academically (e.g., Black & Wiliam, 1998), feedback about errors and improvement should be specific and encouraging (e.g., Bangert-Drowns, Kulick, & Morgan, 1991), and students who understand learning objectives and assessment criteria show greater improvement in performance than those who do not (Frederickson & White, 1997). This section is followed by examples of formative assessment, and the author highlights the importance of classroom discussion and informal classroom/individual observations as formative assessment to guide instruction. Boston offers specific strategies to involve all students (e.g., present several potential answers to a question and ask students to vote on correct response) and to assess students' understanding (e.g., ask students to summarize main ideas of a lesson or assigned reading) and describes how to use tests and portfolios in a formative way. The author concludes by recommending various resources including books, websites, and journal publications for teachers interested in implementing formative assessment practices. (Permission to post article is pending. Link to publisher's website is provided.)

Description: Research

Capacities: Capacity Building, Multiple Measures

Type: Website

Boudett, K. P. & Steele, J. L. (2007). Data wise in action: Stories of schools using data to improve teaching and learning. Cambridge, MA: Harvard Education Press.

Summary: A sequel to 2005's Data Wise: A Step-by-Step Guide to Using Assessment Results to Improve Teaching and Learning, this book provides concrete examples and suggestions for how to implement the Data Wise process of using assessment results to improve instruction and learning. Case studies of eight schools that have implemented the improvement process offer insight into the challenges faced and strategies used in rolling out the process. The examples demonstrate how these schools adapted the process to fit their particular needs and illustrate how the Data Wise culture can begin with one school leader and spread outward.

Description: Guidance

Capacities: Action for Learning, Culture of Collaboration, Culture of Inquiry, Leadership, Transformation Plan

Type: Book

Boudett, K. P., City, E. A., & Murnane, R. J. (Eds.). (2005). Data wise: A step-by-step guide to using assessment results to improve teaching and learning. Cambridge, MA: Harvard Education Publishing Group. 1-891792-67-9

Summary: This book provides several ideas and tools to assist school leaders in utilizing data to improve instruction and student achievement. It includes information and resources on the following topics: collaborative work, assessment literacy, data analysis, examination of instruction and student progress, and action plan development and implementation. Real-life examples are used throughout to illustrate the process, and chapters are written by experts in the field.

Description: Knowledge

Capacities: Culture of Inquiry, Expectations for Data Use, Professional Development, Tools, Transformation Plan

Type: Book

Boudett, K.P., Murnane, R., City, E., & Mood, L. (2005, May). Teaching educators: How to use student assessment data to improve instruction. Phi Delta Kappan 86(9), 700-706.

Summary: The authors describe a course at the Harvard Graduate School of Education in which students are placed on teams with public school teachers where they work together and use student data to solve real problems. The authors review the four main design elements of the course which are as follows: 1) organizing around a clear process; 2) teaching how to use tools for each cycle step; 3) assigning projects that use real school data; and 4) supporting collaborative work. Throughout the course, the instructors collected online surveys, group and individual assignments, notes from group meetings, observations during classes, and focus groups to examine the overall effectiveness of the experience. The authors found a positive change in both graduate students and public school teachers' attitudes towards data use, as participants developed the belief that using data could have a positive impact on teaching and learning and developed the skills needed to engage in this kind of work.

Description: Knowledge

Capacities: Culture of Collaboration, Professional Development, Professional Expertise

Type: Website

Breiter, A., & Light, D. (2006). Data for school improvement: Factors for designing effective information systems to support decision-making in schools. Educational Technology & Society, 9(3), 206-217. Retrieved December 7, 2006 from http://www.ifets.info/journals/9_3/18.pdf

Summary: Describes the process of data driven decision making in schools, and the potential of new technologies to meet the informational needs of educators at different levels of the system.

Description: Guidance

Capacities: Decision-Making, Infrastructure, Professional Development

Type: Website

Brookhart, S. M. (2005, April). Research on formative classroom assessment. Paper presented at the annual meeting of the American Educational Research Association, Montreal.

Summary: (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Research

Capacities: Multiple Measures

Type: Website

Brookhart, S. M. (2007). Expanding views about formative classroom assessment: A review of the literature. In J. H. McMillan (Ed.), Formative classroom assessment: Research, theory and practice. New York, NY: Teachers College Press.

Summary:

Description: Knowledge

Capacities: Multiple Measures

Type:

Brookhart, S. M. (2007). Feedback that fits. Educational Leadership, 64(4), 54-59.

Summary: The purpose of this article is to explore the effectiveness of teacher feedback. The author maintains that feedback must be clear, understandable, meaningful to students, and must focus on specific qualities of student work or on the strategies used to complete the work in order to effectively support student learning. The author includes effective ways to deliver feedback explaining when to give feedback, how much feedback to provide, and the best mode of feedback to maximize student learning and motivation. Brookhart also describes how to relate feedback back to learning goals, and maintains that feedback should be descriptive, without judgment, positive, and specific. The author also includes a vignette of fourth-grade classroom to illustrate effective and ineffective feedback strategies used by a fourth-grade teacher. Brookhart concludes that when providing feedback, teachers must always consider how students will hear, understand, and feel about the feedback. She suggests that with practice, teachers will develop a repertoire of effective feedback strategies to use to better support student learning.

Description: Guidance

Capacities: Professional Expertise, Reciprocal Feedback, Tools

Type: Website

Brookhart, S., Moss, C., & Long, B. (2008). Formative assessment that empowers. Educational Leadership, 66(3), 52-57.

Summary: In this article, the authors describe how formative assessment practices (i.e., regular feedback on how well students are learning) help maximize student learning opportunities in the classroom. The authors maintain that through formative assessment teachers communicate to students learning goals and ways to reach those goals while students show teachers their level of understanding. Through this mutual communication, teachers become more effective and students feel more empowered. The authors describe a district in rural Pennsylvania where teachers were encouraged to transition from using more traditional assessment strategies to formative assessment practices. The district established a partnership with the Center for Advancing the Study of Teaching and Learning in Pittsburgh where six teachers learned about formative assessment strategies. The initial group of teachers then spread the initiative across the district to 85 other teachers using the Teaching as Intentional Learning model which posits that teachers grow through intentional inquiry related to authentic questions that come up in the actual classroom. After implementing these strategies and increasing student-teacher communication, teachers found that students were more motivated, engaged in school, and felt a greater sense of self-efficacy in regards to their own learning. Furthermore, students demonstrated improved performance on state tests, as fewer children scored "Below Basic" at every grade level during the initiative. The authors also describe the teachers' transition from using traditional assessments to formative assessments, discussing the practices and strategies that helped to increase student-teacher communication and give students ownership over their own learning. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Communication, Reciprocal Feedback

Type: Website

Brookhart, S.M. (2003). Developing measurement theory for classroom assessment purposes and uses. Educational Measurement: Issues and Practice, 22(4), 5-1.

Summary: This article examines the degree to which conventions and assumptions of measurement theory apply to classroom assessment. The author argues that because the classroom context is fundamentally different than the large-scale testing context from which classic and modern test theory were derived, measurement theory specific to classroom assessment should be developed. Three contextual considerations are highlighted to emphasize the importance of developing classroom specific theory: the psychosocial context of classroom assessment, the integration of instruction and classroom assessment, and the formative nature of most classroom assessments. The author further supports these ideas by contrasting large-scale and classroom assessment concepts and makes suggestions for ways to develop measurement theory specific to the classroom context. (Permission to post article is pending. Link to publisher's website is provided.)

Description: knowledge

Capacities: Data Quality Assurance, Multi-level Data Use, Multiple Measures

Type: Website

Brown, G. T., Hirschfed, G. H. (2008, March). Students' conceptions of assessment: Links to outcomes. Assessment in Education: Principles, Policy & Practice, 15(1), 3-17.

Summary: In this study of New Zealand secondary school students, the researchers explored youths' ideas about the purpose of assessment and how those conceptions relate to educational outcomes. Conceptions of assessment were measured by students' self-report surveys and were categorized in four ways: 1) assessment holds students accountable; 2) assessment holds schools accountable; 3) assessment is fun; and 4) assessment is ignored. Educational achievement was measured using a standardized curriculum-based reading comprehension test. The researchers used path analysis and found that the conception that assessment holds students accountable was positively related to student achievement, while the conceptions that assessment holds schools accountable, assessment is fun, and assessment is ignored were negatively related to achievement. The authors' interpretation of these results is guided by self-regulation theory. They conclude that students who view assessment as a means of taking responsibility for their learning are more likely to be self-directed learners and will demonstrate higher academic achievement.

Description: Research

Capacities: Balanced Assess't System, Multiple Measures

Type: Website

Brown, S., Rust, C. and Gibbs, G. (1994) Involving students in the assessment process, in Strategies for Diversifying Assessments in Higher Education. Oxford: Oxford Centre for Staff Development, and at DeLiberation.

Summary:

Description: Guidance

Capacities: Multiple Measures

Type:

Brunner, C., Fasca, C., Heinze, J., Honey, M., Light, D., Mandinach, E. et al. (2005). Linking data and learning: The Grow Network study. Journal of Education for Students Placed At Risk, 10, 241-267.

Summary: (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description:

Capacities:

Type: Website

Brunner, C., Fasca, C., Heinze, J., Honey, M., Light, D., Mardinach, E., & Wexler, D.(2005). Linking data and learning: The grow network study. Journal of Education for Students Placed at Risk (JESPAR) 10(3), 241-267. Retrieved February 22, 2007 from http://www.leaonline.com/doi/pdf/10.1207/s15327671espr1003_2

Summary: This article describes findings from a 2-year exploratory study that examined how educators within the New York City public school system are using data-made available to teachers through the print- and Web-based reporting systems of the Grow Network-to inform decisions about teaching and learning. The authors found that data-driven decision-making requires that appropriate data be turned into information that can aid decision-making; technology plays an integral role in information sharing; teachers and administrators use data and data systems for different purposes; and there are limited applications of the particular product in question. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Analysis, Decision-Making, Infrastructure

Type: Website

Buckendahl, C. W., Plake, B. S., & Impara, J. C. (2004). A strategy for evaluating district developed assessments for state accountability. Educational Measurement: Issues and Practices, 23(2), 17-25. Retrieved January 15, 2007 from http://www.blackwell-synergy.com/doi/pdf/10.1111/j.1745-3992.2004.tb00155.x

Summary: This article reviews the evaluation strategy of an assessment and accountability system in Nebraska. While the method emphasizes the use of formative evaluation and promotion of improved assessment quality at the local level, it cannot be used to make refined comparisons of student performance across districts on assessments. Results from the first three years of the evaluation process suggest the need for increased specificity in the review criteria and more reviewer calibration activities. (Permission to post article is pending. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Analysis, Multiple Measures, Quality of Assessment Data, Reporting

Type: Website

Butler, R., & Neuman, O. (1995). Effects of task and ego achievement goals on help-seeking behaviors and attitudes. Journal of Educational Psychology, 87(2), 261-267.

Summary: The purpose of this study was to explore how achievement goals affected child help-seeking behaviors. Participants included 159 second and sixth grade Israeli children. Each child was randomly assigned into one of two conditions (ego-focus achievement or task-focus achievement) and then asked to complete six geometric puzzles of varying difficulty. Children in the ego-focus condition were given a prompt emphasizing the link between intelligence and the ability to solve the puzzles, while children in the task-focus group were told that the puzzle activity was designed to help children learn to solve puzzles. Both groups were told they could ask for help, hints, or the solution at any time. After comparing help-seeking behaviors of children in each condition, the researchers found that both second and sixth grade children in the task-focus group were more likely to request help, demonstrating that children were more likely to seek help when they saw the activity as a way to build competence rather than a way to measure ability. Children were also asked a series of questions about their help-seeking behaviors (e.g., can you tell me why you did not ask for help?) after they finished the task. Results indicated that children in the task-focus group were more likely to attribute their help avoidance to the desire for independent mastery, while children in the ego-focus condition were more likely to explain help avoidance to mask incompetence. Results demonstrate that achievement goals affect help-seeking behaviors and the child's perception of the relationship between help-seeking and competence. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Research

Capacities: Multiple Measures

Type: Website

California Comprehensive Center (June 2006). Data-driven decision making based on curriculum-embedded assessment: Findings from recent California studies. Retrieved from http://www.cacompcenter.org/pdf/cacc_dddm.pdf

Summary: This paper presents findings from three recent research studies examining effective data use in schools. The first study was designed to assess the effects of Proposition 227 on English language learners. The purpose of the second study was to evaluate the Immediate Intervention/ Underperforming Schools Program. (Both evaluations used interviews with administrators at schools characterized by high student performance despite at-risk student populations.) The third study included in this synthesis entailed a large scale survey of low-income elementary schools in California to determine what practices were most highly associated with student achievement. The present paper summarizes several key findings related to data-driven decision making and highlights the role of the administrators, teachers, parents, and students for effective data use. Results demonstrated that educators who report using data describe using the information in numerous ways (e.g., to gauge student understanding, to identify areas for improvement, to form instructional strategies, to create specific strategies for needy students, etc.). Principals emphasized the importance of training and supporting teachers in data use and management through modeling and providing opportunities for staff to practice using assessment data to guide decision-making. The amount of training teachers reported receiving varied considerably, yet teachers at most schools reported discussing data use regularly at various staff events (e.g., grade level or staff meetings). Furthermore, teachers reported engaging in numerous activities to monitor student progress (e.g., modifying instruction, linking instruction to standards) and then using these data to communicate with both students and parents. Overall, principals and teachers in all three studies emphasized the importance of systematic monitoring and data-driven decision making in promoting student achievement gains.

Description: Research

Capacities: Decision-Making, Professional Expertise

Type: Website

Cameron, J., & Pierce, W.D. (1994, Autumn). Reinforcement, reward, and intrinsic motivation: A meta-analysis. Review of Educational Research, 64(3), 363-423.

Summary: In this article, the authors present a meta-analysis of the research on the impact of reinforcement/reward on intrinsic motivation. The authors examined the overall effect of reward on intrinsic motivation and the effects of specific features of reward (e.g., reward type, reward expectancy, reward contingency) on intrinsic motivation. The authors reviewed 96 between-groups experimental studies to compare rewarded versus non-rewarded subjects and found that rewards did not diminish intrinsic motivation. Interaction effects further reveal that verbal praise actually led to increases in intrinsic motivation. A small decrease in intrinsic motivation was observed when individuals receive expected, tangible rewards for only completing a task, yet this effect was small. The authors conclude that rewards/reinforcement do not decrease intrinsic motivation. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Research

Capacities: Multiple Measures

Type: Website

Carless, D. (2007). Conceptualizing pre-emptive formative assessment. Assessment in Education: Principles, Policy & Practice, 14(2), 171-184.

Summary: The purpose of this article is to explore formative assessment, focusing on "pre-emptive formative assessment", meaning teacher actions that attempt to clarify student understanding before their misunderstandings result in inadequate learning or lower grades. The author argues that teachers frequently use these pre-emptive strategies in the classroom, yet the practice has not been well conceptualized in the extant literature. Carless maintains that pre-emptive formative assessment is particularly useful given that it addresses the issue of timely feedback and gives students an opportunity to improve their understanding and performance. The author also discusses strategies to use when implementing pre-emptive formative assessment (e.g., practicing a sub-skill) and provides an example from his own work in teaching college students to learn how to properly use citations and references. Carless then presents limitations of pre-emptive formative assessment including overdependence on the teacher, large variability in assessment knowledge among educators, and the time-consuming nature of these strategies. Overall, however, the author maintains that pre-emptive formative assessment is a particularly useful formative assessment technique given that it addresses the gap in student knowledge or understanding early on so that students have a chance to act on feedback to improve their work. Carless concludes with suggestions for future research. (Permission to post article is pending. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Action for Learning, Reciprocal Feedback

Type: Website

Casserly, M. (2006). Beating the odds: A city-by-city analysis of student performance and achievement gaps on state assessments. Results from the 2004-2005 school year. Washington, DC: Council of Great City Schools.

Summary: This is a useful example of how to examine assessment data at the district and state levels, make comparisons between different groups and across time, and report data on student achievement in-line with the No Child Left Behind Act. Prepared by the Council of the Great City Schools, this report provides detailed information on how major city school systems are performing on state assessments. Data from 65 city school systems in 38 states are presented. The data are broken out by city, year, and grade level for each state test in mathematics and reading. Where the information is available, the data are also reported by race, language, disability and income. In addition to examining student achievement in math and reading over time, this analysis also measures achievement gaps between cities and states and minority populations and whites.

Description: Knowledge

Capacities: Analysis, Professional Development, Skills and Knowledge

Type: Website

Cawelti, G., & Protheroe, N. (2001). High student achievement: How six school districts changed into high-performance systems. Arlington, VA: Educational Research Service.

Summary: Profile of six school districts in Texas, Idaho, and West Virginia that have achieved increased standardized student test scores. A central finding for test score gains is that instruction must align with test content, student assessment responses should be analyzed to ensure coherence, and corrective instruction should be available for students who struggle on assessments.

Description: Knowledge

Capacities: Culture of Inquiry, Expectations for Data Use

Type: Website

Celio, M. B. & Harvey, J. (2005). Buried treasure: Developing a management guide from mountains of school data. University of Washington. Seattle, WA.

Summary: This report provides useful information on developing a school district management guide as well as an actual guide focused on seven evidence-based indicators: achievement, elimination of the achievement gap, student attraction to the school, student engagement with the school, student retention and completion, teacher attraction and retention, and funding equity. The report also includes several implications from the analyses of the role of indicators in the management system described. The authors conclude that indicator development encourages educators to think of new ways to assess accountability and to move beyond bottom-line assessment systems.

Description: Guidance

Capacities: Data Systems, Leadership, Multi-level Data Use, Tools

Type: Website

Center on Education Policy (2007). Educational architects: Do state education agencies have the tools necessary to implement NCLB? Washington, DC: Author. Retrieved March 21, 2008 from http://www.ecs.org/clearinghouse/74/03/7403.pdf

Summary: This report by the Center on Education Policy (CEP) explores whether state education agencies have the capacity (e.g., funding, human resources, technology, etc.) to design and implement the accountability system mandated by the No Child Left Behind Act (NCLB). The researchers collected survey data from officials in all 50 states, conducted interviews with 15 high-ranking state education officials, and collected relevant policy documents to answer this question. They found that state education agencies face many challenges in implementing NCLB, including limitations in staffing and personnel, insufficient funding, and little support and guidance from the U.S. Department of Education. The report recommends that the reauthorized NCLB develop a grant program for states to reassess the role of state agencies in implementing NCLB and amend NCLB to allow differentiated intervention depending on the needs of the schools requiring assistance. They also argue that state agencies should receive additional funding to better implement NCLB and the U.S. Department of Education take a more active role in assisting states in matters regarding school improvement. Overall, they maintain that state education agencies play a key role in implementing federal policy yet they need adequate support to do so effectively.

Description: Research

Capacities: Infrastructure, Multi-level Data Use, Standards

Type: Website

Chan, L. (2004). Preventing problems: The best defense for your information system is good planning and continued care. Retrieved April 8, 2006 from http://www.asbj.com/specialreports/0904SpecialReports/S3.html

Summary: Article about how to plan and support a school management information system for data driven decision making.

Description: Guidance

Capacities: Data Systems, Tools

Type: PDF

Chappuis, J. (2005). Helping students understand assessment. Educational Leadership, 63(3), 39-43. Retrieved September 7, 2007 from http://www.ascd.org/authors/ed_lead/el200511_chappuis.html

Summary: This article identifies seven strategies teachers can use to increase student involvement in the formative assessment process. Research on effective formative assessment suggests that students should be able to answer three basic questions about their learning: Where am I going? Where am I now? and How can I close the gap? In line with these findings, the strategies described are intended to help students identify and better understand their learning goals, recognize their own skill in relation to these goals, and take action to reach these goals. The author contends that students can benefit greatly from formative assessment information and efforts should be made to expand the formative assessment process to systematically involve students in decisions about their learning. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Guidance

Capacities: Action for Learning, Decision-Making, Skills and Knowledge

Type: Website

Chappuis, S. & Chappuis, J. (2007). The best value in formative assessment. Educational Leadership, 65(4), 14-19.

Summary: This article clarifies misunderstandings about formative assessment and highlights the benefits of implementing formative assessment practices in the classroom. The authors discuss the differences between summative assessment (assessment of learning) and formative assessment (assessment for learning) and explain that formative feedback happens when there is still time for teachers to adapt instruction and for students to take steps to improve their performance. They also argue that state/ benchmark tests work better as summative assessments given that results are usually delivered months after administration and teachers may not know how to use results to guide instruction. The authors maintain that the value in formative assessment lies in classroom teachers using results to improve/adapt teaching practices to better support students, and students using results to understand their own learning areas that need to be improved. In sum, formative assessments give teachers an opportunity to better meet the needs of each student and to increase overall learning in school.

Description: Knowledge

Capacities: Multiple Measures

Type: Website

Charles A. Dana Center, University of Texas at Austin, Hope for Urban Education: A Study of Nine High-Performing, High-Poverty, Urban Elementary Schools (Washington, DC: U. S. Department of Education, Planning and Evaluation Service, 1999). Retrieved February 13, 2008 from http://www.ed.gov/pubs/urbanhope/index.html

Summary: This report documents the characteristics and strategies of nine urban elementary schools with high levels of student achievement. Case studies are presented for each school and comparisons in findings are made across schools. Differences in school characteristics suggest that a variety of schools serving poor communities can achieve high levels of student achievement and that they may do this in different ways. Important strategies used by many of the schools to improve academic achievement are highlighted and a set of recommendations based on these findings are provided.

Description: Knowledge

Capacities: Action for Learning, Leadership, Priorities

Type: Website

Chen, E., Heritage, M., & Lee, J. (2005). Identifying and monitoring students' learning needs with technology. Journal of Education for Students Placed at Risk (JESPAR) 10(3), 309-332. Retrieved February 22, 2007 from http://www.leaonline.com/doi/pdf/10.1207/s15327671espr1003_6

Summary: This article presents initial findings from an evaluation research study of the implementation of a Web-based decision support tool, the Quality School Portfolio (QSP), developed at the National Center for Research on Evaluation, Standards, and Student Testing (CRESST) at the University of California, Los Angeles (UCLA). The study focused on users' experiences with the training for and implementation of QSP. The results show that QSP provided educators with enhanced access to extensive data and the ability to analyze the data to identify at-risk students. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Analysis, Decision-Making, Infrastructure

Type: Website

Chester, M. D. (2005). Making valid and consistent inferences about school effectiveness from multiple measures. Educational Measurement: Issues and Practice 24(4), 40-52. Retrieved January 15, 2007 from http://www.blackwell-synergy.com/doi/pdf/10.1111/j.1745-3992.2005.00022.x

Summary: This study explores the use of multiple measures to make judgments about school and district effectiveness. More specifically, the researcher examined data from the state of Ohio's accountability system to determine whether school and district classifications based on multiple measures (i.e., performance indicators, performance index, growth calculation, and adequate yearly progress) were reliable and valid. Among other findings, Chester discovered that the way in which the measures are combined is as important as which measures are chosen. For example, Ohio's method of combining across grades and years for AYP improves the accuracy of classification. Chester also maintains that the degree to which an accountability system leads to improvements in curriculum and instruction and increased student learning is of primary importance. The author concludes that in addition to using multiple measures, states must decide on meaningful, strategic ways to combine those measures. Furthermore, the data from Ohio suggest that well designed state accountability systems can support improvements in schools and student learning.

Description: Knowledge

Capacities: Multiple Measures

Type: Website

Choppin, J. (2002, April 2). Data use in practice: Examples from the school level. Paper presented at the Annual Meeting of the American Educational Research Association, New Orleans, LA. Retrieved April 8, 2006 from http://www.wcer.wisc.edu/archive/mps/AERA2002/data_use_in_practice.htm

Summary: This paper reports results from six Milwaukee Public School case studies examining the use of data to inform decision making in schools. The author begins by exploring differences in the schools involved, obstacles faced during their efforts to improve/adopt data-based decision-making practices/strategies, and relevant trends in their findings. Next, the author uses findings from these six case studies to determine the data needs of schools, how the quality and flow of data to schools could be improved, the level of analysis most useful to schools, and how schools can use data effectively to meet their specific needs. The author concludes by summarizing comments from focus groups from each school regarding conditions needed to support data-informed decision-making efforts, overall impact from the project, and data uses and processes for the future.

Description: Knowledge

Capacities: Data Systems, Infrastructure, Tools

Type: Website

Chrispeels, J. H., Brown, J. H., & Castillo, S. (2000). School leadership teams: Factors that influence their development and effectiveness. In G.M. Crow, & K. Leithwood (Eds.) Understanding Schools as Intelligent Systems, 4, 39-73, JAI Press.

Summary:

Description:

Capacities:

Type:

Clarke, S. (2005). Formative assessment in the secondary classroom. Manchester: Hodder Murray.

Summary: This book is a practical guide to support teachers who want to implement formative assessment practices in secondary classrooms. The author emphasizes learning goals, effective questioning, and feedback as the essential elements of formative assessment practices. Clarke maintains that the ways in which teachers communicate expectations and evaluate performance are key to effective assessment. Clarke also explores different approaches to feedback, highlighting how students respond to various strategies. Clarke includes numerous examples of formative assessment practices from secondary classrooms to further illustrate these practices and ideas.

Description: Guidance

Capacities: Multiple Measures

Type: Book

Clements, B., & Nadeau, G. (2006). Optimal reference guide: Guidelines for accessing student records in a state longitudinal database. Part III. Austin, TX: ESP Solutions Group. Retrieved June 14, 2007 from http://www.espsolutionsgroup.com/resources.php

Summary: This third brief in a series of reports about data-driven decision-making and the development of effective data warehouses reviews the policies and practice issues impacting how education data are managed. Clements and Nadeau discuss laws requiring protection of student records as well as other issues related to longitudinal student record systems including identifying data for inclusion, setting procedures for secure data collection, and determining who has access to state individual student records. The authors suggest using specific levels of access to make clear who has access to what kinds of data. The authors conclude with recommendations for protecting the confidentiality of education records (e.g., create policies to protect access to data and avoid improper disclosures), emphasizing the importance of determining who has access, what data are preserved, how are data used, and what steps are taken to secure data collection and maintenance.

Description: Guidance

Capacities: Data Systems, Infrastructure, Policies

Type: Website

Clymer, J. B., & Wiliam, D. (2007). Improving the way we grade science. Educational Leadership, 64, 36-42.

Summary: Clymer and Wiliam argue that a standards-based grading system provides a better picture of student achievement and helps convey learning goals to students. After reviewing the research on assessment practices and the meaning of feedback, the authors discuss assessment that supports learning, explaining that assessment information must be instructionally meaningful and dynamic. Clymer and Wiliam describe results from a pilot study on grading in an 8th grade science class in which the authors implemented a grading system based on mastery of 10 content standards, rating students as mastery, developing, or beginning, with the final grade reflecting the aggregate level of proficiency in these 10 standards. Students were given multiple chances to show their knowledge or learning and no grade was final until the end of the grading period. Teachers provided students with regular feedback on class work and weekly progress reports. The authors found that after implementing this new grading system, many students changed from a performance oriented approach to a mastery oriented approach to learning. Students also reported a better understanding of the material, were able to focus more on key concepts, and used peer- and teacher- feedback to monitor their own learning. Additionally, this new grading system was associated with improvements in student achievement, particularly for the highest- and lowest- achieving students. The authors conclude that standards-based assessment is an improvement on more traditional assessment methods frequently used in the U.S. and will help teachers use assessment to better support student learning.

Description: Research

Capacities: Multiple Measures, Reciprocal Feedback

Type: Website

Coburn, C. E., & Talbert, J. E. (2006). Conceptions of evidence use in school districts: Mapping the terrain. American Journal of Education 112(4), 469-495.

Summary: This article explores evidence use at multiple levels of the educational system and how various professionals conceive of evidence-based practice, including what makes evidence of student learning valid (e.g., psychometric properties of the test, alignment with grade-level outcomes, etc.) and how evidence of student learning should be used (e.g., meeting accountability demands, informing policy decisions, informing classroom instruction, etc.). They also explore education professionals’ opinions of what constitutes high-quality research (e.g., strong base in theory, scientific rigor, undeveloped conceptions of research, etc.) and how much they believe in research as a guide to practice. The researchers find much variation in educators’ conceptions of evidence and what constitutes high-quality research. This variation depends on their professional role in the educational system, and the organizational pattern and reform history of their district. More specifically, professionals who were in positions requiring more direct interaction with research had more highly developed conceptions of high-quality research, and expressed more faith in research to inform policy and practice. District administrators were more likely to see evidence as valid if it possessed certain psychometric properties and matched learning outcomes, whereas teachers and principals saw evidence as valid if it helped shed light on student thinking and was based in teacher judgment. The researchers maintain that a system of evidence-based practice may require different kinds of evidence for different uses at every level, and administrators in the central office must act as mediators between the top district officials and teachers to facilitate communication about evidence-based practice at all levels. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Culture of Inquiry, Multi-level Data Use, Professional Expertise

Type: Website

Coburn, C. E., Honig, M. I., & Stein, M. K. (in press). What is the evidence on districts' use of evidence? In J. Bransford, L. Gomez, D. Lam, & N. Vye (Eds.) Research and practice: Towards a reconciliation. Cambridge: Harvard Educational Press.

Summary: This chapter documents findings from an extensive review of research on school districts' use of evidence in decision making. The authors discover that the processes by which district personnel use evidence to inform their decisions are often complex and mediated by individual and communal interpretations of the research, as well as by organizational and political conditions. Based on these findings the authors provide a set of key lessons to support evidence use in districts which highlight the importance of understanding the social processes at play.

Description: Knowledge

Capacities: Decision-Making, Leadership, Skills and Knowledge, Standards

Type: Website

Coburn, C.E., Toure, J. & Yamashita, M. (in press). Evidence, interpretation, and persuasion: Instructional decision making at the district central office. Teachers College Record.

Summary: This article focuses on evidence use at the district level and examines how district personnel use research and data when making decisions, and factors influencing the decision-making process. The researchers conducted interviews with various district employees, collected a variety of documents, and observed planning and professional development meetings over a three year period (2002- 2005). The researchers find that while research/evidence often signaled existing problems, district professionals typically used their own content knowledge and beliefs when making decisions about solutions to those issues. Differences of opinion within the district about how to best address problems were influenced by four factors: district leader’s content knowledge, having multiple levels and divisions bringing different backgrounds and ideas, limited resources to interpret data, and changes in upper-level leadership. The authors maintain that decision-making at the district level is often shaped by politics and recommend that districts receive sufficient resources to support complicated decision-making. They also suggest that all district professions be given an opportunity to interact and with research and data, and develop content knowledge to make more informed decisions.

Description: Research

Capacities: Decision-Making, Expectations for Data Use, Skills and Knowledge

Type:

Cohen, D. K. (1995). What is the system in systemic reform? Educational Researcher 24(9), 11-17.

Summary: The author argues that systemic reforms in education often focus on policy instruments as a means of driving instructional change, rather than on the complex relations between policy and practice. He explains that instructional "systems," including teacher knowledge, professional values and committments, and social resources of practice, are weak and that traditional policy is not been designed to address these. Therefore, researchers need to look at the education system from both the perspective of policy reform and instructional change in order to discover ways in which policy can improve practice. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Policies, Standards

Type: Website

Colgan, Craig. (2004). Data on demand: Student information systems become a key tool for crunching and analyzing data. Retrieved April 8, 2006 from http://www.asbj.com/specialreports/0904SpecialReports/S1.html

Summary: This article discusses how to implement and support information technology changes in schools for increased data based decision making.

Description: Guidance

Capacities: Data Systems, Tools

Type: PDF

Consortium for School Networking (2003). Vision to know and do: The power of data as a tool in educational decision making. Washington, DC: Author. Retrieved September 30, 2008, from http://www-03.ibm.com/industries/education/doc/content/bin/dddm_paper.pdf

Summary: This article presents exemplars of data-driven decision making in school districts. District leaders created a culture of data-driven decision making through deliberate and careful planning and encouraging a sense of shared accountability. The districts adopted systems typically used for improving business practices and modified them to meet the needs of an educational system. The article includes several examples of districts using data to transform their organization and improve school performance. Featured districts are diverse in terms of size, geographical area, and student composition. These examples highlight the improvement process and demonstrate how data-driven decision making can be used to improve student learning outcomes, teacher practices, professional development, administrative budget decisions, and community support.

Description: Guidance

Capacities: Data Quality Assurance, Decision-Making, Professional Development

Type: Website

Consortium for School Networking. (n.d.). How ready are you? A special tool for school district leaders to assess readiness to use data effectively. Retrieved July 31, 2006 from http://www.3d2know.org/assessment/survey.cfm

Summary: This is a brief 10-item web survey about district readiness for data-driven decision making.

Description: Tool

Capacities: Action for Learning

Type: Website

Copland, M. (2003). Leadership of inquiry: Building and sustaining capacity for school improvement. Educational Evaluation and Policy Analysis 25(Winter), 375-395. Retrieved May 8, 2007 from http://epa.sagepub.com/cgi/reprint/25/4/375.pdf

Summary: This article reviews a longitudinal mixed-methods study of leadership in the context of a region-wide school reform effort entitled the Bay Area Reform Collaborative (BASRC). Data analysis suggests the use of an inquiry process is paramount for building capacity for school improvement and developing leadership. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities:

Type: Website

Corcoran, T., Fuhrman, S. H., Belcher, C. L. (2001). The district role in instructional improvement. Phi Delta Kappan 83(September), 78-84. Retrieved May 8, 2007 from http://repository.upenn.edu/cgi/viewcontent.cgi?article=1006&context=gse_pubs

Summary: The researchers examined the roles played by district staff members in shaping and supporting instructional classroom reforms using evidence-based decision making in three large urban school districts. (Publisher granted permission to post abstract only. Link to article abstract is provided.)

Description: Knowledge

Capacities:

Type: Website

Council of Chief State School Officers. Administrative data improvement. Retrieved May 19, 2006 from http://www.ccsso.org/projects/administrative_data_improvement/

Summary: This project provides resources and technical assistance to state education agencies for data system improvements and participation in federal data collection efforts.

Description: Tool

Capacities: Infrastructure, Leadership, Professional Development

Type: Website

Cromey, A. (2000). Using student assessment data: What can we learn from schools? Policy Issues, Issue 6.

Summary: Drawing on information from case studies in nine Michigan public schools, the researchers determined that schools can better use student assessment data by limiting assessments to specific purposes, aligning standards with curriculum and instruction, providing professional development, and involving teachers. Successful data use practices and policies also depend on defining the purpose of assessment data in the monitoring process, evaluating assessment convergence or divergence, and the efficacy of local practices. The researchers argue that assessment data can be a valuable tool in promoting educational improvement when it is appropriately defined and used. (Permission to post article is pending.)

Description: Guidance

Capacities: Action for Learning, Culture of Inquiry, Policies, Standards

Type:

Crooks, T. J. (1988, Winter). The impact of classroom evaluation practices on students. Review of Educational Research, 58(4), 438-481.

Summary: This report examines the relationship between classroom evaluation practices and student learning by drawing on results from fourteen fields of research. Divided into four sections, the report first examines the nature, role, and impact of classroom evaluation. The second section, reviews research on the impact of classroom evaluation practices on student learning activities and achievement. The third section reviews research on student motivation and the ways in which it is affected by classroom evaluation practices. The final section focuses on the implications of the preceding sections, specifically with regard to the effective use of classroom evaluation. The author concludes by arguing that classroom evaluation is one of the most significant factors influencing education and therefore, must entail careful planning and be used as a means to assist learning instead of as an end in and of itself. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Research

Capacities: Multiple Measures

Type: Website

Crumrine, T., & Demers, C. (2007, September). Formative assessment: Redirecting the plan. Science Teacher, 74(6), 28-32.

Summary: This article chronicles the experiences of a dedicated group of HS science teachers in New Hampshire as they collaborate to develop a better understanding about teaching and learning. The group meets every Friday morning to engage in "professional storytelling" where they share and discuss personal teaching stories. Although the topics covered in these meetings are diverse, all discussions focus on understanding the learning process as it is happening. The authors then note four common questions addressed in this collaboration: (1) What is important for students to understand?; (2) How do teachers know what students understand?; (3) How do students know what they know?; and (4) What is to be done with all this knowledge?. The authors focus the remainder of the article examining these four questions by describing actual classroom examples and related experiences. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: knowledge

Capacities: Culture of Inquiry, Skills and Knowledge

Type: Website

CUNY: Information Competency Tutorials. Retrieved April 25, 2007 from http://qcpages.qc.edu/library/olstutorial/02_define/M2A.html.

Summary: Tutorial 2: Define Your Research Topics provides straightforward steps for how to develop a research topic, formulate the topic into focused question(s), identify main concepts within each question, and broaden or narrow your topic depending available information. The final page provides a worksheet for users to document specific terms and concepts related to their research topic and related questions.

Description: Tool

Capacities: Professional Development, Skills and Knowledge

Type: Website

Dailey, D., Fleischman, S., Gil, L., Holtzman, D., O'Day, J., & Vosmer, C. (2005). Toward more effective school districts: A review of the knowledge base. Washington, DC: American Institutes for Research. Retrieved May 08, 2007 from http://www.air.org/projects/Toward%20More%20Effective%20Scchool%20Districts-A%20Review%20of%20the%20Knowledge%20Base%206-14-05%20313%20pm.pdf

Summary: The purpose of this paper is to describe how large urban school districts can improve classroom instruction and student performance. The authors review the research on district effectiveness, focusing on gains in student achievement and other outcomes (e.g., lower dropout rates) in high-poverty, underperforming districts. The authors summarize reports, research studies, and policy statements on effective district reform and highlight resources that can be used to foster district improvement. The authors maintain that the research on high performing high poverty districts continues to be somewhat limited so findings should be interpreted with caution. The article concludes with a detailed annotated bibliography.

Description: Knowledge

Capacities: Action for Learning

Type: Website

Data Coach website. Retrieved June 24, 2008 from http://www.aea10.k12.ia.us/datacoach/index.html.

Summary: This website contains useful tools to assist educators in building their capacity to use data in their decision-making processes. The Data Collection Guide can be used to define key questions, to identify information needed to answer the questions and to develop a data collection plan. The Learning Team Process Outline assists educators in documenting progress and establishing next steps in order to collect, organize and analyze information to answer key questions related to student achievement. Lastly, the Data Coach Project Packet includes helpful worksheets and checklists to keep users on track in their efforts to integrate data into their decision-making.

Description: Tool

Capacities: Decision-Making, Leadership, Skills and Knowledge, Tools

Type: Website

Data Quality Campaign. (2005). Creating a longitudinal data system: Using data to improve student achievement. Austin, TX: National Center for Educational Accountability. Retrieved July 31, 2006 from http://www.dataqualitycampaign.org/files/Publications-Creating_Longitudinal_Data_System.pdf

Summary: The purpose of the Data Quality Campaign is "to demonstrate to teachers, administrators, and other education partners that high-quality, easily accessible and functional longitudinal data are among the most powerful tools they can use in their efforts to prepare every student for success in an increasingly knowledge-based world" (p. 6). Drawing on the campaign's mission-statement, this document reviews the fundamentals of longitudinal data systems and describes ten essential elements of a state longitudinal data system. The elements include a unique statewide student identifier; the ability to match student records between P-12 and postsecondary systems; and an audit system to assess data quality, validity and reliability. The authors conclude that while these ten essential elements will create more sustainable systems, states must continue to develop their programs and generate next-generation improvements.

Description: Guidance

Capacities: Capacity Building, Data Systems, Infrastructure

Type: Website

Data Quality Campaign. (2006). Building student-level longitudinal data systems: Lessons learned from four states. Austin, TX: National Center for Educational Accountability.

Summary: The Data Quality Campaign conducted four site visits (Florida, Utah, Virginia, Wisconsin) to document the common challenges and lesson learned as states design and build student-level longitudinal data systems. This report synthesizes the findings from these four visits. First, 10 essential elements for a complete state longitudinal data system (e.g., a unique statewide student identifier, information on untested students, etc.) are presented. Next, the article includes information about why these four states were chosen and how long it takes to build a longitudinal data system highlighting time, funding, and human resource considerations as well as public relations and promotion efforts and plans for ongoing maintenance and change. The article also addresses lessons learned from other states already implementing these systems, focusing on recommendations for data managers (e.g., create internal and external advisory groups, implement data validation processes) and tips for writing requests for proposals (e.g., be knowledgeable about what you are looking for, be specific in delineating your requirements).

Description: Guidance

Capacities: Data Systems, Multi-level Data Use

Type: Website

Datnow, A., Park, V., & Wohlstetter, P. (2007). Achieving with data: How high-performing school systems use data to improve instruction for elementary students. Center on Educational Governance, Rossier School of Education, University of Southern California. Retrieved February 16, 2007 from http://www.newschools.org/viewpoints/AchievingWithData.pdf

Summary: This study explores data-driven instructional decision-making at the classroom, school, and system levels in two urban school districts and two nonprofit charter management organizations. The researchers document effective performance-driven practices, identify salient themes regarding the structure and culture of the systems, examine needs for improvement, and make recommendations for policy and practice.

Description: Knowledge

Capacities: Capacity Building, Expectations for Data Use, Leadership, Policies

Type: Website

Deal, T.E. & Peterson, K.D. (1999). Shaping school culture: The heart of leadership. San Francisco: Jossey-Bass Education.

Summary: This book explores how school leaders can implement reforms within the context of school culture (e.g., traditions, norms, values, etc.), and how they can use their schools' unique cultures to create dynamic communities of educators and learners. Specifically, the authors explore the importance of school values, the relevance of school traditions, the importance of symbols such as mottos and words, and the ways in which leaders can influence school culture to create a thriving learning environment. The authors also examine the less positive aspects of school culture, and strategies for school leaders to avoid or address negativity in teachers, administrators, students, and parents. Peterson and Deal demonstrate these methods by incorporating real-life examples of effective practices in a variety of school cultures.

Description: Knowledge

Capacities: Action for Learning, Culture of Collaboration, Leadership

Type: Book

Dembosky, J. W., Pane, J. F., Barney, H., Christina, R. (2005). Data drive decisionmaking in Southwestern Pennsylvania school districts. RAND Corporation. Retrieved March 2, 2007 from http://www.rand.org/pubs/working_papers/2006/RAND_WR326.sum.pdf

Summary: This report describes the capacity of school districts in Southwestern Pennsylvania to use data and analysis to inform instructional, policy, and evaluation decisions to improve student achievement. Conclusions from this study include: schools and districts are in the early stages of systematically using data to drive decision-making; most teachers and principals value data, but lack sufficient data analysis skills and a process for using data; schools and districts are using data to evaluate and adjust curriculum and programs, but not to evaluate teachers; lack of time presents the biggest challenge to schools and districts in this study. The report further suggests recommendations regarding policy, developing a data-driven decision-making culture, and technology resources.

Description: Knowledge

Capacities: Culture of Collaboration, Culture of Inquiry, Expectations for Data Use

Type: Website

Desimone, L., Porter, A. C., Birman, B.F., Garet, M.S., & Yoon, K.S. (2002). How do district management and implementation strategies relate to the quality of the professional development that districts provide to teachers? Teachers College Record, 104(7), 1265-1312.

Summary: The purpose of this study is to examine district policies for providing high quality professional development for teachers. The researchers found that strategies such as aligning professional development to standards and assessments, continuous improvement efforts, and teacher involvement in planning were associated with higher quality professional development for teachers. Furthermore, high quality professional development is characterized by active learning opportunities, collective participation, and activities such as teacher networks or study groups rather than more traditional approaches. These findings provide evidence for the relationship between federal policies and support strategies, implementation techniques, and the quality of professional development experiences. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Professional Development

Type: Website

Dixon, H. & Williams, R. (2003). Teachers' understanding and use of formative assessment in literacy learning. New Zealand Annual Review of Education,12:2002. Retrieved May 15, 2008 from http://www.tki.org.nz/r/assessment/research/pdf/DixonWilliams_NZ%20Annualreview_article19Dec03.pdf

Summary: The purpose of this study is to explore teachers' understanding about, and use of formative assessment in literacy learning with children in first through eighth grades. The authors conducted semi-structured interviews with 40 randomly selected primary school teachers. Teachers were asked about their perceptions of differences between formative and summative assessments, how they use assessment data in reading, writing, and oral language instruction, and factors that have influenced their understanding of formative assessment. Although past research has shown that teachers are often confused about the nature and purpose of formative assessment, results of the current study indicated that teachers understand key characteristics of formative assessment and how these practices differ from summative measures. When describing their formative assessment practices, however, significant practices were not mentioned (e.g., feedback to promote learning, sharing learning goals with students) demonstrating the need to link teachers' theoretical understanding to classroom practice. The authors also found that while teachers report using some formative assessment strategies in written language instruction (e.g., conferencing, encouraging student self-assessment) and reading instruction (e.g., running records), they were less clear about how to use formative assessment in oral language instruction. The authors conclude that although teachers demonstrate a theoretical understanding of formative assessment, there is a need for a better understanding of how to implement these strategies in the classroom.

Description: Research

Capacities: Capacity Building, Multiple Measures, Professional Development, Professional Expertise, Skills and Knowledge

Type: PDF

Dougherty, C. (2008). The power of longitudinal data: Measuring student academic growth. National Center for Educational Accountability, Data Quality Campaign.

Summary: This article examines the use of growth models to measure student and school progress in meeting the accountability mandates of the No Child Left Behind (NCLB) Act. The author argues that growth models are an effective way to measure student progress, given that some students (particularly those who are behind academically) have made significant progress, even though they still do not meet the goals of basic proficiency in reading and mathematics. Furthermore, growth models also allow educators to see the progress made by students who have already reached proficiency goals. The author describes advantages of growth models, maintaining that growth models take into account students' beginning levels of achievement to show academic growth for students at all levels, and will help identify effective schools and teachers. Dougherty explains that longitudinal data systems are essential for using growth models and highlights key elements for data systems used to report student growth (e.g., unique statewide student identifiers, the ability to link individual student's test score records over time, etc.). The author also presents findings from a 2006 DQC survey assessing states' abilities to measure academic growth. Results indicated that 28 states had the needed longitudinal data elements to measure student academic growth, while 36 states demonstrated these capabilities in 2007. Lastly, the author examines the use of growth models under NCLB, exploring how various states use these models to measure student progress. Dougherty concludes that although growth models are complex and require a longitudinal data system, the information they provide to schools, districts, and states will be very useful in improving educational policy and practice.

Description: Knowledge

Capacities: Analysis, Multi-level Data Use, Reporting

Type: Website

DuFour, R. (2001). In the right context: The effective leader concentrates on a foundation of programs, procedures, beliefs, expectations, and habits. Journal of Staff Development 22(1). Retrieved May 08, 2007 from http://www.nsdc.org/library/publications/jsd/dufour221.cfm (Reprinted with permission of the National Staff Development Council, www.nsdc.org, 2009. All rights reserved.)

Summary: (Reprinted with permission of the National Staff Development Council, www.nsdc.org, 2009. All rights reserved.)

Description: Guidance

Capacities:

Type: Website

Dunifon, R. (2005). How to read a research article. Ithaca, NY: Cornell Cooperative Extension. Retrieved April 25, 2008 from http://www.citra.org/Assets/documents/How%20to%20Read%20a%20Research%20Article.pdf.

Summary: This research brief outlines steps for how to effectively read, understand and critique a scholarly research article. It begins by clarifying the various sections of an article and then takes readers through the process of identifying essential information including: the source of the article, the literature review findings, the data used in the analysis, the measures and definitions of variables, the methods and results, and the conclusions and limitations of a given study. Geared towards educators, this article supports their efforts to navigate through academic research articles and integrate findings into their own work.

Description: Guidance

Capacities: Professional Development, Skills and Knowledge

Type: Website

Dunn, K. E., & Mulvenon, S. W. (2009). A critical review of research on formative assessments: The limited scientific evidence of the impact of formative assessments in education. Practical Assessment, Research, & Evaluation, 14(7), 1-11.

Summary:

Description:

Capacities:

Type: Website

Dweck, C.S. (2000). Self-theories: Their role in motivation, personality and development. Philadelphia: Psychology Press.

Summary: In this book, Dweck presents her program of research on adaptive and maladaptive cognitive-motivational patterns. The author demonstrates how these patterns originate in self theories and how a person's thought process affects achievement, work behaviors or habits, social relationships, perceptions of others, and emotional well-being. Dweck also examines the experiences that shape and influence these cognitive-motivational patterns. Findings are based on extensive research with children and young adults. This book is designed for professionals and researchers in psychology, education, child development, or other related fields.

Description: Knowledge

Capacities: Multiple Measures

Type: Book

Dwyer, C.A. (Ed.). (2006). The future of assessment: Shaping, teaching, and learning. Mahway, NJ: Erlbaum.

Summary: The purpose of this book is to explore the role of assessment to support teaching and learning in the classroom. The essays included are interdisciplinary, representing the fields of education, psychology, economics, statistics, and public policy, and attend to issues regarding the impacts of the No child Left Behind act. More specifically, researchers in these fields discuss changes in ideas about assessment in the last century, the benefits and challenges of incorporating classroom assessments, technologies to support instruction, assessment for learning requiring both teacher and student participation, the alignment of assessment with school standards and reforms, and the overall value for assessment in education. This book is based on papers presented at the Educational testing Service Invitational Conference held in New York, October 2005.

Description: Knowledge

Capacities: Action for Learning, Analysis, Balanced Assess't System, Data Integrity, Data Quality Assurance, Expectations for Data Use, Multiple Measures, Priorities, Professional Development, Professional Expertise, Skills and Knowledge, Tools

Type: Book

Earl, L., & Fullan, M. (2003). Using data in leadership for learning. Cambridge Journal of Education, 33(3), 383-394.

Summary: This article highlights the dilemmas and advantages school leaders experience in using data for school planning and improvement. The researchers examine findings from studies of school reform efforts in England and Manitoba and Ontario, Canada. Findings across these studies reveal common reservations school leaders have about data, including: insecurity about their ability to understand data, uncertainty about the uses and usefulness of data, and concerns about how to translate data into useable knowledge and effectively communicate it to people in their communities. Despite these concerns, leaders in all three locations say data helped focus their thinking and provided them with insights that were useful in making decisions. The article also notes contexts which increase leaders’ tension about data and makes recommendations for how to increase the meaningfulness of data collection. The researchers emphasize the need to strengthen school leaders' assessment literacy in order to enhance the role data plays in school leadership. (Publisher granted permission to post abstract only. Link to article abstract is provided.)

Description: Guidance

Capacities: Action for Learning, Communication, Leadership

Type: Website

Earl, L., & Katz, S. (2002). Leading schools in a data-rich world. In K. Leithwood & P. Hallinger (Eds.), Second international handbook of educational leadership administration (1003-1022). Dordrecht: Kluwer Academic.

Summary:

Description:

Capacities:

Type: Book

Easton, L.B. (2008). Mix it up: Variety is key to a well-rounded data-analysis plan. Journal of Staff Development, 29(4), 21-24.

Summary: In this article, the author argues that educators must use numerous data sources to effectively evaluate school and student progress. Easton argues that test data (at the state, district, or school level) alone are insufficient and that alternative sources should be used to complement student performance measures. The author recommends that educators use focus group discussions and individual student interviews to gain an understanding of students' ideas and perceptions of their learning experience, and examine student work in relation to learning goals to determine each student's level of performance and ways of thinking. Easton also suggests using classroom walk-throughs and shadowing students (i.e., following them throughout the day) to gain information about school processes and a student's actual experience in school. The author maintains that after data have been collected, educators must identify patterns or themes and report findings to the entire school faculty. Easton concludes that data from actual students and real student work are essential to an effective data analysis plan. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Guidance

Capacities: Balanced Assess't System, Expectations for Data Use

Type: Website

Eberle, F., & Keeley, P. (2008, January). Formative assessment probes. Science and Children, 45(5), 50-54.

Summary: This article explores how formative assessment probes can help teachers connect students' initial ideas to complex science concepts. The authors use classroom snapshots to illustrate the use of probes in science instruction. More specifically, they demonstrate how probing fifth grade students understanding in a science class helped develop and refine their ideas and conceptions of rocks and matter. The authors argue that probing furthers student learning by highlighting any misconceptions students might have, and providing an opportunity for students to connect scientific ideas. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Guidance

Capacities: Action for Learning, Multiple Measures

Type: Website

Educational Assessment. (2006). M. Heritage & A. Bailey (Eds.). Special Issue: Assessing to Teach, 3-4(11).

Summary: (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Culture of Inquiry, Multiple Measures, Professional Development, Professional Expertise, Skills and Knowledge, Tools

Type: Website

Elmore, G.C., Holloway, J.R., & Workman, S.B. (2006). Vision, data, and analysis: An administrative structure for decision making. Educational Research Series: Issue 11. Boulder, CO: EDUCAUSE Center for Applied Research. Retrieved September 23, 2008, from http://www.educause.edu/content

Summary: This report presents the decision-making process and strategies used to support reformatting and change in a collegiate information technology (IT) system. More specifically, Indiana University successfully implemented an expansion of their current IT system to better meet the needs of faculty, staff, and students. The practices used to support these improvements, such as developing a detailed plan to achieve goals, using data-driven management strategies, and regularly revisiting goals and priorities, can be applied to foster change in various contexts and organizational structures. The authors highlight steps in the decision-making process, including collecting and analyzing data, prioritizing goals, effectively managing changes, securing resources for support, making informed decisions quickly, and knowing how to measure or monitor progress.

Description: Knowledge

Capacities: Decision-Making, Transformation Plan

Type: Website

Elmore, R. F. (2003). Doing the right thing, knowing the right thing to do: School improvement and performance-based accountability. Washington, DC: National Governors Association Center for Best Practices.

Summary:

Description:

Capacities:

Type: Website

Englert, K., Fries, D., Goodwin, B., & Martin-Glenn, M. (2003). Understanding how superintendents use data in a new environment of accountability. Aurora, CO: Mid-continent Research for Education and Learning.

Summary: This article provides findings from a study that examined superintendents' use of assessment and accountability systems. The study aimed to better understand the assessment and accountability practices and policies superintendents are implementing and to examine whether the policies simply comply with the No Child Left Behind Act or are also effective in furthering school and student improvement.

Description: Knowledge

Capacities: Action for Learning, Multi-level Data Use, Skills and Knowledge

Type: Website

Englert, K., Fries, D., Goodwin, B., Martin-Glenn, M., Michael, S. (2004). Understanding how principals use data in a new environment of accountability. Aurora, CO: Mid-continent Research for Education and Learning.

Summary: This report summarizes the findings of the second in a series of three studies that examines how educators at various levels use assessment data and accountability systems. The study aims to develop a better understanding of the assessment and accountability practices principals are implementing at the school level and to assess the effectiveness of schools' accountability policies in guiding school and student improvement. This study generated descriptive information about the needs schools and districts have for effectively using data, how schools and districts are using data to guide classroom practice, and whether there are differences in data use based on the level of school proficiency.

Description: Knowledge

Capacities: Action for Learning, Multi-level Data Use, Skills and Knowledge

Type: Website

Englert, K., Fries, D., Martin-Glenn, M., & Michael, S. (2005). How are Educators Using Data? A Comparative Analysis of Superintendent, Principal, and Teachers' Perceptions of Accountability Systems. Aurora, CO: Mid-continent Research for Education and Learning.

Summary: This report provides information on the assessment and accountability practices and policies educators are implementing on the classroom, school, and district levels. The study provides descriptive information about the need for schools and districts to effectively use data, how schools and districts use data to guide classroom practice, and the difference in data use based on the level of student proficiency in individual schools.

Description: Knowledge

Capacities: Action for Learning, Multi-level Data Use, Skills and Knowledge

Type: Website

Even, R. (2005). Using assessment to inform instructional decisions: How hard can it be? Mathematics Education Research Journal, 17(3), 45-61.

Summary: This article examines 1) how teachers make sense of student assessment data; and 2) how to best support teachers when adopting more contemporary assessment methods. The author maintains that contemporary assessment practices (e.g., projects, journals, portfolios, conversations, observations, etc.) are changing the purpose of assessment from evaluating student performance to improving student learning and informing teacher instructional decisions. To address the first problem, the author presents four episodes of teacher-student interactions during mathematics instruction in which a teacher misunderstands students during the learning process. The author finds that teachers' ability to make sense of student learning is an active and complicated process, as teachers must be able to understand how students think about mathematics, have a flexible instructional plan that allows for student feedback and redirection, and use flexible solutions to problems. To address the second problem, how to support teachers using contemporary assessment strategies in mathematics, the author examined a professional development course designed to teach primary teachers how to incorporate portfolios into their assessment repertoire. Even found that although contemporary assessment tools and methods were frequently discussed, this was done so with the purpose of more traditional assessment (i.e., to evaluate student performance at the end of a unit). The author also found that after learning to use the portfolio strategy, teachers reported greater integration of assessment and instruction than before taking the course. Even concludes that using more contemporary assessment methods to guide instruction may be somewhat complicated so teachers should be explicitly taught how to use assessment to inform instructional decisions. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Research

Capacities: Analysis, Professional Expertise, Skills and Knowledge

Type: Website

Fashola, O. S. (2004). Being an informed consumer of quantitative educational research. Phi Delta Kappan, 85(7), pp. 532-538.

Summary: This article provides a straightforward description of various components of scientific research, including evaluation designs, measures of student academic achievement, and interpretation of statistical findings. The author also discusses factors that contribute to program effectiveness including clear goals, regular assessments, professional development, and support. (Publisher granted permission to post abstract only. Link to article abstract is provided.)

Description: Guidance

Capacities: Analysis, Professional Development, Skills and Knowledge

Type: Website

Feldman, A., & Capobianco, B. M. (2007, November). Teacher learning of technology enhanced formative assessment. Journal of Science Education and Technology. Retrieved October 28, 2008 from http://www.springerlink.com/content/p8863455668557t2/fulltext.pdf

Summary: In this study, the researchers explored how technology enhanced formative assessment (FA) practices become an integral part of a teacher's classroom practice. Participants were high school physics teachers and data were collected using observations, interviews, and collaborative discussions. Data analysis revealed a model of four technologies (i.e., hardware and software, methods for constructing FA items, pedagogical methods, and curriculum integration) needed to develop the knowledge and skills to integrate technology enhanced FA practices. The researchers conclude that teachers must practice the four technologies, understand FA practices, and collaborate with other teachers in order to make FA a key part of their practice. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Research

Capacities: Culture of Collaboration, Multiple Measures

Type: Website

Feldman, J., & Tung, R. (2001). Whole school reform: How schools use the data-based inquiry and decision making process. Paper presented at the annual meeting of the American Educational Research Association, Seattle, WA.

Summary: This paper examines both the process and impact of implementing data-based inquiry and decision making (DBDM) in six public schools in Massachusetts. The summary includes an in-depth case study of one school that has successfully implemented DBDM and analysis of qualitative data across all six schools. Findings reported across the six schools include the impact the process has on schools and school culture, the kinds of support that facilitate the DBDM process and the barriers schools face when implementing DBDM. The paper concludes with a discussion of the implications for developing an inquiry-based school and ensuring equity of student access and achievement.

Description: Knowledge

Capacities: Culture of Collaboration, Culture of Inquiry, Data Systems, Decision-Making

Type: Website

Felner, R.D., Bolton, N., Seitsinger, A.M., Brand, S., & Burns, A. (2008). Creating a statewide educational data system for accountability and improvement: A comprehensive information and assessment system for making evidence-based changes at school, district, and policy levels. Psychology in the Schools, 45(3), 235-256.

Summary: This article describes statewide efforts to develop the High-Performance Learning Community [HiPlaces] Assessment, a high quality data reporting and utilization system designed to improve educational accountability. The authors describe how psychological theory and empirical research findings were used to inform the assessment method and highlight important features of this system. They also maintain that this system exceeds those typically used in education, as it includes a comprehensive assessment of all levels of the public education system (e.g., fiscal, policy, etc.) as well as the developmental needs and educational attainment of students. Furthermore, this system has undergone significant revisions since its inception and these changes were largely informed by data collected at the school, state and district levels. The authors conclude that this system is more comprehensive than most statewide assessment systems, but it should serve as a model for others to follow. (Permission to post article is pending. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Data Systems, Multi-level Data Use

Type: Website

Foegen, A., & Deno, S.L. (2001). Identifying growth indicators for low-achieving students in middle school mathematics. The Journal of Special Education, 35(1), 4-16.

Summary: This study examines the reliability and criterion validity of four mathematics measures as potential growth indicators in middle school mathematics. Participants included 100 students from an economically and ethnically diverse urban school, all of whom were taught by the same math teacher. Students completed four math measures including a Basic Math Operations Task, Basic Estimation Task and two Modified Estimation Tasks. These measures were completed at two times during one week of school. The reliability of the four math measures was assessed by determining internal consistency, test-retest, and parallel forms reliability. Results revealed adequate reliability for all four measures. Criterion validity was measured with correlations between the four math measures and school record data (i.e., school grades and standardized test scores) as well as teacher ratings of student performance in mathematics. Analyses revealed moderate relations among variables, indicating adequate criterion validity. Lastly, researchers used regression analyses to determine the efficacy of using the four math measures together or separately to predict standardized test scores and teacher ratings. The researchers found that single indicators of math performance predict other measures of math proficiency about as effectively as using the measures in combination. Overall, the researchers conclude that these four measures may be useful as indicators of math performance for middle school students.(Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Research

Capacities: Multiple Measures, Skills and Knowledge

Type: Website

Frisbie, D. A. (2005). Measurement 101: Some fundamentals revisited. Educational Measurement: Issues and Practice 23(2), 21-28. Retrieved January 15, 2007 from http://www.blackwell-synergy.com/doi/pdf/10.1111/j.1745-3992.2005.00016.x

Summary: In this article the author argues that many measurement concepts are misunderstood and misused in the field of education. During his experience as a university professor and director of the Statewide Testing Program in Iowa, Frisbie found that there is much confusion about the terms validity, reliability, and criterion- vs. norm-referenced tests, and he presents potential consequences of these misunderstandings. More specifically, the author maintains that inconsistencies in the use and meaning of "validity", leads to miscommunication within and outside the field, weak validation, and incomplete directions for test administration. He also finds that confusion about the term "reliability" leads to challenges in instrument selection and in establishing technical adequacy. Frisbie discusses the differences between criterion-referenced and norm-referenced tests and argues that using these labels to describe the test rather than the kind of score interpretation leads to problems with test selection, development, scoring, and interpretation. He also addresses additional points of confusion including test quality of a criterion-referenced context versus a norm-referenced context, standardized testing, the meaning of "on grade level", and percentiles versus percentile ranks. Frisbie concludes with suggestions for teachers, journal editors, assessment writers, and professional development providers to help develop a greater awareness of the need to use a common language in the field. (Permission to post article is pending. Link to publisher's website is provided.)

Description: Guidance

Capacities: Data Quality Assurance, Multiple Measures, Quality of Assessment Data

Type: Website

Fuchs, L. S. (2004). The past, present, and future of curriculum-based measurement research. School Psychology Review, 33(2), 188-192.

Summary: This is an introduction to an issue of School Psychology Review focused on curriculum-based measurement (CBM) research. The authors begin by describing how CBM evolved from mastery measurement, but avoids the pitfalls of this traditional approach of single-skill assessment by integrating various skills needed for competence on each weekly test. The author explains that because each test is similar in difficulty, rate of learning can be measured with slope, and the effectiveness of instruction can be assessed. A challenge of this method, however, is to develop assessments that incorporate various skills needed to demonstrate adequate learning at the end of the year. Fuchs describes three research stages used when assessing the effectiveness of CBM tasks: 1) investigation of the technical features of the static score; 2) measurement of technical features (i.e., slope) to determine if academic progress is being made; and 3) assessment of instructional utility to see if educators can use CBM results to inform and improve instruction. The author notes that although a considerably body of CBM research exists, recent studies have focused on the first stage (i.e., investigation of technical features). Fuchs acknowledges that stages 2 and 3 may be more difficult because they require ongoing data collection, yet maintains that understanding these stages is essential to advance CBM research. Fuchs recommends that, in addition to attending to stages 2 and 3, future studies explore the use of computer programs and technology to support CBM strategies (e.g., collecting and managing data, graphing analyses, creating diagnostic profiles for individual students or classrooms, etc.).

Description: Guidance

Capacities: Multiple Measures

Type: Website

Fuchs, L.S. Fuchs, D., Hamlett, C.L., Phillips, N.B., & Bentz, J. (1994). Classwide curriculum-based measurement: Helping general educators meet the challenge of student diversity. Exceptional Children, 60, 518-537. Retrieved November 29, 2008 from http://findarticles.com/p/articles/mi_hb3130/is_n6_v60/ai_n28638481

Summary:

Description: Guidance

Capacities:

Type: Website

Fullan, M. (2000, April). The three stories of education reform. Phi Delta Kappan, 81(8), 581-584. Retrieved February 13, 2008 from http://www.pdkintl.org/kappan/kful0004.htm

Summary: This article outlines three stories of education reform efforts focusing on the internal dynamics of individual schools; the responses of schools to external forces such as parents and community, technology, and government policy; and the efforts of outside agencies, ranging from school districts to entire states, in accomplishing large-scale school reform. Taken together these three stories provide a useful framework for avoiding overload and extreme fragmentation and accomplishing large-scale reform. (Publisher granted permission to post abstract only. Link to article abstract is provided.)

Description: Knowledge

Capacities: Action for Learning, Culture of Collaboration, Culture of Inquiry, Leadership, Stakeholder Collaboration

Type: Website

Gallagher, C., & Worth, P. (2008, December). Formative assessment policies, programs, and practices in the Southwest Region (Issues & Answers Report, REL 2008-No. 041). Washington, D.C.: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southwest. Retrieved October 28, 2008 from http://ies.ed.gov/ncee/edlabs

Summary: This article provides a summary of the policies and programs in five Southwest Region states (Arkansas, Louisiana, New Mexico, Oklahoma, and Texas) used to encourage formative assessment practices in the classroom. Arkansas, Louisiana, and New Mexico have been particularly effective in communicating their formative assessment objectives to stakeholders through state policies or state supported programs. There is, however, considerable variability in education policies regarding formative assessments, the degree to which states control the use of formative assessments at state and local levels, and steps taken to support implementation of formative assessment strategies (e.g., through professional development opportunities, resources, etc.). The authors conclude that overall, Arkansas, Louisiana, and Texas have developed numerous strategies to support formative assessment at the state and local levels, while formative assessment resources in New Mexico and Oklahoma are not as well developed.

Description: Knowledge

Capacities: Communication, Expectations for Data Use, Leadership, Policies, Skills and Knowledge

Type: Website

Gallagher, L., Means, B., & Padilla, C. (2008). Teachers' use of student data systems to improve instruction: 2005 to 2007. Washington, D.C.: U.S. Department of Education, Office of Planning, Evaluation and Policy Development.

Summary: This policy brief is the second in a two-part series examining teachers' use of data from student data systems. This report is based on teacher and technology director survey data collected in 2005 and 2007 from the U.S. Department of Education's National Educational Technology Trends Study (NETTS). This brief addresses how student data systems are being implemented in districts and schools, whether supports for data use and tools for working with data are prevalent, and how school staff use student data systems. Survey results indicated that from 2005 to 2007, teachers report much greater access to electronic student data systems, yet they also report only having access to students’ grades and attendance records rather than achievement data. Additionally, teachers who report receiving better-than-average-support for working with data are more likely to use student data to inform instruction. Teachers reported the desire for more professional development opportunities to strengthen data-use skills and learn how to use data to inform instruction, and cited a lack of time as a barrier to effective data-use in both 2005 and 2007. The authors conclude that there has been progress in the prevalence and use of student data systems in the past two years, yet more systematic strategies are needed to make data use a key part of instructional decisions.

Description: Research

Capacities: Data Systems, Human Resources, Reporting

Type: Website

Gardner, J. (Ed.). (2006). Assessment and Learning. London: Sage Publications, Ltd.

Summary: This book provides an overview of how assessment is used to support learning in the classroom. The book is divided into four parts (practice, theory, formative and summative issues, and policy), and each section includes research-based articles on assessment and learning from a variety of international perspectives. Key topics include learning and assessment in the classroom, professional development, assessment and theories of learning, motivation for learning, formative and summative assessments, and reliability and validity of assessments. The authors also address policy issues in the UK and the U.S. regarding assessment for learning. All contributing authors are members of the internationally known Assessment Reform Group (ARG) and are experts in the field.

Description: Knowledge

Capacities: Capacity Building, Culture of Collaboration, Culture of Inquiry, Data Integrity, Data Quality Assurance, Data Systems, Multi-level Data Use, Multiple Measures, Policies, Professional Expertise, Tools

Type: Book

Garmston, R.J. (2005). Group wise: Create a culture of inquiry and develop productive groups. JSD, 26(2). Retrieved February 13, 2008 from http://www.nsdc.org/library/publications/jsd/garmston262.cfm (Reprinted with permission of the National Staff Development Council, www.nsdc.org, 2009. All rights reserved.)

Summary: This article outlines three factors that are key to creating a culture of inquiry and developing productive groups: engaged leaders who continuously communicate and participate in practices; leaders who provide time and space for teacher collaboration; and the occurrence of continuing self-reflection following professional development. These collaborative efforts are focused on improving instruction, raising student achievement, and enhancing the professional community. (Reprinted with permission of the National Staff Development Council, www.nsdc.org, 2009. All rights reserved.)

Description: Knowledge

Capacities: Communication, Culture of Collaboration, Culture of Inquiry, Leadership

Type: Website

Garrison, C., & Ehringhaus, M. (2009). Formative and summative assessment in the classroom. National Middle School Association. Retrieved October 12, 2009, from http://www.nmsa.org/Publications/WebExclusive/Assessment/tabid/1120/Default.aspx.

Summary: Garrison and Ehringhaus argue that both summative and formative assessments are needed in the classroom to provide a complete view of student learning. The authors begin by defining and providing examples of summative assessment and formative assessment, highlighting distinctions between the two. Next, the authors explain how certain instructional strategies can be used formatively including criteria and goal setting, observations, questioning strategies, self- and peer-assessment, and student record keeping. Garrison and Ehringhaus maintain that these strategies are essential to the formative assessment process. The authors conclude by reemphasizing the importance of balanced assessment practices in the classroom and state that the more information teachers have about individual students as they are learning, the better they can modify instruction to meet student needs and keep learning moving forward.

Description: Guidance

Capacities: Balanced Assess't System, Multiple Measures

Type: Website

Goodman, D. & Hambleton, R. (2004). Student test score reports and interpretive guides: Review of current practices and suggestions for future research. Applied Measurement in Education, 17(2), 145-220. Retrieved September 23, 2008, from http://pdfserve.informaworld.com/789071_775644605_783685471.pdf

Summary: The researchers examined the ways in which large-scale student test scores are reported to parents, teachers, and students. They examined both test reports and interpretative guides from eleven states and two Canadian provinces, and found both promising and problematic features of these documents. More specifically, the researchers found that although score reports and accompanying interpretative guides include organizational tools (e.g., headings, graphics, etc.) to make them more readable and present results in a variety of ways to target diverse audiences, they also include excessive information and statistical jargon and appear cluttered and dense. Furthermore, these reports lack information about test precision and fail to define key terms. The authors recommend that score reports include all necessary information for understanding test results, with a separate interpretative guide providing details about the assessment and results, and that test results be made more personal for each student (e.g., embedding student names). The authors also recommend including a lay-language narrative summary explaining overall results and diagnostic information, strategies parents can use to help their children improve, and examples of various performance levels in the interpretative guides to give clear examples of achievement. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Research

Capacities: Communication, Expectations for Data Use, Reporting

Type: Website

Haladyna, T. M., & Downing, S. M. (2004, March). Construct-irrelevant variance in high-stakes testing. Educational Measurement: Issues and Practice 23(1). Retrieved January 3, 2007 from http://www.blackwell-synergy.com/doi/pdf/10.1111/j.1745-3992.2004.tb00149.x

Summary: The authors define and examine the common threat of construct-irrelevant variance (CIV) to validity, the appropriate interpretation of test scores, and its sources and pervasiveness in high-stakes testing. The authors argue that education leaders need to pay increased attention to this serious problem in order to improve the effectiveness and validity of evidence for high-stakes testing programs. (Permission to post article is pending. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Analysis, Data Quality Assurance, Quality of Assessment Data

Type: Website

Halverson, R. & Thomas, C.N. (2007). The roles and practices of student services staff as data-driven instructional leaders. (WCER Working Paper No. 2007-1). Madison: University of Wisconsin-Madison, Wisconsin Center for Education Research. Retrieved September 15, 2008, from http://www.wcer.wisc.edu/publications/workingPapers/Working_Paper_No_2007_01.pdf

Summary: This report focuses on the recent requirements made by NCLB to use student achievement data to evaluate school performance. The authors explore the role of various school professionals, including school psychologists, special education teachers, and school social workers as potential leaders in the use of data driven instruction and intervention. Unlike general educational practitioners, these professionals often receive training in data use and analysis as part of their more specialized preparatory programs and thus, may provide insights into how to use data effectively to improve teaching practices and student learning. Researchers conducted structured interviews with school and district leaders, recorded observations at various school events (e.g., faculty meetings, professional development meetings, etc.), and collected numerous documents (e.g., school budgets, improvement plans, etc.) to explore these ideas; data were collected from both elementary and middle schools in a Midwestern state. They conclude that these members of the school community are valuable resources for both their data analysis skills and their use of practices typically used to support special needs students (e.g., problem-solving teams, the IEP process). They suggest that school leaders enlist these professionals to help develop effective school-wide intervention and remediation programs and use their expertise to guide general education teachers in data-driven practices.

Description: Knowledge

Capacities: Culture of Inquiry, Human Resources, Professional Expertise

Type: Website

Halverson, R., Grigg, J., Prichett, R., & Thomas, C. (2005). The new instructional leadership: Creating data-driven instructional systems in schools. WCER Working Paper 2005-9, Wisconsin Center for Education Research, University of Wisconsin-Madison. Retrieved May 8, 2007 from http://www.academiccolab.org/resources/documents/HalversonGriggPrichettThomas%20NCPEA.pdf

Summary: The purpose of this paper is to explore data-driven instructional systems (DDIS) in schools and explain how the elements of DDIS may be used to help school leaders use student achievement data to inform quality instructional practices in the classroom. DDIS comprises six interrelated functions including data acquisition, data reflection, program alignment, program design, formative feedback, and test preparation. Through use of these practices, DDIS help make achievement test data accessible to both teachers and students to improve classroom learning and instruction. To explore DDIS, the researchers collected data from four elementary and middle schools in the Midwest known for effective leadership and data-informed decisions. The researchers conducted interviews with a variety of school leaders, observed faculty at school events (e.g., faculty meetings, professional development sessions, etc.), and collected school artifacts (e.g., budget information, school improvement plans, etc.). The researchers found substantial evidence of each DDIS function in each school. They conclude that DDIS encompass a wide variety of practices used by teachers and school leaders to make meaningful decisions and instructional improvements based on student data. They also highlight the significance of teachers (to inform classroom practice) and school leaders (to interpret school-wide performance) throughout this process as well as the interplay between both groups to maximize improvements in teaching and learning.

Description: Knowledge

Capacities: Culture of Inquiry, Decision-Making, Skills and Knowledge

Type: Website

Halverson, R., Prichett, R.B., Watson, J.G. (2007). Formative feedback systems and the new instructional leadership. (WCER Working paper No. 2007-3). Madison: University of Wisconsin-Madison, Wisconsin Center for Education Research.Retrieved September 15, 2008, from http://www.wceruw.net/publications/workingPapers/Working_Paper_No_2007_03.pdf

Summary: This report explores formative feedback systems, networks of structures, practices, and people that help make school testing data accessible and applicable to everyday instruction. The authors use the concept of formative feedback systems to demonstrate the processes through which teachers and administrators can use high-stakes testing information and other data to improve teaching practices and support student learning. They demonstrate this process by focusing on data driven revisions of a literacy program in one elementary school, and describe issues other schools may face making similar adjustments to curriculum and/or instruction. The authors provide this example to help schools better understand how to learn from student performance data and how to modify teaching programs and practices in an era of high-stakes testing. The authors conclude that by using formative feedback systems, schools may gain control over the instructional process, but caution that these specific grade- or school-wide decisions about student learning may also result in a narrowed curriculum.

Description: Knowledge

Capacities: Culture of Inquiry, Data Systems, Skills and Knowledge

Type: Website

Hamilton, L. (2003). Assessment as a policy tool. Review of Research in Education, 27, 25- 68.

Summary:

Description:

Capacities:

Type: Website

Hamilton, L., Halverson, R., Jackson, S., Mandinach, E., Supovitz, J., & Wayman, J. (2009). Using student achievement data to support instructional decision making (NCEE 2009-4067). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/wwc/pdf/practiceguides/dddm_pg_092909.pdf

Summary:

Description:

Capacities:

Type: Website

Harlen, W. (2007). Criteria for evaluating systems for student assessment. Studies in Educational Evaluation, 33(1), 15-28.

Summary: In this article, the author discusses criteria necessary to evaluate the usefulness of student assessment procedures. These criteria include: construct validity, reliability, desired impact, and good use of resources. The author first defines the criteria used to judge the components of an assessment system and then looks at how they apply to student assessment for formative and summative purposes. The author also highlights issues relating to the use of tests and the use of teachers' judgments for summative assessment, concluding that the four criteria are more likely to be met when there is greater use of teachers' judgments rather than external tests. The findings point to the importance of using evaluation to make decisions about appropriate methods of assessing students' achievement. The author contends that evaluators play an important role in informing decision-makers about alternative assessment practices. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Balanced Assess't System, Data Quality Assurance, Multi-level Data Use

Type: Website

Harrison, C., & Bryan, C. (2008). Data dialogue: Focused conversations put evidence to work in the classroom. Journal of Staff Development, 29(4), 15-19.

Summary: In this article, the authors maintain that many districts have implemented data management systems, but now, researchers must focus on understanding how teachers will use that data to improve student performance. Harrison and Bryan present a table summarizing the types of data conversations needed to improve instruction and learning including the type of data dialogue (e.g., whole-school conversations, one-on-one conversations, etc.), the kinds of data used, who is involved, potential conversation topics, and the frequency of these discussion. The authors also present several common features of effective data conversations (e.g., members of the conversation must agree that the data are a good measure of student learning), and elaborate further on school improvement team conversations, teacher- supervisor conversations, department or grade-level conversations (based on individual students and overall instructional strategies), and individual student goal-setting conversations, always including an example to illustrate these practices in a real school setting. Harrison and Bryan maintain that time must be reserved for teachers to engage in these discussions in order to continue making progress in efforts to use data to improve student achievement. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Guidance

Capacities: Action for Learning, Culture of Collaboration, Expectations for Data Use

Type: Website

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112. Retrieved May 15, 2008 from http://rer.sagepub.com/cgi/content/short/77/1/81

Summary: The purpose of this article is to examine how feedback influences learning and achievement. The authors begin by explaining the meaning of feedback and then discuss the effectiveness of feedback by reviewing numerous studies of various kinds of feedback provided in classrooms. They find much variability in the effect sizes of the studies reviewed and conclude that feedback is most effective when students receive specific information about a task and how to complete it effectively, while praise, rewards, and punishment are least effective. Next, the authors present a model of feedback, including both the purpose of feedback and strategies used by teachers and students to reduce the discrepancy between what is known and what needs to be learned. They maintain that effective feedback answers three questions: Where am I going? (i.e., goals); How am I going? (i.e., progress); and Where to next? (i.e., next steps to further learning). They also argue that each feedback question works at four levels (i.e., task level, process level, self-regulation level, and self level). The authors then use their model to address issues regarding feedback (e.g., timing, the role of assessment, etc.). The authors conclude that feedback can effectively enhance learning, though it occurs too rarely in actual classrooms, and must be researched more fully through both qualitative and quantitative methods to better understand how feedback works in the classroom setting. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Research

Capacities: Multiple Measures, Skills and Knowledge

Type: Website

Helwig, R., Anderson, L., & Tindal, G. (2002). Using a concept-grounded curriculum-based measure in mathematics to predict statewide test scores for middle school students with LD. The Journal of Special Education, 36(2), 102-112.

Summary: The purpose of this investigation was to explore the usefulness of curriculum-based measurement (CBM) in predicting student performance on statewide math achievement tests and to determine whether the CBM was more or less predictive for students with and without learning disabilities. The researchers administered an 11-item math concept CBM to eighth grade students (N = 171) who were diverse in terms of achievement, socioeconomic status, race/ethnicity, and geography. Next, the researchers correlated student scores on the CBM with a computer adaptive test created as a proxy for the official statewide mathematics achievement test. Results showed that correlations were high for general education students (r = .80, n = 90) and moderately high for learning disabled student s (r = .61, n = 81). Results from regression analyses indicate that both CBM scores and educational classification (special vs. general education) predicted whether students would meet state requirements for the statewide mathematics achievement test. The authors conclude that CBM tasks measuring conceptual understanding of mathematics may be an important component to include in assessment programs for both learning disabled and non-disabled students.(The publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Research

Capacities: Balanced Assess't System, Multiple Measures

Type: Website

Henderson, S., Petrosino, A., Guckenburg, S., & Hamilton, S. (2007). REL Technical Brief-a second follow-up year for "Measuring how benchmark assessments affect student achievement" (REL Technical Brief, REL Northeast and Islands 2007-No. 002). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Northeast and Islands. Retrieved from http://ies.ed.gov/ncee/edlabs

Summary: This technical brief summarizes findings from a follow-up study to "Measuring how benchmark assessments affect student achievement" (REL 2007- No. 039). The follow-up study examined whether schools in Massachusetts using quarterly benchmark assessments aligned with state standards in middle school mathematics showed gains in student performance two years after initial implementation when compared to schools not using these assessments. Results were similar to the first report- there were no differences in student achievement between schools using benchmark assessments and those without. Authors, however, note numerous limitations to the study including a lack of data on what benchmark assessment practices comparison schools might have been using, and including twice as many comparison schools than treatment schools in the study. Furthermore, the authors suggest that two years may be insufficient time to observe an impact in student performance from the intervention.

Description: Research

Capacities: Balanced Assess't System, Standards

Type: Website

Heritage, M. (2007, October). Formative assessment: What do teachers need to know and do? Phi Delta Kappan, 89(2). Retrieved May 15, 2008 from http://www.pdkintl.org/kappan/k_v89/k0710her.htm

Summary: The purpose of this article is to explore formative assessment practices and highlight the knowledge and skills teachers must have to effectively use these strategies in the classroom. The author begins by defining formative assessment (FA), grouping FA strategies into three broad types: on-the-fly assessment, planned-for interaction, and curriculum-embedded assessments. Next, the author describes the four key elements of formative assessment (i.e., identifying the learning gap, feedback at multiple levels, active involvement of students, and links to learning progressions) and then highlights the knowledge and skills teachers must have to implement FA practices. The author maintains that to use formative assessment successfully, teachers must have domain knowledge, pedagogical content knowledge, knowledge of students' previous learning, and knowledge of assessment practices. Furthermore, teachers must also possess skills to create classroom conditions that allow for successful assessment, to teach the students to assess their own learning and the learning of others, to interpret the evidence, and to match instruction to the gap they see in their students' learning. Heritage notes that in the current accountability environment, assessment is largely viewed as summative, thus, losing the reciprocity between teaching and assessment. Despite these trends, the author maintains that formative assessment practices provide a snapshot of student progress and can be valuable to guide instruction and promote student learning. Heritage concludes that if formative assessment practices are going to be used, an investment in teachers must be made by adding assessment courses to pre-service training programs, providing opportunities for new teachers to develop and practice assessment skills, and supporting continued professional development courses to further refine these skills.

Description: Guidance

Capacities: Expectations for Data Use, Multiple Measures, Professional Development, Professional Expertise, Skills and Knowledge

Type: Website

Heritage, M. (2008). Learning progressions: Supporting instruction and formative assessment. Washington, D.C.: Council of Chief State School Officers. Retrieved May 9, 2008 from http://www.ccsso.org/content/PDFs/FAST%20Learning%20Progressions.pdf

Summary:

Description: Knowledge

Capacities: Culture of Inquiry, Multi-level Data Use, Multiple Measures, Professional Development, Professional Expertise, Reciprocal Feedback, Skills and Knowledge

Type: Website

Heritage, M., & Chen, E. (2005, May). Why data skills matter in school improvement. Phi Delta Kappan, 86(9), 707. (ERIC Document Reproduction Service No. EJ712939)

Summary: In this paper, the authors describe a CRESST initiative designed to help educators develop the skills needed to collect, analyze, and use data to make informed decisions about student learning. Researchers at CRESST developed the Quality School Portfolio (QSP), a web-based program, as well as a complementary training program to help educators learn five core skills needed for effective data use. These skills include determining what you want to know, collecting data, analyzing results, setting goals, and developing strategies for achieving the goals. Survey and interview data from training participants reveals that the program has been effective in helping educators learn the skills needed for effective data use and has helped them develop confidence in working with data and developing improvement plans. The authors maintain that data must be used to inform school improvement efforts, and that educators are more likely to use data if they have the skills and knowledge to do so effectively. (Permission to post article is pending. Link to publisher's website is provided.)

Description: Guidance

Capacities: Data Systems, Expectations for Data Use, Infrastructure, Multiple Measures, Skills and Knowledge, Tools

Type: Website

Heritage, M., & Yeagley. R. (2005). Data use and school improvement: Challenges and prospects. In J.L. Herman & E. Haertel (Eds.), Uses and misuses of data for educational accountability and improvement (National Society for the Study of Education Yearbook, Vol. 104, Issue 2, pp. 320-339 ). Chicago: National Society for the Study of Education. Distributed by Blackwell Publishing.

Summary: This chapter examines the challenges of effective data use and how to address issues that may arise when implementing school-based data-systems. The authors discuss why achievement data must be aligned, valid, reliable, and sensitive to student differences and address specific advantages and disadvantages of the four main types of data (i.e., large-scale achievement tests, benchmark assessments, formative assessment, and school grades). The authors also move beyond assessment data and explore other sources of information that may impact student performance such as demographic data, perception data, and school processes. The chapter concludes with consideration of the ways data use can be integrated into school practice. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Human Resources, Skills and Knowledge

Type: Website

Heritage, M., Chen, E., & LaTorre, D. (2005). Upgrading America's use of information to improve student performance. (CSE Tech. Rep. No. 661). Los Angeles, CA: Center for the Study of Evaluation and National Center for Research on Evaluation, Standards, and Student Testing. Retrieved April 9, 2006 from http://www.cresst.org

Summary: The authors report on the development, implementation, and evaluation of the effectiveness and functionality of Quality School Performance (QSP), a decision support tool for states, districts, and schools. The authors discuss the five objectives of the project: to develop a support tool (QSP version 4.0); to develop a standardized training system for trainers and end users; to implement a national roll-out of QSP; to conduct evaluation research to assess the effectiveness, functions, and benefits of QSP and the training; and to design requirements for a fifth generation of decision-support tools using the results from this project. While the authors were unable to examine the effects of QSP on student learning, as originally planned, they were able to determine that QSP is a useful tool for performing educational data analyses.

Description: Knowledge

Capacities: Culture of Inquiry, Data Systems, Skills and Knowledge

Type: Website

Heritage, M., Kim, J., Vendlinski, T., & Herman, J. (2009). From evidence to action: A seamless process in formative assessment? Educational Measurement: Issues and Practice, 28(3), 24-31.

Summary: Researchers at the National Center for Evaluation, Standards, and Student Testing at UCLA conducted a generalizability study of measures of teacher knowledge for teaching mathematics. More specifically, teachers were asked to use assessment information to assess student learning and then decide what steps to take next to further instruction. The researchers found that while teachers were able to determine student performance levels from assessment data, they had considerable difficulty determining how to modify instruction in response to evidence of student learning. Heritage and colleagues argue that educators must develop an understanding of how learning progresses in a domain so that they know what to do next given a student’s current learning level. The authors also maintain that teachers must have a deep understanding of the actual domain so that they know what is (and is not) "good performance" in that subject area. The authors conclude that in order to move from evidence to action, teachers must have a better understanding of how learning develops and deeper knowledge of the content domain.

Description: Research

Capacities: Expectations for Data Use, Professional Expertise

Type: Website

Herman, J. L. & Haertel, E. H. (Eds.) (2005). Uses and misuses of data for educational accountability and improvement (National Society for the Study of Education Yearbook, Vol. 104, Issue 2). Chicago: National Society for the Study of Education. Distributed by Blackwell Publishing.

Summary: This book examines data use for accountability and improvement in education. Contributing authors discuss the philosophy and history of testing and assessment, test design and validity issues, characteristics of effective data use in schools, and challenges to implementing data-based decision making in schools. They explore other topics as well such as fairness in testing, challenged student populations, social promotion based on test results, and effects of the high-stakes testing environment on students and teachers. The book concludes with chapters highlighting best practices for effective data use in schools.

Description: Knowledge

Capacities: Expectations for Data Use, Quality of Assessment Data

Type: Book

Herman, J., & Gribbons, B. (2001, February). Lessons learned in using data to support school inquiry and continuous improvement: Final report to the Stuart Foundation. (CSE Tech. Rep. No. 535). Los Angeles, CA: Center for the Study of Evaluation and National Center for Research on Evaluation, Standards, and Student Testing. Retrieved April 10, 2006 from http://www.cresst.org

Summary: This report examines how schools can use assessment and accountability practices to improve education for all students. The authors draw on their work in Southern California schools to demonstrate how assessment can be used to help close the achievement gap in schools and districts with high percentages of economically disadvantaged or ethnically diverse students. Focusing on the results of work in two settings, the authors discuss their data-based inquiry process, including available data, initial empirical questions, and the process of inquiry. The authors conclude that schools must continue to build their capacity to respond effectively to the requirements of standards-based reform and propose specific areas to target.

Description: Knowledge

Capacities: Data Systems, Professional Development, Skills and Knowledge

Type: Website

Herman, J.L., & Winters, L. (1992). Tracking your school’s success: A guide to sensible evaluation. Newbury Park, CA: Corwin Press, Inc.

Summary: This book is a guide to help school administrators and teachers track school progress and use evaluation to address school challenges and problems. The authors explain the steps involved in conducting an evaluation, including how to focus the evaluation, identifying tracking strategies, managing instrument development, and collecting, scoring, and summarizing data. They also address how to analyze information, interpret and use findings, and continue monitoring progress. The authors include a sample school report card and sample worksheets to further illustrate evaluation techniques.

Description: Knowledge

Capacities: Analysis, Data Systems

Type: Book

Herman, J.L., Osmundson, E., Ayala, C., Schneider, S., & Timms, M. (2006, December). The nature and impact of teacher's formative assessment practices. (CSE Tech. Rep. No. 703). Los Angeles, CA: Center for the Study of Evaluation, National Center for Research on Evaluation, Standards, and Student Testing, University of California, Los Angeles. Retrieved May 15, 2008 from http://www.cse.ucla.edu/products/reports/R703.pdf

Summary: This paper presents findings from a study conducted to explore how teachers use formative assessment in the classroom, the impact on student learning, and factors that influence its effectiveness. The authors begin by presenting a conceptual model of quality assessment practice to illustrate that sound formative assessment must be comprised of quality assessment tools and strategic use of assessment information. Next, they describe the study methods including the science curriculum unit used as a context for the study, participants (i.e., nine middle school teachers), data collection (i.e., observations, interviews, teacher logs, student assessment data), and data analysis strategies. The authors describe the overall quality of formative assessment practices observed during the study and report much variability in the use of formative assessment. The authors conclude that formative assessment is an iterative process of collecting and using assessment data to inform feedback and guide future instruction, and they discuss potential avenues for future research.

Description: Research

Capacities: Multiple Measures, Professional Development, Professional Expertise, Skills and Knowledge, Tools

Type: Website

Hoff, D.J. (2006). Delving into data. Education Week, 25 (35), 12-22. Retrieved September 15, 2008, from http://search.ebscohost.com/login.aspx?direct=true&db=f5h&AN=20878229&site=ehost-live

Summary: This article highlights the importance of implementing digital data systems to help teachers, administrators, and school board officials organize and use all potential educational data to inform teaching and learning practices. The author recommends that states take advantage of available technology and make a variety of information (e.g., student attendance records, test score history, previous courses taken, etc.) available to teachers to better support data-driven decision making in the classroom. Although some states have begun to use available technology, Hoff recommends that systems must be further developed with additional student information and analysis tools so that teachers understand available data and can use this information to better support student learning. The author maintains that although adequate data exist, they must be made more accessible to teachers, administrators, and policy-makers so that these education professionals can use these data to foster optimal learning outcomes. (Permission to post article is pending. Link to publisher's website is provided.)

Description: Guidance

Capacities: Culture of Inquiry, Data Systems, Multi-level Data Use

Type: Website

Holcomb, E. L. (2001). Asking the right questions: techniques for collaboration and school change. Thousand Oaks, CA: Corwin Press, Inc.

Summary: This book offers real-word examples of collaborative efforts to bring about school change and provides a framework for large- and small-scale reform guided by five key questions: Where are we now? Where do we want to go? How will we get there? How will we know we are getting there? and How can we sustain the focus and momentum? The author also addresses current issues related to academic standards and accountability, recommendations for site-based management, and how districts and schools can interact to improve student performance.

Description: Guidance

Capacities: Culture of Collaboration, Leadership, Stakeholder Collaboration

Type: Book

Holcomb, E. L. (2004). Getting excited about data: Combining people, passion, and proof to maximize student achievement. Corwin Press; Second Edition.

Summary: The purpose of this book is to help educators learn to effectively use school data to improve learning and instruction in the classroom. The author describes the role of data in school improvement and change, and discusses barriers to data use in schools. Holcomb explores how to motivate educators to use data and how to create a culture of collective responsibility among internal and external personnel. The author also discusses the need for proof of success to stimulate progress, the importance of balanced assessments, how to effectively present data and interpret results, and how to use data to establish priorities. Examples, diagrams, and sample questions are included throughout to further illustrate concepts.

Description: Guidance

Capacities: Leadership, Professional Development, Skills and Knowledge, Tools

Type: Book

Honey, M. (2001). The consortium for technology in the preparation of teachers: Exploring the potential of handheld technology for preservice education. New York: EDC/Center for Children and Technology.

Summary:

Description: Knowledge

Capacities:

Type: Website

Honig, M. I., & Coburn, C. E. (2005). When districts use evidence to improve instruction: what do we know and where do we go from here? Voices in Urban Education, 6. Retrieved February 13, 2008 from http://www.annenberginstitute.org/VUE/archives.html.

Summary: This article reports on key findings from a review of 39 empirical studies examining the ways in which district personnel use evidence to inform their decision making. Important lessons emerged including: the use of evidence at the district level is common; evidence can be used for political rather than instructional purposes; prior knowledge impacts how evidence is used; administrators need time and support in using evidence to inform practice; and external support is often necessary to support the systematic use of evidence. Implications for practice are highlighted including opportunities to interpret evidence, increased access to relevant evidence, and the use of models to integrate evidence into daily practice.

Description: Knowledge

Capacities: Decision-Making, Leadership, Skills and Knowledge

Type: Website

Hostetler, K. (2005). What is "good" education research? Educational Research, 34(6), pp.16-21. Retrieved April 25, 2008 from http://edr.sagepub.com/cgi/content/refs/34/6/16.

Summary: This article discusses methodological as well as ethical considerations necessary for assessing the quality of educational research. The author contends that in addition to sound procedures, "good" research also encompasses beneficial aims and results. The case is made for the importance of connecting educational research findings to the ultimate goal of enhancing individual's well-being.

Description: Knowledge

Capacities: Professional Development, Skills and Knowledge

Type: Website

Huffman, D., & Kalnin, J. (2003). Collaborative inquiry to make data-based decisions in schools. Teaching and Teacher Education, 19, 569-580.

Summary: The purpose of this study was to explore the effects of a long-term collaborative inquiry project for teams of educators and parents. The teams included math and science teachers, principals and superintendants, curriculum specialists, and sometimes parents, who worked together to collect and analyze local data to make decisions about how to improve classroom instruction and student learning. The researchers found that the collaborative inquiry led to increased knowledge of teaching, improved teaching methods, changes to teaching philosophies, and connections with other educators. Furthermore, the school also benefitted as teachers began to consider and discuss common school-wide issues. The authors maintain that increased collaboration in schools is essential to improve teaching practices as well as student and school performance.

Description: Research

Capacities: Culture of Collaboration, Decision-Making

Type: Website

Ikemoto, G.S. & Marsh, J.A. (2007). Cutting through the "data-driven" mantra: Different conceptions of data-driven decision making. Yearbook of the National Society for the Study of Education, 106 (1), 105-131.

Summary: This chapter explores the increased emphasis on data-driven decision making (DDDM) and the ways in which educators use data to make informed decisions about teaching and learning. The authors explain the process of DDDM from data collection to implementation and find four general models ranging from simple to complex of DDDM strategies. The authors also highlight factors such as accessibility and timeliness of data, staff support, partnerships with external organizations, analysis tools, etc. that may support or challenge DDDM. They conclude with implications for policy and practice and recommend that policymakers take specific steps (e.g., make data accessible and timely, provide professional development opportunities for teachers to develop necessary data-related skills, etc.) to support educators in the DDDM process. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Guidance

Capacities: Decision-Making, Skills and Knowledge, Transformation Plan

Type: Website

Ingram, D., Louis, K.S., et al. (2004). Accountability policies and teacher decision making: Barriers to the use of data to improve practice. Teachers College Record, 106(6), 1258-1287.

Summary: This paper reviews findings from a longitudinal study of nine high schools nominated as leaders in Continuous Improvement (CI) that considers teacher willingness to use data to assess their own, as well as that of their colleagues and schools, effectiveness and ability to make improvements. It reveals inconsistencies between cultural assumptions and accountability policies, and barriers to the use of data in schools. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Culture of Collaboration, Culture of Inquiry, Expectations for Data Use

Type: Website

Innovation Network, Inc. Data collection tips: Developing a survey. Retrieved January 18, 2008 from the Innovation Network, Inc. website www.innonet.org.

Summary: This document provides an overview of some common issues that nonprofits face when constructing a survey and tips for structuring and administering surveys including: Asking only for the information that you need; Considering a survey's length in terms of time, not pages; Paying attention to how the survey is formatted; Using appropriate language; Making sure you give culturally appropriate options in your answer choices; Checking your assumptions; Avoiding biased questions; Asking questions one at a time; Steering clear of objectionable questions; Considering whether people have enough information to answer the question; And making sure your questions are specific.

Description: Guidance

Capacities: Professional Development, Skills and Knowledge, Tools

Type: Website

Jackl, A., & King, S. (2006). Optimal reference guide: What's behind your data warehouse? Part II. Austin, TX: ESP Solutions Group. Retrieved June 14, 2007 from http://www.espsolutionsgroup.com/resources.php

Summary: The second brief in this series on data warehousing details the design, structure, and configuration parameters of a data warehouse.

Description: Guidance

Capacities: Data Systems, Infrastructure

Type: Website

Jain, H., Nazareth, D., & Nelson, R. (2000). Data warehouses in public schools: Lessons learned in an on-going implementation. Americas Conference on Information Systems (AMCIS. Retrieved April 26, 2006 from http://scholar.google.com/scholar?hl=en&lr=&q=cache:QJ0ca9gw92MJ:is.lse.ac.uk/support/AMCIS/AMCIS2000/pdffiles/papers/026.pdf+Data+Warehouses+in+Public+Schools:+Lessons+Learned+in+an+On-Going+Implementation

Summary: This paper describes the process of creating and implementing a data warehouse in the Milwaukee Public Schools. (Permission to post article is pending. Link to conference website is provided.)

Description: Guidance

Capacities: Data Systems, Quality of Assessment Data, Tools

Type: Website

Jandris, T. P. (2002). Data-based decision-making: Essentials for principals. Alexandria, VA: National Association of Elementary School Principals.

Summary: This is a practical guide for school leaders who want to implement data-based decision-making in their organizations. The purpose of this guide is to help principals understand how to use data to improve learning in their schools. Information covered in this guide includes the "big picture" of assessment in the education system; implementation of a school-wide assessment program; the assessment "process," including reflecting on data and developing appropriate interventions; ways to use classroom assessments; the importance of using data in school reform; and the use of technology to improve the assessment process. This guide also includes a list of web resources and recommended books and articles. (Permission to post article is pending. Link to publisher's website is provided.)

Description: Guidance

Capacities: Culture of Collaboration, Culture of Inquiry, Skills and Knowledge

Type: Website

Jenkins, L. (1997). Improving student learning: Applying Deming's quality principles in classrooms. Milwaukee, WI: ASQC Quality Press. 978-0873895699

Summary: This book provides information on how educators can measure improvement in student learning, providing direct information on what to measure as well as examples and activities that can be used in the classroom. It provides numerous examples of statistical quality tools that have been successfully applied to K-12 classrooms.

Description: Guidance

Capacities: Data Systems, Professional Development, Skills and Knowledge, Tools

Type: Book

Johnson, R. S. (2002). Using data to close the achievement gap: How to measure equity in our schools. Thousand Oaks, CA: Corwin Press, Inc. 978-0761945093

Summary: This book provides helpful information on how educators, policy makers and parents can use data to set achievement goals and to measure school progress towards these goals. This book addresses the issue of using data to understand and solve the problem of disparate levels of achievement among different groups of students and includes many useful tools, templates and examples from schools.

Description: Knowledge

Capacities: Professional Development, Skills and Knowledge

Type: Book

Johnstone, C.J., Altman, J., & Thurlow, M. (2006). A state guide to the development of universally designed assessments. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Summary: This guide provides strategies to support states in designing and examining educational assessments to ensure accessibility for all users. The goal of the universal design approach to educational assessment is to create tests that accurately measure the skills and knowledge of a wide range of students, including those with disabilities and English language learners. Elements of universal design include precise constructs; clear and readable text, format, instructions, and visuals; respect for and inclusion of diverse test populations; and a test format that can be updated and/or changed without major consequences to measures. The authors contend that universal design considerations should be made from the beginning of test design through the entire assessment process.

Description: Guidance

Capacities: Quality of Assessment Data

Type: Website

Joint Committee on Testing Practices. (2005). Code of fair testing practices in education (Revised). Educational Measurement: Issues and Practice 24(1), 23-26. Retrieved January 15, 2007 from http://www.blackwell-synergy.com/doi/pdf/10.1111/j.1745-3992.2005.00004.x

Summary: The goal of the Joint Committee on Testing Practices and the revised Code of Fair Testing Practices in Education is to provide "a means by which professional organizations and test publishers can work together to improve the use of tests in assessment and appraisal" (p. 3). This code provides specific guidance for both educational test developers and test users in four areas: developing and selecting appropriate tests, administering and scoring tests, reporting and interpreting tests, and informing test takers.

Description: Tool

Capacities: Policies, Quality of Assessment Data

Type: PDF

Keeney, L. (1998). Using data for school improvement: Report on the second practitioners' conference for Annenberg challenge sites. Providence, RI: Annenberg Institute for School Reform.

Summary: This report has been produced as part of the Institute's efforts to share information and experiences about how schools are using data to improve their accountability practices and student performance. It describes some of the broad learnings from the colloquium, the Challenge sites in which the participating schools are located, and presents some of their issues related to data collection and analysis.

Description: Guide

Capacities: Expectations for Data Use

Type: PDF

Kerr, K. A., Marsh, J. A., ,Schulyer Ikemoto, G., Darilek, H., & Barney, H. (2006). Strategies to promote data use for instructional improvement: Actions, outcomes, and lessons from three urban districts. American Journal of Education 112(4), 496-520.

Summary: The researchers investigate strategies that are used by three urban school districts to encourage data use and their effects on school culture and educator and administrator practice. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities:

Type: Website

King, S. & Jackl, A. (2006). Optimal reference guide: What's really "in store" for your data warehouse? Part I. Austin, TX: ESP Solutions Group. Retrieved June 14, 2007 from http://www.espsolutionsgroup.com/resources.php

Summary: The paper introduces data warehousing by defining what it is and what it is not, and may help administrators answer whether or not a data warehouse is a necessary component of their overall education information system.

Description: Guidance

Capacities: Data Systems, Infrastructure

Type: Website

Kirkup, C., Sizmur, J., Sturman, L., & Lewis, K. (2005). Schools' use of data in teaching and learning. London: DfES (DfES RR 671).Retrieved September 7, 2007 from http://www.dfes.gov.uk/research/data/uploadfiles/RR671.pdf

Summary: This comprehensive report documents findings from a study conducted by The National Foundation for Educational Research (NFER) on effective data use among a sample of primary, secondary and special schools in England. The study identifies how data is used to promote learning across all types of schools; best practices and main challenges in the effective use of data; and recommendations for school staff, administrators, and policymakers. Key findings suggest that effective data use resulted from meaningful dialogue between staff and was supported by simple and accessible data systems. The report highlights two levels at which data use impacts teaching and learning: interventions targeted at individual students and school-wide approaches.

Description: Knowledge

Capacities: Action for Learning, Communication, Culture of Collaboration, Data Systems, Expectations for Data Use, Multi-level Data Use, Policies, Quality of Assessment Data, Reporting, Stakeholder Collaboration, Tools

Type: Website

Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119, 254-284.

Summary: The purpose of this meta-analysis is to review the findings related to Feedback Interventions (FIs), defined as "actions taken by (an) external agent(s) to provide information regarding some aspect(s) of one’s task performance" (p. 255). Results from the meta-analysis reveal many inconsistent findings in the effects of FIs on student performance, as researchers found that on average, FIs improve performance, yet in more than one third of the studies reviewed, FIs decreased performance. The researchers found that this variability is not attributable to sampling error, feedback sign, or existing theories used to guide the study. After moderator analyses, the researchers conclude that FI effects are moderated by the nature of the learning task, yet the specific properties of the task associated with difference in performance remain unclear. The researchers maintain that even when FIs result in improved performance, effects may be marginal, so evaluating costs and benefits of any FI intervention is essential. Kluger and DeNisi conclude that future research should include control groups as comparisons and should focus more on the process through which FIs lead to improved performance and the degree to which these effects are lasting or transitory. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Multiple Measures

Type: Website

Knapp, M. S., Copland, M. A., & Talbert, J. E. (2003, February). Leading for learning: Reflective tools for school and district leaders. Center for the Study of Teaching and Policy, University of Washington.

Summary: This article provides a framework of reflective ideas and tools to support education leaders' efforts to improve student learning. The framework highlights five areas of action for improvement and various pathways for enhancing student and professional learning. The ideas and tools concentrate on work related to instructional leadership, school reform and renewal, teacher learning and professional community, teacher leadership, organizational learning, policy-practice connections, and education in high-poverty, high-diversity settings.

Description: Guidance

Capacities: Leadership, Professional Development, Tools

Type: Website

Knapp, M. S., Swinnerton, J. A., Copland, M. A., & Monpas-Huber, J. (2006). Data-informed leadership in education. Center for the Study of Teaching and Policy, University of Washington. Retrieved March 2, 2007 from http://depts.washington.edu/ctpmail/PDFs/DataInformed-Nov1.pdf

Summary: This article synthesizes and interprets ideas, frameworks, beliefs, and activities regarding the use of data in educational decision making. The concept of data-informed leadership relates to the availability, quality and use of data among school leaders in order to improve teaching and learning. The broader focus on leadership, rather than just data-based decision making, captures a wide range of purposes data can serve for leaders. This article reviews common practices and emerging strategies that support leaders' use of data on the state, district, and school levels. Specifically, it highlights how data are used and what kinds of data are implied for specific types of leadership activities. The article concludes with noting unanswered questions that warrant further research and the enduring dilemmas in data-informed practice.

Description: Guidance

Capacities: Action for Learning, Expectations for Data Use, Leadership

Type: Website

Lachat, M. A., & Smith, S. (2005). Practices that support data use in urban high schools. Journal of Education for Students Placed at Risk (JESPAR) 10(3), 333-349. Retrieved February 22, 2007 from http://www.leaonline.com/doi/pdf/10.1207/s15327671espr1003_7

Summary: This article presents initial findings of a case study focusing on data use in five low-performing urban high schools undergoing comprehensive school-wide reform. Study findings include several factors that contribute to data use in these study sites: the quality and accuracy of data; staff access to timely data; the collaborative use of data organized around clear objectives; and leadership structures that support the use of data. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Culture of Collaboration, Leadership, Quality of Assessment Data

Type: Website

Lachat, M.A. (2001). Data-driven high school reform: The breaking ranks model. Providence, RI: LAB at Brown University.

Summary: This publication describes how schools can develop the capacity to analyze and use data as a core component of improving secondary schools. It builds on the concepts put forth in Breaking Ranks: Changing an American Institution and provides the reader with a context for how data can be used to support a school reform process. The seven chapters discuss the following topics: the expectations for a 21st century high school; student-centered accountability; resistance to using data-driven reform; the framework and key components of the Breaking Ranks Model; effective use of data in reform efforts; examples of data-driven reform among high school staff; and questions about student performance. The author concludes that the use of data can contribute to more sound decisions about curriculum, instruction, and assessment.

Description: Knowledge

Capacities: Action for Learning, Capacity Building, Culture of Collaboration, Decision-Making, Leadership, Multi-level Data Use, Policies, Stakeholder Collaboration

Type: Website

Laird, E. & Reyna, R. (August 2008). Data governance: Changing culture, breaking down silos and deciding who is in control. National Center for Educational Achievement, Data Quality Campaign.

Summary: The purpose of this article is to examine the issue of data governance and deciding who is responsible for what parts of a data system. The authors begin by discussing the shift in educational data systems from collecting aggregate data to a new emphasis on student-level longitudinal data and how the educational experience relates to various areas of child development over time. The authors argue that because of these changes, the role of data managers is being altered as well, thus necessitating clear and established governance guidelines to help educators at all levels understand who is responsible for each part of the data system. The authors maintain that incorporating clear data governance results in improved data quality, improved access to student-level data, and better alignment of data with school programs. The authors also note that elements such as an agency-wide data governance committee, a data governance director, a clear data architecture system to guide data collection and use, clear security policies regarding access to data, and a state audit system to evaluate data quality, validity, and reliability are essential to successful data management. The authors then present three separate case studies from Kansas, Kentucky, and Tennessee to illustrate specific data governance strategies. The authors conclude by providing various resources and reports that can be used to support better data governance.

Description: Knowledge

Capacities: Data Systems, Multi-level Data Use

Type: Website

Laird, E. (2008). Tapping into the power of longitudinal data: A guide for school leaders. National Center for Educational Accountability, Data Quality Campaign. http://www.dataqualitycampaign.org/files/publications-tapping_into_the_power_of_longitudinal_data-a_guide_for_school_leaders-010108.pdf

Summary: This article explores how school administrators and teachers can use a statewide data system to improve student outcomes. The article begins with a list of 10 essential elements for a comprehensive statewide longitudinal data system (e.g., a unique statewide identifier for each student, an audit system assessing data quality, validity, and reliability). Next, the authors review how teachers, administrators, and policymakers can use a high-quality data system to better support student learning and achievement, focusing on progress monitoring, diagnosis and prescription, internal and external benchmarking, predictive analysis, and evaluation. They emphasize the importance of analyzing growth in student academic performance over time to identify areas for improvement. They also describe how to use longitudinal data in conjunction with snapshot data to more accurately determine where intervention or remediation should take place (e.g., professional development for individual teachers, grade-level intervention, student-level tutoring or remediation, etc.). The article concludes with six action steps (e.g., participate in and provide professional development opportunities to help educators learn to use data to improve student performance) for school leaders working to build and use longitudinal data systems.

Description: Guide

Capacities: Data Systems, Multi-level Data Use

Type: Website

Lang, S., Stanley, T., & Moore, B. (2008). Short cycle assessment: Improving student achievement through formative assessment. Larchmont, NY: Eye on Education.

Summary: This book is designed to teach educators how to better support student achievement through frequent feedback on student class work and performance. The authors use a step-by-step approach to highlight various strategies used to monitor student progress and provide appropriate responses to their work. More specifically, the authors explain how to write good questions that assess student learning, design effective formative assessments, and administer short-cycle tests. The authors also demonstrate how to analyze and interpret data to inform instructional practices and how to adequately prepare students for large-scale, high-stakes tests. Various activities and forms are incorporated throughout to help educators through this process.

Description: Tool

Capacities: Analysis, Data Systems, Policies, Professional Expertise, Skills and Knowledge

Type: Book

LaRocque, M. (2007). Closing the achievement gap: The experience of a middle school. Clearing House: A Journal of Educational Strategies, Issues and Ideas, 80(4), 157-161.

Summary: This article describes the experience of a low-achieving Florida middle school and factors that contributed to its students' improved performance on statewide, high-stakes tests. A research team collected information (e.g., questionnaires, artifacts, etc.) from principals, teachers, parents, and community members. They found that improved school leadership, parent-community school involvement, data-driven decision making, and the celebration of diversity led to higher performance on the Florida Comprehensive Achievement Test, a test that is also used to grade overall school performance. The author concludes with recommendations for practice to help other schools develop similar aspects of school culture to promote student achievement. (Permission to post article is pending. Link to publisher's website is provided.)

Description: Research

Capacities: Communication, Culture of Collaboration, Infrastructure, Transformation Plan

Type: Website

Leahy,S., Lyon, C., Thompson, M., & Wiliam, D.(2005). Classroom assessment: Minute by minute, day by day. Educational Leadership. 63(3), 18-26. *Used by permission. Learn more about ASCD at www.ascd.org

Summary: This article explores ideas about classroom assessment to support and promote student learning. The authors argue for the need to shift from a quality control approach to a quality assurance approach (i.e., assessment for learning), and focus on how students can benefit from assessment practices. The authors present their work with teachers in elementary, middle and high school to highlight five universal classroom assessment strategies that have positive impacts on student learning. These strategies are:1) clarifying and sharing learning intentions and criteria for success;2) engineering effective classroom discussions, questions, and learning tasks;3) providing feedback that moves learners forward;4) activating students as the owners of their own learning;5) and activating students as instructional resources for one another. The authors conclude by describing the importance of using assessment to adapt instruction to meet individual student needs and providing suggestions to help teachers learn the basic principles of assessment for learning and them in their classrooms.

Description: Guidance

Capacities: Capacity Building, Culture of Collaboration, Culture of Inquiry, Multiple Measures, Professional Development, Professional Expertise

Type: PDF

Learning Point Associates & Educational Service Agency (ESA) Alliance of the Midwest. (2006, September). Effective use of electronic data systems: A readiness guide for district and school leaders. Naperville, IL: Learning Point Associates and Educational Service Agency (ESA) Alliance of the Midwest. Retrieved October 16, 2008 from http://www.centerforcsri.org/research/improvement.cgi?st=s&sr=SR005392

Summary: This guide is aimed at school and district leaders considering the adoption of an electronic data system to improve the use of data in their organization. The guide comprises three sections. The first section, tools for district and school leaders, highlights essential elements such as information quality, system quality, readiness to implement a data system and cost. The second section, tools for facilitators, includes a needs assessment for school and district leaders, information about choosing an appropriate system, and worksheets to assist with collaborative discussion and planning. The third section provides various resources to support data use and data systems.

Description: Guidance

Capacities: Capacity Building, Data Systems, Infrastructure

Type: Website

Learning Point Associates. (2004, December). Guide to using data in school improvement efforts: A compilation of knowledge from data retreats and data use at Learning Point Associates. Naperville, IL: Learning Point Associates. Retrieved April 12, 2006, from http://www.ncrel.org/datause/howto/guidebook.pdf

Summary: This guide is designed for educators who are beginning to learn how to use data in their school improvement planning process. It offers information on types of data, strategies for analyzing and understanding data, and methods for determining how these efforts can influence goals and planning. Four broad types of data are discussed: achievement data, demographic data, program data, and perception data.

Description: Knowledge

Capacities: Culture of Inquiry, Data Integrity, Expectations for Data Use

Type: Website

Learning Point Associates. Data use: Data primer. Retrieved January 18, 2008 from the North Central Regional Educational Laboratory website http://www.ncrel.org/datause/dataprimer/overview.php.

Summary: The Data Primer is organized around four modules. Each provides a practical question that educators can ask when developing school improvement plans: Module 1: Where are we? Module 2: Where do we want to go? Module 3: How fast are we moving and in what direction? Module 4: Are we leaving anyone behind? Each module contains three sections starting with the "tutorial" section that illustrates how the question might be answered, followed by the "practice" section that lets users apply their own data to graphing techniques introduced in the tutorial section, and lastly, the "going further" section that acts as a bridge between the sample scenario and actual tools, resources, and services that users can access and implement.

Description: Tool

Capacities: Professional Development, Skills and Knowledge, Tools

Type: Website

Learning Points Associates. (2006). Information systems for school improvement: Resources. Retrieved July 31, 2006 from http://www.learningpt.org/page.php?pageID=26

Summary: This website highlights information resources to support data-driven decision-making in schools and districts. From this website, schools and districts can access professional development programs such as data workshops, comprehensive district improvement audits, and the Survey of Enacted Curriculum (SEC).

Description: Guidance

Capacities: Action for Learning, Culture of Collaboration, Data Systems, Expectations for Data Use, Professional Development

Type: Website

Leavitt, P., McDaniel, R., & Skogstad, E. (2004). Disaggregating data in schools: Leveraging the information you have. Houston, TX: APQC.

Summary: This book helps teachers and other educational professionals better understand how to use student test data to improve teaching and learning in the classroom. The authors include chapters on why it is important to disaggregate data, types and sources of data and tests, breaking down data with technological support, understanding test data, and how data can be used at every level.

Description: Knowledge

Capacities: Multi-level Data Use, Professional Expertise, Skills and Knowledge

Type: Book

Lee, D., & Gavine, D. (2003). Goal-setting and self-assessment in Year 7 students. Educational Research, 45(1), 49-59.

Summary: (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description:

Capacities:

Type: Website

Leedy, P. D. & Ormrod, J. E. (2004). Practical research: Planning and design (8th edition). Upper Saddle River, New Jersey: Prentice Hall. 978-0131108950

Summary: This book provides a broad overview of basic research methodology and takes a hands-on approach to understanding the processes necessary for conducting quality research and generating meaningful results. Topics covered include an overview of the scientific method and research tools, methods for identifying the research problem and conducting a literature review, research design planning and proposal writing, qualitative, historical and descriptive research approaches, experimental and causal-comparative designs, statistical techniques for quantitative data analysis, and writing and publishing the research report.

Description: Knowledge

Capacities: Data Systems, Professional Development, Skills and Knowledge

Type: Book

Levine, E. (2002). Building a data warehouse: How you collect, manage, and report data may be the difference between success and failure. American School Board Journal. Retrieved March 2, 2007 from http://www.asbj.com/2002/11/1102technologyfocus2.html

Summary: This article explains the basics of data warehousing and steps educators and administrators can take to enhance data collection and reporting, as a means to greatly improve their chances of making data-driven decision-making a reality.

Description: Guidance

Capacities: Capacity Building, Data Quality Assurance, Infrastructure

Type: PDF

Linn, R. L, & Miller, D. M. (2005). Measurement and assessment in teaching (9th ed.). Upper Saddle River, N.J.: Prentice Hall.

Summary: This volume introduces prospective teachers to all of the elements of assessment that are essential to good instruction.

Description: Tool

Capacities: Data Quality Assurance, Professional Expertise, Skills and Knowledge

Type: Book

Linn, R. L., Baker, E. L., & Betebenner, D. W. (2002). Accountability systems: Implications of requirements of the No Child Left Behind Act of 2001. Educational Researcher, 31(6), 3-16.

Summary: In this article, the authors discuss the varying impact of NCLB on the content and stringency of state performance standards, test rigor, and implications for achieving adequate yearly progress (AYP) targets. The authors focus on the critical requirements of the law, the challenges for states and schools, and the vast differences between states' performances. They argue that unless NCLB requirements are made flexible, more schools will be categorized as "in need of improvement" than can be assisted effectively. The authors also argue that unless AYP targets take into account the volatility and variance at the school and state level, NCLB may create a sense of defeat and helplessness among organizations that face its unattainable goals. The authors conclude that while the purpose of NCLB to force state accountability is commendable, the future success of NCLB depends on the way in which it is interpreted and applied. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Expectations for Data Use, Policies, Standards

Type: Website

Locke, E., Shaw, K., Saari, L., Latham, G. (1981). Goal-setting and task performance: 1969-1980. Psychological Bulletin, 90, 125-152.

Summary: The purpose of this article is to review the research related to the effects of different types of goals on task performance as well as other factors influencing effectiveness of goal setting. The authors reviewed both laboratory and field studies (N = 110) and found that in the majority of studies, goals that were specific and challenging were more effective and led to higher student performance when compared to easy goals, "do your best" goals, or no goals. Furthermore, goal setting leads to improved task performance when students have the capacity to meet goals, feedback is given in relation to goal attainment, rewards are provided for reaching goals, participants are accepting of the goals, and the manager or experimenter is supportive. The researchers did not, however, find any consistent individual difference variables (besides academic ability) that impacted the effectiveness of goal setting. The researchers conclude that when structured appropriately, goals can help support improved performance in both the classroom and in professional settings. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Research

Capacities: Action for Learning, Multiple Measures

Type: Website

Loeb, S., Beteille, T. & Perez, M. (2008). Building an information system to support continuous improvement in California public schools. PACE Policy Brief 08-2. Retrieved September 25, 2008, from http://gse.berkeley.edu/research/pace/reports/PB.08-2.pdf

Summary: This policy brief explores three key elements of an effective education information system: systematically collecting data on students and education organizations; synthesizing and reporting data to stakeholders and independent evaluators; and implementing policies and programs aimed at improving instructional practices. The authors focus on data collection and maintain that good, quality data are essential to answering key questions about student learning and performance in the classroom. They note that education leaders need various data from the classroom, school, district and state level to accurately evaluate schools and make data-driven policy decisions. The authors argue that California is behind other states in terms of data collection and must improve these efforts to develop an effective information system to better support educational gains.

Description: Knowledge

Capacities: Culture of Collaboration, Data Systems, Multi-level Data Use

Type: Website

Looking at Student Work. Retreived January 18, 2008 from the Looking at Student Work website www.lasw.org.

Summary: This website includes numerous resources to assist educators in looking at student work, emphasizing collaboration amongst teachers in examining student work; focusing on small samples of student work; reflecting on important questions about teaching and learning; and using protocols for looking at and discussing student work.

Description: Guidance

Capacities: Culture of Collaboration, Professional Development, Skills and Knowledge, Tools

Type: Website

Love, N. (2002). Using data/getting results: A practical guide for school improvement in mathematics and science. Norwood, MA: Christopher-Gordon Publishers, Inc.

Summary: Chapter 2 (Data and Inquiry: Turning the Wheels of Reform) discusses the processes and principles of collaborative inquiry to help teachers, schools, and districts solve their problems. The processes include framing a question to investigate, collecting and analyzing data, drawing conclusions, and sharing results. Various phases of inquiry are also discussed, including organizing data-driven dialogue; common obstacles that schools and districts face in using inquiry and data; and strategies to overcome them. Chapter 3 (Keep Your Eyes on the Prize: Student Learning) discusses a process for improving student learning in mathematics and science by 'keeping your eyes on the prize.' The process is an inquiry cycle that starts with mobilizing schools communities around a common vision for student learning in mathematics and science. The next steps involve collecting and analyzing student learning data, formulating a learner-centered problem, setting measurable student learning goals, planning for and taking action to improve the system, and monitoring results. Based on feedback about results, the team makes changes, uncovers new problems, and continues the improvement cycle.

Description: Tool

Capacities: Analysis, Data Integrity, Multi-level Data Use, Multiple Measures, Professional Development, Professional Expertise, Quality of Assessment Data, Reporting, Skills and Knowledge, Tools, Transformation Plan

Type: Book

Love, N. (2004). Taking data to new depths. Journal of Staff Development 25(4), 22-26. Retrieved May 8, 2007 from http://www.nsdc.org/library/publications/jsd/love254.pdf

Summary: The purpose of this article is to discuss the importance of educators knowing how to use data effectively. Love maintains that despite having a wealth of data available, many teachers are not trained to work with or understand school data, often leading to superficial and ineffective analysis. She discusses The Using Data Project, funded by the National Science Foundation (NSF), which helps math and science educators develop the ability to examine, understand, and analyze data and then draw appropriate inferences from the results. Love describes how select teachers attend a series of workshops to become “data facilitators” where they learn to understand, examine and interpret data. The data facilitators then bring their new skills to their schools leading data teams through analysis, supporting team collaboration, and showing how findings can be used to improve school and student performance. Love maintains that The Using Data Project is an effective way to facilitate data use, engage teachers in discussions about data, and build a culture of data use at the school level.

Description: Guidance

Capacities: Professional Development

Type: Website

Love, N., Stile, K., Mundry, S., & DiRanna, K. (2007). The data coach's guide to improving learning for all students: Unleashing the power of collaborative inquiry. Thousand Oaks, CA: Corwin Press.

Summary: This book is a guide for developing a school-wide culture of inquiry and collaboration in regards to student test data, and using test data to improve learning and instruction in the classroom. The authors explain the importance of collaborative inquiry and how to organize and build a foundation within the school to commit to shared goals and ideas. They also take the reader through the process of identifying student learning problems, verifying causes, generating solutions, taking action, and monitoring results. These strategies are designed to help teachers and administrators better meet the needs of all students and support continued progress in student learning and achievement. The accompanying CD-ROM provides templates, handouts, Power Point slides, resources, and sample goals and agendas.

Description: Guidance

Capacities: Culture of Collaboration, Expectations for Data Use, Skills and Knowledge

Type: Book

Love, N., Stiles, K.E., Mundry, S., & DiRanna, K. (2008). Passion and principle: Ground effective data use. Journal of Staff Development, 29(4), 10-14.

Summary: In this article, the authors present six "foundational assumptions", or values associated with ethical data use. These assumptions include that educators are responsible for the learning of every child (regardless of class, culture, or capabilities), data are only given meaning through interpretation, and collaborative inquiry is an important way for teachers to share expertise. Love and colleagues also maintain that while data can be used to highlight areas of weakness, improved teaching practices will only result from discussion and improvements to curriculum, instruction, assessment, and professional development practices. Lastly, the authors argue that all school personnel must take collective responsibility for student learning and that distributed leadership practices will help ensure that staff members have a specific role in promoting academic excellence. Love and colleagues maintain that these values can be used to help educators learn to effectively use data to improve the learning and performance of all students.

Description: Guidance

Capacities: Expectations for Data Use

Type: Website

Mandinach, E. B., Honey, M., & Light, D. (2006). A theoretical framework for data-driven decision making. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA. Retrieved April 6, 2007 from http://cct.edc.org/admin/publications/speeches/DataFrame_AERA06.pdf

Summary: This paper builds on a project sponsored by the National Science Foundation to explore and create an evaluative framework for data-driven decision making. The conceptual model focuses on the classroom, school, and district levels of decision making, and how the use of technology-based tools can facilitate decisions among stakeholders.

Description: Knowledge

Capacities: Culture of Collaboration, Data Systems, Decision-Making, Expectations for Data Use, Infrastructure, Leadership, Reciprocal Feedback, Skills and Knowledge, Tools

Type: Website

Mandinach, E. B., Honey, M., Light, D., Heinze, C., & Rivas, L. (2005). Creating an evaluation framework for data-driven decision-making. EDC Center for Children and Technology. Retrieved May 8, 2007 from http://www.iste.org/Content/NavigationMenu/Research/NECC_Research_Paper_Archives/NECC_2005/Mandinach-Ellen-NECC05.pdf

Summary: This paper outlines a project from the Education Development Center's (EDC) Center for Children and Technology that examines how technology-based tools can facilitate decision-making by exploring the implementation and impact of data tools at different levels of the school system. The purpose of the project is two-fold: to understand how schools use data tools to make evidence-based decisions about assessment and instruction, and to develop an evaluation framework to examine the phenomena that inform the field of data use in schools. The authors briefly describe the main areas of the study, including project tools; research sites and types of data collection; development of methodological, conceptual, and structural functionality frameworks; and the affordances of technology. The results of the study outlined in this paper are examined in "The Impact of Data-Driven Decision Making tools on Educational Practice: A Systems Analysis of Six School Districts" (Mandinach, et al., 2006).

Description: Knowledge

Capacities: Data Systems, Decision-Making, Infrastructure, Tools

Type: Website

Mandinach, E. B., Rivas, L., Light, D., Heinze, C., & Honey, M. (2006). The impact of data-driven decision making tools on educational practice: A systems analysis of six school districts. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA. Retrieved April 6, 2007 from http://cct.edc.org/admin/publications/speeches/Data_AERA06.pdf

Summary: In this study of data use in six school districts, the authors "examine technology-based, data-driven decision making tools, their implementation, and impact on different levels of school systems" (p. 3). The purpose of this examination is to develop and validate an evaluation framework that captures the complex factors that influence the structure and function of schools and districts. The authors present theoretical and actual frameworks from the six districts on how data-driven decision making occurs in school systems, and profile efforts in the Tucson Unified School District to implement a data warehouse and information management system.

Description: Knowledge

Capacities: Decision-Making, Infrastructure, Tools

Type: Website

Mandinach, E.B., & Honey, M. (Eds.) (2008). Data-driven school improvement: Linking data and learning. New York: Teachers College Press.

Summary: In this book, the editors bring together 14 articles, based on results of a three-year project sponsored by the National Science Foundation, which examine data use in schools, and describe the ways in which educators use achievement data and technology to support and improve student learning. The book is divided into sections focusing on the conceptual frameworks foundations for data-driven decision-making, the role of data-driven decision-making and change in action, and digital resources or technology used to support these kinds of decision-making strategies. It includes chapters from a variety of researchers in the field who are experienced in linking research to practice, and overall, provides a comprehensive view of data-drive decision-making in schools.

Description: Knowledge

Capacities: Action for Learning, Decision-Making, Skills and Knowledge

Type: Book

Marsh, J. A., Pane, J. F., & Hamilton, L. S. (2006). Making sense of data-driven decision making in education. RAND Corporation. Retrieved December 14, 2006 from http://www.rand.org/pubs/occasional_papers/2006/RAND_OP170.pdf

Summary: The authors synthesize findings from four RAND studies on data use in education to examine how data is used to inform decisions in schools and districts. The following research questions guided the examination: (1) What types of data are administrators and teachers using?; (2) How are administrators and teachers using these data?; (3) What kinds of support are available to help with data use?; and (4) what factors influence the use of data for decision making? The authors used various data from the four projects including surveys, interviews, focus groups, observations, and school/district documents. Among other findings, the authors reported that of all the potential data sources, achievement test scores receive the most attention, yet test results were often received too late to make adjustments during the current school year. The authors also found that educators frequently used test scores to set improvement goals and monitor school or district performance. Workshops on how to interpret test data were the most common data-driven decision making support available to school staff, yet most teachers and administrators did not find these training sessions helpful. Lastly, the authors found that accessibility of data, quality of data, motivation to use data, timeliness of data, staff capacity and support, curriculum pacing pressures, lack of time, organizational culture and leadership, and history of state accountability all influenced the frequency and level of proficiency with which educators use data. The authors conclude with directions for future research.

Description: Research

Capacities: Decision-Making, Multi-level Data Use

Type: Website

Marsh, J., Kerr, K., Ikemoto, G., Darilek, H., Suttorp, M.J., Zimmer, R., et al. (2005). The role of districts in fostering instructional improvement: Lessons from three urban districts partnered with the Institute for Learning. MG-361-WFHF. Santa Monica, CA: RAND Corporation. Retrieved November 5, 2007, from http://ww.rand.org/pubs/mongraphs/MG361/

Summary: This book documents findings from a study of three urban districts' efforts to improve instructional quality and school performance. The strategies districts employed to improve teaching and learning are divided into four areas: promoting principals' instructional leadership; supporting teachers' professional learning; specifying curriculum; and promoting data-based decision-making for planning and instructional improvement. For each area the authors identify common factors the affected the districts' success, asses the impact of the districts' partnership with the intermediary organization, the Institute for Learning (IFL), and make recommendations for districts carrying out similar instructional reforms.

Description: Knowledge

Capacities: Action for Learning, Communication, Culture of Collaboration, Leadership, Policies

Type: Website

Mason, S. (2002). Turning data into knowledge: Lessons from six Milwaukee public schools. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA. Retrieved April 10, 2006, from http://www.wcer.wisc.edu/archive/mps/AERA2002/Mason%20AERA%202002%20QSP%20Symposium%20Paper.pdf

Summary: This paper reviews a two-year project designed to increase the capacity of six Milwaukee Public Schools to use student, classroom, and school data in their decision making processes. The researchers learned that data must become an active part of school planning and improvement strategies, and integrated and accepted into the school culture and organization. The challenges that schools confront as they build their capacity for data-driven decision-making include: cultivating the desire to transform data into knowledge, focusing on the process of data use, committing to the creation and acquisition of data, organizing systems to manage and analyze data, and applying results to improve student achievement.

Description: Knowledge

Capacities: Analysis, Capacity Building, Culture of Inquiry, Data Systems, Expectations for Data Use, Infrastructure, Professional Development

Type: Website

Massell, D. (2000). The district role in building capacity: Four strategies. (CPRE Policy Brief No. RB-32). Philadelphia: Consortium for Policy Research in Education, University of Pennsylvania. Retrieved October 27, 2008 from http://www.cpre.org/images/stories/cpre_pdfs/rb32.pdf

Summary: This policy brief is part of the larger project, "Education Reform Policy: From Congress to the Classroom," which is designed to examine how various policies foster cohesion and reform in the educational system. In their research of 22 districts in various states, the authors observed four key capacity-building strategies: 1) interpreting and using data (e.g., using test data to evaluate student progress and improve teaching); 2) building teacher knowledge and skills (e.g., expanding professional development opportunities, using on-site and continuous workshops, including teachers as professional development leaders); 3) aligning curriculum and instruction to policies and practice; and 4) targeting interventions on low-performing schools and students. Additionally, the authors discuss the challenges associated with these approaches, including the need to help teachers understand how to select and effectively use relevant data to inform teaching practices, and to coordinate professional development opportunities with district goals and challenges while still meeting individual teacher needs.

Description: Research

Capacities: Culture of Inquiry, Expectations for Data Use, Skills and Knowledge

Type: Website

Massell, D. (2001). The theory and practice of using data to build capacity: State and local strategies and their effects. In Susan H. Fuhrman (Ed.), From the Capitol to the Classroom: Standards-Based Reform in the States. Chicago: University of Chicago Press.

Summary:

Description:

Capacities:

Type: Book

McDonald, B. & Boud, D. (2003). The impact of self-assessment on achievement: The effects of self-assessment training on performance in external examinations. Assessment in Education, 10(2), 209-220.

Summary: The purpose of this study was to explore whether high school students trained to use self-assessment practices demonstrate improved performance on external examinations. The researchers selected students (N = 504) from 10 high schools in Barbados representing high, middle, and low levels of overall student achievement. First, teachers at each school were trained in how to support and foster students' self-assessment skills, and then these teachers trained students in self-assessment practices as part of their regular classroom curriculum. The experimental group (n = 256) was explicitly trained in self-assessment skills over the entire school year by evaluating their own work using specific criteria, monitoring progress towards learning goals, and making informed decisions about how to best further their own learning. Although this training initially took place exclusively in the classroom, students were frequently encouraged to use these skills in a variety of contexts, in and out of the classroom. Students reported generally positive reactions to the training. Student performance on the Caribbean Examinations Council examination, which assesses student knowledge in business studies, humanities, science, and technical studies, was compared with a matched control group (n = 259) to determine program effects. The researchers found that the experimental group had significantly higher mean scores in all curricular areas and in overall performance when compared to the control group, thus lending support for the notion that self-assessment practices lead to improved performance on externally mandated tests. The researchers conclude that training students in self-assessment practices may contribute to improved learning outcomes, though they caution that similar results may not be seen in more challenging or less favorable academic settings, highlighting the importance of replication of study results in different countries and contexts. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Research

Capacities: Action for Learning

Type: Website

McDonald, S., Andal, J., Brown, K., & Schneider, B. (2007). Getting the evidence for evidence-based initiatives: How the Midwest states use data systems to improve education processes and outcomes (Issues & Answers Report, REL 2007–No. 016). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Midwest. Retrieved from http://ies.ed.gov/ncee/edlabs.

Summary: The purpose of this report is to describe the data use practices of states in the Midwest Region (i.e., Illinois, Indiana, Iowa, Michigan, Minnesota, Ohio, and Wisconsin) to demonstrate how to use data systems to improve educational outcomes. The authors maintain that these states are developing innovative approaches for collecting and using data for educational decision making. These practices include establishing longitudinal student-level and teacher-level data collections and connecting data across the educational data system. These states, however, also experience challenges in meeting their data use goals, as data management staff are limited and sometimes inexperienced, and federal and state regulations may not permit key data to be collected. The authors conclude that regional benchmarks should be established to provide a framework for states working towards better data use, and that districts should also attend to specific state requests for various resources and create related training materials when needed. Individual chapters describing each state's exemplary practices are included.

Description: Knowledge

Capacities: Data Systems

Type: Website

McIntire, T. (2004). Eight buying tips: Data warehouses. Technology & Learning 25(1). Retrieved April 24, 2007 from http://www.techlearning.com/showArticle.php?articleID=26806926

Summary: This article presents eight tips for schools, districts, and states considering the purchase of a data warehouse. The author recommends that organizations question whether the warehouse allows integration of data from all relevant systems, if the data model fits all system needs, what data domains (e.g., student demographics, school enrollment, standardized assessment results, etc.) will be included and supported, and how often the system will need to be updated. McIntire also advises potential buyers to consider staff access and available support for using the data warehouse, the length of time the provider will take to implement the system, and the vendor's experience working with K-12 organizations. The author includes a "buyer’s checklist" noting seven attributes a data warehouse should offer and provides the contact information for various companies offering data management tools and services. McIntire concludes with recommended readings to learn more about data collection and management.

Description: Guidance

Capacities: Data Systems

Type: Website

McMillan, J. H. (2001). Essential assessment concepts for teachers and administrators. Thousand Oaks, CA: Corwin Press, Inc. 978-0803968400

Summary: The purpose of this book is to explore how to use assessment practices to improve student achievement and meet educational goals. McMillian describes how to integrate assessment with teaching and learning, and explores the concepts of reliability, validity, and fairness in testing. The author also includes chapters explaining how to understand and use numerical assessment data and how to interpret and use national and state standardized assessments. This book is designed for teachers, administrators, and any other school personnel who hope to better understand assessment and how to make assessment-related decisions.

Description: Knowledge

Capacities: Balanced Assess't System, Professional Development, Skills and Knowledge

Type: Book

McMillan, J.H. (Ed.). (2008). Formative classroom assessment: Theory into practice. New York: Teachers College Columbia University.

Summary: This book comprises a collection of articles exploring formative assessment theory and practice related to teaching in K-12 classrooms. Essay topics include the reasons to use formative assessment practices to improve student learning, theoretical foundations and a review of relevant literature, and Benjamin S. Bloom's work contributing to formative assessment research, theory, and practice. The book also includes articles discussing implications of high-stakes testing for formative classroom assessment, large-scale assessments and implications for research, theory, and practice, and formative classroom assessment in specific subjects including Science, Mathematics, English, the Humanities, and Social Sciences. Essays are written by well-known experts in the field.

Description: Knowledge

Capacities: Multiple Measures

Type: Book

McTighe, J., & O'Connor, K. (2005). Seven practices for effective learning. Educational Leadership. 63(3),10-17. Retrieved May 15, 2008 from http://woodard.latech.edu/~pleonard/epas_report_2005_06/products/additional_resources/jan_28_wkshp/seven_practices.pdf

Summary: The purpose of this article is to present specific assessment and grading practices through which teachers can enhance student learning. The authors argue that students are more likely to be motivated to learn when the task is clear, the learning goals are relevant, and students believe they can be successful. Using these factors, the authors explore seven practices for effective learning: using summative assessments to frame meaningful performance goals, showing criteria and models in advance, assessing before teaching, providing feedback early and often, encouraging self-assessment and goal setting. The authors maintain that by using these strategies in summative, diagnostic and formative assessments, teachers can improve teaching and learning in the classroom.

Description: Guidance

Capacities: Analysis, Decision-Making, Multiple Measures, Professional Development, Professional Expertise, Skills and Knowledge

Type: Website

Means, B., Padilla, C., DeBarger, A., & Bakia, M. (2009). Implementing data-informed decision making in schools: Teacher access, supports, and use. Report prepared for U.S. Department of Education, Office of Planning, Evaluation and Policy Development. Prepared by SRI International, Menlo Park, CA.

Summary: This interim report presents early findings from the National Study of Education Data Systems and Decision Making, sponsored by the U.S. Department of Education. The purpose of this study is to determine the availability of education data systems and the prevalence of organizational supports for data system use. Study goals also include examining how educators are using data systems, whether they know how to interpret student data, and how they use data to inform instruction. This interim report includes case study findings from the first group of site visits which included interviews with teachers, principals, and district staff members as well as tasks to examine teachers' understanding of student data. This report also includes district and teacher survey data collected in spring 2007 from the U.S. Department of Education's National Educational Technology Trends Study. Thus far, the researchers have found that teachers have experienced greater access to student data systems, yet district and school use of data systems focuses more on accountability and curriculum alignment with state standards. Case studies revealed that teachers are using some student data to inform instruction, yet these data are not usually included in statewide data systems. Furthermore, including classroom and state assessment data in one integrated system is rare. These early results as well as more specific findings are discussed in detail.

Description: Knowledge

Capacities: Capacity Building, Data Systems, Human Resources, Multi-level Data Use, Policies, Skills and Knowledge, Tools

Type: Website

Mercurius, N. (2005, October). Scrubbing data for D3M. T.H.E. Journal. Retrieved April 26, 2006 from http://thejournal.com/articles/17433

Summary: In this article, the author discusses the techniques of mopping and scrubbing data in order to prevent some common pitfalls of data use, and ensure data quality for effective data-driven decision-making (DDDM) in educational organizations. Mopping is "the process of locating or identifying where and how data are stored" (p. 2). The information from this procedure helps data personnel develop data plans to address the data collection and analysis needs in their organizations. Scrubbing is the process of removing incorrect information from data sets that is caused by duplication, or inaccurate or incomplete inputting, to guarantee the quality of data and accuracy of data analysis. The author emphasizes that scrubbing is the responsibility of all individuals responsible for DDDM in an organization and that "attention must be paid by all participants in the collection process to the procedures in place at each school site for the standardization of data entry" (p. 5).

Description: Guidance

Capacities: Data Integrity, Data Systems

Type: Website

Mertens, D. M. (2005). Research and evaluation in education and psychology: Integrating diversity with quantitative, and mixed methods, 2nd edition. Thousand Oaks, CA: Sage Publications. 978-0761928058

Summary: This book provides foundational information on different aspects of research. Chapters 4 -10 explain a variety of approaches to systematic inquiry including experimental and quasi-experimental research, causal comparative and correlational research, survey research, qualitative methods, and mixed-methods designs. Chapters 11-13 cover the topics of sampling, data collection, and data analysis, interpretation, and reporting. There is an appendix with detailed information on how to write a research proposal. Each chapter begins with an outline of the main ideas, and includes examples from specific research studies and step-by-step processes for conducting specifc types of research. Each chapter ends with a set of questions for critically analyzing the topic covered as well as a set of questions and activities for discussion and application.

Description: Knowledge

Capacities: Professional Development, Skills and Knowledge

Type: Book

Mertler, C. A. (2007, December). Interpreting standardized test scores: Strategies for data-driven instruction decision making. California: Sage Publications.

Summary: The purpose of this book is to help teachers and administrators better understand standardized tests and test scores. The book supports teachers in data-drive decision making by helping them learn to use standardized test results to guide instruction and meet student needs. The author includes specific examples to demonstrate how educators use test results to plan and revise instruction as well as interviews with teachers and administrators who have successfully used student test scores to make decisions about teaching and learning in the classroom. Overall, this guide helps educators learn to interpret, use, and reflect on test data to improve learning and instruction in schools.

Description: Guidance

Capacities: Decision-Making, Multi-level Data Use, Multiple Measures

Type:

Mieles, T. & Foley, E. (2005). Data warehousing: Preliminary findings from a study of implementing districts. Philadelphia: Annenberg Institute for School Reform. Retrieved April 12, 2006 from http://annenberginstitute.org/EKF/images/DataWarehousing.pdf

Summary: This paper reviews a study of the strategies and challenges of data warehousing for school improvement.

Description: Knowledge

Capacities: Culture of Collaboration, Culture of Inquiry, Data Systems

Type: Website

Mieles, T., & Foley, E. (2005). From data to decisions: Lessons from school districts using data warehousing. Providence, RI: Annenberg.

Summary: This study, conducted in 2004 by the Annenberg Institute for School Reform, looks at eight school districts that have successfully implemented data warehousing technology. Data were gathered through interviews with educators and education technology experts at each district. This report provides information on the key features of a data warehouse, highlights the successes and challenges of data warehousing, documents a set of lessons learned from the experiences of the eight districts, and provides a list of useful resources for school leaders interested in knowledge management tools. The report provides practical information to school leaders and districts interested in implementing their own data warehouse. This study promotes data warehousing as an effective way for school leaders to integrate, analyze, and measure data for knowledge management and policy and decision making purposes.

Description: Tool

Capacities: Action for Learning, Analysis, Data Quality Assurance, Decision-Making, Infrastructure, Leadership, Multi-level Data Use

Type:

Mills, L. B. (2008, March). Getting started with data warehousing: The first in a series on how to manage data efficiently. Technology & Learning, 28(8), 31.

Summary: This brief article describes issues to consider when implementing a data warehouse. The author provides suggestions about how to begin a data warehouse project, such as assessing the quality of existing data and reviewing any existing database systems. The author also includes issues to consider (e.g., cost, functionality) when choosing a data warehouse, and concludes by providing resources/websites for additional information.

Description: Guidance

Capacities: Data Systems, Priorities

Type: Website

Minstrell, J. (1989). Teaching science for understanding. In L. B. Resnick & L. E. Klopfer (Eds.), Toward the thinking curriculum: Current cognitive research (pp. 129-149). Alexandria, VA: Association for Supervision and Curriculum Development.

Summary:

Description: Guidance

Capacities: Multiple Measures, Professional Development, Professional Expertise, Skills and Knowledge

Type:

Moss, P. (2003, Winter). Reconceptualizing validity for classroom assessment. Educational Measurement: Issues and Practice, 22(4), 13-25.

Summary: This article explores the shortcomings of conventional validity theory for guiding classroom assessment practices and provides alternative theoretical frameworks to add to conventional theory. The author reflects on her own experience using classroom assessment practices in a graduate research methods course to illustrate these ideas. The author further discusses the importance of exploring assessment practices in context and drawing from real life examples to inform the development and evaluation of validity theory. She concludes with implications for teaching and research and presents ways that researchers and those with measurement/testing expertise can be more helpful to classroom teachers. (Permission to post article is pending. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Multiple Measures, Professional Expertise, Skills and Knowledge, Tools

Type: Website

National Center for Education Statistics. (2001, January). Technology @ your fingertips, version 2.0. Retrieved April 10, 2006 from http://nces.ed.gov/pubs98/98293rev.pdf

Summary: This guidebook was designed to help state education agencies, school districts, and schools develop and maintain informative and effective web sites. It includes general information about the role of a web site in education and more detailed information regarding web-related security, legal issues, and privacy protection. The guidebook also explores resources for web development, such as training and professional development and purchasing software to host a web site. Chapters 1 through 5 are written in non-technical language and are primarily for administrators, while chapter 6 is written for network personnel who are responsible for ensuring network security. The authors believe the coupling of these two sections helps address the needs of both nontechnical policymakers and network administrators and provides a universal view and understanding of internet use in education and developing web sites for schools.

Description: Guidance

Capacities: Data Systems, Expectations for Data Use, Human Resources

Type: Website

National Center for Education Statistics. NCES handbooks online. Retrieved May 19, 2006 from B191

Summary: The NCES Data Handbooks provide guidance on consistency in data definitions and maintenance for education data, so that such data can be accurately aggregated and analyzed.

Description: Tool

Capacities: Data Quality Assurance, Quality of Assessment Data, Analysis

Type: Website

National Forum on Education Statistics (2002). Safety in numbers: Collecting and using crime, violence, and discipline incident data to make a difference in schools (NCES 2002-312). U.S. Department of Education, National Center for Education Statistics. Washington, DC: U.S. Government Printing Office. Retrieved September 30, 2008, from http://nces.ed.gov/pubs2002/2002312.pdf

Summary: This handbook is intended to help school staff and district office personnel collect and utilize student discipline data. It includes detailed information about what kinds of data to collect, why these kinds of data are important, and how they can be used to help promote school safety. The handbook is a result of contributions from state and school district administrators, education researchers, and Federal Department of Education program and research staff. These education professionals maintain that a safe and organized learning environment is essential to student learning. The handbook is an updated version of the Forum’s 1996 “Recommendations of the Crime, Violence, and Discipline Reporting Task Force” and is adaptable to local school district needs.

Description: Guidance

Capacities: Culture of Inquiry, Data Systems, Multi-level Data Use, Skills and Knowledge

Type: Website

National Forum on Education Statistics (2003). Weaving a secure web around education: A guide to technology standards and security (NCES 2003- 381). U.S. Department of Education, National Center for Education Statistics. Washington, DC: U.S. Government Printing Office. Retrieved September 30, 2008, from http://nces.ed.gov/pubs2003/2003381.pdf

Summary: This guidebook was designed to help state education agencies, school districts, and schools develop and maintain informative and effective web sites. It includes general information about the role of a web site in education and more detailed information regarding web-related security, legal issues, and privacy protection. The guidebook also explores resources for web development, such as training and professional development and purchasing software to host a web site. Chapters 1 through 5 are written in non-technical language and are primarily for administrators, while chapter 6 is written for network personnel who are responsible for ensuring network security. The authors believe the coupling of these two sections helps address the needs of both nontechnical policymakers and network administrators and provides a universal view and understanding of internet use in education and developing web sites for schools.

Description: Guidance

Capacities: Culture of Collaboration, Data Systems, Tools

Type: Website

National Forum on Education Statistics. (2004). Forum guide to building a culture of quality data: A school & district resource (NFES 2005-801). U.S. Department of Education. Washington, DC: National Center for Education Statistics. Retrieved December 1, 2006 from http://nces.ed.gov/pubs2005/2005801.pdf

Summary: The National Forum on Education Statistics developed this guide to help schools and school districts improve the quality of data they collect and to provide processes for ensuring quality data, focusing on data entry. It is key that all staff understand how the data will be used and work collaboratively in order to ensure data quality.

Description: Guidance

Capacities: Culture of Collaboration, Culture of Inquiry, Stakeholder Collaboration

Type: Website

National Forum on Education Statistics. (2004). Forum guide to protecting the privacy of student information: State and local education agencies, (NCES 2004-330). Washington, DC: National Center for Education Statistics. Retrieved December 1, 2006 from http://nces.ed.gov/pubs2004/2004330.pdf

Summary: The purpose of this document is to help state and local education agencies develop policies and procedures to protect student information, while ensuring school officials have adequate information to make management and instructional decisions.

Description: Tool

Capacities: Data Systems, Infrastructure

Type: Website

National Forum on Education Statistics. (2005). Forum guide to education indicators. U.S. Department of Education. Washington, DC: National Center for Education Statistics. Retrieved December 1, 2006 from http://nces.ed.gov/pubs2005/2005802.pdf

Summary: Provides encyclopedia-type entries for 44 commonly used education indicators. Each indicator entry contains a definition, recommended uses, usage caveats and cautions, related policy questions, data element components, a formula, commonly reported subgroups, and display suggestions. The document will help readers better understand how to appropriately develop, apply, and interpret commonly used education indicators.

Description: Guidance

Capacities: Communication, Policies, Reporting

Type: Website

National Research Council. (2001). Knowing what students know: The science and design of educational assessment. Committee on the Foundations of Assessment. Pellegrino, J., Chudowsky, N., & Glaser, R. (Eds.). Board on Testing and Assessment, Center for Education. Division on Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.

Summary: In this book, the Committee on the Foundations of Assessment synthesizes current research in cognitive and measurement science, and explores their implications for current and future approaches to educational assessment in both classroom and large-scale settings, for the purpose of assisting learning, measuring achievement, and evaluating educational programs. The authors emphasize the central role of a model of cognition and learning for good assessment design, and opportunities for advancing assessment through information technologies. They also advocate a series of research, policy, and practice recommendations grounded in two guidelines: increased multidisciplinary collaboration around issues of assessment, and further discussion about the use of assessment in education based on the concepts put forth in book.

Description:

Capacities: Action for Learning, Analysis, Balanced Assess't System, Capacity Building, Culture of Inquiry, Human Resources, Multi-level Data Use, Multiple Measures, Professional Development, Professional Expertise, Skills and Knowledge, Standards

Type: Book

Natriello, G. (1987). The impact of evaluation processes on students. Educational Psychologist, 22, 155-175.

Summary: This review article synthesizes the literature examining the assessment and evaluation of students. The author provides a conceptual framework to illustrate important features of the evaluation process (e.g., establishing purpose, setting standards for performance, providing feedback, etc.) and then describes how students are impacted by each feature of this process. Features affecting students include assessment purpose, task resolution, clarity of criteria, demand level and referents of standards, frequency of sampling, soundness of appraisals, and the differentiation and affective value of feedback. The author argues that studies examining the evaluation process are of limited value to educators and researchers because of a lack of descriptive information, a focus on one or two specific features of evaluation, and a failure to consider the multiple purposes of evaluation in schools. The author concludes by providing recommendations for future research, including consideration of the multiple purposes of evaluation in schools and an increased focus on evaluation as a complex system rather than focusing on individual elements of the process. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Research

Capacities: Multiple Measures, Professional Development, Professional Expertise, Skills and Knowledge

Type: Website

NCES Nonfiscal Data Handbook for Early Childhood, Elementary, and Secondary Education (2007). Retrieved February 28, 2008 from http://nces.ed.gov/forum/pub_2003419.asp

Summary: This handbook is intended to serve as a reference for users of education data and provides definitions and online tools (NCES Handbooks Online) to illustrate standard education terms for students, schools, and local and state education agencies.

Description: Tool

Capacities: Professional Development, Skills and Knowledge, Tools

Type: Website

NEA Foundation for the Improvement of Education. (2003). Using data about classroom practice and student work to improve professional development for educators. Washington, DC: NEA Foundation for the Improvement of Education.

Summary: In this paper, the authors examine how data analysis can help guide decision-making about professional development, review the capacity issues that schools and districts must address before they can use data for professional development purposes, and provide a list of resources and guiding questions regarding the appropriate use of data in education.

Description: Guidance

Capacities: Capacity Building, Culture of Collaboration, Culture of Inquiry, Professional Expertise

Type: Website

Nelson, M., & Eddy, R. M. (2008, Spring). Evaluative thinking and action in the classroom. New Directions for Evaluation, 117, 37-46.

Summary: This chapter presents a case study of a low-achieving California middle school where teachers and administrators facilitated significant improvements in student performance by enhancing evaluative skills at the school level. The authors describe the specific processes that helped foster this change including collaboration between teachers, continued professional development, use of multiple assessment measures, incorporation of an electronic data-management system, and involvement of multiple stakeholders (e.g., teachers, administrators, parents, and students) in understanding how assessments guide instruction. The authors analyze these changes in relation to No Child Left Behind and evaluation capacity building, noting that despite mandated requirements, most new teachers enter the field without the skills to analyze student achievement data and use them to guide instruction. The authors conclude that implementing these changes is a long-term process and schools must develop these practices in response to their own local needs.

Description: Research

Capacities: Culture of Collaboration, Multi-level Data Use, Transformation Plan

Type:

Newmann, F. M., King, M. B., & Rigdon, M. (1996). Accountability and school performance: Implications from restructuring schools. Final deliverable. Wisconsin: February 15, 2007 from ERIC database.

Summary: This paper explains how three main issues prevented the strengthening of school accountability at the following K-12 school sites, including implementation constroversies about standards, incentives, and constituencies; insufficient efforts to build organizational capacity; and failure to recognize the importance of internal school accountability. The findings suggest that external agencies trying to increase school accountability should attend to stimulating the internal accountability that is linked to organizational capacity. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Capacity Building, Leadership, Standards

Type: Website

Nichols, P. D., Meyers, J. L., & Burling, K. S. (2009). A framework for evaluating and planning assessments intended to improve student achievement. Educational Measurement: Issues and Practice, 28(3), 14-23.

Summary: Nichols and colleagues present a framework to help educators determine the validity of claims that student performance data from a specific assessment can be used to improve student achievement. The authors maintain that in order to be labeled "formative", assessment data must be able to be used to improve student performance. After explaining the components of the framework in detail, the authors present an example (i.e., one-on-one tutoring) to illustrate how the framework is used to evaluate the validity of formative claims. The authors also discuss implications for educational test development to better understand when the use of assessment information is likely to improve test performance. The authors conclude that educators should focus on how assessment results are related to improvements in instruction, rather than student outcomes, as improved instructional methods are essential to improve student performance.

Description: Guidance

Capacities: Action for Learning, Expectations for Data Use

Type: Website

O'Donnell, A.M., & King, A. (Eds.). (1999). Cognitive perspectives on peer learning. Mahwah, NJ: Lawrence Erlbaum.

Summary: This book is based on papers presented at the Rutgers Invitational Symposium on Education on Cognitive Skills and Learning with Peers in November, 1996. The book explores peer learning, focusing on the cognitive growth that occurs as a result of peer interactions in the classroom. In Part I, the authors present the work of Piaget and Vygotsky and highlight their contributions to theory on cognitive development and implications for peer learning. The next six chapters, Part II, describe various approaches and techniques to support peer learning and models of collaboration (largely informed by Piaget's and Vygotsky's theories of cognitive development) that can be used to enhance classroom learning and instruction. Part III explores the teacher's role when implementing peer learning techniques in the classroom, the implications of peer learning for teacher education and professional development, and the skills and knowledge needed to effectively use peer learning strategies in the classroom. The book concludes by discussing areas for future research. Overall, this book serves as a guide to help teachers understand better how to support peer learning interactions in the classroom.

Description: Knowledge

Capacities: Culture of Collaboration, Multiple Measures

Type: Book

O'Leary, Z. (2004). The essential guide to doing research. London: SAGE Publications. 978-0761941996

Summary: This guide provides detailed information on several aspects of conducting research including design, implementation, and dissemination. Activities and examples are provided to illustrate and clarify key points. The author encourages readers to think strategically and creatively about the best research methods for their particular projects.

Description: Knowledge

Capacities: Professional Development, Skills and Knowledge

Type: Book

Organisation for Economic Co-operation and Development. (2005). Formative assessment: Improving student learning in secondary classrooms. Centre for Educational Research and Innovation. Retrieved May 9, 2008 from http://www.oecd.org/dataoecd/19/31/35661078.pdf

Summary: This policy brief explores the need for formative assessment to improve learning in secondary classrooms. OECD studied the use formative assessment in Australia, Canada, Denmark, England, Finland, Italy, New Zealand and Scotland, and this brief presents those findings. Topics covered include the benefits of formative assessment (e.g., achievement gains, building learning skills), general practices involved in formative assessment (e.g., establishing and communicating learning goals, assessing student understanding, providing feedback), and specific formative assessment techniques (e.g., portfolios and rubrics, thinking time instead of hands up). The policy brief also discusses barriers to the wider use of formative assessment (e.g., a lack of coordination between assessments at the classroom, school, and policy levels) and ways to address school-level barriers to successful implementation of formative assessment practices (e.g., having students track their own performance and progress, providing opportunities for cooperative learning experiences, and creating a school-wide culture of evaluation). The brief concludes with suggestions of how to promote formative assessment practices, such as aligning summative and formative assessments, providing training and support for formative assessment, building bridges between research, policy, and practice, and involving students and parents in the process.

Description: Research

Capacities: Multiple Measures, Professional Development, Professional Expertise

Type: Website

Palaich, R.M., Good, D.G., & Van Der Ploeg, A. (2004, June). State education data systems that increase learning and improve accountability. North Central Regional Educational Laboratory. Retrieved April 10, 2006 from http://www.ncrel.org/policy/pubs/pdfs/pivol16.pdf

Summary: This paper reviews uses of education data and considers why state education leaders should build and maintain data systems. The author begins by explaining how different levels of the education system (e.g., district leaders, teachers, parents, etc.) have their own uses for student data, noting the type of information relevant to each group and current/potential uses for these kinds of data. Next, the authors discuss past, present, and future uses of education data highlighting the growing importance of new kinds of performance measures, sharing data and accountability among all interested parties, and standards for data use in educational data systems. They also explore challenges to implementation and conclude with policy recommendations (e.g., create an education data oversight commission in each state, develop common data standards and management practice, etc.) to support states in creating effective data systems.

Description: Guidance

Capacities: Data Systems, Expectations for Data Use, Stakeholder Collaboration

Type: Website

Pellegrino, J.W. (2006). Rethinking and redesigning curriculum, instruction, and assessment: What contemporary research and theory suggests (A paper commissioned by the National Center for the New Commission on the Skills of the American Workforce). Retrieved October 29, 2008 from http://www.skillscommission.org/pdf/commissioned_papers/Rethinking%20and%20Redesigning.pdf

Summary: This paper explains how current research and theory about the nature of learning is the key to improving teaching and learning practices. The author begins by describing curriculum, instruction, and assessment as three elements that must be aligned to effectively meet educational goals. He also highlights important principles about learning and understanding, such as students come to the classroom with preconceptions about the world, competence involves knowledge, and meta-cognitive strategies help students control their own learning and monitor progress. Next, the author discusses assessments and argues that measures of student performance should be tied to curriculum and made more meaningful for both teachers and students. Pellegrino notes concerns about the current assessment system (e.g., effectiveness of measurement, utility for improving teaching and learning) and offers technology as a potential solution to address these shortcomings. He concludes with ideas about the future, arguing that increased use of technology systems and advancements in technology will serve as an important source of information for educational improvements in the future.

Description: Guidance

Capacities: Balanced Assess't System, Multiple Measures

Type: Website

Perie, M., Marion, S., & Gong, B. (2009). Moving toward a comprehensive assessment system: A framework for considering interim assessments. Educational Measurement: Issues and Practice, 28(3), 5-13.

Summary: The authors assert that many assessment systems labeled as formative, benchmark, and predictive are categorically different from those studied by Black and Wiliam (1998) and should be labeled as interim assessments. Perie and colleagues begin by distinguishing between assessment types (i.e., summative, interim, formative classroom) and then provide an overview of interim assessments, discussing the definition and purpose of this kind of assessment. The authors explain that interim assessments can be instructional (i.e., to inform curriculum and instruction), evaluative (i.e., to provide evaluative information about the curriculum or instruction), or predictive (i.e., to determine the likelihood of students meeting end-of-the-year criteria), and they maintain that interim assessments, along with summative and formative assessments, are integral to a balanced assessment system. Perie and colleagues also present characteristics of an effective interim assessment system that educators can use to help evaluate systems being considered for purchase or development. The authors conclude with concerns about the present condition of these assessments and directions for future research.

Description: Knowledge

Capacities: Balanced Assess't System

Type: Website

Petrides, L. A., & Guiney, S. Z. (2002, December). Knowledge management for school leaders: An ecological framework for thinking schools. Teachers College Record, 104(8), 1702- 1717.

Summary: In this paper, the authors explain how educational organizations can use information and knowledge management to collect and examine data, and an ecological framework to convert data into meaningful information. They argue for the use of an "ecological framework" that draws on the metaphor of "knowledge ecology" to "illustrate how ideas are exchanged, innovations blossom, value is added to information, and new knowledge is tested and applied through accrued expertise and learning within the rich perspective of the ecosystem" (p. 5). The authors discuss flaws in the current use of information and knowledge management in education organizations, and identify four steps that schools can take to implement an ecological framework for knowledge management, including evaluating the current availability of information; determining information needed to support decision making; operating within the context and perspective of the school’s organizational processes' and assessing the school's information culture and politics. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Guidance

Capacities: Data Systems, Expectations for Data Use, Reporting

Type: Website

Petrides, L., & Nodine, T. (2005). Anatomy of school system improvement: Performance-driven practices in urban school districts. San Francisco, CA: Institute for the Study of Knowledge Management in Education & NewSchools Venture Fund. Retrieved February 15, 2007 from http://www.newschools.org/viewpoints/documents/District_Performance_Practices.pdf

Summary: This study is first in a series of three that seeks to examine how urban school districts across the country are adopting performance-driven practices as a means to raise student achievement levels. The study confirmed that the districts interviewed are attempting to implement performance-driven practices in a variety of ways throught their organizations; becoming a performance-driven organization is closely tied to managing people and processes; adopting performance-driven practices is a district-wide effort; professional development is a crucial tool in the adoption of performance-driven practices; there is a dynamic relationship between district oversight and direction and site-based leadership in the process of adopting performance-driven practices; external factors have had a role in motivating many districts to focus on student achievement outcomes; and districts face significant obstacles in adopting performance-driven practices.

Description: Guidance

Capacities: Expectations for Data Use, Leadership, Professional Development

Type: Website

Pijanowski, L. (2008). Striking a balance: Georgia district adds assessments and transforms classroom practice. Journal of Staff Development, 29(4), 43-46.

Summary: Pijanowski describes a district in Georgia that has demonstrated increased academic success because of a new focus on benchmark assessments and collaborative discussions about assessment. The district implemented a professional learning program, Focused Choice, which provided all district staff with six early release days and two staff development days to learn about standards-based classrooms. The author reports that these professional development experiences led to changes in how assessment was used by teachers. The article includes a table of five assessment measures (i.e., standardized assessment, benchmark assessment, common assessment, classroom assessment, and progress monitoring) and describes the purpose of each, who designs it, and how each is used to support instruction. The author also describes how school personnel were involved in three levels of reflection and dialogue (i.e., individual teacher reflection, grade-level conversation, school-wide dialogue about results) to enhance their ability to understand data and use their findings to improve classroom instruction and student performance. The article concludes with lessons learned from implementation of the benchmark assessments as part of the balanced assessment program. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Research

Capacities: Culture of Collaboration, Professional Development

Type: Website

Popham, W.J. (2003). Test better, teach better: The instructional role of assessment. Alexandria, VA: Association for Supervision and Curriculum Development.

Summary: In this book, the author explores the link between testing and teaching, and describes how teachers can effectively use test data to improve classroom instruction. Popham maintains that the teachers can use classroom-based and externally mandated tests to help identify students' strengths and weaknesses, and make changes in their instruction to improve student learning. Although the author includes some information about state/district-wide testing, he focuses on how teachers can use teacher-made classroom tests to focus instruction and better meet individual student needs. This book is designed for both experienced and novice teachers with little background in educational testing and measurement.

Description: Tools

Capacities: Action for Learning, Professional Expertise, Tools

Type: Book

Popham, W.J. (2008). Transformative assessment. Alexandria, Virginia: Association for Supervision and Curriculum Development.

Summary: The author clarifies that formative assessment is about discovering what students are and are not learning, and adjusting instructional strategies to meet their needs to help students reach their learning goals. He encourages educators to use formative assessment in schools and classrooms to improve educational processes by helping educators and students improve on their approaches to teaching and learning, shifting classroom culture from competition to student-teacher collaboration, and building learning communities to promote teacher professional development.

Description: Guidance

Capacities: Action for Learning, Capacity Building, Culture of Collaboration, Culture of Inquiry, Decision-Making, Multiple Measures, Professional Development, Professional Expertise, Skills and Knowledge

Type: Book

Porter, A. C., Chester, M. D., & Schlesinger, M. D. (2004). Framework for an effective assessment and accountability program: The Philadelphia example. Teachers College Record 106(6), 1358-1400.

Summary: (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description:

Capacities:

Type: Website

Preuss, P.G. (2003). School leader’s guide to root cause analysis: Using data to dissolve problems. Larchmont, NY: Eye On Education, Inc. 978-1930556539

Summary: This book provides information to support educators in focusing on school improvement issues. The author includes several real-life examples of effective strategies schools and educators have used to improve learning for all students. The chapters include the following information: the basics of root cause analysis, identifying key indicators of student success, root cause processes and analysis, considerations, models and stories, and additional resources.

Description: Guidance

Capacities: Data Systems, Professional Development, Skills and Knowledge

Type: Book

Process for Developing Learning Communities. Retrieved January 25, 2008 from the Southwest Educational Development Laboratory (SEDL) website http://www.sedl.org/pubs/change34/6.html

Summary: This article offers a review and synthesis of the literature on learning communities, including suggestions for how to initiate and develop professional learning communities, a summary of findings from five case studies, and identification of the defining characteristics and operational procedures of these communities.

Description: Knowledge

Capacities: Action for Learning, Culture of Collaboration

Type: Website

Protheroe, N. (2001). Improving teaching and learning with data-based decisions: Asking the right questions and acting on the answers. Retrieved May 17, 2007 from Education Research Service http://www.ers.org/spectrum/sum01a.htm

Summary: The author explores data-based decision making in schools, focusing on how to use student assessment data to improve classroom instruction and student performance. Protheroe begins by presenting evidence from various research studies showing that assessment data, when used effectively, can improve instruction. Next, the author discusses important questions to consider when using data to make decisions and recommendations for collecting and analyzing data, emphasizing the need to collect good quality data. Protheroe also explores how to respond to data, highlighting the importance of making sure that state standards are aligned with curriculum, improving teaching strategies, and providing specialized instruction for students who are not mastering learning goals. The author concludes with a section on staff support, noting that teachers and administrators must understand achievement goals and have assessment literacy skills. Overall, Protheroe maintains that with consideration of these essential elements, assessment data can be used to improve learning and instruction.

Description: guidance

Capacities: Action for Learning, Expectations for Data Use

Type: Website

RAND Corporation. (2005). Advancing systemwide instructional reform: Lessons from three urban districts partnered with the Institute for Learning. Santa Monica, CA.

Summary: This article highlights findings from the instructional reform efforts of three urban districts partnered with a university-based organization, the Institute for Learning. These district efforts included the use of school-based coaches to support teachers, specification of curriculum, promoting principals' instructional leadership, and promoting data-based decision-making. Barriers and facilitators of instructional improvement are identified across the three districts. Results suggest that involvement of a third-party organization can help facilitate policy alignment and build the capacity of district staff to lead instructional change.

Description: Knowledge

Capacities: Culture of Collaboration, Leadership, Professional Development, Stakeholder Collaboration

Type:

Rea-Dickens, P. (2001). Mirror, mirror on the wall: Identifying the processes of classroom assessment. Language Testing, 18, 429-462.

Summary: This article explores classroom assessment in an EAL (English as an additional/second language) context. The author draws from research conducted in collaboration with Minority Group Support Services, specifically using interview and lesson transcript extracts, to analyze assessment practices. Using these data, the author describes the different stages of the teacher assessment cycle and includes a working framework for teacher decision-making strategies as a result of assessment data. The author also highlights the multidimensional nature of classroom assessment and labels the different dimensions as bureaucratic, pedagogic, or learning-focused. The author concludes that formative assessment is essential to student learning. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Research

Capacities: Multiple Measures, Skills and Knowledge

Type: Website

Reeves, D. B. (2004). The leader's guide to standards: A blueprint for educational equity and excellence. San Francisco: Jossey-Bass.

Summary: This book provides a good foundation for understanding the use of standards in evaluating student performance. Chapter topics include: the difference between standards and norms; standards-based performance assessment; power standards; instructional leadership; communication with parents, politicians, and community members; data-driven decision making; educational accountability; and the roles of the superintendent, board of education, and state leadership.

Description: Knowledge

Capacities: Decision-Making, Leadership

Type: Book

Regional Educational Laboratory Midwest. (2008). Using multiple levels of data to address educational issues. (Policy Brief No. 1, June 2008). Retrieved from http://www.learningpt.org/aboutus/howwework/centers/RELpubs/June2008Brief.pdf

Summary: This policy brief is based on the 2007 Issues & Answers report, "Getting the Evidence for Evidence-Based Initiatives: How the Midwest States Use Data Systems to Improve Educational Processes and Outcomes". This brief explains the importance of data use and then describes several exemplary practices used by various Midwestern states to address data priorities and meet federal requirements (e.g., linking data across the education system, using accountability data, etc.). Next, the brief includes a section on common data challenges including constraints on resources and capacity, data burden, and implications of state and federal regulations. Specific data priorities and challenges are also highlighted for individual states throughout the region. The brief concludes with a detailed table illustrating the type of student and teacher information recorded in each state data inventory.

Description: Guidance

Capacities: Data Systems, Multi-level Data Use

Type: Website

Reitz, R., & Winter, C. (2004). Growing pains: Piloting a statewide information system is difficult but worth the effort. Retrieved April 10, 2006 from American http://www.asbj.com/specialreports/0904SpecialReports/S4.html

Summary: This article relates the implementation of a statewide information system in North Carolina to educators interested in similar projects.

Description: Guidance

Capacities: Capacity Building, Data Systems, Tools

Type: PDF

Retrieved December 18, 2007 from http://usingdata.terc.edu/

Summary: The Using Data Professional Development Program is a process of systemic reform to help schools and districts turn data they collect into data they use. The program consists of six days of professional development workshops (http://www.terc.edu/work/743.html) sequenced over a calendar year to prepare school leaders and school-based data teams in using data to identify and address student achievement gaps and to establish a culture of collaborative inquiry at their schools. In addition, "Meeting the Challenges of Accountability in Mathematics and Science" is a monograph created by TERC to share lessons learned through the Using Data project. The paper indicates how schools involved with the project used data to effectively and continuously improve results.

Description: Service

Capacities: Professional Development, Skills and Knowledge

Type: Website

Retrieved January 24, 2008 from http://cep.terc.edu/dec/

Summary: The Data on Enacted Curriculum (DEC) project is a professional development model to help schools and districts connect the data they have with the results they want to produce. DEC is a process and materials designed to build the capacity of educators to realize the potential of their data in solving problems in closing the achievement gap and raising the achievement of all students in math and science.

Description: Guidance

Capacities: Professional Development

Type: Website

Retrieved January 25, 2008 from The Education Trust website www2.edtrust.org

Summary: This website describes an inquiry and professional development process called Standards in Practice (SIP), which helps educators examine student and teacher work in the context of what they want students to know and do.

Description: Knowledge

Capacities: Professional Development, Skills and Knowledge, Tools

Type: Website

Retrieved January 25, 2008 from the National School Reform Faculty website www.nsrfhamrony.org.

Summary: This website offers resources for collaborative examination of teacher and student work, including protocols, videos, and training for facilitators. This site also includes information on Critical Friends Groups (CFGs), a professional learning community that offers a formal framework for inquiry.

Description: Guidance

Capacities: Professional Development, Skills and Knowledge, Tools

Type: Website

Retrieved January 25, 2008 from the Technical Education Research Centers (TERC) website http://www.terc.edu/ourwork/usingdata.html.

Summary: This website provides information on how educators can use test data constructively to guide school improvement decisions. Drawing on the process detailed in Nancy Love's book Using Data/Getting Results: A Practical Guide to School Improvement in Mathematics and Science, TERC has created a professional development program to assist teachers and school and district administrators in facilitating the process for using data in their schools. The Using Data Professional Development Program (http://usingdata.terc.edu/) is a process of systemic reform to help schools and districts turn data they collect into data they use. The program consists of six days of professional development workshops sequenced over a calendar year to prepare school leaders and school-based data teams in using data to identify and address student achievement gaps and to establish a culture of collaborative inquiry at their schools.

Description: Guidance

Capacities: Professional Development, Skills and Knowledge, Tools

Type: Website

Rothman, R. (2008). Data-informed decision making: Using data wisely and well. Voices in Urban Education, 18. Retrieved February 28, 2008 from http://www.annenberginstitute.org/VUE/winter08/Rothman.php

Summary: This article provides an overview of the Winter 2008 issue of Voices in Urban Education, focusing on districts and communities that have effectively used data to inform decision-making. The related articles describe how districts identify and analyze relevant data in order to improve student learning. The districts and community organizations included in this issue use multiple measures in different ways. Based in part on a study by the Annenberg Institute for School Reform on district data use, this issue aims to identify leading indicators across districts and communities that point towards effective, sustainable use of data to improve schools and student learning.

Description: Knowledge

Capacities: Culture of Collaboration, Data Systems, Decision-Making, Leadership, Multi-level Data Use, Multiple Measures, Tools

Type: Website

Rowntree, D. (2003). Statistics without tears: A primer for non-mathematicians. Needham Heights, MA: Allyn & Bacon Inc. 978-0024040909

Summary: This book provides an overview of basic statistical topics including descriptive and inferential statistics, collecting and describing a sample, analyzing and summarizing data, comparing samples and estimating to a population, testing for significance, and interpreting analysis results. The information is presented in simple, straightforward prose and is geared towards readers who do not have a background in quantitative research.

Description: Knowledge

Capacities: Analysis, Professional Development, Skills and Knowledge

Type: Book

Rudner, L. M. (1994). Questions to ask when evaluating tests. Practical Assessment, Research & Evaluation 4(2). Retrieved June 5, 2006 from http://PAREonline.net/getvn.asp?v=4&n=2

Summary: The author suggests the scope and types of questions that education leaders should consider when evaluating assessments. Topics covered include test coverage and use, appropriate samples for validating and norming tests, reliability, and validity (criterion, content, and construct). Rudner also discusses test administration, reporting results, and test and item bias. The author concludes with additional resources and recommended readings for additional information.

Description: Guidance

Capacities: Data Quality Assurance, Expectations for Data Use

Type: Website

Ruiz-Primo, M.A., & Furtak, E.M. (2007). Exploring teachers' informal formative assessment practices and students' understanding of the context of scientific inquiry. Journal of Research in Science Teaching, 44(1), 57-84.

Summary: The purpose of this study was to explore teachers' informal formative assessment practices in three middle school science classrooms. Researchers videotaped 30 physical science investigation lessons across the three teachers, transcribing each and focusing on student-teacher assessment conversations. Formative assessment practices were measured using ESRU cycles (teacher Elicits a question; Student responds; teacher Recognizes the student's response; and then Uses the information collected to support student teaching). Student learning was measured through a comprehensive assessment including graphing tasks, predict-observe-explain items, and prediction questions. Results indicated that the teachers who used the complete ESRU cycle more often had students who demonstrated higher performance on the assessment when compared to performance in the other two classes. The researchers conclude that the ESRU model is an effective way to measure differences in teachers' informative assessment practices, and these strategies are associated with improved student learning and performance in science classrooms. (Permission to post article is pending. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Action for Learning, Professional Development

Type: Website

Russell, M., Bebell, D., O'Dwyer, L., & O'Connor, K. (2003). Examining teacher technology use. Journal of Teacher Education, 54(4), 297-310.

Summary: The purpose of this study was to examine the degree to which teachers use technology to support instruction. The researchers collected survey data from 2,894 teachers in Massachusetts to determine how technology is used by teachers and students in and out of the classroom, what factors affect these uses, and how technology impacts student learning. The researchers identified six categories of technology use in schools: 1) teacher use of technology use for preparation; 2) teacher use of technology for delivery; 3) teacher-directed student use of technology; 4) teacher use of technology for special education and accommodation; 5) teacher use of email; and 6) teacher use of technology for recording grades. The researchers also found that overall, teachers were more likely to use technology as a communication tool or to help prepare for lessons rather than during the instructional process or learning activities. When comparing experienced versus novice teachers, however, they found that experienced teachers were more likely to report using technology to aid instruction and activities, while novice teachers reported greater comfort with technology but were more likely to use it in preparation for instruction. The researchers conclude that teacher preparation programs must include opportunities for teachers to see how technology benefits learning and how it can be used as an instructional tool. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Research

Capacities: Action for Learning, Professional Expertise, Skills and Knowledge

Type: Website

Sadler, D.R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119-144.

Summary: This article describes a theory of formative assessment and the role of these practices in the learning process. The author highlights teacher feedback and student self-monitoring as key aspects of formative assessment and describes both in detail. More specifically, Sadler argues that traditional definitions of feedback as “correct” or “incorrect” are too narrow to be helpful and should be expanded to include qualitative judgments about performance requiring knowledge of standards, multi-criteria comparisons, and an understanding of how to narrow the gap between work produced and the desired goal. The author also discusses the need for students to become less dependent on teacher feedback and develop skills and knowledge (through exemplars and explicit instruction) to evaluate their own work and progress. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Multiple Measures, Skills and Knowledge

Type: Website

Sadler, D.R. (1998). Formative assessment: Revisiting the territory. Assessment in Education: Principles, Policy & Practice, 5(1), 77-84.

Summary: In this article, the author comments on a formative assessment review article by Black and Wiliam (1998), highlighting certain points with particular attention to issues regarding quality feedback. The author argues that students must be taught how to understand feedback and how to use suggestions to improve their work. The author also mentions the inequality between student and teacher and how the nature of that relationship should be recognized to help understand how to communicate effective feedback. A description of the three elements that make up feedback (i.e., attention to the learner’s production, appraisal against a reference framework to identify strengths and weakness, and an explicit response) is provided as well as the six resources used by competent teachers to carry out this evaluation. These resources include superior content knowledge, dispositions towards the activity and learners, skills to construct tests and devise tasks, knowledge of criteria and standards, evaluative expertise, and experience determining effective feedback. The author concludes that quality of feedback includes both the accuracy/ appropriateness of the teacher’s response and effectiveness of communication to the learner, and that exploring the quality of feedback should be a key focus of future research. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Multiple Measures

Type: Website

Safer, N., & Fleischman, S. (2005). How student progress monitoring improves instruction. Educational Leadership, 62(5), 81-83. Retrieved October 1, 2008 from http://www.ascd.org/authors/ed_lead/el200502_safer.html

Summary: The purpose of this article is to explore how student progress monitoring (i.e., using student progress data to evaluate teaching and make informed instructional decisions) can be used to promote student achievement. The authors describe how to implement this instructional strategy, explaining how to measure progress regularly and track progress over time. They also compare student progress monitoring with other mastery measurement approaches (e.g., teacher made tests) and maintain that progress monitoring makes it easier to modify instruction given that student progress toward learning goals is continuously being assessed. The authors include a vignette describing how a third-grade teacher uses student progress monitoring to better support student reading progress. They also provide a website for educators who are interested in commercially available products that help teachers monitor student learning. The authors argue that student progress monitoring is effective with all students, though this strategy may be particularly helpful to students with special needs or those struggling to meet learning goals. The authors conclude that student progress monitoring often requires more effort from the teacher, yet the increased effort results in better learning outcomes for all students.

Description: Guidance

Capacities: Multiple Measures

Type: Website

Sarmiento, J. W. (2003). Technology tools for the analysis of achievement data: An introductory guide for educational leaders. Philadelphia: Laboratory for Student Success, Temple University. Retrieved June 14, 2007 from http://www.nwrel.org/scpd/sslc/federal_grantees/cohort2/ data_institutes/binder/resources/C2DataTechToolsforAnalysis.pdf

Summary: This guide serves as an introduction to education information systems, technology tools, and vendor products that assist schools and districts with data analysis. The authors begin by describing the growing need for more complex strategies for collecting, analyzing, reporting, and using achievement data. Next, they present sample scenarios depicting different stages of organization regarding the use of achievement data in learning communities. The authors also discuss School Information Systems and explain functional and technical considerations for selecting technology tools. The authors conclude by highlighting key features to consider when choosing technology systems and data analysis tools and include a description of available products.

Description: Guidance

Capacities: Analysis, Data Systems, Infrastructure

Type: Website

Sato, M., Coffey, J., & Moorthy, M. (2005). Two teachers making assessment for learning their own. The Curriculum Journal, 16(2), 177-191.

Summary: This article highlights the stories of two teachers involved in The Classroom Assessment Project to Improve Teaching and Learning (CAPITAL). The goal of this project was to identify and understand the processes that guide teachers' assessment activities in the classroom. The authors documented the classroom activities and group discussions among middle school science teachers over four years. The authors found significant variations in how the two teachers adapted and incorporated assessment for learning into their everyday teaching practice. These differences emphasize the contextualized and personal nature of change for teachers.

Description: Knowledge

Capacities: Action for Learning, Capacity Building, Skills and Knowledge

Type: Website

Schafer, W.D. (1993). Assessment literacy for teachers. Theory into Practice, 32(2), 118-126. College of Education, The Ohio State University.

Summary: The purpose of this article is to explore the need for assessment literacy for teachers. The article begins with a vignette about a low achieving fourth grade boy whose parents are in a conference with his teacher discussing his academic struggles and asking how to interpret his standardized test scores. Through this brief story, the author illustrates several ways in which assessments may be misunderstood. The author maintains that assessment is an important element in education and, therefore, should be a focus of at least one course in teacher education programs. The author also highlights important topics (e.g., test formats, levels of educational objectives, types of standardized tests, interpreting scores, etc.) that should be taught in an assessment course and discusses assessment topics covered in other teacher preparation courses. The author concludes by describing the assessment practices commonly seen in schools and reviews recommendations to improve the use of assessments including hiring assessment specialists in schools and assessment training for all education professionals. (Permission to post article is pending. Link to publisher's website is provided.)

Description: Guidance

Capacities: Balanced Assess't System, Multiple Measures

Type: Website

Schmoker, M. (1999). Results: The key to continuous school improvement, second edition. Alexandria, VA: Association for Supervision and Curriculum Development.

Summary: This book provides information to support educators in using data to improve student learning, including setting goals, working collaboratively, and keeping track of student-achievement data from many sources. The author includes information on standards and assessments, effective professional development programs, and several examples of successful practices in schools and districts.

Description: Guidance

Capacities: Action for Learning, Culture of Collaboration, Professional Development

Type: Book

Schmoker, M. (2003). First things first: Demystifying data analysis. Educational Leadership, 60(5), pp.22-24. Retrieved January 18, 2008 from website http://mikeschmoker.com/data-analysis.html

Summary: This article argues that schools should focus their school improvement efforts on simple questions of student performance that can be answered using data. The author discusses how educators can set realistic and specific goals regarding student achievement; how they can address areas of weakness in student performance and learn to conduct the analyses that will have greatest impact on their teaching and student achievement; and, lastly, how they can use formative assessment data to improve their lessons and assess their progress.

Description: Knowledge

Capacities: Analysis, Professional Development, Skills and Knowledge

Type: Website

Schools Interoperability Framework Association. Retrieved June 14, 2007 from http://www.sifinfo.org/

Summary: This website describes the Schools Interoperability Framework (SIF) Association, providing both a general overview and specific sections explaining what SIF means for vendors, schools, government agencies, and partners. The website provides a variety of resources in regards to the implementation of SIF including an implementation readiness tool, an implementation toolkit, downloadable text on RFP language, and links to SIF certified applications. The website also includes links to the SIF newsletters, presentations related to the Schools Interoperability Framework, and annual reports.

Description: Tool

Capacities: Data Systems, Infrastructure

Type: Website

Sharkey, N. S., & Murnane, R. J. (2006). Tough choices in designing a formative assessment system. American Journal of Education 112(4), 572-588.

Summary: This article from an issue of the American Journal of Education dedicated to research about data use and data-driven decision-making describes one school district's attempts to implement a math formative assessment system. The authors utilize the current literature on formative assessment and the findings from this qualitative case study to shed light on the challenges educators face in choosing and implementing formative assessments. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities:

Type: Website

Shavelson, R.J., Baxter, G.P., & Pine, J. (1992, May). Performance assessments: Political rhetoric and measurement reality. Educational Researcher 4(21), 22-27.

Summary: In this paper, the authors explore the use of alternative achievement measures by presenting findings from a study examining the reliability and validity of various science performance assessments. The researchers evaluated how hands-on investigations compared with various alternative assessments including notebook reporting of the investigation, computer simulations of the investigation, short-answer questions paralleling the investigation, and multiple choice tests. Participants included fifth and sixth grade students. The researchers found that inter-rater reliability was high for all investigations, yet internal consistency was more difficult to attain, as student performance differed depending on the task. These findings suggest that although raters are able to provide reliable scores on science investigations, numerous tasks may be needed to adequately measure student performance. As for the interchangeability of alternative science assessments, notebooks were closest to the investigations in terms of reliability and validity, followed by the computer simulation, paper and pencil short answer questions, and multiple choice items. Furthermore, multiple-choice test scores were only moderately correlated to the investigations, meaning that these assessments may measure different aspects of science knowledge/achievement. The authors conclude that policy makers and education professionals should focus less on holding students and teachers accountable and focus more on the resources to support teaching and learning in the classroom. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Research

Capacities: Data Quality Assurance, Multiple Measures, Policies

Type: Website

Shepard, L. A. (1993). Evaluating test validity. In L. Darling-Hammond (Ed.), Review in research in education, vol. 19. Washington, DC: AERA.

Summary: In this chapter, the author seeks to trace the evolution of test validity and point it in a new direction based on the work of Samuel Messick.

Description: Guidance

Capacities: Data Quality Assurance, Quality of Assessment Data

Type: Journal

Shepard, L. A. (2009). Commentary: Evaluating the validity of formative and interim assessment. Educational Measurement: Issues and Practice, 28(3), 32-37.

Summary:

Description:

Capacities:

Type: Website

Shepard, L.A. (2000). The role of assessment in a learning culture. Educational Researcher, 29(7), 4-14.

Summary: This article focuses on how classroom assessment can be used as part of instruction to support and enhance learning. The author discusses changes in curriculum, learning theory, and assessment by contrasting the central tenets of the 20th Century dominant paradigm - including behaviorist learning theory and scientific measurement - with those of an emergent paradigm - that draws on cognitive, constructivist and sociocultural theories of learning. This comparison highlights the discrepancy between new views of instruction and traditional views of testing. The author elaborates on the ways assessment practices should change in order to align with and support the social-constructive model of teaching and learning. These suggestions include ways to use assessment in the classroom both to monitor and promote individual students' learning and to examine and improve teaching practices. According to this article, teacher education programs must address the issues surrounding the use of assessment as teachers play a central role in shaping students attitudes about learning. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Expectations for Data Use, Leadership, Quality of Assessment Data

Type: Website

Shepard, L.A. (2005). Linking formative assessment to scaffolding. Educational Leadership, 63(3), 66-70. *Used by permission. Learn more about ASCD at www.ascd.org

Summary: This article highlights the collaborative processes of scaffolding and formative assessment to illustrate how contemporary learning theories affect teaching practices. The author describes four effective strategies common to both formative assessment and scaffolding to demonstrate the strong connection between assessment and research on learning. These strategies involved eliciting prior knowledge from students, providing them with effective feedback, ensuring they transfer knowledge to new situations, and teaching them how to self-assess. According to the author, a cultural shift in learning needs happen in order for these strategies to by truly effective. A learning culture needs to be created where students and teachers have shared expectations about the next steps in learning.

Description: Knowledge

Capacities: Action for Learning, Communication, Culture of Collaboration, Professional Development, Skills and Knowledge

Type: PDF

Shepard, L.A. (2005, October). Will commercialization enable or destroy formative assessment? (Original title: Formative assessment: Caveat emptor). Paper presented at the ETS Invitational Conference 2005: The Future of Assessment, Shaping Teaching and Learning, New York.

Summary:

Description: Knowledge

Capacities: Multiple Measures

Type:

Shepard, L.A. (February 2000). The role of classroom assessment in teaching and learning. (CSE Tech. Rep. No. 517). Los Angeles, CA: Center for the Study of Evaluation, National Center for Research on Evaluation, Standards, and Student Testing, University of California, Los Angeles.

Summary: This paper presents a framework for understanding assessment as a significant part of the teaching and learning process. The author argues that if assessment measures are to be used to inform teaching practices, the content and character of assessments must be improved, and the collection and analysis of assessment data must become part of the ongoing learning process. The paper begins by exploring underlying curriculum and psychological theories that have influenced instruction, conceptions of subject matter, and methods of testing. The author then presents a conceptual framework based on new theories about the relationship between curriculum, assessment, and learning theory. The final sections describe a model for classroom assessment developed from social-constructionist principles and then discuss areas for further research.

Description: Guidance

Capacities: Culture of Collaboration, Culture of Inquiry, Expectations for Data Use, Multiple Measures, Professional Expertise, Skills and Knowledge

Type: Website

Shute, V.J. (2007, March). Focus on formative feedback. (ETS Research Report Number RR-07-01). Princeton, NJ: Educational Testing Service. Retrieved May 15, 2008 from http://www.ets.org/Media/Research/pdf/RR-07-11.pdf

Summary: This paper provides a review of research examining teacher feedback, focusing specifically on formative feedback (i.e., information shared with the learner to improve learning). The review shows that formative feedback should be multidimensional, non-evaluative, supportive, timely, specific, credible, infrequent, and genuine. The author also maintains that feedback comes in a variety of forms (e.g., verification, explanation) and can be given to the learner throughout the learning process. The author also discusses factors that may enhance or challenge the success of formative feedback (e.g., teacher/learner characteristics, task characteristics) to support positive learning outcomes. The review includes figures throughout to illustrate ideas and concludes with suggestions of ways to use formative feedback to foster learning in the classroom.

Description: Guidance

Capacities: Multiple Measures, Professional Development, Professional Expertise, Skills and Knowledge

Type: Website

Sirotnik, K.A., & Kimball, K. (1999, November). Standards for standards-based accountability systems. Phi Delta Kappan, 81(3), 209-214. Retrieved February 13, 2008 from http://www.pdkintl.org/kappan/ksir9911.htm

Summary: This article discusses 11 standards that educators and policy makers should consider in evaluating standards-based accountability systems. The authors raise questions and issues to foster discussion about how to operationalize each standard in order to determine proficiency of assessment and accountability systems. These standards support legislators and education policy makers' efforts to critically and responsibly examine assessment and accountability systems. (Publisher granted permission to post abstract only. Link to article abstract is provided.)

Description: Guidance

Capacities: Leadership, Standards

Type: Website

Smith, E. & Gorard, S. (2005). 'They don't give us our marks': The role of formative feedback in student progress. Assessment in Education, 12(1), 21-38.

Summary: This study examines how students' progress in a comprehensive secondary school is affected when teachers use formative feedback alone without other types of classroom assessments. The researchers divided students (N = 104) into one of four mixed-ability groups, each comprised of 26 students. Three groups were given the standard school marks/grades throughout the school year without many comments from the teacher, while the remaining treatment group received individual formative feedback from trained teachers, but no formal grades or summative marks for any work during the year. At the end of the school year, researchers compared the English, Math, Science and Welsh achievement levels reached by students in both the control and experimental groups and found that for all subjects except Science, the students in the control group made more academic progress. (No differences were observed in science progress.) To better understand these results, the researchers conducted group interviews with members of the treatment group and found that students frequently cited frustration from not receiving any kind of grade on their work and not knowing how to evaluate their own progress and further their own learning. The researchers explain that certain formative assessment techniques such as peer and self-assessment were not employed, thus in the present study, the transfer of some control over learning from teacher to student, as is recommended in formative assessment practices, did not occur. The researchers found that overall, the approach used with the treatment group did not support learning and was somewhat unpopular among students. The researchers note that these findings are not consistent with prior research (e.g., Black & Wiliam, 2003) that has demonstrated the benefits of using formative assessment practices without formal grades, though the teachers in the present study received less support and attention than previous work. The authors conclude that certain interventions may not be as effective in general practice and recommend that future studies consider the applicability of study findings to the real school setting. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Research

Capacities: Balanced Assess't System, Multiple Measures

Type: Website

Smith, J. (2003). Reconsidering reliability in classroom assessment and grading. Educational Measurement: Issues and Practice, 22(4), 26-33.

Summary: This article explores reliability in classroom assessment and grading and whether approaches to reliability from more traditional measurement theory are appropriate for classroom assessments. The author focuses on issues regarding high school grading and assessment exploring the reliability of class rank, grade point average, course grades, and course assessments, and argues that the strong reliance on high school grades to make critical decisions regarding a student's future (e.g., college admissions) may be inappropriate from a reliability standpoint. The author proposes an alternative idea of reliability, labeled "sufficiency of information", based on the basic tenet of reliability theory of having enough information to make decisions or inferences. More specifically, sufficiency of information would mean a reliability index measuring the amount of information used to measure student performance or achievement. The author offers a formula to measure this index, discusses specificities regarding this notion of reliability (e.g., high achieving students would need fewer pieces of information), and mentions how this conceptualization of reliability may be applied to formative assessment practices (e.g., determining whether sufficient information has been gathered to make instructional decisions). The author concludes that a sufficiency of information approach to reliability may be particularly helpful to teachers who must decide whether they have collected enough information to provide accurate and meaningful measures of student performance. (Permission to post article is pending. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Data Integrity, Data Quality Assurance, Multiple Measures, Skills and Knowledge

Type: Website

Snipes, J., Doolittle, F., & Herlihy, C. (2002). Foundations for success: Case studies of how urban school systems improve student achievement. Washington, DC: Council of the Great City Schools. Retrieved April 10, 2006 from http://www.cgcs.org/pdfs/Foundations.pdf

Summary: This report focuses on the role of school districts as change agents and sustainers of school improvement. It examines the experiences of three large urban school districts that have experienced overall student achievement gains. The research found that data use is critical to the decision making processes in successful school districts.

Description: Knowledge

Capacities: Culture of Collaboration, Culture of Inquiry, Expectations for Data Use

Type: Website

Spillane, J.P. (2005). Distributed leadership. The Educational Forum, 69 (2), 143-150. Retrieved October 6, 2008, from http://pdfserve.informaworld.com/745466_770849120_790169944.pdf

Summary: In this paper, the author explores the basic tenets of "distributed leadership" emphasizing the collaborative and reciprocal nature of this idea. The author explains that "distributed leadership" means focusing on interactions between leaders and followers in given situations rather than solely focusing on actions of the individual leader. The author argues that distributed leadership is a helpful perspective for thinking about effective school leadership. (Publisher granted permission to post abstract only. Link to article abstract is provided.)

Description: Guidance

Capacities: Communication, Culture of Collaboration, Leadership

Type: Website

Stein, M. (2003). Making sense of the data: Overview of the K-12 data management and analysis market. Boston, MA: Eduventures. Retrieved December 7, 2006 from http://www.3d2know.org/making_sense_of_the_data.pdf

Summary: This report provides state and district administrators with a comprehensive overview of the K-12 data management and analysis (DMA) market. The author begins by discussing the new data challenges faced by districts and states as a result of the No Child Left Behind Act (NCLB) as well as rising expenses related to data management and analysis. Next, the author describes the benefits of using DMA systems (e.g., efficiency, positive impacts on education), explains DMA components, and defines key terms related to DMA systems. The report also includes a description of vendor approaches and market strategies as well as 23 profiles of businesses involved in the data management and analysis market. The author maintains that administrators must first decide on the necessary DMA system components to meet the goals of NCLB and then decide on appropriate vendors to best meet their needs.

Description: Guidance

Capacities: Data Systems, Infrastructure, Tools

Type: Website

Stiggins, R. (2005). From formative assessment to assessment for learning: A path to success in standards-based schools. Phi Delta Kappan, 87(3).

Summary: This article examines the use of formative assessment and how assessment of learning, traditionally used to motivate students, should be changed to assessment for learning to foster the development of effective schools and competence in all students. The author begins by reviewing the history of assessment in schools, emphasizing how students who experienced success thrived in the school setting while those who did not fare as well often lost confidence in their own abilities and eventually gave up and stopped caring about school. Now, the author argues, there is less emphasis on sorting and ranking students and more emphasis on the potential for success in all students which has given way to interest in formative assessment. Furthermore, although summative assessments have received more attention throughout the history of education, formative assessment practices used to provide teachers and students with continuous evidence of learning and mastery are an increasingly popular tool to improve school success. The author maintains that formative assessment practices lead to more frequent testing, effective management of student data, and assessment to guide and support learning (rather than assessment of learning), all of which help meet the shortcomings of large-scale, annual, standardized tests. The author concludes that students’ emotional response to assessment results and actions taken in response to results are key to the success of formative assessment practices. Therefore, learning goals should always be clear and students must receive continuous, guided feedback to help maintain a positive response so that students are motivated to continue trying and know the next steps to meet their learning goals. (Publisher granted permission to post abstract only. Link to article abstract is provided.)

Description: Knowledge

Capacities: Multiple Measures, Professional Development, Professional Expertise, Skills and Knowledge

Type: Website

Stiggins, R. (2006). Assessment for learning: A key to motivation and achievement. Edge, 2(2), 3-19.

Summary: In this article, the author describes the changing educational system and the need for assessment for learning rather than assessment of learning, given the increased emphasis on school success for all students. The author begins by describing recent shifts in education (e.g., from almost exclusive focus on norm-referenced to criterion-referenced tests) that have occurred that support this changing view of assessment, and then describes qualities of effective assessments (e.g., provides rich description) and ways to appropriately use these assessments (i.e., to inform decisions regarding instruction and to encourage students to want to learn and improve). The author also includes numerous tables to highlight important questions, decisions to be made, information needed, and implications regarding classroom assessments for the student, teacher, parent, principals, curriculum specialists, and district level professionals. The author maintains that all levels of the system have a unique impact on assessment and all must work effectively to maximize benefits of assessment to all students. Next, the author describes three critical foundations of effective assessment systems, including achievement expectations, commitment to developing and implementing school-based standards, and accuracy. He also discusses issues of student motivation related to assessment and argues that assessment for learning, when carried out properly, can motivate students to achieve. Stiggins concludes that courses teaching how to use assessments for learning should be included in teacher preparation and professional development programs so that all teachers have the necessary knowledge and skills to use these practices in the classroom.

Description: Guidance

Capacities: Balanced Assess't System, Multi-level Data Use, Multiple Measures

Type: Website

Stiggins, R. (2006). Balanced assessment systems: Redefining excellence in assessment. Princeton, NJ: Educational Testing Service. *Reprinted by permission of Educational Testing Service, the copyright owner. No endorsement of this web site or publication by Educational Testing Service should be inferred.

Summary: This paper describes a vision for the future of assessment that informs educational decisions at all levels and encourages all students to learn. The author argues for profound changes in assessment in order to create a more balanced assessment system, which includes reshaping assessment priorities and redefining the relationship between assessment and effective schools. The author identifies various components that are essential to bringing about this vision for assessment, and explains how each one will affect school achievement and school effectiveness. These components include balancing summative with formative applications and large-scale with classroom assessment; judging assessment quality by the results of learning and learners; and creating assessments that provide descriptions of student learning and help students improve. The author also argues that assessments must be transformed from isolated to ongoing events in order for them to be truly effective.

Description: Guidance

Capacities: Balanced Assess't System, Expectations for Data Use, Multiple Measures

Type: Website

Stiggins, R. (2007). Assessments through the student’s eyes. Educational Leadership, 64(8), 22-26. Retrieved October 12, 2009, from http://www.ascd.org/publications/educational_leadership/may07/vol64/num08/Assessment_Through_the_Student's_Eyes.aspx

Summary: Stiggins argues that the focus of assessment should shift from ranking students by achievement level to assessment that supports learning for all students. The author begins with a table to highlight the differing assessment experiences of kids who are doing well (e.g., the student thinks I want more success) versus those who struggle academically (e.g., the student thinks I'm confused, I don't like this, help!). Next, Stiggins defines and describes the process of assessment for learning and then presents two scenarios to illustrate how teachers can use assessment to set up students for success and help students turn failure into achievement. Stiggins concludes with recommendations to help redefine assessment practices including expanding the criteria by which teachers evaluate the quality of assessments (i.e., in addition to reliability and validity, considering the effect assessments have on future learning), changing the belief that adults make the most important or critical interpretation of assessment results, and considering students' thoughts, feelings, and reactions to assessment experiences. Stiggins maintains that by changing the purpose of assessments in schools and encouraging assessment for learning, educators will encourage success for all students.

Description: Guidance

Capacities: Balanced Assess't System, Multiple Measures

Type: Website

Stiggins, R. J. (1994). Student-centered classroom assessment. New York: Macmillan College Publishing Company. Inc. 978-0134329314

Summary: This book provides information on student-involved and classroom-based assessment practices. It includes information on how educators can develop quality assessment procedures within the classroom setting. The author gives clear descriptions of various assessment methods and how to align them with relevant achievement goals.

Description: Knowledge

Capacities: Professional Development, Skills and Knowledge, Tools

Type: Book

Stiggins, R., & Duke, D. (2008). Effective instructional leadership requires assessment leadership. Phi Delta Kappan, 90(4), 285-291.

Summary: This article describes the principal's role in implementing assessment for learning. The authors maintain that principals should be instructional leaders and that understanding how to use assessment to improve teaching and learning is an important part of this role. Furthermore, principals should be able to ensure that assessments are of high quality and are implemented effectively in classrooms, and therefore, principal preparation programs must include preparation for this role. The authors argue that principals should understand assessment at the classroom-, program-, and policy-levels and how effective assessments and data-driven decision making will vary at each level. The authors also provide a list of 10 essential leadership competencies related to assessment (e.g., can develop and implement sound assessment policies, can plan, presents, or secure professional development activities that contribute to the use of sound assessment practices). The authors maintain that principal preparation programs must remove barriers to assessment leadership (e.g., limited assessment training) by incorporating assessment knowledge into existing courses or create a course designed to educate principals about assessment and their role in the assessment process at all levels. (Permission to post article is pending. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Multi-level Data Use, Professional Expertise

Type: Website

Stiggins, R.J. (2002, June). Assessment crisis: The absence of assessment FOR learning. Phi Delta Kappan, 83(10), 758-765.

Summary: This article discusses the narrow view of assessment in American education and the need to balance the use of classroom assessments and standardized tests. The author distinguishes between assessment of learning and assessment for learning, arguing that the latter has been largely ignored in education. The author maintains that while both are important, classroom assessment, or assessment for learning, has the potential to impact teachers' daily practice to improve student learning. The author outlines benefits for students, teachers, and parents, and reviews numerous studies demonstrating the positive impacts of an increased emphasis on assessment for learning. The article concludes with an action plan comprised of five practices for example match funding of assessment of learning to assessment for learning and provide professional development opportunities for teachers to enhance their knowledge about classroom assessment, to help balance the types of assessment used in the American educational system. (Publisher granted permission to post abstract only. Link to article abstract is provided.)

Description: Guidance

Capacities: Action for Learning, Multiple Measures

Type: Website

Stipek, D.J. (1998). Motivation to learn (2nd ed). Needham Heights, MA: Allyn and Bacon.

Summary: This book explores achievement motivation by integrating research, theory, and practice. The book includes a review of current theories regarding motivation to achieve and provides explicit examples and guidance to help teachers improve academic motivation in the classroom. Chapter topics include motivational problems, definitions of achievement motivation, and reviews of relevant theories including reinforcement theory, social cognitive theory, and intrinsic motivation. The book also includes other topics related to motivation such as perceptions of ability and achievement anxiety as well as more practical chapters discussing intrinsic motivation in the classroom, maintaining positive achievement related-beliefs in the classroom, communication of expectations, and solutions for low student motivation. In addition to essays, the book also includes various tools (e.g., charts, checklists, rating forms) to support teachers in diagnosing motivation problems and to assess their own use of praise and motivational strategies. Overall this book serves as a guide and resource to help teachers support and enhance students' achievement motivation in the classroom.

Description: Knowledge

Capacities: Multiple Measures, Skills and Knowledge

Type: Book

Streifer, P. A. & Schumann, J. A. (2005). Using data mining to identify actionable information: Breaking new ground in data-driven decision making. Journal of Education for Students Placed at Risk (JESPAR) 10(3), 281-293. Retrieved February 22, 2007 from http://www.leaonline.com/doi/pdf/10.1207/s15327671espr1003_4

Summary: This study addresses the difficulty that school leaders face in using the stores of data they have already collected to analyze the effectiveness of interventions focused on improving achievement. The essential question is whether or not educators can predict student achievement from the disparate variables already stored in typical data warehouses. There are three major findings from this study: the complication of data presentation is often resolvable; it is possible to build data-ming models that produce a predicted target variable that is highly correlated to the actual target variable; and easy to understand methods of data output and presentation permit school personnel to make more strategic curriculum decisions. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Analysis, Decision-Making, Infrastructure

Type: Website

Streifer, P. A. (2004). Tools and techniques for effective data-driven decision-making. Lanham, MD: Scarecrow Press.

Summary: The author provides readers with a comprehensive understanding of what data-driven decision-making can and cannot tell educators about student achievement and addresses the related issues for leadership, policy development, and accountability.

Description: Guidance

Capacities: Expectations for Data Use, Leadership, Policies

Type: Book

Stringfield, S., Wayman, J. C., & Yakimowski, M. (2005). Scaling up data use in classrooms, schools and districts. In Dede, C., Honan, J., & Peters, L. (Eds.), Scaling up success: Lessons learned from technology-based educational innovation. San Francisco: Jossey-Bass.

Summary: This text is a hands-on resource that clearly describes different models for "scaling up" success. It is filled with illustrative examples of best practices that are grounded in real-life case studies of technology-based educational innovation.

Description: Knowledge

Capacities: Infrastructure, Data Systems

Type: Book

Supovitz, J. & Klein, V. (2003, November). Mapping a course for improved student learning: How innovative schools systematically use student performance data to guide improvement. University of Pennsylvania Graduate School of Education: Consortium for Policy Research in Education. Retrieved December 14, 2006 from http://www.cpre.org/Publications/Publications_Research.htm

Summary: This report develops a framework to help educators build more efficient and effective systems for data collection and analysis, and explains how school leaders and teachers can use student performance data to guide instruction and decision making.

Description: Knowledge

Capacities: Culture of Collaboration, Expectations for Data Use, Human Resources

Type: Website

Supovitz, J.A. (2005). Small learning communities that actually learn: Lessons for school leaders. Phi Delta Kappan, 86 (9), 649-651.

Summary: This article discusses the benefits and challenges of establishing collaborative communities of teachers, highlighting findings from a study of teacher communities in Cincinnati and Philadelphia. It is often assumed that such communities improve teaching and learning by allowing teachers to know their students better and respond to their needs in more appropriate ways and by facilitating teachers' sharing of practices, thus creating a culture focused on instructional improvement. The authors find in their study, however, that it takes more than creating a community structure to significantly change instructional practices. (Publisher granted permission to post abstract only. Link to article abstract is provided.)

Description: Knowledge

Capacities: Culture of Collaboration, Leadership

Type: Website

Supovitz, J.A. (2006). The case for district-based reform: Leading, building, and sustaining school improvement. Cambridge, MA: Harvard Education Press.

Summary: This book provides an analysis of the role of the school district in enacting large-scale reform. Drawing on the real-life reform efforts of the Duval County school system in Florida, the author discusses the successful practices, as well as common challenges, faced in large-scale school improvement efforts. He also highlights the role of the district in bringing about meaningful and sustainable reform. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Culture of Collaboration, Leadership, Skills and Knowledge

Type: Website

Sutherland, S. (2004). Creating a culture of data use for continuous improvement: A case study of an Edison Project School. American Journal of Evaluation 25(3), 277-293. Retrieved March 2, 2007 from http://aje.sagepub.com/cgi/reprint/25/3/277

Summary: The author outlines how a school can promote and maintain a culture of evaluative practices and data use for continuous improvement. Key findings from a four year study examing comprehensive school reform in twelve Title I schools in three states include the importance of obtaining a multidimensional view of educational change in order to better investigate and evaluate school change efforts. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Culture of Collaboration, Culture of Inquiry, Expectations for Data Use

Type: Website

Symonds, K. W. (2004). After the test: Closing the achievement gaps with data. San Francisco: Bay Area School Reform Collaborative. Retrieved October 16, 2008 from http://www.ncrel.org/gap/studies/basrc.htm

Summary: The Bay Area School Reform Collaborative (BASRC) surveyed 32 K-8 schools in the San Francisco Bay Area in order to identify effective school-level policies and strategies of schools narrowing the achievement gap between higher- and lower-achieving groups of students. Findings from the study are grouped into three categories, including teacher support for use of data, leadership for equity, and school focus. The authors recommend that schools need frequent and reliable data, teachers need support to use data, schools need to hire and promote people of color and opportunities for faculty to discuss how race and ethnicity affect students' experiences in school, and focus on ensuring that students are mastering reading and literacy skills that are the foundation of learning.

Description: Knowledge

Capacities: Action for Learning, Culture of Collaboration, Expectations for Data Use, Leadership, Professional Development, Transformation Plan

Type: Website

T.H.E. Journal. Retrieved April 9, 2006 from http://www.thejournal.com/

Summary: T.H.E. Journal is an online resource designed to help administrators, technology staff and technology proficient educators use technological innovations to improve learning and instruction. The website provides resources and publications on a variety of topics including (but not limited to) professional development, data management, policy/research, integration/networking, and accountability/assessment. Readers can also find relevant and recent news and briefs, opinion pieces, and access to numerous educational resources.

Description: Guidance

Capacities: Professional Development, Skills and Knowledge, Tools

Type: Website

The Datause Website. Retrieved May 1, 2006 from http://edadmin.edb.utexas.edu/datause/publications.htm

Summary: This website provides research based resources about data-driven decision making and software reviews for analyzing student data.

Description: Knowledge

Capacities: Data Systems, Expectations for Data Use, Skills and Knowledge

Type: Website

Thompson, M., & Wiliam, D. (2007). Tight but loose: A conceptual framework for scaling up school reform. Paper presented at the annual meeting of the American Educational Research Association (AERA), April 9 – April 13, 2007, Chicago, IL. *Reprinted by permission of Educational Testing Service, the copyright owner. No endorsement of this web site or publication by Educational Testing Service should be inferred.

Summary: This report discusses a potential approach to improve formative assessment practices called the Keeping Learning on Track (KLT) program, a sustained professional development program for teachers. KLT is based on three components: A content component (i.e., minute-to-minute and day-by-day assessment for learning), a process component (i.e., an ongoing program of school-based collaborative professional learning), and an empirical/theoretical component (i.e., the intervention’s theory of action supported by empirical research). In addition to describing these components in greater detail, the authors present a theoretical framework called "Tight but Loose" to support the implementation of classroom-based interventions. This framework emphasizes the balance between strict principles or a theory of action vs. local opportunities, needs, and constraints. The authors use this framework to discuss the challenges of implementing an intervention in diverse and complex contexts, focusing specifically on the KLT program. The authors conclude that because of the diversity in school contexts, flexibility is needed when implementing or operating any standard intervention program. With that flexibility, however, comes a need for a thorough understanding of the theory behind the intervention so that key components and principles are not lost.

Description: Knowledge

Capacities: Multiple Measures

Type: Website

Thorn, C. A. (2001, November 19). Knowledge management for educational information systems: What is the state of the field? Education Policy Analysis Archives, 9(47). Retrieved April 10, 2006 from http://epaa.asu.edu/epaa/v9n47/

Summary: This article considers the application of knowledge management techniques to educational information systems for reform efforts. The final section is a case study of a data management system in the Milwaukee Public Schools.

Description: Knowledge

Capacities: Data Integrity, Data Systems, Tools

Type: Website

Thorn, C., Watson, J., & Zeyher, S. (2001, June). Analysis of education information systems in the context of systemic reform. (Research Monograph No. 22). Madison: University of Wisconsin, National Institute for Science Education. Retrieved April 10, 2006 from http://facstaff.wcer.wisc.edu/normw/WebbMonograph22.pdf

Summary: This series of papers from the National Institute for Science Education's (NISE) InfoNet study provides an analytic framework "that can be used to describe and project the ways in which data environments function within education systems ... and support systematic reform" (p. 1). Thorn considers the contexts in which education information networks are developed, and how business and industry thinking about information management can be applied to education organizations. Watson and Zeyher argue that systemic reform efforts can benefit from information systems that support the collection, analysis, and use of data for decision-making. The authors present their four-dimensional framework for analyzing education information systems, which addresses user needs, data characteristics, analytic structures, and technical requirements, as a tool for education leaders to compare and contrast information systems. (Permission to post article is pending. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Data Systems, Skills and Knowledge, Tools

Type: Website

Thorn, C.A. (2002). Data use in the school and classroom: The challenges of implementing data-based decision making inside schools. WCER Working Paper No. 2002-2. Madision, WI: Wisconsin Center for Education Research. Retrieved September 22, 2008, from http://www.wcer.wisc.edu/Publications/workingPapers/Working_Paper_No_2002_2.pdf

Summary: This paper explores the challenges teachers and school-level administrators face when implementing research-based decisions about instructional practices in the classroom. The author presents both individual and group strategic decision making models and maintains that no one model is best. The author concludes with recommendations to improve quality decision making, including greater access to data and high quality professional development opportunities, incentives for short- and long-term gains, and an expanded range of strategies to address plans for school improvement.

Description: Knowledge

Capacities: Decision-Making, Professional Expertise, Transformation Plan

Type: Website

Threlfall, J. (2005). The formative use of assessment information in planning- The notion of contingent planning. British Journal of Educational Studies, 53(1), 54-65.

Summary: The purpose of this article is to explore the relationship between assessment information and teacher planning. The author begins by highlighting changes in formative assessment practices in the UK from 1988 to 2003. Threlfall notes that early conceptions of formative assessment included using assessment information to inform instruction, yet now it is less about planning and more about feedback and self-assessment. The author attributes the shift in focus to teachers' lack of pedagogical knowledge and the challenges they faced interpreting student assessment information to plan instruction. Threlfall proposes 'contingent planning' as a way to use assessment data to plan the next steps of instruction without becoming overwhelmed with unnecessary information or challenged by a lack of expertise. Contingent planning includes having more than one teaching possibility and then using a planned assessment to determine which instructional plan is most appropriate for the students. Threlfall argues that this approach is more manageable because the assessment data gathered is consistent with teachers' understanding of a topic and allows a teacher to take different paths of instruction depending on what the students have learned. The author concludes by describing various instances in which this approach will be more or less helpful to teachers, explaining that this approach is particularly useful when teachers expect differences among students that they believe will be important for instructional decisions. (Permission to post article is pending. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Action for Learning

Type: Website

Tierney, R.D. & Charland, J. (2007, April). Stocks and prospects: Research on formative assessment in secondary education classrooms. Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL.

Summary: This paper provides a comprehensive overview of the recent literature on formative assessment practices in secondary classrooms. The authors reviewed thirty empirical research studies using three questions to guide the review: 1) What is the nature of empirical research done recently on formative assessment in the secondary school context?; 2) What can be understood about formative assessment from this body of work?; 3) What priorities are indicated in and by this body of work for further inquiry?. The researchers found that numerous countries were represented in the literature, and articles covered a variety of topics including formative assessment as a practice, specific aspects of formative assessment (e.g., peer and self-assessment), assessment tools or strategies, and efforts to change assessment culture by increasing formative assessment practices. The studies reviewed included quantitative, qualitative, and mixed- methods designs, and researchers employed a variety of methodologies including observations of classrooms, videotapes of lessons, surveys, interviews, and examinations to assess program or experimental effects. Although there was great diversity in the literature, the reviewers did notice a focus on qualitative methods and the subject of Science. Next, the authors examine the studies in relation to five key elements of formative assessment: student involvement, feedback to students, adjustment of teaching, explicit learning goals, and diverse approaches to elicit learning. They found that student involvement and feedback to students were most commonly seen. The authors conclude by highlighting directions for future research in regards to educational context, assessment methods, and student engagement. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Research

Capacities: Multiple Measures

Type: Website

Tierney, R.D. (2006). Changing practices: Influences on classroom assessment. Assessment in Education, 13(3), 239-264.

Summary: The purpose of this review is to examine the complex processes of classroom assessment and how assessment practices are influenced by various sources of information. More specifically, the authors review 24 articles related to classroom assessment, focusing on how knowledge-generation sources (i.e., educational research, evaluative inquiry, large-scale assessment) and mediating sources (i.e., educational policy, professional development, teachers' beliefs), influence assessment practices. The authors include a detailed table summarizing the purpose, methods, data source, participants, and context of each study included in the review. The authors also highlight noticeable trends in the studies including qualitative data, collaboration among researchers and teachers, and adequate time as an important consideration for thorough data collection and to measure policy impacts. The authors conclude that both knowledge-generating sources and more proximate, mediating sources will inform and influence classroom assessment practices so understanding the complex dynamics and interaction between all influences is key to encouraging formative assessment practices to better support student learning. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Research

Capacities: Culture of Collaboration, Multiple Measures

Type: Website

Tognieri, W., & Anderson, S.E. (2003). Beyond islands of excellence: What districts can do to improve instruction and achievement in all schools. Washington, DC: Learning First Alliance. Retrieved April 8, 2006 from http://www.learningfirst.org/publications/districts/

Summary: This study highlights successful reform efforts in five high poverty school districts from across the United States. The researchers found that the districts used a similar set of strategies to improve student achievement and implement effective professional development practices in their schools, including: acknowledging poor performance and making a commitment to change; implementing a system-wide approach to improving instruction; shaping their visions to focus on student learning; adopting new professional development programs to improve instruction; making decisions based on data; redefining leadership roles; and committing to sustained reform efforts. The authors offer a number of recommendations, based on these "lessons learned," for stakeholders, from school boards to union leaders and superintendents, to address instructional reform efforts in their organizations.

Description: Knowledge

Capacities: Culture of Collaboration, Culture of Inquiry, Leadership, Professional Development

Type: Website

Tomlinson, C. A. (2007). Learning to love assessment. Educational Leadership, 64(4), 8-13.

Summary: In this article, Tomlinson describes her changing feelings about assessment practices throughout her teaching career. She first explains how early in her career, classroom management took up most of her attention while classroom assessment took up the least. She then discusses insights about assessment practices she has learned through the years and offers 10 "understandings" about classroom assessment that have impacted her work as an educator. She uses the term "informative assessment" and explains that informative assessment (1) isn't just about the tests, (2) isn't really about the grade book, (3) isn't always formal, (4) isn't separate from the curriculum, (5) isn't about 'after', (6) isn't an end in itself, (7) isn't separate from instruction, (8) isn't just about student readiness, (9) isn't just about finding weakness, and (10) isn't just for the teacher. Each topic is explained in detail. Tomlinson distinguishes between assessment of learning, assessment for learning, and assessment as learning, showing how her understanding of assessment progressed from assessment to evaluate student performance, to assessment to informing teaching, to assessment to inform learning. She concludes that summative assessment does have a place in education, and formative assessments do promote more effective teaching, yet "when assessment is seen as learning- for students as well as teachers- it becomes most informative and generative for students and teachers alike" (p. 13).

Description: Knowledge

Capacities: Multiple Measures

Type: Website

Torrance, H., & Pryor, J. (1998). Investigating formative assessment. Buckingham, UK: Open University Press.

Summary: The purpose of this book is to explore how assessments of young children are conducted in classrooms. The book was developed from interviews with teachers and students about the purposes of assessment, as well as videos and audio recordings of classroom assessment practices. This book includes ideas from assessment policy, learning theories, and the sociology of the classroom to examine the ways in which assessments are implemented and how student learning is impacted in various subject areas as a result of these assessments. The book includes chapters on defining and investigating formative assessment, teachers' ideas about assessment, formative assessment and learning, and ideas for improving formative assessment practices in the classroom.

Description: Guidance

Capacities: Multiple Measures

Type: Book

Tufte, E.R. (2001). The visual display of quantitative information. Cheshire, CT: Graphics Press.

Summary: This classic book on statistical graphics, charts, tables, provides a background of theory and practice of the design of data graphics, and examples of how to display data for precise, effective, quick analysis.

Description: Guidance

Capacities: Data Systems, Quality of Assessment Data, Tools

Type: Book

Tunstall, P., & Gipps, C.V. (1996). 'How does your teacher help you to make your work better?' Children's understanding of formative assessment. The Curriculum Journal, 7(2), 185-203.

Summary: (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: knowledge

Capacities: Multiple Measures

Type: Website

Tunstall, P., & Gipps, C.V. (1996). Teacher feedback to young children in formative assessment: A typology. British Educational Research Journal, 22(4), 389-404.

Summary: This paper presents results from a study conducted by the Institute of Education at the University of London examining the types of feedback given to students, ages 6 and 7, and the ways in which students interpreted this feedback. The researchers conducted field work including classroom observations, interviews with teachers, examination of student work, and tape recordings of classroom dialogue in 6 London schools. Based on their findings, the researchers were able to develop a typology ranging from evaluative to descriptive types of teacher feedback to young children. More specifically, they found four general categories of assessment feedback rewarding, approving, specifying attainment, and constructing achievement. Each of the categories was subdivided to represent a positive and negative response (e.g., rewarding vs. punishing). The researchers found that all teachers exhibited all feedback categories, and some teachers did demonstrate a particular style of feedback response. Furthermore, the researchers observed all feedback categories in all subject area. They propose that these styles can be extended to other age groups as well. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Research

Capacities: Multiple Measures

Type: Website

U.S. Department of Education, Office of Elementary and Secondary Education. (2006, April). Improving data quality for Title I standards, assessments, and accountability reporting guidelines for states, LEAs, and schools. Washington, D.C.: U.S. Department of Education. Retrieved May 10, 2006 http://www.ed.gov/policy/elsec/guid/standardsassessment/nclbdataguidance.pdf

Summary: This document addresses data quality and data use issues associated with Title I educational agencies. It includes detailed information on collecting, managing, analyzing and reporting on data in accordance with the No Child Left Behind Act. Topics covered include: Issues in data quality, technical infrastructure, data definitions, staff organization and training, data collection processes, collection instruments, student assessment data collection, data quality management controls, data review and validation, and data privacy and security issues. The document concludes with an overview of roles and responsibilities for improving data quality and additional resources.

Description: Guidance

Capacities: Data Quality Assurance, Data Systems, Expectations for Data Use, Professional Expertise, Quality of Assessment Data, Skills and Knowledge

Type:

U.S. Department of Education. (2003, February). U. S. Department of Education information quality guidelines. Retrieved May 19, 2006, from http://www.ed.gov/policy/gen/guid/iq/infoqualguide.pdf

Summary: These guidelines reflect the Department of Education's policies and procedures for reviewing the quality of information it disseminates. The report includes detailed information on how the department ensures the information it disseminates is accurate, reliable and high-quality.

Description: Guidance

Capacities: Data Quality Assurance, Quality of Assessment Data, Skills and Knowledge

Type: Website

Using data. Retrieved May 1, 2006, from http://usingdata.terc.edu/toplevel/home.cfm

Summary: The purpose of this project funded by the National Science Foundation is to provide professional development experiences and sustained learning opportunities to help educational leaders develop the skills, knowledge, and abilities to effectively use science and mathematics student learning data to improve programs, policies, and learning in all classrooms.

Description: Service

Capacities: Culture of Collaboration, Professional Development, Skills and Knowledge

Type: Website

Valli, L. & Buese, D. (2007). The changing role of teachers in an era of high-stakes accountability. American Educational Research Journal, 44(3), 519-558.

Summary: This article examines changing expectations for elementary school teachers' roles and responsibilities in and out of the classroom. After reviewing the changes in required teacher tasks over the past four years, the authors found that because of federal, state, and local policies designed to improve student performance, teachers' work has increased and expanded in four areas: instructional, institutional, collaborative, and learning. The authors find that these increasing demands have unintended costs including negative impacts on pedagogy and teachers' relationships with students. Additionally, many teachers report feeling stressed, frustrated, and overwhelmed by these increasing demands. The authors further illustrate these challenges by examining one policy directive, differentiated instruction, across numerous academic subjects to demonstrate how these demands impact teachers' professional well-being. The authors conclude that more research needs to be done to address the relationship between external educational policies and teacher roles in and out of the classroom. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Research

Capacities: Human Resources

Type: Website

Van Petegem, P., Vanhoof, J., Daems,F., Mahieu, P. (2005). Publishing information on individual schools? Educational Research and Evaluation, 11 (1), 45-60. Retrieved September 15, 2008 from http://web.ebscohost.com/ehost/pdf?vid=2&hid=106&sid=16143c7a-79e7-4891-9fb3-dc683da76594%40sessionmgr109

Summary: In this article, the authors explore the complexities of publishing individual school data in a public forum and present arguments for and against these practices. Their arguments are based on interviews with educational experts from England, Scotland, the Netherlands, and France, and document analysis of publications by the governments, media, and commercial groups in these countries. The authors found four main reasons supporting publication of individual school data: 1) meeting informational needs of parents, students, teachers, school administrators and government officials; 2) increasing school accountability; 3) creating market mechanisms in education (sometimes viewed as a negative result); 4) and encouraging discussions of educational equity. In publishing school data, the researchers stress that schools must avoid unintended effects (e.g., increased competition and decreased cooperation, over-emphasis on test results, etc.), consider criticisms of the existing system (e.g., publications focusing solely on test results, raw data unadjusted for student populations, etc.), and make sure the data are reliable, valid and accessible. The authors conclude that while it may be beneficial to publish individual school data, care must be taken to do so appropriately. (Publisher granted permission to post abstract only. Link to article abstract is provided.)

Description: Knowledge

Capacities: Data Systems, Multi-level Data Use, Reporting

Type: Website

Villano, M. (2007, September). Data-driven decision making: The "other" data. T.H.E. Journal, 34(9), 46-52. Retrieved October 28, 2008 from http://www.thejournal.com/articles/21237_7

Summary: This article explores the importance of collecting a variety of data to support data-driven decision making in schools. The author argues that while student performance data helps to assess student learning, other forms of data should also be collected to better highlight trends and school processes. For example, collecting data to show student demographics, perceptions (i.e., what students, teachers and parents think about the learning environment), and school processes (e.g., school programs, classroom practices) may provide a better and more accurate picture of how a school districting is performing. The author also describes U.S. school district initiatives to help teach educators how to collect and use data effectively. Overall the author maintains that collecting a variety of data will better inform the data-driven decision making process.

Description: Guidance

Capacities: Decision-Making, Expectations for Data Use, Multiple Measures

Type: Website

Vygotsky, L. S. (1962). Thought and language. Cambridge, MA: The MIT Press.

Summary: In his classic book, Vygotsky explores cognitive development by exploring the relationship between words and cognition. Vygotsky argues that speech is social in nature and only becomes internalized verbal thought as children develop. He maintains that inner speech (turning words into thoughts) is qualitatively different from external or social speech (turning thoughts into words) and that thought is mediated by language. This exploration of the relationship between language development and thought is one of Vygotsky's most well-known contributions to cognitive science.

Description: Knowledge

Capacities: Multiple Measures

Type: Book

Walberg, H. J. (2007). Handbook on restructuring and substantial school improvement. Lincoln, IL: Center on Innovation & Improvement. Retrieved August 7, 2007 from http://www.centerii.org/handbook/download/Restructuring%20Handbook.pdf

Summary:

Description:

Capacities:

Type: Website

Wall, J. E., & Waltz, G. R. (Eds.). (2004). Measuring up: Assessment issues for teachers, counselors, and administrators. Austin, TX: PRO-ED, Inc.

Summary: Wall and Waltz explore various assessment concepts by addressing basic information about testing and issues related to assessment in schools and classrooms. The book includes chapters explaining basic testing concepts (e.g., types and uses of tests, reporting and interpreting test results), assessment issues for special populations and audiences (e.g., test and item bias, assessment of students with disabilities or limited language proficiency), and special topics and issues in assessment (e.g., ethics in educational assessment, assessment for learning). The book also addresses the future of assessment (e.g., computer adaptive testing) and provides additional assessment resources for further research and information. This book is designed for teachers, counselors, and school administrators who wish to improve their understanding and knowledge of educational assessment.

Description: Guidance

Capacities: Data Systems, Quality of Assessment Data, Skills and Knowledge

Type: Book

Watson, J. (2002, April). QSP and the MPS information system. Paper presented at the Annual Meeting of the American Educational Research Association, New Orleans, LA. (ERIC Document Reproduction Service No. ED472177)

Summary: This report describes technology tools/systems used in the Milwaukee Public Schools (MPS) and the implementation of a database tool, Quality School Performance (QSP), to illustrate how multiple tools can be used concurrently to support data informed decision-making. The report is composed of three main sections. The first section reviews MSP technology providing an overview of the system and describing key features. The second section presents a four-dimensional framework that describes how QSP compares and contrasts with other information systems. In the third and final section, the authors evaluate the effectiveness of QSP and other MPS tools used in data-driven decision making and conclude that QSP best meets user needs at the school level. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Data Systems, Professional Development

Type: Website

Wayman, J. C. & Conoly, K. (2006). Managing curriculum. Rapid implementation of a districtwide data initiative. ERS Spectrum, 24(2), 4-8. Retrieved October 16, 2008 from http://edadmin.edb.utexas.edu/datause/publications.htm

Summary: This paper describes the process of implementing a curriculum management system that supports both a district curriculum and data initiative in the Corpus Christi Independent School District (CCISD) in Texas. The authors discuss the selection and implementation process, the specific challenges and successes that the district encountered, and questions regarding future sustainability. The district's experience is outlined in five parts: choosing a system, implementing the CMS and Data Initiative, professional development, SchoolNET usage statistics, and discussion of the process in its entirety.

Description: Knowledge

Capacities: Culture of Inquiry, Data Systems, Professional Development

Type: Website

Wayman, J. C. & Stringfield, S. (2003). Teacher-friendly options to improve teaching through student data analysis. Paper presented at the 2003 Annual Meeting of the American Association for Teaching and Curriculum, Baltimore, MD, October. Retrieved January 10, 2007 from http://www.csos.jhu.edu/datause/papers/aatc_teacher-friendly_software.pdf

Summary: This paper explores the accessibility of student data for teachers and potential software programs to facilitate data-driven instruction and decision making in schools. The authors maintain that while adequate data for decision making exist, they must be made more accessible to teachers in order to inform classroom practice. The authors argue that this can be accomplished by acquiring effective software programs. The authors include numerous commercially-available software products and describe ideal features (e.g., quick access to information, affordable, accessible to all levels of educators, etc.) of these programs. While the authors admit that no one program will meet all needs of a school, they maintain that some are more comprehensive than others and should be evaluated given the needs of the school and teachers.

Description: Tool

Capacities: Data Systems, Human Resources, Skills and Knowledge

Type: Website

Wayman, J. C. & Stringfield, S. (2006). Technology-supported involvement of entire faculties in examination of student data for instructional improvement. American Journal of Education 112(4), 549-571. Retrieved December 1, 2006 from http://edadmin.edb.utexas.edu/datause/papers/Wayman-Stringfield-Faculty-Data-Use.pdf

Summary: The authors examine the efforts of three elementary and middle schools to involve faculty in the examination of student data, supported by efficient data and computer-adaptive assessment systems. The results indicate the importance of administrative support, professional development, and a school climate conducive to data use.

Description: Knowledge

Capacities: Culture of Inquiry, Expectations for Data Use, Multi-level Data Use

Type: Website

Wayman, J. C. (2005). Guest editor's introduction. Journal of Education for Students Placed at Risk (JESPAR) 10(3), 235-239. Retrieved February 22, 2007 from http://www.leaonline.com/doi/pdf/10.1207/s15327671espr1003_1

Summary: This special issue concerns the use of student data for educational improvement. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Capacity Building, Expectations for Data Use, Transformation Plan

Type: Website

Wayman, J. C. (2005). Involving teachers in data-based decision-making: Using computer data systems to support teacher inquiry and reflection. Journal of Education for Students Placed At Risk, 10(3), 295-308. Retrieved April 9, 2006 http://edadmin.edb.utexas.edu/datause/publications.htm

Summary: This paper explores teacher participation in data-driven decision making and increasing access to various types of student information. The author focuses on data-warehousing, the collection and organization of all relevant student information into one system. Wayman argues that if data are easily available and accessible to all teachers regardless of technological proficiency, this information can then be used to improve teaching and learning in the classroom. The author also reports briefly on commercially available software and includes suggestions (based on past research on data use) to better support teachers using these data systems, including small-scale, context-relevant, ongoing professional development opportunities, strong leadership, and collaboration with colleagues.

Description: Guidance

Capacities: Data Systems, Professional Development

Type: Website

Wayman, J. C. (2007). Student data systems for school improvement: The state of the field. In TCEA Educational Technology Research Symposium: Vol. 1 (pp. 156-162). Lancaster, PA: ProActive Publications. Retrieved October 16, 2008 from http://edadmin.edb.utexas.edu/datause/publications.htm

Summary: This report provides a broad overview of the use and utility of student data systems in educational settings. The author provides descriptions of different types of student data systems, highlighting important features of such systems, their implementation in educational settings, and future efforts to use technology to improve student learning. The author concludes that these data systems have the potential to improve educators' abilities to manage and report student data, and therefore to effectively use data for educational improvement.

Description: Knowledge

Capacities: Data Systems, Multi-level Data Use, Multiple Measures, Tools

Type: Website

Wayman, J. C., Midgley, S. & Stringfield, S. (2006). Leadership for data-based decision-making: Collaborative data teams. In Danzig, A., Borman, K., Jones, B., & Wright, B. (Eds.), New models of professional development for learner centered leadership. Erlbaum. Retrieved October 16, 2008 from http://edadmin.edb.utexas.edu/datause/publications.htm

Summary: This paper examines the use of collaborative data teams, that is, the shared involvement among teachers, administrators and other school personnel to use student learning data to improve teaching and learning. The authors maintain that effective collaborative teams require alignment of school goals and standards among all school personnel, a focus on student learning data, continued engagement in the process (without imposing excessive structure or rules), and adequate technology and technical support to allow access to all relevant data. They also include suggestions for leaders to help facilitate the process of building collaborative teams. The authors conclude with a discussion of implications for policy and practice.

Description: Knowledge

Capacities: Professional Development, Skills and Knowledge

Type: Website

Wayman, J. C., & Cho, V. (2008). Preparing educators to effectively use student data systems. In Kowalski, T. J., & Lasley, T. J. (Eds.), Handbook on data-based decision-making in education (pp. 89-104). New York: Routledge.

Summary: The purpose of this chapter is to present key issues for district leaders and educators to consider when implementing district-wide data systems. The authors argue that while data systems are an essential part of effective data use, there are many additional issues to consider when implementing these systems. Wayman and Cho present a brief overview of computer data systems and describe how to prepare educators for effective system use. They also discuss ongoing professional development and collaboration among educators at all levels to better support the implementation of data systems. Wayman and Cho discuss institutional structures (i.e., system access, time to engage in data use, principal leadership, and "go-to" individuals to support others) and maintain that these supports are necessary and will improve data use and data systems. They conclude by reiterating that the presence of a data system does not guarantee effective data use, so in addition to financial assistance, these data initiatives require substantial district contributions in terms of support and structure.

Description: Guidance

Capacities: Culture of Collaboration, Data Systems, Professional Expertise

Type: Website

Wayman, J. C., Brewer, C., & Stringfield, S. (2009, April). Leadership for effective data use. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.

Summary:

Description:

Capacities:

Type: Website

Wayman, J. C., Cho, V., & Johnston, M. T. (2007). The data-informed district: A district-wide evaluation of data use in the Natrona County School District. Austin: The University of Texas. Retrieved October 16, 2008 from http://edadmin.edb.utexas.edu/datause/publications.htm

Summary: This report summarizes findings from a district-wide evaluation of data use and data practices. Drawing on interview and survey data, the study examines the culture, expectations, and use of data at every level of the Natrona County School District. The report highlights key findings on data use, opinions, and technological resources. Based on their findings, the researchers provide a set of recommendations to assist the school district in establishing a plan for useful, effective data use. The report also includes an appendix written by district personnel which captures their reactions, current efforts, and future plans 90 days after receiving the report.

Description: Knowledge

Capacities: Action for Learning, Culture of Collaboration, Culture of Inquiry, Multi-level Data Use, Stakeholder Collaboration, Transformation Plan

Type: Website

Wayman, J. C., Stringfield, S., & Yakimowski, M. (2004). Software enabling school improvement through the analysis of student data. Baltimore, MD: Johns Hopkins University Center for Research on the Education of Students Placed At Risk. Retrieved April 9, 2006, from http://edadmin.edb.utexas.edu/datause/publications.htm

Summary: This report, sponsored by the Center for Research on the Education of Students Placed at Risk (CRESPAR), examines issues related to the use of student data and data driven decision-making and presents reviews of commercially available software for analyzing student data. The authors discuss issues in software implementation such as the assessment of a school's basic data needs, the time and cost of implementation, the choice of vendor, and the School's Interoperability Framework (SIF). The authors also provide detailed commentary on important features of efficient software for student data analysis and review specific software programs. The authors maintain that the lack of data-informed decision-making in schools is due to the fact that the data available to schools are not stored in an efficient and accessible way. They suggest that if schools plan to implement/adopt data-informed practices, all relevant data must be organized and accessible for both teachers and administrators in order to do so effectively.

Description: Tool

Capacities: Data Systems, Human Resources, Skills and Knowledge

Type: Website

Wayman, J.C., & Stringfield, S. (2006). Data use for school improvement: School practices and research perspectives. American Journal of Education 112(4), 463-468.

Summary: This introduction to a special issue of the American Journal of Education on data use for school improvement outlines five articles drawn from the 2005 Annual Meeting of the American Educational Research Association that describe the experiences of schools and districts pursuing various ways of using student data to inform educational practices. The article topics include data use at the district level, teacher involvement in data-informed decision making, and one district’s experience establishing a formative assessment system. The introduction also builds on the five articles included in this special issue of the American Journal of Education and includes many potential avenues for future research. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Guidance

Capacities:

Type: Website

WestEd. (2002). Improving districts: Systems that support learning. San Francisco: Author.

Summary: This report commissioned by the U.S. Department of Education reviews nine districts that were honored for their outcomes and practices by the department's National Awards Program for Model Professional Development. These successful school systems are characterized by structured teacher professional development, community-wide vision of continous improvement, and an emphasis on data-driven decision-making.

Description: Knowledge

Capacities: Communication, Culture of Inquiry, Professional Development

Type:

Wiggins, G. P. (1998). Educative assessment: Designing assessments to inform and improve student performance. San Francisco: Jossey-Bass. 978-0787908485

Summary: This book covers all aspects of assessment design, including how to craft performance tasks that meet rigorous educational standards, how to score assessments fairly, and how to structure and judge student portfolios. It also looks at how performance assessment can be used to improve curriculum and instruction, grading and reporting and teacher accountability.

Description: Guidance

Capacities: Professional Development, Skills and Knowledge, Tools

Type: Book

Wiliam, D. (2006, July). Does assessment hinder learning? Paper presented at the ETS Invitational Seminar at the Institute of Civil Engineers, London, UK.

Summary: This paper explores how assessment affects student learning. The author first reviews research and field work in the UK and USA to identify five assessment strategies that, when used appropriately, support, rather than hinder, learning. Strategies include the following: 1) Engineering effective classroom discussions, questions, activities and tasks that elicit evidence of student learning; 2) Providing feedback that moves learning forward rather than responses that have a negative impact; 3) Clarifying and sharing learning intentions and success criteria with learners; 4) Activating students as owners of their own learning; and 5) Activating students as learning resources for one another through cooperative learning activities. Overall, Wiliam maintains that assessment information should be used by students and teachers to guide and improve learning and instruction in the classroom. He also discusses the need for cost-effective educational reforms and argues that creating teacher learning communities may be an effective strategy to foster change in teacher practice. Wiliam concludes by discussing the need for a balanced assessment system and creating high-quality, motivating assessments. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Guidance

Capacities: Communication, Culture of Collaboration, Culture of Inquiry, Expectations for Data Use, Multiple Measures, Professional Development, Professional Expertise, Reciprocal Feedback, Skills and Knowledge

Type: Website

Wiliam, D. (2007). Changing classroom practice. Educational Leadership, 64(4), 36-42.

Summary: This article provides recommendations for incorporating formative assessment practices in the classroom. Wiliam begins by discussing recent changes to educational policy (e.g., NCLB) and school organization (e.g., Charter schools) that were designed to improve student achievement. He maintains, however, that formative assessment, when carried out correctly, is the only development that leads to improved student learning. After working with schools throughout the United States, Wiliam reports that changes in classroom practice are difficult to achieve because many teachers have a hard time changing their traditional strategies. Furthermore, he argues that although many teachers are knowledgeable about the strategies shown to improve learning, they may not know how to implement them in the classroom. Wiliam suggests Teacher Learning Communities (i.e., small teacher teams in which members support their colleagues' efforts to change classroom practice) as an approach to support the implementation of formative assessment strategies. Wiliam provides numerous recommendations (e.g., meet monthly for at least 75 minutes, start with volunteers) to establish these groups. He also includes an example plan of action and an outline to help structure or organize meetings (e.g., introduction 5-10 minutes, how's it going? 30-50 minutes, etc.). Wiliam concludes that formative assessment practices have the potential to improve student performance, yet teachers need effective support to implement these strategies. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Guidance

Capacities: Culture of Collaboration

Type: Website

Wiliam, D. (2007). Keeping learning on track: Classroom assessment and the regulation of learning. In F.K. Lester Jr. (Ed.), Second handbook of mathematics teaching and learning, 1053-1098, Greenwich, CT: Information Age Publishing.

Summary:

Description: Knowledge

Capacities: Multiple Measures

Type: Book

Wiliam, D., & Thompson, M. (2007). Integrating assessment with learning: What will it take to make it work? In C.A. Dwyer (Ed.), The future of assessment: Shaping teaching and learning. Mahway, NJ: Lawrence Erlbaum Associates.

Summary:

Description: Research

Capacities:

Type: Book

Wiliam, D., Lee, C., Harrison, C., & Black, P. (2004, March). Teachers developing assessment for learning: Impact on student achievement. Assessment in Education Principles Policy and Practice 11(1), 49-65.

Summary: The purpose of this study was to examine the effects of teacher-developed formative assessment strategies on student achievement. The researchers recruited twenty four teachers and supported them over a 6 month period as they learned about principles underlying formative assessment and developed their own formative assessment plans. The researchers also observed each teacher's strategies and techniques related to formative assessment in the classroom and provided opportunities to discuss ideas and improve practice. After comparing student achievement data in classrooms with the teacher-implemented formative assessments to student performance in other classrooms at the same schools, the researchers concluded that formative assessment does have a positive effect on student achievement measures, particularly in terms of externally mandated assessments. (Publisher granted permission to post abstract only. Link to article abstract is provided.)

Description: Research

Capacities: Multiple Measures

Type: Website

Williams, T., Kirst, M., Haertel, E., et al. (2005). Similar students, different results: Why do some schools do better? A large-scale survey of California elementary schools serving low-income students. Mountain View, CA: EdSource. Retrieved April 9, 2006 from http://www.edsource.org/pdf/simstusumm06.pdf

Summary: Report explains why some California elementary schools, serving mostly low-income students, score significantly higher on the state's academic performance index (API) than schools with similar student populations. In particular, this report indicates that there is a high correlation between districts and school administrators who use student assessment data to improve instruction and learning, and higher-performing schools.

Description: Knowledge

Capacities: Culture of Collaboration, Culture of Inquiry, Expectations for Data Use

Type: Website

Wilson, Mark (Ed.). (2004). Towards coherence between classroom assessment and accountability. Chicago, IL: The University of Chicago Press.

Summary: This book examines the relationship between classroom assessment and accountability practices in education. The authors maintain that most assessments used in classrooms today are designed, created, and evaluated by teachers. They argue that in order to improve the usefulness of assessment in schools, a two-way information exchange should be utilized in which school/district accountability assessments become relevant for classroom instruction and classroom assessment is used to evaluate system accountability. The book is divided into four sections. The first section is a general introduction, while the second section includes descriptions of five accountability systems, each developed in partnership with teachers and based on daily classroom assessments. The third section includes commentary chapters evaluating the five systems or addressing other issues relevant to assessment and accountability. The book concludes with a summary and discussion of overall coherence between classroom assessment and accountability. This book is designed for educational researchers, education professionals, and policymakers.

Description: Guidance

Capacities: Multiple Measures

Type: Book

Wurtzel, J., Marion, S., Perie, M., & Gong, B. (2009). The role of interim assessments in a comprehensive assessment system. In L.M. Pinkus (Ed.) Meaningful Measurement: The Role of Assessments in Improving High School Education in the Twenty-First Century (pp. 77-94). Washington, DC: Alliance for Excellent Education.

Summary:

Description:

Capacities:

Type: Website

Yackel, E., Cobb, P., & Wood, T. (1991). Small-group interactions as a source of learning opportunities in second-grade mathematics. Journal for Research in Mathematics Education, 22(5), 390-408. *Reprinted with permission from Journal for Research in Mathematics Education, copyright 1991 by the National Council of Teachers of Mathematics. All rights reserved.

Summary: This study examined small-group interactions as a tool for mathematics instruction in a second-grade classroom. The teacher used small-group problem solving activities followed by whole-class discussions as the primary instructional strategy for all math topics and skills throughout the school year. The authors describe how they devised math activities and collected their data, and they highlight the ways in which the teacher established expectations and classroom norms for cooperative group learning, without providing an external reward for individuals or groups. The researchers found that when children were engaged in small-group learning activities, they had opportunities for learning through collaboration and consensus seeking that may not be present in a more traditional classroom setting. Examples from the classroom are included to illustrate these findings. The researchers conclude that creating classroom norms for small-group expectations and cognitively-based educational activities designed to be challenging to students at a variety of levels are key features of a cooperative learning environment without extrinsic rewards.

Description: Research

Capacities: Multiple Measures, Professional Expertise, Skills and Knowledge

Type: PDF

Young, V. M. (2006). Teacher's use of data: Loose coupling, agenda setting, and team norms. American Journal of Education 112(4), 521-548.

Summary: Using an embedded systems perspective, the author explores the influence of grade-level group norms and district and school leadership on teachers' data use practices in four schools located in two different districts. Findings indicate that the extent to which LEAs establish clear reasons for using data, model data use, and structure time for teachers to learn about and use data impacts teachers' collaborative use of data in schools. (Publisher did not grant permission to post article. Link to publisher's website is provided.)

Description: Knowledge

Capacities: Culture of Collaboration, Culture of Inquiry, Expectations for Data Use, Leadership, Professional Development

Type: Website

Zhao, Y., & Frank, K.A. (2003). Factors affecting technology uses in schools: An ecological perspective. American Educational Research Journal, 40(4), 807-840. Retrieved April 9, 2006 http://www.msu.edu/~kenfrank/papers/Factors%20affecting%20technology%20uses%20in%20schools.pdf

Summary: The researchers use an ecological metaphor to consider the factors that affect the use of technology in schools.

Description: Knowledge

Capacities: Culture of Inquiry, Human Resources, Skills and Knowledge

Type: Website

Zimmerman, B.J. (1986). Becoming a self-regulated learner: Which are the key subprocesses? Contemporary Educational Psychology, 11(4), 307-313.

Summary: This article is an introduction to a special issue of Contemporary Educational Psychology focusing on various sub-processes in academic self-regulation (e.g., metacognition, self-verbalization, etc.). The author begins by explaining that self-regulation theory focuses on how students activate and alter their own learning practices in different contexts and contrasts this theoretical stance to other approaches that emphasize teacher/school quality or student intelligence. Zimmerman explains that self-regulated learners monitor their own learning, view themselves as competent and autonomous, and select and create environments that support learning. Next, the author reviews the extant literature demonstrating the importance of self-regulated processes on student learning and achievement. Then he introduces the articles included in the special issue of Contemporary Educational Psychology. Overall, Zimmerman maintains that this may be an effective theoretical framework to guide research aimed at better understanding student learning processes. (Permission to post article is pending. Link to publisher's website is provided.)

Description: Guidance

Capacities: Multiple Measures

Type: Website