25 Formative and Stealth Assessment

Presentation on theme: "25 Formative and Stealth Assessment"— Presentation transcript:

1 25 Formative and Stealth Assessment
Section III Assessment and Evaluation 25 Formative and Stealth Assessment 形成性和隐形评估 (Valerie J. Shute and Yoon Jeon Kim) 万海鹏 School of Educational Technology, Faculty of Education Beijing Normal University

2 Abstract Classroom and other high-stakes assessments are used for purposes of grading, promotion, and placement, but not to enhance learning. In this chapter, we focus on formative assessment which posits that assessment should: (a) encourage and support, not undermine, the learning process for learners and teachers; (b) provide formative information whenever possible (i.e., give useful feedback during the learning process instead of a single judgment at the end); and (c) be responsive to what is known about how people learn, generally and developmentally.

3 Outline Introduction Measurement and Assessment
Summative and Formative Assessment Formative and Stealth Assessment Conclusion

4 学生不应该仅仅是被评估的主体，而应该成为评估的受益者，通过评估来引导 和促进他们的学习(NCTM,2000)。 NCLB法案自2001年实施以来，标准化测试变得十分流行，由此通往学校改革 的道路变成对频繁标准化考试的强烈需求(Stiggins,2002)。 当前评估实践的问题在于对源自评估的信息没有得到应有的利用，不管是从支 持教师教学还是学生学习的角度而言(e.g., Shute,2007 ; Symonds,2004 ; Wiliam & Thompson,2007 )。 尽管有大量研究表明，这种形成性评估方法能够很好地支持学生的学习，但目 前在课堂中仍然缺乏大规模的应用案例，原因在于形成性评估的实施复杂并往往被 人误解为总结性评估。形成性评估是一个基于测试的过程，而不仅仅是一次测试。 本章主要系统阐述形成性评估的相关内容，并介绍了一种实施形成性评估的方 式——stealth assessment，以更好地将形成性评估带入课堂。 美国数学教师委员会The National Council of Teachers of Mathematics No Child Left Behind (NCLB) Act 1. Introduction

5 2. Measurement and Assessment
summative assessment formative assessment 测量的目的在于鉴别学生之间 的不同差异，并根据其成就高 低进行排序 Based On 测量并伴随有意义的反馈，有 益于指导教学和支持学生学习 Reliability refers to the consistency of assessment results—the degree to which they rank order students in the same way. Validity refers to the extent to which the assessment accurately measures what it is supposed to measure, and the accuracy of the inferences made from task or test results to underlying competencies. 衡量评估质量 的核心指标 不仅聚焦于对学生能力的有效准确测量，同时还利用这些测量信息进行诊断和教学支持。 2. Measurement and Assessment

6 3. Summative and Formative Assessment
Assessment-for-learning model(formative assessment) is often implemented in a non-standardized and hence less rigorous manner than summative assessment, and thus can hamper the validity and consistency of the assessment tools and data(Shute & Zapata-Rivera,2010). Benefits of Summative Assessment 1 基于指定的标准，对不同群体学习绩效进行比较 2 为不同层面（班级、学校、地区、州和国家）和不同受益者（学习者、教师、管理者）提供可信的数据（分数） 3 影响教育政策（课程和融资决策等） 3. Summative and Formative Assessment

7 3. Summative and Formative Assessment
形成性评估过程涉及的所有信息当中，最为核心的当属那些被教师(或FA系统)以及学生用于改进教学和学习的反馈信息。 FA System: (for teacher) Implement the full range of assessment types Gather evidence Make sense of the data Adjust instruction accordingly (for learner) improve motivation, volition, self-efficacy, problem-solving skills FA过程的两种理解 Teacher-centric(formal):任务管理、学习测验、学习结果收集，以及基于数据 分析的反馈或教学活动改进； Student-centric(informal):在各种学习环境中自发进行的互动课堂活动和讨论。 3. Summative and Formative Assessment

8 3. Summative and Formative Assessment
常模参照 标准参照 在末期 (单元、学年) 每月、每周、每天 测验、个人档案、自评以及自我展示 形成性评估在课堂以及计算机环境下的有关研究，但只是简要介绍了研究的相关结果，对具体的研究设计没有细致说明。 课堂研究结果表明：成就、自我效能、动机；对于低成就学生的帮助 计算机支持的学习环境下的形成性评估：ASSISTmen系统、ACED系统； 反馈形式以及质量的重要性 客观选择，关注信效度 整体或局部诊断 正误或得分 3. Summative and Formative Assessment

9 3. Summative and Formative Assessment

10 4. Formative and Stealth Assessment
When formative assessments are seamlessly woven into the learning environment and are thus invisible to learners, we call this stealth assessment (Shute,2011 ; Shute, Ventura, Bauer, & Zapata-Rivera,2009 ) . Two major question: how to make sense of rich data collected the best way to communicate a variety of student-performance information in a way Key Elements of approach: Evidence-centered assessment design(ECD) (Mislevy et al.,2003 ) Formative assessment and feedback to support learning(Black & Wiliam,1998 ; Shute,2008) the basis for instructional decisions, such as the delivery of tailored content to learners(e.g., Shute & Towle,2003 ; Shute & Zapata-Rivera,2008 ) 评估的最高境界不是所谓的收集数据，然后提供所需的分析结果；而是把学习和评估融为一体，让评估消失在学习过程当中。 一般而言，隐形评估可以通过自动评分和机器推理来实现 4. Formative and Stealth Assessment

11 4. Formative and Stealth Assessment
Competency model defines variables related to students’ knowledge, skills, abilities Evidence model provides detailed instructions about (a) what the student says or does that can count as evidence for those skills (Steinberg & Gitomer,1996 ) , and(b) how the evidence statistically links to variables in the competency model (Mislevy,1994 ) . Task/action models express situations that can evoke required evidence. Assembly model specifies how the competency, evidence, and task/action models work together to form a valid assessment. Conceptual assessment framework of ECD (adapted from Mislevy et al.,2003 ) 4. Formative and Stealth Assessment

12 4. Formative and Stealth Assessment
Example of a Stealth Assessment Oblivion (The Elder Scrolls IV: Oblivion,2006, by Bethesda Soft works) Illustrative competency model for Oblivion(from Shute et al.,2009 ) 渡河任务 4. Formative and Stealth Assessment

13 4. Formative and Stealth Assessment
Evidence model Bayes net estimates from levitating over the river(from Shute et al.,2009 ) 4. Formative and Stealth Assessment

14 5. Conclusion 教师应该掌握的专业技能：
(a) diagnose students 'competencies (at various grain sizes) based on different sources of information, (b) figure out what to do next given the obtained data, and (c) build up and employ a pool of rich tasks, probing questions, and other instructionally fruitful activities that can serve to elicit more evidence to inform student models and concurrently support students’ learning. 形成性评估在课堂中运用的十大建议： Cognitive research Complex tasks Learning goals Administration 尽管形成性评估和隐形评估具有很大的吸引力，但是要想在课堂教学中大规模运用依然面临很大挑战。（教师需要） Feedback Personal accountability Evidence-based diagnosis Preplan questions and paths Leverage prior knowledge Collaboration 5. Conclusion

15 5. Conclusion Challenges of Implementing stealth assessment：
(a) The competency model, for example, must be developed at an appropriate level of granularity to be implemented in the assessment.,能力模型 (b) developing the evidence model can be rather difficult in a gaming environment when students collaborate on completing quests,证据/行为模型 (c) scoring qualitative products such as essays, student reflections, and online discussions where there remains a high level of subjectivity even when teachers are provided with comprehensive rubrics.质性作品的评分 Shute, V. J., Ventura, M., Bauer, M. I., & Zapata-Rivera, D. (2009).Melding the power of serious games and embedded assessment to monitor and foster learning: Flow and grow. In U. Ritterfeld, M.Cody, & P.Vorderer (Eds.),Serious games: Mechanisms and effects(pp. 295–321). Mahwah, NJ: Routledge, Taylor and Francis. 总之本章主要介绍了形成性评估以及隐形评估的相关内容，所谓隐形评估可以理解为形成性评估在沉浸式环境中的应用；两种评估都能帮助收集学习者的有效数据并促进学习者学习，同时还能减轻教师在管理学生方面的工作负担。 拓展材料 Reference: Handbook of Research on Educational Communications and Technology(4th) 5. Conclusion

16 Thank You! 万海鹏 Doctoral Candidate of Educational Technology