The Application of Web-based Peer Evaluation System 國立台灣師範大學人力資源與發展研究所 The Application of Web-based Peer Evaluation System Yu-Hui Tao National University of Kaohsiung ytao@nuk.edu.tw March 13, 2009 (Your way) 2018/9/19

Motivation Started teaching in 1997 Initial impression of Taiwanese students: low class participation despite what I have done Started adding various activities in class design for increasing students’ responsibility through their performance evaluation Often asked students to criticizing others’ work/statements/… In some courses, students were asked to write down their comments to the presenters for me to evaluate, and sometimes I cut and past these comments to the presenters 2018/9/19

Outline Concept of Peer Evaluation System (PES) Comparison with Online Learning System and Instant Response System PES Use Case Diagram and Process Flow Screenshots PES Examples PES Experience Sharing 2018/9/19

PES Concept in Class Assignment 2018/9/19

Peer Evaluation in Literature 陶幼慧、李兆庭，同儕作業評量之教學策略執行平台，電子商務與數位生活研討會，3/17，台北實踐大學，2007。 同儕評量可培養學生的自主性及高層次思考技能(Heywood, 2000; Tsai, et al., 2001; Tsai, Lin and Yuan, 2002)，也屬於同儕教學的一種，可發展評估、證明正當性以使用學科的知識的能力(Donaldson and Topping, 1996)。 同儕評量的理論支柱包含(徐雍智、蔡今中及陳明璋，2005) Vygotsky 的社會建構主義 Piaget 的認知衝突理論 以及從其他同儕輔助學習而來的教育 理論 其影響範圍包含了認知、後設認知、情意、社會與傳達技能、整體受益等方面向 包括寫作、教學、商務、科學、工程到醫學方面(Falchikov, 1995; Freeman, 1995; Topping, 1998) 一般而言，形成性評量提供意見，而總結性(summative)評量提供分數(Bostock, 2000)。 2018/9/19

Peer Evaluation in Literature 更有學者推薦書面與口頭同儕回饋的運作，因為學生在書面評量時僅專注於評論的撰寫，而口頭報告中學生還同時會質詢、解釋及給建議的活動(Berg, Admiraal and Pilot, 2003)。 同儕評量可能的好處除了成本效益外，最重要的是教師可管理其程序的進行而非自己評量大量學生的作品(Race, 1998)，並可透過網際網路科技來協助同儕評量的進行。 儘管如此，Wong and Ng (2005)的研究指出許多的教學管理系統並不支援正式 的評量(CELT, 2004)，但學生很滿意線上的作業繳交與網路同儕評量的使用。 2018/9/19

Peer Evaluation in Literature 2018/9/19 (Topping, 1998; 林珊如等人; 2000)

Peer Evaluation in Literature 整體而言，在Topping (1998) 分析的109 篇同儕評量的研究中得知，具一定的信度與效度，且其的應用在高等教育許多的領域中已經被證實有正面的學生滿意度與學習效能(Searby & Ewer, 1997; Stanier, 1997; Gatfield, 1999; Hafner & Hafner, 2003) 雖然實務上學生所作評量的效度與可信度，仍可能是一個潛在基本的問題，這部份的判斷可調查受評者對得到同儕的評量是否感覺正確與有價值。而這樣的問題也可透過如明確的條件、雙向匿名與多個評量者來改善。 根據劉旨峰、林珊如及袁賢銘(2001)整理出目前研究中，具高信度的同儕互評的應用範圍有散文寫作(Haaga, 1993; Orpen, 1982)、多媒體創作(Rushton, Ramsey & Rada,1993)、口頭報告(Freeman, 1995; Hughes & Large, 1993a,1993b)、選擇題(Catterall, 1995)、實用報告(Hughes, 1995) 與專業技能(Korman &Stubblefield, 1971; Ramsey et al., 1996)。 低信度的則有散文寫作(Mowl & Pain, 1995)及口頭報告(Taylor,1995; Watson, 1989)等等。所以，文獻調查結果間仍是有部份意見分歧的現象 Bostock (2000) 指出管理學生同儕評量的主要困難為「在學生間傳遞產生評量訊息給受評者和評量者，以及儲存這些資料供日後稽核」，而同儕評量的缺點可透過匿名性、多位評量者及教師調節來避免 2018/9/19

Peer Evaluation in Literature 整體而言，在Topping (1998) 分析的109 篇同儕評量的研究中得知，具一定的信度與效度，且其的應用在高等教育許多的領域中已經被證實有正面的學生滿意度與學習效能(Searby & Ewer, 1997; Stanier, 1997; Gatfield, 1999; Hafner & Hafner, 2003) 雖然實務上學生所作評量的效度與可信度，仍可能是一個潛在基本的問題，這部份的判斷可調查受評者對得到同儕的評量是否感覺正確與有價值。而這樣的問題也可透過如明確的條件、雙向匿名與多個評量者來改善。 根據劉旨峰、林珊如及袁賢銘(2001)整理出目前研究中，具高信度的同儕互評的應用範圍有散文寫作(Haaga, 1993; Orpen, 1982)、多媒體創作(Rushton, Ramsey & Rada,1993)、口頭報告(Freeman, 1995; Hughes & Large, 1993a,1993b)、選擇題(Catterall, 1995)、實用報告(Hughes, 1995) 與專業技能(Korman &Stubblefield, 1971; Ramsey et al., 1996)。 低信度的則有散文寫作(Mowl & Pain, 1995)及口頭報告(Taylor,1995; Watson, 1989)等等。所以，文獻調查結果間仍是有部份意見分歧的現象 Bostock (2000) 指出管理學生同儕評量的主要困難為「在學生間傳遞產生評量訊息給受評者和評量者，以及儲存這些資料供日後稽核」，而同儕評量的缺點可透過匿名性、多位評量者及教師調節來避免 2018/9/19 9

葉盈秀 ，Visual Interface Development for e-Learning Management System, 淡江大學教育科技學系碩士班 ，2006。

Online Learning System Comparison 2018/9/19 12

Online Learning System Comparison 2018/9/19 13

Online Learning System Comparison 2018/9/19 14

IRS (Instant Response System) Source：http://www.habook.com.tw/software/Detail_soft.asp?BookNo=ME010200&B_key=IRS即時反饋系統&CatalogID=12 Receiver IRS System Interactive white board (IWT) Teacher’s remote Learning profile database 2018/9/19 Student’s remote

IRS Strategies Source： HaBook onine DM 2007/09/21 Preview & Q&A before class No preview before class Preview but answer Q’s at beginning of the class IRS Strategies School Harvard University Taipei Medical University Taipei Municipal University of Education Class  Physics course Medical course Information Science course Application Model                                                                            Source： HaBook onine DM 2007/09/21 2018/9/19

Online Learning System vs. IRS vs. PES Level of Interaction (class activities) Online Learning System High IRS PES Low Level of Synchronization Async. Sync. 2018/9/19

PES – How it works? English Manual: http://tao.nuk.edu.tw/matsuyama/PESHelpStudent.ppt 2018/9/19

PES – Home page (http://pes.nuk.edu.tw) Class announcements 2018/9/19

PES – Teacher’s first page Current page and path Crate a new class 2018/9/19

PES –Create a new class Co-teacher setup Class status setup Advanced features setup Evaluation assignment setup 2018/9/19

PES –Advanced features setup Mostly default Depends on class design 2018/9/19

PES–Evaluation assignment setup For individual assignment Three assigning rules: N students before you N students after you Odd for Even #s and vice versa For group assignment Plus self evaluation 2018/9/19

PES–Student account registration The first thing students do 2018/9/19

PES –Student enrolling classes Enroll a class 2018/9/19

PES –Class list for enrollement Available ones 2018/9/19

PES – Teacher’s first page Student list/management 2018/9/19

PES – Student list/management Check the approved students 2018/9/19

PES – Teacher’s assignment screen Insert/Edit/Delete an assignment Version Assignment 2018/9/19

PES – Adding a new assignment Individual Group Test Roll call Assigning discussion Competition (N/A) Free discussion (N/A) Can override the default setup at class creation Click confirm will go to due date setup screen 2018/9/19

For setting upload, pasdue, evaluation, and reply dates & times PES –Due dates setup For every version For setting upload, pasdue, evaluation, and reply dates & times 2018/9/19

PES–Student assignment screen (before due date) This group’s upload link Other group’s uploaded assignment 2018/9/19

PES–Student assignment upload This group’s upload link Other group’s uploaded assignment Browse the ZIP file for upload 2018/9/19

PES–How to zip a file? This group’s upload link Other group’s uploaded assignment 2018/9/19

PES –PES–Student assignment screen (after due date) Past due and disabled 2018/9/19

PES – Student assignment screen Assignment content Assignment in a new large window 2018/9/19 Assignment description Assignment evaluation

PES – Student evaluation Teacher can evaluate this student to other’s evaluations Only shown in teacher’s screen Opinion & reply +/- score 2018/9/19

PES –Student evaluation can be revised before due date& time Edit or delete a comment 2018/9/19

PES – Teacher’s evaluation on each student’s evaluation of others Performance evaluation of this student’s peer evaluation Student’s evaluation of assigned students’ assignments 2018/9/19

PES – Student score ranking Tracing scoring Rank by scores Exporting evaluation performance .xls Exporting Scores .xls Score Ranking 2018/9/19

PES – Parameters of assignment activities 24=16 variations Individual vs. group In-class vs. after-class (vs. combo) Formative vs. summative Competition vs. not-competition 2018/9/19

Formative assignment – upload the revision based on peer feedback Editing this assignment Version listing 2018/9/19

Formative assignment – creating one more version create a new version here 2018/9/19

PES–Competition func. N/A but can be done with regular functions Screening 3 finalist from round 1 3 competitors from version 1 2018/9/19

PES can incorporate what class activities？ Ind. Free Disc. N/A Ind. Comp. Assign. Ind. Test Upload Ind. Async. Assign. Ind. Assigned discus. Ind. Class Rollcall Ind. Form. Assign. Ind. Sync. Assign. … … Grp.Sync. Assign. Grp. Form. Assign. Grp. Assigned discus. Grp. Async. Assign. Ind. Comp. Assign. Grp. Class Disc. upload 2018/9/19

PES – Assigned discussion with students replies Teacher’s posting 2018/9/19

PES - Online test with upload Excel online test 2018/9/19

PES – Class roll call Teacher side Student side 2018/9/19

PES - Exporting assignment with a HTML file linking students’ assignments 2018/9/19

PES – Other possibilities? Online survey Blog Web-based BBS IRS functionalities … 2018/9/19

Online Survey 2018/9/19

PES – What I have learned Teacher： Focusing on managing students’ assignment process instead of grading them all With PES skill, class activity design can be agile and flexible Student： Focusing on training higher-order thinking and peer mutual learning/instructions Focusing on learning process not the final scores PES： A special purposed system, not to replace online learning platform But can gradually incorporate more IRS and online learning functions 2018/9/19

PES–Suggestion You can start anytime now by obtaining a teacher account from me It is free but half finished, so use it at your own convenience (only incomplete Chinese user manual) Each assignment can be exported to be used as a simple HTML page link ---The End--- 2018/9/19