教育測驗與評量研究分組心得報告 The evaluation of different gaming modes and feedback types on game-based formative assessment in an online learning environment

組別:第三組 組員名單: 數位學習組 羅烈允 數位學習組 徐逸娟 數位學習組 吳芝羽

大綱: 線上學習可能遭遇的問題 實驗設計 遊戲導向設計 題目信效度 實驗結果 問題探討

線上學習可能遭遇的問題 學習動機：互動工具─聊天室、視訊會議、討論區。提供多媒體教材引導及時學習─影音、動畫、遊戲設計。 缺乏互動 及時學習導引 形成性評量 自我評量功能建置 許多線上學習者未使用 學習動機：互動工具─聊天室、視訊會議、討論區。提供多媒體教材引導及時學習─影音、動畫、遊戲設計。 形成性評量：線上學習需要建立自我評量、改進系統。並引入遊戲導向形成性評量。

實驗設計 SOG：single-player online game MOG：multiplayer online game Gaming mode： 2(SOG vs MOG) Feedback type： 2(IEF vs no IEF) 不同遊戲模式與回饋方式是否會影響學生能源知識 不同遊戲模式與回饋方式是否會影響學生井字遊戲能力 Learning effectiveness： Tic-tac-toe ability test Participation perceptions questionnaires energy knowledge 不同遊戲模式與回饋方式是否會影響學生遊戲愉悅與參與感 SOG：single-player online game MOG：multiplayer online game IEF：immediate elaborated feedback

遊戲導向設計 formative assessment: combining tic-tac-toe with a formative assessment, enabling users to undertake the assessment as if they were playing tic-tac-toe. in which a player competes either in single-player or multi-player mode against the computer or persons. The first player to form a row, column, or diagonal line wins the game. step 3-1：When a student answered a question incorrectly, the student would have to make a move for the other party. step 3-2：Players could only place their own pieces when they answered correctly.

題目信效度～ Energy knowledge test 40 multiple-choice items was composed. Two junior high school teachers inspected all items‘ content validity. A KR-20 reliability coefficient of 0.85 was obtained through a pre-examination study with 58 ninth-grade students. The average difficulty of 0.70 and discrimination of 0.39 were obtained.

題目信效度～ tic-tac-toe ability test The test consisted of 16 questions. A research professional who specialized in chess games reviewed the appropriateness of the test. A KR-20 reliability coefficient of .71 was obtained through a pre-examination study with 58 ninth-grade students. (a) Know how to win: when an opportunity appears to make a three-piece line, players can identify the correct position. (b) knowhow to block: when an opportunity appears for the opponent to make a three-piece line, the player can identify the correct position to block the opponent from winning. (c) know how to make a fork: when an opportunity appears for making 2 three-piece lines, the player can identify the correct Position. (d) know how to block a fork: when an opportunity appears for the opponent to make 2 three-piece lines, the player can identify the correct position for placing the piece to block the fork.

題目信效度～ perceptions questionnaire 5-point scale that ranged from strongly agree (5) to strongly disagree (1). The Cronbach a for the entire scalewas 0.91, and the Cronbach a for playing enjoyment and involvement perception of the subscale were 0.84 and 0.90, respectively. an open-ended itemwas provided at the end of questionnaire. The items of enjoyment perception were: (1) I enjoyed playing TRIS-Q; (2) I enjoyed the feeling of competition in TRIS-Q; (3) I acquired exciting game experience from TRIS-Q; (4) Playing games similar to TRIS-Q in the future will make me happy. The items of involvement perception were: (1) I like the way I can play the game while taking the test; (2) This kind of game makes mewant to answer the questions thoughtfully; (3) I can recognize the areas I failed to learn well through the game; (4) I attempted to find the answer I did not know through the game; (5) I think this game benefitsmy learning.

實驗結果~ Effects on student knowledge acquisition A two-way analysis of covariance (ANCOVA) no interaction existed between the two factors of gaming mode and feedback type (F(1, 104)=0.061, p =0.805) The main effect of the variable gaming types yielded a test result of F(1, 104) =1.186, p =0.279, The main effect of feedback type generated a result of F(1, 104) = 8.307, p =0.006, h2 =0.072, which attained significance. 1.the assumption of homogeneity of regression was tested (F(3, 101) =0.29, p =0.835) and was not violated. 2. The results of the two-way ANCOVA showed that no interaction existed between the two factors of gaming mode and feedback type (F(1, 104) =0.061, p =0.805). Thus, Hypothesis 1b was not confirmed. 3. The main effect of the variable gaming types yielded a test result of F(1, 104) =1.186, p =0.279, which did not attain significance, suggesting that after eliminating the effects of the covariate, the gaming mode (SOG and MOG) used in the game-based formative assessment did not significantly affect the effectiveness of energy knowledge acquisition. 4. after eliminating the effects of the covariate, the feedback type (IEF and no IEF) of the game-based formative assessment significantly affected the effectiveness of energy knowledge acquisition.

實驗結果~ Effects on student tic-tac-toe ability The results of the two-way ANCOVA showed that no interaction existed between the two factors of gaming mode and feedback type (F(1, 104) =0.715, p =0.400). The main effect of the variable game types yielded a test result of F(1, 104) =1.270, p =0.262, which did not attain significance the main effect of feedback type (F(1, 104) =6.455, p =0.013, h2 =.058) attained significance

實驗結果~ Effects on student participation perception no interaction existed between the two factors of gaming mode and feedback type (F(1, 105) =0.005, p =0.946). The main effect of gaming mode (F(1, 105) =0.126,p =0.723) did not attain significance. The post-hoc test results showed that after adjusting the average, the average scores of SOG and MOG groups on the enjoyment scales were 3.59 and 3.53 respectively.

小組討論心得 考慮immediate elaborated feedback (IEF)可 否應用於眼動研究，探討使用眼動儀作多媒體 互動教學設計，對學習動機與概念形成的影響。 項目分析與信效度檢驗題信效度分析可否實驗 設計是為了興趣而閱讀，學生不會選擇無法理 解的書，因此不會需要教導閱讀技巧。 提升學生閱讀能力最好的方法就是提升他們的 閱讀興趣。 給孩子一些好書跟時間，讓他們沉醉其中。

簡報完畢 敬請指教