人機介面 視線軌跡追蹤 : 原理與應用 授課教師 開課單位. Outline 眼動儀類型 應用領域與最新技術 –Application areas –State-of-the-art technology Eye 1.0 (wearable) Eye tracker.

Presentation on theme: "人機介面 視線軌跡追蹤 : 原理與應用 授課教師 開課單位. Outline 眼動儀類型 應用領域與最新技術 –Application areas –State-of-the-art technology Eye 1.0 (wearable) Eye tracker."— Presentation transcript:

1 人機介面 視線軌跡追蹤 : 原理與應用 授課教師 開課單位

2 Outline 眼動儀類型 應用領域與最新技術 –Application areas –State-of-the-art technology Eye tracker @NCCUCS 1.0 (wearable) Eye tracker @NCCUCS 2.0 (remote) Gaze-based HCI Demo ( 展示 ) Conclusions

3 眼動儀類型 侵入式 – 搜尋線圈法（ Search Coil ） – 眼電圖法（ Electro-Oculography ） 非侵入式 – 紅外線視訊（ Infra-red Video System ） – 光學式瞳位追蹤 (Optical-type Eye Tracking Device) – 紅外線眼動圖法（ Infra-red Oculography ） –Purkinje 影像追蹤法（ Dual-Purkinje-image ）

4 搜尋線圈法 利用電磁感應的原理來測量眼球的移動 優點：  有很好的空間和時間解析度 缺點：  需戴上侵入式鏡片  受到眼球分泌物的影響  固定磁場 搜尋線圈法四週固定磁場示意圖

5 眼電圖法 利用電極的原理來測量眼球的移動 缺點：  必須貼電極貼在皮膚上  皮膚角質分泌影響皮膚電阻 優點：  價格便宜

6 紅外線視訊系統 影像追蹤的方法記錄眼球的移動 缺點：  價格昂貴  空間位置校正 優點：  可計算瞳孔移動位置和大小

7 光學式瞳位追蹤系統 由攝影機所獲得的瞳孔影像，利用影像處理的方式， 分析眼球的移動軌跡 優點：  使用者頭部與攝影機之間的 相對位置不會改變 第四代追瞳器裝置 缺點：  頭盔因本身重量往下滑  光源讓人不舒服  配戴眼鏡無法使用

8 紅外線眼動圖法 利用紅外線光源在眼角膜邊緣的反射差異，來檢測眼 球運動狀況 缺點：  紅外線光對眼睛可能 會造成傷害 優點：  測量範圍廣

9 Purkinje 影像追蹤法 光源經過眼球組織不同的折射率，來計算眼球運動 缺點：  不能和眼球同步  價格極為昂貴 優點：  不易受眼瞼遮蔽準確度高

10 應用領域 醫療 : 漸凍人等身體障礙者輸入介面 EyeMusic- 創作音樂 EyeDraw- 繪畫 Usability: 網頁和廣告的使用性和實用性 電腦 - 替代滑鼠，文字輸入 遊戲 - eye tracking game 心理學 - 眼球測謊儀 閱讀 - text 2.0 遠端遙控 - 遠端電視操控， EyeTech

11 基於影像之眼動追蹤 An eye tracker is a device for measuring eye positions and eye movements.eye movements The most popular variant uses video images from which the eye position is extracted. Input source: visible spectrum vs. infrared

12 眼動 Eye Movements Eye movements Eye movements are typically divided into fixations ( 凝視 when the eye gaze pauses in a certain position) and saccades ( 掃視 when it moves to another position). The resulting series of fixations and saccades is called a scanpath. ( 掃描路徑 )scanpath

13 掃描路徑實例

14 阿春眼動看護

15 EyeWriter Project

16 Web Design (U. of Manchester) Heat mapGaze plot

17 Usability Study: Google Search Evaluation

18 Gaming

19 Driving Behavior [Andrew T. Duchowski]

20 Character Input EyeWrite [Andrew T. Duchowski]

21 Media Research The Poynter Institute : http://www.poynter.org http://www.poynter.org Published first eye track study in 1991. More results published in 2000, 2004 and 2007. Eyetrack ’07: http://www.poynter.org/content/content_vie w.asp?id=105035 http://www.poynter.org/content/content_vie w.asp?id=105035

22 Text 2.0

23 Research Communities ACM SIG/CHI Eye Tracking Research and Applications (ETRA) Symposium: http://www.e-t-r- a.org/http://www.e-t-r- a.org/ COGAIN: Communication by Gaze Interaction http://www.cogain.orghttp://www.cogain.org References: http://www.cogain.org/downloads/ http://www.cogain.org/downloads/

24 State-of-the-art Eye tracking Technology Hi-Speed (SMI iView X™ Hi-Speed) Head-Mounted System Remote tracking Integrated with LCD monitor Wearable glasses

25 Mobile Eye Tracker 1.0 @ NCCUCS eye camera scene camera

26 Key Objectives Low-cost (NTD 10,000 vs. 1,000,000) Mobility Easily customized for specific applications On-line processing Sampling rate? Accuracy?

27 System Architecture Eye image Preprocessing Pupil detection Gaze point projection Scene image Calibration 9 pairs of points Calibration process

28 Pupil Detection (I): extracting feature points

29 Pupil Detection (II): Ellipse fitting using RANSAC

30 Pupil Detection (III): checking fitness measure Ellipse model: Number of bright vs. dark points inside and out the perimeter

31 Results

32 HCI Application: Eye Scrolling

33 HCI Application: Eye Gaming

34 HCI Application: Eye Typing

35 Mobile Eye Tracker 2.0 @ NCCUCS Improve the pupil detection algorithm to alleviate corneal reflection problem. Enhance the accuracy by compensating for head movement. Construct and test a remote eye tracker. More HCI applications using the remote eye tracker. Use the eye tracking device to assist mobile user interface design.

36 Improved Pupil Detection

37 Allowing Head Movement

38 Remote Eye Tracker

39 Experimental Results (Wearable) Calibration point Original error Compensate for head movement (error/standard deviation) 198.1732.91 (12.7) 2137.5455.89 (10.94) 3108.2029.69 (9.57) 476.3028.79 (21.19) 5113.3022.49 (7.79) 6117.8634.83 (10.96) 7116.5333.31 (8.01) 8112.5616.03 (6.78) 9146.7326.49 (9.66) 1 cm = 38 Pixels

40 Accuracy (Wearable)

41 Experimental Results (Remote) Calibration pointOriginal error Compensate for head movement (error/standard deviation) 1238.7339.69 (24.6) 2227.5163.56 (22.82) 3132.9744.30 (21.49) 4222.9751.66 (30.94) 5306.5831.15 (16.71) 6280.7951.66 (21.83) 7311.8969.03 (41.57) 8344.2469.88 (31.71) 9347.0364.66 (24.41)

42 Accuracy (Remote Eye Tracker)

43 Demo: Web Browsing

44 Demo: Photo Viewing

45 Demo: Interactive Story Telling

46 Demo: Tic-Tac-Toe

47 Gaze Path Analysis Recursive intersection –Find the similarity between two scan paths –Order of scanning is irrelevant –Suitable for processing fixation data Modified dynamic time warping –Order of scanning is taken into account –Can handle both fixation and saccade data

48 Recursive Intersection Path 2 Path 1

49 Recursive Intersection: Example

50 Dynamic Time Warping (DTW)

51 Modified DTW (MDTW) X-axis Y-axis time

52 MDTW Result

53 Summary Eye tracking device –Low cost (<$10,000 NTD) –Head-mounted with mobile functionality –Remote eye tracking –Allow slight head movements –Accuracy: suitable for some HCI applications, image viewing tasks, not high enough for reading HCI applications