26.Evaluation of ICT Competencies 信息通信技术的能力评估

Presentation on theme: "26.Evaluation of ICT Competencies 信息通信技术的能力评估"— Presentation transcript:

1 26.Evaluation of ICT Competencies 信息通信技术的能力评估
汇报人：田嵩 Agustin Tristán-López and Miguel Angel Ylizaliturri-Salcedo

2 4/6/2019 Abstract Assessment of ICT (Information and Communication Technology) competencies involves the definition of specific target activities appropriate to the environment in which they are required. The range of ICT competencies also includes the specific abilities needed by professionals responsible for the development of software or communication products and services. This chapter discusses several ICT assessment projects, technical specifications and solutions. These approaches primarily focus on a basic core of skills. In the future, ICT evaluation will involve more intelligent fields. 评估ICT（信息与通讯技术）能力首先要定义与环境相适宜的、特定的目标活动。具体环境包括从家到工作地点的不同活动。ICT能力还包括专业化的软件开发、通信产品或服务等的能力。 文章讨论了几个ICT评估项目，包括其中用于评估的主要技术指标，和测试管理中问题的解决方案。这些测试从类似笔纸的电子测试到使用仿真软件等更加真实的形式。对社会、学院、商业和经济环境的详细数据的评估表明，这些方法主要聚焦于基本的核心技能，包括：浏览网页、发电子邮件、使用Office软件等。将来ICT评估将更多地涉及文档中自然语言响应的自动评分、数学问题的解决、图形或健康有益及其他真实生活情景。

3 1. To provide a panoramic view of the status of ICT assessment
4/6/2019 About ICT ICT curricula are aimed at satisfying the academic and professional motivations of individuals on the development of higher order ICT competencies. In addition, ICT also has the potential to empower minorities and produce benefits resulting from the technology itself. The goals of this chapter are: 1. To provide a panoramic view of the status of ICT assessment 2. To present some of the online, e-portfolios and other Internet applications 3. To show the basic concepts of the platforms based on computational simulators ICT课程的目标是提供高层次ICT能力发展的个体学术化或专业化的动力。另外ICT对少数人群也可以提供潜在的提升，并从技术本身产生收益。ICT的这种潜在的提升能力，对于弥补人与人、国与国间的数字鸿沟非常有用。 本章的目标： 1、通过几个国家的主框架和发展大纲提供一个ICT评估状态的全景视角； 2、当前的一些问卷调查和测验类的在线平台，或电子档案及其他互联网应用； 3、介绍一些基于计算机模拟平台的基本概念。

4 Description of ICT Competencies
4/6/2019 Description of ICT Competencies The first step in an evaluation project should be to describe the set of competencies to be demonstrated by an ICT user. Defining competencies is an important issue.(Table 26.1) Concept Examples of skills or competencies ICT literacy Access, manage, integrate, evaluate, create, transmit, or communicate information. Digital literacy Search, locate, organize, analyze, evaluate, and synthesize information to communicate, manage, produce, and perform other tasks with the information. Information literacy Search, locate, organize, evaluate, and use information, mainly as an information-retrieval activity. Computer literacy Install, apply, and modify commercial or open-source operating systems and programs. Technology literacy or technological literacy Understand, select, apply, use, manage, and evaluate all kinds of technology or technological products and devices. ICT advanced and professional Install, upgrade, and configure hardware and software. ICT评估项目的第一步是描述ICT用户的可证明能力集合。定义有哪些能力是很重要的事情，因为每个作者、研究人员、协会或专业团体都有特定情境下的ICT能力。 表26.1列举了一些特定领域中的重要的ICT能力。

5 Description of ICT Competencies
4/6/2019 Description of ICT Competencies Information technology literacy, originally defined as the set of competencies needed to use digital or electronic devices to manage data and information, has evolved into ICT to explicitly include the use of computers and telecommunications to handle information. ICT competencies are multiple and contextual to the social, educational, professional, or economic environment. These competencies integrate a wide range of tasks from basic manipulation of elementary software to advanced programming for problem solving. 信息技术素养原本被定义为使用数字或电子设备来管理数据或信息的能力集合，如今演变为ICT能力，其中包括计算机的使用和处理信息通讯手段。ICT能力对社会、教育、专业领域或经济环境来说，具有多样性和情境性。这些能力的整合涉及很大地范围，从基本的软件使用到通过高级编程去解决问题。

6 Importance of ICT Assessment
4/6/2019 Importance of ICT Assessment ICT assessment can identify and measure several characteristics of the knowledge and skills of ICT users, as well as their attitudes and values concerning the use of this tool to solve daily problems. Evaluating ICT users’ competency is essential to produce relevant, useful information.Educational institutions may create courses to better prepare students for their society’s current equirements for daily life. In addition, ICT assessment allows measurement of the digital divide resulting from differing digital literacy levels among individuals and countries. ICT评估能够从ICT使用者的知识和技能的多个特征进行定义和测量，例如他们使用工具去解决日常生活中的问题时的态度和价值。评估ICT用户能力对获得相关的、有用的信息是必要的。 教育机构可以开设课程来帮助学生更好地针对社会需求。 另外，ICT评估允许评测由不同个人或国家间信息化能力的差异而带来的数字鸿沟。 26.2表格中部分网站无法打开，部分网站提供的ICT统计数据较陈旧。 International Telecommunication Union部分xls统计数据来源于

7 Frameworks and Syllabus for ICT Evaluation
4/6/2019 Frameworks and Syllabus for ICT Evaluation A framework should be based on a multidimensional model that includes knowledge, skills (cognitive or psychomotor), as well as values and attitudes toward technology; taxonomies can be theoretical, empirical, or mixed, but they can also be developed by vendors of tests and educational materials. Within a framework, the definition of a competency consists of a sentence that describes what an individual knows or is able to do in a particular domain or content and in a specific context or situation. ICT框架应该是一个多维模型，其中包括知识、技能（认识或心理活动）、价值观和对技术的态度。可以从理论、经验、混合等形式分类，同时也应该从考试供应商或教育材料中发展而来。在一个框架中，这种能力的定义由个人可以理解的或在特定情境下的领域和内容中实现的描述性句子组成。

8 For example—— the Scottish Electronic Staff Development Library
4/6/2019 For example—— the Scottish Electronic Staff Development Library communication learning use of the Internet standards educational hardware educational software 例子：苏格兰电子职员发展库 提出的六个类别的信息素养。 沟通 学习 使用互联网 标准化 教育硬件 教育软件

9 The Standards for Technological Literacy
4/6/2019 The Standards for Technological Literacy These Standards provide 20 specifications in five major categories: 1. Fundamentals of technology and connection with other areas 2. Role of technology in society, culture, and the environment 3. Design of applications and solutions to engineering problems 4. Ability to use technological products and systems in everyday life 5. Applications in seven areas where engineering and health represent a primary focus (e.g., transport, construction, health, agriculture, and ICT) 技术能力标准，提出了五大类20项标准。 1、技术的基础和与其他领域的衔接 2、社会、文化、环境中技术的角色 3、应用程序设计和工程问题的解决 4、使用技术在日常生活中生产和系统化的能力 5、当前重点关注的工程和健康的七个领域中的应用（交通、建筑、健康、农业、ICT）

10 Frameworks and Syllabus for ICT Evaluation
4/6/2019 Frameworks and Syllabus for ICT Evaluation Since 1996, more than 60 countries have recognized the certification program known as the International Computer Driving Licence (ICDL or ECDL for its European version). This program defines a core of seven modules that include the basics of ICT, use of the computer and operating system, and a subset of standard office tools (word processor or text editor, slide presentation, spreadsheet, database management, information and communication). 1996年以来，超过60个国家认可国际计算机操作执照（ICDL）作为认证程序。这个认证包括核心的七个模块，包括： ICT基础、计算机和操作系统的使用，标准化Office工具子集（文字处理、幻灯片、电子表格、数据库管理、信息与通信）

11 Frameworks and Syllabus for ICT Evaluation
4/6/2019 Frameworks and Syllabus for ICT Evaluation We propose the following general syllabus for assessment of core ICT competencies: ICT environment Human aspects of ICT Folders, files, and documents Text processing Spreadsheet Computer-assisted presentations Database management Electronic mail Image handling Information technologies 我们结合了前面提到的框架提出了下面这个ICT核心能力评测的通用大纲： 1、ICT环境：Principal components of a computer, use of hardware and peripherals，operating systems，networking，graphical user interface. Career opportunities。 2、ICT的人性化方面：Security and privacy，health and ergonomics，environmental protection。 3、文件夹、文件和文档：Organizing folders and files，Definition of files and the most common file types，Searching for documents and retrieving content，Management of files。 4、文字处理：Templates and document creation，Page configuration and styles，Use of headers, footers, and pagination，Paragraph formatting，Tables，Management of objects，Spell-check and language tools，Document publishing。 5、电子表格：formatting cells. Functions, formulas, and cell references. Spreadsheet formatting. Creation of charts and graphs。 6、计算机辅助演示：Templates and creation of slide presentations，Design of slides and display options. Publishing presentations。 7、数据库管理：Using data fields and clusters. Functions and formulas. Searching, sorting, and retrieving data units. Publishing a database。 8、电子邮件： 9、图像处理：Use of photo editing software 10、信息技术：Internet and access to information，Virtual sites，Web and related tools，Security and the dangers of virtual communities

12 Requirements for the Assessment Environments
4/6/2019 Requirements for the Assessment Environments ICT assessment validity objectivity reliability ICT评估需要满足：可用性、客观性、可靠性，同时要考虑特殊用户和复杂软件的需求。

13 Requirements for the Assessment Environments
4/6/2019 Requirements for the Assessment Environments Virtual learning environments (VLE) or learning management systems (LMS) are software platforms consisting of e-learning tools for authoring, data management, planning and structuring educational curricula, homework tracking, forum/blog/chat communication, Web publication, and course management and assessment. Assessment is a minimal component of these platforms, for two reasons. First, they do not provide relevant psychometric models to score tests. Second, these platforms do not focus on evaluating specific ICT competencies. 虚拟学习环境或学习管理系统是组成e-learning的软件平台，可用于权限管理、数据管理、计划和组织课程、作业追踪、论坛交流、网络发布、课程管理和评测。 评估只占这些平台中很小的一部分，有两个原因：一是他们不提供从心理评测到评分的相关模型，二是这些平台并不专注于评估特定的ICT能力。

14 Requirements for the Assessment Environments
4/6/2019 Requirements for the Assessment Environments Computer-based tests should not be merely computerized versions of paper-and-pencil tests. A good visual interface must allow back-and-forth browsing of items and provide information on test status. Output feedback is important to provide recommendations for individual test takers; however, it seems preferable not to report details about correct and incorrect answers. ICT assessment software must work in multiple environments with a variety of hardware and software. 基于计算机的测试不应该仅仅是纸笔测试的计算机化版本。一个优秀的可视化界面应该允许回溯往复浏览测试项目，并可以提供测试状态的信息。 对个人测试者提供推荐性意见的输出反馈是非常重要的。无论如何，最好不要将正确或错误答案的详情报告出来（特别是当学生可以多次测试的情况）。ICT测试软件一定要可以在多种硬件或软件环境下运行。

15 4/6/2019 Online Testing First-generation online testing platforms merely adapt paper-and-pencil tests, quizzes, and questionnaires to a computer environment. This kind of testing software has two operating modules: management module and testing module. 第一代在线测试平台仅仅是将纸笔测试、测验或调查问卷迁移到计算机环境中。这种测试软件分为两个操作模块：管理端和测试端。

16 4/6/2019 Online Testing Several studies show no significant differences in scores on a single test administered under different management schemes, from paper-and-pencil to online testing. However, security represents a major problem, particularly the need to authenticate student identity. computer-aided testing produces some benefits, for instance: management simplification; reduction of time and effort for teachers and students, elimination of expensive optical forms and optical mark readers; low frequency of errors during input; feasability to use free- response essays. 多个研究展示：从纸笔测试到在线测试，在不同管理方案下的单一测试管理中分数没有显著差异。无论如何，安全性是一个主要的问题，特别是学生身份的认定，或其他作弊情况等，这需要大量的软硬件资金投入。 计算机辅助测试也会带来一些好处，如：简化管理；减少教师和学生的时间和精力；减少昂贵的光学识别设备的投入；降低输入阶段的错误率；使用更加自由的响应文字。 文中提到了类似ETS这样的考试平台（教育自动评分引擎），还有类似的临床病例多媒体仿真、西班牙语文章自适应自动评分等系统。

17 ICT-Simulation Assessment Software
4/6/2019 ICT-Simulation Assessment Software Simulation software requires design tools both for items and tests; options for customizing the appeareance of the environment to be used by the student or candidate to solve the items and perform the requested activities; network management tools to control a local network or an Internet platform; scoring modules with statistical and measurement models to produce feedback reports. ICT仿真考试软件 仿真软件需要针对项目和测试的设计工具。针对学生及解决问题或执行活动的候选人的定制化环境界面；控制本地网络或互联网平台的网络控制工具；带有统计和量化模型的评分模型用来提供反馈报告。

18 ICT-Simulation Assessment Software
4/6/2019 ICT-Simulation Assessment Software ICT-Simulation Assessment Software (ICT-SAS) is not a local solution in a single country， software for ICT-SAS is organized in three modules: In ICT-SAS candidates do not answer questions. Instead, candidates perform a series of tasks simulating a real-life situation. Test design Test administration Test workstations ICT仿真考试软件不是针对单一国家的本地化解决方案，ICT-SAS由三个模块组成： 考试设计 考试管理 考试客户端 在ICT-SAS中，考生不是回答问题，而是完成一系列模拟真实生活情景的任务。每个任务由多个可被归类为动作或展示的活动组成。

19 ICT-Simulation Assessment Software
4/6/2019 ICT-Simulation Assessment Software ICT-SAS syllabus core: Desktop Operating system Notepad Text processor Worksheet Slide presentation Internet Database ICT-SAS遵循基本的核心大纲，包括： 桌面、操作系统、记事本、文字处理、电子表格、幻灯片展示、电子邮件、互联网、数据库

20 ICT-Simulation Assessment Software
4/6/2019 ICT-Simulation Assessment Software ICT-SAS中每条命令都可以获得一个ID编码的功能代码行，考生的每一步操作也会自动编码并与命令序列进行对比。 考试设计人员开发的正确命令序列集与考生操作序列日志集对比和数据挖掘分析预评分结果、评分、最终报告 A command is the combination of the ID number of the application tool plus the code of the function to be performed. Every step produced by the candidate is encoded and can be compared with the sequence of commands that should be produced by the candidate.

21 4/6/2019 Conclusions ICT literacy is a means to empower people in situations where minorities suffer from unfavorable conditions, for example, in countries where women’s rights are constrained by law or tradition. ICT assessment can be used for academic and professional certification purposes, or even as a way to increase self-esteem at home, regardless of people’s age or social condition at any level of education. Online testing is a feasible method to assess ICT competencies. Include submit questionnaires and quizzes ， the use of e-portfolios and other Internet tools, and up to simulated environments. ICT能力是可以帮助哪些处于不利环境的人们更加有力，例如那些受到法律或传统观念限制的国家中的妇女。 ICT评测能够用于学院或专业评测的目的，甚至可以在家里提高自尊心的一种途径，无论任何教育水平下人们的年龄或社会条件。 在线测试是一种评估ICT能力的可靠的方法，包括提交问卷或测试，使用电子档案或其他互联网工具，以及仿真模拟环境。

22 4/6/2019 Conclusions ICT assessment instruments and software should incorporate psychometric models to obtain valid, objective, and reliable measures of the competency level of students or professionals in specific contexts. Existing developments focus primarily on simulated environment software, because it allows actions performed in the simulation to closely mirror those performed with real tools. And may integrate measuring models in a powerful way, including statistical and psychometric models for item and test calibration. ICT测试工具和软件应该与心理学模型关联以获取学生或特定情景下专业人员的能力水平的有效性、客观性和可靠性。 当前的发展主要聚焦于软件仿真环境，因为这种环境能够实现接近于使用真实工具的模拟活动的评测。并且可以集成非常有用的评测模型，包括统计分析和针对项目或测试矫正的心理评测模型。

23 疑问 ICT-SAS平台的测试分类是否可以有效的评测ICT素养能力？ 是否有针对移动环境的ICT能力评测？

24 谢谢观看！