Stealing Machine Learning Models via Prediction APIs 此页可以删除 徐誉畅 016033910001

Model Extraction Attacks 目录 Contents 1 Introduction 2 Background 3 Model Extraction Attacks 4 Extraction with Confidence Values 5 Online Model Extraction Attacks

Extraction Given Class Labels Only 目录 Contents 6 Extraction Given Class Labels Only 7 Extraction Countermeasures

Introduction Machine learning (ML) aims to provide automated extraction of insights from data by means of a predictive model. A predictive model is a function that maps feature vectors to a categorical or real-valued output. Popular models include support vector machines (SVMs), logistic regressions, neural networks, and decision trees. 机器学习（ML）旨在通过预测模型从数据中自动提取信息。 预测模型是将特征向量映射到分类或实值输出的函数。 流行模型包括支持向量机（SVM），逻辑回归，神经网络和决策树。

Introduction AIM of model extraction attacks Free of charge leak information about training data. evade detection (such as spam detection) Model extraction attacks: Adversary can query an ML model through API to obtain predictions on input feature vectors. The adversary’s goal is to extract an equivalent or near- equivalent ML model. 模型提取攻击的目的 免费获取模型服务 关于训练数据的信息泄露。 逃避检测（如垃圾邮件检测） 模型提取攻击： 攻击者可以通过API查询ML模型，以获得关于输入特征向量的预测。 攻击者的目标是提取等效或近似等效的ML模型。

Introduction Information-rich outputs by APIs: class labels high-precision confidence values response to partial queries Two approaches: For logistic regression, multiclass logistic regressions and neural networks: querying d +1 random d-dimensional inputs For decision trees: use adaptive, iterative search algorithms to discover paths in a tree. 信息丰富的API输出： 类标签 高精度置信度值 响应部分查询 两种模型提取攻击方法： 对于逻辑回归，多元逻辑回归和神经网络：对d维输入数据模型进行d + 1次随机query,联立方程组求解 对于决策树：使用自适应迭代搜索算法来发现树中的路径。

Introduction Model extraction attacks under omission of confidence values New attacks extract models matching targets on >99% of the input space for a variety of model classes, but need up to 100 times more queries than equation-solving attacks While less effective than equation-solving, these attacks remain attractive for certain types of adversary. 针对以上的攻击，常采用的措施是API只提供类别标签作为输出 这里对缺少置信值的模型提取攻击进行讨论： 发现新的攻击提取模型可以匹配目标模型精度99％以上，但需要比之前的模型多100倍query 但这些攻击对某些攻击者仍然具有吸引力。

Introduction Experiment Locally experiment: using standard ML libraries Case study: BigML and Amazon. Show computationally fast attacks that use a small number of queries to extract models matching the targets on 100% of tested inputs. 实验分为两种情况 本地实验：使用标准ML库 案例研究：使用BigML和Amazon提供的API。 实现快速而高效的攻击，实现仅使用少量查询来提取与目标模型100％匹配的模型。

Introduction Main contribution: Simple equation-solving model extraction attacks that use non-adaptive, random queries to solve for the parameters of a target model. A new path-finding algorithm for extracting decision trees that abuses confidence values as quasi-identifiers for paths. Model extraction attacks against models that output only class labels, 主要贡献： 对回归模型，使用非自适应随机查询求解目标模型参数的简单方程求解模型。 对树模型，采用一种新的路径搜索算法，从而得到将置信度值作为路径标识符的决策树。 探究仅输出类标签的模型进行模型提取攻击的算法

Background ML model： Input: a d-dimensional vector in the feature space Outputs: in the range Y 机器学习模型：从输入到输出空间的映射 输入：d维特征向量

Background Features: categorical features: assume one of a finite set of values (whose set size is the arity of the feature), continuous features：assume a value in a bounded subset of the real numbers Distance measure : 0-1 distance, 两种特征： 分类特征：有限集合中的一个值 连续特征：实数的有界区间中的某个值 距离度量：本文采用0-1距离，

Model Extraction Attacks Features: an adversary obtains black-box access to some target model f attempts to learn a model 𝑓 that closely approximates or even matches f 模型提取攻击： 对手获得对某个目标模型f的黑盒访问 并试图学习一个逼近甚至完全匹配f的模型f

Model Extraction Attacks Machine learning services: cloud-based ML services provide Simple-to-use Web APIs. Details vary reatly across services： white-box & black-box synchronous requests & asynchronous ‘batch’ requests 机器学习服务： 基于云的ML服务提供简单易用的Web API。 服务之间的细节差异很大。文章中涉及的机器学习服务的特征如表中所示 如果用户可以下载适合本地使用的模型，则模型为白盒。 如果只能通过API访问，则为黑盒。 研究的服务同时接受多个预测的同步请求和异步“批处理”请求。

Model Extraction Attacks Apart from PredictionIO, all of the services we examined respond to prediction queries with not only class labels, but a variety f additional information 除了PredictionIO，涉及的所有服务不仅对类别标签进行预测查询，而且对预测输出提供了各种附加信息，包括置信度分数（对于逻辑回归就是类概率）。

Model Extraction Attacks Attack scenarios. Avoiding query charges. A model extraction attack will undermine the provider’s business model if a malicious user pays less for training and extracting than for paying per-query charges. Violating training-data privacy. Model extraction could leak information about sensitive training data. Stepping stone to evasion. An adversary may use knowledge of the ML model to avoid detection by it. Such as identification of spam, malware classification, and network anomaly detection 攻击场景： 避免查询费用。 如果恶意用户的训练和提取费用低于每个查询的费用，则模型提取攻击会破坏提供商的业务模式。 违反培训数据隐私。 模型提取可能泄露关于敏感的训练数据的信息。 逃避检测。 对手可以使用ML模型的知识来避免它的检测。 如识别垃圾邮件，恶意软件分类和网络异常检测

Model Extraction Attacks Two distinct adversarial models: Direct queries： providing an arbitrary input x to a model f and obtaining the output f (x). Indirect queries： queries on points in input space M yielding outputs API提供的查询模式包括： 直接查询： 向模型f提供任意输入x并获得输出f（x）。 间接查询： 对输入空间M中的样本首先进行映射到M空间再产生输出

Model Extraction Attacks Error measures: Test error Rtest ： the average error over a test set D 1 Uniform error Runif： a set U of vectors uniformly chosen in X estimates the fraction of the full feature space on which f and 𝑓 disagree. 错误度量： 测试错误Rtest：测试集D的平均误差 均匀误差Runif：在X中均匀选择的样本集合U； 估计在完整特征空间中提取的模型和真实模型不相同的程度

Extraction with Confidence Values Equation-Solving Attacks Application： all logistic models (LR and neural networks) API reveals class probabilities providing an adversary with samples (x, f (x)) that can be viewed as equations in the unknown model parameters. These equation systems can be efficiently solved, thus recovering f。 . 方程式解决攻击： 应用：所有logictic模型（LR和神经网络） 利用样本特征和API提供的类别概率（置信分数）及标签，从而可以建立关于模型中未知参数的等式； 通过多次query，可以联立求出未知参数从而得到模型。

Extraction with Confidence Values Equation-Solving Attacks--Binary logistic regression A LR model performs binary classification (c = 2) LR model is defined by 𝑤𝜖 𝑅 𝑑 ，𝛽𝜖𝑅 and outputs a probability LR is a linear classifier: it defines a hyperplane in the feature space 方程式解决攻击--二值逻辑回归 类别个数为二 参数。。。输出类别概率 是一种线性分类模型 定义特征空间的线性分界面

Extraction with Confidence Values Equation-Solving Attacks-- Binary logistic regression Given an oracle sample (x, f (x)), we get a linear equation. d+1 samples are both necessary and sufficient (if the queried x are linearly independent) to recover w and b 给定一个样本（x，f（x）），得到一个对应的关于未知参数的线性方程。 证明至少进行d + 1次query（如果查询的x是线性独立的）就可以以恢复所有的参数w和b

Extraction with Confidence Values Equation-Solving Attacks-- Binary logistic regression Train a LR model and extract it given d +1 predictions. In all cases, we achieve Rtest = Runif = 0. the attack requires only 41queries on average, and 113 at most. 训练LR模型，并利用d+1个样本进行模型提取 在所有情况下，最终可以实现实现Rtest = Runif = 0。也就是对模型的完全提取 攻击平均只需要41次查询，最多只有113次。

Extraction with Confidence Values Equation-Solving Attacks-- MLRs and Multilayer Perceptrons multiclass logistic regression (MLR) : combines c（c>2） binary models to form a multiclass model Two types of MLR: softmax model one-vs-rest (OvR) 等式解决攻击 - MLR和多层感知器 多元逻辑回归（MLR）：结合c个（c> 2）二进制模型形成多类模型 两种类型的MLR： softmax模型 OvR

Extraction with Confidence Values Equation-Solving Attacks-- MLRs and Multilayer Perceptrons Softmax model: fits a joint multinomial distribution to all training samples The equation system is non-linear By minimizing an appropriate loss function with a regularization term, the loss function is strongly convex, and the optimization thus converges to a global minimum. Softmax模型： 使用所有训练样本的联合多项分布 得到的方程式是非线性的 通过使用正则化项最小化损失函数对模型求解，损失函数是强凸的，所以可以收敛到全局最小值。

Extraction with Confidence Values Equation-Solving Attacks-- MLRs and Multilayer Perceptrons OvR model： trains a separate binary LR for each class then normalizes the class probabilities. OvR model： 为每个类训练单独的二值逻辑分类器 然后normalize类别概率。

Extraction with Confidence Values Equation-Solving Attacks-- MLRs and Multilayer Perceptrons Multilayer perceptrons (MLP): a non-linear transform to all inputs (the hidden layer), a softmax regression in the transformed space The equation system is non-linear MLPs and MLRs mainly differ in the number of unknowns in the system to solve. the loss function for MLPs is not strongly convex. 多层感知器（MLP）： 首先对所有输入（在隐层中）进行非线性变换， 然后在输出层进行softmax回归 方程式是非线性的 MLP和MLR主要区别在系统中未知数的数量上有所不同。由于有隐含层，MLP的参数更多 MLP的损失函数不是强凸函数，所以不一定取得全局最优。

Extraction with Confidence Values Equation-Solving Attacks-- MLRs and Multilayer Perceptrons For the non-linear equation systems we obtain, we do not know a priori how many samples we need to find a solution. explore various query budgets of the form a*k, where k is the number of unknown model parameters, and a is a budget scaling factor. 对于我们获得的非线性方程系统，所以不能确定需要进行多少次query对方程组进行求解 定义查询预算次数为a * k，其中k是未知模型参数的数量，a是预算缩放因子

Extraction with Confidence Values Equation-Solving Attacks-- MLRs and Multilayer Perceptrons Experiment： softmax regression, a OvR regression and a MLP target ‘Race’ (c = 5). vary a from 0:5 to at most 5. 实验： 分别对三种模型进行实验 样本输出类别个数为5 预算缩放因子最小是0.5 最大是5

Extraction with Confidence Values Equation-Solving Attacks-- MLRs and Multilayer Perceptrons Experiment： For MLR models the attack is extremely efficient, requiring around one query per unknown parameter of f. For MLPs, the system to solve is more complex, with about 4 times more unknown parameters 对于MLR模型（多元逻辑回归），攻击非常有效，每个f的未知参数大约需要一个查询。 对于MLP多层感知器，由于解决的系统更复杂，query次数约为未知参数的4倍

Extraction with Confidence Values Equation-Solving Attacks-Training Data Leakage for Kernel LR When a model’s outputs are directly computed as a function of that data, extraction attacks can leak private training data. By querying the model, adversary obtains a non-linear equation system, the solution of which leaks training data. 方程解决攻击 – 利用kernel逻辑回归模型进行训练数据泄漏研究 当模型的输出直接作为该输入特征的映射时，提取模型的攻击可能获得训练数据的信息。 通过对目标模型进行query，攻击者获得非线性方程组，从而提取模型，利用提取的模型获得训练数据信息。

Extraction with Confidence Values Equation-Solving Attacks-Training Data Leakage for Kernel LR (a) Displays 5 of 20 training samples used as representers in a KLR model (top) and 5 of 20 extracted representers (bottom). (b) the average image of training points classified as a 3;4;5;6 or 7 by the target model f , along with 5 extracted representers . （a）在Kernel LR模型中选择20个训练样本中的5个作为代表（上图），20个提取的结果中的5个代表（底部）。 显然泄露了训练样本信息。 （b）由目标模型f分类为3; 4; 5，6或7的训练点的平均图像，以及f的5个提取的结果代表。 攻击泄漏了训练数据每个类别的“平均值”。

Extraction with Confidence Values Equation-Solving Attacks-Model Inversion Attacks on Extracted Models Model inversion attack: Uses access to a classifier f to find the input that maximizes the class probability for class i： The model work best in a white-box setting, where the attacker knows f and its parameters. extract the model and then uses it with the white-box inversion attack. case study ：faster and requires 20 times fewer online queries. 等式解决攻击 - 模型反转攻击提取模型 模型反演攻击：使用分类模型f来找到使类i的类概率最大化的输入： 该模型在白盒设置场景中效果最好，即攻击者知道f及其参数。 组合攻击：首先尝试提取模型，然后将其作为模型反演攻击的基础。 案例研究显示，在先前提取的模型上执行模型反演导致的攻击速度更快，并且需要少20倍的在线查询。

Extraction with Confidence Values Decision Tree Path-Finding Attacks Decision trees partition the input space into discrete regions, each of which is assigned a label and confidence score. Path-finding attack: exploits API particularities to extract the ‘decisions’ taken by a tree when classifying an input. 决策树路径查找攻击 决策树将输入空间分成离散区域，每个区域都分配一个标签和置信区间。 路径查找攻击：利用API提供的信息来提取树分类时的“决策路径。

Extraction with Confidence Values Decision Tree Path-Finding Attacks The attack starts with a random input x search for all constraints on x LINE SEARCH (for continuous features) CAT SPLIT (for categorical features) create new queries for unvisited leaves. Once all leaves have been found, the algorithm returns the corresponding constraints on x. 攻击从随机输入x开始，并获取叶节点ID。 然后，使用下面描述的算法LINE SEARCH（用于连续功能）和CAT SPLIT（用于分类功能），搜索该类别必须满足的所有约束条件并保留在该叶节点中。 然后为未访问的树叶创建新的查询。 一旦找到所有的叶子，算法就为每个叶子节点返回x特征空间上的相应约束。

Extraction with Confidence Values Decision Tree Path-Finding Attacks Toy example : current query is x = {Size =50; Color = R} and O(x) = id2. LINE SEARCH程序（第12行）用于搜索连续特征。 在我们的例子中，我们将Size的值设置为0到100，并获得id1和id5。 由于id不匹配，SIZE的分割发生在id2的路径上。 通过对特征尺寸（和x固定的所有其他特征）的二进制搜索，我们发现导致不同节点的所有区间，即[0; 40]，（40，60]，（60; 100） ，我们找到当前叶的所属区间（即Size =（40; 60）），并构建查询来探索新的树路径。 CATEGORY SPLIT过程（第20行）查找分类特征的拆分。 在我们的例子中，我们改变了在x中的颜色值和查询O，以获得每个值的叶子ID。 然后，我们建立一组导致当前叶的值C，即C={R,G,B}，以及为探索其他没有遍历过的结点的color集合V={B,G,Y} V={B,G,O}

Extraction with Confidence Values Decision Tree Path-Finding Attacks A top-down approach It extracts the tree ‘layer by layer’ starting at the root with an empty query then set each feature in turn and query again. With similar procedures as described previously, extract the root’s splitting criterion recursively search lower layers of the tree. 采用自上而下的方法，在根节点用空特征集查询获得id信息，然后依次添加特征，直到找到根节点的区分特征。 在之后的每一个节点处也使用相同的过程。

Extraction with Confidence Values Decision Tree Path-Finding Attacks Duplicate identities attacks are resilient to some nodes or leaves sharing the same id. checking whether a leaf with the current id was already visited whether the current query violates that leaf’s predicates 允许某些节点拥有相同的id 不仅检查当前id的叶子结点是否访问过，而且判断当前的query是否和之前设置的该叶子结点的判别相符 但是如果相同id的query只在单一特征上不同 LINE SEARCH and CATEGORY SPLIT就会在特征区分上出现问题

Extraction with Confidence Values Decision Tree Path-Finding Attacks Attack Evaluation Querying partial inputs vastly improves attack: require far less queries and achieve higher accuracy for trees with duplicate leaf ids. both attacks achieve perfect extraction when all leaves have unique ids. 结果： 对于有冗余的叶子结点，部分查询可以减少query次数并提高精度 两种方法在所有叶节点没有冗余时都取得很好的效果

Online Model Extraction Attacks Case Study 1: BigML Decision trees model train a tree on the German Credit data. Two using respectively 1722 and 1150 queries. Both attacks’ duration (1030 seconds and 631 seconds) is dominated by query latency. 案例研究1：BigML 使用决策树模型 在德国信用数据上训练一棵树，并将其设置为黑盒模型。 两个攻击（算法1和自顶向下的变体）分别使用1722和1150个查询来提取树的路径的精确描述。 两次攻击的延迟时间（1030秒和631秒）主要是查询延迟。

Online Model Extraction Attacks Case Study 2: Amazon Web Services logistic regression model In all cases, the extracted model matches f on 100% of tested inputs. 逻辑回归模型 在所有情况下，提取的模型可以达到100％匹配。

Online Model Extraction Attacks Learning unknown model classes or hyper- parameters. extract-and-test approach: each attack is applied in turn, and evaluated by computing Rtest or Runif over a chosen set of points. The adversary succeeds if any of the strategies achieves a low error. 学习未知模型类或超参数。 提取与测试方法： 先提出假设 再对假设进行验证 error很低的时候取得成功

Extraction Given Class Labels Only explore model extraction in a setting with no confidence scores The Lowd-Meek attack present an attack on any linear classifier, A linear classifier classifies an instance x as positive if and negative otherwise. SVMs with linear kernels and binary LRs are examples of linear classifiers. Their attack uses line searches to find points arbitrarily close to decision boundary straightforward extension : polynomial kernel SVMs. 在无置信分数信息的情况下探索模型提取 Lowd-Meek攻击 对任何线性分类器进行攻击，线性分类器将实例x分为正，否则为负。 具有线性内核和二进制LR的SVM是线性分类器的示例。 使用线搜索来查找任意靠近决策边界的点 一些非线性模型的直接扩展：多项式核SVM。 将内核SVM转换为线性SVM，然后可以使用Lowd-Meek攻击来提取参数。

Extraction Given Class Labels Only Strategy of re-training a model locally Retraining with uniform queries: sampling m points uniformly at random and training a model on these samples. Line-search retraining: issues m adaptive queries using line search techniques, to find samples close to the decision boundaries. A model is then trained on the m queried samples. Adaptive retraining.: applies techniques from active learning. For some number r of rounds and a query budget m, it queries the oracle on m/r uniform points, and trains a model. 三种Retraining 模型 统一查询重新训练：随机采样m个点，并对这些样本进行训练。 线搜索再培训：使用线搜索技术发现自适应查询，找到接近决策边界的样本。 然后对模型样本进行培训。 适应性再培训：应用主动学习技巧。 对于一些轮次和查询预算m，它会在m / r均匀点上查询oracle，并训练一个模型。 在总共r轮中，沿着决策边界选择m / r个新点 在第r轮训练中选择决策边界附近的m/r个新样本重新加入样本集进行训练

Extraction Given Class Labels Only Experiment--Linear Binary Models The retraining strategies perform better than simple uniform retraining. The adaptive strategy is the most efficient of three retraining strategies. However, for budgets large ，the Lowd-Meek attack is clearly the most efficient 在决策边界附近搜索点的retrain策略明显比简单的uniform retrain 算法表现得更好。 适应策略是三项retrain策略中最有效率的。 阿尔法较小时，它甚至超过了Lowd-Meek攻击。 然而，阿尔法较大时，Lowd-Meek攻击显然是最有效的

Extraction Given Class Labels Only Experiment--Multiclass LR Models the adaptive strategy clearly performs best and that the line-search strategy does not improve over uniform retraining all strategies achieve lower Rtest than Runif. It thus appears that for the models we trained, points from the test set are on average ‘far’ from the decision boundaries of f 我们观察到自适应策略明显表现最佳，线搜索策略在整体再培训方面并没有得到改善，这可能是因为线搜索必须在多个决策界限之间i及进行分割。 我们进一步注意到，所有策略Runif>Rtest。 因此看来，对于训练的模型，测试集中的点的均值距离f的决策边界比较“远”

Extraction Given Class Labels Only Experiment--Neural Networks For a budget of 100 * k, where k is the number of model parameters Rtest = 99:16% and Runif = 98:24%, using 108200 queries per model on average. 对于100 * k的预算，其中k是模型参数的数量， 我们得到Rtest = 99：16％，Runif = 98：24％， 平均每个模型使用108; 200个查询。

Extraction Given Class Labels Only Experiment--RBF Kernel SVMs adaptive retraining performs best, adaptive retraining表现最好

Extraction Countermeasures Rounding confidences. A possible defense is to round confidence scores to some fixed precision. class probabilities rounded to 4 or 5 decimal places (as done already in BigML) have no effect on the attack’s success. When rounding further to 3 and 2 decimal places, the attack is weakened, but still vastly outperforms adaptive retraining using class labels only. 舍入情况。 可能的防御是将置信度得分转化为某种固定精度。 我们观察到，类概率四舍五入到4或5个小数位（已经在BigML中已经完成）对攻击的成功没有影响。 当四舍五入到小数点后加上3位和2位时，攻击力就会减弱，但仍然大大超过只使用类标签的retrain 方法

Extraction Countermeasures Differential privacy Differential privacy (DP) decrease the ability of an adversary A to learn information about training set elements, when given access to prediction queries. Ensemble methods. may be more resilient to extraction attacks, in the sense that attackers will only be able to obtain relatively coarse approximations of the target function. 差分隐私 差异隐私（DP）降低在获得预测查询时可以了解有关训练集样本信息的能力。 集成学习方法。 可能对提取攻击更具弹性，因为攻击者只能获得目标功能的较粗略的近似值。

谢 谢！