Automating Visual Privacy Protection Using a Smart LED Shilin Zhu∗ University of California-San Diego shz338@eng.ucsd.edu Chi Zhang∗ University of Wisconsin- Madison czhang296@wisc.edu Xinyu Zhang University of California-San Diego xiz368@eng.ucsd.edu

摄像头无处不在，隐私空间容易暴露

This work We propose LiShield, a system that deters photographing of sensitive indoor physical space, and automatically enforces location-bound visual privacy protection. Our key idea is to illuminate the environment using smart LEDs, and make it is imperceptible by human eyes, but can interfere with the image sensors on mobile camera devices

objective 1 2 3 Striping the captured image of rolling-shutter camera. Allowing the authorized camera to recover the image. cannot ensure full protection in strong ambient light, but still can embeds invisible “barcode” into the physical environment. 1 2 3

Human eyes Vs. cameras Human eyes Not sensitive to higher frequency flickers beyond 80 Hz. Human eyes Rolling-shutter sampling Can easily pick up flickers above a few kHz. cameras

Basic waveform Basic waveform follows an ON-OFF modulation, which causes the reflection intensity of the scene to “flicker” at high frequency. LiShield will impose a striping effect on the captured image, as long as its flickering frequency exceeds the camera frame rate.

Maximizing Image Quality Degradation Exposure time 𝑡 𝑒 can effect the image quality.

Maximizing Image Quality Degradation If there are other parameters can influence the image capturing quality?

Maximizing Image Quality Degradation A model driven approach to optimize the waveform

Numerical model Captured image quality Parameters of waveform Parameters of camera Captured image quality

Numerical model PSNR: 峰值信噪比 SSIM: 结构相似性

Model analysis result (1) A single frequency cannot ensure robust protection LiShield includes a countermeasure called frequency randomization 若固定闪光频率，暴力搜索可以找到恰好曝光时间等于闪光周期的情况，此时可获得高图片质量。

Model analysis result (2) LiShield must prevent attackers from using long exposures. LiShield should leverage overexposure to limit attacker’s exposure time. 观察发现：曝光时间越长，照片质量越高 但是曝光时间越长也就越容易过曝。 利用过曝来限制攻击者的曝光时长。

Model analysis result (3) LiShield should keep a high peak intensity to expand the overexposure zone. With power efficiency and eye health in mind, LiShield sets peak intensity to 20 kLx by default. 峰值越高越容易过曝

Circumventing Potential Attacks Manual exposure attack Multi-frame attack Post-processing attack Manual exposure attack: 暴力搜索合适的曝光时长 Multi-frame attack：从多帧中找出清晰的条，拼接在一起 Post-processing attack： 图像处理方法 去噪、去条带

Manual exposure attack Frequency Scrambling

Multi-frame attack Illumination Intensity Randomization For static sene, attackers may find at least one clean version for each row across all frames, thus recovering the image. Use K intensity levels, and extends the attacker’s search space

Scene recovery with authorized cameras Authorized Video Recording Static Scene Recovery

Authorized Video Recording The authorized camera needs to convey its exposure time setting 𝑡 𝑢 𝑒 to the smart LED, and synchronize its clock with the smart LED’s clock So to authorize the user with exposure 𝑡 𝑢 𝑒 , LiShield simply needs to set its flickering frequency 𝑓 𝑎 =1/ 𝑡 𝑙 =N/ 𝑡 𝑢 𝑒 (N =1, 2, . . .)

Authorized Video Recording when the authorized camera is not recording at its maximum possible rate, there will be an interval where the camera pauses capturing. LiShield packs random flickering frequencies other than 𝑓 𝑎 into the inter-frame gap, so as to achieve the same scrambling effect on the attackers. (e.g., a 30 fps camera recording at 25 fps)

Static Scene Recovery To prevent attackers from launching the multi-frame attack, the timing of the critical frames is negotiated only between the smart LED and the authorized user through the secure side channel.

Automatic physical watermarking for privacy enforcement 不同的相机参数会解出不同的频率值，但是频率的比值是一致的。 LiShield embeds two waveforms with frequencies 𝐹 0 and 𝐹 1 , 𝑭 𝟏 / 𝑭 𝟎 equals to a value 𝑅 𝑝 well known to the policy enforcers.

Automatic physical watermarking for privacy enforcement Detect 流程 把图片划分成许多blocks 算出每个block中的闪光频率 计算所有block中频率的两两比值 看这些比值中有多少个能和 𝑅 𝑝 中的值匹配 如果匹配的值达到一定数量，则通过判断

Implementation

Experimental evaluation Impact of flickering frequency

Experimental evaluation Effectiveness of user authorization

Experimental evaluation Effectiveness of user authorization

Experimental evaluation 单色的不如RGB的 We conclude that LiShield’s barcode detector provides reliable detection

Experimental evaluation --Counteracting Attacks Manual exposure attack Vs. Frequency scrambling 单色的不如RGB的

Experimental evaluation --Counteracting Attacks Denoising 去噪，这种条带不符合已知噪声类型 去条带的两个也不行，差值无法补上失去的像素。 提高了其中一个质量值，也无法提高其他的 It is practically impossible to remove LiShield’s impact by image processing

Conlusion In this paper we propose LiShield, which uses smart-LEDs and specialized intensity waveforms to disrupt unauthorized cameras, while allowing authorized users to record high quality image and video. We consider LiShield as a first exploration of automatic visual privacy enforcement and expect it can inspire more research along the same direction.

Thanks