Learning Lightweight Lane Detection CNNs by Self Attention Distillation Yuenan Hou, Zheng Ma, Chunxiao Liu, and Chen Change Loy The Chinese University of Hong Kong SenseTime Group Limited Nanyang Technological University
1 Main idea SAD:Self Attention Distillation 使用深层block来监督浅层block
2 Implementation Details 从block的C*H*W输出得到H*W的activation map p越大->越集中于highly activated area sum相比于max->less bias 选择p=2,sum进行实验 Mimic path 引入SAD时间
3 Experiment results
3 Experiment results Mimic path: 浅层block包含较多底层语义,不适合直接用深层block监督 学习相邻层结果更佳
3 Experiment results Time to introduce:过早引入影响收敛速度