李宏毅專題 Track A, B, C 的時間、地點開學前通知 深度學習 暑期訓練 (2016) 李宏毅專題 Track A, B, C 的時間、地點開學前通知

作業繳交 四個作業 因為是暑假，所以並不強迫繳交 請加入 FB 社團「深度學習暑期訓練 (2016)」 建議修李宏毅專題的同學如果對deep learning 沒有概念 的話一定要做 等一下會講作業大概的內容 都很簡單 作業詳細內容和做法提示預計下周二(8/09)公告在社團上 會使用 keras 這套工具 (歡迎自學 Tensorflow )

學習教材 「資料科學愛好者年會」六小時演講錄影 嚴禁外流

學習教材 Lecture 1 深度學習簡介 Lecture 2 深度學習技巧 Lecture 3 Convolutional Neural Network (CNN) 和 Recurrent Neural Network (RNN) Lecture 4 深度學習應用與展望 投影片: http://www.slideshare.net/tw_dsconf/ss-62245351 (可公開)

助教 作業一: 盧柏儒 作業二: 黃邦齊 作業三: 李致緯  作業四: 萬家宏  其他助教：莊舜博、鍾佩宏、張瓊之、楊棋宇 作業問題在社團上討論

工作站 專題工作站 在週四 (8/04) 中午前以下 google doc 登記 下週二前(8/09)開帳號 如果每組自己有 linux 系統會比較快樂 如果每組自己有 GPU 會更快樂

作業說明

Machine Learning You said “Hello” Learning ...... “Hi” “How are you” “Good bye” You write the program for learning. A large amount of audio data

Machine Learning This is “cat” Learning ...... “monkey” “cat” “dog” You write the program for learning. A large amount of images

Machine Learning ≈ Looking for a Function Speech Recognition Image Recognition Playing Go Dialogue System “How are you” “Cat” “5-5” (next move) “Hi” “Hello” (what the user said) (system response)

Framework Model Image Recognition: “cat” A set of function “cat” “money” Repeat again “dot” “snake”

Framework Model Supervised Learning Image Recognition: “cat” A set of function Model Better! Goodness of function f Repeat again Supervised Learning Training Data function input: function output: “monkey” “cat” “dog”

Pick the “Best” Function Image Recognition: Framework “cat” A set of function Neural Network Training Testing Model “cat” Step 1 Goodness of function f Using Pick the “Best” Function Repeat again Step 2 Step 3 Training Data “monkey” “cat” “dog”

Deep means many hidden layers Neural Network neuron Input Layer 1 …… Layer 2 …… Layer L …… Output …… y1 …… y2 …… …… You can connect the neurons by other ways you like  How many layer is deep? CNN just another way to connect the neuros. You can always connect the neurons in your own way. “+” is ignored Each dimension corresponds to a digit (10 dimension is needed) …… yM Input Layer Output Layer Hidden Layers Deep means many hidden layers

作業一 教材: lecture 1, lecture 2 Task 1 “5” “0” “4” “1” basic：higher than a specific accuracy (等助教公布) option：analyze the output of each layer

作業一 教材: lecture 1, lecture 2 Task 2 Network 政治 體育 政治 財經 http://top-breaking-news.com/ 體育 政治 財經 basic：higher than a specific accuracy (等助教公布) option：analyze the output of each layer

作業一 教材: lecture 1, lecture 2 More reference: Example code of task 1: https://github.com/fchollet/keras/blob/master/example s/mnist_mlp.py Example code of task 2: https://github.com/fchollet/keras/blob/master/example s/reuters_mlp.py Neural Networks and Deep Learning http://neuralnetworksanddeeplearning.com/ Chapter 1 - 3

作業二 教材: lecture 3 – Convolutional Neural Network Image Recognition “monkey” “monkey” Network “cat” “cat” “dog” “dog”

作業二 教材: lecture 3 – Convolutional Neural Network Basic：higher than a specific accuracy (等助教公 布) option：analyze the functionality of “filter” More Reference: Example code: https://github.com/fchollet/keras/blob/master/example s/cifar10_cnn.py http://cs231n.github.io/convolutional-networks/ Neural Networks and Deep Learning (Chapter 6)

作業三 教材: lecture 3 – Recurrent neural network Machine learns human language Machine writes documents The life is ……. You do not have to teach machine grammars ……. Basic: Let machine generate an English sentence Option: Let machine generate a Chinese sentence

作業三 教材: lecture 3 – Recurrent neural network Machine learns human language More reference Example code: https://github.com/fchollet/keras/blob/master/ examples/lstm_text_generation.py http://karpathy.github.io/2015/05/21/rnn- effectiveness/ http://colah.github.io/posts/2015-08- Understanding-LSTMs/

作業四 教材: lecture 4 Auto-encoder: unsupervised learning NN Encoder code Basic: visualize the “code”. Does different digits represent by different “code”? NN Decoder code Option: Given a “code”, can machine write a digit?

作業四 教材: lecture 4 Auto-encoder: unsupervised learning More reference: https://blog.keras.io/building-autoencoders-in- keras.html Advanced: Auto-Encoding Variational Bayes, https://arxiv.org/abs/1312.6114 Generative Adversarial Networks, http://arxiv.org/abs/1406.2661 Replacing “digits” with “images”

Have Fun! 