《Deep Learning》

0.书本介绍

作者: Ian Goodfellow / Yoshua
Bengio / Aaron Courville

内容简介：

"Written by three experts in the field,
Deep Learning is the only comprehensive book on the subject." -- Elon Musk, co-
chair of OpenAI; co-founder and CEO of Tesla and SpaceX
Deep learning is a form
of machine learning that enables computers to learn from experience and
understand the world in terms of a hierarchy of concepts. Because the computer
gathers knowledge from experience, there is no need for a human computer
operator to formally specify all the knowledge that the computer needs. The
hierarchy of concepts allows the computer to learn complicated concepts by
building them out of simpler ones; a graph of these hierarchies would be many
layers deep. This book introduces a broad range of topics in deep learning.
The
text offers mathematical and conceptual background, covering relevant concepts
in linear algebra, probability theory and information theory, numerical
computation, and machine learning. It describes deep learning techniques used by
practitioners in industry, including deep feedforward networks, regularization,
optimization algorithms, convolutional networks, sequence modeling, and
practical methodology; and it surveys such applications as natural language
processing, speech recognition, computer vision, online recommendation systems,
bioinformatics, and videogames. Finally, the book offers research perspectives,
covering such theoretical topics as linear factor models, autoencoders,
representation learning, structured probabilistic models, Monte Carlo methods,
the partition function, approximate inference, and deep generative models.
Deep
Learning can be used by undergraduate or graduate students planning careers in
either industry or research, and by software engineers who want to begin using
deep learning in their products or platforms. A website offers supplementary
material for both readers and instructors.

这本书由GAN的发明人Ian Goodfellow主写，系统地介绍了深度学习的基础知识和后续发展，是一本值得反复读的好书。

这里我做的笔记是基于本书的框架，但内容不限于来自本书，最终目的是加深对知识本身的理解。

笔记也会不断更新，只要在今后在工作中碰到其中的问题需要进一步研究学习，就会继续丰富此笔记的内容。

1. Introduction

什么是machine learning?

在原始的AI系统中，定义不同的case使用不同的解决方法，这称为“hard code”。进一步的AI系统需要一种去获取知识的能力，也就是从原始数据中发现模式（“Pattern”），这种能力就是machine learning。

但是，一般的machine learning算法严重依赖于数据的表示(representation)，表示中包含的每份信息又称为feature。

这又引发了一个新的问题，对于很多task，我们不知道应该提取什么样的特征（只能经验主义）。

于是又有了representation learning，即使用machine learning不光光是学习reprsentation到output的映射（mapping），还要学习data到representation的映射。

典型的表示学习算法是autoencoder。encoder函数是将输入数据映射成表示;decoder函数将表示映射回原始数据的格式。
representation learning的难点：表示是多种多样的，一种表示学习算法很难覆盖多种层次和不同类型的表示。

Deep Learning：使用多层次的结构，用简单的表示来获取高层的表示。这样，解决了上面的问题（一种方法）。