Coursera Deep Learning Specialization

This is the repository for my implementations on the Deep Learning Specialization from Coursera.

Taught by Andrew Ng

Syllabus

Course 1. Neural Networks and Deep Learning

Foundations of Deep Learning:

• Understand the major technology trends driving Deep Learning
• Be able to build, train and apply fully connected deep neural networks
• Know how to implement efficient (vectorized) neural networks
• Understand the key parameters in a neural network's architecture

Codes:

• Week2: Neural Network Basics
• Week3: Shallow Neural Network Implementation
• Week4: Deep Neural Network Implementation
  • Mathematical demonstration: Backpropagation
  • Mathematical demonstration: Cross-entropy & Softmax gradients

Course 2. Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization

• Understand industry best-practices for building deep learning applications.
• Be able to effectively use the common neural network "tricks", including initialization, L2 and dropout regularization, Batch normalization, gradient checking,
• Be able to implement and apply a variety of optimization algorithms, such as mini-batch gradient descent, Momentum, RMSprop and Adam, and check for their convergence.
• Understand new best-practices for the deep learning era of how to set up train/dev/test sets and analyze bias/variance
• Be able to implement a neural network in TensorFlow.

Codes:

• Week1: Initialization, Regularization & Gradient Check
• Week2: Optimization Algorithms
• Week3: Hyperparameter tuning, Batch Normalization & Tensorflow Implementation
  • Mathematical demostration: Batch Normalization Gradient
  • Paper: Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Course 3. Structuring Machine Learning Projects

• Understand how to diagnose errors in a machine learning system, and
• Be able to prioritize the most promising directions for reducing error
• Understand complex ML settings, such as mismatched training/test sets, and comparing to and/or surpassing human-level performance
• Know how to apply end-to-end learning, transfer learning, and multi-task learning

Course 4. Convolutional Neural Networks

• Understand how to build a convolutional neural network, including recent variations such as residual networks.
• Know how to apply convolutional networks to visual detection and recognition tasks.
• Know to use neural style transfer to generate art.
• Be able to apply these algorithms to a variety of image, video, and other 2D or 3D data.

Codes:

• Week1: Convolutional Neural Network Implementation in Numpy
• Week2:
  • Keras CNN Implementation
  • ResNet Keras Implementation
  • Paper: Deep Residual Learning for Image Recognition
• Week3:
  • YOLO Implementation(You Only Look Once)
  • Paper: You Only Look Once: Unified, Real-Time Object Detection
  • Paper: YOLO9000: Better, Faster, Stronger
• Week4:
  • Neural Style Transfer
  • Paper: A Neural Algorithm of Artistic Style
  • Face Recognition
  • Paper: FaceNet: A Unified Embedding for Face Recognition and Clustering
  • Paper: Going deeper with convolutions (Inception Networks)

Course 5. Sequential Models

• Understand how to build and train Recurrent Neural Networks (RNNs), and commonly-used variants such as GRUs and LSTMs.
• Be able to apply sequence models to natural language problems, including text synthesis.
• Be able to apply sequence models to audio applications, including speech recognition and music synthesis.

Codes:

• Week1:
  • RNN & LSTM Implementation in Numpy (Including backpropagation)
    • Mathematical demonstration: RNN gradient through time
    • Mathematical demonstration: LSTM gradient through time
    • Mathematical demonstration: GRU gradient through time
    • Paper: Vanishing/Exploding gradient & Clipping
  • Character-Level Language Modeling
  • Sequence Sampling Generation LSTM
• Week2:
  • Natural Language Processing & Word Embeddings
  • Operations on word vectors - Debiasing
• Week3:
  • Neural Machine Translation with Attention
  • Trigger word detection