[AI] Interesting Deep Learning Algorithms

4  Interesting Deep Learning Algorithms

1. Artificial Neural Networks (ANNs)

👉 With 3 Layers

- Input layer: No computation is performed,  just used for passing information from the outside to the network.

- Hidden layer: Deriving complex relationships between input and output. Identifies the pattern in the dataset, learning the data representation and for extracting the features. We cloud use many hidden layers, each layer is responsible for extracting important features.

- Output layer: Number of neurons in the output layer is based on the type of problem we want our network to solve (Example: binary classification, number of neurons in the output layer is one). 

⏩ Other Concept:

- Activation functions (ex: sigmoid, tanh, ReLU, softmax): To introduce a non-linear transformation to learn the complex underlying patterns in the data.

- Cost function: Use Optimization Algorithms (Gradient descent) to minimize cost so that Network predictions will be better. 

2. Recurrent Neural Networks (RNN)

Predict output not only based on the current input, but also on the previous hidden state (which captured the contextual information and state as memory). 

Face with vanishing gradient problem (gradients will tend to get smaller and smaller).

3. Long Short-Term Memory (LSTM): 

Use Three gates are responsible for deciding what information to add, output, and forget from the memory:

- Forget gate: deciding what information should not be in the cell state. 

- Input gate: deciding what information should be stored in the memory.

- Output gate: deciding what information should be shown from the cell state at a time.

4.  Convolutional Neural Network (CNN or ConvNet)

💬 Consists of three important layers:

- Convolutional layer: Extracting important features from the image. New matrix (call as feature map or activation map) created by convolution operation. We can use multiple filters to extract different features.

- Pooling layer: Reduce the dimensions of feature maps by perform a pooling operation (also call as downsampling or subsampling operation). it has different types of pooling operations,  max/average  pooling: take maximum/average value from the filter window; sum pooling: sum all the values of the input matrix within the filter window.

- Fully Connected layer: Multiple convolutional layers x pooling layers. need an algorithm can classify these extracted features. Can use many Convolutional layers followed by a pooling layer (not necessarily have to be there after each convolutional layer). Basically a feedforward neural network that classifies the given input image based on the feature maps.