2022 Neural Networks Architecture


  1. Introduction
    • Neural Networks review. Deep Learning. Types of neurons. Feed-forward networks. Backpropagation. DL frameworks. Theano. AutoDiff. TensorFlow. Torch. JAX.
  2. Popular Network Architectures
    • Multi-task learning. Siamese Networks. Generative Adversarial Networks (GAN). Style Transfer. Disentangled Representation Learning.
  3. Structure Learning Networks
    • Hopfield Networks. RBMs. SOMs. Autoencoders. VAE.
  4. Convolution Networks
    • CNNs. DL image processing. YOLO. Retina-Net. SpineNet.
  5. Recurrent Networks
    • RNNs. Training by unrolling. Internal memory, access and update LSTMs. GRUs. Encoder/Decoder. Attention in Encoder/Decoder systems.
  6. Transformer Networks
    • Attention is all you need. Transformers. BERT. GPT. T5. Pretraining. Transfer learning. Zero-shot and few shot learning.
  7. Graph Processing Networks
    • Graph processing architectures. Local vs. global. GNNs. DGCNNs. GCN. MPNN.
  8. Multimedia Processing Networks
    • Multimedia processing architectures. VQA. NMNs. Hierarchical co-attention. Dall-e. ImageGen.
  9. AutoDL
    • Learning architectures. AutoDL. NEAT. DARTS. GDAS.