Course Outline
DAY 1 - ARTIFICIAL NEURAL NETWORKS
Introduction and ANN Architecture.
- Comparison between biological and artificial neurons.
- The conceptual model of an ANN.
- Activation functions employed in ANNs.
- Standard categories of network architectures.
Mathematical Underpinnings and Learning Mechanisms.
- Review of vector and matrix algebra.
- State-space principles.
- Optimization concepts.
- Error-driven learning methods.
- Memory-driven learning approaches.
- Hebbian learning rules.
- Competitive learning strategies.
Single-Layer Perceptrons.
- Perceptron structure and learning processes.
- Introduction to pattern classifiers and Bayes' classifiers.
- Utilizing perceptrons as pattern classifiers.
- Perceptron convergence properties.
- Inherent limitations of perceptrons.
Feedforward ANNs.
- Architecture of multi-layer feedforward networks.
- The backpropagation algorithm.
- Training and convergence in backpropagation.
- Functional approximation via backpropagation.
- Practical considerations and design challenges in backpropagation learning.
Radial Basis Function (RBF) Networks.
- Pattern separability and interpolation techniques.
- Theory of regularization.
- Regularization applied to RBF networks.
- Design and training of RBF networks.
- Approximation capabilities of RBFs.
Competitive Learning and Self-Organizing ANNs.
- General clustering methodologies.
- Learning Vector Quantization (LVQ).
- Competitive learning algorithms and architectures.
- Self-organizing feature maps.
- Characteristics of feature maps.
Fuzzy Neural Networks.
- Neuro-fuzzy systems.
- Foundations of fuzzy sets and logic.
- Design of fuzzy systems.
- Architecture of fuzzy ANNs.
Practical Applications
- Discussion of select Neural Network applications, highlighting their benefits and potential challenges.
DAY 2 - MACHINE LEARNING
- The PAC Learning Framework
- Guarantees for finite hypothesis sets – consistent scenarios
- Guarantees for finite hypothesis sets – inconsistent scenarios
- General principles
- Deterministic vs. Stochastic scenarios
- Bayes error and noise
- Estimation and approximation errors
- Model selection strategies
- Rademacher Complexity and VC Dimension
- Bias-Variance tradeoff
- Regularization techniques
- Overfitting prevention
- Validation methods
- Support Vector Machines
- Kriging (Gaussian Process Regression)
- PCA and Kernel PCA
- Self-Organizing Maps (SOM)
- Kernel-induced vector spaces
- Mercer Kernels and kernel-induced similarity metrics
- Reinforcement Learning
DAY 3 - DEEP LEARNING
Content will be taught in context with topics covered on Days 1 and 2
- Logistic and Softmax Regression
- Sparse Autoencoders
- Vectorization, PCA, and Whitening
- Self-Taught Learning
- Deep Network Architectures
- Linear Decoders
- Convolution and Pooling
- Sparse Coding
- Independent Component Analysis
- Canonical Correlation Analysis
- Demonstrations and Real-world Applications
Requirements
A solid grasp of mathematics is essential.
A strong comprehension of fundamental statistical principles is required.
While basic programming skills are not mandatory, they are highly recommended.
Testimonials (2)
Working from first principles in a focused way, and moving to applying case studies within the same day
Maggie Webb - Department of Jobs, Regions, and Precincts
Course - Artificial Neural Networks, Machine Learning, Deep Thinking
It was very interactive and more relaxed and informal than expected. We covered lots of topics in the time and the trainer was always receptive to talking more in detail or more generally about the topics and how they were related. I feel the training has given me the tools to continue learning as opposed to it being a one off session where learning stops once you've finished which is very important given the scale and complexity of the topic.
