Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 14 hours
Course Outline
Foundations of Parameter-Efficient Fine-Tuning (PEFT)
- The motivation for PEFT and the limitations of full fine-tuning
- An overview of PEFT objectives and advantages
- Real-world industry applications and use cases
LoRA (Low-Rank Adaptation)
- The conceptual framework and intuition behind LoRA
- Implementation of LoRA using Hugging Face and PyTorch
- Practical exercise: Fine-tuning a model with LoRA
Adapter Tuning
- The mechanics of adapter modules
- Integration strategies for transformer-based models
- Practical exercise: Applying Adapter Tuning to a transformer model
Prefix Tuning
- Leveraging soft prompts for the fine-tuning process
- Comparative strengths and limitations relative to LoRA and adapters
- Practical exercise: Executing Prefix Tuning on an LLM task
Evaluation and Comparative Analysis of PEFT Methods
- Key metrics for assessing performance and efficiency
- Balancing trade-offs in training speed, memory consumption, and accuracy
- Conducting benchmarking experiments and interpreting results
Deployment of Fine-Tuned Models
- Procedures for saving and loading fine-tuned models
- Considerations for deploying PEFT-based models
- Integration into production applications and workflows
Best Practices and Advanced Extensions
- Combining PEFT with quantization and distillation techniques
- Applications in low-resource and multilingual environments
- Exploring future trends and active areas of research
Requirements
- A solid foundation in machine learning principles
- Practical experience in working with Large Language Models (LLMs)
- Proficiency in Python and PyTorch
Target Audience
- Data Scientists
- AI Engineers
