Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 14 hours
Course Outline
Introduction to Open-Source LLMs
- The definition of open-weight models and their strategic importance.
- An overview of LLaMA, Mistral, Qwen, and other community-driven models.
- Key use cases for private, on-premise, or secure deployments.
Environment Setup and Tools
- Installation and configuration of Transformers, Datasets, and PEFT libraries.
- Selecting appropriate hardware specifications for fine-tuning tasks.
- Loading pre-trained models from Hugging Face or alternative repositories.
Data Preparation and Preprocessing
- Understanding dataset formats, including instruction tuning, chat data, and text-only inputs.
- Managing tokenization and sequence lengths effectively.
- Constructing custom datasets and efficient data loaders.
Fine-Tuning Techniques
- Comparing standard full fine-tuning with parameter-efficient methods.
- Implementing LoRA and QLoRA for efficient fine-tuning processes.
- Leveraging the Trainer API for rapid experimentation.
Model Evaluation and Optimization
- Assessing fine-tuned models using generation quality and accuracy metrics.
- Managing overfitting, generalization capabilities, and validation sets.
- Strategies for performance tuning and comprehensive logging.
Deployment and Private Use
- Procedures for saving and loading models optimized for inference.
- Deploying fine-tuned models within secure enterprise environments.
- Evaluating on-premise versus cloud deployment strategies.
Case Studies and Use Cases
- Real-world examples of enterprise adoption of LLaMA, Mistral, and Qwen.
- Approaches to handling multilingual and domain-specific fine-tuning.
- Discussion on the trade-offs between open and closed models.
Summary and Next Steps
Requirements
- A solid understanding of large language models (LLMs) and their underlying architecture.
- Practical experience with Python and PyTorch.
- Basic familiarity with the Hugging Face ecosystem.
Target Audience
- ML practitioners.
- AI developers.
