Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 14 hours
Course Outline
Introduction to Multimodal LLMs in Vertex AI
- Overview of multimodal features available in Vertex AI
- Gemini models and their supported modalities
- Enterprise and research application scenarios
Configuring the Development Environment
- Setting up Vertex AI for multimodal workflows
- Handling datasets across different modalities
- Lab: Environment setup and dataset preparation
Long Context Windows and Advanced Reasoning
- Concepts behind long-context workflows
- Applications in strategic planning and decision-making
- Lab: Implementing long-context analysis
Designing Cross-Modal Workflows
- Integrating text, audio, and image analysis
- Chaining multimodal steps within pipelines
- Lab: Designing a multimodal pipeline
Managing Gemini API Parameters
- Configuring multimodal input and output specifications
- Optimizing inference speed and resource efficiency
- Lab: Tuning Gemini API parameters
Advanced Applications and Integrations
- Building interactive multimodal agents and assistants
- Integrating with external APIs and toolsets
- Lab: Developing a complete multimodal application
Evaluation and Iterative Improvement
- Assessing multimodal system performance
- Key metrics for accuracy, alignment, and drift
- Lab: Evaluating multimodal workflows
Recap and Future Directions
Requirements
- Strong proficiency in Python programming
- Practical experience in developing machine learning models
- Working knowledge of multimodal data types (text, audio, and image)
Target Audience
- AI Researchers
- Senior Developers
- Machine Learning Scientists
