Get in Touch
 Duration 14 hours

Course Outline

Prep Machine Learning Models for Production Deployment

  • Model packaging using Docker
  • Exporting models from TensorFlow and PyTorch
  • Considerations for versioning and storage

Serving Models on Kubernetes

  • An overview of inference servers
  • Deploying TensorFlow Serving and TorchServe
  • Establishing model endpoints

Optimizing Inference Performance

  • Strategies for batching requests
  • Managing concurrent request handling
  • Tuning latency and throughput

Autoscaling ML Workloads

  • Horizontal Pod Autoscaler (HPA)
  • Vertical Pod Autoscaler (VPA)
  • Kubernetes Event-Driven Autoscaling (KEDA)

GPU Provisioning and Resource Administration

  • Configuration of GPU nodes
  • Overview of the NVIDIA device plugin
  • Defining resource requests and limits for ML workloads

Strategies for Model Rollout and Release

  • Blue/green deployment patterns
  • Canary rollout methodologies
  • A/B testing for model assessment

Monitoring and Observability for Production ML

  • Key metrics for inference workloads
  • Best practices for logging and tracing
  • Creating dashboards and setting alerts

Addressing Security and Reliability

  • Protecting model endpoints
  • Implementing network policies and access controls
  • Ensuring high availability

Summary and Future Steps

Requirements

  • A solid understanding of containerized application workflows
  • Practical experience with Python-based machine learning models
  • Familiarity with core Kubernetes concepts

Target Audience

  • ML engineers
  • DevOps engineers
  • Platform engineering teams

Number of participants


Price per participant (excl. VAT)

Testimonials (1)

Upcoming Courses

Related Categories