Kafka for Administrators Training Course

Fundamentals of Kafka Administration

• The role of Kafka in modern data platforms and typical production responsibilities
• Core operator concepts: brokers, topics, partitions, offsets, and consumer groups
• Replication basics: leaders and followers, in-sync replicas, and availability trade-offs
• Key operational aspects and standard terminology found in runbooks

KRaft Mode and Cluster Architecture

• KRaft fundamentals: controllers, metadata quorum, elections, and their operational significance
• Deployment strategy: sizing for throughput, partitioning, retention policies, and scalability
• Node roles and configurations: combined vs. dedicated controllers, and fault domain planning
• Practical exercise: Inspect KRaft metadata, verify quorum health, and analyze controller logs

Installation, Configuration, and Routine Operations

• Deployment methods (packages, tarballs, containers) and standards for enterprise environments
• Essential broker settings affecting reliability: listeners, replication, log directories, and retention
• Safe service management: startup sequences, graceful shutdowns, and verification procedures
• Practical exercise: Deploy a multi-node cluster, confirm broker registration, and validate basic produce/consume workflows

Topic, Partition, and Data Management

• Topic lifecycle management via the Kafka CLI: creation, description, configuration updates, and deletion
• Selecting appropriate partition counts and replication factors for specific workloads, while avoiding common pitfalls
• Partition reassignment and balancing: identifying when to move partitions and safely tracking progress
• Practical exercise: Create topics, initiate a partition reassignment, simulate a broker failure, and verify recovery

Production Security for Kafka

• Implementing TLS for client and inter-broker traffic: certificate management, trust chains, and validation
• SASL authentication: selecting appropriate mechanisms and preventing misconfigurations
• ACL-based authorization: enforcing least-privilege principles for admins, producers, and consumers
• Practical exercise: Enable TLS and SASL, test client connectivity, and apply ACLs for specific application roles

Observability, Reliability, and Diagnostics

• Key monitoring indicators: controller health, under-replicated partitions, request latency, and disk/network saturation
• Log analysis and metrics: interpreting broker logs and integrating metrics via JMX exporter into observability stacks
• Operational playbooks: managing rolling restarts, safe configuration changes, and resolving disk-full and ISR issues
• Practical exercise: Establish a basic alert framework, diagnose a degraded cluster, and restore healthy replication

Upgrades and Disaster Recovery Preparedness

• Kafka upgrade strategy: compatibility verification, staging processes, and rollback procedures
• Backup and recovery expectations: identifying recoverable data, limitations, and configuration restoration basics
• Overview of cross-cluster replication and the application of MirrorMaker 2 for DR and migration scenarios
• Summary: Operational checklist, handover documentation, and next steps for production implementation