Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 21 hours
Course Outline
Foundations of Data-Intensive Platform Engineering
- Introduction to data-intensive applications.
- Key challenges in platform engineering for big data.
- Overview of data processing architectures.
Data Modeling and Management
- Core principles of data modeling for scalability.
- Exploring data storage options and optimization techniques.
- Managing the data lifecycle within a distributed environment.
Big Data Processing Frameworks
- Overview of major big data processing tools (Hadoop, Spark, Flink).
- Distinguishing between batch and stream processing.
- Setting up a functional big data processing pipeline.
Real-Time Analytics Platforms
- Architectural strategies for real-time analytics.
- Utilizing stream processing engines (Kafka Streams, Apache Storm).
- Building real-time dashboards and visualizations.
Data Pipeline Orchestration
- Managing workflows with Apache Airflow and similar tools.
- Automating data pipelines to enhance efficiency.
- Implementing monitoring and alerting mechanisms for data pipelines.
Platform Security and Compliance
- Security best practices specific to data platforms.
- Safeguarding data privacy and ensuring regulatory compliance.
- Implementing robust data access controls.
Performance Tuning and Optimization
- Techniques for improving data throughput and reducing latency.
- Scaling strategies for data-intensive platforms.
- Conducting performance benchmarking and continuous monitoring.
Case Studies and Best Practices
- Analyzing successful data platform implementations.
- Key lessons learned from industry leaders.
- Emerging trends in data-intensive platform engineering.
Capstone Project
- Designing a platform solution for a specific data-intensive application.
- Implementing a prototype of the data processing pipeline.
- Evaluating the platform's overall performance and scalability.
Summary and Next Steps
Requirements
- A solid grasp of fundamental data structures and algorithms.
- Proficiency in programming with Java, Scala, or Python.
- A good working knowledge of core database concepts and SQL.
Target Audience
- Software developers.
- Data engineers.
- Technical leads.
Testimonials (2)
the overall concept of DevOps/DevSecOps is not a team but a culture, a mind set
Mpho Mashele - South Africa Reserve Bank
Course - DevOps and Platform Engineering: A Collaborative Approach
About the microservices and how to maintenance kubernetes