Get in Touch
 Duration 21 hours

Course Outline

Foundations of Data-Intensive Platform Engineering

  • Introduction to data-intensive applications.
  • Key challenges in platform engineering for big data.
  • Overview of data processing architectures.

Data Modeling and Management

  • Core principles of data modeling for scalability.
  • Exploring data storage options and optimization techniques.
  • Managing the data lifecycle within a distributed environment.

Big Data Processing Frameworks

  • Overview of major big data processing tools (Hadoop, Spark, Flink).
  • Distinguishing between batch and stream processing.
  • Setting up a functional big data processing pipeline.

Real-Time Analytics Platforms

  • Architectural strategies for real-time analytics.
  • Utilizing stream processing engines (Kafka Streams, Apache Storm).
  • Building real-time dashboards and visualizations.

Data Pipeline Orchestration

  • Managing workflows with Apache Airflow and similar tools.
  • Automating data pipelines to enhance efficiency.
  • Implementing monitoring and alerting mechanisms for data pipelines.

Platform Security and Compliance

  • Security best practices specific to data platforms.
  • Safeguarding data privacy and ensuring regulatory compliance.
  • Implementing robust data access controls.

Performance Tuning and Optimization

  • Techniques for improving data throughput and reducing latency.
  • Scaling strategies for data-intensive platforms.
  • Conducting performance benchmarking and continuous monitoring.

Case Studies and Best Practices

  • Analyzing successful data platform implementations.
  • Key lessons learned from industry leaders.
  • Emerging trends in data-intensive platform engineering.

Capstone Project

  • Designing a platform solution for a specific data-intensive application.
  • Implementing a prototype of the data processing pipeline.
  • Evaluating the platform's overall performance and scalability.

Summary and Next Steps

Requirements

  • A solid grasp of fundamental data structures and algorithms.
  • Proficiency in programming with Java, Scala, or Python.
  • A good working knowledge of core database concepts and SQL.

Target Audience

  • Software developers.
  • Data engineers.
  • Technical leads.

Number of participants


Price per participant (excl. VAT)

Testimonials (2)

Upcoming Courses

Related Categories