Self-study course · Google Cloud

GCP Data Engineering

Twelve interactive chapters that turn one book into practice: explore each idea, decide scenarios, run the hands-on steps in your own project, then test yourself. Progress is saved in this browser.

Part I Getting started

Fundamentals of Data Engineering

The data life cycle, lakes vs warehouses, ETL vs ELT, and how big data is stored.

Big Data Capabilities on GCP

What the cloud is, how a GCP project works, and which service does which job.

Part II Building solutions with GCP components

Building a Data Warehouse in BigQuery

Cloud Storage and BigQuery, loading data, and modelling a warehouse layer by layer.

Orchestrating Batch Loads with Cloud Composer

Airflow concepts (DAGs, operators, dependencies) and scheduling a real pipeline.

Building a Data Lake with Dataproc

Hadoop and Spark on GCP, jobs on a cluster, and why ephemeral clusters save money.

Streaming with Pub/Sub and Dataflow

Event streams, topics and subscriptions, windows, and a streaming pipeline.

Visualizing Data with Data Studio (now Looker Studio)

From a BigQuery table to a dashboard, and how BI tools affect BigQuery cost.

Machine Learning on GCP

The MLOps landscape: pre-built models, AutoML, and Vertex AI pipelines.

Part III Architecting top-notch pipelines

User and Project Management in GCP

IAM roles, how to structure projects, controlling data access, and Terraform.

Cost Strategy in GCP

Estimating an end-to-end solution and optimizing BigQuery with partitions and clusters.

CI/CD on Google Cloud for Data Engineers

Continuous integration and delivery with Cloud Build, including Composer deployments.

Boosting Your Confidence as a Data Engineer

The Professional Data Engineer exam, a review quiz, and where the field is heading.

Path

Suggested order

The book runs chapter 1 to 12, but you will learn faster if you meet identity and project structure (chapter 9) before you create anything in the cloud.

Guide

How to use this

Explore the concept in the interactive panels before reading anything long.
Decide the scenarios. Choosing first, then reading why, is what makes it stick.
Do the hands-on steps in your own Google Cloud project. Set a budget alert first, and delete what you create.
Quiz yourself last. Reset and retry later; spaced repetition beats one long session.

Based on Data Engineering with Google Cloud Platform by Adi Widjaja (Packt, 2022), rewritten in my own words. Some services have been renamed or replaced since, so each chapter flags what has changed. Always confirm names, prices and console steps in the current Google Cloud documentation.