Become a successful data engineer by building and deploying your own data pipelines on Google Cloud, including making key architectural decisionsKey FeaturesGet up to speed with data governance on Google CloudLearn how to use various Google Cloud products like Dataform, DLP, Dataplex, Dataproc Serverless, and DatastreamBoost your confidence by getting Google Cloud data engineering certification guidance from real exam experiencesPurchase of the print or Kindle book includes a free PDF eBookBook DescriptionThe second edition of Data Engineering with Google Cloud builds upon the success of the first edition by offering enhanced clarity and depth to data professionals navigating the intricate landscape of data engineering. Beyond its foundational lessons, this new edition delves into the essential realm of data governance within Google Cloud, providing you invaluable insights into managing and optimizing data resources effectively. Furthermore, this book helps you stay ahead of the curve by guiding you through the latest technological advancements in the Google Cloud ecosystem. You'll cover essential aspects, from exploring Cloud Composer 2 to the evolution of Airflow 2.5. Additionally, you'll explore how to work with cutting-edge tools like Dataform, DLP, Dataplex, Dataproc Serverless, and Datastream to perform data governance on datasets. By the end of this book, you'll be equipped to navigate the ever-evolving world of data engineering on Google Cloud, from foundational principles to cutting-edge practices.

What you will learnLoad data into BigQuery and materialize its outputFocus on data pipeline orchestration using Cloud ComposerFormulate Airflow jobs to orchestrate and automate a data warehouseEstablish a Hadoop data lake, generate ephemeral clusters, and execute jobs on the Dataproc clusterHarness Pub/Sub for messaging and ingestion for event-driven systemsApply Dataflow to conduct ETL on streaming dataImplement data governance services on Google CloudWho this book is forData analysts, IT practitioners, software engineers, or any data enthusiasts looking to have a successful data engineering career will find this book invaluable. Additionally, experienced data professionals who want to start using Google Cloud to build data platforms will get clear insights on how to navigate the path. Whether you're a beginner who wants to explore the fundamentals or a seasoned professional seeking to learn the latest data engineering concepts, this book is for you.

Table of ContentsFundamentals of Data engineering with GCPBig Data Capabilities on GCP Building a data warehouse in BigQueryBuild Orchestration for Batch Data Loading Using Cloud ComposerBuilding a Data Lake using DataprocProcess Streaming Data with Datastream, Pub/Sub and DataflowVisualizing Data for Making Data-Driven Decisions with Looker StudioBuild machine learning solutions on GCPUser and Project Management on GCPData Governance in GCPCost Strategy in GCPCI/CD on Google Cloud Platform for Data EngineersBoost your confidence as a Data Engineer