Databricks Certified Data Engineer Associate Exam Prep

What's inside

1. Level 1: The Exam, the Guide and How This Course WorksFree
   [Exam craft] Learn how the Databricks Certified Data Engineer Associate exam works. It has 45 scored multiple-choice questions in 90 minutes, costs $200, allows no test aids and may include unscored items. Data manipulation code is shown in SQL where possible and in Python otherwise. The seven section weightings are 6/21/22/16/10/10/15. The official exam guide belongs to Databricks, is not reproduced here and is free on the certification page. This course is independent of Databricks and does not promise a pass.
2. Level 2: The Lakehouse and the Data Intelligence PlatformFree
   [Databricks Intelligence Platform] Learn what the lakehouse is and why it replaces separate data lakes and warehouses. See how the Databricks Data Intelligence Platform is built: a control plane, a compute plane, and cloud object storage. Delta Lake and Unity Catalog sit at its core.
3. Level 3: Navigating the Databricks Workspace
   [Databricks Intelligence Platform] Tour the workspace features the exam names: notebooks with mixed SQL and Python cells and magic commands, the SQL editor, Catalog Explorer, Jobs & Pipelines, and the catalog.schema.table namespace.
4. Level 4: Compute Types, Serverless and Cost Models
   [Databricks Intelligence Platform] Compare the compute options: all-purpose (classic) clusters, jobs compute, SQL warehouses and serverless compute. Cover access modes, autoscaling, auto-termination and how DBU-based cost changes the best choice.
5. Level 5: Reading Files: Formats, Volumes and read_files
   [Data Ingestion and Loading] Load CSV, JSON, Parquet and text files from Unity Catalog volumes with spark.read in PySpark and read_files in Spark SQL. Learn why self-describing formats like Parquet behave differently from CSV.
6. Level 6: Semi-Structured and Nested JSON
   [Data Ingestion and Loading] Ingest JSON and nested data and query it. Covers colon and dot field access, from_json with a schema, STRUCT and ARRAY types, explode, and loading the result into Unity Catalog-governed Delta tables.
7. Level 7: Idempotent Batch Loading with COPY INTO
   [Data Ingestion and Loading] Use COPY INTO to load files from cloud storage or volumes into Delta tables idempotently. Cover FILEFORMAT, FORMAT_OPTIONS, COPY_OPTIONS ('mergeSchema'), and how files that were already loaded get skipped.
8. Level 8: Auto Loader: Incremental File Ingestion
   [Data Ingestion and Loading] Ingest new files incrementally with Auto Loader (format 'cloudFiles'). Covers checkpoints for exactly-once processing, trigger(availableNow=True) for batch-style runs, and directory listing versus file notification mode.
9. Level 9: Auto Loader Schema Inference, Evolution, Rescued Data
   [Data Ingestion and Loading] Control how Auto Loader handles schemas with cloudFiles.schemaLocation, schema hints, and cloudFiles.schemaEvolutionMode (addNewColumns, rescue, failOnNewColumns, none). Learn how the _rescued_data column catches values that don't match the schema.
10. Level 10: Schema Enforcement and Evolution on Delta Writes
    [Data Ingestion and Loading] Understand how Delta Lake's schema enforcement rejects writes that don't match the table. Learn when to allow schema evolution with mergeSchema, overwriteSchema or automatic schema evolution, and the risk of letting bad columns in.
11. Level 11: Streaming Tables and Batch vs Streaming Ingestion
    [Data Ingestion and Loading] Create streaming tables in Spark SQL with CREATE STREAMING TABLE ... AS SELECT * FROM STREAM read_files(...). Compare incremental streaming ingestion with full batch reloads in cost, latency and complexity.
12. Level 12: Lakeflow Connect: Standard and Managed Connectors
    [Data Ingestion and Loading] Learn Lakeflow Connect's two connector families. Managed connectors ingest from SaaS applications and databases such as Salesforce and SQL Server into Unity Catalog-governed Delta tables. Standard connectors cover file and streaming sources such as cloud storage and message buses.
13. Level 13: JDBC/ODBC Sources and Partner Connectors
    [Data Ingestion and Loading] Read from relational databases over JDBC with spark.read.format('jdbc') and parallel partitioning options. See how ODBC/JDBC drivers and partner connectors fit in next to native ingestion.
14. Level 14: Choosing the Right Ingestion Method
    [Data Ingestion and Loading] Bring the section together in exam-style scenarios. Choose between Auto Loader, COPY INTO, Lakeflow Connect (standard and managed connectors), partner connectors and JDBC based on data volume, ingestion frequency, data types (including unstructured files) and Unity Catalog governance needs.
15. Level 15: Delta Lake Tables, History and Time Travel
    [Data Transformation and Modeling] Work with Delta Lake tables as the foundation for transformations. Covers the transaction log, CREATE TABLE AS SELECT versus INSERT OVERWRITE, DESCRIBE HISTORY and DESCRIBE DETAIL, time travel with VERSION AS OF and TIMESTAMP AS OF, and RESTORE.
16. Level 16: The Medallion Architecture: Bronze, Silver, Gold
    [Data Transformation and Modeling] Design bronze/silver/gold medallion layers: raw data landed as is, then cleaned and conformed, then business-level aggregates. Decide which transformations belong in each layer and why.
17. Level 17: Spark SQL and DataFrame Transformations
    [Data Transformation and Modeling] Clean and reshape data with matching Spark SQL and PySpark DataFrame code: select, withColumn, filter/where, cast, CASE WHEN/when, null handling and date functions. Learn lazy evaluation and the difference between transformations and actions.
18. Level 18: Joins: Types, Pitfalls and Broadcast
    [Data Transformation and Modeling] Join datasets in SQL and PySpark with inner, left, full, cross, left semi and left anti joins. Avoid duplicate-key fan-out and ambiguous column names, and use broadcast joins when one side is small.
19. Level 19: Aggregations, Window Functions and Deduplication
    [Data Transformation and Modeling] Summarise and deduplicate data with GROUP BY/groupBy().agg() and window functions such as ROW_NUMBER, RANK and LAG over PARTITION BY. Covers dropDuplicates, DISTINCT, and keeping the latest record per key.
20. Level 20: Higher-Order Functions on Arrays and Maps
    [Data Transformation and Modeling] Transform nested arrays without exploding them, using Spark SQL higher-order functions: transform, filter, exists, aggregate (reduce) and lambda syntax. Compare these with explode-then-regroup and with Python UDFs.
21. Level 21: MERGE INTO: Upserts into Delta Tables
    [Data Transformation and Modeling] Learn how MERGE INTO updates, inserts and deletes rows in one atomic Delta operation, including WHEN MATCHED, WHEN NOT MATCHED and WHEN NOT MATCHED BY SOURCE clauses, why it fails when several source rows match one target row, and how to remove duplicates from the source first.
22. Level 22: Change Data Capture: Change Data Feed and AUTO CDC
    [Data Transformation and Modeling] Learn change data capture on Databricks: turning on Delta Change Data Feed and reading it with table_changes(), and applying ordered insert, update and delete events with AUTO CDC (formerly APPLY CHANGES INTO) as SCD Type 1 or Type 2.
23. Level 23: Gold Layer: Materialized Views vs Streaming Tables
    [Data Transformation and Modeling] Learn which object to build in the Gold layer: a materialized view that recomputes or incrementally refreshes aggregates, or a streaming table that processes each append-only input record once, and what each costs in freshness, compute and flexibility.
24. Level 24: Data Quality Checks in Transformations
    [Data Transformation and Modeling] Learn to enforce and test data quality while transforming data: NOT NULL and CHECK constraints on Delta tables, filtering bad rows into a quarantine table, and validation queries that count nulls, duplicates and out-of-range values before data is promoted from Silver to Gold.
25. Level 25: Liquid Clustering and Table Tuning
    [Data Transformation and Modeling] Learn how liquid clustering (CLUSTER BY, including CLUSTER BY AUTO) replaces Hive-style partitioning and ZORDER for organising data layout, how to change clustering keys without rewriting the table, and the common tuning settings the exam expects you to recognise, such as shuffle partitions.
26. Level 26: Lakeflow Jobs: Jobs, Tasks and Compute
    [Working with Lakeflow Jobs] Learn the anatomy of a Lakeflow Job (formerly Databricks Workflows): a job contains tasks, each task has a type (notebook, Python script, SQL, pipeline, dbt, JAR and others) and runs on serverless jobs compute or job clusters, and the cost and isolation reasons for not running production jobs on all-purpose clusters.
27. Level 27: Task Dependencies and Run-If Conditions
    [Working with Lakeflow Jobs] Learn how tasks form a directed acyclic graph (DAG) through depends_on, how fan-out and fan-in run tasks in parallel, and how Run if conditions (All succeeded, At least one succeeded, None failed, All done, At least one failed, All failed) control what runs after an upstream failure.
28. Level 28: Control Flow: If/else, For Each and Run Job
    [Working with Lakeflow Jobs] Learn the control-flow task types in Lakeflow Jobs: If/else condition tasks that branch on a value, For each tasks that loop a nested task over a list with set concurrency, and Run Job tasks that call another job, so you can tell which one a scenario needs.
29. Level 29: Job Parameters, Task Values and Dynamic References
    [Working with Lakeflow Jobs] Learn how to pass data into and between tasks: job-level parameters that override task parameters, dbutils.jobs.taskValues to share small values downstream, and dynamic value references such as {{job.run_id}} and {{tasks.<name>.values.<key>}}.
30. Level 30: Triggers and Scheduling
    [Working with Lakeflow Jobs] Learn how jobs are started: scheduled triggers with Quartz cron syntax and time zones, file arrival triggers on Unity Catalog volumes or external locations, table update triggers, continuous mode, and manual or API runs, and how to match the trigger to the data's arrival pattern.
31. Level 31: Retries, Timeouts, Notifications and Repair Runs
    [Working with Lakeflow Jobs] Learn how to make jobs resilient and observable: task retry policies, timeouts and duration-threshold warnings, email and webhook notifications for start, success and failure, and repair runs that re-run only failed and skipped tasks rather than the whole job.
32. Level 32: Lakeflow Spark Declarative Pipelines
    [Working with Lakeflow Jobs] Learn Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables): declaring streaming tables, materialized views and flows in SQL or Python (the pyspark.pipelines module, formerly import dlt), how the pipeline works out the dependency graph, triggered versus continuous mode, development versus production mode, and running a pipeline as a job task.
33. Level 33: Pipeline Expectations and Event Log
    [Working with Lakeflow Jobs] Learn to enforce data quality inside declarative pipelines with expectations: CONSTRAINT … EXPECT with the default warn behaviour, ON VIOLATION DROP ROW and ON VIOLATION FAIL UPDATE, the Python @dp.expect family, and how to read violation metrics in the pipeline event log.
34. Level 34: Git Folders: Version Control in the Workspace
    [Implementing CI/CD] Learn Databricks Git folders (formerly Repos): connecting a Git provider, cloning a repository, creating branches, committing, pushing and pulling from the workspace, resolving conflicts, and why development happens in personal Git folders while production deploys from the remote repository.
35. Level 35: Declarative Automation Bundles: Anatomy
    [Implementing CI/CD] Learn the structure of Declarative Automation Bundles (renamed from Databricks Asset Bundles in March 2026; both names and the DAB abbreviation are still in use): databricks.yml, the bundle and include sections, and resources that declare jobs, pipelines and other workspace objects as source-controlled YAML next to the code.
36. Level 36: Bundle Targets and Variables
    [Implementing CI/CD] Learn how bundle targets model environments such as dev, staging and prod, with per-target workspace hosts, mode: development versus mode: production behaviour, custom variables with defaults and per-target overrides, and substitutions such as ${bundle.target} and ${var.catalog}.
37. Level 37: Deploying with the Databricks CLI
    [Implementing CI/CD] Learn the Databricks CLI bundle lifecycle: authenticating, databricks bundle init from templates, validate, deploy -t <target>, run and destroy, and what each command creates, changes or removes in the workspace.
38. Level 38: Promoting Code from Dev to Prod
    [Implementing CI/CD] Learn an end-to-end CI/CD flow: pull request, automated tests, and bundle deployment to staging and prod from a CI system such as GitHub Actions, running as a service principal with run_as, so that production never depends on a person's identity or workspace copy.
39. Level 39: Monitoring Jobs: Run History and Failures
    [Troubleshooting, Monitoring, and Optimization] Learn to monitor and debug Lakeflow Jobs: reading the job runs list and matrix view, opening a failed task run to find its error, output and logs, spotting slow or flaky tasks from run durations, and querying job run history from system tables.
40. Level 40: Reading the Spark UI: Skew, Shuffle and Spill
    [Troubleshooting, Monitoring, and Optimization] Learn to read the Spark UI's Jobs, Stages, Tasks, SQL/DataFrame and Executors tabs, and to diagnose the three most tested problems: data skew (one task much longer than the median), heavy shuffle, and spill to disk, along with fixes such as AQE skew-join handling, broadcast joins and right-sized compute.
41. Level 41: Data Skipping, OPTIMIZE and Predictive Optimization
    [Troubleshooting, Monitoring, and Optimization] Learn how Delta file statistics let a query skip files, how the small-files problem breaks that, and how OPTIMIZE, liquid clustering maintenance and predictive optimization fix it, and how to spot each case in an exam scenario.
42. Level 42: VACUUM, Retention and Caching
    [Troubleshooting, Monitoring, and Optimization] Learn how VACUUM removes unreferenced data files, how its retention threshold limits time travel, and when the disk cache helps with repeated reads of Delta data and when it doesn't.
43. Level 43: Diagnosing Failed Jobs and Clusters
    [Troubleshooting, Monitoring, and Optimization] Use job run history, task output, driver logs and cluster event logs to find out why a Lakeflow Jobs run or a compute resource failed, then choose the right fix.
44. Level 44: The Unity Catalog Object Hierarchy
    [Governance and Security] Learn the Unity Catalog metastore and the three-level catalog.schema.object namespace, plus the securable objects under it: tables, views, volumes, functions and models.
45. Level 45: Privileges: GRANT, REVOKE and Ownership
    [Governance and Security] Learn how Unity Catalog privileges such as USE CATALOG, USE SCHEMA, SELECT and MODIFY are granted, revoked and inherited down the hierarchy, and what an object's owner can do.
46. Level 46: Managed vs External Tables, Volumes and Locations
    [Governance and Security] Compare managed and external tables and volumes, and learn how storage credentials and external locations govern access to cloud storage paths.
47. Level 47: Row Filters, Column Masks and Dynamic Views
    [Governance and Security] Learn fine-grained access control: row filters and column masks built from SQL UDFs, and dynamic views that use functions such as is_account_group_member, to limit what each user sees in one table.
48. Level 48: Attribute-Based Access Control with Tags
    [Governance and Security] Learn how Unity Catalog ABAC policies use governed tags on catalogs, schemas, tables and columns to apply row filters and column masks at scale, rather than attaching them table by table.
49. Level 49: Lineage, Audit Logs and System Tables
    [Governance and Security] Use Unity Catalog's automatic table and column lineage in Catalog Explorer, and query the audit and other system tables, to find out where data came from, who touched it and what a change will break downstream.
50. Level 50: Delta Sharing: Shares, Recipients and Providers
    [Governance and Security] Learn how Delta Sharing shares live tables securely without copying them, comparing Databricks-to-Databricks sharing with open sharing to non-Databricks recipients.
51. Level 51: Timed Mock Exam Timed mock
    [Exam craft] A timed mock exam of 45 original multiple-choice scenario and code-reading questions in 90 minutes, drawn from all seven exam sections in proportion to their weightings, with a 70% pass mark and every distractor explained.

Access

The first 2 levels are free with a free account. Every level, the podcast edition and the AI tutor come with All Access at £4.99/month or any Creator plan — see pricing.

Start free →