Data Engineering Fundamentals on AWS

Please provide your name *

1. Which of the following is the Data Engineer’s responsibility rather than the Data Scientist’s? *

Choosing the evaluation metric for a credit risk model Delivering clean, documented tables that refresh reliably against an agreed SLA Tuning the hyperparameters of a Random Forest Interpreting feature importance for the business

2. In a medallion architecture, what does the Silver layer contain? *

Raw data exactly as it landed from the source system Cleaned, validated and conformed data ready to be modelled or aggregated Business-level aggregates built specifically for reporting and dashboards Archived data that has been transitioned to S3 Glacier for retention

3. Why is Parquet generally preferred over CSV for data lake tables that are queried with Athena? *

It is human-readable, which makes debugging pipelines easier It enforces referential integrity between related tables It stores data column by column and compresses well, so queries scan far less data It removes the need to register the table in the Glue Data Catalog

4. What is the primary purpose of an AWS Glue Crawler? *

To scan data in S3 and create or update the table definitions in the Glue Data Catalog To copy files between S3 prefixes on a schedule To run PySpark transformations across a distributed cluster To enforce column-level access permissions on catalogued tables

5. What do Glue job bookmarks allow a job to do? *

Roll a Glue job back to a previously deployed version Process only the new data that has arrived since the last successful run Schedule a Glue job without needing EventBridge Reprocess the entire source dataset on every run to guarantee correctness

6. What is DQDL used for in AWS Glue? *

Defining the schema and partition keys of a Glue Data Catalog table Writing the PySpark transformation logic inside a Glue ETL job Describing the state machine that orchestrates a pipeline Defining data quality rules — such as completeness, uniqueness and value ranges — that are evaluated against a dataset

7. In an AWS Step Functions state machine, which state type is used to branch the workflow based on a condition, for example whether a data quality check passed? *

Parallel Map Choice Wait

8. Which statement best describes when to use Athena rather than Redshift Serverless? *

Athena is better suited to ad-hoc and infrequent queries run directly over S3, billed per query Athena is better suited to high-concurrency BI workloads with complex joins Athena requires the data to be loaded into a cluster before it can be queried Athena is the only one of the two that can read Parquet files

9. What is Amazon Kinesis Data Firehose primarily used for? *

Running Spark Streaming transformations with checkpointing Delivering streaming records into destinations such as S3 or Redshift, with buffering, compression and format conversion, and no servers to manage Storing streaming records indefinitely so they can be replayed at any time Orchestrating multi-step batch pipelines across AWS services

10. What does AWS Lake Formation give you that plain S3 bucket policies and IAM alone do not? *

Automatic conversion of CSV files into Parquet as they land Automatic scaling of the number of workers on a Glue job Cheaper storage through automatic tiering of infrequently accessed objects Fine-grained column-level and row-level access control over catalogued data