최신Cloudera CDP Data Engineer - Certification - CDP-3002무료샘플문제

A. Spark History Server

B. Apache Ambari

C. Hadoop YARN ResourceManager UI

D. spark Web UI

A. To visualize task execution and logs

B. To retry failed tasks automatically

C. To connect to external systems and databases

D. To define the execution order of tasks

A. Leverage Spark SQL's built-in JSON support with appropriate schema definition

B. Implement a custom parser for the specific JSON structure

C. Use generic string manipulation functions to extract data from JSON

D. Convert the entire dataset to a single string and process it line by line

A. coalesce reduces the number of partitions without a full data shuffle, enhancing performance.

B. coalesce triggers a full shuffle of the data, improving data distribution.

C. coalesce can increase the number of partitions without shuffling.

D. repartition is incapable of reducing the number of partitions.

A. Decreasing the number of partitions

B. Filtering data after a wide transformation

C. Using broadcast variables for small data

D. Increasing the number of partitions

A. Leverage XCom to share data between tasks and define dependencies based on the data availability.

B. All of the above

C. Use the Airflow UI to visually connect the tasks with arrows, indicating their execution order.

D. Define the tasks directly within the DAG code using Python operators and set their dependencies manually.

A. Use rdd.persist() with the appropriate storage level based on your needs

B. Implement custom logic to save the data to HDFS between stages

C. Leverage Spark's automatic checkpointing mechanism

D. Store the data in a temporary table using Spark SQL

A. Optimistic locking in Iceberg introduces a significant performance overhead.

B. Iceberg's optimistic locking requires manual conflict resolution.

C. Optimistic locking might result in many failed transactions and retries, potentially leading to job instability.

D. Iceberg's optimistic locking isn't suitable for distributed environments like CDP.

A. Use HiveQL's ALTER TABLE statement to modify the Hive table schema

B. Modify the Spark DataFrame schema to match the Hive table schema manually

C. Leverage Spark SQL's schema inference capabilities to automatically adjust the DataFrame schema

D. Ignore the schema mismatch and write the data anyway, potentially leading to data corruption

A. Data Replication

B. Master Node Redundancy

C. Task Re-execution

D. Checkpointing

A. Convert the "purchase_date" column to a string and filter based on the year

B. Loop through the DataFrame and group purchases by month, calculating the average manually

C. Use Spark SQL's MONTH function and AVG function with appropriate windowing

D. Utilize Spark's machine learning library (MLIiB. for time series analysis

A. DataFrames require less memory and compute resources compared to RDDs.

B. DataFrames automatically optimize queries using the Catalyst optimizer and Tungsten execution engine.

C. RDDs are deprecated and will be removed in future versions of Spark.

D. DataFrames provide more fine-grained control over partitioning and parallelism.

A. By organizing data into fixed-size segments, or buckets, based on a hash function of a column

B. By encrypting data to reduce 1/0 operations

C. By replicating data across multiple nodes to improve access speed

D. By compressing data automatically to reduce storage space