Google GCP-DE : Data Engineer

You are developing a software application using Google's Dataflow SDK, and want to use conditional, for loops and other complex programming structures to create a branching pipeline. Which component will be used for the data processing operation?

• A. Pipeline
• B. Transform
• C. PCollection
• D. Sink API

As your organization expands its usage of GCP, many teams have started to create their own projects. Projects are further multiplied to accommodate different stages of deployments and target audiences. Each project requires unique access control configurations. The central IT team needs to have access to all projects. Furthermore, data from Cloud Storage buckets and BigQuery datasets must be shared for use in other projects in an ad hoc way. You want to simplify access control management by minimizing the number of policies. Which two steps should you take? Choose 2 answers.

• A. Create distinct groups for various teams, and specify groups in Cloud IAM policies.
• B. Find all the active members who have access to these projects, and create a Cloud IAM policy to grant access to all these users.
• C. Use Cloud Deployment Manager to automate access provision.
• D. For each Cloud Storage bucket or BigQuery dataset, decide which projects need acces
• E. Only use service accounts when sharing data for Cloud Storage buckets and BigQuery datasets.
• F. Introduce resource hierarchy to leverage access control policy inheritance.

Which is not a valid reason for poor Cloud Bigtable performance?

• A. The table's schema is not designed correctly.
• B. The Cloud Bigtable cluster has too many nodes.
• C. There are issues with the network connection.
• D. The workload isn't appropriate for Cloud Bigtable.

You operate an IoT pipeline built around Apache Kafka that normally receives around 5000 messages per second. You want to use Google Cloud Platform to create an alert as soon as the moving average over 1 hour drops below 4000 messages per second. What should you do?

• A. If that number falls below 4000, send an alert.
• B. Use a Cloud Dataflow template to write your messages from Cloud Pub/Sub to BigQuer
• C. Use Cloud Scheduler to run a script every five minutes that counts the number of rows created in BigQuery in the last hour
• D. Use Kafka Connect to link your Kafka message queue to Cloud Pub/Su
• E. Set a sliding time window of 1 hour every 5 minute
• F. Use Kafka Connect to link your Kafka message queue to Cloud Pub/Su
• G. If that number falls below 4000, send an alert.
• H. Use Cloud Scheduler to run a script every hour that counts the number of rows created in Cloud Bigtable in the last hour
• I. Use a Cloud Dataflow template to write your messages from Cloud Pub/Sub to Cloud Bigtabl
• J. Set a fixed time window of 1 hour.Compute the average when the window closes, and send an alert if the average is less than 4000 messages.
• K. Consume the stream of data in Cloud Dataflow using Kafka I
• L. Compute the average when the window closes, and send an alert if the average is less than 4000 messages.
• M. Consume the stream of data in Cloud Dataflow using Kafka I

You are building a new application that you need to collect data from in a scalable way. Data arrives continuously from the application throughout the day, and you expect to generate approximately 150 GB of JSON data per day by the end of the year. Your requirements are: Decoupling producer from consumer Space and cost-efficient storage of the raw ingested data, which is to be stored indefinitely Near real-time SQL query Maintain at least 2 years of historical data, which will be queried with SQ Which pipeline should you use to meet these requirements?

• A. Create an application that publishes events to Cloud Pub/Sub, and create Spark jobs on Cloud Dataproc to convert the JSON data to Avro format, stored on HDFS on Persistent Disk.
• B. Create an application that publishes events to Cloud Pub/Sub, and create a Cloud Dataflow pipeline that transforms the JSON event payloads to Avro, writing the data to Cloud Storage and BigQuery.
• C. Set up periodic exports of the database to write to Cloud Storage and load into BigQuery.
• D. Create an application that provides an AP
• E. Write a tool to poll the API and write data to Cloud Storage as gzipped JSON files.
• F. Create an application that writes to a Cloud SQL database to store the dat