[Q30-Q49] Try 100% Updated Associate-Developer-Apache-Spark Exam Questions [2023]

Associate-Developer-Apache-Spark Databricks

admin -

May 26, 2023

4/5 - (2 votes)

Try 100% Updated Associate-Developer-Apache-Spark Exam Questions [2023]

Pass Associate-Developer-Apache-Spark Exam – Real Questions and Answers

Preparing for the Databricks Associate-Developer-Apache-Spark exam requires a solid understanding of Spark’s core concepts and APIs. Candidates should have experience working with Spark in a production environment and be familiar with Spark’s data processing capabilities. They should also have a good understanding of SQL and data analysis techniques.

The Databricks Certified Associate Developer for Apache Spark 3.0 exam assesses the individual’s knowledge of Spark architecture, Spark internals, Spark SQL, Spark Streaming, and Spark MLlib. The exam is composed of multiple-choice questions, and the passing score is 70%. The certification is valid for two years, after which the candidate will need to take the exam again to renew their certification. The certification is an excellent way for professionals to demonstrate their skills in Apache Spark and improve their career prospects in big data and analytics.

QUESTION 30
Which of the following code blocks can be used to save DataFrame transactionsDf to memory only, recalculating partitions that do not fit in memory when they are needed?

from pyspark import StorageLevel
transactionsDf.cache(StorageLevel.MEMORY_ONLY)

transactionsDf.cache()

transactionsDf.storage_level(‘MEMORY_ONLY’)

transactionsDf.persist()

transactionsDf.clear_persist()

from pyspark import StorageLevel
transactionsDf.persist(StorageLevel.MEMORY_ONLY)

QUESTION 31
Which of the following describes a narrow transformation?

narrow transformation is an operation in which data is exchanged across partitions.

A narrow transformation is a process in which data from multiple RDDs is used.

A narrow transformation is a process in which 32-bit float variables are cast to smaller float variables, like 16-bit or 8-bit float variables.

A narrow transformation is an operation in which data is exchanged across the cluster.

A narrow transformation is an operation in which no data is exchanged across the cluster.

QUESTION 32
Which of the following code blocks returns about 150 randomly selected rows from the 1000-row DataFrame transactionsDf, assuming that any row can appear more than once in the returned DataFrame?

transactionsDf.resample(0.15, False, 3142)

transactionsDf.sample(0.15, False, 3142)

transactionsDf.sample(0.15)

transactionsDf.sample(0.85, 8429)

transactionsDf.sample(True, 0.15, 8261)

QUESTION 33
Which of the following code blocks concatenates rows of DataFrames transactionsDf and transactionsNewDf, omitting any duplicates?

transactionsDf.concat(transactionsNewDf).unique()

transactionsDf.union(transactionsNewDf).distinct()

spark.union(transactionsDf, transactionsNewDf).distinct()

transactionsDf.join(transactionsNewDf, how=”union”).distinct()

transactionsDf.union(transactionsNewDf).unique()

QUESTION 34
The code block shown below should store DataFrame transactionsDf on two different executors, utilizing the executors’ memory as much as possible, but not writing anything to disk. Choose the answer that correctly fills the blanks in the code block to accomplish this.
1.from pyspark import StorageLevel
2.transactionsDf.__1__(StorageLevel.__2__).__3__

1. cache
2. MEMORY_ONLY_2
3. count()

1. persist
2. DISK_ONLY_2
3. count()

1. persist
2. MEMORY_ONLY_2
3. select()

1. cache
2. DISK_ONLY_2
3. count()

1. persist
2. MEMORY_ONLY_2
3. count()

QUESTION 35
Which of the following code blocks returns a copy of DataFrame transactionsDf where the column storeId has been converted to string type?

transactionsDf.withColumn(“storeId”, convert(“storeId”, “string”))

transactionsDf.withColumn(“storeId”, col(“storeId”, “string”))

transactionsDf.withColumn(“storeId”, col(“storeId”).convert(“string”))

transactionsDf.withColumn(“storeId”, col(“storeId”).cast(“string”))

transactionsDf.withColumn(“storeId”, convert(“storeId”).as(“string”))

QUESTION 36
Which of the following DataFrame operators is never classified as a wide transformation?

DataFrame.sort()

DataFrame.aggregate()

DataFrame.repartition()

DataFrame.select()

DataFrame.join()

Explanation
As a general rule: After having gone through the practice tests you probably have a good feeling for what classifies as a wide and what classifies as a narrow transformation. If you are unsure, feel free to play around in Spark and display the explanation of the Spark execution plan via DataFrame.[operation, for example sort()].explain(). If repartitioning is involved, it would count as a wide transformation.
DataFrame.select()
Correct! A wide transformation includes a shuffle, meaning that an input partition maps to one or more output partitions. This is expensive and causes traffic across the cluster. With the select() operation however, you pass commands to Spark that tell Spark to perform an operation on a specific slice of any partition. For this, Spark does not need to exchange data across partitions, each partition can be worked on independently. Thus, you do not cause a wide transformation.
DataFrame.repartition()
Incorrect. When you repartition a DataFrame, you redefine partition boundaries. Data will flow across your cluster and end up in different partitions after the repartitioning is completed. This is known as a shuffle and, in turn, is classified as a wide transformation.
DataFrame.aggregate()
No. When you aggregate, you may compare and summarize data across partitions. In the process, data are exchanged across the cluster, and newly formed output partitions depend on one or more input partitions. This is a typical characteristic of a shuffle, meaning that the aggregate operation may classify as a wide transformation.
DataFrame.join()
Wrong. Joining multiple DataFrames usually means that large amounts of data are exchanged across the cluster, as new partitions are formed. This is a shuffle and therefore DataFrame.join() counts as a wide transformation.
DataFrame.sort()
False. When sorting, Spark needs to compare many rows across all partitions to each other. This is an expensive operation, since data is exchanged across the cluster and new partitions are formed as data is reordered. This process classifies as a shuffle and, as a result, DataFrame.sort() counts as wide transformation.
More info: Understanding Apache Spark Shuffle | Philipp Brunenberg

QUESTION 37
Which of the following describes a way for resizing a DataFrame from 16 to 8 partitions in the most efficient way?

Use operation DataFrame.repartition(8) to shuffle the DataFrame and reduce the number of partitions.

Use operation DataFrame.coalesce(8) to fully shuffle the DataFrame and reduce the number of partitions.

Use a narrow transformation to reduce the number of partitions.

Use a wide transformation to reduce the number of partitions.
Use operation DataFrame.coalesce(0.5) to halve the number of partitions in the DataFrame.

QUESTION 38
The code block displayed below contains multiple errors. The code block should return a DataFrame that contains only columns transactionId, predError, value and storeId of DataFrame transactionsDf. Find the errors.
Code block:
transactionsDf.select([col(productId), col(f)])
Sample of transactionsDf:
1.+————-+———+—–+——-+———+—-+
2.|transactionId|predError|value|storeId|productId| f|
3.+————-+———+—–+——-+———+—-+
4.| 1| 3| 4| 25| 1|null|
5.| 2| 6| 7| 2| 2|null|
6.| 3| 3| null| 25| 3|null|
7.+————-+———+—–+——-+———+—-+

The column names should be listed directly as arguments to the operator and not as a list.

The select operator should be replaced by a drop operator, the column names should be listed directly as arguments to the operator and not as a list, and all column names should be expressed as strings without being wrapped in a col() operator.

The select operator should be replaced by a drop operator.

The column names should be listed directly as arguments to the operator and not as a list and following the pattern of how column names are expressed in the code block, columns productId and f should be replaced by transactionId, predError, value and storeId.

The select operator should be replaced by a drop operator, the column names should be listed directly as arguments to the operator and not as a list, and all col() operators should be removed.

Explanation
Correct code block: transactionsDf.drop(“productId”, “f”)
This question requires a lot of thinking to get right. For solving it, you may take advantage of the digital notepad that is provided to you during the test. You have probably seen that the code block includes multiple errors. In the test, you are usually confronted with a code block that only contains a single error. However, since you are practicing here, this challenging multi-error question will make it easier for you to deal with single-error questions in the real exam.
The select operator should be replaced by a drop operator, the column names should be listed directly as arguments to the operator and not as a list, and all column names should be expressed as strings without being wrapped in a col() operator.
Correct! Here, you need to figure out the many, many things that are wrong with the initial code block. While the question can be solved by using a select statement, a drop statement, given the answer options, is the correct one. Then, you can read in the documentation that drop does not take a list as an argument, but just the column names that should be dropped. Finally, the column names should be expressed as strings and not as Python variable names as in the original code block.
The column names should be listed directly as arguments to the operator and not as a list.
Incorrect. While this is a good first step and part of the correct solution (see above), this modification is insufficient to solve the question.
The column names should be listed directly as arguments to the operator and not as a list and following the pattern of how column names are expressed in the code block, columns productId and f should be replaced by transactionId, predError, value and storeId.
Wrong. If you use the same pattern as in the original code block (col(productId), col(f)), you are still making a mistake. col(productId) will trigger Python to search for the content of a variable named productId instead of telling Spark to use the column productId – for that, you need to express it as a string.
The select operator should be replaced by a drop operator, the column names should be listed directly as arguments to the operator and not as a list, and all col() operators should be removed.
No. This still leaves you with Python trying to interpret the column names as Python variables (see above).
The select operator should be replaced by a drop operator.
Wrong, this is not enough to solve the question. If you do this, you will still face problems since you are passing a Python list to drop and the column names are still interpreted as Python variables (see above).
More info: pyspark.sql.DataFrame.drop – PySpark 3.1.2 documentation
Static notebook | Dynamic notebook: See test 3

QUESTION 39
Which of the following describes the characteristics of accumulators?

Accumulators are used to pass around lookup tables across the cluster.

All accumulators used in a Spark application are listed in the Spark UI.

Accumulators can be instantiated directly via the accumulator(n) method of the pyspark.RDD module.

Accumulators are immutable.

If an action including an accumulator fails during execution and Spark manages to restart the action and complete it successfully, only the successful attempt will be counted in the accumulator.

QUESTION 40
The code block shown below should add a column itemNameBetweenSeparators to DataFrame itemsDf. The column should contain arrays of maximum 4 strings. The arrays should be composed of the values in column itemsDf which are separated at – or whitespace characters. Choose the answer that correctly fills the blanks in the code block to accomplish this.
Sample of DataFrame itemsDf:
1.+——+———————————-+——————-+
2.|itemId|itemName |supplier |
3.+——+———————————-+——————-+
4.|1 |Thick Coat for Walking in the Snow|Sports Company Inc.|
5.|2 |Elegant Outdoors Summer Dress |YetiX |
6.|3 |Outdoors Backpack |Sports Company Inc.|
7.+——+———————————-+——————-+
Code block:
itemsDf.__1__(__2__, __3__(__4__, “[s-]”, __5__))

1. withColumn
2. “itemNameBetweenSeparators”
3. split
4. “itemName”
5. 4
(Correct)

1. withColumnRenamed
2. “itemNameBetweenSeparators”
3. split
4. “itemName”
5. 4

1. withColumnRenamed
2. “itemName”
3. split
4. “itemNameBetweenSeparators”
5. 4

1. withColumn
2. “itemNameBetweenSeparators”
3. split
4. “itemName”
5. 5

1. withColumn
2. itemNameBetweenSeparators
3. str_split
4. “itemName”
5. 5

QUESTION 41
Which of the elements in the labeled panels represent the operation performed for broadcast variables?
Larger image

2, 5

2, 3

1, 2

1, 3, 4

QUESTION 42
Which of the following statements about Spark’s DataFrames is incorrect?

Spark’s DataFrames are immutable.

Spark’s DataFrames are equal to Python’s DataFrames.

Data in DataFrames is organized into named columns.

RDDs are at the core of DataFrames.

The data in DataFrames may be split into multiple chunks.

QUESTION 43
The code block displayed below contains an error. The code block should return a new DataFrame that only contains rows from DataFrame transactionsDf in which the value in column predError is at least 5. Find the error.
Code block:
transactionsDf.where(“col(predError) >= 5”)

The argument to the where method should be “predError >= 5”.

Instead of where(), filter() should be used.

The expression returns the original DataFrame transactionsDf and not a new DataFrame. To avoid this, the code block should be transactionsDf.toNewDataFrame().where(“col(predError) >= 5”).

The argument to the where method cannot be a string.

Instead of >=, the SQL operator GEQ should be used.

QUESTION 44
Which of the following code blocks reads in the two-partition parquet file stored at filePath, making sure all columns are included exactly once even though each partition has a different schema?
Schema of first partition:
1.root
2. |– transactionId: integer (nullable = true)
3. |– predError: integer (nullable = true)
4. |– value: integer (nullable = true)
5. |– storeId: integer (nullable = true)
6. |– productId: integer (nullable = true)
7. |– f: integer (nullable = true)
Schema of second partition:
1.root
2. |– transactionId: integer (nullable = true)
3. |– predError: integer (nullable = true)
4. |– value: integer (nullable = true)
5. |– storeId: integer (nullable = true)
6. |– rollId: integer (nullable = true)
7. |– f: integer (nullable = true)
8. |– tax_id: integer (nullable = false)

spark.read.parquet(filePath, mergeSchema=’y’)

spark.read.option(“mergeSchema”, “true”).parquet(filePath)

spark.read.parquet(filePath)

1.nx = 0
2.for file in dbutils.fs.ls(filePath):
3. if not file.name.endswith(“.parquet”):
4. continue
5. df_temp = spark.read.parquet(file.path)
6. if nx == 0:
7. df = df_temp
8. else:
9. df = df.union(df_temp)
10. nx = nx+1
11.df

QUESTION 45
The code block displayed below contains an error. The code block should return the average of rows in column value grouped by unique storeId. Find the error.
Code block:
transactionsDf.agg(“storeId”).avg(“value”)

Instead of avg(“value”), avg(col(“value”)) should be used.

The avg(“value”) should be specified as a second argument to agg() instead of being appended to it.

All column names should be wrapped in col() operators.

agg should be replaced by groupBy.

“storeId” and “value” should be swapped.

1. filter
2. col(“supplier”).isin(“Sports”)
3. “itemName”
4. explode(col(“attributes”))

1. where
2. col(“supplier”).contains(“Sports”)
3. “itemName”
4. “attributes”

1. where
2. col(supplier).contains(“Sports”)
3. explode(attributes)
4. itemName

1. where
2. “Sports”.isin(col(“Supplier”))
3. “itemName”
4. array_explode(“attributes”)

1. filter
2. col(“supplier”).contains(“Sports”)
3. “itemName”
4. explode(“attributes”)

Explanation
Output of correct code block:
+———————————-+——+
|itemName |col |
+———————————-+——+
|Thick Coat for Walking in the Snow|blue |
|Thick Coat for Walking in the Snow|winter|
|Thick Coat for Walking in the Snow|cozy |
|Outdoors Backpack |green |
|Outdoors Backpack |summer|
|Outdoors Backpack |travel|
+———————————-+——+
The key to solving this question is knowing about Spark’s explode operator. Using this operator, you can extract values from arrays into single rows. The following guidance steps through the answers systematically from the first to the last gap. Note that there are many ways to solving the gap questions and filtering out wrong answers, you do not always have to start filtering out from the first gap, but can also exclude some answers based on obvious problems you see with them.
The answers to the first gap present you with two options: filter and where. These two are actually synonyms in PySpark, so using either of those is fine. The answer options to this gap therefore do not help us in selecting the right answer.
The second gap is more interesting. One answer option includes “Sports”.isin(col(“Supplier”)). This construct does not work, since Python’s string does not have an isin method. Another option contains col(supplier). Here, Python will try to interpret supplier as a variable. We have not set this variable, so this is not a viable answer. Then, you are left with answers options that include col (“supplier”).contains(“Sports”) and col(“supplier”).isin(“Sports”). The question states that we are looking for suppliers whose name includes Sports, so we have to go for the contains operator here.
We would use the isin operator if we wanted to filter out for supplier names that match any entries in a list of supplier names.
Finally, we are left with two answers that fill the third gap both with “itemName” and the fourth gap either with explode(“attributes”) or “attributes”. While both are correct Spark syntax, only explode (“attributes”) will help us achieve our goal. Specifically, the question asks for one attribute from column attributes per row – this is what the explode() operator does.
One answer option also includes array_explode() which is not a valid operator in PySpark.
More info: pyspark.sql.functions.explode – PySpark 3.1.2 documentation
Static notebook | Dynamic notebook: See test 3

QUESTION 47
The code block displayed below contains at least one error. The code block should return a DataFrame with only one column, result. That column should include all values in column value from DataFrame transactionsDf raised to the power of 5, and a null value for rows in which there is no value in column value. Find the error(s).
Code block:
1.from pyspark.sql.functions import udf
2.from pyspark.sql import types as T
3.
4.transactionsDf.createOrReplaceTempView(‘transactions’)
5.
6.def pow_5(x):
7. return x**5
8.
9.spark.udf.register(pow_5, ‘power_5_udf’, T.LongType())
10.spark.sql(‘SELECT power_5_udf(value) FROM transactions’)

The pow_5 method is unable to handle empty values in column value and the name of the column in the returned DataFrame is not result.

The returned DataFrame includes multiple columns instead of just one column.

The pow_5 method is unable to handle empty values in column value, the name of the column in the returned DataFrame is not result, and the SparkSession cannot access the transactionsDf DataFrame.

The pow_5 method is unable to handle empty values in column value, the name of the column in the returned DataFrame is not result, and Spark driver does not call the UDF function appropriately.

The pow_5 method is unable to handle empty values in column value, the UDF function is not registered properly with the Spark driver, and the name of the column in the returned DataFrame is not result.

QUESTION 48
Which of the following statements about lazy evaluation is incorrect?

Predicate pushdown is a feature resulting from lazy evaluation.

Execution is triggered by transformations.

Spark will fail a job only during execution, but not during definition.

Accumulators do not change the lazy evaluation model of Spark.

Lineages allow Spark to coalesce transformations into stages

QUESTION 49
Which of the following describes a difference between Spark’s cluster and client execution modes?

In cluster mode, the cluster manager resides on a worker node, while it resides on an edge node in client mode.

In cluster mode, executor processes run on worker nodes, while they run on gateway nodes in client mode.

In cluster mode, the driver resides on a worker node, while it resides on an edge node in client mode.

In cluster mode, a gateway machine hosts the driver, while it is co-located with the executor in client mode.

In cluster mode, the Spark driver is not co-located with the cluster manager, while it is co-located in client mode.

The Databricks Associate-Developer-Apache-Spark exam is an excellent opportunity for developers to validate their skills in working with Apache Spark. It is a globally recognized certification that demonstrates the ability to work with Spark’s core APIs and perform data analysis tasks using Spark SQL and DataFrames. With the increasing demand for data analytics professionals, this certification can help developers stand out in the job market and enhance their career prospects.

Associate-Developer-Apache-Spark Exam Questions Get Updated [2023] with Correct Answers: https://www.dumpstests.com/Associate-Developer-Apache-Spark-latest-test-dumps.html

#Associate-Developer-Apache-Spark certification dump Associate-Developer-Apache-Spark latest exam dumps file Associate-Developer-Apache-Spark reliable study plan Associate-Developer-Apache-Spark test king Associate-Developer-Apache-Spark valid exam duration Associate-Developer-Apache-Spark valid study guide ebook Associate-Developer-Apache-Spark valid test answers new Associate-Developer-Apache-Spark exam voucher

[Q30-Q49] Try 100% Updated Associate-Developer-Apache-Spark Exam Questions [2023]

No Responses

Leave a Reply Cancel reply

Related Certifications

Associate-Developer-Apache-Spark Practice Tests

Recent Posts

Recent Comments

Categories