CCA-500 Sample Questions Answers

For a 1GB child heap size an io.sort.mb of 128 MB will always maximize memory to disk I/O

Increase the io.sort.mb to 1GB

Decrease the io.sort.mb value to 0

Tune the io.sort.mb value until you observe that the number of spilled records equals (or is as close to equals) the number of map output records.

You can specify new queue name when user submits a job and new queue can be created dynamically if the property yarn.scheduler.fair.allow-undecleared-pools = true

Yarn.scheduler.fair.user.fair-as-default-queue = false and yarn.scheduler.fair.allow-undecleared-pools = true

You can specify new queue name when user submits a job and new queue can be created dynamically if yarn .schedule.fair.user-as-default-queue = false

You can specify new queue name per application in allocations.xml file and have new jobs automatically assigned to the application queue

fstime

VERSION

Fsimage_N (where N reflects transactions up to transaction ID N)

Edits_N-M (where N-M transactions between transaction ID N and transaction ID N)

25GB on each hard drive may not be used to store HDFS blocks

100GB on each hard drive may not be used to store HDFS blocks

All hard drives may be used to store HDFS blocks as long as at least 100 GB in total is available on the node

A maximum if 100 GB on each hard drive may be used to store HDFS blocks

Configuration of rack awareness is accomplished using a configuration file. You cannot use a rack topology script.

Hadoop gives preference to intra-rack data transfer in order to conserve bandwidth

Rack location is considered in the HDFS block placement policy

HDFS is rack aware but MapReduce daemon are not

Even for small clusters on a single rack, configuring rack awareness will improve performance

Ingest with Hadoop streaming

Ingest using Hive’s IQAD DATA command

Ingest with sqoop import

Ingest with Pig’s LOAD command

Ingest using the HDFS put command

You must modify the configuration files on the NameNode only. DataNodes read their configuration from the master nodes

You must modify the configuration files on each of the DataNodes machines

You don’t need to restart any daemon, as they will pick up changes automatically

You must restart the NameNode daemon to apply the changes to the cluster

You must restart all six DatNode daemon to apply the changes to the cluster

We cannot say; the number of Mappers is determined by the ResourceManager

We cannot say; the number of Mappers is determined by the developer

30

3

10

We cannot say; the number of mappers is determined by the ApplicationMaster

Stored as metadata on the NameNode

Stored along with the data in HDFS

Stored in the Metadata

Stored in ZooKeeper

Run hdfs fs –du / and locate the DFS Remaining value

Run hdfs dfsadmin –report and locate the DFS Remaining value

Run hdfs dfs / and subtract NDFS Used from configured Capacity

Connect to http://mynamenode:50070/dfshealth.jsp and locate the DFS remaining value

NodeManager

ApplicationMaster

ApplicationManager

ResourceManager

Configure yarn.applicationmaster.resource.memory-mb and yarn.applicationmaster.resource.cpu-vcores so that ApplicationMaster container allocations match the capacity you require.

You don’t need to configure or balance these properties in YARN as YARN dynamically balances resource management capabilities on your cluster

Configure mapred.tasktracker.map.tasks.maximum and mapred.tasktracker.reduce.tasks.maximum ub yarn-site.xml to match your cluster’s capacity set by the yarn-scheduler.minimum-allocation

Configure yarn.nodemanager.resource.memory-mb and yarn.nodemanager.resource.cpu-vcores to match the capacity you require under YARN for each NodeManager

You cannot enforce this, since client code can always override this value

Set dfs.block.size to 128M on all the worker nodes, on all client machines, and on the NameNode, and set the parameter to final

Set dfs.block.size to 128 M on all the worker nodes and client machines, and set the parameter to final. You do not need to set this value on the NameNode

Set dfs.block.size to 134217728 on all the worker nodes, on all client machines, and on the NameNode, and set the parameter to final

Set dfs.block.size to 134217728 on all the worker nodes and client machines, and set the parameter to final. You do not need to set this value on the NameNode

When your workload generates a large amount of output data, significantly larger than the amount of intermediate data

When your workload consumes a large amount of input data, relative to the entire capacity if HDFS

When your workload consists of processor-intensive tasks

When your workload generates a large amount of intermediate data, on the order of the input data itself

MapReduce version 2 (MRv2) on YARN abstracts resource allocation away from the idea of “tasks” into memory and virtual cores, thus eliminating the need for a developer to specify the number of reduce tasks, and indeed preventing the developer from specifying the number of reduce tasks.

In YARN, resource allocations is a function of megabytes of memory in multiples of 1024mb. Thus, they should specify the amount of memory resource they need by executing –D mapreduce-reduces.memory-mb-2048

In YARN, the ApplicationMaster is responsible for requesting the resource required for a specific launch. Thus, executing –D yarn.applicationmaster.reduce.tasks=2 will specify that the ApplicationMaster launch two task contains on the worker nodes.

Developers specify reduce tasks in the exact same way for both MapReduce version 1 (MRv1) and MapReduce version 2 (MRv2) on YARN. Thus, executing –D mapreduce.job.reduces-2 will specify reduce tasks.

In YARN, resource allocation is function of virtual cores specified by the ApplicationManager making requests to the NodeManager where a reduce task is handeled by a single container (and thus a single virtual core). Thus, the developer needs to specify the number of virtual cores to the NodeManager by executing –p yarn.nodemanager.cpu-vcores=2