Falanguage icon
×
Products
Services
Resources
About Us
Apache Hadoop Yarn backgroundApache Hadoop Yarn
Apache Hadoop Yarn backgroundApache Hadoop Yarn
Data Analysis Tools
KibanaNiFiApache SparkApache Hadoop HDFSApache Hadoop Yarn

Apache Hadoop Yarn Monitoring

YARN governs and orchestrates the processing of data in Hadoop. The YARN cluster architecture is a master-slave cluster framework like HDFS, with a master node daemon called the ResourceManager and one or more slave node daemons called NodeManagers running on worker or slave nodes in the cluster. Moein monitoring system is capable of monitoring YARN Cluster. Performance metrics of ResourceManager and NodeManagers, and their JVM metrics. The following is a list of the performance metrics of Apache YARN:

ResourceManager Metrics:

JVM Metrics:

A complete description of the JVM-related metrics and parameters is provided in the JVM object.

Cluster Metrics:

  1. ResourceManager Host Name
  2. Number Of Active NodeManagers
  3. Number Of Decommissioned NodeManagers
  4. Number Of Lost NodeManagers
  5. Number Of Unhealthy NodeManagers
  6. Number Of Rebooted NodeManagers
  7. Number Of Launched AMs
  8. AMs Average Launch Time
  9. Number Of Registered AMs
  10. AMs Average Register Time
  11. Submitted Applications
  12. Running Applications
  13. Pending Applications
  14. Completed Applications
  15. Killed Applications

Yarn Resource Manager.overview.en

Live Nodes Metrics:

  1. NodeManager Host Name
  2. NodeManager Rack Name
  3. NodeManager State
  4. NodeManager ID
  5. NodeManager HTTP Address
  6. NodeManager Last Health Update
  7. NodeManager Health Report
  8. NodeManager Version
  9. NodeManager Number Of Containers
  10. NodeManager Used Memory
  11. NodeManager Free Memory Space
  12. NodeManager Used Memory Percentage

Yarn Resource Manager.node Manager.en

General information from ResourceManager:

  1. ResourceManager State
  2. Security Activation Status
  3. Host Name
  4. Port

General information from Queues:

  1. Queue Name
  2. Queue Host Name
  3. Running Applications With Elapsed Time Less Than 60 Min
  4. Running Applications With Elapsed Time Between 60 And 300 Min
  5. Running Applications With Elapsed Time Between 300 And 1440 Min
  6. Running Applications With Elapsed Time More Than 1440 Min
  7. Submitted Applications
  8. Running Applications
  9. Pending Applications
  10. Completed Applications
  11. Killed Applications
  12. Allocated Memory
  13. Allocated CPU
  14. Allocated Containers
  15. Total Allocated Containers
  16. Number Of Rack Local Containers Allocated
  17. Number Of Off Switch Containers Allocated
  18. Number Of Released Containers
  19. Available Memory
  20. Available CPU
  21. Pending Requested Memory
  22. Pending Requested CPU
  23. Number Of Pending Containers
  24. Reserved Memory
  25. Reserved CPU
  26. Reserved Containers
  27. Active Users
  28. Active Applications

Yarn Resource Manager.queue.en

NodeManager Metrics:

JVM Metrics:

A complete description of the JVM-related metrics and parameters is provided in the JVM object.

NodeManager Metrics:

  1. Launched Containers
  2. Completed Containers
  3. Failed Containers
  4. Killed Containers
  5. Initializing Containers
  6. Running Containers
  7. Allocated Memory
  8. Allocated Containers
  9. Available Memory
  10. Used VCores
  11. Available VCores

Yarn Node Manager.overview.en

Communication Protocols:

  • JMX
Data Analysis Tools
KibanaNiFiApache SparkApache Hadoop HDFSApache Hadoop Yarn
Address
3rd floor, No. 8, 2nd dead-end, Sadeghi St., Azadi Ave., Tehran, Iran, Postal code 1458846155