[ Prometheus ] Jmx exporter Hadoop설정

[ prerequisite ]

• hadoop설치 
• hadoop_env.sh에 jmx port 설정

jmx설정 예시

export HADOOP_DATANODE_OPTS="-Dcom.sun.management.jmxremote 
-Dcom.sun.management.jmxremote.ssl=false 
-Dcom.sun.management.jmxremote.authenticate=false 
-Dcom.sun.management.jmxremote.port=19004 
$HADOOP_DATANODE_OPTS"

jmx데이터를 바로 prometheus가 읽을 수 없으니 

jmx_exporter를 통해 jmx->metric으로 변환시켜야 한다.

[ jmx_export설치 ]

wget https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/0.16.1/jmx_prometheus_javaagent-0.16.1.jar

참조 : https://github.com/prometheus/jmx_exporter

[ hadoop-env.sh ]

각 옵션마다 jar파일을 실행하게 설정 값을 변경

예시 : -javaagent:<jmx exporter jar file path>=<port>:<configure file path>

# Name node jmx export 설정
export HDFS_NAMENODE_OPTS="
...
-javaagent:/path/jmx_prometheus_javaagent-0.16.1.jar=5566:/path/config.yml
$HDFS_NAMENODE_OPTS"

# Data node jmx export 설정
export HDFS_DATANODE_OPTS="
...
-javaagent:/path/jmx_prometheus_javaagent-0.16.1.jar=5577:/path/config.yml
$HDFS_DATANODE_OPTS"

configure file 참조: https://github.com/prometheus/jmx_exporter

---
startDelaySeconds: 0
hostPort: 127.0.0.1:1234
username: 
password: 
jmxUrl: service:jmx:rmi:///jndi/rmi://127.0.0.1:1234/jmxrmi
ssl: false
lowercaseOutputName: false
lowercaseOutputLabelNames: false
whitelistObjectNames: ["org.apache.cassandra.metrics:*"]
blacklistObjectNames: ["org.apache.cassandra.metrics:type=ColumnFamily,*"]
rules:
  - pattern: 'org.apache.cassandra.metrics<type=(\w+), name=(\w+)><>Value: (\d+)'
    name: cassandra_$1_$2
    value: $3
    valueFactor: 0.001
    labels: {}
    help: "Cassandra metric $1 $2"
    cache: false
    type: GAUGE
    attrNameSnakeCase: false

[ web에서 확인 ]

http://YourHostIp:5566

[ prometheus.yml ]

jmx exporter데이터를 http로 받을 수 있으니 prometheus가 가져올 수 있게 설정

scrape_configs:
  - job_name: "namenode"
    static_configs:
      - targets: ["namenode-ip:5566"]

  - job_name: "datanode"
    static_configs:
      - targets: ["datanode-ip:5577"]

[ prometheus web에서 확인 ]

프로메테우스 재시작 후 -> http:prometheus_host:9090 -> target메뉴 에서 

수집데이터 확인

참조 :

https://www.fatalerrors.org/a/hdfs-yarn-on-promethues-grafana-monitoring-promethues-data-source-configuration.html

https://programming.vip/docs/how-to-use-jmx_-exporter-prometheus-grafana-monitor-hadoop-cluster.html