Blackbox 网络监控

145次阅读
没有评论

共计 3716 个字符,预计需要花费 10 分钟才能阅读完成。

Blackbox Exporter 是什么

参考链接:Github Blackbox Exporter

顾名思义,像 node_exporter 或mysql_exporter等,可以直接从内部获取到资源运行状态指标,此类可称为白盒。

而黑盒则是模拟用户身份探测服务的外部可见性。

常用的模块包含 TCP,HTTP,DNS,ICMP等。

部署

如需使用 ICMP 模块作为探针,需预先修改内核参数:

# /etc/sysctl.conf
net.ipv4.ping_group_range=0 2147483647
net.ipv4.ip_forward = 1

# sysctl -p

配置文件 yaml 示例:

 # cat /etc/blackbox_exporter.yml
modules:
  icmp:
    prober: icmp
    timeout: 5s
    icmp:
      preferred_ip_protocol: "ip4"
      source_ip_address: "192.168.2.10"

  http_2xx:
    prober: http
    http:
      preferred_ip_protocol: "ip4"
      follow_redirects: false
      tls_config:
        insecure_skip_verify: true

  http_post_2xx:
    prober: http
    http:
      method: POST

  tcp_connect: 
    prober: tcp
    tcp: 
      ip_protocol_fallback: false
      preferred_ip_protocol: ip4
      tls: true
      tls_config: 
        insecure_skip_verify: true
    timeout: 15s

  pop3s_banner:
    prober: tcp
    tcp:
      query_response:
      - expect: "^+OK"
      tls: true
      tls_config:
        insecure_skip_verify: false

  ssh_banner:
    prober: tcp
    tcp:
      query_response:
      - expect: "^SSH-2.0-"
      - send: "SSH-2.0-blackbox-ssh-check"

  irc_banner:
    prober: tcp
    tcp:
      query_response:
      - send: "NICK prober"
      - send: "USER prober prober prober :prober"
      - expect: "PING :([^ ]+)"
        send: "PONG ${1}"
      - expect: "^:[^ ]+ 001"

prometheus 配置示例:

- job_name: 'blackbox_http'
  scrape_interval: 60s
  metrics_path: /probe
  params:
    module: [http_2xx]
  file_sd_configs:
    - files:
      - '/etc/prometheus/file_sd/blackbox_http.yml'
  relabel_configs:
    - source_labels: [__address__]
      target_label: __param_target
    - source_labels: [__param_target]
      target_label: instance
    - target_label: __address__
      replacement: 127.0.0.1:9119

- job_name: 'blackbox_ping'
  scrape_interval: 60s
  metrics_path: /probe
  params:
    module: [icmp]
  file_sd_configs:
    - files:
        - '/etc/prometheus/file_sd/blackbox_ping.yml'
  relabel_configs:
    - source_labels: [__address__]
      target_label: __param_target
    - source_labels: [__param_target]
      target_label: instance
    - target_label: __address__
      replacement: 127.0.0.1:9119

- job_name: 'blackbox_tcp'
  scrape_interval: 30s
  metrics_path: /probe
  params:
    module: [tcp_connect]
  file_sd_configs:
    - files:
      - '/etc/prometheus/file_sd/blackbox_tcp.yml'
  relabel_configs:
    - source_labels: [__address__]
      target_label: __param_target
    - source_labels: [__param_target]
      target_label: instance
    - target_label: __address__
      replacement: 127.0.0.1:9119

- job_name: 'blackbox_tls'
  scrape_interval: 30s
  metrics_path: /probe
  params:
    module: [http_2xx]
  file_sd_configs:
    - files:
      - '/etc/prometheus/file_sd/blackbox_tls.yml'
  relabel_configs:
    - source_labels: [__address__]
      target_label: __param_target
    - source_labels: [__param_target]
      target_label: instance
    - target_label: __address__
      replacement: 127.0.0.1:9119

systemd 单元文件示例:

# cat /etc/systemd/system/blackbox_exporter.service 
[Unit]
Description=Blackbox Exporter
After=network-online.target

[Service]
Type=simple
PIDFile=/var/run/blackbox_exporter.pid
PermissionsStartOnly=true
ExecReload=/bin/kill -HUP $MAINPID
ExecStart=/usr/local/bin/blackbox_exporter \
  --config.file=/etc/blackbox_exporter.yml \
  --web.listen-address=0.0.0.0:9119

SyslogIdentifier=blackbox_exporter
KillMode=process
Restart=always
RestartSec=5


NoNewPrivileges=true
PrivateTmp=true
ProtectHome=true

ProtectSystem=full

[Install]
WantedBy=multi-user.target

常用PromQL

  • ICMP metrics:指定时间间隔内的 ping 成功率
    sum_over_time(probe_success{job="blackbox_ping"}[$time_range]) / count_over_time(probe_success{job="blackbox_ping"}[$time_range])
    

  • ICMP metrics:ping 延时
    probe_icmp_duration_seconds{phase="rtt",job="blackbox_ping"}
    

  • ICMP metrics:ping 实时丢包率
    1 - avg_over_time(probe_success{job="blackbox_ping"}[60s])
    

  • HTTP metrics:连通性
    probe_success{job="blackbox_http"} - 0
    

  • HTTP metrics:返回状态码
    probe_http_status_code{job="blackbox_http"}
    

  • HTTP metrics:响应时间
    probe_duration_seconds{job="blackbox_http"}
    

  • HTTP metrics:指定时间间隔内的成功率
    sum_over_time(probe_success{job="blackbox_http"}[$time_range]) / count_over_time(probe_success{job="blackbox_http"}[$time_range])
    

Grafana 绘图

Blackbox 网络监控

总结

如果聚焦更细致的 ping 测试结果,可以尝试使用 smokeping。参考链接:github.com/oetiker/SmokePing

对比 blackbox,它能够获取到ping的连续结果,就类似我们在命令行终端直接使用ping命令。

而 blackbox 受限于 scrape_interval,只能够抓取有间隔的时间点数据。

本文属于专题:Prometheus Exporter

引用链接

正文完
 
pengyinwei
版权声明:本站原创文章,由 pengyinwei 2024-10-16发表,共计3716字。
转载说明:除特殊说明外本站文章皆由CC-4.0协议发布,转载请注明出处:https://www.opshub.cn
评论(没有评论)