Add Prometheus and Grafana to your voice stack

Prometheus collects numbers over time: how many calls are up, how many failed, how busy each process is. It reads them from each program’s metrics endpoint every few seconds and keeps the history. Grafana draws that history as dashboards and alerts on it.

This guide stands up both beside an OpenSIPS proxy, has OpenSIPS publish its statistics, and shows a test call in the numbers. With Kamailio instead, one section replaces the OpenSIPS steps. This guide does not use sipnab. When you have this working, Add sipnab’s metrics to Prometheus adds sipnab’s own.

The parts, and what each one does:

Part	Role
Prometheus	Scrapes each metrics endpoint every 5 seconds and stores the series, on port 9090.
Grafana	Dashboards over what Prometheus stored, on port 3000.
OpenSIPS	Your SIP proxy. Its prometheus module publishes its statistics over its httpd module, on 127.0.0.1:8888.

Everything below runs on one machine. The last section says what changes when they are apart.

Tested on

Every command on this page ran as written, in order, on x86_64 virtual machines with 2 cores and 3 GB of memory:

• With OpenSIPS built in step 2: on a clean Debian 13 (kernel 6.12.63) and Ubuntu 24.04.5 (kernel 6.8.0) on 2026-09-26, with an earlier step 3 that named the source build’s paths; and on Debian 13 on 2026-09-27, rebuilding the tree the rtpengine guide had built.
• With the OpenSIPS 4.0 packages: on Ubuntu 24.04.5, on 2026-09-27.
• With Kamailio: on Debian 13, and beside OpenSIPS on Ubuntu 24.04.5, on 2026-09-27.

The commands pin the components to the versions below.

Software	Version or commit
Docker Engine / Compose	from the Docker apt repository
Prometheus	prom/prometheus:v3.15.0
Grafana	grafana/grafana:13.2.2
OpenSIPS, built here	f46ef9337b, master, 4.1.0-dev
OpenSIPS, from packages	4.0.2, installed as the OpenSIPS guide installs it
Kamailio	6.1.4, installed as the Kamailio guide installs it
SIPp (for the test call)	the distribution’s sip-tester

The examples use 192.0.2.10 as the machine’s address. Replace it with yours everywhere it appears.

Before you start: your SIP server

Find your case, and follow the steps it names:

• No SIP server yet. Follow every step. Step 2 builds OpenSIPS for you. If you would rather run the OpenSIPS 4.0 packages, install them with step 1 of the OpenSIPS guide, then take the packages’ route in step 2.
• OpenSIPS already runs, from the packages or built from source. Step 2 adds the two modules the metrics need. Step 3 finds your configuration file and replaces it. On a machine whose script you want to keep, add the lines step 3 marks to your own script instead.
• Kamailio already runs, or you want Kamailio. Follow step 1, then With Kamailio, which takes you through the rest.
• Both OpenSIPS and Kamailio on one machine. Set up OpenSIPS as above, then see With Kamailio, which covers both.

1. Install Docker

Prometheus and Grafana run as containers. Install Docker Engine and the Compose plugin from the Docker repository:

# Run all of these, in order.
sudo apt-get update
sudo apt-get install -y ca-certificates curl git
sudo install -m 0755 -d /etc/apt/keyrings
. /etc/os-release
sudo curl -fsSL "https://download.docker.com/linux/$ID/gpg" -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/$ID $VERSION_CODENAME stable" \
  | sudo tee /etc/apt/sources.list.d/docker.list >/dev/null
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo usermod -aG docker "$USER"

Log out and back in so that your account’s new docker group takes effect, then check:

docker compose version

2. Install OpenSIPS’s metrics modules

OpenSIPS publishes its statistics with its prometheus module, over HTTP served by its httpd module.

With the OpenSIPS packages, the two modules come in two more packages:

sudo apt-get install -y opensips-http-modules opensips-prometheus-module

Or build OpenSIPS with them. Skip this if you installed the packages. The prometheus module is part of OpenSIPS’s default build, but the httpd module it publishes through is not: it needs libmicrohttpd. Install that and build httpd too. If you already built OpenSIPS from source by one of these guides, the block reuses that tree, /usr/local/src/voice/opensips, and rebuilds it with httpd. With a tree of your own, change /usr/local/src/voice in the block to the directory that holds it. OpenSIPS master also builds with compiler optimizations turned off, which is right for OpenSIPS’s own developers and wrong for a proxy carrying calls, so turn them back on:

# Run all of these, in order.
sudo apt-get install -y --no-install-recommends build-essential bison flex uuid-dev pkg-config libncurses-dev libssl-dev libmicrohttpd-dev
sudo mkdir -p /usr/local/src/voice && sudo chown "$USER": /usr/local/src/voice
cd /usr/local/src/voice
[ -d opensips ] || git clone https://github.com/OpenSIPS/opensips.git
cd opensips
git checkout f46ef9337b
make Makefile.conf
sed -i 's/^DEFS+= -DCC_O0/#DEFS+= -DCC_O0/' Makefile.conf
make -j2 all include_modules="httpd"
sudo make install include_modules="httpd"
/usr/local/sbin/opensips -V | head -2

Leave DBG_MALLOC as it is. With both it and CC_O0 switched off, this commit of master does not compile: net/tcp_conn_defs.h calls get_ticks() without including the header that declares it, and only DBG_MALLOC’s headers happen to supply it.

3. Configure OpenSIPS to publish its statistics

This configuration is a minimal proxy with the metrics endpoint added. The metrics part is the block of loadmodule and modparam lines marked below.

Write the configuration

The packages and the source build keep their configuration and their modules in different places. The first two lines find them: C is your configuration file, and M the directory your install loads modules from, which the script’s mpath names.

# Run all of these, in order.
for f in /etc/opensips/opensips.cfg /usr/local/etc/opensips/opensips.cfg; do sudo test -f "$f" && C=$f && break; done
for d in /usr/lib/*/opensips/modules /usr/local/lib64/opensips/modules; do [ -f "$d/tm.so" ] && M=$d && break; done
echo "configuration: $C   modules: $M"
sudo tee "$C" >/dev/null <<'EOF'
# OpenSIPS as a SIP proxy that publishes its statistics for Prometheus.
log_level=3
stderror_enabled=no
syslog_enabled=yes
syslog_facility=LOG_LOCAL0
udp_workers=2
open_files_limit=4096

socket=udp:192.0.2.10:5060   # the address your phones and carriers reach

mpath="MODULES/"

loadmodule "proto_udp.so"   # built into the core, but still loaded by name
loadmodule "signaling.so"
loadmodule "sl.so"
loadmodule "tm.so"
loadmodule "rr.so"
loadmodule "maxfwd.so"
loadmodule "sipmsgops.so"
loadmodule "dialog.so"

loadmodule "mi_fifo.so"
modparam("mi_fifo", "fifo_name", "/run/opensips/opensips_fifo")

# --- metrics start here ---
loadmodule "httpd.so"
modparam("httpd", "ip", "127.0.0.1")
modparam("httpd", "port", 8888)
loadmodule "prometheus.so"
modparam("prometheus", "statistics", "core: dialog: tm: shmem: load:")
# --- metrics end here ---

route {
	if (!mf_process_maxfwd_header(10)) {
		send_reply(483, "Too Many Hops");
		exit;
	}

	if (has_totag()) {
		if (is_method("ACK") && t_check_trans()) {
			t_relay();
			exit;
		}
		if (!loose_route()) {
			send_reply(404, "Not here");
			exit;
		}
		t_relay();
		exit;
	}

	if (is_method("CANCEL")) {
		if (t_check_trans())
			t_relay();
		exit;
	}
	t_check_trans();

	if (!is_method("INVITE")) {
		send_reply(405, "Method Not Allowed");
		exit;
	}

	record_route();
	create_dialog();

	# Where the call goes. Here, a test callee on this machine; in your
	# stack, lookup("location"), dispatcher or a carrier.
	$du = "sip:127.0.0.1:5070";
	t_relay();
}
EOF
sudo sed -i "s|^mpath=\"MODULES/\"|mpath=\"$M/\"|" "$C"
sudo grep '^mpath=' "$C"

What the metrics lines do:

• httpd serves HTTP on 127.0.0.1:8888, the loopback address, so only this machine can read the statistics.
• prometheus publishes them at /metrics on that server. It publishes nothing until statistics names what to publish; here the core, dialog, transaction, memory and load groups. Each statistic becomes a series named opensips_ and the statistic’s name, such as opensips_processed_dialogs.
• create_dialog() makes OpenSIPS track each call, which is what the dialog statistics count.

If you built OpenSIPS in step 2: give it a user and a unit

The packages come with an opensips user and a systemd unit. If you installed them, or if your source build already runs as a service, skip to Check the configuration and start OpenSIPS. A fresh build has neither. Run OpenSIPS as its own user, under systemd:

# Run all of these, in order.
sudo useradd --system --home-dir /run/opensips --shell /usr/sbin/nologin opensips
sudo chown root:opensips /usr/local/etc/opensips
sudo chmod 750 /usr/local/etc/opensips
sudo tee /etc/systemd/system/opensips.service >/dev/null <<'EOF'
[Unit]
Description=OpenSIPS SIP server
After=network.target

[Service]
Type=forking
User=opensips
Group=opensips
RuntimeDirectory=opensips
RuntimeDirectoryMode=775
PIDFile=/run/opensips/opensips.pid
ExecStart=/usr/local/sbin/opensips -P /run/opensips/opensips.pid -f /usr/local/etc/opensips/opensips.cfg -m 64 -M 8
Restart=always
TimeoutStopSec=30s
LimitNOFILE=262144

[Install]
WantedBy=multi-user.target
EOF
sudo systemctl daemon-reload

Check the configuration and start OpenSIPS

# Run all of these, in order.
for f in /etc/opensips/opensips.cfg /usr/local/etc/opensips/opensips.cfg; do sudo test -f "$f" && C=$f && break; done
sudo chown root:opensips "$C"
sudo chmod 640 "$C"
sudo opensips -C -f "$C"
sudo systemctl enable opensips
sudo systemctl restart opensips
systemctl is-active opensips
curl -s 127.0.0.1:8888/metrics | awk '/^opensips_/ && n++ < 3'

The last line prints the first three series OpenSIPS publishes.

4. Start Prometheus and Grafana

Both run with the host’s own network, so that Prometheus can reach OpenSIPS’s endpoint on the loopback address. Choose a Grafana admin password and put it in .env, where Compose reads it:

# Run all of these, in order.
sudo mkdir -p /opt/monitoring && sudo chown "$USER": /opt/monitoring
cd /opt/monitoring
echo "GF_PASS=$(openssl rand -hex 12)" > .env
chmod 600 .env
mkdir -p grafana/datasources
cat > prometheus.yml <<'EOF'
global:
  scrape_interval: 5s

scrape_configs:
  - job_name: opensips
    static_configs:
      - targets: ["127.0.0.1:8888"]
EOF
cat > grafana/datasources/prometheus.yml <<'EOF'
apiVersion: 1
datasources:
  - name: Prometheus
    type: prometheus
    uid: prometheus
    url: http://127.0.0.1:9090
    isDefault: true
EOF
cat > docker-compose.yml <<'EOF'
services:
  prometheus:
    image: prom/prometheus:v3.15.0
    network_mode: host
    command:
      - --config.file=/etc/prometheus/prometheus.yml
      - --storage.tsdb.retention.time=15d
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
      - prometheus-data:/prometheus
    restart: unless-stopped

  grafana:
    image: grafana/grafana:13.2.2
    network_mode: host
    environment:
      GF_SECURITY_ADMIN_PASSWORD: ${GF_PASS}
    volumes:
      - ./grafana/datasources:/etc/grafana/provisioning/datasources:ro
      - grafana-data:/var/lib/grafana
    restart: unless-stopped

volumes:
  prometheus-data:
  grafana-data:
EOF
docker compose up -d
until curl -fs -o /dev/null localhost:9090/-/ready; do sleep 2; done
until curl -fs -o /dev/null localhost:3000/api/health; do sleep 2; done
echo ready

--storage.tsdb.retention.time=15d is how long Prometheus keeps the history.

Check that Prometheus reaches OpenSIPS:

curl -s localhost:9090/api/v1/targets | python3 -c 'import sys,json; [print(t["labels"]["job"], t["health"]) for t in json.load(sys.stdin)["data"]["activeTargets"]]'

It prints opensips up. Open Grafana at http://192.0.2.10:3000 and log in as admin with the password in /opt/monitoring/.env. The Prometheus data source is already there.

5. Place a test call and watch the counter

SIPp plays both ends: a callee on this machine, and a caller that dials through OpenSIPS. SIPp’s built-in caller ignores the Record-Route header OpenSIPS adds, so its BYE would miss the proxy and draw 404 Not here. The two sed lines make it honor the route set, the way a real phone does:

# Run all of these, in order.
sudo apt-get install -y sip-tester
mkdir -p ~/sipp && cd ~/sipp
sipp -sd uac > uac_rr.xml
sed -i 's|<recv response="200" rtd="true">|<recv response="200" rtd="true" rrs="true">|' uac_rr.xml
sed -i -E 's#^( *)(ACK|BYE) sip:\[service\]@\[remote_ip\]:\[remote_port\] SIP/2.0#\1\2 [next_url] SIP/2.0\n\1[routes]#' uac_rr.xml
sipp -sn uas -i 127.0.0.1 -p 5070 -m 1 -bg
sipp -sf uac_rr.xml 192.0.2.10:5060 -i 192.0.2.10 -p 5080 -m 1 -d 1000 -timeout 20s

Prometheus scrapes every 5 seconds. Wait for one scrape, then ask it how many dialogs OpenSIPS has processed:

# Run all of these, in order.
sleep 6
curl -s localhost:9090/api/v1/query --data-urlencode 'query=opensips_processed_dialogs' \
  | python3 -c 'import sys,json; print(json.load(sys.stdin)["data"]["result"][0]["value"][1])'

It prints 1, the test call.

6. Operate it

Check health and read the logs.

# Run all of these, in order.
cd /opt/monitoring && docker compose ps
docker compose logs --tail 20 prometheus
systemctl is-active opensips

Restart. Prometheus misses the scrapes while it is down, and shows a gap.

# Run all of these, in order.
cd /opt/monitoring && docker compose restart
sudo systemctl restart opensips

A restart of OpenSIPS starts its counters again from zero.

Uninstall. docker compose down -v removes the containers and their volumes, which is the stored history and Grafana’s settings:

# Run all of these, in order.
cd /opt/monitoring && docker compose down -v
sudo systemctl disable --now opensips

With Kamailio

Kamailio publishes its statistics with its xhttp_prom module, over HTTP served by its xhttp module. Both come with the kamailio package. Set Kamailio up as the Kamailio guide does, through its step 2, and follow step 1 here for Docker. In place of steps 2 and 3, add the metrics endpoint to Kamailio’s configuration. The lines marked metrics are the ones to add to your own script:

# Run all of these, in order.
sudo tee /etc/kamailio/kamailio.cfg >/dev/null <<'EOF'
#!KAMAILIO
# Kamailio as a SIP proxy that publishes its statistics for Prometheus.
debug=2
log_stderror=no
log_facility=LOG_LOCAL0
children=2

listen=udp:192.0.2.10:5060   # the address your phones and carriers reach
listen=tcp:127.0.0.1:8888    # metrics: the endpoint, on the loopback address
tcp_accept_no_cl=yes         # metrics: HTTP requests carry no Content-Length

loadmodule "tm.so"
loadmodule "sl.so"
loadmodule "rr.so"
loadmodule "maxfwd.so"
loadmodule "siputils.so"
loadmodule "textops.so"
loadmodule "pv.so"
loadmodule "kex.so"
loadmodule "corex.so"
loadmodule "ctl.so"

# metrics
loadmodule "xhttp.so"
loadmodule "xhttp_prom.so"
modparam("xhttp_prom", "xhttp_prom_stats", "all")

request_route {
	if (!mf_process_maxfwd_header("10")) {
		sl_send_reply("483", "Too Many Hops");
		exit;
	}

	if (has_totag()) {
		if (loose_route()) {
			t_relay();
			exit;
		}
		if (is_method("ACK") && t_check_trans()) {
			t_relay();
		}
		exit;
	}

	if (is_method("CANCEL")) {
		if (t_check_trans()) {
			t_relay();
		}
		exit;
	}
	t_check_trans();

	if (!is_method("INVITE")) {
		sl_send_reply("405", "Method Not Allowed");
		exit;
	}

	record_route();

	# Where the call goes. Here, a test callee on this machine; in your
	# stack, lookup("location"), dispatcher or a carrier.
	$du = "sip:127.0.0.1:5070";
	t_relay();
}

# metrics: answer HTTP requests for /metrics
event_route[xhttp:request] {
	if ($hu =~ "^/metrics") {
		prom_dispatch();
		exit;
	}
	xhttp_reply("404", "Not Found", "text/plain", "not found\n");
}
EOF
sudo kamailio -c -f /etc/kamailio/kamailio.cfg
sudo systemctl enable kamailio
sudo systemctl restart kamailio
systemctl is-active kamailio
curl -s 127.0.0.1:8888/metrics | awk '/^kamailio_/ && n++ < 3'

What the metrics lines do:

• listen=tcp:127.0.0.1:8888 gives Kamailio a TCP socket on the loopback address, and xhttp answers the HTTP requests that arrive on it. Only this machine can read the statistics.
• xhttp_prom_stats "all" publishes every statistic Kamailio keeps. Each becomes a series named kamailio_, the group and the statistic’s name, such as kamailio_core_rcv_requests_invite.
• event_route[xhttp:request] is where Kamailio handles each HTTP request, and prom_dispatch() answers the ones for /metrics.

Then follow step 4, and name the scrape job after Kamailio:

# Run all of these, in order.
cd /opt/monitoring
sed -i 's/job_name: opensips/job_name: kamailio/' prometheus.yml
docker compose restart prometheus
until curl -fs -o /dev/null localhost:9090/-/ready; do sleep 2; done

Place step 5’s test call, then ask Prometheus how many INVITE requests Kamailio has received:

# Run all of these, in order.
sleep 6
curl -s localhost:9090/api/v1/query --data-urlencode 'query=kamailio_core_rcv_requests_invite{job="kamailio"}' \
  | python3 -c 'import sys,json; print(json.load(sys.stdin)["data"]["result"][0]["value"][1])'

It prints 1, the test call. The query names the kamailio job because, for the few minutes before the rename, Prometheus scraped the same endpoint as opensips, and a query still finds that series for up to five minutes after its last sample. Step 6 then works unchanged, with kamailio in place of opensips in the systemctl commands.

With OpenSIPS on the same machine, OpenSIPS already serves its metrics on 127.0.0.1:8888, and Kamailio listens for SIP on 5062, as OpenSIPS and Kamailio on one machine sets it up. Give Kamailio’s endpoint its own port too: in the configuration above, change 5060 to 5062 in the first listen line, and 8888 to 8889 in the second listen line and in the curl line at the end. Then keep the opensips scrape job and add one for Kamailio, in place of renaming it:

# Run all of these, in order.
cd /opt/monitoring
cat >> prometheus.yml <<'EOF'
  - job_name: kamailio
    static_configs:
      - targets: ["127.0.0.1:8889"]
EOF
docker compose restart prometheus
until curl -fs -o /dev/null localhost:9090/-/ready; do sleep 2; done

Prometheus then shows both targets up, and each proxy’s series under its own name: opensips_ and kamailio_.

Put the parts on different machines

• Prometheus on its own machine. OpenSIPS’s endpoint has no authentication, so keep it on an address only Prometheus can reach: set modparam("httpd", "ip", ...) to an internal address, allow port 8888 from the Prometheus machine only, and change the scrape target to that address.
• Several SIP servers. Add each one to the targets list. Prometheus labels every series with the target it came from.

When something does not work

• curl 127.0.0.1:8888/metrics prints no opensips_ lines. The statistics parameter is missing. The prometheus module publishes nothing until it names what to publish.
• OpenSIPS does not start: failed to load module 'httpd.so'. It was not built. Build with include_modules="httpd", which needs libmicrohttpd-dev.
• The target shows down. Check curl 127.0.0.1:8888/metrics on the OpenSIPS machine, then that the target address in prometheus.yml matches.