How to Monitor Azure Kubernetes Service (AKS) Node and Pod Health: Step-by-Step Guide 2026

Without proper monitoring, a NotReady node or an OOMKilled pod can sit undetected for 20 minutes while your application silently degrades. By the time an alert fires or a user complains, the cascading failure has already impacted revenue. According to the CNCF Annual Survey 2023, 89% of organizations now use Kubernetes in production, making effective node and pod health monitoring a baseline requirement for operational reliability.

This guide walks through how to monitor AKS node and pod health using Azure Monitor Container Insights, Managed Prometheus, and third party tools like CubeAPM. You will learn how to track node conditions, identify failing pods, set up proactive alerts, and correlate infrastructure signals with application performance data.

Prerequisites

Before starting, ensure you have the following in place:

• An active Azure subscription with an AKS cluster deployed
• Azure CLI installed and authenticated (az login completed)
• kubectl installed and configured to access your AKS cluster (az aks get-credentials)
• Contributor or Owner role on the AKS cluster resource group
• Basic familiarity with Kubernetes concepts (nodes, pods, namespaces, deployments)
• A working understanding of Prometheus and PromQL (optional but recommended for Managed Prometheus sections)

Step 1: Enable Azure Monitor Container Insights on Your AKS Cluster

Azure Monitor Container Insights is the native monitoring solution for AKS. It collects node metrics, pod metrics, logs, and Kubernetes events from your cluster and stores them in a Log Analytics workspace.

Navigate to your AKS cluster in the Azure portal. Go to Monitoring in the left sidebar, then select Insights. If Container Insights is not enabled, click Enable and select an existing Log Analytics workspace or create a new one. Azure will deploy the monitoring agent (a DaemonSet) to every node in your cluster.

Verify the deployment by running:

kubectl get daemonset -n kube-system

You should see ama-logs or microsoft-oms-agent running on all nodes. This agent collects metrics and logs and sends them to Azure Monitor.

Once enabled, return to the Insights blade in the Azure portal. You will see cluster level views showing CPU and memory usage, node count, pod count, and active alerts. This is your primary dashboard for AKS health monitoring.

Container Insights automatically surfaces critical signals: node conditions (Ready, MemoryPressure, DiskPressure, PIDPressure), pod status (Running, Pending, Failed, CrashLoopBackOff), and container restart counts. These signals are indexed and queryable via Log Analytics using Kusto Query Language (KQL).

Step 2: Track Node Health Metrics and Conditions

AKS nodes report health through a set of conditions that Kubernetes surfaces via the node status API. The most important condition is Ready. A node in NotReady state cannot accept new pod assignments and may already be evicting existing pods.

From the Insights blade in the Azure portal, click on the Nodes tab. You will see a list of all nodes with their current CPU percentage, memory percentage, and status. Click on any node to drill into its detailed metrics: CPU usage over time, memory working set, disk used percentage, and network bytes sent/received.

To query node conditions directly using Log Analytics, go to Monitoring > Logs and run this KQL query:

KubeNodeInventory
| where TimeGenerated > ago(1h)
| where Status contains "NotReady" or Status contains "MemoryPressure" or Status contains "DiskPressure"
| project TimeGenerated, Computer, Status
| order by TimeGenerated desc

This query returns all nodes that reported a non-healthy condition in the last hour. You can adjust the time window and add filters by node pool or namespace.

For real time node metrics, you can also use kubectl:

kubectl top nodes
kubectl describe node <node-name>

The describe output includes the full list of conditions, allocated resources, and recent events. Look for conditions like MemoryPressure: True or DiskPressure: True. These indicate the node is under resource pressure and may start evicting pods.

Node OS disk space is a common failure point. Monitor the Disk Used Percentage metric in Container Insights. If a node’s disk utilization exceeds 85%, it will trigger DiskPressure and begin pod eviction. Set an alert threshold at 80% to catch this early.

Step 3: Monitor Pod Health, Restarts, and OOMKills

Pod health monitoring focuses on three signals: pod phase (Running, Pending, Failed), container restart count, and OOMKill events. A pod stuck in Pending state indicates scheduling failure due to insufficient resources or unmet affinity rules. A high restart count suggests application crashes or liveness probe failures. An OOMKill means the container exceeded its memory limit and was terminated by the kernel.

In the Azure portal Insights blade, click the Containers tab. Filter by namespace, deployment, or pod name to narrow the view. Look for the Restarts column. Any value above zero in the last 30 minutes warrants investigation.

To query pod health using Log Analytics, run:

KubePodInventory
| where TimeGenerated > ago(1h)
| where PodStatus != "Running"
| project TimeGenerated, Namespace, Name, PodStatus, ContainerRestartCount
| order by TimeGenerated desc

This returns all pods not in Running state in the last hour. Add | where ContainerRestartCount > 0 to focus on restarting pods.

To detect OOMKills, query Kubernetes events:

KubeEvents
| where TimeGenerated > ago(1h)
| where Reason == "OOMKilled"
| project TimeGenerated, Namespace, Name, Message
| order by TimeGenerated desc

OOMKills are a production red flag. They mean your memory limits are set too low, or your application has a memory leak. Correlate OOMKill events with pod metrics to identify which container caused the kill.

Use kubectl to inspect pod status directly:

kubectl get pods --all-namespaces -o wide
kubectl describe pod <pod-name> -n <namespace>

The describe output shows recent events, including scheduling failures, liveness probe failures, and OOMKills. Look for the Last State field under container status. If it shows Terminated (OOMKilled), the container was killed for exceeding memory.

Pod restarts are not always bad. A single restart after a deployment is normal. But if a pod restarts every 10 minutes, it indicates a crash loop. Check application logs:

kubectl logs <pod-name> -n <namespace> --previous

The --previous flag shows logs from the last terminated container, which is critical for debugging crash loops.

Step 4: Set Up Proactive Alerts for Node and Pod Issues

Monitoring without alerts is just dashboards. Alerts turn signals into actions. Azure Monitor supports metric alerts and log query alerts. For AKS, you should create alerts on node conditions, pod restart counts, and OOMKill events.

Navigate to Monitoring > Alerts in the Azure portal for your AKS cluster. Click New alert rule. Select the metric or log query you want to alert on.

For a node NotReady alert, use this setup:

• Signal type: Metric
• Metric namespace: Microsoft.ContainerService/managedClusters
• Metric name: kube_node_status_condition (filter by condition=Ready, status=false)
• Threshold: Static, greater than 0
• Aggregation: Maximum
• Evaluation frequency: 5 minutes
• Window: 10 minutes

This fires an alert if any node stays in NotReady state for more than 10 minutes.

For pod restart alerts, use a log query alert:

KubePodInventory
| where TimeGenerated > ago(10m)
| where ContainerRestartCount > 2
| summarize RestartCount=max(ContainerRestartCount) by Name, Namespace

Set the alert threshold to fire when RestartCount exceeds 2 in a 10 minute window. This catches pods in crash loops before they degrade service.

For OOMKill alerts, use:

KubeEvents
| where TimeGenerated > ago(5m)
| where Reason == "OOMKilled"
| summarize Count=count() by Namespace, Name

Fire an alert when Count is greater than 0. OOMKills are always worth an immediate page.

Route alerts to your incident management system. Azure Monitor integrates with PagerDuty, Slack, Microsoft Teams, email, webhooks, and Azure Logic Apps. Configure an action group with your preferred notification channel.

Alert noise is a real problem. Start with high severity alerts only (NotReady nodes, OOMKills, sustained high CPU above 90%). Add lower priority alerts once you have validated thresholds and response runbooks. Over-alerting trains teams to ignore alerts.

Step 5: Enable AKS Managed Prometheus for Deeper Metrics

Azure Monitor Container Insights provides node and pod level metrics, but Managed Prometheus gives you access to the full Prometheus ecosystem: Kubernetes metrics from kube-state-metrics, custom application metrics, and PromQL query flexibility.

To enable Managed Prometheus, navigate to your AKS cluster in the Azure portal. Go to Monitoring > Insights, then click Configure Monitoring at the top. Select Enable Prometheus metrics. Azure will deploy the Prometheus agent and create an Azure Monitor workspace to store metrics.

Once enabled, you can query Prometheus metrics using Azure Managed Grafana (deployed alongside Prometheus) or via the Azure Monitor workspace query interface.

Key Prometheus metrics for node and pod health:

• kube_node_status_condition{condition="Ready",status="true"} — current Ready state of each node
• node_cpu_seconds_total — CPU usage per node (requires node exporter)
• node_memory_MemAvailable_bytes — available memory per node
• kube_pod_status_phase{phase="Running"} — count of pods in Running state
• kube_pod_container_status_restarts_total — cumulative restart count per container
• container_memory_working_set_bytes — memory usage per container
• kube_pod_container_status_last_terminated_reason{reason="OOMKilled"} — OOMKill events

Example PromQL query to track node CPU pressure:

(1 - avg(rate(node_cpu_seconds_total{mode="idle"}[5m])) by (node)) * 100

This calculates CPU utilization percentage per node over a 5 minute window. Use it to alert when any node exceeds 90% CPU for more than 10 minutes.

Managed Prometheus integrates with Azure Monitor Alerts. Create a Prometheus rule group to define alert conditions in PromQL, then route alerts to the same action groups you configured for Container Insights.

Prometheus adds query flexibility Container Insights cannot match, especially for high cardinality labels (pod labels, namespace tags, custom annotations). But it also adds storage and query cost. Plan retention and sampling rates accordingly.

Step 6: Correlate Node and Pod Health with Application Performance

Node pressure and pod restarts often manifest as application latency spikes or error rate increases before they show up in Kubernetes metrics. To catch these early, correlate AKS infrastructure signals with application performance monitoring data.

If you are using an APM tool that supports infrastructure monitoring alongside distributed tracing, link pod metrics to traces. For example, if a pod restarts at 10:15 AM, check your APM tool for error spikes or latency increases in that pod’s service at the same time. This correlation speeds up root cause analysis by confirming whether the restart was a symptom or a cause.

CubeAPM integrates Kubernetes monitoring with APM, logs, and infrastructure metrics in a single self hosted platform. It collects pod labels, node names, and container IDs as trace attributes, so you can filter traces by Kubernetes context. When a pod restarts, you can immediately see which API endpoints were affected and drill into the exact traces that failed during the restart window.

To set this up in CubeAPM, enable the Kubernetes integration by deploying the CubeAPM agent as a DaemonSet on your AKS cluster. The agent auto-discovers pods and enriches telemetry with Kubernetes metadata. From the CubeAPM UI, create a dashboard that shows pod restart count alongside API error rate and P99 latency for the affected service. This unified view eliminates the need to context-switch between Azure Monitor and your APM tool.

For teams using Azure Monitor exclusively, you can achieve partial correlation by querying Container Insights and Application Insights logs together in the same Log Analytics workspace. Use a join query to link pod events to application exceptions by timestamp and pod name. This requires manual instrumentation to tag logs with pod metadata, which is where a Kubernetes aware APM platform saves significant time.

Troubleshooting Common Issues

This section covers the most frequent node and pod health problems teams encounter on AKS, with specific steps to diagnose and resolve each.

Node stuck in NotReady state

Check node conditions:

kubectl describe node <node-name>

Look for Conditions section. Common causes: kubelet stopped, network connectivity lost, or disk full. If DiskPressure: True, the node has run out of disk space. SSH into the node (if enabled) or use kubectl debug to create a debugging pod and inspect disk usage. Delete old logs or unused images to free space. If the issue persists, cordon and drain the node, then delete and recreate it.

Pods stuck in Pending state

Run:

kubectl describe pod <pod-name> -n <namespace>

Check the Events section. Common reasons: insufficient CPU or memory in the cluster, unsatisfied affinity rules, or no nodes matching the pod’s node selector. If the message says 0/3 nodes are available: 3 Insufficient memory, scale your node pool or reduce pod resource requests. If it says 0/3 nodes are available: 3 node(s) didn't match pod affinity rules, revise your affinity or anti-affinity configuration.

High pod restart count due to liveness probe failures

Check probe configuration:

kubectl get pod <pod-name> -n <namespace> -o yaml | grep -A 10 livenessProbe

Liveness probes that are too aggressive (short initialDelaySeconds or timeoutSeconds) cause unnecessary restarts. If your application takes 30 seconds to start but the liveness probe starts after 10 seconds, it will kill the pod before it is ready. Increase initialDelaySeconds to match your application’s actual startup time. Also verify the probe endpoint returns 200 within the timeout window.

OOMKills on specific pods

Query memory usage before the kill:

Perf
| where TimeGenerated > ago(1h)
| where ObjectName == "K8SContainer"
| where CounterName == "memoryWorkingSetBytes"
| where InstanceName contains "<pod-name>"
| summarize max(CounterValue) by bin(TimeGenerated, 1m)
| render timechart

If memory usage trends upward over time, you have a memory leak. If it spikes suddenly, you may have a traffic surge or a batch job that exceeds limits. Increase the memory limit in the pod spec, or fix the leak in the application code. Never remove memory limits entirely. Unlimited memory pods can consume all node memory and crash the entire node.

Node CPU throttling causing application slowness

Check CPU throttling per container:

rate(container_cpu_cfs_throttled_seconds_total[5m])

If this metric is non-zero, the container is hitting its CPU limit and being throttled by the kernel. This causes latency spikes. Increase the CPU limit in the pod spec, or optimize the application to use less CPU. Note that CPU throttling does not cause pod restarts, but it degrades performance.

Node disk pressure causing pod evictions

Check disk usage per node:

kubectl top nodes
kubectl describe node <node-name>

If DiskPressure: True, the node has less than 10% disk space available. Kubernetes will evict low priority pods to free space. Clean up old container images:

kubectl exec -it <pod-name> -n kube-system -- crictl rmi --prune

Or increase the node OS disk size in the AKS node pool configuration. Disk pressure alerts should fire before evictions start.

Monitoring AKS Node and Pod Health with CubeAPM

CubeAPM provides unified Kubernetes monitoring alongside APM, logs, and infrastructure metrics, all self hosted inside your Azure VPC. It eliminates the need to stitch together Azure Monitor, Application Insights, and third party log tools by consolidating telemetry into a single platform with full data residency and predictable $0.15/GB pricing.

For AKS monitoring, CubeAPM deploys as a DaemonSet that collects node metrics, pod metrics, Kubernetes events, and container logs. It auto-discovers services running in your cluster and links pod restarts, OOMKills, and resource saturation events directly to distributed traces and error logs for the affected service. This correlation speeds up root cause analysis by showing infrastructure and application context in the same view.

To set up AKS monitoring in CubeAPM, deploy the agent using the provided Helm chart:

helm repo add cubeapm https://charts.cubeapm.com
helm install cubeapm-agent cubeapm/cubeapm-agent \
  --set clusterName=your-aks-cluster \
  --set endpoint=your-cubeapm-backend-url \
  --set apiKey=your-api-key

The agent will begin collecting metrics, logs, and traces within minutes. Navigate to the CubeAPM Kubernetes dashboard to see cluster health, node CPU and memory usage, pod status, and restart counts. Create alerts on node NotReady state, pod restart count, or OOMKill events using the same alert templates you would configure in Azure Monitor, but with the added benefit of trace and log correlation.

CubeAPM integrates with Azure Monitor if you want to keep existing dashboards, but most teams migrate fully to CubeAPM to reduce tool sprawl. Because it runs inside your VPC, there are no Azure egress fees for telemetry data. One customer (a 200 node AKS cluster running 800 microservices) documented 73% cost savings after replacing Azure Monitor and Application Insights with CubeAPM, while also reducing MTTR by 40% due to faster trace to log to metric correlation.

For teams evaluating Azure monitoring tools, CubeAPM ranks high for AKS workloads because it handles the Kubernetes specific signals (pod labels, namespace tags, node conditions) natively, without requiring custom KQL queries or manual dashboard construction.

AKS monitoring is complex because failures cascade. A node with disk pressure can trigger pod evictions, which cause service restarts, which manifest as API errors. CubeAPM surfaces the entire chain in a single timeline view, making it easier to see that the root cause was disk space, not application code.

Pricing follows the standard CubeAPM model: $0.15/GB of telemetry ingested (metrics, logs, traces), with unlimited retention and no per-host or per-user fees. For a 50 node AKS cluster ingesting 2TB of telemetry per month, total cost is $300 per month. Compare this to Azure Monitor Container Insights at $0.40/GB for logs plus $2.50 per cluster per day plus $0.50 per monitored node per day, which totals $1,575 per month for the same 50 node cluster.

This estimate models a production ready AKS setup with high availability and standard telemetry volumes. A smaller cluster or simpler deployment may cost significantly less.

Conclusion

Monitoring AKS node and pod health requires tracking multiple layers: node conditions, pod status, container restarts, OOMKills, resource utilization, and Kubernetes events. Azure Monitor Container Insights provides the baseline with automatic node and pod metric collection and Log Analytics integration. Managed Prometheus adds query flexibility for high cardinality metrics and custom application instrumentation. Tools like CubeAPM consolidate all telemetry into a single self hosted platform with full data residency and predictable pricing, which matters for teams operating under strict compliance requirements or cloud cost controls.

Set alerts on NotReady nodes, pod restart counts, and OOMKill events. Correlate these infrastructure signals with application performance data to catch cascading failures early. Most production incidents on Kubernetes manifest as a combination of infrastructure pressure and application errors. Monitoring both in the same tool reduces MTTR and prevents alert fatigue.

Disclaimer: The information in this article reflects the latest details available at the time of publication and may change as technologies and products evolve. Features, pricing, and plan limits can change over time. Always verify the latest information directly with the vendor before making purchasing or deployment decisions.

Frequently Asked Questions