How do I set up Prometheus monitoring on AWS EKS?

If you are running workloads on Amazon EKS, you need visibility into pod health, node resource usage, and application performance. Prometheus is the most widely adopted open-source monitoring tool for Kubernetes, and this guide shows you exactly how to set it up. You will deploy Prometheus using the kube-prometheus-stack Helm chart and configure persistent storage so your metrics survive pod restarts.

What Is EKS Monitoring and Why Does It Matter?

EKS monitoring refers to collecting, storing, and visualizing metrics from your Amazon Elastic Kubernetes Service cluster. Without monitoring, you cannot reliably answer questions like: Is my cluster running out of memory? Which pod is consuming the most CPU? Why did my deployment fail at 2 a.m.?

Prometheus addresses this by continuously scraping metrics exposed by your nodes, pods, and Kubernetes API objects. It stores these in a time-series database and lets you query them with PromQL. From there you can build dashboards, set alert rules, and track resource usage across your entire cluster.

According to the AWS EKS documentation, Prometheus is the recommended self-managed monitoring approach for EKS clusters, with Amazon Managed Service for Prometheus (AMP) available as the fully managed alternative.

Prerequisites

Before you start, make sure you have the following in place:

• A running Amazon EKS cluster (version 1.21 or later recommended)
• kubectl: installed and configured with access to your cluster. Verify with: kubectl cluster-info
• Helm v3+: the Kubernetes package manager. Install from https://helm.sh/docs/intro/install/
• AWS CLI: configured with credentials and the correct region
• IAM permissions to create namespaces and deploy Helm charts

Step 1: Create a Dedicated Monitoring Namespace

Isolating your monitoring stack in its own namespace keeps it separate from application workloads and simplifies access control.

kubectl create namespace monitoring

kubectl create namespace monitoring

Confirm the namespace exists:

kubectl get namespace monitoring

kubectl get namespace monitoring

Step 2: Add the Prometheus Helm Repository

Helm charts for Prometheus are maintained by the prometheus-community organization. Add the repository and update the local chart index:

helm repo add prometheus-community https://prometheus-community.github.io/helm-chartshelm repo update

helm repo add prometheus-community https://prometheus-community.github.io/helm-chartshelm repo update

Step 3: Install kube-prometheus-stack

The kube-prometheus-stack Helm chart is the recommended way to deploy Prometheus on Kubernetes. It installs the following components in a single command:

• Prometheus Server
• Alertmanager
• Prometheus Operator
• Node Exporter (collects node-level metrics like CPU and memory)
• kube-state-metrics (exposes Kubernetes API object state)

Run the install command:

helm install kube-prometheus-stack \  prometheus-community/kube-prometheus-stack \  --namespace monitoring \  --create-namespace

helm install kube-prometheus-stack \  prometheus-community/kube-prometheus-stack \  --namespace monitoring \  --create-namespace

Helm will output the release name and status. The –create-namespace flag ensures the namespace is created if it does not already exist.

Step 4: Verify That All Pods Are Running

Check the health of all pods in the monitoring namespace:

kubectl get pods -n monitoring

kubectl get pods -n monitoring

You should see output similar to this:

NAME                                                READY   STATUS    RESTARTSalertmanager-kube-prometheus-stack-alertmanager-0   2/2     Running   0kube-prometheus-stack-kube-state-metrics-XXXXXX     1/1     Running   0kube-prometheus-stack-operator-XXXXXXXXXXX          1/1     Running   0kube-prometheus-stack-prometheus-node-exporter-X    1/1     Running   0prometheus-kube-prometheus-stack-prometheus-0       2/2     Running   0

NAME                                                READY   STATUS    RESTARTSalertmanager-kube-prometheus-stack-alertmanager-0   2/2     Running   0kube-prometheus-stack-kube-state-metrics-XXXXXX     1/1     Running   0kube-prometheus-stack-operator-XXXXXXXXXXX          1/1     Running   0kube-prometheus-stack-prometheus-node-exporter-X    1/1     Running   0prometheus-kube-prometheus-stack-prometheus-0       2/2     Running   0

If any pod shows Pending status, check whether PersistentVolumeClaims are bound (see Step 6). If a pod shows CrashLoopBackOff, inspect its logs with: kubectl logs <pod-name> -n monitoring

Step 5: Access the Prometheus UI

Access Prometheus UI

Use kubectl port-forward to reach the Prometheus UI from your local machine:

kubectl port-forward -n monitoring \  svc/kube-prometheus-stack-prometheus 9090:9090

kubectl port-forward -n monitoring \  svc/kube-prometheus-stack-prometheus 9090:9090

Open your browser and go to http://localhost:9090. You can now run PromQL queries, browse scrape targets under Status > Targets, and view alert rules.

Step 6: Configure Persistent Storage (Production)

By default, Prometheus stores metrics in memory and local ephemeral storage. When a pod restarts, all historical data is lost. In production, configure a PersistentVolumeClaim backed by Amazon EBS so your metrics survive restarts.

Create a values.yaml file with the following storage configuration:

prometheus:  prometheusSpec:    storageSpec:      volumeClaimTemplate:        spec:          storageClassName: gp2          accessModes: ["ReadWriteOnce"]          resources:            requests:              storage: 20Gi

prometheus:  prometheusSpec:    storageSpec:      volumeClaimTemplate:        spec:          storageClassName: gp2          accessModes: ["ReadWriteOnce"]          resources:            requests:              storage: 20Gi

Install or upgrade the chart with this values file:

helm upgrade --install kube-prometheus-stack \  prometheus-community/kube-prometheus-stack \  --namespace monitoring \  -f values.yaml

helm upgrade --install kube-prometheus-stack \  prometheus-community/kube-prometheus-stack \  --namespace monitoring \  -f values.yaml

Step 7: Set Up Basic Alerting with Alertmanager

The kube-prometheus-stack includes Alertmanager. You can configure it to send notifications to Slack, PagerDuty, or email. The following values.yaml snippet routes alerts to a Slack webhook:

alertmanager:  config:    global:      resolve_timeout: 5m    route:      group_by: ['alertname', 'namespace']      group_wait: 30s      group_interval: 5m      repeat_interval: 12h      receiver: 'slack'    receivers:    - name: 'slack'      slack_configs:      - api_url: 'https://hooks.slack.com/services/YOUR/WEBHOOK/URL'        channel: '#alerts'        send_resolved: true

alertmanager:  config:    global:      resolve_timeout: 5m    route:      group_by: ['alertname', 'namespace']      group_wait: 30s      group_interval: 5m      repeat_interval: 12h      receiver: 'slack'    receivers:    - name: 'slack'      slack_configs:      - api_url: 'https://hooks.slack.com/services/YOUR/WEBHOOK/URL'        channel: '#alerts'        send_resolved: true

Apply the updated values:

helm upgrade kube-prometheus-stack \  prometheus-community/kube-prometheus-stack \  --namespace monitoring \  -f values.yaml

helm upgrade kube-prometheus-stack \  prometheus-community/kube-prometheus-stack \  --namespace monitoring \  -f values.yaml

Step 8: Expose Prometheus Externally (Optional)

If your team needs to access Prometheus from outside the cluster, you have two options:

Option A: LoadBalancer Service

Patch the Prometheus service to create an AWS Application Load Balancer:

kubectl patch svc kube-prometheus-stack-prometheus \  -n monitoring \  -p '{"spec": {"type": "LoadBalancer"}}'
kubectl get svc -n monitoring

kubectl patch svc kube-prometheus-stack-prometheus \  -n monitoring \  -p '{"spec": {"type": "LoadBalancer"}}'
kubectl get svc -n monitoring

The EXTERNAL-IP column will show the AWS ELB DNS name once provisioned. Restrict access using Security Groups so only authorized IP ranges can reach the endpoint.

Option B: AWS Load Balancer Controller with Ingress

For HTTPS and path-based routing, use the AWS Load Balancer Controller with an Ingress resource. This approach lets you terminate TLS at the ALB and restrict access with AWS WAF.

Alternative: Amazon Managed Service for Prometheus (AMP)

If you do not want to manage Prometheus storage and scaling yourself, AWS offers Amazon Managed Service for Prometheus (AMP). AMP is a fully managed, Prometheus-compatible monitoring service that scales automatically and stores metrics durably in AWS infrastructure.

With AMP, you still run a Prometheus agent or the AWS Distro for OpenTelemetry (ADOT) Collector inside your EKS cluster to scrape and forward metrics. AMP then handles long-term storage and querying. Any Prometheus-compatible tool can connect to AMP using SigV4 authentication.

Self-managed Prometheus vs Amazon Managed Service for Prometheus

Prometheus vs CloudWatch for EKS Monitoring

A common question is whether to use CloudWatch instead of Prometheus. Both work, but there are meaningful differences.

CloudWatch integrates natively with AWS services, requires no additional tooling, and is a good fit for small EKS clusters where you are already invested in the AWS ecosystem. However, CloudWatch costs can escalate quickly. Custom metrics are priced at roughly $0.30 per metric per month, and costs multiply across dimensions (nodes, pods, namespaces). Teams managing clusters with many custom application metrics have reported monthly bills in the thousands of dollars.

Prometheus stores metrics locally, which makes it free at the metric-collection level. The tradeoff is operational overhead: you manage storage, upgrades, and high availability yourself. For teams already using Kubernetes, the Prometheus ecosystem is well-supported and provides richer Kubernetes-native metrics out of the box.

Common Issues and Fixes

Pods Stuck in Pending: PersistentVolumeClaim Not Bound

If the Prometheus server or Alertmanager pod shows Pending status, check the PVC:

kubectl get pvc -n monitoring

kubectl get pvc -n monitoring

If the PVC is in Pending state, your cluster may not have a default StorageClass, or the EBS CSI driver is not installed. On EKS 1.23+, install the Amazon EBS CSI add-on through the EKS console or with:

eksctl create addon \  --name aws-ebs-csi-driver \  --cluster <your-cluster-name> \  --region <your-region> \  --force

eksctl create addon \  --name aws-ebs-csi-driver \  --cluster <your-cluster-name> \  --region <your-region> \  --force

Your Observability Tool Shows No Metrics

If your observability tool is not receiving data from Prometheus, verify the Prometheus service endpoint is reachable inside the cluster. The internal service URL for Prometheus is:

http://kube-prometheus-stack-prometheus.monitoring.svc.cluster.local:9090

http://kube-prometheus-stack-prometheus.monitoring.svc.cluster.local:9090

Point your tool to this URL as the Prometheus data source. If scrape targets are missing, check Status > Targets in the Prometheus UI at http://localhost:9090 after port-forwarding.

Port-Forward Disconnects After Idle Period

kubectl port-forward is designed for short-term local access. In production, expose services using a LoadBalancer or an Ingress controller rather than relying on port-forwarding.

Useful PromQL Queries for EKS Monitoring

Once Prometheus is running, use these queries in the Prometheus UI to get immediate visibility into your cluster:

CPU usage per node:

100 - (avg by(instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)

100 - (avg by(instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)

Memory usage per pod:

sum by(pod, namespace) (container_memory_working_set_bytes{container!=""})

sum by(pod, namespace) (container_memory_working_set_bytes{container!=""})

Number of pods not in Running state:

count by(namespace) (kube_pod_status_phase{phase!="Running"})

count by(namespace) (kube_pod_status_phase{phase!="Running"})

Node disk pressure:

kube_node_status_condition{condition="DiskPressure",status="true"}

kube_node_status_condition{condition="DiskPressure",status="true"}

Summary

Setting up Prometheus monitoring on AWS EKS comes down to four core steps: create a monitoring namespace, add the Helm repository, install kube-prometheus-stack, and access the dashboards. From there, configure persistent storage for production, set up Alertmanager for notifications, and use PromQL queries to monitor the metrics that matter to your workloads.

For teams that prefer not to manage the monitoring stack themselves, Amazon Managed Service for Prometheus is a fully managed alternative that requires no storage configuration and scales automatically.

Disclaimer: This article contains pricing estimates based on publicly available AWS CloudWatch Logs rates as of May 2026. Actual costs may vary by AWS region, account type, and usage patterns. Always verify current pricing before making infrastructure decisions.

FAQs