How to Monitor AWS Route 53 Health Checks and DNS Failover: Step-by-Step Guide

AWS Route 53 sits at the edge of your infrastructure. Every latency spike, health check failure, or misconfigured failover policy here affects every downstream service. According to the 2024 CNCF Annual Survey, 78% of organizations now run production workloads in the cloud, and DNS health monitoring remains a top cause of cascading failures when not configured correctly. Most teams realize too late that Route 53’s default health check metrics are too coarse to diagnose real problems.

This guide walks through how to set up Route 53 health check monitoring, configure DNS failover, integrate CloudWatch alarms, and troubleshoot the most common failure scenarios. By the end, you will know how to detect failover events, alert on health check status changes, and validate that your failover configuration actually works under load.

Prerequisites

Before setting up Route 53 health check monitoring, ensure you have:

• An AWS account with Route 53 access
• At least two resources configured for failover (primary and secondary endpoints)
• IAM permissions for Route 53, CloudWatch, and SNS (if configuring alerts)
• AWS CLI installed and configured, or access to the AWS Management Console
• Basic familiarity with DNS concepts and health check intervals
• An SNS topic created if you want email or Slack alerts on health check failures

Step 1: Create a Route 53 Health Check for Your Primary Endpoint

Route 53 health checks monitor endpoints by sending automated requests to specified IP addresses or domain names. You configure the health check to verify your primary resource is responding correctly before Route 53 routes traffic to it.

Navigate to the Route 53 console, select Health Checks, and click Create Health Check. Choose either Monitor an endpoint (for HTTP, HTTPS, or TCP checks) or Monitor other health checks (for calculated checks that aggregate multiple health check statuses).

For endpoint monitoring, configure:

• Protocol: HTTP, HTTPS, or TCP
• IP address or domain name: Your primary resource endpoint
• Port: 80 for HTTP, 443 for HTTPS, or custom port
• Path: The URI path Route 53 will request (e.g., /health or /status)
• Request interval: Standard (30 seconds) or Fast (10 seconds)
• Failure threshold: Number of consecutive failures before marking unhealthy (default is 3)

Fast interval health checks cost $1 per month per health check compared to $0.50 for standard. For production failover, fast interval detects failures 3x faster and is usually worth the cost.

Enable string matching if your health endpoint returns a specific string like "status":"ok". Route 53 will only mark the check healthy if the response body contains that exact string.

After creating the health check, Route 53 begins sending requests from multiple locations globally. The health check is marked healthy only if the specified number of health checkers receive a valid response.

Step 2: Configure DNS Failover with Primary and Secondary Records

DNS failover requires two Route 53 records with the same name and type but different failover routing policies. One record is designated Primary, the other Secondary. When the primary health check fails, Route 53 automatically routes traffic to the secondary record.

In the Route 53 hosted zone for your domain, create a new record with:

• Record name: app.example.com (or your subdomain)
• Record type: A (for IPv4) or AAAA (for IPv6)
• Routing policy: Failover
• Failover record type: Primary
• Value: IP address or alias target of your primary resource
• Health check ID: Select the health check created in Step 1
• Record ID: A unique identifier like primary-us-east-1

Then create a second record with the same name and type:

• Failover record type: Secondary
• Value: IP address or alias target of your backup resource
• Health check ID: Leave blank or associate a separate health check for the secondary
• Record ID: A unique identifier like secondary-us-west-2

When the primary health check fails, Route 53 stops returning the primary record in DNS responses and returns the secondary instead. Failover typically completes within 60 seconds for standard health checks and 30 seconds for fast interval checks, assuming your DNS TTL is low.

Set the TTL (Time to Live) on both records to 60 seconds or lower. A high TTL means clients cache the old IP address longer, delaying failover.

Step 3: Integrate Route 53 Health Checks with CloudWatch

Route 53 automatically publishes health check status metrics to CloudWatch. Each health check creates a metric named HealthCheckStatus in the AWS/Route53 namespace. The metric value is 1 when healthy and 0 when unhealthy.

To view health check metrics, open the CloudWatch console, navigate to Metrics, and select Route 53. You will see all active health checks listed. Select a health check to view its status over time.

Key metrics to monitor:

• HealthCheckStatus: Binary metric (1 = healthy, 0 = unhealthy)
• HealthCheckPercentageHealthy: Percentage of Route 53 health checkers reporting the endpoint as healthy
• ConnectionTime: Time taken to establish a TCP connection (useful for diagnosing latency issues)
• SSLHandshakeTime: Time taken for TLS handshake (HTTPS health checks only)
• TimeToFirstByte: Time from connection to first byte received

The HealthCheckPercentageHealthy metric is especially useful. If 80% of health checkers report healthy but 20% report unhealthy, the health check overall remains healthy but you have a regional connectivity issue worth investigating.

These metrics update every 60 seconds for standard health checks and every 10 seconds for fast interval checks.

Step 4: Create CloudWatch Alarms for Health Check Failures

CloudWatch alarms notify you when a health check transitions from healthy to unhealthy. This is critical because DNS failover happens automatically, but you need to know when your primary resource failed so you can investigate and restore it.

In the CloudWatch console, select Alarms and click Create Alarm. Choose the Route 53 HealthCheckStatus metric for your primary endpoint. Configure the alarm:

• Statistic: Minimum (detects even brief unhealthy periods)
• Period: 1 minute
• Threshold: HealthCheckStatus < 1 for 1 consecutive period
• Alarm name: Route53-Primary-Unhealthy-app.example.com

For the alarm action, select an SNS topic to send notifications via email, Slack, or PagerDuty. You can also trigger Lambda functions or Auto Scaling actions based on health check status.

A second useful alarm monitors the HealthCheckPercentageHealthy metric. Set a threshold like < 80 to alert when more than 20% of Route 53 health checkers report the endpoint as unhealthy. This catches partial failures that do not trigger full failover but indicate degraded service in specific regions.

Always create alarms for both primary and secondary health checks. If the secondary also fails, you have no failback and DNS returns NXDOMAIN, effectively causing a total outage.

Step 5: Validate Failover Behavior

After configuring health checks and failover records, validate that failover works as expected under failure conditions. The simplest test is to force the primary health check to fail and confirm Route 53 switches to the secondary.

Option 1: Temporarily block the health check path on your primary resource. If your health check queries /health, configure your web server to return a 500 error or block that path entirely. Within 60 seconds (or 30 seconds for fast interval), the health check should report unhealthy and Route 53 should begin returning the secondary record.

Option 2: Use the Route 53 console to invert the health check temporarily. Edit the health check, expand Advanced Configuration, and enable Invert health check status. This marks a healthy endpoint as unhealthy for testing purposes without touching your application.

To verify failover, query DNS from multiple locations:

dig app.example.com @8.8.8.8
nslookup app.example.com 8.8.8.8

Before failover, the A record should return your primary IP. After the health check fails, the same query should return your secondary IP.

Check the CloudWatch alarm. It should transition to ALARM state within 1 minute of the health check failing. If the alarm does not fire, verify the alarm threshold and the SNS topic subscription.

Monitor the failover in the Route 53 console under Traffic Flow or Health Checks. The health check status changes to Unhealthy and the associated DNS query count for the secondary record increases.

After confirming failover works, restore the primary health check (remove the block or disable the invert setting). Route 53 automatically fails back to the primary once it reports healthy again.

Step 6: Set Up Calculated Health Checks for Complex Failover

For more complex architectures, you may need calculated health checks that aggregate the status of multiple child health checks. A calculated health check is healthy only if a specified number of child checks are healthy.

Use cases:

• Multi-region failover where the primary region has multiple availability zones. The calculated check is healthy only if at least 2 out of 3 AZ health checks are healthy.
• Application stacks with multiple dependencies. The overall health depends on the database, cache, and application server all being healthy.

To create a calculated health check, select Monitor other health checks when creating the health check. Add the child health checks and specify the threshold. For example, require at least 2 out of 3 child checks to be healthy.

Calculated health checks help reduce false positives. A single transient failure in one AZ does not trigger failover if the other AZs remain healthy.

Calculated checks add one layer of indirection, which means failover detection is slightly slower. If a child check fails, the calculated check waits one evaluation period before reflecting the failure.

Troubleshooting Common Issues

Health check remains unhealthy but the resource is responding correctly

Route 53 sends health check requests from multiple IP ranges globally. If your security group or firewall blocks Route 53 health checker IPs, the checks fail even though the endpoint works. Ensure your security group allows inbound traffic on the health check port from the Route 53 health checker IP ranges. Filter for service: ROUTE53_HEALTHCHECKS.

Failover is slow or does not happen

Check the DNS TTL on your Route 53 records. A TTL of 300 seconds means clients cache the old IP for up to 5 minutes even after Route 53 updates the record. Lower the TTL to 60 seconds for faster failover. Also verify the health check interval. Standard checks evaluate every 30 seconds; fast checks every 10 seconds. Failover cannot happen faster than the check interval plus failure threshold multiplied together.

Health check reports unhealthy intermittently

Intermittent failures usually indicate the resource is under load or experiencing transient network issues. Check the HealthCheckPercentageHealthy metric. If it fluctuates between 60% and 100%, the issue is likely regional network latency. Consider raising the failure threshold from 3 to 4 or 5 to avoid unnecessary failovers caused by brief network hiccups.

Secondary resource also fails during failover

If both primary and secondary health checks fail, Route 53 has no healthy record to return. DNS queries receive no answer or NXDOMAIN, causing a complete outage. Always monitor secondary health checks with CloudWatch alarms. Consider adding a third tertiary fallback or a static maintenance page hosted on S3 and CloudFront that never fails.

Health check passes but application returns errors

Route 53 health checks verify network reachability and HTTP status codes but do not validate application logic. If your health endpoint returns 200 OK but the application has a database connection failure, Route 53 still marks it healthy. Implement a deep health check endpoint that validates all critical dependencies database, cache, external APIs and return 500 if any are unreachable.

CloudWatch alarm does not fire

Verify the SNS topic has a confirmed subscription. Check the alarm configuration in CloudWatch and ensure the metric namespace is AWS/Route53 and the dimension is the correct health check ID. Test the SNS topic manually by publishing a test message.

Monitoring Route 53 health checks and DNS failover is critical for maintaining uptime in distributed systems. This guide covered creating health checks, configuring failover routing, integrating CloudWatch alarms, and validating that failover actually works. Teams using infrastructure monitoring platforms benefit from unified visibility across DNS, application, and infrastructure layers. For broader visibility across AWS services, AWS monitoring tools help correlate Route 53 health with EC2, RDS, and load balancer metrics in one dashboard.

Disclaimer: The information in this article reflects the latest details available at the time of publication and may change as technologies and products evolve. Features, pricing, and plan limits can change over time. Always verify the latest information directly with the vendor before making purchasing or deployment decisions.

Frequently Asked Questions