Blue-green deployment: two interchangeable environments for zero downtime Blue-green deployment rests on a simple but powerful principle: maintaining two identical production environments simultaneously, called Blue and Green. At each deployment, you release the new version to the inactive environment (for example Green), test it under production conditions without shifting user traffic, then instantly switch all traffic (via a load balancer or a DNS update) to the new environment as soon as you are certain it works. If an anomaly appears after the switch, you instantly reactivate the old environment. This model completely eliminates maintenance windows: your users experience no service interruption.
The main advantage of blue-green deployment lies in its immediate reversibility. Unlike a classic deployment where fixing an error in production can take several minutes (or hours), you can switch back in a few seconds by simply changing the traffic destination. You thus gain the time to diagnose the problem without the service being degraded for your users. This rapid recovery capability considerably reduces team stress during a critical incident and limits the financial losses tied to failures. For an e-commerce platform, for example, every minute of downtime represents lost revenue.
On AWS, implementing blue-green deployment generally goes through an Application Load Balancer (ALB) or Network Load Balancer (NLB) that routes traffic to one target or the other (Blue or Green). You can also use Amazon Route 53 with a simple failover routing policy to direct DNS requests. The virtual machine images (AMIs) or Docker containers deployed via ECS/EKS run in both environments in parallel. During a deployment, you update the dormant version, run smoke tests (quick integrity checks), then shift the traffic weight from 0 to 100% in a single atomic action. If you have prepared an automated rollback strategy, the system can even revert to Blue automatically if the health metrics degrade in the first seconds following the switch.
The cost of this strategy is its main drawback: you must provision double the resources (two complete environments at all times). For a massive application or a team on a constrained budget, this duplication can be problematic. That is why blue-green is often chosen for critical or high-volume services, where the risk of service loss justifies the infrastructure investment.
Canary release: progressive testing in production with a subset of users The canary release is a strategy where the new version is deployed to production, but only a small percentage of user traffic (for example 5% to 10%) is initially routed to it. Meanwhile, the majority of users (90% to 95%) continue to use the old stable version. You observe the critical metrics (error rate, latency, CPU, memory) of this small group of canaries. If all goes well after a few minutes or hours, you gradually increase the percentage: 25%, then 50%, then 100%. If an anomaly is detected, you stop the deployment and immediately switch the affected minority back to the old version. The maximum impact is therefore limited to a few percent of user traffic.
The appeal of the canary release is that it lets you validate a new version directly with real users, in their real conditions, before rolling it out broadly. Tests in a pre-production environment never capture the full complexity of production traffic: unexpected usage patterns, client-side version combinations, particular cache states, exotic network configurations. A canary exposes the new version to a share of this raw reality. In addition, the canary is more economical than a blue-green: you only double your infrastructure gradually, as the deployment accelerates.
On AWS, a canary release is orchestrated via an Application Load Balancer with weighted target groups or via AWS CodeDeploy with gradual deployment strategies. You first configure two target groups: one for the old version, one for the new. The load balancer initially sends 5% of requests to the new version and 95% to the old. You then integrate a monitoring system (CloudWatch, Prometheus, Datadog, etc.) that watches key metrics of the new version compared with the old. If the error rate stays low, latency does not spike, and no alarms fire, a Lambda function or a custom controller gradually increases the traffic weight. This process can be fully automated or semi-automatic (the team validates manually before each step). Tools such as Flagger (for Kubernetes) or AWS AppConfig can orchestrate this progression in an elegant and reproducible way.
The canary differs from blue-green on one fundamental point: there is only one production, not two. The two versions coexist in the same production during the deployment, routed differently according to the configured weight. This coexistence requires careful management of backward compatibility (the new version must be able to talk to databases and services as they existed with the old version) and makes deployments slightly more complex to schedule. In return, if an anomaly detected after 10 minutes affects 5% of traffic, the impact on users is minimal and you have had time to identify the problem without serving the degradation to 100% of your audience.
Comparison and choice between blue-green and canary depending on your context Blue-green and canary do not address the same constraints, and the choice between them depends mainly on your risk appetite, your budget, and the criticality of your service. Blue-green is ideal if you must be absolutely certain that the new version works before shifting traffic: you have the time to test it fully in the dormant environment, possibly by injecting real replayed traffic (traffic replay) or by having a team validate manually. It is the strategy of banks, insurers, or healthcare services, where an error can cause massive financial or trust losses. It also suits deployments where the cost of the additional infrastructure is negligible compared with the company's revenue.
The canary, on the other hand, shines when you are willing to accept testing in real conditions but want to minimize exposure in case of a problem. This is the philosophy of many scale-ups and startups: deploy often, observe quickly, and fix on the fly. The canary costs less (no doubling of infrastructure) and accelerates the feedback loops. You discover production-specific bugs much earlier than with a purely pre-production approach. If your service is very critical but you have responsive teams capable of diagnosing and fixing an incident in minutes, the canary is relevant. For a less critical service, or one you trust more, it also allows you to be more relaxed.
Many organizations combine the two depending on the change being deployed. You can use a blue-green for major architectural changes or database migrations (too much risk for a gradual canary), but switch to a canary for minor business logic updates or bug fixes. Some organizations always start with a canary (deploy 1% of traffic to the new version, observe for 5 minutes), then accelerate to 100% if nothing abnormal appears, while keeping the blue-green infrastructure ready in the background for an instant rollback should a problem nonetheless emerge an hour later. This "accelerated canary with a blue-green safety net" approach offers a good balance between speed, cost, and safety.
The choice also depends on your existing deployment chain. If you already have two identical environments (one for each AWS availability zone, for example), moving to a true blue-green is natural. If you have a single Kubernetes cluster with limited resources, a canary using an ingress controller (Istio, Linkerd) is more accessible. Finally, some contexts impose legal or contractual constraints: for example, if you serve sensitive data and a regulation forbids exposing even 5% of traffic to an uncertified version, only a fully pre-tested blue-green is allowed.
Implementing blue-green deployment in an AWS architecture Setting up a blue-green deployment on AWS starts with an architectural decision: are you going to maintain two complete sets of resources (compute, database, cache, etc.) or only double the application layer? For most cases, you double only the application (EC2, ECS, EKS) and share the stateful services (RDS, ElastiCache, S3). This reduces costs and simplifies state synchronization.
Here is a concrete architecture: you have an Application Load Balancer that receives all traffic. Behind it, two target groups: one named "Blue" containing EC2 instances (or ECS tasks) running version v1, the other named "Green" containing identical instances running version v2 (not yet active). Initially, the ALB routes 100% of traffic to Blue. During a deployment, you launch the Green instances, connect them to the same RDS database and the same ElastiCache, then orchestrate smoke tests (simple HTTP calls verifying that the application starts and responds to basic requests). Once Green is stable, you change the ALB target group rule: switch from "100% to Blue, 0% to Green" to "0% to Blue, 100% to Green" in a single atomic action. Traffic switches instantly.
To automate this cycle, you write a script (bash, Python, or better, a Lambda function orchestrating the AWS SDK) that executes the following steps: read the current version from a tag or an environment variable, create the Green instances/tasks with the new version, wait for their readiness (positive health checks), run smoke tests, modify the ALB listener rule to redirect to Green, then archive or stop the Blue instances (without deleting them outright, in case you need to revert to them). This script is triggered either manually (via a declarative GitOps approach) or at the end of your CI/CD pipeline (CodePipeline, GitHub Actions, etc.).
One important subtlety: the active TCP/HTTP connections. When you switch the ALB traffic, the connections established with Blue are not interrupted instantly, they persist until the timeout expires. If you brutally delete the Blue instances after the switch, you risk leaving clients with zombie connections. The best practice is to wait for a connection draining period (generally 300 seconds), during which the ALB no longer sends NEW traffic to Blue but lets the existing connections finish naturally. Then you stop the Blue instances. They remain available if you need to roll back within a few minutes.
Another challenge: state synchronization. If your application writes to cache (ElastiCache) or database (RDS), and a user switches between Blue and Green in the middle of a transaction, they could see inconsistencies. The solutions: (1) use a strong transactional database (RDS with Multi-AZ), (2) ensure that your application is idempotent and can handle repeated reads, (3) add a step of "draining" the cached sessions before the switch (force all users to reconnect after the switch). Many modern systems with in-memory sessions (instead of the database) handle this by placing the session in a distributed store (Redis) rather than in the application itself.
Orchestrating the canary release with CloudWatch and CodeDeploy A canary release on AWS is commonly orchestrated via AWS CodeDeploy, which natively supports gradual deployments ("Canary" and "Linear" are built-in options). For this, you prepare a deployment script (AppSpec) that describes how to launch the new version, then you configure CodeDeploy to roll out this deployment along a curve of progressive traffic increase.
Here is the concrete flow: you send an artifact (for example, a ZIP file containing the new application version) to S3, then you trigger CodeDeploy specifying the "Canary" strategy with parameters such as Canary Percentage (e.g. 10%) and Canary Interval Minutes (e.g. 5 minutes). CodeDeploy launches the new version on 10% of your instances/tasks, waits 5 minutes, then observes the metrics (CloudWatch Alarms configured upstream). If no alarm has turned red, it increases to 90% and waits another 5 minutes. Then it switches the remaining 10% to 100%. If an alarm fires during the first phase (10%), CodeDeploy stops and rolls back all changes.
The key to this automated orchestration lies in your CloudWatch alarms. You must define metrics that truly reflect the health of your application: ALB Target Group Unhealthy Host Count (if it exceeds 0), Application Error Rate (if it exceeds 1%), Average Response Time (if it exceeds a threshold), or custom application metrics (number of failed transactions, memory usage, etc.). These alarms are the signal CodeDeploy uses to decide whether to continue or stop the deployment. If you configure alarms that are too strict, you risk blocking valid deployments (false positives). Too permissive, and you will let bugs through (false negatives). The team must find the balance, often by trial and error during the first canary deployments.
A typical use case: you have a Node.js application served by ECS tasks. You prepare a new version with a database query optimization. You trigger CodeDeploy in canary 10% for 5 minutes. The new version starts on one task, and CloudWatch verifies that the average latency stays below 200 ms and that the error rate is zero. After 5 minutes, CodeDeploy checks the alarms: all is well, it switches to 90% (9 tasks out of 10). You wait another 5 minutes, the metrics stay good, the deployment finishes at 100%. If, on the contrary, 2 minutes after the canary deployment you notice that database access errors spike (the new query is not compatible with your database schema), the CloudWatch alarm fires automatically and CodeDeploy switches all instances back to the old version in under a minute. Exposure was limited to 10% of traffic for 2 minutes.
A common extension: combine CodeDeploy canary with a tool like AWS Lambda or a custom application that enriches the decisions. For example, instead of simply checking that the alarms are silent, you can integrate automated regression tests (GET and POST requests for critical scenarios) to validate the canaries before progressing. Or you can configure Slack/Teams notifications that your team receives, "Canary in phase 2/2, traffic at 10%, approve to move to 90%?", for a human check if the change is sensitive.
Monitoring and critical alarms to automatically trigger a rollback Whatever your deployment strategy (blue-green or canary), monitoring determines your ability to recover quickly in case of a problem. A deployment may seem successful at the moment of the switch, but an anomaly can appear only a few minutes later, affecting specific use cases. That is why you must define, well before writing the pipeline, a list of key indicators that describe your application's health, then monitor them continuously after each deployment.
The essential metrics include: (1) Error rate, generally the most critical (how many HTTP requests return 5XX?). Common threshold: alert if > 1% for 2 consecutive minutes. (2) Latency (p95, p99 of response times): rarely changes abruptly after a deployment, but a spike can indicate a database query that drags on or a progressive memory leak. Threshold: alert if p95 > 200ms (to adapt to your business; a real-time system could be stricter). (3) Instance availability (unhealthy hosts in your target group): if the health checks fail, the instances crash or return 503s, the infrastructure is broken. Threshold: alert immediately if a host becomes unhealthy. (4) Resource utilization (CPU, memory, disk I/O): stable performance is a good sign. If CPU suddenly spikes, an infinite loop may have been introduced. Threshold: alert if average CPU > 80% for 5 minutes (adapt to your capacity).
Beyond these technical metrics, integrate business metrics suited to your context: number of successful transactions per minute (if you are a payment service), number of search requests per second (for a search engine), event aggregation rate (for an analytics platform), etc. These metrics often reveal a functional anomaly faster than a technical metric, and they justify an immediate rollback if they collapse.
To automate the rollback, you configure CloudWatch Alarms that encapsulate these thresholds, then you link them to actions: SNS notification (alert the team), Lambda function (trigger a rollback script), or direct configuration in CodeDeploy/CodePipeline (stop the deployment if an alarm fires). Imagine a canary: you have configured an alarm "Error Rate > 2% for 1 minute." During the canary, 10% of traffic is routed to the new version. If this alarm fires, AWS CodeDeploy stops the deployment and switches the affected instances back to the old version. A Lambda can also be associated to execute additional actions: notify Slack, open an incident ticket, run post-rollback regression tests, etc.
The duration of post-deployment monitoring depends on your strategy and your confidence. A canary generally lasts 5-15 minutes (observing the reduced percentage), then you accelerate to 100%. A blue-green can take 15-60 minutes depending on your caution. But it is never the end of monitoring: even after routing 100% of traffic, you keep watching the metrics for the following 24-48 hours. Subtle bugs can take hours to appear (e.g. a memory leak that accumulates during the usage peaks the next day). Using CloudWatch dashboards that compare current metrics (post-deployment) with the previous day's baseline gives a clear view of any unexpected degradation.
Integration into a GitOps pipeline and deployment immutability When blue-green and canary release are integrated into a GitOps approach, the notion of deployment immutability becomes central. In strict GitOps, you never manually describe what percentage of traffic to send to which version; instead, you describe the DESIRED STATE (e.g. "100% of traffic must go to v2") in a declaration (a YAML file in Git), and the asynchronous system realizes that state progressively. This means your deployment strategies (blue-green, canary) are no longer ad hoc actions, but reproducible, versioned declarative patterns.
In a Kubernetes cluster with Istio or Linkerd, this is expressed via resources of type VirtualService or Flagger. Instead of calling a CodeDeploy script manually, you commit a Git change modifying the traffic distribution (e.g. moving from 'spec.hosts.weight: [0, 100]' to '[10, 90]' for a canary). A controller (Flagger) observes this declaration, detects the change, and orchestrates the canary deployment: it launches the v2 pods, checks the Prometheus metrics, and increases the traffic weight in steps if all goes well. The advantage: the deployment intent is tracked in Git, auditable, and replayable exactly as-is.
For AWS without Kubernetes, the GitOps approach is less native, but it remains possible. You can use a tool like Terraform + AWS CDK to declare the architecture (blue-green or canary) as code, commit this code to Git, and a CI/CD pipeline (GitHub Actions, CodePipeline) applies the changes. For example, a Terraform file declares "the ALB must route 100% to blue"; during a deployment, you modify this file to "100% to green," commit, and the tool detects the divergence and applies it. This turns the deployment from an imperative action ("run this script") into a state declaration ("this is the goal"). The benefits: complete traceability via Git history, the ability to roll back by reverting a commit, and easier compliance.
In this context, container images (Docker) play a key role. Each build produces an immutable image identified by a SHA hash or a tag. Your GitOps declaration references that specific image, not "latest." Thus, deploying a version amounts to updating the reference in the declaration file and committing. No surprises: you know exactly what code is running in production at any moment.
The complete integration of a GitOps pipeline with blue-green/canary: (1) A developer commits code to main. (2) The CI pipeline builds the code, runs tests, produces the immutable Docker image (tagged v2.5.0), and pushes it to ECR. (3) The developer or an automated process updates the GitOps manifest (Terraform, Helm, Kustomize, CDK) to point to the new image and configures the strategy (canary with weight initially 10%). (4) The CD platform (ArgoCD, Flux, Pulumi, or a custom CodePipeline) detects the Git change and rolls out the canary. (5) The monitoring controllers (CloudWatch, Prometheus) observe the metrics. (6) If all goes well, a second Git update raises the weight to 100%. If an alarm fires, a Git revert restores the previous state. The entire cycle is reproducible, versioned, and auditable.
Managing limitations and common pitfalls, then production best practices Even with elaborate orchestration, blue-green and canary deployments run into practical challenges. The most common: managing database schema migrations. If your new version expects an additional column in the database, but your old v1 code (still active in blue) is unaware of its existence, you risk errors. The commonly adopted solution: decouple application deployments from schema migrations. First, you migrate the database in backward-compatible mode (add the column with a default value, without changing constraints) and wait 24-48 hours to ensure the replicas have caught up. Then, once the database is stable, you deploy the new application version. Finally, after verification in production, you clean up the obsolete columns. This approach is called database versioning or the expand/contract pattern.
A second challenge: warm caches. If your application uses a cache (Redis, Memcached), and the new version serializes data differently, the existing caches become invalid. Two approaches: (1) purge the cache at deployment time (simple, but can create a load spike if your cache was huge), (2) accept temporary cache misses and let the new version repopulate the cache little by little. The second is more elegant and does not impact the user directly.
A third pitfall: websockets and long-lived connections. When you switch the traffic of a load balancer (blue-green) or increase a canary percentage, standard HTTP connections are redirected without a problem. But websockets (long-term persistent connections) are different: they establish a long TCP connection that does not close. If you deploy a new version and switch, the old clients connected via websocket stay attached to the old version and do not receive the messages of the new version. The solution: put in place a graceful shutdown strategy where the old instances send a message "I am closing in 30 seconds, reconnect" to the websocket clients, who then reconnect (and potentially land on a new-version instance). Or, more simply, accept that a small portion of your clients experience a brief and transparent reconnection during a deployment.
Best practices to limit these pitfalls and gain peace of mind: (1) Test in pre-production the EXACT same deployment you are going to do in production (same orchestration, same blue-green or canary strategy). Do not discover the bugs during the production deployment. (2) Automate the alarms and rollbacks. Avoid manual operations that take time and are error-prone. An automated rollback in 10 seconds is better than a manual rollback in 10 minutes. (3) Maintain strict API backward compatibility between successive versions for at least two deployment cycles, to allow a temporary mix of versions. (4) Version your application images and your infrastructure in tandem. Never use a "latest" tag in production; it makes rollbacks ambiguous. (5) Regularly clean up old resources (terminated Blue instances, unused Docker images) to prevent the infrastructure from becoming cluttered and costly. (6) Document the manual rollback procedures for the day automation fails. A night team without access to dashboards must be able to return to the previous version in under 5 minutes.
When and how to combine blue-green and canary for a robust hybrid deployment Many sophisticated organizations do not adopt one OR the other, but both in sequence. Here is why and how. A hybrid Blue-Green + Canary deployment proceeds as follows: (1) You prepare the complete Green environment (application, synchronized database, populated cache) with the new version. This Green is the exact mirror of Blue but with v2. At this stage, ZERO user traffic goes to Green. (2) You run an exhaustive battery of tests against Green: smoke tests, integration tests, production traffic replay, load tests simulating 10,000 concurrent users. The goal: to ensure that Green starts and functions under near-real conditions. (3) Once Green is validated, instead of instantly switching 100% of traffic (pure blue-green), you activate a canary: direct 5% of REAL traffic to Green, observe the anomalies detected by this real traffic for 10 minutes. (4) If no alarm fires and the metrics stay stable, accelerate to 50%, then 100%. (5) At the end, you complete the deployment and stop the Blue instances.
This "blue-green upstream, progressive canary downstream" scheme offers several advantages. First, Green is a fully configurable and complete environment, which allows thorough tests and simulations that are not possible with a pure canary (where you deploy directly to production). Second, the progressive canary reduces the residual risk: even if Green passed all the pre-production tests (a false negative), you only expose 5% of traffic in the first minute. Third, you combine the economic advantages of both: Green is maintained only during the deployment hours (not permanently as in a strict blue-green), so the additional infrastructure is temporary and cheap. Fourth, human feedback loops are integrated: after the automated tests but before the canary switch, the team can validate manually, "all is well, approve for canary 5%?"
To implement this on AWS, you orchestrate two stages via CodePipeline or an orchestration Lambda. Stage 1 (blue-green preparation): launch complete Green, run tests, wait for human validation. Stage 2 (canary progression): once validated, CodeDeploy canary starts at 5%, monitors for 10 minutes, then accelerates according to the preconfigured parameters. If an alarm fires at any stage (tests fail, canary metrics spike), everything stops and you return to Blue in a stable state. This redundancy of the safety net makes the deployment considerably more robust than a pure canary approach, while remaining faster and cheaper than a permanent blue-green.
Distributed observability: tracing users and requests across versions To quickly determine whether a problem affects 5% of users (canary) or 100% (post-deployment), you must be able to isolate the requests that went through one version or the other. That is the role of distributed observability, which aggregates the traces and logs of each user across multiple services. Tools like Datadog, New Relic, Prometheus + Jaeger, or AWS X-Ray make it possible to trace a user request end to end, noting which version of which service it passed through. When a canary deployment begins, you configure the tool to add an attribute like 'version: v2' or 'deployment_id: abc123' to each trace emitted by the new version. During the 10 minutes of canary observation, you create an ad hoc dashboard that filters: "show only the traces of version v2 and the associated errors." If you see an error rate spiking specifically in v2, the diagnosis is instant. If the error rate stays identical to that of v1, you have confirmation that the detected anomalies do not come from the deployment.
This distributed visibility is CRITICALLY important for canary releases, because it lets you distinguish signal from noise. Without it, you observe that "an error occurred" but do not know whether it comes from the new version or from an unrelated event (a momentary network outage, a disk timeout, a traffic surge). With properly configured distributed observability, you know EXACTLY what proportion of the anomalies is attributable to the new version, and you can make fast and reliable decisions.
Automating promotion decisions with gates and explicit exit criteria To move from one canary step to the next (5% to 25% to 50% to 100%) or from an observation phase to a full deployment, define EXPLICITLY the criteria that must be met. These criteria are called gates or exit criteria. Without explicit gates, decisions become arbitrary and human ("wait, does it look good?"), which slows the deployment and increases the risk of error. With gates, each progression is mechanical: if the criteria are met, the tool continues; otherwise, the tool stops and alerts.
Examples of robust gates: (1) Error rate of the new version == error rate of the old version (within 0.5%) for 5 minutes. (2) p95 latency of the new version <= p95 latency of the old version + 10% for 5 minutes. (3) No critical alarm (CPU, memory, I/O) has fired. (4) All the health checks of the Green instances return "healthy." (5) Distributed observability: 0 unhandled exceptions logged by the downstream services correlated with the new version. (6) Business gate (optional): number of successful transactions per minute in v2 >= number in v1. Logical combinations (AND, OR) can combine these criteria: for example, "continue if (Error Rate OK AND Latency OK) OR (business gate OK)."
A tool like Flagger (for Kubernetes) or AWS CodeDeploy automates these gates. You describe the criteria in a YAML or JSON configuration, and the tool evaluates them continuously during the canary. Some gates are quantitative thresholds (error rate < 1%), others qualitative (health check == healthy). The advantage: as soon as a gate is violated, the deployment stops immediately. No waiting, no ambiguity, no chance of forgetting to ask someone "is it OK to continue?". For very sensitive changes, you can configure an additional gate that stops the deployment and requests manual approval before progressing (for example, a Slack notification: "Canary OK, move to 50%? Click to approve").
Cost-saving strategies: reducing the infrastructure cost of blue-green while keeping safety Blue-green deployment involves provisioning double the resources. For a massive application on AWS, this can mean doubling the EC2/ECS/EKS cost. If the budget is limited, several tactics reduce this cost without sacrificing safety. (1) Partial blue-green: instead of doubling EVERYTHING (instances, databases, cache), double only the application and share the stateful services (RDS, ElastiCache). The marginal cost drops drastically. (2) Green right-sizing: during the test phase, Green does not need to handle all production traffic; you can configure it with smaller instances, then scale it up just before the switch. (3) Blue-green deployment limited to off-peak hours: if your service has peaks and troughs (e.g. troughs at night, peaks in the morning), deploy Green only at night, test and switch, then you are ready before the morning peak. Green can be terminated after the switch, with no permanent cost. (4) Canary without blue-green: accept the slightly higher risk of a pure canary so as not to duplicate the infrastructure. Compensate by reinforcing the alarms and minimizing the duration of the initial canary. (5) Spot instances for Green: Green does not need stability (you can relaunch it quickly), so use AWS Spot instances (60% cheaper) rather than on-demand instances. (6) Containerization and bin packing: on ECS/EKS, use an auto-scaler to create the Green resources on demand, then delete them once the deployment is finished; this avoids a permanent allocation.
The right balance depends on your context: a startup with a limited budget will accept the slightly higher risk of the canary to save money. A bank will pay the cost of blue-green without flinching. An SME will find a middle ground such as partial blue-green or limited hours.