Why data validation is crucial after a migration During an infrastructure migration to AWS, data often passes through several stages (replication, transformation, synchronization) before it lands in the target production environment. Each of these stages carries the risk of loss, corruption or desynchronization. A missing or altered piece of data can go unnoticed for days or weeks in production, causing cascading failures, a loss of customer trust, or regulatory compliance problems (particularly GDPR or sector-specific rules). This is why data validation is not an optional step to be rushed: it must be systematic, exhaustive and formally documented before you consider the migration closed. This validation covers three dimensions: structural integrity (every expected piece of data is present and well formed), business consistency (the application logic rules are respected) and synchronization (the data stays up to date in real time or according to the planned schedule if a cutover window took place). The teams that neglect this phase are the ones that face the most serious production incidents, often a few weeks after go-live when the migration is supposed to be "closed."
Key steps in validating data integrity Validation begins even before the production cutover. During the initial replication or synchronization phase in a test environment, you have the opportunity to compare source and target exhaustively. The first step is counting by table or entity: verify that each source table contains the same number of rows in the target. This simple, fast check immediately reveals massive losses. Next, perform a primary-key validation: make sure that all the expected unique keys are present and that none has been accidentally duplicated by the replication process. Many migrations fail silently because of undetected key conflicts. Then, validate the referential constraints: if table A holds foreign keys pointing to table B, verify that all references remain valid after migration (no orphans, no broken references). Finally, audit a representative sample of rows column by column, particularly for sensitive data (financial amounts, critical dates, customer identifiers): compare the raw values, the formats and the data types between source and target to detect silent transformations or unexpected rounding. If you use a database (SQL Server, PostgreSQL, Oracle, DynamoDB, etc.), each system has native tools to ease this comparison: hashing scripts, exports with checksums, or direct comparison queries. Do not forget the metadata either: column comments, default values and validation rules must migrate intact.
Reconciliation and handling discrepancies after cutover After the production cutover, a critical time window opens: for a few hours or days, depending on your migration strategy (big bang, phased or progressive), the data may diverge between the old infrastructure and AWS. If you opted for a migration with a downtime window, that window must be short and precisely controlled: stop writes on the source, run the final synchronization, switch application traffic to AWS. If you chose a progressive migration or continuous replication, reconciliation becomes more complex: you must identify and handle the data modified during the transition window, particularly the inserts, updates or deletes that occurred at the same time on source and target. A reconciliation tool (a specialized ETL or a custom script) must compare the in-flight data in near real time and flag divergences. The discrepancies detected must be classified: are they expected temporary divergences (in-flight data, delayed arrival due to network latency), or anomalies that require intervention? Each discrepancy detected must be logged precisely, with the timestamp, the affected fields and the direction of the divergence (source ahead vs target ahead). A real-time reconciliation dashboard, even a simple one (an Excel table updated every hour or a script returning a discrepancy count), lets you watch the trend: a steady decrease in discrepancies is normal, while a plateau or an increase demands immediate corrective action. If discrepancies remain after the synchronization window, you face a decision: launch a catch-up replay (replaying the missing mutations) or accept a known and documented loss (rare, but it can be justified if the affected data is non-critical and a manual recovery plan is in place). This decision must be made collectively by the business and technical teams, never alone.
Business-consistency and application-logic testing Beyond structural checks, you must validate that the migrated data respects the business rules that govern it. A migration can succeed in transferring every row intact yet fail to preserve the logic that binds them together. For example, in an e-commerce system, the total stock of an item must match the sum of its variants stored in several warehouses; a migration can break this relationship. Identify the business rules that are critical for your domain: accounting balances (debits = credits), category totals, date checks (closing date >= opening date), range limits (no negative amount where it is forbidden). For each of these rules, write validation queries that scan the entire dataset in the target and raise an alert if a violation is found. These queries must be run as soon as the cutover is done, then repeated daily for at least a week after go-live, because anomalies can emerge as the application processes the migrated data. Also test the application's own critical paths: run complete transactions (an end-to-end customer order, processing a supplier invoice, etc.) using the migrated data, and verify that the results produced are identical to what the old infrastructure would have produced. These deep functional tests are slower to set up than pure SQL checks, but they detect subtle defects that the structural schema alone does not reveal: for example, an interest calculation function that rounded differently, or a third-party service called on every sale that now returns a NULL value rather than an exception.
A strategy for continuous monitoring and post-validation alerting Validation is not a one-off event: it must continue as ongoing monitoring after the formal close of the migration. During the first weeks in production, keep watching data integrity through repeated checks, run on a schedule: for example, every night or every hour, rerun your count, key and business-rule checks. If a check fails, trigger an alert immediately (email, Slack, PagerDuty) so the team can investigate. This continuity is crucial because damage can occur after the migration due to a misconfigured application, a synchronization left active by mistake, or a cron job launched on both infrastructures at once. Set up a visible dashboard of validation status: number of rows in source vs target (with a trend chart), number of business-rule violations, list of unresolved discrepancies. This permanent display reminds the whole team that the migration is not "finished" as long as this dashboard does not show green on every indicator. Gradually, as the weeks pass without incident, reduce the frequency of the checks (from hourly to daily, then to weekly), but never remove them completely: even after three months of stability, maintain at least a monthly audit of critical data. This prolonged vigilance is what separates a successful migration from one that appeared successful at first but failed silently after a few months.
Documentation and formal closure of the validation All this validation work must be documented exhaustively. Create a formal validation document (often called a "migration sign-off" or "validation report") that records: the list of checks performed, the date and time of each check, the results obtained (row counts, discrepancies found), the justification for any tolerated discrepancy (if a discrepancy of 5 rows was accepted, why and by whom), and the signatures of the business and technical owners approving that the data migration meets expectations. This documentation serves several purposes: it proves due diligence if an anomaly is discovered later (you can trace that the problem did not exist at validation time), it eases internal or external audits, and it creates a reference for future migrations of the same type. Store this document somewhere accessible to the team (a wiki, Confluence, a git repository) and archive it after the project closes. Also include a remediation plan for any discrepancy detected after closure: for example, if a count reveals 100 missing rows two weeks after go-live, who would investigate, how and on what schedule? Documenting this process in advance avoids confusion and delays in case of an incident. Finally, organize a post-migration retrospective involving the engineers, the DBAs, the business owners and the owners of critical data, to discuss what worked well and what could be improved in the validation process for the next migration.