Data migration without surprises
By Redaktion techport.ai, IT-Beratung · Last updated on
In introduction projects, migrating the data is the most common reason for delay and at the same time the part that gets started last. The reason is understandable: while the new system is not yet standing, working on old data feels pointless. The opposite would be correct. Data analysis belongs at the beginning, because its result changes the project plan.
A migration is not a technical exercise but a business decision about what comes along, in what quality and in what structure. Only the department can make that decision.
How you notice it
- Migration appears in the plan as one work package shortly before go-live.
- Nobody can say how many customer, article or document records are actually active.
- There is no rule for what happens to legacy data that is not migrated.
- The first test run is also the rehearsal for the real switch.
Why this happens
Old data holds the history of the company, including every special case, workaround and mistyped entry from twenty years. That variety is invisible in daily work because people compensate for it as they read. A migration compensates for nothing. It transfers exactly what is there into a structure defined more tightly than the old one. That is why problems only appear in the test run, and why the effort cannot be estimated without prior analysis.
How we go about it
- Analyse before planning. We record volumes, completeness, duplicates, formats and special cases per data object. The result is a statement of how many records really have to be migrated and where rework is needed. Those numbers set the schedule.
- Decide what comes along. We define with the departments which objects are migrated, at what depth and for which period. History frequently moves into an archive rather than the new system. That includes deciding which legacy data has to stay available for legal reasons.
- Cleanse and test repeatedly. We cleanse in the source system as far as possible and run at least three complete test migrations, with increasing accuracy and with checks by the departments. Every run is logged, deviations are counted and dealt with.
- Make the reconciliation provable. Before the switch we define how success is measured: record counts per object, totals per account, samples following a defined pattern. After the switch the reconciliation is documented and signed off before the legacy system is switched off.
What you gain
- A schedule based on counted data rather than assumptions.
- Markedly less rework after go-live.
- A documented reconciliation that also holds up in an audit.
From our projects
The number of genuinely active records is almost always well below the number present, often a fraction of it. That insight is usually the best news of the whole project because it saves effort, and it only emerges if someone asks what active actually means. The second recurring finding concerns fields that have been repurposed: a comments field that has held delivery terms for years, or a customer number whose last digit denotes a category. Those cases matter to the business and appear in no documentation, only in conversation with the people who work with them daily.
Good to know
Data relevant for tax and commercial law is subject to retention obligations that can restrict switching off a legacy system. Since the Fourth Bureaucracy Relief Act, accounting vouchers and invoices generally have to be retained for eight years in Germany, other documents such as commercial books and process documentation for ten years. What matters is that the data remains machine readable throughout the retention period. A printout is not sufficient. Clarify early with your tax advisers whether history belongs in the new system, in an archive system or in continued read access to the legacy system.
Häufige Fragen
Should we cleanse legacy data before or after the migration?
Before, in the source system, because the departments know the data there and because every correction after the switch has to be checked twice. The exception is structural changes that cannot be represented in the legacy system at all. Those belong in the transformation rules of the migration and are documented there.
How long should we keep the legacy system available?
As long as retention obligations and follow-up questions require, typically between one and ten years depending on the type of data. What matters is reducing access early to read only for a small number of people, so nobody is tempted to keep working there.
Let us talk about Data migration without surprises
In a thirty minute first call we work out where your biggest lever sits and whether we are the right people for it.
Further reading
Back to the field Rollout and Change