This is the seventh post in a series I am writing to convey lessons I have learned about Data Migration. We started the discussion by exploring your data through a migration lens. We recommended you identify key Business Data Objects, prioritize them, and walk through each using the destination system’s table and field structure as a guide. The next three posts examined larger patterns and processes in Data Migration and discussed how the challenges and opportunities might shape your approach to the project. If you’re just joining us on this journey, feel free to take a few minutes to read up on Knowing Your Data and the Extract/Transform/Load patterns in my previous posts.
We at ArgonDigital are always happy to discuss data migration with you and explore ideas about making your project go smoothly.
In each post, I want to highlight the brilliance and achievements of the teams I have been honored to work with. Over the next few weeks, I want to switch gears a bit to talk about those teams and the way they built technology to drive their very large, complex projects.
Data Migration Is a Fulltime Job
Data migration is a fulltime job.
To put it another way: if you want to succeed in migrating data to your new system, accept that data migration is a fulltime job.
It may be that your project is very straightforward – virtually identical source and destination data models, modest number of records, rigorously maintained data – then you can ignore this advice. But if you’re reading this post, I doubt this applies to you.
Please take the advice: your data migration project may require you to dedicate several fulltime employees. By “dedicate” and “fulltime”, I mean that the people driving the data migration workstream should have no other duties than successfully moving your data from old to new.
- They should not own migration along with another workstream.
- They should not span multiple projects as free-range data migration experts.
- They should not be assigned for only a first phase, or only right at the end.
Trust me from having lived through four years of data migration: there will be plenty for them to do. They will not be underutilized. There will be analysis: every object, table, and field on (at least) two systems; every record on the source system; and every incremental pull, transformation, and load. There will be planning: each phase, each increment, each dry run, and each validation.
Data Migration Requires a Balanced Team
If your data migration project involves any scale or complexity, your migration team will need multiple skillsets and, hence, multiple dedicated people:
- Make sure you have a strong product manager who can envision the holistic end result and has the strong agile skills to guide the team to deliver it.
- Assign an experienced program manager with strong project and process leadership skills who can align and integrate the data migration workstream with the larger system replacement project and the other workstreams.
- Assign a strong technical leader with deep knowledge of either the source or destination system, as well as database skills and experience.
- Finally, ensure there is at least one business subject matter expert with deep understanding of the current system and business processes.
More of each skillset may be needed – especially in the engineering area – but you should assign at least one fulltime contributor of each type to the effort.
Experience with Teams
In the three large migration efforts in which I have served, there has been variation in the commitment each team has made.
- A large media company which was replacing a digital rights system made an enormous commitment from the start. They assigned a director-level product manager, a tech lead with three additional engineers, a scrum master, and a tester. At various times, they added several business analysts to perform detailed assessments, groom stories, and analyze data extracts. The commitment allowed the team to lean into and drive the pace of the overall project. It also positioned the team to support a dramatic acceleration of the project timeline when corporate strategy demanded it.
- A financial services company also committed significant resources, adjusted for the unique circumstances. A long-tenured product manager with deep experience in system migrations led the team, along with a tech lead who had significant engineering experience with the legacy system. Several business SMEs were assigned, given the number and complexity of data objects and processes involved. The destination system vendor supported the migration with a toolkit – including an intermediate environment and loading functionality, and the commitment to add or change functionality according to the client’s needs. Business workstreams were enlisted to test migration increments.
- A separate media company was replacing its in-house digital asset management system with a new cloud-based system built and maintained by a vendor. The project was not initially funded with enough staff to commit a dedicated data migration team as I described above. The initial commitment was limited to one fulltime engineer/analyst with long experience in the source system. He was able to build a tool which would address the critical need to cleanse asset metadata. However, he could not accommodate the many additional requests for functionality in the tool. Moreover, lack of dedicated product and program managers limited the organization’s ability to set aggressive goals for the initial increment, to build a clear roadmap for subsequent increments, and to learn and grow over time.
We’ll pick up the story again in the next post: Invest in Technology!
