I'm about to start a new data migration project. It's kind of my thing at the moment. I'll basically have to migrate a data model from an Oracle 11g database to a different data model in an Oracle 12c database and somewhere in between will take into account all the business rules and surrounding application ecosystem with all its constraints and challenges.
In my previous project, we were migrating from and to 11g. We used PL/SQL to extract and perform some data preparation. The core data transformation was later performed in a C++ module, from which XML files were generated and finally imported. It was far from being a great solution; instead, it was basically some code rework from a past project.
What about you guys? I'm looking forward to knowing what kind of experiences you had with this kind of project!
Photo credit: renatela (https://www.flickr.com/photos/renatela/1069651852/) via VisualHunt / CC BY-NC-ND
Top comments (2)
Hey Miguel. I'm fairly involved with data analysis and modeling but have never had to migrate data (or take care of a database in any fashion, really). What are the challenges associated with this kind of transition? What keeps it from being completely 1-to-1?
Hi, thanks for the reply!
It would be great if it was 1-to-1 but this project is just one more small step of a longer path. Basically this is a consequence of a merger between two companies and now they are reengeneering, adapting and reorganizing their business processes. In this case in particular I'm migrating the data to a new system where some of the business rules are new or different. This is actually the real and interesting challenge of this project: to be able to execute a migration and ensuring that all keeps working as usual. Basically the number of applications that communicate with the one I'll be migrating is considerable and some of the business rules may impact the ability to migrate some of the data on a given date, so a one shot migration is also off the table. It will surely be an interesting project :)