McMahon, P., Chiorean, M., Coleman, S., & Askoolum, A. (2018, November 30). Bye bye Mongo, Hello Postgres | Digital blog. Retrieved January 4, 2019, from https://www.theguardian.com/info/2018/nov/30/bye-bye-mongo-hello-postgres.
My Top Seven Takeaways
- This was massive, downtime wasn’t affordable, most organizations won’t face such a problem, still they used some interesting approaches for such large data migration that can be useful in other contexts.
- Don’t manage your own database, outsource that at a scale, otherwise you will end spending a lot of effort on it without really providing business value.
- Postgres rocks, under AWS RDS rocks even better, JSONB columns provides a lot of flexibility for your not-so-structured data.
- Current API, even Controllers were all smudged with DB specific artifacts, they had to draw a hard line bumping a major version of the API. A clean, streamlined replica of the original API was built.
- Data migration was accomplished via API, basically duplicating and comparing API calls to the different backends. Very, very nice, and even nicer as they produced structured logs to track down the data discrepancies.
- Instead of updating the API clients, a Proxy was built in front of both versions of the API, this had some strings attached as it was supposed to be a transient artifact so a compromise on its functionality was accepted in the form of some minor downtime.
- “Trying to think of things we could have done to avoid this error, we realized that when starting a big piece of work you also have to accept that you’re going to make mistakes”.
It is a really good article, with a lot of technical insights on MongoDB, PostgreSQL, data migration including architectural