As described last week, we have a planned upgrade happening on our databases this week. We have done a number of tests and are hopeful for a quick and non-impactful transition. We also have planned and tested fallbacks as well.
We will be doing the Database Shard (less important database tables) on Tuesday morning, and then the main database on Wednesday morning.
As detailed in the earlier blog:
There should be minimal impact with the whole upgrade process switch taking less than 5 minutes (details below). The final switch is done from Aurora 1 to Aurora 2 will pause writes to the database. This has different impacts to different parts of the system:
- Registration and Ticket Purchases – All of these database calls use a Queuing mechanism (SQS) that essentially will buffer a transaction waiting for the database to become available again. So users signing up for an event will likely see a pause in completing the transaction.
- Event Webpage Views – There should be no impact because these are all database reads (and often come from our cache layer). Aurora will give us the full access to the read only database replicas we use during the upgrade so there will be no impact on people viewing your website.
- Dashboard Updates – These database calls typically do not use Queues as there is less need for reliability. If a Director tries to make an update to something on the dashboard during the transition, they will get an error message.
- Dashboard Views and Reports – Like webpage views, these only read from the database and there should be no impact.
We will also be doing the Analytics database upgrade probably on Thursday. That is a less critical database because it is not used for transactions and only really tracks page views and provides estimated transaction totals by source. We will likely lose a few minutes of tracking information when we do the upgrade as it is not worth the investment to try to have a zero downtime type of switch, so we intend to use the standard Amazon AWS path for upgrading which involves some downtime.
Keep your fingers crossed for us, and we will report the final stats on our blogs.