Support Cockroach DB
Summary
Support a DB technology that can reduce DB downtime of the service and allow the DB to horizontally scale.
Proposal
We should support Cockroach DB in addition to Postgres. Cockroach shards and replicates over several work nodes allowing it to be horizontally scalable and have higher availability than traditional single node databases. It also support doing zero downtime upgrades of the database.
Cockroach is already highly compatible with Postgres, it uses the same client drivers and supports most of the same SQL dialect.
Known Issues and Limitations
- GORM automigrations of unique fields breaks Cockroach DB #5752. So you have to create the indexes manually.
- Cockroach DB's implements serializable optimistic transactions. So if your transactions have read/write contention, some of the transactions will fail. The apiserver should retry the transaction using the
crdbgorm.ExecuteTx(...)helper. - There are statement input size limits: 16 Mib : dont't think this should affect us.
Alternatives Considered
- Don't support it - this would allow us to stay more nimble by only having to support one database type, but keeping the sevice up with a high level of 9's might be harder.