A solution is no solution if it is too expensive. And if price was not a consideration, most things you imagine could be created. I remember an advertisement from long ago for a water bottle which could create water from air in the middle of the desert. I’ve always wondered what happened to them!
This article is written by our sponsor nautilus2k.com which is a new startup trying to help other startups building data-driven apps on the Microsoft .NET Core/ Azure tech stack.
Engineers coming from big companies have answers to such problems. But the reason startups still fail is because their solution is too big, complex and expensive for startups.
Relational Databases are still relevant in the age of NoSQL because companies still have important information like employees, customers, security, etc which need to be stored in a system which ensures referential integrity & consistency.
While NoSQL databases like MongoDB have made inroads in these areas, it is best to leave them for use cases they are best suited for. The holy grail is integration between best of breed technologies & vendors — not one vendor trying to do everything for everyone, and failing at most of it.
So, what’s the best way to scale up a relational database like SQL Server, MySQL, Oracle or Sybase?
No, it’s not by sharding. SQL is a dead end, because it may have solutions to everything, but beyond a certain cost, these become useless to most SMBs.
Remember, the vast majority of engineers are actually at the level of junior engineers or lower — so the solution must be usable by them as well.
The best way to scale up SQL server is:
- #1 Redirect the read load to a distributed cache
- #2 Redirect the search load to Azure Search/ similar
- #3 Redirect the storage of fast-changing metadata to a time series or NoSQL database which is also cheaper for Big Data.
To do #1, you have to implement a change management service that will keep your cache up to date with the database at all times. This would be a near real-time…
