How technical conservatism helps us evolve faster and better

At Intercom, we are looking to the future and we are taking bold steps to get there. But when making technical decisions, we like to be conservative.

In practice, being technically conservative means reusing existing technologies and frameworks in our stack, or promoting proven patterns and solutions. We appreciate this familiarity because we understand that the important problems to solve are the ones that bring value to the customer or the company.

Instead of evaluating new technologies and spending time on previously solved operational issues that ultimately bring little value to the customer, we can focus on improving the product by creating, publishing and by iterating through solutions.

This is the sixth article in a series exploring our product principles. Here, Waheed discusses our "Be Technically Conservative" engineering principle.

Technical conservatism has many long-term benefits

This principle is best illustrated by a few examples over the years, demonstrating how "being technically conservative" allows us to scale quickly while ultimately not being a constraint. I've already talked about our experience designing our reporting system, where we evaluated the benefits of introducing a new data store to our stack - Redshift. This would have meant introducing a new type of database into our system that had never been tested in production. Additionally, we should have spent a lot of time building operational knowledge, keeping the clusters in production, and dealing with unforeseen issues with running Redshift at scale.

"We leveraged our existing infrastructure to speed the process from about six months to just six weeks"

In the end, we decided that a familiar data store was better suited to the task. We leveraged our existing Elasticsearch infrastructure, which powers the majority of Intercom's search capabilities, to speed the process from about six months to just six weeks.

The initial version of the reporting system was delivered a few years ago now, so we had a chance to reflect on some of the longer-term benefits of our application of technical conservatism in this case:

We solved the customer's problem faster

With our existing infrastructure, we avoided wasting time learning about a new data store and dealing with all the inevitable setbacks. We were able to immediately focus on the customer's problem to be solved and ship it quickly, reducing the delivery time by more than four months.

We made the most of the team's time

Our data infrastructure team was able to continue to focus on a small set of familiar technologies instead of being spread across multiple technologies. As a result, they had - and still have - more time to ensure the health of our existing systems and optimize our use of each technology.

"Because our set of technologies is relatively small, improvements are regularly made"

We have increased the value of continuous improvements

As our set of technologies is relatively small, improvements are regularly made. The product leverages these technologies, and therefore the impact of these enhancements is compounded on anything built on it. A small improvement can have a massive, positive ripple effect on the entire product.

More teams had more contributions

Using common technologies means more engineers and teams feel confident and empowered to work with them. We have seen frequent improvements to the reporting product from teams across the company rather than just one team owning a particular part of the system.

Remember that principles are not rules, they are guidelines

Principles are an amazing way to align teams and have worked great for Intercom. But there are times when it may make more sense not to follow them. As a business evolves, there is a risk that some team members will dogmatically follow the principles or interpret them incorrectly. Technical conservatism by default doesn't mean we never introduce something new.

"Technical conservatism means favoring a technology that's already in your stack - but only if it's the best option"

Technical conservatism means favoring a technology that's already in your stack, but only if it's the best option. In some situations, an existing technology may not...