How Observability as Code (OaC) Helps Hometree Change its Engineering Culture
About
Founded in 2016, Hometree makes home ownership simple & sustainable by offering a suite of insurance and maintenance products that provide 24/7 breakdown cover on a range of essential household items such as heating, electrics, and plumbing.
The Challenge
The engineering team at Hometree needed an observability solution that allowed them to both validate the adequate performance and ensure the successful delivery of the functionalities their platform offers in the quickest way possible.
" We have faster response times to errors now, thanks to both the alerting and linking of error events, which has reduced downtime. "
Tech Stack
- JavaScript
- AWS Lambda
- CloudWatch
- RDS
- DynamoDB
- API Gateway
Context
Earlier this year, Hometree shifted their attention to resolving performance issues reported by customers. CloudWatch alerts in the form of vague metrics (e.g., number of failed invocations) offered limited value in defect investigation. Despite having telemetry data in the form of structured logs, running queries and searches proved difficult, particularly across different services and squads. Identifying and resolving issues was slow and sometimes took several hours.
Goals
- 1 - Promptly receive alerts for specific defects of applications
- 2 - Quickly troubleshoot and find the root cause of defects in production
- 3 - Cover all services with observability configurations as their architecture scales
Hometree 🤝 Baselime
After connecting their AWS account to Baselime, Hometree was able to identify the root cause of the majority of defects with no additional changes to their code within minutes.
"
Baselime brings together all our telemetry events (Lambda logs, CloudTrail, API Gateway logs, etc.) as opposed to seeing them as individual streams.
We can link together different Lambdas, create alerts, and search through all our events at the same time.
"
- ✔️ Tackle complexity brought by distributed architecture
- ✔️ Obtain accuracy & full context of distributed systems
Moreover, this allowed Hometree to shift left observability in their processes, making it a core part of their application development rather than an afterthought.
"
By switching to Observability as Code, observability and application code live in the same place so we can assess observability coverage faster.
It also facilitates creating queries and alerts when developing features because observability is not treated as a secondary step that happens eventually.
"
- ✔️ Shift left observability in the development cycle
- ✔️ Increase the quality of observability configurations
- ✔️ Deploy in production with greater confidence at a greater velocity
Adoption has also spread beyond the DevOps-minded members of squads. Given that observability configurations are code, developers from all levels of seniority and familiarity with observability are getting involved.
" The team has an improved incentive to set up monitoring in new areas of the architecture since it’s so easy to start and easy to remain consistent with the codebase. "
- ✔️ Effect positive cultural change towards observability
Results
By using Baselime's Observability as Code solution, Hometree both significantly improved its incident response time and incorporated best practice observability across their development life cycle. This cultural shift will continue serving the team in the future to deliver the best service to their customers.
What's Next?
Observability as Code is only the first step towards shifting observability left in development cycles. There is a need for deeper integrations in the current workflows, especially within CI/CD pipelines.
" The big feature we're looking forward to is platform agnostic CI/CD integration for Baselime. I'm excited to see the plans. "