How We’re Building a Data-Driven Software Engineering Culture
“When you can measure what you are speaking about, and express it in numbers, you know something about it.” - Lord Kelvin
Hi there! My name is Jorge Arturo, I’m a Senior Software Engineer at Appian, and I’d consider myself a data-driven person. The above quote is one of my favorites. It sums up the power of measurement and knowledge, and something I’ve carried with me in my work.
At Appian, we are building a data-driven culture by embedding metrics in what we do and using data to drive decision-making and maximize our impact.
Since I joined Appian in April 2021, one of the most amazing things I’ve found is how everyone seems extremely committed to both self-improvement and improving the product. A particular manifestation of this is how people embrace the many learning opportunities offered throughout the company, like access to 3,000+ free online learning courses, presentation skills training and tuition reimbursement.
Qualitatively, this translates into better practices being followed and better decisions being made. However, what about the quantitative aspect? Well, turns out there is also a strong commitment to this as well.
Using data to drive design decisions/
Due to the intangible nature of software engineering, I’ve found that quantitative data can be a very powerful tool to characterize software processes and artifacts, as well as drive design decisions.
While this is a key aspect of engineering in general, this approach is not as frequently found in software engineering as it is in other engineering disciplines.
To generate this data, software products must be instrumented with tools that will store, aggregate, manipulate, and help visualize it.
Many types of data elements can be collected, such as simple counts of specific actions and traces. This emphasizes the correlation and latency of operations over time and various forms of logging which are invaluable to investigate incidents and debug processes.
Prioritizing measurement as a habit, not an afterthought.
Very early in my career, I learned the importance of instrumenting software for data collection. I very deliberately sought to inject metrics collection into my work and present the data in useful dashboards, making it easier to understand. In many organizations, the instrumentation of software artifacts happens as an afterthought or in response to specific incidents with little standardization, if done at all.
However, at Appian, I have seen this data-driven attitude with teams of engineers dedicated to leveraging these tools and using the insights to improve the engineering experience.
This type of metrics-driven mindset needs to extend to all operational aspects. Engineering resources are scarce and expensive so deciding how to allocate them optimally requires the use of data rather than hunches.
How we improved efficiency in ticket triaging.
One example of how this data-driven culture works in action is how our team recently changed our ticket triaging system to be more effective and efficient.
Our team was tasked with identifying possible conditions that increased the time between when an issue was reported and resolved. We began by categorizing several types of tickets and measuring the spans for each transition.
We identified that one particular type of ticket with a longer time for completion was related to broken builds. This typically happens as a side effect of a commit that alters the conditions expected by a particular job, even in areas of the code that are not seemingly related to the change that triggered the issue. These tickets are created automatically and handed over to Appian Quality Engineers for triage, but this operation can take time, and be expensive in terms of human involvement.
However, we also observed that in many cases (as many as 50%) these tickets could be closed in an equally automatic fashion if successive automated tests showed them to be resolved. This was usually because the Developer became aware of the issue and provided a suitable fix, so attention by the Quality Engineer would have gone to waste.
The process could then be optimized by automating this data gathering so that triaging could proceed more efficiently. The effects of this improvement were also quantified and reflected in an easy-to-read dashboard. With that kink out of the way, the data pointed to further opportunities for improvement creating a virtuous cycle.
What it takes to be a data-driven culture.
Creating a data-driven culture is not something that happens overnight. It takes a while for it to become a habit and at Appian, we’re moving in the right direction while being committed to continuous improvement.
In my previous experience, the beginning of this process tends to be a hard sell since it’s perceived as deviating resources that could be otherwise employed in the core product features. But once you get enough people on board by promoting some of the quantifiable benefits, it becomes easier and easier to get more buy-in. That’s because once you have some metrics to share, stakeholders wonder how life could have been possible without them!
Read more stories about working at Appian on our Engineering and Product team Careers blog.