Dora Metrics & Pull Requests
🟣 Cycle Time​
Cycle Time measures the efficiency and speed of the development process. It helps identify bottlenecks, process inefficiencies, and areas for improvement. Shorter Cycle Time indicates faster delivery and quicker feedback loops.
How is it calculated?
- It is the time taken from when a PR (Pull Request) is created until it is merged.
If a PR was created on day 1, first reviewed on day 3, and finally merged on day 5, the cycle time of this PR is 5 days.
🟣 First Response Time​
First Response Time refers to the duration it takes for a team or individual to respond to an initial request or inquiry. It is crucial for maintaining good customer or user satisfaction. It is commonly used as a metric in customer support, helpdesk, and communication systems. A shorter First Response Time indicates prompt and efficient customer service.
How is it calculated?
- It is the time taken from when the PR was created to when it received its first review or comment.
If a PR was created on day 1 and first reviewed on day 2, the first response time is 1 day.
🟣 Rework Time​
Rework Time refers to the time spent on correcting and revising work that did not meet the desired requirements or quality standards. It includes activities such as bug fixing, defect resolution, and making changes to address feedback or errors. Rework Time impacts project timelines, resources, and overall productivity. A high amount of Rework Time indicates potential issues in the development process, requirements gathering, or communication.
How is it calculated?
- If a PR had a review which "Requested Changes" on it, then the time from when that review was made till the PR was finally approved and merged is the Rework Time.
If a PR was reviewed with changes requested on Day 2 and finally approved and merged on Day 4, the Rework Time is 2 days.
🟣 Merge Time​
Merge Time refers to the time it takes to merge changes into the base branches since approval. Ideally, merge time should be as close to 0 as possible, since it would be ideal for a PR to not remain unmerged if it's been approved to go ahead. Efficient and streamlined merge/deployment processes can help reduce Merge Time and improve development velocity.
How is it calculated?
- It is the time elapsed between the PR being approved (and hence being ready for merging) and it getting actually merged.
If a PR was approved on day 3 and finally merged on day 5, the merge time is 2 days.
🟣 Lead Time​
Lead Time measures the total time taken from when the first commit is made until the code is deployed to production. It reflects the overall responsiveness of the development process and is critical for understanding the end-to-end delivery timeline.
How is it calculated?
- It is the time taken from when the first commit is made until the PR implementing that task is merged (or until the workflow is deployed to production, as set by the user).
If the first commit was made on day 1, the related PR was created on day 3, first reviewed on day 5, and finally merged on day 7, the lead time for this task is 7 days.
🟣 Deployment Frequency​
Deployment Frequency refers to how often new changes or updates are deployed to a production environment. It measures the frequency of releasing new features, bug fixes, or enhancements to end-users. High Deployment Frequency indicates a fast-paced and agile development process with shorter release cycles.
How is it calculated?
- This requires deployment workflows to be configured (limited release). Deployment frequency is calculated by simply using the count of deployments over a duration, for all codebases assigned to the team.
Example
If a team has two repos: Product-Web and Infra-Platform, which are deployed 20 and 10 times respectively over a period of two weeks, then this team deploys around 15 times per week.
🟣 Incidents​
Incidents refer to unplanned disruptions or events that cause a deviation from normal system behavior, resulting in service disruptions or failures. They can include issues such as application crashes, service outages, data breaches, or performance degradation. Incidents require immediate attention and resolution to restore normal system functionality and minimize the impact on users or customers.
How is it used?
- Incidents are loaded from your Incidents provider (such as Pagerduty, Zenduty, or Ops Genie). It's used to show your Change Failure Rate, or Mean Time to Recovery from an incident, which form key parts of DORA metrics.
Example
A sudden server outage that affects the availability of a website is an example of an incident.
🟣 Change Failure Rate​
Change Failure Rate measures the percentage of changes (e.g., deployments or releases) that fail after being deployed to production. Failed changes may be identified through incidents or alerts from integrations such as PagerDuty or OpsGenie, or through Revert PRs. A lower Change Failure Rate indicates more reliable and well-tested changes, while a higher rate suggests potential issues with testing or the quality of the changes.
How is it calculated?
- It is the ratio of failed changes to the total number of changes deployed over a specific period. Failed changes may be tracked via incident reports, alerts, or reverted PRs.
If a team deploys 10 changes in a month and 2 of those changes result in incidents or require rollbacks (as identified through PagerDuty or revert PRs), the Change Failure Rate is 20%.
🟣 Mean Time to Recovery (MTTR)​
Mean Time to Recovery (MTTR) measures the average time it takes to restore service after an incident or failure occurs. It is calculated from when an incident is created until it is resolved. If incidents are linked to PRs, MTTR can be approximated by measuring the time between the creation of a problematic PR and the subsequent reverted PR, assuming the last PR may have caused the issue. However, using incident providers for tracking MTTR is recommended for more accurate measurements.
How is it calculated?
- If using incident providers, MTTR is the time from incident creation to resolution. For PR-related incidents, it can be the time between the reverted PR and the PR before it. This provides a rough estimate of recovery time.
If an incident is created at 2 PM and resolved at 5 PM, the MTTR for that incident is 3 hours. If a PR causes issues leading to a revert, the MTTR can be approximated by the time between the problematic PR and its revert, though this method is less precise.