OpenCV Dora Metrics: Impressive Merge and Rework Times; Merged PRs Still Lagging Under 50
9 min read
Isn’t it cool when a car identifies a pedestrian, stays in its lane, or recognizes a stop sign?
Or for that matter
That satisfying moment when your phone recognizes your face and unlocks instantly. Cool! Right?
Yep, that’s computer vision at work! It’s not just magic; it’s technology identifying unique patterns in your face through sophisticated algorithms.
Tools like OpenCV make this possible by empowering machines to interpret and analyze visual data, whether it's facial recognition, object tracking, or motion detection.
What is OpenCV?
OpenCV (Open Source Computer Vision Library) is a versatile and powerful toolkit designed to help machines understand the visual world. With over 2500 algorithms for tasks like object recognition, motion tracking, and image processing, OpenCV enables real-time applications in industries like robotics, healthcare, and automotive. It’s available for multiple platforms—Windows, Linux, macOS, iOS, and Android—and supports languages like Python, C++, and Java.
Whether you’re building self-driving cars, smart cameras, or augmented reality apps, OpenCV is your one-stop library for bringing computer vision to life.
With over 1.5k contributors, 78.6k stars, this magic tool has quite a fanbase to pique anyone’s interest. So, I being that curious one, dug into their software delivery pipeline using my all-time favorite engineering productivity tracking tool, Middleware OSS. I checked their Dora Metrics and got some interesting insights.
What are Dora Metrics?
Dora metrics are key performance indicators that measure software delivery efficiency and team performance through four key metrics: Deployment Frequency, Lead Time for Changes, Change Failure Rate, and Mean Time to Recovery (MTTR).
Also read: What are Dora Metrics? - Master DORA Metrics to Master World Class Software Delivery Processes
OpenCV: Balancing Speed, Efficiency, and Code Quality – Strengths and Opportunities
In the world of open-source software, speed and efficiency are as important as the quality of the code itself. When it comes to the OpenCV repository, their recent delivery metrics reflect a team that’s dialed into these ideals, managing to handle an incredibly diverse workload with the kind of precision that others might envy. However, even with impressive numbers on some fronts, there are areas where their performance could be benchmarked against industry standards to reveal some intriguing insights.
Merging PRs: An Efficient Powerhouse
The 2023 State of DevOps report set a benchmark where elite performers merged PRs within 24 hours. OpenCV’s numbers blow this out of the water, merging PRs in a fraction of the time. This level of efficiency, especially with over 1,500 contributors, is a testament to their CI/CD pipeline and automated testing framework. Quick merges not only mean faster feature rollouts and bug fixes but also indicate a healthy, responsive repository that values developer time.
The Real Hero: Zero Rework Time
While merge times are stellar, what really makes OpenCV stand out is its average rework time—or, more accurately, the complete lack of it. Over the last three months, the average rework time has consistently been 0 hours. Only in July, it was a meager 0.96 hours. This means the code quality and review processes are so stringent that once something gets merged, it’s rarely revisited for further tweaking. This is a rarity in most open-source projects and is a strong indicator that the team’s code review process is both thorough and effective.
The Nature of the Work: Diverse, But Efficient
The work being done in the OpenCV repository isn’t just run-of-the-mill bug fixes; it’s a well-rounded combination of different types of contributions, each requiring a unique approach. For instance, 40% of the contributions are bug fixes, as seen in PRs like “Avoid future integer overflow in _OutputArray::create.” These bug fixes are critical to maintaining the reliability of the platform, and their timely handling is crucial.
On the flip side, 30% of PRs are for documentation improvements, such as the Update orbbec(uvc) tutorial.
Documentation might not always be the most glamorous part of software development, but it’s essential, especially for a repository as widely used as OpenCV. Platform support and optimizations, like the PR to “Fix compilation problem with v_erf” make up another 30% of contributions. The diversity in PR types shows that the repository isn’t just focused on quick wins but is consistently improving across all fronts, from performance optimizations to user-friendly documentation.
Throughput Challenges: Fewer PRs Merged
While OpenCV has achieved remarkable gains in efficiency and contributor engagement, one area shows room for improvement: throughput, specifically the volume of merged PRs.
In September 2024, only 39 PRs were merged, a notable decrease from 57 in July. This drop points to potential bottlenecks, like extended code reviews or resource limitations, that could slow down the flow of contributions.
By enhancing throughput through targeted improvements—such as more streamlined review processes, automated testing, or resource allocation—OpenCV can ensure that valuable contributions aren’t left waiting in the queue.
While the merge time itself has improved, the number of PRs being handled has dropped. When compared to the benchmarks set by elite performers in the 2023 State of DevOps report, which suggests elite teams regularly merge higher volumes of PRs, OpenCV seems to be struggling to scale its PR throughput.
This drop in volume could be due to a variety of reasons, such as the complexity of PRs or a bottleneck in the review process. However, this is an area that could use more focus, especially considering the large pool of contributors the project has. By streamlining this part of their process, OpenCV could align its PR volume with its efficiency, becoming a true powerhouse in software delivery.
A Testament to Collaboration
With contributions from experts specializing in CUDA compatibility, Python bindings, and even hardware abstraction layers, OpenCV has shown it can effectively manage a large, diverse contributor base. PRs like “ix CUDA for old GPUs without FP16 support by Jamim and Add support for boolan input/outputs in python bindings developed by Abdurrahheem demonstrate the wide range of skills being brought to the table. These contributors aren’t just submitting simple changes—they’re tackling highly technical and specialized areas, further underlining the importance of a robust and responsive development pipeline.
Also read: Ceph Repo Dora Metrics: Solid Deployments, but First Response and Merge Times Are Dragging
8 Strategies to Boost OpenCV’s Merged PRs and Keep Metrics in Top Shape
To improve OpenCV's merged PR count and maintain strong performance across all their key metrics, a strategic focus on process optimization and contributor engagement is essential. Here are some specific recommendations:
1. Enhance Contributor Onboarding and Engagement:
OpenCV has a massive pool of 1,500 contributors, yet the number of merged PRs remains below 50 in most months. One way to increase the number of successful PRs is to streamline and simplify the onboarding process for new contributors.
Recommendation: Create detailed guidelines and contributor-friendly documentation that explain how to navigate the PR process efficiently. Incorporate videos or tutorials to guide contributors through common pain points like code standards or testing procedures.
Benefit: This will reduce the number of stalled PRs due to missed requirements or misunderstood processes and potentially increase the volume of merged PRs by empowering contributors to submit high-quality PRs more frequently.
2. Optimize the Review Process
A quick and consistent merge time (under 3 hours) shows that OpenCV already has a robust system for integrating PRs once they reach the review stage. However, the number of merged PRs still fluctuates, suggesting bottlenecks earlier in the pipeline.
Recommendation: Introduce automated PR triaging to categorize and prioritize PRs based on urgency or impact. This will help ensure that high-priority PRs are merged quickly, and contributors working on less critical PRs receive timely feedback to keep the ball rolling.
Benefit: A more structured review process can prevent the backlog of PRs waiting for reviews, resulting in more PRs getting merged without increasing the workload on reviewers.
3. Encourage Smaller, More Incremental PRs
Larger PRs take more time to review and are prone to delays due to their complexity. Encouraging contributors to submit smaller, bite-sized PRs can help reviewers process them faster and make merging smoother.
Recommendation: Set explicit guidelines on the preferred size and scope of PRs to make it easier for contributors to submit small, modular changes rather than large, multifaceted updates.
Benefit: This reduces cognitive load on reviewers and keeps the merge pipeline moving quickly, helping to maintain the PR count while keeping the merge time efficient.
4. Improve Contributor Feedback Loop
Contributors often stall in the process because they wait for feedback on their PRs. A quicker feedback loop can keep contributors engaged and prevent PRs from languishing.
Recommendation: Implement automated feedback tools, like linting and pre-merge checks, to catch issues early and provide immediate feedback to contributors. Additionally, you could set up a dedicated team of volunteer reviewers whose primary focus is to provide initial feedback on new PRs within 24 hours.
Benefit: A faster feedback loop will lead to quicker iterations, helping to improve both the quality and number of merged PRs.
5. Set Contributor Incentives
Open-source contributions can often feel thankless, especially when a PR doesn't get merged quickly. Offering incentives for frequent contributors can boost their motivation.
Recommendation: Introduce a recognition program that highlights top contributors of the month based on the number of accepted PRs, the impact of their contributions, or overall code quality. You could also consider offering rewards, like branded merchandise or profile features, for consistently high-quality contributions.
Benefit: Creating a more engaging and rewarding experience for contributors can help drive more activity in the project, boosting the number of PRs submitted and ultimately merged.
6. Focus on Automated Tests & CI Enhancements
OpenCV’s successful zero rework time is impressive, and maintaining it will be key to long-term efficiency. Expanding automated tests and continuous integration (CI) capabilities can ensure that all submitted code continues to meet high standards while reducing the workload on human reviewers.
Recommendation: Invest in expanding test coverage to handle more edge cases and ensure that all PRs are automatically tested for potential integration issues. Additionally, automated performance benchmarking for PRs, especially for platform support and optimizations, can be implemented to speed up the validation of PRs before review.
Benefit: More comprehensive testing and benchmarking will minimize delays and reduce manual intervention, allowing more PRs to be merged quickly and efficiently.
Also read: Swift Package Manager Dora Metrics: Swift Cycle Times, But Updates Could Be Swifter
OpenCV: Impressive Merge and Rework Times; Merged PRs Still Lagging Under 50
OpenCV's performance metrics, particularly their exceptional merge times and zero rework rate, highlight the efficiency of their CI/CD processes. However, with the merged PR count consistently below 50, there's still room for improvement in optimizing contributor engagement and streamlining the review process. By focusing on onboarding, automated triaging, smaller PRs, and expanding testing capabilities, OpenCV can ensure that its key metrics not only remain strong but also drive an increase in high-quality, merged PRs.
If you find these learnings interesting, we’d really encourage you to give a shot at Dora metrics using Middleware Open Source. You could follow this guide to analyze your team or write to our team at productivity@middlewarehq.com with your questions and we’ll be happy to generate a suggestion study for your repo — free!
Did you know?
OpenCV was originally developed by Intel in 1999 as a way to accelerate CPU-heavy applications, especially for real-time computer vision. It was open-sourced in 2000 and has since become one of the most popular libraries for image processing.
Further Resources