Performance Bugs and Anti-Patterns

A performance bug is a programming error that causes significant performance degradation without producing a wrong answer [1]. A performance anti-pattern is a recurring design that causes such degradation and cannot be removed by tuning alone [2].

1. Anti-patterns are design faults

Smith and Williams catalogued designs that make systems slow however well each method is coded [2]. Four are common.

Anti-pattern What goes wrong Example
God Class one class does all the work and asks others for their data a controller that reads a valve’s status and then tells it to open needs two messages where one would do; a controller that asks 20 cart items for price, discount and stock makes 60 calls where cart.total() makes one
Circuitous Treasure Hunt data is found by following a long chain of lookups drawing one engineering model took 6,001 database calls: 1 for the model, 2,000 for its beams, 4,000 for their nodes
One-Lane Bridge all traffic must pass one narrow point a single lock or connection that every request needs
Excessive Dynamic Allocation objects are created and destroyed on every call a service that opens a new database connection object for every request pays the set-up cost each time, and the garbage collector must pause to reclaim the discarded objects; in the paper, service time (the processing time per call) grows with call depth

The Treasure Hunt is today’s “N + 1 queries” problem. For example, an endpoint loads 100 user ids with one query and then each profile with its own query: 101 round trips where one join would do. At 5 ms per round trip that is about 0.5 s spent on network waits alone. The fix is a redesign (one query, or a batch), not a faster database.

2. What empirical studies of performance bugs found

Four studies of open-source bug trackers describe how performance bugs differ from other bugs.

Study Data Finding
Jin et al. 2012 [3] 109 bugs from Apache, Chrome, GCC, Mozilla and MySQL two thirds came from developers misunderstanding the workload or an API’s performance; over three quarters sat in an input-dependent loop or input-event handler; the median fix was 8 lines
Nistor et al. 2013 [1] performance and other bugs from Eclipse JDT, Eclipse SWT and Mozilla 33.9–57.3% of performance bugs were found by reasoning about code, only 5.5–10.4% with a profiler; fixing them was no more likely to introduce new bugs
Zaman et al. 2011 [4] 4,293 performance and 847 security bugs in Firefox security bugs were triaged 3.64 times faster; more than 62% of performance bugs were never assigned to a developer
Zaman et al. 2012 [5] sampled performance bugs from Firefox and Chrome performance bugs needed more discussion, were harder to reproduce, and in Chrome were more often regressions

Three lessons follow. First, a profiler finds the hot spot of a run (the code where that run spent most of its time), but a third to over half of the bugs were found by someone reading the code and noticing work that grows with the input, against 5–10% with a profiler: review for performance, not just test for it. Second, because the typical fix is small, the cost lies in finding the bug, which is an argument for performance budgets (see §3) and early review. Third, since performance bugs are often never assigned, a team has to track them deliberately.

3. Why late discovery is expensive

Anti-patterns are fixed by changing the design, which is cheap on a whiteboard and costly after the code depends on it. Smith and Williams’ Software Performance Engineering therefore models a design’s expected response time before it is built, from estimated execution counts and resource demands [6]. Jewell’s performance budget assigns each component a share of the response time, tracks actual against budget during development, and puts the cost of this work at 1–5% of the project [7].

Practice lags behind. In a survey of 26 DevOps practitioners, only about a third evaluated performance regularly, half spent less than 5% of their time on it, and 88% used no performance model, although about 70% said they would like to [8].

When to use it — and when not

  • Use the anti-pattern list in design and code reviews: look for loops that issue a request per item and for classes that pull other objects’ data.
  • Do not expect a profiler run to replace this review; it shows where time goes in one workload, not which code will grow with the next one.

Limitations

The studies cover open-source desktop and server projects and their bug trackers; bugs fixed before they were reported, and closed-source systems, are not in the data.

How solid is this?

  • Where it comes from. Four empirical studies [3], [1], [4], [5], the anti-pattern catalogue [2] and the DevOps survey [8].
  • What is small. The DevOps survey rests on 26 complete responses, so its percentages are indicative only.
  • What varies. Nistor’s discovery shares are ranges because they differ across the three projects studied.

References

  1. A. Nistor, T. Jiang, and L. Tan,
    “Discovering, Reporting, and Fixing Performance Bugs,” in Proceedings of the 10th Working Conference on Mining Software Repositories (MSR), 2013, pp. 237–246. doi: 10.1109/MSR.2013.6624035.
  2. C. U. Smith and L. G. Williams,
    “Software Performance Antipatterns,” in Proceedings of the 2nd International Workshop on Software and Performance (WOSP), 2000, pp. 127–136. doi: 10.1145/350391.350420.
  3. G. Jin, L. Song, X. Shi, J. Scherpelz, and S. Lu,
    “Understanding and Detecting Real-World Performance Bugs,” in Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), 2012, pp. 77–88. doi: 10.1145/2254064.2254075.
  4. S. Zaman, B. Adams, and A. E. Hassan,
    “Security versus Performance Bugs: A Case Study on Firefox,” in Proceedings of the 8th Working Conference on Mining Software Repositories (MSR), 2011, pp. 93–102. doi: 10.1145/1985441.1985457.
  5. S. Zaman, B. Adams, and A. E. Hassan,
    “A Qualitative Study on Performance Bugs,” in Proceedings of the 9th IEEE Working Conference on Mining Software Repositories (MSR), 2012, pp. 199–208. doi: 10.1109/MSR.2012.6224281.
  6. C. U. Smith and L. G. Williams,
    Performance Solutions: A Practical Guide to Creating Responsive, Scalable Software. Addison-Wesley, 2001.
  7. D. Jewell,
    “Performance Engineering and Management Method — A Holistic Approach to Performance Engineering,” in Performance Modeling and Engineering, Springer, 2008, pp. 29–55. doi: 10.1007/978-0-387-79361-0_2.
  8. C.-P. Bezemer et al.,
    “How is Performance Addressed in DevOps?,” in Proceedings of the ACM/SPEC International Conference on Performance Engineering (ICPE), 2019, pp. 45–50. doi: 10.1145/3297663.3309672.

Disclaimer: AI is used for text summarization, polishing and explaining. Authors have verified all facts and claims. In case of an error, feel free to file an issue.

This site uses Just the Docs, a documentation theme for Jekyll.