Performance from Day 1

Performance is important. You can meet all of the requirements, and be completely bug free, but if a page takes 20 seconds to render, the customer won't be happy. As Jeff Atwood wrote, speed still matters. Performance is the functional requirement that every customer forgets to mention, and every developer forgets to ask about. Customers generally just assume the performance will be adequate.

Performance is so important that I suggest we change the standard user story template to:

As a <user> I want to <action> so that <goal> within <performance expectation>

When discussing performance, this quote is often tossed around:

premature optimization is the root of all evil

The full quote, however, is (emphasis mine):

"We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil." - Donald Knuth

Small efficiencies in an algorithm are one thing, but often we go to the other extreme: we make architectural choices that make decent performance impossible.

  • We introduce unnecessary layers, for the sake of architectural purity, which often have to be torn down to get halfway decent performance at the end of the project
  • We don't look out for stupid bugs that lead to common problems SELECT N+1 and memory leaks
  • We don't make provision for the simplest performance improvements, like caching, in our frameworks, so they have to be scattered throughout the code

Most projects end up having a sprint that is devoted wholly to fixing performance in the application. That's not "tweaking 10% extra". It's "making the home page render without 72 SQL queries". It's a sad fact that most performance problems aren't fixed by rocket science micro-optimization, but by undoing dumb architecture decisions.

A simple performance check-list

During the "sprint 0" backlog building stage, there's a few simple questions we should ask the customer:

  1. How many concurrent users should they expect to serve?
  2. Will there be periods of major increase in demand (e.g., Christmas sales)
  3. What is their maximum response time (usually this should be no more than a few seconds)
  4. What costs are they likely to wear as far as server costs, bandwidth costs, etc., so we can keep an eye on them

At the end of each sprint, as a bare minimum, we should:

  1. Measure the number of SQL queries that are issued as we browse the most common pages
  2. Measure the number of network requests (browser->web->app server) necessary to serve a single request
  3. Keep an eye on our memory and CPU usage, and watching how they change as more users are added

This steps won't uncover every potential performance problem, but they take about 10 minutes to do at the end of a sprint, and will uncover the most basic performance problems caused by the architecture, at the best time to fix them. Windows Performance and Reliability Monitor, SQL Profiler, NHibernate Profiler and the "network" tab of your favorite browser's debugging tools are all you really need.

A picture of me

Welcome, my name is Paul Stovell. I live in Brisbane and work on Octopus Deploy, an automated deployment tool for .NET applications.

Prior to founding Octopus Deploy, I worked for an investment bank in London building WPF applications, and before that I worked for Readify, an Australian .NET consulting firm. I also worked on a number of open source projects and was an active user group presenter. I was a Microsoft MVP for WPF from 2006 to 2013.

Ujjaval Suthar
Ujjaval Suthar
29 Apr 2011

Hi Paul,

Very well laid out thoughts about performance.

Just curious, that you mentioned about Donald Knuth. On a side note, have you read any of his book from series "The art of computer programming" completely?

Just asked, in case if I have met a person who has read one of those fully.. :-)

Great post ..

Thanks,

29 Apr 2011

Ujjaval, I have a 3-volume set from his series, and spent a few hours reading it before getting bored and heading back to XAML :-)

29 Apr 2011

Hey Paul,

Nice post and is along the same lines I encourage in my book. The only thing I would add is that it is not necessarily stupid architecture but is often the "current best practice/Blogged/industry favourite" architecture that does it. Quite often it is implemented from a purist perspective without a view to practicality. I have seen this many times, usually in the form of over-engineered architecture that seems golden in theory, but bad in practice.

if your doing web apps. Start with caching (output caching, and query caching) and get used to looking at page event times through trace.axd. Its so easy, it is sad its not done enough.

29 Apr 2011

@Paul, funny you mention this:

The only thing I would add is that it is not necessarily stupid architecture but is often the "current best practice/Blogged/industry favourite" architecture that does it

A friend and I were talking the other day about how a solution can be called "best practice" while performing terribly and being a poor solution. The "state of the art" in enterprise application architecture is over-complicated, purity driven layer cakes.

29 Apr 2011

For performance key point is measure and, we need to build metrics application service first. This service fill up data of application usage. Service raise notification like bug on TFS if something is above form expectations in first day.

Every user interaction with user need to be measure. This service wrap every major component like render, command, repository etc.

Like business users we need to monitor, analyses and act base on this metric data. One day in every week should be a performance day, and every week we can change expectation.

Continue the good work, @Paul

Vijay Santhanam
Vijay Santhanam
01 May 2011

Good post, but perf isn't a functional requirement per se. It's a system quality as most of the literature/standards put it. And there are other qualities too For run-time: Reliability, Usability, Perf, Security/Safety and for non-run-time: Maintainability, Testability, Scalability, Reusability, and many more...

If any of them are important to a success of a product, they should be storied out as well - not just perf.

I guess you're singling out perf cos it's regularly hidden/implicit, but so are many of the others.

Would like to see your Reliability From Day 1, Security from Day 1, etc posts too.