What is the benefit of doing performance testing earlier in the application lifecycle?


How is your performance?

Many studies have shown that the cost of finding defects in applications increases exponentially the further the defect remains undetected through the lifecycle. Some studies (Microsoft, National Institute of Science and Technology) have estimated that this may be as much as 100 times more expensive to fix a post-deployment issue compared to finding it at the point the error was inserted.

Performance issues are pernicious because, while the application meets the functional requirements, it does not deliver that functionality within the expected (or demanded) time frame. But what really matters with performance issues is that the solution is frequently not correctable with code changes. Too often the remedy involves architectural reformation, changes to the footprint (memory usage typically) or demands for higher and more specific topologies with greater horsepower. This kind of repair is costly to do and can take a long time to effect.

In the case of a performance issue in embedded code it may just not be possible to solve without re-architecting the silicon. How many times have we been forced to upgrade the hardware to achieve reasonable performance from the software?

So performance testing early in the lifecycle is critical if project costs are to be managed. Many tools exist that can simulate load on the application as well as throttling the resources (bandwidth, data transfer speeds, latency) so that realistic expectations of the resulting performance can be observed.

Simulated normal loads are one thing but stressing an application is vital too. Seeing how an application performs under extreme loads will help us understand what the resulting performance envelope looks like so that we can design in ways to balance the load over time. It also helps us to identify where we might instrument the code so that we can give live telemetry from the application about when it is reaching execution limits that are known to cause the application to fall out of the operational envelope.

Load testing, stress testing, performance testing are all essential parts of the test lifecycle that we can do today with very sophisticated and capable tools across a wide range of platforms. With test automation it should be possible to make the code, build and test cycle fully repeatable so that performance issues are detected early enough to do something about them.

About Kevin

In the past year Kevin has spoken at 20 conferences and seminars on a range of leading IT topics, including methodologies, business analysis, quality assurance techniques, governance, open source issues, tool interoperability, release management, DevOps, Agile, ITIL, from the mainframe to distributed platforms to the web, mobile, wearable and embedded systems. He is a much sought after speaker, recognized around the world for his provocative and entertaining style. Kevin is a 40 year industry veteran, holder of three technology patents and today is VP of Worldwide Marketing and Chief Evangelist at leading Application Development and Deployment vendor Serena Software. In the past decade he has been crossing the globe and has met with over 4,000 people. At Serena he works closely with industry analysts, the press, customers, partners and employees to exchange ideas about industry direction and business issues. He was born and educated in the UK and lives and works in the Bay Area, California.
This entry was posted in Business and Technology. Bookmark the permalink.

One Response to What is the benefit of doing performance testing earlier in the application lifecycle?

  1. Jack Martin says:

    Very informative post and it was quite helpful to me. I also wrote something on similar lines on performance testing benefits – http://bit.ly/1ppqqMp

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s