ETM’S ALI KLAVER talks to KEVIN PARKER about application development  and delivery. You can listen to the podcast here.

AK: Kevin, as Chief Evangelist at Serena you must get to meet a lot of customers in your travels, so what trends are you seeing consistently across the globe?

KP: For the last decade there’s been a huge compression of IT budgets. I was talking to a customer recently who was saying that we’ve got to the point with IT budgets that “the lemon has been squeezed so tightly that there is not a drop left. In fact all you can actually hear are the pips squeaking!”

Our customers are under this enormous compression of the budgets they have available, but at the same time the lines of business are bringing all those ideas forward for IT to implement and create solutions for—and they want them faster than ever.  The whole pace and cadence of business continues to accelerate. We now have tighter budgets and demand for faster time to market.

We are living in the most highly regulated, compliance-required environment we have ever seen. This in itself is piling up on the IT departments and then the business is also asking them to implement cool social media stuff and mashups.

We’re now escalating the technology delivery and complexity, and we’re adding delivery to the cloud, mobile devices and embedded systems and suddenly we’ve got increasing orders of magnitude and complexity in everything we do.

It doesn’t matter whether I’m in the Far East talking to a manufacturing company, whether I’m in Europe talking to somebody in finance, or here in North America in health care—everybody is facing these challenges and it’s got to the point now where there is nothing left that we can do to improve delivery speed.

Remember the things we’ve tried, such as outsourcing—but there’s really not very much more that we can outsource. We bought every package we can so we put every ERP and CRM system in so that we don’t have the development time, and people are now asking what else they can do with less money, especially when “the pips are squeaking.”

AK: We always try to think of ways to deal with these sorts of challenges, so what approaches do you commonly find that organizations take to address the things you just mentioned?

KP: As I travel the world it seems to be an almost universal truth that organizations are all looking to solve the same problems at the same time.

Year on year, as I travel around, there seems to be a focus on the same topics, and perhaps that’s just the fashion. I suspect it’s actually more about maturity and people evolving their IT solutions and their approach to delivering them in a consistent way.

But right now I can tell you that it isn’t the core of application delivery that people are focusing on—it’s the book ends; the front end of the lifecycle, demand management and the back end of the lifecycle, release management.

At the front end there are a lot of people trying to work out how to make sure that the money that they spend in IT is being spent on the right things. So there is an intense focus and emphasis on understanding the demand management part of the lifecycle.

How can they make sure that, out of the business ideas that are coming in, they pick the winners? The projects they select need to move the dial for the business the most with the least amount of investment, so where is that increased ROI from these projects, not only in terms of increasing the return on investment, but also the new ROI which is eliminating the risk of incarceration.

So there is a big focus on the front end of the lifecycle and a lot of customers are trying to get systems in place so there is greater visibility.

In fact, if there were a keyword for what has been going on in 2010 with the customers I meet, it would be “visibility”. People want to see where their projects comes in the queue; where the “waterline” is that IT is setting in terms of projects they’re going to do this year; and how they can negotiate and arbitrate with their peers in other lines of business.

Without a doubt, the biggest focus, and this is universally true with every customer I visit, is that the complexity of delivering applications into production today is so massive that it’s not something that we can hand to humans any longer.

We have an interesting situation in most organizations on a Friday afternoon when the release engineer comes in and shepherds all those deployments through until Sunday morning, makes sure that the code gets in the right place, that all the patches get applied, and that the servers get stopped and started in the right order.

But because of the complexity of applications today, and due to the interdependence between and among all the applications, we’ve reached the point now where it’s beyond the ability of individual humans to do any more.

So most of the customers I talk to are looking for two things; a way to automate the lifecycle process  of release management so they can make sure that the right people sign off on the releases, that people downstream are getting clear visibility into the release calendar so they can smooth out the peaks and troughs in terms of demand for new releases, and then at the lowest level there is an intense focus on making sure that we automate so we get complete repeatability, traceability and predictability for how we deliver software.

We’re now deploying into this heterogeneous set of target environments that have their own intrinsic topological and technology needs, so we have to encapsulate those needs inside technology. Everyone is looking for solutions that can make that happen.

One of the critical drivers of that is ITIL. There was lot of hype around ITIL three or four years ago, and now that it’s gone mainstream and a lot of my customers say that they’re doing ITIL. They’re implementing it their own way and aren’t using all the parts of ITIL out of the book, but essentially following the discipline that comes from ITIL.

Now that there is this focus on managing changes into production, it requires everyone in the lifecycle, both in the production and application of delivery environments, to get much more automated so that it’s possible to seamlessly increase the velocity of delivering to production.

There used to be this sense that release management was a bottleneck and the point at which everything stopped. It was a big grind to get things out into production and that is still true in many places, but the really smart people have realized that if you can increase the velocity and the ability to deliver releases into production with crisp execution every time, you can start to get changes into the production environments that can meet the rapidly changing needs that the business has.

We talked about those a moment ago, and of course with new approaches for software delivery like lean and agile, they absolutely demand that we pick up the pace of change and the ability to put smaller batches of change into production much more quickly.

If you don’t automate it you are going to bury the release engineers with changes and you will grind to a halt. This is  the number one focus customers have these days.

AK: So this is a massive undertaking and I guess there are quite a number of pitfalls that you might have seen in the last few months, if not years. Can you talk about those?

KP: Because this is such a massive problem, people are looking at automating from end to end so they’re looking for one size fits all solutions. The great thing about one size fits all solutions is that you have one common technology usually on one common platform and it’s designed to automate the hand-off from the beginning to the end of a lifecycle.

The problem with these solutions is that in all cases they are designed for an average kind of environment—no customer is an average kind of environment.

A lot of these products are massive and have 1001 different features and functions of which we use only 5-10%. So customers are struggling to make these one size fits all solutions work for themselves.

The other approach is to put all of our artefacts in a single repository, because if they’re all in one place then it’s easy to see what the artefacts are, who’s got them checked out and who’s working them.

What the repository doesn’t do is give you any meta data about the actual state of the project—it just tells you where your artefacts are. So this idea of trying to have consolidation down to fewer tools seems instinctively right, but the pitfalls are that you end up spending an awful lot of money on brand new tools and retraining your people, and then you have to migrate your current dataset into the new repository.

We all know how much we love migration because we can put all of this effort into migration, but we don’t actually get tested on anything. Moving the data over never ends up with a full fidelity so you lose some of your histories, connections and relationships.

You have all of these things in the one bucket so it sounds good, but the reality is that if you are in automotive, the requirement management tools that you need are different to healthcare. If you’re developing software for aerospace your configuration management tool better be industrial strength, but if you’re in retail then you want to have a tool that enables you to have a very nimble infrastructure so you can go at the speed of the business.

You need tools that are specific to your industry, risk profile and development methodology, not one size fits all. Customers are really struggling with this dichotomy between wanting best in class around the different parts of the lifecycle, and wanting all the parts to be joined together.

AK: So no two companies are the same, they all have unique components, and they really do need those unique tools that you were talking about. Let’s step out of what we’ve just been talking about because I know at Serena that you’ve just introduced “Orchestrated Application Delivery”, so tell us what that is.

KP: Actually I think it’s related to the last point, because if you think about it we have lots of very talented people that are very well trained, well educated, professional and highly paid who build software solutions for the business units. But they tend to operate in their own independent silos and have great tools.

The largest part of the problem is not the individual person in the lifecycle, it’s not the individual test engineer who has great testing tools, it’s the way we include the testing part of the lifecycle in the overall lifecycle.

At Serena we are of the view that you have to take your existing infrastructure of great tools to enable you to connect those things together. We often forget that “lifecycle” is really just a synonym for the word “process”, and the idea that the software development lifecycle is about the tools is missing the point completely.

The real point is that software delivery is about the process that we follow, and if we can have a repeatable and automated process. That’s why I think this is the last automation frontier.

It is as though, as IT people, we have been the cobbler’s children these past 40 years. We’ve been out in the business looking at how they run their process and we’ve been connecting their processes together with automation but we’ve never turned the spotlight on ourselves.

All the things that we think about automating as financial business processes apply directly to how we develop software. So making sure there is a crisp and clear handoff from one point in the lifecycle to the next, and making sure that handoff is fully automated, the data that is created in that part of the lifecycle are automatically transferred and moved down the lifecycle.

Let me give you a perfect example of this. Once application delivery starts, the world doesn’t freeze. A requirement may well change in the business and so the business should be able to change that requirement and the process should there so that Whoever is currently working on the requirement, whether a designer, developer or a test engineer, should be notified automatically that the requirement has changed and that they can make an intelligent decision about starting again, ignoring it, getting approval for it and so on.

Whatever the process is it should be fully automated, but for most customers the requirements tool is completely disconnected from the configuration management tool or the test tool or whatever it may be.

Orchestrated application delivery is about taking those best in class tools that our customers have selected to support their very skilled individuals and connecting them through process automation so that there is completely traceable, repeatable, auditable and visible handoff throughout the entire lifecycle.

It is a strange irony to me that the person who has the word “information” in her job title, the CIO, runs her business based on the project management office team going to project status meetings and asking people where they are. Asking somebody where you are on a particular task is not a status, it is an opinion, and when you aggregate those opinions all you get is a big opinion.

We have to get to the point where the tools inform us about where we are in the process, then we’ll understand where our business it, and then we’ll be able to understand how to optimize the process and how to get more juice out of the lemon.

AK: Can you talk about some of the customers who are exploiting this successfully today?

KP: We’re working with a customer right now in the mid-West and they recently realized that the software delivery process they evolved over the last 30 years has reached a point now where the individual parts of the lifecycle are fully optimized.

But there are huge barriers to increase the velocity of change inherent in that system because the points of intersection between the various parts of the lifecycle are largely manual.

For example, in their release management lifecycle they have a document that is easily 20 pages long and has upwards of 200 bullet points in it. The things they have to do in order to prepare for a release, and many of those things are completely repeatable.

They also have what they call their million dollar meetings. These are meetings that happen every Friday from 9-11am with about 80 people on the phone for two hours. The agenda is about 200 items of things that are going to change this weekend and they march down the spreadsheet and basically check that everyone is okay with item 72, for example. There are business stakeholders sitting on the phone for two hours waiting for their item to come up so they can say that it looks good or that they need to delay it another week or whatever else their contribution might be.

They do this every Friday because they don’t have any real visibility into what’s going on. There is no mechanism that aggregates the data that each individual person needs to make an informed decision. If I’m a VP of a business unit the information I need about making an approval is different to if I was the manager of the system of the team that’s going to be using that system.

We’ve got to understand that we can’t carry on in the old way, and this particular client believes fundamentally that if we don’t automate the business process of developing applications we will continue to get way down the lifecycle and discover that we’ve missed something and have to go back to the beginning and do it again.

He believes that the manual handoffs in the process are flawed because they involve humans, and that wherever possible it should be system to system handoffs, system to human handoffs, or human to system handoffs. If it’s ever a human to human handoff then it needs to be automated and visible and trackable in every other way.

This client is also really concerned that wherever possible we do everything we can to make sure that the process is streamlined, and that where there are exceptions to the process that we understand what the exception is and try and work out how we can eliminate them.

The truth of it is that the 80/20 rule makes a lot of sense but when it comes to business processes it’s much closer to a 99/1 law. Out of the things we do, 99% are entirely repeatable and predictable, and the 1% exceptions we just have to handle. This client is hoping that in the first year they are able to save around about $15 million from their IT budget that they can reinvest in innovation or new systems that will continue to make them a world class provider in that particular market.

AK: So that automation and simplicity that you’re talking about can only speak to the bottom line. For my last questions, if you could leave our listeners with one piece of advice from today’s podcast what would it be?

KP: That’s a good question. When I first started out as a programmer 30 years ago I had a wonderful chief programmer, his name was Paul Studd. He said to me: “Kevin, don’t do—document”.

What he meant was that when I was a in a piece of code and found something wrong, I was to document the error, assign he task to myself, and then make the change. Because when you publish that code into production the data center folk need to know what all the behavioural changes are.

In those days, Paul was right, and that was absolutely the professional way to do software development. I want to bring Paul’s mantra up to date and say that now it’s not: “Don’t do—document”; it’s “don’t document—automate”.

We are all in the habit of sending memos out, and if you send a memo out to 1000 software developers in your organization about a change in the process, I guarantee you that 900 of them won’t actually open the email; out the 100 that do read it probably 10 of them will actually remember what it was about; and one of them will actually do something about it.

As humans we are basically recidivists, we keep doing the same things the same way, but if we have an automated process and can see a way of improving that process, then we can go into the process and change it so that the blue button doesn’t appear anymore in this circumstance; or you’re forced to answer yes or no before you transition the task to the next person down the lifecycle.

The short answer to your question, Ali, is that if there is an opportunity to automate any part of the lifecycle—do it. Over time, when you’ve joined all of those pieces of automation of your lifecycle together, you are absolutely going to find that the quality of the work that you do, the velocity at which you are able to do it, and the actual throughput of work you are able to do is going to be much higher.

This is absolutely the way in which we will be able to do more with less in the future.

AK: Kevin, thank you so much.

KP: My pleasure.