Technical Debt re-visited

There has been some renewed discussion recently on the subject of technical debt, this has been played out on twitter as well as a variety of blog posts including this one.

The technical debt metaphor is one I use often and I believe that there is some clarification required around this subject.

At it’s broadest, technical debt is any trait of the current system that is considered sub optimal from a technical perspective. The debt aspect reflects the cost of ownership of that trait. For example and overly complex and untidy method is sub optimal, it incurs cost each time it needs to be re-visited and re-understood due to either a defect in the logic or a new requirement. This method obviously incurs more cost if it is re-visited more often.

Much of the debate recently has been focused on the times that it might be good to incur technical debt. The debt metaphor reminds us of borrowing money from the bank to get that new toy a little sooner. The question here is two fold, does technical borrowing exists and how can we view the rate of interest or total cost.

Before we can consider technical borrowing we need to split technical debt into two categories. First, the design or implementation decision that was sound the day it was made but now, with new information, is constraining our ability to progress. Second, the quick and dirty approach that we can see on the day of writing is gong to cost us very soon.

The first category of debt is a direct result of doing the simplest thing that could work and resisting the temptation to predict the future. We develop simple software that is easily changed through effective use of tests, the code that is written is written well. We categorise these design traits as debt when new information presents its self and we identify a change that will enable the system to serve us better. We may decide to make the change or not based on the ongoing cost of maintaining an design that no longer reflects our understanding of the problem or technology.

The second category is often the result of a deadline looming. A common chorus of “lets hack this in now and we’ll fix it later” is coupled with the nagging concern that later never arrives. In my mind, this category of debt can give the first category a bad name. But, is this thinking sound? Surely there are times when the wrong technical solution is the right business decision. It is my belief that there are situations where a quick fix is the right decision, imagine a major flaw that brings down a trading system. However, there are too many instances where code bases become un-maintainable through a feeling of time pressure that leads developers to incur debt with little or no benefit. In fact, in most cases, untidy, poorly designed software incurs cost both short and long term. In the short term, defects delay the release of the software and in the long term software is difficult to maintain and rigid in the face of changing business need.

So we arrive at technical borrowing. The decision to borrow must be explicit. As a team, along with appropriate stake-holders consider the decision. For example,

we have selected framework X for the next release, it does not offer the scaling flexibility we would like but we believe that our early entry to the market is a higher priority. We plan to move to framework Y in the next release window to achieve our target scaleability.

There are less extreme examples of refactorings that should be done but won’t fit in this iteration. Again, be explicit. Discuss the need for refactoring with your Product Owner or Project Manager, consider the cost of the refactoring, the cost incurred by not doing so, the risk of refactoring. One approach to maintaining a healthy code base is to keep a backlog of refactoring and allocate time in each iteration.

However, beware the temptation to save time, get it done, do a quick fix or tidy it up later. In almost all cases code that is hurried, poorly designed, overly complex or untidy will incur far more cost both short and long term than the time saved today by not doing it right.

Finally, this can be most difficult in a legacy code base where technical debt abounds. I use a simple motto that has served me well with a few teams:

Do no harm!

Where you cannot fix it all be sure to agree, as a team, that you won’t make it worse. In fact agree to making one small aspect of everything you touch better for the next person.

Lean at starbucks (lean without the social side isn’t lean at all)

Valtech UK Blog

Valtech now have a blog that aggregates from my own blog and those of a number of colleagues.

There are some smart guys on there and more have promised to contribute so I suggest you take a look and stay in touch as more consultants are added.

Tweetboard up and running

I’m experimenting with tweetboard. Take a look here if you’re interested or follow the link above.

Drop me a line if you visit, if only to let me know if the thing is working

What is Agile?

I met with a potential client last week and found my self answering the question, “what, in a nutshell, does agile mean?”.

To date I’ve never had a canned answer to this and can come from various directions perhaps talking about the history and how it’s an umbrella term for a whole bunch of things or maybe talk about what it’s like working on an agile team. On this occasion I was quite pleased with the answer that fell out:

Agile is about delivering business value!
Businesses tend to exist in a state of constant change so if we want to deliver something valuable to them then we need to be willing and able to embrace that change and deliver quickly.
In order to deliver quickly and deal with change we have found that quality is a necessary. This is particularly true if we want to continue to deliver over a sustained period.
At the bottom of this stack, underpinning all the elements mentioned so far is the need to treat workers with respect and empower them to do the right thing. Without this you wont achieve the quality or speed that you need.

So:
Value <- Embrace Change <- Quality <- Respect & Empowerment.

Is there any more to it?

To SCRUM or to Kanban? – Which one fits.

SCRUM and Kanban (as applied to software development) are different. Proponents of the two drift from staunch opposition to declaring Kanban as a special case of Scrum.

I have been practising and coaching Scrum for a number of years and I delight in the effect it has on project teams. While I practice Scrum I find that Lean Thinking can both help me communicate Scrum and help teams to become more effective in applying Scrum. This way of thinking led to me becoming interested in Kanban a year or so ago since when I’ve being most concerned with where I ought to recommend Scrum and where Kanban.

Recently I have spent time with maintenance groups for whom Scrum would be a poor fit. Kanban offered me an alternative when the client was leaning towards a ScrumBut implementation.

There are some fundamental differences between early project / product development work and the later stages and I don’t see this as a problem. Toyota themselves hold two separate approaches. The Toyota Production System (TPS) is the most widely known Lean Manufacturing implementation. This is the world of work cells, standardised work, Kanban boards and Andon lights. However, when Toyota want to create something new they use the Toyota Product Development System. This approach focuses on a chief engineer holding a vision and rallying a team around him. High bandwidth communication, discovery and experimentation are the order of the day.

So back in a software development context Scrum was designed to manage uncertainty and complexity, to foster high band width communication and seems a valuable fit for early development.

Corey Ladas in his ScrumBan Book declares the following Axiom for applying his application of Lean to software development

Axiom 2: It is possible to develop any value adding increment in a continuous flow from requirement to deployment

Corey makes it clear as he progresses that this well defined workflow is critical to his desired approach. In a new development that workflow is often not known and usually not linear. Rather the team swarm over the requirement often returning to the customer for feedback before offering a completed deployable feature.

My suggestion is this, Scrum is a powerful approach to managing uncertainty as well as being a supportive environment for building a team. I believe that one goal of product development should be to reduce the uncertainty and evolve the workflow. As adding features becomes more straightforward we may find that we are in a state akin to production and can take advantage of a Kanban approach to further optimise for reduced cycle time, increased throughput and just in time scheduling.

Prioritisation with MMFs

For the majority of agile teams user stories are the preferred tool for requirements management. User stories provide numerous benefits including; a focus on the business problem being solved, a just in time approach to specification, amenability to splitting that supports working in short time-boxes.

The problem with user stories in practice is that the desire to work in short cycles drives teams to split stories to a degree that they can loose sight of the value element from the perspective of a customer or product owner. This is often alleviated through themes or story hierarchies but maybe there is a better way.

The minimum marketable feature (MMF) provides a connection between the high level vision of the project and the detailed user story.

According to Software by Numbers a Minimal Marketable Feature is:

… a chunk of functionality that delivers a subset of the customers requirements and that is capable of returning value to the customer when released as an independent entity.

I have found that this construct offers a powerful approach to assessing, understanding and prioritising feature sets. I believe that the benefits of considering MMFs applies to both software projects and product development activities. Due to the focus that is placed on the need rather than the solution this technique continues to apply where we consider projects with little bespoke software.

In essence given some form of need we must pursue it’s decomposition repeating the challenge; “is the described feature or feature set really so minimal as to be un-deployable should any aspect be missing”. We gain further benefit by asking; “is there any aspect of this specification that is over specified”.

I guess at this stage an example is called for. Imagine for a moment that I am a traditional retailer considering gaining a web presence. I have a vision to develop an online presence in order to increase revenue by attracting a wider customer base.

I can begin assessing my need and decomposing features while ensuring that the resultant chunk remains marketable.

MMF 0
Help prospective customers find out about the business through basic online presence.

MMF 1
Enable online purchasing of restricted set of products
Offer a single form of payment

MMF 2
Broaden product range

MMF 3
Increase payment options

MMF 4
Make special time limited offers

MMF 5
Provide vouchers to encourage repeat custom

I am still experimenting with this approach, however, it seems that as a product increases in maturity MMFs can reduce in size (probably to the point where stories would be adequate). It is early in the process where we can get the most value.

In my experience release planning is an area of weakness for many organisations. Releases are large for a variety of reasons including funding, deployment effort and fear of not getting what you really want. I believe that an aspect of this is that once assigned a project it’s team want it to live on independent of the rest of the organisations portfolio. Applying an MMF approach should ease the job of portfolio management since MMFs have a very tangible value proposition.

Managing WIP with Scrum

Recently I’ve mentioned, both on groups and blog posts, the fact that many limiting Work In Progress in reserved for those taking on Kanban Software Development.

In this post I’d like to share a couple of the techniques / approaches that I use in managing WIP within a Scrum framework. These tools may be useful to you in order to improve an existing Scrum team or as a step in moving towards Kanban (see also Scrumban by Corey Ladas).

Daily Management

Members of an agile team should be empowered and trusted to work on the right thing. This may be helping out a colleague or fixing a broken build as well as building new features.

In order to encourage the right behaviour I propose the following filter to new teams. When looking for the next task to work on:

• Is there anything I can do to assist a colleague in moving a high priority task to Done?
• What is the highest priority task on the board that I have the skills to take on?

This has the effect of keeping the task in progress to a minimum as well as weighting the teams effort towards the top of the board (i.e. towards higher priority stories).

Some of the teams I have coached have had a code review column that development tasks must flow through, reviewing some code would be one way of assisting in getting to Done.

The effect of this approach is to minimise the stories in progress, maximise the chances of completing the higher priority stories in the iteration, encourage swarming over tasks and stories.

Project tracking

Many Scrum teams use burn down charts. Consider using a burn up chart, this provides visibility into changes in scope as well as room for further improvements as you’ll see in a moment. By tracking two lines representing scope and done we can our progress towards some milestone. In the example below blue shows the known project scope and green is work done by the dev team.

It is common to do some early analysis work before an iteration to ensure that the acceptance criteria of a story are known before the sprint planning meeting. We want to ensure that we don’t over invest in this analysis work retaining a healthy buffer of stories ready for development.
This part of the scope that has been analysed can be shown on our burn up chart, yellow in the example below. Here we can see where analysed work is building up.

Of course if we are doing this we may also add an analysis column to the task board along with a ready for development column.

Another category of work common in Scrum teams is post iteration testing. After the demo we hand off work to a QA or UAT group that validate the requirement has been met. By recognising this work we continue to expand our view along the value stream.
Below I show in red the work that has been accepted by QA. This demonstrates a problem in the system which was not made visible by the original chart. We can see that work that is developed but not tested is building up. If we care about WIP and cycle time then this is a problem, also if we care about project success we should consider this to be a build up of risk.

A final line could be added to show the deployment of software. A delay in deployment can mean missed revenue earning opportunities as well as a missed opportunity to learn and react.

I use these type of charts (Cumulative Flow Diagrams) as an aspect of project reporting in order to expose a more complete view of the development process beyond the confines of the Scrum team.

Also, where applicable I will augment the task-board with these extra states to show more project context to the team.

My first open space

I ran my first open space the other week.

What is open space?

For those that haven’t experienced it this is an approach to running meetings or conferences that leaves the agenda open for the participants. The way I have seen this applied is for attendees to do a very short elevator pitch regarding a subject for discussion at the beginning of the day. They then place a card on a large wall plan of the day indicating time slot and location of the session. This free form approach allows for small group discussions to be held in parallel. Throughout the day attendees move from session to session selecting subjects of interest from the board. Sessions tend to be run as open rooms so attendees are free to drift between sessions if they wish.

With some trepidation I requested that time was put aside in the schedule of my companies quarterly gathering. As you would expect I had some concern that no suggestions would be forth coming and the days schedule would have holes in it. In fact, attendees came up with a wide variety of ideas for discussion and all concerned appeared to enjoy the day.

If you have considered running an open space either for a local user group or at your place of work I can recommend it highly. I believe that the key to achieving active participation in a work environment is to be clear that sessions are informal, thus minimising prep work, and enable many sessions to run in parallel, enabling small group discussions on niche subjects.

Agile coaches gathering

Last weekend was the agile coaches gathering in Bletchley, UK.

This was a small gathering of practising agile coaches. The event was run as open space with the subject matter focussing more on coaching than on agile.

The image below is the open space schedule, while details are sparse this may give you an insight into the variety of subjects being discussed.

I’ll try to offer some of my take away thoughts following the day below.
Read the rest of this entry »