Building software isn’t like building a bridge. This is not new thinking, this is well understood and has been a factor of software development since the 70’s.
When building a bridge someone has a vision that is part creative and part engineering on how the bridge should be built. The engineers then take this vision and run calculations, build models, run stress tests and so on to make sure that the design will stand up in the real world. They then produce a detailed schedule of work on what needs to happen each week to get the bridge built – there is no deviation from the plan, no changing your mind about how the bridge should be implemented and, beyond the initial vision and problem solving – very little creativity.
Building software on the other hand is a creative process. Developers who write code can approach a problem in a number of different ways, some good, some bad, some passable – but all creative, drawn from their bag of experience and knowledge. Yes, it’s engineering, but I propose it’s also extremely creative.
The case for up-front-design (but not BIG up-front-design)
I’ll even go so far as to argue that planning a bridge is easier than planning software – we already know every little detail about the bridge design up front, we know that design isn’t going to change and we know it will take 4 hours for Dave with a spanner to attach two I beams together, typically it takes 3 hours to weld the plates that make a section, but sometimes that weld is poor, so you build in a contingency factor and so on – easy. (Aside: for any bridge builders out there who just happen to have stumbled on this post, I’m sure it’s really not that easy!)
Planning and managing software development on the other hand – well, I’m an advocate of SCRUM and agile approaches, sure, and that’s a great way to run a project, but I’m not so sure that it’s a great way to start a project from the beginning. XP would have us diving straight into code, SCRUM we’d already have a backlog to pick some items off ready to start building in iteration 1, so I guess this post is about how that backlog get’s filled to start with – an iteration 0 - just diving headlong into code without first understanding the holistic picture of the app you’re going to build, the pieces that will make up the whole and the general stages you need to go through is a recipe for disaster on a project that lasts say 2 years with 3 workstreams (custom dev, business intelligence and data migration).
Also, I’m a consultant and I generally work with a team of consultants from my company (often with other developers supplied by the client or with an offshore development team) and these people cost money – it’s extremely rare for a client to engage say 4 consultants at £1,000 a day without having some sort of idea of how long we’ll take to deliver and how much it will cost.
So, I’m sorry XP purists, but you can’t escape the fact that we need up front design.
Up-front-design in perspective
I’m not saying we need big thick, 000’s page long manuals with detailed conceptual, logical and physical designs down to sequence diagrams for individual methods, no, but we do need to;
- Gain insight into the overall application surface
- Identify the pieces of the application – essentially the sub-applications within the whole – we’ll call these domain components.
- Flesh the high level use cases/user stories for each domain component.
From this the project can be planned into iterations and roughly estimated – the estimate won’t be 100% but it will be an indication and a yardstick to measure change by. In other words, if the customer during a sprint decides to change tack on part of the application, its quite easy to see that adding that extra work to the next iteration will make something fall off the back end of the project and either not get delivered or will cost more. The exercise of defining the high level project will give you a good idea of how best to structure delivery in iterations and let you populate a backlog so you can see the effect of change and re-plan accordingly during the execution of the build.
Alongside the identification of the big picture, the application architect will also be considering the approach to development – will the user want a rich UI interface, will it be a desktop application, a web based application, maybe SilverLight would be a good idea – should the business logic be distributed and exposed as services – will this application expose itself to other applications – how will we handle workflow – should we surface through a SharePoint portal? - all these things will become clearer as the high level design above is being done, but at the same time, the architect will be making some technical decisions that are based on assumption and are therefore risky for the success of the project – we need a way to mitigate that risk, less we head off in the wrong direction only to find that the assumption was wrong in 10 months and have a lot of re-engineering to do to fix it!
Projects need iterations but they also need phases
All of this indicates what we already know - that projects need phases. They need this start up phase to identify the big picture, estimate the whole and de-risk technical assumptions, but what are the phases of the project from there? Well, if you subscribe to any of the methodologies like RUP, Open/UP, MSF etc, you will already be familiar with project phases – the list below describes the phases of a project how I personally like to work (but I’m not precious or religious about these in any way – call them whatever works for you!).
- Inception
- During this phase you identify the big picture – application surface, domain components and drill into use case titles/user stories
- You also identify the proposed architecture for the solution
- Build a proof of concept to start to de-risk some of the technical assumptions
- Deliver
- 30,000ft map of the entire application
- High level use cases
- Iteration plan
- Proposed costs and timescales
- Use case detail, wireframes and specification detail for the first elaboration sprint.
- Elaboration
- This phase we start to work in iterations, using SCRUM to manage those iterations.
- Each iteration delivers the use case detail (building on the high level headers), lo-fidelity screen wireframes and other pertinent specification detail for the next iteration.
- De-risk any additional technical risks
- Build a fully functioning vertical slice through the architecture and problem domain
- Construction
- SCRUM sprints/iterations through the build of the product.
- Stringent quality control – continuous builds, nightly build released to testing.
- Continue to provide design detail 1 sprint ahead.
- Transition
- Releasing the product – a release sprint.
- In some cases you can’t release at the end of each 2-4 week sprint – the application may be too large and what you can build in an iteration too small.
- Instead each iteration should deliver demoware – a presentation or demo to some of the users.
- Some applications can be extremely complex involving multiple deployment steps, these should be planned.
- Support
- Handing over to the people responsible for the upkeep of the system.
Next
In future posts I’ll cover some of these phases, how we go about running the inception phase for example, key things to do in workshops with customers, and how best to start the process of identifying the big picture.
To conclude though, I will say this - yes, the above is applicable to larger sized projects. If you’re writing a simple expense tracking application for the sales department or a personal journal application for yourself then yes, go pure agile and start coding. If you’re working in a larger team on a larger project, you’re going to have to put some effort in up front I’m afraid to get the project rolling – you can run the project as SCRUM from then on and get all the juicy benefits from it, whilst also understanding the bigger picture, budget and timescale implications.
