Get Control! An Introduction to Process and Documentation Parts 1 - 5 - Connecting the Dots

Part 2 - Common Process Methodologies

Web development and software development are closely related, but distinctly different industries. Although the tools, languages, applications, and skills used by Web developers differ from those used by their software development counterparts, there is great similarity in that both industries rely on proper planning, clarification and understanding of the project to be successful.

In this section, we'll take a look at several methodologies used by both the software and Web development industries, and see what lessons are to be learned from both.

Due to the differences between the types of clients each attracts, these two industries operate quite separately and only rarely compare notes. This is an unfortunate reality for Web developers, because there is so much to be learned from the software industry, namely development processes that are proven to work, and are directly applicable to the Web developer's work.

Remember, the software industry has been evolving since the late 60s -- before most Web developers were born. And, in reality, most Web applications can easily be categorized as a software product with a browser-based client.

Email, Excel, and Word

Microsoft Office is by far the most accessible, most used, yet least utilized project management tool available today. The combination of Outlook, Excel, and Word can be formidable if used correctly, but the vast majority of Office users operate the tools in a perfunctory manner, failing to employ them strategically at all.

By far the most common method of capturing project requirements and specifications is in email threads. While convenient, this technique typically results in disorganized batches of information, haphazardly stored in various inbox folders. It's not surprising that most developers leave all the information in email format, since the majority of communications are via email in the first place. That, however, is a trap. While emails can certainly provide useful reference when organized effectively, project requirements and specifications should always be documented more formally.

Is imperative that all information stored in email be migrated into the appropriate document, to eliminate the need to scour past emails for project information (or at least make it rare). Email should never be used as an 'storage' mechanism for information that belongs in a specific project document. It is, however, a convenient way of passively documenting all project activity and communication over a long time period. Getting the information out of emails and into a more useful document format takes some discipline, but your efforts will be rewarded handsomely once the benefits of a well-documented project start to kick in.

The use of Excel is also widespread – we create spreadsheets for everything from requirements documentation to bug tracking. Unlike Word, Excel has few capabilities that are truly helpful in the realm of Web development, and should be avoided unless you truly need spreadsheet functionality. As a rule, if you're not adding up numbers or creating complex charts with formulae, resist the urge to document your projects in Excel. Exceptions to this include project estimations, budgeting, and other math-oriented applications.

Waterfall Technique

Highly popular until the mid 90s, the waterfall or 'traditional' method provides a robust but inflexible approach to development, in which a sequence of phases is completed resulting in a final launch. Each phase results in a milestone and deliverable, usually consisting of a document and some meetings to discuss the next phase. The project moves from planning, to design, to build, to testing, to migration, to release, and is only actually usable after all the phases are complete.

Although the waterfall technique goes against current thinking by establishing a rigid, inflexible development process, it also provides some benefits. Given the nature of the process, it's much more difficult to encounter scope-creep or other extensions to the project timeline using the waterfall technique. This results in shorter development cycles, but frequently forces development groups to miss out on useful enhancements, redesigns, or quality improvement during the process. The entire team gets one slow-and-steady chance to build the application the first time, and there simply is no second chance.

The drawbacks of this method strongly outweigh, in my opinion, the advantages and the waterfall technique should only be used for small to medium sized projects in which the requirements are crystal clear, and are unlikely to change at all -- a situation in which the waterfall technique would result in the earliest possible release.

However, most developers know how incredibly rare it is for anything to be 'crystal-clear' on day one of a project. So, the potential risks of the waterfall technique should generally be avoided when building Web applications. Such risks include:

• general inflexibility
• lack of early prototypes to prove design concepts and gain client buy-in
• exaggerated and sometimes unnecessary volume of paperwork
• inability to add features once development has begun

Change orders can be particularly tricky when using this technique, as even if everyone is in agreement on a desired change, there is no time to implement it without moving the entire project back.

Russian Style

A reference to the techniques employed by the Russian space program, this development approach is based on the 'prototype early and often' mentality. Contrary to most other methodologies, this approach urges developers to perform a meaningful, but 'short as possible' documentation and planning phase before embarking on a limited prototyping effort.

In practice, there is still considerable documentation involved in the early stages of development when working this way. However, no attempt is made to capture all required information prior to the programming phase. Thus, design flaws and other unpredictable issues are illuminated early and handled accordingly -- all within a highly flexible mindset in which all developers expect to be building the same thing repeatedly, with improvements seen with each iteration.

The early prototypes are obviously not feature-complete and are usually limited to one area of concentration rather than the full-blown application. The learning curve moves into full force early in the project, however, and the developers gain an intimate knowledge of the project and its intended functionality much earlier than they may in other methods. This allows projects with a clear goal (like, 'land on the moon') but ill-defined tasks (like, 'build spacesuit') to experiment and perform research along the way.
Although counter-intuitive to traditional waterfall-method fans, this approach has several benefits to certain types of projects:

1. It's very useful when the specific requirements are unknown and cannot be documented. Prototyping early and often allows the developers to experiment with various techniques without investing heavily in any particular solution.
2. It's effective when used with projects that are so complex that there is little chance of documenting the full requirements unless a prototype is created for purposes of discussion and comparison. If the application is complex enough, it's very helpful to provide the stakeholders with a beta version and allow them to say, 'It's missing this button'. This typically reveals requirements that may not have been considered during a pure-documentation phase.
3. The learning curve of developers is much faster, and knowledge accumulates more quickly within the project team, than is the case with other methods.

The Russian style also has its pitfalls, and can prove to be an expensive and lengthy approach for many applications. As a general rule, if the project manager feels that there is a good chance of capturing the majority of the requirements and specifications during the documentation phase, there is no reason to use the Russian technique. If, on the other hand, it seems daunting to make absolute recommendations for application architecture and platform commitments, some early prototyping might be in order and the Russian style could be useful.

Iterations: Rational Unified Process

Developed by the Rational Software Corporation, RUP is a project methodology based upon the UML documentation system, and supported by a large number of Rational Software tools such as ClearQuest, ClearCase, Rational Rose, RequisitePro, and more.

What is UML? UML stands for Unified Modeling Language. In short, UML is a markup and notation system, which allows software developers, Web developers, and others to create clear, concise, easy-to-understand diagrams and documentation for systems, applications, processes, and more. UML can be useful at any level and even the basic set of flowchart-type icons is very helpful. There are many tools to facilitate UML and ample online and print resources, making it easy to adopt. It's a sure-fire way to increase your documentation quality and save you time. Learn more about UML here: uml.org

There is ongoing debate about the merits of the Rational Toolset, one of the most expensive, complex and heavily integrated product suites available today. Nevertheless, Rational Software continues to be a leader in the world of enterprise level software process infrastructure tools and shows no sign of slowing down. Tools aside, Rational has successfully managed to shift the entire paradigm of software development into a mentality of iterative development cycles.

Iterations are the new mantra of the software industry, and have replaced the rigid waterfall technique as the process paradigm of choice for most programming groups. The iterative approach takes many forms (including RUP) but can also be adapted by individuals simply by planning projects with multiple planning/programming/release cycles. UML has also evolved into a world-standard for documentation and is amazingly useful for large, complex documentation needs.

Ironically, the software process system (RUP) that brought some of the most useful lessons to the world of project management is amongst the most inapplicable to Web development, because RUP and Rational Tools are much too expensive and difficult for most Web developers to use effectively.

XP: Extreme Programming

Extreme Programming offers a streamlined approach to development, and is a self-proclaimed 'lightweight' methodology. It is highly focused on agility and flexibility, with few critical practices and an emphasis on iterations. It's very powerful in the hands of a professional programmer with lots of formal methodology experience, but slightly tricky for self-trained or junior techs. Nevertheless, XP works for many teams and is gaining popularity. XP is debatably geared towards general software development rather than Web applications, but has been adopted by all types of teams throughout the world.

The ideal way to evaluate the XP technique is to visit Extreme Programming and click on the interactive 'map', which visually explains the general project structure. It provides an intelligent approach that revolves around a combination of key documents and processes, and is most suitable for an aggressive, process-oriented programmer.

The Cowboy Technique (AKA"faking it")

Still the most popular technique among independent and boutique Web developers, the cowboy technique usually consists of a simple contract, some kind of specifications document, and that's about it. One the project begins, the vendor uses email and phone to keep things going, gets an occasional signoff perhaps, and just pushes toward the end without a clear path.

Fans of the cowboy technique also tend to underestimate project values, and provide risky schedules with aggressive deadlines.

Invariably, the cowboy technique of project management will result in chaos to the greatest extent possible, as determined by the scope of the project. On a tiny brochure-ware project, there is little to go wrong and the cowboy technique will cause few problems. On a large project, the cowboy technique usually kills the profit margin, burns out the employees, and lets down the client.

There Is No Best Way, Only Best Practices

Having now reviewed a series of popular methodologies, it's important to point out that none of these is perfect, as each project has its own unique needs, which are the product of the project type, client personality, and vendor skills.

The best methodology for you and your clients will be determined on a project-by-project basis, and might involve an adaptation or variation of any of the above methodologies. There are many more to learn from, and most have something unique to offer. In the next part of this series, we'll discuss some concepts that will help you take the standard documents and process systems and adapt them to your needs.