Yes you are comprehensively documenting your agile project--just not in English

By Hillary Johnson

If I were re-writing the Agile Manifesto and I wanted to express agile values in terms that spoke positively to the project manager's realm, rather than defining agile in opposition to it, I'd state the third value ("Working Software over Comprehensive Documentation") like this: The best documentation is the product itself.

How did I get here? We had a fantastic yesterday, delivering our Agile Project Management workshop to 13 quite knowledgeable individuals from all sorts of backgrounds. When the group is this experienced, we always learn as much as they do, and one of my "ahas" came when Lisa Winter expressed a pain point PMPs often experience during agile adoptions, especially when the organization is large and long in the tooth:

What do you do when you're caught between the rock and the hard place? ie, you need to run an agile project in an agile manner, but you also need to provide traditional reporting upstream. Do you make like Allen Stanford or the Mafia and keep double books? Or clone yourself and give the waterfall stuff to your evil twin? I hope not!

What Lisa wanted to know was, how can I cut down on this particular form of overhead? Where do agile metrics map to traditional metrics and where don't they?

So how do these things map? Take documentation. A great deal of confusion surrounds the nature of requirements and documentation in Agile, and it’s often put forth—mistakenly—that Agile eschews documentation. While it is true that in Agile development, working software is favored over extensive documentation, this doesn’t mean you don’t document requirements—it means that you and your team document these requirements quite comprehensively indeed at the end of every sprint--but you document them in Ruby, Perl or C++, not English!

This isn't some crazy cowboy agilista indulgence. In fact, there are industries built upon this notion of product as documentation, aka copyright law. And with the recent extension of copyright law to cover software it's no longer mere analogy. The code itself is the sole document on which basis protection is granted--just as the text of The Shining is the only documentation Stephen King produced to obtain his copyright. This is why I think that, while I heartily endorse it's intended meaning, the agile manifesto's implication that comprehensive documentation is frowned upon isn't an entirely accurate statement--what more comprehensive documentation is there, after all, than working software? And if it's proof enough for Uncle Sam, it might need to be proof enough for the VP of Product Development.

But, you say, this isn't documenting requirements exactly, because it comes after the fact. Does it really? Your first working software will be delivered months, possibly years, before a traditional requirements document would have, so maybe the working software really does fulfill the requirements model after all.

Maybe the answer to Lisa's question lies in translating Agile concepts and mapping them to traditional  values, if not practices.  It might be worthwhile to demonstrate to people who are, quite reasonably, invested in their own histories and bodies of knowledge, that despite radically different approaches, the actual values--quality, timeliness, transparency, value, etc., are the same.

Or am I getting it all wrong by implying that Agile means wrapping your ankle behind your ear to make a point?

Comments

[Denis]

Hilary you sound just like a developer who has been asked to write a user manual ;)

Seriously, unless the developers are exceptional and the code reads like plain english and is layered to support various levels of reading, it is not the best documentation.

Documentation is a projection of the product. It's goal is to communicate something in simpler terms than the code. Agile makes a couple of things clearer: documentation is perishable and documentation is costly to create so you should create only the one that is really useful. If you can generate it automatically from the product, great.

[Michael Dowling]

Hi Hillary,

I work for one of those organizations that are huge and "long in the tooth," and I absolutely relate to the PMO's need for extensive documentation for requirements. Only recently have I been able to document our requirements as tests, in the form of BDD and FIT/Robot tests along with simple business rules for each story. Before doing this, we were, well, writing use cases as a task on our backlog.

The great thing about the BDD and FIT/Robot tests as documentation is that you can *generate* the official documentation from the tests in the dev repository. Imagine: with business, QA, and the rest of the team, you write out business logic in plain text, BDD and FIT/Robot tests to prove the point, and once the tests pass to everyone's satisfaction, generate the documentation as an aggregate in a word doc to appease the PMO. (FYI, this generation is the subject of my new open source project. Boring, I know. But necessary for organizations like mine).

Audit requirements state that the documention must describe *what* the business contracted a developer to create, the design to describe *how* it got done, and change-controls to describe how it got to production. BDD and FIT/Robot tests describe the *what*, generated UML class diagrams and sequence diagrams from unit tests show *how*. I haven't found the right "agile replacement" for change-controls, though done correctly and with the previous 2 artifacts auto-generates, it tends to be less of a headache.

Something to ponder...

[Jay Conne]

Hi Hilary,

You wrote: "I'd state the third value ("Working Software over Comprehensive Documentation") like this: The best documentation is the product itself."

As Denis noted, code is unlikely to be sufficiently accessible for some practical needs. In my team training and coaching, I identify three objectives that documentation must serve to be respectful to your employer as a professional. As with any artifact, start with the question, "who is going to use it and how?"

Those three are:
1) Maintainability - since apps cost more to maintain over their life than to build initially, do what yields cost effectiveness here. If the code and test suite is sufficient, the so be it. Perhaps the restated User Stories and Conditions-of-Satisfaction will help. And then whatever else serves the goal. Note that this is NOT the traditional requirements document.

2) Operations - since typically others need to install and serve the application to the user community including handling any errors that occur regardless of cause - code or environment.

3) Usability - introducing users to the application and supporting their continuous learning, matched to the complexity. Is this a separate doc or is it embedded in the code and help system. The more integrated the better. I would want a clear statement of the context to start. This includes a statement of the purpose, scope and limitations of the app.

Keeping these three in sync as the code evolves needs to be part of one's DONE criteria for any changes just like the test suite and other potential debt (in XP terms).

Hope that helps.

Jay

[Jay Conne]

One additional thought - re keeping double books.

That's exactly what I recommend. Do the right thing and separately do the wasteful thing if that's necessary to keep your job.

Meanwhile track the cost (AKA waste) including the usage of each of the artifacts. This supports management in making evidence-based decisions, to take a term from current medical fashion.

Jay

[Project Management]

Thanks for sharing this informative post