Acceptance criteria in Agile user stories are funny things when you think about it. On one hand we describe our stories as 'placeholders for a conversation' and strive to minimise the documentation that we pass around the organisation in favour of face to face communication. On the other we insist on having a set of clear and unambiguous testable statements written up against each of these stories that we can use to define whether the story is 'done' or not. This could be viewed as something of contradiction. Whether it is or not, I believe, depends on how, and more importantly when, those criteria are created.
There is no shortage of material on Agile software development sites suggesting that a complete set of acceptance criteria should be in place before any attempt is made to size stories or plan sprints. This article from the scrum alliance, for example, states
"I can't overemphasize just how crucial it is to have proper acceptance criteria in place before user stories are relatively sized using story points!"
The comments that follow the article provide some interesting perspectives on the subject, both from other Agile practitioners and the author himself, who adds a significant caveat to the earlier statement
"Teams that have been together a long time can look at a new story and immediately see that it's relative to another story without the need for acceptance criteria"
Another here on backlog refinement recommends a "definition of ready" that includes, amongst other things
The story should be testable (that means acceptance criteria is defined).
I'd disagree with this sentiment. I don't personally adhere to the approach of adding acceptance criteria to stories on the product backlog. I perceive a number of issues around adding acceptance criteria before the story is sized which I think can undermine core agile and lean principles.
A replaceholder for a conversation
My primary concern is that once acceptance criteria are added to a story it ceases to be a "placeholder for a conversation". The product owner, in order to get stories to meet the 'definition of ready' can be pressured to add the criteria to the story themselves, outside of any conversations with the other key role in the team. There is then potential for developers to pick up and deliver the story based solely on these acceptance criteria, without at any point needing to discuss the story directly with the Product Owner or tester. There is a natural tendency when working to take the path of least resistance, and if a set of written criteria exist on the story that the developer can use to progress their work, without having to talk to anyone else, then many will be inclined to do so.
This problem is apparent in both directions. There is also an inevitable temptation for the PO to treat the creation of acceptance criteria as the means of communicating what is needed to the team. This could be seen as a means to 'make sure' the team do exactly what the PO wants, whilst reducing the personal drag that comes from having to have conversations with them. This undermines the primary value of the criteria, in my opinion, which is to capture the consensus reached between the PO and the team on the scope and value of that piece of work.
The purpose of the user story is not to define exactly what we want or how we want the software developed, but to strive to capture the essence of who it is for and why they want it. In the situation where the PO creates the criteria away from the team then essentially all that we have done is to relegate acceptance criteria to becoming a new flavour of up front written specification. In doing so we embed in them all of the risks that are inherent in such artefacts, such as inconsistent assumptions and inconsistent interpretation of ambiguous terms.
We also run the risk of contradicting at least one of the agile principles -
"The most efficient and effective method of conveying information to and within a development team is face-to-face conversation."
I'm sure you will have noticed that I've used terms such as "risk" and "tendency" liberally here. A well disciplined team can avoid the pitfalls I've highlighted by insisting on conversations at appropriate points and collaborating on the creation of the criteria. This brings me on to my second concern with fully defining acceptance criteria for backlog stories, that this practice contradicts the principles of lean development.
Eliminating waste
The first of the seven principles of lean software development, as defined by Tom and Mary Poppendieck, is to eliminate waste. Waste is anything that does not add value, for example work that is done unnecessarily or in excess to what is required. Given that there is no guarantee that every story that is sized on the backlog will be completed then putting the effort into establishing acceptance criteria for all of these is potentially wasteful. Learn principles originated in manufacturing, and one of the sources of waste identified in that context is "inventory". Whilst I do have many concerns with comparing software development to manufacturing, I think that considering detailed acceptance criteria on a backlog story as inventory, in that they are an intermediate work item that doesn't add value and may not get used, does highlight the wasteful nature of this practice. Investment in time in creating documentation that we are uncertain whether or not we want to use is a waste which we should avoid if we can refer that cost until such time as we're more confident that it will add value.
It is also wasteful to have more people than needed in conversations. If the entire team is present in creating a full set of acceptance criteria for every story as part of backlog refinement or sprint planning, then this could constitute a lot of waste as only 3 people are required for these conversations, the PO and the developer and tester on the story, or the 3-Amigos. While the whole team may be required for sprint planning, I don't believe that the whole team is required to elaborate each story in more detail and define the acceptance criteria. Having them do this presents a significant cost that could be avoided.
Waste also occurs in revisiting previous work. When we do progress a user story, I would expect diligent teams to want to revisit any pre-existing acceptance criteria with the Product Owner at that point to confirm that they are still valid and whether any additions or alterations are required. This again introduces an element of waste in revisiting prior conversations if acceptance criteria have been created in their entirety in advance - if no changes are required then we've wasted time revisiting, and if there are then we wasted time initially in doing the work that subsequently had to be changed. This neatly brings me on to my next argument against adding acceptance criteria to backlog stories.
Deferring decisions
The Lean principles of Software Development, as defined by the Poppendiecks, also advocate deferring commitments until we need to - the "last responsible moment". There's good reasons for this. Generally the later we can make a decision, the more relevant information we have to hand and the better educated that decision is.
Whilst user stories in a backlog should be independent, there will inevitably be some interdependence. For example in a database development we might have stories for supporting user defined views, and others around table level security principles. Depending on which feature was developed first, I'd certainly want to include criteria to test compatibility with that first feature when the second was delivered. If I was writing the acceptance criteria at the point of picking up the stories, this would be easily factored in, however if writing criteria for a wealth of backlog items up front then we're not in a position to know which would be delivered first. We would therefore need to introduce conditional elements into the criteria which needed to be revisited later to cater for the status of the other stories.
Not all such decisions will be as visible in advance. There will also be a process of learning around the development of a piece of software which may influence the criteria we place on the success of later stories. By establishing the details on what constitutes acceptable for a story long before the story is picked up and worked on, we're embedding decisions into the backlog long before we need to make them which may prove to be erroneous at a later stage depending on the discoveries that we have made in the interim. I've seen situations where the lessons that we learn from testing certain features can massively influence the criteria we place on others. For criteria embedded in the backlog this could involve significant rewriting should that occur.
Ensuring a conversation
Historically I've never found the need to add acceptance criteria to backlog items. I've always sized stories and estimated backlog items based on the story title and a conversation with the relevant individuals. These individuals, as a group, should have the expertise to understand the value to the user, and the relative complexity of that item to similar items that we've previously developed. Will the sizing be as 'accurate' as if we'd created detailed acceptance criteria, possibly not, however I'd suggest that the difference is negligible and the benefits minimal as compared to the potential wastes that I've described.
So when would I add acceptance criteria to a story? My personal preference would be to add acceptance criteria to a story at the point at which the story is actually picked up to be worked on. This is the point at which the story changes from being a placeholder on a backlog used for sizing and planning, into an active piece of work which requires elaboration before it is progressed. At this point we also have more information about the developers/testers working on the story and can have a more focused conversation between these individuals and the product owner. I describe in this post a structure that I've used at the point of starting work on a story to identify the acceptance criteria and ensure the individuals that are working on that piece have a good understanding of the value they are trying to deliver. If a user story is a placeholder for a conversation then, for me, this is the intiation of that conversation. This is where we reach mutual consensus on the finer details of the user story and establish an appropriate set of test criteria as a framework under which we can assess the delivery. At the same time we can establish what the team feel are the relevant risks that the work will present to the Product, and any assumptions in our understanding that require confirmation activity by the Product Owner. Creating the acceptance criteria collaboratively at this point ensures that the memories of that conversation are still fresh in the mind of the individuals as they are delivering the work. The conversation is the true the mechanism by which we convey understanding and agree on what needs to be delivered, and the acceptance criteria are rightly mere reminders of this conversation. Isn't that so much better than using them as the primary means of communicating this information?
References
- This article provides a great "back story" from Mary Poppendieck on Lean Software Development- http://www.leanessays.com/2015/06/lean-software-development-history.html
- I'm not keen on comparing manufacturing to software development, however some interesting history on the "Toyota Production System" is available on the Toyota Website : http://www.toyota-global.com/company/visionphilosophy/toyotaproductionsystem/originofthetoyotaproductionsystem.html
- One of the best descriptions I've found about how and when to add Acceptance Criteria to user stories can be found here - highly recommended: https://www.scrumalliance.org/community/articles/2010/april/new-to-user-stories
Excellent thought provoking post Adam. Thanks for sharing.
Post a Comment