A challenge most of us face at some time when working in software is the pervading fear that everyone else is doing it better than you. It's far too easy to fall into the trap of believing that everyone else is diligently applying the latest methods and techniques while you're stuck fighting the day to day challenges that dominate your time. Spotify are an excellent example of a company whose posts and content imply an purist agile process nirvana, where anecdotal reports from the trenches reveal a far more pragmatic set of working practices that face just the same challenges as everyone else.
The most effective tactic I've found to address the risk of this kind of paranoia is to network with peers. It doesn't take long in a room full of people in similar roles to appreciate that your challenges are no worse than others', and few have the luxury of applying the idealistic techniques of the textbooks and corporate marketing content. I experienced an excellent example of this a lifetime ago (well pre-covid which feels like the same thing) during a product management peer 'un-conference' discussion on Hypothesis Testing.
A False Hypothesis
The format of the 'un-conference' was based on attendees posting their own topics for open or semi-structured discussion sessions - and one of the most well attended was on the subject of 'Hypothesis testing'. My discussions with people prior to the session revealed this as a subject that qualified as an area of paranoia among product people - it was clearly something that folks thought they should be doing. Most product people I spoke to who intended to go to the session fully expected the other delegates to be more comfortable than them with the topic, and wanted to see and understand how others were approaching it.
What actually emerged from the discussion was that here was a technique that clearly more were talking about than were actually doing themselves:-
- Very few product people were adopting hypothesis testing - At least in a formal sense in their day-to-day work. In response to the question "Who here is actually doing structured hypothesis testing?" only about 3 hands went up with any conviction, out of a room of I'd guess between 20 and 30.
- Attempts to implement it hadn't stuck - even when the discussion host claimed to have introduce a structured approach in their product team, details emerged from another attendee from the same company that the team in question hadn't persisted with the approach after the instigator had moved on.
- Few people were clear how to do it - as the discussion progressed it became clear that few were confident in how to go about establishing a structure for product hypothesis testing. Most folks in the room had a good grasp of what a hypothesis was but there was a lot of uncertainty around how to actually construct a formal approach and so few had actually tried anything.
This shape of everyone assuming everyone else is succeeding with a technique while few actually are is one I've seen before. The idea of Product people writing natural language BDD tests to drive automation was hugely popular a few years ago yet everyone I spoke to that had attempted it saw little benefit in essentially asking product roles to write pseudo-code. The reality is that implementing any new structured approach, particularly one that involves adoption and input from other roles, is a significant undertaking that a lot of folks simply don't have time to do.
Do you need structure?
Given how much easier it is to write about how good something is than to actually show it, it's understandable that such situations occur, but where does that leave the majority of folks who don't have the time or autonomy to drive the implementation of new methods?
For me it starts with an understanding that you don't need to implement a formal approach to get value from experimenting with a technique. If you build flexibility into your process then applying new ideas in context can be done in an ad-hoc way when appropriate to the context of the problem at hand.
The discussion in the conference room was based on an assumption that adopting a hypothesis based approach required a defined structure and process to document and process hypotheses and testing outcomes. People discussed hypotheses as a separate entity from stories, spikes and other backlog work. There was lots of talk around where people should document their hypotheses - with spreadsheets and confluence being mentioned. While this may be integral to the work of a devoted UX researcher testing the impact of isolated changes in the user experience, for many product people, having another separate list to focus on may not be practical. It's enough to be prioritising between progressing the feature roadmap, maintenance work, bug fixing and ops/infrastructure improvements without having a separate list of hypotheses to juggle as well.
Just do it
My suggestion to the room was, rather than fight a battle to introduce a new structure into already busy teams - a more pragmatic approach is to introduce work into the existing team backlog to validate hypotheses and test assumptions. In my current organisation we took the approach of creating a backlog item simply called "Work". It's not a bug, or a story (we have those as well), but simply a loosely defined work item of flexible shape. This could be investigating an area of concern, running a technical research spike or gathering evidence to prove/disprove a hypothesis - anything that will take the team forward in knowledge and learning is a candidate. In this flexible way documenting a hypothesis and the steps to validate becomes part of the acceptance criteria of the individual work item, rather than on a separate list somewhere. The learning goes into the 'research' area of the team wiki and the relevant work items are created to follow up on the evidence gathered.
Adding research items into team backlogs in this manner provides a low key, defensible route to incorporate hypothesis testing into a team's existing processes, when time and resources may be stretched. These can be standalone investigations or form part of the initial research to support decisions around a larger development. The approach handily circumvents any accusations of arbitrarily forcing in new processes - after all it would be hard for anyone to argue against taking the initiative to identify and test that your assumptions around a development are valid.
Work works
I've spent a lot of my career arguing that tests don't need to be formally structured and documented to get value from them, and for me hypothesis testing in product is no exception. As with many methods, actually starting small and simple may be a better way for many to start given the barriers that can be thrown up when trying to introduce large changes. Additionally an exploratory approach actually accommodates better for context as exploration naturally shapes itself around the problem at hand.
At the risk of seeming to advocating a method to avoid other methods - I can recommend having a loose backlog item of 'work' (or if you insist on a more specific name, maybe 'research') as an option in your backlog tracking system for exactly this reason. Investigation, testing and exploration can take many forms, and flexibility of tools and techniques in context wins out over formality in my book every time.
Thanks to National Cancer Institute for this Photo on Unsplash
Post a Comment