What should we measure?
Flow Metrics show the state of the product, how quickly our work flows through the system, whether it’s likely that features will be completed by a date, and what kind of work is being prioritized.
They are tied to business value and provide us with the feedback we need to make decisions about the next steps for our product.
Perhaps more important, Flow metrics are based on data, not forecasting, and have consistently been shown to be more accurate than forecasting techniques.
The key metrics are:
- Flow (Work In Progress) Load: How much work is in our system
- Cycle Time: The average time that stories take to complete
- Throughput: How much work we typically complete in a defined time (week, sprint, quarter etc.).
- Flow Distribution: What kind of work is being prioritised (Features, Risk (including Tech Debt), Compliance & Bugs)
- Flow Efficiency: How much wait time we have in the system
Flow Load: How much work is in our system → How long it will take to get to Done
Why it matters: “The single most important factor that affects wait time is capacity utilization.” – Dominica DeGrandis
If the amount of Work in Progress is greater than your team size (taking pairs & mobs into account), then you won’t be able to predict the delivery date of the work. Look for bottlenecks where work is getting stuck, and focus on creating flow there.
How to see it: You can either simply count the items in progress per week, and chart this over time, or use the Cumulative Flow Diagram that is generated automatically by Azure DevOps and Jira.
Some ways to improve: We want our WIP to be just a bit smaller than the team size (including Pairs). If work is queuing in one area, this is a Bottleneck; get the whole team to look at the best way to unblock items in Wait status.
The most effective way to clear excess WIP is move to a pull-based approach, only taking in the top priority stories and working on those till complete, before picking up new items.
This flow-based approach means more work gets completed at a higher quality.
If our stories per sprint are higher than our throughput, this is usually an indicator of high WIP as well. Even with the best of intentions, teams discover new requirements during the sprint, or have unexpected bugs etc. One way to accommodate unforeseen work as a (historically based) percentage at the start of the sprint; the other is to ensure a level of clarity (eg. using BDD) during the refinement ceremony.
Cycle Time: The average time that stories take to complete → How likely we’ll meet a date
Why it matters: Cycle time uses our historic performance to allow us to forecast average delivery dates of stories. It’s more reliable than velocity, as it is objective data based on actual duration that includes the realities of wait times, dependencies, rework etc. that we tend not to include in our forecasts, rather than being based on guesses about the future which are seldom accurate.
How to see it: Azure DevOps and Jira calculate this automatically, you just need to define the parameters.
- Define what your start date is: Usually dev teams work from the point that they start backlog refinement; some teams work from when an item is brought into a sprint.
- Define your end point – this should be ‘ In Production’ but for teams where this doesn’t seem feasible use LCT Complete / Tested in PPE etc.
Some ways to improve: The biggest challenge to Cycle Time is unaccounted waiting time.
Wait time increases wherever we have bottlenecks – a Value Stream Map exercise calculating Value Adding time (item being worked on) vs Wait time (giving you Total Time) will show where your longest area of delays are, i.e. bottlenecks to reduce. Both Jira & Azure DevOps can be queried to show when items are being worked on, and the total time.
Also, If you’re not using ‘Production’ as the end date, you may be hiding other bottlenecks in the overall flow that could be improved.
Throughput: How much work we usually complete in a defined time (cadence) → How many items we can expect to be able to do
Why it matters: If the number of stories per sprint is greater than our Throughput, we are creating a bottleneck in the system which will actually slow down our capacity to deliver work, so it decreases productivityinstead of increasing it.
If our sprint stories match our historic Throughput, we should be confident of completing work within the defined time.
How to see it: Throughput is simply an average of the number of items completed in the defined time. If you use Sprints as the defined time, Azure DevOps & Jira should show this automatically. For other time periods, you can write a quick query to calculate this using the Sprint definition, and show it in a widget.
Some ways to improve: The fastest and most effective way to improve the Throughput: WIP ratio is to move to a PULL based system: only take on work when the system has capacity, don’t allow queues to build up, get all ‘waiting’ work moving and to complete. This increases predictability and allows us to respond effectively to change. You can also reduce queues by reducing bottlenecks; these two often work hand-in-hand. The most common appraoch is simply to increase capacity – while this can have an overall improvement, it is the least predictable.
Flow Distribution: What kind of work is being prioritised → What we should do next
Why it matters: This metric helps us to prioritize upcoming work. Flow Distribution is an indicator about the health / state of the product. It helps us understand what kinds of priorities are currently being focused on, and what we should focus on next to maximise customer happiness: delivering features they need while mitigating risk.
This metric is very context dependent, for example:
- If only Features are being worked on, you may be ignoring risk or compliance items, which carries it’s own risk – or you may have addressed them early already, in which case you might be moving very smoothly.
- If there is a large proportion of bugs, you may have tech debt items that need to be prioritized.
- If most items are Risk / Compliance, you may have a deadline that you have to meet
How to see it: This is a count of the different kinds of work completed during your time period (e.g. a sprint / Quarter), shown as a percentage (100% stack bar is very effective)
For this you need to categorise the work items, and then insert a widget in (both Azure DevOps and Jira have Stack Bars) displaying the data over time.
Typical Categories include Revenue Generation, Revenue Protection and Failure Demand.
- Risk (including Compliance)
- Tech Debt
Some ways to improve: Each product will have a different balance these work types at different levels of product maturity, and when facing external demands (new regulations; product competition; technical changes etc.).
Since this is a very context-specific metric, reviewing the Flow Distribution for the last few quarters along with priorites for the next quarter, can introduce healthy conversations about the next work coming up.
Reviewing Flow Distribution during a quarter is a useful way to see whether your aims for the quarter are being met; and adjust near-term priorities accordingly.
Flow Efficiency: How much wait time we have in the system → If there are lots of bottlenecks / obstacles
Why it matters: This metric tells us how much we could improve our delivery speed. And calculating this by co-creating a Value Stream Map, reveals us where best we could focus our attention to improve our effeciency.
The theory of constraints tells us that any system is capable of delivering at the Throughput rate of its weakest link.
How to see it: Flow Efficiency is Value Added time (the time that an item is being worked on) divided by the Total Time. The average flow time in organizations is 15%; this can go as low as 5% or up to about 40%.
To calculate yours:
- Do a Value Stream Map exercise showing each step the work goes through, from the customer requesting an item to the customer receiving it, and the average time* spent in each phase.
- Add up the Value Adding time (the time that an item is being worked on) and Total Time
* If you have data, use data! Jira & Azure DevOps will give you this detail for time in development, but you may need to estimate on the rest of the work.
This 12 minute video explains Flow Metrics really well: what they tell us, and how we get to them: https://www.youtube.com/watch?v=uBEZoXc4A5w
Slides from Dominica DeGrandis: Agile2019_Dominica_Flow_metrics
It’s also my first public talk about agile education and the work we’re doing at codeX :D
The slides are up on slideshare, and Lukasz Machowski did a great writeup of the talk: Can Agility Change the World? – Notes from Scrum Gathering.
At codeX we’re developing a breakthrough education model to address the skills shortage and the digital divide, using our experience training agile teams.
We believe in changing the future, and this is a story about what we’ve learnt about agility, diversity and making real change.
Quick post for the slides from my talk at AgileIndia2014.
The talk is in three sections:
– How do we generate new ideas?
– Motivation at Work
– Rethinking the Retro as a creative tool
Slideshare link: Building Creative Teams: Motivation, Engagement & Retrospectives
What does neuroscience tell us about facilitation?
In Methods & Tools’ Winter 2012 magazine I explore how neuroscience supports facilitation methods, and use this to make a stab at categorizing facilitation activities according to different levels of interaction.
The journey into neuroscience was prompted by trying to explain why agile facilitation methods work so well. While there is much evidence that they do, there is very little rationale as to why they should. Looking at how our brains process information provides fascinating insight into the great results they generate.
Update: here’s a direct link to the article: Agile Facilitation & Neuroscience: Transforming Information into Action
The end and beginning marked by the New Year provide a useful point to stop and reflect on our daily lives.
I don’t subscribe to the ‘give something up’ approach to New Year’s resolutions. But I do like to identify a theme for development – a kind of big-reaching goal that has a year to unfold. Of course, the hard part is identifying which areas of change are significant and supported, and what we can do with them. Not unlike a retrospective, really.
Powerful Questions are tools for exploring our world from a new perspective. Using subtle shifts, they help us to see our situation differently, and open our minds to options we might otherwise miss – which makes them inherently creative.
I was flicking through my set on New Year’s Eve and noticed that a few were particularly fitting for a personal retrospective. These gently thought-provoking questions provide great insight for an annual reflection and planning session:
- What new skill can you learn?
- Who can you partner with?
- What’s incongruent?
- What brings the most value?
- Who wants you to succeed?
- How do you get your energy?
- A small game is safe; a big game is meaningful, exciting. What’s your Big Game?
- What would a simpler way look like?
- What’s your bottom line?
- What’s the theme song today?
- Where is the fun?
- What could be scrapped without loss?
- Where is courage needed?
- What if it’s good enough?
- What offer can you make?
- What’s already working that you can build on?
- Where are you rushing to?
- Where’s your growing edge?
- What’s so important, you’d leave your comfort zone to make it happen?
- What would a wise person whisper in your ear?
Loosely themed, these questions cover:
– What is calling for your attention?
– What’s exciting to you?
– What’s non-negotiable?
– What possibilities are lurking around the edges?
– What themes are present already?
Without stepping into cliché or worrying about resolutions, spending some time chewing on these will help to surface trends and frame intentions, which is a great way to start anything…
Here’s to finding meaningful insights and exciting paths for the year ahead.
The collaboration that underpins agile development depends on strong facilitation methods that ensure that all aspects of development are approached in an informed, focused and inclusive manner.
That’s a bit of a mouthful, but it’s still easier said than done. The titles below are my “go to” books for understanding the breadth, possibilities and challenges of facilitation. Together they cover a wide variety of tools for both the practices and (rather badly named) ‘soft skills’ of facilitation:
- Facilitator’s Guide to Participatory Decision-Making – a facilitation classic, this comprehensive book covers a variety of methods and practices as well as case studies of real scenarios, with particular attention on handling difficult dynamics to ensure the focus remains on collaborative outcomes (Sam Kaner, Second Edition 2007)
- Agile Retrospectives: Making Good Teams Great – mentioned frequently throughout this blog, it’s a must-have that covers facilitation skills for retrospectives, the principle of ‘Inspect and Adapt’ in software development, as well as plenty of activity ideas (Esther Derby & Diana Larson, 2006)
- Gamestorming – A Playbook for Innovators, Rulebreakers and Changemakers – the first facilitation book I bought, which inspired an entire focus shift; covers how facilitation ‘games’ work, an excellent explanation of how metaphor works in meetings, and even basic drawing skills, in addition to an extensive array of interactive games (Dave Gray, Sunni Brown & James Macanufo, 2010)
- Visual Meetings – How Graphics, Sticky Notes & Idea Mapping Can Transform Group Productivity – another great facilitation book, with insight into the role visual elements play in facilitating, and a variety of formats for different scenarios (David Sibbet, 2010)
- Facilitator’s Pocketbook – surprisingly useful little book covering preparation, skills required, varying techniques according to group maturity, as well as some session templates (John Townsend & Paul Donovan, 2nd Edition 2009; publisher’s extract)
- Collaboration Explained: Facilitation Skills for Collaborative Leaders – comprehensive workbook-like investigation into collaboration for high performing teams; tackles facilitation practices from the view of the various agile meetings, as well as the skills required and challenges faced (Jean Tabaka, 2006)
- Agile Software Development – The Cooperative Game – an indepth exploration of cooperative games and how agile principles embrace these for both learning and delivering excellence; includes an interesting addendum on the creation of the Agile Manifesto (Alistair Cockburn, Second Edition 2007)
Books on management, motivation and ‘soft skills’
The more I work with facilitation techniques and practices, the more I think of them as a management style for collaborative organizations. The books below have been instrumental in shaping my perspective, and have been invaluable when facilitating conversations beyond the standard agile meetings.
- The Leader’s Guide to Radical Management – an excellent and extremely readable book which addresses the challenge creative and complex work poses to traditional management styles, by applying the tenets of agility to leadership across industries (Steve Denning, 2010 – youtube overview here)
- Drive: The Surprising Truth About What Motivates Us – a powerful book bringing together a number of studies that ultimately show autonomy, mastery and purpose to be our true intrinsic motivators, and what that means for knowledge workers (Daniel Pink, 2010 – basics covered in this RSA Animate Video)
- Why We Do What We Do – Understanding Self-Motivation – an indepth look at intrinsic vs extrinsic motivation covering extensive research by Deci and his colleagues; forms the basis of sections of Daniel Pink’s ‘Drive’ (Edward L. Deci, 1996)
- Leadership and Self-Deception – an unexpected, honest and refreshing insight into the seeds of conflict, and averting and resolving unnecessary conflict (The Arbinger Institute, 2006)
- Crucial Conversations – Tools for talking when stakes are high – another conflict resolution book, presenting a useful approach for tackling the hard conversations (Kerry Patterson, Joseph Grenny, Ron McMillan & Al Switzler, 2002)
- Brain Rules – very readable and highly informative book on the neuroscience of how we learn, retain information and make connections between concepts, with insight into issues such as why multitasking is a myth and continuous learning is inherent in our makeup (John Medina , 2009)
- Improv Wisdom – Don’t Prepare, Just Show Up – lessons from Improv theatre on developing a flexible and spontaneous mindset able to respond to opportunities as they arise (Patricia Ryan Madson, 2005)
- Complexity: The Emerging Science at the Edge of Order and Chaos – the story of the Santa Fe Institute functions as an informative introduction to the study of complex systems, which has significant relevance to software development (M. Mitchell Waldrop, 1993)
Closing activities form the “future” section of the agenda. Following the Exploring phase, they are focused around the question ‘How do our new insights help us move into the future?’
As with Opening and ‘Exploring-Divergent’ activities, there is a lot of overlap between ‘Exploring-Convergent’ and Closing activities. For me the distinction is the move to a planning phase: establishing a goal to move forward with, and the activities to support it. If the focus of the session is Planning, this could use up to half the allotted time, for others around a third to a quarter.
It’s also important to be aware of the time span available to implement change, and have the group select the most valuable area of focus within that context.
Ranking / Selecting:
Finding the right focus is much more reliable when multiple interests are represented – it’s easier to avoid personal agendas and generates more discussion around what really is valuable and possible. Selection criteria, such as ‘what we are able to do now,’ ‘what fits best with our team objectives’ and ‘what do we have most passion for’, play a significant part in identifying an achievable goal the whole team is committed to.
- Ordering – Simply placing the ideas in priority order on the board is a good quick way to agree on importance – a variation is to have the group do this in silence at first, to identify debate-worthy areas.
- Dot voting – quick technique using scarcity to clarify personal commitment
- Impact / Effort Matrix – timeless sorting method for short / medium / long term plans (Innovgames.com has an online version as well)
- Other decision-making grids: using the framework above, rank items by ‘Risk vs Impact’, ‘Importance vs Urgency’ and so on
- Thirty-Five – great technique for sharing and ranking ideas in one, esp with an added twist of selling each idea as you go along. Works as a quick-and-dirty ranking method for generating and selecting ideas for discussion in groups of 10 and up – not ideal for finalizing critical issues
While it’s generally agreed that we should create SMART goals, it’s hard to find activities that support goal clarification. Esther Derby’s article on Double Loop Learning provides some excellent questions to interrogate the goals for validity; and I use this format in retrospectives:
- Goal Focus – Friendly way to ensure goals are SMART: once you have consensus on a goal, ask the questions: Can we do it? Why is it important? What could stop us from achieving it? How will we know when it’s done? Clarifies that the goal is appropriate for the skills & responsibilities of the team, as well as the time allotted.
Physical interaction tends to be a more effective way to indicate the level of commitment or agreement than a purely verbal response, and is more likely to surface any hesitation, making it easier to clarify the boundaries of what can be achieved.
- Thumb vote – powerfully simple indication of agreement: Thumb up: I agree / support this goal; Sideways: neither agree nor disagree – will vote with whatever the group chooses; Down: I don’t support this goal / don’t think we should do it now.
- Fist of Five – more detailed than a thumb vote, showing degree of alignment
- Vote with your feet – Physical & spatial vote, nice for large groups: for comparative values, have participants stand somewhere along a line indicating their position on an issue. Gets the group moving, shows physical choice of position, and creates a 3D graph of the spread of commitment
- Jump vote – Fun physical ice-breaker type vote in which each member jumps to indicate their level of commitment, enjoyment, intention etc. – ideally for lighthearted topics and large groups. Use a circle format for a quick opening or closing.
This nuts-and-bolts section identifies how to take a new possibility to a new reality, and could feel tiring or exciting – it helps to get this pace right. Again, it’s important to limit the actions to a realistic number.
- Problem Solving Tree – Consistently asking “How can we do that?” generates detailed actions for solving issues (another version here)
- Graphic Gameplan – Identify major steps required for meeting each objective – nice for high-level project views, and a good way to generate collaborative plans; also a nice Exploring activity for focused planning sessions
- Who / What / When Matrix – For simpler goals, generate the action plan collaboratively
I try to close all facilitated sessions with a quick feedback format that allows participants to review the experience, helps me to get to know the teams better, and helps me improve as a facilitator. The higher the trust relationship, the better the feedback, the more trust is built … and so on.
- Plus / Delta or Keep / Change: What was positive, what to change
- Start / Stop / Continue: Quick version of the Exploring game
- Helped / Hindered / Hypothesis: helps frame how well participants were able to contribute (from Agile Retrospectives)
- WIIFM – Review the What’s In It For Me notes created in the Check In phase – gives feedback on whether expectations were met, and generates interesting discussion around what was learned in the course.
Another wrap-up mechanism is sharing individual perspectives; I do these in call-out fashion:
- Three wishes – Some things I’d love to see happen (from Agile Retrospectives)
- 1 Takeaway – One thing I’ve gained from this session
- Temperature Reading – Good debrief technique for longer sessions
- Appreciations – Nice way to highlight contributions that otherwise would likely go unnoticed, and end the session on a positive note (from Agile Retrospectives)
A strong closing session helps to build confidence that the way forward is relevant and attainable. Following thorough Opening and Exploring sections, this creates a reliable process for implementing beneficial action … and repeated consistently in retrospective format puts us well on the journey of effective, directed Continuous Improvement.
Most of these activities come from books, blogs or training sessions I’ve been part of; some I’ve created to meet specific needs. Where I can find attributions they are noted; if you see any I’ve missed, or know of links I haven’t found, please let me know in the comments below.
Exploring is essentially the ‘Present’ phase of facilitation, with two major sections within it: Exploring: Divergent and Exploring: Convergent.
Divergent games feel a lot like ‘Part 2’ to Data Gathering – and I think it is a bit of a grey area: I often find them so closely linked that the two sections can be combined, but sometimes there is value in having them both. Then Convergent exercises consolidate our findings in preparation for moving to the “Future” phase.
As I understand complexity theory in software development, the Exploring section relates to managing emergence, and ‘sensing’ in Cynefin’s Probe – Sense – Respond model. It’s this level of investigation that helps us to see what effects our actions are really having, identify positive and negative patterns that may be developing, as well as highlight unexpected areas of potential.
Here, we want to delve further into issues that are important, extending our understanding by looking through a different lens – of brainstorming, understanding risk, or in-depth analysis.
Generate Ideas / Breakthrough thinking:
I was fortunate to attend a session with Darian Rashied called “Facilitating Creativity for Breakthrough Problem Solving” at the London Scrum Gathering last year. In it, Darian explained how unexpected connections work to generate ideas: things that make no sense keep us occupied, we can’t walk away from them. This means we reach deeper and cross boundaries we would usually stay well within, in order to resolve the senselessness. According to John Medina in Brain Rules, this can even carry through to our sleep, hence the term “sleep on it”.
Using de Bono’s framework, Darian reinterpreted the game phases as follows:
Opening > Exploring (divergent) > Exploring (cohesive) > Closing
Provocation > Movement > Harvesting > Treatment
Provocation: ridiculous, fun, laughing – getting out of the serious mode activates a different part of brain which frees up our imagination
Movement: activities that stimulate mental leaps help us escape our normal, tried-and-tested thought patterns
Harvesting: reaping the benefit of our slightly altered viewpoints by creating space for the ridiculous, accepting and investigating all ideas
Treatment: taking ridiculous ideas and reshaping them back to practical applications
While some of the ideas below may seem whacky, they really do generate at the very least some interesting new viewpoints.
- Random Input: Idea generating technique that works by aligning the problem with the attributes of a random object (played in Darian’s workshop, from Edward de Bono’s How To Have Creative Ideas)
- Brainwriting: Generate ideas and share them, expanding on each others ideas as you go (also in Gamestorming)
- Anti-Problem: Solving the opposite of your problem helps to view the issue from an ‘opposite’ perspective, and promotes both idea generation and a “re-view” of your current processes
- Destruction Plan: Taking the Anti-Problem to an extreme, this game harnesses our apparently abundant destructive abilities to create awareness of tools, processes & ideas we don’t normally consider, then reframe them for positive effect (played in Darian’s workshop).
Traditional Risk Matrixes and Risk Mitigation Strategies tend to fall far short of the mark for the complex work that makes up most of software development. The activities below work well as collaborative approaches for surfacing risks and assumptions, and are really valuable at the start of a project or in a planning phase for resolving rocky ground.
- Speedboat – what’s weighing us down, why we aren’t moving at the speed we expected
- Pre-Mortem – ‘remember the future’ technique – an imagined post-mortem on a failed project – surfaces underlying problems with our current approach (I blogged on this here)
- Ritual dissent – an idea generating-and-then-bashing activity, that helps to challenge assumptions & highlight risks – this requires a high-trust environment and at least 3 or 4 teams (another description of Ritual Dissent)
Avoiding failure is apparently a better evolutionary tactic than building on successes1 and this may be why we find it easier to visualize disaster than success. Whatever the reason, once we’re given permission to identify things that can go wrong, these activities can unleash a wealth of information. Be sure to create safety first and follow on with identifying mitigating actions, so that no-one is left with a sense of impending doom…
1 Mostly from Dave Snowden’s Podcasts discussing resilience and exaptation.
Root Cause Analysis2:
Sometimes we encounter issues that are really symptoms of deeply rooted organizational impediments. This is especially valid for recurring issues, as well as catastrophic events. Here we need to dig deep to unpack the root cause of the problem.
- 5 Why’s: Originally from the Toyota kata, this method uses 5 as a heuristic number of times we need to ask “Why” something happened to get to a level of human cause(s), such as flawed process or incorrect assumptions
- Cause and Effect Diagrams: These diagrams make visible all the ‘symptoms’ that need to be addressed, their apparent causes and their effects, and highlight the reinforcing loops. This is a valuable aid for identifying ‘attractors’ to enhance positive loops, and ‘dampeners’ to weaken the negative – making it possible to make decisions on what should be resolved in which order, and how we expect it to impact the rest of the system. (Ishikawa diagrams are an alternative form of this, which I haven’t worked with)
2 I owe this section to Carlo Kruger’s A3 Thinking session which he presented to SUGSA at the beginning of this month – Thanks Carlo!
These two formats are complete facilitation plans for generating insight from opposite standpoints: a strength-based, imagi-planning approach, and analytic problem analysis:
- Appreciative Inquiry – Highly creative full retrospective technique for generating new ideas from a positive foundation. I find that future-specting is quite hard to grasp for teams unfamiliar with agile games, so for less exploratory teams I’ve replaced this with an outcomes-focused exercise.
- A3 Thinking – Toyota’s methodical ‘total picture’ approach to understanding issues and testing corrective measures. Incorporates root cause analysis techniques into a detailed improvement plan. Great for significant issues, overkill for small ones.
Once we’ve expanded our view, we need to start the converging process, making sense of what we have uncovered. These sorting exercises help to clarify where ideas are overlapping and identify dominant themes and needs. I typically do all of these in a session, with more or less detail as time allows.
Grouping > Clarifying > Interpreting
- Clustering – The process of grouping associated ideas provides a the first step to identifying underlying causes, and also highlights unity and differences in a group
- Identify Themes – Having the team name the clusters of post-its encourages discussion about and agreement on the grouping, providing a nice organic approach to determining categories (from Agile Retrospectives)
- Debrief – Open questions that bring attention to emerging patterns – can be done as a quick round-robin or murmur group exercise (from Agile Retrospectives)
Through exploring our situation we seem to be answering the question ‘What does what we know about the Past tell us about the Present?’. By uncovering underlying themes and discovering experiments yet to be tried, we put ourselves in a position of strength – able to apply our insights in a way that can shape our future.
We take this information into the Closing section to identify specific probes to set up and actions to take that will help us get there.
Most of these activities come from books, blogs or training sessions I’ve been part of; some I’ve created to meet specific needs. Where I can find attributions they are noted; if you see any I’ve missed, or know of links I haven’t found, please let me know in the comments below.