Why “Monte Carlo” Forecast?
I get a lot of questions from people, and I always try and respond. Sometimes though my thinking takes too long and I forget, and if that has happened to you, please just re-email me. The question I got via LinkedIn this week was roughly as follows:
“How does Monte Carlo forecasting differ from a team using historical card counts or velocity data?”
This great question gets to the point of what forecasting is trying to achieve, and why team forecasting is different from organizational forecasting.
(If you don’t understand what we are talking about, what Monte Carlo forecasting is, see this interactive article for the low-down)
Average plans fail on average
If we use a team’s historical throughput or velocity data and project that pace forward to predict “when we will be done” (typically burndown or burnup charts) we land on zero (shipped) with the same chance as a coin-toss. Monte Carlo allows us to choose what degree of certainty we think appropriate to communicate using the same inputs. The goal of probabilistic forecasting is to take the variability of the inputs and project a set of answers that show the accumulation of that variability of the result. This solves the problem of giving a delivery date that already suffers from 50/50 chance of failing; we get to choose a higher degree of certainty, like 85% or 95% that better accounts for all sources of things going pear-shaped.
The complexity of considering variability quickly nukes intuition
When teams project historical pace data, that have just one uncertain input - that story count or velocity data. But start adding more uncertain inputs, the scope is uncertain, the growth of that scope is uncertain, the defects that need fixing are uncertain, the pace in December or August versus other months is uncertain, and so on. Considering these without the help of a computer spreadsheet is tough to impossible. The benefit of Monte Carlo and spreadsheets like mine is they add this capability without blowing up our brains with complex math.
All models are wrong - that is the BLOODY point
The other reason for using Monte Carlo isn’t really the Monte Carlo part, but that using a spreadsheet model means you consider more factors and can spot when things are going pear-shaped earlier. Upon spotting that delivery isn’t going as expected means you now have a set of inputs to ascertain what guess was wrong. Is scope changing, did defect count through the roof. Models help us see “oh crap” earlier when there is time to do something about it. I like to teach that the models help have the right conversations earlier because they say “Nope” earlier which causes discussion as to what needs to be done to make a “Yep.”
TLDR; Spotting forecast and reality diverging earlier is the key capability. Earlier identification means a faster reaction.
One team’s predictability doesn’t mean product delivery predictability
The more we scale organizations the less relevant any single team’s work has on overall delivery. Dependencies between teams’ and outside the technical organization begin to impact delivery more than any single teams success. Agile is a team sport (Klaus Leopold) - we need to use Monte Carlo to model the uncertainty through the entire delivery chain, not “my part.”
Over the years I’ve helped hundreds of teams make the move to Monte Carlo forecasting, and here is the main point I need to stress. Use the simplest method you can that gets the results you want. If Monte Carlo gives you added benefit for no extra effort, and if it sparks the right discussions, then you should consider using it. I’ve tried to create self-explanatory tools and training that balances complexity with rewards. If you haven’t checked them out, they are free to download and use here -
PS. Training isn’t necessary to get started. If you want to learn more in-depth usage we have two courses available. More info available here: