This post aims to give the people what they want: a product experience metric for flow based on a brief, handy measurement tool. Although developed via quantitative studies, this flow scale has also proved useful in face-to-face interviews and smaller scale usability sessions. Anyone is welcome to take these questions into their own user studies. Let me know how it goes. Extra credit to any team that wants to follow the method described to derive their own flow scale (I'd be happy to collaborate there).
The following 4 questions are ones I use to quickly track a product's propensity to deliver flow. What about calling the scale FlowMoBI (Flow & Motivation Brief Index)?
For a straightforward flow index, make each of the 4 questions’ scores worth 25 points. That will nicely sum the 4 up to 100. It feels much better to think about normalized scores. The raw scores (for questions 1 + 2 + 3 + 4) need only be multiplied by 5, to generate a 100-point index. It's awkward to assess the significance of a raw score moving from 15 to 17. Yet we effortlessly intuit the meaning of an index going up from 75 to 85.
The remainder of this post is for readers who like to see how the sausage gets made.
My original objective was to find out how much flow contributes to a good experience. These steps outline my method:
1- Select an online population (volunteer customers, paid panelists, or Mechanical Turk workers)
2- Assign sample with performing a task
a- Direct sample to a specified site to perform assignment (e.g., map a journey, schedule a workout, build a list of favorites)
b- Define the criteria for success (e.g., email a screen shot of completed task)
c- Upon task completion, respondents return to answer a set of questions about their experience
3- At a minimum, ask the 4 FlowMoBI questions plus some kind of aggregate UX measure (in our example, NPS). [Initially, I tested all 49 questions from published studies. You don’t have to start with such a shot-gun, but feel free to make up and test alternative phrasing that might better fit your situation.]
4- Analyze your data using your favorite method(s): correlation matrix, regression model, factor analysis, reliability rating, etc
In studies I’ve run, flow’s relationship with positive user experience has been robustly significant. The correlation between flow and NPS typically stands at about 50% (observed values between .42 and .55 across dozens of studies). When I began, I was working with a very large battery of 49 questions, so it made sense to run a factor analysis. The data revealed 2 powerful dimensions: Fluency and Absorption. In the Flow/Motivation Brief Index (FlowMoBI) the 4 questions are split: 2 measure aspects of absorption and 2 for fluidity. Fortunately, 2 factors sufficed to capture most of the value. Running a statistical regression showed that adding the 4-question FlowMoBI scale significantly increases predictive power. So, if a firm were tracking NPS already, and using that as a correlate of their future success, adding FlowMoBI would give them more precise information about that effort to predict the user's relationship to the product.
While my data isn’t shareable, anyone so inclined should be able to repeat this for themselves. I'm not at all in love with the nonce term, FlowMoBI, so perhaps we can come up with a better label together.
If you'd like to test your product's flow (perhaps in comparison to competitors), I'd be happy to help you run the experiments involved in learning this.