How do you measure and report the results of a cognitive walkthrough?

1 Define success criteria

Before you start a cognitive walkthrough, define what constitutes a successful outcome for the task and the user. This will help you evaluate whether each step is clear, relevant, and supportive of the user's goals. Success criteria can be based on factors like time, accuracy, satisfaction, completion, or error rates. For example, a success criterion for the task of booking a flight online could be: "The user can book a flight with their preferred dates, destination, and budget in less than 10 minutes with no errors or confusion."

2 Collect and document data

As you perform a cognitive walkthrough, collect and document data that reflects the user's experience. This can include qualitative data, such as comments, questions, or suggestions, and quantitative data, like the number of steps, errors, or backtracks. Use tools such as screen recording, note-taking, or rating scales to capture and organize the data. For instance, you can use a 5-point Likert scale to rate how easy or difficult each step is for the user, from 1 (very easy) to 5 (very difficult).

4 Report and communicate results

Communicate the results of the cognitive walkthrough to relevant stakeholders such as designers or managers. You can use formats like slides, reports, or dashboards to present the data in a clear and visual way. Storytelling techniques, such as scenarios, personas, or quotes, can also help illustrate the user's perspective. The main goal of reporting and communicating the results is to highlight the strengths and weaknesses of the product or interface, and to provide actionable recommendations for improvement. For example, you can use a table to show the ease ratings for each step, and use color coding to indicate which steps need more attention or revision.