What are some common pitfalls to avoid when defining acceptance criteria?

Ambiguity in acceptance criteria leads to interpretation issues, impacting Agile development's effectiveness. Specificity is key, as vague statements like "the app should be fast" lack measurable benchmarks. Drawing from experience, defining criteria as "the app should load within 3 seconds on a 4G network" offers clarity and testability, aligning with Scrum principles for clear goals. This approach ensures team alignment, facilitating a smoother development process and enhancing product quality. Clear, specific acceptance criteria are essential for successful Agile projects, ensuring expectations are well-defined and met.

The internet definition is "'Vague' is used where something lacks precision or detail, while 'ambiguous' is something that could have two meanings, or is open to interpretation." Rather than look at the effect, which is where everyone rushes to, it is prudent to look at the cause so as to avoid this pitfall the next time around. While it is also known as Definition of Done, acceptance criteria being crystal clear is incumbent on the Sprint team - this would require some amount of discipline during the grooming meetings. The stories would then start getting developed and imagine if they were not defined appropriately! Release miss, quality gone, client confidence dented. Rewind to present day - customer PoV/clear/testable/specific.

In addition to providing more detailed criteria, it's also helpful to specify how the current solution is performing for comparison. If the new requirement is 3 seconds on a 4G network, how's that compare to current application performance? Also, it's good to understand WHY you're testing this? What is your customer trying to accomplish? Are you taking that in mind when defining success criteria?

Crafting clear and measurable acceptance criteria is crucial to avoid ambiguity in project development. Specific benchmarks, like loading time on designated networks, ensure alignment and quality outcomes while minimizing misunderstandings.

One of the most frequent issues we've encountered is the creation of ambiguous or vague acceptance criteria. This lack of clarity can lead to misunderstandings and misaligned expectations, ultimately resulting in features that do not meet the needs of the users. For example, a criterion like "the system should load efficiently" is subjective—what does "efficiently" mean? To avoid this, we specify measurable outcomes, such as "the system should load within two seconds on a 3G connection," which leaves no room for misinterpretation.

Focus the use case on WHY you are delivering and build the acceptance criteria around making that successful. This will help both business users and developers understand the importance of their work and typically deliver a better product.

Balancing acceptance criteria in user stories is crucial; too many or too few can lead to inefficiencies or ambiguity. Adhering to the INVEST principle ensures clarity and effectiveness, aiming for 3 to 8 well-defined criteria per story for focused development and thorough testing.

Working software being the primary measure of progress, wouldn't be possible without a clear DoD int he first place. The INVEST principles will help POs specify product backlog requirements - epics, features & user stories. Edge cases are crucial to factor - else the risk of a risky sprint demo. Striking a balance of well-thought of acceptance criteria is a skill the sprint team would do well to master. That brings me to my fav Agile principles that apply here - Business people & devs to work together daily in an environment to thrive, via F2F converations, by ensuring technical excellence. Excellence does not allow for mediocre acceptance criteria or too many/too few of them. It means hitting the right mark - repeatedly. INVEST in SCRUM.

Having too many or too few acceptance criteria can hinder a project's success. Too many criteria can overwhelm and confuse the team, making it difficult to focus on key requirements and leading to potential delays. On the other hand, too few criteria can leave important aspects undefined, resulting in incomplete or inadequate deliverables. Striking the right balance is essential. Include enough criteria to cover all critical aspects of the feature without overloading with unnecessary details. Aim for clarity and completeness to ensure that the team understands the requirements fully and can deliver a high-quality product that meets stakeholders' expectations.

Developers don't want to be told how to do their job, they want to know what is wanted (and why). If you're putting technical or implementation details in your acceptance criteria, you're telling the development team that their only real value is doing what they're told.

Crafting acceptance criteria centered on user needs, not technical details, ensures relevance and clarity in development. Prioritizing user goals over implementation specifics fosters efficiency and enhances user satisfaction.

See even here, it is emphasized to look at it from the customer's PoV - not the dev team's! What's the customer wanting to achieve as an end-user - not what technology to be used. Here the PO's role is crucial. They keep the team grounded and within the guardrails of the objective. The DoD/Acceptance criteria need to be a healthy balance of technical details + customer's objectives/desired outcomes. Technical excellence drives us techies to go all "technical" in the DoD - forgetting that the end-user has probably no idea of technology - they just want to use the functionality successfully. So how should my stories be written. Enter Backlog grooming/refinement meeting!

Incorporating technical or implementation specifics into acceptance criteria can severely limit the flexibility of the development team. This was evident in a project where the criteria specified using a particular technology stack that proved to be unsuitable mid-project due to performance issues. Instead, acceptance criteria should focus on what the feature does, not how it is done. We ensure criteria are written from the user's perspective, such as "the user can easily access the search function," without specifying the under-the-hood methods to achieve this functionality.

Including technical or implementation details in acceptance criteria can be problematic. When criteria are overloaded with such details, they may limit the development team's flexibility and creativity, leading to rigid solutions. Conversely, having too few criteria can leave crucial functional aspects undefined, risking incomplete or misaligned outcomes. Acceptance criteria should focus on what needs to be achieved rather than how to achieve it. Clearly state the desired behavior, outcomes, and any specific constraints, but leave technical implementation to the developers' expertise. This balance ensures clarity on requirements while allowing the team to choose the best technical approach.

Explicitly state assumptions that underlie the acceptance criteria and then gather feedback from real users or data to ensure that the assumptions are valid. Also write down potential risks if the assumptions are incorrect, so you can plan mitigation strategies

Pitfall 4 warns against overlooking or misjudging assumptions in acceptance criteria, like assuming all users have email access. To prevent this, rigorously validate and document assumptions, ensuring inclusivity and accuracy in criteria.

There's another one - over-confidence. We all grew up with the adage of "what 'assume' means". It is true to this day. The sprint team would do well to role-play customer v/s dev team (a.k.a. devil's advocate) so as to ensure all assumptions are factored in & tried and tested thoroughly so the user stories are fit for purpose. Part of technical excellence/motivated individuals wanting to provide an excellently developed bug-free feature. One of the team members could call out what they thought was missing or wrong and the team would be expected to incorporate the same with lessons-learnt in mind. Attention to detail is a key skill the team must ensure they have - or proactively demonstrate.

Assumptions are often an overlooked aspect of acceptance criteria, yet missing or incorrect assumptions can derail a project quickly. A stark example occurred when we assumed that all users would have access to a certain type of hardware which, in reality, was only available to a subset. This oversight led to a redesign of the feature. Now, we make it a practice to explicitly state assumptions in the criteria, like "assuming users have devices capable of supporting AR technology," to ensure all team members and stakeholders have a unified understanding.

Now here's where the SCRUM Master/Coach would have to unleash their creative power to keep the team motivated and facilitate learning/unblock anything they perceive as an impediment to delivering a successful sprint. The whole goal is for business people (PO) and developers must work together daily throughout the project. Very often I have seen the PO and the dev team being disconnected and they not willing to accept the PO as their own. This is a VERY important bridge to build and cross both ways. Synergies must be created by the SCRUM Master/Coach in such situations. Frequent breaks to let their hair down along with the PO develops a bond w/the team difficult to break over time. Then the team starts building trust to get the job done!

While freezing acceptance criteria early is a must, you need to work on the criteria iteratively beforehand, to refine them on the different feedbacks from the stakeholders

Collaborative acceptance criteria writing involving all stakeholders ensures alignment and accuracy in product development. Neglecting feedback may lead to misalignment and errors, highlighting the importance of inclusive processes.

Acceptance criteria should be developed collaboratively with input from stakeholders, including end-users, product owners, and developers. Lack of collaboration can result in criteria that do not fully reflect user needs or business objectives.

Common pitfalls when defining acceptance criteria include being too vague, not aligning criteria with business goals, setting unrealistic expectations, neglecting edge cases, and forgetting to involve relevant stakeholders. Clear, specific, and measurable acceptance criteria are essential to ensure effective communication and successful project outcomes.

Lack of collaboration or feedback during the definition of acceptance criteria often leads to criteria that are out of sync with user needs or technical realities. In one instance, criteria defined without input from the technical team failed to consider the limitations of the existing database structure, resulting in costly and time-consuming reworks. To combat this, we hold collaborative sessions involving all stakeholders, including developers, testers, product owners, and, where possible, users, to gather diverse perspectives and ensure the criteria are realistic and comprehensive.

Changing or adding acceptance criteria after development has begun can disrupt the development process and lead to delays and rework. It's essential to finalize acceptance criteria before development starts and avoid making significant changes during implementation.

Every story must have clear acceptance criteria that’s explicit, clear and achievable which is accepted by both the product owner and the development team. Sometimes requirements evolve and there maybe a need to adjust the acceptance criteria but this shouldn’t be the norm. If such a need arises, the development team must be given the opportunity to adjust the feature as needed either in the next sprint and complete the story. If there is a need to adjust the acceptance criteria mid sprint, the work on the story should be stopped and should only resume (or start afresh) once the acceptance criteria is agreed once again.

Pitfall 6 in agile development involves changing acceptance criteria mid-development, causing delays and scope creep. Effective communication and judicious management of changes are vital to navigate evolving requirements while maintaining project integrity.

Changing or adding to acceptance criteria after development has started is another common pitfall. This typically leads to scope creep and project delays. On several occasions, we've had to extend project timelines significantly due to late changes in criteria, impacting budget and stakeholder satisfaction. Our approach now includes locking down acceptance criteria during the sprint planning phase and only revising them during the sprint review if absolutely necessary, with all implications clearly communicated and agreed upon.

Agile is about delivering value. Value is delivered when features / changes are delivered to production which the business cares about. Hence to begin with, Stories need to be defined at a level and granularity which deliver business value. Once that is in place, Acceptance criteria needs to include all aspects which makes the deliverable production ready without any exception. The fundamental requirement for this is to define a measurable and unambiguous Definition of Done. This definition of done needs to be shared by all parties including business (not just the product owner).

In general, the pitfalls discussed above all come back to one key consideration, the user and their experience. Acceptance criteria should be about how THEY see success and that revolves around WHY this user story is important to them. If you work with the representative(s) from the user community, and you focus on why they want this user story delivered, you are much more likely to have a happy user. And that is the definition of success.

Lest we forget the fundamentals here .. as in let's first make sure every story *has* an acceptance criteria. Hand in hand with that, for stories that touch on the UI aspect of the software being developed, let's make sure there are mocks and styles defined as part of the acceptance criteria. Without now slicing and dicing, too much or too little content in the AC, levels of articulation and such, a general rule of thumb is that the acceptance criteria should, once met, leave the story with *nothing else* to be done on it. Hence, it fits right into the infamous Definition of Done and is an integral part of it.