The biggest challenge in Agile testing is to know how to integrate testing into the Agile process. In this article I will present how I have successfully integrated testing into the Agile artefacts.
What is Agile software development?
Agile software development refers to software development methodologies based around the idea of iterative development, where requirements and solutions evolve through collaboration between self-organising cross functional teams and stakeholders. The focus of the Agile manifesto is “Continuously delivering working software while allowing for and supporting changing requirements”.
More and more organisations are adopting Agile processes due to the benefits of faster application development cycles and quicker turnaround. However, shorter and faster development cycles are generally questioned for quality, hence it is important to integrate testing into the process rather than as an activity that occurs at the end of the development phase. That is where Quality Assurance (QA) comes in.
Quality throughout the development cycle
QA activities need to be integrated into the entire development cycle in order to assure the quality occurring in the software. Quality must be on the minds of all and built into the team’s day-to-day activities rather than occurring at the end. It should be built into project processes from initial requirements gathering, through to development and testing activities.
Quality in User Stories
A user story is a description of a software feature from an end user perspective. It describes the type of user, what they want and why. A user story helps to create a simplified description of the requirement.
User stories should take account of INVEST principles (but not necessarily be rigidly held to them as not all stories will satisfy all criteria, but they should satisfy most).
Importantly, A user story may be refined many times before being implemented. A good user story should contain details such as a description, Acceptance criteria, background material and relevant screenshots or wireframes.
Acceptance criteria are the conditions that have to be fulfilled in order to mark the story as “Done”, which is exposed to users, stakeholders and the team. They contain different scenarios explaining the workflow of the feature and can be written using BDD Gherkin syntax following “Given, When, Then” format. The acceptance criteria should cover the positive (happy path) and negative scenarios. It is also advised to include any specific non-functional requirements that are associated with the user story.
Definition of ready
A Definition of ready is the set of criteria agreed amongst the team in order for the story to be considered ready for development. A definition of ready could stipulate for example that the story should contain a proper description, acceptance criteria are complete, relevant screenshots or wireframes have been attached, and business rules have been fully documented.
Definition of done
Definition of Done (DoD) is a shared understanding of what it means for a user story to be complete. Each Agile team will have its own DoD checklist of features and activities. Basically, a team agrees on a list of criteria which must be met before product increment can be considered “done”.
The idea of DoD is that it ensures everyone on the team knows exactly what is expected of everything the team delivers. It ensures transparency and quality fit for the purpose of the product and organisation.
Here is the checklist for definition of done:
- Coding is completed for the presumed functionality
- Peer code review performed
- Project builds without errors
- Unit tests written and passing
- Integration tests written and passing
- Project deployed to a test environment
- Testing performed and issues resolved
- Accessibility tests passed
- Feature meets the acceptance criteria
- Project documentation updated
- Relevant automation test scripts created, peer reviewed and checked into version control
- Tested on devices/browsers listed in the project assumptions
Each team should collaborate and come up with the definition of done that suits its unique environment.
It is always useful to maintain project documents such as User research out comes, Technical implementations, Requirements, Service standards, etc. in a central repository such as Confluence or SharePoint. This documentation can be used for knowledge sharing and can create a living document for the team.
Source code should be self-explanatory. Whilst comments serve to document the code, it is advised to use comments only where it adds value and to avoid liberally commenting code that is already readable.
Manual and Automation test scripts can be written using BDD Gherkin syntax following “Given, When, Then” format which can be used as a living document for the application.
It is advised to have the test environments setup and deployments tested even before the developers start writing code. This ensures that there is an environment available to test the code once the code is developed. You can have different environments such as local, Integration (QA or System test environment), Staging (pre-production) and Production.
Local environments are used by the developers and testers to run the application in their local machines for development and testing.
The Integration environment is where the code is deployed through Continuous Integration / Continuous Deployment (CI/CD) pipelines for System testing. This environment is used by the QA team to perform manual testing and to run automated Integration (API) tests and Smoke tests ideally every time a code change is pushed to the remote repository.
Ticket workflow in a Sprint or Kanban
Ticket workflow is an assortment of statuses, which are used to track the progress of a ticket. The below table illustrates the ticket workflow that can be used in Agile Scrum model or in Kanban model. This workflow board can be created in any project management tool like Jira or Azure DevOps.
|To Do||In Dev||Blocked||Ready for code review||In code review||Awaiting QA||In QA||Done|
To Do: This is the first state, indicating it is ready to be picked up by a developer. The user story in this state should meet the definition of ready.
In Dev: This state indicates that the feature is under development.
Blocked: This state indicates that the respective story is being blocked and cannot proceed, for example if further information is required to progress or a dependency exits.
Ready for code review: Once the feature is developed, the developer moves the ticket to this state indicating that the ticket is ready for review by a peer.
In code review: The ticket which is currently under peer review is moved to this state with the assignee specified. If there are any comments, then relevant comments are posted against the ticket and assigned back to the appropriate developer.
Once the review comments are fixed, the ticket is moved to “Ready for code review“, then moved to “In code review” by a peer. Finally if it has passed the review, then the code is approved by the peer and merged, ready for QA.
Awaiting QA: Tickets in this state are ready for testing.
In QA: Tickets in this state will be picked by the QA team and undergo thorough manual testing. If the acceptance criteria are not met, then it has failed QA and is moved back to “To Do” and assigned back to the developer with relevant comments, steps to reproduce the issue, relevant screenshots and expected behaviour.
After fixing any defects, the ticket undergoes all the above steps and is retested.
You can follow the Exploratory Testing technique for manual testing.
Done: After the ticket has passed QA, it is moved to “Done” with any relevant comments, and closed. In principal, a feature should be moved to “Done” by, or in collaboration with the Product Owner to indicate the Product Owner has accepted this feature as complete (as opposed to the development team just saying it is complete). Depending on the project team structure, an additional status ‘Ready for PO review’ could be introduced.
Exploratory testing is about discovery, investigation and learning. As the name suggests, it’s about testers exploring the application to identify potential edge cases.
Exploratory testing is the simultaneous process of test design and test execution. It is also complementary to test automation; that is while automated checks are checking for regression issues, exploratory testing focuses on new features which have been developed.
The major challenge for QAs on any Agile projects is the amount of regression testing that needs to be performed before each release; this can also be addressed using automation tests.
In order to get quick feedback, automated tests need to be executed as often as possible, ideally after every code change that is pushed to the repository. Tests need to be fast, delivering consistent and reliable results.
It is important to prepare a test automation strategy that is result-oriented and fits well with the team and organisation as a whole. You can read more about creating an affective Automation test strategy in my blog post “How to get ROI from Test Automation in Agile projects”.
No matter how good the process is, if there is a lack of communication amongst the team members then it is difficult to produce a high quality product. Co-locating the team in a shared workspace will improve face-to-face communication and helps to build trust among the team members. Wherever co-locating is not possible you can use team collaboration tools such as Skype, Slack, Microsoft Teams or similar.
In Agile where requirements and solutions evolve through collaboration between self-organising cross functional teams and stakeholders. This makes it important to have a regular interaction between QAs, developers, the BA/Product Owner and stakeholders through daily stand-ups, planning meetings, backlog refinement meetings, regular demos and retrospectives.
Your team can enjoy the benefits of the Agile development process when you produce a high-quality product frequently and repeatedly. To achieve this, the QA process needs to be integrated into the development cycle and should be in the minds of all team members. It should be part of the team’s day-to-day work rather than something that occurs at the end of development. It is also equally important to have effective communication amongst the team to improve the quality of the product.