Creating a Test Strategy For Agile Teams

6 min readJul 19, 2019

In my opinion, a fitting analogy for a Test Strategy document would be that pair of socks in your wardrobe that are omnipresent, but never there simultaneously. Imagine my plight when I was suddenly responsible for creating one!

I felt like I had been set up for failure.

I have bad feelings associated with the phrase ‘test strategy’ as it has always been created as a tester-specific artifact and not the team(s’) specific document. Whilst I have encouraged and supported teams I have worked for in establishing a quality focused, collaborative approach to development, it never crossed my mind that having a document was maybe useful.

In all honesty, when I had this task staring me in the face, I did feel that a unified approach of doing things at an strategic level was required, as we were changing how we were doing things. At this particular juncture, a document was a required protocol.

Creating a Framework

When I set about writing my test strategy, I wrote a list of things that should be supported by my test strategy so that I had a framework in which my test strategy would fit , the list included the following :

The test strategy should convey that testing is a part of the development process and not something to be considered after the code has been written
The test strategy should not be role-specific
That every member of the cross-functional team can relate to it and feel positive about contributing to it and establishing good test approach in their teams using my document as a guideline.
The document should not require more than 2 or 3 minutes to read and should focus primarily on the key goals that the strategy helps to achieve
The desired outcome should be clear
That the document is not static and can be improved continually
The document doesn’t restrict the agile teams from doing what they do best as in the agile teams should still have the flexibility to design their test approach based on what helps them with faster feedback and deploying services without obstructions albeit the document focuses on a unified goal for testing.
That I, as the practice lead, believe in it to a level that I can crown it champion

I then decided to focus on next steps that I wanted to take. There were so many invaluable inputs from various experienced colleagues, even before the requirement for a test strategy became increasingly apparent. Although my colleagues’ inputs helped me prioritise the key goals, I had to ensure that I believed in it too and subsequently made a decision based on what synergised the best with the team(s).

I had previously analysed and made a list of key issues I spotted in the teams and I categorised them into three themes, this was something I have always been doing and was not an task I took up specifically for writing this. I was keen on making our test process better. I was clear that I was not selling a document but championing a positive change.

Grouping The Key Issues Into Goals

As I have been continually monitoring issues within teams , grouping these issues into themes was not difficult . Every team had different challenges , some had process issues , some had issues around the way they designed their automation test suites , some were restricted to use a persistent environment, etc. And all teams competently found an appropriate work around to get things done on time — however, without an unified approach and lack of guidance things were getting more challenging for teams consequently generating a lot of frustration . It was clear that this needed to be fixed and having something documented to initiate this change was the first important task. I have listed few examples on what types of issues were grouped into goals under themes below.

Process Goals:

This included all the good practices we could encourage in teams to make the development process better. Some examples were :

That quality is owned by the team and not the individual.
More collaboration at every stage of development with focus on quality.
Good analysis sessions that help understanding the requirement better and encouraging discussion on how to test and identifying key quality aspects to be considered for every feature under development
Tickets with a well-defined scope
Encouraging knowledge sharing
Identifying and removing single point failures in teams.
Improving collaboration by encouraging activities like pair programming / mobbing /peer reviews
Having structured and high impact meetings

Testing Goals

The goals were around testing from developer tests to exploratory tests :

Increasing the number of fast tests and reducing the number of slow tests
Improving our exploratory testing efforts so the problems spotted can then feed into improving the maturity of out test pipelines
Encouraging continuous testing , setting up better test pipelines for faster feedback.
Designing test frameworks such that the purpose is well defined, as in when a automation test suite is designed to test integration of services , then it focuses on the integration and does not include regression or business logic
Code reviews on any piece of code that’s written whether source code or test automation code.
Investing time and effort on creating test automation suites and maintaining them like source code
Moving away from one test framework maintained by a person , driven at the UI level which does all and protects everything and tests tons and tons of things and takes 2 hrs to run.
Focus on service layer tests, consumer tests and non functional tests
Creating better logging and monitoring mechanism
Creating and using synthetic data for automation and exploratory testing .

Non Production Environments Goals

There were several important points to consider here of which few examples are as follows :

Aligning the use of non production environments across all teams based on purpose they are meant to be used for like separating training requirements from development requirements
Creating non persistent test automation environments before test runs and tearing them down after test execution would help reduce dependency on persistent shared environments and reducing the risk of creating brittle tests
The more closer the environment is to production , the more stable it is expected to be and most risks are mitigated and issues are resolved earlier.
Having standard naming conventions for the non production test environments which reflected their intent clearly to everyone

Documenting The Test Strategy

I wrote the test strategy based on all the above discoveries I made. The task of documenting and presenting the test strategy was not under my radar initially, but a sudden change of events resulted in that task being thrusted into my hands. I had very limited time to write and present something. Because I had been continually analysing and identifying issues which I wanted to turn into goals, I only spent most of the time I had in creating the framework (described above) and then fitting my goals within that framework to produce a readable simple wiki page, which I then presented to various groups of people and at Community of Practices meetings.

I did have help from our dev coach and other colleagues who helped me proofread the document and refine it before I presented it.

Modelling Change

However, people are prepared to change but they do not want to be forced. A change process takes patience, knowledge about behaviour and flexibility from both the person who wants to change and from her environment.

It was important to be patient, to persevere and to be consistent and more importantly be the change I wanted to see.

I had the initial thumbs up from all those who would be a part of this change but the journey had just begun. It was also important that I had to be open to any new suggestions and other constructive criticism.

Probably another detailed blog post about modelling change might be better than cramming all the details here.

Reflections

One thing which I didn’t talk about here and mentioned in the real document was the famous test automation pyramid . It is recommended and talked about a lot by many in the wider development and test communities. I do feel it was an really useful thing to have in agile test strategy documents .

But it is really important to remember that a good test pyramid is achieved :

1:When each layer is built based on the layer below

2:Each layer has a unique purpose and adds value

3:When the testability of the product is considered from discovery to delivery such that there are more than 2 layers of test ( as in not just unit tests and UI tests)

A team might have built a good measurable pyramid with most of the test being implemented as developer tests and very thin slice UI end to end journey tests. But the tests could have been designed without following the recommended best practices ; e.g. good discovery sessions, pair programming, good collaboration and many factors mentioned in the above goals might have been omitted. Overlooking various important aspects of what helps us, achieving our development goals is not the best course of action.

Test pyramid is an outcome of good development and test practices, from Analysing→ Developing →Delivering→ Maintaining the product/service.

Overall it was a very rewarding task, and my test strategy document no longer emulates a pair of disused socks!