Finding the right balance

What should one look for when recruiting for a test team?
Do you take the company profile of "Software Tester", which probably reads something like:-

• Bachelors degree in computer science or a scientific/mathematical subject
• 2 or more years experience of software testing
• Experience of automated testing tools
• Experience of your documentation tool of choice
• ISEB foundation certificate

and hand this to the recruitment agents?

Personally I try to avoid this approach. I see little benefit in populating a team with multiple versions of the same skill set. Instead I prefer to take the opportunity to put together a team who together have a range of skills, the members of which can all make valuable and different contributions to testing the product.

In order to do this, the first question that must be tackled is:-

What qualifies someone as a software tester?

My answer would be:-

Anyone who can make an educated assessment of the value of the system under test to a stakeholder of that system, or anyone who can increase the ability of the team as a whole to achieve that assessment.

The first category could contain a number of different abilities such as:-

1. Experience in software testing, particularly working with the technologies in question
2. Experience of working in the industry or business area in which the product is targeted
3. Anyone with experience of using the systems and tools (rival or complementary) which the targeted users of the system will have experience of.
4. Experience of administering the environments, operating systems or types of network within which the system will operate
5. Anyone with experience of working in the legal or legislative environment in which the system will operate

The second may contain abilities such as:-

1. The ability to create and maintain automated test harnesses/suites
2. The ability to configure test environments

Now, clearly if we are putting a team together to critically assess and automate ongoing checking of a system then again just putting together a team of testers with specific domain knowledge but no ability to automate or maintain test infrastructure then that is not going to achieve the team goals. What we require is a team whose combined skills will achieve the goals of the team as a whole.

Creating a skills matrix to identify needs

My approach to identifying the skills required to build such a team revolves around the creation of a skills matrix:-

First I create a list of skills which I feel are relevant to the team and the work that we have to do. I generally group these under headings such as "Testing Skills", "Domain Knowledge", "Test Automation" or similar. I will obtain the input of the current team members on this as well. The result will be a list of skills E.g.

Testing
	Test Scripting
	Estimation
	Team Admin
	Test Automation
	Performance Testing
	Multi-User/Load testing
	Web Testing
Test Technical
	Unix Shell scripting
	Windows Shell scripting
	Perl
	Java
	VBA/VBS
	Dot Net
	Python
	Selenium
Database Related
	Oracle
	ASE
	SQL Server

Then, I rate these skills in terms of importance to the team. I use 2 ratings, both on a scale of 1-3:-

- Relevance - How relevant this skill is to the team
- Weight - how frequently the skill is required. This essentially translates to the number of team members that are required to possess the skill

Then I rate each member of the team according to their ability. I do this in discussion with each individual, according to the following ratings:-

0 - I would not know where to start doing/using this
1 - I would need help if asked to do/use this
2 - I would be able to do/use this on my own but am no expert
3 - I would be able to do/use this without any help and know what I was doing

By putting these ratings into a spreadsheet and using simple formulae of

total score=SUM(team ratings) - (relevance X weight)
peak score=MAX(team ratings) - (relevance)

The Total column gives us a score relating to the coverage of that skill in our team relative to the weight. The "Max" column is simply the difference between the relevance of the skill and the ability of the highest scoring team member, thus identifying if there is a general ability in the team but we may be lacking a specialist.

A score of zero and above will indicate adequate coverage, below zero indicates a need in the team. I appreciate that the scoring system is arbitrary,it is a useful indicator of need. If it is not indicating useful information then I suggest adjusting the weighting/relevance ratings. The result of this analysis will be a matrix similar to:-

Area	Skill	Relevance	Weight	Alan	Bob	Carl		Total	Max
Testing
	Test Scripting	3	2	3	3	3		3	0
	Estimation	3	2	3	3	2		2	0
	Test Automation	3	2	3	2	2		1	0
	Performance Testing	3	1	2	2	2		3	-1
	Multi-User/Load testing	2	1	1	1	2		2	0
	Web Testing	1	2	2	1	1		2	1
Domain Knowledge									0
	SQL92	3	2	1	2	2		-1	-1
	ODBC Connectivity	2	2	3	1	3		3	1
	Business Intelligence	3	2	1	2	0		-3	-1

This provides a simple yet highly visible representation of the current position of the team and where the shortfalls are relative to where you want the team to be.

This approach to identifying the skill needs of the team is useful immediately for personal development plans, to see if the existing team members have an interest in learning or developing on one of the new skills.

The approach is especially useful at the point of recruiting new members to the team. Whereas the temptation is often to recruit based on a role "template" as described at the start of this post, an exercise such as this can sometimes reveal unexpected results in that the needs of the team lie in complementary areas to the traditional testing skills. Recently I recruited an excellent addition to my team who had very little testing experience at all. He did, however, possess system administration skills in the operating environments that our software is installed into, which gives him an excellent insight into the desirable characteristics of the system from the perspective of a valuable stakeholder, the system admin. Similarly another of the best testers in my team has a high level of specific domain knowledge in the area of databases, which is a desirable skill in critically assessing the behaviour of the product from the perspective of a DBA.

I would add the caveat that I have only used this on small teams so am not sure how well it scales, although on larger projects it could be interesting to apply the same approach recursively at the individual and team levels to identify team skills and needs as part of a larger group.

This approach takes some courage, as it requires looking at the combined skills of a team to meet the challenges faced rather than the skills of individual members. There may also be some push back from management, who prefer to see things simply by fitting individuals into role shaped compartments. However the benefits to the company that can be brought by putting together a multi-skilled team to address a multi-faceted task will pay dividends in the long term.

Copyright (c) Adam Knight 2009-2010