There are definitely some very passionate souls out there debating the issues of
* what to automate
□ how much to automate
◊ who should automate
○ who should own the automation
People such as James and Jonathan Bach and others, who are awesome lecturers and educators in defining and terming test techniques, can add to the battling camps of automation versus manual or artificial intelligence versus sapient. If that wasn’t enough fuel for the fire, you have edges versus test step descriptions or test case scripting versus exploratory checklists or modeling. It’s like a techy version of the Hatfields and McCoys!!!! Everyone is entitled to their own view but this overwhelming battle has made its way onto the working environments and interviewing arenas placing some people in an almost no win situation. I will touch on this topic later.
Boxing Test Automation
One thing that tugs my frustration strings are those testing advocates who want to put test automation in a little box and dictate its limitations by making statements like tools are only script checkers, they only do what you tell them to do or you can only use them for regression testing. I am going to step out of the norm of refraining from a personal opinion and say that is because they lack the insight and imagination to truly maximize the potential that test automation and other tools can offer. I agree with leading test advocates that testing is a sapient activity one that is in my opinion, philosophical, scientific, intellectual and creative in its nature. Testing should and must go beyond the ‘if then else’ flow of the code and this demand should influence test automation. When you combine the intellectual and analytical input of a tester with the agility of test automation, endless possibilities occur.
In my career, I have worked for a variety of industries, working with different kinds of products using different development methodologies both traditional and agile. Each opportunity introduced so many new challenges, techniques and terms and methods they became cyclical in my head…techniques, terms and methods oh my! Many of us are sitting in the eye of the storm struggling to do our jobs day to day can become overwhelmed with ideas and opinions especially regarding test automation, test management and other related topics. Check out the LinkedIn discussions. One simple question can result in a firestorm of responses that can often overwhelm the original question asked!
I say damn it all to hell!!!
Stimulate your own brain juices to decide what it is you truly want to accomplish and then leverage all of the great information and misinformation to decide what tools you require for your testing toolbox that will help you accomplish our desired goals. If you are part of a team that will make this decision together, then you will need to provide supporting information to what you will add to the toolbox.
Test automation is not just about picking a tool that will let you either record some or program actions and play it back. It is about leveraging all the tools you have available to you to help you manage the development process, manage the testing process, keep people informed, manage current and historical data, eliminating the waste of performing routine mundane tasks, helping to mitigate problems and yes of course test execution. Think about the simple activities surrounding setting up for a test run. You have to check if a build is ready, download if it is, perhaps clean the machine, install or update it, license it, start it and then perform a smoke test to see if it is ready for testing. Do you know how much time these activities take if they are not a part of the requirement or user story you are targeting for testing? Why not have an automated method perform these tasks for you and alert you that a new build is ready and verified for you for testing! Come on even die hard manual and/or exploratory testers can appreciate that!
So Who Owns Test Automation?
That is an easy answer. The team! Regardless if it is an agile or traditional team with one or many people, the team owns it. The team’s goal should be to mature and maintain their toolbox and the solutions they have developed using their tools. There should be collaboration on how test automation will support the overall development and testing process. The team should always proactively seek updates or new tools to support future needs. Last, the team should ensure they code and version manage their efforts to ensure they have stable tests to always support their test effort.
Who Should Develop Test Automation?
Oh my this is another interesting debate! Some argue it requires developers while others say testers can do automation. Although I am a tester who can develop advanced test automation frameworks, I have been on the side of the interview table desperately seeking testers with some development skills.
People have solved the dilemma by hiring testing specialist with a background in test automation or programming, developers with a background in testing, or create a collaborative union between developers and testers. Regardless of how a solution is discovered, the individual or union of individuals should have the following skills to contribute to the development group:
- Experienced with testing fundamentals such as planning, test environment planning and maintenance, test development, test execution, bug and test reporting within the preferable development methodology such as waterfall (yes there are still people using waterfall) or agile.
- Experienced with quantifying and reporting test results.
- Experienced using either commercial or open source tools.
- Experienced programming preferably using the desired language, i.e. Java, Python, C#, C/C++, etc.
- Experienced managing the test process preferably within the desired methodology such as agile or traditional.
- Knowledgeable using management systems such as code, build, test, application life cycle, project, etc.
- Experienced with identifying and selecting the right tools and processes that will contribute to doing the job and leveraging the skills available to utilize the tools.
- The willingness to learn and mature their own knowledge to support growing and maturing development endeavors.
- Possess imagination to innovate ways to improve and support the overall development process.
As long as the individual(s) or team possesses these skills then you have the right team or group to support not just the automation effort but the overall development and testing effort within a given project.
A Tale of Interviewing Dilemma
In the past few months of seeking new employment after my last role ended, I have had the craziest interviewing experiences of my life! I would invest time, energy and money going to 2 or 3 interviews for several companies and the reasons why I didn’t get the job were just astounding! The reasons fell into the following categories:
- They did not like ONE technique used for managing the testing process.
- I wasn’t enough of a developer although I was applying for a manual testing role.
- I had TOO MUCH test automation experience.
- I didn’t have enough programming experience for test automation (although I have single handedly innovated and developed test automation frameworks since 1997 using both commercial and open source tools).
- I was too agile and they required someone to be both traditional and agile
- I was not agile enough.
In seemed the interviewers were forgetting the basics. They should seek someone who has the fundamental skills and knowledge for the job and of course if they seem to be a good fit for the team. It is awesome when someone can also bring with them the experience and skills that made them successful in prior roles but they most certainly will have to learn to adopt new techniques to fulfill their new role. One example would be, to adopt new methods and techniques to support agile test efforts. No one comes pre-packaged with 100% of what the interviewer would like to have for the role. However, it seems that is what interviewers would like. I laugh because I remember for one particular interviewer who required someone who had Windows 8 experience. That was the most important requirement that needed fulfilled. Windows 8 was just released. I thought to myself so it is more important that a person has this one particular knowledge of an OS and not a person who has demonstrated the ability to acquire the knowledge and skills needed to do the job…what happens when Windows 9 comes out….that person is no longer qualified because we have a new OS? As you can see, I was frustrated meeting people who didn’t know what they wanted, who truly didn’t examine a person’s background before inviting them to an interview or simply seeking compartmentalized people components they could plug in to do single tasked software factory work.
At the end of the day we need a job and these people have the keys to giving us a job so please use this information to help you better prepare for the challenges of today’s interviewing requirements.
How Much Should be Automated?
Honestly, if I could automate a rabbit jumping over a leprechaun’s rainbow I would :). Today, we can leverage many tools to assist our automation endeavor. We are not limited by our choice of one test automation tool. In the world of APIs, web services and open source solutions we can leverage many tools to help us automate even the most challenging of things to automate such as real time data, video streams, voice streams, complex calculations, object metamorphosis and more. If it can be programmed, it can be automated.
NOW WAIT before you guys start the screaming and crucifying me!
This does not mean automation should replace the intellectual abilities of a tester, NO NO NO NO! The tester is an important part of building good and robust test automation. The tester is key to managing the changes our automation must take on build after build and release after release. The tester is imperative to ensuring test automation is routinely testing high usage features and ensuring if it is not meant to be broken, it shouldn’t be.
Test Automation is a tester’s extension of his knowledge of what to test, when to test and how much to test. Test automation should allow the tester to regularly and continuously identify more clever ways to test instead of over-investing his time to performing repeated manual tests of an application. Last test automation helps the tester verify processes and functions not reachable by a client interface.
I am a true advocate of different things work for different people, different companies and different situations. Although I am a true test automation advocate, it is not my role to force my ideas on someone else. I love the online discussions when people ask advice on what tool they should use to support an automation endeavor. Tons of people and vendors respond giving their biased opinion of what the person should use because they use it or trying to sell it. It is even funnier to see those people who will respond disagreeing and proudly saying no don’t use that, use this I prefer this tool it’s better. I love this article by the Software Testing Club and The Ministry of Testing list of test tools. Input is always great! However in the end, regardless of which camp you prefer to join, it is up to you to choose the right tools you require to support your job because even in the manual camp, you are using some type of tools to help you with your job.