Saturday, September 14, 2013

Hi I am UX, Nice to Meet You Automation


In this era of evolving development methodologies from the traditional waterfall type methods to its progeny agile project and development methodologies, we are introduced to a new role known as UX.  Well, the concept is not new but for many of us, we are just now becoming acquainted with it.  This clever acronym can be expanded to describe a team, a concept, a process and a methodology that supports today’s customer-facing type products.  UX is short for user experience and can be defined as “any aspect of a person’s interaction with a given system, including the interface, graphics, industrial design, physical interaction, and the manual.”  The full definition and more information can be found here.

As testers, we have lately been introduced and bombarded with new terms such as:  
  1. Flows and Navigations Maps
  2. Persona
  3. Prototypes
  4. User Experience
  5. User Journey
  6. User Stories
  7. UX
  8.  UX Designs
  9. UX Designers
  10. UX Developer
  11. Wireframes
The above is not an exhaustive list but it identifies people, techniques and processes the agile concept is suggesting to invest in to improve the design-ability and tested feasibility of a desired  innovation to develop a new or improve an existing system.  The output of a UX team can prove to be an invaluable input into the test team’s efforts. 

So what!  What can the UX team provide that I can use for my test effort?

From my short experience working with UX teams, I have identified three major outputs that can help re-invent the testing and test automation approaches.  Those outputs are:

  • Project specifications and user stories
  • Wireframes
  • User journeys
All testers know it is (or should be) a requirement to involve testers in the earlier stages of the development process.  So any and all information shared is of value to our efforts in understanding how to develop the required tests.  Traditionally, in the early stages of receiving the information, it was used primarily as analysis and testers would provide feedback based on their experience and knowledge of the business and solutions already utilized.  We can debate, if there is any value in defining tests at the earlier design stages that are prone to frequent changes.  If one is an automation specialist, then based on many debates I have read, it is way too early for most to begin thinking about test automation.  However, as the above design type documents are finalized and approved, they provide a wealth of input that can both expedite and advance the testing and test automation practices.  For the sake of this article and the subject of my blog, I will limit this discovery to how UX can support test automation specifically.

Project Specifications and User Stories

This output from the UX team provides helpful and invaluable information source to both functional automated and performance testers.  These documents often include important details such as:

  • Design characteristics
  • System flow
  • Error and warning handling
  • Expected performance metrics

If we, automation specialists, have incorporated a test management solution as part of our test automation, then this is a great opportunity to leverage entering the required information to support:  sprint/release/build handling, test user stories/requirements, configurations, testset types, and test case organization.  For more information, review the article Agile Test Planning (using SmartBear’s ALMComplete).  At this early stage, it is not required for us to begin creating code, but we can mature the supplemental test artifacts our framework leverages to support our automated test runs and reporting.
I am not an expert in performance testing however I would imagine the defined performance goals will allow the performance testers to begin creating their approach to the project by identifying how performance can help the project meet the identified goals, if the goals are reasonable and achievable given the expected user load and architecture of the system, and begin identifying the metrics and boundaries in which they should test in and/or surpass to test the system’s performance and scalability.


Wireframe design or modeling provides a visual guide of a customer-facing product.  As both testers and test automation specialist, the wireframes can provide us details we can capture at an early stage of how to expand our exploratory efforts as well as what type of framework support may be required. 
For automated exploratory testing, we can begin to identify characteristics we may wish to routinely verify the look, feel and usability of the system.  We can identify a number of checks we wish to perform such as field min max input, text size and color, select listbox items, clicking buttons, system intrusion features, etc.  For visual characteristics, the test automation specialist can use a number of techniques to retrieve the values of property settings to ensure it adheres to the agreed guidelines instead of relying 100% of a person’s sight and attention span especially after the fifth time someone exercised the same exploratory tests.

In addition, the test automation specialist can begin to identify what framework upgrades or designs that are required as well as collaborate with the developer to identify what techniques and/or tools he will use to bring the designs to life as a way to ensure test automation compatibility.  This information can be used to identify if an upgrade of the existing tools are required or the acquisition of a new tool.

User Journeys

This is perhaps the most exciting output from a UX team.  The current UX team prefers to model a user experience based on a persona type.  These models are often created using diagramming tools such as Visio or SmartDraw.  With the innovation and growing usage of model based test automation tools, this creates a wonderful collaboration between the UX designer, the tester and the test automation specialist!  These user journeys should model how a particular type of person will utilize the system.  This means the model has captured the expected behavior of the system as the user makes the journey through it.
In a nutshell, here are the basic sets of test cases required for testing the new or changed system!  As the tester, why re-invent the wheel!?

As the manual agile tester, we can use the user journey models to add additional models to add more sophisticated journey paths as well as test for negative conditions.  We can enhance the models by injecting keywords, test data or test data location, trace to requirements and defining the edges (actions) and vertexes (verifications) of the models.  If a test management system is incorporated, a reference locater to the model can be included in a defined test case.  The benefit of leveraging the test management system is to have the ability to link the automated tests to other project artifacts.

As the test automation specialist, we can use the information provided to modify and/or add the keyword support required by the models.  Additionally this technique will allow the framework to walk through a model sequentially or randomly which means it will possess a level of artificial intelligence to identify the required permutations for the defined user journey for both smoke and regression testing with a bit of spice by using random generated test runs that can identify unanticipated permutations and lead to discovering unanticipated defects.


The invention of UX provides an invaluable support organization for the testing organization.  If I had my way, the development process would place the test team in a collaborative state with the UX team which empowers testing to be positioned as ready or near ready for testing on deployment from development.   This means we can further support the idea of sophisticated automatic testing on deployment of changes and depending on the results, acceptable pass rates can deploy those changes to production right away which supports the idea of continuous testing and delivery.  This provides greater band-width of time to the test effort for manual exploratory testing, analyzing non-acceptable pass rates, manual bug verification, reacting to production issues, etc.  

For us long time testers, this may feel like a scary trend as we feel our importance is diminished but on the contrary.  I believe it elevates the importance to our role yet sophisticates our efforts so we are part of the desire for continuous delivery and improvements to meet the immediate needs of the customer.

Margaret Thomas has worked in the testing profession since 1996 and worked within a number of industries including mobile, finance, medical and insurance.  She developed a strong desire for automated testing when she began working within finance and has utilized a number of testing tools both commercial and open source.  She is also the author of the technical blogs found at

No comments:

Post a Comment