Monday, April 22, 2013

The Journey to Marrying Continuous Delivery to Regulations

In my previous article, Welcome Aboard...Now Hit the Ground Running!, I began sharing how I was introduced to some new techniques that offered some interesting new ideas for test automation.  I was introduced to methods such as:
  • Continuous delivery
  • A variety of teams using different agile methods
  • Different approaches to test automation such as continuous testing and mobile remote test management
  • Automated IT functions
  • Pipeline
all within an environment that must meet governmental regulatory guidelines.  I can’t lie, I was overwhelmed!  So I reached out on LinkedIn to start a discussion to learn how other companies were rising to the challenge of being agile and using continuous delivery with the overhead of meeting regulatory guidelines.  I didn’t get much feedback but the people who did respond all stated automation was the key. 

In order to rise to the challenge of continuous delivery, the company adopted a method of product delivery that seems quite similar to how a factory manufacturers goods.  A conveyor belt is started and each product component team designs, builds, test and release their components to the conveyor belt which passes the QA and change management stations into production.  Of course it is not as simple as that but you get the idea.  Each station, on the conveyor belt, uses a different agile method, is organized as an independent agile team with all the required roles, and the release frequency can vary from station to station.  No matter how they elect to manufacture their component, their delivery process is squeezed into quality assurance for qa test certification and internal auditing and a change management program responsible for ensuring regulations are adhered to and the actual release of the components into production.  Automation is imperative to improve the efficiency for both qa and change management.  To date, I can share some of the automation strategies we are using to rise to the challenge.  

Strategy One:  Maximize the hell out of your Application Life Cycle Management, Jira or Test Management tool.  These tools independently or as a joint effort can manage the entire line organization process to manage the life cycle of the proposed changes from inception, design, development, QA, change management and then released into production.  They allow us to customize how we capture information used for regulatory purposes and to be an intuitive part of the test planning process.  The tools’ filtering features allows us to create reports required for auditing purposes as well as provide traceability to all information related to the change.

Strategy Two:  Automated system monitoring.  A continuous delivery environment requires that all test and production environments are always up and available.  Of course we utilize professional system monitoring tools to keep a watchful eye on the system’s health.  However we also leverage a very cost-effective way using tools like PsTools to do high-level system monitoring to inform testers of system unavailability during a test run.

Strategy Three:  Automated system deployment using continuous delivery tools to move changes to desired environments.  Setting up or updating a test or production environment is as easy as the click of a play button.  Another great advantage to using these types of tools is to automatically deploy and run automated tests to verify the new/updated deployment.  This continuous automated test method provides immediate test results complete with supporting information for any test that might have failed.

Strategy Four:  Of course you are going to automate tests.  How else can manual testers maximize the use of their time?  One method that is very appealing is using test modeling in addition to the continuous automated testing methods.  Test modeling can support both manual and automated tests and allow the tester to create various scenarios of tests instead of creating an exhaustive list of test cases to be organized into test sets. 

Strategy Five:  Make the investment in time to learn how much auditing is truly required.  This way you can tailor your QA and change management process to meet the auditing requirements.  One technique auditors suggest is to break your releases down to identifiable and manageable components that will allow you to apply a significance rating to each of the components.  For example, you can have a login component.  However your login component can be a combination of a number of components and perhaps components within components.  Here is an example

The significance rating identifies which components actually qualify for external auditing.  At the same time, internally the rating can help to recommend exactly how much internal auditing and test certification is required before it can be allowed to be released into production.  This way, you invest the time and effort on the most critical and significant components of the system.  To be honest, it is really nothing more than old-fashioned risk assessment which quality assurance and testers have always used as a way to identify where to invest the most time and effort.

So this is where I am now with the journey towards working in a company who is doing continuous delivery to regulated markets.  Right now I am working on setting up the Agile Tester’s or Software Development in Test workstation for true continuous test management, continuous testing and continuous test automation leveraging some of the new techniques I have learned.  

Thank you for reading!
Margaret Thomas has worked in the testing profession since 1996 and worked within a number of industries including mobile, finance, medical and insurance.  She developed a strong desire for automated testing when she began working within finance and has utilized a number of testing tools both commercial and open source.  She is also the author of the technical blogs found at


No comments:

Post a Comment