Tuesday, January 29, 2013

From Genesis to Programmatic Test Management




In the beginning….


…there was man.  No kidding.  Actually in the beginning of test automation efforts there was this concept of creating arrays that served as test data containers when we began growing up from record and play back to data driven testing.  Then we thought, this is ridiculous so we began using comma delimited files to reduce the stagnation of our automated scripts.  As our test automation efforts continued to mature to handle greater testing complexities, we required more formalized techniques for capturing test data (see article, TestCase Data Management).  In this article, I would like to share with you one of my favorite methods for not just capturing test data, but capturing test cases that can either provide the test data or make a call for test data.  My favorite method is to retrieve test cases from a Test Management or an Application LifeCycle Management System.  Going forward, I will refer to both as a Test Management system.

Why you ask?  Because I enjoyed the benefits of having one testcase library to support both manual and automated tests that can easily demonstrate the traceability to other project artifacts within our development process.  I also enjoyed having a dynamic library of tests to support continued growth, reusability and perform any type of desired test effort.

Many of today’s Test Management systems include support to both manage and synchronize an automated test run especially with their own functional test tool.  However, I often found it quite limiting and restrictive to support my desired test process.  If you find you desire greater functionality than the default interface provided by the Test Management system then may I recommend fashioning your own Test Management Service Layer that can provide you the desired interactivity features between your Test Management system and your automated tests aka TestFramework.

The Use Cases


Before we delve into the idea of developing a service layer, let’s understand the use cases (requirements) we used to support our endeavor.

1.0    As a tester, I prefer to utilize the same format and technique for developing both manual tests and tests designated for test automation.

2.0    As a tester, I would like to follow the same process for incorporating these tests into the overall development and testing process and linking these tests to other supporting artifacts, e.g. release, user stories/requirements, planning, defects, configurations, etc.

3.0    As a test automation developer, I would like to retrieve the test cases, as developed by the tester, and process them utilizing the TestFramework.

4.0    As a test automation developer, I would like to return the results from the TestFramework to the TestManagement system and provide all supporting documentation such as logs, actual test step results and screen shots for failed tests.

5.0    As a tester, I would like to easily analyze failed tests and generate a defect if the finding proves to be an application failure.

Please feel free modify to fit your unique requirements.

The Service Layer


Preconditions


a.       This service layer idea is based on the product SpiraTeam Test Management system.  The same idea can be applied to any Test Management system that provides a web service API.
b.       It was developed using Java.
c.        For this example, we omitted using log4j and outputted errors and exceptions to the console.
d.       The step for incorporating the wsdl into our project was omitted for this article.

Sample Service Layer Code


Below is a sample service layer developed to interact with a test management system that contains features we desire to have to support our TestFramework.  Although I didn’t include all of the features used to support our effort, you can build features to support activities such as:

·         Collect configuration information stored in the Test Management system to support a test run.  Reference the article Ready, Set...Go Test Run and read the section “The Configuration Matrix” for more ideas.
·         Automatically report failed test cases as defects.  See the article Automatic Bug Reporting for more information.
·         Sprint or release management.  See the article One Click Sprint Management.
·         Validate execution of a test case against a specific requirement set in the test case details.
·         Ability to provide detailed findings such as step by step recreation, actual results and providing supporting screen shots and logs.  See the article Test Automation Reporting for more details.

The additional benefits inherited by incorporating your Test Management system with your TestFramework is:

  1. Leveraging the given test case characteristics for reporting purposes; and
  2. Have the ability to leverage the Test Management’s systems features to support other aspects of the testing process as well as to support other processes which provides another method of interactivity with a project’s stakeholders.

The Sample Code

package se.servicelayer.importexport;

/*
The service layer imports the libraries from SpiraTeam to access the webservice api methods that will allow us to interact and/or manage the artifacts of the test management system from our TestFramework.
*/

import com.inflectra.www.SpiraTest.Services.v3_0.IImportExport;
import com.inflectra.www.SpiraTest.Services.v3_0.ImportExportLocator;
import org.datacontract.schemas._2004._07.Inflectra_SpiraTest_Web_Services_v3_0.ServiceFaultMessage;
import org.datacontract.schemas._2004._07.Inflectra_SpiraTest_Web_Services_v3_0_DataObjects.*;
import se.servicelayer.readerElement.ReaderListElements;
import se.servicelayer.readerElement.ReaderListTestSteps;
import javax.xml.rpc.ServiceException;
import java.rmi.RemoteException;
import java.util.ArrayList;
import java.util.Calendar;
import java.util.List;

public class Spirateam {

    /*
The class contains several constants that are available to all instances of the class to provide the use of the importExportLocator and the correct credentials to access the server.  The public fields can be unique for each instance of the class such as which project to use.  
   */

    private static final ImportExportLocator exportLocator = new ImportExportLocator();
    private static final IImportExport proxy = null;
    private static final String user = “st_webserviceuser”;
    private static final String password = “st_password”;
    private String project = “defaultProject”;
    private Integer projectId = 0;
    private Integer releaseId = 0;

//**********************************************************************************************************************
//Property constructor & initializers
    /*
The constructor sets the project, projectId and releaseId to reference in the Test Management system.
   */
   public Spirateam (changeProject, changeProjectId, changeReleaseId) {
       project = changeProject;
       projectId = changeProjectId;
       releaseId = changeReleaseId;
  }

    //We utilized the following methods to get or set the private class fields.

    public String getUser() {
        return user;
    }
    public void setUser(String user) {
        this.user = user;
    }
    public String getPassword() {
        return password;
    }
    public void setPassword(String password) {
        this.password = password;
    }
    public String getProject() {
        return project;
    }
    public void setProject(String changeProject) {
        project = changeProject;
    }
    public String getProjectId() {
        return projectId;
    }
    public void setProjectId(String changeProjectId) {
        projectId = changeProjectId;
    }
    public String getReleaseId() {
        return releaseId;
    }
    public void setReleaseId(String changeReleaseId) {
        releaseId = changeReleaseId;
    }


//**********************************************************************************************************************
/*
SpiraTeam connection functions

The connectToSpiraTeamProject method makes a connection using the given credentials set above for the current instance of the class.  It allows the subsequent methods created below to access the designated artifacts of the test management system after a successful connection to the system.
*/
//**********************************************************************************************************************

    public static Boolean connectToSpiraTeamProject () {
        Boolean connected = false;
        try {
            exportLocator.setMaintainSession(true);
            proxy = (IImportExport) exportLocator.getPort(IImportExport.class);

            if (!proxy.connection_Authenticate(user, password)) {
                connected = false;
                System.out.println("Authentication failure!");
            } else {
                connected = true;
            }

            if (connected) {
                RemoteProject[] remoteProjects = proxy.project_Retrieve();
                for (int i = 0; i < remoteProjects.length; i++) {
                    if (project.equals(remoteProjects[i].getName())) {
                        pId = remoteProjects[i].getProjectId();
                        break;
                    }
                }
                proxy.connection_ConnectToProject(pId);
            } else {
                System.out.println("Failed to connected to SpiraTeam project");
            }

        } catch (ServiceFaultMessage e) {
            System.out.println(e);
        } catch (ServiceException e) {
            System.out.println(e);
        } catch (RemoteException e) {
            System.out.println(e);

        } finally {
            try {
                if (proxy != null) {
                    proxy.connection_Disconnect();
            }
            } catch (Exception e) {
                System.out.println("Disconnect failed: " + e);
            }
        }
        return connected;
    }

   //On conclusion of the test run, the TestFramework can disconnect properly from the Test Management system.

    public static void disconnectFromSpiraTeam () {
        try {
            if (proxy != null) {
                proxy.connection_Disconnect();
            }
        } catch (Exception e) {
            System.out.println("Disconnect failed: " + e);
        }
    }

//**********************************************************************************************************************
//SpiraTeam document functions

    /*
The uploadFileToSpiraTeamTestCase was used to upload supporting documents or screen shots to failed test cases.
    */
//**********************************************************************************************************************

    public static Integer uploadFileToSpiraTeamTestCase (String testCaseId, byte[] bytes, String filename, String type) {
        Integer docId = 0;
        try {
            RemoteDocument remoteDocument = new RemoteDocument();
            remoteDocument.setAttachmentTypeId(1);
            remoteDocument.setFilenameOrUrl(filename);
            remoteDocument.setDescription("Error report, " + type + " for testcase id:  " + testCaseId);
            remoteDocument.setArtifactId(Integer.parseInt(testCaseId));
            remoteDocument.setArtifactTypeId(2);
            docId = proxy.document_AddFile(remoteDocument, bytes).getAttachmentId();
        } catch (RemoteException r) {
            System.out.println("Error adding file to SpiraTeam. " + r);
        }
        return docId;
    }

//**********************************************************************************************************************
//SpiraTeam Release Functions
   /*
createSpiraTeamRelease was used to support no touch smoke testing which could automatically create a new release in the test management system when a new build or updated test environment was detected.
   */
//**********************************************************************************************************************


    public static RemoteRelease createSpiraTeamRelease (String releaseId, Boolean iteration, String parentReleaseId) {
        RemoteRelease remoteRelease = new RemoteRelease();
        Calendar date = Calendar.getInstance();
        Integer rId = 0;

        if (connectToSpiraTeamProject()) {
            try {
                remoteRelease.setActive(true);
                remoteRelease.setName(releaseId);
                remoteRelease.setCreationDate(date);
                remoteRelease.setDaysNonWorking(0);
                remoteRelease.setEndDate(date);
                remoteRelease.setIteration(iteration);
                remoteRelease.setLastUpdateDate(date);
                remoteRelease.setResourceCount(1);
                remoteRelease.setSummary(iteration);
                remoteRelease.setStartDate(date);
                remoteRelease.setVersionNumber(releaseId);

                if (!parentReleaseId.equals("")) {
                    rId = getSpiraTeamRelease(parentReleaseId);
                }
                if (rId > 0) {
                    remoteRelease = proxy.release_Create(remoteRelease, rId);
                } else {
                    remoteRelease = proxy.release_Create(remoteRelease, null);
                }
            } catch (RemoteException r) {
                System.out.println("Failure creating Spirateam release for id:  " + releaseId + ", " + r);
            }
        }
        return remoteRelease;
    }



//**********************************************************************************************************************
//SpiraTeam Testcase Functions


   /*
getSpiraTeamTestCasesIdsByTestSet was one method used for retrieving the desired tests to be processed by the automated test run.
  */

//**********************************************************************************************************************

    public static int[] getSpiraTeamTestCasesIdsByTestSet (Integer testSetId) {
        int [] releaseId = null;
        if (connectToSpiraTeamProject()) {
            try {
                RemoteTestCase[] remoteTestCases = proxy.testCase_RetrieveByTestSetId(testSetId);
                releaseId = new int[remoteTestCases.length];
                Integer totalCases = remoteTestCases.length;
                totalCases = totalCases - 1;
                for (int i = 0; i <= totalCases; i++) {
                    releaseId[i] = remoteTestCases[i].getTestCaseId();
                }
            } catch (RemoteException r) {
                System.out.println("Failed to connect and get testcases");
            }
            return (releaseId);
        } else {
            return releaseId;
        }
    }
}

Conclusion


Some of today’s Test Management systems provide a robust web service api one can use to both manage and interact almost every feature of the system.  This allows the TestFramework to provide added support to lean and agile processes by reducing the need to perform mundane or repetitive tasks in addition to providing regular updates to the system.  This allows all stakeholders and teams to have up to date information about the progress of the overall test runs and development process from the testing perspective.  So from the inception of the test automation efforts of executing recorded scripts to perhaps utilizing a robust TestFramework solution to support some Test Management efforts, we can enjoy the results of our evolution.

Happy Automation!

__________________________________________________________________________________
Margaret Thomas has worked in the testing profession since 1996 and worked within a number of industries including mobile, finance, medical and insurance.  She developed a strong desire for automated testing when she began working within finance and has utilized a number of testing tools both commercial and open source.  She is also the author of the technical blogs found at http://mag4automation.blogspot.se/.

2 comments:

  1. eduExam is custom Online Examination Software to help examiners conduct tests and exams for student or employee a crosses globally. EduExam is a very powerful educational softwares for e-Learning and online education. EduExam is a blessing for both - the creators and the takers of the tests. The excellent analytical reports make quality improvement in organizations now at your fingertips.

    ReplyDelete
  2. Education is taking a new dimension in terms of learning or acquiring knowledge.There are e-learning tools available online with free online learning. Training-online is providing such a service. For more details contact us.

    ReplyDelete