What is Continuous Testing

Continuous Testing Developers? Wait what ...

We’ve been talking about the "Nirvana" of Continuous Testing for a very long time now. I remember...

A story that I heard last year about how students in the Navy SEAL program are expected to make their own beds every morning. It seemed like an obvious task at first, but then thinking more about it, how often do we do that? Maybe just push it to a bit later or not do it at all considering that you’d be back at night? A tiny habit of achieving the first success of the day – making your bed so that you can move on to other tasks with the idea that you’ve already had one success that day! So how does this tie into the testing concept?

In software development world, it is easy for QA/QE testers to say "let the Dev folks finish their delivery of the code and then we will do the testing". However we’ve heard from so many customers who are on their DevOps journey that QA/Testing is ‘the bottleneck’ which consumes much of the cycles in their SDLC. There have been numerous research topics on how much savings are seen by shifting left – that means Testing is done at various times in your SDLC shows a huge benefit. Testing done at various times equates to the concept of Continuous Testing (CT) which is an integral aspect of what needs to be done throughout the SDLC cycle rather than wait towards the end.

So the obvious conclusion is that even though Continuous Integration (CI) and Continuous Delivery (CD) seem to accelerate software releases, Continuous Testing (CT) is an integral part that will help deliver quality of your product!

I whole heartedly believe that as more and more organizations are moving into shorter development/testing cycles by adoption of newer processes, such as Agile and DevOps, it becomes important for each of us to adopt better and optimized development and testing practices, such as Continuous Testing. Before we go into deep let me start from the beginning.

What is Continuous Testing?

What you may not realize is that the concept of Continuous Testing has been around for nearly 15 years now, first presented in a paper titled "Reducing Wasted Development Time via Continuous Testing" at the 2003 International Symposium on Software Reliability Engineering. The concept was really simple to use spare CPU resources to continuously run test in the background which provides "rapid feedback about test failures as source code is edited".

A more updated and practical definition of Continuous Testing was presented in a recent SD Times article titles "Continuous testing enabled the promise of Agile". What I particularly like about the article is that it gives a simplistic view of the term, where it is "testing that happens continuously and automatically as code is developed".

Why are more people not doing Continuous Testing?

I wonder why more people are not doing continuous testing and the best reason I can come up with is because of testing constraints, environments, issues, anything that is keeping our testing colleagues from being able to test, and continue developing for that matter.

A 2017 Computing Research report shows that "63% of companies developing software, have had delays in Test/QA stage of their development cycle". In the ideal world of Agile and DevOps there shouldn’t be such a bottleneck because, in theory, testing should be done continuously, rather than at the end of the cycle. Don’t you think?

So why is it that testing is not able to be done continuously?

Well, it may be because the component you want to test is not yet available or you are dependent on other teams to finish their work before you can begin your testing. Another reason for our inability to test is the fact that we may lack the appropriate test environments (Mainframe, 3^rd Party, Performance, or other) or even lack the connectivity to internal or 3^rd party APIs that constrain our ability to test continuous. A third possibility may be the lack of test data for you to quickly and continuously test. (This will only get worse with GDPR, don’t you think?)

Enter Service Virtualization for Continuous Testing

We’ve been working with Service Virtualization for over 10 years (over 6 years since we acquired our solution from ITKO) and believe that it’s only by using Service Virtualization that organizations will be able to test continuously.

Before I go any further, let me explain what Service Virtualization really is and how it is different than mocking and subbing, which you may be more accustomed to. Service Virtualization is the practice of capturing and simulating the behavior, data and performance characteristics of dependent systems, essentially creating a "virtual copy" of those dependent systems. These virtual facsimiles behave "just like the real thing" and can be used independently of those actual live systems to continue working, even testing continuously.

The litmus test of a Virtual Service is that the system consuming the service cannot tell it is not the real system, while Stubs and Mocks fail this test because they lack robustness needed by the consumer of the service.

Service Virtualization takes a more modern approach to the traditional practice of mocking and stubbing. Application developers and quality engineers using Service Virtualization can enjoy underlying services with dynamic and robust capabilities, sufficiently similar to the "real" underlying systems, but without depending on the complexity and constraints of real hardware and software. These virtual services allow consumers of servicing being written and consumed in parallel to "shift-left", or remove linear project dependences to afford earlier delivery in the software development lifecycle.

With Service Virtualization, development and testing teams can overcome many constraint obstacles, including these three I mentioned earlier.

• Missing Components – with the ability service virtualization provides you to create facsimiles of other services, components or systems, sometimes even better versions of the real thing, you then have everything you need when you need it in order to continue development or testing work, without depending on other people’s work.
• Unavailable Test Environments – so the excuse of having to wait for test environments, system or access won’t fly anymore. With Service Virtualization, you are able to isolate the system under development/test from all of the back end systems, essentially freeing it from having to access any restricted or unavailable test environments or systems.
• Lacking Test Data – setting up, running through and refreshing test data in no longer an issue when using Service Virtualization, because you almost don’t need a separate set of data, given you have virtual services that have the same characteristics, and even the data profile of the real thing. So, no need to manage test data samples and protect them with your life before someone takes your sample and modifies, alters or expunges it without knowing they are yours.

Now that you know how powerful Service Virtualization can be to help you overcome these and many other developing or testing constraints, what is keeping you from testing continuously?

Two free ways to start with Service Virtualization from CA Technologies (Shameless Plug)

Your excuse to not using Service Virtualization cannot be that you can’t afford the solution... because I have a sweet (and free) proposition for you. And here is my shameless plug to two free tools CA Technologies has made available to the developer/tester community:

CA Service Virtualization Community Edition – This is a self-contained, free service virtualization tool that was optimized for HTTP/HTTPS web services and common protocols like SOAP, REST, JSON and XML.

With Community Edition you can create and save transactions by invoking REST APIs and SOAP-based web services. You can also use live traffic recording, API design, Request/Response Pairs or network captures to create virtual services.

CodeProject has a more in-depth product review here in case you want more details on the product. For your free copy of CA Service Virtualization Community Edition visit our product page.

CodeSV – This is another free Java library we created as a lightweight way for developers to define and use virtual services within your own JAVA IDE.

CodeSV allows you to create and run HTTP virtual service definitions directly in your unit testing code and integrate virtual services into your CI/CD process.

CodeProject also had an in depth look at CodeSV if you like here. This tool is available for free download from www.codesv.io

Here is an example of how to use CodeSV.

Firstly to setup CodeSV, you can choose from a bundle of choices - Maven, Gradle or manual setup. Our Quick Start Guide on the CodeSV GitHub page will guide you thru the steps. In this case, let’s say I am in my IntelliJ IDEA and choose to create a Gradle project as a first step. Once the project is created, there are 2 simple steps required to get started –

1. Add Maven repository to our build.gradle file
2. Add CodeSV dependency definition

Next step is to create a Unit Test by creating a Class

Before moving on we need to make sure the Test class is configured to work with CodeSV

1. Add a couple of import statements
2. Add a VirtualServerRule class member inside your Test class

Now we are ready to write our first Unit Test. We will simulate the example of calling a service still under construction. Let’s create a test that will send a request to a non-existent endpoint. The service will be available at the following URL as defined below –

Now let us create a test that calls the non-existent endpoint that we just defined. Inside the test, create a URL object to open a HTTP Connection

Then we will set the Request method – here it is the Get method. To make it easier for us, we will print out the response code

Now let’s set the Response body. Close the reader and print out the response.

Now let’s run the Test to make sure it fails. Since we have not virtualized our service yet, we are making a call to a non-existent entity. You will see that the result is unknown host exception.

To overcome this, we will use CodeSV to virtualize and return a meaningful response. So let’s create a virtual service on our custom endpoint. This service will virtualize a Get request which is sent to our custom URL. We want it to return response code of 200 and with a string body of ‘Success’.

Now let’s run the test again. This time the test will pass. As expected the virtualized Get request was sent to the custom URL and returned the response we just defined. The response code is 200 and the body contains the ‘Success’ string.

Wait - there’s more to this! Examples on how to virtualize forGet(String url), forPost(String url), Request-Response pairs export, support for HTTPS and more please refer to http://codesv.io/index.php/how-to-videos/

Now that you’ve read this post I hope you agree with me that Continuous Testing is imperative to our organization’s success and our ability to develop, test and deliver better software faster. I hope that you’ve learned more about Service Virtualization and how it can help with Continuous testing. I want to hear from you (Koustubh.Warty@ca.com) to let me know what you think of CA Service Virtualization Community Edition and CodeSV after you’ve have taken them for a spin.