DEVELOPMENT OF TEST AUTOMATION WITH DISTRIBUTED

DEVELOPMENT OF TEST AUTOMATION WITH DISTRIBUTED

Temirlan Shaikenov

student, Suleyman Demirel University,

Kazakhstan, Almaty

ABSTRACT

Nowadays quality assurance automation has many approaches to develop test automation. We can develop with using different frameworks, applications and programming languages. We can run automation tests in parallel and series mode. Many test automation developers do not know which approach to choose for their development. We are interested in developing test automation with distributed systems which run automation tests in parallel on multiple machines and multiple browsers. A key element of this work is the design of effective architecture with integration different applications that showing development of test automation with distributed systems. To distribute automation tests we used Selenium grid framework for running tests in multiple machines which containerized in Docker. All process we run with using maven in Jenkins Continuous Integration (CI) and Continuous Delivery (CD) environment. In results we made development test automation with distributed systems that run in parallel automation tests which speed up testing and executed on different browsers and operating systems at the same time.

Keywords: Quality assurance automation, distributed systems, Selenium grid, Docker, maven, Jenkins.

INTRODUCTION

Quality assurance (QA) area growing every day and it has many different approaches to design test automation environment. We can build automation tests with using Java, Python, JavaScript and others programming languages. Also, for these programming languages we have many different frameworks, applications and approaches. The general research purpose is to show how we can develop test automation with distributed systems. This approach will help to run automation tests in parallel mode, we can run tests on multiple machines with what we want version and operating system. If one machine broke, it will not affect to other machines. The importance of this topic is that we design effective architecture with integration different applications that showing development of test automation with distributed systems. This is help for test automation developers to develop effective and stable test automation environment.

To achieve this, we integrate applications Jenkins CI/CD, Docker containers, Docker images, Selenium grid, Eclipse integrated development environment (IDE), Maven, test runner TestNG and browsers. Jenkins CI/CD automation server that helps automate the development process related to building, testing and deploying. Docker is a platform that use OS-level virtualization with software in packages called containers. Selenium grid is a server that it run tests in parallel on multiple machines. Also, Selenium grid have main node called hub and workers nodes called Grid nodes. Hub node controls Grid nodes in Selenium Grid. Maven helps us to build Java projects in easy way. TestNG is a test runner which helps us to run automation tests.

We have different research papers about automation test area and they show basically how to develop in series mode test automation. Some papers explain about that how to use automation with different frameworks based on computer languages, and other papers explain advantages and disadvantages of methods in test automation area. This paper focuses on development test automation with distributed systems which helps to run stable tests in parallel. The remainder of this paper is structured as follows: In the section 2 we will have methods for this topic. Section 3 presents results and discussion of this framework and Section 4 gives us conclusion. Finally, Section 5 presents references and last section gives abstract.

METHODS

For making development test automation with distributed systems we need to download all application from official website. First of all, we work with Docker containers which you can pack and set up applications with all requirements like libraries, dependencies and databases. For configuring Selenium grid, we should to host multiple virtual machines as Grid nodes which connect to hub node. [4] Also, we need to download the Selenium server jar which we include to each machines where we run tests in Selenium grid. For the testers, it takes a lot of time to configure in this way and Docker containers helps us to solve this configure time problem. For integration Selenium grid and Docker containers we install images the hub and browser nodes into Docker containers. For this research we installed three images for running tests: Selenium hub image, Selenium node-chrome image and Selenium node-firefox image. [1] To find this images you can install from official Docker Hub website or from terminal with using pull commands:

-docker pull selenium/hub

-docker pull selenium/node-chrome

-docker pull selenium/node-firefox

After installing all images, we can check with using terminal command in Figure.1.

Figure 1. Docker images

For running Docker images, we need to configure Docker networking for recognizing hub and nodes by their container name. [2] We write this terminal command: docker network create name. Our Docker network name is grid. For running hub image, we need to specify port, Docker network and name in Figure.2.

Figure 2. Run Selenium hub images

For running browser nodes image, we need to specify port, Docker network, name of Selenium hub image, publish and subscribe ports and memory size in Figure.3.

Figure 3. Run Selenium hub images

We run main hub image node and three browser image nodes are shown in Figure.4 and Figure.5.

Figure 4. Docker running images

Figure 5. Docker running images status and ports

For checking that everything running we go to default local website “http://localhost:4444” where we saw three remote nodes are shown in Figure.6.

Figure 6. Selenium grid local website

In this way Docker containers helps us to create remote machines where we run automation tests with Selenium grid in Figure.7.

Figure 7. Selenium grid without and with Docker containers

For the next step we start to develop automation tests that will running in distributed way. [3] We need to create maven Java project where in default pom.xml file we need to install test runner TestNG and Selenium for integration with browsers in Figure.8.

Figure 8. Selenium and TestNG in pom.xml file

Also, we need to configure that this project will run from maven command. [6] We adding additional configuration to pom.xml and specify to run tests with using test runner TestNG in Figure.9.

Figure 9. Maven with test runner TestNG in pom.xml file

We need to create testng.xml file where we configure test runner that we run in parallel mode in Figure.10. [5]

Figure 10. Testng.xml file

Also, we need to create Java class where we can write automation tests for running. These tests will connect to Selenium grid hub. Remote web driver helps us in connection remote server. We send commands request to hub node and the Selenium grid hub distributed these automation tests by free browser nodes. In automation test we can specify which browser we can use, we can add additional browser arguments with using chrome options. We need to create chrome options object and all methods from this object we can use. Then we use Remote Web Driver where we provide Selenium grid URL and browser options. From this driver we can integrate and making somethings with browser for test in Figure.11.

Figure 11. Automation test in Java

When all automation tests created we make integration with Jenkins for running tests remotely from server. [7] For integration we configure and adding project path and maven command: mvn clean test. In Jenkins we run automation tests in distributed way in Figure.12. We can configure that Jenkins can be work locally and globally. For opening Jenkins local server, we go to local website “http://localhost:8080”.

Figure 12. Jenkins local server

RESULTS AND DISCUSSION

We made development test automation with distributed systems which helps to run tests in parallel on multiple machines and different browsers. With using Selenium grid, we had main hub node which control other nodes. If one or several nodes failed, Selenium grid hub will switch to another free node which work. This way gave us independent automation tests environment and also, this way show us distributing area.

We compared running time automation tests in series mode and in distributed mode. We found that distributed mode or parallel mode faster than series mode or sequential mode. We can save a lot of time when we run automation tests with distributed systems in Figure.13.

Figure 13. Series and distributed mode

We created 50 automation tests. One test for running took about 3 minutes. For series mode we multiply 50 tests by 3 minutes and overall time took 150 minutes. In distributed mode we have 3 remote containerized Docker machines and we divide 150 minutes by 3 machines and overall time took 50 minutes for running all automation tests in Table 1.

Table 1.

Automation tests

We expected this result that development test automation with distributed systems more effective than series mode running. We saved our time for running all automation tests. Also, we can check run tests in different browsers at the same time with using multiple machines.

Previous similar research papers show different frameworks or how to run in series mode but in this research we focused and understood how to develop automation tests with distributed systems.

In the future, we can say that this distributed method will be relevant because when quantity of automation tests increasing hence timing will increase, for decreasing timing of running automation tests we should use distributed mode for running on multiple machines.

CONCLUSION

We developed automation tests with distributed systems which helps automation test developers to design effective test automation environment. This way better than when we implement all the test cases in one machine at some point there might be some limitations, and sometimes one single machine will not be sufficient enough to run all the test cases. We run all automation tests in different remote machines and in different browsers like Chrome and Firefox. We decreased time of running automation tests and checked in different browsers.

In methods section we design automation test with distributed systems integrated with Jenkins CI/CD, Docker containers, Docker images, Selenium grid, Eclipse IDE, Maven, test runner TestNG and browsers.

In future work if we have a lot of automation tests, we can use Kubernetes for managing containerized applications at scale because we had Docker containers which containerized Selenium grid hub and browser nodes. 

References:

1. Software Testing, “Integrate Selenium Grid With Docker”, April 3, 2022. [Online] Available: https://www.softwaretestinghelp.com/docker-selenium-tutorial/
2. SeleniumHQ docker selenium, “Docker images for the Selenium Grid Server”, 2022. [Online] Available: https://github.com/SeleniumHQ/docker-selenium
3. Damm L.-O., Lundberg L.: Quality Impact of Introducing Component-Level Test Automation and Test-Driven Development, Proc. EuroSPI, Springer, 2007
4. Selenium dev, “Selenium Grid 4”, 2022. [Online] Available: https://www.selenium.dev/documentation/grid/
5. TestNG, “TestNG documents”, 2022. [Online] Available: https://testng.org/doc/
6. Maven, “Maven documents”, 2022. [Online] Available: https://maven.apache.org/
7. Jenkins, “Jenkins documents”, 2022. [Online] Available: https://www.jenkins.io/