Friday, September 19, 2014

How to create testing infrastructure?

There are many cases in automated testing where you need to write multiple tests which are basically the same, except for some minor changes.

For example, you need to write a test which inserts a value to a filed, and the value should change every time. If you write the whole thing in one test, a simple loop will solve the issue. But if you want a separate test for each value, you’ll need to create many test files that will include the test code and will differ only in the desired value.

Here is an example for a simple python script which takes its “values” from a simple JSON file and creates siesta tests for as many values as there are in the JSON file, a test for each value: (A short explanation regarding the values and flow of the script will follow)


#!/usr/bin/python
import json
import sys
import getopt
import time
import os.path


#######################################
## Declaring default values
#######################################


## This is the default template to generate test files.
testTemplate = """
    StartTest(function(t) {{
       t.chain(
          //here we click on the button which will open a window
          function (next) {{
             var fieldToClick = Ext.ComponentQuery.query('element')[0];
             t.click(fieldToClick,next);
          }},          
          //wait for the window to open
          {{ waitFor : 'componentQuery', args : "windowToOpen" }},
          //insert to the text field in the window a value
          function (next) {{
              var fieldToType = Ext.ComponentQuery.query('textfield')[0];
              t.type (fieldToType, '{0}', next);
          }}
        );
    }})
"""
## Default JSON input file name
inputJSONfile = '/home/user/values.json'

## The default template for test's file names
testFileNameTemplate = "Insert_Value_{0}_To_Textfield.t.js"


#######################################
## This is the script's actual code (The MAIN function)
#######################################


## Printing the final configuration
print ("#######################################\nConfiguration\n#######################################")
print 'Input JSON file is:', inputJSONfile
print 'Template for test''s file names:' , testFileNameTemplate
print 'Test template used for generating test files:', testTemplate
print ("#######################################\n")

## Open the attributes json file
with open(inputJSONfile) as data_file:   

    ## Parsing the JSON data
    data = json.load(data_file)

    # Loop on all the keys in the json
    for k in data.keys():

        ## Extract the attribute name for each record in the json
        valueToInsert = data[k]
                               
        ## Generating the file name
        fileName = testFileNameTemplate.format(valueToInsert)

        ## Creating the test file (This will OVERWRITE EXISTING FILES)
        ## Injecting the values into the test template
        with open(fileName, "w") as fo:
             fo.write(testTemplate.format(valueToInsert))
             
             print (fileName + " created.")



So what exactly happens in this script?

Each test clicks on an element, and insert a value into the textbox which opens.

The python scripts creates all the test files in the same directory from which it runs, and you can supply the JSON file path in the script itself.

The testTemplate is the code which you will be included in each test file.

The inputJSONfile is the path of the JSON file

The testFileNameTemplate is the file name for each test (and can include the value from the JSON, as you can see on this example).


You can manipulate the script and add more code and more parameters to it; the purpose of this post is to give you the main idea for creating such an infrastructure.

Tuesday, August 12, 2014

Continues Integration – how to run siesta via Jenkins?

As you’ve seen in the previous post (Running siesta from a remote computer), you can run siesta tests remotely. Moreover, you can create a Jenkins job which will run those tests for you, and will indicate for any failed test.

In order to do that, you need first to complete the setup in the “Running siesta from a remote computer” post, so that you’ll have server and client configuration.

After getting that stage done, you’ll need to:
  1.  Install siesta on your Jenkins machine.
  2.  Install java on your Jenkins machine.
  3.  Validate that you can run the webdriver commands from your Jenkins machine. it means that from the path where siesta is installed on your Jenkins, you need to go to the “bin” directory and run from there a test with webdriver. for example:

# webdriver http://IP/siestaTests --include=MytestFile.t.js --host YourHostName --port 4444 --browser=ie --report-format=JUnit --report-file=MytestFile_log_

(If you experience any problems, please refer to the “Running siesta from a remote computer” post).

After you successfully managed to run the webdriver command, you can start to configure the Jenkins job:

1. Access your Jenkins server -> click on “new item” -> choose the “Build a free style software project” option, and give the job a name.

2. In the job configuration window (which will open as you click on the “ok” button), check the “This build is parameterized” option.

3. The parameters are used for your comfort only, and they are NOT a must. I recommend configuring them so that you can run the same build with different options (such as different browsers or different clients) without needing to create a new job for each option.

4. You’ll need to configure 5 parameters, and in order to do so click on “Add parameter” button under the option you checked in step 2.

·         Parameter type:  Text Parameter
Name: CLIENT_NAME
Default value: insert here your client http address, which is the URL of your siesta GUI.
·         Parameter type:  Choice
Name: BROWSER
Choices: insert here the browsers you want to test on, with a separation of a new line between each.
·         Parameter type:  String Parameter
Name: SERVER_NAME
Default value: insert here your server IP address, which is the machine you want the tests to run against.
·         Parameter type:  String Parameter
Name: PORT
Default value: insert here the webderiver server port, which is 4444 by default.
·         Parameter type:  String Parameter
Name: TESTS_TO_INCLUDE
Default value: insert here the path of the tests you want to run. You can remove this parameter and just run all the tests. If you are using this parameter, note that the path you give here is relative to the index.js file.

5. Now add a very simple shell script which will run the webdriver command on your Jenkins. in order to do that you need to click on “Add build step” under the Build section on your job configuration.

6. Choose “execute shell” and in the window that opens, insert the following:

export JAVA_HOME={the path of your java installation, i.e. /opt/java/jdk1.7.0_51}
export PATH=$JAVA_HOME/bin:$PATH
{full location of siesta webdriver, i.e. /var/siesta/bin/webdriver } ${ CLIENT_NAME } --report-format JUnit --report-file-prefix junit- --browser ${BROWSER} --host ${ SERVER_NAME } --port ${ PORT } --no-color --include=${ TESTS_TO_INCLUDE }

7. Now all that’s left is to get the tests results in Jenkins. to do that go to “Post-build Actions” and click on “Add post-build action”. Choose “Publish JUnit test result report” and in the “Test report XMLs” field insert “junit-*”.

8. Click on “save”.


That’s it! All configuration is done, and from now on you can use siesta tests as a part of the continues integration process, by running the siesta-tests build before/after other builds.

Sunday, April 13, 2014

Waiting for components in Siesta

“waitFor” is one of the most important methods for testing UI elements, since there are many occasions in which there can be race conditions in UI testing (which are machine-induced and not user induced). For example, if you click on a button which opens a window with a combo-box in it, and you want to click on that combo-box - you must wait until the window is rendered to the DOM in order to click on its combo-box.

How can you make sure that the window is there? That’s what “waitFor” does for you – it waits until the elements you want are visible (or not visible) and only then runs the code after it.

There are a few options to set waiting times in siesta. The simple and basic option is to set the “waitFor” attribute for a certain amount of time:

StartTest(function(t) {
                t.chain(
{ waitFor : 2000 },
functionX ()
                );
};

This will force the test to wait 2 seconds before executing the next function in the chain.

Another option is to start the test only after a certain attribute is rendered to the screen. If you can query this attribute with ComponentQuery, you can also set the “waitFor” method to wait until it appears:

StartTest(function(t) {
                t.chain(
{ waitFor : 'componentQuery', args :  "button[itemId=Foo]" },
functionY()
                );
};

This will force the test to wait until it finds the “Foo” button and only then the test will execute “functionY”.

·         You can also wait for “componentVisible”, which will wait until the element will be rendered to the screen and not only until it can query for it.

Third option is the opposite - to start the test after some attribute is no longer appears on screen. For example, if there’s a progress bar on the page and you want some tests to run after the progress bar disappears, you can use a different form of the “waitFor” method:

StartTest(function(t) {
                t.chain(
{ waitFor : 'componentQueryNotFound', args :  "progressBar" },
functionY()
                );
};


This will make the tests wait until the progress bar (which has an xtype of “progressBar” in ComponentQuery) disappears and only then execute “functionY”. 

Sunday, February 16, 2014

How to run some code before each test?



Let’s suppose you need to run some general “code” (a few commands), like a login process or a button click before every test in siesta. You want that each and every test in your test suite will be executed from where the “general code” stopped.

For example, assume that there are two types of tests suites – the first test suite is running on a specific window, and the other is running from a different window. The windows are two different windows entirely, and each of them can be reached from a “main menu” which has two buttons – one button for each window. This means that you need a click on button “x” to be performed before the first type of tests, and a click on button “y” to be performed before the other type of tests. 

In siesta there isn’t yet a solution for this kind of problems, but we can overcome them by adding a new test class (extending the test class) and by overriding the isReady function, so before each test will start to run the code in the new test class will be executed. 

Setting the environment for a new test class

Now it’s time to configure the environment to use a new test class. There are several phases to do this:

1       1. You need to add the test class name to the harness.configure function in index.js file:

Harness.configure({
    title       : ‘This is my test Suite’,
    testClass  : Siesta.Test.Whatever,
});

NOTE: the new test class MUST extend the siesta test class, so its name should be inside the siesta name space. For example, names like “This.Test.Class” will not work, but names like “Siesta.Test.Whatever” will.

         2. You need to create a file which will contain the new test class, and locate it in the webapp folder. Since all the paths are relative to the index.js file, you can create it in the same folder as index.js located in.

3       3. Add this line (which is the new file you created) to the index.html file, right after the “siesta-all.js”  file and under “script” tags:

<script type="text/javascript" src="Pre-Code.js"></script>

          4. In the new class file, you need to override the “isReady” function. You can read more about the function here: http://www.bryntum.com/docs/siesta/#!/api/Siesta.Test-method-isReady

There are a few important things to note when using siesta to test ExtJS and when wanting to use Ext commands or siesta commands (like Ext.ComponentQuery or t.click):

  • The “isa” value should be Siesta.Test.ExtJS since we are testing ExtJS application.
  • When using Ext commands, instead of the usual “Ext.ComponentQuery” you should use “this.getExt().ComponentQuery”.
  • When using variables that are defined in the preload files, you should add “this.global” before their names.
  • When using siesta methods (like click or chain), you should add “this” before the method instead on the usual “t”.
  • After you finish writing the code you want to do before each test, don’t forget to set the value of “isCustomSetupDone” to true – this will start the test itself.

Saturday, January 25, 2014

Callback Method

If you’re a frequent siesta user, you are probably familiar with the “callback” parameter – you can find it in almost all common methods like “click”, “type”, “drag” etc.

Here is the official documentation:

·         callback : Function (optional)
A function to call after click.

In other words, the callback is a method to be called after the current operation is complete.

To better understand that let’s stop for a moment and explain how these common methods behave. When you call the “type” method in your test code some ‘magic’ happens: the browser will start the operation of typing as you requested and running the code that should run as a result of this. At the same time, your test code will continue to run and the next line in your code will be executed. This is best described as an asynchronous method because it is running “in the background” while your next line of code is already being executed.

Now suppose you want a click to be performed after a type operation. That means that you want the test to “wait” and execute the click only after the typing operation is complete. In order to do that you need to provide a function as a callback to the type operation, so when the typing operation is complete it will call this function which in our examples will perform the click.

Let’s take a look at some examples:

if you want to run click after type you can provide the “next” function as a callback in the type method:

t.chain(
       function (next) {
             t.type(typeField,'text to type',next);
       },
       function (next) {
             t.click(button, next);
       }
);      
                    
You can also call the “next” function yourself after a method that does not have a callback option, like the diag method:

t.chain(
          function (next) {
                  t.diag('Im going to click a button!');
                  next();
          },
          function (next) {
                  t.click(button, next);
          }
);
                          
if you want to write a function of your own which uses the callback option – for example, a function that clicks on a certain button, you can write this in your preload file (if you don’t know what a preload file is or why do you need it – go back to the previous posts):

function clickOnFooButton (t, callback) {
var fooBtn = Ext.ComponentQuery.query(‘button[itemId=Foo]’);
    t.chain(
function (next) {
                  t.click(fooBtn[0],next)
            },
            function (next) {
                  callback();
            }
    );
};

And on your test file you can simply call the “clickOnFooButton” function the same way as you would click on a button:

StartTest(function(t) {
     t.chain(
            function (next) {
                  clickOnFooButton (t,  next);
            },
            function (next) {
                   t.type(typeArea,‘hello’); 
            }
     );
};


This will make the test to click on the button which has the itemId of “Foo” and only after that, will type ‘hello’ to the typeArea.