Edit this page Non-Regression Automation for User Interfaces

Overview

NoraUi, for NOn-Regression Automation for User Interfaces, is a Java framework based on Selenium, Cucumber and Gherkin stack to create GUI testing projects that can be included in the continuous integration chain of single/multi applications web solution builds.
It ensures applications non-regression throughout their life taking into account code evolutions and acceptance of defined business client criteria.

Manage your data


Choose your Input Data Provider
Excel (xls, xlsx, xlsm), CSV
REST Web Services
SQL (MySql, Postgres, Oracle)
...
or implement yours

Added features


Reworked Selenium methods
Error management
Reporting
Conditioned steps
Loop steps
Configuration management

Run from anywhere on everything


Windows/Unix/Mac OS portability
Independent of front-end web apps technologies

Many, perhaps most, software applications today are written as web-based applications to be run in an Internet browser. The effectiveness of testing these applications varies widely among companies and organizations. In an era of highly interactive and responsive software processes where many organizations are using some form of Agile methodology, test automation is frequently becoming a requirement for software projects. Test automation is often the answer. Test automation means using a software tool to run repeatable tests against the application to be tested. For regression testing this provides that responsiveness.

There are many advantages to test automation. Most are related to the repeatability of the tests and the speed at which the tests can be executed. There are a number of commercial and open source tools available for assisting with the development of test automation. Nora-UI is probably the open source solution you can use. This user's guide will assist new users in learning effective techniques in building test automation for web applications.

This user's guide introduces Nora-UI, teaches its features, and presents commonly used best practices accumulated from the Nora-UI community. Many examples are provided. Also, technical information on the internal structure of Nora-UI and recommended uses of Nora-UI are provided.

Test automation has specific advantages for improving the long-term efficiency of a software team's testing processes. Test automation supports:

It is not always advantageous to automate test cases. There are times when manual testing may be more appropriate. For instance, if the application's user interface will change considerably in the near future, then any automation might need to be rewritten anyway. Also, sometimes there simply is not enough time to build test automation. For the short term, manual testing may be more effective. If an application has a very tight deadline, there is currently no test automation available, and it's imperative that the testing get done within that time frame, then manual testing is the best solution.


Concepts

Page Object Pattern

Page Object Pattern, the term that Selenium users keep buzzing. Page object is a design pattern that can be implemented as a Selenium best practices and also in NoraUi. The functionality classes (Page/PageElement) in this design represent a logical relationship between the pages of the application. Steps, regarding to them, encapsulate behaviors using one or several Pages.

The Page Object Pattern represents the screens of your web app as a series of objects and encapsulates the components represented by a page. It allows us to model the UI in our tests. A page object is an object-oriented class that serves as an interface to a page of your application unit tests.

Layered architecture

NoraUi framework introduces some concepts (Step, Page, PageElement, Selectors, packages...) all useful at different levels of your project. So let's explain quickly the different layers of NoraUi structure.

archetype
The 4 NoraUi architecture layers
  • Gherkin Scenario

    The Gherkin layer represents the functional actions to execute within your test. It is the highest abstraction level of your scenario.

  • Steps

    Steps layer represents all the glue code that will be available to be used from Gherkin scenario features. All Steps classes are included under the 'steps' package. This is true for both your project and NoraUi framework. The Steps will contain all your tests behaviors (web driver actions, checks...) that you can split into different classes and sub-package related to your business needs.

  • Pages

    Pages layer represents your target applications by defining their structures in PageElements that are used in step actions. All Pages classes can be distributed into differents 'page' sub-packages depending on the business domains they belong to. No behavior should be done in these classes except some minor validations. Behaviors are done in the upper layer (Steps).

  • Selectors

    The last layer is the one which is the closest to remote applications. It contains versioned .ini files by application which define how to retrieve a DOM element using the upper layer (Pages) PageElements. Several solutions are implemented like search by xpaths, class, id...

To conclude this point, every implemented scenario will run the following flow for each line:

Read the current Gherkin line

Found the matching Steps method to run

Get the concerned Page and/or PagesElement

Retrieve linked selectors

Go back up to the Steps layer to execute the action

So the framework is built to be used beginning from the top layer (closer to functional/business needs) to the bottom one (closer to target applications structures).

Enough of theory, lets get into practical implementation.


Installation Instructions

NORA-UI is a Maven project, You can use Noraui Archetype to create your robot.
In this chapter we will see together how to create a new Nora-UI project.

Archetype catalog configuration
Create or edit the archetype-catalog.xml file in your .m2 repository and the following content:

<?xml version="1.0" encoding="UTF-8"?>
<archetype-catalog
xsi:schemaLocation="http://maven.apache.org/plugins/maven-archetype-plugin/archetype-catalog/1.0.0 http://maven.apache.org/xsd/archetype-catalog-1.0.0.xsd"
xmlns="http://maven.apache.org/plugins/maven-archetype-plugin/archetype-catalog/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <archetypes>
        <archetype>
          <groupId>com.github.noraui</groupId>
          <artifactId>noraui-archetype</artifactId>
          <version>3.3.0-SNAPSHOT</version>
          <description>Maven archetype for use a NORA-UI project</description>
        </archetype>
    </archetypes>
</archetype-catalog>

On Eclipse:
New > Project... > Maven > Maven Project > Next > Next
Select Group Id: com.github.noraui, Artifact Id: noraui-archetype and Version: 3.3.0-SNAPSHOT
Next

archetype
Using NoraUi archetype in new Maven project Eclipse wizard

Finish


Quick Start Guide

Create a new test scenario

1. Create a feature file

In your src/test/resources/steps folder, add a new file with the extension .feature.
Name it as the tag it will contain. This tag will be used after to launch a scenario.
You can move your scenario in any folder structure below /step folder.

archetype
Adding a new feature in your project
The next step is to modify the src/main/resources/scenarios.properties configuration file in order to indicate the path to the feature file among existing ones.
This configuration file is used by NoraUi to map a tag and a path.
archetype
Setting up scenarios.properties
We can now go back to our feature and add some basic content and a new step.
Copy these few lines to MyNewFeature.feature.
@MyNewFeature
Feature: My new feature
  Scenario Outline: My new feature scenario
    When I click on 'mypackage.MyPage-myElement'.
    Examples:
    #DATA
    |id|
    |1|
    #END
You can notice that we used "I click on 'mypackage.MyPage-myElement'." step which comes directly from NoraUI framework.
When you write a feature, you have 2 possibilities when searching for steps:

Here is a piece of advice for Eclipse: there is a very useful Cucumber JVM Eclipse Plugin that you can install via "Install New Software" menu via this address: https://cucumber.io/cucumber-eclipse/update-site.
The plugin indicates for example, non-implemented steps via highlighting, or existing steps via auto-completion (Ctrl+Space) as you can see on the screenshot below.

To use external libraries as NoraUi within your Gherkin files, in Eclipse preferences go to Cucumber > User Settings. Then add 'noraui' root package to your classpath. Now your are able to use generic steps (click, type, check, open a page...) from NoraUi dependency in your own project.

archetype
Example of auto-completion for existing steps
You can find all methods here

Let's focus now on the structure of the added steps within our new scenario.
When I click on 'mypackage.MyPage-myElement'.
The above step allows you to click on a specific element of a web page. It applies for any Page, for any of its PageElement from any 'pages' subpackages of your project.

Then we want to continue on writing a more complex scenario in MyFeature.feature. So let's go back to it and complete the existing steps.
The objective will be to do a simple google research and check the results to find the first link.
The Gherkin content will look like this:

@MyNewFeature
Feature: My new feature
  Scenario Outline: My new feature scenario
    Given 'GOOGLE_HOME' is opened.
    # When I update text 'google.SearchPage-searchField' with '<search>'.
    # And I click on 'google.SearchPage-searchButton'.
    # The 2 above lines have been replaced due to google search page behavior change
    When I update text 'google.SearchPage-searchField' and type ENTER with '<search>'.
    Then I check that 'google.SearchPage-googleLink' is present.

    And I go back to 'GOOGLE_HOME'
  Examples:
    #DATA
    |id|search|Result|
    |1|test|
    #END

2. Create a Page class

In order to reference target Page elements, we need to instantiate a new SearchPage in a new google package (under application.pages).

Paste the following Java code into SeachPage class:

public class SearchPage extends Page {

    public final PageElement searchField = new PageElement("-searchField", "Search field");
    public final PageElement searchButton = new PageElement("-searchButton", "Search button");
    public final PageElement googleLink = new PageElement("-googleLink", "Google first link");

    public SearchPage() {
        super();
        this.application = "google";
        this.pageKey = "GOOGLE_SEA";
        this.callBack = Context.getCallBack("RESTART_WEB_DRIVER");
    }

    @Override
    public boolean checkPage(Object... elements) {
        return true;
    }
}

Your application pages must always extend abstract class Page.
From top to bottom, several instances of PageElement are created to make the link Gherkin feature.
The constructor defines the application this Page comes from (we will see after how to define applications in XXXContext class), its unique key and a callback in case of failure of any behavior related to this Page.
Finally the checkPage() method can be override to use your own way to validate the Page is correctly opened in the web browser driver (by checking current page title for instance).

3. Define selectors

Once your new Page implemented, the next action consists of defining the way to select its PageElements when it is targeted by the web driver.
There are several ways to look for a HTML markup in a DOM in NoraUi (CSS selector, link content, id, name, CSS class or xpath which is the most complete mode but quite more difficult to describe).

Under src/main/resources/selectors/V1, create a new file named google.ini and paste the following content:

[GOOGLE_SEA-searchField]
id=lst-ib

[GOOGLE_SEA-searchButton]
name=btnK

[GOOGLE_SEA-googleLink]
xpath=(//h3//a)[1]

All you have to know about selectors is here. We easily locate searchField and searchButton using id and name HTML attributes.
For the googleLink element, we use xpath to get the first link displayed on the results page.

4. Define your own steps

You should use as much as possible the NoraUi generic steps for basic purposes.
This can be made directly with Gherkin language as we have done before, or within your own steps. Steps definition are simple classes under application.steps package. These classes should extend Step class. From them, you can use Pages and code specific actions and expectations:

public class MyOwnSteps extends Step {
    
    @Inject
    private SearchPage searchPage;

    @Then("The GOOGLE search page is displayed")
    public void checkGoogleSearchPage() throws FailureException {
        if (!searchPage.checkPage()) {
            new Result.Failure<>("google", "ERROR_MESSAGE", true, searchPage.getCallBack());
        }
    }
In this sample, we retrieve the SearchPage from Page factory and use it after in an Gherkin exposed (@Then...) public method to do a simple validation. If the validation fails, a new Result.Failure object is created to manage error reporting, screenshooting and callback invocation (we will see error management and reporting in incoming parts).

You can go further by combining NoraUi generic methods calls inside one Step method.
Example:

Create a new method in your code that will simply call existing generic methods of the framework as below:
@Given("custom application action with click on '(.*)-(.*)', check value on '(.*)-(.*)' and update value on '(.*)-(.*)'")
public void customApplicationAction(String pageA, String toClickA, String pageB, String elementNameB, String pageC, String elementNameC) {
    clickOn(Page.getInstance(pageA).getPageElementByKey('-' + toClickA));
    if (!checkInputText(Page.getInstance(pageB).getPageElementByKey('-' + elementNameB), "RED")) {
        updateText(Page.getInstance(pageC).getPageElementByKey('-' + elementNameC), "GREEN");
    } else {
        updateText(Page.getInstance(pageC).getPageElementByKey('-' + elementNameC), "ERROR");
    }
}
"clickOn", "checkInputText" and "updateText" already exist. You can find all methods here

Then from the Gherkin feature file, you just need to call your code like this:

@MyNewFeature
Feature: My new feature
  Scenario Outline: My new feature scenario
    ...
    custom application action with click on 'package.Page-elementA', check value on 'package.Page-elementB' and update value on 'package.Page-elementC'
    ...
    Examples:
    #DATA
    |id|
    |1|
    #END

Sometimes you have no choice but to develop by yourself complex methods.

The principle stays the same:
@Then("custom application action with '(.*)'")
public void cancelOrNotAllSteps(String fooJsonString) throws FailureException {
    Foos foos = new Foos();
    foos.deserialize(fooJsonString);
    try {
        for (Foo foo : foos) {
            List<WebElement> rows = Context.waitUntil(ExpectedConditions.presenceOfAllElementsLocatedBy(Utilities.getLocator(myappHomePage.cancelStepList)));
            if ("to cancel".equals(foo.getActionOfStepToBeProcessed())) {
                for (int i = 0; i < rows.size(); i++) {
                    WebElement row = rows.get(i);
                    if (row.getText().startsWith(foo.getStepToBeProcessed())) {
                        selectCheckbox(myappHomePage.cancelStepCheckbox, true, i + 1);
                    }
                }
            } else if ("to achieve".equals(foo.getActionOfStepToBeProcessed())) {
                for (int i = 0; i < rows.size(); i++) {
                    WebElement row = rows.get(i);
                    if (row.getText().startsWith(foo.getStepToBeProcessed())) {
                        selectCheckbox(myappHomePage.cancelStepCheckbox, false, i + 1);
                    }
                }
            }
        }
    } catch (TechnicalException e) {
        new Result.Failure<>(fooJsonString, NoraRobotMessages.FAIL_MESSAGE_CLICK_ON_MY_APPLICATION, true, myappHomePage.getCallBack());
    }
}
    

In our google search page testing scenario, we don't need to create new Steps methods for the moment. We will only use existing framework ones.

5. Manage your applications

Open your XXXContext.java class and add the following code in it (see lines with '/** Added line(s) **/' comment):

...
    /** Added lines **/
    public static final String GOOGLE_HOME = "GOOGLE_HOME";
    public static final String GOOGLE_KEY = "google";
    private String googleHome; // GOOGLE home url

    /**
     * Constructor is useless because all attributes are static
     */
    private NoraRobotContext() {
        super();
    }

    /**
     * {@inheritDoc}
     */
    @Override
    public synchronized void initializeRobot(Class clazz) {
        super.initializeRobot(clazz);
        logger.info("NoraRobotContext > initializeRobot()");

        // Urls configuration
		/** Added line or use the NORAUI CLI directly **/
        googleHome = getProperty(GOOGLE_KEY, applicationProperties);
        logogameHome = getProperty(LOGOGAME_KEY, applicationProperties);

        // Selectors configuration
		/** Added line or use the NORAUI CLI directly **/
        initApplicationDom(clazz.getClassLoader(), selectorsVersion, GOOGLE_KEY);
        initApplicationDom(clazz.getClassLoader(), selectorsVersion, LOGOGAME_KEY);

		// Exception Callbacks
		/** Added line or use the NORAUI CLI directly **/
		exceptionCallbacks.put(GO_TO_GOOGLE_HOME, STEPS_BROWSER_STEPS_CLASS_QUALIFIED_NAME, GO_TO_URL_METHOD_NAME, GOOGLE_HOME);
		/** Added line or use the NORAUI CLI directly **/
		exceptionCallbacks.put(CLOSE_WINDOW_AND_SWITCH_TO_GOOGLE_HOME, STEPS_BROWSER_STEPS_CLASS_QUALIFIED_NAME, "closeWindowAndSwitchTo", GOOGLE_KEY, GOOGLE_HOME);
		/** Added line or use the NORAUI CLI directly **/
		exceptionCallbacks.put(CLOSE_ALL_WINDOWS_AND_SWITCH_TO_GOOGLE_HOME, STEPS_BROWSER_STEPS_CLASS_QUALIFIED_NAME, "closeAllWindowsAndSwitchTo", GOOGLE_KEY);
        exceptionCallbacks.put(GO_TO_LOGOGAME_HOME, STEPS_BROWSER_STEPS_CLASS_QUALIFIED_NAME, GO_TO_URL_METHOD_NAME, LOGOGAME_HOME);
        exceptionCallbacks.put(CLOSE_WINDOW_AND_SWITCH_TO_LOGOGAME_HOME, STEPS_BROWSER_STEPS_CLASS_QUALIFIED_NAME, "closeWindowAndSwitchTo", LOGOGAME_KEY, LOGOGAME_HOME);
        exceptionCallbacks.put(CLOSE_ALL_WINDOWS_AND_SWITCH_TO_LOGOGAME_HOME, STEPS_BROWSER_STEPS_CLASS_QUALIFIED_NAME, "closeAllWindowsAndSwitchTo", LOGOGAME_KEY);

		// applications mapping
		/** Added line or use the NORAUI CLI directly **/
        applications.put(GOOGLE_KEY, new Application(GOOGLE_HOME, googleHome));
        applications.put(LOGOGAME_KEY, new Application(LOGOGAME_HOME, logogameHome));
        
        Page.setPageMainPackage("com.your.company.application.pages.");
    }
	...	
    // home getters
	/** Added block or use the NORAUI CLI directly **/
    public String getGoogleHome() {
        return googleHome;
    }
    public String getLogogameHome() {
        return logogameHome;
    }

Here, we just have defined in the robot Context a new "google" application linked with a selectors file named "google.ini" and pointing at a URI named "google" in the robot properties file (src/main/resources/XXX.properties).
So, in this file, add the following content: google=${google} and in src/test/resources/environments filter files, you can simply set a value to the new google variable. An environment filter file should look like this (for instance 'dev.properties'):

#phantom,chrome,ie,htmlunit
browser=chrome
timeout=20
selectors.version=V1
display.stacktrace=true
logogame=https://noraui.github.io/demo/logogame/v3/
google=https://www.google.com/
http_proxy=
https_proxy=
no_proxy=
locale=en

These different filter files are necessary to run your scenarios on several platforms and manage the global configuration dynamically depending on the phase in which the robot is running. Commonly, the phases are dev for development, ci for continuous integration and prod for production if the robot aims for executing operations on applications in production.

6. Choose data providers

Before launching the test, we need to define one last thing, input and output data providers.
Input data providers are plugged to provide a data set on which the scenario will loop as many times as the lines it contains.
Several types are available: EXCEL, CSV, DB (ORACLE, MYSQL, POSTGRE), REST, GHERKIN. You can also create a specific one regarding your needs (see Custom data provider).
Output data providers are mainly used to write results of each run on the provided data set. Other information can also be returned to the user during the execution (for instance a value read on a page element).
Several types are available: EXCEL, CSV, REST, CONSOLE. You can also create a specific one regarding your needs (see Custom data provider).

For the rest of this exercise, we will configure GHERKIN as the input data provider and CONSOLE as the output data provider.
That means testing data will be included directly within the 'MyNewFeature' feature file and results will be written in the default console.

Update your XXXRobot.properties and change these lines to set providers as described above:

...
# type of input dataProvider (EXCEL, CSV, DB, REST, GHERKIN, noraui.data.xxx.YourCustomInputDataProvider)
dataProvider.in.type=GHERKIN
# type of output dataProvider (EXCEL, CSV, REST, CONSOLE, noraui.data.xxx.YourCustomOutputDataProvider)
dataProvider.out.type=CONSOLE
...

We are now ready to run our first test.

Run the scenario

1. From Eclipse run configuration

If you initialized the maven project by using the archetype, you should already have an available run configuration in Eclipse. Otherwise, it should look like this:

Eclipse run configuration
Eclipse run configuration

Replace "norarobot" by the name of your project.
Then by running this configuration, a prompt will appear asking for the name of the feature to launch (exactly the @tag associated to the feature file on the top of it). Type "MyNewFeature" and the scenario is starting.

2. From Maven command

If you want to run you tests from a command prompt, the Eclipse run configuration equivalent would be:

mvn clean verify -PscenarioInitiator,dev,norarobot -Dcucumber.options="--tags '@MyNewFeature'" -Dmaven.test.failure.ignore=true

Analyse results

As the input data are placed in src/test/resources/data/in, result files are placed in src/test/resources/data/out in the case you are using files as output (EXCEL or CSV).
For this tutorial, we chose the CONSOLE data output provider. So after the run, you should see the following content in the console:

[2017-09-24 13:08:43]: > When I update text 'google.SearchPage-searchField' with 'test'
[2017-09-24 13:08:43]: > When I click on 'google.SearchPage-searchButton'
[2017-09-24 13:08:44]: > And I check that 'google.SearchPage-googleLink' is present
[2017-09-24 13:08:44]: > And I go back to 'GOOGLE_HOME'
[2017-09-24 13:08:47]: ****************************************************************************************
[2017-09-24 13:08:47]: *                                                                                      *
[2017-09-24 13:08:47]: *  Scenario: [@MyNewFeature] step 1 of 1 with 0 error(s) and 0 alert(s). 0s remaining  *
[2017-09-24 13:08:47]: *                                                                                      *
[2017-09-24 13:08:47]: ****************************************************************************************

1 Scenarios (1 passed)
5 Steps (5 passed)
0m10,508s

In addition to providers, Cucumber generates 3 types of reports that you can find in target/reports folder. The HTML report is more appropriate for the majority of users. Json and Junit reports can be reinjected thereafter in other applications like dashboards or continuous integration tools (Jenkins, Travis CI...).

To go further

Take a look at nora-academy project to create a new testing project on a local JHipster application.


Command Line Interface

Nora-UI framework a CLI (Command Line Interface) for developers.

You can add a new target application, add a new scenario, ... using a command line.

Command Line Interface

cli

  • Add a new target application

    Add a new target application

  • Add a new scenario

    Add a new scenario

  • Add new model

    Add new model

  • Remove application

    Remove application

  • Remove scenario

    Remove scenario

  • Remove model

    Remove model

  • Encrypt data

    Encrypt data

  • Decrypt data

    Decrypt data


Official java doc

Nora-UI framework has a complete technical documentation for developers.

You can find all official java doc here


Continuous Integration

The use of Nora-UI is achieved through continuous integration.

In this chapter we will see together how to configure a Jenkins and reap the results on a daily basis.


OWASP Zed Attack Proxy (ZAP)

Nora-UI framework can be use to OWASP Zed Attack Proxy

You can find all official doc here

Nora-UI configuration (robot\src\test\resources\environments\dev.properties):

# proxy configuration
http_proxy=http://localhost:8087
https_proxy=http://localhost:8087
no_proxy=

Start Zed Attack Proxy (ZAP) on a particular address and port. ZAP provides a setting for that: Open Options and then Local Proxies and specify your proxy settings.
zap-local-proxies
Besides that it should be possible to configure the proxy used by ZAP like you would do for any other Java based application. ZAP provides a setting for that: Open Options and then Connection and specify your proxy settings.
zap-proxy-yourcompany


Custom data provider

You can create a custom data provider (Java language).

# type of input dataProvider (EXCEL, CSV, SQL, REST, GHERKIN, com.your.company.data.xxx.YourCustomInputDataProvider)
dataProvider.in.type=com.your.company.data.json.CustomJsonDataProvider
# type of output dataProvider (EXCEL, CSV, REST, CONSOLE, com.your.company.data.xxx.YourCustomOutputDataProvider)
dataProvider.out.type=com.your.company.data.json.CustomJsonDataProvider

You can find a sample here


Our clients

They trust us.

Our clients