agile

Automatic testing with Selenium

Quality Assurance: Automatic testing of web applications with Selenium

 

Introduction

Every web application needs proper testing to ensure that it works like expected. Therefore you should have a set of test-cases, that verify the functionality of the application. For large applications you often need a separate test planning phase, where many test cases are designed – there can be  many hundreds of them.

A proper design of stable and useful test-cases is required – but that’s not part of this article. But at least we can say that:

  • Most of the test-cases should focus on a single feature or requirement
  • In order to keep the overview test cases should have speaking names
  • test-cases should be grouped in test suites

Before a new release of a website or web-application is released, the test cases should be executed to ensure the quality of the application. For business critical websites, this phase is even more important! It’s obvious that this process can take a lot of time and effort. If you want to keep up with your competitors you do not only need a robust and high quality application - but you also need a short release cycles that adds new innovations and features to your website.

 

The key to solve both requirements, short release cycles and proper quality assurance, is automated frontend testing. Automated tests for web applications are furthermore also very useful in agile development processes: In an optimal way they can be used together with continuous integration, to provide early feedback for changes in the source.

Getting started with Selenium

Selenium is a suite of tools to automate web application testing across many platforms. The easiest way to start with it, is to install the Firefox add-on Selenium IDE. With this IDE you have an easy way to record and edit your test cases.

Let’s start with a simple test:

1)      Open your IDE and select "new" > "testsuite". A new test suite is created, with a first empty test-case.

2)      Now just click the red record button and switch to your browser window.

3)      Open www.google.de

4)      Do a search for "typo3".

5)      On the result page highlight the website "typo3.org" and click the left mouse button. Select "assertTextPresent" from the context menu. (screenshot)

6)      Now save your test and give it the name "can search for typo3.org".

7)      Save your test suite and give it the name "google search".

 

Every time you want to check if google still can search for typo3 and returns the expected result "typo3.org" you can run this test case. Do this by clicking the play button in the IDE.


Speaking Selenese

If you have a closer look at your test case, you see that it consist of a list of commands. The Selenium IDE records your actions in the browser and adds the correct commands to the test-case. All commands support one or two parameters.

In fact the commands create a testing language - and they are often referred to as "selenese". For having robust test cases, it’s often necessary to tune your test case commands after recording them with the Selenium IDE. Robust means that the test cases are stable against small changes in the markup. Also while running a test case, selenium executes the commands faster than a human  - that might cause unexpected behavior, of course a test case or the application should be insensible against this too.

Therefore let’s have a closer look at the commands. In general commands are divided into actions, accessors and assertions. As mentioned commands can have 1 or 2 parameters. In most cases the first parameter is a locator to a specific UI element and the second parameter is a text pattern.

Actions do something in the browser, like clicking (command click) a link or selecting some Selectbox entry (command select). Mostly it is necessary that your test-case will wait after an action, until the web application is ready doing something. Most of the action commands can be used with the suffix "AndWait". That means that Selenium waits till the server has send back the response for the action. (See below for an example how to deal with ajax)

Assertions are mainly the verification* or assert* commands (Like verifyTextPresent). The difference between assert and verify is, that a failed assert ends the test case and a verification only marks the test case as failed - but the following commands are still executed.

Simple Selenium test case example

Let’s have a look at a simple test-case example

1

open

/download/

 

2

assertTitle

Downloads

 

3

verifyText

//h2

Downloads

4

clickAndWait

link=next

 

5

assertText

 

showing*page 2

6

 

 

 

In this testcase we start to open the url "/download/" (btw. the test case can be executed on different base Urls). The next commands checks that the title of the page is really "Downloads" (2) and that a text "Download" is somewhere in a H" Tag on the website(3). After that the link with the text "next" is clicked (4), and after the page is reloaded we assert that somewhere on the website a text "showing*page 2" is present. (Where the asterix is a wildcard).

A more advanced selenese example

Let’s have a look at a more advanced example:

1

open

/ ajaxtest /

 

2

assertTitle

*ajax*

 

3

click

link=glob:*ajax

 

4

waitForElementPresent

//div[@id="ajaxresult"]/span[contains(@class, 'activated')]

 

5

pause

1000

showing*page 2

6

store

javascript{'test_'+ new Date().getTime()+ Math.floor(Math.random()*1000)}

$userName

7

echo

 

generated Username: ${$userName}

8

type

id=login

${$userName}

Again we start with opening some URL. Than a click action is performed on a link that has a text ending with "ajax" (3). In AJAX applications you cannot use the "*AndWait" actions, since the website is not reloaded. Here it is better to place a seperate WaitFor* command after the action command. That’s what we did on line (4), we expect that after the Ajax Response is finished when a "span" element with a class "activated" is present. The "pause" command right after a waitFor* command tends to be useful to have a better robustness of the test case.

The next commands show what else is possible. In line (6) a random value is stored in the variable $userName. This variable is used in the echo command on line (7) and then an inputfield with the id "login" is filled with this value (8).

 

Interesting are the possibilities of the different locators. Here some examples:

  • Using the identifier locator like: “identifier=loginForm”  (identifier prefix is optional)
  • Using CSS Selectors: “css=.content div.grid”
  • Selecting a link with a special text: “link=my link”
  • DOM Selektors: “dom=document.forms[0].username”
  • Using XPath Selektors: “xpath=//form[@id='loginForm']” (xpath prefix is optional, a “//” is sufficient)

Whats next

If you want to run your tests in other browser you can use Selenium Core. Selenium Core comes with a HTML based Test-Suite Runner that can simply be called from other browsers too:

 

 

There are more useful Selenium Tools. First to be mentioned is the Selenium RC Server. This is a Java based Server, that can run tests on multiple platforms and browsers. The cool thing is that you can connect to this server from other applications. So it is possible to run Unit Tests that use Selenese to test your web application.

The TYPO3 extension “selenium”

The TYPO3 extension “selenium” can help to build Selenium Tests for TYPO3 extensions. It has:

  • A backendmodul that scans for *TestSuite*.html files inside “tests/selenium” folders and links them with the Selenium Core Runner. So you can easily start Selenium tests from the backend of TYPO3.
  • In the extension configuration you can configure a Selenium RC Server (default port 4444) and you can use this in your phpunit tests.
  • Therefore two abstract testsuite classes are provided.
    • tx_selenium_phpunittestcase: Uses the Selenium Testcase class that is part of phpunit.
    •  tx_selenium_testcase: Uses the PHP selenium Driver, that is part of the Selenium RC package.
    • Each of this classes has a method initializeSelenium($browser,$url) that instanciates the $this->selenium based on the configured RC Server.

 

 

 

 

Links

http://seleniumhq.org/

http://seleniumhq.org/docs/

 

blog comments powered by Disqus
  1. Jarrod http://www.searchthelake.com 29.03.11 05:53

    I like the idea of having this automated. I just had a new site built that we launched 3 weeks ago and we are still finding bugs in it. It would be nice to have an application that could test drive the site and find these kind of things but I'm not sure it could work in my case.

  2. Daniel 19.03.11 19:22

    Well - you can assert for nearly anything (elementVisible, elementPresent, alert boxes ...)
    So theoretically you can test all functionalities - the only thing that cannot be detected is design and layout bugs

  3. shingles info http://www.checkshinglessymptoms.com 14.03.11 13:30

    The core assertion are that either text is present, or that it is not and it can get a very long way, it is no silver bullet as the application can break in many ways!

  4. Michele http://www.michelebaratta.com/ 19.10.10 16:02

    That was a good read. I find writing selenium tests very tricky. The core assertion are that either text is present, or that it is not and it can get a very long way, it is no silver bullet as the application can break in many ways!

  5. Freelance web designer http://www.echofreelance.com/ 23.07.10 09:21

    This is interesting to know that selenium is a power tool of testing.

  6. Andreas 13.07.10 15:18

    Thank you for the article.

    Point 5 can be improved (see underscores):

    5) On the result page highlight the _url_ "typo3.org" and click the _right_ mouse button. Select "assertTextPresent" from the context menu. (screenshot)

    And the screenshot is a bit small. ;)

  7. Dirk http://www.steffes-tun.de 08.04.10 08:51

    THX for this selenium idea. Will check it out!

  8. Daniel 06.10.09 21:10

    Hey

    Its now in TER:

    http://typo3.org/extensions/repository/view/selenium/current/

  9. Bastian 06.10.09 17:03

    Nice article, thanks!
    Where can we find the "selenium" extension?

blogroll