Getting started writing DSLs using Selenium on Rails

In previous posts we have discussed the reasons for using Domain Specific Languages (DSLs) to make implementation of Selenium tests much simpler and more maintainable. This post is going to talk about how to go about actually starting to write Selenium DSLs using Selenium Core and Selenium on Rails. This technique relies on the fact that either you are testing a rails application or you have a method for ensuring that the domain that your application to be tested is on the same domain as your rails app that will host Selenium on Rails (the way that we have done this is by using Apache and mod_proxy).

For this example we will use one CustomInks internal Vendor Management system as the demonstration. Exactly what is being tested is not as important here, this same set of steps should work for anything - just how you lay out your DSL commands will differ.

Step 1: Install Selenium on Rails

The first thing to do is to install selenium on rails. This is easily done:

script plugin/install \
http://svn.openqa.org/svn/selenium-on-rails/selenium-on-rails

This will install both Selenium on Rails and Selenium-Core into your rails project. Next what we typically do (at least for the development stage of the DSL) is to make it so that Selenium will work in development mode as well as in testing mode. Normally Selenium on Rails will only function when rails is started in testing mode both to make sure that it was using the right database and to be sure that the tests were not accessible in production. This is accomplished by first copying the config.yml.example that comes with Selenium on Rails to just config.yml, so:

mv vendor/plugins/selenium-on-rails/config.yml.example \
vendor/plugins/selenium-on-rails/config.yml

Once this has been accomplished you then edit config.yml and uncomment the line under 'environments' that says development (line 9 in my version). So it would look like:

environments:
- test
- development

Now start webrick(or your rails server of choice) and point your browser to http://localhost:3000/selenium. You will see the basic Selenium startup page with no tests to write.

Step 2: Write a very simple test

Now lets write a simple test to make sure that things are working. We are going to author a RSel file, which is really nothing more than a ruby file with the Selenium API automatically loaded. Note: There are several additional ways to write tests in Selenium on Rails, using RSel is the way that we have found to be the most effective.

Selenium on Rails expects all of the RSel files to be in the directory: test/selenium. Selenium on Rails does not automatically create that directory so go ahead and create it:

mkdir test/selenium

Now create a new file in the test/selenium directory called yahoo.rsel. yahoo.rsel should contain one line:

open 'http://www.yahoo.com'

Save this file and point your browser back to http://localhost:3000/selenium. Now you should see one test called 'Yahoo'. Hit the Play button the right side and you should see the Yahoo home page load.

Step 3: Install basic infrastructure for your DSL

The simple Yahoo test case shows basically how Selenium on Rails works. The basic flow is as follows:

1. Write RSel test case
2. Load Test in Browser
1. Browser makes request to Selenium on Rails for Test Case
2. Selenium on Rails reads RSel file and compiles it to Selenium-Core table based structure
3. Execute Selenium Test in Browser
1. Javascript parses table
2. Evaluates Command
3. Loads www.yahoo.com
4. Tests finish

The important thing to realize here is that the RSel files are compiled into a static HTML table before the test starts to execute. This means that dynamic conditions in tests are not possible with this method (look into using Selenium Remote Control if you are interested in doing this).

A DSL really is just a series of custom commands that emulate your business language and model your domain. In this case we are writing a DSL for testing but the actual task of writing tests in your own DSL will happen in the RSel files. The way that CustomInk built our DSL was to expose language commands using a Ruby module, then automatically mix that module into all of the Rsel files. This allows your Rsel files to look like this:

load_lab
bird_clipart = add_clipart_to_product( '26602' )
verify_location on product_of bird_clipart, 0, 0
click_on bird_clipart
click_button 'Center'
verify_centered bird_clipart

To do this we first need to create a module in our Rails project where we can put all of our commands. To do this first make a directory under your Rails lib folder called Selenium:

mkdir lib/selenium

Now create a dsl.rb in the lib/selenium directory. We normally start with a template for our DSLs that looks like this:

module Selenium
  module DSL
   module DSLCommands
     # Insert your commands here
   end
   def method_missing( symbol, *args )
     if( ( not @delegate) )
       @delegate = Delegate.new( self)
     end
     args_for_print = (args.size == 1) ? args.first() : args.join(', ')
     header "#{symbol.to_s.titleize} #{args_for_print.to_s}"
     to_return = @delegate.method( symbol ).call( *args )
     header "End #{symbol.to_s.titleize}: #{args_for_print.to_s}"
     to_return
   end
   # Add a comment into the table
   def comment text
     @xml.comment! text
   end
   # Add a pretty header
   def header text
     command 'echo', text
   end
   class Delegate
     include Selenium::DSL::DSLCommands
     def initialize( parent_delegate )
       @parent_delegate = parent_delegate
     end
     def method_missing( symbol, *args )
         @parent_delegate.method( symbol ).call( *args )
     end
     def open( uri )
       @parent_delegate.open(uri)
     end
     def type( id, text )
       @parent_delegate.type(id,text)
     end
   end
  end
end

This template does a couple of things. First it adds in two additional methods to the Selenium on Rails API, header and comment. These actually used to be methods in Selenium on Rails but aren't anymore (for whatever reason). Header inserts a header into the table (creating comments that can be seen) and comment inserts a HTML comment that can not bee seen. Both of these are used as debugging methods. This template also makes it so that every DSL command that you write will be generated with a header comment before and after all of the individual Selenium commands. This also allows for easier debugging.

Finally we need to modify Selenium or Rails to automatically include our DSL so that all of the tests we write have access to the new commands. To do this you need to monkey patch: vendor/plugins/selenium-on-rails/lib/selenium_on_rails/rselenese.rb. You need to make two edits. First, on the first line of rselenes.rb add:

require 'selenium/dsl'

Then around line 15, make it so that SeleniumOnRails::RSelenese includes Selenium::DSL. So the class declaration should now look like this:

class SeleniumOnRails::RSelenese
attr_accessor :view
include Selenium::DSL

Or approximately anyway. What this does is to make all of your DSL command top level Rsel commands that can be accessed as though they were built directly into Selenium.

Step 4: Write your first DSL commands

Now everything should be in place to write your first DSL command. A DSL command in this context is nothing more than a Ruby method that is designed to fit together with a set of other methods to make a coherent sub-language. At the end of the day your Rsel files are still Ruby (meaning that you get all of the built in Ruby language constructs) but you have developed your own set of commands (or keywords) that more closely model your business language. Remember that Selenium is all Javascript and in order to avoid cross-domain security problems with Javascript your Selenium suite and your application must be on the same domain. Hence the reason that the examples that follow do not test www.yahoo.com.

Lets start with something to load our application, since we are testing CustomInks internal vendor manager the command will be: load_vendor_manager. More broadly lets imagine that we want to test the fact that vendor capacity (the amount of work that a given vendor can be given) is set correctly. Before we even start writing DSL commands we imagine what the test should look like and then write the test with no implementation. This is an important step. Always remember that the goal of your DSL is to make test writing easy, not to make DSL implementation easy. So here is what we want our test to look like:

load_vendor_manager
select_group "East"
select_vendor "Digital"
verify_vendor_capacity "15"

The nice thing about this test is that it seems to be obvious about what it is doing, it works in the language of the business and introduces commands that should be useful in other tests. To start the actual implementation go back to lib/selenium/dsl.rb and find the comment that says: Insert your commands here (around line 4). This is where the Ruby code that implements the DSL belongs. Here is our implementation for load_vendor_manager:

def load_vendor_manager()
command 'setTimeout', '60000'
open '/backend/vendor/details'
wait_for_element_to_not_exist 'id=vendorLoader_c'
assert_element_not_present 'id=vendorLoader_c'
assert_element_present 'id=vendorGroups'
end

This is a simple command that makes first sets the timeout, opens the page, waits for the pre-loader to disappear and then verifies that the page loaded correctly. Notice how we directly refer to element id's in this command but not in the tests. This is important as it separates out the actual implementation of the application from the tests so that as the application changes the tests don't have to be modified, only the details of the DSL have to be modified. Now if we add another Rsel file under test/selenium with the command load_vendor_manager in it Selenium on Rails will execute the more finely grained Selenium commands provided in the implementation and load the vendor manager.

Notice that in this command we also verify that the vendor manager is in a steady state - we wait for the page to completely load and then assert that it is correct. This is important to do. Since the complexity is hidden behind by the DSL, the more asserts that are baked into the DSL the more confident you can be of the state of your application.

The actual code that we implemented for load_vendor_manager is not what is really important here though. What is important is that this provides a very simple framework that you can add an arbitrary amount of complexity to in order to make test writing ease. The resulting tests are straight forward and complexity is hidden in the DSL.

Hopefully this helps people get started writing DSLs and easy to maintain Selenium tests. Our next post will cover more techniques for the implementation of DSL commands in Selenium on Rails.


Tags: selenium, selenium testing,ruby on rails,
white box testing, selenium on rails, dsl, domain specific language, customink, tshirts, custom tshirts