The first test that we tried to automate was one that added a piece of clipart to a shirt and then made sure that the clipart could not be drug off of the shirt. It took two developers, three days to write one test and this is what it looked like:
Don't bother trying to read that, the point is that it was super complicated and very painful to write. Here are some highlights of it's most awful parts:
- Doing one thing always took multiple commands - A simple statement like "Load the lab' would often turn into 5 - 10 Selenium commands
- Copy and Paste == Selenium code reuse
- Selenium commands require xpaths or css locators that tightly couple your tests to the underlying implementation. Basically this means that every time an implementation detail changes (like the id of a div) all of your tests will break. Search and replace will then become your friend
This approach had several advantages:
- The keywords were easy to use and made sense in the context of our application. The words used here were the words of the business.
- It provides a layer of abstraction on top of the implementation details. In no place do be reference elements by name or use xpaths or css selectors. This is all handled in the DSL itself.
- No more copy and pasting, instead commonly copied and pasted Selenium code is rolled up into a single command that actually makes sense when you read it.
- The ability to actually use Ruby code when necessary. If you want to do one operation 5000 times you can with a simple for loop around that keyword.
The idea of a Domain Specific Language (DSL) is actually not a new one at all and in fact the term means different things to different people. In this case we are using it to mean a High Level Language, based on the language of your business, that is built on top of another high level language (Ruby in this case). Really it is a way for you to construct commands so that people can write programs that look and flow like a conversation about your business. This is why a DSL was perfect for our situation, we had a generic framework that we wanted to make test a specific application. That is what a DSL does, take a generic programming language and allow you to make one that is specific for your needs.
Our DSL is now around 80 commands and supports a couple of hundred tests. We run it on multiple operating systems on multiple browsers in parallel. Whole books have been written on DSL construction but here are our tips on getting started writing your own DSL for Selenium tests:
- The goal here is to make test writing easy, so start by writing up three or four tests that cover the basic core functionality of your application. Then read them out loud, see if they actually make sense. Only once you have these tests just the way you want them should you sit down and try to make them work.
- When you do actually start implementation, just worry about getting the basic functionality for your sample tests up and working. Once you have the initial commands working then add new commands as needed.
- Don't try to design all your commands up front - you will never be able to. The people who should be designing the new commands should be the ones who will be writing the tests - not a sub-committee.
- Always error on the side of adding another command. Yes you could make due with a command called "verify_position" and then just do the math to figure out if it is centered or not but that makes tests harder to write. Just go ahead and add a "verify_centered" command and make it easy to do forever.
- Every command should contain at least one assert. You want to assert as much as possible. Since you are writing a DSL anyway why not have each command assert that it is working correctly? Then, any situation that you use it in will always assert that the application is in the correct state.
Our next post is going to talk about the specifics of implementing a DSL using Selenium on Rails.
Tags: selenium, selenium testing,ruby on rails,
white box testing, selenium on rails, dsl, domain specific language, test writing easy, customink, tshirts, custom tshirts