Adventures in Functional Tests
In lieu of writing something overtly informative, I would like to provide a sort of narrative about what I’ve been up to lately, in hopes that there is some inherent lesson there. Maybe we can even have some fun along the way. The truth of the matter is, I am no expert on functional testing, but I’ve done a good deal of learning and growing over the past month in hopes of providing some value to the company that has given me the opportunity to expand the scope of my position. It’s a story about me, from me; so here I will preemptively apologize for any excessive self-indulgence.
The Premise
The area of opportunity I was given to explore was writing functional tests for our catalog of wickedly dynamic and ever expanding web applications. As we are always growing, improving, and changing our solutions, we need a way to ensure that they are making it to our clients in tip top shape; that existing functionality is not broken and new functionality is performing in the manner it was designed to.
Understanding that we would like to manually touch every part of an application before its every release, sometimes the demands of the market and constraints of a given sprint only afford us the opportunity to physically touch so much of it.
At Mercury New Media, we like to live the DevOps lifestyle, which encompasses the idea of continuous delivery. Continuous delivery promises new features and bug fixes into a production environment for use in a timely, safe, and reliable way. One way to accomplish this is by employing automated testing of applications.
How We Do It
For our functional tests, we utilize Selenium, C#, Nunit, and Visual Studio. With these technologies I was only barely familiar or had no familiarity at all, so there was some homework that I needed to do. Luckily, I work with a team of All-Stars, and I wasn’t the first person to write functional tests at the company, so I did a little bit of leaning to supplement my research.
My first assignment was to write tests for an existing application of ours to ensure the functionality would remain intact with new releases. I literally copied a previous test in another part of the application and went through it replacing browser elements that corresponded to the new section I was working on.
To prepare for this new assignment of mine I had been cramming C# in hopes that I could fumble around the application in some sort of competent manner, but what I found was that most commands were utilizing Selenese and browser elements. The Selenium WebDriver sends commands to the browser, looking for elements that you name on the page, and performing some sort of action that you dictate in the code. So basically, I tell Selenium to go look for this field, input some text in that field, and go to this button and click that.
With the appropriate references in the solution, and using Visual Studio’s IntelliSense I was able to figure out some neat ways to find and interact with elements on the page. W3Schools provided a nice crutch for helping find the tough-to-get-to elements. I struggled with getting tests to pass because of element specificity, and learned the concept that sometimes searching by an ID is more malleable than pulling out a complicated Xpath. I found some helpful classes by John Sonmez in the Pluralsight library to boost my skills here. I utilized some of the C# I had learned and threw in a new method here and there to automate some repeatable tasks and make myself feel like a real programmer!
But Wait, There’s More
Selenium isn’t just a language for sending commands to a browser, Mercury also utilizes a Selenium grid for capturing videos of our tests in action. Thank goodness for this fact, because without being able to check and see exactly where some of my test were failing, I would have had no idea where to go next. To put it very simply, because that is how I understand things, a Selenium grid establishes some virtual machines that are browsers waiting on these Selenese commands. So I create a test, and once that test runs it will send the Selenese commands to a virtual machine waiting on the grid. The machine, or node, will get the commands, execute the test, and record the results. In the code I can see where a test is failing, for example if the driver couldn’t see a certain web element. Perhaps it couldn’t see that element because a previous element hadn’t been properly interacted with, perhaps something else. With access to the hub and the recorded videos I can see exactly what was going on so that I can easily fix my commands and make the test better.
Another pain point we encountered was that we had a large number of tests that were taking a good deal of time to run. We weren’t making use of our entire grid while testing, so each test would run individually even though we had X amount of browsers waiting-in-the-wings.
My first inclination was that this was a setting handled in the grid hub, but this was not the case. The Nunit framework allows for running test in parallel. After placing code in the tests to command Nunit to run X amount of tests in parallel we were able to utilize all the nodes on the grid at the same time. This cut our time for testing significantly, and that means releases go out to market even quicker. If we fail, we fail fast, which is the best way to do so.
Another way to cut down on the time that tests were running was to substitute sleep elements for wait elements. When I used wait elements the page didn’t seem to get them. A Band-Aid in the interim was to use sleep to make sure that browser elements had time to show up before moving on to the next task. As I moved through more and more tests and became further engrossed in Selenese, I was able to find that there were many different ways to wait for elements, and waits could be done for an element to be visible, an element to be clicked, and element to exist, etcetera, etcetera. Instead of waiting a predefined amount of time for an element to get in the state that I needed it, I was waiting only for the instant that that element was available, then moving on.
A Solution of My Own
Eager to employ my newly found skill set to other areas of my work I searched for additional opportunities. As part of Online Care Plans offered to subscribing clients we create a monthly report that details website health. One of the plan features includes videos and screenshots of key functional areas of a client site to show the site across multiple browsers and devices over time.
To do this, we have been utilizing the Selenium Builder plugin for Firefox and running JSON scripts using Sauce on Demand, the videos and screenshots are then stored in a Sauce Labs account. The Selenium Builder plugin is great because you can easily record your real world steps and translate those into Selenese commands, then run those tests and send the results directly to Sauce Labs and utilize their vast variety of different browser and device emulators.
Unfortunately, many of our scripted tests were failing so I worked some Selenese magic on the JSON files to update the scripts and get them passing. Since I was becoming a Selenese wizard I couldn’t bear the idea of manually recording my steps.
One major problem we were experiencing was our automated device tests were timing out. While this tool was working great for desktop tests, we still had many videos to run manually for mobile devices. I was personally feeling the pain of running these mobile tests manually, as it was part of my responsibilities, and it certainly wasn’t living the DevOps lifestyle. Considering my exploration into functional tests, I now intuitively knew what we could do to overcome this.
I decided to create my very own solution in Visual Studio to house these tests. I created a new project and downloaded the appropriate references for Selenium and Nunit. In addition, I downloaded the Appium framework, for good measure. The next thing I needed to do was setup the capabilities for my browser and tie them into Sauce Labs. Sauce Labs provided a Platform Configurator which made the process almost like cheating. Now I was armed with all the tools to go about creating tests in Selenese and having the results transferred to our Sauce Labs account. Once the test ran I could log in and grab the video and screenshots to include in our Online Care reports.
In Summation
As someone who fancies himself a hobbyist creative writer, I feel like there needs to be a moral-of-the-story ending to this piece. Maybe something that isn’t so Stuart Smalley, “I’m good enough, I’m smart enough, and doggone, people like me”. More to the effect that an automated testing framework can provide an immense amount of value to your team and the clients that you serve.
There is no better way to systematically ensure that your applications functionality is kept intact as it makes its way to market. Automation saves time and frees up resources to allow for further wicked problem solving. The cream-of-the-crop web companies are employing the DevOps lifestyle and this practice is a great way to be a part of that. As for me, the reward is being given this exciting and new opportunity to further grow and expand my skill set. I would like to thank you for the read, as I return to writing in Selenese.