Today we’ve added a form on the homepage for visitors to try out our service.
This simple demo will open the website you’ve entered in Internet Explorer 8 and perform a basic test.
It should give you a sense of how easy it is to use our service and test your websites.
Once you entered a website address, you will see this page. It will take up to one minute for the test to finish.
As soon as the test is finished, you will see the results together with screenshots and a video.
In this blogpost we’ll talk about creating your first Selenium test, running the test on your own computer and then later run it on several browsers at once.
It shouldn’t be too hard to follow along, but if you have any questions or remarks, please leave a comment.
Before we can create a test, we first need a website and an idea of what to test on the website.
Let’s say we have an e-commerce website selling products.
An important part of your e-commerce website is the shopping cart, this virtual cart keeps track of the products the visitor indicated he/she wants to purchase.
If this shopping cart functionality should fail, for whatever reason (bug, code changes, human error, …) the owner of the website will immediately start losing money and customers.
In an effort to prevent this, we decide to create a functional test with Selenium to verify if the shopping cart is still functioning like it should.
The first thing we need is to download the Selenium IDE addon for Firefox. It will help us in creating our first Selenium test.
The IDE has a function to record your browsing activity. All you need to do is launch the addon, click the record button and start performing actions.
Our first test
In our case, we would click the record button, go to our e-commerce website and click “Add to cart” on a specific product.
We’ll then go to the shopping cart page and verify if our product is in the cart. Right click the title of the product in the cart and select “verifyTextPresent”.
This will verify if the product is in fact in our cart.
Stop the recording of the test. Your first Selenium test is ready!
You can now choose to run this test in your browser. It will mimic the actions you just performed and verify that the product is in fact in your shopping cart.
We could stop here and run this test manually every time we want to test our shopping cart. But what about other browsers, maybe the cart is broken in Internet Explorer, a browser used by a lot of people.
Run your test on multiple browsers
That’s where TestingBot.com shines. You simply add your test to our website, choose which browsers you want to run it on and how many times per day/week/month.
We run the tests for you on the browsers you want. If a test should fail, we’ll alert you so you know that something is wrong with your shopping cart.
You can then fix the problem without losing visitors and revenue.
We’ve made a screencast of the above example so you can follow along easily:
The way it works is: we changed some Selenium code to capture the PID (Process ID) of the browser that is being used by using Java’s JNA library.
We pass this PID to our tool, which uses the PID to correctly identify the unique browser window. We then get its Hwnd and use PrintWindow to capture the contents of the window.
By releasing this code as open source, we thought it would possibly help others. Let us know what you think!
Today we’re examining the captureNetworkTraffic selenium command, a little known gem in Selenium’s command list.
Selenium offers a feature, captureNetworkTraffic, which allows you to intercept the network traffic as seen by the browser running your test.
The response includes headers, status codes, timings and ajax requests. It’s especially useful if you’re debugging requests in your browser.
We have just finished optimizing the video recording in our Selenium grid.
Implementing this was harder than we thought, so here are some tips on how we implemented this.
To create a video, we used AForge’s AviWriter, which makes creating avi videos very easy in C#.
We first started off with using C# and Windows’ PrintWindow function. This worked in some cases but would sometimes show a dark square in the content area.
PrintWindow was also very slow when appending it to a video.
Next, we tried to use Bitblt and GDI with C#, together with AviWriter. This was still too slow for us as it caused tests to run twice as long. We want your tests to run as fast as possible and did not want to hog the CPU because of video recording.
After searching on Google, we found several costly screen capture SDKs which promised fast video recording by using special drivers. This turned out to be working in some cases, but was still too slow.
Finally, we investigated Microsoft Expression Encoder which offers a screen-capture SDK. This turned out to be a winner since it recorded video without hogging the CPU.
The only pitfall is that the video encoding used by this SDK is a proprietary encoding by Windows. To convert the video to FLV, we needed to encode it after recording, something that requires quite a lot of CPU.
We choose to do the video recording on a different high CPU server, so that your tests will run as fast as possible without a screen recording slowing down the CPU.
The program we’re using to take screenshots is MiniCap, which is a lightweight command line program to easily take screenshots. Highly recommended.
It was quite a journey, but we think we now have a good system to record tests without slowing down your tests.
After some more optimizations in our code and infrastructure we’ve now reached the point where the average simple test takes 9 seconds.
Compared with competitors, we’re 2 to 3 times faster in running your test.
This means that you can run more tests in less time at testingbot.com
Here are some benchmarks with a simple test that starts an Internet Explorer browser (7) and verifies the title of google.
0.0496 tests/s, 0.0496 assertions/s.
0.12 tests/s, 0.12 assertions/s
We’ll be experimenting with a quick-launch feature soon where we’ll speed up the startup of a browser (the most time-consuming event during testing).
We’ve just finished upgrading our Selenium grid (the hub and all its RC nodes) to Selenium v2.15.0
This upgrade should provide improved stability.
According to the Selenium changelog these are the changes in the v.2.15 version:
Now supports up to and including Firefox 11.
Changed order of FirefoxProfile initialization attempts on a
remote host: 1) try to load from capabilities, 2) try to load
locally if webdriver.firefox.profile is specified, 3) create an
Added the beginning of infrastructure for gathering logs from
Added an UnreachableBrowserException.
Additional work on the WebDriver JS console.
Implicit waits now change how long we wait for alerts. This
functionality will change in 2.16
FIXED: 2700: The infamous scrolling issue.
FIXED: 2816: ClassCastException when calling switchTo().alert().
Grid now distinguishes between CLIENT_GONE and
FIXED: issue where older RCs could not connect to a newer hub.
Selenium-backed WebDriver now uses atoms for typing.
Significantly reworked input device abstractions.
HTA mode for Selenium RC (*iexplore) leaves a browser window open.
Selenium is the industry standard for automatically testing websites in browsers.
A selenium test is a set of instructions that will be sent to a browser of your choice.
Most major programming language offer support for Selenium. Either by plugins or by default.
These plugins will translate language specific syntax to Selenium commands which can be understood by a Selenium Grid.
For example, let’s say we want a test where we will test Google on Firefox. We want to check if the title of the homepage on Google is “Google”.
We start of by specifying which browser we want to use. In Selenese (the syntax used by Selenium), we would use the command getNewBrowserSession together with the options: Firefox and browser version.
This command is sent to the Selenium grid, which is managed by a hub.
The hub will receive the command, check if its grid supports the browser and assign the requester (client) a session.
The client now needs to use this session to further instruct the grid. An example of a session-id would be sofpa92f3fj023jfodsf9.
When we’ve reached this point, the hub has instructed one of its node on the grid (which supports the browser we requested) to start the browser.
Since we want to test Google, we need to instruct the browser on the node to go to google.com.
We do this by sending the “open” command to the grid, together with the URL and our sessionid.
The hub will receive the request, check the sessionid and knows that we previously requested a Firefox browser on a specific node.
Our hub will forward the request to the node, which in its turn will instruct Firefox to open the page and go to the requested URL.
To end our test, we need to verify if the title of the page which is now opened on the node does in fact equal “Google”.
We send a request to the grid, asking our hub to contact the correct node and ask for the title of the current page.
We receive the title back from the grid and verify if the title does in fact equal the title we expect.