This is the general CI flow that you might have:
1. You commit your changes to Git
2. Your CI server picks up the changes and builds the app
3. It's time to run some tests. Now is the time to trigger the call to the Selenium grid to perform the UI/browser testing.
4. If all is well, you deploy your app to the next tier (say production or staging).
In order for the Selenium grid to test your app, your app must be reachable somehow. This usually means that you would have to deploy it to a publicly accessible URL (for example: qa.awesomesite.com, because otherwise the Selenium grid, that sits on a cloud somewhere, won't be able to reach your app. Deploying my code to a test environment just to test it when I can run the webpage containing it here and now? ugghhhh.
Enter SSH tunnel. This black magic of a protocol allows the cloud provider to connect to your server (this can be your machine or CI server) and access local resources (for example, it could load http://localhost:8080/).
This feature extremely simplifies the testing procedure. Without it, we would have to deploy our code before it was tested to a testing environment. With this, we can run the tests directly from our CI server.
The Problem
But there are some complications. Some of the SSH tunnelling servers are implemented using proxies that would not allow this trick to work when using certain features (for example, SauceLabs is using Squid). This means that in order to test things like WebSockets for example, you would still have to deploy your bit of JavaScript code to a publicly available domain.
To workaround this problem, we devised a strategy that allows us to keep on using SSH tunnelling for testing our app on the CI server by dynamically injecting locally hosted scripts contained in a designated publicly hosted web page.
1. There is a single, empty web page that is deployed to a publicly accessible address (e.g. qa.awesomesite.com). This web page contains an empty head and body, with the exception of a script that is referencing a fixed address: http://localhost:1234/qa/injected.js.
2. When the testing phase begins, the SSH tunnel is opened. If you are using SauceLabs, you would want to make sure you start the Sauce Connect plugin with the -D option to exclude the publicly accessible page from being transferred through the tunnel. For example, you would start connect by doing this:
/bin/sc -D *.awesomesite.com
You would setup your selenium listener to your localhost (on SauceLabs Connect it's on port 4445):
browser = wd.promiseChainRemote('localhost', 4445, username, accessKey);
This is where we want to make sure our injected.js script is indeed hosted for the period of the test. If you use grunt, you can achieve this quite easily by using the grunt-connect plugin, starting the static server before the test and closing it after the test.
Your grunt task can basically look like this:
- JsHint your code to make sure you did not to anything stupid
- Browserify your CommonJS modules into browser-compatible JS
- Raise the static server with grunt-contrib-connect
- Call the SauceLabs Selenium grid to action
grunt.registerTask('test', ['jshint', 'browserify', 'connect', 'test:sauce:chrome']);
3. At this stage, the remote selenium cloud loads up qa.awesomesite.com from the remote location, but the page on that site includes a reference to http://localhost:1234/qa/injected.js, which is loaded from the local server (or your machine for that matter) via the SSH tunnel
4. The injected.js script is a generic script whose entire purpose is to fire up a test. When building the app, before the tests begin, we use grunt to browserify two CommonJS scripts:
- loader.js, which only loads the script for the test, and can look roughly like this:
var $ = require('jquery');
var Tester = require('./tester.js');
$(document).ready(function() {
$('<ul class="test"></ul>').appendTo('body');
$('<li id="scriptLoaded">script loaded</li>').appendTo('ul.test');
var tester = new Tester();
tester.test();
});
- tester.js, which is responsible for generating the conditions that the test-spec is set to validate:
var moduleToTest = require('../../../index.js'); module.exports = Tester; function Tester() { this.test = function(argument) { var testedModule = new TestedModule(); testedModule.doStuffThatSshTunnelCannotHandle(); } }
The highlighted row is where you would listen for WebSocket messages emitted by the server.
5. Now, in our test-spec, where we have our web driver code, we can test out that the remote script was injected and that the SSH-tunnel incompatible operations is in fact executed successfully (testPageUrl holds the URL to the publicly accessible HTML that references injected.js):
it('should load the injected script via the ssh tunnel', function(done) {
browser
.get(testPageUrl)
.waitForElementById('scriptLoaded', 10000)
.text()
.should.become('script loaded')
.nodeify(done);
});
it('should should display peerID within 10 seconds', function(done) {
browser
.get(testPageUrl)
.waitForElementById('peerid', 10000)
.text()
.should.eventually.have.length(24)
.nodeify(done);
});
Gotcha's:
- If using SauceLabs, make sure you do not neglect the -D flag! without it, the publicly hosted URL will be loaded through the SSH tunnel and you would not be able achieve your goal of testing local resources.
- If you are using Travis CI, do not use the Travis Sauce Connect plugin (you will not be able to apply the -D command). Instead, fork this Gist used by Travis and add your -D flag. Then run this script on the "before_script" section of the travis.yml file. Travis is merely running this to start and stop the Sauce Connect plugin:
- If you are using Travis CI, do not use the Travis Sauce Connect plugin (you will not be able to apply the -D command). Instead, fork this Gist used by Travis and add your -D flag. Then run this script on the "before_script" section of the travis.yml file. Travis is merely running this to start and stop the Sauce Connect plugin:
No comments:
Post a Comment