Tue 26 Feb 2013
The other day I got asked if I knew of a tool that would notice changes to IDs of elements and update Selenium tests accordingly because there was an incurred maintenance cost in updating these all the time because the test will fail.
The tl;dr; is there isn't a tool and I don't think there should ever be one
Why?
HTML documents are not complex things, far from it, so when we change them we should think about how this is going to impact everything that hangs off a page.
If a designer or front end engineer changes the ID of elements then they go around updating the JavaScript references to that element and then update any CSS that works with those elements!
The non-obvious thing that they tend to forget is the test automation like, Selenium, that uses that element too!
Why? Well 9/10 times its because they don't have any ownership in the process! They don't submit tests nor do they run tests so they don't see the problem.
Now you are probably going to say "We use XPath/ CSS/ Names because (insert excuse here)!"
The point is still the same, your testing process and your development process are silo'ed and you're feeling the affects of this! Just remember everyone involved has a stake in the final product and QA aren't gate keepers for the product.
Area: blog
Mon 18 Feb 2013
Recently there was a bit of a rant on the Selenium users mailing list about how there were a few bugs irritating the person and, because they are only testers with not enough development experience, didnt feel that they could help with fixing these issues.
Note: this person was not rude and was not trolling so was happy to reach out.
One thing that I want people to know is that you don't have to be a brilliant developer to work on Open Source projects. You don't even have to have any experience as a software developer. You are now wondering how you can help?
By submitting a patch, even with mistakes an inexperienced would do, you are solving the problem. Once you have submitted it one thing is going to happen. Someone is going to look at your patch, eventually and reply. The reply will be we landed it or your patch needs some work. The latter is a good thing, even if you see it as a negative, since you will be learning.
Now, and is the most important thing to remember, is that the developer looking at your patch could very well learn something from you! Committers on Open Source projects tend to have come up through the ranks, unless they created the project, and have learnt what code does what where. They have learnt from other developers and want to share the knowledge back.
Going through all of these processes is a good thing. As people in software we have to be constantly learning or we fall behind. Once you are behind it can be hard to get employment in some companies you would love to work.
Next time you feel negative about contributing to Open Source, try change the thought to how you can learn something new and make a difference to your favourite projects. Selenium has getting involved bugs and Mozilla has its good first bugs that people can solve.
Don't be afraid to contribute to numerous projects because you will likely learn from a number of different people. This way you will get many years worth of meaningful experience instead of just many years of doing the same thing over and over.
Area: blog
Wed 06 Feb 2013
Could CSS 3, while is a great thing for the Internet and for web developers, be making websites that are extremely hard to automate?
As most of you know, in Selenium WebDriver we try an emulate what elements that a user can interact with. This means that we do a lot of DOM walking and gathering important little bits about the CSS on each of the elements to make sure that they are visible. You can see the details of what we do in the Determining Visibility section of the W3C Browser Automation Specification. Unfortunately every so often we get a situation where people start raising bugs about their Selenium WebDriver script with respect to allowing or allowing interaction with elements.
Recently we have started getting issues with how we handle CSS transforms. The first bug report came from the Web QA Team creating tests for Firefox OS. They were automating the Music app and noticed that tests were saying that an element was visible when it wasn't. I recreate the issue and make a minimised test case. You can see the element is miles away from the rest of the DOM below in the minimised test case below.
Marionette, the future of FirefoxDriver and what we use to automate Firefox OS, thinks that is visible. So down the slippery slope we go, add in support CSS Transforms, because that is the root cause for this bug. Now if you are wondering how complex transforms can get have a look at this MDN article.
It's just Algebraic transforms, no big deal right? But it can be when you start thinking in terms of cascading. Parent, or even great great grandparent element of the element we need to interact with could have the style applied. Now we need to walk the DOM from the element we want all the way back to body.
NOTE: Selenium has to walk the DOM a few times during various parts of the visibility checks which on large pages can be very expensive (therefore slow!). I am looking at you in particular large table element! We do the same when getting the text of elements
So while we are doing all of this work we used to assume that anything that mutates the element, like moving it across the page or changing its visibility, is going to happen in JavaScript and since the visibility code for Selenium is written in JavaScript, we will be blocking the JavaScript thread. CSS 3 has created things that have meant that we don't need JavaScript anymore to do animations
That means working with Selenium on something that is pure CSS animations might not work as you expect. So next time you decide to have a play with CSS 3 and it's awesomeness, have a play with Selenium at the same time and see how we fare. I suspect there are a lot of cases we don't cater for and we should!
Area: blog
Wed 16 Jan 2013
As many of you know I have recently released the 2nd edition of my Selenium book. When I wrote my first book I looked for a way to do version control of my book. Being a developer and test type, I used git to do all of my version control but I found that I would sometimes forget to commit and push my changes. This broke my own rule of keeping commits small.
The other problem is that if the text editor crashes, or even the OS crashes or I get a hard drive crash, there is a chance that I might only be able to go back to my last commit and push. Since I already said I am rubbish at doing the commit and push thing with documents git was definitely out of question.
I went to look for a different solution. I remembered that Dropbox keeps a history of files when syncing. So I went and tested the theory with a directory within my Dropbox folder. After a few times of playing with a file and syncing and reverting, I thought I would give it a go with my first chapter. I instantly fell in love with this approach. My publisher wanted me to use Microsoft Word to write all of chapters I instantly reverted to my old university habits of ctrl+s every few minutes. And every time I saved my chapter Dropbox would instantly synchronise the file and update the history of the file.
This meant that just by saving my file I was getting the small commit chunks I like without the worry of remembering to push my changes. I have lost the ability to do commit messages but in this scenario I honestly don't care. All I worry about is making sure I have the most up to date files I can if something were to go wrong.
I highly recommend people using Dropbox in this way.
Area: blog
Thu 03 Jan 2013
Building Software can be a very expensive thing to do! Especially if you are building a web application that needs to be deployed.
An EC2 micro instance or Linode 512 machine will cost ~USD20 a month. This then needs to have all monitoring apps installed and maintained which costs more money. This might not be a lot to some of us but growing up in South Africa, that always felt like a lot of money. When you are building a new app for Firefox OS, paying for a server to serve your app might not be in your costs for building, testing and deploying your app.
What if you could limit your development and deployment costs to £0? Note your app needs to be open source to make the cost £0.
Source Control
The current "go-to" source control at the moment is GitHub. I quite like it and for our free CI and deployment, we are going to have to have to use GitHub.
Project template
The FirefoxOS Marketplace team have created some template projects called mortar that give you all the necessary pieces to build an app. There are a number of different templates, from normal to games, for all your different needs.
Continuous Integration
Travis-CI is my favourite CI as a service at the moment. They are running the company as an open source company so I have a lot of love for these guys but I am digressing.
If you are using mortar then your .travis.yml will look like the following
language: node_js
node_js:
- 0.8
And you will also need to update your package.json file to specify how to run tests. I wrote my tests with python so it's the following
"scripts": {
"test": "py.test -n 3"
}
Now, if you want to maximise the use of your app you will need to make sure it works on as many browsers as possible. My friends at Saucelabs have recently created an Open Sauce account giving you free usage and it works well with Travis CI. To get it working you just add the following to your .travis.yml
env:
global:
- secure: "sauce user name"
- secure: "sauce labs key"
matrix:
- SAUCE_PLATFORM='Windows 2003' SAUCE_BROWSER_VERSION=18
You can see a working example here.
And the have something like the following in your test setup. It will also set the browser size to the default FirefoxOS view port
def setUp(self):
try:
os.environ['SAUCE_USERNAME']
desired_capabilities = webdriver.DesiredCapabilities.FIREFOX
desired_capabilities['version'] = os.environ['SAUCE_BROWSER_VERSION']
desired_capabilities['platform'] = os.environ['SAUCE_PLATFORM']
desired_capabilities['name'] = 'Testing Get In The Habit'
self.driver = webdriver.Remote(
desired_capabilities=desired_capabilities,
command_executor="http://%s:%s@ondemand.saucelabs.com:80/wd/hub" % \
(os.environ['SAUCE_USERNAME'],
os.environ['SAUCE_ACCESS_KEY'])
)
except Exception as e:
print e
self.driver = webdriver.Firefox()
self.driver.set_window_size(320, 480)
self.driver.get("http://localhost:8008")
Now that we have built and tested our app it's time to deploy!
Deployment
One of the things about our apps for Firefox OS is that it can be packaged on to the phone or can be served from a web server. GitHub pages make this a rather trivial exercise and also allows you to grow your user base without having to know how to maintain servers. You can also have a look at things like Launcher.io / AppFog / Heroku that have free tiers. If you need to have a backend for your app for synchronising of data then I highly recommend one of them.
Area: blog
