Explaining how the real world works!     RSS Feed The Automated Tester on Twitter The Automated Tester on LinkedIn The AutomatedTester on github

How I got involved with Mozilla

Thu 08 Dec 2011

How did you get involved with your favourite Open Source project? I started helping on the Selenium project in 2006 by helping out in the forums and then over time submitted patches and eventually got my commit bit by helping Jim Evans . I have always been a huge proponent Selenium even when it made up a small amount of my job. However I never looked to help Mozilla for some strange reason even though my personal views and the company mission were ideally matched.

Lately there have been a number of different blog posts from Mozillians about how they got involved with Mozilla. The posts describe how these people went from being volunteers, or using forks of Mozilla code, to being employees. A good example is Clint Talbert's post. Clint wrote his post in response to David Boswell's post. These stories make me feel like I have missed a step in getting into Mozilla. And to be honest it does make me a little sad.

One of the hardest things to do, and most OSS committers on any project will agree, is creating a community around your project. Mozilla has over the years done an amazing job at getting contributors. One of the main pulls is the Mozilla Misson! The simple message is "Mozilla's mission is to promote openness, innovation and opportunity on the web.". This is something that I didn't really know before joining Mozilla. I started using Mozilla Suite when it came out many many years ago. I was in University at the time and it made my life easier. I, like most students, had an amazing amount of free time and spent it playing with new technology and learning to program better but not contribute back to an OSS project.. In other words I could have given back.

I have been working on other OSS projects as and when I came across them but didn't see the OSS project that was taking me to the ones I was working on. Next time that you feel like that you want to help out on an OSS project but don't know which one have a look at have a look at Mozilla. If you want to see about helping my team, have a look at our team site and come have a chat on irc.mozilla.org#automation.

    Area: blog

Hug an OSS Developer or just don't be a dick

Wed 30 Nov 2011

I am one of the few privileged people in the world to only work on open source software for a living. All the work I get paid for as well as all my private projects is open source. It was one of the main things that drew me to work at Mozilla. Creating software that I can give away and when there are bugs I can fix, or even better receive a patch, and I can learn from my mistakes all in the open. I can show other people my approach to problems and perhaps they can learn from it too! I find this really exciting for me.

OSS is what makes the software world tick. Let's face it, a large number of libraries, frameworks or even our key datastores that we use in our day to day jobs are open source! There is also a chance that you use Linux as your main OS. A lot of these projects are run entirely by volunteers who freely give up there time to make something that they think is useful to the world!

Unfortunately there does appear to be a increasing trend for users of the OSS projects to feel they are allowed to treat developers as if they were paying for the code. Perhaps I am more aware of this trend since I am part of OSS landscape.

I am constantly seeing messages that say "how can I use your product that is full of bugs", "I am embarrassed to see this happening", "Urgently fix my problem...".

To be honest, that is the extent of the bug or query to the mailing list and looking past the trolling comments I couldn't help if I wanted to.

We have seen examples of this quite publicly with posts about Node.Js being cancerous and the article about Mozilla Places database crashing one users Firefox instance. But if you were to go through bug reports of your favourite OSS tool you will see these comments. They are not useful, helpful and can be down right hurtful.

Speaking to Simon Stewart he said sign of a OSS product making it big is the amount of comments that come in that are not useful and/or trolling increases. While I can understand that I don't understand the need to make comments like that.

We can see this a similar idea from Mikeal Rogers in one of his tweets.

Both my Selenium colleagues and my Mozilla team are made up of some of the cleverest and friendliest people who bend over backwards for the community at the drop of a hat.

So next time you make a comment on OSS think about how much you would feel if you gave up your free time to be treated this way by users. Be constructive in your criticism and make sure to make sure to give steps to replicate bugs even down to the smallest detail!

    Area: blog

Thoughts on GTAC 2011

Thu 03 Nov 2011

GTAC is one of my favourite conferences that I like to go to. It has the right mix of people who attend. The one thing they have in common is that they want to learn and don't want any sales pitches.

This year did not disappoint. I have put my thoughts on each of the talks below that really stuck out.

Testing is Dead - Alberto Savio

This was an interesting talk about how we need to stop thinking being gatekeepers of products. We need to rethink our approach to creating products as a whole and then go from there. He started talking about prototyping and getting feedback. We should constantly dogfooding our application. Dogfooding, a term coined by Microsoft, is where you get people from with in the organisation using the application as soon as humanly possible.

Those of you who follow Eric Ries and the whole Startup movement will be well aware this idea. Eric pushes the concept of a minimum viable product so that you, as an entrepreneur or intrapreneur, can get people using it and feeding back ideas and bugs. This approach won't work with all applications but it solves the 80% problem.

Lightning Talks

The talk that stood out the most was a project called scriptcoverage. It is a Chrome extension that when coupled with testing tools could tell how much of your JavaScript is being exercised. This is a project that hopefully I will have time to port to a Firefox extension.

WebConsistency - Kevin Menard

Most people will know Kevin for being one of the key people on Selenium Grid. Kevin talked about how we need to stop worrying about pixel perfection between multiple browsers and that we can automate some checks. Since we access the DOM we can work out what CSS is being applied where and how this influences everything.

This is the basis for Kevin's start up, and MogoTest, so suggest signing up and giving it a try. In Testing Tools at Google talk the Googlers mentioned it was a much better product than they had created internally. Great comment from a company that suffers from "Not invented here"

How hackers see bugs - Hugh Thompson

Hugh did a really great talk on Security. Hugh talked about how numbers can really skew what is really happening in the real world. He was also talking about how social engineering is becoming the tool of choice. Years ago we had emails from Nigeria were sending fairly standard emails with no personalised parts and a number spelling mistakes. That has changed to perfect English with personalisation.

This was the most entertaining talk of the whole conference. When the videos are out I will be watching this again.

WebDriver - Simon Stewart

Simon did good talk on how to scale out your tests. Simon and a number of other Googlers have built out a Selenium farm so that they can run tests on many different browsers and OSes. While Simon was saying that a number of tools that Google use were built in house there are many different OSS alternatives. Selenium Grid and the plugins that can pull VMs easily. Simon did mention that people should be wary of running tests in the cloud if you don't have there own. Latency can make tests run slowly. Simon phrased it as "You are putting 4x the Internet in the way". Something we should be aware of when using cloud providers.

Lightning Talks Day Two

The lighting talks on day two where quite good, especially the ones for mobile by the Googlers. It was good to see the hard work that they had done to make WebDriver API work on Mobile is great. I did a talk on how we can create HeatMaps from WebDriver tests and David Clarke from my team did a talk on JsTestNet. It's a way to drive JsTestDriver in a more distributed fashion.

Jellyfish and other JavaScript testing tools - Adam Christian

Adam did a great job showing the power of JavaScript. Adam is the creator of numerous testing frameworks, and tells people not to do it, shows off his latest tool. Jellyfish can run tests on any device. Adam did a very similar talk at Selenium Conference but has updated it quite a bit so won't go on too much. Adam was extolling the virtues of doing very good JavaScript testing.

Conclusion

The other talks weren't really my thing but was surprised by how Google heavy this conference was but happy there wasn't the obvious recruiters. I would say for a feature talk make sure you practice and don't do fake banter.

    Area: blog

A basic continuous deployment process

Mon 12 Sep 2011

One of the major buzzwords floating around the Internet these days is continuous deployment. Continuous Deployment is the process of pushing code to servers straight from trunk at any moment. A number of companies do this on a commit/push and most of those companies have chosen to push the code straight to their production servers. This means they could be releasing up to once every 11 seconds. For those who want to learn more about it I suggest that you look up continuous deployment at Imvu, easy, flickr, Amazon and Mozilla.

The approach that I took for my setup was to have a deployment process similar to if you were to use Heroku or Joyent's no.de service. When I am ready to deploy I just want to do git push myserver master. I know this process won't scale well but for most sites this is all you need.

I have been developing an IRC bot for my team to use. It does lookups in Bugzilla, it can give people links to our Pivotal Tracker projects and does logging of what is happening in the channel and a few other things. So far my team has been making really good use of it, and being people who work in QA, they tend to find bugs or think of new features.

I wanted a way that I could fix bugs, or create features required, and release with out too much effort.

The following is what I do on my free AWS EC2 Micro instance. Please note that my IRC bot is written to run with Node.Js so some things will have a Node.Js slant but the my overall process will work for every one.

On the server you want to deploy do the following

 
mkdir project-folder.git # note the .git prefix 
mkdir project-folder 
cd project-folder.git 
git init --bare 
git config core.worktree project-folder
git config core.bare false
git config receive.denycurrentbranch ignore
cd hooks 
mv post-receive.sample post-receive 
Edit the post-receive file and add git checkout -f to the file

Now on our development machine a simple

 
git remote add micro ssh://user@aws-instance/~/project-folder.git 
git push micro master 

If you go into project-folder on your server you should see your code now. If you received a security exception make sure that you have copied your ssh public key, like the one you may have put on Github, in to ~/.ssh/authorized_keys on your server. Add a new line and then the ssh key. Try push again once that is correct.

Now we need a way to restart Node.Js when there is new code is deployed. I used Nodemon by Remy Sharp. Instead of doing node myapp.js just do nodemon myapp.js. Nodemon will restart the node process running you application when it sees changes to the JavaScript files.

If you do all of the above things you should have a basic Continuous Deployment process. I haven't created any rollback scripts for if something goes wrong but since you can push fixes quickly that shouldn't matter too much.

    Area: blog

Selenium Advanced User Interactions

Mon 27 Jun 2011

HTML5 is becoming on the of the biggest things to hit the internet. Unfortunately it has also meant that Selenium 1 has hit a wall in with which technologies it can interact with. It can't play with these new technologies like canvas.

The new Selenium 2 API is trying to solve a number of these problems with the way that it drives the browser. One of the good things that has come out of the work on Selenium 2 is the ability to control elements on the page like a user would. This could be dragging items around a page or even working with canvas applications. Using the mouse, or keyboard, at a low-level to do some really good things.

The Advanced User Interactions API allows us to do things like dragging and dropping as well as gives us the ability to do something like hold down a key and move the mouse around or carry on typing. Below I have put together a couple examples below that you can how we can use the new Advanced User Interaction API that is part of the Selenium 2 API. A quick note, this API only works on Linux and Windows.

The first example uses the first demo page. The page allows you to draw what you want on a canvas element on the page. The script below will draw a triangle on the canvas.

  
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains

class TestCanvas:
     def setup_method(self, method):
          self.firefox = webdriver.Firefox()

     def teardown_method(self, method):
          self.firefox.quit()

     def test_that_we_can_draw_on_canvas(self):
          self.firefox.get('http://www.theautomatedtester.co.uk/demo1.html')
          canvas = self.firefox.find_element_by_id("tutorial")
          drawing = ActionChains(self.firefox)\
                    .click_and_hold(canvas)\
                    .move_by_offset(-40, -60)\
                    .move_by_offset(30, 20)\
                    .move_by_offset(100, 200)\
                    .release(canvas)
          #Now we know what we want to happen, let's perform the actions
          drawing.perform()

Now that we have done something cool, let's have a look at using the API to do every day things with our sites. The next example will drag an element on the page. The following demos will use the following demo page

  
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains

class TestDraggingAround:

     def setup_method(self, method):
          self.firefox = webdriver.Firefox()

     def teardown_method(self, method):
          self.firefox.quit()

     def test_that_we_can_drag_a_div_around_the_page(self):
          self.firefox.get('http://www.theautomatedtester.co.uk/demo2.html')
          draggable = self.firefox.find_element_by_class_name("draggable")
          dragging = ActionChains(self.firefox)\
                    .click_and_hold(draggable)\
                    .move_by_offset(30, 20)\
                    .move_by_offset(100, 200)\
                    .release(draggable)

          #Now we know what we want to happen, let's perform the actions
          dragging.perform()

But there are times where we need to be able to drag and element and drop it on another element. To do this there is a basic method called drag_and_drop(). See the example below.

  
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains

class TestDragAndDrop:
     def setup_method(self, method):
          self.firefox = webdriver.Firefox()

     def teardown_method(self, method):
          self.firefox.quit()

     def test_that_we_can_drag and_drop(self):
          self.firefox.get('http://www.theautomatedtester.co.uk/demo2.html')
          draggable = self.firefox.find_element_by_class_name("draggable")
          droppable = self.firefox.find_element_by_id("droppable")
          dragdrop = ActionChains(self.firefox)\
                         .drag_and_drop(draggable, droppable)
                         
          #Now we know what we want to happen, let's perform the actions
          dragdrop.perform()

I suggest having a play with this.

    Area: blog

How to organise a Test day

Fri 10 Jun 2011

After the success of my last test day I thought I would share some how I, and my fellow Mozillians, go about organising an event.

For those who don't know what a test day, let me explain. A test day is a day where the Mozilla asks community members to help test something. The day is spent focusing everyone's effort on this application. The application could be a desktop, mobile or web application.

So what are the benefits of a test day? The main benefits are that we will have worked through an application and raised bugs or triaged them.

As an example, the last test day for automation ended with over 14 patches from employees, contractors and contributors. That is phenomenal. Another example is the test day on Friday that had 27 bugs raised or triaged while working through the various about: pages within Firefox. It has a huge benefit in that people learn from each other. The test day is run on IRC with everyone discussing bugs, asking for thoughts on the look and feel. For those new to testing we help with areas to focus on. We guide people through creating high quality bug reports with all the bits we need. For Mozilla QA we get the much needed, and much appreciated, help from the community as well as making them feel part of making great product.

So, how do you organise a test day?

  • pick a product or feature
  • pick a day to hold it on
  • advertise it.
  • learn from it

The advertise step may seem obvious but this is where you get people really excited about a project or feature. My approach is, and I think it is a great selling point to community members, is reminding them that all the bugs, bug triaging, patches are all publicly available so can be placed on your CV. Visible proof you know how to do what you do makes a huge difference when it comes to looking for your next job.

Read up on our Wiki page for more info, our products wouldn't be as good as they are without these test days!

P.s. There is a test day today for Developer tools and would be great if you could help out!

    Area: blog

Wow! What a Test Day!

Wed 25 May 2011

Yesterday, 24 May 2011, was the second test day that I had organised for Mozilla WebQA team. This is my team that I work in mainly trying to help them with their automation.

Yesterday was the turn of Addons.mozilla.org (AMO) to get the attention. And attention it got. Over the day, we managed to get a 13 pull requests from Mozilla employees, contractors and contributors. For a 24 hour period I think this is great. A small project, dedicated to Selenium tests, can get that attention just astounds me!

We have in the region of 10 new tests, tests that we never had before. I know this doesn't sound a lot but one thing that many people don't realise is that Mozilla has a large number of websites that need to have test coverage, so having one site gain 10 tests over a day means that we can concentrate on the more mission critical parts of sites. As we all know, AMO is quite a mission critical site for Mozilla.

On the side of the test day, Marlena Compton ran a GitHub WTF course to help people get up to speed with using Git. This course was quite useful both internally at Mozilla for people wanting to learn but also to contributors on the day.

Remember that a Test day is a focused day for contributors but you don't need a test day to contribute. We are always available on IRC if you don't know where to start. I look forward to seeing more faces at the next test day!

    Area: blog

Selenium Test Day for Addons.Mozilla.org

Thu 12 May 2011

Since the last exercise at doing Selenium tests with Mozillians was a success I want to do it again!

On Tuesday 24 May 2011, I will be hosting a Test Day for Mozilla with help of some of my colleagues to help expand coverage on one of our projects and would love to have the Selenium Community help out!

So what is a test day? A test day is where Mozilla asks the community to help out on a project so we can make sure that it is properly tested and share ideas about testing the application. In this case we would like to expand our selenium coverage for http://addons.mozilla.org and you can help by going through the details from our Quality.mozilla.org (QMO) blog post. All details will be updated there if need be so register and keep an eye on it!

It starts around 10am GMT on the Tuesday through to 1am GMT on the Wednesday so hopefully giving everyone from around the world a chance to participate. Come talk Selenium and help us build some amazing tests. Your name could be added to the contributors list if we use the test!

P.s. This is a great way to practise Selenium and show what you can do to future employers!

    Area: blog

University of Sussex and Mozilla Labs Student Outreach Talk

Mon 28 Mar 2011

The other week I was fortunate to be able to give a guest lecture at the University of Sussex. Mozilla Labs Student Outreach Programme had organised for me to go talk about Test Automation. I think I shocked the students with how much needs testing by so few Mozillians.

I decided to do my talk on how we test at Mozilla WebQA. I described all of what we test and how we go about testing. I showed how we build a DSL on top of Selenium to help keep our tests DRY. I then showed the students how we can go about testing web applications via mobile devices and Selenium 2.

I have put the video below with a link to my slides. Thanks to Martin Berger for letting me talk to his students

Slides are available here.

    Area: blog

Selenium Test Day with Mozilla WebQA

Thu 24 Mar 2011

Fancy writing Selenium tests along side Mozillians?

Next week Tuesday, 29 March 2011, I will be hosting a Test Day for Mozilla with help of some of my colleagues to help expand coverage on one of our projects and would love to have the Selenium Community help out!

So what is a test day? A test day is where Mozilla asks the community to help out on a project so we can make sure that it is properly tested and share ideas about testing the application. In this case we would like to expand our selenium coverage for http://input.mozilla.com and you can help by going through the details from our Quality.mozilla.org (QMO) blog post. All details will be updated there if need be so register and keep an eye on it!

It starts around 10am GMT on the Tuesday through to 1am GMT on the Wednesday so hopefully giving everyone from around the world a chance to participate. Come talk Selenium and help us build some amazing tests. Your name could be added to the contributors list if we use the test!

    Area: blog

Selenium Camp - Slides and Thoughts

Mon 28 Feb 2011

On Saturday I was fortunate enough to open the main track of Selenium Camp. Selenium Camp was the first Selenium Conference. It was held in Kiev, Ukraine. I was invited to go do a talk there by the organisers of the event a couple months ago and accepted their gracious offer.

I did my talk on Selenium 2 and was show casing all the differences between Selenium 1 and Selenium 2. My slides are available on here for everyone to see. I did demo's on the bits that Selenium 1 fails horribly with Selenium 2 does quite easy. I also did a demo on how to create tests for mobile devices showing off the iPhone support.

The event was extremely well organised and the organisers bent over backwards for all the speakers to get things done so I have am extremely thankful to them for a really great conference. They managed to have a conference with a good mix of people and, unheard of in a lot of western tech conferences, had a near 50/50 split of male/female.

While the talks were all in Russian or Ukrainian, the ones that I watched seemed to be easily translatable by seeing what was on the slides so I could get a gist of what the speaker was saying. They were all discussing the problems that every Selenium user has and how they have overcome it. What a Selenium Conference should be!

One of the things that worried me about going out to Selenium Camp was that I can say 2 things in Ukrainian. Da and Neit. This is something that I shouldn't have been worried about, they all spoke really good English and there were a lot of really smart people out there.

Thanks again to all the organisers for fetching me from the Airport and making sure I got back there safely. Thanks for organising a really great event and I hope that I can go to the next one!

    Area: blog

Py.Test and -XDist Plugin

Mon 14 Feb 2011

The other day I was working on trying to find a decent way to start parallelising our Selenium Tests within Mozilla. One thing to know is that the team I am in, WebQA, does the QA work for all of the Mozilla Web properties.

The thing that people don't probably realise is that Mozilla has a lot sites. Lots and lots of sites so we need to make sure that we run our Selenium tests really quickly. The next thing that we need to do is to make sure that we can report on all our tests. I have a QA Director that likes to have a look at the test reports every so often so metrics are very important.

So what have we gone for to make sure we get awesome parallelism and good metrics? Well we had been using Nose as our test runner because it has good test discoverability and comes with a nice Xunit reporting plugin. Nose has an awesome ability to run tests in parallel with its Processes plugin.

Unfortunately these two plugins don't work together. The XML file is created for the results but its empty which means when the CI server picks it up it thinks something is wrong and goes red. This is not ideal because "cry wolf" situations on a CI are always bad.

On Selenium we use Py.Test to run our tests because it too has really good discoverability of tests. It has a really good way to store the results into to XUnit style report that CI can use. This is obviously a good thing! The thing I personally like about Py.Test is the verbose messaging when a test fails but thats just me!

Next I went to play with moving the tests to run in Parallel. The -XDist plugin has a nice way to run tests in parallel and handles the scheduling of tests so that they run in Parallel. It then send the details, by the looks, back to Py.Test and suddenly we have some metrics.

So if you ever want to contribute Selenium tests to Mozilla, we're open source remember!!!, you will ideally need to install those 2 packages so you know it works beautifully! Keep your eyes open for details of an upcoming Test Day!

    Area: blog

DDD9 - Slides and thoughts

Mon 31 Jan 2011

On Saturday I did a talk at DDD9, a .NET Community driven conference, about writing maintainable tests with the main focus on Page Object Pattern for tests.

My slides and Demo are available for all to see. Unfortunately I made the mistake in assuming that Microsoft would have a guest wifi connection for all people to use so people in my talk couldn't see my example. I have been spoiled by speaking at other venues and having to worry. Sorry all! P.S. If presenting with a Mac take a MiniDisplay Port to VGA converter. I only had a MiniDisplay to DVI and had to go hunting for one with one of the organisers.

The talks on the day were not bad for what they were talking about. Unfortunately I did fall into the trap of believing talk would be on what they title was. To be fair to Gary Short he did say it was the first time that he did his talk. I did learn a few things from his talk but was hoping to see more of a deep dive into .NET Collections and not micro-optimisations. All of this was made up by Gary's excitable talking style of course.

The next talk that enjoyed was by Helen Emerson. She did a talk on "CSS is Code, How do we avoid the usual problems". CSS admittedly has always been something I knew enough to get me by and always then asked others when I got stuck. Helen did show the usual mistakes and how the order of styles is important from the file. Yes I do know its Cascading Style Sheet and it does what says on the tin, but I still learnt that! I did also do my part in selling a fellow Selenium committer's startup, MogoTest from Kevin Menard, as a way to check layouts in different browsers.

Thanks to all of those who came to my talk and to the organisers of the day

    Area: blog

Selenium Conferences

Fri 21 Jan 2011

Wow, what a year this is going to be! For those that have been using Selenium for years, there has been something that a lot of people asking for. A conference just for Selenium. There have been none for a while and now, like buses, we have a couple of conferences

Firstly there is Selenium Camp that is being organised in Kiev, Ukraine. Beware the site is written in Russian but there is a browser from a search engine that will open the site and translate it to your language of choice. There are talks that match every type of user for Selenium. There are talks that show how to do low level things as well as talks that shows how to Page Objects.

Secondly we have Selenium Conference that is being organised by a number of Selenium Comitters and the companies that they work for/with. The program has not been released because they are still looking for people to send in proposals for talks. If you fancy submitting a talk I suggest filling in this form straight away!

Both of these conference are the type of conferences that I love. They are not there for companies to do Sales Pitches! They are talks about technologies or about people's experience. I know that the program for Selenium Conference is not out but I have been assured that "Experiences and Technological Talks" will be the accepted over anything else. I will be speaking at Selenium Camp and have submitted a proposal to Selenium Conference so hopefully can speak there too!

I can't wait for these conferences and if you see me at either of these conferences don't be afraid to say Hi!

    Area: blog

P2PU - My course in School of WebCraft

Wed 19 Jan 2011

Since joining Mozilla I have seen all the great things that they do and how their mission to promote openness, innovation and opportunity on the web works first hand. With this in mind I have decided to do a P2PU course in their School of WebCraft.

The School of WebCraft is a partnership between Mozilla and P2PU as part of Mozilla's Drumbeat project. The Drumbeat project is all about keeping the web open. It has a number of projects to make this happen and if you feel you agree with cause get involved!

Teaching via P2PU is a great opportunity for me to be able to share my knowledge and love of UI automation with people that want to learn. The course is a beginners course on how to use Selenium and is totally free!

Since this is my first course with P2PU I have limited the amount of people that can join to 10 but if this course works well then hopefully I will be able to advertise it and get a lot more people involved!

For those that didn't make it into my class hopefully you can sign up again in the future. There was nearly 40 people that signed up for the course that only accepted 10. This was just people that found the course by navigating the the P2PU site. A number of the courses were extremely over subscribed so there are definitely lots of people wanting to learn new things!

    Area: blog