Thu 26 Jan 2012
Back in 2008, David Henderson and I wanted to try automate collecting client side performance data for the web application that we were working on at the time. We were getting a large number of complaints from users about load time and we need to try solve this. The way that we decided on doing this was to use Selenium, which was already running our tests, and hack the Selenium server to give us the information we needed. What we came up with was showcased at GTAC 2009. It was really good for the time and we were happy.
About a year later just as the new movement for HTTP Archive or HAR as its commonly known was taking off I found that Jan "Honza" Odvarko had created a Firebug add on to export the network tab to HAR. This, when used with Selenium, could mean that we could get the same data that the browser wanted with little to no effort. Run your tests as you were with the new WebDriver API and get it to programmatically install the necessary addons as well as set all the preferences needed. I wrote this for Python and for .NET. This works really well but now limits this type of data to only being collected within Firefox. Some modern web applications can send back totally different javascript and sprites dependant on which browser hits.
What if you could run your Selenium tests and collect the same info as the if you were using the Firebug Net tab but using any browser. Browsermob Proxy is a good way to collect this information and it has a programmatic interface that allows us to set it before our tests start. It can also return a HTTP Archive of the traffic that it is routing. I have released the Python Library for BrowserMob Proxy that can be injected into tests quite easily. I have put an example below.
from browsermobproxy import Server
server = Server("path/to/browsermob-proxy")
server.start()
proxy = server.create_proxy
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.set_proxy(proxy.selenium_proxy())
driver = webdriver.Firefox(firefox_profile=profile)
proxy.new_har("google")
driver.get("http://www.google.co.uk")
proxy.har # returns a HAR JSON blob
proxy.stop()
driver.quit()
If we now feed the data from proxy.har into HAR Viewer we can get a nice waterfall of the traffic as below! Jari Bakken has a useful commandline version called Har.
I recently showed this off at London Web Performance. Unfortunately it wasn't recorded but you can view the slides here
Area: blog
Thu 12 Jan 2012
WebDriver, the browser automation framework we all love, is on its way to becoming a browser standard. WebDriver as an OSS project is 5 years old and started being merged into the Selenium project about 3 years ago. From there we have seen WebDriver grow. It has had a lot updates in that time. The team is currently averaging ~100 commits a week.
In the short time, from GTAC 2009 when they were created, the Ruby bindings created by Jari Bakken have had over a million downloads! The Python bindings have around 3000 downloads a week and growing. Jim Evans work on the .NET bindings is great, I don't have figures because of Nuget. Same with the Java bindings. Maven makes it difficult to know how many downloads.
We have also seen that Selenium Jobs have overtook the commercial counterparts. All of these things point to us needing a standard to make sure if we can do the same thing on multiple devices and multiple OSes.
Yesterday we held a W3 meeting in the Google UK office. It was great to be able to get in a room and discuss what needed. A plan was hatched on what needs to be done next, and it is going to be a lot of work.You can see the minutes of the meeting here to see what we discussed.
Mozilla, Opera and Google are happy with the direction we are going and long may it last! It will be great to get Microsoft and Apple involved soon.
Remember that all of this work is happening out in the open and you can watch the Hg repository for the editors draft updates.
Area: blog
Thu 08 Dec 2011
How did you get involved with your favourite Open Source project? I started helping on the Selenium project in 2006 by helping out in the forums and then over time submitted patches and eventually got my commit bit by helping Jim Evans . I have always been a huge proponent Selenium even when it made up a small amount of my job. However I never looked to help Mozilla for some strange reason even though my personal views and the company mission were ideally matched.
Lately there have been a number of different blog posts from Mozillians about how they got involved with Mozilla. The posts describe how these people went from being volunteers, or using forks of Mozilla code, to being employees. A good example is Clint Talbert's post. Clint wrote his post in response to David Boswell's post. These stories make me feel like I have missed a step in getting into Mozilla. And to be honest it does make me a little sad.
One of the hardest things to do, and most OSS committers on any project will agree, is creating a community around your project. Mozilla has over the years done an amazing job at getting contributors. One of the main pulls is the Mozilla Misson! The simple message is "Mozilla's mission is to promote openness, innovation and opportunity on the web.". This is something that I didn't really know before joining Mozilla. I started using Mozilla Suite when it came out many many years ago. I was in University at the time and it made my life easier. I, like most students, had an amazing amount of free time and spent it playing with new technology and learning to program better but not contribute back to an OSS project.. In other words I could have given back.
I have been working on other OSS projects as and when I came across them but didn't see the OSS project that was taking me to the ones I was working on. Next time that you feel like that you want to help out on an OSS project but don't know which one have a look at have a look at Mozilla. If you want to see about helping my team, have a look at our team site and come have a chat on irc.mozilla.org#automation.
Area: blog
Wed 30 Nov 2011
I am one of the few privileged people in the world to only work on open source software for a living. All the work I get paid for as well as all my private projects is open source. It was one of the main things that drew me to work at Mozilla. Creating software that I can give away and when there are bugs I can fix, or even better receive a patch, and I can learn from my mistakes all in the open. I can show other people my approach to problems and perhaps they can learn from it too! I find this really exciting for me.
OSS is what makes the software world tick. Let's face it, a large number of libraries, frameworks or even our key datastores that we use in our day to day jobs are open source! There is also a chance that you use Linux as your main OS. A lot of these projects are run entirely by volunteers who freely give up there time to make something that they think is useful to the world!
Unfortunately there does appear to be a increasing trend for users of the OSS projects to feel they are allowed to treat developers as if they were paying for the code. Perhaps I am more aware of this trend since I am part of OSS landscape.
I am constantly seeing messages that say "how can I use your product that is full of bugs", "I am embarrassed to see this happening", "Urgently fix my problem...".
To be honest, that is the extent of the bug or query to the mailing list and looking past the trolling comments I couldn't help if I wanted to.
We have seen examples of this quite publicly with posts about Node.Js being cancerous and the article about Mozilla Places database crashing one users Firefox instance. But if you were to go through bug reports of your favourite OSS tool you will see these comments. They are not useful, helpful and can be down right hurtful.
Speaking to Simon Stewart he said sign of a OSS product making it big is the amount of comments that come in that are not useful and/or trolling increases. While I can understand that I don't understand the need to make comments like that.
We can see this a similar idea from Mikeal Rogers in one of his tweets.
Both my Selenium colleagues and my Mozilla team are made up of some of the cleverest and friendliest people who bend over backwards for the community at the drop of a hat.
So next time you make a comment on OSS think about how much you would feel if you gave up your free time to be treated this way by users. Be constructive in your criticism and make sure to make sure to give steps to replicate bugs even down to the smallest detail!
Area: blog
Thu 03 Nov 2011
GTAC is one of my favourite conferences that I like to go to. It has the right mix of people who attend. The one thing they have in common is that they want to learn and don't want any sales pitches.
This year did not disappoint. I have put my thoughts on each of the talks below that really stuck out.
Testing is Dead - Alberto Savio
This was an interesting talk about how we need to stop thinking being gatekeepers of products. We need to rethink our approach to creating products as a whole and then go from there. He started talking about prototyping and getting feedback. We should constantly dogfooding our application. Dogfooding, a term coined by Microsoft, is where you get people from with in the organisation using the application as soon as humanly possible.
Those of you who follow Eric Ries and the whole Startup movement will be well aware this idea. Eric pushes the concept of a minimum viable product so that you, as an entrepreneur or intrapreneur, can get people using it and feeding back ideas and bugs. This approach won't work with all applications but it solves the 80% problem.
Lightning Talks
The talk that stood out the most was a project called scriptcoverage. It is a Chrome extension that when coupled with testing tools could tell how much of your JavaScript is being exercised. This is a project that hopefully I will have time to port to a Firefox extension.
WebConsistency - Kevin Menard
Most people will know Kevin for being one of the key people on Selenium Grid. Kevin talked about how we need to stop worrying about pixel perfection between multiple browsers and that we can automate some checks. Since we access the DOM we can work out what CSS is being applied where and how this influences everything.
This is the basis for Kevin's start up, and MogoTest, so suggest signing up and giving it a try. In Testing Tools at Google talk the Googlers mentioned it was a much better product than they had created internally. Great comment from a company that suffers from "Not invented here"
How hackers see bugs - Hugh Thompson
Hugh did a really great talk on Security. Hugh talked about how numbers can really skew what is really happening in the real world. He was also talking about how social engineering is becoming the tool of choice. Years ago we had emails from Nigeria were sending fairly standard emails with no personalised parts and a number spelling mistakes. That has changed to perfect English with personalisation.
This was the most entertaining talk of the whole conference. When the videos are out I will be watching this again.
WebDriver - Simon Stewart
Simon did good talk on how to scale out your tests. Simon and a number of other Googlers have built out a Selenium farm so that they can run tests on many different browsers and OSes. While Simon was saying that a number of tools that Google use were built in house there are many different OSS alternatives. Selenium Grid and the plugins that can pull VMs easily. Simon did mention that people should be wary of running tests in the cloud if you don't have there own. Latency can make tests run slowly. Simon phrased it as "You are putting 4x the Internet in the way". Something we should be aware of when using cloud providers.
Lightning Talks Day Two
The lighting talks on day two where quite good, especially the ones for mobile by the Googlers. It was good to see the hard work that they had done to make WebDriver API work on Mobile is great. I did a talk on how we can create HeatMaps from WebDriver tests and David Clarke from my team did a talk on JsTestNet. It's a way to drive JsTestDriver in a more distributed fashion.
Jellyfish and other JavaScript testing tools - Adam Christian
Adam did a great job showing the power of JavaScript. Adam is the creator of numerous testing frameworks, and tells people not to do it, shows off his latest tool. Jellyfish can run tests on any device. Adam did a very similar talk at Selenium Conference but has updated it quite a bit so won't go on too much. Adam was extolling the virtues of doing very good JavaScript testing.
Conclusion
The other talks weren't really my thing but was surprised by how Google heavy this conference was but happy there wasn't the obvious recruiters. I would say for a feature talk make sure you practice and don't do fake banter.
Area: blog
