Thu 29 Mar 2012
This is 2nd to last week of the quarter and we have been busy closing off projects!
Henrik has implemented the dynamic job execution per node group for running daily and L10N Firefox builds. This means that we can consume Pulse messages and trigger tests for that specific platform. We are also looking at how we can find a solution for running scripts cross platform without having to duplicate certain parts of the code. This goal should be finished by the end of the quarter. He is also wanting to use a new tool called Pulse build monitor from the Automation and Tools Team to make processing of Pulse messages easier.
I have pretty much finished the validation code for Mozmill Dashboard. This goal should be finished too by the end of the quarter and then we just need a quick security review we can finally close it off!
Memchaser now has support for the new GC/CC API by listening for Observer notifications. We have also made updates for what info is being logged after a request from the JavaScript team. We have also been working on a few different features and still on track for a 3 April release!
We have also successfully published a new version of Nightly Tester Tools Addon and have passed review so make sure to get the update!
We have a number of projects that could do with some help so if you have free time have a look and see how you can help us!
Look out for future updates and if you want to join our team meetings you can get all of the details at here and you can also get notes from previous meetings.
Area: blog
Thu 22 Mar 2012
The team continues to close out projects for the quarter.
Henrik has continued working on the MozMill CI automation. He has added nodes for items to be run on windows and is now
moving on to having items setup in Linux. He will then be working on code so that we distribute all the testing across
all of the machines. This is really coming together and I look forward to the end result.
I have been working on getting things ready for working with Simon Stewart on the W3C Browser Automation Spec which will
be doing next week! Hopefully not long after you read this you will be able to see it nearly complete!!
I have also been working on the Mozmill Dashboard validation code. This will hopefully be a trivial amount of work to
finish off and then I will be submitting it for a security review with the Mozilla InfraSec team.
Henrik has released MemChaser 0.2.1 last week to be able to use the new console parser code that was added. Have a look at all the
new details in his blog post.
Dave has been investigating a regression in memory usage within Firefox. All the details can be found in Bug 724553
We have a number of projects that could do with some help so if you have free time have a look and see how you can help us!
Look out for future updates and if you want to join our team meetings you can get all of the details at here and you can also get notes from previous meetings.
Area: blog
Thu 15 Mar 2012
This week the team has been busy starting to wrap up projects as we near the end of the quarter.
We are releasing a new version of Nightly Tester Tools after have made a few minor changes. Henrik has been working with a couple contributors to get it done. You can download the latest version from AMO!
Geo, Henrik and Dave have been working to finish off our Mozmill CI system. Geo has been trying to solve and issue with Windows 2000 machines executing tests from Jenkins. Henrik has managed to get specific tests to be executed for different branches when Mozilla Pulse notifies about finished Firefox builds. This is all starting to look very good!
I have started work on data validation for the Mozmill Dashboard, the web application we use to view results. This is a minor task that is needed for the dashboard to become visible to the outside world!
Memchaser has had a couple pull requests opened recently. We are looking to add a few more features by the end of the month and release another version out on AMO!
We have a number of projects that could do with some help so if you have free time have a look and see how you can help us!
Look out for future updates and if you want to join our team meetings you can get all of the details at here and you can also get notes from previous meetings.
Area: blog
Thu 08 Mar 2012
It's amazing how quickly a week can go by!
The team has been hard at work on all of their projects.
Cameron has managed to get a stage and dev server up and running for Case Conductor so that people can help test it.
Dave has also done a proof of concept of a Proposal he blogged about a while back. With this, and being on track for a 1.0 release of Case Conductor at the end of the month this is looking great!
Our CI projects for Mozill are coming along nicely. Geo has been testing on getting this multiple Windows versions. Multi-platform products can create an interesting matrix of what needs testing.Henrik added support for Firefox ESR 10.0 to the CI system and will finishing that off soon! I am really excited by this project.
If you missed it last week, Memchaser 0.2 has been released and blogged about it! Thanks to the entire team for contributing to this!
Finally, David Guo, our intern, has been working on moving our Mozmill tests over to work with Mozmill 2.0. Currently we are working with 1.5.x and have been waiting for 2.0 to be stablised. The A*team have done a great job of stabising the code for us to now update!
We have a number of projects that could do with some help so if you have free time have a look and see how you can help us!
Look out for future updates and if you want to join our team meetings you can get all of the details at here and you can also get notes from previous meetings.
Area: blog
Fri 02 Mar 2012
The team have been really busy over the last week. Dave Hunt joined the WebQA work week. They had their first ever live test day. I suggest that you go have a read of what happened on Dave's blog. I for one am really glad that it was success and hope that it can be emulated again in the new London office soon!
The team has also been really hard at work getting Memchaser 0.2 finished. Henrik Skupin has blogged about what we improvements have been made. This has been approved on AMO so you should be told about the update soon!!.
Cameron has been working on Case Conductor and has release 0.8. Major kudos to him on getting it all the way to this point and still be on track for a 1.0 release at the end of the month!
We have a number of projects that could do with some help so if you have free time have a look and see how you can help us!
Look out for future updates and if you want to join our team meetings you can get all of the details at here and you can also get notes from previous meetings.
Area: blog
Tue 28 Feb 2012
Let me start by explaining what Travis CI is. Travis CI is an open source cloud CI solution that integrates with GitHub so you can build and test your applications. It is free to use for OSS projects. They started only supporting Ruby and have grown really quickly to support other languages.
The other week I explained what my team is up to and I mentioned that we are working on a Firefox Addon to visualise what is happening with Garbage collection in the browser. This is very important since garbage collection taking a lot of time can have a negative impact on the user experience.
This week I wanted to make sure that we have a CI for the work that we are doing so we can spot if we have any regressions. We all know that manual testing is long, tedious and can miss things. Below are the steps that you need to take to get a JetPack addon built and tested. This is for Addons not built on the Builder site. In theory this will work on older addons too.
You can see an example of this here. Thanks to Travis CI guys for making this possible and if you find it useful show them some Love!
Area: blog
Fri 24 Feb 2012
This week has been relatively quiet for the team in terms of their usual projects mainly because there was a chemsplill last week. A chemsplill is where a major exploit has been found and we need to get this closed as quickly as possible so that Firefox users are safe from malicious code execution. Unfortunately details of this have not been released yet but you will have noticed that 10.0.2 was released last week.
The team has also been working hard on getting more tasks for contributors. We have also added more mentored projects, below is links to projects that have tasks community members can get involved in
Cameron has been working hard on Case Conductor, the a test case management tool. 0.8 is now done and ready for deployment. Cameron is also working with WebQA to get more test coverage on it and get it out the door. Cameron has been discussing with the Case Conductor team about how we can have a 1.0 release by the end of March. Look out for more on this in weeks to come.
Look out for future updates and if you want to join our team meetings you can get all of the details at
here and you can also get notes from previous meetings.
Area: blog
Wed 15 Feb 2012
Automation Services, the team I co-lead with Henrik Skupin, are going to start sending out weekly updates so that
everyone can be aware of what we have been upto in the last week on our main projects.
Nightly Tester Tools
Our team is now taking ownership of the addon from Heather Arthur. This extension has over 100000 active users at the
moment and we want to grow that community. We need to still do a few logistical things with Heather to get it moved over
and we hope to have this sorted in the next few weeks. Once that is done we will do some house keeping on the project like changing the repository
structure and getting build scripts in place. Next quarter we hope to do some work on the project and will create some
issues out that will be mentored by someone in the team for anyone to work on!
Actually we will be creating mentored issues for a lot of our projects at the moment and will have a central place for these soon! In the mean
time you can have a look at:
MemChaser
One of the major projects on the go at the moment is a project to see where we have large GC/CC pauses in the browser.
Long pauses lead to the browser feeling jerky which isn't a great user experience. Last week Henrik blogged about the MemChaser 0.1.1
release and got a couple
of good comments! It has helped identify a few bugs. A really good example is where we identified memory leak with
Add-ons SDK bug, which is already partly fixed for SDK v1.6 and even for the upcoming v1.5 release.
Currently we have over 1000 downloads and nearing 400 active daily users
(statistics). We need to get docs in place for
further planning on the project but this is a great start!
Look out for future updates and if you want to join our team meetings you can get all of the details at
here and you can also get notes from previous
meetings.
Area: blog
Thu 26 Jan 2012
Back in 2008, David Henderson and I wanted to try automate collecting client side performance data for the web application that we were working on at the time. We were getting a large number of complaints from users about load time and we need to try solve this. The way that we decided on doing this was to use Selenium, which was already running our tests, and hack the Selenium server to give us the information we needed. What we came up with was showcased at GTAC 2009. It was really good for the time and we were happy.
About a year later just as the new movement for HTTP Archive or HAR as its commonly known was taking off I found that Jan "Honza" Odvarko had created a Firebug add on to export the network tab to HAR. This, when used with Selenium, could mean that we could get the same data that the browser wanted with little to no effort. Run your tests as you were with the new WebDriver API and get it to programmatically install the necessary addons as well as set all the preferences needed. I wrote this for Python and for .NET. This works really well but now limits this type of data to only being collected within Firefox. Some modern web applications can send back totally different javascript and sprites dependant on which browser hits.
What if you could run your Selenium tests and collect the same info as the if you were using the Firebug Net tab but using any browser. Browsermob Proxy is a good way to collect this information and it has a programmatic interface that allows us to set it before our tests start. It can also return a HTTP Archive of the traffic that it is routing. I have released the Python Library for BrowserMob Proxy that can be injected into tests quite easily. I have put an example below.
from browsermobproxy import Server
server = Server("path/to/browsermob-proxy")
server.start()
proxy = server.create_proxy
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.set_proxy(proxy.selenium_proxy())
driver = webdriver.Firefox(firefox_profile=profile)
proxy.new_har("google")
driver.get("http://www.google.co.uk")
proxy.har # returns a HAR JSON blob
proxy.stop()
driver.quit()
If we now feed the data from proxy.har into HAR Viewer we can get a nice waterfall of the traffic as below! Jari Bakken has a useful commandline version called Har.
I recently showed this off at London Web Performance. Unfortunately it wasn't recorded but you can view the slides here
Area: blog
Thu 12 Jan 2012
WebDriver, the browser automation framework we all love, is on its way to becoming a browser standard. WebDriver as an OSS project is 5 years old and started being merged into the Selenium project about 3 years ago. From there we have seen WebDriver grow. It has had a lot updates in that time. The team is currently averaging ~100 commits a week.
In the short time, from GTAC 2009 when they were created, the Ruby bindings created by Jari Bakken have had over a million downloads! The Python bindings have around 3000 downloads a week and growing. Jim Evans work on the .NET bindings is great, I don't have figures because of Nuget. Same with the Java bindings. Maven makes it difficult to know how many downloads.
We have also seen that Selenium Jobs have overtook the commercial counterparts. All of these things point to us needing a standard to make sure if we can do the same thing on multiple devices and multiple OSes.
Yesterday we held a W3 meeting in the Google UK office. It was great to be able to get in a room and discuss what needed. A plan was hatched on what needs to be done next, and it is going to be a lot of work.You can see the minutes of the meeting here to see what we discussed.
Mozilla, Opera and Google are happy with the direction we are going and long may it last! It will be great to get Microsoft and Apple involved soon.
Remember that all of this work is happening out in the open and you can watch the Hg repository for the editors draft updates.
Area: blog
