Mach is an amazing tool which facilitates a large number of common user stories in the mozilla source tree. You can perform initial setup, execute a build, run tests, examine diagnostics, even search Google. Many of these things require an object directory. This can potentially lead to some confusion if you typically have more than one object directory at any given time. How does mach know which object directory to operate on?

This probably isn’t a groundbreaking revelation to most Mozillians, but it’s something I haven’t quite managed to put into words until now.

On the Tools and Automation team at Mozilla, the easiest way to describe what we do is to say “We try to make the lives of others within the community easier”. This is a vague statement, but it’s about as close as we can get without writing a small dissertation.

Before now running ‘classic’ unittests (mochitest, reftest, xpcshell, etc.) on B2G emulators has been a massive pain. The new recommended way of running them is through mach.

1. Update B2G repo if you haven’t already: git pull
2. Configure an emulator: BRANCH=master ./config.sh emulator
3. Build: ./build.sh
4. Run: ./mach mochitest-remote

OUTDATED

The steps in this article are no longer accurate. Pushing a custom command line is easier now. See here for more details.

Have you ever wanted to see the test results of a custom command line in try? Things like –test-manifest, –shuffle or –run-slower? Now you can! The process isn’t exactly optimized for the developer use case, but neither is it really difficult to do once you know how.

qqver is a Mercurial extension that forces mq to version all patch queues in the same repository. The syntax is the exact same as when doing normal patch queue versioning (i.e ‘hg init –mq’), except the repo is created one directory higher so it is able to track all patch queues. As with stock queue repos, new and existing patches are added automatically.

I came across an odd python problem the other day. I wanted to format a log message without printing it with python’s logging module. But I couldn’t find any examples or others who wanted to do the same thing.

You might ask why I would want to do this. In my case, I was using sys.stdout.write() to display the progress of a download as a percentage. Python’s logging module automatically appends a newline to your message and there didn’t seem to be a way to change this without a subclass. Anytime you want to overwrite something on the previous line with python logging, you will run into trouble.

Hey! Check it out, I have crash stacks in my logs now! I know, I know, I took my sweet time. I’m sorry. I hope this doesn’t damage our relationship. Is there anything I can do to make it up to you? I want to help. I really do. It’s just. I don’t know. I guess I’ve been feeling under the weather lately. I must have caught a bug.

Throughout most of our B2G test automation deployment, we’ve been very conscious about not enabling too many tests simply because we didn’t have enough capacity on our test slaves to run them all. Regardless it was still bad enough as it was (many of you probably experienced very long wait times for results). Thanks to releng (and especially Rail Aliiev) we are now running most of our B2G tests in Amazon AWS which means we can be much more flexible in accomodating load.

As B2G continues to trod onwards to its release, there is still a lot of confusion about the level and state of test coverage it has. Back in November we started running mochitests, reftests and marionette/webapi tests on ARM emulators. Now we’ve also added xpcshell tests and for the most part we have these nice green letters to look at on TBPL that make us feel good about ourselves. But what is really being run? What is the meaning behind these letters “M”, “R”, “Mn” and “X”? Are there any causes for concern? Are there other tests being run that don’t show up on TBPL? What are the current automation priorities? What are the next platforms to use after emulators?

This blog post aims to answer these questions and more. It is a comprehensive snapshot of the current state of automated testing on B2G.

Contrary to popular belief, we (the A-Team) have been running mochitests, reftests, marionette tests and webapi tests on B2G in some form of continuous integration or another for about 5 months now. They’ve been reporting results to a TBPL look-alike called autolog, and were run on Amazon EC2 VM’s with emulators. This was a temporary solution to get something stood up quickly while we moved towards our ultimate B2G automation goal - tests running on Pandaboards and reporting to TBPL.

As of this week, while there are still no tests running on Pandaboards, I’m happy to say we have emulators running mochitests, reftests and marionette/webapi tests, all reporting to TBPL.