One of the most painful aspects of a developer’s work cycle is trying to fix failures that show up on try, but which can’t be reproduced locally. When this happens, there were really only two options (neither of them nice):

1. You could spam try with print debugging. But this isn’t very powerful, and takes forever to get feedback.
2. You could request a loaner from releng. But this is a heavy handed process, and once you have the loaner it is very hard to get tests up and running.

I’m pleased to announce there is now a third option, which is easy, powerful and 100% self-serve. Rather than trying to explain it in words, here is a ~5 minute demo:

Mach is the Mozilla developer’s swiss army knife. It gathers all the important commands you’ll ever need to run, and puts them in one convenient place. Instead of hunting down documentation, or asking for help on irc, often a simple |mach help| is all that’s needed to get you started. Mach is great. But lately, mach is becoming more like the Mozilla developer’s toolbox. It still has everything you need but it weighs a ton, and it takes a good deal of rummaging around to find anything.

Frankly, a good deal of the mach commands that exist now are either poorly written, confusing to use, or even have no business being mach commands in the first place. Why is this important? What’s wrong with having a toolbox?

Today marks the 5 year anniversary of try syntax. For the uninitiated, try syntax is a string that you put into your commit message which a parser then uses to determine the set of builds and tests to run on your try push. A common try syntax might look like this:

try: -b o -p linux -u mochitest -t none

Since inception, it has been a core part of the Mozilla development workflow. For many years it has served us well, and even today it serves us passably. But it is almost time for try syntax to don the wooden overcoat, and this post will explain why.

I mentioned in my previous post a mercurial extension I wrote for making bookmarks easier to manipulate. Since then it has undergone a large overhaul, and I believe it is now stable and intuitive enough to advertise a bit more widely.

Introducing bookbinder

When working with bookmarks (or anonymous heads) I often wanted to operate on the entire series of commits within the feature I was working on. I often found myself digging out revision numbers to find the first commit in a bookmark to do things like rebasing, grafting or diffing. This was annoying. I wanted bookmarks to work more like a git-style branch, that has a definite start as well as an end. And I wanted to be able to easily refer to the set of commits contained within. Enter bookbinder.

This is a continuation of my previous post called The New Mercurial Workflow. It assumes that you have at least read and experimented with it a bit. If you haven’t, stop right now, read it, get set up and try playing around with bookmarks and mozreview a bit.

You may not know that most of our test harnesses are now outputting structured logs (thanks in large part to :chmanchester’s tireless work). Saying a log is structured simply means that it is in a machine readable format, in our case each log line is a JSON object. When streamed to a terminal or treeherder log, these JSON objects are first formatted into something that is human readable, aka the same log format you’re already familiar with (which is why you may not have noticed this).

There’s a good chance you’ve heard something about a new review tool coming to Mozilla and how it will change everything. There’s an even better chance you’ve stumbled across one of gps’ blog posts on how we use mercurial at Mozilla.

With mozreview entering beta, I decided to throw out my old mq based workflow and try to use all the latest and greatest tools. That means mercurial bookmarks, a unified mozilla-central, using mozreview and completely expunging mq from my workflow.

tl;dr Look for reports like this in the near future!

At Mozilla, platform developers are culturally bound to tbpl. We spend a lot of time staring at those bright little letters, and their colour can mean the difference between hours, days or even weeks of work. With so many people performing over 420 pushes per day, all watching, praying, rejoicing and cursing, it’s paramount that the whole process operates like a well oiled machine.

That’s not a rhetorical question. I’d like to know in which scenarios a mixin in python really is the best option. I can’t seem to think of any, but maybe I’m not thinking outside the box enough.

The basic idea of a mixin is to create a small re-usable class that can “plug-in” to other larger classes. From the wikipedia definition, a mixin is a way to compose classes together without using inheritance. The problem is unlike ruby, python mixins are a purely conceptual construct. Python mixins are inheritance (the only difference is that the class name usually contains ‘Mixin’). It is up to the developer to remember this, and to manually avoid all of the common pitfalls of multiple inheritance. This kind of defeats the whole purpose of the mixin in the first place. What’s more is that most people use python mixins improperly.

My last post was basically a very long winded way of saying, “we have a problem”. It kind of did a little dance around “why is there a problem” and “how do we fix it”, but I want to explore these two questions in a bit more detail. Specifically, I want to return to the two case studies and explore why our test harnesses don’t work and why mozharness does work even though both have IFFY (in flux for years) requirements. Then I will explore how to use the lessons learned to improve our general test harness design.