Hunting the Shmoo

It’s no secret that I’m not a fan of try syntax, it’s a topic I’ve blogged about on several occasions before. Today, I’m pleased to announce that there’s a real alternative now landed on mozilla-central. It works on all platforms with mercurial and git. For those who just like to dive in:

$ mach mercurial-setup --update  # only if using hg
$ mach try fuzzy

This will prompt you to install fzf. After bootstrapping is finished, you’ll enter an interface populated with a list of all possible taskcluster tasks. Start typing and the list will be filtered down using a fuzzy matching algorithm. I won’t go into details on how to use this tool in this blog post, for that see:

$ mach try fuzzy --help  # or
$ man fzf

I’ve previously blogged about why I believe try syntax is an antiquated development process that should be replaced with something more modern and flexible. What follows is a series of ideas that I’m trying to convert into a concrete plan of action to bring this about. This is not an Intent to Implement or anything like that, but my hope is that this outline is detailed enough that it could be used as a solid starting point by someone with enough time and motivation to work on it.

One of the most painful aspects of a developer’s work cycle is trying to fix failures that show up on try, but which can’t be reproduced locally. When this happens, there were really only two options (neither of them nice):

1. You could spam try with print debugging. But this isn’t very powerful, and takes forever to get feedback.
2. You could request a loaner from releng. But this is a heavy handed process, and once you have the loaner it is very hard to get tests up and running.

I’m pleased to announce there is now a third option, which is easy, powerful and 100% self-serve. Rather than trying to explain it in words, here is a ~5 minute demo:

Mach is the Mozilla developer’s swiss army knife. It gathers all the important commands you’ll ever need to run, and puts them in one convenient place. Instead of hunting down documentation, or asking for help on irc, often a simple |mach help| is all that’s needed to get you started. Mach is great. But lately, mach is becoming more like the Mozilla developer’s toolbox. It still has everything you need but it weighs a ton, and it takes a good deal of rummaging around to find anything.

Frankly, a good deal of the mach commands that exist now are either poorly written, confusing to use, or even have no business being mach commands in the first place. Why is this important? What’s wrong with having a toolbox?

Today marks the 5 year anniversary of try syntax. For the uninitiated, try syntax is a string that you put into your commit message which a parser then uses to determine the set of builds and tests to run on your try push. A common try syntax might look like this:

try: -b o -p linux -u mochitest -t none

Since inception, it has been a core part of the Mozilla development workflow. For many years it has served us well, and even today it serves us passably. But it is almost time for try syntax to don the wooden overcoat, and this post will explain why.

You may not know that most of our test harnesses are now outputting structured logs (thanks in large part to :chmanchester’s tireless work). Saying a log is structured simply means that it is in a machine readable format, in our case each log line is a JSON object. When streamed to a terminal or treeherder log, these JSON objects are first formatted into something that is human readable, aka the same log format you’re already familiar with (which is why you may not have noticed this).

There’s a good chance you’ve heard something about a new review tool coming to Mozilla and how it will change everything. There’s an even better chance you’ve stumbled across one of gps’ blog posts on how we use mercurial at Mozilla.

With mozreview entering beta, I decided to throw out my old mq based workflow and try to use all the latest and greatest tools. That means mercurial bookmarks, a unified mozilla-central, using mozreview and completely expunging mq from my workflow.

tl;dr Look for reports like this in the near future!

At Mozilla, platform developers are culturally bound to tbpl. We spend a lot of time staring at those bright little letters, and their colour can mean the difference between hours, days or even weeks of work. With so many people performing over 420 pushes per day, all watching, praying, rejoicing and cursing, it’s paramount that the whole process operates like a well oiled machine.

That’s not a rhetorical question. I’d like to know in which scenarios a mixin in python really is the best option. I can’t seem to think of any, but maybe I’m not thinking outside the box enough.

The basic idea of a mixin is to create a small re-usable class that can “plug-in” to other larger classes. From the wikipedia definition, a mixin is a way to compose classes together without using inheritance. The problem is unlike ruby, python mixins are a purely conceptual construct. Python mixins are inheritance (the only difference is that the class name usually contains ‘Mixin’). It is up to the developer to remember this, and to manually avoid all of the common pitfalls of multiple inheritance. This kind of defeats the whole purpose of the mixin in the first place. What’s more is that most people use python mixins improperly.

My last post was basically a very long winded way of saying, “we have a problem”. It kind of did a little dance around “why is there a problem” and “how do we fix it”, but I want to explore these two questions in a bit more detail. Specifically, I want to return to the two case studies and explore why our test harnesses don’t work and why mozharness does work even though both have IFFY (in flux for years) requirements. Then I will explore how to use the lessons learned to improve our general test harness design.