Code V.igoro.us

Buildbot's WebStatus display has, for a long time, had an allowForce option which controls what kind of mayhem can be wrought via the web interface. Historically, this has been a boolean option: either web users can do everything (force builds and shut down slaves) or nothing. Bug 701 asks that we change that to give more granular access control.

Buildbot has an interesting way of separating the status display from the control functionality. It has two parallel interface hierarchies, IStatus and IControl, implementing the necessary methods. The IStatus hierarchy is illustrated with the orange bubbles here:

The IControl hierarchy is similar, although it only goes down to the Build level right now.

When allowForce is true, the WebStatus object adapts the buildmaster to the IControl interface and adds a link to the result in its control attribute. Forcing a build or shutting down a slave then uses this object to navigate to the appropriate control object and calls a method from the corresponding interface. If the control attribute is None, no access is allowed.

This scheme has the advantage that it is difficult to accidentally expose functionality, since when allowForce is false, the control methods are inaccessible. However, it has the disadvantage of not allowing any more granular level of access control.

I just reworked the web status to have a more flexible authorization mechanism, and while I wasn't able to remove the IControl hierarchy entirely, I was able to marginalize it to only those code blocks that need to perform controlled actions, instead of passing control objects all over the place.

Charlieplexing is a technique to allow n tristate digital output pins to control n(n-1) LEDs, by exploiting the high-impedence state of the output pins and the reverse-bias tolerance of LEDs.

I put together a circuit to test this out, similar to the 6-LED diagram in the Wikipedia entry. I added three 220Ω resistors, though. This ASCII art sums it up nicely:

      +--->|---+          +--->|---+          +--->|---+    
      |  LED1  |          |  LED3  |          |  LED5  |
 <----|        |-/\/\-+---|        |-/\/\-+---|        |-/\/\-+-->
      |  LED2  |      |   |  LED4  |      |   |  LED6  |      |   
      +---|<---+      R   b---|<---+      G   +---|<---+      B   

where R, G, and B are the three connections to the Arduino, and the arrows on the left and right side are joined by a jumper. I put together some simple Arduino-level code to drive this layout in a simple counter sequence, and fired it up.

One thing I will say about the Arduino: tinkering pays off. After the usual forgotten-semicolon cleanup, LEDs started blinking immediately. This is good, because I don't have much of an Arduino-debugging toolbox at this point!

Anyway, there's a problem. When an LED is fully illuminated, two other LEDs glow at about 50% brightness. For example, when LED1 is on, LED4 and LED6 glow too. It's not hard to see why: to turn LED1 on, the arduino drives R low and B high. The path through LED1 is intended, but LED4 and LED6 are also forward-biased in series, albeit with more resistance inline and the voltage-drop from two LEDs. Can this be prevented?

Let's work out the math here, using the resistance r as the only parameter I can change. The LEDs I'm using (5mm Green LEDs from CHINA YOUNG SUN) don't have a lot of performance data, but do list a forward voltage drop of 1.8-2.2V, so let's use 2V. My Arduino is running from the USB power at 5V. The resistor to the left of R defines the current through that leg for the remaining 5-2 = 3V, so I₁ is 3/r. On the leg containing LED4 and LED6 a total of 1V remains across two resistors, so I₄ and I₆ are both 1/2r.

The LED datasheet lists a maximum average current (20ma) and peak current (30ma), but doesn't list a minimum current. Guessing 1ma, I need 1/2r < 0.001, so r > 500. At that point, though, I₁ is just 6ma - not nearly bright enough to impress, particularly with a 16% duty cycle when multiplexing all 6 LEDs. All the same, I figured I'd try it. The closest resistor I had was 470Ω, so I gave that a shot. The result was bad on all fronts: the expected LED is dim, and the unwanted LEDs are still visible.

I suspect that, to get this to work, I'd need to reduce the supply voltage to something less than twice the LEDs' forward voltage drop. Then two LEDs in series would carry no current. I considered doing this with a simple voltage divider on each microcontroller pin, but this of course ruins the tri-state behavior that's critical to charlieplexing. I could build that by using two transistors for each pin, but by that point the benefits of charlieplexing in terms of component count are long lost.

So, in short, for good charlieplexing, make sure that your source voltage is less than twice the V_f of your LEDs.

[EDIT: it turns out, after some futzing that this also "works fine" if you just leave out the ballast resistors. Presumably this relies on a limited duty cycle and a "high enough" internal resistance to not melt the LEDs]

I have an early Mac Mini that's not good for much of anything anymore. It's not too fast, and it's a 32-bit PPC chip, so most proprietary-blob software is off-limits, meaning it doesn't make a good media PC. I put Gentoo on it, and decided to use it to interface with my new Arduino Duemilanove. I'm more interested in the Arduino as a processor I can play with, rather than a gateway to flashing LEDs, so I don't want the full IDE - just a compiler and assembler, and a USB-based programmer.

This post documents the process I followed to set all of this up.

I ran across a surprising weakness in Perl, regarding object identity. I was writing a function to handle XMsgs, which are messages used by the Amanda transfer architecture to indicate the progress of a transfer. Messages sent from any transfer element are delivered to the same handler, and in this case I needed to know which element had sent the message. Fortunately, XMsg objects have a elt attribute pointing to the sending element.

Svnmerge has always been the kid brother of Subversion. It's in the project's contrib directory, and is thus unversioned. Since it's a single Python script, users just download it directly from the repository. In the last year, the script has only seen a few non-trivial commits, and the mailing list has been almost silent.

There are some simple reasons for decline. First, Subversion itself, in version 1.5, has adopted some of the functionality of Svnmerge. In particular, svn now uses properties (svn:mergeinfo) to track the revisions that have and have not been merged into a particular branch. There are some limitations, of course. Most obviously, a branch can only be "reintegrated" into trunk once, which is not the workflow of many Svnmerge users. Also, to my knowledge, Subversion cannot merge between repositories, while Svnmerge can.

Second, Git, Mercurial, and other DVCS's now provide strong support for merge-heavy workflows. Both tools also have excellent "gateways" to Subversion. For example, Amanda's Github repository tracks the Subversion repository using some simple shell scripts.

Between these two forces, I think that much of the audience for Svnmerge has disappeared. Those left, sadly, will see even less support.

This contest recently came to my attention. I think I'll give it a shot. The challenge isn't too clear -- exactly what constitutes a "superseded" order is left pretty vague -- but there are lots of opportunities for underhandedness in the task.

I love the fact that GCC has basically eliminated the need to run lint, by adding every imaginable warning. But there are two problems.

First, generated code (at this point I'm being bitten by code generated by rpcgen for NDMJOB) seldom runs without warnings. Usually these are the "unused XXX" warnings, but some generated code is really and truly broken - K&R style definitions, signed/unsigned confusions, the works.

Second, warnings come and go between GCC versions, so just because it runs fine on my system does not mean it will run fine on all of the GCC's Amanda gets built on (never mind any other compilers!). Here's where I find myself trying to figure out how to disable a particular bogus warning for some ancient version of gcc, which is naturally not in the manuals now available on the web. And here's where my favorite GCC trick is handy: -fdiagnostics-show-option will cause gcc to display the -W option that controls that particular warning. From there, you can google the warning, or just disable it if you've determined that it's spurious and not worth the code changes required to make it go away.

I thought I'd throw that up here so I don't forget about it again.

I stumbled upon a summary of the Drefyus model of skill acquisition by Andrew Hunt today, copied out of the powerpoint here. There's a bullet point in there that I hadn't seen before:

1. Level 1: Beginner
   • Little or no previous experience
   • Doesn't want to learn: wants to accomplish a goal
   • No discretionary judgement
   • Rigid adherence to rules

I've been toying with the idea of a beginner's guide to Amanda for some time. My vision was of a shallow but broad description of how Amanda fits together: clients and servers, applications, holding disks, changers, devices, and so on. Scaffolding, in the constructivist sense. The intent was to head off some of the more ignorant questions about Amanda, and give beginners a base on which to build their own Amanda configurations.

But this won't work! A beginner does not want to learn, and anything that tries to "teach" a beginner is just so much noise to him or her. In fact, most beginner guides are nothing of the sort -- they are probably only useful to those already at level 2.

This is all the more the case with Amanda: like plumbing, Amanda users just want backups to happen so they can concentrate on their core business, so even smart folks will happily remain at level 1 if they can make it work. So what can we do to support these users? And how can we encourage people to move to the next level?

(from a comment on Zen and the Art of Nonprofit Technology)

I've realized recently that open-source projects have developed a class structure. First, consider the broad middle-class of little-known but functional projects that have a number of users and several developers (Amanda, Buildbot, Cfengine, Sphinx, Curl, the autoconf archive, maybe GMT). Above these projects are the superprojects that nearly everyone uses and that have hundreds of contributors (Firefox, Linux, Python, Perl, to name a few). These are all islands in a squalid sea of abandonware and single-developer projects -- we've all got one or two, right?

What are the keys to mobility between these classes? What factors might keep an otherwise-deserving project in the lower or middle classes?

Now that I'm using git all the time, it's helpful to have bash show me what branch I'm on, and whether the working directory is dirty. I found a gist by Henrik Nyh that does just this. I souped it up a little bit to be more efficient and handle a few extra situations. Here's the result: [EDIT: I have updated this gist since this post was made; the current version is based on git-completion.sh in the git distribution]

Please use it if you like, and fork the gist if you make any improvements!

Several Amanda developers use git internally, but the Amanda source code is in Subversion at SourceForge. Git-svn manages bidirectional mirroring for us, and works flawlessly.

Recently, however, we introduced a problem through user error: due to a typo in our git-authors file, a bunch of revisions were mirrored with incorrect author information. This was more than a cosmetic problem because it caused the SHA1 hashes to differ between developers who fixed the typo at different times (because the author information is included in the data that feeds the hash algorithm). This would leave us with divergent commits forever.

The challenge, then, is to make git-svn think that it never fetched that revision. HEAD was at r1413, but r1391 had the bad author. We branched for release right after the bum commit, so there are two branches to deal with. Here's what I did:

First, roll back the remote branches (e945b67d78c239b42cb882e5c28e24354d0c05f0 is r1390)

$ git update-ref -m "roll back git-svn" ext/trunk e945b67d78c239b42cb882e5c28e24354d0c05f0
$ git update-ref -m "roll back git-svn" -d ext/amanda-261 cad126843a7649ca3e05088dd46ee41d7f17e7e2

Next, edit both maxRev values in git-svn's metadata (.git/svn/.metadata):

[svn-remote "ext"]
    reposRoot = https://amanda.svn.sourceforge.net/svnroot/amanda
    uuid = a8d146d6-cc15-0410-8900-af154a0219e0
    branches-maxRev = 1413
    tags-maxRev = 1413

Finally, delete the .reflog

$ rm .git/svn/ext/trunk/.rev_map*
$ rm .git/svn/ext/amanda-261/.rev_map*

Then just re-run the fetch:

$ git svn fetch ext

I'm watching the DNC's Rules & Bylaws Committee meeting, discussing the fate of the Michigan and Florida delegations. This committee consists of the elite among the Democratic superdelegates. These are all powerful, intelligent, important people (including the wonderful Donna Brazile).

What has me upset is that I've now heard two women explain that they don't have the "mathematical genius" to work out the proper representation for these states; no men have made such a declaration. First of all, that's bull: anyone can work out what 33% of 128 is. The difficult part is in the politics, and everyone on this committee is an expert on politics.

More importantly, though: why is it OK for these women to brag about their mathematical ignorance? What message does this send to PoliSci students struggling through calculus? One of the women to mention this was Alice Travis Germond, who was a VP of NARAL -- hardly an advocate of keeping women barefoot, pregnant, and in the kitchen. What is she thinking?

[UPDATE] The other woman to brag about her innumeracy was Tina Flournoy -- Gore's finance director! Finance!

I just stumbled on Ted T'so's Organic vs. Non-Organic Open Source, Revisited. The distinction is that an organic open source community has a diverse development community, in terms of motivations or, more concretely, employers. Conversely, a non-organic community is dominated by developers from a single company.

This has put words to a question I've been wrestling with for a long time. In any community, those who do the most work have the strongest voices and the most power. As someone who knows his way around the Amanda codebase, I hold the fate of a user's feature request in my hands: if I like it, I can implement it, and if I don't, it will be relegated to the dusty archives of the amanda-users list. Similarly, when I make a proposal, non-developers must respond in a subordinate voice: "well, I won't be writing it, but I think .." This is not due to any malice on my part, but simply a fact of the relationship of consumer to producer.

The bulk of Amanda development is currently performed by Zmanda employees. That's a simple fact that Ohloh can verify for you. That makes the Amanda development community non-organic, in Ted's terminology. In Raymond's terms, we're all cathedral and no bazaar. As Ted points out, this can be a good thing -- I think that we do great work, and that we go out of our way to listen and respond to the user community's ideas and requests. A tight group of developers can move quickly, and consensus decisions are easier and less time-consuming. However, power corrupts, and I would like to have other developers out there to tell me, "no," or, "I have a better way," occasionally.

But how?

Some time back, I had been thinking of trying to trace the "social network" of open source developers. I'm interested questions like

• How many open-source developers contribute significantly to multiple projects? How many projects?
• Is the set of developers reasonably well-connected by the "works on a project with" relation?
• What proportion of projects are basically single-developer projects?
• What is the distribution of number of developers per project, and relative levels of contribution from those developers? How unique is the Linux kernel in having a large set of substantial contributors?
• How many open-source projects are there (above some level of completeness and activity)?

Long story short, I had grand plans for parsing VC histories, ChangeLogs, AUTHORS files, and so on, but never got around to actually coding anything. Then I found Ohloh. They've taken the liberty of indexing a lot of publicly available code, and have some web-based knobs that you can tweak to clean up some of their data. They have REST API available to access their data, and some of their code (their SLOC counter) is open source. I'm pretty impressed.

I will probably try out the API and see if I can answer some of these questions, but I worry that their database is quite dirty. They rely on individual users to log in to the site and "claim" their identity -- connecting e.g., their SourceForge login to their Berlios.de login to their logins on project-specific sites. I would like to see them apply some additional intelligence to matching users up; for example, email addresses appear in lots of locations (ChangeLogs, AUTHORS, commit messages), and provide a fairly reliable proxy for a particular person.

Every girl gets the memo:

To: Girls
From: The Establishment
Re: Math

Math is hard, and you'll never need it anyway.  
You'll just be cooking and making babies, after all.

cc: Boys

But most teen girls, being the modern anti-establishment type, reject this sentiment as "the man" trying to keep them down. They look to the female role models around them for guidance as to what "real life" is like. That comes down to teachers, moms, aunties, and big sisters.

Last week, while I was working on an algebra problem with a group of 7th-grade girls, the classroom teacher (a woman) said, "I'm not good at math."

To: Girls
From: Grown-up Girl
Re: Math

I never learned math, and now I'm a teacher, so you can safely ignore it.

But what did this teacher really mean? She can solve the problem we were working on, no problem. What's hard is teaching math. As with most subjects, a teacher needs to be agile enough to dance around the math, attacking it from all angles, spotting the misconceptions and shreds of understanding in a student's responses, and asking the right questions.

That's legitimately hard, and I think it's unreasonable to ask someone to be this good at more than one subject. I'm a pretty OK math teacher, but if you plop me down in front of a language arts class, I'll struggle. I'm OK with that, but the proper phrase is "I'm not good at teaching language arts," not "I can't read."