Category: Technical

Day 53: An Engineering Mindset

I have spoken out many times about how much I dislike software developers granting themselves titles that include the terms “engineer” or “architect” in them. Those are titles that are earned through years of hard study and certification, just like doctors and lawyers.

Software development is not engineering. If I want to build a vaulted ceiling in my home, there are tables of the properties of materials that give the acceptable spans for any given size of component. If I have a 20-foot span, I can look at those tables and determine what I would need with steel beams, engineered wood beams, or plain 2×10 lumber.

Engineered Lumber & Beams | NICHOLS LUMBER AND HARDWARE
Each of these engineered wood beams has a known maximum span.

There’s no need to debug this; these are known values, along with gravity, weight, and all the other things that go into building design. And you could use a table for materials from 100 years ago, and it would still work today. Knowing how to apply these values in the structure behind the design of buildings takes years of study before one can legally call themselves an architect or an engineer.

The most apt term I’ve heard for a software developer is “gardener”. It’s accurate, as gardens take up-front planning, and like software, they need to be maintained. Both also have their share of bugs!

So while I won’t call myself an engineer, I do have an engineering mindset. What I mean by that is I strive to be methodical about things like program structure, testing, code review, version control, and the like. I enjoy working with teams who are like-minded in that regard.

An engineering mindset isn’t limited to software development, though. I find I do that with every task I do. Take washing dishes, for example. We don’t have a dishwasher, so they all need to be washed by hand. Nor do we have a drainboard, so they have to go on a towel by the sink. I don’t just wash them in the order that they are piled in the sink; instead, I re-arrange them so that they are stacked in the order to be washed in the left sink basin, with the right sink basin empty. Why? Because the drying area is on the right, and this makes for a neat left-to-right flow. First the flat dishes get washed and placed in the right sink. They are then rinsed off, then placed vertically leaning against the wall to help them dry evenly and quickly. Then the bowls and other things with depth are washed, with the eventual placement in the drying area determining the order. Finally the small things like utensils are washed and placed in any available spot left on the towel. This allows the most efficient use of the limited drying area while still allowing things to dry fully.

The other thing I try to optimize for is to minimize the use of hot water. I’ve come up with some routines for group wetting and rinsing that has cut the amount of hot water usage. I’ve even played around with how an item is handled while being rinsed in order to get the most efficient rinsing of all surfaces.

I have been developing this process over the 4 years since we moved into this small house, and I believe I have it down to as efficient a flow as possible. I’ve mentioned this to people, and their reaction is along the lines of “wow, you’re pretty OCD!”. But this isn’t anything like OCD: people who suffer from that condition say that they are perfectly aware that their actions are silly or even harmful, but are unable to stop doing them. I choose every one of these steps because it makes sense. If someone were to show me an even better way to do it, I’d switch in an instant.

That’s an engineering mindset: taking a process, even one as mundane as washing dishes, and always thinking of ways to optimize it. It doesn’t feel like work, though, it’s more like solving a puzzle. Some people like crossword puzzles or jigsaw puzzles; I like solving dish washing puzzles!

Day 52: Happy 10th Birthday, OpenStack!

I just saw this announcement from the OpenStack Foundation about OpenStack’s 10th birthday! Yes, 10 years ago this week was the first OpenStack Summit, in Austin, TX, with the public announcement the following week at O’Rielly OSCON. Yet most people don’t know that I played a very critical role in the beginning!

OpenStack began as a joint venture between Rackspace (my employer at the time) and NASA. I was on the team at Rackspace that developed and supported its aging cloud compute services, and we were looking to develop something from scratch that could be much more scalable than our current system. Around that time Thierry Carrez saw an announcement from a group at NASA about their development of a compute virtualization system, and suggested to the powers that be at Rackspace that this might be a better way to go instead of developing the whole thing ourselves. From that followed a lot of discussion among the executives at Rackspace, as well as some conversations with NASA, and the conclusion was that we would team up. One of the first things to do was to get the developers for both groups together to discuss things from a more technical perspective. And this is where I believe that I made a critical decision that, had I chosen wrong, might have resulted in OpenStack never happening.

The NASA team was a consulting group, Anso Labs, and they were arriving in San Antonio, and we had plans to take them out to dinner, but no idea where. It was then that I suggested The Cove, a local place with lots of outdoor seating, a relaxed atmosphere, good beer, and delicious food. We had a great time that evening, and we all got to know each other. Had we been in a more conventional restaurant, people may have only gotten to know the people sitting next to them, but since The Cove is open seating, we moved around a lot, talking about both the technical stuff and personal stuff.

Over the next few days we began reviewing the code and exchanging idea on what needed to be developed next, and those discussions went very smoothly, getting a lot done in a short period of time. I still maintain to this day that without that first night having beer and food at The Cove, OpenStack might never have become the success that it did.

You’re welcome.

Day 47: Python Virtualenvs

If you’re not a Python programmer, you probably won’t find much in this post. Sorry.

If you are a Python programmer, then you probably know about virtualenvs, or virtual environments. They allow you to create several different Python environments to work in: each can have it’s own version of Python, as well as its own installed packages. This means I can work on a project that has particular requirements, and then switch to a project with completely different requirements, and the two won’t affect each other.

A while ago I used to use vitualenvwrapper, which made working with virtualenvs a lot easier. But as I switched to Python 3 over the past few years, I started to have some issues where it didn’t work correctly (sorry, I no longer recall precisely what those issues were). I’m sure a lot had to do with the addition of the venv module into Python 3, which allowed you to create a virtualenv by running python -m venv /path/to/virtualenv.

For a while I ran all the commands manually, but I have the quality that makes for good programmers: I’m lazy. A lazy person doesn’t want to do any more work than they have to, so if I can automate something to save time over the long run, I will. And here’s what I came up with:

export VENV_HOME=$HOME/venvs

function workon { 
    if [ -z $1 ]
    then
        ls -1 $VENV_HOME
    else
        source $VENV_HOME/$1/bin/activate
    fi
}

function mkvenv {
    python3 -m venv $VENV_HOME/$1
    workon $1
    pip install -U pip setuptools wheel
    pip install ipython pytest-pudb requests
}

function rmvenv {
    command -v deactivate
    rm -rf $VENV_HOME/$1
}

_venvdirs()
{
    local cur="$2"
    COMPREPLY=( $(cd $HOME/venvs && compgen -d -- "${cur}" ) );
}
complete -F _venvdirs workon rmvenv 
Code language: Bash (bash)

These lines should be added to your .bashrc in Linux, or your .bash_profile for Macs. I haven’t tried them with zsh yet, so no guarantees there. Let’s go over what these lines do.

Line #1 defines the directory where the virtualenvs will be stored. You can store them anywhere; it doesn’t make any difference.

Lines #3–10 define the workon function, which activates the specified virtualenv, or lists all virtualenvs if none is specified. Lines 12–17 define the mkvenv function, which creates a new venv, and lines 19–22 define rmvenv, which deletes virtualenvs when you no longer need them.

I’d like to point out 2 lines in mkvenv that you can customize. Line #15 updates the installed versions of pip, setuptools, and wheel. If for some reason you don’t want the latest versions of these, edit or remove that line.

Line #16 is more interesting: nearly every virtualenv I create needs these packages installed. Rather than install them one-by-one, I add them when I create the virtualenv. If you have different packages you always want available, edit this line.

Finally, lines #24–29 are of utmost importance to someone lazy like myself: they provide auto-completion for the other commands. I had to learn about bash completion to get that working, but it turned out to be much easier than I had imagined.

Here is a gif showing it in action:

Try it out! Let me know if you find this useful, or if you have suggestions for improvement.

Day 45: Composing Music and Code

I am not a musician. Yeah, I was a vocalist in a couple of bands in high school and college, but all I really did is copy the original song. The notion of coming up with a new approach to a song was way beyond my imagination.

I am amazed by people who can write their own original songs. How can you create a melody from… nothing? It really is beyond my ability to comprehend. Lyrics seem straightforward enough, but the music?

A few years ago at a conference I was hanging around in the evening with some friends, several of whom were musicians that had brought their guitars. One of them played a song, and I asked him where the song was from. He replied that it was one of his own. “Wait: you wrote that yourself?” That surprised the hell out of me!

Now that my curiosity was piqued, I couldn’t help but ask him how he does it. He replied that sometimes he has some little bits of music in his head, just a couple of notes. When he’s practicing his guitar, he’d just sit around playing some small riffs; just a few notes here and there. He’ll play one, then change it up a little bit, and if it sounds better, he’ll start working with that. Over time he’ll have a few more such bits floating around in his head, and every now and then he’ll play a few that just start to flow together. At that moment he can hear the overall melody, and he has to write it down, or record it – anything to capture it before it fades. It doesn’t matter what time of day it is, or what else is happening; he can’t do anything else until he has it down on paper/tape/whatever.

Now I understood: his process for composing music was very similar to what I experience when coming up with a coding design to a given problem. I’ll play around in my mind with a few approaches I’ve used in the past, changing them around as needed to see if that would work. At some point, though, the pieces seem to align themselves into the solution I need, and I can see it clearly in my mind. At that point, I have to write down what I’m seeing before I lose it. I can remember times when I missed appointments because I was heads-down writing such a solution.

The act of creation is remarkably similar for both cases: playing around with ideas/riffs, and once the solution “appears”, a furious effort to record it. And it’s that “playing around” part that is really hard to explain to others: to them, it just looks like you’re screwing around instead of working.

I do my best thinking when I’m walking, not when I’m planted in front of a keyboard. If you’re ever stuck trying to come up with a solution, try changing what you’re doing. Go for a walk, work on your garden, vacuum the floor – anything to move away from the computer and get your body moving. That act alone helps free the brain to make connections that it might otherwise never make.

Day 35: Chopped Candidate

Have you ever watched the TV show Chopped? If you haven’t, it’s a competition among 4 chefs. There are 3 rounds, and after each round, one of them is chopped (eliminated), until one remains. The winner gets a cash prize. This would seem like a good way to determine who is the best of the group, right?

The problem is how the competition is run: each round the chefs are given a basket of “mystery” ingredients that they can’t see until the round begins. And more often than not, the basket contains, shall we say, “odd” combinations. One such basket contained blood orange syrup, the African spice blend ras el hanout, hot cross buns, and lamb testicles. The chefs can add other staple ingredients, but those four flavors have to be featured prominently in the result.

And if that isn’t difficult enough, there is a time limit that is always ridiculously short. The chefs had 20 minutes to create an appetizer from the basket I described above: 20 minutes to create a recipe, determine what other ingredients to add, prepare and cook the food, and then plate it for a beautiful presentation.

I must confess that I find the show very entertaining, and have watched countless episodes. And I’m not alone: the show has been running for 44 seasons over the past 11 years. But let me ask you: if you were opening a restaurant, would this be the way you would select your head chef? I would hope not! Any restaurant that would spring surprises on their chefs and expect them to deliver first-rate food in impossibly short time limits wouldn’t last very long.

Which brings me to the point of all this: if you are interviewing for a programmer, do your interviews actually determine how well they would be able to work in your team? How positive their contribution will be?

Making a candidate live code a solution to a problem they’ve never seen before in a short period of time with people watching their every keystroke is the software development equivalent to being on Chopped. I certainly hope that your work environment isn’t anything like that. So why would you think that a live coding session in an interview tells you anything about their potential?

What artificial scenarios like Chopped or live coding interviews do is test a candidate’s ability to handle stress. Personally, I’ve never had a problem with live coding, but then again I’ve never had test anxiety in school, either. I’ve seen many talented developers choke under those circumstances, but that doesn’t mean that I wouldn’t want to have them on my team.

What does it say about your company as a place to work if the bar they have to clear is how well they can handle high levels of stress?

When I first started interviewing candidates when I was at Rackspace, the standard was to have one interviewer do a live coding challenge, and another ask one of those bizarre, abstract brainteasers (“Walk us through your thoughts…”). Once again, these practices just show how nervous someone is in what is already an inherently stressful situation. That link includes a juicy quote:

These types of questions are likely to frustrate some interviewees so watch out for those who aren’t willing to play the game. It’s an interview after all and you make the rules.

Mark Wilkinson, head of recruitment, Coburg Banks

It’s all a game to him, and if asking questions with no right answers eliminates potentially good candidates, tough. It sounds like he is more interested in seeing who can tolerate being bullied than finding the best people for his company.

After sitting through some of these types of interviews at Rackspace, I campaigned internally to change these practices, because I saw some intelligent and capable candidates get flustered and end up looking dumb. I found that there are better ways to determine if someone is a good addition to your team. Perhaps I’ll elaborate more about these in a future post…