Day 31: Using etcd As a Mediator

etcd is a database originally developed by CoreOS, and is most famously used as the database at the heart of Kubernetes. It is a distributed key-value store, which in itself is not all that remarkable. The thing about etcd that makes it so attractive is the ability to watch a key for changes.

Other key/value stores, such as Redis, have implemented a similar feature, and may work just as well for you. I’ve been using etcd for years, and it’s worked well for me, so I’ve never had a reason to try these other tools.

For most data stores, the only way to find if a particular value has changed is to poll. You issue a query for that value on a regular basis, and compare it to the last value returned to see if it has changed. This is terribly inefficient, especially with values that don’t change often. It’s also inexact with respect time, because your system’s reaction to a changed value depends on the interval between polls. Longer intervals, while less chatty, mean that more time will elapse between when the value changes, and when your application responds to that change.

Enter etcd. Instead of polling an etcd server for changes, you can watch for changes. This is essentially a pubsub system that requires almost no configuration to work. When a key is written to etcd, if there are any watchers for that key, a message is sent to them with the new value.

This is kind of dry in theory, so let’s look at a real-world application using this system: my photoviewer and photoserver applications. These applications allow me to display photographs on monitors that can be anywhere with an internet connection, and control each of those displays from a central server. They represent the ultimate convergence of my work as an artist and my love of programming.

Each display consists of a monitor (actually a TV, but all I want is an HDMI input) and a Raspberry Pi that runs the photoviewer application. Each display has a unique ID to identify it, and when a display starts up, it registers itself with the server. The server contains the settings for that display, such as the list of photos to display, and how often to change the displayed photo.

Photoviewer running in my kitchen

I have one such display in the kitchen of my home, and like to change the photos displayed on it from time to time. To do that, I go into my photoserver app and change the album for that display. Almost instantly the image on the display changes. How did that happen? The server is a virtual machine running in the Digital Ocean cloud, not local to the kitchen display.

The reason this works is that I’m also running an etcd server on another cloud instance. When I change any setting for a display, the photoserver app writes a new value for that display’s key. The key consists of the unique ID of the display plus the type of value being changed. For example, if I change the photos I want displayed for a display with the ID of 65febdde-3e8a-4c76-ab8f-d8a653e466c7, the server would write a value of a list of the names of those images to the key /551a441f-8aba-44b5-b70b-349af0be5b67:images.

That application uses the etcd3 library to watch my etcd server for changes to any key beginning with /<unique ID>. The watch() method is called with a callback method, and when a new key is written beginning with that display’s prefix, the value is sent to the callback.

The callback method sees that the full key ends with :images, so it passes the value (the list of image names) to the photo display method, which then retrieves the image and displays it. This happens in real time, without any polling of the server needed.

The original version of these apps used the traditional polling method, which seemed wasteful, considering that it was typically weeks between any changes being made. Switching to an etcd watch makes much more sense from a design perspective, and it greatly simplified the code.

Look for cases in your applications where a response is needed to a change in data. Using etcd as a mediator might be a good approach.

Day 30: It Ain’t Over

It’s tiring. I get it. Dealing with the pandemic is exhausting: wondering if any time you touch something that you’re picking up the virus, or if that person is walking by a little to close, or if it’s worth the risk to pick up some fresh vegetables from the store. It’s been months!

And it’s going to be many months more, thanks to the completely inept handing of things by the Republicans in this country. I single them out because they took common-sense safety precautions and politicized them. How is wearing a mask to limit the spreading of the virus a political question?

But thanks to Governor Abbott here in Texas, the number of new cases of COVID-19 infection is climbing rapidly. Here in San Antonio we had the biggest number of new cases (311) on Tuesday, and the most significant increase in the number of people on ventilators. Houston has the world’s largest medical center, but is so overwhelmed by COVID-19 cases that its Children’s Hospital has started admitting adult patients. Back in April Abbott issued an order that forbids localities from enforcing rules requiring face masks, in an effort “open Texas back up”. There was no evidence that the risk of contagion was gone; in fact, everything said the exact opposite. But in keeping with the Republican strategy of pretending it’s gone, he kept lifting the restrictions that had kept the infection rates low. Now the infection rates are spiking, and they’re acting surprised. The same pattern is happening in Florida, where Governor DeSantis is likewise pretending that things are normal.

And this isn’t a case of “how could we have known?”. Every epidemiologist was saying that re-opening was the wrong thing to do. Every medical authority was saying that we needed more testing to understand the extent of the disease, and that we needed stricter precautions: masks that cover the nose and mouth, full social distancing, and banning public gatherings. We have the evidence from the last global pandemic from 100 years ago: relaxing the precautions led to a second wave of infections. This may be the first time any of us have experienced a pandemic, but it isn’t the first time it’s happened, so we have historical data.

But instead the Republicans, following Trump’s lead, ignored that advice in the hope of giving a boost to the economy.

Priorities!

How is this anything but criminal? If I kill someone so that I can get some money, that’s murder. But if I kill someone to boost the economy (and, by extension, help myself get re-elected), that’s just politics?

Nobody could have prevented this pandemic. I’m certainly not blaming Trump, Abbott, DeSantis, or anyone else for it. But it could have been handled in a way to minimize the suffering and death, as we’ve seen in other countries that treat this like the medical issue it is, rather than political football.

Day 28: The Impossible Butterfly

I wrote this last Saturday evening right after it happened, and posted it to Twitter. I had plans for writing something different today, but our cat had other plans, and became ill. So I’ve just spent most of the day in the “waiting room” (sitting in the car until called), and then having the cat treated. It’s odd how doing almost nothing all day long can exhaust you!

So rather than write something brand-new, I thought I’d re-post my incredible experience. If you’ve followed this blog, you’d know that I raise caterpillars of the Easter Black Swallowtail butterfly, from here, here, here, and here.


I’m at a complete loss for words. I just experienced something I didn’t think was possible.

Earlier today a butterfly emerged from one of the pupas in my nursery. He had a wing deformity (yes, it was a male), and when he attempted to fly, he couldn’t get more than a few feet before returning to the ground

Resigned that he wasn’t going to survive, I placed him on a flowering plant in the yard, and assumed that nature would take its course, and the butterfly would soon die.

I kept returning every hour or so throughout the evening to check on him, and he hadn’t moved very far on the plant. As the evening grew late, I wanted to say my goodbyes. I know it seems silly, but raising these marvelous creatures from egg to adult creates an attachment. I placed my fingers in front of him, and he instinctively climbed up onto my hand. I spoke to him, telling him how sorry I was that his wing was deformed, so he wasn’t going to be able to fly and live his life.

After a minute or so, he started flapping his wings, and flew for a few feet, as he had done before, but quickly landed on the ground. I again placed my hand in front of him, and again he climbed aboard. I talked to him a little bit more, and again he started to flap his wings. This time, though, he flew!

I watched him fly across the yard, up over the garage, and then over the back fence. He quickly disappeared into the distance, and I couldn’t see him any longer.

I don’t know what kind of life he will have, but to be able to fly away on his own is something I truly never thought he would be capable of. I am still feeling that warm giddiness at witnessing what I had thought was impossible.

Day 27: Disc Golf Numbers, Part 2

A few days ago we covered the first two of the four mysterious numbers that are on each disc: Speed and Glide. Today we’ll cover the other two: Turn and Fade.

Turn is the ability of a disc to turn toward the direction of its rotation during the initial high-speed part of its flight. That’s a mouthful! But what does it mean?

Let’s assume that you throw the disc with your right arm, and use the common backhand delivery. When you throw it, the disc will be spinning clockwise when viewed from above, and the front edge is spinning toward your right side. That’s why we say that the “direction of its rotation” is to your right.

Turn values range from 1 to -5, where 1 is the most resistant to turn, and a -5 will turn the most. Having some Turn is good for beginners, as it tends to offset the Fade at the end of the flight, making a more or less straight throw. One problem, though, is that high Turn discs are not as stable when throwing into the wind, so in general you should avoid high Turn discs in that situation.

Let’s compare the flights of two discs that have nearly the same numbers, but differ significantly in their Turn:

The disc on the left, the Thunderbird, has a Turn of 0. Note how its initial flight is fairly straight, while the disc on the right, the Roadrunner, bends to the right for the first half of the flight. That is consistent with a Turn value of -4 for the Roadrunner. I own a Roadrunner, and I can testify that it is very hard to control when throwing into the wind.

The fourth and final number to understand is Fade. Fade represents the tendency of the disc to turn away from its direction of rotation at the end of its flight as it slows down. In other words, Turn and Fade move the disc in opposite directions.

Fade values range from 0 to 5, with 0 finishing the straightest, and 5 bending hard at the end of the flight. It might seem odd to want your disc to fade, but it can be helpful for distance control, where you want to ensure that the disc doesn’t keep sailing past your target. It’s also very useful for when you need to shape your shot around an obstacle such as a tree, as you can throw to the side of the tree, and let the disc’s natural fade bring it back towards the target.

We’ve covered what the numbers on your discs mean. So, as a beginner, how do you use that information? First and most importantly, avoid buying high Speed discs. Until you develop your technique and strength, you simply won’t be able to throw them reliably. Start with a lower-speed disc, such as a fairway driver.

Next: practice! Find an open space where you can practice without having your mistakes (you will make plenty!) cause any problems. Not too far from my house is a park that has a baseball field without any outfield fence:

I’ve circled two trees that I measured to be 240 ft. apart. This is perfect, as I can rarely throw more than 250 ft. I take my discs, noting the numbers on them, and imagine how I need to throw them to get close to the target tree. I have to also factor in the wind, if any. When I make a good throw, I then observe how the disc’s flight goes, and whether it matches what I was expecting. I gather my discs, and now throw from the other tree back towards the first. This way I can practice both throwing with headwinds and tailwinds, or sidewinds coming from each side. I repeat this several times with a set of 3 discs, and then when I have a good feel for them, try with a different set of discs. It’s really helped me understand what disc works best in different conditions. Now I just need to improve my consistency, which means: more practice!

Day 26: June2020.org

I was planning on finishing the post about disc golf today, but I started watching the live stream of The Poor People’s Campaign: A National Call for Moral Revival, and felt as though there were more important things to write about.

If you haven’t heard of this movement, please take a few minutes to go to june2020.org, and read up on it. And if at all possible, please donate what you can to help.

I watched the live stream for over an hour, and am more convinced than ever that we cannot improve our country by looking at one issue and focusing on that. Instead, it is an over-arching, systemic issue that has not arisen by accident. It is by conscious design. It needs to be addressed as a system: things such as poverty, mass incarceration, abandoned veterans, gerrymandering, unending war, etc., are not individual issues, but all symptoms of the same evil design of the society.

I chose the word “evil” to describe it, as I can think of nothing that better exemplifies what evil means. Imagine not caring about what happens to some people, as long as you profit from it. Imagine not caring about millions of people living in poverty so you can have more. Imagine not caring about people escaping violence and persecution and denying them help just so you can keep your race’s percentage of the population from diminishing even further. Imagine not caring about democracy so that you can prevent people from voting who might not want to sustain the status quo. Anyone as cold an uncaring as that, who would willfully cause millions upon millions to suffer, is the epitome of evil.

Please go to june2020.org and help in any way you can. Then continue to speak out against this unjust society, and vote for people who will at least keep it from getting worse until we can nurture a new generation of leaders who can lead us to where we need to go.