Hubot Inspiration

At puppetconf I had the pleasure of attending Phil Zimmerman’s awesome Killer R10K Workflow session.

While R10K, and his voodoo were awesome, the use of Hubot as kind of the center spoke for the communications of the workflow had me feeling a little inspired.

I am unable, however to use anything as-is in the workflow because we can’t use external services (github, hipchat, etc.).

We have set up a workgroup XMPP host (running Openfire). And since we both develop Ops infrastructure and run a R&D lab, we set up a couple chatrooms for SysAdmin topics, and Development topics.

So now using Hubot and hubot-xmpp, we have our own friendly chatbot, Virgil. It’s a play on Dante’s Divine Comedy: Virgil being the guide through heaven and hell. Also the historical poet Virgil himself, and the Southern use of the name. I think it captures what I call the “redneck scholar” personality well… The guy who well read, and can operate a tractor. Theory and practice in a single person. In a way, what the whole devops movement strives for.

Currently Virgil is tied into our git repositories with a post-receive hook. He has the typical hubot, and random quote functionality.

In particular I saw one I liked that was made to cheer people up who mentioned failure. But it just spit out a single quote. So I wanted a little variety. I would have just used the msg.random piece, but one thing that bugged me was the single string. So I made him pull a random array from an array so I could store quotes and attribution reasonably, and deal with them as distinct, but related pieces of data.

Nothing crazy. Just pick a random number between 0 and n (and make sure it’s an int) and grab that element from the outer array. This lets me manage quotes and attribution in a more flexible way and give Virgil some personality.

I’m now polishing off putting him into the escalation chain on our monitoring system that uses a convoluted email->procmail->python script->json-via-http->hubot. It sounds more complicated than it is, as basically it’s a script feeding the hubot script. Procmail also buys me the ability to take all those annoying things that have to use email for notification and reduce them to just being chat notifications, where more of them belong.

The key being that I snarf the data as data, and use the bot script to present with personality.

Next up will be tying him into our API in front of Foreman and Puppet to magically provision machines for us. I can’t wait to ask him for 3 VMs with Tomcat.

Introducing GardenBuddy

I’ve put together a little project for the Raspberry Pi to monitor environmental conditions for my garden. For now I’ m calling it garden buddy, and the code is available here:

Using a few sensors (light, soil temperature, moisture) and available weather data from NOAA, I can monitor my garden and look for trending data.

The software is all in python and includes the little daemon for stuffing the data into RRD files, and a couple CGI’s (kickin’ it old-skool) for viewing the graphs.

I’ve made it so the sensor and graph configuration is all done in an INI formatted config file, so you don’t have to necessarily know Python to use it.

It’s like performance monitoring, but for your tomatoes.

I dream of one day intelligently managing a watering system with it, but for now semi-pretty graphs will do.

Some TODO items I have are:

  • interfacing with more sensor types
  • making it prettier
  • Unrolling the requirement for a “real” webserver
  • A rainbarrel/soakerhose/valve management piece

Battery Replacement on the Nexus 4

Just wanted to note I found this article, which describes the battery replacement process well (the Youtube clip helps immensely). One piece of errata, however: you need a 00 Philips not a 0 as described for removing the battery connection itself. My ebay-bought battery seems to be working great. Hopefully I can get a bit more life out of the thing before I buy my Nexus Eleventy-two.


I’ve started on cloning ksb’s excellent xapply in python for two reasons:

  1. It’s an interesting exercise
  2. There are many times I don’t have msrc or want to bring msrc with me for a one-off usage, where a python script would be perfect

Currently it just requires Python 2.7+ (I really love argparse).

I currently support:

  • Parallel jobs!
  • Input from command arguments
  • Input from arbitrarily many files
  • Fancy dicer syntax (eg %[2,4])

So far it does most of what I need, but it is nowhere near feature parity yet. I was considering going with different command line arguments, but I decided to stay as close to the original as I can (although I cannot guarantee argparse will behave the same as ksb’s getopts behavior).

Feel free to contribute if you’re bored. Feel free to use if it helps. I’m releasing it under the standard 3-clause BSD License.

retro-cool: tcpmux

TCPMUX is a wonderful (and potentially terrible) protocol for one-off network services. It’s described in RFC1078.

Basically TCPMUX is a service itself (usually built into or run from inetd) that listens on port 1. To access a particular service it provides you give it the name of the service plus a CRLF. ‘help’ is a special service that lists all available services.

So for example I wanted a way for one host to poll a list of ports installed on another host. I have two lines in my /etc/inetd.conf file:

I have a dumb little script that generates the output:

Then I can quickly get this data from everywhere like so:

Now most implementations are a little forgiving on the newline sequence but YMMV.

xinetd doesn’t provide ‘help’ typically and has been known to just segfault sometimes, although I think this had been fixed in recent versions.

But this is a great alternative to setting up a user with SSH keys or doing something more complicated for passing data that should be allowed to go across the wire plain-text unauthenticated. nc + tcpmux is an incredibly handy (and potentially powerful) combination.

Now there are some obvious limitations here:

  • Be very, very careful with user input. Acting on user input in something like a shell script is fraught with danger.
  • Passing tcpmux through a perimeter firewall is probably not the best plan unless you have control of everything. tcpmux can be abused to provide ANY network service.

systemd enters the real world

Despite my frustrations with systemd and the attitudes surrounding it, it has now been accepted by both Debian and Ubuntu in addition to Fedora. And this is a great thing.

Read what is going through the community now. Things like this blog post are floating around now. The conversation is happening. The concerns are out there. And now they have to be addressed. Now the concerns being addressed are not the concerns of a small group but the concerns of a greater community. It’s the old if “you can’t beat them, join them”…. and change them.

Debian alone introduces a large stabilizing force. Up until now, systemd has been controlled by a community of like-minded people. That’s normal. But when it becomes Linux infrastructure the variety of minds contributing and consuming increases. And heterogeneity is a wonderful thing. I just may hold off using it myself, until some of this takes shape. :)

FreeBSD’s pkgng keeps being awesome

So there is plenty of work left to do, but I keep finding new ways to love FreeBSD’s pkgng.

Now that the official repo is up and running I use it rather than building everything from ports. I only build two packages now because I use options that aren’t selected by default: php5 (for mod_php) and mutt (mostly for IMAP header caching).

So I build them from ports, and use the ‘pkg lock’ command to keep pkg from updating them incorrectly.

So I couldn’t remember if I had anything else locked and looked into the ‘pkg query’ command. It takes an amazing set of format string options which allowed me to make a quick one-liner:

And when I want to check for vulnerabilities I can just ‘pkg audit’ which is just amazing. Imagine being able to just run this across a server farm without needing to buy or build something. Or like I imagine many do, just version scanning network services.

They really didn’t just reinvent the wheel here. They really have put in a lot of effort and learning from others (like yum and apt) into creating a best-of-breed package management tool that actually integrates with ports very well.

Systemd is not Unix

OK. So now I feel like this is getting past ridiculous, but people really seem to be buying what is selling here. But one of the basic Unix principles is to create small general purpose tools that can be recombined and systemd takes on the very un-Unixy approach of being all things to everyone.

So far the list of basic Unix components that systemd replaces or forces removal of:

  1. init (this one I get)
  2. cron
  3. at
  4. batch
  5. nice
  6. an MTA
  7. syslog
  8. fstab
  9. the LSB filesystem heirarchy

At this point I think it’s fair to really call a distro with systemd GNU/systemd/Linux.

Read on for explanation.
Continue reading Systemd is not Unix

Making urxvt a little more modern

I was working on trying to setup a slim desktop configuration for a VM and my Raspberry Pi and wanted to use a simple terminal instead of Konsole or gnome-terminal.

But I like black backgrounds, white text and a blue color that doesn’t make my eyes bleed. I wandered into this excellent post with a Tango-ized color palette for urxvt, and now my .Xdefaults file has:

The ‘^H’ is a literal control character so it won’t copy/paste (type Control-v Control-h in insert mode in vi). But that just keeps my backspace consistent everywhere.
Continue reading Making urxvt a little more modern