The problem: You have a puppet function you find useful. At this point custom functions are really easy to create and distribute via modules so there’s really no reason not to be using them. However, you want to use the result of a function in your templates.

A real-world example of this is sharing authentication information. (For example, crypted password hashes. Plain text, as always, would be evil no matter where it was living.)
You really have 3 choices:

• Coding it right into your puppet code and checking that into your version control system
• Not distributing that information with puppet
• Using puppet to distribute the information, but store it outside of your puppet modules somehow.

We chose the third way, and store this kind of information in YAML files outside the puppet modules tree.
Anyway. So we want to access this info inside a template.

Assume you have a function called ‘get_extdata’.
You normally call from your ‘.pp’ manifests with something like

$data = get_extdata('mymodule', 'path:to:data')

You can make a call like that from your template by doing this wonderful fragment of magic-wand-waving to get this function accessible:

In mytemplate.erb:

<% Puppet::Parser::Functions.autoloader.loadall %>
Now you can use the 'scope' variable to get at your function.
This is the equivalent of the call get_extdata('mymodule',  'path:to:data'):
<%= scope.function_get_extdata('mymodule',  'path:to:data') %>

This works like a charm, and means you don’t have to create a bunch of variables that you don’t really need before you load the template.

So if you were creating an .htpasswd style file with puppet, you could do

In your init.pp

$users = ['steve', 'paul', 'stu']

htpasswd.erb:

<% Puppet::Parser::Functions.autoloader.loadall %>
<% users.each do |user| -%>
<%= user %>:<%= scope.function_get_extdata('authmodule',  user) %>
<% end -%>

Since then, the world has changed. I’ve upgraded to Snow Leopard, and Quicksilver has essentially stopped working out for me. (I’ve been trying to keep it up to date, and sometimes it kind of works, but my confidence levels are really low.) On the other hand, I’m loving me some Google Quick Search Box (a new project from the original creator of Quicksilver). The only feature that I still wanted Quicksilver for was triggers. And until today, I was still running it for trigger support to implement my Inbox Zero trick.

And then triggers broke when I upgraded it. And I’m out.

So here’s how to get Inbox Zero magic in Mail.app using only tools native to Snow Leopard.

First, launch Automator and create a new Service workflow.

Select “Utilities” and then drag in “Run Applescript” to the window on the right.

Fill in your applescript, replacing only the center “comment” block — leave the other autogenerated lines there.

The core code again, (tweak to fit your accounts and archive mailbox names)

tell application "Mail"
	set theSelectedMessages to selection
	set myAccount to "zimbra"
	set myMailbox to "Archive"
	repeat with theMessage in theSelectedMessages
		move theMessage to mailbox myMailbox of account myAccount
	end repeat
end tell

It should look something like this:

And that’s it! Save it, and give it a name you’ll remember. (I chose “Archive Selected Mail.”)

Now it’s actually in Mail’s Menu (under ‘Services’) and we can use the built in OSX hotkey support to launch it.

Go to the System Preferences Menu, click “Keyboard Shortcuts”, and add one for the menu item you created. (It needs to be the same exact name you saved your service as.)

Badda bing. Feature complete, and I can (sadly) retire Quicksilver forever. This technique should be extensible to all kinds of other cases when you want to add new features to applications, for access either through the menus or with hot keys.

Subscribe over there if you’re interested in that kind of thing.

Here are some screenshots from my strace-based ‘magic profiler’ (which I still promise to talk more about later.)
It lets you attach to a process which is about to render your website, and then trace all the interactions it has with the underlying system while it renders it.

That includes network traffic, file server traffic, DNS resolver lookups — everything your app does to communicate to the outside world. The timing is always going to look slower than normal (at least a bit) as the tracing does impose some overhead, but it’s relatively pretty accurate.

Here’s what I saw right after telling WP Widget Cache to clear the caches.

And here is what I saw on the second hit:

I have widgets to load Flickr, (the entry with ‘flickr’ in it) delicious bookmarks, (the yahoo one) and Twitter. (The naked IP. Apparently they don’t like PTR records.) Those three external calls dominated my total time to generate the page. Now, it’s down to a very quick MySQL lookup and a chatty conversation with a fileserver.

One feature I especially appreciate is the ability to tune cache times per widget. If I want new Flickr photos to be available hourly, sure. If my tweets need to be there within 5 minutes, not a problem. And because the widgets are there every time anyone uses any part of the site (even the rarer ones) it plays really well with the other WP Cache Plugins.

The issue was that the sensor was fine, except when it was rapidly transitioning from one temperature to another. For example, if the ambient temperature was 74, and suddenly a can of soda fresh from the fridge was placed on it, you’d get several “transitional readings” (70, 68, 52, 48, …) until the temperature finally stabilized.

We talked about various ways of detecting this case, but the most straightforward one seemed to be Standard Deviation, which Wikipedia explains much more clearly than I could.

I googled around for sample code or a library and didn’t find any. So even if my Google-fu is weak and there are great resources out there, now there’s another one.

The basic idea is that we take 10 samples quickly (20 ms apart), figure out the Standard Deviation, and if that’s close enough to zero, we can call this a “stable temperature.”

The full code is available from my git repository, but here’s the core of it:

  // Gather sample data

  float sampleSum = 0;
  for(int i = 0; i < SAMPLES; i++) {
    s_val[i] = analogRead(TS);
    sampleSum += s_val[i];
    delay(20); // set this to whatever you want
  }
  float meanSample = sampleSum/float(SAMPLES);

  // HOW TO FIND STANDARD DEVIATION
  // STEP 1, FIND THE MEAN. (We Just did.)

  // STEP 2, sum the squares of the differences from the mean

  float sqDevSum = 0.0;

  for(int i = 0; i < SAMPLES; i++) {
    // pow(x, 2) is x squared.
    sqDevSum += pow((meanSample - float(s_val[i])), 2);
  }

  // STEP 3, FIND THE MEAN OF THAT
  // STEP 4, TAKE THE SQUARE ROOT OF THAT

  float stDev = sqrt(sqDevSum/float(SAMPLES));

  // TADA, STANDARD DEVIATION.
  // this is in units of sensor ticks (0-1023)

So, hopefully the comments are self-explanatory (when combined with Wikipedia, if you’ve never seen Standard Deviation before.)

At the end of this block you have 2 useful variables defined: ‘meanSample’, which is the mean (average) value of all the samples you polled, and ’stDev’ — the standard deviation amongst all the samples.

This allows you to do things like

if(stDev < TOLERANCE) {
    // reading is stable enough
    fireMissleAt(meanSample);
}

An important note about the units — the Arduino analogRead values are 12 bit, meaning they range from 0-1023.
In general, those numbers will “mean something” to you. Perhaps you can convert them to a temperature as we were doing, or a direction froma compass, or a position on a pot. I chose to do the math with the numbers in as raw a form as possible. This means that if you have a sensorToTemp() function, you can call that on meanSample, but you’d also want to call that on stDev as well. Make sure to
convert both values into meaningful values for your application. If you just want to know if the measurement is stable, then perhaps just knowing that the (12 bit) version of stDev is over your comfort level is enough.

Along with Standard Deviation came the need to print out some floating point numbers, so I also included some sample code to do that.
It’s hard coded to send things to the serial port, but could be easily tweaked to fabricate a return string or print somewhere else.

// This is a utility function for printing out floating point values
// Fixed at %0.2f form. (XX.YY, 2 digits after whatever decimal part there is.)
void printFloat(float var) {

  int int_part = int(var);
  int float_part = 100*var - (100*int_part);

  Serial.print(int_part, DEC);
  Serial.print(".");
  if(float_part < 10) {
    Serial.print("0");
  }
  Serial.print(float_part, DEC);
}

Hopefully this helps someone else out. It was fun to write and I’m sure I’ll have a use for it someday!

Photo Credit (no that’s not me!)

So, we still have shared hosting, in it’s various forms. I like the (mt) Grid Server, but I will admit to being biased (a lot.)

So with shared hosting, you don’t have root. And managing software you need (to get useful web applications up and running) can be a pain — a lot of the documentation is biased towards the idea that you have a full server, be it virtual or physical.

If you don’t have root on your server, and you’re using perl (for any of the amazing Modern Perl tools and stacks out there,) I recommend checking out the awesome local::lib.

Once you follow their ‘bootstrapping config’ and set up your bashrc, it makes things feel like you’re on your own server. Just ‘cpan -i WordPress::Post’ or ‘./Build install’, and it just works!

So a full install using local::lib would look like:
Download and unpack local::lib:
(Warning, update the link to the latest version, this is the current release:)

mkdir tmp
 cd tmp
wget http://search.cpan.org/CPAN/authors/id/A/AP/APEIRON/local-lib-1.004008.tar.gz
tar -zxvf local-lib-1.004008.tar.gz

cd into the local::lib directory, and then run:

perl Makefile.PL --bootstrap
make test && make install
echo 'eval $(perl -I$HOME/perl5/lib/perl5 -Mlocal::lib)' >>~/.bash_profile
. ~/.bash_profile

And you’re done! Every time you ssh in, your environment will be set up to use your whole local tree. So even if you want perl’s New Fancy Hotness, all it takes is:

cpan -i Moose

One caveat if you’re using local::lib and trying to run this tool from cron, is that you’ll probably want to use a wrapper script to correctly set up the environment. (Sometimes you’ll need to set up the environment for your web apps, too — that can be done via .htaccess.)

Here’s an example wrapper script, set up to work on my MediaTemple (gs) account and work with my local::lib setup:

#!/bin/bash

#set up environment
HOME=/home/12345/users/.home
eval $(perl -I$HOME/perl5/lib/perl5 -Mlocal::lib)

# run the script
$HOME/perl5/bin/mycron.pl

To make things work with your webapp, you need some ‘SetEnv’ commands in the .htaccess. You can do this by just loading the local::lib module and checking the output:

%> perl -I$HOME/perl5/lib/perl5 -Mlocal::lib
export MODULEBUILDRC="/home/12345/users/.home/perl5/.modulebuildrc"
export PERL_MM_OPT="INSTALL_BASE=/home/12345/users/.home/perl5"
export PERL5LIB="/home/68601/users/.home/perl5/lib/perl5:/home/12345/users/.home/perl5/lib/perl5/i386-linux-thread-multi:$PERL5LIB"
export PATH="/home/12345/users/.home/perl5/bin:$PATH"

Each one of those EXPORT statements needs to become a SetEnv statement.

# so this:
#export MODULEBUILDRC="/home/12345/users/.home/perl5/.modulebuildrc"
# becomes:
SetEnv MODULEBUILDRC "/home/12345/users/.home/perl5/.modulebuildrc"

Repeat that for each one of the variable ‘export’ lines, and you should be good to go! Affordable, scalable, low management overhead shared hosting and an easy way to get modern perl.

Recently, I passed around this video from Sweet Maria’s, which is one of the bean providers the most enthusiastic of said co-workers shops with.

The Sweet Maria’s gent claimed as an aside in that video (which really raises the bar on coffee geekery, truly impressive) that one should always weigh one’s water, as the volume changes when it is heated.

I get that the volume changes, but it seemed to me that it can’t change that much. I realized this weekend that I had a way to find out exactly how: WolframAlpha. I just had to ask it Volume of 1kg of water at 90 degrees Celsius and, lo and behold, it knows!

Curious, I punched in a few more numbers, and made the following chart:

So the water expands a pretty impressive 3.5% on it’s way from “very cold” to “coffee temp”. Assuming you start with water which is not totally chilled, however, the expansion starts to become less of an issue.

So, does it matter to most of us — no. We probably don’t measure our water within 3.5% accuracy on the best of days, anyway. (And if you have a hot water machine, like we do, which happens to dispense perfect temp water anyway, you’re measuring pre-expanded.)

But if you’re making any kind of serious volume, 3.5% might actually start to have an impact. It’s at least a sip out of a cup.

I’d say we have to call this geek “Plausible.” But I’m still not weighing my water.

Aside: if anyone knows how to use WolframAlpha to actually make a chart like this, let me know. It seems like it should be doable, I just couldn’t finesse my query in such a way to make it happen. OmniGraphSketcher to the rescue.

After using WordPress from both the browser and the iPhone client, I just wasn’t that happy using it for our pictures. On the other hand, we both love (and already use, and have Pro accounts on) Flickr.

Having the photos “live” on Flickr means some handy things.

1. They let you get an RSS feed of a tag. (We use ‘carterbarratt.’) This means we don’t even have to use an API key, which is convenient.
2. They always have an image available that’s sized to 500 on the longest side. This turns out to be a perfect image size (either height or width) for a lot of WordPress themes.
3. They handle video and make the thumbnails look exactly like a “non-video image” in the RSS feed.
4. Lots of things can upload to Flickr — we use mostly the desktop uploader (after exporting from Lightroom) and the new Flickr native client on our iPhones.
5. The exact same stuff (title, description) that I’d want on a photo blog is available when I upload to Flickr.

So, FlickrPress was born. It’s a fairly simple perl script, intended to be run as a cron job, which uses WordPress::Post to create new blog posts for every flickr photo it finds with a certain tag, from certain users.

Check out the FlickrPress page for downloads and installation instructions.

In the old days, I probably would have just used perl backticks to run the command and capture the output. I’d “test” on a machine that had those tools installed, and call it a day.

my $output = `some system command`;
# process the output here

However, that’s pretty evil, and it’s essentially untestable.

Here’s a pattern that works pretty well.

The IPC::Run module includes a ‘run’ command, which in the simple form takes 4 references: To a list of a command and it’s arguments, and then scalar references for the contents of Standard In, Standard Out, and Standard Error.

If all you care about is capturing STDOUT, you might call it like:

my @raidstate = ("hpacucli", "'controller slot=0 physicaldrive all show'")
my $raid_output;
IPC::Run::run(\@raidstate, \undef, \$raid_output, \undef);

Ok, but how does that help us test?
Enter the venerable Test::MockModule.

All I need to do is override the run() function for the module in my test!

use Test::MockModule;

my $module = new Test::MockModule('IPC::Run');
$module->mock('run', \&mock_ipc_run);

Ok, so how do I get the right content to get in the $raid_output?

It depends on how exactly you’re calling run: I hand-tuned this for the specific use case I’m doing right now, where I just care about capturing output.

What we do is store a list of command lines in the data block, followed by their output. When someone calls ‘run()’ with one of those command lines, we send that output back to them.

sub mock_ipc_run {
    my($cmd, $stdin, $stdout, $stderr) = @_;
    # very important to rewind DATA so this subroutine can be called more than once
    seek(DATA, 0, 0);
    my $cmd_str = join(" ", @$cmd);
    while(<DATA>) {
        # iterate until we match the command name with '*' @ the beginning
        if(/^\* $cmd_str/) {
            while(<DATA>) {
                # until we find another command entry...
                if(/\*\s/) {
                    return;
                } else {
                    # pack the results into the $stdout reference we were given
                    $$stdout .= $_;
                }
            }
        }
    }
}

And the matching section in the DATA block:

__END__
* hpacucli 'controller slot=0 physicaldrive all show'

Smart Array P400i in Slot 0

  array A
   physicaldrive 1I:1:1 (port 1I:box 1:bay 1, SAS, 73.4 GB, OK)
   physicaldrive 1I:1:2 (port 1I:box 1:bay 2, SAS, 73.4 GB, OK)
   physicaldrive 1I:1:3 (port 1I:box 1:bay 3, SAS, 73.4 GB, OK)
   physicaldrive 1I:1:4 (port 1I:box 1:bay 4, SAS, 73.4 GB, OK)
   physicaldrive 2I:1:5 (port 2I:box 1:bay 5, SAS, 73.4 GB, OK)
   physicaldrive 2I:1:6 (port 2I:box 1:bay 6, SAS, 73.4 GB, OK)

And you’re done! This is nice and easily extensible. If you have several specific use cases (like output that actually starts with ‘* ‘, for example, or needing STDIN/STDERR) you’d need to modify it. It’s a handy general pattern for making “sysadmin code” a lot more like “developer code”, though. (The great thing is I was able to write and test this whole code suite on a VM that didn’t even have an HP raid controller, let alone hpacucli installed, for example.)

The short version is this:

1. You donate
2. I go and REALLY REALLY bust my butt
3. I vomit (optional)
4. We help out some deserving people and (for those of us with prostates) maybe even ourselves.

“But Josh, how do I do this?” You ask.
Good question! Just go to my easy-to-remember donation page:
http://j.mp/WatchJoshSweat
In 30 seconds you can enter the amount you’d like to donate and a credit card number. Very easy!

Update: Some people were curious about what the workout itself is.
Straight from the Crossfit FAQ: