Cronologue

Came across cronologger (blog post) recently (via Dean Wilson), which is a simple wrapper script you use around your cron(8) jobs, which captures any stdout and stderr output and logs it to a couchdb database, instead of the traditional behaviour of sending it to you as email.

It's a nice idea, particularly for jobs with important output where it would be nice to able to look back in time more easily than by trawling through a noisy inbox, or for sites with lots of cron jobs where the sheer volume is difficult to handle usefully as email.

Cronologger comes with a simple web interface for displaying your cron jobs, but so far it's pretty rudimentary. I quickly realised that this was another place (cf. blosxom4nagios) where blosxom could be used to provide a pretty useful gui with very little work.

Thus: cronologue.

cronologue(1) is the wrapper, written in perl, which logs job records and and stdout/stderr output via standard HTTP PUTs back to a designated apache server, as flat text files. Parameters can be used to control whether job records are always created, or only when there is output produced. There's also a --passthru mode in which stdout and stderr streams are still output, allowing both email and cronologue output to be produced.

On the server side a custom blosxom install is used to display the job records, which can be filtered by hostname or by date. There's also an RSS feed available.

Obligatory screenshot:

Cronologue GUI

Update: I should add that RPMs for CentOS5 (but which will probably work on most RPM-based distros) are available from my yum repository.

Blosxom4Nagios

A little whole ago at one of my client sites we decided that we wanted to monitor the bulk of our Nagios notifications via RSS rather than via email or jabber. The centralised river-of-news architecture of RSS is just a much better fit for high-volume notification flow than the individual silos and persistent messaging nature of email.

There are a few nice RSS solutions out there for Nagios, including the following:

We quite liked Altinity's RSS4NAGIOS because it was notifications (rather than alerts) based, and because it was pretty easy to just drop in and use. The only thing we didn't really like was that it was still relatively static - it provides nice per-user feeds, but you can't carve those feeds up or drill down to subsets of the data very easily. We wanted to be able to slice-and-dice things a bit more dynamically - be able to look at feeds for per-host, per-hostgroup, per-status, or date-filtered notifications, for example.

I'm also a blosxom developer, so I quickly realised that I could do everything I wanted pretty trivially using blosxom. All I needed was a script to capture and tag notifications as blosxom posts, and then I could have dynamic filtering, multi-dimensional tag-based feeds, etc., all pretty much for free.

So a couple of afternoons later, Blosxom4Nagios was up and running. It's basically a single-purpose blosxom install that you drop into a directory somewhere (/var/log/nagios/blosxom, by default), hook into apache for display, hook into nagios to accept notifications, and away you go.

Notifications are tagged by type (host/service), state (OK, WARNING, CRITICAL etc.), hostname, hostgroup, service name, service group, contact, and date, so you can filter notifications based on any of these, and produce feeds (both RSS2 and atom) on any of them as well e.g.

etc.

Screenshots:

Default View

Filtering by date

Filtering by host

Blosxom4Nagios is available here:

and is licensed under the same MIT Licence as blosxom itself.

Comments and feedback welcome.

Blosxom hcard plugin

Building on my initial set of blosxom microformat plugins, the hcard plugin provides a global hcard variable for inclusion in your blosxom templates.

To use it, you simply define the set of hcard data to use in an 'hcard.yml' file in your blosxom data directory, and then include $hcard::hcard somewhere in your blosxom flavours/template. An example hcard.yml for me might be:

Name: Gavin Carr
Organisation: Open Fusion
Role: Chief Geek
Email: gavin@openfusion.com.au
URL: http://www.openfusion.net/
Suburb: Wahroonga
State: NSW
Postcode: 2076
Country: Australia
Latitude: -33.717718
Longitude: 151.117158
HCard-Class: nodisplay
HCard-Style: div-span

I'm using hcard here, so if you have microformat support in your browser (e.g. via the Operator plugin, if using firefox) you should be able to see my hcard on this page.

As usual, available in the blosxom sourceforge CVS repository.

Blosxom Microformat Plugins

I've been messing around recently with some ideas on adding some initial microformats support to blosxom.

Microformats are fragments of html marked up with some standardised html class names, providing a minimalist method of adding simple structured data to html pages, primarily for machine parsing (try out the firefox Operator plugin to see microformats in action). Some examples of currently defined microformats are contact details (hcard), events (hcalendar), links or bookmarks (xfolk), geolocation (geo), etc. See the main microformats website for more.

With blosxom, one simple approach is to allow microformat attributes to be defined within story metadata, and either autoappend the microformat to the story itself, or simply define the microformat in a variable for explicit inclusion in the story. So for example, if you wanted to geocode a particular story, you could just add:

Latitude: -33.717770
Longitude: 151.115886

or

meta-latitude: -33.717770
meta-longitude: 151.115886

to your story headers (depending on which metadata plugin you're using).

This is the initial approach I've taken, allowing you to attach microformats to stories with a minimum of fuss. So far, the following blosxom microformat plugins are available:

  • uf_adr_meta - adr support
  • uf_geo_meta - geo support
  • uf_hcalendar_meta - hcalendar support
  • uf_hcard_meta - hcard support
  • uf_xfolk_meta - xfolk support

Note that these are beta quality, and may well contain bugs. Feedback especially welcome from microformat gurus. There's also a lot of other ways we might like to handle or integrate microformats - this is just a useful first step.

All plugins are available in blosxom sourceforge CVS repository.

Blosxom Tags

I've been using tags here right from the beginning, because they provide a much more powerful and flexible way of categorising content than do simpler more static categories. This seems to be pretty much the consensus in the blogosphere now.

I started off using xtaran's tagging plugin. The one thing I didn't like about tagging was that it has a fairly brute-force approach to doing tag filtering - it basically just iterates over the set of candidate files and opens up and checks them all, every time.

So I started messing around with adding some kind of tag cache to tagging, so that the set of tags on a post could be captured when a post was created or updated, and thereafter tag filtering could be done by just referencing the tag cache. That means that if you've got 100 posts, your tag query only needs to read one file - the tag cache - instead of all 100 posts.

En route I realised I really wanted a more modular approach to tagging than the tagging plugin uses as well. For instance, I'm experimenting with various kinds of data blogging, like using dedicated special-purpose blogs for recording bookmarks or books or photos. And for some of these blogs I wanted to be able to do basic tagging and querying, but didn't need fancier interface stuff like tagclouds.

So I've ended up creating a small set of blosxom plugins that provide most of the functionality of tagging using a tag cache. The plugins are:

  • tags - provides base tag functionality, including checking for new and updated stories, maintaining the tag cache, and providing tag-based filtering. Requires my metamail plugin.

  • storytags - provides a story level $storytags::taglist variable containing a formatted list of tags, suitable for inclusion in a story template. Requires tags.

  • tagcloud - provides a $tagcloud::cloud variable containing a formatted wikipedia:"tagcloud" of tags and counts, suitable for inclusion in a template somewhere. Requires a hashref of tags and counts, which tags provides, but should be able to work with other plugins.

Note that these plugins are typically less featureful than the tagging plugin, and that tagging includes functionality (related tag functionality, in particular) not provided by any of these plugins. So tagging is still probably a good choice for many people. Nice to have choice, though, ain't it?

All plugins are available in blosxom sourceforge CVS repository.

'entries_timestamp' blosxom plugin

I've tried all three of the current blosxom 'entries' plugins on my blog in the last few months: entries_cache_meta, entries_cache, and the original entries_index.

entries_cache_meta is pretty nice, but it doesn't work in static mode, and its method of capturing the modification date as metadata didn't quite work how I wanted. I had similar problems with the entries_cache metadata features, and its caching and reindexing didn't seem to work reliably for me. entries_index is the simplest of the three, and offers no caching features, but it's pretty dense code, and didn't offer the killer feature I was after: the ability to easily update and maintain the publication timestamps it was indexing.

Thus entries_timestamp is born.

entries_timestamp is based on Rael's entries_index, and like it offers no caching facilites (at least currently). Its main point of difference from entries_index is that it maintains two sets of creation timestamps for each post - a machine-friendly one (a gmtime timestamp) and a human-friendly one (a timestamp string).

In normal use blosoxm just uses the machine timestamps and works just like entries_index, just using the timestamps to order posts for presentation. entries_timestamp also allows modification of the human timestamps, however, so that if you want to tweak the publish date you just modify the timestamp string in the entries_timestamp.index metadata file, and then tell blosxom to update its machine-timestamps from the human- ones by passing a reindex=<$entries_timestamp::reindex_password> argument to blosxom i.e.

http://www.domain.com/blosxom.cgi?reindex=mypassword

It also supports migration from an entries_index index file, explicit symlink support (so you don't have to update timestamps to symlinked posts explicitly), and has been mostly rewritten to be (hopefully) easier to read and maintain.

It's available in the blosxom sourceforge CVS repository.

Multiple Blosxom Instances

The blosxom SourceForge developers have been foolish enough to give me a commit bit, so I've been doing some work lately on better separating code and configuration, primarily with a view to making blosxom easier to package.

One of the consequences of these changes is that it's now reasonably easy to run multiple blosxom instances on the same host from a single blosxom.cgi executable.

A typical cgi apache blosxom.conf might look something like this:

SetEnv BLOSXOM_CONFIG_DIR /etc/blosxom
Alias /blog /usr/share/blosxom/cgi
<Directory /usr/share/blosxom/cgi>
  DirectoryIndex blosxom.cgi
  RewriteEngine on
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteRule ^(.*)$ /blog/blosxom.cgi/$1 [L,QSA]
  <FilesMatch "\.cgi$">
    Options +ExecCGI
  </FilesMatch>
</Directory>

The only slightly tricky thing here is the use of mod_rewrite to allow the blosxom.cgi part to be omitted, so we can use URLs like:

http://www.example.com/blog/foo/bar

instead of:

http://www.example.com/blog/blosxom.cgi/foo/bar

That's nice, but completely optional.

The SetEnv BLOSXOM_CONFIG_DIR setting is the important bit for running multiple instances - it allows you to specify a location blosxom should look for all its configuration settings. If we can set this multiple times to different paths we get multiple blosxom instances quite straightforwardly.

With separate virtual hosts this is easy - just put the SetEnv BLOSXOM_CONFIG_DIR inside your virtual host declaration and it gets scoped properly and everything just works e.g.

<VirtualHost *:80>
ServerName bookmarks.example.com
DocumentRoot /usr/share/blosxom/cgi
AddHandler cgi-script .cgi
SetEnv BLOSXOM_CONFIG_DIR '/home/gavin/bloglets/bookmarks/config'
<Directory /usr/share/blosxom/cgi>
  DirectoryIndex blosxom.cgi
  RewriteEngine on
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteRule ^(.*)$ /blosxom.cgi/$1 [L,QSA]
  <FilesMatch "\.cgi$">
    Options +ExecCGI
  </FilesMatch>
</Directory>
</VirtualHost>

It's not quite that easy if you want two instances on same virtual host e.g. /blog for your blog proper, and /bookmarks for your link blog. You don't want the SetEnv to be global anymore, and you can't put it inside the <Directory> section either since you can't repeat that with a single directory.

One solution - the hack - would be to just make another copy your blosxom.cgi somewhere else, and use that to give you two separate directory sections.

The better solution, though, is to use an additional <Location> section for each of your instances. The only extra wrinkle with this is if you're using those optional rewrite rules, in which case you have to duplicate and further qualify them as well, since the rewrite rule itself is namespaced i.e.

Alias /blog /usr/share/blosxom/cgi
Alias /bookmarks /usr/share/blosxom/cgi
<Directory /usr/share/blosxom/cgi>
  DirectoryIndex blosxom.cgi
  RewriteEngine on
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_URI} ^/blog
  RewriteRule ^(.*)$ /blog/blosxom.cgi/$1 [L,QSA]
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_URI} ^/bookmarks
  RewriteRule ^(.*)$ /bookmarks/blosxom.cgi/$1 [L,QSA]
  <FilesMatch "\.cgi$">
    Options +ExecCGI
  </FilesMatch>
</Directory>
<Location /blog>
  SetEnv BLOSXOM_CONFIG_DIR /home/gavin/blog/config
</Location>
<Location /bookmarks>
  SetEnv BLOSXOM_CONFIG_DIR /home/gavin/bloglets/bookmarks/config
</Location>

Because one blosxom just ain't enough ...

'mason_blocks' blosxom plugin

I've just released my first blosxom plugin into the wild. 'mason_blocks' is a blosxom plugin implementing simple conditional and comment blocks using HTML::Mason-style syntax, for use in blosxom flavour and template files.

Examples:

# Mason-style conditionals
% if ($pagetype::pagetype ne 'story') {
<a href="$permalink::story#comments">Comments ($feedback::count)</a>
% } else {
<a href="$permalink::story#leave_comment">Leave a comment</a>
% }

# Mason-style comments
%# Only show a comments section if there are comments
% if ($feedback::count > 0) {
$feedback::comments
% }

# Mason-style block comments

I wrote it when I couldn't get the interpolate_fancy plugin to work properly with nested tags, and because I wanted proper perl conditions and if-else support. mason_blocks provides all the conditional functionality of interpolate_fancy, but not other stuff like 'actions'.

mason_blocks is available from the blosxom plugins CVS repository.

Blosphemy

I've been trying out a few of my blosxom wishlist ideas over the last few days, and have now got an experimental version of blosxom I'm calling blosphemy (Gr. to speak against, to speak evil of).

It supports the following features over current blosxom:

  • loads the main blosxom config from an external config file (e.g. blosxom.conf) rather than from inline in blosxom.cgi. This is similar to what is currently done in the debian blosxom package.

  • supports loading the list of plugins to use from an external config file (e.g. plugins.conf) rather than deriving it by walking the plugin directory (but falls back to current behaviour for backwards compatibility).

  • uses standard perl @INC to load blosxom plugins, instead of hardcoding the blosxom plugin directory. This allows blosxom to support CPAN blosxom plugins as well as stock $plugin_dir ones.

  • uses a multi-value $plugin_path instead of a single value $plugin_dir to search for plugins. The intention with this is to allow, for instance, standard plugins to reside in /var/www/blosxom/plugins, but to allow the user to add their own or modify existing ones by copying them to (say) $HOME/blosxom/plugins.

These changes isolate blosxom configuration from the cgi and plugin directories (configs can live in e.g. $HOME/blosxom/config for tarball/home directory installs, or /etc/blosxom for package installs), allowing nice clean upgrades. I've been upgrading using RPMs while developing, and the RPM upgrades are now working really smoothly.

If anyone would like to try it out, releases are at:

I've tried to keep the changes fairly minimalist and clean, so that some or all of them can be migrated upstream easily if desired. They should also be pretty much fully backward compatible with the current blosxom.

Comments and feedback welcome.

Packaging Blosxom

I'm currently working on packaging blosxom as an RPM for deployment on a few different RedHat/CentOS servers I administer. With most small-medium software packages this is pretty straightforward - write a simple spec file, double-check the INSTALL instructions, and replicate those in the spec file. It's rather more challenging with blosxom.

blosxom's roots are in supporting extremely minimalist environments. It's reasonably straightforward to setup blosxom on a 1990s shared web hosting account with only the most basic CGI support, and only FTP access to the server for your files.

Blosxom itself is a single perl CGI script, which you configure by setting a few variables at the top of the script. Blosxom plugins, which are used to implement lots of the functionality in blosxom, are likewise little perl modules configured (if necessary) at the beginning of each plugin. In a shared web hosting environment you'd configure blosxom itself and your plugins the way you'd like, and then upload them to your server home directory via FTP.

Fast forward to 2007, where virtual linux servers with full root access are available for US$15/month, with prices continually dropping. In this kind of environment the whole mixing-configuration-and-code thing becomes much more of a liability than a feature.

There's a debian package available, so the debian guys have made a start of wrestling with some of these issues - they patch blosxom to allow it to use an external config, for example. I've done something similar, and am realising I'm going to want to support the same kind of thing with plugins.

So here's my current wishlist for a blosxom RPM:

  • the ability to install one of more blosxom packages and get blosxom itself, a good set of blosxom plugins, and a good set of blosxom flavours and themes all ready to go

  • a proper separation between config and code, so that I can upgrade any of my blosxom packages without having to worry about losing config settings

  • an easy way of configuring exactly what plugins and themes are used for my blog

  • most standard modern blog features available more-or-less out-of-the-box (e.g. comments and spam protection, support for sending "trackback":wikipedia:Trackback pings, support for receiving trackbacks and "pingbacks":wikipedia:Pingback, OpenID support, support for microformats, etc.)

  • multi-user and multi-blog support, so that an installed blosxom can be used for multiple blogs

  • mod_perl support, for scalability

That's my current wishlist anyway. I'm still trying to figure out whether others in the blosxom development community are interested in any of this stuff too, or whether they all just still use FTP. ;-)

Why Blosxom?

I'm using blosxom for this blog. I'd played with it a while ago and really liked its simplicity and ethos, but never got it working quite the way I wanted. When returning to the blogging world recently I went and looked a few of the popular alternatives - Typo, Wordpress, Movable Type - and didn't find anything that really grabbed me.

Yes, all three are slicker, more modern, and have a lot more functionality out-of-the-box than blosxom, as far as I can tell. So why am I back with blosxom?

For me, blosxom has two killer features:

  • you can write your blog entries offline, using a real editor, and using nice sane rich-text formats like Markdown

  • it is simple and pluggable, by design, which makes it immensely hackable

In fact, blosxom isn't really full-blown blogging software at all, especially as it's presently packaged and distributed. Instead it's a lightweight pluggable toolkit with which to build a blog. If you're after something that Just Works, it's probably a bad choice; if you're after something you can play with and bend to your will, it's really nice.

Blosxom's also suffered a bit from not having had much development love over the last few years. Be nice to see blosxom get a bit more support for the modern blogging world - have to see if I can help stir things up a bit ...