Browsing man pages in vim

So I've been playing around a lot with my .vimrc lately, and this is one of the more useful things I've added. I have forgotten where it came from exactly, but here it is:

let $GROFF_NO_SGR=1
source $VIMRUNTIME/ftplugin/man.vim
nmap K :Man <cword><CR>

What's it good for? The default key binding for K (that's "shift-k" I guess) in vim is to look up the word under the cursor using the man command. The sad thing about this process is that vim gets replaced by less (or whatever pager you happen to be using), and that once you're done reading the page, you have to press "Enter" one additional time to get back into vim and back to whatever you were doing. Kinda breaks your flow, you know?

Once you've added the three lines above to your .vimrc things are quite different. When you hit K, the man page opens as a new split window inside of vim so you're staying in the same environment. All the usual binds for switching between windows work, so you can keep the man page open while going back to your code. Better yet, the man page will be "syntax highlighted" using different colors for headings, text, and (you guessed it) references to other man pages. And the best thing? You can browse man pages the same way you browse tags: use "ctrl-]" to open another man page and use "ctrl-t" to "go back" to the previous one.

Now that's how man pages were supposed to be integrated with your editor. Very nice indeed... :-D

There's one small problem that I have not been able to work around yet: The original K could be preceded by the section number to look in, but this won't work in the replacement above. I am not enough of a vim hacker yet to add that capability. Shame on me?

Update: Actually, I forced myself to learn just enough of vimscript to cobble together something ugly for section numbers:

" experimental hack to get section numbers to work as well

function ManWrapper(n, w)
  if a:n > 0
    let cnt = a:n-line(".")+1
    execute "Man" cnt a:w
  else
    execute "Man" a:w 
  endif
endfunction 

com -count=0 -nargs=+ CMan :call ManWrapper(<count>, <f-args>)
nmap X :CMan <cword><cr>

Yes, I know, it's quite horrific! If you know this dreadful language better, please tell me how to rewrite this cleanly.

Random Design Patterns, Part 1

I have no special reason to start writing these, except that I've been re-reading some patterns stuff recently. And while they are "warm" in my brain, I might as well try to write them down as that always seems to help me "solidify" things. None of the patterns I'll write about are new in any way, so feel free to skip these posts you pattern gurus!

First pattern, simple as can be: Null Object. Say you have some operation that returns a Sprite object that you then do something to. What if there is no sprite? You could return NULL or nil or None or whatever your language of choice calls the thing. But then you have to check the returned value:

sprite = some_operation()
if sprite:
    sprite.do_something()

If instead you return a Sprite instance that simply doesn't do anything, your code becomes a little more straightforward:

sprite = some_operation()
sprite.do_something()

Not exactly a big deal, but of course it could add up to something more significant if you were using this in a more complicated way.

It's certainly not a good idea to always ignore the fact that you didn't find the sprite in question, for example you don't want to keep inserting NullSprite objects into a list over and over. So when you do care, you need a way of telling that it's a real sprite. One way is to guarantee that only a single NullSprite is ever created and to make that one globally accessible. Then, in places where you care, you can say this:

sprite = some_operation()
if sprite is not Sprite.NULL:
    sprite.do_something()

Whether Null Object is particularly useful therefore depends on how often you care versus how often you don't care (but still have to check if you use the language's builtin version of "no such object"). As with all design patterns: Think before you apply the pattern!

Sets of Dictionaries in Python

I may come to regret this post in the future, we'll see. So I was hacking on some sysadmin tool that collects data. Each item is a dictionary of various things, and I had all those dictionaries in a list. Wait! What if I parse another piece of data that results in an identical dictionary? I don't want to keep growing the list to infinity with duplicates, do I? So without much thought I replaced the list with a set, but that doesn't work:

>>> set([{}])
Traceback (most recent call last):
  File "", line 1, in 
TypeError: unhashable type: 'dict'

Of course this makes sense: Python implements sets as dictionaries and dictionaries as hash tables and you cannot use something mutable as a key in a hash table. (If you don't see why that's so, think harder.) But sensible or not, what I certainly don't want to do is search my list of dictionaries for duplicates before every single insertion! So I came up with a little hack to make dictionaries hashable:

def hash_string_dict(obj):
    """
    Return a unique-ish string for a dictionary mapping
    string-ish things to string-ish things.
    """
    import hashlib
    k = "".join(sorted(str(obj.keys())))
    v = "".join(sorted(str(obj.values())))
    digest = hashlib.sha1(k+v).hexdigest()
    return digest

Alright, so the dictionaries are not really hashable, instead I produce a hashable digest of a dictionary's contents. You can now give me a lecture on how this is not very efficient, and some part of me would agree. However, given a long enough list of dictionaries, the time I spend on computing these digests will be less than looking through the whole list for a duplicate.

So instead of a list of dictionaries or a set of dictionaries, I end up with a dictionary of dictionaries: The key in the outer dictionary is the digest of the inner dictionary. If a duplicate comes along it'll produce an identical digest and I can forget about it after one (expected!) constant time lookup.

Of course all of this depends crucially on my dictionaries being immutable as far as my application is concerned. Python itself doesn't have the luxury of wondering about this, it has to "worst-case" it and assume all dictionaries are mutable, period. I wonder: Should I package this as a container class? :-D

Update: Note that it's sort of important that the keys and values you have in that dictionary produce "useful" string representations. In other words, don't apply this trick without thinking through the kind of data you're pushing around and whether the digest has a reasonable chance of being accurate enough for duplicate detection.

And a new blog?

I used to have a tiny blog of my own at the now-defunct http://blogs.jhu.edu/ which was run by our local student ACM chapter. For some reason JHU decided that they want the name back to run their own blogging service, so the students (and me, and a whole bunch of other people who were using their service) got booted off that URL. Of course JHU never did set up their own service, sigh.

Anyway, apparently the students got frustrated enough to not re-open the blogging service under a different name, so for the past year or so I've not had a blog. Fast-forward to yesterday: I needed some information from one of my old blog posts, so I decided to request the raw data (thanks for the SQL dump Rich!) and to start over somewhere else. So here I am.

Not sure how long it'll take me to import the old posts, but I guess eventually you'll be able to read them here. Not that they were terribly interesting before, but hey, gotta have a blog, no?

Self-Signed SSL Certificates

So you'd think there are enough blog posts about this already. Well, think again, since this particular summary is what actually ended up working for me. Obviously your mileage may vary.

1. openssl genrsa 1024 > host.key
2. openssl req -new -key host.key -out host.csr
3. openssl x509 -req -days 730 -in host.csr -signkey host.key -out host.crt

Of course that's what everybody has, so why write about this? Three reasons:

• Make sure you chmod 400 host.key since you don't want anybody to see that.
• Using lighttpd? Do a cat host.key host.crt > host.pem and chmod 400 that as well.
• The "Common Name" you have to enter in step 2. If you have various subdomains like www.example.com and mail.example.com and so on, you don't want to enter "example.com" here. Instead you'd enter something globtastic like "*.example.com". But wait, that doesn't match just plain example.com anymore! Better use "*example.com" and wow, that actually works.

An Internet. Wow! It's so pretty... Who would've thunk? :-D

Program like it's 1975?

Looking through various proxies and caches (don't ask why) I ran across Varnish and was struck by a short piece one of their developers wrote in 2006. Let me quote a line or two (emphasis mine):

Take Squid for instance, a 1975 program if I ever saw one: You tell it how much RAM it can use and how much disk it can use. It will then spend inordinate amounts of time keeping track of what HTTP objects are in RAM and which are on disk and it will move them forth and back depending on traffic patterns. Well, today computers really only have one kind of storage, and it is usually some sort of disk, the operating system and the virtual memory management hardware has converted the RAM to a cache for the disk storage.

You should read the whole thing, it's a nice summary of technological changes that still haven't made it into everyone's head. I feel guilty myself: I recently taught our OS course, but I don't think I ever made this sufficiently clear when we talked about virtual memory. I'll try to add a relevant assignment to the Unix course next semester... :-D

Pocco: Another Take on Documentation

I just discovered a cute tool called pocco on github. Apparently pocco is the Python version of what is also known as docco, rocco, and (gasp) shocco out there. Quite the zoo really! :-D So what does it do? It parses Python source and separates comments from code. It then dumps everything into HTML with a little CSS, making the comments appear on the left lined up with the corresponding code on the right. Neato.

Or it is? What could this possibly be useful for? Comments in Python are not usually used for documentation: we have docstrings after all! But docstrings are not perfect, for example you can only (by "natural" means anyway) attach a docstring to a module or class or function: What about those globals over there? Do I really have to document them in the module docstring?

But I realized that something like pocco is actually useful in Python. After all, the documentation you put into a docstring should be concise and clean: it should say exactly what it needs to say for someone else to use your stuff, nothing else. Boring! What about all those humorous anecdotes you accumulate while you hack? What about false starts, things that others should be told about before they head down the same wrong way. And so on, and so forth. With pocco, you can use comments for commentary and docstrings for documentation yet extract and (nicely?) format both. How cool is that? :-D Let's just hope that pocco will leave docstrings alone for good...

The other use I could come up with is code handouts in programming courses. Often the comments I write in this context have more of a "let me take you by the hand and work this through with you" feeling. Reading them in a nicely formatted and syntax-highlighted way may be nicer for everyone involved. True, since all comments are extracted it's hard to "comment on comments" that way... But that's okay, at least for me. Now all I need for this second application is Java, C, and C++ support! :-D