code-imports

I finally put up the emacs based code import-management solution I've been using for a while. I wrote a similar thing a few years ago at Google, but I rethought and rewrote this, plus added a bunch of ert tests. I've put it up as code-imports in github.  The module allows you to organize imports according to predefined rules, as well as grab other files as imports and add them as imports to a target file.

As opposed to the earlier incarnation, I've extracted as much as the logic as possible into purely functional functions, which makes it easy to test.  In my experience, import management is complicated enough that manual testing is just not enough to ensure a stable product.  Testing the code thoroughly is one way to solve this problem.  I've also simplified the number of concepts in the code, and the number of variable to tweak to the bare minimum.

I'm going to refine this a bit, then announce it more broadly shortly.

A brief demonstration of emacs new lexical bindings

It wasn't a joke.  On April 1st, Stefan Monnier merged the lexical binding branch into the emacs trunk.  Unless it needs to be rolled back, emacs 24 should have lexical binding, which is a huge improvement for elisp.  If you want to try it out, get the latest emacs from source, and compile it yourself.

To enable it, you need to set lexical-binding to t.  This is buffer local, so you need to set it for every buffer that you want to have lexical binding in.

Here's a demonstration in an ielm session (M-x ielm).

ELISP> (setq lexical-binding t)
t
ELISP> (defun make-adder (base) (lambda (num) (+ num base)))
make-adder
ELISP> (setq x 1)
1
ELISP> (setq 1-adder (make-adder x))
(closure
((base . 1)
t)
(num)
(+ num base))
ELISP> (funcall 1-adder 10)
11
ELISP> (funcall 1-adder 100)
101
ELISP> (setq x 100)
100
ELISP> (funcall 1-adder 100)
101

Without lexical binding, the variable x would be a dynamic variable, and so when we set it to 100 later in the session, the 1-adder would change to a 100-adder inadvertently.  In fact, if we try, we can't even get to that point.  Here's the same session without lexical binding:

ELISP> lexical-binding
nil
ELISP> (defun make-adder (base) (lambda (num) (+ num base)))
make-adder
ELISP> (setq x 1)
1
ELISP> (setq 1-adder (make-adder x))
(lambda
(num)
(+ num base))
ELISP> (funcall 1-adder 10)
*** Eval error *** Symbol's value as variable is void: base

What's going on here?  Notice that when we make the 1-adder without lexical binding, we don't get a closure.   Instead, we just get a function that references num and base.  num is passed into the function, but base is a dynamic variable, so it is interpreted in the emacs global context, as opposed to a context inside a closure.  The variable base is not bound to a value in the global context, and when we execute the lambda, which refers to base, emacs looks for the base variable, cannot find it, and throws an error.

This doesn't happen with lexical binding.  In that case, we can actually see the closure itself being returned when there is lexical binding:

(closure
((base . 1)
t)
(num)
(+ num base))

The closure here shows a structure in which we can interpret to mean that base is being bound to 1.  You can also see num is the argument (since it is not in the same s-expression as the binding of base), and then it proceeds with the simple addition function.

I haven't yet made use of closures in my emacs code.  I don't really need it in my initialization files, and if I writes modules that assume it then almost no one can yet use it.  So I'll give it time, but it's a huge step in making elisp a better lisp.  Read more about lexical bindings Paul Graham's wonderful free book On Lisp.

Lexical bindings are not only a good feature in themselves, they are also is an important step on the way to multithreading.  Having nothing but global variables makes threading quite challenging, and lexical bindings in the standard elisp packages can help make multithreading easier.

Data

Humans are not very good at remembering event frequency statistics. We tend to only remember the unusual, and forget about the common case.  There are plenty of other cognitive errors we make, but this seems to me one of the most troublesome.

An example: Every day I take the subway from Forest Hills in Queens to Manhattan.  Either the E or F train will arrive first, and I need the E train. I was noticing, though, that it always seemed to be the F train arriving first, making me wait for an extra few minutes.

Why would that be? Could it be a situation like Feynman's girlfriend puzzle, where the E train closely follows the F?  I decided to keep some statistics to find out.

Every day, I'd record which train came first. After a few days, I noticed the E had been coming first lately. Surely a fluke.

I kept at it, and before long the trend was clear. The E comes first almost twice as often! I counted 18 E firsts, and only 10 F firsts.

Obviously, I was completely wrong about even the most basic statistics for my morning commute, and that's about something I experience every single workday morning. Think how wrong I can be about everything else!

I take this as a warning about not trusting experiences when making decisions. Get data instead. Even just some data will be better than opinion or memories.

Does everything have to be social?

All I wanted to do was to download a document. To do that, I had to sign up for Scribd via my Facebook account. Next thing I know, I am getting follower notifications emails from some of my Facebook friends.

I just wanted to download a document. Now I'm connected to all sorts of friends. What do they want from me? Did they even explicitly choose to friend me or was it automatically done by Scribd?

For the time being, I'm just going to ignore it all and hope it goes away.

Macros and meta-macros

Life is repetition.  We do the same things, day after day.  Within each day at work, we find ourselves performing the same actions, more or less.   The great thing about computers is that tasks performed on computers can be automated.  So any time we find that we are repeated ourselves on the computer, we have an opportunity of creating a tool that will allow us to eliminate repetition.

Macros are a great example of this, and one of the defining feature of lisp.  Sure, c++ has macros as well, and they share same of the same capabilities as lisp's macros, but in general the power of lisp macros make them better for everyday use.  Macros are great tools to reduce repetitive code to simple forms.  Code such as:


Seems reasonable enough if you just are opening up one file.  But what if you do this all the time?  Even the simple boilerplate here starts to add up.  That's why common lisp created a macro of an easy way to do this:


What a breath of fresh air that is... there is absolutely nothing wasted here.  All repetitive and wasted code has been removed, leaving only the essence of the task.  That's the art of the macro.  It's one thing to be able to write macros, but to recognize patterns of repetition is a skill worth practicing.

Meta-macros are a term I just made up for this post, which describes repetitive coding tasks we have to perform.  To take a simple example, perhaps you are always navigating to the same files.  Like macros, it is essential that we realize we are doing something repetitive, and that this task can be removed with automation.  In this case, emacs provides a solution: bookmarks.  Many times the solution isn't provided by emacs, though, or the patterns are more complicated.   Every time I change or add a new function in my .cc file, I have to edit my .h file as well.  This is just a natural part of editing C++ code, something so ingrained in us that it's hard to even imagine not going through the trouble of essentially adding function definitions twice.  It's a plainly repetitive task, which could certainly be automated with a meta-macro, by which I mean some elisp code.

Sometimes these meta-macro's automation can increase complexity.  It works fine 98% of the time, but 2% of the time it screws you up and you stop using the meta-macro in frustration.  But macros can have the same trait.  Good macros are actually a bit hard to write, and can be very confusing when they fail.  Meta-macros can be even worse, due to the complexity of the meta level in which we operate.  It takes dedication to solve each new problem you encounter, making your meta-macro more and more robust.  But, with enough effort, you will have an elegant solution that makes you more productive, and able to focus on problem solving, not mere coding.

When you do, please share it with the word.  We all need the help.

Breadcrumb mode for emacs

I found this useful mode via reddit: breadcrumb.  It allows you to define bookmarks with a keypress.  The bookmarks are unnamed, but you can jump between them, either between-buffer or within-buffer.  This should be really useful for working within a changelist.

One thing that may be useful would be loading or saving breadcrumb state, as I switch between working on different projects.  So I may have to add this (it probably isn't that hard).

Saner ansi-term in emacs

I like ansi-term in emacs. It is the closest to a "real" terminal, offering all the goodness that bash, zsh, or whatever your favorite shell can give you. However, you definitely lose a few things in going to it. The following emacs code restores at least the things I find most useful: dabbrev-completion, yanking, and correct colors.

When you use this code, note that dabbrev-completion is C-c /, and yanking is C-c C-y.