When there is only one way of doing things, it is easier to modify and reuse code. When code is reused, programs are easier to change and most importantly, shrink. When a program shrinks its construction and maintenance requires fewer people which allows for more opportunities for reuse to be found. Consistency leads to reuse, reuse leads to conciseness, conciseness leads to understanding.
I like the meaning of ‘reuse’ in this quote. This is not the ‘reuse’
promised by early OO proponents. They promised1: that one would be
able to create objects which would be reused from one project to
another. This is the reuse which means that code is reused in a
project. This is the reuse which I believe to be attainable.
It is like a inward-spiral: the more reuse of code the fewer ways to
do a thing there will be which leads to only one way.
The sentence about fewer people is a bit odd - but I feel it is does
make sense. The fewer people on the team the more likey there will be
only one way to do things which means there is by definition more
Perhaps more important than reuse though is consistency. If code is
consistent I have found it to be easier to reason about, because
reasonable assumptions can be made when reading it. Being able to make
reasonable assumptions, and have those borne out, is very important to
me when reading code.
Conciseness and Understanding
And it follows that with reuse and consistency one will get
consiceness. But does that necessarily lead to underderstanding? That
is something I feel is right, but I have seen some concise code
which is hard to understand.
“Programming as Experience: The Inspiration of Self”, 1995, Smith,
Randall B and Ungar, David
Or perhaps that is not what they said, but what we heard.↩
While avoiding family this Christmas-day I started trying to figure out way
for me to do TDD in Common Lisp (as one does…). That led to SLIME1 and
QuickLisp2 and lisp-unit3 and asdf4 (&c. &c).
…it’s yaks the whole way down.
The hardest part for me was to find a simple way to load my code and run the
tests. ASDF ships with asdf:test-system defined but it does nothing by
default5. So I did some digging around and saw some other’s solutions and
synthesized my version.
Here is an example ASDF system definition for a Game of Life6
(asdf:defsystem#:gol:description"Conways Game of Life in Lisp":author"Mark Simpson <firstname.lastname@example.org>":version"0.0.0":depends-on("lisp-unit"):components((:module"src":serialt:components((:file"package")(:file"gol"))(:module"test":serialt:depends-on("src"):components((:file"package")(:file"gol-test")))):in-order-to((test-op(load-op:gol))):perform(test-op(oc)(progv(list(intern"*PRINT-ERRORS*":lisp-unit)(intern"*PRINT-FAILURES*":lisp-unit))'(tt)(funcall(find-symbol"RUN-TESTS":lisp-unit):all:gol-test))))
My God… it’s full of yaks
I’m just going to let that sink in for a bit while I get some coffee.
Ok, now that I’m back I will explain each part. The first interesting bit is
on line 6:
This line tells ASDF that it should find and load a system called lisp-unit
because our system depends upon it. This is where QuickLisp makes things
simple since that is its job and it does a fine one at that.
I’ve decided that my project should have two main directories, one to contain
all the production source code and one for all the test source code. Each one
will be in their own packages, with the :gol-test package :use-ing the
:gol package. :serial t by the way tells ASDF that each item in the
component list depends upon the one before it.
Now the hairier parts which tell ASDF to run our tests:
The first line is simply stating that in order to do test-op ASDF needs to
perform load-op on the :gol system. This way every time we run the tests
we’ll reload the system if needed.
Now… this next bit probably shows how rusty my Lisp is. Maybe there is an
easier way. Basically we want to run (lisp-unit:run-tests :all :gol-test)
with lisp-unit:*print-errors* and lisp-unit:*print-failures* both bound to
t (they default to nil and that doesn’t give enough info IMNSHO). However
when Lisp is reading and evaluating this code lisp-unit has probably not yet
been loaded - so we have to be crafty and find the symbol by name in the
package then funcall it. Binding the variables was trickier but I stumbled
upon progv7 and that seems to be just what I needed.
progv creates new dynamic bindings for the symbols in the list which is its
first argument. These symbols are determined at runtime. That means we can use
intern to find or create these symbols in the right package8. progv
binds these symbols to the values in the list which is its second parameter
and then finally executes the sexp which is its third argument in an
environment which contains these new bindings.
With this work done I can now type in (asdf:test-system 'gol) into my REPL
and it loads and runs my tests and gives me useful output of those tests. I
might go so far as to bind that to a very short function name, or even a key
There are still more yaks to shave.
What is left to be determined is if this is the best way to do things.
Some documentation/blogs I read lead me to believe that putting the tests in
the same system with the production code is verboten (or at least a
no-no). I tried having the tests be in their own system definition
(depending upon the production code) but then ASDF & QuickLisp both strongly
pushed me to have that definition it its own .asd file9 which seemed
awkward to me. I personally have no problem shipping tests with the production
code, and I believe that if needed all the symbols of the test package could
be unintern-ed before an image was saved if desired.
I’ll play around with this setup and see how things go.
Being able to read source code2 is very important. So how does
one do it well?
Just like reading any other material - first decide why is one reading it.
Why is one reading the listing? Is one debugging a problem or trying to gain
specific knowedge?; or reading to get general or pleasure? One takes different
approaches depending upon the desired goal.
The first step is to find an interesting spot to start.
For Debugging (or to Gain Specific Knowledge)
Start by skimming of the code, grep3 and tags4 tables are useful for
this. This will help one find the right, or at least likely entry point.
There is a lot of instinct5 involved in this step. One gets better at it
the more one needs to do it6.
To Get General Knowledge (or for Pleasure)
Code is for reading7 so read it. Skim the code (including the tests8).
Look for something interesting, get the lay of the land, the shape of the
code. Note the Dramatis Personæ. Perhaps the code seems to be about widgets
and frobbing, oh and here’s something about twiddling that looks interesting.
Pick one of these that is most intriguing (e.g. One wonders what happens when
a widget is frobbed.) and use that as one’s entry point. This is a matter of
Now that One has One’s Entry Point…
Now that one is staring in the face a chunk, hopefully a small chunk, of code;
how does one read it?
Read the first line of code, what does it say? Believe it. That can be risky -
but at this point one must do it. What does the next line say, and so on.
Follow function calls11 if they seem interesting. Skip over things that are
not, or do not seem relevant. If one’s interest is in frobbing ignore code
about tweaking or twiddling. Perhaps one makes a note that frobbing seems to
involve tweaking or twiddling; but one must know about frobbing now so those
are not relevant. This is also very tied to instinct.
It is this point here which I think is most important. One must gain a sense
of what is relevant when reading code. One cannot read every code path and
think that will lead to understanding. That would be like reading the
dictionary and assuming one knows the language. This is where I have seen
programmers fail in their reading.
This is a very good reason why naming is so important12.
It has occurred to me that my style of reading code is not the same as
some other’s. I have been told by a person I think highly of that they
feel my way is the/a right way. I have finally gotten around to writing it
up as a quick blog post.↩
Is it time for us to stop calling it ‘code’ given its implications of
obfuscation? I think it is still appropriate as long as we remember that
the definition is not about obfuscation per se but about using a set of
symbols to mean other words/letters/symbols. This is, IMNSHO, exactly what
we are doing. ↩
This year’s SCNA Conference seemed to have several talks which touched upon
the idea of experts having implicit knowledge. That is to say: they know
things but can’t necessarily explain what they know, or why. Ken Auer and Dave
Thomas touched upon it; during the panel on Software Quality it was also
brought up (e.g. Corey Haines’ amazing reference to Chicken Sexing).
But I felt that Brian Marick’s talk took the idea and then added to it.
Bright vs. Dull Cows
Brian started talking about this implicit knowledge idea - giving his wife as
an example. His wife (a veterinary professor) teaches her students to
determine if a cow is ‘bright’ or ‘dull’ (a subjective diagnosis). Her
students specify, day-after-day, if cows are dull or bright. She then corrects
them. They usually ask how she could tell and she tells them something
(drooping ears, nose-cleaning, etc.). But actually these specific clues are
not really the way to tell. Those who can diagnose a cow as bright or dull
cannot explain why.
This anecdote seems to show that the implicit knowledge of the expert is
something that can be trained. When discussing this idea after the talk Corey
Haines mentioned a conference session in which Michael Feathers gave where he
had the attendees give quick good/bad judgment calls on snippets of code. Then
tallied those up and found that some snippets had a large amount of agreement
as to their good-/bad-ness. They then discussed and tried to determine why.
In light of Brian’s anecdote however it seems that they might never find out
why, but maybe they could train themselves by looking at a lot of code and
being corrected when they get the judgment wrong.
(Brian hopes to use this sort of idea to be able to train himself to have a
nearly physical reaction to bad code. While that sounds cool, I worry about a
‘Ludovico Technique’ for code.)
Effortful vs. Automatic
Brian then talked about two types of thinking: Effortful and Automatic.
Effortful is the sort when you need to do a non-trivial arithmetic, or a
parking in a tight spot. Automatic is when you do simple math like 2+2 or easy
driving where you are sort of on autopilot. When you use the Effortful way, it
actually takes effort. It is slower seems to be only possible to do it
serially. Automatic thinking is faster, takes little effort, and seems to be
If you don’t have something in Automatic then your brain shifts to Effortful;
sort of a cache miss. If you do something enough, or become good enough at
it, you can put stuff into your Automatic cache.
Switching to Automatic
So how can this be done? While Brian offers the usual ‘practice, practice,
practice’, he adds to it an interesting idea about ‘Set and Setting’. ‘Set and
Setting’ is an idea from Timothy Leary about why drugs such as LSD seem to
have very different effects in different people. The mindset (Set) of the
person (which includes things like rituals which build up that mindset) and
the Setting (environment) have a strong influence on the experiences of that
By being aware of which mindsets (and how to enter them) and what settings are
most beneficial for programming (and even different types of programming) he
feels he can move some programming tasks into his Automatic thinking area.
It seems to me that the idea of ‘Automatic’ and ‘Effortful’ adds to the idea
of the expert’s implicit knowledge. Perhaps the expert’s knowledge is just an
expression of Automatic thinking?
I’m interested in trying out the practice of being more aware of my own ‘Set
and Setting’ with respect to code.
I am once again jumping into the Clojure world and trying to learn a bit of
it. I am doing so via a small project called defclink
(source). I am also attempting to get back into blogging. So I
thought I’d combine the two items into this short post.
Goals of my project.
My goal of this project is to learn some Clojure building a webapp. I decided
to also use Clojurescript since it seemed like an interesting idea. I wanted
to keep the number of libraries and frameworks low to maximize the Clojure
code I would need to write, thus learning Clojure. That being said I allowed
myself the luxury of the Compojure library (with Ring under it) to handle the
HTML protocol stuffs.
I’ll start by explaining how to get a brand new Compojure project.
and furthermore will setup your project.clj file to pull in compojure.
It sets up simple tests that going to “/” will result in a success page with
“Hello World” and that going to a non-existant page will result in a 404.
There is a defined set of routes to accomplish this.
To run the server at this point it is easy to do:
lein ring server
and then go to localhost:3000 to see the awesome_thing
At this point you probably want to push this to awesome thing to Heroku to
start get VC funding. This is easy to do (the pushing part).
First create your heroku app in the usual manner (at the time of this writing
you don’t even need to choose the cedar stack - but check the docs to make
sure you are Clojure compatible).
Now create a Procfile with the following contents:
web: lein with-profile production trampoline run -m defclink.main $PORT
Now you should be able to push to Heroku and sit back waiting for the funding
to roll in.
“When you find yourself in a hole, quit digging”
– Will Rogers
Sometimes I wonder what the important skills for a developer are. I now
believe that one of the important ones (perhaps most important) is knowing
how/when to give up and start again. This contradicts another important
skill/trait for a developer to have: tenacity. But I think they are balancing
forces with giving up being slightly more important. Let me explain.
Often when working on a problem, it takes longer; is more complicated and
difficult than expected. At this point there are many files modified and it
looks like there are many more to modify; plenty of broken tests, maybe an
unknown quantity. There are two options: keep going, or give up.
This is the point I like to remind myself (and my coworkers) of Will Rogers
quote. Another thing I do is ask “Are we digging a tunnel, or a hole?” If we
think we are digging a tunnel then we can give ourself a time limit for
further digging; if it pays off – great! otherwise we need to climb out of
I used to believe, quite strongly, that tenacity was more important. To
persevere through the mountain of outstanding changes and get it done was
admirable. I now believe that giving up is the better option. What changed my
mind? I now see more benefit in taking the knowledge acquired from the work
done and reapply it to the problem from a fresh start. Powering through the
situation is deciding at the current way is the only way; starting again is
deciding that there is a different way, with the knowledge that the current
way has giving you (which is at least that the current way isn’t working out).
As a footnote I want to add that I also thinking giving up in another way is
important. When working on something collaboratively (such as when pairing)
one can find oneself at odds with another developer about how to proceed. I
believe that unless the idea is very obviously bad that to stop
arguing/discussing and do something is better that spending time discussing.
Especially when there is source code involved - any code is better than no
code. Any code, can then be changed into better code.
In the interest in breaking my current blogging writing block I’ll record for
posterity my ‘recipe’ for chicken soup.
First you need to eat some chicken so you have bones. I usually get rotisserie
chickens for another recipe (Skillet Chicken Pot Pie) and then save the
bones in the freezer. I use two carcasses for this recipe.
So take the bones and entomb them in a big pot with some carrots, celery,
garlic and onion. Don’t worry about chopping up the vegetables or even peeling
them - this step is making stock.
(The clever ones among you will note that I didn’t tell you how much of these
things to put in. I am doing this in honor of my mother on this
Now put in enough water to drown it all…
and bring it to a boil.
Let it boil for as long as you can stand to. You want it to get all the good
tastes out of the chicken bones and vegetables. I usually let it go for 2-4
hours and stir it occasionally. I don’t let it boil down more than half-way
At this point it should be filling up your house with a very
After boiling it down you need to strain out all the solid bits and throw them
away. I have not included a picture of this because it looks rather unpleasant
and might put you off the recipe. But take my word for it - this bit of
unpleasantness will be worth it.
What you have now is chicken stock. Do with it what you will!
I usually choose to turn right around and create soup. I now put in carrots,
celery, onion, garlic, maybe peas. These, since you will be eating them should
be peeled, cleaned, etc.
I usually, because I am lazy, add a pasta to this soup, but potato or rice
could be used instead.
As everyone that knows me knows, I have become enamored with cocktails.
So I thought, with that amazing hubris encapsulated within the novice, I would
tell you, the reader, how to stock a bar if you are new to cocktails yourself.
I’ll split this amateur’s amateur guide into the following sections:
I will start by listing the equipment you need. This will seem like a lot; but
you can find some of it in ‘kits’ which will help you get started (replacing
crappy items as you realize that there are better tools).
This seems like a lot of stuff - but like I said you can find it in ‘kits’. If
you can get better tools from a specialty store (such as The Boston Shaker).
You should start with one or two drinks you like and that have only a few
ingredients. For example a Manhattan or Negroni only have 3 ingredients; a Gin
and Tonic only two (three if you count the lime). Pick drinks you really like
because you will be making them a lot. It also helps if you pick drinks with
‘standard’ ingredients, like Gin, Vodka, Rye, Bourbon, Whisky, Campari etc.
This makes it easier to buy these ingredients from your local liquor store.
Now that you have the equipment and the ingredients; make yourself a drink –
you deserve it! Then make it again, and again, and again…
As you practice, you’ll figure out how to measure (and when you don’t care
about measuring), you’ll figure out how to stir or shake well, how much ice
you might need etc.
Once you have gotten good a drink (or perhaps sick of it) look for a drink
which has all the same ingredients but in different proportions; or a drink
that has a different ingredient you don’t have yet.
In this way your repertoire will grow and your skill will as well.
Hippocrates noted that there was so much to his craft to learn, but only such
a short life to learn it in1. Chaucer paraphrased it as ‘Life is so
short, and the craft takes so long to learn’2.
While I can totally relate to that idea these days I didn’t always. At one
point I was a foolish person who thought they were good at what they did and
just slid along, month after month, year upon year not improving. That changed
several years ago when I was introduced to a bunch of new (to me) ideas and
techniques and the scales fell from my eyes and I realized how little I knew.
I feel I know so little sometimes; that my skills as a developer are limited.
I know that is not entirely true, my self-deprecation rearing its ugly head.
But it is a frequent feeling.
There is so much to learn and one’s life is so short; why then should I
even bother? I enjoy programming, and I, as part of my character, prefer to do
the best job I can. Thus I am left with no choice but to do everything I can
to improve my skills.
There is so much to learn and one’s life is so short; how then can I move
forward? I have been focusing on the basics (such as simple design and clean
code) - these will be skills applicable to any programming language I may use
or project I may be on. From this base I can expand into areas which are less
familiar to me (such as functional and logic programming) - these will broaden
the mind and help clarify existing skills in the familiar areas.
Whilst fixing some bugs in data file parsing code I found that the bugs lie in
the fact that the types of data files were more similar than not; but the code
said that they were more dissimilar than similar. Refactoring to make same
code same fixed at least part of the bugs on its own.
On my current project we have some code that reads data files and puts the
data into the database. The files come in different formats and layouts,
however there are only really two different kinds of files. I have always
maintained that the processing of these two types of files are more same than
different. i.e. How to read an excel file with the data laid out just so
and put that data into the database is all the same whereas the way to
validate if this line in the file a data line or something to ignore, or how
to split all the data into larger logical chunks (required before putting into
the database) is perhaps different between these two types of files.
I was very involved in the creation of the code for the reading of the first
type of data file; but not very involved for the second. Recently when we
shook out some bugs in the parsing of the second type of file I found myself
in the code for both types of files fixing the bugs.
I found that the code implied that the two files were more different than I
had imagined. Code that I thought would be similar or even exactly the same
was not. This difference could be seen on multiple levels; from the naming of
things up to the factoring of the code into methods and classes.
As I reviewed the bugs I had to fix I saw that the bugs were stating that the
second type of file was in fact not so different from the first type. As far
as the user was concerned much of the “rules” of parsing were exactly the
The first thing I did was make the code for the second type of file much more
similar to the first type. This involved renaming and refactoring. Then once
things were much more the same I extracted the sameness into the base class
(which had already existed). By doing this refactoring at least parts of the
bugs were fixed. All that was left was some tweaking to the code left behind
which defined the difference.
This coding really showed to me the importance of keeping a consistency in the
code. Things which are the same should be named and look the same. Things
which are different should NOT named the same or look the same; they must be
kept dissimilar. The compiler doesn’t care of course; but I think it will
help the programmer.
Not keeping to this ‘rule’ I think will lead to bugs where things which should
be the handled in the same or similar manner are mistakenly not since they
appear to the following developers to be different. Or even more simply
because the commonality was not extracted earlier because it was not noticed.
The way to keep this in mind is not not just think if the code is factored
well or has good naming; but also if the code you wrote is similar or
dissimilar to other code around it - and if that is correct or not.