A few weeks ago I was playing around with a new stack: Node-Express-Angular all in CoffeeScript¹. I found it quite easy to have the Express server compile my CoffeeScript files and then return the JavaScript to the browser. I then found myself last week in a position at work where I wanted a little test page to hang the Angular directives I was working on so I quickly used the same trick.

I’m not saying this trick is bullet-proof, or even a necessarily good idea for production. But it seems like a good thing for testing and perhaps for getting a project off the ground with some prototyping. It also serves as a small example of how to write an Express Middleware².

With that caveat I will present you with the code and explanation of it³.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

express = require 'express'
coffee = require 'coffee-script'
fs = require 'fs'

app = express()

app.configure ->
  app.set 'port', process.env.PORT ? 3000

app.use express.logger()

app.use '/client', (request, response, next) ->
  coffeeFile = path.join __dirname, "../client", request.path
  file = fs.read coffeeFile, "utf-8", (err, data) ->
    return next() if err?
    response
      .contentType('text/javascript')
      .send coffee.compile data

app.listen app.get('port'), ->
  console.log 'listening on port %d', app.get('port')

The part that concerns us are lines 12-18. This section is telling Express that if a request comes in with a path starting with /client that we will want to do some special processing on it.

• Line 13 determines the path to the CoffeeScript file (which is in the ../client directory relative to our current directory). Of note here is that request.path does not contain the /client part of the path and __dirname is the directory of the currently executing file.

• Line 14 sets up the reading of this CoffeeScript file. It is important to specify the encoding otherwise the Coffeescript compile will throw an error.

• If there is an error reading the file (line 15), such as the file not existing, then we tell express to run whatever is the next rule (eventually if no other rules work Express will send a 404 for us).

• If we can read the file then we use CoffeeScript to compile that file (line 18) and send that data back as the response (lines 16-18). (Of note here is that we set the content type to be text/javascript (line 17) to make sure that the browser does the right thing (it doesn’t appear necessary with Chrome at least, but probably best to do it.)).

Congratulations, we’ve just written an Express Middleware!

When there is only one way of doing things, it is easier to
modify and reuse code. When code is reused, programs are easier
to change and most importantly, shrink. When a program shrinks
its construction and maintenance requires fewer people which
allows for more opportunities for reuse to be found. Consistency
leads to reuse, reuse leads to conciseness, conciseness leads to
understanding.

Resuse

I like the meaning of ‘reuse’ in this quote. This is not the ‘reuse’ promised by early OO proponents. They promised¹: that one would be able to create objects which would be reused from one project to another. This is the reuse which means that code is reused in a project. This is the reuse which I believe to be attainable.

It is like a inward-spiral: the more reuse of code the fewer ways to do a thing there will be which leads to only one way.

The sentence about fewer people is a bit odd - but I feel it is does make sense. The fewer people on the team the more likey there will be only one way to do things which means there is by definition more reuse.

Consistency

Perhaps more important than reuse though is consistency. If code is consistent I have found it to be easier to reason about, because reasonable assumptions can be made when reading it. Being able to make reasonable assumptions, and have those borne out, is very important to me when reading code.

Conciseness and Understanding

And it follows that with reuse and consistency one will get consiceness. But does that necessarily lead to underderstanding? That is something I feel is right, but I have seen some concise code which is hard to understand.

References:

• “Programming as Experience: The Inspiration of Self”, 1995, Smith, Randall B and Ungar, David

While avoiding family this Christmas-day I started trying to figure out way for me to do TDD in Common Lisp (as one does…). That led to SLIME¹ and QuickLisp² and lisp-unit³ and asdf⁴ (&c. &c).

…it’s yaks the whole way down.

The hardest part for me was to find a simple way to load my code and run the tests. ASDF ships with asdf:test-system defined but it does nothing by default⁵. So I did some digging around and saw some other’s solutions and synthesized my version.

Here is an example ASDF system definition for a Game of Life⁶ implementation.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

(asdf:defsystem #:gol
  :description "Conways Game of Life in Lisp"
  :author "Mark Simpson <verdammelt@gmail.com>"
  :version "0.0.0"
  :depends-on ("lisp-unit")
  :components ((:module "src"
			:serial t
			:components ((:file "package")
				     (:file "gol"))
                (:module "test"
			:serial t
            :depends-on ("src")
			:components ((:file "package")
				     (:file "gol-test"))))
  :in-order-to ((test-op (load-op :gol)))
  :perform (test-op (o c)
		    (progv
			(list (intern "*PRINT-ERRORS*" :lisp-unit)
			      (intern "*PRINT-FAILURES*" :lisp-unit))
			'(t t)
		      (funcall (find-symbol "RUN-TESTS" :lisp-unit)
			       :all :gol-test))))

My God… it’s full of yaks

I’m just going to let that sink in for a bit while I get some coffee.

Ok, now that I’m back I will explain each part. The first interesting bit is on line 6:

1

  :depends-on ("lisp-unit")

This line tells ASDF that it should find and load a system called lisp-unit because our system depends upon it. This is where QuickLisp makes things simple since that is its job and it does a fine one at that.

Next is the definition of our components:

1
2
3
4
5
6
7
8
9

  :components ((:module "src"
			:serial t
			:components ((:file "package")
				     (:file "gol"))
                (:module "test"
			:serial t
            :depends-on ("src")
			:components ((:file "package")
				     (:file "gol-test"))))

I’ve decided that my project should have two main directories, one to contain all the production source code and one for all the test source code. Each one will be in their own packages, with the :gol-test package :use-ing the :gol package. :serial t by the way tells ASDF that each item in the component list depends upon the one before it.

Now the hairier parts which tell ASDF to run our tests:

1
2
3
4
5
6
7
8

  :in-order-to ((test-op (load-op :gol)))
  :perform (test-op (o c)
		    (progv
			(list (intern "*PRINT-ERRORS*" :lisp-unit)
			      (intern "*PRINT-FAILURES*" :lisp-unit))
			'(t t)
		      (funcall (find-symbol "RUN-TESTS" :lisp-unit)
			       :all :gol-test))))

The first line is simply stating that in order to do test-op ASDF needs to perform load-op on the :gol system. This way every time we run the tests we’ll reload the system if needed.

Now… this next bit probably shows how rusty my Lisp is. Maybe there is an easier way. Basically we want to run (lisp-unit:run-tests :all :gol-test) with lisp-unit:*print-errors* and lisp-unit:*print-failures* both bound to t (they default to nil and that doesn’t give enough info IMNSHO). However when Lisp is reading and evaluating this code lisp-unit has probably not yet been loaded - so we have to be crafty and find the symbol by name in the package then funcall it. Binding the variables was trickier but I stumbled upon progv⁷ and that seems to be just what I needed.

progv creates new dynamic bindings for the symbols in the list which is its first argument. These symbols are determined at runtime. That means we can use intern to find or create these symbols in the right package⁸. progv binds these symbols to the values in the list which is its second parameter and then finally executes the sexp which is its third argument in an environment which contains these new bindings.

With this work done I can now type in (asdf:test-system 'gol) into my REPL and it loads and runs my tests and gives me useful output of those tests. I might go so far as to bind that to a very short function name, or even a key in Emacs.

There are still more yaks to shave.

What is left to be determined is if this is the best way to do things.

Some documentation/blogs I read lead me to believe that putting the tests in the same system with the production code is verboten (or at least a no-no). I tried having the tests be in their own system definition (depending upon the production code) but then ASDF & QuickLisp both strongly pushed me to have that definition it its own .asd file⁹ which seemed awkward to me. I personally have no problem shipping tests with the production code, and I believe that if needed all the symbols of the test package could be unintern-ed before an image was saved if desired.

I’ll play around with this setup and see how things go.

(How It Ought To Be Done)

IMNSHO¹.

Being able to read source code² is very important. So how does one do it well?

Just like reading any other material - first decide why is one reading it. Why is one reading the listing? Is one debugging a problem or trying to gain specific knowedge?; or reading to get general or pleasure? One takes different approaches depending upon the desired goal.

The first step is to find an interesting spot to start.

For Debugging (or to Gain Specific Knowledge)

Start by skimming of the code, grep³ and tags⁴ tables are useful for this. This will help one find the right, or at least likely entry point. There is a lot of instinct⁵ involved in this step. One gets better at it the more one needs to do it⁶.

To Get General Knowledge (or for Pleasure)

Code is for reading⁷ so read it. Skim the code (including the tests⁸). Look for something interesting, get the lay of the land, the shape of the code. Note the Dramatis Personæ. Perhaps the code seems to be about widgets and frobbing, oh and here’s something about twiddling that looks interesting.

Pick one of these that is most intriguing (e.g. One wonders what happens when a widget is frobbed.) and use that as one’s entry point. This is a matter of personal taste/fetish⁹.

Now that One has One’s Entry Point…

Now that one is staring in the face a chunk, hopefully a small chunk, of code; how does one read it?

Top Down. It is as simple as that¹⁰.

Read the first line of code, what does it say? Believe it. That can be risky - but at this point one must do it. What does the next line say, and so on. Follow function calls¹¹ if they seem interesting. Skip over things that are not, or do not seem relevant. If one’s interest is in frobbing ignore code about tweaking or twiddling. Perhaps one makes a note that frobbing seems to involve tweaking or twiddling; but one must know about frobbing now so those are not relevant. This is also very tied to instinct.

It is this point here which I think is most important. One must gain a sense of what is relevant when reading code. One cannot read every code path and think that will lead to understanding. That would be like reading the dictionary and assuming one knows the language. This is where I have seen programmers fail in their reading.

This is a very good reason why naming is so important¹².

Experts Don’t Know what they Know

This year’s SCNA Conference seemed to have several talks which touched upon the idea of experts having implicit knowledge. That is to say: they know things but can’t necessarily explain what they know, or why. Ken Auer and Dave Thomas touched upon it; during the panel on Software Quality it was also brought up (e.g. Corey Haines’ amazing reference to Chicken Sexing). But I felt that Brian Marick’s talk took the idea and then added to it.

Bright vs. Dull Cows

Brian started talking about this implicit knowledge idea - giving his wife as an example. His wife (a veterinary professor) teaches her students to determine if a cow is ‘bright’ or ‘dull’ (a subjective diagnosis). Her students specify, day-after-day, if cows are dull or bright. She then corrects them. They usually ask how she could tell and she tells them something (drooping ears, nose-cleaning, etc.). But actually these specific clues are not really the way to tell. Those who can diagnose a cow as bright or dull cannot explain why.

This anecdote seems to show that the implicit knowledge of the expert is something that can be trained. When discussing this idea after the talk Corey Haines mentioned a conference session in which Michael Feathers gave where he had the attendees give quick good/bad judgment calls on snippets of code. Then tallied those up and found that some snippets had a large amount of agreement as to their good-/bad-ness. They then discussed and tried to determine why. In light of Brian’s anecdote however it seems that they might never find out why, but maybe they could train themselves by looking at a lot of code and being corrected when they get the judgment wrong.

(Brian hopes to use this sort of idea to be able to train himself to have a nearly physical reaction to bad code. While that sounds cool, I worry about a ‘Ludovico Technique’ for code.)

Effortful vs. Automatic

Brian then talked about two types of thinking: Effortful and Automatic. Effortful is the sort when you need to do a non-trivial arithmetic, or a parking in a tight spot. Automatic is when you do simple math like 2+2 or easy driving where you are sort of on autopilot. When you use the Effortful way, it actually takes effort. It is slower seems to be only possible to do it serially. Automatic thinking is faster, takes little effort, and seems to be parallelizable.

If you don’t have something in Automatic then your brain shifts to Effortful; sort of a cache miss. If you do something enough, or become good enough at it, you can put stuff into your Automatic cache.

Switching to Automatic

So how can this be done? While Brian offers the usual ‘practice, practice, practice’, he adds to it an interesting idea about ‘Set and Setting’. ‘Set and Setting’ is an idea from Timothy Leary about why drugs such as LSD seem to have very different effects in different people. The mindset (Set) of the person (which includes things like rituals which build up that mindset) and the Setting (environment) have a strong influence on the experiences of that person.

By being aware of which mindsets (and how to enter them) and what settings are most beneficial for programming (and even different types of programming) he feels he can move some programming tasks into his Automatic thinking area.

Conclusion

It seems to me that the idea of ‘Automatic’ and ‘Effortful’ adds to the idea of the expert’s implicit knowledge. Perhaps the expert’s knowledge is just an expression of Automatic thinking?

I’m interested in trying out the practice of being more aware of my own ‘Set and Setting’ with respect to code.

I am once again jumping into the Clojure world and trying to learn a bit of it. I am doing so via a small project called defclink (source). I am also attempting to get back into blogging. So I thought I’d combine the two items into this short post.

Goals of my project.

My goal of this project is to learn some Clojure building a webapp. I decided to also use Clojurescript since it seemed like an interesting idea. I wanted to keep the number of libraries and frameworks low to maximize the Clojure code I would need to write, thus learning Clojure. That being said I allowed myself the luxury of the Compojure library (with Ring under it) to handle the HTML protocol stuffs.

I’ll start by explaining how to get a brand new Compojure project.

How to set up a Compojure project

Starting a Compojure project is very simple:

lein new compojure awesome-thing

This will create:

README.md    project.clj  resources/   src/         test/

./resources:
public/

./resources/public:

./src:
awesome_thing/

./src/awesome_thing:
handler.clj

./test:
awesome_thing/

./test/awesome_thing:
test/

./test/awesome_thing/test:
handler.clj

and furthermore will setup your project.clj file to pull in compojure.

It sets up simple tests that going to “/” will result in a success page with “Hello World” and that going to a non-existant page will result in a 404. There is a defined set of routes to accomplish this.

To run the server at this point it is easy to do:

lein ring server

and then go to localhost:3000 to see the awesome_thing

Heroku?

At this point you probably want to push this to awesome thing to Heroku to start get VC funding. This is easy to do (the pushing part).

First create your heroku app in the usual manner (at the time of this writing you don’t even need to choose the cedar stack - but check the docs to make sure you are Clojure compatible).

Now create a Procfile with the following contents:

web: lein with-profile production trampoline run -m defclink.main $PORT

Now you should be able to push to Heroku and sit back waiting for the funding to roll in.

“When you find yourself in a hole, quit digging” – Will Rogers

Sometimes I wonder what the important skills for a developer are. I now believe that one of the important ones (perhaps most important) is knowing how/when to give up and start again. This contradicts another important skill/trait for a developer to have: tenacity. But I think they are balancing forces with giving up being slightly more important. Let me explain.

Often when working on a problem, it takes longer; is more complicated and difficult than expected. At this point there are many files modified and it looks like there are many more to modify; plenty of broken tests, maybe an unknown quantity. There are two options: keep going, or give up.

This is the point I like to remind myself (and my coworkers) of Will Rogers quote. Another thing I do is ask “Are we digging a tunnel, or a hole?” If we think we are digging a tunnel then we can give ourself a time limit for further digging; if it pays off – great! otherwise we need to climb out of the hole.

I used to believe, quite strongly, that tenacity was more important. To persevere through the mountain of outstanding changes and get it done was admirable. I now believe that giving up is the better option. What changed my mind? I now see more benefit in taking the knowledge acquired from the work done and reapply it to the problem from a fresh start. Powering through the situation is deciding at the current way is the only way; starting again is deciding that there is a different way, with the knowledge that the current way has giving you (which is at least that the current way isn’t working out).

As a footnote I want to add that I also thinking giving up in another way is important. When working on something collaboratively (such as when pairing) one can find oneself at odds with another developer about how to proceed. I believe that unless the idea is very obviously bad that to stop arguing/discussing and do something is better that spending time discussing. Especially when there is source code involved - any code is better than no code. Any code, can then be changed into better code.

In the interest in breaking my current blogging writing block I’ll record for posterity my ‘recipe’ for chicken soup.

First you need to eat some chicken so you have bones. I usually get rotisserie chickens for another recipe (Skillet Chicken Pot Pie) and then save the bones in the freezer. I use two carcasses for this recipe.

So take the bones and entomb them in a big pot with some carrots, celery, garlic and onion. Don’t worry about chopping up the vegetables or even peeling them - this step is making stock.

(The clever ones among you will note that I didn’t tell you how much of these things to put in. I am doing this in honor of my mother on this Mother’s Day.)

Now put in enough water to drown it all…

and bring it to a boil.

Let it boil for as long as you can stand to. You want it to get all the good tastes out of the chicken bones and vegetables. I usually let it go for 2-4 hours and stir it occasionally. I don’t let it boil down more than half-way however.

At this point it should be filling up your house with a very delicious aroma.

After boiling it down you need to strain out all the solid bits and throw them away. I have not included a picture of this because it looks rather unpleasant and might put you off the recipe. But take my word for it - this bit of unpleasantness will be worth it.

What you have now is chicken stock. Do with it what you will!

I usually choose to turn right around and create soup. I now put in carrots, celery, onion, garlic, maybe peas. These, since you will be eating them should be peeled, cleaned, etc.

I usually, because I am lazy, add a pasta to this soup, but potato or rice could be used instead.

Bon Appetit!

Introduction

As everyone that knows me knows, I have become enamored with cocktails.

So I thought, with that amazing hubris encapsulated within the novice, I would tell you, the reader, how to stock a bar if you are new to cocktails yourself.

I’ll split this amateur’s amateur guide into the following sections:

• The Equipment
• The Drink
• The Practice
• The Expansion

The Equipment

I will start by listing the equipment you need. This will seem like a lot; but you can find some of it in ‘kits’ which will help you get started (replacing crappy items as you realize that there are better tools).

• a jigger¹
• a shaker set. (I prefer a Boston Shaker²)
• a strainer (such as a Hawthorne Strainer³)
• a bar spoon⁴

This seems like a lot of stuff - but like I said you can find it in ‘kits’. If you can get better tools from a specialty store (such as The Boston Shaker).

The Drink

You should start with one or two drinks you like and that have only a few ingredients. For example a Manhattan or Negroni only have 3 ingredients; a Gin and Tonic only two (three if you count the lime). Pick drinks you really like because you will be making them a lot. It also helps if you pick drinks with ‘standard’ ingredients, like Gin, Vodka, Rye, Bourbon, Whisky, Campari etc. This makes it easier to buy these ingredients from your local liquor store.

The Practice

Now that you have the equipment and the ingredients; make yourself a drink – you deserve it! Then make it again, and again, and again…

As you practice, you’ll figure out how to measure (and when you don’t care about measuring), you’ll figure out how to stir or shake well, how much ice you might need etc.

The Expansion

Once you have gotten good a drink (or perhaps sick of it) look for a drink which has all the same ingredients but in different proportions; or a drink that has a different ingredient you don’t have yet.

In this way your repertoire will grow and your skill will as well.

Hippocrates noted that there was so much to his craft to learn, but only such a short life to learn it in¹. Chaucer paraphrased it as ‘Life is so short, and the craft takes so long to learn’².

While I can totally relate to that idea these days I didn’t always. At one point I was a foolish person who thought they were good at what they did and just slid along, month after month, year upon year not improving. That changed several years ago when I was introduced to a bunch of new (to me) ideas and techniques and the scales fell from my eyes and I realized how little I knew.

I feel I know so little sometimes; that my skills as a developer are limited. I know that is not entirely true, my self-deprecation rearing its ugly head. But it is a frequent feeling.

There is so much to learn and one’s life is so short; why then should I even bother? I enjoy programming, and I, as part of my character, prefer to do the best job I can. Thus I am left with no choice but to do everything I can to improve my skills.

There is so much to learn and one’s life is so short; how then can I move forward? I have been focusing on the basics (such as simple design and clean code) - these will be skills applicable to any programming language I may use or project I may be on. From this base I can expand into areas which are less familiar to me (such as functional and logic programming) - these will broaden the mind and help clarify existing skills in the familiar areas.