Eppur Si Muove

I was reading the above article this morning.  The undercurrent of the article is that certain evolved systems grow in size while at the same time growing in efficiency.  Examples are animals and cities.  On the other hand, some systems grow in size while losing efficiency; not surprisingly, one example is mature companies.
My interpretation of the results was that strong evolutionary pressure, coupled with the continuous experiments in creative destruction are the prerequisites for growing a system that wont destroy itself in the long run.
There's also another type of system that tends to kill itself off as it grows; these are software systems and to counteract this tendency software engineers also introduce evolutionary pressure and creative destruction.
Unit tests are the technology that enables creative destruction, or refactoring in software talk.  A lot of talk promotes unit tests as a way to cut down on defects.  That's a lower level use of the technology.  A far more important use is to allow an engineer to take existing code and rework the design to support change.  Without units tests our only option for enhancing existing code is to kludge an addition in the safest looking place and hope we have understood all the side effects.
As for evolutionary pressures, software engineers often introduce those into systems they are working on in order to drive the creative destruction and force the systems they build to have superlinear (see the article above) growth characteristics.
Continuous deployment is a pressure that forces a system to evolve along the delivery axis.  I have read that Paul Graham keeps track of lines of code as he develops his Arc language.  He wants to create a succinct and expressive language and that's one metric that he uses to force his language to grow in fitness.  The developers of project Oberon wanted a compiler that was both powerful and simple so they forced themselves to add functionality to their compiler while maintaining limits for self compilation speed and self compilation object size.
The Haskell language is also an evolutionary experiment with corresponding pressures.  They wanted to see what software development innovations they could discover in an environment that would only allow functional, lazy programming techniques.  Im sure that decision has forced many discoveries but one that I'm aware of is Haskell's use of Monads.
If we think about our own software efforts we can also think of places where we can apply evolutionary pressure to our systems.  Making sure code passes automated checks like Checkstyle's metrics can help one to write code within complexity limits.  Lines of code limits or a tool like Simian would be a way to force systems into simplicity and expressive power.  Code coverage targets would encourage adaptability and robustness.  Performance testing like JMeter forces software to maintain computing efficiency. 
Tomorrow is a new week ---- what forces can you apply to your systems to force them onto a long term fitness curve.

Marco

Ive been coding in clojure for the past year in my personal projects.  I just thought I would review a bit about the language and my experiences with it now that I have several small and medium sized projects under my belt.
Im pretty happy with the language itself.  I have a repl for interactive programming.  I have adynamic language but I can add optional typing in.  It seems strange to me the no one ever mentions clojure's optional typing as it would seem like a big selling point in a dynamic language.  I like a lot that pre-conditions and post-conditions are part of the language.  With the last release, protocols were added to the language, so what was originally just a functional language has now become something of a object / functional language.  I like a language that has a philosophy but also offers multiple paradyms to developers.  And lastly there is good concurrency support and macros.  I have only used the last two features moderately.  Both are features are easy enough to use with macros, at least for me, requiring a bit more effort to work with.  IMO the above list is a fantastic set of features.
Another big plus working with clojure is that Im running code on a mature and fast virtual machine.  And I also have the entire ecosystem of java code out there that I can leverage in my work.  Calling java from clojure is well explained in the documentation for the language and once the syntax is explained its pretty intuitive.  Best of all everything just works.  The only trouble I ever had with java interop was in recent problem where I wanted to reference and inner class.  It wasn't explained in the documentation how to do that -- (one has to add a '$' between the inner and outer class names it turns out).
All of the above are to good things about clojure but not everything is roses and puppy dogs.  I have never found a way to debug clojure code that fully satisfies me.  On the main clojure.org site RH tells us he uses jswat.  I have never been able to use jswat and set breakpoints in clojure code though.  There is debug-repl which I have played with a bit -- that works...But in the end, what I find I end up using is the clojure.contrib.trace module to introspect my code.  Now I know there are IDE's for clojure now and perhaps their debugging functionality works well.  I use just emacs and swank so maybe others have a better way.  Also as a final note, among those of us that write unit tests there is the common perception that once one has tests one never has to use a debugger.  I dont buy it.  I just finished a first pass on an interpreter in clojure last week.  It has lots of tests but I still ran into two situations where it really would have saved me a lot of time to walk through the recursive calls in a nice debugger.  All tools have their usages.
My biggest project with clojure has been a web application.  I decided to use clojure / compojure server side, YUI as a javascript library, and couchdb for storage.  This was a learning experience on a lot of fronts and if I had to do it again I would probably skip compojure but its a hard call.  I would at least seriously consider if I could start with one of the existing java frameworks and build in clojure from there.  Here are a few things that I just couldn't warm up to enough about compojure.  It uses an old version of Jetty so I cant use things I might want to like web sockets.  The http request / session information is abstracted away into a map.  That works fine but in some cases I felt I didn't have the information a wanted and I didn't have the control I needed to do what I wanted to do.  Lastly the documentation is just ok.  I had to read the code to figure stuff out, especially when starting, and there are a lot of macros in there.  That means that, for people starting out reading, the code is not going to be all that intuitive.  Lastly, even starting with compojure, I found out that I had to build a lot of the infrastructure for a web application on my own.  The compojure / ring / hiccup modules all together give one
a set of routing procedures for urls and a way to write html in clojure.  The rest one does on ones own.  This is not ruby on rails or django.
One more note on web application development with clojure.  My original hope in using clojure for web development was that I could make changes on the server side and see those changes immediately on my web pages.  Naturally one can reload modules in clojure so this quickly becomes a natural part of development.  In reality, as my web application started growing larger and more complex, though I was able to reload modules,  I lost that ability to see code changes immediately.  In order to see changes I had to start rebooting and restarting.  These days I have less and less patience for that sort of stuff - especially when Im working in emacs.  Web development is already such a polyglot of languages and cluges and ajax and different browser behaviour; everything about web development seems to already conspire towards wasting time as it is.
Well thats my experience, good and bad.  Im glad that I have spent the time to learn clojure.  Like everything else its no magic bullet.

Marco