Brool brool (n.) : a low roar; a deep murmur or humming

A Modest Proposal

 |  clojure coding compojure php

Warning: a very rambling article; less a solid proposal than me just exploring an idea that may lead to a dead end in a week or two after I’ve thought about it.

I really enjoy Clojure. Everything seems so well thought out and well designed; in a lot of ways it reminds me of Python, which is three ways to ironic because back in the day I started using Python because it was very Lispy. The cycle of life turns even on computer languages.

But unfortunately, I get the sense that there’s probably only room for one JVM language to make it big. As much as I like Clojure, I think that Scala will probably be the one that succeeds. It’s got the static typing, it’s got the Java-like performance, and it’s got a syntax that’s close-ish to Java, so it fills the niche that Java fills so well: a programming language that is ideal for large groups of developers.

However, it’s early enough in the development of Clojure that there’s the sense that maybe individual efforts could make a difference. (As a counterexample, Haskell is too far along — it seems like everything has been done already, or anything that needs to be done requires a deep knowledge of category theory and the latest functional design papers.)

Assuming that I would want to help out Clojure, what could be done?

Donate? Actually, I did that. I figured that I got as many hours of enjoyment out of Clojure as I did out of a typical game, so I contributed the price of a game.

Libraries? Well, I’ve written one or two. Will probably write more.

If Clojure is going to succeed, it needs a niche, an area in which it performs so well that it blows everything else away. Concurrent programming is one possibility, but there is a lot of competition there.

I have a modest proposal:

Clojure should be more like PHP.

Yes, yes, I know that programming in PHP is socially reprehensible, just one step above people that shop without their shirt while in Walmart and car salesmen.

But what if we were to give Clojure the trappings that make starting developers try it? Metaphorically, it’s like Gulliver’s Travels; it looks like a humorous, light-hearted adventure story, but underneath it’s biting political satire and deep thoughts. We lure developers in with the possibility of an easy, light-hearted way to write web sites… and then once they’re in there, once they’ve progressed from manipulating web sites and start to realize the benefits of advanced languages, they can deal with a sane language, built on top of the JVM and with the full power of a Lisp.

So call this Gulliver.

Why Did PHP Succeed?

My impression, based on absolutely no citations or hard evidence, is that PHP became popular primarily because it is:

So, continuing the gedankenexperiment: if we were designing something that would be very approachable and very easy to use, what would it look like?

Making Compojure More Approachable

Compojure is an excellent base to start with, but it strays away from templates, and instead tries to do everything with compojure.html, a library that allows you to form everything with vectors and lists:

(html [:p "2 + 2 = " (+ 2 2)])

Handy? Yes, if you’re a developer. But if a web designer is doing your HTML, or if you’re just starting out programming and you already know HTML, though, it seems an additional barrier to entry. So what if we turn it inside out and do it like PHP does it?

2 + 2 = <?= (+ 2 2) ?>

Trying something more complicated, in compojure.html this:

(html [:ul (map (fn [x] [:li x]) (range 10))])

becomes:


  <? (dotimes [x 10] ?>
  <?= x ?>
  <? ) ?>

A little bit longer, but treating it as HTML makes it conceptually easier for beginning programmers to understand, especially if you want to do something like changing the id or class with every row so your CSS can make it all pretty:


  <? (dotimes [x 10] ?>
  "><?= x ?>
  <? ) ?>

Doing this is compojure.html is trickier:

(html [:ul (map (fn [x] [:li {:id (str "list-" x) :class (if (even? x) "even" "odd")} x]) (range 10))])

So, let’s lure people in with easy to use, and they can pick up vectors and S-exprs and notational convenience later. Converting a template file into the basic code is easy:

(defn- transform-string "Internal. Transform a string into the appropriate echo statement. An empty string translates into a single space." [s] (if (> (.length s) 0) (format "(gulliver/echo %s)" (pr-str s)) " ")) (defn- transform-code "Internal. Transform code so that it can run as a render routine." [s] (if (= (first s) \=) (format "(gulliver/echo %s)" (.substring s 1)) s)) (defn- convert* "Internal. Given a string that represents a Clojure template, produce a flat list of the statements necessary to produce that template." [s] (let [m (re-matcher #"(?s)(.*?)<\?(.+?)\?>|(.+?)$" s)] (loop [next (re-find m) buffer ["'("]] (cond (not (nil? (nth next 3))) (eval (read-string (apply str (conj buffer (transform-string (nth next 3)) ")" )))) :otherwise (recur (re-find m) (conj buffer (transform-string (nth next 1)) (transform-code (nth next 2)))) ))))

Resulting in…

==> (convert* "") ((gulliver/echo "") (dotimes [x 10] (gulliver/echo "") (gulliver/echo x) (gulliver/echo "")) (gulliver/echo ""))

Once you have the basic code generation, it’s easy enough to hook it up to a default servlet that automatically invokes it when necessary. There’s a fair bit of handwaving going on here — regexes are questionable for this, there are some interesting issues with namespaces, and it would require some method of resolving dependencies between files, but heck, it’s only 100 lines of code; it was really quick to put together and gives us a sense of whether this can work out.

So, with the source on github, your basic server becomes…

(use 'gulliver) (defserver web-server {:port 8080} "/*" (servlet gulliver/template-servlet))

Easy! As an test, an example file:

Delivered via Gulliver Expressions at work: Looping: Here's a date:

… does the right thing. How about a time test?

Starting time test... Adding 1..100000: (took ms)

… runs in about 50ms. PHP takes about 130ms on the same sequence on my laptop, so, while there are tons of optimizations we could do (like not doing a stat on the file every time the page is rendered (!)), it nonetheless performs well enough that maybe this is a viable option. Finally, as a special bonus, a stupid eval trick that evaluates Clojure code:

<? ) ?>

Take that, non-dynamic languages!

Github copy of the source

.

Next part: Schema-ing Against MySQL

Discussion

Comments are moderated whenever I remember that I have a blog.

Shantanu Kumar | 2010-09-01 18:04:13
@tim This is great. I would suggest that you separate this out from compojure and servlet stuff. The reasons are: 1. People can use it for template processing and offline doc generation. 2. Web frameworks other than Compojure can use it, thus fueling innovation. I would be eager to know when/if you factor this out into a compojure/servlet independent template library.
Reply
Stephen | 2011-05-13 11:17:57
I can't believe there is more of a movement out there to make Clojure more designer-friendly. Sad to see the project didn't take off in the past couple years, but I'm hoping folks will start to realize that this kind of thing is needed. The project was a great start, though! If more effort gets put into it, I will definitely try it on some big projects.
Reply
Stephen | 2011-05-13 11:18:26
Erg.. major type-o -- I meant to say, "I can't believe there isn't more of a movement"
Reply
Antti Rasinen | 2009-09-01 20:25:22
I think Clojure is already "made it". It seems modern and fresh in the sense Arc failed to be. Clojure has the potential to become the biggest Lisp around. That is an achievement in itself. To make it in the real world is of course a different matter. There may be truth in the statement that Clojure needs a niche. PHP is a good analogy. Other "Worse is better" examples apply too. But the rest of your article seems to describe PHP made out of Lisp? That is such a bad idea on at least three levels. 1. Is HTML templating truly an area where Clojure could shine? 2. Web frameworks and HTML templating are no niches anymore. These days you can't swing a dead cat without hitting a web framework. Heck, you can't even pick up the cat. 3. (Related to #1) Trying to ape another API or programming model in a whole different programming paradigm will lead to poor implementations and un-idiomic code. It'd be great to see a popular Lisp and I hope Clojure makes it. But I do hope it makes it on its own merits. I'll make one prediction here. The niche, whatever it will be, will be a field where current programmatic solutions involve lot of repetition or drudge work. The Clojure/Lisp solution will remove that drudge. (Think Rails and CRUD, for example.)
Reply
tim | 2009-09-01 21:29:37
@Antti: I hope you're right -- I really think highly of the language. It just seems like Scala has more momentum in the shops that are already using Java; it's possible that Clojure could find success outside of established Java shops, but it's a harder sell, maybe. It's possible this whole exercise might be a bad idea -- it's kind of a work in progress -- but nonetheless I think that making Clojure web apps drop-dead simple is not inherently bad. Yes, there are 300 advantages to clojure.html, but it looks scary to developers: "What, I don't even get to use HTML, I have to use <i>another</i> language?" So why not make it more approachable?
Reply
Cliff Wells | 2009-09-12 02:54:33
Your arguments echo many I've heard in the past with regard to Python. In fact, it was possible, using mod_python and some horrible PHP-like languages to get pretty close to the PHP-experience". In the end, almost everyone rejected these approaches as they create horrible-looking mish-mashes of the two languages (yes, I consider HTML a language). I think that while a good segment of the market still embraces the PHP way, we're seeing a migration away from this type of development (even in the PHP world). I don't think that reinventing it in any language is a good way to promote that language, especially when the language itself tend to attract the more esoteric crowd. It's a bit like putting sweats on Michelangelo's David. You might attract a certain type, but you'll probably alienate your real target audience. I was personally attracted to Clojure for the following reasons: 1) functional language features 2) comprehensive and modern libraries, thanks to Java integration (but without the pain of Java) 3) modern MVC web framework being developed using it (arguably a sub-category of (2) but important enough in its own right to qualify as a separate point) 4) S-expression based HTML generation (yes, I much prefer this approach, see http://breve.twisty-industries.com for my previous attempt to bring this type of functionality to Python). We've seen widespread migration (largely thanks to Rails) away from the PHP way. I don't think there's a future in it and it appears the world at large is starting to agree.
Reply
Greg | 2009-10-09 20:03:56
Tim, thank you for doing this, I was afraid I'd have to do this from scratch myself, but now that you've done this it makes things much easier, *and* I'd be happy to contribute to the project as well. I completely agree with your conclusion, we need a great PHP-style system to compliment Compojure, and this seems to be it (unless there have been other attempts that I'm not aware of). I look forward to how this progresses, great job!
Reply
Greg | 2009-10-09 20:08:50
BTW, I can't believe you're getting naysayers, this sort of stuff is essential for making great dynamic website (IMO), especially when *designers* are making website -- they're used to the PHP-way of doing things, and personally I don't see any problem mixing HTML with another language, I don't think it's ugly at all, but as Cliff pointed out, there are plenty of people who do. To each his own, thanks again!
Reply
Dan | 2009-12-01 19:38:46
I think the most fundamental place where Clojure can make a serious contribution is in the sciences. Currently, code is not treated as data in the sciences, and this is a problem. Much of the new code in my area (machine learning / neuroscience) is written in Python, where code is most definitely not data. This makes it incredibly difficult to write programs that can keep track of how different data sets were arrived at. When I transform a data set in some way (usually a whole series of them), I want my operations to be published with the new data set. I want them to be published in a format that is executable, easily legible, and most importantly easily parsable - I want to be able to use other code to reason about these transfomations, try new ones, etc. etc. In other words, I want to use the code as data. Clojure (especially with Incanter), offers the possibility of doing this. I can write programs that compose programs, reason about them, search through them. I can use the same language to write and read each layer of program. I can fairly easily distribute my programs over the cluster thanks to integration with Cascading and Hadoop.
Reply
John Hunter | 2010-02-06 02:53:15
I think this is a great idea ! There is something satisfying about getting instant feedback as in php. There's no reason something like this "Clojure Server Pages" and compojure couldn't both be useful. Maybe you begin with this and later head into compojure. Did anything ever happen with this, Tim? Cheers, John
Reply
Corwin Jensen | 2010-06-15 17:06:25
<i>I think that while a good segment of the market still embraces the PHP way, we&#8217;re seeing a migration away from this type of development (even in the PHP world). I don&#8217;t think that reinventing it in any language is a good way to promote that language, especially when the language itself tend to attract the more esoteric crowd. It&#8217;s a bit like putting sweats on Michelangelo&#8217;s David. You might attract a certain type, but you&#8217;ll probably alienate your real target audience. I was personally attracted to Clojure for the following reasons:</i> +1
Reply
Add a comment