"... a computer language is not just a way of getting a computer to perform operations, but rather ... it is a novel formal medium for expressing ideas about methodology"
Abelson/Sussman "Structure and Interpretation of Computer Programs".

"Conceptually FORTRAN remained on familiar grounds in the sense that its purpose was to aid the mechanization of computational processes we used to do with pen and paper (and mechanical desk calculators if you could afford them). This was in strong contrast to LISP whose purpose was to enable the execution of processes that no one would dream of performing with pen and paper."
Dijkstra "Keynote address to be given on 1 March 1999 at the ACM Symposium on Applied Computing at San Antonio, TX"

Index

  1. what is a good language?
  2. why good people use bad languages?
  3. performace issues
  4. catchy quotes
  5. relevant links

The Right Tool For The Job

My programming language tastes are heavily tilted towards the Lisp family of programming languages. Yes, I like recursion (although I use it only when necessary), but this is not the only reason. My requirements to a high-level programming language are five-fold:

[1]Memory management:
I don't want to mess with memory allocation, so the language must provide some sort of garbage collection; also, I don't want to get segmentation faults, so no pointers should be present either (well, you cannot have garbage collection if you have pointers anyway).
[2]Objects:
the language must be OO. As Paul Graham wrote in his book ANSI Common Lisp: With macros, closures, and run-time typing, Lisp transcends object-oriented programming. The generic function model is preferred to the message passing one since it is more general and functional (in both senses of the word).
[3]Egalitarianism:
everything in the language must be a first class object, including functions (including the ability to create a function on the fly and return it as a value containing persistent local states, so closures are a must), structures, arrays etc (none of the three objects mentioned is first class in C, and only structures, renamed "classes", are first class in C++ and Java).
[4]Library:
I do not want to re-invent the wheel. The language must have hash-tables, lists, arrays, conditions &c standard.
[5]Introspection:
The program must be available as data, with as little structure as possible, but no less that that. Lisp is perfect - a program is a list. Perl and Tcl use strings - too little structure: a string is not a unit of evalutation or compilation; OpenJava offers too much structure.

If you know of a language outside the Lisp family that satisfies all of the above requirements, please tell me! Most of the examples you will be able to come up with (like Dylan, Smalltalk, Haskell and ML) will be functional languages heavily influenced by the Lisp heritage.

Another nice looking language is Python. The biggest problem with it is that it is defined by its reference implementation, which does not do native code compilation (only byte-code compilation).

Yet another nice looking language is Ruby. It suffers from the same problems as Python (except that it does not even have a byte-code compilation).

Some people use TCL. Since it is even worse than Python, what else do I have to say?

There are probably more languages du jour defined by their unique implementations. Just ignore them.

See also why OCaml Language Sucks.

The Swiss Army Chain Saw - All the stupidities of UNIX in one!

The over-hyped Practical Extraction and Report Language - Perl is praised for its very powerful regular expressions and a good system interface (thus it is supposed to be very convenient for system administration work). There are three major problems with it:

  1. It is burdened by a bizarre syntax, making it a write-only language: it is virtually unreadable (and people love and perpetuate this!), and no editor can be relied upon to auto-indent and highlight the syntax correctly. Moreover, Perl itself cannot handle its own syntax properly: often it will accept a "scalar" (like $var) and barf on a list member (like $vector[$index]), such as when you try to give print the first argument.
  2. Far too much of the implementation is exposed to the programmer, making it very hard to write an optimizing compiler; this is probably the bane of the languages defined by their unique implementations. Interestingly, some implementation details are unjustly hidden - linked lists and vectors are the same in Perl despite their obvious and vast differencies (put it another way, linked list-specific operations, like splice, are available for vectors, creating a false impression that they are fast).
  3. Perl is Object-Disoriented: by sticking the value type into the variable name it offers the "gurus" a wonderful opportunity to answer questions like "how do I create an array of file-handles" - as if an array of file-handles were conceptually different from an array of numbers.

If you are about to say something like "I really want to use a real language, but I am stuck with Perl because I need its regular expressions", you should try the CLISP regexp module. It supports the full strength of POSIX regexp syntax, including {} etc:

CL-USER[1]> (use-package :regexp)
T
CL-USER[2]> (match "\\(abc\\)\\{2,4\\}" "abcabcabc")
#S(REGEXP:MATCH :START 0 :END 9) ;
#S(REGEXP:MATCH :START 6 :END 9)
CL-USER[3]>
(let ((h (make-hash-table :test #'equal :size 10)) (n 0))
  (with-open-file (f "/etc/passwd")
    (with-loop-split (s f ":")
      (incf (gethash (seventh s) h 0))))
  (with-hash-table-iterator (i h)
    (loop (multiple-value-bind (r k v) (i)
            (unless r (return))
            (format t "[~d] ~s~30t == ~5:d~%" (incf n) k v)))))
[1] "/usr/local/bin/tcsh"      ==     2
[2] "/bin/sh"                  ==     1
[3] "/bin/ksh"                 ==     2
[4] "/usr/local/bin/bash"      ==     4
[5] "/bin/csh"                 ==    21
[6] "/usr/lib/uucp/uucico"     ==     1
[7] ""                         ==    11
[8] "/sbin/sh"                 ==     1
NIL
CL-USER[4]>

If you think you must have the Perl regexp syntax, you can use the CLISP PCRE module. Alternatively, you can use Dorai Sitaram's pregexp which is a Perl regexp implementation for both Lisp and Scheme. Another option is CL-PPCRE - a portable regular expression library for Common Lisp by Edi Weitz.

So, why is Perl so popular?

A Swiss Army Knife is a perfect tool for many small jobs. Nevertheless, one would hardly expect a professional plumber or a car mechanic to use it on the job. There are better tools for these jobs, right?

This is not the case in the area of software development. People start learning the trade with toy tools, like Basic, and then they are trying (and succeeding!) in keeping them forever, just because they know and love them.

Another aspect is a self-defeating good idea, like the truly brilliant idea of RMS to make a powerful programming language (as opposed to a "scripting language") the extension language to Emacs. The idea is that whenever an extension language is present, the users will use it to the full extent, so if the language is not "Turing complete", the users will soon be upset. (In a recent interview, RMS explains, inter alia, what makes Lisp such a good development platform.)

The result was that people started writing everything in this language, Emacs-Lisp, including a full-fledged news-reader, a WWW browser, a computer algebra system and a spreadsheet. The problem is that Emacs (the implementation, not the Emacs-Lisp language!) is heavily tilted towards editing operations, and not particularly good with general-purpose computations. It is time to make the next step: from using an "extension" language (like the guile) to extend applications to a full-fledged powerful general-purpose language with a good library (I hint in the direction of ANSI Common Lisp here, but I would settle for any similarly powerful language). With shared libraries, it doesn't matter whether your applications are extended with a 1MB libguile.so or 2MB libclisp.so, but it will matter for the users whether they will have to reinvent hash tables, structures and arrays. Yeah, I know, you don't need full Common Lisp in your window manager, but you will not have a separate copy of CL for each program! The same libclisp.so should be used by the window manager, the editor, the shell &c.

C gives yet another good example, similar to Emacs-Lisp. Essentially, it is a very good assembly-like language, designed for UNIX kernel and utilities development - so we can call it "UNIX extension language", like Emacs-Lisp is the Emacs extension language. But eventually C was used to write large programs (word processors, computer algebra systems, computer-aided design systems, etc). C is a totally inappropriate language for large projects: it lacks all the aspects of a high-level language, which means that the programmer has to design a work-around for all of them. C++ is a slight improvement in some areas, but only a slight, and at a great cost of enormous complexity (see The Anatomy of the Assignment Operator and The Assignment Operator Revisited for examples). Java is better, but it is still strongly lacking in functionality.

A high-level powerful advanced language is more appropriate for many tasks where C/C++/Java are employed now, but there are plenty of people with a passing knowledge of C/C++/Java and only a few with a grasp of Lisp (this is actually a chicken-and-egg problem.) My estimate is that an average task requiring 6 months and 10 C/C++/Java programmers at $60,000 a year each would require 3 months and 3 Lisp programmers at $100,000 a year. But the businesses are so conservative...

Fortunately, there are some signs of (modest) progress. The following mainstream applications are written in Common Lisp:

There have been many failed efforts to re-implement the LispOS. It is nice that people are still thinking about it, but it is a pity that they just talked and wrote no code. By the way, you can help make a difference!

Performance

I can hear you cry: "But Lisp is SLOW!" You are wrong. Lisp is a family of languages, it cannot be slow. Languages (and their families) are not slow. Implementations are.

The longer answer goes along the lines of the Pot Principle.

Remember 20 years ago, when people were writing in Assembler because C is slow? The history repeats itself, but this time it is Lisp versus C/C++ instead of Assembler versus C.

Greenspun's Tenth Rule of Programming: any sufficiently complicated C or Fortran program contains an ad hoc informally-specified bug-ridden slow implementation of half of Common Lisp.

Java Is Dead - Buried Under Its Own Hype

Now (Dec 1999) that Sun has withdrawn Java from the standards process and slapped a "Java branding royalty", we can say that Java is dead as an open hardware-independent development platform. Lisp, on the other hand, having been standartized by ANSI and being offered by several commercial vendors, such as Franz and XANALYS (formerly known as Harlequin), is such a platform, and it has Corba, ODBMS access, GUI, IDE and whatever else you might think of.

The bottom line

Lisp is a dynamic language - it grows up to meet your needs.

John Foderaro, as quoted by Paul Graham in On Lisp:

Lisp is a programmable programming language.

Kent M. Pitman <kent@nhplace.com> in <Message-ID: sfwher7c9d4.fsf@shell01.TheWorld.com>:

Lisp is a language for doing what you've been told is impossible.

See Lisp History for the proof of the surprising fact - there is more literature on Lisp than on Java and C++ combined!

And, a final quote from Erik Naggum in Message-ID: <3150208907886348@naggum.no> (emphasis added - me):

C/C++/Java/Perl/etc are for people who want to make things that work.
Common Lisp is for peple who want to make things that don't break.

"The Humble Programmer", E. Dijkstra, CACM, vol. 15, n. 10, 1972, page 859:

Lisp has jokingly been called "the most intelligent way to misuse a computer". I think that description is a great compliment because it transmits the full flavor of liberation: it has assisted a number of our most gifted fellow humans in thinking previously impossible thoughts.

Relevant Links



Sam Steingold
<sds@gnu.org>
 Valid XHTML Use CSS Valid CSS created: 1999-01-01