In my experience, the meta-issue is that there are multiple substantial barriers to Lisp adoption, and an uncertain payoff, leading to the question, "Is this really the best use of my time?"
The first substantial barrier is Emacs. Before learning Lisp, first you must learn Emacs. I've tried several times, but keep deciding it's not worth the effort. Imagine for an instant that in order to use Ruby, you need to use some strange Japanese editor, but it's really not bad once you get used to the cryptic commands and rebind your keyboard so it is usable. Do you think this might hamper the adoption of Ruby?
Next, the IDE. I'm willing to put up with Slime's 1980's styling, but I keep ending up in mysterious inferior modes that don't seem to be documented. After reading through Slime documentation, it seems I need to understand Emacs's Lisp modes first.
Then, the REPL. Sure, it's nice to be able to interact. But sometimes I need to write a self-contained script that can take part in the whole Unix environment: things like being part of a pipeline, running as a cron job, running through CGI, running as a command. Python makes this trivial; Lisp makes this difficult or impossible.
Then there's the payoff. Will Lisp actually help me solve any of my problems? That's still extremely unclear. Perl or Python are great for random problems I have, such as "Fetch a web page, pull stuff out with regular expressions, and convert it from iso-8859-1 to UTF-8" or "Analyze a pcap dump file". This is where Lisp runs into the infamous library issues. The problems Lisp solves seem to not be the problems I want to solve.
The issue of a dialect/implementation is somewhat vexing: try reading SICP and Practical Common Lisp at the same time, and discovering that everything is different. Also, the expectation that whatever implementation you pick will be the wrong one for the libraries you want is a bit scary. But overall, I'd say this isn't a fundamental barrier.
The above is meant as an honest answer to your question; I'm not interested in starting a flame war.
"But sometimes I need to write a self-contained script that can take part in the whole Unix environment: things like being part of a pipeline, running as a cron job, running through CGI, running as a command. Python makes this trivial; Lisp makes this difficult or impossible."
This is hugely under-estimated as a reason for lack of Lisp adoption. I would put this way ahead of lack of IDEs.
"The first substantial barrier is Emacs. Before learning Lisp, first you must learn Emacs. I've tried several times, but keep deciding it's not worth the effort."
First of all, I don't think it's accurate to say that you must learn Emacs before learning Lisp. Other editors such as vi or IDEs such as ACL & LispWorks are used successfully.
On the other hand, after using IDEs for over a decade, I switched to vim for a couple of years and more recently switched to Emacs. Get the "Learning GNU Emacs" book and it will be trivial to pick up Emacs.
