Assessing open source software as a scholarly contribution

A post that is not strictly related to Computer Graphics, 3D or Cultural Heritage.
Just a small note/rant to point out a recent paper:

Lou Hafer and Arthur E. Kirkpatrick
"Assessing open source software as a scholarly contribution"
Communications of the ACM, Volume 52 ,  Issue 12  (December 2009)

It is an interesting discussion on the fact that "Academic computer science has an odd relationship with software: Publishing papers about software is considered a distinctly stronger contribution than publishing the software".
 
Being a senior researcher before being the lead developer of MeshLab, I have to say that I totally agree with those feelings. I have often thought that devoting a significant portion of my time to the MeshLab project is not a 100% wise move from a career point of view; probably writing a bunch of easy minor-variation papers is much more rewarding and is evaluated better when running for higher positions.


The sad thing is that there are people thinking that the citations coming from the paper you have written about your software are more than enough to reward you for your effort of writing it. Usually these considerations came from computer scientists who do not have perfectly clear what means writing and maintaining real software tools.
Some bare facts:

1. If you write and maintain significant software tools/library then you are serving the research community in a way that is more significant that writing a paper.
2. The time required to develop and maintain sw tools is much larger than the time required to write one paper.
3. Assessing the importance/significance of software is more difficult than assessing the value of papers, no common bibliometric tools (obviously download count is not a good metric).
4. Commissions evaluating people careers usually ignore sw and concentrate on other, more standard, research products (papers, editorial boards, commitee, teaching, prizes, etc).

As a simple consequence, of 2,3,4 and despite of 1, with current career evaluation habits, developing and maintaining sw tools that are significantly useful for the research community is NOT a career maximizing move. And this is, in my humble opinion, definitely, completely, utterly WRONG.

Now when you stumble upon a discontinued piece of code that you would have loved to have maintained, you have an hint of why the original author abandoned it.