Comparing OLS and Usenix

Yesterday was the last day of

href=”http://linuxsymposium.org/”>Ottawa Linux Symposium, and I

can’t help comparing it with the

href=”http://www.usenix.org/publications/library/proceedings/usenix04/”>Usenix

Annual Technical Conference, which was only a few weeks earlier. The

difference in the quality and relevance (at least as far as my interests

are concerned) of the papers, the energy of the attendees, and the

overall atmosphere of the two conferences were distinctly different.

A look at this year’s

href=”http://www.finux.org/proceedings”>conference proceedings for

OLS is enlightening; two volumes, with a total of 600 pages and 50

papers. In the past, some academics had sneered at OLS and

other Linux conferences because the papers were not well written, at

least by academic standards. This criticism is now much less

valid — if it ever was valid at all. As Clem Cole has pointed out, the NFS

paper, which is now considered seminal, is by academic standards a lousy

paper, and it is doubtful it would have been accepted by today’s program

committees under the guise of “raising quality of Usenix’s

papers”. More importantly for OLS however, beyond the

quality of the writing, was how vital and relevant the topics of the

papers.

Two people (one from IBM, and one from EMC) independently commented

to me that they had thought they stepped into a time-warp; the energy,

exchange of ideas, and excitement that they felt at OLS was reminiscent

of a Usenix conference from the early 1980’s. Two others (both previous

Usenix or Freenix program committee members and/or chairs) agreed with

these sentiments and said that they had pretty much given up any hopes

of “salvaging” Usenix and encouraged me to skip the

href=”http://www.usenix.org/events/usenix05″>2005 Usenix ATC and go

to Linux.conf.au instead. Yet

another old-timer stated that Usenix ’04 was a great place to meet old

friends, but OLS and LCA is where the action is at.

That’s one possible solution, I suppose — after all, if all I care

about is Linux and Open Source Software (OSS) development, OLS (and to a

lesser extent, the [Linux

Kongress]2 and linux.conf.au) is the primiere place to submit papers

and meet with colleagues doing the leading-edge development work in

Linux, X Windows, and other OSS packages.

Unfortunately, there are a number of reasons why this might not be a

satisfactory or complete answer:

So, what can be done? I spent

quite a few hours brain-storming with Val Henson, Paul McKenney, and a

few other interested parties during the course of OLS. Val recorded the

results of some of these brain-storms in her

href=”http://blogs.sun.com/roller/page/val/20040722#the_freenix_track_has_it”>web

log. As we had discussed it, the “Product Track” (which I jokingly called

“Realnix”) would contain technologies that either are in a shipping

product, or is intended to be integrated into a shipping product, where

for the purposes of this criteria, the mainline Linux, NetBSD, GNOME,

KDE, X Windows systems, would be considered “shipping product”.

Some might argue that this separation of “academic papers” and

“industry papers” might not be such a great idea. The argument has been

made that the academic track might be viewed as a “Golgafrincham B ark” where papers

with extensive bibliographies, meticulously enumerated related work

sections, and results describing 3% improvements on microbenchmarks

would be relegated. Perhaps such a perception is unfair, but whether or

not Rob Pike’s description of

href=”http://www.cs.bell-labs.com/who/rob/utah2000.pdf”>Systems Software

Research as Being Irrelevant is accurate is a topic for another day

and time. If Rob’s observations have any validity, however, then it is

up to those in the academic world to address it, and the health and

vitality of the “academic track” will ultimately be their

responsibility, for good or for ill.

If we don’t create separate tracks for academic and product papers,

it’s going to be important to make sure the program committees are

balanced with representatives not just from universities and research

labs, but also from product groups. Bryan Cantrill has put together

href=”http://blogs.sun.com/roller/page/bmc/20040708#check_this_out_a_img”>some

data showing how poorly the computer industry in general, and

product groups specifically have been representated on the Usenix

program committees. My concern is that the academic and

product-group communities, as well as their standards for papers, have

diverged too much already, and trying to manage this tension may prove

to be too much for many program committees. This may result in a

lurching back and forth of the acceptance criteria and overall balance of

the Usenix ATC, which would not be a good thing. Consistency

and predictability is extremely important to potential authors who are

deciding whether or not to submit a paper proposal to a conference or

track.

From a practical perspective, given how strong the OLS conference has

been this year, it will be harder and harder for Freenix to attract

papers about Linux and Open Source Software; OLS has proven itself as

the place for the Linux/OSS community to meet, and I predict that this

will if not starve Freenix submissions, cause it to be extremely

lopsided with very few relevant and interesting Linux papers. Hence,

expanding the scope of Freenix to include product papers may make a lot

of sense.

Of course, making room for papers from product groups (no matter how

we do it) isn’t going to be enough by itself. We will also need to find

ambassadors who work at each of various companies in the computer

industry to market to engineers and to managers why submitting papers

and serving on program committees is a good and useful thing to do —

both for the potential paper author as well as their employer. Val

Henson has put together a set of

href=”http://blogs.sun.com/roller/resources/val/slides.pdf”>slides

for doing just that; see her

href=”http://blogs.sun.com/roller/page/val/20040713#program_committees_superpages_and_compare1″>web

log entry for more discussion and the LaTeX sources to the

slides. (She has given permission for others to adapt her slides

for their company; you should credit her if you use them, though!)

Is this going to be enough to keep Usenix relevant and

interesting? To be honest, I don’t know. I sure hope

so. The good news is that there are more and more people thinking

about the problem, as some of the links to various web logs have

demonstrated. So even if we don’t have the complete solution to

the problem right now, we do have a growing consensus that there is a

problem, and what the scope of the problem is likely to be, and that’s

always a better place to be than being unwilling to acknowledge that

there is a problem in the first place.

Update (7/27): I’ve since found the following

[

blog entry]3 which presents

an opposite, or at least slightly different, point of view.

In it, Werner Vogel argues that the problem is that

program committee members have tried, but failed to get industry (product group)

participation — either on program committees or via submissions. I think he makes some

good points, although I think I differ about which is the chicken and

which is the egg here. It may be that having a separate track just for industry/product-group

representatives may make it easier to both recruit program committee members, and

to have enough people to actively solicit and arm-twist paper submissions. Also, perhaps

a program committee will have to use more creative ways of soliciting papers that

might not occur to a group of academics — for example, such as going through

the product marketing group to arm-twist the engineers to write and submit real papers (not just white papers!)

describing the sexy new technology in a recently-released product. Anyway, Werner’s comments are thoughtful

and should be carefully considered.