Toward a more robust law review process

In reading legal blogs, I’ve noticed some dissatisfaction with the law review publishing process, primarily on the part of law professors. I won’t attempt to comprehensively document this dissatisfaction, but a few recent examples include Alfred Brophy’s multi-stage postings of advice for law reviews on PropertyProf Blog and Jeff Harrison’s questioning of the rigor of the law review process as well as the worthwhileness their the aggregate product on Moneylaw blog. Certainly, there are plenty of opportunities to publish in law reviews. Harrison estimates the number as follows:

These numbers must be off but, think about it: 180 law schools, 2 journals per school, 5 issues a year, 4 article per issue. I think that is a demand for 7200 articles per year. (72,000 since 1997) That may be conservative.

This suggests that there is quite a demand for manuscripts. However, as we know, not all journals are created equal. Certainly, placement in top tier law reviews is taken seriously in a range of important decisions such as promotion and hiring. Further, it is likely that a publication in a high tier journal tends to result in more citations in subsequent journals. Citation count studies like Brian Leiter’s are apparently also taken seriously in important decisions although their usefulness is contested by some. All of this is to say that there is something important at stake in determining the relative perceived quality of law reviews and the articles contained therein.

I won’t take on the task of comprehensively listing the alleged shortcomings of law reviews. Some of the main concern-themes I have seen have focused on the perceived unfairness and arbitrariness of the selection of articles for publication. I will detail below my argument for how law reviews (or at least some law reviews) could become more robust, fair, and accurate in selecting quality articles. If this can in fact be done, then we perhaps have a better basis for employing a hierarchy of law reviews basis for making important hiring and promotion decisions. I am quite sure that there will be plenty of people who will be willing to correct me where I am mistaken.

First, there seems to be no reason that at least a critical number of law reviews (maybe 50 to 75) could not employ a peer review process for article selection. If we assume that the average law review receives around 2000 submission a year, then this number could be whittled down through a blind internal review process by a group of law review editors (with the guidance of faculty, to be discussed later). This would perhaps bring the total submission number to about 700-800 submissions, which would not be too different than that of American Journal of Political Science, which handles that number with a much smaller staff than most law reviews. An important component of the process at this juncture is that the sorting process is accomplished without editor reference to the author, their affiliation, or silly footnotes strategically thanking well known scholars for their input.

Second, the above winnowing process and the process of selecting peer reviewers for remaining submissions would be guided by a faculty editorial board as well as an in-house faculty Editor who has final say on all manuscript decisions. Of course, the student editors would still have input on such matters. Editorial board members could be called upon to provide advice in their areas of expertise and, as stated earlier, to serve as a core set of peer reviewers and suggest others. Faculty editorial board members could include the law school’s faculty, but could also draw from other departments on campus or other school’s faculty.

Third, reviewers would be selected for the remaining 700 or so manuscripts and decisions would be largely driven by the results of these reviews. There is likely no reason that a standard set of guidelines for article quality could not be used; consider the following guidelines employed at the American Journal of Political Science:

AJPS requests that reviews address the following points about the merits of the manuscript in as long a review as a reviewer chooses to provide (although a 1-1.5 page length is typical). The points include, but are not limited, to:

• The extent to which the manuscript addresses an interesting and important research problem or question.

• The amount of creativity or innovation of research informing the manuscript.

• The extent to which the manuscript engages the relevant research literature and contributes to the accumulation of knowledge.

• The quality of:

-Thought and/or theorizing (as appropriate).

-Conceptual development or use (as appropriate).

-Analysis or methodological use (as appropriate).

-Evidence bearing on the argument, theory, or rival hypotheses, models, or theories introduced (as appropriate).

-Organization of the manuscript.

-Communication or written expression in the manuscript.

Simultaneous submission might be possible, but would probably work consistent with the rules set forth at the Journal of Empirical Legal Studies:

Simultaneous submission policy: Simultaneous submission of papers to JELS and other journals is permitted. JELS, however, requires that if a submission is accepted for publication in JELS before acceptance by another journal, the author commits to publishing the article in JELS. Thus, if JELS accepts an article before other journals have acted, the author must publish in JELS. If an author receives an acceptance before JELS has acted, the author is free to withdraw the submission from JELS, or to request an expedited review from JELS. Please contact JELS for details.

Frankly, I don’t know how JELS handles expedites, but the normal process could be problematic, given that peer reviewers are apt to not be happy about having to get their reviews back in a week. A law review might have to make a decision based on one reviewer’s decision, or require a longer expedite time (2-3 weeks), or may have to limit its recognition for expedites to the 50-75 fellow peer review law reviews. A final aspect in the review process that deviates from most peer review journals, but would make the process faster, would be to do away with the “revise and resubmit” process. The suggestions of peer reviewers would be taken into consideration in making the editorial decision and editors may require authors to make revisions in response to reviewers’ suggestions, but there would be no more peer review – editors would see to it that changes were made (much like a “conditional acceptance”).

Finally, I’ll address some implications of the proposed new law review. For one, it would require that law professors serve as peer reviewers and on editorial boards. Peer review detracts from getting your own work done, is a pain, and is almost never compensated. But, it’s part of being an academic and should be considered by schools in yearly evaluations. I do about 15-20 of these a year and I know some people who do many more. Despite simultaneous submission, it shouldn’t be too onerous. If we consider 180 law schools multiplied by about 20 faculty per school, that’s 3,600 potential reviewers; plus, they could seek peer review from other fields as well. Another aspect of this system is that it could lengthen the review time for manuscripts. Hopefully, review times wouldn’t be as long as they are in social science fields, but this would be an adjustment that could take some getting used to. The proposed process might also amplify recent movements to shorten article length (even further). This makes review time quicker, allows for more articles to be published per volume, and, according to Ann Althouse, might make people more apt to actually read the articles.

Would this cure the ills that have been alleged? Would it truly create a more robust and rigorous law review selection process? Would it mean a better final product? I don’t know, but perhaps it’s time that we started exploring these issues.