Note: First Things just released an issue in which they ranked colleges based on academic, social, and religious criteria. The rankings are not available online unless you are a subscriber, but for people at Calvin, I can send a copy of the article from our library upon request. What follows is my response. Thanks to Kurt and Irene for helpful editing.
To the Editors of First Things:
First, I would like to thank you for doing a service to parents and educators in the United States: by publishing rankings of schools that explicitly engage the academic culture and treatment of faith on campuses, you have provided sorely needed information to those making difficult choices. I am sure that the process of producing these rankings was expensive and time-consuming, but I hope that you will consider making them a regular feature.
Second, as a faculty member at Calvin college, I would like to thank you for the great compliment you have paid my institution. Placing us 13th overall and 2nd among Protestant schools is a generous recognition of what we do here, given the number of fine institutions that you examined. The point of this letter is not to argue that we were ranked unfairly. This is most certainly not the case.
The process of producing rankings is always one in which false precision is inevitable, and the methods of ranking will never make everyone happy. This is particularly true when the rankings are intended to express a set of priorities different from those that are common in academia. But let me, as an empirical social scientist, offer some suggestions for improving your methods, should you choose to repeat this process.
First, rankings are only useful if a reader can understand the criteria used to rank the schools. There are a couple of ways by which you could make these criteria clearer, and thus make the rankings more useful. First, your inclusion of “descriptions from our FT experts” mixed in with the other data makes it difficult to interpret the final rankings. If your goal was to produce a ranking of “the favorite schools of FT editors and friends,” then you really don’t need all of the empirical apparatus that you include, such as explicit weights on different surveys. In fact, to those of us who respect what FT does, it would be more helpful to produce a separate “FT Favorites” set of rankings, so that we can see those data in isolation.
The second move that you could make for more clarity is to share, somewhere, some or all of the questions that were used to arrive at the scores given in various categories. One example will illustrate the point. Hillsdale College, a fine institution in most respects, inexplicably has a higher “religious” score than Calvin College. This may be a testament to Hillsdale’s ability to be hospitable to Christian students outside of a confessional framework, something for which they deserve credit. Without more knowledge of the questions in the surveys, however, a reader can only be left to speculate about why a college that does so much to integrate faith and learning (Calvin) ends up ranked lower than a school with no explicit institutional faith commitments.
One more note of caution. In surveying people in very different institutional contexts, the students' context can affect the comparability of their answers. A student at a religious school might have very different reference points for questions about faith and social life than a student at a non-religious institution.
Again, thanks for the hard work. This particular issue was especially interesting and insightful.
Sincerely,
Steven McMullen
Department of Economics
Calvin College
No comments:
Post a Comment