UPDATE-This is the review of Forbes 2008 America’s Best Colleges. Here is the link to the America’s Best Colleges 2009 .
Forbes magazine has come out with a new college ranking system in an apparent attempt to compete with US News. The system used by Forbes is in my opinion the most ridiculous system yet devised. I would be more impressed if they just wandered down the street asking people for their opinion on the best colleges.
Here is what the system is based on: 25% of the ranking is based on evaluations from RateMyProfessor, a website that purports to allow students to rate their professors. Of course the website has virtually no control over who posts so anyone, whether a student or not, can go in and post on a particular professor. Even the professor in question can post if they would wish. If you want to post about a professor multiple times, go ahead, we won’t stop you.
The second criteria, also accounting for 25% of the rating, is how many of the college’s alumni, adjusted for enrollment, are listed in Who’s Who in America. For those of you who are not familiar with this list of supposedly impressive people, here is a review on why Who’s Who is totally aburd. You should note that the article, showing how lame Who’s Who is, was published by none other than Forbes magazine.
The final 50% of the ranking is based equally on three factors including the average amount of student debt, the percentage of students graduating in four years, and number of students and faculty, adjusted for enrollment, who have won nationally competitive awards like Rhodes Scholarships.
The last three criteria, particularly the first two, actually make some sense in ranking a college. A student should know, and compare, the average amount of student debt and the four year graduation rate. But to consider these factors in the rating system as less important than ratemyprofessor and Who’s Who demeans even those positive aspects of this system.
Although I am not a big fan of the various ranking systems, at least some of the better known systems attempt to use valid numbers. Forbes has taken a few important issues and combined them with the least reliable numbers they could find. All to make money off the anxiety of students and families attempting to find the best college. Forbes should be ashamed and everyone should boycott this issue of their magazine. Moreover, if they are willing to use such sloppy numbers here, I have to question everything Forbes publishes.
I recently came accross your blog and have been reading along. I thought I would leave my first comment. I dont know what to say except that I have enjoyed reading. Nice blog.
Tim Ramsey
Aren’t college rankings by definition absurd? They purport to put all the colleges in order as if they were sports teams.
I do agree that the RatemyProfessor and Who’s Who parts of the system are especially suspect…
The Center for College Affordability and Productivity and Forbes.com carefully considered the issues you have raised about ratemyprofessor and Who’s Who. Please see our extensive methodology section for more details:
http://tinyurl.com/6hm98o
Many thanks for your comments,
Carl
Carl Lavin
Managing editor,
Forbes.com
Carl,
Thank you for your comment and the address for the methodology that you used in coming up with the rankings. I understand the argument but still don’t feel that it is reasonable to allocate 50% of a college ranking to such questionable sources. Of course U.S. News does some of the same thing by allocating 25% of the ranking to peer assessment. That does not make it a good measure. I would suggest that for future editions Forbes focus on some of the issues that comprise your other 50% of the ranking and rather than rank colleges 1,2,3 etc, bunch them into groups. Top 20 if necessary, that sort of thing.
PS for Carl. Arkansas is not abbreviated correctly in the rankings list. AR not AK. AK is Alaska.
To add one tidbit, the “debt” ranking is based entirely on the average amount of debt of the average student borrower. Students who pay their own way or students who receive full scholarships are excluded from the calculations. Thus, the amount is not the average debt for all graduating students, but only the average debt of students who borrow money from certain identifiable sources.
In addition, the debt calculation does not take into account loan forgiveness or loan capping programs that involve the school paying back student debt at or after graduation. It is inexplicable that a school like Grinnell College would be ranked 409th in the student debt category, when it: (1) has a policy in place to cap need-based student loans at $4,000 per year; (2) has some of the most generous financial aid programs; and, for the past several years, has received alumni-funding to pay off the complete loan debt of dozens of graduating students.
The Forbes rankings are totally ludicrous. It makes US News and Washington Monthly look good by comparison.
The rankings really don’t indicate how good the education. They seem more to judge how much the student likes the education but do not take into account the the opportunities school provide.
Every student deciding on a college must take into account the fundamental tradeoff between a big school and small school. A small school will give you more personal attention but a big school has more opportunities. I noticed that MIT is rated right below Centre College. Centre College probably provides more personal attention than MIT does but can Centre college give students the opportunity to work on cutting edge research in every field imaginable with some of the most sophisticated laboratory equipment in the world? No. Small liberal arts schools are very common both in the US and abroad, large well respected research institutions are fewer the opportunities they provide can not be ignored.
Forbes has lost a great deal of credibility in my eyes. To Carl Lavin, explaining your methodology doesn’t make your logic sound.
Look at Emory & Henry College
Open admissions and ranked higher than some
Ivy League Schools. What a joke!!!
This ranking is just messed up in so many ways. And the financial information on there is so wrong. For Michigan University, the in-state and out-state costs were averaged together so the college looks affordable. I definitely prefer US News’ version, and I’ve heard all kinds of complaints about Forbes’
The ideal college ranking system would be difficult to game, reflective of something that is actually relevant to students, and implemented consistently each year. We have implemented this type of system at http://college.mychances.net/college-rankings.php
This college rankings system is based on revealed student preference . In this system, the college admissions process is treated like a chess tournament. The colleges play matches (which occur when 2 colleges admit the same student). In each match, there is a winner (the college that the student ends up attending) and a loser. The winner gains points; the loser forfeits them. After many games, the colleges that students prefer rise to the top.
If the students behave rationally by choosing the college that they prefer – and if their preferences are dictated by some meaningful characteristics of the school – then this set of rankings should be rather meaningful.
James,
I agree with you that a good ranking system would be one not capable of manipulation. However, I don’t believe that there is only one way to rank colleges since everyone has different needs from the college experience.
Student preferences can be some indication but the problem with that approach is that if a student doesn’t know what they want to begin with from college, then their choice of college is already questionable. In an ideal world where all students adequately investigated their choice of college based on their particular needs, student preference might have some more weight.
That being said, I took a quick look at your website and it looked like you have some really good information there. Congratulations on putting together a good website with helpful information for students.
Todd, thanks for your feedback. You raise a good point with regards to the limitations of student preference, and the fact that each student will have their own personal set of rankings based on factors that matter to them.
Your feedback made me consider another limitation to our preference method: the biases inherited by students over the years can be propagated. If people’s preferences are driven by US News instead of by some factors that actually matter to their education or their wellbeing for the next 4 years, then our list will just end up recapitulating that of US News. Depending on your vantage point, that would either be an extremely unfortunate sign of the times or a validation of the utility of the US News list.
Forbes could have been like everyone else and look at useless input information such as sat scores. Instead they focused more on the outputs. Sure a school that admits students with high sat scores are not gonna make the students dumber while they attend. I do think when evaluating a college the people who would know best would be the students……with the millions of entrys on ratemyprofessor its hard to say that it isnt a good estimate of how well the faculty is performing. I also think that looking at student debt acquired is an interesting but effective way to look at how well a college performs with regard to the students.
Don,
I agree that having student input might have some validity to rating a college. However, ratemyprofessor has absolutely no control over who uses their services. So, if I want to, I can go onto any college site and rate any professor I want to. That is the problem with what Forbes did.
As I said in the original article, evaluating student debt does have some merit but the factors for rating that made some sense had less impact on Forbes ratings than the factors that should never have been used.
58 George Fox University ???
61 New College of Florida ???
75 Juniata College ???
104 Duke University
106 Georgetown University
173 Johns Hopkins University
200 University of Michigan, Ann Arbor
207 Cornell University
267 Carnegie Mellon University !!!
355 New York University
Really?
Ratemyprofessor.com is used by widely varying amounts in different colleges, and is only even remotely valid from a statistical or scientific perspective when comparing professors from the same department in the same institution. Based on my review of results, professors who inflate grades and don’t challenge students get the best reviews, while those who actually teach students effectively and hold them to a normal grading standard get the worst.
The Forbes page explains that RMP scores correlate to some degree with standard student evaluations, but I don’t think Forbes understands why universities collect student evaluations and how they’re read. Suffice it to say, each grain of student evaluation is taken with a 5 pound block of salt. Generally the comments are the only things that are useful.
My conclusion is maybe the Forbes staff were all rejected form Ivy League Schools
Great Article by the way.
US News bases muh of their rankings off school reputation. How is this “fair”? Just because Ivy League schools have excellent reputations, it doesn’t mean they provide better educations that lesser known private colleges.
By the way, Juniata College is an excellent school and is a tier 1 liberal arts college.
Kathryn,
Don’t mistake what US News does for trying to be fair. The do the rankings to make money. And they know that they have to rate certain schools high. The best example was a number of years ago when they changed the criteria for the rankings and Caltech came out as number 1. Everyone “knew” that couldn’t be right so US News changed the criteria back the next year and Caltech went to its “rightful” place about number 9.
And yes, I agree with you that Juniata is a great school that should be considered by more students.
This is absurdly horrifying as it gets. Military Academy and Air Force Academy are in top 10 just because the graduating students face no incurring tuition debts reflects the paradigm of a Communist Society not one like ours. I agree that best colleges are the most expensive ones but the students graduating from these institutes land in the top jobs. Besides, a major part of your first few years salary goes towards the eradication of the college debts.
One more thing, MIT at number 11; below Caltech, Harvard, Princeton, Yale, Stanford!!!!! Are you serious?????? Instead of relying on ratemyprof.com, Forbes should lay down some other ground rules for determining the status of faculty in an institute like number of Nobel Laureates, Louis Jeantet awardees, Millenium Award winners, National Medal of Science papers published in top rated Journals like Nature, grant received from the Federal Government, etc.. For students too the publications should be the foremost criteria and don’t get me started on Scholarships. You can not deduce anything from them as most of these scholarships have AMERICAN CITIZENSHIP as the first criteria and that singles out the vast majority of the International Students who throng the campuses.
As stated in another comment above, college rankings tend to be skewed for one school or another in any regards. For example, yes a degree from Harvard or some other Ivy League school helps a candidate get to the next level a little easier than other schools, but is the candidate really any better or is the halo effect playing into the hiring or acceptance decision? Also, The US News rankings are not perfect either, they use total building square footage as one of their criteria, so wasting money received from tax payers is a good thing? Another is graduation rate. In a time of “grade creep” (increasing of average grades given to rate higher rankings) why does graduation rate matter? If anything, a more difficult academic university where competition is high should techincally be producing better students than some 2nd rate degree factory like University of Pheonix or Penn State.