Date:
Monday , May 07, 2007
Author:
Jason Wall
Editor:
Josh Norem
Google +1

XP vs. Vista - A Tale of Framerates

Seldom has the rumor mill turned faster than when gamers talk about gaming in Vista. Some folks are staying away from the new OS simply because they feel it doesn’t game well. We set out to put some hard numbers on those claims.

Introduction

A hotly debated and often-lamented issue with Vista is that it doesn’t appear to perform as well in 3D gaming as its predecessor, Windows XP. While some have reported no change in performance, we’ve seen poor driver support in Vista cause serious problems with performance and overall compatibility in various gaming titles. While we’d like to take readers’ testimony at face-value, we felt the need to go and find out what the hub-bub is about and put hard numbers on this supposed performance discrepancy. We’ll be taking both OSes, side by side, and running them through the exact same gaming gauntlet. Afterwards, we’ll look at the framerates and see what’s what. In addition to capturing framerates, we’ll also be looking at general image quality between the OSes.

Test Set-Up

We know from the get-go that this is going to be a difficult task. There are literally countless configurations that folks game on, and obviously, not everyone’s experience is going to be the same. There’s also the issue of the games we wished to test. Each of them put their own demands on a system and the OS, so some games may show a dramatic performance delta while others may not. Driver support is another issue. We wanted to publish this article in a timely manner, but with enough of a “grace period,” if you will, that publishers and graphics card manufacturers would have stable and optimized drivers for Vista. And finally, we need to make sure that our gaming experiences between the operating systems are as exact as possible so that a direct comparison can be made.

To address the hardware concern, we’re using two different video cards: a 7600 GS and an 8800 GTS 320MB, graciously provided by AVADirect. These represent the mid-to-low range and upper range of current video cards. The rest of the system was supplied to us by Puget Custom Computers and represents a solid mid-range computer – the specifications are below. The idea behind using different video cards was two-fold. First, we get to test how different cards will play the same games under the two OS environments. Also, however, we’re testing two different drivers from NVIDIA. One of these cards is also a DX10 card, so we were interested to see how/if this made a difference. Below the specifications are the Vista system performance ratings for the 7600 GS and 8800 GTS configurations, respectively.

Article Image

Article Image Article Image

For our game title selection, we used our [H] Consumer line-up. These games were chosen with great deliberation as to what kind of games they represent, and especially for their popularity in the marketplace. These are the games that most people are playing according to sales figures. If you're looking for a great discussion of how Supreme Commander is doing with Vista and Intel's quad-core technology, check out this article from Brent.

There wasn’t much we could do with the driver situation. We waited what we thought was a reasonable amount of time for game publishers and NVIDIA to get their act together with regards to game patches and drivers, and went with the most current version of both. If a new driver was released, we’d install it and start all over. This happened more than once, which is why this article isn’t what we would call “timely.”

In regards to ensuring that we have an exact gaming experience between the OSes, we did our best. In AI-intensive games such as Company of Heroes and The Sims 2, we used saved games to recreate the exact same situation instead of letting the game AI build from scratch again. This locked the computer into the same strategy so that we wouldn’t be testing different loads on the system. Prey, Need for Speed, Oblivion, and FSX were easy because the bots do essentially the same thing every time. BF2142 was a bit difficult because the AI behaves differently and reacts to troop movements, which the player doesn't have much control over. World of Warcraft is even worse because server loads and time of day can affect game performance drastically. We did our best to test the game at the same time of day on weekdays so that the server load would be approximately the same while we performed our framerate capture. Also, the author is the only one who performed testing on these games, and he generally did it in the same day so that it was fresh in his mind exactly how the tests were performed.

Also, to avoid preconceptions or points of reference, we always tested Vista first. FRAPS hides the framerate counter if you wish it to, so we did for the majority of our experiences so that we wouldn’t pay attention to it. Whatever graphics settings we had in Vista, we copied them exactly in XP and captured the framerates without making any adjustments. This had to be done twice over because of the different video cards. For each game, the settings used on each card will be noted.

Prelude

This article is not going to have a lot of editorial and verbose pontificating upon the results. We intend for the numbers to speak for themselves. We’d be foolish to brand Vista with a “sucks for gaming” label if at some point in the future, whatever performance delta may or may not be present is closed with driver revisions or a service package.

So, with that said, here are our experiences.