Rancho*: It just looks like, that benchmarking the way we used to, is more than obsolete. When Degger* first posted this, I remembered a german article posted some days back in mid May. I didn't bother about it too much because I thought Trident's market share was way too low to be of any interest to our readers. But recent developments in the industry force the media, semi professional reviewers and also end users to change their point of view on established benchmarking solutions currently available. There is some small hope that I put into RightMark3D, an open source benchmark approach. However real word applications will be what we have to look for in performance evaluation of graphics chips in the future. The Trident article was posted by Chip.de, Germany's largest online IT portal. The Toshiba Tecra M1 notebook uses an Trident XP4m32 graphics chip that produces inflated benchmarks in 3DMark2K1SE. Once you renamed the 3DMark.exe file, scores dropped more than 50%! The exploit revealed to the public by The Tech-Report using 3DMark03 and Nvidia graphics cards functions in the same way. Thanks to Thorz for the submission.
If you're old like me, you may remember the day when ATI was caught optimizing its Radeon 8500 drivers for Quake III Arena timedemos. The trick was simple: ATI's drivers looked for an executable named "quake3.exe" and turned down texture quality when "quake3.exe" started. Kyle Bennett at the cold, HardOCP renamed "quake3.exe" to "quack3.exe" and ran some benchmarks. ATI was busted. In a funny twist of fate, I got a tip earlier this week about NVIDIA's Detonator FX drivers. The allegation: if you rename 3DMark03.exe to something else and run the benchmark with anisotropic filtering enabled in the drivers, test scores drop. In other words, NVIDIA appears to be using the same lame technique ATI did way back when: keying on the program's filename in order to trigger benchmark "optimizations." In this case, those optimizations appear to be a lower quality form of texture filtering than the anisotropic filtering method selected in the driver control panel. Many review sites like us benchmark cards with anisotropic filtering and edge antialiasing turned on, so these things do matter. Read the whole story over @ The Tech Report Discuss
If you're old like me, you may remember the day when ATI was caught optimizing its Radeon 8500 drivers for Quake III Arena timedemos. The trick was simple: ATI's drivers looked for an executable named "quake3.exe" and turned down texture quality when "quake3.exe" started. Kyle Bennett at the cold, HardOCP renamed "quake3.exe" to "quack3.exe" and ran some benchmarks. ATI was busted. In a funny twist of fate, I got a tip earlier this week about NVIDIA's Detonator FX drivers. The allegation: if you rename 3DMark03.exe to something else and run the benchmark with anisotropic filtering enabled in the drivers, test scores drop. In other words, NVIDIA appears to be using the same lame technique ATI did way back when: keying on the program's filename in order to trigger benchmark "optimizations." In this case, those optimizations appear to be a lower quality form of texture filtering than the anisotropic filtering method selected in the driver control panel. Many review sites like us benchmark cards with anisotropic filtering and edge antialiasing turned on, so these things do matter. Read the whole story over @ The Tech Report Discuss
