Linux Benchmark deficiency

[FalconFly]

As there currently is no Ticket despite the Problem being very annoying since more than 2 years now :

Linux Benchmarks are still too inconsistent compared to Win32 Benchmarks, Scores per FPU and ALU need to be matched up as close as possible (i.e. within 5% or less)...
Example (AMD Athlon64 3200+) :

Linux 5.8.16 : 1744 FPU / 2921 INT / 4665 Combined
Linux 5.8.17 : 2016 FPU / 4742 INT / 6758 Combined
Linux 5.9. 3 : 1563 FPU / 2670 INT / 4233 Combined
Linux 5.9. 4 : 2013 FPU / 4843 INT / 6856 Combined
Linux 5.9. 5 : 2014 FPU / 4742 INT / 6756 Combined

Desired* : 1880 FPU / 3344 INT / 5224 Combined

• desired as to match corresponding Win32 Benchmark result

Notes : Different CPU Models may exhibit higher or lower deficiency using existing Linux BOINC Benchmarks, therefor a comprehensive List needs to be built in order to determine what percentage of improvement is required at least for the most common CPUs currently in use.

Options : 1) modify Compiler optimize Flags to achieve best possible match to known Win32 Scores 2) implement correction Factors to the known, deficient Benchmarks (in the end, this 'might' be the easiest solution, the most realistic chance of success is likely a combination of both Options above) 3) modify Benchmark Code to assist Cross-Platform consistency (probably the most promising but least desirable Option due to amount of work required, should be reserved as last-ditch option if Options 1 and 2 fail)

Requirements :

• Developer with time at hand and persistency in mind to solve a 2+ year old Problem
• sufficient amount of Volunteer Testers to report Results into Linux CPU benchmark results Database

Priority :

• assessed as High (Solution is long overdue)

[romw]

Is the desired score based on 5.9.4 and older or 5.9.5 and newer?

If older, do you happen to know what the desired score would be if tested against 5.9.5?

[FalconFly]

Replying to romw:

Is the desired score based on 5.9.4 and older or 5.9.5 and newer?

If older, do you happen to know what the desired score would be if tested against 5.9.5?

Good point, I missed that.

The desired Scores were derived from Win32 V5.8.16 comparison.

Based on Win32 V5.9.5 benchmark comparison, the desired score is amended to : 1996 FPU / 3703 INT / 5699 Combined

[FalconFly]

Forgot :

For testing, I can currently offer AthlonXP, Athlon64 and Athlon64 X2 and Pentium M CPU series CPUs. Most of my Systems run Linux, but I can easily take a dedicated 2nd Harddrive and Install Win2000 for testing & comparison if more numbers are needed.

[romw]

Okay just to clairify it appears we are doing good in the FPU front:

Linux 5.9.5 : 2014 FPU / 4742 INT / 6756 Combined[[BR]]
Win   5.9.5 : 1996 FPU / 3703 INT / 5699 Combined[[BR]]

I believe Windows is comming up short due to the extra security checks VS 2005 introduced into the compiler.

[romw]

BOINC w/Compiler Runtime Checks disabled

[romw]

I have posted a copy of BOINC 5.9.6 with the compiler runtime checks disabled.

FalconFly?, anybody else, could you run the benchmarks on your machine(s) where you can dual boot Linux and Windows and report the revised scores?

Both binaries should be digitally signed with a valid certificate.

[FalconFly]

Replying to romw:

I have posted a copy of BOINC 5.9.6 with the compiler runtime checks disabled.

FalconFly?, anybody else, could you run the benchmarks on your machine(s) where you can dual boot Linux and Windows and report the revised scores?

Both binaries should be digitally signed with a valid certificate.

(It's getting late in Germany, I will test the attached File tommorow on Dual boot)

Anyway, looks very good on the FPU side with V5.9.5 now.

I've ran some comparison Synthetic Benchmarks (SiSoft? Sandra XI) to get a second Opinion about the Relation between Integer ALU and FPU performance levels :

Win32 BOINC V5.8.16 had a 1.78:1 INT:FPU Ratio... Win32 BOINC V5.9.5 has a 1.86:1 INT:FPU Ratio...

SiSoft? Sandra Results from that System : Dhrystone (ALU) : 7909 MIPS Whetstone (FPU) : 4606 MFLOPS Whetstone (SSE2) : 6387 MFLOPS Whetstone (SSE3) : 6689 MFLOPS

Judging from these relative numbers, the ALU:FPU Ratio w/out SSE enhancements seems comparable, SiSoft? Sandra clocks in at a 1.72:1 Ratio. Taking that into account, the current Win32 BOINC Benchmarks results seem okay and right in the ballpark, as the SiSoft? Sandra Utility is known to squeeze out just about the maximum possible when it comes to Synthetic Benchmarks since several years.

The current 5.9.5 Linux clocks in at 2.35:1 ALU/FPU Ratio, so (take it for what it's worth) I 'believe' Win32 5.9.5 is doing perfect, it's just Linux 5.9.5 seems to get too much Dhrystone performance out of the compiler, based on what I see.

[FalconFly]

Did a quick, last Test for tonight on the Windows Athlon64 3700 :

5.9.5 : 2196 FPU / 4074 INT / 6270 Combined 5.9.6 : 2207 FPU / 4042 INT / 6249 Combined

Basically no change, well within Benchmark tolerances...

[anonymous]

Replying to romw:

Both binaries should be digitally signed with a valid certificate.

Hm, just to avoid misunderstandings : the .zip File contains only one Binary (Win32 5.9.6 + associated .dll's )

Any Linux 5.9.6 that should be there, or what Linux Version do you want compared to that Win32 5.9,6?

[romw]

You could use Linux 9.9.5.

EXEs and DLLs are both executables/binaries.

[FalconFly]

I lost tons of time today as Ubuntu 7 LiveCD repeatedly crapped out on me (couldn't execute the boinc binary from the Terminal - it kept claiming 'File not found' *ugh* ?!) ...I'm on 24h shift tomorrow, so I'll have to postpone the complete test series to Saturday :(

I'll see to it that I get the dual Boot results done for Pentium M, AthlonXP, Athlon64 and Athlon64 X2 that day. That'll be Linux 5.9.5 vs. Win32 5.9.5, unless other specific Versions are requested.

[FalconFly]

Alright, got some data from dual Boot :

Athlon64 3000+ (Socket939 1800MHz, 512k L2)

Linux V5.9.5 : 1811FPU | 4352INT

Win32 V5.9.5 : 1839FPU | 3392INT

Athlon64 X2 4400+ (Socket939 2200MHz, 1024k L2)

Linux V5.9.5 : 4428FPU | 10680INT

Win32 V5.9.5 : 4500FPU | 4064-8312INT

I don't know if Win2000 SP4 with all Updates does not like DualCores? or if the Win32 Benchmark still has Problems here. The Integer Results were just plain unreliable despite me trying to work around it all day. Good news is that my observation of the Benchmark scaling with CPU Core Clock was correct, therefor (for our Linux vs. Win32 purposes) an Athlon64 X2 behaves no different than a Single Core Athlon64 - therefor no need or its results separately in this respect. As no Linux Live CD worked out for me, I can't deliver numbers for the Pentium M. The AthlonXP Results will follow on Monday.

[FalconFly]

Addendum :

Thinking about getting more Data from other CPU Models, there is still the Option of searching through the Project Pages' Host Listings. That'd be alot (!) of work finding Systems running Linux and Win32 5.9.5, but it's feasible as a backup plan. A good statistical average from a sufficient number of Systems should yield good numbers.

[FalconFly]

As promised :

AMD AthlonXP 3000+ (SocketA 2166MHz, 512k L2)
Linux V5.9.5 : 2139FPU | 4414INT
Win32 V5.9.5 : 2162FPU | 3685INT

[FalconFly]

And one more :

Intel Pentium M M750 (1866MHz, 2048k L2)
Linux V5.9.5 : 1890FPU | 4284INT
Win32 V5.9.5 : 1728FPU | 3490INT

That's AthlonXP, Athlon64 and Pentium M. These are all CPUs I can provide by myself.
Averaging these 3 CPU types, Linux 5.9.5 Integer Benchmarks are rated 23.6% over its Win32 counterpart.

[romw]

Since the FP benchmarks are within 1% across both platforms, and the credit granting calculations are weighted for FLOPS, we are going to punt the Integer work to 6.0.

[romw]

This issue should now be resolved with the latest Linux clients using the latest GCC.