Opened 18 years ago

Closed 18 years ago

#122 closed Defect (fixed)

Windows benhcmark differences between VS 2003 and VS 2005

Reported by: romw Owned by: romw
Priority: Blocker Milestone: 5.10
Component: Client - Daemon Version:
Keywords: Cc: davea

Description (last modified by Nicolas)

Default optimizations have changed between Visual Studio 2003 and Visual Studio 2005 which has lead to people reporting that their benchmark scores are worse then before.

Rebirther states: 

Benchmark results:
Number of CPUs: 2
829 floating point MIPS (Whetstone) per CPU (before 1400)
1119 integer MIPS (Dhrystone) per CPU (before 1200)

Looks like we'll need to play around with the optimization settings in 2005 a bit.

Anybody else want to report their scores?

Change History (17)

comment:1 Changed 18 years ago by romw

Status: new → assigned

comment:2 Changed 18 years ago by Contact

WinXP on Processor: 1 AuthenticAMD Mobile AMD Athlon(tm) XP-M Processor 2800+ [x86 Family 15 Model 8 Stepping 2]

5.9.4:

1302 floating point MIPS (Whetstone) per CPU 2908 integer MIPS (Dhrystone) per CPU

5.8.16:

1505 floating point MIPS (Whetstone) per CPU 2921 integer MIPS (Dhrystone) per CPU

comment:3 in reply to:  description Changed 18 years ago by Contact

Replying to romw:

Anybody else want to report their scores?

WinXP on Processor: 1 GenuineIntel? x86 Family 6 Model 8 Stepping 6 696MHz [x86 Family 6 Model 8 Stepping 6]

Number of CPUs: 1

5.9.4:

391 floating point MIPS (Whetstone) per CPU 954 integer MIPS (Dhrystone) per CPU

5.8.16:

593 floating point MIPS (Whetstone) per CPU 1040 integer MIPS (Dhrystone) per CPU

comment:4 Changed 18 years ago by romw

Resolution: → fixed
Status: assigned → closed

(In [12465]) Rom 24 Apr 2007

  • fixes #122 - Windows benhcmark differences between VS 2003 and

VS 2005.

The default FP model has changed since 2003, so change how the

BOINC components are compiled so that they use /fp:fast instead of /fp:precise.

win_build/

*.vcproj

comment:5 Changed 18 years ago by KSMarksPsych

4/30/2007 3:29:06 PM
Processor: 1 GenuineIntel Intel(R) Pentium(R) 4 CPU 2.80GHz
[x86 Family 15 Model 2 Stepping 7]
4/30/2007 3:29:06 PM		Processor features: fpu tsc sse sse2 mmx
4/30/2007 3:29:06 PM		Memory: 446.98 MB physical, 1.78 GB virtual
4/30/2007 3:29:06 PM		Disk: 55.88 GB total, 38.34 GB free

2007-04-20 16:06:58 [---] Version change (5.9.3 -> 5.9.4)
2007-04-20 16:06:59 [---] Running CPU benchmarks
2007-04-20 16:08:00 [---] Benchmark results:
2007-04-20 16:08:00 [---] Number of CPUs: 1
2007-04-20 16:08:00 [---] 1217 floating point MIPS (Whetstone) per CPU
2007-04-20 16:08:00 [---] 2275 integer MIPS (Dhrystone) per CPU

2007-04-30 15:29:06 [---] Version change (5.9.4 -> 5.9.5)
2007-04-30 15:29:07 [---] Running CPU benchmarks
2007-04-30 15:30:08 [---] Benchmark results:
2007-04-30 15:30:08 [---] Number of CPUs: 1
2007-04-30 15:30:08 [---] 1508 floating point MIPS (Whetstone) per CPU
2007-04-30 15:30:08 [---] 2676 integer MIPS (Dhrystone) per CPU

2007-04-30 16:09:46 [---] Version change (5.9.5 -> 5.8.16)
2007-04-30 16:09:46 [---] Running CPU benchmarks
2007-04-30 16:10:48 [---] Benchmark results:
2007-04-30 16:10:48 [---] Number of CPUs: 1
2007-04-30 16:10:48 [---] 1390 floating point MIPS (Whetstone) per CPU
2007-04-30 16:10:48 [---] 2502 integer MIPS (Dhrystone) per CPU




Take these with a grain of salt though... this computer has always benchmarked on the low side

comment:6 Changed 18 years ago by romw

Resolution: fixed
Status: closed → reopened

Reopening to track the differences between 5.9.4 and 5.9.5.

Please use the same format as Kathy. Thanks in advance.

comment:7 Changed 18 years ago by MarekZ

Windows XP SP2 on AMD Athlon 64 X2 3800+ (Manchester):
5.9.4:
Number of CPUs: 2
1724 floating point MIPS (Whetstone) per CPU
2495 integer MIPS (Dhrystone) per CPU
5.9.5: Number of CPUs: 2
2153 floating point MIPS (Whetstone) per CPU
2887 integer MIPS (Dhrystone) per CPU

comment:8 Changed 18 years ago by romw

MarekZ, Could you also post the 5.8.16 benchmark scores?

comment:9 Changed 18 years ago by marekz

The same box:
5.8.16 results:
Number of CPUs: 2
2015 floating point MIPS (Whetstone) per CPU
3687 integer MIPS (Dhrystone) per CPU <--- (looks quite strange, but I get it 4 times +/- 5 points)

Results for the 5.9.x are also medians calculated from four measurements (each after computer restart)

comment:10 Changed 18 years ago by marekz

I see, the variability of the Dhrystone score for the 5.9.5 is big enough:

02:47:24| Starting BOINC client version 5.9.5 for windows_intelx86
02:47:24| Processor: 2 AuthenticAMD AMD Athlon(tm) 64 X2 Dual Core Processor 3800+
[x86 Family 15 Model 43 Stepping 1]
02:47:24| Processor features: fpu tsc pae nx sse sse2 3dnow mmx
03:02:26| Benchmark results:
03:02:26| Number of CPUs: 2
03:02:26| 2148 floating point MIPS (Whetstone) per CPU
03:02:26| 3945 integer MIPS (Dhrystone) per CPU
03:04:19| Benchmark results:
03:04:19| Number of CPUs: 2
03:04:19| 2145 floating point MIPS (Whetstone) per CPU
03:04:19| 2776 integer MIPS (Dhrystone) per CPU
03:05:43| Benchmark results:
03:05:43| Number of CPUs: 2
03:05:43| 2146 floating point MIPS (Whetstone) per CPU
03:05:43| 3947 integer MIPS (Dhrystone) per CPU
03:07:10| Benchmark results:
03:07:10| Number of CPUs: 2
03:07:10| 2144 floating point MIPS (Whetstone) per CPU
03:07:10| 2850 integer MIPS (Dhrystone) per CPU
03:08:36| Benchmark results:
03:08:36| Number of CPUs: 2
03:08:36| 2144 floating point MIPS (Whetstone) per CPU
03:08:36| 2832 integer MIPS (Dhrystone) per CPU
...
I'm not sure why, but no other strong CPU-using programs running, no mouse or keyboard use, no strong network activity (like eMule etc.), no computer restarts this time. Really don't know what's happening here. :-)

comment:11 Changed 18 years ago by Contact

30/04/07 11:22:57 PM|Processor: 1 AuthenticAMD Mobile AMD Athlon(tm)XP-M Processor 2800+ [x86 Family 15 Model 8 Stepping 2]
30/04/07 11:22:57 PM|Processor features: fpu tsc pae nx sse sse2 3dnow mmx
30/04/07 11:22:57 PM|Memory: 383.43 MB physical, 1.04 GB virtual
30/04/07 11:22:57 PM|Disk: 111.78 GB total, 22.45 GB free 

30/04/07 11:44:10 PM||Version change (5.9.4 -> 5.8.16)
30/04/07 11:44:10 PM||Running CPU benchmarks
30/04/07 11:44:10 PM||Suspending network activity - user request
30/04/07 11:45:12 PM||Benchmark results:
30/04/07 11:45:12 PM||   Number of CPUs: 1
30/04/07 11:45:12 PM||   1484 floating point MIPS (Whetstone) per CPU
30/04/07 11:45:12 PM||   2608 integer MIPS (Dhrystone) per CPU

30/04/07 11:48:45 PM||Version change (5.8.16 -> 5.9.4)
30/04/07 11:48:45 PM||Running CPU benchmarks
30/04/07 11:48:45 PM||Suspending network activity - user request
30/04/07 11:49:47 PM||Benchmark results:
30/04/07 11:49:47 PM||   Number of CPUs: 1
30/04/07 11:49:47 PM||   1299 floating point MIPS (Whetstone) per CPU
30/04/07 11:49:47 PM||   2745 integer MIPS (Dhrystone) per CPU

30/04/07 11:52:41 PM||Version change (5.9.4 -> 5.9.5)
30/04/07 11:52:42 PM||Running CPU benchmarks
30/04/07 11:52:42 PM||Suspending network activity - user request
30/04/07 11:53:43 PM||Benchmark results:
30/04/07 11:53:43 PM||   Number of CPUs: 1
30/04/07 11:53:43 PM||   1611 floating point MIPS (Whetstone) per CPU
30/04/07 11:53:43 PM||   2967 integer MIPS (Dhrystone) per CPU

comment:12 Changed 18 years ago by Ageless 

01/05/2007 01:18:37[----]Processor: 1 GenuineIntel Intel(R) Pentium(R) 4 CPU 2.80GHz [x86 Family 15 Model 2 Stepping 7]
01/05/2007 01:18:37[----]Processor features: fpu tsc sse mmx
01/05/2007 01:18:37[----]Memory: 1023.46 MB physical, 1.90 GB virtual
01/05/2007 01:18:37[----]Disk: 12.79 GB total, 5.97 GB free
01/05/2007 01:25:35[----] Version change: 5.9.4 -> '''5.8.16'''
01/05/2007 01:26:37[----] Benchmark results: 
01/05/2007 01:26:37[----] Number of CPUs: 1
01/05/2007 01:26:37[----] 1461 floating point MIPS (Whetstone) per CPU
01/05/2007 01:26:37[----] 2868 integer MIPS (Dhrystone) per CPU
01/05/2007 01:32:48[----] Version change: 5.8.16 -> '''5.9.4'''
01/05/2007 01:33:50[----] Benchmark results: 
01/05/2007 01:33:50[----] Number of CPUs: 1
01/05/2007 01:33:50[----] 1274.12 floating point MIPS (Whetstone) per CPU
01/05/2007 01:33:50[----] 2781.39 integer MIPS (Dhrystone) per CPU

The latter are 3 runs I did earlier, plus another one I just ran. 

2007-04-30 20:44:23[----] Number of CPUs: 1
2007-04-30 20:44:23[----] 1569 floating point MIPS (Whetstone) per CPU
2007-04-30 20:44:23[----] 3071 integer MIPS (Dhrystone) per CPU
30/04/2007 20:53:47[----] Number of CPUs: 1
30/04/2007 20:53:47[----] 1570 floating point MIPS (Whetstone) per CPU
30/04/2007 20:53:47[----] 3065 integer MIPS (Dhrystone) per CPU
30/04/2007 22:02:30[----] Number of CPUs: 1
30/04/2007 22:02:30[----] 1526 floating point MIPS (Whetstone) per CPU
30/04/2007 22:02:30[----] 3013 integer MIPS (Dhrystone) per CPU 
01/05/2007 01:35:15[----] Version change: 5.9.4 -> '''5.9.5'''
01/05/2007 01:36:17[----] Benchmark results:
01/05/2007 01:36:17[----] Number of CPUs: 1
01/05/2007 01:36:17[----] 1574 floating point MIPS (Whetstone) per CPU
01/05/2007 01:36:17[----] 3074 integer MIPS (Dhrystone) per CPU

comment:13 Changed 18 years ago by romw

Resolution: → fixed
Status: reopened → closed

Okay it appears the slower benchmark problem is now resolved.

At this point a 5-6% increase in benchmark scores it alright, it'll provide an incentive to upgrade to the latest release.

Now we'll just need to normalize the scores between Windows and Linux.

comment:14 Changed 18 years ago by Ageless

Resolution: fixed
Status: closed → reopened
04/05/2007 22:01:1604/05/2007 22:01:1604/05/2007 22:01:1604/05/2007 22:01:1604/05/2007 22:01:1604/05/2007 22:01:1604/05/2007 22:01:16
Starting BOINC client version 5.9.7 for windows_intelx86
log flags: task, file_xfer, sched_ops, checkpoint_debug
Libraries: libcurl/7.16.1 OpenSSL/0.9.8e zlib/1.2.3
Processor: 1 GenuineIntel? Intel(R) Pentium(R) 4 CPU 2.80GHz [x86 Family 15 Model 2 Stepping 7]
Processor features: fpu tsc sse mmx
Memory: 1023.46 MB physical, 1.90 GB virtual
Disk: 12.79 GB total, 5.96 GB free
04/05/2007 22:01:1604/05/2007 22:01:1604/05/2007 22:02:1804/05/2007 22:02:1804/05/2007 22:02:1804/05/2007 22:02:18
Version change (5.9.5 -> 5.9.7)
Running CPU benchmarks
Benchmark results:
Number of CPUs: 1
1532 floating point MIPS (Whetstone) per CPU
3091 integer MIPS (Dhrystone) per CPU

Fixed too soon, I would say. My benchmarks went UP when checked the last time.

comment:15 Changed 18 years ago by KSMarksPsych

2007-05-04 15:31:21 [---] Processor: 1 GenuineIntel Intel(R) Pentium(R) 4 CPU 2.80GHz [x86 Family 15 Model 2 Stepping 7]
2007-05-04 15:31:21 [---] Processor features: fpu tsc sse sse2 mmx
2007-05-04 15:31:21 [---] Memory: 446.98 MB physical, 1.78 GB virtual
2007-05-04 15:31:21 [---] Disk: 55.88 GB total, 38.39 GB free
2007-05-04 15:31:22 [---] Version change (5.9.5 -> 5.9.7)
2007-05-04 15:32:24 [---] Benchmark results:
2007-05-04 15:32:24 [---] Number of CPUs: 1
2007-05-04 15:32:24 [---] 776 floating point MIPS (Whetstone) per CPU
2007-05-04 15:32:24 [---] 2635 integer MIPS (Dhrystone) per CPU

2007-05-04 15:35:39 [---] Running CPU benchmarks
2007-05-04 15:35:40 [---] Suspending computation - running CPU benchmarks
2007-05-04 15:36:40 [---] Benchmark results:
2007-05-04 15:36:40 [---] Number of CPUs: 1
2007-05-04 15:36:40 [---] 1505 floating point MIPS (Whetstone) per CPU
2007-05-04 15:36:40 [---] 2614 integer MIPS (Dhrystone) per CPU

The first run was done with nothing extraneous running in the background and no activity on the computer. Skype and Firefox were shut down. Second run was done with normal programs running and typing in Open Office.

Compared to the the results for 5.9.5, they are consistent but still higher than 5.8.16.

2007-04-30 15:29:06 [---] Version change (5.9.4 -> 5.9.5)
2007-04-30 15:29:07 [---] Running CPU benchmarks
2007-04-30 15:30:08 [---] Benchmark results:
2007-04-30 15:30:08 [---] Number of CPUs: 1
2007-04-30 15:30:08 [---] 1508 floating point MIPS (Whetstone) per CPU
2007-04-30 15:30:08 [---] 2676 integer MIPS (Dhrystone) per CPU

2007-04-30 16:09:46 [---] Version change (5.9.5 -> 5.8.16)
2007-04-30 16:09:46 [---] Running CPU benchmarks
2007-04-30 16:10:48 [---] Benchmark results:
2007-04-30 16:10:48 [---] Number of CPUs: 1
2007-04-30 16:10:48 [---] 1390 floating point MIPS (Whetstone) per CPU
2007-04-30 16:10:48 [---] 2502 integer MIPS (Dhrystone) per CPU

Again, take the absolute numbers with a grain of salt as the computer has always benchmarked low

comment:16 Changed 18 years ago by Nicolas

Description: modified (diff)

Typo fix on ticket description.

comment:17 Changed 18 years ago by romw

Resolution: → fixed
Status: reopened → closed

As previously mentioned an increase of 5-6% in scores over VS 2003 is alright.

If anything it'll encourage people to upgrade.

What makes it so much better is we closed the FLOP scores to within 1% of the linux benchmark scores.

Note: See TracTickets for help on using tickets.