« Internet Explorer 8 using Restart APIs t... | Best source code comments »

Measuring performance under Visual Studio

Mon, April 6, 2009, 07:58 PM under dotNET
When I created my Visual Studio C# demos for showing off Tasks versus Threads, I of course created the project in Release mode, but there were 2 additional things that I had to do.

One had little impact, but is important to remember: set the power plan on Windows to "High Performance". Otherwise CPU usage can be throttled by the OS, which is not what you want when you are demonstrating parallelism and 100% CPU usage ;-)

The other had a profound impact and had me puzzled for a bit: use Ctrl+F5 instead of F5. The difference in the time the threads scenario took was humongous. That is because the naïve threads demo creates around 1023 threads and it turns out creating threads under the debugger (F5 case, even in release mode) is measurably slower than not. I should have known that, but I didn't, so sharing it here in case you didn't either: when measuring perf, always use Ctrl+F5 in release mode.

For a lot more on the performance difference between running under the debugger and not, I recommend Joc's blog post: Why is my app so much slower under the debugger.
Comments [3] |#Permalink
Tuesday, 07 April 2009 08:44:00 (Pacific Daylight Time, UTC-07:00)
Hey Daniel just noticed you've touched the perf thing :)

Yonks ago, Vance Morrison implemented his Code Timers for perf measurement: http://blogs.msdn.com/vancem/archive/2006/09/21/765648.aspx

It's great as it incorporates the basic statistics, all those mean/median/standard deviation things that give you a much richer understanding of a measurement.

(And you can see how they work in real code: Mocking Frameworks Compare project uses it - see Performance Comparison at http://code.google.com/p/mocking-frameworks-compare/source/browse/trunk/Tests/CS/Performance/Program.cs)
Andrew
Thursday, 09 April 2009 05:02:00 (Pacific Daylight Time, UTC-07:00)
Daniel,

That reminds me of the difference of behaviour in the GC between debug and release modes that Jeffrey Ritcher mentions in his CLR via C# book.

http://stupiddumbguy.blogspot.com/2008/05/net-garbage-collection-behavior-for.html discusses it in more detail for those that don't have access to the book.

Daniel did you ever buy/read that book- I remember lending you the first edition way back in 2002? :-)
Richard OD
Saturday, 09 May 2009 13:45:57 (Pacific Daylight Time, UTC-07:00)
Andrew: thanks for sharing

Richard: Yes I read it… wow, that was a long time ago :-)
Daniel Moth
Comments are closed.