Thursday, April 06, 2006

My new book is finally on book shelves and in my hands. What a relief.

Now it's time to rinse and repeat. I've been quite lazy with regards to the new book project, but it's time to get serious. I'm laying down some pretty intense milestones over the next few months. We'll see if I can hit the dates.

While the .NET Framework book used primarily a breadth-oriented approach, the Concurrency book is quite different. It covers a smaller set of topics, albeit very depth-oriented.

4/6/2006 8:16:42 PM (Pacific Daylight Time, UTC-07:00)  #   

 Saturday, April 01, 2006

I was in Las Vegas for the better part of last week. Aside from winning money (for once), I also ate at some great places: the 5-course prix fixe menu at Michael Mina (northwest seafood, at the Bellagio), Bartolotta (modern Italian, at the Wynn), brunch at the Mesa Grill (southwestern, at Caesar's Palace), Olives (new Mediterranean, at the Bellagio), and the best of all, the 16-course prix fixe at Joël Robuchon at The Mansion (modern French, at the MGM Grand).

In fact, Joël Robuchon's beat my favorite two dining spots to date: Aujourd'hui, and the Herbfarm. Not to mention that they were nice enough to accomodate and craft up an entirely custom 16-course vegetarian tasting menu. Dinner came complete with fancy scroll-like copies of the menus to take home, wrapped in purple silk. But I figured I'd also digitize it to help remember. Here's the non-vegetarian menu:

Joël Robuchon at The Mansion
March 25th, 2006

Menu Dégustation
Tasting Menu

La Pomme
cuillère de perles, de son jus rafraîchi d’un granite de vodka
Apple pearl, vodka granite

Le Caviar Osciètre
dans une délicate gelée recouverte d’une onctueuse crème de chou-fleur
Oscetra caviar topped with a delicate gelée and a smooth cauliflower cream

Le Foie Gras
en mille-feuille caramélise d’anguille fumée aux saveurs orientales
Foie gras, mille-feuille of smoked eel with oriental flavors

Le Thon
en tartare, poivron rouge confit a la bergamote et au jambon sèche
Tuna tartar, cold red bell pepper confit with bergamot and dry cured ham

La Langoustine
truffée et cuite en ravioli a l’étuvée de chou vert
Truffled langoustine ravioli with steamed green cabbage

La Laitue
en fin veloute sur un flan tremblotant a l’oignon doux
Light lettuce cream on top of a delicate sweet onion custard

La Noix de Saint-Jacques
en cannelloni aux courgettes sous un voile de lard d’Arnad et une émulsion de parmigiano
Cannelloni of scallops and zucchini, parmesan emulsion

Le Homard
au coulis de pissenlit avec quelques feuilles crues de barbes-de-capucin relevées d’une vinaigrette coralline
Lobster, pissenlit coulis, capucin leaves and sea urchin vinaigrette

L’Os a Mœlle
de bœuf de Kobe aux légumes printaniers
Kobe beef bone marrow, spring vegetables

L’Ormeau
et l’artichaut poivrade dans un court bouillon au gingembre
Abalone, baby artichokes in a ginger bouillon

Le Bar
pole a la citronnelle avec une étuvée de jeunes poireaux
Pan-fried sea bass with a lemon grass foam and stewed baby leeks

L’Amadai
cuit en écailles et servi sur une nage au yuriné
Amadai in a lily bulb broth

Le Veau
en cote au plat avec un jus gras et escorte de taglierinis de légumes au pistou
Sautéed veal chop with natural jus and vegetable taglierinis flavored with pesto

L’Epeautre
du pays de Sault mitonne et dore a la feuille d’or
Sault wild oatmeal, gold leaf

Le Bahia
en fin crémeaux de papaye, jus de cassis
Guava and papaya granite, cream of cassis and orange macaroon

La Fraise
glacée aux coquelicots, en popcorns caramélises, sirop de cachaça
Poppy sorbet, caramelized popcorns, cachaça syrup

Le Café Express
Espresso

Petits Fours

Yes, this was a tasting menu. It was not a la carte. I ate each of those dishes in the course of about 4 1/2 hours. And had a 2002 Puligny-Montrachet 1er Cru, Les Pucelles to go along with all of it. Yes, just one bottle. Excess was not on the agenda for that night...

4/1/2006 10:26:56 PM (Pacific Daylight Time, UTC-07:00)  #   

 Thursday, March 30, 2006

Wow, not only does Vance Morrison have a blog--he's a performance architect on the CLR team--but he recently wrote two articles on reader-writer locks:

In them, he walks through a custom implementation of a lock, and then does some insightful performance analysis on it. As usual with Vance's writing, it's very detailed and precise.

3/30/2006 1:07:33 PM (Pacific Daylight Time, UTC-07:00)  #   

 Saturday, March 25, 2006

The profiler that ships with Visual Studio is great for "real" CPU profiling. But let's face it: there are still some situations where a good ole' stopwatch works just fine (of the System.Diagnostics.Stopwatch variety). For example, when you're trying to do some quick and dirty measurement on a very specific region of code, and don't want to deal with the rest of the noise.

The BCL stopwatch isn't inherently thread-safe. Even if you protect access to it (and somehow account for the overhead of doing so), it maintains a single counter. In the past, I've wanted to measure the total amount of time spent inside a select few regions of code, across all threads. The profiler works for this, but you need to get the sampling granularity right, and deal with all of the extra data collected. Then you have to mine it.

So I whipped up a stopwatch that maintains a counter that is the cumulative total of all threads that have started/stopped it, across an entire AppDomain. It's nothing incredibly clever, but I've found it to be quite useful. In many code projects I have, I've simply set up a file that declares a bunch of static ThreadSafeStopwatches, which I can then just call from anywhere.

Here's the code (also available for download here: ThreadSafeStopwatch.cs):

using System;

using System.Collections.Generic;

using System.Diagnostics;

using System.Threading;

 

/// <summary>

/// This class enables AppDomain-wide profiling of multi-threaded

/// code, by tracking a cumulative number of ticks spent across all

/// threads. If multiple threads start the same watch in parallel,

/// this class ensures that we count time for both threads, and that

/// they do not interfere with each other. It does this by storing an

/// internal stop-watch in TLS, and an instance-wide tick counter.

///

/// Note that the cumulative count is only ever incremented if a

/// thread actually calls Stop in its own stop-watch. If a thread

/// routine terminates w/out stopping the watch, it's as if it never

/// began.

/// </summary>

public sealed class ThreadSafeStopwatch : IDisposable

{

 

    /** Fields **/

    private long ticks;

 

    // These thread-specific fields are used to maintain a cache

    // of thread-safe stopwatches to actual stopwatches. For long

    // running threads, we can build up some amount of trash over

    // time, so a cache management scheme is implemented via a

    // combination of mechanisms: Dispose, ~ThreadSafeStopwatch,

    // GetWatch, and PruneCache.

    [ThreadStatic]

    private static Dictionary<WeakReference, Stopwatch> threadWatches;

    [ThreadStatic]

    private static int cacheCounter;

 

    /** Properties **/

    public long ElapsedTicks

    {

        get { return ticks; }

    }

    public float ElapsedMilliseconds

    {

        get { return (float)ticks / TimeSpan.TicksPerMillisecond; }

    }

 

    /** Methods **/

    ~ThreadSafeStopwatch()

    {

        Dispose(false);

    }

 

    public void Dispose()

    {

        Dispose(true);

    }

 

    private void Dispose(bool disposing)

    {

        // Clean up the associated cache entry with this object.

 

        // NOTE: I am explicitly not guarding this code-path by if

        // (disposing) { ... } because the Dictionary is not finaliz-

        // able. Thus, I know it is safe to access it. In the case of

        // non-process-exit finalizers, we do want to clean up the

        // associated cache entry, thus we access another object on

        // the finalizer code-path.

 

        if (!Environment.HasShutdownStarted)

        {

            Dictionary<WeakReference, Stopwatch> watches = threadWatches;

            WeakReference thisRef = new WeakReference(this);

            if (watches != null && watches.ContainsKey(thisRef))

            {

                // Deallocate the cache entry:

                watches.Remove(thisRef);

            }

        }

    }

 

    const int threadCachePruneCount = 100;

    private Stopwatch GetWatch()

    {

        if (threadWatches == null)

        {

            // First time called on this thread, allocate a new dictionary:

            threadWatches = new Dictionary<WeakReference, Stopwatch>(

                WeakRefEqualityComparer.Comparer);

        }

        else

        {

            // This has been called before. Increment the thread counter;

            // every so often, we prune out trash being held alive by our

            // cache.

            if (++cacheCounter % threadCachePruneCount == 0)

            {

                PruneCache();

                cacheCounter = 0;

            }

        }

 

        // Now look for the associated stopwatch:

        Stopwatch sw;

        if (threadWatches.TryGetValue(new WeakReference(this), out sw))

            return sw;

 

        // If we didn't find the stopwatch, simply return null to

        // indicate that the caller needs to allocate a new one:

        return null;

    }

 

    private void PruneCache()

    {

        // BUGBUG: This is probably a poor cache management policy. But

        // I'm only enabling this in DEBUG builds for now, and until I

        // run into a real problem, I'm not spending time on it.

 

        List<WeakReference> toRemoveWrefs = null;

 

        // Look for dead references, and add them to the list (which is

        // lazily created, by the way).

        foreach (WeakReference wr in threadWatches.Keys)

        {

            // If the weak-reference is no longer alive, we add it to the

            // list of 'to-remove' stop-watches.

            if (!wr.IsAlive)

            {

                if (toRemoveWrefs == null)

                    toRemoveWrefs = new List<WeakReference>();

                toRemoveWrefs.Add(wr);

            }

        }

 

        // If we found any dead entries, remove them now.

        if (toRemoveWrefs != null)

        {

            foreach (WeakReference wr in toRemoveWrefs)

            {

                threadWatches.Remove(wr);

            }

        }

    }

 

    [Conditional("DEBUG")]

    public void Start()

    {

        // We look in TLS to see if this thread has already allocated a

        // stopwatch for the current thread-safe stopwatch. This is

        // thread-safe, of course, since each thread gets their own list

        // of Stopwatches (reentrancy aside--there aren't any blocking

        // points below):

 

        // Since we are about to retrieve something from TLS, and use it

        // across a set of paired operations (Start/Stop), we mark the

        // beginning of a thread-affinity region.

        Thread.BeginThreadAffinity();

 

        // Access TLS:

        Stopwatch sw = GetWatch();

 

        if (sw == null)

        {

            // No watch was found, allocate a new one and publish it.

            sw = new Stopwatch();

            threadWatches.Add(new WeakReference(this), sw);

        }

 

        // First, ensure we haven't begun it yet. If the stopwatch is

        // already running, we ignore this call. This is consistent with

        the System.Diagnostics.Stopwatch

        // class's behavior.

        if (!sw.IsRunning)

        {

            // And if that check succeeds, start the stop-watch ticking.

            sw.Start();

        }

    }

 

    [Conditional("DEBUG")]

    public void Reset()

    {

        // Get the current stopwatch in TLS -- see above comments (in

        // Start) for details on thread-safety.

        Stopwatch sw = GetWatch();

 

        // If we found one, reset it.

        if (sw != null)

            sw.Reset();

 

        // And also set our cumulative ticks to 0.

        ticks = 0;

    }

 

    [Conditional("DEBUG")]

    public void Stop()

    {

        // Get the current stopwatch in TLS -- see above comments (in

        // Start) for details on thread-safety.

        Stopwatch sw = GetWatch();

 

        // First, ensure we are running. If the stopwatch isn't running

        // yet, we ignore this call. This is consistent with the System.

        // Diagnostics.Stopwatch class's behavior.

        if (sw != null && sw.IsRunning)

        {

            // Add the stopwatch's total time to our instance counter.

            // This has to be an interlocked operation, because the whole

            // point of this class is to be shared across threads. 'ticks'

            // is the only instance state.

            Interlocked.Add(ref ticks, sw.ElapsedTicks);

 

            // We reset the stopwatch because we want to start at 0 upon

            // the next invocation to 'Start' -- the cumulative time is

            // kept in the 'ticks' variable.

            sw.Reset();

 

            // We can now end the thread-affinity that was started in the

            // Start operation above.

            Thread.EndThreadAffinity();

        }

    }

 

    class WeakRefEqualityComparer : IEqualityComparer<WeakReference>

    {

        internal static WeakRefEqualityComparer Comparer =

            new WeakRefEqualityComparer();

 

        public bool Equals(WeakReference wr1, WeakReference wr2)

        {

            // For purposes of our hash-table, if two weak-references

            // refer to the same object, we consider them equal.

            object o1 = wr1.Target;

            object o2 = wr2.Target;

 

            if (!wr1.IsAlive || !wr2.IsAlive)

                return false;

 

            // We shouldn't ever have null weak-references that aren't

            // dead.

            Debug.Assert(o1 != null && o2 != null);

 

            // If the two underlying objects are equal, we pretend the

            // weak-refs are too.

            return o1.Equals(o2);

        }

 

        public int GetHashCode(WeakReference wr)

        {

            object o = wr.Target;

 

            // Just return 0 for dead objects. We actually shouldn't

            // ever use a dead object for hashing, although there could

            // be some benign races above that result in this case. I

            // haven't convinced myself otherwise, and they will get clean-

            // ed up with the normal finalization code-path. It's A-OK.

            if (!wr.IsAlive)

                return 0;

 

            // Again, shouldn't get a live weak-ref that has a null

            // object ref.

            Debug.Assert(o != null);

 

            // Now, simply return the underlying object's hash-code.

            return o.GetHashCode();

        }

    }

}

3/25/2006 3:19:52 PM (Pacific Daylight Time, UTC-07:00)  #   

The Whidbey version of Rotor just went up for download on MSDN on Thursday. I downloaded it and built it this morning.

While some of the big rock features are getting press (e.g. generics, LCG, anonymous methods/closures, etc.), some of the smaller features are plenty cool, and can be grokked in their entirety in much less time. For example, do a 'grep -i nullable clr/src/*/*.*' if you want to see what went into implementing the Nullable DCR that Soma mentioned over here. And check out stuff like the WrapNonCompliantException function in vm/excep.cpp (and its callsites) to see how non-CLS exceptions get wrapped. And of course there's all that reliability work that went into Whidbey, leading to things like SafeHandles (vm/safehandle.cpp), and OOM and SO hardening.

Most of it's there for you to tinker with. Or to simply print out and enjoy, reading it as you sit beside the fireplace with a nice Bourdeaux. To each his (or her) own.

3/25/2006 3:00:51 PM (Pacific Daylight Time, UTC-07:00)  #   

 Tuesday, March 21, 2006

Ahh, I fondly remember the days of all-nighters.

I found this great interview, done in September '05, where Max (co-founder of PayPal) discusses the virtues of entrepreneurship and late-night hacking.

I miss those days. Oh, how I miss them.

3/21/2006 11:14:07 PM (Pacific Daylight Time, UTC-07:00)  #   

 Wednesday, March 15, 2006

Jim recently asked me to put together a few stand-alone articles, loosely excerpted from my upcoming book.

The first one just went online: One Platform to Rule Them All. This is based on a section from pretty early in the book. In fact, it's near the front of the first chapter.

I chose the title based on T-shirts the CLR team had printed up a few years back, which prominently displays this slogan on it along with some nifty graphics (e.g. a ring inscribed with code). As you can imagine, it's based on a famous book series, and more recently, movie series...

3/15/2006 11:59:07 PM (Pacific Daylight Time, UTC-07:00)  #   

 Wednesday, March 08, 2006

Here are a few recent happenings.

And... my new book is just about to come out (late March/early April). I'm happy to report that I recently accepted an offer to write another, Concurrent Programming on Windows, the goal for which is to be the definitive guide to concurrent programming on Windows, WinFX, and the .NET Framework. It'll show you how to write kick-butt software on these new shnazzy multi-core machines.

3/8/2006 4:16:15 PM (Pacific Standard Time, UTC-08:00)  #   

 Thursday, February 23, 2006

When you perform a wait on the CLR, we make sure it happens in an STA-friendly manner. This entails using msg-waits, such as MsgWaitForMultipleObjectsEx and/or CoWaitForMultipleHandles. Doing so ensures we pick up and dispatch incoming RPC work mid-stack, while the STA isn't necessarily sitting in a top-level message loop. In fact, an STA that doesn't pump temporarily can easily lead to temporary and permanent hangs (i.e. deadlocks), especially in common COM scenarios where reentrant calls across apartments are made (e.g. MTA->STA->MTA->STA). Even where deadlock isn't possible, failing to pump can have a ripple effect across your process, as components wait for other components to complete intensive work.

I was recently writing about this fact for an article, and realized I had leaped to an incorrect assumption. OLE creates special RPC windows for processing these apartment transitions. I knew that. But I had assumed that we blatently pump for messages for any windows. But in reality, we don't. We only pump the special "WIN95 RPC Wmsg", "OleMainThreadWndClass", and "OleObjectRpcWindow" RPC windows. Consider why.

If you were on a UI and you pumped your window's message queue, you could end up dispatching new events before old events had completed. If you dispatched a WM_CLOSE message on the same stack you were doing some other UI processing, you'd destroy the window before that other processing was done. Without the GUI message loop taking this into account somehow, you'd crash. There are other factors. Imagine you were processing a click event that required movement of several UI elements. If some bit of infrastructure--or perhaps your code--ended up pumping, UI invariants could be broken. (Lots of FX code pumps for RPC, by the way.) At best, this could lead to strange visual artifacts, and at worst a crash.

You'd also have to deal with the subtleties of reentrancy. I've written about this before. Imagine a UI event had waited on an auto-reset event, did some work, and then set the event. If another event--perhaps the same type--were dispatched while it was doing "some work," it might try to wait on this same event. This would be a deadlock. A pretty difficult one to track down too, especially if it only occurred if the user clicked on certain elements at precise timings to get the reentrancy to occur. This is already a problem with COM interop. But thankfully we don't burden Windows Forms and WPF programming with it too.

2/23/2006 12:37:08 PM (Pacific Standard Time, UTC-08:00)  #   

 Tuesday, February 21, 2006

I've been pretty busy lately orchestrating all the concurrency work the CLR is undertaking for the next few versions.

We have several long lead incubation projects underway to explore new programming models which--some of us at least--believe will make concurrent programming at least an order of magnitude easier. While I hope we ship them, the fate of those projects is currently unknown. We'll understand better over time what is feasible and in what timeframe.

But there's also a set of features and work items we are actively planning for in the mainline product. This includes looking at new abstractions (e.g. futures, barriers, better lock factoring, a richer variety of locks, lock leveling), fixing existing ones (e.g. nagging issues and obvious missing features (e.g. prioritization, isolation, diagnostics) with our thread-pool, improving RWL's performance), debugging and tool support, and general cleanup activities (e.g. getting our own locking practices in order).

If there's anything you'd like to see us consider in this category, please let me know. Either leave a comment here or send me an email at joedu at microsoft. I'll share it with the rest of the concurrency team, and we'll seriously consider it.

I also have a couple related articles to watch out for: one on deadlock avoidance and detection in next month's MSDN magainzine, and another on application responsiveness in WPF in DDJ. I'm also going to be on .NET Rocks next week discussing some of this.

Unfortunately, all of this has left very little time to blog. If you have ideas for topics, drop me a line.

2/21/2006 7:55:42 PM (Pacific Standard Time, UTC-08:00)  #   

 

RSS 2.0

Me
 

Joe Send mail to the author(s) is an architect and developer on a systems incubation project at Microsoft.

Recent

Search

Browse

Disclaimer:
The content of this site are my own personal opinions and do not represent my employer's view in anyway.

© 2013, Joe Duffy