Wednesday, June 22, 2005

Contrary to popular belief, I am still alive.

Is it me or has the whole blogsphere gone quiet over the past few weeks? Guess everybody I subscribe to is heads down shipping Whidbey.

I'm coming up on my 1yr anniversary with Microsoft. This is damned amazing, it really feels like only a few months. (Doesn't everybody say that?) It's been fun thus far, and I think it's only going to get better.

I can't come up with anything intelligent to say right now, so I'm just going go byte-byte (sic).

Byte-byte.

6/22/2005 9:08:22 PM (Pacific Daylight Time, UTC-07:00)  #   

 Tuesday, June 14, 2005

I'm on a CLR Road Trip right now, which basically means a bunch of us on the team are out visiting customers. We're trying to get a clearer understanding of how folks use (or would like to use) our technology, and also to get feedback on our future direction. Despite the 19 hour day yesterday (no joke, 6:30am-1:30am--that doesn't even include any flying or anything, we flew in the night before!), I'm having a blast. We've done a couple of these in the past.

I gave a "CLR Internals" talk to the Vancouver .NET user group last night. Thanks to those who showed up! I found othis MSDN article which drills nicely into some of my main topics: http://msdn.microsoft.com/msdnmag/issues/05/05/JITCompiler/. I'll try to post a deck in the next few weeks.

We're travelling to Calgary tonight and will be talking at the Calgary .NET user group in the next couple days.

6/14/2005 8:21:05 AM (Pacific Daylight Time, UTC-07:00)  #   

 Sunday, June 05, 2005

I used to play guitar quite a bit. I also used to create a lot of industrial music using my computer with a whole host of techniques that ranged from sampling and messing with random non-sound binary files, writing programs to generate sounds, sample munging, and plain old recording. I was also in a metal band somewhere late in high school. We played around at local clubs (Worcester/Boston, MA), and released a tiny album that went nowhere. I did the lead guitar, some of the remastering, and a lot of the sampling that made it onto the record.

We broke up, and I dropped the guitar in favor of a keyboard. (I had dropped the keyboard in favor of the guitar mid-high school, so I was technically "returning to my roots.")

A couple weeks back, I picked up the guitar again. I gave most of my recording and guitar equipment to my brother (I saved a guitar and small amp for myself). So I went to Guitar Center and picked up a Cry Baby and Metal Box. I also still have the Acid and Sound Forge software (a couple versions behind now), so I'm a one man band again.

It's refreshing to strum away in an attempt to relearn the scales and random tabs I used to know by heart. And I'm hoping to create some more industrial tunes. The recent NiN release, I think, reminded me of how fun this can be.

6/5/2005 2:37:11 PM (Pacific Daylight Time, UTC-07:00)  #   

 Saturday, June 04, 2005

I remember the distinct feeling I got the first time I entered 'csc.exe' at the command line and realized that the C# compiler wasn’t doing any exception checking for me. Surely I had done something incorrectly. Or so I thought. After a bit of time searching around, asking around, and banging my head against the wall (still got the fracture in my skull), I came to the realization that C# had chosen not to support checked exceptions. Hmm.

Surprised? Sure. Confused? Yep. Sad? Quite.

Once I began to understand the implications of this design choice, I just became more and more confused. How the hell do I know in what manner this method could fail?! Trial and error? Manually fuzzing an API and interpreting the errors it throws? Wow, that seems like a great process, eh? Guessing? Not caring? And nevermind the problem of it changing the exceptions it decides to throw in the next version once I managed to figure it all out. (Remember: exceptions aren’t a static part of a method’s signature like they are in Java, so the implementation is free to silently alter its exception throwing policy without notice.) The horridness of this situation seemed to spiral into a rotting pit of smelly bannana peels.

The next phase of my surprise, disgust, confusion, <insert word here> was to scour the tools and documentation. How could a hole this huge be left gaping open? Surely either the Object Browser in VS or the SDK documentation would fully expose this data, or maybe a magical Intellisense switch that revealed the Truth. Well, the answer was a resounding no. The SDK did an OK job (they’re getting better over time, but not nearly as good as JavaDoc can do with the information stored in metadata), but they weren’t complete, had to be grokked out of band, and were free to change silently (all still significant problems IMHO). And it didn’t do much good for my own code and its exceptions!

I learned the ropes programming in x86 ASM and C, when I was ~13 and into hacking MUDs and other games on Amiga, Linux, and then DOS. Then I moved to C++. And then I moved to Java, and I stayed there for 5-ish years. Along the way I experimented with LISP and Smalltalk, but never to a large degree. My first professional programming experience was with C++ and COM, but by and large I’ve written more lines of real project/product code in Java.

So my move to C# was one made with a lot of expectations around how things worked, and it took me some time to get through a whole slew of these little gotchas—small discrepancies between the JVM/Java and CLI/C#. But you know what? There’s only one that stands out in my mind today, and continues to bother the hell out of me: Checked exceptions.

Arguments against checked exceptions.

If you haven’t already, you should read this interview [http://www.artima.com/intv/handcuffs.html] with Anders Hejlsberg, the topic of which is C#’s decision not to support checked exceptions. I keep referencing C# as being the decision maker; while it’s true that the CLI could have natively support them and thus it's in part their decision too, I have little doubt they’d be there if C# 1.0 wanted them. Further, Java's implementation is compiler-specific, and has no JVM support other than the metadta, so it's not clear the CLI would have even had to provide support.

Anders is a smart guy, one of very few Distinguished Engineers at Microsoft, and has his head more than screwed on right and tight. The article sounds reasonable, although a number of times I'm left thinking the data is incomplete. Maybe my head's not quite right. Or maybe Java has corrupted my mind.

I honestly see this debate as another incarnation of the static vs. dynamic typing debate that often plagues the language space. There's not a right answer in principle, but there's certainly an answer to what's right for the majority of users of your language. Anders' certainly understands the target audience of C# more than I do, so his call was probably right. But I'm left feeling neglected, poor little Java programmer man.

Many people jump on the anti-checked exceptions bandwagon without ever having done significant programming in Java. I’m sure Anders isn’t in this camp. But a lot of folks are. And until you’ve done significant programming and maintenance on a complex system with checked exceptions, you are probably not going to appreciate the safety and self-documentation of intent that it provides. You absolutely cannot understand the benefits of checked exceptions by just writing a 10-20 line program.

Some common claims made against checked exceptions come down to:

• Most developers subvert them.
• They make it difficult to version code.
• You usually don’t want to catch an exception, you want to let it leak.

I disagree wholeheartedly with each of these statements. And here’s why.

Most developers subvert them.

Based on a decent amount of Java experience, this is incorrect by a long shot. In those 10-20 line programs I mentioned, yes a lot of folks write "throws Exception" at the end of their methods, swallow exceptions, or generally subvert the checked exceptions system. But do we really want to tune the C# language for such small programs? I would argue Python, Perl, or a lighter weight language that doesn’t even do static type checking (since that’s another similar "annoyance" which prevents programs from compiling) for this category of programs. When checked exceptions are understood or provide value, subversion is unnecessary. I admit some users don't understand them (and hence use them incorrectly in the way somebody might use any language feature incorrectly), and that they don't provide value in the small, simple, script-ish program cases.

They make it difficult to version code.

Hell yes they do! But it's "difficult" in a good way, the same way that it's "difficult" to change the semantics or signature of a piece of code which is relied on throughout a complex system. If you could change it without compiler help, well, you'd be working in a dynamic language (and we ain't going there right now buddy).

There are two basic cases to consider: versioning public APIs consumed by somebody else and internal methods.

The API case is just like any other static feature of an API. Can you change the signature of your method after you ship it? Yes you can, if you want to break people. The same is true of checked exceptions. If you don’t have a statically checked list of exceptions you can throw, you’re going to break somebody anyhow.

If my API does this today:

object Foo()
{
    // …
    if (theNetworkIsUnavailable)
        throw new FooException();
    // …
}

And tomorrow it does:

object Foo(int x)
{
    // …
    if (theNetworkIsUnavailable)
        throw new NetworkConnectionUnavailable();
    // …
}

Your code that used to say:

try
{
    object o = Foo();
}
catch (FooException e)
{
    // Do something with it
}

Will no longer catch the error condition, since NetworkConnectionUnavailable will just head right past the FooException catch block. Hopefully your program has a backstop to catch this and respond accordingly, but depending on your error handling logic, this is likely to result in bugs either way. Is this the type of thing you want silently slipping past a compiler? Probably not. If you’re writing an API, error conditions are like any other pre-/post-condition, and should be treated as such. If the type system can enforce this, all the better for program correctness I say. (Static vs. dynamic languages again, ugh...)

The implementation case is simply not a problem. An implementation, by its definition, is a method which other programs or units don't get to see or use—i.e. non-public—and thus any problems are caught at compile time. You don't have to recompile dependencies, for example, which you may or may not have access to. This is just like changing the type or parameter list for a method… You make the change, see where things fall out, and fix them. It's the ordinary static language "beat the compiler" game. This is ordinarily a good thing, as it forces you to take a look at the exception handling code at the call sites to ensure they remain correct with the new exception behavior.

You usually don’t want to catch an exception, you want to let it leak.

I would argue about the use of "usually" here, a more correct word being "sometimes." And because this is a less-than-average situation, I don’t think the language should be tuned for it. It needs to support it, sure, but I am arguing it’s not the common case.

Note that Java supports unchecked exceptions in the type system. Basically, anything that derives from RuntimeException needn’t be checked. These are common errors like OutOfMemory, StackOverflow, NullRef, and so on, which a program almost never catches. Note that it does derive from Exception, so they don’t escape past general error handling code. These are by and large the errors that developers merely want to leak, so the checked exception subsystem doesn’t get in your way here at all!

I also believe the style of coding which has a main entrypoint wrapped in an exception handler is not the best way to do exceptions management. Perhaps I’m whacked, but in most Java systems I’ve worked on, letting an exception rip up the callstack to the toplevel is not the preferred approach, and wouldn't make it past a single code review. Usually there’s a common function used to publish an error (e.g. pop UI, write to the log, and so on) once it gets caught, but unless we need to tear down the application, the exception never shoots through the entire program to the top, leaving holes in the brains of your callers. Sure we have a handler at the top in case something escapes, but it’s not relied on as a crutch when we might be able to recover.

But it ain’t perfect.

I agree that Java’s implementation of checked exceptions isn’t perfect. But I feel much more comfortable in it than I do in C#’s fully unchecked system. Both suffer from a common problem of code duplication or over-catching in the handlers. My experience also shows that Java’s exceptions class hierarchy is better in the JSE (I know for sure at least two other smart people agree with this), and its users tend to get their own exception hierarchies right. Without a good factoring, a scaling problem ensues as you need to deal with a larger quantity of exceptions. Your catch block has to either have 10 incarnations of similar code for each exception type that can get thrown, you over-catch, or you give up and subvert the system. But provided that the factoring is clean, you can easily skirt this issue.

Still, having a "catch (Exception ex) where ex : FooException, BarException, FooBarException { }" syntax which caught any instance of those three and stored it into an Exception-typed variable would help to eliminate some of the nastiness of exception handling code. An implementation of this using exception filters in CLI would be trivial.

Having to deal with checked exceptions when you don’t care would be a nice thing to be able to express, for example, in the case of smaller programs. A compiler switch would suffice here so long as the runtime doesn’t actually enforce the "handle everything but what you’ve said you’re going to throw" policy. The JVM itself doesn’t do this check, that it’s a compiler-only enforced policy, so it's reasonable to expect that a CLI implementation would be a compiler deicision. This JVM behavior exists so that versioning can occur without recompilation.

There are other possible improvements, and certainly a whole category of other statically-detectable things (pre-/post-conditions, invariants), but I’ll stop here. Since this is mostly a language decision, it's interesting to see some approaches to solving the problem. For example, check out Spec# [http://research.microsoft.com/specsharp/] a nifty rifty roo MSR project.

Checked exceptions as implemented in Java ain’t perfect, but sufficiently close that I miss it.

6/4/2005 9:59:52 PM (Pacific Daylight Time, UTC-07:00)  #   

 Sunday, May 29, 2005

Spring Release weekend in Walla Walla, WA happened a couple weeks back. This is the time of year when all of the local wineries open their doors for tastings and highlight recent vintage releases for an extended weekend. We went down for the entire event. It got really crowded on Sat & Sun, so our strategy--to head down on Thu afternoon--was definitely the right call. We were able to get some f2f time in with some of the winemakers and owners.

We picked up a slew of great tastes in the process. Our collection has slowly been growing (or rather, not depleting faster than it grows ;) ), and has surged past the 100-mark to ~115-ish bottles. Since we've moved out west, the number of WA and CA wines has grown past half of our collection. This is surprising to me since nearly all of our wines were from Italy and France prior to the move. But my pivot table says we have 56 US bottles, followed by 21 Italian, and 16 French. The rest are from Germany, Australia, Portugal, Spain, Chile, and Canada, in descending order.

We came home with just over 2 cases total of:

All in all, a great addition to our collection. Isenhower and K were my absolute favorites by quite a distance. I'd recommend getting your hands on the latest vintages for these folks--Isenhower sold out completely while we were down there and K sold out on most of their selections, so I'm assuming a bunch of local wine shops purchased 'em up. Good luck on the search! By the way, if you see the House Wine by the Magnificent Wine Co., buy it up. These puppies are roughly $10/bottle, are hand crafted by the same guy who does the other K wines, and have a great full mouth feel with amazing rich body, something you typically don't find in such an inexpensive wine. It's a steal.

Anybody out there big grape-heads like myself?

5/29/2005 2:01:22 PM (Pacific Daylight Time, UTC-07:00)  #   

 Saturday, May 28, 2005

My implementation of Software Transactional Memory (STM) on Rotor is coming along nicely. I mentioned it briefly here and here. I've taken a different approach than most, actually taking advantage of the JIT and EE to do my dirty work for me. Some prefer to stay inside the cozy confines of managed code, but I'd like to understand better the impact that my design might have on non-transactional code. I admit that my approach is likely not viable for real commercial use mostly due to its intrusive nature. Unless all code were transactional, which it's not. But this is partly what I'd like to understand better.

Most people confuse the idea of memory transactions with, say, database transactions. The concepts are very similar, but memory transactions are about dealing with concurrency at a much finer grained level (just your machine). Rather than using locks to protect shared memory, STM prefers to enable possibly conflicting operations to happen (inside a transaction), and then to resolve them by comparing transaction records to memory at commit time. The theory behind this work has been heavily influenced by database software, and the techniques it uses for its own memory consistency models. You can read more about STM on Tim Harris's site (MSR "smart dude"), e.g. these research papers.

My particular STM design affects the JIT, some core parts of the EE, requires new managed classes, and also a bunch of new FCalls. I'm writing up a paper on this project as I go, but in a few bullets here are the main points:

  • Blocks of code can be marked as "atomic", along with options such as what style of locking (optimistic, pessimistic), granularity (memory location/field, object), and retry semantics (0..n).
    • Atomic blocks get hoisted into methods of their own and annotated with a System.Threading.AtomicTransactionAttribute. This requires a hack to the C# compiler; for now, I've simply required that people write the method and tag it with [AtomicTransaction] on their own.
    • The runtime knows about [AtomicTransaction] and treats these methods differently (details below).
    • Note: I would like to understand how best to take advantage of and integrate with System.Transactions, but haven't gotten around to it yet.
  • The MethodTable in the EE now contains a second set of vtable slots.
    • This second vtable contains atomic versions of the JITted code. We'll see what this means later.
    • All of the atomic vtable slots get pre-populated with a special JIT stub that invokes the JIT with a flag to indicate it desires atomic semantics. If you never call a method atomically, the stub will never get replaced with the JITted code, so this might not be as heavyweight as you first thought.
    • Any methods marked with [AtomicTransaction] get the atomic JIT stub for their ordinary vtable slot too.
  • When the JIT is called with the atomic flag, it does a couple things out of the ordinary:
    • Method calls dispatch using the atomic vtable instead of the plain ole' vanilla vtable.
    • Any calls to ldfld, ldsfld, ldelem, ... get routed through a runtime helper. This helper knows about previous reads/writes in the same transaction by consulting our transaction records in TLS. If we have written, it retrieves that. Otherwise, it reads the underlying memory, records the value read, and loads the value.
    • Any calls to stfld, stsfld, stelem, ... get run through a similar runtime helper. This simply records the written value in the active transaction.
  • Then there are Commit/Rollback APIs which resolve conflicting read/writes, and actually writes any updates to the memory. It does this through a nonblocking algorithm (assuming optimistic locking) which is guaranteed to never deadlock.
    • If conflicts are found (memory value is different than the recorded read), it consults the transaction's retry options. If we haven't spent our retries left count, we loop back and attempt the transaction over again.
    • Otherwise, we blit any writes to memory, mark the transaction as committed, and move on.
    • Note: I also support nested transactions which changes the behavior of commit slightly. Firstly, when a new [AtomicTransaction] method is reached, we allocate a child transaction. Any reads must consult the parent chain for writes that parent transactions have recorded. Then during commits for children, we simply copy the contents of child reads/writes to the parent. The parent is then responsible for detecting consistency with memory when it decides to commit.
5/28/2005 1:20:06 PM (Pacific Daylight Time, UTC-07:00)  #   

 Friday, May 27, 2005

What Video Game Character Are You? I am Pacman.I am Pacman.

I am an aggressive sort of personality, out to get what I can, when I can. I prefer to avoid confrontation, but sometimes when it's called for, I can be a powerful character. I tend to be afflicted with munchies constantly. What Video Game Character Are You?
5/27/2005 12:46:55 AM (Pacific Daylight Time, UTC-07:00)  #   

 Saturday, May 21, 2005

It's been a while since I've posted anything about my book.

I'm wrapping up my next chapter this weekend (I'm over half way done page-count-wise :P), and was interested in any opinions out there on the content. To reiterate: my goal is to cover CLR and FCL topics which haven't gotten a lot of love to date, with a focus on v2.0, while still painting a complete enough picture to be useful mostly on its own (with some previous experience with the technology of course). I think assemblies, loading internals, and how to deal with versioning problems are some that have been neglected (except for on Suzanne Cook's blog, of course).

Also, if you're interested in being a reviewer, drop me a line at joedu (at@at) bluebytesoftware (DOT.DOT) com. Please provide a brief bio of yourself so I know who's serious and who isn't.

Here's a rough layout. Comments welcome (uninteresting bullets, missed items, etc.):

Assemblies: Packaging & Deployment

  • Units of Execution & Reuse
    • A Look Inside Portable Executable (PE) Files
    • Strong Naming
    • Metadata, Metadata, Metadata
    • Manifests & Modules
    • Dynamic Assemblies
  • Asssembly Loading
    • Loading By Hand
    • Inside the Bind, Map, Load Process
    • Probing ("Fusion")
    • Custom Assembly Binding
    • Execution
      • EXE Bootstrapping
      • Other Hosts (IE, SQL Server "Yukon")
      • Code Sharing
      • AppDomain Isolation
  • Deployment & Configuration
    • XCopy
    • The GAC
    • ClickOnce
    • Configuration
  • Versioning
    • AppCompat
    • Rolling Forward
    • Versioning Libraries
    • Versioning Applications
    • Servicing Deployed Apps

 

5/21/2005 11:46:47 PM (Pacific Daylight Time, UTC-07:00)  #   

Structured exception handling and all of the challenges associated with it aren't new topics about which developers must worry. I was flipping through my copy of The C++ Programming Language (3rd Edition) this morning and ended up rediscovering Appendex E on C++ Standard-Library Exception Safety. Understanding what consistency and failure guarantees you intend to make with your library and then actually making them, knowing what to expect from a pre-written library and when and how to program defensively against it, and all such related topics were challenging with C++ and remain a challenge with C#. It's amazing how many of these things are directly transferrable between technologies.

He mentions in the book that all of the appendices are available on the web, so I did a quick search. Voila! Here it is [PDF].

Even if you're not a C++-er, you'll find a ton of great information in that article. Enjoy. I did (again).

5/21/2005 1:20:57 PM (Pacific Daylight Time, UTC-07:00)  #   

 Saturday, May 14, 2005

What is this program's output and why?

using System;

public class NullableTest
{

    static void Main()
    {
        int? x = null;
        object y = x;

        Console.WriteLine("Test #1: {0}", x == null);
        Console.WriteLine("Test #2: {0}", y == null);
        Console.WriteLine("Test #3: {0}", IsNull<int?>(x));
    }

    static bool IsNull<T>(T t)
    {
        return t == null;
    }

}

Is this intuitive, confusing, or <something else>?

5/14/2005 8:49:07 AM (Pacific Daylight Time, UTC-07:00)  #   

 

RSS 2.0

Me
 

Joe Send mail to the author(s) is an architect and developer on a systems incubation project at Microsoft.

Recent

Search

Browse

Disclaimer:
The content of this site are my own personal opinions and do not represent my employer's view in anyway.

© 2013, Joe Duffy