|
Personal Info:
Joe  leads the architecture of an experimental OS's developer platform, where
he is also chief architect of its programming language. His current mission is to enable
writing large-scale software that is reliable, secure, and scalable by-construction. Before this, Joe
founded the Parallel Extensions to .NET project.
He has been granted 19 patents, with 49 pending. When not working, Joe enjoys travelling with his wife,
writing books, writing music,
studying music theory & mathematics, and doing anything involving food & wine.
My books
My music
Disclaimer:
The content of this site are my own personal opinions and do
not represent my employer's view in anyway.
© 2012, Joe Duffy
|
|
 Sunday, July 10, 2005
I'm wrapping up a chapter in my book on Unmanaged Interop this evening. In the process, I've fallen back in love with some classics on my bookshelf:
COM is still cool in my book. (Pun not intended.)
And furthermore, I can't tell you what a blessing it is to be able to write about a topic, encounter a question or two, and walk right down the hall to the guy's office who 1) knows the absolute most about a specific technology and 2) is kind enough to answer questions in exceeding detail. I hope this translates into a better end product.
Here are just a few (externally available) amazing resources related to hosting, CERs, and SafeHandles, the new face of Unmanaged Interop for V2.0:
The Designing .NET Framework Class Libraries series that aired on MSDN a while back was good fun. The format was fun for us here at Microsoft (video on Friday, chat on the following Wednesday). I hope those who participated agreed. If not, I'd love to get your feedback: what did you like, what didn't you like about the format? Should we do precisely the same way if we do that type of thing in the future... make a couple tweaks?
Here's a cross index to each of the talks with associated chat transcripts:
- Setting the Stage
- Naming Conventions
- Rich Type System
- Member Types
- Designing Inheritence Hierarchies
- API Usability
- Designing Progressive APIs
- CLR Performance Tips**
- Designing for a Managed Memory World**
- Understanding Interoperability**
- Packaging, Assemblies, and Namespaces
- FxCop in Depth
- Enabling Development Tools**
- Security**
- Q&A**
We had a great viewing count (I don't have the #s off hand), comparable to some of the more popular MSDN articles and downloads.
Unfortunately, a few chat transcripts are still missing from the home page. I apolgoize for this, it's in part my fault. The good news: I've been assured they'll be up this week.
In the interim, this cross index should suffice. Those marked with ** are currently missing their chat transcripts. The videos for all are available from the Talk Homepage links. Enjoy!
Friday, July 08, 2005
Brad just pointed out the recent DotNetRocks episode with the PDC planning folks. They give a good insight into the insanity that goes behind planning such a huge event. I'm helping to organize the CLR Team's presence there, but my part is minimal compared to some guys (like the folks in the interview).
I'm really psyched about PDC this year. We've got a plethora of great talks lined up, and some of the best technical speakers you'll find this side of tha' Missisippi. Just take a look at the talks we've release thus far...
And they have an RSS feed so you can keep an eye on the talks as they are released!
If you haven't convinced your boss to pay yet, better work harder (and faster).
The fun begins 2 months from this weekend!
When access to a location in memory is shared among multiple tasks executing in parallel, some form of serialization is necessary in order to guarantee consistent and predictable logic. Furthermore, in many situations, a number of such reads and writes to shared memory are expected to happen “all at once,” in other words atomically. Serializability and atomicity are both often implemented using mutual exclusion locks. This is bread and butter stuff.
An important concept in concurrent programming is forward progress. This is the idea that the largest number of parallel tasks should make the most amount of progress towards their goal as possible for every given time unit of execution. If you can manage to divvy up the work such that all tasks can execute completely logically independently from each other—called linear parallelization, something that is actually difficult to achieve in practice—then sharing resources such as memory can quickly bog down your theoretical linear speedup in practice. Shared memory prevents each task from making forward progress because there are points of execution where access to resources must be serialized. That means code has to wait in line in order to execute. That’s generally bad.
What an ambitious introduction. Unfortunately, I must constrain the rest of this particular article to some very precise, more manageable topics… Else I would never complete it, and might end up with a book on my hands. And furthermore, I am going to constrain my conversation to the CLR, with a focus on the Monitor APIs. I intend to write a series of these articles over the coming months, since I’ve been writing a lot about the topic in general lately.
Eliminating deadlocks
Deadlocks are well documented out there, and are simple to understand. Thus I will start with them. Deadlocks are by far the #1 forward progress inhibitors. While contention over shared memory can prevent all but one parallel unit from making forward progress (in the extreme case, where all tasks request access to the same resource simultaneously), deadlocks prevent all units involved in the access from making forward progress. Without detection and correction logic, your program is likely to come to a grinding halt.
For example, consider two bits of code running in parallel:
#1: #2
lock (a) lock (b)
{ {
lock (b) lock (a)
{ {
// atomic code // more atomic code
} }
} }
As written, these can easily get into a so called deadly embrace. Because they acquire and release locks in the opposite order, it’s not a difficult stretch to imagine #1 acquiring a, #2 acquiring b, and then #1 trying to acquire b (blocking forever), and #2 trying to acquire a (also blocking forever). The result is often a hung application or background worker thread. The result is a frustrated user having to open up Task Manager so they can slam the End Task button tens of times… and then waiting for dumprep.exe to get done with its jazz.
The solution to this problem is often “acquire and release locks in the same order,” but that’s seldom achievable in practice. It’s more likely that a and b are acquired in entirely separate functions, deep in some complex call-stack, which can moreover alter the flow of control at runtime. It’s not always a statically detectable situation. Another solution is to write your code so that it can back off of lock acquisitions if it suspects a deadlock has occurred. With the new Monitor.TryEnter API, this is relatively trivial to do (in the simple case).
Regardless of how ridiculously simplified this scenario is, let’s start here. It’s easier to understand and solve.
A quick note on SQL Server
Through the CLR’s hosting APIs, you can actually hook all blocking points, including Monitor.Enter calls. SQL Server (and possible other sophisticated hosts) use this to detect deadlocks and prevent them from occurring. Unfortunately, I don’t know their policy for handling, but presumably it is a fair one whereby a victim is chosen at random and killed. This is consistent with the way SQL Server handles deadlocks pertaining to data transactional deadlocks. Chris Brumme’s weblog entry on Hosting has a plethora of related information.
Lock ordering and optimistic deadlock back-off
An old fashioned solution to this problem is to mentally tag all locks in your program, and ensure that you acquire them in a consistent manner. You could use a simple algorithm, such as “sort by variable name.” This works so long as you never alias a memory location. Oh, and so long as you don’t make a mistake when you’re writing the code (and anybody else who is touching your program). But this would be error prone and laborious. We can do better.
We could, for example, write a function that accepts a list of objects and does a few things in the process of locking on them:
- Sorts the objects in identity order to ensure consistent lock acquisition ordering;
- Uses a simple back-off strategy in case there are other lockers not using our ordered locking scheme.
The code might look like this:
static int deadlockWait = 15;
static bool EnterLocks(params object[] locks)
{
return EnterLocks(-1, locks);
}
static bool EnterLocks(int retryCount, params object[] locks)
{
// Clone and sort our locks by object identity.
object[] locksCopy = (object[])locks.Clone();
Array.Sort<object>(locksCopy, delegate(object x, object y)
{
int hx = x == null ? 0 : RuntimeHelpers.GetHashCode(x);
int hy = y == null ? 0 : RuntimeHelpers.GetHashCode(y);
return hx.CompareTo(hy);
});
// Now begin the lock acquisition process.
bool successful = false;
for (int i = 0; !successful && (retryCount == -1 || i < retryCount); i++)
{
successful = true;
for (int j = 0; j < locksCopy.Length; j++)
{
try
{
if (!Monitor.TryEnter(locksCopy[j], deadlockWait))
{
// We couldn't acquire this lock, ensure we back off.
successful = false;
break;
}
}
catch
{
// An exception occurred--we don't know whether we got the last lock
// or not. Assume we did. We indicate that by incrementing the counter.
j++;
successful = false;
throw;
}
finally
{
if (!successful)
{
for (int k = 0; k < j; k++)
{
try { Monitor.Exit(locksCopy[k]); }
catch (SynchronizationLockException) { /* eat it */ }
}
Thread.Sleep(0); // Might increase chances that a thread will steal a lock (good).
}
}
}
}
return successful;
}
This method is actually sufficiently complex that it warrants a bit of discussion. Most of the complexity stems from our paranoia about orphaning locks coupled with the back-off algorithm. Notice that we first sort the list of locks, using the System.Runtime.CompilerServices.RuntimeHelpers.GetHashCode method for comparisons (this function returns a unique hash code based on an object’s identity). We then use a loop to try acquisition of the locks. If an acquisition fails, we begin the back-off logic by unraveling any locks we had acquired previously, yielding the thread to increase the chance that another possibly deadlocked thread is able to make forward progress, and starting over again.
Of course, a real function would probably offer a timeout variant. The timout for the Monitor.TryEnter isn’t configurable, the retry Count is near meaningless to the user, and the routine is still subject to denial of service attacks whereby somebody grabs a lock and holds on to it forever. In that case, we’ll loop forever (unless an explicit retryCount is provided, it defaults to -1 which means infinite). We also need a similar, although much simpler, ExitLocks mechanism. I’ve omitted these implementations for brevity. Lastly, in the face of asynchronous aborts, this code falls on its face. Nevertheless, it demonstrates the concepts (I hope).
Cross call-stack ordering and back-off
Again, this strategy works only if you know all of your locks up front. With deep call-stacks, this may not be the case. For example, consider:
void f(bool b)
{
if (b)
{
lock (a)
{
g(!b);
}
}
else
{
lock (b)
{
g(!b);
}
}
}
void g(bool b)
{
if (b)
{
lock (a)
{
// some atomic function
}
}
else
{
lock (b)
{
// some other atomic function
}
}
}
If these were called from two parallel tasks, one task run as f(true) the other as f(false), you’d have a similar, although much more complex and difficult to follow, deadlock scenario. We might be able to (almost) solve this, too, however, with some really ugly hacks that I wouldn’t suggest anybody uses in real code. With that caveat, let’s take a look at them…
We could learn a thing or two from STM. If we performed only idempotent and reversible operations inside the atomic (lock protected block), you could imagine a more complex back-off strategy that spanned multiple levels of a call-stack. This requires you to make a lot of assumptions, use exceptions for control flow, and quite truthfully some unorthodox strategies (including polluting your thread with state). Moreover, without some form of transactional memory, rollback in the case of failure has to be done manually. These are in general bad practices, but the result seems to exhibit some redeeming qualities.
Here’s a big steaming pile of code that attempts to demonstrate a possible implementation:
static LocalDataStoreSlot atomicSlot;
static Par()
{
atomicSlot = Thread.AllocateNamedDataSlot("AtomicContext");
}
internal class AtomicFailedException : Exception
{
public AtomicFailedException() {}
}
internal class AtomicContext
{
internal AtomicContext parent;
internal List<object> toLock = new List<object>();
}
static bool DoAtomically(Action<object> action, params object[] locks)
{
return DoAtomically(action, null, locks);
}
static bool DoAtomically(Action<object> action, Action<object> cleanup, params object[] locks)
{
return DoAtomically(action, cleanup, 10, locks);
}
static bool DoAtomically(Action<object> action, Action<object> cleanup, int retryCount, params object[] locks)
{
bool entered = false;
// We have to maintain our context so that we can unravel the parent correctly.
AtomicContext ctx = new AtomicContext();
ctx.toLock.AddRange(locks);
ctx.parent = (AtomicContext)Thread.GetData(atomicSlot);
Thread.SetData(atomicSlot, ctx);
try
{
for (int i = 0; !entered && i < retryCount; i++)
{
if (entered = EnterLocks(10, ctx.toLock.ToArray()))
{
bool retryRequested = false;
try
{
action(null);
}
catch (AtomicFailedException)
{
if (cleanup != null)
cleanup(null);
retryRequested = true;
}
finally
{
if (entered)
ExitLocks(locks);
if (retryRequested)
entered = false;
}
}
}
}
finally
{
// Reset the context to what it was before we polluted it.
AtomicContext cctx = (AtomicContext)Thread.GetData(atomicSlot);
Thread.SetData(atomicSlot, cctx.parent);
if (!entered && cctx.parent != null)
{
cctx.parent.toLock.AddRange(cctx.toLock);
throw new AtomicFailedException();
}
}
return entered;
}
The last overload is obviously the most complex, and the meat of the implementation. DoAtomically uses a back-off strategy not unlike the first EnterLocks function. In fact, it uses EnterLocks for lock acquisition. DoAtomically maintains a context of the locks that must be acquired, and can be chained such that there is a parent/child relationship between two contexts (representing multiple DoAtomically calls in a single call stack).
The function then goes ahead and attempts to acquire each object that much be locked. If it succeeds, it calls the delegate that was supplied as an argument. This delegate can likewise make DoAtomically calls which will recursively detect deadlocks and perform escalation if they occur. Note: there is some noise here. Because of the small timeout we use, a function that holds a lock for an extended period of time can give the impression of a deadlock. This number could probably use some tuning. Further, I haven’t tested the interaction between this code and non-DoAtomically code. Presumably, it would be more succeptable to livelock, but wouldn’t actually fail or deadlock (assuming the other code doesn’t mount a denial of service).
The escalation policy we use is to perform cleanup logic (since we tried to execute the action, there could be broken invariants that must be restored), mutate the parent context so that it will attempt to acquire the locks the child tried to acquire (and failed), and essentially unravel the stack to the parent (using an exception—ugh—I think continuations would make this a much prettier situation). The parent then tries to acquire its own locks in addition to the child locks that got escalated. This can be an arbitrarily nested call-stack, so a parent could end up with more than just a single child’s locks to acquire. But this ensures an entire call stack’s worth of locks are acquired in an ordered fashion, and furthermore backed off of all at once. The obvious downside to this approach is that you end up taking a coarser grained lock than necessary, but with the benefit of avoiding deadlocks.
Assuming all of the back off and retry succeeds, it will return a true indicating success. If it doesn’t, and it’s exhausted all of its retries and escalation space, the topmost atomic block will simply return false to indicate failure. Honestly, an exception in this case might be more appropriate.
An overly simple example
A small test function that uses this (sorry, I didn't have time to write up a more complex one), is as follows:
static int i = 0;
static object x = new object();
static object y = new object();
static void Main()
{
List<Thread> ts = new List<Thread>();
for (int j = 0; j < 20; j++)
{
Thread t = new Thread(new ThreadStart(delegate {
DoAtomically(delegate {
i++; Console.WriteLine("{0}, {1}", Thread.CurrentThread.ManagedThreadId, i); i--;
}, x, y);
}));
ts.Add(t);
}
ts.ForEach(delegate (Thread t) { t.Start(); });
ts.ForEach(delegate (Thread t) { t.Join(); });
}
Of course, all threads should print out the number 1.
A brief word on livelock
A quick word on livelock with the above design. With an escalation policy as defined above—your standard back-off, yield, and retry—it is highly susceptible to live-lock. This is a situation where code is trying to make progress, but chasing its own tail, or continually hitting conflicts. Consider what happens if a very long block and a very short block are competing in a deadlock fashion for the same resources.
The policy defined above will always back-off and retry, meaning that a short transaction has less work to do in order to perform its task. If the larger block is higher priority than the smaller one, we’re unfairly favoring the small block simply due to its size. But similarly, a long running block could acquire a lot of resources, and the smaller block could quickly try (and retry) to acquire locks, fail, and give up.
Lock leveling or some more intelligent queuing system might help out here. But I’ve written enough already.
Future topics
If you’re interested in a particular concurrency-related topic, let me know!
I’d like to spend more time in the future on:
- Events and signaling;
- Managing large groups of complex parallel tasks;
- Implicit parallelism, e.g. using compiler code generation and IL rewriting;
- STA, COM and UI programming, reentrancy;
- More on livelock—it happens in a lot of contexts—and some ideas on how to solve them;
- Lock free programming, and why you should avoid it.
Feedback will help me write about things you want to know about.
Happy hacking!
Saturday, July 02, 2005
Those guys on the VC++ team have been busy workers. In the Whidbey release of C++/CLI (was Managed C++), they've added a whole big batch of new features. The best part? Some of these are things you simply can't do in C#. Put another way, C++/CLI exposes a larger set of the underlying features that the CTS/CLI has to offer.
For example, want to create a ref type on the stack? Fine.
MyType mt("Foo");
On the GC heap? Alright.
MyType^ mt = gcnew MyType("Foo");
(Note if you're wondering "how'd they do that?" The answer is that the first case only has stack semantics. It still lives on the GC heap. In other words, it acts very similar to 'using' in C#, but it maps nicely to the C++ programmer's existing understanding of stack versus heap semantics.)
Similarly, did you want to maintain a typed reference to a boxed value type? Ok.
int^ i = gcnew int(5);
This compiles to IL which uses modopts to store typing and boxing information so that the runtime/JIT know how to treat it, and for the verifier so that it knows it's being used in a type-sound manner. Did you need a Nullable<int>? Nonsense! Just set your reference to null and you've got it:
int^ i = nullptr; // now it's null
i = gcnew int(5); // now it's not
Furthermore, with stack semantics for ref types, deterministic finalization is simple. Just write a destructor for your type (it gets compiled down to a Dispose method), and it gets invoked for you when leaving the scope. Just like the old C++ days. This means you can say:
{
StreamReader sr(...);
// do some stuff with stream
}
And sr gets disposed just prior to leaving the block scope. You can also create your own standard resource mgmt wrappers like that come with TR1 (e.g. tr1::shared_ptr<>). Using the terms that Rico Mariani came up with in a meeting a while back, you've got "the bang" and "the twiddle"...
!MyType() {} // the bang: a finalizer
~MyType() {} // the twiddle: a Dispose method
They've also implemented STL with full interoperability with Whidbey's generics.
They've also implemented OpenMP, a fairly ubiquitous shared memory parallelism library that I've been using a lot for research recently. Now they just need MPI and the world would be complete.
I'm using C++ for many things lately (mostly due to my Rotor work), and I have to say: as I use it more and more, I am starting to miss it. But admittedly I do sometimes prefer the cozy confines of managed code. C++/CLI enables me to nicely sit in between the two worlds, getting the best of both (and leaving behind the worst). There's a hell of a lot more to it than this post surfaces. Check it out.
Happy hacking!
Monday, June 27, 2005
I need some feedback from the community here.
You see, we've got a veeerry interesting late-game DCR that we're wrapping up this week. It was on the order of 5 weeks of development work to implement. In other words, not small. (DCR == "Design Change Request," i.e. not a bug. We're changing the design of a feature we've already implemented.) I'll be less secretive once I am able to. In fact, I can't wait to blog about this one in detail...
Anyhow.
We're fundamentally mucking with the type system. (Yes, this makes me quite queasy.) In doing so we've introducing a bit of an oddity. The unfortunate thing is that we won't have a Beta with this functionality included. So... software being the inexact science it is, I figured maybe I'd get a response or two from folks out there. Not quite the same, but better than nothing I proclaim!
Now that I've played it up, it's really quite simple. What if
(new T()).GetType() == typeof(T)
evaluated to false? How horrible would that be? Think about it. I believe this is a fairly fundamental invariant we preserve when spanning the static and dynamic type systems.
If we broke it, would you lose sleep? Hate us? Throw your copy of Whidbey into the trash compactor and pick up Java 5 instead? (Go back to COM, VB6, ... DOS programming perhaps?)
I'm being flippant. Mostly because I'm extraordinarily tired. But I really wonder.
Wednesday, June 22, 2005
Contrary to popular belief, I am still alive.
Is it me or has the whole blogsphere gone quiet over the past few weeks? Guess everybody I subscribe to is heads down shipping Whidbey.
I'm coming up on my 1yr anniversary with Microsoft. This is damned amazing, it really feels like only a few months. (Doesn't everybody say that?) It's been fun thus far, and I think it's only going to get better.
I can't come up with anything intelligent to say right now, so I'm just going go byte-byte (sic).
Byte-byte.
Tuesday, June 14, 2005
I'm on a CLR Road Trip right now, which basically means a bunch of us on the team are out visiting customers. We're trying to get a clearer understanding of how folks use (or would like to use) our technology, and also to get feedback on our future direction. Despite the 19 hour day yesterday (no joke, 6:30am-1:30am--that doesn't even include any flying or anything, we flew in the night before!), I'm having a blast. We've done a couple of these in the past.
I gave a "CLR Internals" talk to the Vancouver .NET user group last night. Thanks to those who showed up! I found othis MSDN article which drills nicely into some of my main topics: http://msdn.microsoft.com/msdnmag/issues/05/05/JITCompiler/. I'll try to post a deck in the next few weeks.
We're travelling to Calgary tonight and will be talking at the Calgary .NET user group in the next couple days.
Sunday, June 05, 2005
I used to play guitar quite a bit. I also used to create a lot of industrial music using my computer with a whole host of techniques that ranged from sampling and messing with random non-sound binary files, writing programs to generate sounds, sample munging, and plain old recording. I was also in a metal band somewhere late in high school. We played around at local clubs (Worcester/Boston, MA), and released a tiny album that went nowhere. I did the lead guitar, some of the remastering, and a lot of the sampling that made it onto the record.
We broke up, and I dropped the guitar in favor of a keyboard. (I had dropped the keyboard in favor of the guitar mid-high school, so I was technically "returning to my roots.")
A couple weeks back, I picked up the guitar again. I gave most of my recording and guitar equipment to my brother (I saved a guitar and small amp for myself). So I went to Guitar Center and picked up a Cry Baby and Metal Box. I also still have the Acid and Sound Forge software (a couple versions behind now), so I'm a one man band again.
It's refreshing to strum away in an attempt to relearn the scales and random tabs I used to know by heart. And I'm hoping to create some more industrial tunes. The recent NiN release, I think, reminded me of how fun this can be.
Saturday, June 04, 2005
I remember the distinct feeling I got the first time I entered 'csc.exe' at the command line and realized that the C# compiler wasn’t doing any exception checking for me. Surely I had done something incorrectly. Or so I thought. After a bit of time searching around, asking around, and banging my head against the wall (still got the fracture in my skull), I came to the realization that C# had chosen not to support checked exceptions. Hmm.
Surprised? Sure. Confused? Yep. Sad? Quite.
Once I began to understand the implications of this design choice, I just became more and more confused. How the hell do I know in what manner this method could fail?! Trial and error? Manually fuzzing an API and interpreting the errors it throws? Wow, that seems like a great process, eh? Guessing? Not caring? And nevermind the problem of it changing the exceptions it decides to throw in the next version once I managed to figure it all out. (Remember: exceptions aren’t a static part of a method’s signature like they are in Java, so the implementation is free to silently alter its exception throwing policy without notice.) The horridness of this situation seemed to spiral into a rotting pit of smelly bannana peels.
The next phase of my surprise, disgust, confusion, <insert word here> was to scour the tools and documentation. How could a hole this huge be left gaping open? Surely either the Object Browser in VS or the SDK documentation would fully expose this data, or maybe a magical Intellisense switch that revealed the Truth. Well, the answer was a resounding no. The SDK did an OK job (they’re getting better over time, but not nearly as good as JavaDoc can do with the information stored in metadata), but they weren’t complete, had to be grokked out of band, and were free to change silently (all still significant problems IMHO). And it didn’t do much good for my own code and its exceptions!
I learned the ropes programming in x86 ASM and C, when I was ~13 and into hacking MUDs and other games on Amiga, Linux, and then DOS. Then I moved to C++. And then I moved to Java, and I stayed there for 5-ish years. Along the way I experimented with LISP and Smalltalk, but never to a large degree. My first professional programming experience was with C++ and COM, but by and large I’ve written more lines of real project/product code in Java.
So my move to C# was one made with a lot of expectations around how things worked, and it took me some time to get through a whole slew of these little gotchas—small discrepancies between the JVM/Java and CLI/C#. But you know what? There’s only one that stands out in my mind today, and continues to bother the hell out of me: Checked exceptions.
Arguments against checked exceptions.
If you haven’t already, you should read this interview [http://www.artima.com/intv/handcuffs.html] with Anders Hejlsberg, the topic of which is C#’s decision not to support checked exceptions. I keep referencing C# as being the decision maker; while it’s true that the CLI could have natively support them and thus it's in part their decision too, I have little doubt they’d be there if C# 1.0 wanted them. Further, Java's implementation is compiler-specific, and has no JVM support other than the metadta, so it's not clear the CLI would have even had to provide support.
Anders is a smart guy, one of very few Distinguished Engineers at Microsoft, and has his head more than screwed on right and tight. The article sounds reasonable, although a number of times I'm left thinking the data is incomplete. Maybe my head's not quite right. Or maybe Java has corrupted my mind.
I honestly see this debate as another incarnation of the static vs. dynamic typing debate that often plagues the language space. There's not a right answer in principle, but there's certainly an answer to what's right for the majority of users of your language. Anders' certainly understands the target audience of C# more than I do, so his call was probably right. But I'm left feeling neglected, poor little Java programmer man.
Many people jump on the anti-checked exceptions bandwagon without ever having done significant programming in Java. I’m sure Anders isn’t in this camp. But a lot of folks are. And until you’ve done significant programming and maintenance on a complex system with checked exceptions, you are probably not going to appreciate the safety and self-documentation of intent that it provides. You absolutely cannot understand the benefits of checked exceptions by just writing a 10-20 line program.
Some common claims made against checked exceptions come down to:
• Most developers subvert them.
• They make it difficult to version code.
• You usually don’t want to catch an exception, you want to let it leak.
I disagree wholeheartedly with each of these statements. And here’s why.
Most developers subvert them.
Based on a decent amount of Java experience, this is incorrect by a long shot. In those 10-20 line programs I mentioned, yes a lot of folks write "throws Exception" at the end of their methods, swallow exceptions, or generally subvert the checked exceptions system. But do we really want to tune the C# language for such small programs? I would argue Python, Perl, or a lighter weight language that doesn’t even do static type checking (since that’s another similar "annoyance" which prevents programs from compiling) for this category of programs. When checked exceptions are understood or provide value, subversion is unnecessary. I admit some users don't understand them (and hence use them incorrectly in the way somebody might use any language feature incorrectly), and that they don't provide value in the small, simple, script-ish program cases.
They make it difficult to version code.
Hell yes they do! But it's "difficult" in a good way, the same way that it's "difficult" to change the semantics or signature of a piece of code which is relied on throughout a complex system. If you could change it without compiler help, well, you'd be working in a dynamic language (and we ain't going there right now buddy).
There are two basic cases to consider: versioning public APIs consumed by somebody else and internal methods.
The API case is just like any other static feature of an API. Can you change the signature of your method after you ship it? Yes you can, if you want to break people. The same is true of checked exceptions. If you don’t have a statically checked list of exceptions you can throw, you’re going to break somebody anyhow.
If my API does this today:
object Foo()
{
// …
if (theNetworkIsUnavailable)
throw new FooException();
// …
}
And tomorrow it does:
object Foo(int x)
{
// …
if (theNetworkIsUnavailable)
throw new NetworkConnectionUnavailable();
// …
}
Your code that used to say:
try
{
object o = Foo();
}
catch (FooException e)
{
// Do something with it
}
Will no longer catch the error condition, since NetworkConnectionUnavailable will just head right past the FooException catch block. Hopefully your program has a backstop to catch this and respond accordingly, but depending on your error handling logic, this is likely to result in bugs either way. Is this the type of thing you want silently slipping past a compiler? Probably not. If you’re writing an API, error conditions are like any other pre-/post-condition, and should be treated as such. If the type system can enforce this, all the better for program correctness I say. (Static vs. dynamic languages again, ugh...)
The implementation case is simply not a problem. An implementation, by its definition, is a method which other programs or units don't get to see or use—i.e. non-public—and thus any problems are caught at compile time. You don't have to recompile dependencies, for example, which you may or may not have access to. This is just like changing the type or parameter list for a method… You make the change, see where things fall out, and fix them. It's the ordinary static language "beat the compiler" game. This is ordinarily a good thing, as it forces you to take a look at the exception handling code at the call sites to ensure they remain correct with the new exception behavior.
You usually don’t want to catch an exception, you want to let it leak.
I would argue about the use of "usually" here, a more correct word being "sometimes." And because this is a less-than-average situation, I don’t think the language should be tuned for it. It needs to support it, sure, but I am arguing it’s not the common case.
Note that Java supports unchecked exceptions in the type system. Basically, anything that derives from RuntimeException needn’t be checked. These are common errors like OutOfMemory, StackOverflow, NullRef, and so on, which a program almost never catches. Note that it does derive from Exception, so they don’t escape past general error handling code. These are by and large the errors that developers merely want to leak, so the checked exception subsystem doesn’t get in your way here at all!
I also believe the style of coding which has a main entrypoint wrapped in an exception handler is not the best way to do exceptions management. Perhaps I’m whacked, but in most Java systems I’ve worked on, letting an exception rip up the callstack to the toplevel is not the preferred approach, and wouldn't make it past a single code review. Usually there’s a common function used to publish an error (e.g. pop UI, write to the log, and so on) once it gets caught, but unless we need to tear down the application, the exception never shoots through the entire program to the top, leaving holes in the brains of your callers. Sure we have a handler at the top in case something escapes, but it’s not relied on as a crutch when we might be able to recover.
But it ain’t perfect.
I agree that Java’s implementation of checked exceptions isn’t perfect. But I feel much more comfortable in it than I do in C#’s fully unchecked system. Both suffer from a common problem of code duplication or over-catching in the handlers. My experience also shows that Java’s exceptions class hierarchy is better in the JSE (I know for sure at least two other smart people agree with this), and its users tend to get their own exception hierarchies right. Without a good factoring, a scaling problem ensues as you need to deal with a larger quantity of exceptions. Your catch block has to either have 10 incarnations of similar code for each exception type that can get thrown, you over-catch, or you give up and subvert the system. But provided that the factoring is clean, you can easily skirt this issue.
Still, having a "catch (Exception ex) where ex : FooException, BarException, FooBarException { }" syntax which caught any instance of those three and stored it into an Exception-typed variable would help to eliminate some of the nastiness of exception handling code. An implementation of this using exception filters in CLI would be trivial.
Having to deal with checked exceptions when you don’t care would be a nice thing to be able to express, for example, in the case of smaller programs. A compiler switch would suffice here so long as the runtime doesn’t actually enforce the "handle everything but what you’ve said you’re going to throw" policy. The JVM itself doesn’t do this check, that it’s a compiler-only enforced policy, so it's reasonable to expect that a CLI implementation would be a compiler deicision. This JVM behavior exists so that versioning can occur without recompilation.
There are other possible improvements, and certainly a whole category of other statically-detectable things (pre-/post-conditions, invariants), but I’ll stop here. Since this is mostly a language decision, it's interesting to see some approaches to solving the problem. For example, check out Spec# [http://research.microsoft.com/specsharp/] a nifty rifty roo MSR project.
Checked exceptions as implemented in Java ain’t perfect, but sufficiently close that I miss it.
|
|
Recent Entries:
Search:
Browse by Date:
| | Sun | Mon | Tue | Wed | Thu | Fri | Sat | | 26 | 27 | 28 | 29 | 30 | 1 | 2 | | 3 | 4 | 5 | 6 | 7 | 8 | 9 | | 10 | 11 | 12 | 13 | 14 | 15 | 16 | | 17 | 18 | 19 | 20 | 21 | 22 | 23 | | 24 | 25 | 26 | 27 | 28 | 29 | 30 | | 31 | 1 | 2 | 3 | 4 | 5 | 6 |
Browse by Category:
Notables:
|