the excel syndrome

What is the excel syndrome?

It's an executive manager who uses Excel and is convinced he "knows" computers. You see it a lot now. And it's one of the big things that's changed in the last few years. Ten years ago, if you said "e-commerce" everyone's eyes glazed over and they said "just make something work".

Today, all the execs use Excel and buy things from Amazon. So when an IT professional says : "setting up this database correctly and adding accounts with the right security and tables with the right schemas will take... er... 2 weeks", the exec says "WHAT!?? that's insane! Why... I can do it in Excel in 10 minutes". Then he (or sometimes she, but that's pretty rare), slashes the project estimate.

Oh, I know I'm exaggerating for emphasis. But the syndrome exists. It really does. My CEO just said today that we weren't delivering fast enough. Fast enough compared to ... what? How does he have a feel for how long it *should* take?

I started noticing this a few years ago. Non-IT people started to think they understood IT well enough to second guess their IT staff. I see it among IT people too -- especially those who have worked in small environments and with older, more limited systems. Sure, it may be possible to export a file from DBase from a live system with a simple real-time query. But.. um... DBase isn't relational? Oh, and there's that little thing called "security"? And the DBase database that only held 2MB of data is a slightly different creature than a SqlServer database that holds 200GB.

It is part of a trend in the industry. IT has become big business. Consulting companies and vendors are big business. As the money that goes into IT grows, more and more people are trying to get their little part of it. Naturally, that includes the execs and the bean counters.

I'm not sure where IT will go in the future, but it's days of free-wheeling are done, I think. It will become more and more driven by people who have an opinion on how the money is being spent, even if they don't know what their opinion means. It's kind of like those Windows7 commercials where the people are saying the invented Windows7, because they sent an email to Microsoft telling them to make computers better.

There are a lot of those people.....

data access

there's lots of ways to do data access, but here's something I've been doing that seems kinda cool.

First, create a namespace (DataObjects, in my case) .

Next, a BaseDataObject this is the parent that all the data accessors will inherit from.

In the BaseDataObject go the common things. For example, I have a public property called "DatabaseName". Using this, the code will query for the correct connection string parameters when needed.

The BaseDataObject also owns the actual connection. This is interesting. The Connection goes into a thread static collection of connections (which is hard to say quickly :D ). When the connection is needed, the collection is searched. If the connection isn't there, it gets created and inserted for the life of the thread. Took a while to nail this, but this means that the connections are correctly cached by database name for the life of the thread and no longer.

In order for this to work, the connection is actually of type IDbConnection. Anything else will bind the data object to one database type.

All of the methods of the BaseDataObject follow that pattern -- each uses IDbCommand, IDataReader and the like.

The simplest subclasses only override the connection itself. For example, I've got a MySqlDataObject that allows connection to a MySql database. It overrides the "connectToDatabase" method to secretly append a mySql designation to the DatabaseName. This means the connection string resolution returns a MySql compatible connection string, not a SqlServer one. In addition, when it creates the database connection, it creates a "MySqlConnection", which gets implicitly cast upward to a IDBConnection on the return.

The more complex subclasses (the SqlServer one, for example) have some additional functionality. For example, I've added an ExecuteNonQueryWithParameters method to the BaseDataObject, which accepts a Dictionary of key-value parameters and then uses this to populate the Sql Statement. The BaseDataObject resolves all this by looping through the dictionary and ingloriously building the sql string. The SqlDataObject overrides this to use the Parameters.AddWithValue method on the SqlCommand, since this is available.

In addition, I've occasionally further subclassed the child data objects to allow versions of the objects that toss exceptions back to the caller and versions that don't, but log the exceptions and return a null result.

To wrap this all up, I added an object factory to the DataObjects namespace. Pass in the DatabaseName, and you get an appropriate subclass of BaseDataObject, based on whatever business rules are applied to decide this. The returning object is basically a live connection ready to be used.

So far, this has worked pretty well. My big frustration is the lookup on the database name -> connection string. I have been using a Microsoft "Settings" object for that, but, frankly, it sucks and doesn't provide any real benefit. The good news is that, since this is hidden by the objects in the namespace, I can change it anytime without issue. Moreover, the structure allows me to reference databases as objects. Not only do I like that, but the polymorphism and use of object factories allows that I can move data from one database to another with no changes to code.

Just thought it was cool enough to post.

today

just something to share that doesn't mean anything.

I spent about 3 hours today debugging a workflow. I was furious at Microsoft.

The issue was that the AutoResetEvent wasn't getting set, but timing out. So, after the flow was complete, the "oncomplete" delegate would fire, but then sit there and wait for the AutoResetEvent to timeout.

I checked the code. I checked the flow. I debugged. I searched the web. I was typing up a post on MSDN, when I thought I'd better double check my logic.

My flow was being called from a subclass of a util class that I had written. After some additional hair pulling, I realized that the subclass was hiding the AutoResetEvent with a variable named the same. What was really stupid was that in C#, you have to create the variable in the subclass with the "new" key word:

private new AutoResetEvent waitHandle = new AutoResetEvent(false);

Geeze. SO I *intentionally* was hiding the variable that I needed then wondering why it wasn't there.

My official thought: "DOH!"

So here's my takeaway. It's getting easier to build complex functionality with the new tools. They do a ton for you. Awesome.

But they hide things. Nothing in my debugging would show me which variable was getting updated or how.

I'm not blaming Microsoft. Well.. ok... I am, simply because I need someone to blame.

But it's a challenge that developers face now. Devs have always had to use code we didn't control. But the degree to which its needed is getting much higher. Whether we're using EJB or Windows something Foundation, you just have no clue what the code is really doing. If you make a simple mistake setting something up -- a hidden variable, a mistaken delegate, a copy/paste error in a config file -- can cost days in developer time to find. In the end, it could even be a bug in the toolset or a mistake in the docs.

It gives me pause for thought. I really wonder if this is the best way to approach things.

--kevin

Workflow and Exceptions

This guys says lots of cool, good, happy things about handing exceptions in Windows Workflow.

http://www.microsoft.com/uk/msdn/nuggets/nugget/60/Windows-Workflow-Foundation-Exceptions.aspx

AND he says it with a cool accent!

The basics are pretty simple, but there are some weird gotchas. Like you have to have a different Exception Handler for each kind of exception you want to handle, not just different code for each.

There are a couple additional things to keep in mind:

first, Studio (2008, anyway) doesn't seem to recognize that you have a workflow exception handler in place. So when you run the code in debug mode, it will stop at the exception and tell you that it's unhanded. If you hit F5, it'll will step into the handler. Seems like a disconnect between the development tool and the workflow foundation.

There are a bunch of things you can do with the exceptions, of course. But my real desire is usually to trap them, log them, then return them gracefully to the calling process.

The second thing to be aware of is how to do this. Lots of options, but if you create a public, read-only property to your workflows called, say "Exceptions", and defined as , say, a custom collection (or generic) to hold the exceptions, these will get auto-magically returned to the caller. However, they are cleverly hidden so that it's not obvious where they are.

Upon completion of the workflow, all the publics get serialized and passed as a parameter to the delegates registered for the workflowRuntime.WorkflowCompleted, workflowRuntime.WorkflowTerminated, workflowRuntime.WorkflowAborted events.

So somewhere in the code, there should be a something like this:

workflowRuntime.WorkflowCompleted += OnWorkflowCompleted;

OnWorkflowCompleted is the delegate defined by:

protected void OnWorkflowCompleted(object sender, WorkflowCompletedEventArgs e)

Typically, here is where you will set the wait handle:

waitHandle.Set();

where waitHandle is defined as :

protected AutoResetEvent waitHandle = new AutoResetEvent(false);

But the 2nd parameter (WorkflowCompletedEventArgs e) is really just a serialized representation of the workflow itself. So all the public properties are available as properties.

So, in my case, I just have my exception handlers add the exception to the public collection owned by the flow. This is really cool for capturing all the exceptions in one place. For example, say I'm persisting some data elements. It could be that many of the data elements are bad. If I throw an unhandled exception for the first one, I never see the rest until the data gets scrubbed and reposted. Then I bomb on the 2nd one. But with this approach, I can just add the exceptions to the output collection and de-serialize in the calling code.