Here's a story. Many years ago a very smart young man went to work in IT. Naturally, he developed software in the language of the future (PL/1, with a bit of assembler thrown in). He worked with the computing gods, and aspired to become one.
But along the way, something happened. Maybe he just didn't have divine attributes, maybe he got bored or outsourced or decided he wanted a life. So he changed his aspirations. Rather than being one of the gods of computing Olympus, he would manage them.
He was good at management. He was hard-working and politically savvy. Over time, he worked his way up from Junior Assistant Lackey Manager to Senior Assistant Lackey. Pretty soon, he found himself in a Vice President's office.
But he had one problem. He had no clue what his employee's did. He didn't remember much about computers, but he knew terms like "libraries" and "frames" and maybe even remembered a bit of JCL. They were talking about EJB's and Ajax (wasn't that a cleaning product?) and SSIS and PCI and data warehouses.
Conceptually, he kind of got it, being a smart fellow. But the details were all a mixture of magic words and flowcharts. And he couldn't understand why IT had gotten so expensive or why it took so long to get things done. Why not just toss together some JCL and run the silly thing? Why did everything take 4 months?
Enter the vendors. They walk into his office with really spiffy demos. It's not a flow chart, it's something he can see. And it's pretty. The reports are all graphical and the demo shows how easy they are to use. And time to market? Oh, the vendor insists that they have a crack consulting team that can deliver on the fly. Why, as part of the demo, the vendor even builds a really cool report in minutes.
Our hero is hooked. He signs on and spends a zillion dollars.
Then he gets the product to his IT staff and they tell him that they can do the same work already with other tools. Moreover, they insist that the complexity isn't producing the pretty report, but aggregating the data from the 16 different heterogeneous data stores they have, manipulating it to make it actually match, then (this is the really hard part), making it mean something. It's easy to sum a table column. It's a lot harder to know what the data in the column means or what the rules are around summing it.
The problem is that there is a gap between the people who work in IT and the folks who manage it. The vendors and consultants make a ton of money trying to fill that gap. Each year, there's a bevy of products aimed at it. They each hold the promise of being able to close the gap -- and turn the IT systems from a collection of disparate pieces into a contiguous whole. Only, in reality, the vendors don't really care much if they work. Oh, I'm not saying they are sloppy or bad people - although some are, I'm sure. But I'm just saying that in the end, the vendors have very little incentive to really close the gap, since that's basically how they make their living. Without the gap, consulting services have to shift their work to being little more than resource augmentation providers. And they have no real competitive edge. Today, if a consultant can get certified in the latest hot product, they can bill themselves as experts. If the gap closes, that will hold much less value. The vendors are in the same position. If the gap closes, then the only way for them to make a living would be to be truly creative with their products or to build tools that actually provide value for the developers. Don't get me wrong, some companies do that. Microsoft has a bunch of them. Oracle does too, although they have a lot of "gap-closers" too. Sun has a few, although they've been hamstrung by financial issues. But other companies would have to pony up and compete with the IBMs of the world.
And it's so much easier to simply pull together a good demo.
