Friday, April 13, 2012

anonymous types & C#

My blog updates have been sparse lately. And by sparse, I mean non-existent. I mean rarer than a Google+ post.
There have been a lot of reasons for this – it’s been a rough winter and a busy spring so far.

But I’ve stacked up a lot of geek things to blog. I’ve got a little C# stuff, some SQL tricks, and a bit of general industry and gaming thoughts (for example, how Apple is now becoming more evil than Microsoft). For now, let me start with the C# stuff.

Anonymous types. My guess is you either
a) have no clue what they are,
b) think they’re the most evil thing since 8am meetings and are probably responsible for global warming or
c) love them so much you’re thinking about rewriting every line of code you’ve ever written (even things for companies you don’t work for anymore) to change all variables to anonymous types.

I’ve been a little surprised how little understanding people have of them. I blame Microsoft (as I often do). Why, oh why, did they select the “var” keyword when declaring anonymous types? I’ve had to explain to even senior technical people that they are *not* “variant” data types – like they kind you’d find in Javascript, for example. If you write
var x=””;
you *cannot* ever do
x=new Datetime();

the compiler will stop you.

Anonymous data types are bound at *compile* time, not run time. As I had to explain to someone just this week, there is no binary difference in the assembly between saying:
string x=””;
and
var x=””;
If you look at the MSIL, they are exactly the same.

So, as someone asked me this week: why use them?
Primarily, of course, they’re used for Linq. Because, at design time, the data set you are getting back from a Linq query isn’t known, you can’t very well use named data types. This is where it gets cool though.

If you think about what Linq does, it returns a set of data. That set is unknown at design time (as mentioned), but known at compile time through introspection. So, .Net uses reflection to type the data coming out of the Linq query and then creates class to contain it. The compiler does the work of creating the class definition and setting up a property (get()/set()) for each data element in the set.

The resulting class is “hidden” in that it’s not generated until compile time and in that it’s not named. Of course, names don’t really mean anything to the compiler anyway, so it doesn’t care, but that makes it complicated to actually refer to the specific class in code.
Enter the badly named “var”.

Here’s the cool thing. Since Linq is requiring the compiler to generate a class (really, it could *almost* use a struct, but it doesn’t for another reason which I’ll mention later), this means that the compiler now has the ability to generate class code. This means you can use that outside of Linq queries.

Why would I care? (I mean besides my own mental issues?)

Suppose you have a simple method which, say, reads the lines from a file and stores some info about them. For the sake of simplicity, lets say you want to know the original line number and the actual text from the input file—and you’ll be re-ordering the lines later, so you can’t just use a simple count, you have to persist the original.

You *could* create a class like this:
Class LinesAndCounts
{
int _lineNum =0;
int LineNumber
{
get
{
return _lineNum;
}
set
{
_lineNum=value;
}
}




And so forth.
It’s a lot of typing. And the class gets scoped to the assembly or the project or the namespace, not just the method. What if you’re just wanting to read a file, make some modifications to it, and re-write it, with the original line numbers. That shouldn’t require a class that gets scoped to the whole world, right?

What anonymous types let you do is this:
var x = new {LineNumber=0, LineText=”abc”};

The compiler will automagically generate code more or less the same as what I started to write above (before I got lazy). Frankly, I think that is so beautiful, it brings tears to my eyes (*sniff*,*sniff*). In one line of code, I’ve created a name-less class and assigned values that I can use later. Moreover, that class is scoped to the method/scope-block that created it. Once execution leaves the method, not only does “x” go out of scope, but the class itself gets de-scoped, and the garbage collector takes it back to the city dump.

So let me take a contrived, concrete example. Suppose I want to read all the lines of an input file. Let’s take a simple one:
its line one
its line two

And I want to print all these lines with their corresponding line numbers reversed. So the “its line two” would have line number 1, and “its line one” would have line number 2. And then I’m going to add a header line to throw this number scheme off.
OK, this is an overly simple example, and there are lots of efficient ways to do this using the file system objects and such. But let’s pretend that the operations are more complex than they really are.

I could create a class (call it MyClass) that would have 3 properties: the original line number, the text of the line, and the new line number. Then, I could add a method to this class that would transform the line numbers by taking the original line number and subtracting it from the total number of lines like

int Transform( int origLineNumber, totalLines)
{
return totalLines-origLineNumber;
}

I could create a List to hold the objects, and (for the sake of the demo) step through this original list to move the items to a new List with the re-numbered lines. Pretty inefficient, but demo-worthy and useful sometimes.

Doing that with anonymous types, though, I don’t need to create any of the classes. The code is here:


public class AnnonExample
{
public void messWithFile()
{
int lineNumber = 0;
var firstLine = new { LineNumber = lineNumber, LineText = "this is the new file", NewLineNumber=0 };
var lineList = (new[] { firstLine }).ToList();
lineNumber++;

foreach (string line in System.IO.File.ReadAllLines("fileStuff.txt"))
{
var theLine=new {LineNumber=lineNumber, LineText=line, NewLineNumber=0};
lineList.Add(theLine);
lineNumber++;
}

for (int i=0; i < lineList.Count; i++)
{
var l = lineList.ElementAt(i);
var p2 = Transform(l, (p) => new { LineNumber = p.LineNumber, LineText=p.LineText, NewLineNumber=((lineList.Count ) - (p.LineNumber)) });
Console.WriteLine(p2.LineNumber + ":" + p2.NewLineNumber + ":" + p2.LineText);
}


}
static T Transform(T element, Func transformFunc)
{
return transformFunc(element);
}
}

Note that the classes are all anonymous and loaded into lists and manipulated later. The “Transform” is accepting a lambda expression to do the dirty work, so it could be reused generically.

Under the covers, .Net is using the same anonymous class for each of the lines. It generates it once, then re-uses it. And it enforces type-checking on it – try to assign theLine=”x” at the bottom of the first loop and you’ll get a compile error. The generic list is the same way. You can’t fill it half-way with the lines above, then fill the rest with strings. It’ll bark.

My jury is still out on anonymous types. I can see their uses. But I do think they have the tendency to reduce readability if they’re over used. Still, they can be incredibly useful for the quickie class that you need just to hang onto for one iteration of a loop so that you can modify something, grab the results, then throw it away. In the example above, I could’ve done the same thing with parallel arrays, but I think this is much less messy. It isn’t always obvious how parallel arrays are connected. Here, it’s very straightforward.

In any case, it’s very cool stuff.

--kevin

Saturday, September 24, 2011

Security Hack

Just an interesting hack that people could use.
step1:
write a web page. On the page, put a bunch of links that you care about. Make them hidden, though.

step2:
Create a style sheet and set up a link visited font and color. Doesn't really matter which one, you just have to know it.
like this:
a:visited { color : red; }
a { color : orange; }

Step3:
In the web page, write some client-side script that loops through each of the hidden links and checks the color.
(you can use this: element.style.color) and compares it to the color of the visited link in step2. If the colors are the same,
that means the link has been visited.

Now, for every person who hits your site, you know whether a lot about their browser history.

If you sell cars, you can tell if the visitor has been looking for cars. Or..whatever interests you. If you're a site that hosts some free service, you can partner with companies who provide your ads to figure out which product(s) or services they visitor may be willing to buy.

It's a kind of slimy way to profile a site visitor. And kind of a slimy way to be profiled.
Next time, I'll maybe post a way to get even more data on the visitor.

Meanwhile, you can see a nice post on it here:
http://blog.mozilla.com/security/2010/03/31/plugging-the-css-history-leak/

Mozilla has some plugins to help with this. And that's a good thing.
--kevin

Monday, September 5, 2011

HP

Reviving the blog after taking the Summer off with this thought.
It's widely known that HP will stop making most hardware. (http://news.cnet.com/8301-1001_3-20094222-92/hp-halts-webos-business-spins-off-pc-unit/)

Although, they may un-retire their notepad now that they think they can actually make a profit, but they have announced that this may be short-lived.

Meanwhile they've been buying software companies. ( For example, Autonomy : http://www.bloomberg.com/news/2011-08-18/hp-said-to-be-near-10-billion-autonomy-takeover-spinoff-of-pc-business.html). In short, they're trying to become IBM. Spin off the PC business, like IBM did to Lenovo, focus on servers and corporate software and support. Make money from consulting.

From a business perspective that makes sense. In fact, it's real winner. But I have to think about HP's origins. In the 1960's it was a small company that built testing hardware for technical applications (oscilloscopes, etc.). One of their engineers decided to invent this thing called a "programmable calculator". The marketing department shot it down -- it could never make money. They cancelled the project. But Mr. Hewlett was an engineer at heart. When he saw it, he loved it and decided that he wanted one. What Mr. Hewlett wanted, Mr. Hewlett got. So, the marketing types rolled their eyes and went along with it, sure they couldn't ever sell more than 2.

Of course, 20 years later, HP was the leading manufacturer of programmable calculators and a multimillion dollar company. Bill Hewlett was right. He was a geek at heart and a bit ahead of his time.

That HP is stepping away from hardware implies to me that the geeks have lost. The accountants are twitting their pencils and deciding that there's more money in software and consulting than hardware. And they're right. I agree. But I'm sad.

I think it speaks of a shift in the industry that saddens me. The days when IT folks could just build cool stuff and figure out how to make money later are mostly gone. (well, Google may be the last one standing, but I think they've even shifted from an "innovative" approach to a "profit" approach about 70% of the time now).

Hopefully there will still be opportunities to make a lot of money doing something that's "just cool", rather than something that balances the cost-benefit sheets.

We'll see.
--kevin

Friday, April 29, 2011

amazon

Hypothetical situation. You're an IT executive. You're tasked to add a bunch of functionality to the internal systems. A sales rep comes in and shows some of your bosses how you can save a zillion $$ by moving the data to the cloud. You look over the numbers and find out that they’re basically right.

Now what do you do? You generally don’t get into the executive position if you’re not a bit self-focused. Here’s the problem. If you push your data to cloud, it’s reasonably safe, relatively stable and pretty cheap. But the people who maintain it don’t report to you. That means your org chart shrinks and you lose control over your budget and infrastructure. It means you can’t pull resources away from the most important work in order to focus on your pet project. In short, it means you lose power and position. Think of it. If all the functionality in the company moved into the cloud, you’d be out of a job. Or even if not, you can kiss your bonuses good bye. Who’s going to give you a raise from something your vendor did? Of course, if they fail, you’ll hear about it loudly.

So, this week Amazon’s cloud crashed. In truth, it just proves they’re human. I like to collect those stories so that when the bosses demand perfection, I can tell them that if they give me as much money as the folks at Amazon gets, I may have a shot at making it well, but even then, it won’t be perfect.

From a cloud perspective, the Amazon crash hurts. The mythical VP above doesn’t want to move anything to the cloud anyway. Now he has an excuse.

Will be interesting to see what the fallout is…
--kevin

Wednesday, April 13, 2011

Consoles and streams

Just a small .Net thing that's kinda cool.
Lots of people write messages to the console of console apps by using Console.Write()/WriteLine()
If you're really working in a console app, that's great. Of course, the problem is that a lot of these console apps get converted to services or windows apps or web pages or whatever. Console.Write() in an aspx page will actually work and not throw an error, but it adds no value.
Even if you're really working in a true console app, the messages you write often scroll by and get lost.

So lots of people write messages to log files. Typically, someone will creat a "Log" class with a "WriteLog" method or something. I've done this, and it's handy. At one point what was cool is that my Logger class was just a wrapper to Log4Net, so I was able to trap the exceptions, put them into the database and into a local text file at the same time (in case the database was down). I even had a method to accept Exceptions and write them out. Cool stuff.

But what if you're working with old code filled with Console.Write() and you want to avoid a global replace? Or what if you've got developers working with you and just can't break them of the habit of Console.Write()?

I ran across this solution and thought it was worth a post.
The Console.Write uses a Console.Out object under it to actually do the write. Console.Out is just a TextWriter. And it can be overridden. (Don't you *love* OO?) So you can say:
FileStream fs = new FileStream("Test.txt", FileMode.Create);

StreamWriter sw = new StreamWriter(fs);

Console.SetOut(sw);

From that point on, anything that runs within the scope of this will write Console information to the "Test.txt" file. For example, if you do it in the Main of your program, anything in the application will write it's console messages to this file.

I love that. I think it's handy and cool. How do you set it back?

Before you change it, you need to store the old value of Console.Out somewhere, then just re-SetOut() it back to what it was, and all new Console messages go to the Console. So you can have the Main() write a couple things to the Console (starting messsages, etc), then have everything else write to the text file. Then, re-set back to the console to put out the ending messages.

The really cool thing is that the writer doesn't need to be a system object. As long as it implements the same interface as TextWriter, the Console class doesn't care. So I could , for example, take my Logger class and make it implement the TextWriter API, then pass that into the SetOut() and all the console messages would get logged through Log4Net wherever I wanted them. Note that TextWriter is an abstract class, so it's all set up to subclass and override.

Very cool stuff... enjoy,
--kevin

Friday, March 25, 2011

IPV4

I mentioned this a while ago that I saw a black market opportunity for IPV4 addresses. Microsoft is one of the first to jump into the pool, it seems:
http://www.pcmag.com/article2/0,2817,2382616,00.asp

MS picks up 666,000 (ominous number, no?) IP addresses from Nortel. I’d expect this to continue. Look for big companies to buy other companies just to get their IP addresses.
In the long term, they’ll likely switch to IPV6, but it’s much cheaper to buy a range of IPV4’s in the short term than it is to upgrade every piece of computer hardware in the company.

Just thought it was interesting enough to post.
--kevin

Tuesday, February 8, 2011

Excel Perspective revisited

Last week, I was asked to do something really simple. Most of my team was out of town and I was on “hold down the fort” duty. So my boss’ boss comes over with an emergency that needs immediate attention. Seems there’s a report that produces a spreadsheet that lists users who have done something bad, but doesn’t add any contact information. It needs two more columns so that the users’ bosses can actually call and yell at them.

So far, so good. Two more columns to a spreadsheet sounds easy enough. It takes me a while to actually find out how the report is generated, but I do.

Problem1: the report is basically a DTS package – not SSIS or anything else supported in the last 30 years – but an old version of Microsoft DTS that hasn’t been supported since the Lincoln administration. I’ve no clue how to work with the thing or how to even open it.
I button hole one poor, unfortunate comrade who got stuck in Ohio with me and he is almost as clueless as I, but suggests that maybe we can convert the package to SSIS so that we can manipulated it. I spend the rest of the day trying.
After a few hours smashing my head on my desk as hard as I can, I realize that the issue I’m having is that

Problem 2: the DTS package itself is corrupt, so it won’t port. I pull the DTS package from production and not only find out that the development version is corrupt, but that the development version is wrong. The flows are almost entirely different, and it’s pretty obvious that the dev version is way out of synch.
Armed with the “correct” version, I end up deciding not to upgrade to SSIS, since… hey.. I can actually open this package.

Problem 3: The package isn’t launched from Sql Server. It’s launched from this in-house scheduling tool that someone wrote in the late 90’s (in Power Builder, I think, if that tells you anything). And it’s all kicked off by a batch file… yes… a .bat… a 300-line .bat file.
I figure out the data relationships and modify the queries in the DTS and push to our dev server only to find out that the thing won’t run. Not only that, but I’m not getting any error trapping from the actual DTS – that is, the bat file is swallowing them.
I spend another couple hours trying to get the batch file to correctly report the errors when I noticed

Problem4: The batch file on dev is out of synch with the one in production. I pull down the prod one, add some error handling and (*ding*) now it runs and I have a valid error message logged.

Problem 5: I spend another hour tracking down the error message only to realize that the DTS package modification I did, didn’t “take” – meaning there were 4 queries and 3 of them saved when I pressed “save”, but the other didn’t. There was, after all, a reason that Microsoft decommissioned DTS.

Problem 6: once I got all this resolved, I was so excited that everything actually worked that I … well.. didn’t noticed that I’d actually messed up the query. It’s partly because I’d misunderstood the data relationships. It’s partly because I didn’t account for one of the error conditions. But it’s mostly because I was so distracted by the stupid plumbing that I’d missed the fact that water was coming through the roof.

I guess part of my point is to rant. But mostly it’s to point out how simple things can drive you nuts. From my boss’ boss’ perspective, it’s just two more columns to a spreadsheet –what can be so hard about that? In fact, I thought the same thing when I looked at it.

Gets back to the “Excel Perspective” that I’ve blogged about before. If you can do it in excel, why does it take 4 months to do it on a server?
This is why.
--kevin