I DIDN'T DO IT!

Turns out that right after I get the email telling me my project has been accepted and approved by SourceForge and posted to their "OK" list, their site got hacked.
http://www.theinquirer.net/inquirer/news/1940350/sourceforge-takes-developer-services-security-breach

http://www.techeye.net/security/hackers-hit-sourceforge

Just for the record -- I didn't do it!
:)

Wait, let me practice.
"To the best of my recollection, Senator, I did not have any particpation in nor knowledge of the events described and on the advice of my...."


--kevin

Keep On Top and Raaaave reviews

Well, ok.. not so rave.
But I got an email from Source Forge saying that Keep On Top has been added to their "official" list of Windows products.
Said they:
"We are impressed with the quality of your product and encourage you to keep these high standards in the future"

..like they really looked at it :)

They did test it for spyware, malware, and otherware and proclaimed it "clean", which is good.

Just thought it was kinda fun and worth a blog entry.

cheers,
--kevin

Keep On Top

It’s been over 2 months since my last post, but I’m still kickin’.
The last 2 months has seen a lot activity – the holidays, a new job, some winter travel, a lot of football and a bit of geek stuff.

So for my first post of 2011, I decided to share some open source code I wrote. Here’s the problem space:

You’re doing something that requires attention, but not interaction. For me, this may involve watching a process run or tailing a log file. Or maybe it’s installing some new software. Whatever.
But I hate just sitting there watching the stuff. Who wants to sit and watch log files tail for 2 hours straight? All I really want to know is when it’s done or I want to see if it stops or gets any obvious errors. So I open up my email and fire off a couple. I pop open a browser. I get ping-ed on a chat client by a friend. And, maybe I play a computer game or something.
While I do this, the window I’m watching gets lost under the rest of them. So I … do what? I can randomly switch to the “watch window” and hope to catch it at the right time to see the error, I suppose. But what I really want is for that window to stay in sight while I’m doing something else.

So I wrote KeepOnTop. KeepOnTop starts up with a list of all your running applications. You pick one and it locks that window at the top of all the others. It does not give the window focus. And, in fact, the window stays on top even when you’re typing somewhere else.

The assumption is that you resize the window to be reasonably small and move it off to the side, Then you can type away at an email or whatever, while watching the log file tail. The only impact to the email app is that the part directly under the top window won’t be visible. Of course, you can move the top window anywhere you like anytime you want, so you can get by that.

KeepOnTop uses some native Windows APIs by importing user32.dll into C# as an “extern” and then exposing 3 or 4 static functions in it.

It’s not quite finished yet. I’m thinking about doing a bit of Windows explorer integration so you can right-click an executable and say “keepOnTop”, which would cause the exe to be run and automagically pushed to the top. But we’ll see.

Meanwhile, you can find it on SourceForge at
http://sourceforge.net/projects/keepontop/

The source is in the subversion area. The exe is in a zip file under the “files” area.

threadPools again

Here's a supremely annoying ThreadPool undocumented/under-documented feature that cost me hours.

If you have a resource you're using -- say a database connection -- and you want to limit the resource -- say because you have hard limit on the number of simultaneous database connections you can open -- then you can limit the number of simultaneous threads that a ThreadPool can open by using the SetMaxThread method:

ThreadPool.SetMaxThread(X,Y);

All good. Only:

-- under-documented feature: If you set the max threads less than the number of CPU cores on your system, the setting is ignored. That is, if you're running on a quad-core machine, and you set the max threads to be 2, the ThreadPool assumes you don't know what you're talking about, assumes it can think for you, and disregards your setting.

-- un-documented feature: if you set the max threads less than the number of cores, the max threads setting *doesn't* default to the number of cores, it's ignored completely. Basically, the ThreadPool doesn't default to the lowest possible value, but ramps up the max threads to whatever it feels like it wants to do.

This makes the ThreadPool mostly useless.

Look, if I'm writing an app that uses no resources except the CPU, that's great. But who writes apps that don't use any resources.

Moreover, most software developers don't have any control over the hardware on which their app runs. If I'm a typical line of business developer, I'm writing code that is set up to run on a server somewhere against a database. Likely, I have no control over the server. And it can be upgraded at any time.

In the days of virtualization, an administrator can update the number of CPU cores to a virtual machine on the fly. So if my app runs on a server, and an unrelated app runs on same server, and unrelated app needs more horsepower, admins can add CPU cores without even telling me.

If they do that, then all of a sudden my app -- which was throttled to a small number of threads so that it wouldn't smash the database connections -- now has the MaxThreads set too low. In that case, ThreadPool completely ignores the setting and spins off a zillion threads if it feels like the CPU can take it... completely crushing the database.

It's so frustrating when Microsoft does this. They make a great product, with a great feature, then make it totally unusable in real-world, line-of-business solutions.

GRRRRRRRRRRRR

--kevin

threads and C#

I'm old school, i guess. Probably because I'm old. But, whenever I need to do some mult-threaded programming, I open up the System.Threading namespace and start to create threads. So that I don't overwhelm the system, I usually throttle these threads and create a thread collection somewhere .

It's old school.

I only recently found out that there is a slick way to do this in .Net.

First, you'll need a way to know when the thread has finished. Since there will be n of them, you'll need some type of collection.

Specifically, this will need to be a collection of ManualResetEvent objects.

For example,

System.Collections.Generic.List al = new List();

each time a new thread is needed, you'll need to create a new ManualResetEvent object, like this:

ManualResetEvent m = new ManualResetEvent(false);

(the boolean indicates whether to set the initial state).

Then you'll need to add this to the list:

al.Add(m);

ok, now that some of the thread junk is out of the way, we'll want to actually *do* something.

For example, lets stay we create a class called threadClass.

threadplay.threadClass c = new threadplay.threadClass(m);

in this example, i'm passing the ManualResetEvent to the constructor. The object will need a reference to it somewhere.

Next, you'll need a callback method. This will be what the thread calls to do it's work. It's the "DoStuff" method.

In my class, I called it "theCallback". It will need to have this signature:

public void theCallback(Object x)

Inside the callback method, you can typecast the Object to whatever makes sense. So you can pass in data connections, collections, or just about anything you like.

In my case, I created a class called "playObj" with some boring properties which i passed into the constructor:

new threadplay.playObj("kjh", i)

To set up this to run from a thread, just pop it onto the ThreadPool:

ThreadPool.QueueUserWorkItem(c.theCallback, new threadplay.playObj("kjh", i));

This will enqueue the thread to be run at the next possible moment. By default .Net will manage this thread pool for you. You can see or change the max or min number of threads if you want to, but .Net is suppose to correctly scale this based on system resources. If you're contenting with a resource .Net doesnt' know about (for example a hard limit on number of open database connections), you may want to modify the default behavior. But it is suppose to account for CPU utilization and be optimized already.

Inside the callback method, you'll want to call the Set() on the ManualResetEvent. Do it at the bottom of the method, when the execution of the method is done.

Generally, the call to QueueUserWorkItem will happen in a loop (otherwise, what's the point?).

The purpose of the ManualResetEvent is to allow you to wait for each thread to finish. Since you're not controlling the thread pool, you can't get to the actual threads easily, so you can't do a join(). Rather, you can do use WaitHandle.WaitAll() to wait for all the threads to finish (or WaitAny(), if you only want the first one).

Note that WaitAll() expects and Array, not a List<>. But you can convert.

WaitHandle.WaitAll(al.ToArray());

(for some types of data collections, you may need to typecast this to (ManualResetEvent[]). But since i was using a collection of type List, .Net does this for me).

That's it. The hardest part is the ManualReset event stuff.

This will create a pool of n-threads, enqueue any added after the nth until they can run, and wait for all to finish before continuing. Wiring it together is a bit of a pain because of the ManualResetEvent complexity, but besides that, it's very clean and removes the overhead of dealing with management of system resources.

--kevin

Linux security

Thought this was interesting. To sum it up, there's a huge flaw in the 64-bit Linux Kernel, specifically related to running 32-bit apps. With the right hack, you can create a buffer overflow, which will allow you to do just about anything you want.

As I've noted before, I think the whole idea of a root account is one of Linux's most concerning features. The Debian-based distros have tried to hide that by not allowing root to actually log in (ie, sudo), but all that does is add a level of abstraction on top of the issue.

Making matters more concerning is the groundless opinion of the Linux faithful that Linux is so solid, no one ever needs a virus protection tool. So a flaw like this can go uncaught. In addition, with this bug, a hacker can create a root session and persist that to disk via a binary. The impact of this is that even after the patch is applied, the root session may still be available, if you've already been hacked. It's just an executable file on the disk that contains a shell with root privs. It could be called anything, making it just about impossible to find.

Anyway, thought it was interesting.

http://sota.gen.nz/compat1/

CLR

Had the opportunity to work with a few CLR stored procedures in SQL server.

Cool stuff.

Here's a quick primer on how to do it, what to do, what not to.

First, something simple, but useful. We've got a need to pass in a string, do "something" to it, then return the modified string.

First, I'll pull together the CLR/.Net code.

Let me put it here, then I'll explain a bit.

using System;

using System.Data;

using Microsoft.SqlServer.Server;

using System.Data.SqlTypes;

public class HelloWorldProc

{

[Microsoft.SqlServer.Server.SqlProcedure]

public static void HelloWorld(SqlString x, out SqlString sOut)

{

sOut = x

}

}

The System.Data.SqlTypes is important, as is the SqlServer.Server using statement.

I think ( ? ) the class needs to be in the default namespace. I couldn't get it to work any other way.

The method needs to be static, which does make sense, I think.

Also, note the weird output thing I'm doing. I have to set up the output as a void, then deal with the output as an "output" parameter.

The reason is the limitation placed on Sql CLRs. They can only return integer types (for example, System.Int32, SqlInt32, etc) or voids. Frankly, I've no clue why. Seems silly to me that they can handle a reference parameter, but not an output parameter.

Note that the parameter types are SqlStrings, not strings. But the assignment doesn't even do an explicit typecast.

From the SqlServer side, the assembly has to be created in SqlServer.

CREATE ASSEMBLY HelloWorld from 'c:\temp\HelloWorld.dll' WITH PERMISSION_SET = SAFE;

Next I created a stored procedure to wrap the assembly.

CREATE PROCEDURE SerializeXmlNodes

@x NVARCHAR(1000),

@sOutNVARCHAR(1000) OUTPUT

AS

EXTERNAL NAME HelloWorld.HelloWorldProc.HelloWorld;

The input and output of the stored procedure match the input and output parameters of the .Net method.

Calling the stored proc is done about the same way as calling a "normal" stored proc:

EXEC SerializeXmlNodes @x = 'one', @sOut = @sOut OUTPUT

Select @sOut

in this case, @sOut will be whatever you set @x to, but that's because it's a trivial example.

To get rid of the CLR, you need to drop the stored procedure first, then drop the assembly ("DROP ASSEMBLY HelloWorld").

Very cool stuff, but a little confusing to set it up. Still, it allows string manipulations not available through TSql without serious pain.

--kevin