1-800-MAGIC

Sunday, May 31, 2009

Israel threatens the US

"We want to reach an agreement with the United States on ways to advance the peace process," said a senior Jerusalem official. The U.S. stance, he said, "will stall the process and bring about tension and stagnation, which will hurt both Israel and the United States."

(Emphasis mine.)

http://www.haaretz.com/hasen/spages/1089205.html

Saturday, May 30, 2009

Free web development tools for Microsoft platform

This article is about building websites using ASP.NET. "ASP.NET???" - you would exclaim in disbelief - "But this is 2009! Ruby! Rails! Python and PHP! Why would anyone even look at this boring last-century technology for building HR applications???"

There are several reasons why ASP.NET is worth more than a shrug in today's web development.

First, the tools are REALLY nice. Visual Studio absolutely rocks as a development environment.

Because of the Intellisense magic, most lines of code take only a few keystrokes to enter. Here is an example:

(1) 'c':

(2) '.':

(3) '.':

(4) '.(DownArrow':

(5) ');'

...and a line of code has been entered in roughly 8 keystrokes and under 5 seconds. When you type say a thousand lines of code per day, believe it or not, this makes a huge difference.

The debugger is fantastic - you can set breakpoint on the client and on the server, in C# and in JavaScript, and it works the way you'd expect it to work. In my days in Gmail the debugging experience was Firebug and even that involved an inordinate amount of prayer to my atheistic gods. Debugging in IE required making a pact with the devil. And as you know, Python has no debugger at all, because Pythonic code does not require debugging by definition :-).

Second, there is a lot to be said for a type-safe language.

In the world where most development is confined to 500-lines-of-Javascript-and-a-lot-of-JPEGs Facebook apps, the idea that every object is a hash map, and every squence of keystrokes is a variable makes perfect sense. Why spend time thinking about class hierarchies when we could be delighting the customers instead?

But suppose your app is 100000 lines of code. Standard software development principles - code isolation, compiler-time type checking, interfaces - become incredibly important.

How many huge applications have you seen that are written in dynamic language? I've worked on one - Gmail - which was around 250Kloc of JavaScript at the time. The only way doing anything there was possible was because of Google's internal JavaScript tools that did a lot of static code-checking at compile time, and because the v2 code base was built on object-oriented concepts - class libraries, inheritance, etc. It was very unusual - and very restricted - JavaScript for sure!

But even with all of these great tools, the life was very hard - I don't think Gmail team will be able to increase their code base very much further...

One huge advantage of ASP.NET is its ability to model an HTML page in C# - every HTML element is created as a .NET object on the server, and is then rendered into HTML. This allows your compiler to be very upfront about things that you would otherwise be finding through testing or user feedback!

For example, this is a code snippet that creates a very simple web page:


    Label l = new Label();
    l.Text = "<h1>Welcome!</h1>";
    ActivePage.Controls.Add(l);
    TextBox t = new TextBox();
    t.ID  = "textbox";
    ActivePage.Controls.Add(t);
    Button b = new Button();
    b.Text = "Submit";
    b.ID = "submit";
    b.Click += new EventHandler(clickProcessor);
    ActivePage.Controls.Add(b);

I think it was primarily because of the tools and the language that I was able to clone Google Mondrian in C# in less than a quarter of time it took Guido to write it in Python. (http://malevich.codeplex.com)

Alright, alright, you would say, but don't I have to pay Micro$oft thousands of dollars for the privilege? Ruby, PHP, Python, and MySQL are free, and have you checked the going rate for SQL Server recently?

This used to be true, and it was my constant mantra at Microsoft that we should ship at least the basic dev tools for free. It was always my opinion that if we did ship free C compiler with DOS and Windows, GNU tool chain - and, consequently, Linux - would not have existed.

Maybe I was not alone in thinking this, and better late than never, so starting around 2005 Microsoft began shipping free Express version of Visual Studio Suite, as well as SQL Server Express Edition. This articles is an introduction to these tools.

Without much ado, let's dig into free development with Microsoft tools.

First, you need a free OS. Windows 7 RC is free (for a year :-)) and is available here:
http://www.microsoft.com/Windows/Windows-7/download.aspx

On a modern DSL connection it takes about an hour to download, and another 30 minutes to install (including the updates) - at least on a virtual machine I used.

After the OS is installed, go here: http://www.microsoft.com/web/downloads/platform.aspx and download Microsoft Web Platform using "Installer 2.0 Beta". Make sure you choose everything under the Database tab, and "Visual Web Developer 2008 Express with SP1" under Tools.

Before the installation, it will ask you what authentication scheme to use for SQL Server. Select Windows Authentication.

During the installation, it will complain multiple times about "Known compatibility issues" between Win7 and various SQL Server components. Click "Continue" every time - but you will need to install SQL Server SP1 (later).

Once the installation is complete, go to Windows Updates, and click "Get updates for other Microsoft products". Install all default and optional updates (that are not language packs). Pay attention to the optional updates - the list should include SQL Server SP1. Make sure this gets in.

Your setup is now complete! From start to finish, on a decent DSL connection, it should take no more than 2.5 hours do download and install everything.

You can use free version of Visual Studio for building web applications, and a free version of SQL Server as a backend for them.

As an example of using these tools, we'll build a simple guest book app.

Our app will consists of a database that will store comment records in our guest book, and a web front-end that would allow visitors to enter comments (as well as their names and email addresses).

Let's create the database!

Our database will be really simple. It will contain records which will consists of the record key (an integer), a 50-character name, a 50-character email alias, and an unlimited comment string.

Run Microsoft SQL Server Management Studio (from Programs->Microsoft SQL Server 2008). Right-click on the list of the databases, and select "New Database...". Enter the name of the database (GuestBook), and click OK.

This creates an empty database. Now expand its tree in the left pane, and click on the table. Select "New table...". A windows that allows you to edit the database structure will open. DO NOT save the table until all the editing is done!

First, let's create an identity field - the name would be "Id", the type "int". Right-click on the field and chose "Set Primary Key".

Then, in the list of options below, make this an identity record (a unique key that identifies the record in the database).

Then create the other fields: Author and Email (both nvarchar(50)), and Comment ((nvarchar(MAX)) - all three of them NOT NULL.

Save the table (Control-S) and name it CommentRecord:

Now that we have the place to store data, we should create a security model for it. SQL Server has two security primitives that are of interest here - database users, and SQL Server logins. "A login" is a SQL equivalent of a Windows user. When you allow a user to connect to the database engine, you create a login for this user. In our case, we want to allow IIS user to connect to SQL database, so we create a login for "IIS APPPOOL\DefaultAppPool" by right-clicking on Security/Logins and selecting New... from the menu:

Logins are global to the database engines, and are a way to identify external users to SQL. The databases also have a concept of users, which are projections of logins on the individual databases (why SQL developers decided to have two entities instead of one, beats me). So we now have to create a user for the GuestBook database, and associate it with the login we just created. We will name the user "GuestBookUser". Expand the GuestBook database tree, Security, and right click on Users, and select New. Enter the logon name (IIS APPPOOL\DefaultAppPool), the user name (GuestBookUser), and check db_datareader role membership below, as follows:

.

Note that we have not granted the database user any "write" rights yet, but we expect it to be able to add the records. We will do it by creating a stored procedure (a piece of code inside our database) that would add records, and grant the GuestBookUser the right to execute this code. This way, any visitor to the web site would be able to add data, but not change anything that is already in the database.

Under the database tree, expand Programmability, and right click on New Stored Procedure. The nice thing about SQL Server 2008 is that its Management Studio supports Intellisense:

Enter the following stored procedure into the query window, and click on Execute:

Now the only thing left is to grant GuestBookUser the execute access to this stored procedure. Expand Stored Procedures under Programmability, right-click on its name (refresh - F5 - if it's not there), then select Properties, and then Permissions:

We're done! Once you do it once or twice, creating simple databases like this only takes 5 to 10 minutes.

We can now create the front-end application.

Execute Visual Web Developer 2008 Express Edition AS ADMINISTRATOR. You will need this privilege level to publish the web site.

From the File menu select New Website. Pick ASP.NET Web Site template, set the language to Visual C#, and put it in a directory called GuestBook:

We will be using LINQ to access the database from our code. LINQ To SQL makes a database accessible by generating a C# object model for it. It's actually extremely nice. You can get at the database data by executing SQL-like queries directly from C#, by writing something like this:


    var query = from cc in context.CommentRecords select cc;
    foreach (CommentRecord r in query)
    {
        Console.WriteLine("Name: {0} Email: {1} Comment: {2}",
            r.Author, r.Email, r.Comment);
    }

LINQ supports much more complicated queries, of course, with conditions and even joins.

To add the database model, right-click on the website name in Solution Explorer, and click on Add new item. Pick LINQ To SQL Classes, and change the name to GuestBook.dbml:

We now need to connect to our database. Click the Database Explorer, then Add Connection:

Pick Microsoft SQL Server:

Set the SQL Server instance as YOURMACHINENAME\SQLEXPRESS, and select GuestBook from the list of databases. Click on Test Connection to make sure everything works:

You can now see the database in the database explorer. Expand the nodes for tables and stored procedures, and drag the CommentRecord table and AddComment SP over to their places on the designer surface:

Save everything (Control-S). The database classes are ready for use!

We need an element on the form to use as a root for our HTML object model. Every HTML element we create will attach to this element. Open Default.aspx file, and convert the empty div element inside the form to asp:PlaceHolder. It needs to have properties runat="server" and id="ActivePage". Save!

Open the Default.aspx.cs file now. This is the code we're going to write - the whole thing is barely over 100 lines long, and almost 20 of that is generated as part of the template:


using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;

public partial class _Default : System.Web.UI.Page 
{
    //
    // This renders the page. Page_Load gets called both when
    // the page is first displayed, and also on post-backs.
    // The postback cycle initializes the page object model by running
    // the code below, then sets the state of the objects from user
    // input, then the handlers for the controls are called (in our case,
    // it is the Click handler for the button). The handlers may (and often
    // do redirect page to cause re-rendering with the new information.
    //
    protected void Page_Load(object sender, EventArgs e)
    {
        Label l = new Label();
        l.Text = "<h1>Welcome to my Guest Book!</h1>";
        ActivePage.Controls.Add(l);

        //
        // Open the connection to the database.
        //
        GuestBookDataContext context = new GuestBookDataContext(
            System.Web.Configuration.WebConfigurationManager.ConnectionStrings[
            "GuestBookConnectionString"].ConnectionString);

        Table t = new Table();
        ActivePage.Controls.Add(t);

        //
        // First, display all existing comments
        //
        var commentsQuery = from cc in context.CommentRecords select cc;
        foreach (CommentRecord record in commentsQuery)
        {
            TableRow row = new TableRow();
            t.Rows.Add(row);
            TableCell cell = new TableCell();
            row.Cells.Add(cell);
            cell.Text = Server.HtmlEncode(record.Author) + " says...";
            cell.Style.Add(HtmlTextWriterStyle.FontStyle, "italic");
            cell.Style.Add(HtmlTextWriterStyle.Color, "blue");
            cell.ColumnSpan = 2;

            row = new TableRow();
            t.Rows.Add(row);
            cell = new TableCell();
            row.Cells.Add(cell);
            cell.Text = Server.HtmlEncode(record.Comment);
            cell.ColumnSpan = 2;
        }

        context.Dispose();

        //
        // Now create the form for a user to enter comments.
        //
        TableRow author = new TableRow();
        t.Rows.Add(author);
        TableCell name = new TableCell();
        author.Cells.Add(name);

        Label authorNameLabel = new Label();
        authorNameLabel.Text = "Your name:";
        name.Controls.Add(authorNameLabel);

        TextBox authorNameInput = new TextBox();
        authorNameInput.ID = "name";
        authorNameInput.MaxLength = 50;
        name.Controls.Add(authorNameInput);

        TableCell email = new TableCell();
        author.Cells.Add(email);

        Label authorEmailLabel = new Label();
        authorEmailLabel.Text = "Email (will not show):";
        email.Controls.Add(authorEmailLabel);

        TextBox authorEmailInput = new TextBox();
        authorEmailInput.ID = "email";
        authorEmailInput.MaxLength = 50;
        email.Controls.Add(authorEmailInput);
        
        TableRow comment = new TableRow();
        t.Rows.Add(comment);

        TableCell commentCell = new TableCell();
        commentCell.ColumnSpan = 2;
        comment.Cells.Add(commentCell);

        TextBox commentInput = new TextBox();
        commentInput.Columns = 80;
        commentInput.Rows = 20;
        commentInput.TextMode = TextBoxMode.MultiLine;
        commentInput.ID = "comment";
        commentCell.Controls.Add(commentInput);

        TableRow submit = new TableRow();
        t.Rows.Add(submit);

        TableCell submitCell = new TableCell();
        submitCell.ColumnSpan = 2;
        submit.Cells.Add(submitCell);

        Button submitButton = new Button();
        submitButton.Click += new EventHandler(submitButton_click);
        submitButton.Text = "Submit comment.";
        submitCell.Controls.Add(submitButton);

        if (Page.IsPostBack)
        {
            TableRow error = new TableRow();
            t.Rows.Add(error);

            TableCell errorCell = new TableCell();
            errorCell.ColumnSpan = 2;
            errorCell.ID = "error";
            error.Cells.Add(errorCell);
        }
    }

    //
    // This is run when the user clicks on the Submit button
    //
    void submitButton_click(object source, EventArgs e)
    {
        string comment = ((TextBox)FindControl("comment")).Text;
        string name = ((TextBox)FindControl("name")).Text;
        string email = ((TextBox)FindControl("email")).Text;
        if (String.IsNullOrEmpty(comment) || String.IsNullOrEmpty(name) ||
            String.IsNullOrEmpty(email))
        {
            TableCell error = (TableCell)FindControl("error");
            Label l = new Label();
            l.Text = "You need all three values!";
            l.Style.Add(HtmlTextWriterStyle.Color, "red");
            error.Controls.Add(l);
            return;
        }

        GuestBookDataContext context = new GuestBookDataContext(
            System.Web.Configuration.WebConfigurationManager.ConnectionStrings[
            "GuestBookConnectionString"].ConnectionString);

        context.AddComment(name, email, comment);

        context.Dispose();

        //
        // This causes the page to be redrawn with the new comment in it.
        //
        Response.Redirect(Request.FilePath);
    }
}

When the code is done, you can run it directly from Visual Studio (F5), and all normal debugging primitives (breakpoints, stepping, etc) work as you'd expect them to work.

Let's deploy the web site to the real web server now. Create a directory under c:\inetpub called GuestBook. Right-click on the web site name in the Solution Explorer, and pick Copy Website option. In the dialog that opens, click on the connect button, select File System, and navigate to c:\inetpub\GuestBook. Select all the files on the left and click on the right arrow to copy them to the target directory:

Open IIS manager, navigate to Default Website, right-click on it, and pick Add Application. In the dialog box that opens enter GuestBook as the name, and c:\inetpub\GuestBook as a path to the application.

Now go to the Firewall Control Panel Applet, and open the port for IIS:

This is it!

Google I/O 2009

An interesting, if long, video: http://www.youtube.com/watch?v=S5aJAaGZIvk&feature=PlayList&p=41F4CEB92D80C4B7&index=0

Main theme is HTML 5. The consequences of it for Silverlight merit quite a bit of thought. Silverlight comes with far better tools, of course - the dev experience will be very hard to compete against. But HTML 5 will have the advantage of ubiquity.

The other very interesting demo - for me, at least - was Java support in Google App Engine. It's an Eclipse plug-in with a built-in deployment support, GWT (http://en.wikipedia.org/wiki/Google_Web_Toolkit) and server-side support all built-in. Not nearly as slick as ASP.NET, but definitely a step in that direction.

A minor thing that I found quite funny, actually, was that the main presenter on the conference was Vic Gundotra - a former marketing/evangelist guy from MSFT who is now a VP of Engineering at Google. The silliness of Google management strikes again - say what you want about Microsoft, but at least our engineering managers are real engineers :-).

Wednesday, May 27, 2009

In Korean news...

Absolutely hilarious piece...
http://www.kcna.co.jp/index-e.htm

Incidentally, I cannot help noticing that "Korean Central News Agency" has a Japanese domain suffix... This is what Wikipedia has to say about Internet in North Korea:

"North Korea's first Internet café opened in 2002 as a joint venture with South Korean internet company Hoonnet. It is connected via a line to China. Foreign visitors can link their computers to the Internet through international phone lines available in a few hotels in Pyongyang. In 2005 a new internet café opened in Pyongyang, connected not through China, but through the North Korean satellite link. Content is most likely filtered by North Korean government agencies.[3][4] In 2003 a joint venture called KCC Europe between businessman Jan Holterman in Berlin and the North Korean government brought the commercial Internet to North Korea. The connection is established through a satellite link from North Korea to servers located in Germany. This link ended the need to dial ISPs in China.[5]

KCC Europe is attempting to regulate the .kp country code top-level domain (ccTLD); as of 2008[update] its site (kcce.kp) and Naenara (naenara.kp) are the only known to be active in the .kp domain. Its IP address resolves not to Asia but to servers at Internet Provider Berlin (ipberlin.com) in the German capital."

http://en.wikipedia.org/wiki/Communications_in_North_Korea

Tuesday, May 26, 2009

No sex for iPhone users (via Reddit)

"Apple has rejected Eucalyptus, an ebook reader that facilitates downloading public domain books from Project Gutenberg, because some Victorian books mention sex."

http://www.boingboing.net/2009/05/22/apple-says-no-projec.html

Monday, May 25, 2009

Priceless! (from Reddit today)

Filed under Dear Jimmy...

http://www.reddit.com/r/politics/comments/8n75m/dear_jimmy_pic/

And the first comment:
"Dear Jamail, I hope all is well in Guantanamo, though we all know better. We're very proud that you've not lied while being tortured. It could be a disaster if you give them a reason to attack more innocent people in order to make the suffering stop.

Dad was arrested because our neighbor covets our farmland, even though it hasn't seen water in 6 months. Your little brother decided to join the Baghdad police, but became a militant when he discovered that he could make more money from bribes from the soldiers. He lost his leg when an IED went off. Your sister was raped by mercenaries and then killed to cover it up. I tried to appeal to the government, but apparently they're not subject to our laws. There's still so much death. It's hard to stay hopeful.

There's an American soldier next to me reading a note complaining about seatbelts. I wonder if they even know why they're here."

http://www.reddit.com/r/politics/comments/8n75m/dear_jimmy_pic/c09th7o

Sunday, May 24, 2009

Why you should always buy memory separately

Dell, Latitude E6500, (through large enterprise portal, today): 4GB is $135

Newegg: 4GB is <$50

US health care by the numbers

Per-capita US health care expenditures in 2002: $5,267 ("Health Care Politics and Policy in America", Patel & Rushefsky, 2006)

Percentage by which it is higher than OECD average: 140%

Per-capita expenditures in Germany, France, Canada, and Great Britain: $2,817, $2,736, $2,931, and $2,160.

Percentage of health care costs that are administrative: 31%. (http://www.pnhp.org/publications/nejmadmin.pdf)

The same in Canada: 16.7%

Percentage of the total US health care costs last year that went to pay salary of UnitedHealth CEO: 0.14% (http://wonkroom.thinkprogress.org/2009/05/21/elizabeth-edwards-1-of-every-700-went-to-pay-salary-of-unitedhealth-ceo/)

US health care ranking in the world, according to WHO, in 2000: 37 (http://www.photius.com/rankings/healthranks.html)

French health care ranking: 1

Number of developed countries ranked lower than US: 0

US population longevity ranking in the world: 33 (tied with Cuba, http://www.usatoday.com/news/health/2007-05-18-un-life-expectancy_N.htm)

US rank in infant mortality: 29 (tied with Slovakia and Poland, http://www.webmd.com/parenting/baby/news/20081015/infant-mortality-us-ranks-29th)

Number of bankruptcies in US that are medical: 50% (http://www.pbs.org/moyers/journal/05222009/transcript4.html)

The share of those where people had health insurance: 3/4

Giving back capital

Constantly in the news here is the push from the banks to return government investments - the TARP funds - to escape from the strings that were attached to that money - specifically, the limits on executive compensation.

This is being discussed as if it were the most natural thing in the world - of course bankers want to be paid more! The question of how returning cheap capital benefits the shareholders (obviously, it does not) - ostensibly, the owners of these enterpirses - is never even mentioned.

It's a really weird concept of capitalism that we have here!

http://1-800-magic.blogspot.com/2009/04/capitalism-socialism.html

Wednesday, May 20, 2009

Breaking change in Malevich

This will read as a daily WTF, unfortunately... but after 6 months of development, I found a really, really nasty bug in Malevich, and if you're maintaining an installation, you should read this.

Interestingly, the bug was in my face all this time, I just wasn't paying attention.

Malevich stores a bunch of timestamps in its database - one for change list itself, one for each review iteration, and one for each file version. Normally these are stored in UTC - after all, how else would you store times in a web application where users are scattered around the globe?

Well, as it turned out, all but not all. Change list time stamp was stored as local time. Of course the bug was on display on the dashboard all this time, literally in my face. But I think this was because all the rest of the dates were in UTC and therefore not very useful - I just taught myself to ignore all time stamps. So I never noticed that unlike all the rest, these ones looked correct - and they should not have!

Anyway, for quite a while Malevich fielded complaints that the time stamps are silly, and I was thinking on and off on how I might fix that. You see, .NET runtime which Malevich is built upon is great at manipulating the time, except for one thing - the web site has no idea what user's time zone is - this is by design, a privacy issue.

The client side does know what the local time offset is, so the only way to transform time correctly must lay with the client.

So this is what Malevich does - each time stamp is wrapped in a <span id='timestamp' name='timestamp> element. When the page loads, I get all the elements by the name 'timestamp', and convert them to local time.

The conversion is pretty crappy from the globalization perspective (heck, but this is *free* software :-)) - because default JavaScript time conversion functions are terrible - they produce very long strings (for example, days of the week are spelled out by some browsers).

So I compose the date/time string myself, in US format. The only thing where I try to be smart is determine if the format is 12- or 24-hours upfront and then generate the date that would use the same format. Which does not work on Chrome because it uses 24-hour format no matter what.


//-----------------------------------------------------------------------
// 
// Copyright (C) Sergey Solyanik for The Malevich Project.
//
// This file is subject to the terms and conditions of the Microsoft
// Public License (MS-PL).
// See http://www.microsoft.com/opensource/licenses.mspx#Ms-PL for
// more details.
// 
//----------------------------------------------------------------------- 
//
// Javascript module to recalculate times to browser local.
//

//
// Is time a 24-hour format or AM/PM?
//
var TIME_FORMAT_24HR = ComputeDateFormat();

//
// Recomputes all date times from UTC to local.
//
function RebuildDates()
{
    var dates = document.getElementsByName('timestamp');
    for (var i = 0; i < dates.length; ++i)
    {
        var date = new Date(dates[i].firstChild.nodeValue);

        var month = date.getMonth() + 1
        var year = date.getYear()
        var day = date.getDate()

        if (day < 10) day = '0' + day
        if (month < 10) month = '0' + month
        if (year < 1000) year += 1900

        var hour = date.getHours();
        var minute = date.getMinutes();
        var seconds = date.getSeconds();

        var suffix = '';
        if (TIME_FORMAT_24HR)
        {
            if (hour < 10) hour = '0' + hour;
        }
        else
        {
            if (hour < 12) suffix = ' AM'; else suffix = ' PM';
            if (hour == 0) hour = 12;
            if (hour > 12) hour -= 12;
            if (hour < 10) hour = '  ' + hour;
        }
        
        if (minute < 10) minute = '0' + minute;
        if (seconds < 10) seconds = '0' + seconds;

        dates[i].firstChild.nodeValue =
            [month, '/', day, '/', year, ' ', hour, ':', minute,
            ':', seconds, suffix].join('');
    }
}

//
// Checks if the date in the current locale is AM/PM or 24 hours.
//
function ComputeDateFormat()
{
    var dateString = (new Date()).toLocaleTimeString();
    if (dateString.indexOf(' AM') == -1 && dateString.indexOf(' PM') == -1)
        return true;
    return false;
}

//
// Makes RebuildDates be called on page load.
//
if (window.attachEvent)
{
    window.attachEvent('onload', RebuildDates);
}
else
{
    if (window.onload)
    {
        var curronload = window.onload;
        window.onload = function()
        {
            curronload();
            RebuildDates();
        };
    }
    else
    {
        window.onload = RebuildDates;
    }
}

Anyway, so I make this change, and now I start looking at dates very closely, and - crrrrap! - all the time stamps now look great, except the most important one - the change list time stamp!

And the worst part of course is that it's in the database! Moreover, it's data! And it's now on a few dozen servers at Microsoft alone, and I have no idea how to even find them.

Anyway, I put notices on the web site (http://malevich.codeplex.com/Wiki/View.aspx?title=special%20note%20on%20Malevich%20upgrades, and on this blog (you're reading it), and in the change list description, and on Microsoft's internal share point site for Malevich.

I modify the installer so that it detects this condition (or, rather, the right version transition), and tries to fix up the problem:


USE [CodeReview]

DECLARE @UTCDate datetime
DECLARE @LocalDate datetime
DECLARE @TimeDiff int 
  
SET @UTCDate = GETUTCDATE()
SET @LocalDate = GETDATE()
SET @TimeDiff = DATEDIFF(hh, @LocalDate, @UTCDate) 

UPDATE [dbo].[ChangeList] SET TimeStamp = DATEADD(hh, @TimeDiff, TimeStamp)

What else can I do?

Of course, the script above does not work completely, either, because it does not account for the daylight savings time (which is really, really hard to compute in T-SQL - I love human time system!), and because it assumes that local time is the same as server time, which is of course incorrect, because a bunch of teams use the tool between here and China... But it does make the time somewhat more right than it was before.

So this is the story of the worst bug in Malevich in the 6 months since it was born.

And the moral of the story... what is the moral of this story? I wrote it primarily to attract the attention of Malevich admins to the fact that they need to be thoughtful about their next upgrade. But if you have a suggestion for a witty ending, offer it in the comments section! :-)

Why do Catholic clergy not oppose gay marriage in NY?

"The state’s Roman Catholic bishops have been somewhat distracted, too, having focused their lobbying energies this session on defeating a bill that would extend the statute of limitations for victims of sexual abuse to bring civil claims, and have appeared unprepared for the battle over marriage."

http://www.nytimes.com/2009/05/20/nyregion/20marriage.html?hp

Wow! Just... wow!

Tuesday, May 19, 2009

Serious play (TED)

Fair warning: allocate 30 minutes upfront. You won't be able to stop once this starts!

Going-away party

"Green was convicted last week in U.S. District Court in Kentucky of murder, rape, conspiracy and obstruction of justice in connection with a 2006 rape-and-murder south of Baghdad. A jury found him guilty of raping a 14-year-old girl, then killing her and setting her body on fire to destroy evidence. Green also was found guilty of killing the girl's parents and 6-year-old sister.
...
Ruth, who is John Green's sister, noted for the jury that Green's mother is not at the trial this week. The mother is moving and had to attend a going-away party, Ruth said."

http://www.cnn.com/2009/CRIME/05/18/kentucky.iraq.murder/index.html

Monday, May 18, 2009

You know that things are hitting the bottom...

...when even New York Times can not spell...

Monday, May 11, 2009

Telescopes can see... photons? (via Reddit).

http://government.zdnet.com/?p=4765

"Thursday, the Europeans will launch two much more advanced telescopes (right) that can see photons, even those reaching back to the Big Bang. Hubble sees visual light."

I kid you not.

This is a "technical" publication...

Where are you, Robin Crusoe? Where are you? Where have you been?

Monday, May 4, 2009

Stretched on $500k/year

"Nor does he appreciate being branded as "rich" when it's far from certain he'll ever build the kind of lavish nest egg the truly wealthy enjoy, especially after the current market meltdown."

"Kelly Lynch, the owner of a commercial maintenance company in Redondo Beach, Calif., is raising two kids with her partner, Jill Fenske, on a household income of $400,000. She's saving $800 a month for the children's college fund and $4,000 a month for retirement - a number that someday might make her rich. "If I blew my money like other people, I'd feel rich," says Lynch. Her views on taxes are befitting a born entrepreneur: "I think it would be unfair if someone tried to raise my taxes," says Lynch. "I don't think people should be penalized because they earn more.""

"Even at the upper end of the HENRY group, our cover subjects, Lindsay Mayer and her husband, Zach, a Dallas attorney, feel stretched on $500,000 a year."

"Tony Molino, 50, an attorney in Rancho Palos Verdes, Calif., speaks for legions of HENRYs: "I've worked 50 to 60 hours my entire life, and I don't have a lot left over at the end of the month. I'm comfortable, but when Joe Biden talks about sucking it up, getting patriotic, and paying more taxes, I get livid.""

http://money.cnn.com/2008/10/24/magazines/fortune/tully_henrys.fortune/index.htm

Poor people, not sure if they will ever be TRULY rich.

Here's a book that shows what really means to work hard: http://www.amazon.com/Nickel-Dimed-Not-Getting-America/dp/0805063897. And it is not sitting at the desk for 2 hours more a day. It's having 3 jobs as a janitor, a waitress, a maid, and despite working 14 hours a day not having enough money to rent an apartment or go to a doctor...

Sunday, May 3, 2009

To Joe the Plumbers among us...

...who vote for lower taxes for the rich because some day they expect to be rich themselves: why not buy TWO lottery tickets instead of one? The extra money from the second win will take care of the taxes!

http://www.americanprogress.org/issues/2006/04/b1579981.html

"By international standards, the United States has an unusually low level of intergenerational mobility: our parents’ income is highly predictive of our incomes as adults. Intergenerational mobility in the United States is lower than in France, Germany, Sweden, Canada, Finland, Norway and Denmark. Among high-income countries for which comparable estimates are available, only the United Kingdom had a lower rate of mobility than the United States."

Saturday, May 2, 2009

Why economic and social liberalism trends are incompatible

US has two dominant parties - the economically liberal but socially conservative Republicans, and socially liberal and economically conservative Democrats.

Many in my circle - the professional class - were long yearning for the third alternative - a party that is liberal both economically and socially. Someone to represent me :-)!

Why is this not happening? Unfortunately, for a reason.

Unrestricted economic liberalism mostly benefits the upper classes.

The lower classes are less educated, more religious (church and especially Christianity historically being used as a way to keep people in their place), and therefore more socially conservative.

Since the upper classes do not have enough votes to advance their agenda, they need to coopt the lower classes to vote against the lower classes' economic interests.

This of course is done by throwing down whatever red meat issues are currently at hand - "family values", gay marriage, "security", "morality", "gun rights", etc. Most of it is either social conservatism or fear because this is what the masses respond to.

And this is why economically liberal parties are bound to be socially conservative. There is simply no way for them to attract enough followers to matter otherwise.

Which is why you have such a high incidents of hypocrisy among Republican leaders - from Rush Limbaugh abusing drugs while preaching death to (other) drug addicts, to Larry Craig's restroom incident. They don't really give a flying f*ck about morals (or anything else they sell - gun rights, security, etc). They are just using this to get more idiots vote for them.

What does this say about the rich? Not much, unfortunately (I cannot bring myself to call Wall Street bankers and automaker CEOs "the elites" which to me means "the best"). By trying to extract the last penny while stirring the worst instincts in the society, they retard the progress of civilization.

Yes, they are rich, but as a result time moves slower for everybody. But you know what? The quality of life and the average longevity of the middle class in the age of antibiotics turns out to be better than that of the royalty in the Middle Ages.

Trading stem cell research for a few thousand bucks in taxes is not smart, and proves once again that IQ and compensation in today's America are unfortunately not correlated.

And this is usually the beginning of the end...

Why is it socially acceptable to be a conservative?

You generally don't see many people go around and say "I am a fascist!" or "I am a racist!". And yet there are plenty of people who proudly declare themselves to be conservative.

Let's take a brief look at the history.

Spanish Inquisition's goal was to suppress heresy - or, in other words, preserve traditional religious beliefs. It was a conservative movement that resulted in executions of thousands of people.

US itself was founded by progressives - conservatives were known as Loyalists at the time.

Over the last couple hundred years conservatives, standing athwart history (in the words of William Buckley), have opposed most of the civilization advances that we now take for granted:

emancipation of slaves
women's suffrage
8 hour workday
limitations on child labour
desegregation and racial equality
Social Security
Medicare

And you don't have to go too far back in time - it is probably safe to say that in 1960s most conservatives supported segregation, and that in 1920s most of them were against women's suffrage. You just cannot say the same about liberals.

Today, history is repeating itself with conservative attitudes towards everything from torture and gay rights to healthcare reform.

So why is the conservative brand still acceptable in the mainstream society after all of this? What am I missing?

Friday, April 24, 2009

The future is now!

Two links:

http://1-800-magic.blogspot.com/2007/12/windows-8-or-9-or-10.html

...and...

http://www.withinwindows.com/2009/04/24/secret-no-more-revealing-windows-xp-mode-for-windows-7/

A whole version earlier than I predicted.

Wednesday, April 22, 2009

Israel to US: shut up and do what I say (via Reddit)

"The Obama Administration will put forth new peace initiatives only if Israel wants it to, said Foreign Minister Avigdor Lieberman in his first comprehensive interview on foreign policy since taking office.

"Believe me, America accepts all our decisions," Lieberman told the Russian daily Moskovskiy Komosolets."

http://www.haaretz.com/hasen/spages/1080097.html

Tuesday, April 21, 2009

Dell Latitude: 8 hours on a single charge!

Tonight (by accident) I left my laptop unplugged and in a state where it could not go to sleep automatically. In the morning I found the laptop with its battery completely depleted, but still running!

Kudos to power management systems in Core 2 Duo and Windows Server 2008!

Monday, April 20, 2009

Laissez-faire (via Reddit)

"In Bucharest, you can buy a young girl for 8,000 euros. I mean buy them as in you own them forever or until you sell them to someone else. Most of them are sold by their parents in Moldova when they are little children. As soon as they’re bought, the owner brands them. This is normally a tattoo of his name."

http://www.viceland.com/int/v13n9/htdocs/slavery1.php?country=uk

While this sounds (reads?) insane, it is not really that unusual for an unregulated market to produce this phenomenon. It was not all this different in US in 1890s, and in Russia in 1990s - just the percentage of population is smaller.

Do laws have priority over money, or does money have priority over laws?

The Lifestyle

"It is a tricky situation in which some Americans find themselves after a long boom: They are by no means struggling, compared with the 98% of Americans who make far less, but depending on where they live and the lifestyle choices they have made, they don't necessarily feel rich, either."

http://finance.yahoo.com/focus-retirement/article/106934/Wealth-Less-Effect-Earning-Well-Feeling-Otherwise

Oh, yes, the life style choices... Take me, for instance. My personal choice, as anyone who knows me well will confirm, is to fly my own 747. I love to travel, and the fact that I have to wait in airports, sleep in a chair on long flights, and stand in long lines is simply unacceptable. It makes me feel positively broke. Middle class you say? As far as I am concerned, I am a pauper.

Now, Obama thinks that we are rich and wants to tax our hard earned money (that is, the amount above $250k) at 3% more. Which means that if we were to make, say, $350k next year, we'd be forking over $3000 more to the government in taxes. Instead of buying one of these: http://1-800-magic.blogspot.com/2009/04/computer-porn.html, which my chosen lifestyle dictates that I have. Talk about socialism!

Malevich has setup!

I've been working on Malevich - my code review system - on and off for almost 5 months now.

It turned out to be very successful - my team was using it for all of our first milestone, and was very happy about it (we're now pushing 1500 change lists that went through the system). Many other Microsoft teams adopted it since, to the tune of at least a couple hundred people - and these are just the people who contacted me.

The biggest problem so far was the setup. The deployment was hard - it required Visual Studio, a service pack, an expansion pack, and tons of manual work. A lot of errors were made, and in many cases I had to drop by and set things right afterwards.

Not any more! The first official build is out, and it has a real setup! The setup checks the dependencies, deploys the database, publishes the web site, and configures notifier automatically, no Visual Studio is required!

So if you wanted to try Malevich but were intimidated by the complexity of installation - the wait is over. You still do have to install Server 2008 with IIS and SQL Server 2008, but the rest is done for you.

Just get the MSI here: http://malevich.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=26374

Happy code reviewing!

Sunday, April 19, 2009

How is this not treason? (via Reddit)

"Rep. Jane Harman , the California Democrat with a longtime involvement in intelligence issues, was overheard on an NSA wiretap telling a suspected Israeli agent that she would lobby the Justice Department reduce espionage-related charges against two officials of the American Israeli Public Affairs Committee, the most powerful pro-Israel organization in Washington."

...

"In exchange for Harman’s help, the sources said, the suspected Israeli agent pledged to help lobby Nancy Pelosi, D-Calif., then-House minority leader, to appoint Harman chair of the Intelligence Committee after the 2006 elections, which the Democrats were heavily favored to win."

http://www.cqpolitics.com/wmspage.cfm?docID=hsnews-000003098436

Saturday, April 18, 2009

The new government

"As far as the most important matter is concerned, there is complete unanimity. Liberman, Netanyahu, Barak, Ellie Yishai of Shas and Danny Hershkovitz of the “Jewish Home” party are in total agreement about the Palestinians. All of them agree on the need to prevent the establishment of a real Palestinian state. All of them agree not to talk with Hamas. All of them support the settlement enterprise. During Barak’s stint as Prime Minister, the settlements grew even faster than during Netanyahu’s tenure. Liberman is himself a settler, Hershkovitz’s party represents the settlers. All of them believe that there is no need for peace, that peace is bad for us. (After all, it was Barak, not Netanyahu or Liberman, who coined the phrase “We Have No Partner for Peace”.)"

http://zope.gush-shalom.org/home/en/channels/avnery/1238277190

Computer porn

2x	Intel Xeon E5410 Harpertown 2.33GHz 12MB L2 Cache LGA 771 80W Quad-Core Processor	http://www.newegg.com/Product/Product.aspx?Item=N82E16819117150	$590
1x	SUPERMICRO MBD-X7DCL-I Dual LGA 771 Intel 5100 ATX Server Motherboard	http://www.newegg.com/Product/Product.aspx?Item=N82E16813182146	$300
6x	Kingston 4GB 240-Pin DDR2 SDRAM ECC Registered DDR2 667 (PC2 5300) Server Memory	http://www.newegg.com/Product/Product.aspx?Item=N82E16820134835	$504
5x	Western Digital VelociRaptor WD1500HLFS 150GB 10000 RPM 16MB Cache SATA 3.0Gb/s 3.5" Hard Drive	http://www.newegg.com/Product/Product.aspx?Item=N82E16822136296	$900
1x	Athena Power AP-P4ATX85FE 20+4Pin 850W Single ATX12V & EPS12V Power Supply	http://www.newegg.com/Product/Product.aspx?Item=N82E16817104126	$155
1x	SILVERSTONE KUBLAI Series KL03-B Black 2.5mm aluminum front door, 0.8mm SECC body ATX Full Tower Computer Case	http://www.newegg.com/Product/Product.aspx?Item=N82E16811163100	$155

...yields an 8-core, 24GB monster capable of hosting 10 VMs comfortably for $2600.

Saturday, April 11, 2009

Capitalism, socialism

Wikipedia defines capitalism thus:

"Capitalism is an economic system in which wealth, and the means of producing wealth, are privately owned and controlled rather than state owned and controlled. Through capitalism, the land, labor, and capital are owned, operated, and traded by private individuals either singly or jointly, and investments, distribution, income, production, pricing and supply of goods, commodities and services are determined by voluntary private decision in a market economy. A distinguishing feature of capitalism is that each person owns his or her own labor and therefore is allowed to sell the use of it to employers." (http://en.wikipedia.org/wiki/Capitalism).

The definition puts "ownership" and "control" of the capital in the same sentence, but does not stipulate that it is the same individual that both owns and controls the resources.

And therein lies our problem.

Commonly accepted theory of why capitalism is preferable to socialism goes like this: the owners generally make better decisions about allocating the resources than non-owners. When the owners do make sub-optimal decision, they lose their ownership stake to the smarter and more responsible new owners.

So far, so good. Except this is not at all the system that we have in the United States. In the US, the majority of the capital is owned by one set of people - the shareholders - and managed by an entirely different set of people - the corporate management.

The interests of these groups of people are not at all aligned (http://1-800-magic.blogspot.com/2007/12/risk-vs-reward.html), and, what's worse, neither group is truly interested in the long-term success of the business.

As a result, while the executives in younger companies which are still managed by the original owners do often tend to do the right thing, I cannot call the behavior of the managerial class in most American enterprises anything other than looting.

This approach to management has been most visible on Wall Street, but look at any older US enterprise, and you will see the same looting of company resources everywhere, from car manufacturers to steel mills to utility companies.

Comparing to the old Soviet days, the system as we have it right now may actually be worse. At least the bureaucrats in Soviet Union were not paying themselves multimillion dollar bonuses and did not zip around in private planes as the economy crumbled.

Thursday, April 9, 2009

Discounting risk

I wrote this article in December 2007 and it describes perfectly the situation we're in now: http://1-800-magic.blogspot.com/2007/12/risk-vs-reward.html

Wednesday, April 8, 2009

If you had only two news articles to read this week (via Reddit)...

...I would recommend "Ten principles for a Black Swan-proof world" by Nicolas Taleb (best known for his book "The Black Swan: The Impact of the Highly Improbable" about underpricing risk) and The dark side of Dubai. This last one is absolutely surreal. Make sure you read it at least through "The Lifestyle" chapter. It is really, truly, absolutely unbelievable.

Tuesday, April 7, 2009

We need this guy back!

http://images.pcworld.com/news/graphics/162166-billgatesFINAL2_original.jpg

Thursday, April 2, 2009

Visual Studio SP1 installer blues

I have but one question - what is taking it so long?

It takes 1.5 hours on my 2.5GHz, 4GB Core 2 Duo laptop running Windows Server 2008.

In 1.5 hours one could, at a typical speed of my laptop:
- Write 263GB of data to disk, at 50MBps
- Transfer 131GB of data over the network, at 200Mbps
- Copy 15 TB (!) of memory (on 800Mhz bus)
- Execute a whopping 26 trillion instructions (2 cores retiring 2 instructions per core per clock).

WHAT DOES THIS THING DO???

Wednesday, April 1, 2009

Weather plays April Fool's in Seattle

It's April 1st, and it's SNOWING! Here's what it looks like from our window.

Tuesday, March 31, 2009

The Lord of the Rings and Atlas Shrugged (via Reddit)

"There are two novels that can change a bookish fourteen-year old’s life: The Lord of the Rings and Atlas Shrugged. One is a childish fantasy that often engenders a lifelong obsession with its unbelievable heroes, leading to an emotionally stunted, socially crippled adulthood, unable to deal with the real world. The other, of course, involves orcs."

http://www.amptoons.com/blog/archives/2009/03/29/quote-8/

Justice and accountability, the American way (via Reddit)

1. The hospital makes a medical error; patient dies.
2. The hospital falsifies the medical record, including the testimony of the doctor given during the investigation.
3. The doctor who gave the testimony produces the original copy.
4. The hospital settles out of the court. The article does not say this, but I bet that the people who were involved in hiding the evidence probably still work for that hospital.

“There isn’t going to be a trial,” he said. “The hospital offered the patient’s family an 8-figure settlement, and they have accepted.”

http://open.salon.com/blog/amytuteurmd/2009/03/30/they_killed_my_patient_then_they_tried_to_hide_it

Malevich hits 1000th code review!

We've been using it for 3 months now in our team of roughly 45 people - 20 devs and 25 testers. This week we've had 1000th code review completed.

Here are a few stats, as of right now:

1008 change lists
17081 file versions
2863 review iterations
7851 comments

If you work at Microsoft and would like to try Malevich, I've set up a test change list here: http://sergeysomsg2/Malevich/default.aspx?cid=299. You can add yourself as a reviewer, leave comments in files, and submit review iterations.

If you're outside Microsoft and want to try it, the source and installation instructions are here: http://www.codeplex.com/Malevich.

Monday, March 30, 2009

Welcome to the ultimate in banana republics! (via Reddit)

Written by a former chief economist at IMF.

"Typically, these countries are in a desperate economic situation for one simple reason—the powerful elites within them overreached in good times and took too many risks. Emerging-market governments and their private-sector allies commonly form a tight-knit—and, most of the time, genteel—oligarchy, running the country rather like a profit-seeking company in which they are the controlling shareholders. When a country like Indonesia or South Korea or Russia grows, so do the ambitions of its captains of industry. As masters of their mini-universe, these people make some investments that clearly benefit the broader economy, but they also start making bigger and riskier bets. They reckon — correctly, in most cases — that their political connections will allow them to push onto the government any substantial problems that arise."

(emphasis mine)

http://www.theatlantic.com/doc/print/200905/imf-advice

Sunday, March 29, 2009

Empire strikes forward

It seems that vast majority of Star Wars technology was designed and built by the marketing department. Here are a few engineering suggestions that could help the bad guys win in Epsiodes -3 to 0...

1) Use firearms. Blasters are slow, not very powerful, and easily deflected by a light saber. An AK-47 would be far, far more effective. And don't forget the training! Despite the advantage in fire power, the storm troopers clearly can't shoot straight. Take them out to a range once in a while, and see how much more effective they become only after a couple training sessions!

2) If you use droids to fight your wars, build the best kind - do not waste time and money on anything else. It looks like the Destroyer model is more effective than50 units of the other kind (and even more effective than a Jedi). I doubt it could be more than 3 times more expensive. Why not just build Destroyers?

3) And while we are on this subject, let's have droids use more effective communication channels than human speech when they talk to each other. The same goes for all interfaces between the droids and everything else - spaceships, vehicles, and weapons. There is no reason while a droid should press a button to fire a weapon - a wireless interface would control it far more efficiently. Also, do invest in computerized targeting for your weapons. If your droids can already see, making them very accurate shooters is a relatively trivial task - ask any engineer.

4) Missiles seem to be an extremely effective weapon in Star Wars, except that there are very few of them. Why not build more of them and launch, say, 20 or 30 at a time? I doubt the Jedi, who have visible trouble dealing with one or two, would be able to escape ten. Also, consider building missiles that are FASTER than the star fighters they attack.

5) Consider using encryption. It looks like anyone can plug into the system anywhere and take control of everything in the space ships. This problem has been solved years ago! It really is not that hard.

6) Pack animals are slow, can't carry much, and are very, very hard to maintain. They need food, living compartments, etc. Consider eliminating them from the army in favor of mechanized transporters. Animals are not very effective in executions either - this has been proven many, many times. Instead of staging elaborate shows with giant predators that always fail to kill the prisoners, just shoot your captives, and put their heads on a stick.

7) I cannot overemphasize the importance of conventional firearms. If the probe sent to assassinate Princess Amidala had used a regular rifle instead of the poisonous centipedes, the subsequent events might have taken a very different turn. And in space, consider using nuclear weapons. The laser guns you currenlty mount on your ships are massively underpowered.

8) Did you know that space ships do not need wings to fly in space? Once you get entirely comfortable with this (yes, I do know it's extremely hard, given that they even make fying noises while moving through vacuum, but making this leap of faith might be crucial to your survival - do it!) you can start using very different design paradigms - like, for example, minimizing the surface area so your spacecraft is easier to protect and harder to target.

9) On the subject of spaceship designs - there must be something in the way you build them that makes them explode after they are hit. Unfortunately, this applies to all other vehicles as well. While this generates impressive visuals, this design point leads to unnecessary casualties among your troops. Consider enclosing whatever it is that explores in extra layers of protection. You have already solved this for your small weapons, which do not seem to have the same problem. Why not use similar design everywhere?

10) This is more of a tactical rather than an engineering advice, but I will give it anyway. Instead of deploying the Walkers and other weird ground assault vehicles, consider attacking from the air to suppress the adversary's ground troops.

Follow these simple steps, and you will be invincible - at least as long as the Good Guys(TM) are controlled by the marketing people.

Saturday, March 28, 2009

Free as in beer!

The GNU connotation of software as a form of speech never made any sense to me.

"Free speech" means the freedom of a person to express himself or herself without a fear of persecution. It does not mean the freedom to use the results of this expression by everyone.

Quite the contrary - the literary works by the greatest writers are the results of exercising free speech. But rip "Harry Potter", and you will - deservedly - find yourself slapped with a lawsuit, a fine, and maybe a jail sentence.

The free software movement has produced a lot of great things - from GNU development tools to Linux, BSD, Apache, and Firefox. It has also produced legions of activists that bring in the level of politics, acrimony, and fanboyism that is unbecoming to honorable competition in an area that is as much a science as it is an art.

By doing so they actually take the freedom away from people who chose to distribute the results of their work through channels that are different from the ones favored by the Free Software Foundation.

So here's my variant of "free software". The code distributed on this blog is free - free as in beer. You can redistribute it in full or in parts, with or without source code. You can use it in any derivative work without giving me any credit.


//-----------------------------------------------------------------------
// <copyright>
// Copyright (C) Sergey Solyanik.
//
// This software is in public domain and is "free as in beer". It can be
// redistributed in full or in parts for free and without any preconditions.
// </copyright>
//-----------------------------------------------------------------------

De-gnoming your blog

My blog has been filling up with spam over the last few months - the nasty, long messages that pollute the comment space and are a pain to delete because the bastards were programmatically sticking them in tens and tens of posts.

Luckily, Google has APIs to Blogger, and better yet, they even have C# libraries than make coding against these API easy. I didn't even have to dust off Eclipse!

So an hour later I had the program that allowed me to clean up the spam quickly.

The code sucks in the entire blog (including the abstracts of posts and comments) and allows one to quickly search and delete spam. Type 'help' on the command prompt, and it will explain the usage.

A few convenient shortcuts:


List or delete all comments from someone:
    list|delete comments by name
e.g.
    list comments by peter.w

List or delete all comments that have the same author as well as the text:

    list|delete comments like sample_comment_uri
e.g.
    list comments like http://www.blogger.com/feeds/3554166144204741789/3010398536200323405/comments/default/4945114575786176865

List or delete all comments with a phrase in title:
    list|delete comments with text_string
e.g.
    list comments with wow gold

List comments in a blog or a message:
    list comments in blog_name|first_words_of_post_title
e.g.
    list comments in Back to Microsoft
    list comments in 1-800-MAGIC

If you're one of the truly insane types that would run code compiled by a random guy from the Internet, the built version is here: http://www.solyanik.com/drop/BlogDeSpammer.zip. Otherwise, the code is below, free as in beer!

Incidentally, it might make a good primer on how to retrieve and manipulate Blogger posts and comments programmatically.


//-----------------------------------------------------------------------
// <copyright>
// Copyright (C) Sergey Solyanik.
//
// This software is in public domain and is "free as in beer". It can be
// redistributed in full or in parts for free and without any preconditions.
// </copyright>
//----------------------------------------------------------------------- 
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Google.GData.Client;

namespace BlogDeSpammer
{
    /// <summary>
    /// Comment class. Just a trivial holder of essential comment data.
    /// </summary>
    class Comment
    {
        public string Text;
        public string Author;
        public string EditUri;

        public Comment(string text, string author, string editUri)
        {
            Text = text;
            Author = author;
            EditUri = editUri;
        }
    }

    /// <summary>
    /// Post class. Just a trivial holder of essential post data.
    /// </summary>
    class Post
    {
        public string Title;
        public string CommentsFeed;
        public ICollection<Comment> Comments;

        public Post(string title, string commentsFeed)
        {
            Title = title;
            CommentsFeed = commentsFeed;
            Comments = null;
        }
    }

    /// <summary>
    /// Blog class. Just a trivial holder of essential blog data.
    /// </summary>
    class Blog
    {
        public string Name;
        public string Feed;
        public ICollection<Post> Posts;
        public Blog(string name, string feed)
        {
            Name = name;
            Feed = feed;
            Posts = null;
        }
    }

    /// <summary>
    /// Main functionality.
    /// </summary>
    class Program
    {
        /// <summary>
        /// Gets collection of blogs belonging to the user.
        /// </summary>
        /// <param name="blogger"> Blogger service. Must be initialized with the parameters.
        /// </param>
        /// <returns> Collection of blogs. </returns>
        private static ICollection<Blog> GetBlogs(Service blogger)
        {
            List<Blog> blogs = new List<Blog>();

            FeedQuery query = new FeedQuery();
            query.Uri = new Uri("http://www.blogger.com/feeds/default/blogs");
            AtomFeed results = blogger.Query(query);
            while (results != null && results.Entries.Count > 0)
            {
                foreach (AtomEntry entry in results.Entries)
                {
                    string blogFeedLink = null;
                    foreach (AtomLink link in entry.Links)
                    {
                        if (BaseNameTable.ServiceFeed.Equals(link.Rel))
                        {
                            blogFeedLink = link.HRef.ToString();
                            break;
                        }
                    }
                    blogs.Add(new Blog(entry.Title.Text, blogFeedLink));
                }

                if (results.NextChunk == null)
                    break;

                query.Uri = new Uri(results.NextChunk);
                results = blogger.Query(query);
            }

            return blogs;
        }

        /// <summary>
        /// Retrieves all the posts for the blog.
        /// </summary>
        /// <param name="blogger"></param>
        /// <param name="blog"></param>
        /// <returns> Collection of posts. </returns>
        private static ICollection<Post> GetPosts(Service blogger, Blog blog)
        {
            List<Post> posts = new List<Post>();

            FeedQuery query = new FeedQuery();
            query.Uri = new Uri(blog.Feed);
            AtomFeed results = blogger.Query(query);
            while (results != null && results.Entries.Count > 0)
            {
                foreach (AtomEntry entry in results.Entries)
                {
                    string commentsFeedLink = null;
                    foreach (AtomLink link in entry.Links)
                    {
                        if ("replies".Equals(link.Rel) &&
                            "application/atom+xml".Equals(link.Type))
                        {
                            commentsFeedLink = link.HRef.ToString();
                            break;
                        }
                    }
                    Console.Write(".");
                    if (commentsFeedLink != null)
                        posts.Add(new Post(entry.Title.Text, commentsFeedLink));
                }

                if (results.NextChunk == null)
                    break;

                query.Uri = new Uri(results.NextChunk);
                results = blogger.Query(query);
            }

            return posts;
        }

        /// <summary>
        /// Gets comments for the given post.
        /// </summary>
        /// <param name="blogger"> Blogger service, initialized with credentials. </param>
        /// <param name="post"> The post. </param>
        /// <returns> Collection of credentials. </returns>
        private static ICollection<Comment> GetComments(Service blogger, Post post)
        {
            List<Comment> comments = new List<Comment>();

            FeedQuery query = new FeedQuery();
            query.Uri = new Uri(post.CommentsFeed);
            AtomFeed results = blogger.Query(query);
            while (results != null && results.Entries.Count > 0)
            {
                foreach (AtomEntry entry in results.Entries)
                {
                    Console.Write(".");
                    comments.Add(new Comment(entry.Title.Text, entry.Authors[0].Name,
                        entry.EditUri.ToString()));
                }

                if (results.NextChunk == null)
                    break;

                query.Uri = new Uri(results.NextChunk);
                results = blogger.Query(query);
            }
            return comments;
        }

        /// <summary>
        /// Main. Does all the work.
        /// </summary>
        /// <param name="args"> Command line parameters. </param>
        static void Main(string[] args)
        {
            if (args.Length != 2)
            {
                Console.WriteLine("Usage: blogdespammer username password");
                return;
            }

            string username = args[0];
            string password = args[1];

            Console.Write("Loading");

            Service blogger = new Service("blogger", "BlogDeSpammer");
            blogger.Credentials = new GDataCredentials(username, password);

            ICollection<Blog> blogs = GetBlogs(blogger);
            foreach (Blog blog in blogs)
            {
                blog.Posts = GetPosts(blogger, blog);
                foreach (Post post in blog.Posts)
                    post.Comments = GetComments(blogger, post);
            }

            Console.WriteLine("\n\nType 'help' for help, 'exit' to quit.");
            for (; ; )
            {
                Console.Write("> ");
                string input = Console.ReadLine();
                if (input.Equals("exit", StringComparison.CurrentCultureIgnoreCase))
                    break;

                if (input.Equals("help", StringComparison.CurrentCultureIgnoreCase))
                {
                    Console.WriteLine(
                        "Type:\n\texit\n\t\t-- to quit.\n\thelp\n\t\t--to get this text.\n");
                    Console.WriteLine(
                        "\tlist blogs\n\t\t-- to list available blogs.\n");
                    Console.WriteLine(
                        "\tlist posts in blog\n\t\t-- to list posts in a blog.\n");
                    Console.WriteLine(
                        "\tlist comments in blog\n\t\t-- to list comments in a blog.\n");
                    Console.WriteLine(
                        "\tlist comments in post\n\t\t-- to list comments in a post.\n");
                    Console.WriteLine(
                        "\tlist comments by author\n\t\t-- to list comments by " +
                        "a particular author.\n");
                    Console.WriteLine(
                        "\tlist comments like commenturi\n\t\t-- to list comments " +
                        "identical to a comment.\n");
                    Console.WriteLine(
                        "\tlist comments with text\n\t\t-- to list comments " +
                        "containing text segment.\n");
                    Console.WriteLine(
                        "\tdelete comments by author\n\t\t-- to delete all " +
                        "comments by a particular author.\n");
                    Console.WriteLine(
                        "\tdelete comments like commenturi\n\t\t-- to delete " +
                        "all comments that are the same as a " +
                        "comment with this URI.\n");
                    Console.WriteLine(
                        "\tdelete comments with text\n\t\t-- to list comments " +
                        "containing text segment.\n");
                    Console.WriteLine("Names above could be either text names or URIs.");
                    continue;
                }

                if (input.Equals("list blogs", 
                    StringComparison.CurrentCultureIgnoreCase))
                {
                    foreach (Blog blog in blogs)
                        Console.WriteLine("{0} ({1})", blog.Name, blog.Feed);
                    continue;
                }

                if (input.StartsWith("list posts in ", 
                    StringComparison.CurrentCultureIgnoreCase))
                {
                    string blogname = input.Substring(14);
                    foreach (Blog blog in blogs)
                    {
                        if (blog.Name.StartsWith(blogname) ||
                            blog.Feed.Equals(blogname))
                        {
                            foreach (Post post in blog.Posts)
                                Console.WriteLine("{0} ({1})", post.Title, 
                                    post.CommentsFeed);
                        }
                    }
                    continue;
                }

                if (input.StartsWith("list comments in ", 
                    StringComparison.CurrentCultureIgnoreCase))
                {
                    string name = input.Substring(17);
                    foreach (Blog blog in blogs)
                    {
                        if (blog.Name.StartsWith(name) || blog.Feed.Equals(name))
                        {
                            foreach (Post post in blog.Posts)
                            {
                                Console.WriteLine("{0} {1}", post.Title, 
                                    post.CommentsFeed);
                                foreach (Comment comment in post.Comments)
                                    Console.WriteLine("-- {0} ({1})\n{2}\n", 
                                        comment.Author, comment.EditUri,
                                        comment.Text);
                            }
                            break;
                        }

                        foreach (Post post in blog.Posts)
                        {
                            if (post.Title.StartsWith(name) || 
                                post.CommentsFeed.Equals(name))
                            {
                                Console.WriteLine("{0} {1}", post.Title, 
                                    post.CommentsFeed);
                                foreach (Comment comment in post.Comments)
                                    Console.WriteLine("-- {0} ({1})\n{2}\n", 
                                        comment.Author, comment.EditUri,
                                        comment.Text);
                            }
                        }
                    }
                    continue;
                }

                if (input.StartsWith("list comments by ", 
                    StringComparison.CurrentCultureIgnoreCase))
                {
                    string author = input.Substring(17);
                    foreach (Blog blog in blogs)
                    {
                        foreach (Post post in blog.Posts)
                        {
                            foreach (Comment comment in post.Comments)
                            {
                                if (comment.Author.Equals(author, 
                                    StringComparison.CurrentCultureIgnoreCase))
                                {
                                    Console.WriteLine("{0} : {1}",
                                        blog.Name, post.Title);
                                    Console.WriteLine("-- {0} ({1})\n{2}\n",
                                        comment.Author, comment.EditUri,
                                        comment.Text);
                                }
                            }
                        }
                    }
                    continue;
                }

                if (input.StartsWith("list comments with ", 
                    StringComparison.CurrentCultureIgnoreCase))
                {
                    string text = input.Substring(19);
                    foreach (Blog blog in blogs)
                    {
                        foreach (Post post in blog.Posts)
                        {
                            foreach (Comment comment in post.Comments)
                            {
                                if (comment.Author.Contains(text) || 
                                    comment.Text.Contains(text))
                                {
                                    Console.WriteLine("{0} : {1}",
                                        blog.Name, post.Title);
                                    Console.WriteLine("-- {0} ({1})\n{2}\n",
                                        comment.Author, comment.EditUri,
                                        comment.Text);
                                }
                            }
                        }
                    }
                    continue;
                }

                if (input.StartsWith("list comments like ", 
                    StringComparison.CurrentCultureIgnoreCase))
                {
                    string prototypeUri = input.Substring(19);
                    Comment prototypeComment = null;

                    foreach (Blog blog in blogs)
                    {
                        foreach (Post post in blog.Posts)
                        {
                            foreach (Comment comment in post.Comments)
                            {
                                if (prototypeUri.Equals(comment.EditUri))
                                {
                                    prototypeComment = comment;
                                    goto found;
                                }
                            }
                        }
                    }
                    continue;

                found:
                    foreach (Blog blog in blogs)
                    {
                        foreach (Post post in blog.Posts)
                        {
                            foreach (Comment comment in post.Comments)
                            {
                                if (comment.Author.Equals(prototypeComment.Author,
                                    StringComparison.CurrentCultureIgnoreCase) &&
                                    comment.Text.Equals(prototypeComment.Text))
                                {
                                    Console.WriteLine("{0} : {1}", blog.Name, post.Title);
                                    Console.WriteLine("-- {0} ({1})\n{2}\n", 
                                        comment.Author, comment.EditUri,
                                        comment.Text);
                                }
                            }
                        }
                    }
                    continue;
                }

                if (input.StartsWith("delete comments by ", 
                    StringComparison.CurrentCultureIgnoreCase))
                {
                    string author = input.Substring(19);

                    foreach (Blog blog in blogs)
                    {
                        foreach (Post post in blog.Posts)
                        {
                            List<Comment> deleted = new List<Comment>();

                            foreach (Comment comment in post.Comments)
                            {
                                if (comment.Author.Equals(author, 
                                    StringComparison.CurrentCultureIgnoreCase))
                                {
                                    Console.WriteLine("{0} : {1}", blog.Name, post.Title);
                                    Console.WriteLine("-- {0} ({1})\n{2}\n", 
                                        comment.Author, comment.EditUri,
                                        comment.Text);
                                    blogger.Delete(new Uri(comment.EditUri));
                                    deleted.Add(comment);
                                }
                            }

                            foreach (Comment comment in deleted)
                                post.Comments.Remove(comment);
                        }
                    }
                    continue;
                }

                if (input.StartsWith("delete comments like ", 
                    StringComparison.CurrentCultureIgnoreCase))
                {
                    string prototypeUri = input.Substring(21);
                    Comment prototypeComment = null;

                    foreach (Blog blog in blogs)
                    {
                        foreach (Post post in blog.Posts)
                        {
                            foreach (Comment comment in post.Comments)
                            {
                                if (prototypeUri.Equals(comment.EditUri))
                                {
                                    prototypeComment = comment;
                                    goto found;
                                }
                            }
                        }
                    }
                    continue;

                found:
                    foreach (Blog blog in blogs)
                    {
                        foreach (Post post in blog.Posts)
                        {
                            List<Comment> deleted = new List<Comment>();

                            foreach (Comment comment in post.Comments)
                            {
                                if (comment.Author.Equals(prototypeComment.Author,
                                    StringComparison.CurrentCultureIgnoreCase) &&
                                    comment.Text.Equals(prototypeComment.Text))
                                {
                                    blogger.Delete(new Uri(comment.EditUri));
                                    deleted.Add(comment);
                                    Console.WriteLine("{0} : {1}", blog.Name, post.Title);
                                    Console.WriteLine("-- {0} ({1})\n{2}\n", 
                                        comment.Author, comment.EditUri,
                                        comment.Text);
                                }
                            }
                            foreach (Comment comment in deleted)
                                post.Comments.Remove(comment);
                        }
                    }
                    continue;
                }

                if (input.StartsWith("delete comments with ", 
                    StringComparison.CurrentCultureIgnoreCase))
                {
                    string text = input.Substring(21);
                    foreach (Blog blog in blogs)
                    {
                        foreach (Post post in blog.Posts)
                        {
                            List<Comment> deleted = new List<Comment>();

                            foreach (Comment comment in post.Comments)
                            {
                                if (comment.Author.Contains(text) || 
                                    comment.Text.Contains(text))
                                {
                                    blogger.Delete(new Uri(comment.EditUri));
                                    deleted.Add(comment);
                                    Console.WriteLine("{0} : {1}", blog.Name, post.Title);
                                    Console.WriteLine("-- {0} ({1})\n{2}\n", 
                                        comment.Author, comment.EditUri,
                                        comment.Text);
                                }
                            }

                            foreach (Comment comment in deleted)
                                post.Comments.Remove(comment);
                        }
                    }
                    continue;
                }

                Console.WriteLine(
                    "Can't parse. Please use 'help' to look up valid commands.");
            }


            Console.WriteLine("Done!");
        }
    }
}