Among all the idiocy printed today about Chrome OS

...finally, the voice of reason! Ladies and Gentlemen, I give you... fake Steve Jobs!

http://fakesteve.blogspot.com/2009/07/lets-all-take-deep-breath-and-get-some.html

The mother of all bull...

"Google Drops A Nuclear Bomb On Microsoft. And It’s Made of Chrome."

http://www.techcrunch.com/2009/07/07/google-drops-a-nuclear-bomb-on-microsoft-and-its-made-of-chrome/

The idiots in the press are at it again, cooking a sensation by blowing up an interesting tidbit of information way out of proportion.

Let me point out two obvious facts.

(1) The entire consumer market is rather small as a share of Microsoft revenue (10%?). The netbooks most likely represent less than 1% of the company's revenue stream. You cannot possibly call a "nuclear bomb" something that targets so little money.

(2) The smart phone market will always be much bigger than a netbook market. So if the "nuclear bomb" metaphor made any sense, Apple has dropped it years ago with iPhone.

Here's another stupid quote of the day:

'"One of Google's major goals is to take Microsoft out, to systematically destroy their hold on the market," said Mr Enderle.

"Google wants to eliminate Microsoft and it's a unique battle. The strategy is good. The big question is, will it work?"'

http://news.bbc.co.uk/2/hi/technology/8139711.stm

When I was at Google, the last thing people there were thinking about was Microsoft. I maybe have heard Microsoft mentioned a grand total of 10 times in my year plus there. What Googlers do care about is building cool things that attract attention and make customers come to their sites. THAT strategy clearly works. Destroying Microsoft - not so much (Netscape tried that approach).

My own take on this - thank you, Google! Windows 8/IE 9 will be better for your efforts. It often takes a competitor to persuade us that a segment of a market is important (unfortunate, but true). With this announcement Google did just that.

Do you have a health insurance?

Don't be so sure. You might lose it when you actually need it. Apparently, insurance companies slap a $1M surcharge on corporate policies that carry expensive patients. The companies then face a choice of whether to essentially pay you a $1M+ salary or...

http://www.dailykos.com/storyonly/2009/7/7/751100/-How-I-lost-my-health-insurance-at-the-hairstylists

Incidentally, in 3/4 of all medical bankruptcies (which are half of all bankruptcies in the US) people had health insurance.

http://1-800-magic.blogspot.com/2009/05/us-healthcare-by-numbers.html

BMI is bogus... because it embarrasses USA

It was making sense up to a point where an author claimed that 200 years ago most people led sedentiary life styles, although I had to ignore his quip on "if the formula does not describe the data, rig the formula" (this, of course, is what science - at least theoretical physics - is all about).

But when I got to the end, it was this: BMI does not make sense because...

"10. It embarrasses the U.S.

It is embarrassing for one of the most scientifically, technologically and medicinally advanced nations in the world to base advice on how to prevent one of the leading causes of poor health and premature death (obesity) on a 200-year-old numerical hack developed by a mathematician who was not even an expert in what little was known about the human body back then."

http://www.npr.org/templates/story/story.php?storyId=106268439&sc=fb&cc=fp

Come to think about it, an even more ridiculous fact is that our entire space program is based on a 300-year-old formula developed by a theologian!



This pearl of logical reasoning comes to you directly from a Stanford (!) Professor (!) of Mathematics (!) Keith Devlin...

http://www.stanford.edu/~kdevlin/

P.S. The author of this blog takes no position on the validity of BMI as a measure of human obesity, only on the validity of the referenced above argument against it.

Scriptster: C# as a scripting language

I love Python! Unlike the vast majority of script languages that evolved ad-hoc, Python was built in a controlled way and has features, syntax, and runtime which click together.

I learned it at Google, got readability in it (http://1-800-magic.blogspot.com/2008/01/after-8-months-no-longer-noogler.html, and have written a few thousand lines of code since. I must say that of all the scripting languages, Python is probably the most amenable to writing thousand-plus line programs (maybe with the exception of Ruby).

As far as I am concerned, it would be nice if all other command interpreter scripting died (sorry, PowerShell) and Python were integrated into shells everywhere.

There is one problem with Python, however - it is yet another language to learn, and yet another development/runtime environment to maintain. Its integration with Windows is good, but not nearly as good as C# which was literally made for Windows. And so after I came back to Microsoft (http://1-800-magic.blogspot.com/2008/06/back-to-microsoft.html) and found myself doing most things in C# and C++, I started getting more and more rusty with Python.

Of course, being a dev manager does not help - my opportunities to code are few and far between.

What that meant that scripting became harder and harder. Soon I found myself writing small executables in C# instead of scripts. Which was all good, except that you end up with two things - a source file and an executable, which now need to be maintained together, checked in together, etc. And for very small things, it's a considerable overhead.

And then I had an idea. .NET, you see, ships with a C# compiler in the box - it is present on every (updated) Windows system. What if I were to write a small program, a C# "interpreter" of sorts, that would run C# programs directly from a command line as if they were batch files?

And so Scriptster was born.

Scriptster is a single executable that allows you to run C# programs directly, without manually compiling them. It automatically compiles C# code before running (and caches the compiled versions so the next execution is faster), but to the user it is completely transparent. It is fast, too - even the first, compiling, invocation takes fractions of a second. Subsequent runs are instantaneous.

Scripter is copy-deployable: simply copy it into a directory of your choice, run

    scriptster --install

(from a command prompt started with administrator privileges - "Run as Administrator"), and close and reopen CMD windows where you expect to be using it (CMD needs update to PATHEXT environment variable, which it reads on load).

After this, you can author and run C# scripts. For example, open Notepad, and type the following:

using System;

class Program
{
    static void Main(string[] args)
    {
        Console.WriteLine("This program was invoked with the following command line parameters:");
        foreach(string s in args)
            Console.WriteLine("{0}", s);
    }
}

Save this file as testscript.csscript (the files need to have .csscript extension to work with Scriptster), and you can run it directly from the command line:

    c:\temp> testscript Blah blah blah
    This program was invoked with the following command line parameters:
    Blah
    blah
    blah

You can run considerably more involved scripts with Scriptster of course, in fact, any C# program can be run, as long as it is (1) in one file, and (2) only relies on assemblies in GAC.

Here's an example of a program that queries LDAP for user alias:

//#ref System.DirectoryServices.dll
using System;
using System.Collections.Generic;
using System.DirectoryServices;
using System.Text;

class Program
{
    static void Main(string[] args)
    {
        DirectorySearcher ds = new DirectorySearcher();

        ds.PropertiesToLoad.Add("mail");

        foreach (string alias in args)
        {
            ds.Filter = "(SAMAccountName=" + alias + ")";
            SearchResult result = ds.FindOne();
            if (result == null)
            {
                Console.Error.WriteLine("Could not resolve {0}", alias);
                continue;
            }
            Console.WriteLine("{0}'s email is {1}", alias,
                result.Properties["mail"][0].ToString());
        }
    }
}

Notice "//#ref" at the top of the file? This is how you tell Scriptster about referenced assembly.

You can edit scripts in Visual Studio, debug them, then rename the files to .csscript extensions and run them right from the command line. You can take existing small programs, and run them from the command line, too.

Interested? You can download Scriptster, including the source code, from CodePlex: http://scriptster.codeplex.com.

Give it a whirl, and let me know how it works out.

The story of Mel - oldie but goodie

http://www.pbm.com//~lindahl/mel.html

Jeff Immelt (GE's CEO) on the road ahead

http://www.businessweek.com/the_thread/economicsunbound/archives/2009/06/immelt_speech_u.html

Obama has the Ring...

• Indefinite detentions: check!
• Broad executive powers: check!
• Secrecy: check!
• Signing statements: check!
• Escalation of foreign wars: check!
• ...

http://sipseystreetirregulars.blogspot.com/2009/04/obama-and-ring-of-power.html

Couple of consumer guides

A must read for information on how to use your credit card (fine print translated into human-readable form)...

http://www.mint.com/blog/finance-core/the-descent-into-credit-card-debt/

... and on how to use your iPhone...

http://www.reddit.com/r/WTF/comments/8w094/att_charges_adam_savage_of_mythbusters_11k/c0alsiz

"A friend of mine used to work for AT&T customer service. He had a call one day from a guy working for a very small company. A company size of five people, in fact. All of them decided to get iPhones and get a shared plan. Then they all decided to go out of the country...apparently for over a month.

Well, they spent a little less than a month over seas when one of them called my friend to ask some innocuous support question. They had not yet seen their bill of over $300,000. My friend did not say a word to him, but hung up and laughed his ass off.

Originally I thought this story was unbelievable, and I doubted it. I found it was possible after some rough calculations, but still it was kind of an extreme case.

Now, I see more and more of these stories popping up. Five people on the same plan, out of country, and all using their iPhones extensively...I'm starting to believe."

ActiveDirectory and disk imaging: not a happy combination

When I first heard about Hyper-V snapshotting I was extremely excited. This feature allows one to freeze an image of the virtual machine's hard drive (take a snapshot), and revert back to it at any point in time.

Moreover, it supports a snapshot tree: you can install the OS, take a snapshot, install one application, take a snapshot, revert back to the original image, install another application, and, again, take a snapshot. As a result, you now have three images which you can boot at any point in time (although not simultaneously): clean OS, app1 install, and independent - and clean - app2 install.

If you ever had to test your software in multiple environments, this is an absolute Holy Grail.

So I did it and it worked - for a while.

Unfortunately, one of the security features of the NT domain is that machine accounts periodically (once a month) change their passwords. This is driven by the client, not AD server (as described here: http://blogs.technet.com/askds/archive/2009/02/13/machine-account-password-process.aspx - which is a good introduction on how machine passwords work), and can - in theory - be turned off. But it's on by default, and is probably on as a security policy at most actual corporate installations.

So in a month the currently running version of VM changes its password. Which then renders all the rest of the snapshots useless: they have the old passwords. If you boot any of the snapshot, your VM can no longer connect to the domain. If you disconnect and rejoin, it gets a new machine SID and a new password. Which means that the password (and SID) of the version of the VM that was running previously - and all other snapshots, as a matter of fact, - is now bad.

All this means that after one month, the snapshot tree that you've just invested so much time building becomes completely useless.

The problem is not limited to Hyper-V per se. It manifests in every imaging solution - Vista/Server 2008 backup, Norton Ghost, etc. The only way to fix it - if your domain policy allows it - is to disable password change. Which brings us back to the link I mentioned earlier.

http://blogs.technet.com/askds/archive/2009/02/13/machine-account-password-process.aspx

The difference between United States and Soviet Union...

...might be that in the US most people wholeheartedly believe the propaganda, where's in SU most people didn't. As far as the foreign policy goes, the actions of the two countries seem to be about the same.

http://digbysblog.blogspot.com/2007/04/truths-consequences-by-digby-since.html

There is a duality of 1984 and Brave New World. In 1984 the government rules because everybody is afraid. In Brave New World it rules because nobody cares. But the net result is still the same.

New US Motto: We are too big to fail

http://freakonomics.blogs.nytimes.com/2009/02/24/the-new-six-word-motto-for-the-us-is/

Open source in China

This place: http://ostatic.com/blog/actuate-survey-open-source-booming-in-china-germany-and-other-regions says that 80% of Chinese use Open Source software.

Yet this impromptu Google survey says that only 8% of people around Times Square know what a browser is: http://1-800-magic.blogspot.com/2009/06/oh-customers.html.

Doesn't compute, does it? People who don't know what a browser is would know about Open Source and be able to distinguish it from other forms of software licenses?

I think I know what's going on here: 80% of Chinese users think that Windows and Office are open source products because they got pirated copies of them for free :-)!

We're going to be proud of our OS again!

This is Win7 running on a VM with Visual Studio and an instance of SQL server. The CPU spikes are compiles. Notice 0% CPU and only 1.32GB of RAM.

Cost of crapware in battery life

I watched a presentation about power last week. Among other interesting numbers in it - a clean install of Vista has less than 1% of CPU utilization on idle; an image from an OEM (including a bunch of 3rd party software) had ~7%. An increment of 10% CPU utilization leads to 8% less battery life...

Oh, customers... Part Deux.

A picture is worth 1000 words...

But 1000 words is roughly 5K, and a moderately-sized picture is at least 50K. Plus - unlike pictures - the text is searchable. Go figure...

Talking to a wall... (via Reddit)

In Jerusalem, a female journalist heard about an old Jew who had been going to the Western Wall to pray, twice a day, everyday, for a long, long time. So she went to check it out. She goes to the Western Wall and there he is! She watches him pray and after about 45 minutes, when he turns to leave, she approaches him for an interview.
"I'm Rebecca Smith from CNN. Sir, how long have you been coming to the Western Wall and praying?"
"For about 50 years."
"50 years! That's amazing! What do you pray for?"
"I pray for peace between the Jews and the Arabs. I pray for all the hatred to stop and I pray for our children to grow up in safety and friendship."
"How do you feel after doing this for 50 years?"
"Like I'm talking to a fuckin' wall."

http://www.reddit.com/r/atheism/comments/8tkf8/in_jerusalem_a_female_journalist_heard_about_an/

Oh, customers...

I kept maybe five textbooks from college. One of them was Ordinary Differential Equations (via Reddit)

http://www.reddit.com/r/AskReddit/comments/8sios/who_was_the_best_teacher_youve_ever_had_what_made/c0aamlv

I hate reposting, but this is too good to leave it up to Reddit's comments retention policy. Reproduced for posterity. But if you like it, do go to the link above and upvote...

By kleinbl00:
"The guy on the left.

He was a graduate from the University of Zagreb or something and he had an awesome accent. And he was beanpole tall and twitched around. He was like a cross between Cosmo Kramer and The Count from Sesame Street.

He was incredibly passionate about what he tought. He would bang on the chalkboard, breaking the chalk, and say "DoyounderstandTHIS? DoyouGETthis?" and look at us all intense. And then he'd roll on with what he was going.

The dude fucking loved math. He was teaching ordinary diff EQ and you'd think he was Beethoven explaining crescendos. And he really didn't give a shit about homework. He'd sit there and drill us through stuff as if our life depended on us understanding. I saw that guy tear up a couple times. More than once, the professor next door would step in and ask him to keep it down. He spent maybe an hour explaining Euler's identity - and I shit you not, he got us to tear up, too.

We had one homework assignment. It was given about halfway through the class. We had a week to do it. And it took that entire week, in groups of two or three, five to six hours a day to do it. And we handed it in, and he didn't even grade it for like three more weeks.

When we got it back, there was a pallor over the class (what was left of it - a third of the class had dropped out). I had a 23%. I'm sure I turned gray. I went to see him - what the hell could I do? I mean, I needed to pass -

"DoNotWorryaboudit. AveragewaseighTEEEN. Yooodooverygooood."

When it came down to the final, it was one, simple, benign sheet of paper. It had one problem on it. There were absolutely no numbers on it other than (1). It started with "imagine a function..."

We had two hours. At 1:15, nobody had handed in a thing. I was just sitting there stunned, grinding through the first half. At 1:30, nobody had handed in a thing. At 1:45, he said

"Eeef...youtakezeetest choam wityou and feenishit, I veel passyou."

Nobody got up. We sat there and cranked through the fucking test. The survivors, anyway. We started with 30 people in the class. We finished with twelve.


--------------------------------------------------------------------------------

I kept maybe five textbooks from college. One of them was Ordinary Differential Equations. It was an expensive book - too expensive for me to afford. The first time I went to see him, I apologized for not having the book.

He gave me his."

Barak Hoover Obama

A very interesting article in this month's Harper's (http://www.harpers.org/archive/2009/07/0082562?redirect=1022411470, subscription required) contrasts Hoover - a deliberative, progressive, well-meaning technocrat - with Barak Obama.

The article is making the case that, although Obama is often compared with FDR, by trying to take the "middle road" and avoiding the open warfare with his detractors, he is actually emulating Herbert Hoover, and that this approach is doomed to failure.

"Franklin Roosevelt also took office imagining that he could bring all classes of Americans together in some big, mushy, cooperative scheme. Quickly disabused of this notion, he threw himself into the bumptious give-and-take of practical politics; lying, deceiving, manipulating, arraying one group after another on his side—a transit encapsulated by how, at the end of his first term, his outraged opponents were calling him a “traitor to his class” and he was gleefully inveighing against “economic royalists” and announcing, “They are unanimous in their hatred for me—and I welcome their hatred.”

Obama should not deceive himself into thinking that such interest-group politics can be banished any more than can the cycles of Wall Street. It is not too late for him to change direction and seize the radical moment at hand. But for the moment, just like another very good man, Barack Obama is moving prudently, carefully, reasonably toward disaster."

More TFS blues - adding a user

The more I use this system, the more I get a suspicion that it was not designed by developers. Or maybe we found a totally clean room bunch of developers that have never used a source control system before? (Would that be... PMs?)

I wrote previously (http://1-800-magic.blogspot.com/2009/03/adventures-in-tfs-continued.html) about the incredible amount of pain TFS is to install, and put together a simple step-by-step guide on how to install it here: http://1-800-magic.blogspot.com/2009/03/how-to-install-tfs-on-single-domain.html. (Incidentally, this is now one of the most popular articles on the blog.)

Yesterday I spent about an hour trying to figure out how to allow my daughter to use TFS server that I've set up for our home projects.

In Perforce (Microsoft uses a derivative of Perforce for quite a few of its internal projects) everything is simple: you type "p4 protect" (or "sd protect" at Microsoft), it opens a file in a notepad, and you add a line that looks like this:

write user sergey * //depot/...

And you are done.

Here's what you have to do in TFS:

(1) Add a user to a list of "licensed users". This list is not displayed by default when you navigate to the project's security settings, you have to click on a checkbox to make it display all groups.

I missed this step, and it was not mentioned on the TFS documentation page that deals with setting permissions (http://msdn.microsoft.com/en-us/library/bb558971.aspx). I performed all the magic incantations from that page, but TFS still would not connect.

And of course the error that it was showing listed security among three other options, but gave no suggestions of what might go wrong.

Eventually, a Google search on the error code led me to the words "licensed user", and then here: http://msdn.microsoft.com/en-us/library/ms404880.aspx.

(2) You have to add the user to Contributor group in your project using TFS Explorer.

This is described here: http://msdn.microsoft.com/en-us/library/bb558971.aspx.

(3) Separately, you have to grant the user access to the sharepoint site.

Also here: http://msdn.microsoft.com/en-us/library/bb558971.aspx.

(4) Separately, you have to grant the user access to the reporting portal.

And again, here: http://msdn.microsoft.com/en-us/library/bb558971.aspx.

I understand cutting features to make the deadlines, but c'mon, ladies and gentlemen of TFS, does this thing really have to be such a pain in the butt for the administrator? Especially that we're competing with Perforce where deployment is done by a single double-click?

What is your number of nines?

Ran into an interesting page today - a list of scheduled down times for Blogger: http://status.blogger.com/.

It looks like Blogger is down for roughly 10 minutes once a month (in addition to a Picasa downtime that impairs its ability to accept images).

10 minutes a month does not look like much, but it does amount to about 2 hours of downtime per year. Is two hours a year good or bad?

The system's availability is defined as the ratio of uptime to the total time:

                      MTTF
    Availability = -----------
                   MTTF + MTTR

where MTTF is the mean time between failures, and MTTR is the mean time that takes to bring the system back online. The "failure" here should be understood as a measure of system ability to process requests rather than a fault: a scheduled downtime is not a bug, but the system is not available nevertheless.

2 hours of downtime in a year yield the availability of 365.25 * 24 / (2 + 365.25 * 24) = 99.9%, or "3 nines", which puts Blogger in a category of "Well-managed" systems.

Here are the definitions of various levels of availability given in Jim Gray's famous book on transaction processing (http://www.amazon.com/Transaction-Processing-Concepts-Techniques-Management/dp/1558601902):

System type	Unavailability (min/year)	Availability	Class
Unmanaged	52560	90%	1
Managed	5256	99%	2
Well-managed	526	99.9%	3
Fault-tolerant	53	99.99%	4
High-availability	5	99.999%	5
Very-high-availability	0.5	99.9999%	6
Ultra-availability	0.05	99.99999%	7

As the Blogger's example shows, it's fairly hard to create a fault-tolerant (or above) system - you have to account for things that range from OS and software patching to the maintenance of the power equipment in the data centers.

One might think that the hardware failures and software bugs cause most of the availability problems, but it is actually the scheduled maintenance that creates majority of work, because it causes a lot of downtime. Once you figured out how to deal with the maintenance, the unavailability due to bugs is probably already taken care of by the same measures.

And at server MTTF of roughly 14 years, one should only be worrying about hardware (assuming that the failure can be detected and the job reallocated within one hour) when availability starts approaching 5 nines.

How many nines does your system have?

Navigating the Dell price labyrinth

Last week I bought a couple of big (8-core, 32GB, fast disk) boxes from Dell because they closely match the hardware that we're going to be running on in our data centers. For the speed and quality of the hardware the price ended up being very reasonable $4k/box (when bought with Microsoft discount).

While doing it, I discovered a curious thing - if you configure the box with 32GB RAM upfront, the memory comes up quite costly. If you just buy the server with 8GB default, and buy 32GB RAM separately - the peripherals section of the same Dell web site, the Dell-recommended RAM upgrade for this very workstation - the total cost is over $800 less (and you end up with 8GB of unused RAM that originally was there).

If you work at Microsoft, use this trick and watch our stock price go up 50 cents!

If, furthermore, you buy disks separately, you save another $150 or more over the price of pre-installed hard drives.

These are Microsoft-internal prices, which - for obvious reasons - I can not quote, but the problem is even bigger on the external Dell web site, because everything is even more expensive there.

Here, for example, is the price for memory - preconfigured - for Precision T7400.


You can see that the price for 32GB is $2960, and for 64GB it is a whopping $17870 (!).

Alternatively you can buy the same RAM on Newegg, so for 32GB you will by 4 8GB kits at $165-$240 for a total of $660 to $960:


Or, for 64GB you would buy 8 of these, at $420 each - $3360 - almost $15000 cheaper than on the Dell's web site!


It goes beyond RAM.

Dell wants $550 for 1TB hard drive (although they give their small business buyer a break - a 1 TB drive for the same T7400 there is "only" $430).


The prices on Newegg for 1TB hard drive range from $110-150 for a retail box to $74-$90 for OEM packaging.

Morale - if you are buying Dell computers, getting the parts on the side will save you a bundle. It is much cheaper to buy the minimal configuration, throw away the memory and the hard drive it comes with, and buy the replacement RAM and disks from Newegg (or anywhere else).

Note that the same is not true with CPUs - Dell CPUs are ~$200 more expensive than the same parts on Newegg, BUT you have to have a non-standard Dell heatsink, which - when bought separately - is very pricey. Plus replacing CPUs is not as trivial as RAM and disks.

Another interesting observation is that prices on Dell's home/small business site are often - usually - considerably less than on corporate web site. Most likely Dell uses this sales tactic to give its corporate users a "discount". Recently I bought a laptop using Microsoft EPP program, just to discover that the 7% "discount" that Dell provides simply matches the price that is available on its small business site for all.

Finally, for peripherals - docking stations and the like - it pays to check eBay. A $199 (plus shipping, handling, and tax) advanced port replicator for Latitude can be easily had there - new - for $129 (reasonable shipping, and no tax).

Overlapped I/O in Windows

One of the current puzzles that our team is dealing with is database performance. As part of our platform we're building a performance counter collection system, most of which lives in a datawarehouse-like structure in SQL server.

The specific problem we're facing is a lack of parallelism on data loads. Our usage does not quite fit standard database models, of which two are typical - OLAP (on-line analytical processing - essentially read-only database that is updated, say, nightly, contains tons of data, and is queried frequently using specific set of queries for which it was designed) and OLTP (on-line transaction processing - where the "hot" subset of the data is smaller, but is read and updated very frequently).

Our database is the worst of both worlds - it writes about 1 million rows a minute, and the reads are rather infrequent. (An argument that SQL server is not the right technology for this can be made, but this discussion is outside the context of this post).

There are two basic problems.

First is the lack of parallelism in bulk transfers on the SQL server - on an 8-core, 32GB machine with 3 10k RPM data drives (which is what $4k currently buys at Dell) the load (and index update) part is completely single-threaded. Which means that one core runs at 100% CPU, and the rest are doing nothing.

I wrote a very simple program that does all the data transforms in memory, and this part is completely parallel - the server runs at 60-70% CPU utilization - and is very fast. Unfortunately, the very last insert into SQL - Amdahl's law! - is now controls the overall performance.

The second - bigger - problem is that eventually the index no longer fits in the RAM, and due to the nature of the data it seems to be rewritten almost entirely on every upload. If I restrict SQL memory to 6GB, this covers roughly 2 hours of input, the disk starts thrashing really badly, and the load times go to hell - what initially would take only 30 seconds becomes 3 minutes.

One potential solution is to partition the database, but once we do it, it starts negatively affect the performance of our queries.

The data flow through the system is really quite small relative to the power of the hardware. The rows we're writing are only a few dozens of bytes each, and the aggregate data flow is less than 1MBps. But because the entire index is being rewritten all the time, the box keeps writing at at the rate of 20-30 MBps for several minutes.

This seemed kinda slow (although speeding it up is of course not going to be a part of the solution - we need to figure out a more global approach) - so I wanted to check what the hardware is capable of doing in terms of disk throughput, so I quickly typed up a piece of code that also makes a reasonable tutorial on how to use overlapped I/O on Windows. Hence this article.

The essential steps as as follows.

First, open the file using FILE_FLAG_OVERLAPPED | FILE_FLAG_NO_BUFFERING:

HANDLE hFile = CreateFileW(szFileName, GENERIC_WRITE, 0, NULL, CREATE_ALWAYS, FILE_FLAG_OVERLAPPED | FILE_FLAG_NO_BUFFERING, NULL);

Second, you want to use DMA for the data transfer, to avoid copying, and for that the data buffer should be aligned on a sector boundary. The easiest way is to VirtualAlloc it, since this will force the buffer to be contiguous and aligned on 64k:

buffer = VirtualAlloc(NULL, bufferSize, MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE);

Use WriteFileEx to schedule your I/O. The hEvent part of the overlapped is not used, so a developer can use it to pass context to the completion routine. The API is quite weird this way - you'd expect that hEvent would take an event to signal on I/O completion - but that's not how it works.

overlapped.hEvent = (HANDLE)1;
overlapped.Offset = (DWORD)offset;
overlapped.OffsetHigh = (DWORD)(offset >> 32);
offset += bufferSize;
WriteFileEx(hFile, buffers, bufferSize, &overlapped, WriteFinished);

where WriteFinished gets called when the write is done:

static void CALLBACK WriteFinished(DWORD dwErrorCode,
DWORD dwNumberOfBytesTransfered,
LPOVERLAPPED lpOverlapped)
{
lpOverlapped->hEvent = (HANDLE)2;
}

Finally, once the write is scheduled, the thread must be put into alertable sleep (or wait - use the SleepEx/WaitForSingleObjectEx/WaitForMultipleObjectEx functions that take the alertable flag:

SleepEx(INFINITE, TRUE);

or

DWORD dwRes = WaitForSingleObjectEx(hSomeEvent, INFINITE, TRUE);
if (dwRes == WAIT_IO_COMPLETION)
{
// write is done
...

Obviously, you will be scheduling multiple I/Os - using an array of OVERLAPPED structures and hEvent fields inside them to keep track of what has finished and what has not is handy.

Below is the full text.

Oh, yes, and that server does ~110 MBps writes on a 10K RPM disk using the code below (and ~80 MBps writes on 7200 RPM disk), with practically zero CPU utilization.

//-----------------------------------------------------------------------
// <copyright>
// Copyright (C) Sergey Solyanik. All rights reserved.
//
// This software is in public domain and is "free as in beer". It can be
// redistributed in full or in parts for free and without any preconditions.
// </copyright>
//----------------------------------------------------------------------- 
#include <windows.h>
#include <stdio.h>

#define MAX_OUTSTANDING_WRITES 64

enum WriteProgress
{
    WriteScheduled = 1,
    WriteSucceeded = 2,
    WriteError = 3
};

static void CALLBACK WriteFinished(DWORD dwErrorCode,
                                   DWORD dwNumberOfBytesTransfered,
                                   LPOVERLAPPED lpOverlapped)
{
    if (dwErrorCode == 0)
    {
        lpOverlapped->hEvent = (HANDLE)WriteSucceeded;
        return;
    }

    wprintf(L"Error: %d\n", dwErrorCode);
    lpOverlapped->hEvent = (HANDLE)WriteError;
}

int wmain(int argc, WCHAR* argv[])
{
    if (argc != 5)
    {
        wprintf(L"Usage: writedata filename total_size chunk_size "
            L"number_of_writes\n");
        wprintf(L"Note: total_size is in megabytes\n");
        wprintf(L"      chunk_size is in bytes and must be a power of 2 "
            L"greater than 2048\n");
        wprintf(L"      number_of_writes is the number of writes "
            L"that are scheduled simultaneously\n");
        return 1;
    }

    if (GetFileAttributesW(argv[1]) != 0xffffffff)
    {
        wprintf(L"%s already exists.", argv[1]);
        return 2;
    }

    int size = _wtoi(argv[2]);
    if (size <= 0)
    {
        wprintf(L"Size must be a positive number.");
        return 3;
    }

    LARGE_INTEGER bytes;
    bytes.QuadPart = (__int64)size * 1024L * 1024L;

    int bufferSize = _wtoi(argv[3]);
    if (bufferSize <= 0)
    {
        wprintf(L"Buffer size should be a positive number.");
        return 4;
    }

    if (bufferSize & (bufferSize - 1))
    {
        wprintf(L"Buffer size must be power of 2");
        return 4;
    }

    if (bufferSize < 4096)
    {
        wprintf(L"Buffer size is too small");
        return 4;
    }

    int simwrites = _wtoi(argv[4]);
    if (simwrites <= 0)
    {
        wprintf(L"Number of simultaneous writes should be a "
            L"positive number.");
        return 5;
    }

    if (simwrites > MAX_OUTSTANDING_WRITES)
    {
        wprintf(L"Number of simultaneous writes is too large.");
        return 5;
    }

    HANDLE hFile = CreateFileW(argv[1], GENERIC_WRITE, 0, NULL,
        CREATE_ALWAYS, FILE_FLAG_OVERLAPPED | FILE_FLAG_NO_BUFFERING,
        NULL);

    SetFilePointerEx(hFile, bytes, NULL, FILE_BEGIN);
    SetEndOfFile(hFile);

    OVERLAPPED overlappeds[MAX_OUTSTANDING_WRITES];
    memset(overlappeds, 0, sizeof(overlappeds));

    void *buffers[MAX_OUTSTANDING_WRITES];
    for (int i = 0 ; i < simwrites ; ++i)
    {
        buffers[i] = VirtualAlloc(NULL, bufferSize,
            MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE);
        memset(buffers[i], i, bufferSize);
    }

    DWORD tick = GetTickCount();

    __int64 totalScheduled = 0;
    __int64 totalWritten = 0;
    __int64 nWrites = bytes.QuadPart / bufferSize;
    int nOutstanding = 0;
    for (;;)
    {
        int nScheduled = 0;
        for (int i = 0; i < simwrites; ++i)
        {
            if ((int)overlappeds[i].hEvent == WriteScheduled)
            {
                ++nScheduled;
                continue;
            }

            if ((int)overlappeds[i].hEvent == WriteSucceeded)
            {
                totalWritten += bufferSize;
                wprintf(L"\r%I64d", totalWritten);
                memset(&overlappeds[i], 0, sizeof(OVERLAPPED));
            }

            if ((int)overlappeds[i].hEvent == WriteError)
            {
                CancelIo(hFile);
                goto finished;
            }

            if (nWrites > 0)
            {
                overlappeds[i].hEvent = (HANDLE)WriteScheduled;
                overlappeds[i].Offset = (DWORD)totalScheduled;
                overlappeds[i].OffsetHigh =
                    (DWORD)(totalScheduled >> 32);
                totalScheduled += bufferSize;
                --nWrites;
                ++nScheduled;

                if (!WriteFileEx(hFile, buffers[i], bufferSize,
                    &overlappeds[i], WriteFinished))
                {
                    DWORD dwErr = GetLastError();
                    wprintf(L"Write error %d\n", dwErr);

                    CancelIo(hFile);
                    goto finished;
                }
            }
        }

        if (nScheduled == 0)
            break;

        SleepEx(INFINITE, TRUE);
    }

finished:
    for (int i = 0 ; i < simwrites; ++i)
        VirtualFree(buffers[i], 0, MEM_RELEASE);

    CloseHandle(hFile);

    int seconds = (GetTickCount() - tick) / 1000;
    if (seconds <= 0)
        seconds = 1;

    wprintf (L" in %d seconds (%d MBps)\n", seconds,
        size / seconds);

 return 0;
}

In US religion is not...

...the opiate of the people (http://en.wikipedia.org/wiki/Opium_of_the_people) - it's Coca-Cola's corporate partner!

Prepare to believe:
http://blogs.answersingenesis.org/museum/2007/07/13/thirsty-museum-guests-choose-coke/

Canadian healthcare

Various myths regarding single-payer system in Canada
http://www.denverpost.com/opinion/ci_12523427

Coding for Kim Jong-il

You are on the plane from Seattle to Beijing when your 747 makes an emergency landing at a military landing strip in North Korea. Korean security forces quickly dispose of the crew and the passengers (the "Dear Leader" (http://en.wikipedia.org/wiki/Kim_Jong-il) likes the plane and wants to keep it for himself by faking a crash), but you are spared because they learn that you are a software developer working one of the best software companies in the world.

As you learn shortly from a personal audience with the "Dear Leader" himself, Kim Jong-il - protected by a nuclear shield - is now ready to branch into software development. His goal is to build North Korea into a technology powerhouse - with a market share as strong that of Microsoft and Google, and with employees as tightly controlled as Apple's.

So you are given a choice of potential technologies one of which you will need to build in the span of one year. At the end of the year, your work will be evaluated solely on a technical success (i.e. you are not expected to win a competition with existing products, but you are expected to build something that is competitive purely from the technical point of view).

If your code passes this evaluation, you live and potentially even reap some undisclosed benefits (the "Dear Leader" is vague on this point). If it does not, a standard process for "people who know too much" is applied to you. If you decline to cooperate, the aforementioned process is applied to you right now.

If you do agree to work on the project, you are given all possible technical means to achieve your task - an unlimited number of computers (clients and server), special dedicated fiber to the Internet, etc. You are allowed to read research papers, but not source code - including, but not limited to, any open source project. Your code must be 100% clean room implementation!

No pre-existing software infrastructure is available, other than a C++/Java/C# compiler, a text editor of your choice, and your choice of a version of Windows or Linux as an operating system.

As a first step, you get to chose the project you will be working on. The options are:
(1) C++ compiler
(2) A general-purpose database engine
(3) A general-purpose Internet search engine
(4) An operating system

Which project would you choose, and why?

Today's installer WTF

We bought my parents-in-law a Zune as a present (MSFT people get a discount at the company store). They have a pretty slow DSL connection, so downloading 131MB Zune software took almost 30 minutes.

But guess what happened next? When the setup started, it started downloading again - for another half an hour.


This begs two questions:
(1) What was in the original 130MB file - just a setup program? In 131MB???
(2) How big can a media player possibly be? Media Player Classic is less than 2MB, and VLC can play absolutely every file known to humanity in 16. Yes, Zune software also has transcoders, but I've got a whole bunch of these, too - and they are just a few megs in other packages. Why do we need a good quarter of a gigabyte?

LDAP, .NET, and Annual Review

It's that time of the year again - the annual performance review started at Microsoft today.

Annual review is an important part of every Microsoftie's career - it defines not only one's compensation, but, more importantly, reputation and mobility. It's much easier to move around the company if the review history is good. The more fun the project is, the bigger competition it attracts, and the more discriminating the hiring managers are when it comes to review history.

Good reviews are required (but, of course, not sufficient) for a great career at Microsoft.

Our team practices peer performance reviews. This is one of the Googleisms that I brought back from my year there. The theory is that peers know more than a manager about the work done by the employee, and are harder to fool, so majority of the review feedback and the ultimate review grade comes from them.

(See more here: http://1-800-magic.blogspot.com/2007/12/life-by-committee-or-management-google.html, and if you work at Microsoft and want to know more, drop me an email - I have written a whitepaper on peer review system and will be glad to share it with you.)

One of the artifacts of the peer review model is the amount of mail every manager has to send to request it - say, 5 employees times 6 reviewers per employee - that's 30 emails.

With tons of cut and paste, it took me over an hour of frantic typing last time around when we started using peer reviews during the mid-year career discussion cycle.

Then of course there is the fun of verifying that the right mail goes to the right person - individually, because you don't want to share everybody's reviewer list with everybody else, - that I didn't by mistake send a mail that is intended for one reviewer to the other, that all names in the email are correct, etc.

This time I was not about to sit and cut and paste for hours again - the problem called for a programmatic solution.

I could have probably easily put together a web site, but ensuring security a bunch of super-important personal information for a few dozen people is not exactly my idea of fun - and can't be reasonably done in under 30 minutes. The data had to stay in email.

But at least I could send the invitations automatically.

A tool was pretty quickly born that took an alias of the reviewee, and a list of aliases of the reviewers, formatted email from a template, and sent it out. In the process, I found out how to extract information from Active Directory using LDAP - I needed to get the names of the users from their aliases.

The job is surprisingly easy using DirectorySearcher class of the System.DirectoryServices namespace (you do need to add the reference to it before using).

Here's a code snippet that extracts and prints a bunch of interesting information about a user from its AD record.

Happy LDAP'ing!

//-----------------------------------------------------------------------
// <copyright>
// Copyright (C) Sergey Solyanik. All rights reserved.
//
// This software is in public domain and is "free as in beer". It can be
// redistributed in full or in parts for free and without any preconditions.
// </copyright>
//----------------------------------------------------------------------- 
namespace ldapquery
{
    using System;
    using System.Collections.Generic;
    using System.DirectoryServices;
    using System.Linq;
    using System.Text;

    /// <summary>
    /// Sample for LDAP usage in C#. Looks up a name, and then 
    /// </summary>
    class Program
    {
        /// <summary>
        /// Prints out various interesting user properties from ActiveDirectory
        /// </summary>
        /// <param name="args"></param>
        static void Main(string[] args)
        {
            if (args.Length == 0)
            {
                Console.WriteLine("Usage: ldapquery alias1 alias2...");
                return;
            }

            DirectorySearcher ds = new DirectorySearcher();

            ds.PropertiesToLoad.Add("displayName");
            ds.PropertiesToLoad.Add("givenName");
            ds.PropertiesToLoad.Add("telephoneNumber");
            ds.PropertiesToLoad.Add("mobile");
            ds.PropertiesToLoad.Add("homephone");
            ds.PropertiesToLoad.Add("mail");
            ds.PropertiesToLoad.Add("title");
            ds.PropertiesToLoad.Add("department");
            ds.PropertiesToLoad.Add("manager");

            foreach (string alias in args)
            {
                ds.Filter = "(SAMAccountName=" + alias + ")";
                SearchResult result = ds.FindOne();
                if (result == null)
                {
                    Console.Error.WriteLine("Could not resolve {0}", alias);
                    continue;
                }

                Console.WriteLine("{0}:", alias);

                if (result.Properties["displayname"].Count > 0)
                {
                    Console.WriteLine("Display name: {0}",
                        result.Properties["displayname"][0].ToString());
                }

                if (result.Properties["givenname"].Count > 0)
                {
                    Console.WriteLine("Given name: {0}",
                        result.Properties["givenname"][0].ToString());
                }

                if (result.Properties["telephonenumber"].Count > 0)
                {
                    Console.WriteLine("Office Phone: {0}",
                        result.Properties["telephonenumber"][0].ToString());
                }

                if (result.Properties["mobile"].Count > 0)
                {
                    Console.WriteLine("Mobile Phone: {0}",
                        result.Properties["mobile"][0].ToString());
                }

                if (result.Properties["homephone"].Count > 0)
                {
                    Console.WriteLine("Display name: {0}",
                        result.Properties["homephone"][0].ToString());
                }

                if (result.Properties["mail"].Count > 0)
                {
                    Console.WriteLine("Email: {0}",
                        result.Properties["mail"][0].ToString());
                }

                if (result.Properties["title"].Count > 0)
                {
                    Console.WriteLine("Title: {0}",
                        result.Properties["title"][0].ToString());
                }

                if (result.Properties["department"].Count > 0)
                {
                    Console.WriteLine("Department: {0}",
                        result.Properties["department"][0].ToString());
                }

                if (result.Properties["manager"].Count > 0)
                {
                    Console.WriteLine("Manager: {0}",
                        result.Properties["manager"][0].ToString());
                }

                Console.WriteLine();
            }
        }
    }
}

SSL certs @ GoDaddy are ~$13 a year...

...up to 10 years.

I finally bit the bullet and bought a real cert for my server at home... And VPN to my home network now works from Windows 7. Yay!

Israel threatens the US

"We want to reach an agreement with the United States on ways to advance the peace process," said a senior Jerusalem official. The U.S. stance, he said, "will stall the process and bring about tension and stagnation, which will hurt both Israel and the United States."

(Emphasis mine.)

http://www.haaretz.com/hasen/spages/1089205.html

Free web development tools for Microsoft platform

This article is about building websites using ASP.NET. "ASP.NET???" - you would exclaim in disbelief - "But this is 2009! Ruby! Rails! Python and PHP! Why would anyone even look at this boring last-century technology for building HR applications???"

There are several reasons why ASP.NET is worth more than a shrug in today's web development.

First, the tools are REALLY nice. Visual Studio absolutely rocks as a development environment.

Because of the Intellisense magic, most lines of code take only a few keystrokes to enter. Here is an example:

(1) 'c':



(2) '.':



(3) '.':



(4) '.(DownArrow':



(5) ');'

...and a line of code has been entered in roughly 8 keystrokes and under 5 seconds. When you type say a thousand lines of code per day, believe it or not, this makes a huge difference.

The debugger is fantastic - you can set breakpoint on the client and on the server, in C# and in JavaScript, and it works the way you'd expect it to work. In my days in Gmail the debugging experience was Firebug and even that involved an inordinate amount of prayer to my atheistic gods. Debugging in IE required making a pact with the devil. And as you know, Python has no debugger at all, because Pythonic code does not require debugging by definition :-).

Second, there is a lot to be said for a type-safe language.

In the world where most development is confined to 500-lines-of-Javascript-and-a-lot-of-JPEGs Facebook apps, the idea that every object is a hash map, and every squence of keystrokes is a variable makes perfect sense. Why spend time thinking about class hierarchies when we could be delighting the customers instead?

But suppose your app is 100000 lines of code. Standard software development principles - code isolation, compiler-time type checking, interfaces - become incredibly important.

How many huge applications have you seen that are written in dynamic language? I've worked on one - Gmail - which was around 250Kloc of JavaScript at the time. The only way doing anything there was possible was because of Google's internal JavaScript tools that did a lot of static code-checking at compile time, and because the v2 code base was built on object-oriented concepts - class libraries, inheritance, etc. It was very unusual - and very restricted - JavaScript for sure!

But even with all of these great tools, the life was very hard - I don't think Gmail team will be able to increase their code base very much further...

One huge advantage of ASP.NET is its ability to model an HTML page in C# - every HTML element is created as a .NET object on the server, and is then rendered into HTML. This allows your compiler to be very upfront about things that you would otherwise be finding through testing or user feedback!

For example, this is a code snippet that creates a very simple web page:

    Label l = new Label();
    l.Text = "<h1>Welcome!</h1>";
    ActivePage.Controls.Add(l);
    TextBox t = new TextBox();
    t.ID  = "textbox";
    ActivePage.Controls.Add(t);
    Button b = new Button();
    b.Text = "Submit";
    b.ID = "submit";
    b.Click += new EventHandler(clickProcessor);
    ActivePage.Controls.Add(b);

I think it was primarily because of the tools and the language that I was able to clone Google Mondrian in C# in less than a quarter of time it took Guido to write it in Python. (http://malevich.codeplex.com)

Alright, alright, you would say, but don't I have to pay Micro$oft thousands of dollars for the privilege? Ruby, PHP, Python, and MySQL are free, and have you checked the going rate for SQL Server recently?

This used to be true, and it was my constant mantra at Microsoft that we should ship at least the basic dev tools for free. It was always my opinion that if we did ship free C compiler with DOS and Windows, GNU tool chain - and, consequently, Linux - would not have existed.

Maybe I was not alone in thinking this, and better late than never, so starting around 2005 Microsoft began shipping free Express version of Visual Studio Suite, as well as SQL Server Express Edition. This articles is an introduction to these tools.

Without much ado, let's dig into free development with Microsoft tools.

First, you need a free OS. Windows 7 RC is free (for a year :-)) and is available here:
http://www.microsoft.com/Windows/Windows-7/download.aspx

On a modern DSL connection it takes about an hour to download, and another 30 minutes to install (including the updates) - at least on a virtual machine I used.


After the OS is installed, go here: http://www.microsoft.com/web/downloads/platform.aspx and download Microsoft Web Platform using "Installer 2.0 Beta". Make sure you choose everything under the Database tab, and "Visual Web Developer 2008 Express with SP1" under Tools.

Before the installation, it will ask you what authentication scheme to use for SQL Server. Select Windows Authentication.



During the installation, it will complain multiple times about "Known compatibility issues" between Win7 and various SQL Server components. Click "Continue" every time - but you will need to install SQL Server SP1 (later).

Once the installation is complete, go to Windows Updates, and click "Get updates for other Microsoft products". Install all default and optional updates (that are not language packs). Pay attention to the optional updates - the list should include SQL Server SP1. Make sure this gets in.

Your setup is now complete! From start to finish, on a decent DSL connection, it should take no more than 2.5 hours do download and install everything.

You can use free version of Visual Studio for building web applications, and a free version of SQL Server as a backend for them.

As an example of using these tools, we'll build a simple guest book app.

Our app will consists of a database that will store comment records in our guest book, and a web front-end that would allow visitors to enter comments (as well as their names and email addresses).

Let's create the database!

Our database will be really simple. It will contain records which will consists of the record key (an integer), a 50-character name, a 50-character email alias, and an unlimited comment string.

Run Microsoft SQL Server Management Studio (from Programs->Microsoft SQL Server 2008). Right-click on the list of the databases, and select "New Database...". Enter the name of the database (GuestBook), and click OK.


This creates an empty database. Now expand its tree in the left pane, and click on the table. Select "New table...". A windows that allows you to edit the database structure will open. DO NOT save the table until all the editing is done!

First, let's create an identity field - the name would be "Id", the type "int". Right-click on the field and chose "Set Primary Key".


Then, in the list of options below, make this an identity record (a unique key that identifies the record in the database).


Then create the other fields: Author and Email (both nvarchar(50)), and Comment ((nvarchar(MAX)) - all three of them NOT NULL.


Save the table (Control-S) and name it CommentRecord:


Now that we have the place to store data, we should create a security model for it. SQL Server has two security primitives that are of interest here - database users, and SQL Server logins. "A login" is a SQL equivalent of a Windows user. When you allow a user to connect to the database engine, you create a login for this user. In our case, we want to allow IIS user to connect to SQL database, so we create a login for "IIS APPPOOL\DefaultAppPool" by right-clicking on Security/Logins and selecting New... from the menu:


Logins are global to the database engines, and are a way to identify external users to SQL. The databases also have a concept of users, which are projections of logins on the individual databases (why SQL developers decided to have two entities instead of one, beats me). So we now have to create a user for the GuestBook database, and associate it with the login we just created. We will name the user "GuestBookUser". Expand the GuestBook database tree, Security, and right click on Users, and select New. Enter the logon name (IIS APPPOOL\DefaultAppPool), the user name (GuestBookUser), and check db_datareader role membership below, as follows:
.

Note that we have not granted the database user any "write" rights yet, but we expect it to be able to add the records. We will do it by creating a stored procedure (a piece of code inside our database) that would add records, and grant the GuestBookUser the right to execute this code. This way, any visitor to the web site would be able to add data, but not change anything that is already in the database.

Under the database tree, expand Programmability, and right click on New Stored Procedure. The nice thing about SQL Server 2008 is that its Management Studio supports Intellisense:


Enter the following stored procedure into the query window, and click on Execute:


Now the only thing left is to grant GuestBookUser the execute access to this stored procedure. Expand Stored Procedures under Programmability, right-click on its name (refresh - F5 - if it's not there), then select Properties, and then Permissions:


We're done! Once you do it once or twice, creating simple databases like this only takes 5 to 10 minutes.

We can now create the front-end application.

Execute Visual Web Developer 2008 Express Edition AS ADMINISTRATOR. You will need this privilege level to publish the web site.

From the File menu select New Website. Pick ASP.NET Web Site template, set the language to Visual C#, and put it in a directory called GuestBook:


We will be using LINQ to access the database from our code. LINQ To SQL makes a database accessible by generating a C# object model for it. It's actually extremely nice. You can get at the database data by executing SQL-like queries directly from C#, by writing something like this:

    var query = from cc in context.CommentRecords select cc;
    foreach (CommentRecord r in query)
    {
        Console.WriteLine("Name: {0} Email: {1} Comment: {2}",
            r.Author, r.Email, r.Comment);
    }

LINQ supports much more complicated queries, of course, with conditions and even joins.

To add the database model, right-click on the website name in Solution Explorer, and click on Add new item. Pick LINQ To SQL Classes, and change the name to GuestBook.dbml:


We now need to connect to our database. Click the Database Explorer, then Add Connection:


Pick Microsoft SQL Server:


Set the SQL Server instance as YOURMACHINENAME\SQLEXPRESS, and select GuestBook from the list of databases. Click on Test Connection to make sure everything works:


You can now see the database in the database explorer. Expand the nodes for tables and stored procedures, and drag the CommentRecord table and AddComment SP over to their places on the designer surface:


Save everything (Control-S). The database classes are ready for use!

We need an element on the form to use as a root for our HTML object model. Every HTML element we create will attach to this element. Open Default.aspx file, and convert the empty div element inside the form to asp:PlaceHolder. It needs to have properties runat="server" and id="ActivePage". Save!


Open the Default.aspx.cs file now. This is the code we're going to write - the whole thing is barely over 100 lines long, and almost 20 of that is generated as part of the template:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;

public partial class _Default : System.Web.UI.Page 
{
    //
    // This renders the page. Page_Load gets called both when
    // the page is first displayed, and also on post-backs.
    // The postback cycle initializes the page object model by running
    // the code below, then sets the state of the objects from user
    // input, then the handlers for the controls are called (in our case,
    // it is the Click handler for the button). The handlers may (and often
    // do redirect page to cause re-rendering with the new information.
    //
    protected void Page_Load(object sender, EventArgs e)
    {
        Label l = new Label();
        l.Text = "<h1>Welcome to my Guest Book!</h1>";
        ActivePage.Controls.Add(l);

        //
        // Open the connection to the database.
        //
        GuestBookDataContext context = new GuestBookDataContext(
            System.Web.Configuration.WebConfigurationManager.ConnectionStrings[
            "GuestBookConnectionString"].ConnectionString);

        Table t = new Table();
        ActivePage.Controls.Add(t);

        //
        // First, display all existing comments
        //
        var commentsQuery = from cc in context.CommentRecords select cc;
        foreach (CommentRecord record in commentsQuery)
        {
            TableRow row = new TableRow();
            t.Rows.Add(row);
            TableCell cell = new TableCell();
            row.Cells.Add(cell);
            cell.Text = Server.HtmlEncode(record.Author) + " says...";
            cell.Style.Add(HtmlTextWriterStyle.FontStyle, "italic");
            cell.Style.Add(HtmlTextWriterStyle.Color, "blue");
            cell.ColumnSpan = 2;

            row = new TableRow();
            t.Rows.Add(row);
            cell = new TableCell();
            row.Cells.Add(cell);
            cell.Text = Server.HtmlEncode(record.Comment);
            cell.ColumnSpan = 2;
        }

        context.Dispose();

        //
        // Now create the form for a user to enter comments.
        //
        TableRow author = new TableRow();
        t.Rows.Add(author);
        TableCell name = new TableCell();
        author.Cells.Add(name);

        Label authorNameLabel = new Label();
        authorNameLabel.Text = "Your name:";
        name.Controls.Add(authorNameLabel);

        TextBox authorNameInput = new TextBox();
        authorNameInput.ID = "name";
        authorNameInput.MaxLength = 50;
        name.Controls.Add(authorNameInput);

        TableCell email = new TableCell();
        author.Cells.Add(email);

        Label authorEmailLabel = new Label();
        authorEmailLabel.Text = "Email (will not show):";
        email.Controls.Add(authorEmailLabel);

        TextBox authorEmailInput = new TextBox();
        authorEmailInput.ID = "email";
        authorEmailInput.MaxLength = 50;
        email.Controls.Add(authorEmailInput);
        
        TableRow comment = new TableRow();
        t.Rows.Add(comment);

        TableCell commentCell = new TableCell();
        commentCell.ColumnSpan = 2;
        comment.Cells.Add(commentCell);

        TextBox commentInput = new TextBox();
        commentInput.Columns = 80;
        commentInput.Rows = 20;
        commentInput.TextMode = TextBoxMode.MultiLine;
        commentInput.ID = "comment";
        commentCell.Controls.Add(commentInput);

        TableRow submit = new TableRow();
        t.Rows.Add(submit);

        TableCell submitCell = new TableCell();
        submitCell.ColumnSpan = 2;
        submit.Cells.Add(submitCell);

        Button submitButton = new Button();
        submitButton.Click += new EventHandler(submitButton_click);
        submitButton.Text = "Submit comment.";
        submitCell.Controls.Add(submitButton);

        if (Page.IsPostBack)
        {
            TableRow error = new TableRow();
            t.Rows.Add(error);

            TableCell errorCell = new TableCell();
            errorCell.ColumnSpan = 2;
            errorCell.ID = "error";
            error.Cells.Add(errorCell);
        }
    }

    //
    // This is run when the user clicks on the Submit button
    //
    void submitButton_click(object source, EventArgs e)
    {
        string comment = ((TextBox)FindControl("comment")).Text;
        string name = ((TextBox)FindControl("name")).Text;
        string email = ((TextBox)FindControl("email")).Text;
        if (String.IsNullOrEmpty(comment) || String.IsNullOrEmpty(name) ||
            String.IsNullOrEmpty(email))
        {
            TableCell error = (TableCell)FindControl("error");
            Label l = new Label();
            l.Text = "You need all three values!";
            l.Style.Add(HtmlTextWriterStyle.Color, "red");
            error.Controls.Add(l);
            return;
        }

        GuestBookDataContext context = new GuestBookDataContext(
            System.Web.Configuration.WebConfigurationManager.ConnectionStrings[
            "GuestBookConnectionString"].ConnectionString);

        context.AddComment(name, email, comment);

        context.Dispose();

        //
        // This causes the page to be redrawn with the new comment in it.
        //
        Response.Redirect(Request.FilePath);
    }
}

When the code is done, you can run it directly from Visual Studio (F5), and all normal debugging primitives (breakpoints, stepping, etc) work as you'd expect them to work.

Let's deploy the web site to the real web server now. Create a directory under c:\inetpub called GuestBook. Right-click on the web site name in the Solution Explorer, and pick Copy Website option. In the dialog that opens, click on the connect button, select File System, and navigate to c:\inetpub\GuestBook. Select all the files on the left and click on the right arrow to copy them to the target directory:


Open IIS manager, navigate to Default Website, right-click on it, and pick Add Application. In the dialog box that opens enter GuestBook as the name, and c:\inetpub\GuestBook as a path to the application.


Now go to the Firewall Control Panel Applet, and open the port for IIS:


This is it!

Google I/O 2009

An interesting, if long, video: http://www.youtube.com/watch?v=S5aJAaGZIvk&feature=PlayList&p=41F4CEB92D80C4B7&index=0

Main theme is HTML 5. The consequences of it for Silverlight merit quite a bit of thought. Silverlight comes with far better tools, of course - the dev experience will be very hard to compete against. But HTML 5 will have the advantage of ubiquity.

The other very interesting demo - for me, at least - was Java support in Google App Engine. It's an Eclipse plug-in with a built-in deployment support, GWT (http://en.wikipedia.org/wiki/Google_Web_Toolkit) and server-side support all built-in. Not nearly as slick as ASP.NET, but definitely a step in that direction.

A minor thing that I found quite funny, actually, was that the main presenter on the conference was Vic Gundotra - a former marketing/evangelist guy from MSFT who is now a VP of Engineering at Google. The silliness of Google management strikes again - say what you want about Microsoft, but at least our engineering managers are real engineers :-).

In Korean news...

Absolutely hilarious piece...
http://www.kcna.co.jp/index-e.htm

Incidentally, I cannot help noticing that "Korean Central News Agency" has a Japanese domain suffix... This is what Wikipedia has to say about Internet in North Korea:

"North Korea's first Internet café opened in 2002 as a joint venture with South Korean internet company Hoonnet. It is connected via a line to China. Foreign visitors can link their computers to the Internet through international phone lines available in a few hotels in Pyongyang. In 2005 a new internet café opened in Pyongyang, connected not through China, but through the North Korean satellite link. Content is most likely filtered by North Korean government agencies.[3][4] In 2003 a joint venture called KCC Europe between businessman Jan Holterman in Berlin and the North Korean government brought the commercial Internet to North Korea. The connection is established through a satellite link from North Korea to servers located in Germany. This link ended the need to dial ISPs in China.[5]

KCC Europe is attempting to regulate the .kp country code top-level domain (ccTLD); as of 2008[update] its site (kcce.kp) and Naenara (naenara.kp) are the only known to be active in the .kp domain. Its IP address resolves not to Asia but to servers at Internet Provider Berlin (ipberlin.com) in the German capital."

http://en.wikipedia.org/wiki/Communications_in_North_Korea

No sex for iPhone users (via Reddit)

"Apple has rejected Eucalyptus, an ebook reader that facilitates downloading public domain books from Project Gutenberg, because some Victorian books mention sex."

http://www.boingboing.net/2009/05/22/apple-says-no-projec.html

Priceless! (from Reddit today)

Filed under Dear Jimmy...


http://www.reddit.com/r/politics/comments/8n75m/dear_jimmy_pic/

And the first comment:
"Dear Jamail, I hope all is well in Guantanamo, though we all know better. We're very proud that you've not lied while being tortured. It could be a disaster if you give them a reason to attack more innocent people in order to make the suffering stop.

Dad was arrested because our neighbor covets our farmland, even though it hasn't seen water in 6 months. Your little brother decided to join the Baghdad police, but became a militant when he discovered that he could make more money from bribes from the soldiers. He lost his leg when an IED went off. Your sister was raped by mercenaries and then killed to cover it up. I tried to appeal to the government, but apparently they're not subject to our laws. There's still so much death. It's hard to stay hopeful.

There's an American soldier next to me reading a note complaining about seatbelts. I wonder if they even know why they're here."

http://www.reddit.com/r/politics/comments/8n75m/dear_jimmy_pic/c09th7o

Why you should always buy memory separately

Dell, Latitude E6500, (through large enterprise portal, today): 4GB is $135


Newegg: 4GB is <$50

US health care by the numbers

• Per-capita US health care expenditures in 2002: $5,267 ("Health Care Politics and Policy in America", Patel & Rushefsky, 2006)


• Percentage by which it is higher than OECD average: 140%


• Per-capita expenditures in Germany, France, Canada, and Great Britain: $2,817, $2,736, $2,931, and $2,160.


• Percentage of health care costs that are administrative: 31%. (http://www.pnhp.org/publications/nejmadmin.pdf)


• The same in Canada: 16.7%


• Percentage of the total US health care costs last year that went to pay salary of UnitedHealth CEO: 0.14% (http://wonkroom.thinkprogress.org/2009/05/21/elizabeth-edwards-1-of-every-700-went-to-pay-salary-of-unitedhealth-ceo/)


• US health care ranking in the world, according to WHO, in 2000: 37 (http://www.photius.com/rankings/healthranks.html)


• French health care ranking: 1


• Number of developed countries ranked lower than US: 0


• US population longevity ranking in the world: 33 (tied with Cuba, http://www.usatoday.com/news/health/2007-05-18-un-life-expectancy_N.htm)


• US rank in infant mortality: 29 (tied with Slovakia and Poland, http://www.webmd.com/parenting/baby/news/20081015/infant-mortality-us-ranks-29th)


• Number of bankruptcies in US that are medical: 50% (http://www.pbs.org/moyers/journal/05222009/transcript4.html)


• The share of those where people had health insurance: 3/4

Giving back capital

Constantly in the news here is the push from the banks to return government investments - the TARP funds - to escape from the strings that were attached to that money - specifically, the limits on executive compensation.

This is being discussed as if it were the most natural thing in the world - of course bankers want to be paid more! The question of how returning cheap capital benefits the shareholders (obviously, it does not) - ostensibly, the owners of these enterpirses - is never even mentioned.

It's a really weird concept of capitalism that we have here!

http://1-800-magic.blogspot.com/2009/04/capitalism-socialism.html

Breaking change in Malevich

This will read as a daily WTF, unfortunately... but after 6 months of development, I found a really, really nasty bug in Malevich, and if you're maintaining an installation, you should read this.

Interestingly, the bug was in my face all this time, I just wasn't paying attention.

Malevich stores a bunch of timestamps in its database - one for change list itself, one for each review iteration, and one for each file version. Normally these are stored in UTC - after all, how else would you store times in a web application where users are scattered around the globe?

Well, as it turned out, all but not all. Change list time stamp was stored as local time. Of course the bug was on display on the dashboard all this time, literally in my face. But I think this was because all the rest of the dates were in UTC and therefore not very useful - I just taught myself to ignore all time stamps. So I never noticed that unlike all the rest, these ones looked correct - and they should not have!

Anyway, for quite a while Malevich fielded complaints that the time stamps are silly, and I was thinking on and off on how I might fix that. You see, .NET runtime which Malevich is built upon is great at manipulating the time, except for one thing - the web site has no idea what user's time zone is - this is by design, a privacy issue.

The client side does know what the local time offset is, so the only way to transform time correctly must lay with the client.

So this is what Malevich does - each time stamp is wrapped in a <span id='timestamp' name='timestamp> element. When the page loads, I get all the elements by the name 'timestamp', and convert them to local time.

The conversion is pretty crappy from the globalization perspective (heck, but this is *free* software :-)) - because default JavaScript time conversion functions are terrible - they produce very long strings (for example, days of the week are spelled out by some browsers).

So I compose the date/time string myself, in US format. The only thing where I try to be smart is determine if the format is 12- or 24-hours upfront and then generate the date that would use the same format. Which does not work on Chrome because it uses 24-hour format no matter what.

//-----------------------------------------------------------------------
// 
// Copyright (C) Sergey Solyanik for The Malevich Project.
//
// This file is subject to the terms and conditions of the Microsoft
// Public License (MS-PL).
// See http://www.microsoft.com/opensource/licenses.mspx#Ms-PL for
// more details.
// 
//----------------------------------------------------------------------- 
//
// Javascript module to recalculate times to browser local.
//

//
// Is time a 24-hour format or AM/PM?
//
var TIME_FORMAT_24HR = ComputeDateFormat();

//
// Recomputes all date times from UTC to local.
//
function RebuildDates()
{
    var dates = document.getElementsByName('timestamp');
    for (var i = 0; i < dates.length; ++i)
    {
        var date = new Date(dates[i].firstChild.nodeValue);

        var month = date.getMonth() + 1
        var year = date.getYear()
        var day = date.getDate()

        if (day < 10) day = '0' + day
        if (month < 10) month = '0' + month
        if (year < 1000) year += 1900

        var hour = date.getHours();
        var minute = date.getMinutes();
        var seconds = date.getSeconds();

        var suffix = '';
        if (TIME_FORMAT_24HR)
        {
            if (hour < 10) hour = '0' + hour;
        }
        else
        {
            if (hour < 12) suffix = ' AM'; else suffix = ' PM';
            if (hour == 0) hour = 12;
            if (hour > 12) hour -= 12;
            if (hour < 10) hour = '  ' + hour;
        }
        
        if (minute < 10) minute = '0' + minute;
        if (seconds < 10) seconds = '0' + seconds;

        dates[i].firstChild.nodeValue =
            [month, '/', day, '/', year, ' ', hour, ':', minute,
            ':', seconds, suffix].join('');
    }
}

//
// Checks if the date in the current locale is AM/PM or 24 hours.
//
function ComputeDateFormat()
{
    var dateString = (new Date()).toLocaleTimeString();
    if (dateString.indexOf(' AM') == -1 && dateString.indexOf(' PM') == -1)
        return true;
    return false;
}

//
// Makes RebuildDates be called on page load.
//
if (window.attachEvent)
{
    window.attachEvent('onload', RebuildDates);
}
else
{
    if (window.onload)
    {
        var curronload = window.onload;
        window.onload = function()
        {
            curronload();
            RebuildDates();
        };
    }
    else
    {
        window.onload = RebuildDates;
    }
}

Anyway, so I make this change, and now I start looking at dates very closely, and - crrrrap! - all the time stamps now look great, except the most important one - the change list time stamp!

And the worst part of course is that it's in the database! Moreover, it's data! And it's now on a few dozen servers at Microsoft alone, and I have no idea how to even find them.

Anyway, I put notices on the web site (http://malevich.codeplex.com/Wiki/View.aspx?title=special%20note%20on%20Malevich%20upgrades, and on this blog (you're reading it), and in the change list description, and on Microsoft's internal share point site for Malevich.

I modify the installer so that it detects this condition (or, rather, the right version transition), and tries to fix up the problem:

USE [CodeReview]

DECLARE @UTCDate datetime
DECLARE @LocalDate datetime
DECLARE @TimeDiff int 
  
SET @UTCDate = GETUTCDATE()
SET @LocalDate = GETDATE()
SET @TimeDiff = DATEDIFF(hh, @LocalDate, @UTCDate) 

UPDATE [dbo].[ChangeList] SET TimeStamp = DATEADD(hh, @TimeDiff, TimeStamp)

What else can I do?

Of course, the script above does not work completely, either, because it does not account for the daylight savings time (which is really, really hard to compute in T-SQL - I love human time system!), and because it assumes that local time is the same as server time, which is of course incorrect, because a bunch of teams use the tool between here and China... But it does make the time somewhat more right than it was before.

So this is the story of the worst bug in Malevich in the 6 months since it was born.

And the moral of the story... what is the moral of this story? I wrote it primarily to attract the attention of Malevich admins to the fact that they need to be thoughtful about their next upgrade. But if you have a suggestion for a witty ending, offer it in the comments section! :-)

Why do Catholic clergy not oppose gay marriage in NY?

"The state’s Roman Catholic bishops have been somewhat distracted, too, having focused their lobbying energies this session on defeating a bill that would extend the statute of limitations for victims of sexual abuse to bring civil claims, and have appeared unprepared for the battle over marriage."

http://www.nytimes.com/2009/05/20/nyregion/20marriage.html?hp

Wow! Just... wow!

Serious play (TED)

Fair warning: allocate 30 minutes upfront. You won't be able to stop once this starts!

Going-away party

"Green was convicted last week in U.S. District Court in Kentucky of murder, rape, conspiracy and obstruction of justice in connection with a 2006 rape-and-murder south of Baghdad. A jury found him guilty of raping a 14-year-old girl, then killing her and setting her body on fire to destroy evidence. Green also was found guilty of killing the girl's parents and 6-year-old sister.
...
Ruth, who is John Green's sister, noted for the jury that Green's mother is not at the trial this week. The mother is moving and had to attend a going-away party, Ruth said."

http://www.cnn.com/2009/CRIME/05/18/kentucky.iraq.murder/index.html

You know that things are hitting the bottom...

...when even New York Times can not spell...

Telescopes can see... photons? (via Reddit).

http://government.zdnet.com/?p=4765

"Thursday, the Europeans will launch two much more advanced telescopes (right) that can see photons, even those reaching back to the Big Bang. Hubble sees visual light."

I kid you not.

This is a "technical" publication...

Where are you, Robin Crusoe? Where are you? Where have you been?

Stretched on $500k/year

"Nor does he appreciate being branded as "rich" when it's far from certain he'll ever build the kind of lavish nest egg the truly wealthy enjoy, especially after the current market meltdown."

"Kelly Lynch, the owner of a commercial maintenance company in Redondo Beach, Calif., is raising two kids with her partner, Jill Fenske, on a household income of $400,000. She's saving $800 a month for the children's college fund and $4,000 a month for retirement - a number that someday might make her rich. "If I blew my money like other people, I'd feel rich," says Lynch. Her views on taxes are befitting a born entrepreneur: "I think it would be unfair if someone tried to raise my taxes," says Lynch. "I don't think people should be penalized because they earn more.""

"Even at the upper end of the HENRY group, our cover subjects, Lindsay Mayer and her husband, Zach, a Dallas attorney, feel stretched on $500,000 a year."

"Tony Molino, 50, an attorney in Rancho Palos Verdes, Calif., speaks for legions of HENRYs: "I've worked 50 to 60 hours my entire life, and I don't have a lot left over at the end of the month. I'm comfortable, but when Joe Biden talks about sucking it up, getting patriotic, and paying more taxes, I get livid.""

http://money.cnn.com/2008/10/24/magazines/fortune/tully_henrys.fortune/index.htm

Poor people, not sure if they will ever be TRULY rich.



Here's a book that shows what really means to work hard: http://www.amazon.com/Nickel-Dimed-Not-Getting-America/dp/0805063897. And it is not sitting at the desk for 2 hours more a day. It's having 3 jobs as a janitor, a waitress, a maid, and despite working 14 hours a day not having enough money to rent an apartment or go to a doctor...

To Joe the Plumbers among us...

...who vote for lower taxes for the rich because some day they expect to be rich themselves: why not buy TWO lottery tickets instead of one? The extra money from the second win will take care of the taxes!

http://www.americanprogress.org/issues/2006/04/b1579981.html

"By international standards, the United States has an unusually low level of intergenerational mobility: our parents’ income is highly predictive of our incomes as adults. Intergenerational mobility in the United States is lower than in France, Germany, Sweden, Canada, Finland, Norway and Denmark. Among high-income countries for which comparable estimates are available, only the United Kingdom had a lower rate of mobility than the United States."

Why economic and social liberalism trends are incompatible

US has two dominant parties - the economically liberal but socially conservative Republicans, and socially liberal and economically conservative Democrats.

Many in my circle - the professional class - were long yearning for the third alternative - a party that is liberal both economically and socially. Someone to represent me :-)!

Why is this not happening? Unfortunately, for a reason.

Unrestricted economic liberalism mostly benefits the upper classes.

The lower classes are less educated, more religious (church and especially Christianity historically being used as a way to keep people in their place), and therefore more socially conservative.

Since the upper classes do not have enough votes to advance their agenda, they need to coopt the lower classes to vote against the lower classes' economic interests.

This of course is done by throwing down whatever red meat issues are currently at hand - "family values", gay marriage, "security", "morality", "gun rights", etc. Most of it is either social conservatism or fear because this is what the masses respond to.

And this is why economically liberal parties are bound to be socially conservative. There is simply no way for them to attract enough followers to matter otherwise.

Which is why you have such a high incidents of hypocrisy among Republican leaders - from Rush Limbaugh abusing drugs while preaching death to (other) drug addicts, to Larry Craig's restroom incident. They don't really give a flying f*ck about morals (or anything else they sell - gun rights, security, etc). They are just using this to get more idiots vote for them.

What does this say about the rich? Not much, unfortunately (I cannot bring myself to call Wall Street bankers and automaker CEOs "the elites" which to me means "the best"). By trying to extract the last penny while stirring the worst instincts in the society, they retard the progress of civilization.

Yes, they are rich, but as a result time moves slower for everybody. But you know what? The quality of life and the average longevity of the middle class in the age of antibiotics turns out to be better than that of the royalty in the Middle Ages.

Trading stem cell research for a few thousand bucks in taxes is not smart, and proves once again that IQ and compensation in today's America are unfortunately not correlated.

And this is usually the beginning of the end...

Why is it socially acceptable to be a conservative?

You generally don't see many people go around and say "I am a fascist!" or "I am a racist!". And yet there are plenty of people who proudly declare themselves to be conservative.

Let's take a brief look at the history.

Spanish Inquisition's goal was to suppress heresy - or, in other words, preserve traditional religious beliefs. It was a conservative movement that resulted in executions of thousands of people.

US itself was founded by progressives - conservatives were known as Loyalists at the time.

Over the last couple hundred years conservatives, standing athwart history (in the words of William Buckley), have opposed most of the civilization advances that we now take for granted:

• emancipation of slaves
• women's suffrage
• 8 hour workday
• limitations on child labour
• desegregation and racial equality
• Social Security
• Medicare

And you don't have to go too far back in time - it is probably safe to say that in 1960s most conservatives supported segregation, and that in 1920s most of them were against women's suffrage. You just cannot say the same about liberals.


Today, history is repeating itself with conservative attitudes towards everything from torture and gay rights to healthcare reform.

So why is the conservative brand still acceptable in the mainstream society after all of this? What am I missing?

The future is now!

Two links:

http://1-800-magic.blogspot.com/2007/12/windows-8-or-9-or-10.html

...and...

http://www.withinwindows.com/2009/04/24/secret-no-more-revealing-windows-xp-mode-for-windows-7/

A whole version earlier than I predicted.

Israel to US: shut up and do what I say (via Reddit)

"The Obama Administration will put forth new peace initiatives only if Israel wants it to, said Foreign Minister Avigdor Lieberman in his first comprehensive interview on foreign policy since taking office.

"Believe me, America accepts all our decisions," Lieberman told the Russian daily Moskovskiy Komosolets."

http://www.haaretz.com/hasen/spages/1080097.html

Dell Latitude: 8 hours on a single charge!

Tonight (by accident) I left my laptop unplugged and in a state where it could not go to sleep automatically. In the morning I found the laptop with its battery completely depleted, but still running!

Kudos to power management systems in Core 2 Duo and Windows Server 2008!

Laissez-faire (via Reddit)

"In Bucharest, you can buy a young girl for 8,000 euros. I mean buy them as in you own them forever or until you sell them to someone else. Most of them are sold by their parents in Moldova when they are little children. As soon as they’re bought, the owner brands them. This is normally a tattoo of his name."

http://www.viceland.com/int/v13n9/htdocs/slavery1.php?country=uk

While this sounds (reads?) insane, it is not really that unusual for an unregulated market to produce this phenomenon. It was not all this different in US in 1890s, and in Russia in 1990s - just the percentage of population is smaller.

Do laws have priority over money, or does money have priority over laws?

The Lifestyle

"It is a tricky situation in which some Americans find themselves after a long boom: They are by no means struggling, compared with the 98% of Americans who make far less, but depending on where they live and the lifestyle choices they have made, they don't necessarily feel rich, either."

http://finance.yahoo.com/focus-retirement/article/106934/Wealth-Less-Effect-Earning-Well-Feeling-Otherwise

Oh, yes, the life style choices... Take me, for instance. My personal choice, as anyone who knows me well will confirm, is to fly my own 747. I love to travel, and the fact that I have to wait in airports, sleep in a chair on long flights, and stand in long lines is simply unacceptable. It makes me feel positively broke. Middle class you say? As far as I am concerned, I am a pauper.

Now, Obama thinks that we are rich and wants to tax our hard earned money (that is, the amount above $250k) at 3% more. Which means that if we were to make, say, $350k next year, we'd be forking over $3000 more to the government in taxes. Instead of buying one of these: http://1-800-magic.blogspot.com/2009/04/computer-porn.html, which my chosen lifestyle dictates that I have. Talk about socialism!

Malevich has setup!

I've been working on Malevich - my code review system - on and off for almost 5 months now.

It turned out to be very successful - my team was using it for all of our first milestone, and was very happy about it (we're now pushing 1500 change lists that went through the system). Many other Microsoft teams adopted it since, to the tune of at least a couple hundred people - and these are just the people who contacted me.

The biggest problem so far was the setup. The deployment was hard - it required Visual Studio, a service pack, an expansion pack, and tons of manual work. A lot of errors were made, and in many cases I had to drop by and set things right afterwards.

Not any more! The first official build is out, and it has a real setup! The setup checks the dependencies, deploys the database, publishes the web site, and configures notifier automatically, no Visual Studio is required!

So if you wanted to try Malevich but were intimidated by the complexity of installation - the wait is over. You still do have to install Server 2008 with IIS and SQL Server 2008, but the rest is done for you.

Just get the MSI here: http://malevich.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=26374

Happy code reviewing!

How is this not treason? (via Reddit)

"Rep. Jane Harman , the California Democrat with a longtime involvement in intelligence issues, was overheard on an NSA wiretap telling a suspected Israeli agent that she would lobby the Justice Department reduce espionage-related charges against two officials of the American Israeli Public Affairs Committee, the most powerful pro-Israel organization in Washington."

...

"In exchange for Harman’s help, the sources said, the suspected Israeli agent pledged to help lobby Nancy Pelosi, D-Calif., then-House minority leader, to appoint Harman chair of the Intelligence Committee after the 2006 elections, which the Democrats were heavily favored to win."

http://www.cqpolitics.com/wmspage.cfm?docID=hsnews-000003098436

The new government

"As far as the most important matter is concerned, there is complete unanimity. Liberman, Netanyahu, Barak, Ellie Yishai of Shas and Danny Hershkovitz of the “Jewish Home” party are in total agreement about the Palestinians. All of them agree on the need to prevent the establishment of a real Palestinian state. All of them agree not to talk with Hamas. All of them support the settlement enterprise. During Barak’s stint as Prime Minister, the settlements grew even faster than during Netanyahu’s tenure. Liberman is himself a settler, Hershkovitz’s party represents the settlers. All of them believe that there is no need for peace, that peace is bad for us. (After all, it was Barak, not Netanyahu or Liberman, who coined the phrase “We Have No Partner for Peace”.)"

http://zope.gush-shalom.org/home/en/channels/avnery/1238277190

Computer porn

2x	Intel Xeon E5410 Harpertown 2.33GHz 12MB L2 Cache LGA 771 80W Quad-Core Processor	http://www.newegg.com/Product/Product.aspx?Item=N82E16819117150	$590
1x	SUPERMICRO MBD-X7DCL-I Dual LGA 771 Intel 5100 ATX Server Motherboard	http://www.newegg.com/Product/Product.aspx?Item=N82E16813182146	$300
6x	Kingston 4GB 240-Pin DDR2 SDRAM ECC Registered DDR2 667 (PC2 5300) Server Memory	http://www.newegg.com/Product/Product.aspx?Item=N82E16820134835	$504
5x	Western Digital VelociRaptor WD1500HLFS 150GB 10000 RPM 16MB Cache SATA 3.0Gb/s 3.5" Hard Drive	http://www.newegg.com/Product/Product.aspx?Item=N82E16822136296	$900
1x	Athena Power AP-P4ATX85FE 20+4Pin 850W Single ATX12V & EPS12V Power Supply	http://www.newegg.com/Product/Product.aspx?Item=N82E16817104126	$155
1x	SILVERSTONE KUBLAI Series KL03-B Black 2.5mm aluminum front door, 0.8mm SECC body ATX Full Tower Computer Case	http://www.newegg.com/Product/Product.aspx?Item=N82E16811163100	$155

...yields an 8-core, 24GB monster capable of hosting 10 VMs comfortably for $2600.

Capitalism, socialism

Wikipedia defines capitalism thus:

"Capitalism is an economic system in which wealth, and the means of producing wealth, are privately owned and controlled rather than state owned and controlled. Through capitalism, the land, labor, and capital are owned, operated, and traded by private individuals either singly or jointly, and investments, distribution, income, production, pricing and supply of goods, commodities and services are determined by voluntary private decision in a market economy. A distinguishing feature of capitalism is that each person owns his or her own labor and therefore is allowed to sell the use of it to employers." (http://en.wikipedia.org/wiki/Capitalism).

The definition puts "ownership" and "control" of the capital in the same sentence, but does not stipulate that it is the same individual that both owns and controls the resources.

And therein lies our problem.

Commonly accepted theory of why capitalism is preferable to socialism goes like this: the owners generally make better decisions about allocating the resources than non-owners. When the owners do make sub-optimal decision, they lose their ownership stake to the smarter and more responsible new owners.

So far, so good. Except this is not at all the system that we have in the United States. In the US, the majority of the capital is owned by one set of people - the shareholders - and managed by an entirely different set of people - the corporate management.

The interests of these groups of people are not at all aligned (http://1-800-magic.blogspot.com/2007/12/risk-vs-reward.html), and, what's worse, neither group is truly interested in the long-term success of the business.

As a result, while the executives in younger companies which are still managed by the original owners do often tend to do the right thing, I cannot call the behavior of the managerial class in most American enterprises anything other than looting.

This approach to management has been most visible on Wall Street, but look at any older US enterprise, and you will see the same looting of company resources everywhere, from car manufacturers to steel mills to utility companies.

Comparing to the old Soviet days, the system as we have it right now may actually be worse. At least the bureaucrats in Soviet Union were not paying themselves multimillion dollar bonuses and did not zip around in private planes as the economy crumbled.

Discounting risk

I wrote this article in December 2007 and it describes perfectly the situation we're in now: http://1-800-magic.blogspot.com/2007/12/risk-vs-reward.html

If you had only two news articles to read this week (via Reddit)...

...I would recommend "Ten principles for a Black Swan-proof world" by Nicolas Taleb (best known for his book "The Black Swan: The Impact of the Highly Improbable" about underpricing risk) and The dark side of Dubai. This last one is absolutely surreal. Make sure you read it at least through "The Lifestyle" chapter. It is really, truly, absolutely unbelievable.

We need this guy back!

http://images.pcworld.com/news/graphics/162166-billgatesFINAL2_original.jpg

Visual Studio SP1 installer blues

I have but one question - what is taking it so long?

It takes 1.5 hours on my 2.5GHz, 4GB Core 2 Duo laptop running Windows Server 2008.

In 1.5 hours one could, at a typical speed of my laptop:
- Write 263GB of data to disk, at 50MBps
- Transfer 131GB of data over the network, at 200Mbps
- Copy 15 TB (!) of memory (on 800Mhz bus)
- Execute a whopping 26 trillion instructions (2 cores retiring 2 instructions per core per clock).

WHAT DOES THIS THING DO???

Weather plays April Fool's in Seattle

It's April 1st, and it's SNOWING! Here's what it looks like from our window.

The Lord of the Rings and Atlas Shrugged (via Reddit)

"There are two novels that can change a bookish fourteen-year old’s life: The Lord of the Rings and Atlas Shrugged. One is a childish fantasy that often engenders a lifelong obsession with its unbelievable heroes, leading to an emotionally stunted, socially crippled adulthood, unable to deal with the real world. The other, of course, involves orcs."

http://www.amptoons.com/blog/archives/2009/03/29/quote-8/

Justice and accountability, the American way (via Reddit)

1. The hospital makes a medical error; patient dies.
2. The hospital falsifies the medical record, including the testimony of the doctor given during the investigation.
3. The doctor who gave the testimony produces the original copy.
4. The hospital settles out of the court. The article does not say this, but I bet that the people who were involved in hiding the evidence probably still work for that hospital.

“There isn’t going to be a trial,” he said. “The hospital offered the patient’s family an 8-figure settlement, and they have accepted.”

http://open.salon.com/blog/amytuteurmd/2009/03/30/they_killed_my_patient_then_they_tried_to_hide_it

Malevich hits 1000th code review!

We've been using it for 3 months now in our team of roughly 45 people - 20 devs and 25 testers. This week we've had 1000th code review completed.

Here are a few stats, as of right now:

1008 change lists
17081 file versions
2863 review iterations
7851 comments

If you work at Microsoft and would like to try Malevich, I've set up a test change list here: http://sergeysomsg2/Malevich/default.aspx?cid=299. You can add yourself as a reviewer, leave comments in files, and submit review iterations.

If you're outside Microsoft and want to try it, the source and installation instructions are here: http://www.codeplex.com/Malevich.

Welcome to the ultimate in banana republics! (via Reddit)

Written by a former chief economist at IMF.

"Typically, these countries are in a desperate economic situation for one simple reason—the powerful elites within them overreached in good times and took too many risks. Emerging-market governments and their private-sector allies commonly form a tight-knit—and, most of the time, genteel—oligarchy, running the country rather like a profit-seeking company in which they are the controlling shareholders. When a country like Indonesia or South Korea or Russia grows, so do the ambitions of its captains of industry. As masters of their mini-universe, these people make some investments that clearly benefit the broader economy, but they also start making bigger and riskier bets. They reckon — correctly, in most cases — that their political connections will allow them to push onto the government any substantial problems that arise."

(emphasis mine)

http://www.theatlantic.com/doc/print/200905/imf-advice

Empire strikes forward

It seems that vast majority of Star Wars technology was designed and built by the marketing department. Here are a few engineering suggestions that could help the bad guys win in Epsiodes -3 to 0...

1) Use firearms. Blasters are slow, not very powerful, and easily deflected by a light saber. An AK-47 would be far, far more effective. And don't forget the training! Despite the advantage in fire power, the storm troopers clearly can't shoot straight. Take them out to a range once in a while, and see how much more effective they become only after a couple training sessions!

2) If you use droids to fight your wars, build the best kind - do not waste time and money on anything else. It looks like the Destroyer model is more effective than50 units of the other kind (and even more effective than a Jedi). I doubt it could be more than 3 times more expensive. Why not just build Destroyers?

3) And while we are on this subject, let's have droids use more effective communication channels than human speech when they talk to each other. The same goes for all interfaces between the droids and everything else - spaceships, vehicles, and weapons. There is no reason while a droid should press a button to fire a weapon - a wireless interface would control it far more efficiently. Also, do invest in computerized targeting for your weapons. If your droids can already see, making them very accurate shooters is a relatively trivial task - ask any engineer.

4) Missiles seem to be an extremely effective weapon in Star Wars, except that there are very few of them. Why not build more of them and launch, say, 20 or 30 at a time? I doubt the Jedi, who have visible trouble dealing with one or two, would be able to escape ten. Also, consider building missiles that are FASTER than the star fighters they attack.

5) Consider using encryption. It looks like anyone can plug into the system anywhere and take control of everything in the space ships. This problem has been solved years ago! It really is not that hard.

6) Pack animals are slow, can't carry much, and are very, very hard to maintain. They need food, living compartments, etc. Consider eliminating them from the army in favor of mechanized transporters. Animals are not very effective in executions either - this has been proven many, many times. Instead of staging elaborate shows with giant predators that always fail to kill the prisoners, just shoot your captives, and put their heads on a stick.

7) I cannot overemphasize the importance of conventional firearms. If the probe sent to assassinate Princess Amidala had used a regular rifle instead of the poisonous centipedes, the subsequent events might have taken a very different turn. And in space, consider using nuclear weapons. The laser guns you currenlty mount on your ships are massively underpowered.

8) Did you know that space ships do not need wings to fly in space? Once you get entirely comfortable with this (yes, I do know it's extremely hard, given that they even make fying noises while moving through vacuum, but making this leap of faith might be crucial to your survival - do it!) you can start using very different design paradigms - like, for example, minimizing the surface area so your spacecraft is easier to protect and harder to target.

9) On the subject of spaceship designs - there must be something in the way you build them that makes them explode after they are hit. Unfortunately, this applies to all other vehicles as well. While this generates impressive visuals, this design point leads to unnecessary casualties among your troops. Consider enclosing whatever it is that explores in extra layers of protection. You have already solved this for your small weapons, which do not seem to have the same problem. Why not use similar design everywhere?

10) This is more of a tactical rather than an engineering advice, but I will give it anyway. Instead of deploying the Walkers and other weird ground assault vehicles, consider attacking from the air to suppress the adversary's ground troops.

Follow these simple steps, and you will be invincible - at least as long as the Good Guys(TM) are controlled by the marketing people.

Free as in beer!

The GNU connotation of software as a form of speech never made any sense to me.

"Free speech" means the freedom of a person to express himself or herself without a fear of persecution. It does not mean the freedom to use the results of this expression by everyone.

Quite the contrary - the literary works by the greatest writers are the results of exercising free speech. But rip "Harry Potter", and you will - deservedly - find yourself slapped with a lawsuit, a fine, and maybe a jail sentence.

The free software movement has produced a lot of great things - from GNU development tools to Linux, BSD, Apache, and Firefox. It has also produced legions of activists that bring in the level of politics, acrimony, and fanboyism that is unbecoming to honorable competition in an area that is as much a science as it is an art.

By doing so they actually take the freedom away from people who chose to distribute the results of their work through channels that are different from the ones favored by the Free Software Foundation.

So here's my variant of "free software". The code distributed on this blog is free - free as in beer. You can redistribute it in full or in parts, with or without source code. You can use it in any derivative work without giving me any credit.

//-----------------------------------------------------------------------
// <copyright>
// Copyright (C) Sergey Solyanik.
//
// This software is in public domain and is "free as in beer". It can be
// redistributed in full or in parts for free and without any preconditions.
// </copyright>
//----------------------------------------------------------------------- 

De-gnoming your blog

My blog has been filling up with spam over the last few months - the nasty, long messages that pollute the comment space and are a pain to delete because the bastards were programmatically sticking them in tens and tens of posts.

Luckily, Google has APIs to Blogger, and better yet, they even have C# libraries than make coding against these API easy. I didn't even have to dust off Eclipse!

So an hour later I had the program that allowed me to clean up the spam quickly.

The code sucks in the entire blog (including the abstracts of posts and comments) and allows one to quickly search and delete spam. Type 'help' on the command prompt, and it will explain the usage.

A few convenient shortcuts:

List or delete all comments from someone:
    list|delete comments by name
e.g.
    list comments by peter.w

List or delete all comments that have the same author as well as the text:

    list|delete comments like sample_comment_uri
e.g.
    list comments like http://www.blogger.com/feeds/3554166144204741789/3010398536200323405/comments/default/4945114575786176865

List or delete all comments with a phrase in title:
    list|delete comments with text_string
e.g.
    list comments with wow gold

List comments in a blog or a message:
    list comments in blog_name|first_words_of_post_title
e.g.
    list comments in Back to Microsoft
    list comments in 1-800-MAGIC

If you're one of the truly insane types that would run code compiled by a random guy from the Internet, the built version is here: http://www.solyanik.com/drop/BlogDeSpammer.zip. Otherwise, the code is below, free as in beer!

Incidentally, it might make a good primer on how to retrieve and manipulate Blogger posts and comments programmatically.

//-----------------------------------------------------------------------
// <copyright>
// Copyright (C) Sergey Solyanik.
//
// This software is in public domain and is "free as in beer". It can be
// redistributed in full or in parts for free and without any preconditions.
// </copyright>
//----------------------------------------------------------------------- 
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Google.GData.Client;

namespace BlogDeSpammer
{
    /// <summary>
    /// Comment class. Just a trivial holder of essential comment data.
    /// </summary>
    class Comment
    {
        public string Text;
        public string Author;
        public string EditUri;

        public Comment(string text, string author, string editUri)
        {
            Text = text;
            Author = author;
            EditUri = editUri;
        }
    }

    /// <summary>
    /// Post class. Just a trivial holder of essential post data.
    /// </summary>
    class Post
    {
        public string Title;
        public string CommentsFeed;
        public ICollection<Comment> Comments;

        public Post(string title, string commentsFeed)
        {
            Title = title;
            CommentsFeed = commentsFeed;
            Comments = null;
        }
    }

    /// <summary>
    /// Blog class. Just a trivial holder of essential blog data.
    /// </summary>
    class Blog
    {
        public string Name;
        public string Feed;
        public ICollection<Post> Posts;
        public Blog(string name, string feed)
        {
            Name = name;
            Feed = feed;
            Posts = null;
        }
    }

    /// <summary>
    /// Main functionality.
    /// </summary>
    class Program
    {
        /// <summary>
        /// Gets collection of blogs belonging to the user.
        /// </summary>
        /// <param name="blogger"> Blogger service. Must be initialized with the parameters.
        /// </param>
        /// <returns> Collection of blogs. </returns>
        private static ICollection<Blog> GetBlogs(Service blogger)
        {
            List<Blog> blogs = new List<Blog>();

            FeedQuery query = new FeedQuery();
            query.Uri = new Uri("http://www.blogger.com/feeds/default/blogs");
            AtomFeed results = blogger.Query(query);
            while (results != null && results.Entries.Count > 0)
            {
                foreach (AtomEntry entry in results.Entries)
                {
                    string blogFeedLink = null;
                    foreach (AtomLink link in entry.Links)
                    {
                        if (BaseNameTable.ServiceFeed.Equals(link.Rel))
                        {
                            blogFeedLink = link.HRef.ToString();
                            break;
                        }
                    }
                    blogs.Add(new Blog(entry.Title.Text, blogFeedLink));
                }

                if (results.NextChunk == null)
                    break;

                query.Uri = new Uri(results.NextChunk);
                results = blogger.Query(query);
            }

            return blogs;
        }

        /// <summary>
        /// Retrieves all the posts for the blog.
        /// </summary>
        /// <param name="blogger"></param>
        /// <param name="blog"></param>
        /// <returns> Collection of posts. </returns>
        private static ICollection<Post> GetPosts(Service blogger, Blog blog)
        {
            List<Post> posts = new List<Post>();

            FeedQuery query = new FeedQuery();
            query.Uri = new Uri(blog.Feed);
            AtomFeed results = blogger.Query(query);
            while (results != null && results.Entries.Count > 0)
            {
                foreach (AtomEntry entry in results.Entries)
                {
                    string commentsFeedLink = null;
                    foreach (AtomLink link in entry.Links)
                    {
                        if ("replies".Equals(link.Rel) &&
                            "application/atom+xml".Equals(link.Type))
                        {
                            commentsFeedLink = link.HRef.ToString();
                            break;
                        }
                    }
                    Console.Write(".");
                    if (commentsFeedLink != null)
                        posts.Add(new Post(entry.Title.Text, commentsFeedLink));
                }

                if (results.NextChunk == null)
                    break;

                query.Uri = new Uri(results.NextChunk);
                results = blogger.Query(query);
            }

            return posts;
        }

        /// <summary>
        /// Gets comments for the given post.
        /// </summary>
        /// <param name="blogger"> Blogger service, initialized with credentials. </param>
        /// <param name="post"> The post. </param>
        /// <returns> Collection of credentials. </returns>
        private static ICollection<Comment> GetComments(Service blogger, Post post)
        {
            List<Comment> comments = new List<Comment>();

            FeedQuery query = new FeedQuery();
            query.Uri = new Uri(post.CommentsFeed);
            AtomFeed results = blogger.Query(query);
            while (results != null && results.Entries.Count > 0)
            {
                foreach (AtomEntry entry in results.Entries)
                {
                    Console.Write(".");
                    comments.Add(new Comment(entry.Title.Text, entry.Authors[0].Name,
                        entry.EditUri.ToString()));
                }

                if (results.NextChunk == null)
                    break;

                query.Uri = new Uri(results.NextChunk);
                results = blogger.Query(query);
            }
            return comments;
        }

        /// <summary>
        /// Main. Does all the work.
        /// </summary>
        /// <param name="args"> Command line parameters. </param>
        static void Main(string[] args)
        {
            if (args.Length != 2)
            {
                Console.WriteLine("Usage: blogdespammer username password");
                return;
            }

            string username = args[0];
            string password = args[1];

            Console.Write("Loading");

            Service blogger = new Service("blogger", "BlogDeSpammer");
            blogger.Credentials = new GDataCredentials(username, password);

            ICollection<Blog> blogs = GetBlogs(blogger);
            foreach (Blog blog in blogs)
            {
                blog.Posts = GetPosts(blogger, blog);
                foreach (Post post in blog.Posts)
                    post.Comments = GetComments(blogger, post);
            }

            Console.WriteLine("\n\nType 'help' for help, 'exit' to quit.");
            for (; ; )
            {
                Console.Write("> ");
                string input = Console.ReadLine();
                if (input.Equals("exit", StringComparison.CurrentCultureIgnoreCase))
                    break;

                if (input.Equals("help", StringComparison.CurrentCultureIgnoreCase))
                {
                    Console.WriteLine(
                        "Type:\n\texit\n\t\t-- to quit.\n\thelp\n\t\t--to get this text.\n");
                    Console.WriteLine(
                        "\tlist blogs\n\t\t-- to list available blogs.\n");
                    Console.WriteLine(
                        "\tlist posts in blog\n\t\t-- to list posts in a blog.\n");
                    Console.WriteLine(
                        "\tlist comments in blog\n\t\t-- to list comments in a blog.\n");
                    Console.WriteLine(
                        "\tlist comments in post\n\t\t-- to list comments in a post.\n");
                    Console.WriteLine(
                        "\tlist comments by author\n\t\t-- to list comments by " +
                        "a particular author.\n");
                    Console.WriteLine(
                        "\tlist comments like commenturi\n\t\t-- to list comments " +
                        "identical to a comment.\n");
                    Console.WriteLine(
                        "\tlist comments with text\n\t\t-- to list comments " +
                        "containing text segment.\n");
                    Console.WriteLine(
                        "\tdelete comments by author\n\t\t-- to delete all " +
                        "comments by a particular author.\n");
                    Console.WriteLine(
                        "\tdelete comments like commenturi\n\t\t-- to delete " +
                        "all comments that are the same as a " +
                        "comment with this URI.\n");
                    Console.WriteLine(
                        "\tdelete comments with text\n\t\t-- to list comments " +
                        "containing text segment.\n");
                    Console.WriteLine("Names above could be either text names or URIs.");
                    continue;
                }

                if (input.Equals("list blogs", 
                    StringComparison.CurrentCultureIgnoreCase))
                {
                    foreach (Blog blog in blogs)
                        Console.WriteLine("{0} ({1})", blog.Name, blog.Feed);
                    continue;
                }

                if (input.StartsWith("list posts in ", 
                    StringComparison.CurrentCultureIgnoreCase))
                {
                    string blogname = input.Substring(14);
                    foreach (Blog blog in blogs)
                    {
                        if (blog.Name.StartsWith(blogname) ||
                            blog.Feed.Equals(blogname))
                        {
                            foreach (Post post in blog.Posts)
                                Console.WriteLine("{0} ({1})", post.Title, 
                                    post.CommentsFeed);
                        }
                    }
                    continue;
                }

                if (input.StartsWith("list comments in ", 
                    StringComparison.CurrentCultureIgnoreCase))
                {
                    string name = input.Substring(17);
                    foreach (Blog blog in blogs)
                    {
                        if (blog.Name.StartsWith(name) || blog.Feed.Equals(name))
                        {
                            foreach (Post post in blog.Posts)
                            {
                                Console.WriteLine("{0} {1}", post.Title, 
                                    post.CommentsFeed);
                                foreach (Comment comment in post.Comments)
                                    Console.WriteLine("-- {0} ({1})\n{2}\n", 
                                        comment.Author, comment.EditUri,
                                        comment.Text);
                            }
                            break;
                        }

                        foreach (Post post in blog.Posts)
                        {
                            if (post.Title.StartsWith(name) || 
                                post.CommentsFeed.Equals(name))
                            {
                                Console.WriteLine("{0} {1}", post.Title, 
                                    post.CommentsFeed);
                                foreach (Comment comment in post.Comments)
                                    Console.WriteLine("-- {0} ({1})\n{2}\n", 
                                        comment.Author, comment.EditUri,
                                        comment.Text);
                            }
                        }
                    }
                    continue;
                }

                if (input.StartsWith("list comments by ", 
                    StringComparison.CurrentCultureIgnoreCase))
                {
                    string author = input.Substring(17);
                    foreach (Blog blog in blogs)
                    {
                        foreach (Post post in blog.Posts)
                        {
                            foreach (Comment comment in post.Comments)
                            {
                                if (comment.Author.Equals(author, 
                                    StringComparison.CurrentCultureIgnoreCase))
                                {
                                    Console.WriteLine("{0} : {1}",
                                        blog.Name, post.Title);
                                    Console.WriteLine("-- {0} ({1})\n{2}\n",
                                        comment.Author, comment.EditUri,
                                        comment.Text);
                                }
                            }
                        }
                    }
                    continue;
                }

                if (input.StartsWith("list comments with ", 
                    StringComparison.CurrentCultureIgnoreCase))
                {
                    string text = input.Substring(19);
                    foreach (Blog blog in blogs)
                    {
                        foreach (Post post in blog.Posts)
                        {
                            foreach (Comment comment in post.Comments)
                            {
                                if (comment.Author.Contains(text) || 
                                    comment.Text.Contains(text))
                                {
                                    Console.WriteLine("{0} : {1}",
                                        blog.Name, post.Title);
                                    Console.WriteLine("-- {0} ({1})\n{2}\n",
                                        comment.Author, comment.EditUri,
                                        comment.Text);
                                }
                            }
                        }
                    }
                    continue;
                }

                if (input.StartsWith("list comments like ", 
                    StringComparison.CurrentCultureIgnoreCase))
                {
                    string prototypeUri = input.Substring(19);
                    Comment prototypeComment = null;

                    foreach (Blog blog in blogs)
                    {
                        foreach (Post post in blog.Posts)
                        {
                            foreach (Comment comment in post.Comments)
                            {
                                if (prototypeUri.Equals(comment.EditUri))
                                {
                                    prototypeComment = comment;
                                    goto found;
                                }
                            }
                        }
                    }
                    continue;

                found:
                    foreach (Blog blog in blogs)
                    {
                        foreach (Post post in blog.Posts)
                        {
                            foreach (Comment comment in post.Comments)
                            {
                                if (comment.Author.Equals(prototypeComment.Author,
                                    StringComparison.CurrentCultureIgnoreCase) &&
                                    comment.Text.Equals(prototypeComment.Text))
                                {
                                    Console.WriteLine("{0} : {1}", blog.Name, post.Title);
                                    Console.WriteLine("-- {0} ({1})\n{2}\n", 
                                        comment.Author, comment.EditUri,
                                        comment.Text);
                                }
                            }
                        }
                    }
                    continue;
                }

                if (input.StartsWith("delete comments by ", 
                    StringComparison.CurrentCultureIgnoreCase))
                {
                    string author = input.Substring(19);

                    foreach (Blog blog in blogs)
                    {
                        foreach (Post post in blog.Posts)
                        {
                            List<Comment> deleted = new List<Comment>();

                            foreach (Comment comment in post.Comments)
                            {
                                if (comment.Author.Equals(author, 
                                    StringComparison.CurrentCultureIgnoreCase))
                                {
                                    Console.WriteLine("{0} : {1}", blog.Name, post.Title);
                                    Console.WriteLine("-- {0} ({1})\n{2}\n", 
                                        comment.Author, comment.EditUri,
                                        comment.Text);
                                    blogger.Delete(new Uri(comment.EditUri));
                                    deleted.Add(comment);
                                }
                            }

                            foreach (Comment comment in deleted)
                                post.Comments.Remove(comment);
                        }
                    }
                    continue;
                }

                if (input.StartsWith("delete comments like ", 
                    StringComparison.CurrentCultureIgnoreCase))
                {
                    string prototypeUri = input.Substring(21);
                    Comment prototypeComment = null;

                    foreach (Blog blog in blogs)
                    {
                        foreach (Post post in blog.Posts)
                        {
                            foreach (Comment comment in post.Comments)
                            {
                                if (prototypeUri.Equals(comment.EditUri))
                                {
                                    prototypeComment = comment;
                                    goto found;
                                }
                            }
                        }
                    }
                    continue;

                found:
                    foreach (Blog blog in blogs)
                    {
                        foreach (Post post in blog.Posts)
                        {
                            List<Comment> deleted = new List<Comment>();

                            foreach (Comment comment in post.Comments)
                            {
                                if (comment.Author.Equals(prototypeComment.Author,
                                    StringComparison.CurrentCultureIgnoreCase) &&
                                    comment.Text.Equals(prototypeComment.Text))
                                {
                                    blogger.Delete(new Uri(comment.EditUri));
                                    deleted.Add(comment);
                                    Console.WriteLine("{0} : {1}", blog.Name, post.Title);
                                    Console.WriteLine("-- {0} ({1})\n{2}\n", 
                                        comment.Author, comment.EditUri,
                                        comment.Text);
                                }
                            }
                            foreach (Comment comment in deleted)
                                post.Comments.Remove(comment);
                        }
                    }
                    continue;
                }

                if (input.StartsWith("delete comments with ", 
                    StringComparison.CurrentCultureIgnoreCase))
                {
                    string text = input.Substring(21);
                    foreach (Blog blog in blogs)
                    {
                        foreach (Post post in blog.Posts)
                        {
                            List<Comment> deleted = new List<Comment>();

                            foreach (Comment comment in post.Comments)
                            {
                                if (comment.Author.Contains(text) || 
                                    comment.Text.Contains(text))
                                {
                                    blogger.Delete(new Uri(comment.EditUri));
                                    deleted.Add(comment);
                                    Console.WriteLine("{0} : {1}", blog.Name, post.Title);
                                    Console.WriteLine("-- {0} ({1})\n{2}\n", 
                                        comment.Author, comment.EditUri,
                                        comment.Text);
                                }
                            }

                            foreach (Comment comment in deleted)
                                post.Comments.Remove(comment);
                        }
                    }
                    continue;
                }

                Console.WriteLine(
                    "Can't parse. Please use 'help' to look up valid commands.");
            }


            Console.WriteLine("Done!");
        }
    }
}

Funny quote: arguing with idiots

"Never argue with an idiot. They bring you down to their level and beat you with experience."

Source unknown.

No Country For Old Men (via Reddit)

The pictures speak for themselves...

http://www.boston.com/bigpicture/2009/03/mexicos_drug_war.html

Death of journalism

So this is what passes for publishable news these days - Paul Krugman writes two pages in NYT Opinions section (http://www.nytimes.com/2009/03/23/opinion/23krugman.html?_r=1), and Reuters immediately quotes half of it on its web site: http://www.reuters.com/article/GCA-CreditCrisis/idUSTRE52M4SS20090323.

Guess which one of them was cross-linked from Reddit? You guessed right - the Reuters one.

What should have been a link to the original article, has become a copycat in its own right. A page was added to the Web. A "journalist" was paid for it. The reporting standards went down one notch further. The life continues...

The big takeover (via Reddit)

"These people need their trips to Baja, their spa treatments, their hand jobs," says an official involved in the AIG bailout, a serious look on his face, apparently not even half-kidding. "They don't function well without them."

http://www.rollingstone.com/politics/story/26793903/the_big_takeover/print

If you thought iFart and iBoobs were weird...

...behold the Cylon Detector iPhone App!

http://www.scifi.com/battlestar/iphone/

Hooray for long American tradition for consumer-driven innovation!
----
Thought about it some more, and came to conclusion that the consumers of this stuff are no weirder than Sarah Palin voters after all :-)...

On hacking cute shiny objects (via Reddit)

"Safari on the Mac is easier to exploit. The things that Windows do to make it harder (for an exploit to work), Macs don’t do. Hacking into Macs is so much easier. You don’t have to jump through hoops and deal with all the anti-exploit mitigations you’d find in Windows.

It’s more about the operating system than the (target) program. Firefox on Mac is pretty easy too. The underlying OS doesn’t have anti-exploit stuff built into it."

http://blogs.zdnet.com/security/?p=2941

This free software is very, very expensive!

New TFS documentation pushed

I blogged earlier (http://1-800-magic.blogspot.com/2009/02/everything-you-know-about-diffing-files.html) about how terrible TFS API documentation was.

This is no longer true - the latest round that just got pushed on MSDN is solid. Good job, team!

http://msdn.microsoft.com/en-us/library/bb130146.aspx

How to install TFS on a single domain-joined server

Since TFS documentation is so convoluted, I thought I would put together a SIMPLE list of what needs to be done.

Software required:

• Windows Server 2008 Standard


• SQL Server 2008 Standard Edition


• Sharepoint Server 2007 WITH SP1


• TFS 2008


• TFS 2008 SP1

All this software is available from MSDN.

Also: one domain account different from the installing user. Should NOT have admin rights. Should NOT be a local account. This is very important.

Prework:
Prepare TFS 2008 slipstreamed with SP1 as follows:

1. Copy the contents of TFS DVD AT folder to a directory, say c:\tfs\base:
       xcopy /die d:\at c:\tfs\base


2. Unpack the SP1 installer to a different directory, say c:\tfs\sp1:
       en_visual_studio_team_system_2008_team_foundation_server_service_pack_1_x86_x64wow.exe /extract:c:\tfs\sp1


3. Slip-stream the sp1:
       msiexec /a c:\tfs\base\vs_setup.msi /p TFS90sp1-KB949786.msp TARGETDIR=c:\tfs\slipstream


4. Create ISO and burn it on a DVD using ISO Recorder (http://isorecorder.alexfeinman.com/W7.htm)

Installation steps:

You must be logged as a domain user and have admin rights to the machine.

If you are installing on a VM, take snapshots between the installation steps, so if something goes wrong, you can restart easily.

1. Install Server 2008. Configure, join the domain, apply all updates. Reboot if necessary.


2. Add Web Server (IIS) role from Server Manager->Roles->Add roles. Select all components.


3. Install Windows Sharepoint Server 2007 SP1 using all defaults (non-SP1 build will not install on Server 2008).


4. Install SQL Server 2008.
   
   • Select all components to install.
   
   
   • On the page that allows you to specify the instance name, select the default instance.
   
   
   • On service accounts page, specify "Network Service" everywher (where it's editable) and make all services that can be made autostart do so.
   
   
   • Ensure Windows authentication is selected (the default), and click "Add current user" to administrators of the computer on the two pages which ask you to specify the admin user (one for SQL, one for analysis services).
   
   
   • On the page that asks you about configuring reporting, select "Install, but leave Reporting Services unconfigured" option.


5. Install TFS.
   
   • Do not change the default SQL instance name (it will be your computer name - definitely DO NOT switch it to '.').
   
   
   • Use "Network Service" where it allows you to (the default), and specify that non-admin user account from pre-requisitive section on the next page.
   
   
   • The installation will also ask you for the sharepoint admin URL. The default is incorrect. You can find the right port by opening the sharepoint admin link from the start menu.


6. RESTART THE SERVER. TFS does not prompt you to do this, but the permissions are synchronized on the service restart, so you won't be able to create a new project until you do this.

You're done.

Ensure that the client computer has Team Explorer that is upgraded to SP1 (creating project will fail in misterious ways if server has SP1 and client does not), and create a new project to test the newly installed system.

Adventures in TFS, continued

I was chronicling my attempt to install TFS on a VM at home so I have a better platform for fixing bugs related to that source control system. The beginning of this adventure is here: http://1-800-magic.blogspot.com/2009/03/adventures-in-tfs.html.

I should have given up on this thing a long, long time ago, but it started feeling like a technical challenge. One man vs. TFS. "The old man and the sea" of sorts. Although there the challenge was much less wanton :-).

Day 7. Create a new VM. This time I take a snapshot in Hyper-V every time something new gets installed.

• Windows Server 2008 + all updates


• Snapshot


• Sharepoint Server 2007 SP1


• Snapshot


• SQL Server 2008 Standard


• Snapshot

This time around TFS setup crashes after complaining that a command with half-a-page worth of command line options returned a non-zero result. Looking at winerror.h, the returned result is an unhandled exception. When I run the same command from a prompt (lots of retyping!), the exception has something to do with SQL reporting services.

At this point I give up and start reading the manual. TFS installation manual is an exercise in hyper-linking, forming a complex graph with many cycles in it. Instead of having 2-3 lists (if you are creating a single server deployment, do this, this, and this), it is an unwieldy bowl of spaghetti docs, each step cross-linking at least 5 more subtopics.

After spending 30 minutes down this maze of twisty little passages, all alike, the only difference I can find is about installing SQL server with unconfigured reporting services.

I roll back to the checkpoint after sharepoint, and this time I choose "Install but not configure reporting services".

TFS install can now complete, but I am back to the square one - trying to create the project in Visual Studio does not work - again, the error message is vague, something about reporting services.

Day 8.

I suddenly remember that I applied Visual Studio SP1 BEFORE I installed TFS client on my client machine. Everything worked so far just editing/checking files in, but I decide to reinstall SP1 just to make sure.

An hour and a half later - yes, this is how long it takes VS SP1 to install - everything works! That was the problem...

This is why we're not winning in the online world...

Adventures in TFS

Now that more and more teams are using Malevich (http://www.codeplex.com/Malevich), I am getting more and more TFS users. TFS had become quite popular at Microsoft because it integrates very nicely with Visual Studio, and of course DevDiv is using it (as they should) for dogfood.

TFS has many idiosyncrasies that Perforce does not, so occasionally I am getting bug reports for things that work nicely in Perforce/Source Depot, but not in TFS.

For a while, I was using CodePlex's Malevich enlistment for fixing bugs, but now that the bugs had become harder and harder to reproduce and require more and more setup, I decided to install my own version of TFS.

One thing that most TFS developers usually don't have to do, is TFS setup. Perforce is simple, really simple. You just copy an executable to a computer and run it - that's all. Well, you do have to open a port in the firewall, then the first time you log in, set up an admin account.

Here's what I had to go through to get TFS to work. Of course I was only working at home, at night, so a day is not really a day - it's just a few hours, sometimes even less.

Day 1. Try to install TFS on one of my servers. Discover that it does not work on 64-bit machines (huh???).

Day 2. Try to install TFS on one of my 32-bit servers. The setup at least starts now. If I use MYCOMPUTERNAME as a SQL Server target (the default instance is SQL 2008 on that box), the setup dies with many complaints about SQL Full Text Search not being installed/started/set up to start automatically. If I use '.' as an instance name, the installation goes through - up to the point where it wants a Sharepoint server. I don't have the Sharepoint server.

Day 3. I am turning on Hyper-V on one of my bigger servers. It's a dual Xeon5030 box, so 4 cores. Nice machine. Server 2008 installs almost instantaneously in Hyper-V, and the speed - I can't tell the difference. I also install SQL Server 2008.

Day 4. I have Sharepoint 2007 from MSDN, but it does not install on Server 2008. Need SP1. So I download Sharepoint 2007 with SP1. Install it. Trying to install TFS. No luck - the installation fails because it cannot find SQL Reporting Services. I am verifying - yes, reporting services are installed.

Day 5. I realize that when I use '.' it is finding a SQL 2005 instance brought in by Sharepoint. When I use MACHINENAME, it is finding SQL 2008 instance that I installed for it. SQL 2005 (that instance) does not have reporting services. SQL 2008 does not have Full Text Search as a separate service, so TFS setup is incapable to detect it. Google says I need SP1 for TFS 2008 to install on SQL Server 2008.

Day 6. Checking MSDN - MSDN does not have TFS with slip-streamed SP1. Google gets me an article that describes how to do it myself. The article is more than a bit imprecise, so I am reproducing the actual steps here:

• Copy the contents of TFS DVD AT folder to a directory, say c:\tfs\base (xcopy /die d:\at c:\tfs\base).


• Unpack the SP1 installer to a different directory, say c:\tfs\sp1 ( en_visual_studio_team_system_2008_team_foundation_server_service_pack_1_x86_x64wow.exe /extract:c:\tfs\sp1). Why does this have x64wow in its name despite the fact that it won't work on x64, wow or no wow - no idea.


• Slip-stream the sp1 (msiexec /a c:\tfs\base\vs_setup.msi /p TFS90sp1-KB949786.msp TARGETDIR=c:\tfs\slipstream)

You're done, the slipstream directory contains TFS 2008 SP1. The rest of the stuff on the TFS DVD (build, TE) is not slip-streamable, so just the TFS part is what you need. Make an image from this directory and voila - this will actually install on Server 2008 with SQL Server 2008!

I am trying to now create a new TFS project. I can connect to the server, but when I do try to create a project, it fails.

TF30004: The New Team Project Wizard encountered an unexpected error while initializing the Microsoft.ProjectCreationWizard.Reporting plug-in.  

Explanation 
TF30171: The Microsoft.ProjectCreationWizard.Reporting plug-in used to create the new team project could not be initialized and returned the following error: TF30224: Failed to retrieve projects from the report server. Please check that the SQL Server Reporting Services Web and Windows services are running and you have sufficient privileges for creating a project..  

User Action 
Contact your Team Foundation Server administrator.  

I am trying to investigate this. It turns out that I have no access to SQL Reporting services - I've installed SQL server as a local admin, and despite the fact that I've added Domain Admins, my user name, and everything to the admins, it did not work.

I decide that the easiest thing would be to just uninstall everything, and reinstall it from a normal user account.

Unfortunately, I start uninstallation from SQL Server. After I uninstall SQL Server, TFS installer crashes on uninstall. It looks like the VM is toast...

To be continued...

A very wonderful opportunity...

...has just presented itself in the form of you potentially paying $1000 for 4 gigs of RAM:

http://store.apple.com/us/configure/MB419LL/A?mco=NDE4Mzg5MA

Also, did you know that FileMaker is "#1 name in databases"? You do now!

Just how dumb are we, really?

http://www.gallup.com/poll/116065/Americans-Views-Bank-Takeovers-Appear-Fluid.aspx

Gallup poll asks half of their polling sample whether they are for or against temporary "takeover" of the banks, and the other half whether they are for or against temporary "nationalization". The rest of the question is identical.

On the takover question: 54% for, 44% against, 3% no opinion.

On the nationalization question: 37% for, 57% against, 6% no opinion.

These people are allowed to vote?

Leading scientists still reject God

"You clearly can be a scientist and have religious beliefs. But I don't think you can be a real scientist in the deepest sense of the word because they are such alien categories of knowledge."

http://www.stephenjaygould.org/ctrl/news/file002.html

Eventually we must stop the stupid blabber about how religion and science are compatible, and just admit that they are not.

Bleak outlook for press accuracy

This is among today's headlines in NYTimes:


I was wondering if "unusually gradual and prolonged period of recovery" is new Fedspeak for a "big, terrible depression". So I searched the speech, which was helpfully linked off the same headline:
http://www.federalreserve.gov/newsevents/speech/bernanke20090218a.htm.

The speech contains no words "unusually", "gradual", or "prolonged". It does contain the word "and".

Meet the new boss, part II

"Hertz's statement mirrors a statement from Obama White House Counsel Gregory Craig published Saturday.

"The president is very sympathetic to those who want to find out what happened," Craig told The Washington Post. "But he is also mindful as president of the United States not to do anything that would undermine or weaken the institution of the presidency. So, for that reason, he is urging both sides of this to settle.""

(Emphasis mine)

http://rawstory.com/news/2008/Obama_not_Bush_now_seeking_delay_0217.html

I thought we elected this guy because we wanted a president, not an emperor? Actually, hold this thought. We elected him because we were scared - again. Except this time we were not afraid of the terrorists, but for our money.

America, the land of the brave...

Recession as an effective garbage collector

"The diminishment has not, God knows, been quantitative. Never has there been so much product. Never has the American art world functioned so efficiently as a full-service marketing industry on the corporate model.

Every year art schools across the country spit out thousands of groomed-for-success graduates, whose job it is to supply galleries and auction houses with desirable retail. They are backed up by cadres of public relations specialists — otherwise known as critics, curators, editors, publishers and career theorists — who provide timely updates on what desirable means."

http://www.nytimes.com/2009/02/15/arts/design/15cott.html?em

I hate HTML

Passionately!

Especially the table layout. This thing was conceived by total morons. I would build a memorial to the authors of this abomination with inscription: "Spit here!"

Consider this:

<style>
    .CssTable
    {
        table-layout: fixed;
        width: 750px;
        border: solid black 1 px;
    }
    
    .CssCell1
    {
        width: 150px;
    }
    .CssCell2
    {
        width: 600px;
    }
</style>
<table class = "CssTable">
<tr><td class = "CssCell1">Cell 1</td>>td class="CssCell2">Cell 2</td></tr>
</table>

It does the right thing: the table is 750 pixels, the first column is 150 and the second one is 600. All is well. Now insert this line right after the table tag (as the first row):

<tr><td colspan=2>Long line</td></tr>

...and all the alignemnt just goes to hell. The columns become equally sized. Why?!

There is a solution - it is to create table header and set its sizes to what the desired sizes of columns are. Still, WTF?

The biggest experiment in compensation rules

Now that hundreds of banks had the compensation for their highest paid people severely limited, we will see how would that translate into financial performance of these institutions.

http://www.washingtonpost.com/wp-dyn/content/article/2009/02/13/AR2009021303288.html?hpid=topnews

My bet - they will do either the same or better for shareholders.

Everything you know about diffing files in TFS

MSDN support for TFS is ridiculous - it is essentially an automated dump of class structure with no information whatsoever. Shame on you TFS PMs/management for releasing a product with no documentation. I thought we were a developer-oriented company!

EDIT 3/12/09 This is no longer true - MSDN has pushed the new round of TFS docs and they are great. Excellent job!

Buck Hodges has a couple of great posts, however, that saved the day for me:

• http://blogs.msdn.com/buckh/archive/2006/05/01/diff-shelveset.aspx


• http://blogs.msdn.com/buckh/archive/2006/04/06/project_diff.aspx

We're so lucky to live in these enlightened times...

...in an enlightened country!

For example, they used to burn you at the stake for witchcraft. Now - you just get expelled from college...

http://www.clearwater.edu/The_Guide.pdf

Happy 1234567890!

http://en.wikipedia.org/wiki/Unix_time

Bi-partisanship

"As someone who appeared numerous times on the 700 Club with Pat Robertson, as someone for whom Jerry Falwell used to send his private jet to bring me to speak at his college, as an author who had James Dobson giveaway 150,000 copies of my one of my fundamentalist "books" allow me to explain something: the Republican Party is controlled by two ideological groups. First, is the Religious Right. Second, are the neoconservatives. Both groups share one thing in common: they are driven by fear and paranoia. Between them there is no Republican "center" for you to appeal to, just two versions of hate-filled extremes."

(Emphasis mine.)

http://www.huffingtonpost.com/frank-schaeffer/an-open-letter-to-preside_b_165359.html

Incidentally, below is a screen shot of commentary that appeared on a right-wing talk show's website following the news that an "anti-racist activist" (sic) was attacked by a black man. Check it out - is is enlightening.

So if you do consider youself a "moderate" Republican, think about who are the people that you rely on to get your agenda passed. Just like in the early thirties in Germany, these forces might turn out to be bigger than you...

Meet the new boss...

"the new administration -- the new DOJ -- had actively reviewed this case and vetted the Bush positions and decisively opted to embrace the same positions."

http://www.salon.com/opinion/greenwald/2009/02/09/state_secrets/index.html

How much does a poor investment banker need to just get by?

I asked this question yesterday, and - lo and behold - today's New York Times has an answer! I am generally not a big fan of long quotes, but this is so surreal, that I see no way around.

The whole story is here: http://www.nytimes.com/2009/02/08/fashion/08halfmill.html?_r=1.

So these are the basic expenses:

"Barbara Corcoran, a real estate executive, said that most well-to-do families take at least two vacations a year, a winter trip to the sun and a spring trip to the ski slopes.

Total minimum cost: $16,000.

A modest three-bedroom apartment, she said, which was purchased for $1.5 million, not the top of the market at all, carries a monthly mortgage of about $8,000 and a co-op maintenance fee of $8,000 a month. Total cost: $192,000. A summer house in Southampton that cost $4 million, again not the top of the market, carries annual mortgage payments of $240,000.

Many top executives have cars and drivers. A chauffeur’s pay is between $75,000 and $125,000 a year, the higher end for former police officers who can double as bodyguards, said a limousine driver who spoke anonymously because he does not want to alienate his society customers.

“Some of them want their drivers to have guns,” the driver said. “You get a cop and you have a driver.” To garage that car is about $700 a month.

A personal trainer at $80 an hour three times a week comes to about $12,000 a year.

The work in the gym pays off when one must don a formal gown for a charity gala. “Going to those parties,” said David Patrick Columbia, who is the editor of the New York Social Diary (newyorksocialdiary.com), “a woman can spend $10,000 or $15,000 on a dress. If she goes to three or four of those a year, she’s not going to wear the same dress.”

Total cost for three gowns: about $35,000.

Not every bank executive has school-age children, but for those who do, offspring can be expensive. In addition to paying tuition, “You’re not going to get through private school without tutoring a kid,” said Sandy Bass, the editor of Private School Insider, a newsletter that covers private schools in the New York City area. One hour of tutoring once a week is $125. “That’s the low end,” she said. “The higher end is 150, 175.” SAT tutors are about $250 an hour. Total cost for 30 weeks of regular tutoring: $3,750.

Two children in private school: $64,000.

Nanny: $45,000.

Ms. Bass, whose husband is an accountant with many high-end clients, said she spends about $425 every 10 days on groceries for her family. Annual cost: about $15,000.

More? Restaurants. Dry cleaning. Each Brooks Brothers suit costs about $1,000. If you run a bank, you can’t look like a slob.

The total costs here, which do not include a lot of things, like kennels for the dog when the family is away, summer camp, spas and other grooming for the human members of the family, donations to charity, and frozen hot chocolates at Serendipity, are $790,750, which would require about a $1.6-million salary to compensate for taxes. Give or take a few score thousand of dollars."

See how much it takes in living expenses just to gamble your money away? I think I start to understand how it came to be that so many heads were chopped off during French Revolution...

Compensation and job performance

The opinion page at WSJ was always a steaming heap of crap, but it looks like under Murdoch the rest of the - once very respectable - paper is moving fast to close the gap. Here's the recent "gem": http://online.wsj.com/article/SB123396915233059229.html.

It starts with a tearful personal story of a former investment banker.

"I was a 35-year-old, nonpartner investment banker then and was horrified to learn that my annual take-home pay would be limited to my small salary, which accounted for about a quarter of my previous year's income. Fortunately the partners decided to pay a small bonus out of their capital that year to help employees like me get by. The next year was no better. Several colleagues with good prospects left the firm and the industry for good. We learned that strong pay-for-performance compensation incentives could cut both ways."

Which actually begs an outright question - how much was this "small salary" anyway? What does it take for an investment banker to "get by"? $200k? $300k? $1m?

This idiotic idea that once you start paying less, the smart people will either leave or automatically become dumber was somehow sold to our populace, but it is belied by an every day experience.

When a person comes to a job interview and his/her answer to the question "why are you in this industry" is "because it pays a lot", the interview ends right there, right? How many companies did you see where this would not be true?

Studies upon studies show that there is NO direct correlation between job performance and the salary. There is a lot of correlation between overall level of happiness, and a dollar spent on benefits achieves far more towards that end than a dollar spent on salary.

Google is an ideal case in point. It employs some of the top engineers, but it pays slightly below the industry average. Almost everyone who works at Google could get more money elsewhere. The same is true for Microsoft.

When I interviewed at Google, I had a competing offer from Wall Street. Between the much higher cash and bonus, it was twice as much as what I'd be making at Google - and 50% more than what I was making at Microsoft. I chose Google, because while the salary was important, job satisfaction was even more so.

When I went back to Microsoft, I could have gone to Wall Street instead, and, again, get more money. I am sure that my original offer was still standing. But I went back to Microsoft, not to Wall Street.

Now, to hell with us mortals. Here's what correlates best with the CEO salary, and it's not the company performance. It's the golf scores: http://www.cerge.cuni.cz/pdf/events/papers/090112_t.pdf.

Yes... Yes! Yes!!!

We should reflect on how broken our system of government is when more and more people start saying - please, take our money!

http://www.nytimes.com/2009/02/06/opinion/06hastings.html?_r=1

And this is what Warren Buffett had to say about this a bit earlier.

http://www.nytimes.com/2006/11/26/business/yourmoney/26every.html

Malevich stats

Malevich is a code review system that I wrote initially for our team because I was missing Monrian - a Google tool that I learned to love when I was there.

We've been using it quite successfully for a month and a half now. Here are a few stats, as of right now, just from my team (a few other teams at Microsoft started using it as well):
Change lists reviewed: 217
Review iterations: 706
Files reviewed: 2386
Comments made: 2968

You can read more about the project here: http://1-800-magic.blogspot.com/2009/01/malevich-introduction.html, or get the code and documentation here: http://www.codeplex.com/Malevich.

Customizing TouchFlo 3d on HTC Fuze device

Fuze comes with a nice shell from HTC which replaces standard Windows Mobile Today screen. It functions as a collection of shortcuts to a bunch of applications and functions of the phone. Unfortunately, it is also used by AT&T (and/or other network operators) to push a bunch of usual OEM "extensions" on an unsuspecting customer - usually, solicitations to buy more services and software.

Luckily, there is a nice customization program that helps remove this crap. It is called Diamond TF3D Config, and is available from here: http://forum.xda-developers.com/showthread.php?t=405749.

Despite the fact that it is called "Diamond", it does appear to work - at least as far as removing the tab with AT&T stuff on it - on the Fuze device as well.

Next step - I need to figure out how to configure the Internet tab in TouchFlo so it does not go to their page...

Reprogramming push-to-talk (PTT) button on HTC Fuze

This will probably work on most other HTC devices (daimond, etc).

The PTT button is I think one of the stupidest creations in Windows CE world. It allows one to program a list of people who - at an extra cost - one would be able to phone with a single button.

Without passing the judgement on the usefulness of this service, the chief annoyance of it is that it has a specially dedicated button which is often very easy to push accidentally (on the Cingular 85xx device, for example, it is right next to the power button), and - the worst part - is not remappable. Unlike all other feature-related buttons.

This last one I completely fail to understand, and I really, really, really hope that we (Microsoft) didn't make it so. Because if we did, it would probably be - for me at least - the one of the most embarrassing features to ever come out of the Redmond campus, right there with Microsoft Bob.

Anyway, this is what one needs to do make the PTT button do something useful.
(1) Disable PTT service. To do this, modify device registry value "Dll" in HKLM\Services\PTT by appending ".sav" to the value of the registry ("PTTService.dll.sav"). This will prevent the PTT service from getting loaded - and will make it easy to restore should you ever want it back (just remove the ".sav").

If you have Visual Studio, you can use Remote Registry Editor in Visual Studio Remote Tools to edit the registry. Otherwise, google for "Windows CE Registry Editor". There are plenty of tools in existence.

(2) Open File Explorer on the device. Go to \Windows directory. Click-and-hold on voicecommand (you will have to scroll down a long, long list of files). Select "Copy" from the context menu. Then scroll down to the very bottom of the screen, and click-and-hold on the empty space. Select "Paste" from the context menu. Rename the resultant shortcut to "Short_PTT".

Note: this works only because voicecommand is already a shortcut. If you want to map an application (executable file) to the button, the instructions are as follows. Click-and-hold on the name of the application. Select "Copy". Then scroll down to the very bottom of the screen, and click-and-hold on the empty space. Select "Paste shortcut" from the context menu. Rename the resultant shortcut to "Short_PTT".

(3) Reboot the device to eradicate the PTT service. You are done.

E=MO2

"When I was a kid, I used to pray every night for a new bicycle. Then I realized that the Lord, in his wisdom, doesn't work that way. So I just stole one and asked him to forgive me!"

http://en.wikiquote.org/wiki/Emo_Philips

Idiocracy

"What happened to 30%? Lies!" shouted customer Gabriel Ifrah, 52, at the Circuit City on La Cienega Boulevard in Los Angeles on Monday, where most items were priced at 10% off.

...

"I've been waiting in line for half an hour each day based on employees' promises that prices could come down, but they haven't," said McGinness, a TV commercial producer from Los Angeles. "It's very disappointing."

http://www.chicagotribune.com/business/la-fi-liquidation20-2009jan20,0,497931.story

Is it me, or does this really sound insanely stupid?

By the way, if you need any proof that recession has not REALLY started yet, this is it. I mean, people waiting in line for cheap electronics?

DIY: Enriching uranium

http://www.amazon.com/Uranium-Ore/dp/B000796XXM/

Redirection with query parameters bug in IE

I really, really, really hate this browser.

Today's bug: if you do meta redirection to the same path with different query parameters, it ignores and passes the same parameters that the original page had.

Suppose your page, accessed as http://foo.com/bar?baz=1&bleh=2 has the following redirect tag: <meta http-equiv="Refresh" content="5;http://foo.com/bar?baz=1" />.

The request the server gets? It's the original http://foo.com/bar?baz=1&bleh=2.

I really, really, really hate this browser...

More people speak on leaving Google

An interesting compendium of opinions from people who worked at Google. A lot of negativity which is perhaps natural for the "former" team.

http://www.techcrunch.com/2009/01/18/why-google-employees-quit/

Still, I had a feeling that a lot of it is driven by the misunderstanding of Google psyche.

Google is not your typical software company - it is an advertising company. It is very much in the nature of advertising to tell the truth in a way that makes the advertiser looks as positive as possible.

It reflects in a way that the software is written, it reflects in compensation.

For example, when Microsoft tells you that they will pay for X (for example, moving you to/from a locale) they mean that the net economic effect to you will be X. Which means grossing up the taxes. The fact that they had to pay more is an implementation detail that is hidden from the user.

When Google tells you that they will pay for X (for example, a subsidy for buying a Prius), they mean that the net economic effect to them would be X. What you will get is really .6X - after taxes.

And so it goes ;-)... People who do not read the small script are bound to be disappointed.

It's not at all that the company is evil - far from it. Google tries to work very hard to appeal to the employees. It's just the advertising nature that show no matter what you do :-).

You have to read it all to fully appreciate the depth of stupidity of this guy...

http://www.windsorstar.com/Leamington+loses+Nigerian+scam/1173799/story.html

Your taxes at work, Part III

The messages on Arab TV teaching children the "art of martyrdom" (AKA becoming the suicide bombers) has been dutifully covered by US media, and have become the part of the lore on why US should support Israel.

Here's the other side of the medal.

http://www.mystudydate.com/pg/blog/Martini/read/1462/message-from-israeli-children-to-arabs-die

Let's hope it is covered with an equal zeal.

Free code! Send email from your Exchage account - programmatically!

Note: This requires Exchange 2007.

In Visual Studio 2008, create a Visual C# Console application project. Call it SendMail for simplicity.

References->Add Service Reference->Advanced->Add Web Reference

In URL box, put a server name on which you have an account. The format of the URL should be something like this https://mail.microsoft.com/EWS/Exchange.asmx

using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Text;

using SendMail.com.microsoft.mail;

namespace SendMail
{
    class Program
    {
        /// <summary>
        /// Shows usage.
        /// </summary>
        private static void DisplayUsage()
        {
            Console.WriteLine("Usage:\n");
            Console.WriteLine("    sendmail [--to emailaddress]+\n");
            Console.WriteLine("             [--cc emailaddress]*\n");
            Console.WriteLine("             [--from emailaddress]?\n");
            Console.WriteLine("             [--subject textlineinquotes]?\n");
            Console.WriteLine("             [--user username]\n");
            Console.WriteLine("             [--domain domain]?\n");
            Console.WriteLine("             [--password password]\n");
            Console.WriteLine("             [--ews exchangewebserviceurl]\n");
            Console.WriteLine("                 Text to send in email\n");
            Console.WriteLine("+ = one or more, * = zero or more, ? = zero or one");
            Console.WriteLine("Note: username and password are required.");
        }

        /// <summary>
        /// Processes command line args, sends one email.
        /// </summary>
        /// <param name="args"></param>
        static void Main(string[] args)
        {
            string user = null;
            string password = null;
            string domain = null;
            string ews = null;
            List<string> to = new List<string>();
            List<string> cc = new List<string>();
            string from = null;
            string subject = null;
            StringBuilder text = new StringBuilder();
            for (int i = 0; i < args.Length; ++i)
            {
                if (i < args.Length - 1)
                {
                    if (args[i].Equals(
                        "--user", StringComparison.InvariantCultureIgnoreCase))
                    {
                        user = args[++i];
                        continue;
                    }
                    if (args[i].Equals(
                        "--domain", StringComparison.InvariantCultureIgnoreCase))
                    {
                        domain = args[++i];
                        continue;
                    }
                    if (args[i].Equals(
                        "--password", StringComparison.InvariantCultureIgnoreCase))
                    {
                        password = args[++i];
                        continue;
                    }
                    if (args[i].Equals(
                        "--ews", StringComparison.InvariantCultureIgnoreCase))
                    {
                        ews = args[++i];
                        continue;
                    }
                    if (args[i].Equals(
                        "--to", StringComparison.InvariantCultureIgnoreCase))
                    {
                        to.Add(args[++i]);
                        continue;
                    }
                    if (args[i].Equals(
                        "--cc", StringComparison.InvariantCultureIgnoreCase))
                    {
                        cc.Add(args[++i]);
                        continue;
                    }
                    if (args[i].Equals(
                        "--from", StringComparison.InvariantCultureIgnoreCase))
                    {
                        from = args[++i];
                        continue;
                    }
                    if (args[i].Equals(
                        "--subject", StringComparison.InvariantCultureIgnoreCase))
                    {
                        subject = args[++i];
                        continue;
                    }
                }

                if (text.Length > 0)
                    text.Append(" ");

                text.Append(args[i]);
            }

            if (to.Count == 0 || user == null || password == null || ews == null)
            {
                DisplayUsage();
                return;
            }

            ExchangeServiceBinding binding = new ExchangeServiceBinding();
            binding.Credentials = new NetworkCredential(user, password, domain);
            binding.Url = ews;

            DistinguishedFolderIdType folder = new DistinguishedFolderIdType();
            folder.Id = DistinguishedFolderIdNameType.sentitems;

            TargetFolderIdType targetFolder = new TargetFolderIdType();
            targetFolder.Item = folder;

            CreateItemType createItem = new CreateItemType();
            createItem.MessageDisposition = MessageDispositionType.SendAndSaveCopy;
            createItem.MessageDispositionSpecified = true;
            createItem.SavedItemFolderId = targetFolder;

            MessageType message = new MessageType();

            List<EmailAddressType> recipients = new List<EmailAddressType>();
            foreach (string s in to)
            {
                EmailAddressType recipient = new EmailAddressType();
                recipient.EmailAddress = s;
                recipients.Add(recipient);
            }

            message.ToRecipients = recipients.ToArray();

            if (cc.Count > 0)
            {
                recipients = new List<EmailAddressType>();
                foreach (string s in cc)
                {
                    EmailAddressType recipient = new EmailAddressType();
                    recipient.EmailAddress = s;
                    recipients.Add(recipient);
                }

                message.CcRecipients = recipients.ToArray();
            }

            if (from != null)
            {
                message.From = new SingleRecipientType();
                message.From.Item = new EmailAddressType();
                message.From.Item.EmailAddress = from;
            }

            message.Subject = subject;
            message.Sensitivity = SensitivityChoicesType.Normal;

            message.Body = new BodyType();
            message.Body.BodyType1 = BodyTypeType.Text;
            message.Body.Value = text.ToString();

            createItem.Items = new NonEmptyArrayOfAllItemsType();
            createItem.Items.Items = new MessageType[1];
            createItem.Items.Items[0] = message;

            CreateItemResponseType response = binding.CreateItem(createItem);
            foreach (ResponseMessageType r in response.ResponseMessages.Items)
            {
                if (r.ResponseClass == ResponseClassType.Success)
                {
                    Console.WriteLine("Message sent.");
                }
                else
                {
                    Console.WriteLine("Failed to send the message. ");
                    Console.WriteLine(r.MessageText);
                }
            }
        }
    }
}

Diamonds are forever?

Yes, but only because you can't sell them!

Just read the absolutely captivating article in The Atlantic Monthly about the value of diamonds as an investment tool. The article http://www.theatlantic.com/doc/print/198202/diamond was written in 1982, but a casual Google search on "historical proces of diamonds" quickly confirms that all the basic numbers are still true, as well as its central premise, which is that diamonds is just about a notch below fertilizer as far as the quality of investment goes.

This is what they say, in short...

First, the diamonds are not actually all that rare. They were, before the beginning of the 20th century. But the since discovered reserves in Africa, Russia, and Australia are pretty big. What's more, the amount of diamonds that is held privately (extracted from these mines over the last 100 years) is astounding.

The diamond market is wholly owned by DeBeers. They fix the price by limiting production, marketing the diamonds as essentially a requirement for marriage in US and Japan (and thus attaching the "sentimental value" to them which is well beyond the real value).

Most importantly, they limit the liquidity of the market, which prevents the influx of second-hand diamonds which would certainly drop the price.

This is where it becomes really fun. How do you prevent the private diamonds from being sold? The scheme is brilliant: if you make a spread between retail and wholesale price huge, people - who paid the retail price when they acquired the stones - would find it very difficult to sell them, because when selling, they would be paying the wholesale price.

And this wholesale price is 2-3 times less than the retail.

In addition, DeBeers makes it hard for dealers to pay even the wholesale price: when dealers get their stones from DeBeers, they get them on consignment: they pay DeBeers when the diamonds are sold. There is no capital on their own involved. But if the dealer is buying the stone from the street, they have to put up their own capital.

The net result? That stone for which you paid $25000 at Costco? If you can get $7000 back for it, consider yourself lucky!

And based on the historical trends (http://minerals.usgs.gov/ds/2005/140/diamondindustrial.pdf), it will not be until the Universe collapses and is reborn in the next Big Bang that the prices will get to the levels where the investment will recoup itself :-).

Malevich - a new code review system

We were using it in our team for the last week, with pretty good success. Over 30 code reviews went through it. A couple of bugs were exposed, and quickly fixed. Most importantly, the central premise held - making comments is easy, and people make a lot of them - many more than before!

So today I finally added the last thing I wanted before taking the project public - support for TFS. It seems to be working now, although I have not done a lot of testing with it.

The code is now on CodePlex!

You can read the history of the project here:

  http://1-800-magic.blogspot.com/2008/12/one-google-thing-i-miss-most.html

  http://1-800-magic.blogspot.com/2009/01/malevich-introduction.html


Or, you can jump in and install it from here:

  http://www.codeplex.com/Malevich.

Happy reviewing!

Better than Vista!

http://xkcd.com/528/

:-)

[Proudly running Windows Server 2008 on my laptop :-)]

End of an era, part II

Within two month of the closing of print publication of the PC Magazine, the bell is now tolling for Dr. Dobb's...

http://ztrek.blogspot.com/2008/12/dr-dobbs-soon-to-become-monthly-section.html

Malevich, an introduction

For the last several weeks I was working on a code review system. I got sucked into it gradually: similar software that I used at Google took Guido van Rossum - a much better developer than me - almost a year to implement as his starter project. You can read about it here: http://code.google.com/p/rietveld/.

So I didn't expect that I would be able to do it, at least not in a reasonable time that is left from work.

But the team needed a code review system, and I was playing with ASP.NET. At some point, I was wondering how hard it is - really - to display a diff view of a file. Two hours later I had a working prototype. A localized success.

This got me thinking. Displaying the diff file must be about the hardest part of the system, isn't it? But what about entering the comments? It turned out that I did not completely forget the JavaScript yet, and an hour or so later I was able to click on my ASP.NET-generated page to enter the comments in a text box, and have them show permanently on a page after clicking a Submit button, or disappear on Cancel or Remove.



Ok, so far so good. Now I needed

1. Free time.


2. A database.


3. A source control interface.

The first one was solved quickly - the team has finished our first milestone, a lot of people went on their Christmas vacations, and Seattle was thoroughly snowed in, all at the same time. So the evenings had become much longer. There was at a very minimum an opportunity to take the project further.

One big problem is that I am a database noob. I used SQL Server once before, and not in production environment, but rather for another pet project of mine - a status reporting application for the team. I did learn how to create tables, stored procedures, and security objects back then, but I have no idea how good (or, rather, how bad) my SQL really is.

Luckily, I did discover the database extension for Visual Studio 2008 in my previous run-in with SQL Server. It is called Visual Studio 2008 Database Edition GDR (German Democratic Republic? I have no idea what GDR is), and you can download it from here: http://www.microsoft.com/downloads/details.aspx?FamilyID=bb3ad767-5f69-4db9-b1c9-8f55759846ed&displaylang=en.

The really nice feature of this GDR thing is that it makes updating the database really, really easy: you modify T-SQL that creates the database objects, and hit Deploy, after which it figures out what NEEDS to be changed, and changes the database accordingly, without destroying the database contents. It even does data conversions automagically!

The downside of editing database code inside GDR vs. SQL Server Management Studio is that Intellisense does not work.

The database took some time - mostly T-SQL in the stored procedures. But eventually it worked, and I needed to start storing real data in it.

Most Microsoft teams use Source Depot - a Perforce derivative - as their source control system. I don't know if Perforce has an API today. Source Depot was snapped from Perforce a while ago, and the API available for it is beyond horrific. Really, truly terrible. A command is issued by calling a function with p4 command line, and you get back semi-parsed data. Except some data gets dumped into standard output anyway.

So I ended up just running the command line parameter, capturing the output and parsing it.

By the way, before I forget, here's a way to capture a process output in C# that actually works (just reading standard handles blocks forever if there is a lot of output on both stdout and stderr channels):

private delegate string StringDelegate();

public static string ReadProcessOutput(Process proc,
    bool eatFirstLine, ref string errorMessage)
{
    StringDelegate outputStreamAsyncReader =
        new StringDelegate(proc.StandardOutput.ReadToEnd);
    StringDelegate errorStreamAsyncReader =
        new StringDelegate(proc.StandardError.ReadToEnd);
    IAsyncResult outAsyncResult =
        outputStreamAsyncReader.BeginInvoke(null, null);
    IAsyncResult errAsyncResult =
        errorStreamAsyncReader.BeginInvoke(null, null);

    // WaitHandle.WaitAll does not work in STA.
    if (Thread.CurrentThread.GetApartmentState() ==
        ApartmentState.STA)
    {
        while (!(outAsyncResult.IsCompleted &&
                errAsyncResult.IsCompleted))
            Thread.Sleep(500);
    }
    else
    {
        WaitHandle[] handles = {
            outAsyncResult.AsyncWaitHandle,
            errAsyncResult.AsyncWaitHandle
        };
        if (!WaitHandle.WaitAll(handles))
        {
            Console.WriteLine("Execution aborted!");
            return null;
        }
    }

    string results = outputStreamAsyncReader.EndInvoke(
        outAsyncResult);
    errorMessage = errorStreamAsyncReader.EndInvoke(
        errAsyncResult);

    proc.WaitForExit();

    return results;
}

Anyway, dealing with the source control turned out to be the nastiest part of the whole project. But in the end, it finally worked, and I could parse the change descriptions, get file versions, and so on.


At this point, I was very, very close. Now I needed a name.

Google code review system was called Mondrian - for Piet Mondrian, an early XX century Dutch painter (http://en.wikipedia.org/wiki/Piet_Mondrian). So Malevich (http://en.wikipedia.org/wiki/Kazimir_Malevich) - a Russian painter from approximately the same era was a logical choice. It also had an added benefit: his most famous painting was the "Black Square". Here it is:


I love "Black Square" because, being really, really, really bad at anything that involves any sort of art, and doubly bad with painting, images, and graphical design (as you can see by looking at the screen shots) this is the one image that even I can reproduce. So if this software project will ever need an icon, or a logo, or any sort of graphical representation, I revel in knowing that I will be able to do it :-).

Anyway, after the name was picked, two other things remained - a program to upload files, and finishing the web site.

I've got to say - I LOVE ASP.NET. I have no idea how good it is in terms of performance, but man it is easy to develop on! Very logical API, easy to learn and use, and Intellisense absolutely rocks! Without much previous experience, I was able to put out roughly 2000 lines of code that implemented the web site and web service to talk to JavaScript code in the browser in perhaps a couple of weekends.



So where are we now? The code is deployed, I demoed it to the team, and a few reviews had flown through it. A couple of bugs found and fixed within minutes, complete with redeployment of the app. I changed the database to accomodate - in the future - the addition of TFS. I've added perforce support, this took less than one evening.



All in all, I've spent less than 80 hours working on it, which I think is not bad. I can say that Visual Studio, SQL, LINQ and ASP.NET mattered a lot - there's no way in hell I could produce anything even close to it even in twice the time without these tools. Windows is not free, but man it makes developers productive :-).

I've uploaded Malevich to CodePlex (http://www.codeplex.com), but have not published the project yet - I think I will add TFS support before I do. The rule on CodePlex is that it must be published within a month, so by the 1st of February it has to be up, or it will get deleted. Check back before then for the link...

Update: It's been published now: http://www.codeplex.com/Malevich. And we have successfully used it for a months and a half now on my team for more than 200 reviews, 2000 files, and 3000 comments. Give it a whirl!

A slideshow

Today we were having a review of M0 results with our product unit manager - feature teams talking about the design work they did over the last couple of months that resulted in current implementation plans.

Cool stuff.

For a change we get the devs and testers, as opposed to PMs, to talk - to make sure they have high-level view of how parts of the overall project fits together.

Big room, about 30-40 people showed up. Overhead projector, all the standard meeting/team review attributes.

I am giving the introduction: where we are at the macro level, what went well, what didn't go so well, the schedule, the resource allocations.

Connect my laptop, fire the Powerpoint, and start talking. All in all, about 15 minutes, maybe a bit less. Not so many slides. But slides nevertheless.

The next presenter is a dev on one of our features. He is to use my laptop, so I am handing the microphone over and heading to my seat when he tells me that he needs the projection.

Turns out I was showing slides with the projector off. For 15 minutes. Lost, I look around. "Did you not see my slides?" -- "You had slides???" Homeric laughter and general hilarity ensues...

True story.

More Michael Lewis

"A lot has been said and written, for instance, about the corrupting effects on Wall Street of gigantic bonuses. What happened inside the major Wall Street firms, though, was more deeply unsettling than greedy people lusting for big checks: leaders of public corporations, especially financial corporations, are as good as required to lead for the short term.

Richard Fuld, the former chief executive of Lehman Brothers, E. Stanley O’Neal, the former chief executive of Merrill Lynch, and Charles O. Prince III, Citigroup’s chief executive, may have paid themselves humongous sums of money at the end of each year, as a result of the bond market bonanza. But if any one of them had set himself up as a whistleblower — had stood up and said “this business is irresponsible and we are not going to participate in it” — he would probably have been fired. Not immediately, perhaps. But a few quarters of earnings that lagged behind those of every other Wall Street firm would invite outrage from subordinates, who would flee for other, less responsible firms, and from shareholders, who would call for his resignation. Eventually he’d be replaced by someone willing to make money from the credit bubble."

http://www.nytimes.com/2009/01/04/opinion/04lewiseinhorn.html?_r=1&pagewanted=1

Your taxes at work, part II

http://www.liveleak.com/view?i=3b3_1230864719

I hate it when the mass media censors so called "disturbing" videos and photographs of casualties resulting from from military attacks. People who are disturbed by the actions of the army which they support sent by the president they elected should not be allowed to vote...

It is comforting to know that there are governments that are even stupider than ours...

http://www.inquisitr.com/12909/great-firewall-of-australia-whats-not-mentioned-makes-it-even-more-scary/

CNN headlines

This is what CNN front page looks like today.



And this is the article to which it links.



Funny how the wording of the title changed enroute, huh?

Israel

"The tale of Israel can be very much compared to someone who was beaten, tortured, molested and raped as a boy by his family members. You don't expect someone like that to grow up to be "normal". The founding of the state of Israel right after the holocaust was a big mistake. The boy grew up, learned martial arts, amassed weapons and became extremely aggressive towards his surroundings, seeing the ghosts of his tormentors in every face that he sees around him. Now he is channeling his childhood pain towards inflicting the same pain upon others. Desensitized to human sufferings, he is now capable of enormous acts of cruelty."

http://www.reddit.com/r/worldnews/comments/7m6m4/today_i_end_my_support_of_israel/647s

...in response to...

http://www.dailykos.com/storyonly/2008/12/28/114432/83/489/677860

I kid you not...

"Sometimes these dollars go to projects that have little or nothing to do with the public good. Things like fruit fly research in Paris, France. I kid you not," Ms Palin said.

http://www.independent.co.uk/news/science/scientific-illiteracy-all-the-rage-among-the-glitterati-1212406.html

Biology is clearly not on the list of graduation requirements on whatever "school" this woman went to... I am just wondering - who ARE these people in Alaska? This whole thing starts looking very much like Russia in 1920.

The reason to learn math...

http://abstrusegoose.com/96

I am so using it on my kids!

SQL nullable data items and LINQ

While working on Malevich - my new code review program, - I was hit by a runtime error when doing LINQ operation on a table that contained null datetime items. In C#, DateTime is a structure - not a class - and therefore can not be null.

Classes in C# are always allocated in the heap, and the variable that holds class object is really a pointer, whereas structures can be allocated on stack (just like classes and structures in C++), and variable that references a structure is really the memory representing this structure.

If C# had a sizeof operator, the sizeof of a variable containing class instance would always be a sizeof of a pointer - 4 or 8 bytes, - whereas a sizeof of a variable containing a structure would be the number of bytes that it takes to store this structure in memory.

Which is why string x = null is valid, but DateTime x = null is not. You can, however, explicitly create a "pointer" to DateTime (or any structure or primitive data type) by using the question mark after the type, for example, DateTime? x = null. This is called a "nullable" data type.

It turns out that the logic in Visual Studio that imports the database model (dbml) does not properly recognize nullable data in SQL - it generates not nullable prototypes for it.

For most common data items it is not an issue. Strings are class instances, and so they can be null in C#. It is not usual I suppose to store nullable numeric types - it would likely be more expensive to store the fact that it is or is not null rather than ignore the value when it is not needed.

DateTime is the only important type that really has a problem - it is common for it to be null in the database, and if you have a row that has null value for the date column, and try to instantiate it using VisualStudio-generated database interop class, you will get a runtime exception saying that DateTime field cannot be assigned a null value.

In SQL:

CREATE TABLE [dbo].[DateTimeTest] (
    [Id]            INT             IDENTITY (1, 1) NOT NULL,
    [TimeStamp]     DATETIME        NULL
);

INSERT INTO [dbo].[DateTimeTest] (TimeStamp) VALUES(NULL)

In C#:

// Watch this crash!
DateTimeTest[] x = (from dt in context.DateTimeTests select dt).ToArray();

I've done a few searches, and there are quite a few cases of people reporting the problem, but no good solutions for it - most people tend to rebuild their classes by hand when they need nullability, or fudge the database.

The solution does exist, however, and it is quite simple - in the graphical representation of database dbml, right-click on the offending field, select properties, and change Nullable field to true.

This is all it takes! After that, the date field is modeled as DateTime?, and everything works just fine.

A year of blogging

So I've been at it for slightly more than a year. I started this blog for three reasons.

First, I wanted to learn to write. In the very beginning, I had to correct or rewrite almost every sentence before making a post. There were typos, awkward sentence construction, and every stylistic problem known to humanity. Some amount of it is still there - after all, English is not my native language - but I think I am a lot better now than I used to be.

Second, I needed to have a place where I could jot things down for future reference. Some of them were simple facts that I would needed to recall periodically. Some were convenient shortcuts in coding. Some where points of view that it was convenient to have written down, for future reference.

Third, I wanted to write down a few lessons that I've learned in my career in computer industry. I was lucky to have been part of a whole bunch of interesting projects, the rise and the fall of the dot com boom, the emergence of broadband, the mobile phone revolution. It felt like a bad thing to have it go to waste: fools learn from their own experience, smart people learn from that of others. Hopefully, smart people everywhere will learn from my experiences. We need more smart people!

Attracting a lot of readers was never a goal. Yet over one hundred thousands people visited it since January 2008. Many left insightful comments from which I learned a lot. Quite a few served as a motivation for further blog entries.

The most controversial was the blog entry where I duscuss the reasons I went back to Microsoft after a year at Google (http://1-800-magic.blogspot.com/2008/06/back-to-microsoft.html). Alone, it accounted for 85000 page views out of 150000 site total. It elicited 180 comments on this site, spawned a small copycat industry of cross-posters, and was ultimately misquoted from ZDNet to NY Times.

The most visited technical article is the one on STL vector performance (http://1-800-magic.blogspot.com/2008/02/stl-vector-performance.html). It was viewed 6500 times, and is currently #1 entry on Google searches on "C++ vector performance", "STL vector performance", and just "vector performance" at Google.


A far less interesting article (http://1-800-magic.blogspot.com/2008/02/down-with-atoi.html) where I gripe about poor design of atoi API was read 3000 times, despite the fact that it is buried on the second page of Google search results on atoi.

The two articles on memory management that I consider my best technical entries - "Guerilla guide to native memory management" (http://1-800-magic.blogspot.com/2007/11/guerilla-guide-to-native-memory.html) and "Memory is not free (more on Vista performance) (http://1-800-magic.blogspot.com/2007/12/memory-is-not-free-more-on-vista.html) - have been read only ~1000 times each, despite prominent position on Google search.


I also used the blog to share a few bits of code I was toying with. About a year ago I got my hands on Sony Reader, and made a web proxy for Project Gutenberg that adds SONY LRF format to the list of available download options (http://1-800-magic.blogspot.com/2008/01/gutenberg-for-sony-pre-alpha.html). This post solicited the largest percentage of positive feedback :-). The site is still up, it has served hundreds of people who downloaded thousands of books through it.


Finally, the least popular post? This one: http://1-800-magic.blogspot.com/2007/12/us-electoral-politics.html. It's only been read 6 times :-)...

Non Piu Andrai

Why we should not let iPhone win...

Two articles that I spotted recently about iPhone apps, one rejected by Apple (http://www.theregister.co.uk/2008/12/23/iboobs/), another (actually, a bunch of others) accepted (http://venturebeat.com/2008/12/23/iphone-fart-app-pulls-in-nearly-10000-a-day/).

While one might observe that the classiness of the Apple crowd appears to be long gone, the important thing is not that.

The ONLY way for a developer to release an app for iPhone is through the Apple store (contrast it to any other smartphone where any piece of software can be installed from any location, be it a PC, or the Internet, and anybody can build and release applications).

And for Apple to distribute an application, someone at Apple must bless the morality of its contents. In this specific instance, Apple decided that farting is OK, while boobs (even the covered ones) are not.

And this is why we cannot let Apple win in this game. Because if we do, we will be putting one company as a judge of what is appropriate, and what is not. And I don't think there's an entity in the world which is qualified to do this.

GM is us....

"To top it off, we’ve fallen into a trend of diverting and rewarding the best of our collective I.Q. to people doing financial engineering rather than real engineering. These rocket scientists and engineers were designing complex financial instruments to make money out of money — rather than designing cars, phones, computers, teaching tools, Internet programs and medical equipment that could improve the lives and productivity of millions.

For all these reasons, our present crisis is not just a financial meltdown crying out for a cash injection. We are in much deeper trouble. In fact, we as a country have become General Motors — as a result of our national drift. Look in the mirror: G.M. is us."

http://www.nytimes.com/2008/12/24/opinion/24friedman.html?_r=1&ref=opinion

Incidentally, a whole lot of smart engineers that we have still left in computer science works on stuff that is very much akin to the financial engineering Friendman is writing about - chasing eyeballs, not users, producing Facebook apps that help people waste time rather than being productive...

The one Google thing I miss the most

This week turns out to be exactly six months since I left Google to return to Microsoft. The year at Google was fun, and I learned a lot of new stuff. I am glad that I took this detour from my Microsoft career - it gave me a lot of new perspectives.

Six months later, what aspects of "The Google Way (TM)" do I miss the most?

Surprisingly, the free food is not it - not even close: luckily, most software engineers are paid enough so that the food costs are not really visible in our budgets.

Google's focus on engineering, where a developer owns everything about the product, including testing and the feature set, is nice at first, until you realize that it comes at a heavy price.

The products that can be built within this model tend to be smallish and not very well integrated. Every project that I've done at Microsoft was bigger in scope, more interesting as an engineering challenge, and all but two made far more money than anything I created at Google. And I am glad to be working on something really big again.

I miss the peer review system. But it is easy to replicate, and Microsoft already gives me all the necessary tools, and we are going to be trying it in my team, although I will implement it slightly differently.

At Google one needs to get positive feedback from people who are one's senior in order to be promoted. Feedback from peers and junior people does not matter very much. The problem is that at some point there are just not enough senior people around you - there are only a handful of technical fellows at Google, and less than 10 distinguished engineers, if I remember correctly.

As a result I have seen mediocre engineers grow very quickly just because they happened to work on a project that had a DE, and much better engineers stuck at lower levels because they were not so lucky.

So in my team we're going to value peer feedback much more uniformly, and assign a bigger weight to the manager feedback than the 10-15% that is customary at Google.

I don't miss my Google managers at all. They were nice people, but they didn't have much impact on my life. After a year at Google, I still do not quite understand what the role of a manager there is. Whatever they do, they certainly do not have the tools to succeed, and their impact on teams seems to start and end with allocating resources.

I worked with a bunch of very smart, talented, and passionate developers at Google. I even had the privilege of meeting Craig Silverstein through my work on the readability team and exchanging a couple of change lists with him. The amount of energy and brain power collected there is staggering.

But I also worked with a few people who were the weakest developers I'd seen in my entire career. They would show up at work around noon, eat the free lunch, browse the web, attend a few meetings, and generally be gone by 6-ish. In the 4 months I worked with them, they had each written a couple of hundred lines of code, and copied and pasted a few hundred lines more. I don't miss that project at all.

I don't miss Linux very much either - especially the C/C++ development tools. I now understand why so many smart people that I met at Google said that they are never going back to C++ after Java. I do not miss debugging with printfs.

The fact that Google had either reimplemented or wrapped every OS API - from thread and process creation to HTTP protocol handlers and RPC - did not help me appreciate the beauty of this extremely lean operating system :-). Yes, the OS was small, fast, (insert your own epithet describing efficiency here)...

But the applications that compensated for this leanness ended up being massive lumbering beasts, requiring hundreds of megabytes of RAM and a minute or more to start. And it was not because these apps were written by idiots - no, the people who created them were actually quite good. It was because they had to do a lot of things that the "bloated" Windows OS does for you.

I miss Python somewhat. It is a nice, effective script language that I learned to love. Python is a corenerstone in Google's infrastructure - everything, from the check-in tool to makefiles, is written in Python. The best thing about Python is its consistency - you can clearly see that the language had one designer, who actually DESIGNED it, rather than just stuffing random unrelated features in a la C++ (or Perl).

I think Linux people should just adopt Python as "the one script engine," and dispense with the proliferation of *sh interpreters with their horrible, horrible scripting support that leaves no doubt that it was simply bandaged on top of something that was never designed to do scripting. And after 25 years, the bandage is wearing off and the puss is seeping all around it.

But on the other hand, there is nothing - at least on Windows - that is doable with Python and cannot be done with C# in an equivalent amount of time, or faster. So while I like the language a lot, I found myself mostly using C# anyway because of it's "built-in" status on Windows and really nice Visual Studio support.

There is one thing at Google that I miss a lot, without any qualifications, with no "if"s or "but"s. It's Mondrian, the Google code review system.

It was written by Guido van Rossum, the creator of Python. It was his starter project, and it demonstrates another admirable thing that Google does, which is investing very heavily in development productivity. I would say that by creating Mondrian, Guido single-handedly improved the quality of Google code by at least 20%.

Guido describes it here: http://video.google.com/videoplay?docid=-8502904076440714866 in great detail.

To summarize this presentation in just a few sentences: you create a change list in perforce. It automatically shows on the Mondrian web site. You go to this web and request code reviews by typing people's aliases. The mail gets sent with the link to the change list representation on this web site.

The web site shows - for every file in the change list - what it was before and what it is now, side by side. To comment, you click anywhere and start typing. The comment gets attached to the line of source on which you clicked. When you're done, submit the result, and the reviewee as well as the other reviewers are notified, and can see your comments on the same web site, right inside the files that are reviewed.

The reason Mondrian works so well is because the cost of making a comment is nearly zero - which is not true for over-the-shoulder code reviews, or trading files - where someone needs to take a note, fix the code, and then the reviewer has no easy way of verifying that the suggestion has actually been followed on. The point, click, and type UI paradigm that has been successfully driving the computer industry for so many years, has proven itself indispensable once again.

Unfortunately, Mondrian is an internal Google tool, not accessible from the outside. And even if it were, it is built on internal Google infrastructure, so it will be a cold day in hell before software organizations begin sending all of their code through Google servers. Obviously, this is not likely to happen at Microsoft :-).

So now that we've reached the end of M0 and a vast majority of my team is on their well-deserved vacations, I have a bunch of free time that I can spend on toying around. I have started playing with building my own Mondrian-like system that you can actually deploy on stock software such as IIS and SQL Server - stuff that is easily available to developers worldwide.

I have several objectives.

First, I want to learn ASP.NET and SQL. For the most part of my career, I was a very low-level developer - I know a lot about operating systems, implementation of compilers, low-level network protocols and file systems. I don't know very much about high-level abstractions - I have never written anything serious in C#, my experience building web services is confined to Google's infrastructure (and as such is not very useful outside Google), and I had never used a database until about a month ago.

Second, I want my team to have a nice code review system. We've had a few attempts at reusing tools that had already existed at Microsoft, but they all proved to be either very inconvenient, or very unstable. A good functioning code review system is a boon to developer productivity. This much I learned at Google.

Third, I think the world will benefit from a nice local, retargetable code review system that can be built on top of widely available software components. So when I am almost finished, I will be releasing it on Codeplex - Microsoft's shared code repository (http://www.codeplex.com/).

Meanwhile, you can read about my progress here. :-)

Update: It's up and running. We've used it for a month and a half, for more than 200 code reviews, 2000 files, and 3000 comments. Get it here: http://www.codeplex.com/Malevich.

Interview with a Chinese banker

"Individually, everyone needs to be compensated. But collectively, this directs the resources of the country. It distorts the talents of the country. The best and brightest minds go to lawyering, go to M.B.A.s. And that affects our country, too! Many of the brightest youngsters come to me and say, “Okay, I want to go to the U.S. and get into business school, or law school.” I say, “Why? Why not science and engineering?” They say, “Look at some of my primary-school classmates. Their IQ is half of mine, but they’re in finance and now they’re making all this money.” So you have all these clever people going into financial engineering, where they come up with all these complicated products to sell to people."

http://www.theatlantic.com/doc/print/200812/fallows-chinese-banker

Stuck in the Middle Ages

No wonder we're still debating whether evolution is real or not.

We're using the British Imperial measurement system that results in pearls like the following:

"To examine innovation in steam technology, we need a measure of how good a steam engine is. One important measure is the amount of work delivered by a given amount of fuel. This can be measured by the duty of a steam engine: the number of pounds of water that can be lifted one foot for each 94 pounds of coal consumed."

http://www.fee.org/publications/the-freeman/article.asp?aid=8370.

Let's try it again: duty of a steam engine is the number of pounds of water that can be lifted one foot for each 94 pounds of coal consumed.

Think how hard physics - no, any science! - in these units...