Time sensitive material

I received this prospectus, dated 12/2009, yesterday.

World's most inefficient way to check for primeness (via Reddit)

http://www.noulakaz.net/weblog/2007/03/18/a-regular-expression-to-check-for-prime-numbers/

Fire your perl interpreter, feed this function 982451653, and watch your computer die a slow and painful death...

The only democracy in the Middle East (TM) (via Reddit)

A Palestinian man has been convicted of rape after having consensual sex with an Israeli woman who believed he was Jewish because he introduced himself as "Daniel".

http://www.telegraph.co.uk/news/worldnews/middleeast/israel/7901025/Palestinian-jailed-for-rape-after-claiming-to-be-Jewish.html

Inception

Watched the highly acclaimed "Inception" on Friday. Below is my take.

Short version: "Titanic" is better.

And I don't say this because I like "Titanic".

My two favorite stories about "Titanic" are as follows. When the movie was first released on VHS, the MicroNews - internal MS paper that back then was a print edition - published this classified ad: "Titanic on VHS. First tape watched once. Second tape never watched".

Second story: among Windows developers Intel's Itanium chip is known as "Itanic". Although this one has nothing to do with the movie...

Long version: If you consider "Matrix" brainy and captivating, "Inception" is for you!

It is extremely similar to "Matrix". Both movies take a trivial idea ("Matrix": reality is actually a computer generated dream; "Inception": reality is influenced by a human-induced shared dreams) and make a movie out of it by adding large number of superficialities.

In "Matrix" it's tough looking impeccably clothed men beating main heroes up in highly choreographed fight scenes. I guess it must be the Hollywood's idea of what's going on inside a computer. I suppose if I had majored in communications, I would have thought about computers like that, too.

In "Inception" it is impeccably "architected" dreamscapes where the heroes get to confront - and conquer! - their demons. I guess it must be the Hollywood's idea of what's going on inside a human brain. I suppose if I had majored in said communications, I would have thought about the mechanics of human brain like that as well.

Why do I think "Titanic" is better? There, if one abstracts from the plot, one still gets to enjoy the vistas. In "Inception", unfortunately, there is no way to abstract from the plot...

How to make a service in .NET

Here's a complete, self-contained way to build a system service using .NET. I was looking for a way to do it on the internets, but most of the examples rely on .NET template (which relies on designer, which is ugly) and don't have a way to install the service programmatically.

Without much ado, here's the code. All of it. Just replace ServiceMainThread with your code, and you're done. It even supports installing multiple instances of itself.

//-----------------------------------------------------------------------
// <copyright>
// Copyright (C) Sergey Solyanik.
//
// This file is subject to the terms and conditions of the Microsoft Public License (MS-PL).
// See http://www.microsoft.com/opensource/licenses.mspx#Ms-PL for more details.
// </copyright>
//----------------------------------------------------------------------- 
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Runtime.InteropServices;
using System.ServiceProcess;
using System.Text;
using System.Threading;

namespace CSServiceHost
{
    /// <summary>
    /// Main service class.
    /// </summary>
    public class MyService : ServiceBase
    {
        /// <summary>
        /// The thread that contains the execution path for the service.
        /// </summary>
        Thread runner;

        /// <summary>
        /// Event which gets signalled when the service stops.
        /// </summary>
        EventWaitHandle stop;

        /// <summary>
        /// Processes the start event for service.
        /// </summary>
        /// <param name="args"></param>
        protected override void OnStart(string[] args)
        {
            stop = new EventWaitHandle(false, EventResetMode.ManualReset);
            runner = new Thread(ServiceMainThread);
            runner.Start();
        }

        /// <summary>
        /// Processes the stop event for service.
        /// </summary>
        protected override void OnStop()
        {
            stop.Set();
            runner.Join();
        }

        /// <summary>
        /// The main entry point for the application.
        /// </summary>
        /// <param name="args"> Program arguments. See help.</param>
        static void Main(string[] args)
        {
            if (args.Length > 0)
            {
                ProcessServiceCommand(args);
                return;
            }

            ServiceBase[] ServicesToRun;
            ServicesToRun = new ServiceBase[] 
            { 
                new MyService() 
            };

            ServiceBase.Run(ServicesToRun);
        }

        /// <summary>
        /// Executes service installation/uinstallation, or runs it as a process.
        /// </summary>
        /// <param name="args"> Program arguments.</param>
        private static void ProcessServiceCommand(string[] args)
        {
            string exe = Assembly.GetExecutingAssembly().Location;

            if ("/install".Equals(args[0], StringComparison.OrdinalIgnoreCase))
            {
                object[] assemblyAttributes =
                    Assembly.GetExecutingAssembly().GetCustomAttributes(false);

                string instance =
                    (from a in assemblyAttributes
                     where a is AssemblyTitleAttribute
                     select ((AssemblyTitleAttribute)a).Title).SingleOrDefault();
                string name =
                    (from a in assemblyAttributes
                     where a is AssemblyDescriptionAttribute
                     select ((AssemblyDescriptionAttribute)a).Description).SingleOrDefault();
                string account = null;
                string password = String.Empty;
                for (int i = 1; i < args.Length; ++i)
                {
                    if (args[i].StartsWith(
                        "/instance:", StringComparison.OrdinalIgnoreCase))
                    {
                        instance = args[i].Substring(10);
                    }
                    else if (args[i].StartsWith(
                        "/name:", StringComparison.OrdinalIgnoreCase))
                    {
                        name = args[i].Substring(6);
                    }
                    else if (args[i].StartsWith(
                        "/account:", StringComparison.OrdinalIgnoreCase))
                    {
                        account = args[i].Substring(9);
                    }
                    else if (args[i].StartsWith(
                        "/password:", StringComparison.OrdinalIgnoreCase))
                    {
                        password = args[i].Substring(10);
                    }
                    else
                    {
                        Console.Error.WriteLine("Could not parse: {0}", args[i]);
                    }
                }

                InstallService(exe, instance, name, account, password);
            }
            else if ("/uninstall".Equals(
                args[0], StringComparison.OrdinalIgnoreCase))
            {
                object[] assemblyAttributes =
                    Assembly.GetExecutingAssembly().GetCustomAttributes(false);
                string instance =
                    (from a in assemblyAttributes
                     where a is AssemblyTitleAttribute
                     select ((AssemblyTitleAttribute)a).Title).SingleOrDefault();

                for (int i = 1; i < args.Length; ++i)
                {
                    if (args[i].StartsWith(
                        "/instance:", StringComparison.OrdinalIgnoreCase))
                    {
                        instance = args[i].Substring(10);
                    }
                    else
                    {
                        Console.Error.WriteLine("Could not parse: {0}", args[i]);
                    }
                }

                UninstallService(instance);
            }
            else if ("/run".Equals(args[0], StringComparison.OrdinalIgnoreCase))
            {
                MyService service = new MyService();
                service.OnStart(new string[0]);
                Console.WriteLine("Service is running as a process.");
                Console.WriteLine("Press <ENTER> to stop and exit.");
                Console.ReadLine();
                service.OnStop();
            }
            else
            {
                Console.WriteLine("To install service:");
                Console.WriteLine("    {0} /install", exe);
                Console.WriteLine("        [/instance:instance_name [/name:display_name]]");
                Console.WriteLine("        [/account:account [/password:password]]");
                Console.WriteLine("To uninstall service:");
                Console.WriteLine("    {0} /uninstall [/instance:instance_name]", exe);
                Console.WriteLine("To run as a regular process:");
                Console.WriteLine("    {0} /run", exe);
            }
        }

        /// <summary>
        /// Installs service.
        /// </summary>
        /// <param name="exe"> Path to the executable. </param>
        /// <param name="instance"> Name of the service instance. </param>
        /// <param name="name"> Display name of the service. </param>
        /// <param name="account"> Account name or NULL for LocalSystem. </param>
        /// <param name="password"> Password or empty string if any of
        /// the machine accounts. </param>
        private static void InstallService(
            string exe,
            string instance,
            string name,
            string account,
            string password)
        {
            IntPtr scm = Win32.OpenSCManager(
                null, null, Win32.SC_MANAGER_CREATE_SERVICE);
            if (scm.ToInt32() == 0)
            {
                Console.Error.WriteLine(
                    "Failed to open SCM (error {0}).", Win32.GetLastError());
                return;
            }

            try
            {
                IntPtr service = Win32.CreateService(
                    scm,
                    instance,
                    name,
                    Win32.SERVICE_ALL_ACCESS,
                    Win32.SERVICE_WIN32_OWN_PROCESS,
                    Win32.SERVICE_AUTO_START,
                    Win32.SERVICE_ERROR_NORMAL,
                    exe,
                    null,
                    0,
                    null,
                    account,
                    password);

                if (service.ToInt32() == 0)
                {
                    Console.Error.WriteLine(
                        "Failed to create service (error {0}).",
                        Win32.GetLastError());
                    return;
                }

                try
                {
                    if (Win32.StartService(service, 0, null) == 0)
                    {
                        Console.Error.WriteLine(
                            "Failed to start service (error {0}).",
                            Win32.GetLastError());
                    }
                    else
                    {
                        Console.WriteLine(
                            "Service installed successfully.");
                    }
                }
                finally
                {
                    Win32.CloseServiceHandle(service);
                }
            }
            finally
            {
                Win32.CloseServiceHandle(scm);
            }
        }

        /// <summary>
        /// Uninstalls service.
        /// </summary>
        /// <param name="instance"> Service instance. </param>
        private static void UninstallService(string instance)
        {
            IntPtr scm = Win32.OpenSCManager(
                null, null, Win32.SC_MANAGER_ALL_ACCESS);
            if(scm.ToInt32() == 0)
            {
                Console.Error.WriteLine(
                    "Failed to open SCM (error {0}).",
                    Win32.GetLastError());
                return;
            }

            try
            {
                IntPtr service = Win32.OpenService(
                    scm, instance, Win32.DELETE | Win32.SERVICE_STOP);
                if (service.ToInt32() == 0)
                {
                    Console.Error.WriteLine(
                        "Failed to open service (error {0}).",
                        Win32.GetLastError());
                    return;
                }

                try
                {
                    Win32.SERVICE_STATUS stat;
                    if (0 == Win32.ControlService(
                        service, Win32.SERVICE_CONTROL_STOP, out stat))
                    {
                        Console.Error.WriteLine(
                            "Could not stop the service (error {0}).",
                            Win32.GetLastError());
                    }

                    while (Win32.QueryServiceStatus(service, out stat) != 0
                        && stat.dwCurrentState != Win32.SERVICE_STOPPED)
                    {
                        Thread.Sleep(1000);
                    }

                    if (Win32.DeleteService(service) == 0)
                    {
                        Console.Error.WriteLine(
                            "Failed to delete service (error {0}).",
                            Win32.GetLastError());
                    }
                    else
                    {
                        Console.WriteLine(
                            "Service successfully uninstalled.");
                    }
                }
                finally
                {
                    Win32.CloseServiceHandle(service);
                }
            }
            finally
            {
                Win32.CloseServiceHandle(scm);
            }
        }

        /// <summary>
        /// The actual logic.
        /// </summary>
        private void ServiceMainThread()
        {
            for (; ; )
            {
                if (stop.WaitOne(10000))
                {
                    break;
                }

                using (StreamWriter w =
                    new StreamWriter(@"c:\temp\testservice.txt", true))
                    w.WriteLine(
                        "Tick {0} {1}",
                        DateTime.Now,
                        Environment.UserName);
            }
        }

        /// <summary>
        /// Win32 thunks.
        /// </summary>
        private static class Win32
        {
            public const UInt32 SC_MANAGER_ALL_ACCESS = 0xF003F;
            public const UInt32 SC_MANAGER_CREATE_SERVICE = 0x0002;

            public const UInt32 SERVICE_WIN32_OWN_PROCESS = 0x00000010;
            public const UInt32 SERVICE_AUTO_START = 0x00000002;
            public const UInt32 SERVICE_ERROR_NORMAL = 0x00000001;

            public const UInt32 STANDARD_RIGHTS_REQUIRED = 0xF0000;
            public const UInt32 SERVICE_QUERY_CONFIG = 0x0001;
            public const UInt32 SERVICE_CHANGE_CONFIG = 0x0002;
            public const UInt32 SERVICE_QUERY_STATUS = 0x0004;
            public const UInt32 SERVICE_ENUMERATE_DEPENDENTS = 0x0008;
            public const UInt32 SERVICE_START = 0x0010;
            public const UInt32 SERVICE_STOP = 0x0020;
            public const UInt32 SERVICE_PAUSE_CONTINUE = 0x0040;
            public const UInt32 SERVICE_INTERROGATE = 0x0080;
            public const UInt32 SERVICE_USER_DEFINED_CONTROL = 0x0100;
            public const UInt32 SERVICE_ALL_ACCESS = STANDARD_RIGHTS_REQUIRED |
                SERVICE_QUERY_CONFIG | SERVICE_CHANGE_CONFIG |
                SERVICE_QUERY_STATUS | SERVICE_ENUMERATE_DEPENDENTS |
                SERVICE_START | SERVICE_STOP | SERVICE_PAUSE_CONTINUE |
                SERVICE_INTERROGATE | SERVICE_USER_DEFINED_CONTROL;

            public const UInt32 DELETE = 0x10000;

            public const UInt32 SERVICE_CONTROL_STOP = 0x00000001;
            public const UInt32 SERVICE_STOPPED = 0x00000001;

            [StructLayout(LayoutKind.Sequential)]
            public struct SERVICE_STATUS
            {
                public UInt32 dwServiceType;
                public UInt32 dwCurrentState;
                public UInt32 dwControlAccepted;
                public UInt32 dwWin32ExitCode;
                public UInt32 dwServiceSpecificExitCode;
                public UInt32 dwCheckPoint;
                public UInt32 dwWaitHint;
            };

            [DllImport("advapi32.dll")]
            public static extern IntPtr OpenSCManager(
                string lpMachineName,
                string lpSCDB,
                UInt32 scParameter);
            
            [DllImport("advapi32.dll")]
            public static extern IntPtr CreateService(
                IntPtr SC_HANDLE,
                string lpSvcName,
                string lpDisplayName,
                UInt32 dwDesiredAccess,
                UInt32 dwServiceType,
                UInt32 dwStartType,
                UInt32 dwErrorControl,
                string lpPathName,
                string lpLoadOrderGroup,
                int lpdwTagId,
                string lpDependencies,
                string lpServiceStartName,
                string lpPassword);

            [DllImport("advapi32.dll")]
            public static extern void CloseServiceHandle(IntPtr SCHANDLE);

            [DllImport("advapi32.dll")]
            public static extern int StartService(
                IntPtr SVHANDLE,
                UInt32 dwNumServiceArgs,
                string lpServiceArgVectors);

            [DllImport("advapi32.dll", SetLastError = true)]
            public static extern IntPtr OpenService(
                IntPtr SCHANDLE,
                string lpSvcName,
                UInt32 dwNumServiceArgs);

            [DllImport("advapi32.dll", SetLastError = true)]
            public static extern int ControlService(
                IntPtr SCHANDLE,
                UInt32 dwControl,
                [MarshalAs(UnmanagedType.Struct)]
                out SERVICE_STATUS lpServiceStatus);

            [DllImport("advapi32.dll", SetLastError = true)]
            public static extern int QueryServiceStatus(
                IntPtr SCHANDLE,
                [MarshalAs(UnmanagedType.Struct)]
                out SERVICE_STATUS lpServiceStatus);

            [DllImport("advapi32.dll")]
            public static extern int DeleteService(IntPtr SVHANDLE);

            [DllImport("kernel32.dll")]
            public static extern int GetLastError();
        }
    }
}

Microsoft Mini

Microsoft Mini blog has a lot of angst today with new round of layoffs at Microsoft.

http://minimsft.blogspot.com/2010/07/kin-fusing-kin-clusion-to-kin-and-fy11.html

I tried to comment, but the Blogger was broken and wouldn't accept it. Since I already typed it up, I am posting it here instead...

I worked at Microsoft for 10 years, then went to Google, then back to Microsoft.

All companies have problems. Apple. Google. Microsoft. They are just different problems, and they look bigger when you are closer to them.

Yes, Microsoft is failing in consumer markets, always have, maybe always will. We don't get consumer. Vista (AKA Abby & Toby platform), Windows Mobile, Kin... Even XBox - it is successful because it was built more for a typical Microsoft employee than for a regular person, it's just MS people love the same kind of videogames that 14 years old males do :-). But if you look at kids or family games on XBox - total failure.

The problem is that we target some mythical "dumb" customer, we don't really know who that is, and we overshoot the level of dumbness by a wide margin. I worked on the first version of Windows Home Server and we had people on the team - not developers, obviously - who seriously tried to argue that our customers don't know what a file share is. I kid you not.

However, just like Microsoft doesn't get consumers, Google and Apple don't get the enterprise. I have participated in creation of a business product at Google, and the people around me did not understand basic concepts like the need for customer service, a refund process, or the like. The entire Google internal system is antithetic to schedule predictability and release stability that is required for a corporate product. People who say that Gmail and Google Docs somehow threaten Exchange and Office have obviously never used these products in a work setting for an extended period of time.

One thing that is going for Microsoft is the plentitude of cultures. We have Xbox team, and an Office team, and Windows team, and Bing, and all these organizations are as unlike each others as they can be. My advice to people, especially developers, who complain about politics, poor managers, boring products, etc - check out the career site! Plenty of teams are hiring, and there are tons of really fun places which will match your preferred style, values, or culture. Just keep moving until you find the right place for YOU. Trust me, it does exist. (By the way, I am hiring, too! If you dream in code and can implement a semaphore if I woke you up at 3am, and like a blend of "old Microsoft" and "new Google" cultures, look me up on career site!)

Finally, it is true that often leaders make companies/armies/countries great. But not always, and never alone, and certainly not in democratic societies :-). I don't think that success of Microsoft in the 90's is directly attributable to BillG and BillG alone, and the steam somehow magically went out of the company the day he left the building. Yes, we have plenty of people at the high places that probably should not have been there. So does Google, so does Apple, so does Oracle, Intel, ..., ..., ... These aren't the people who (most of) you work with, they aren't the people who you meet every day, and I'll let you in on a secret - they aren't the people who make YOUR product a success or a failure. YOU do. They can't affect the stock price much - YOU can, by shipping great products, and by making environment around YOU better, so it attracts more people like YOU.

So don't get consumed by paranoia and politics, focus on your job and your team, and everything else will follow.

And if not, as long as you do the above, you will still be very employable. At Google, at Amazon, or in my team :-).

Republican party, circa 1956

Republican Party Platform of 1956

http://www.presidency.ucsb.edu/ws/print.php?pid=25838

This is what DemocraticUnderground.com had to say about it:

"By these standards, modern Democrats have become Republicans, and modern Republicans have become batshit crazy. I think I’ve heard Kucinich mention Taft-Hartley, but you get a blank stare if you mention it to anybody under the age of 40. And to think that it used to be a topic of polite conversation among the political classes only 54 years ago."

This: http://journals.democraticunderground.com/eridani/449 has TL;DR.

Incidentally, I am hearing that the health care reform passed by Obama is very, very similar to the Republican counter-offer to the Clinton health plan.

Microsoft by the numbers

"The reports of my death have been greatly exaggerated."
-- Mark Twain

http://www.businessinsider.com/microsoft-by-the-numbers-2010-6

Scott Adams on investing (via Reddit)

Absolutely hilarious!

http://online.wsj.com/article/SB10001424052748704025304575285000265955016.html

Terrorism...

Thirty years ago Chief of Staff Mordechai Gur observed that since 1948, "we have been fighting against a population that lives in villages and cities." As Israel's most prominent military analyst, Zeev Schiff, summarized his remarks, "the Israeli Army has always struck civilian populations, purposely and consciously...the Army, he said, has never distinguished civilian [from military] targets...[but] purposely attacked civilian targets."

http://www.chomsky.info/articles/20090119.htm

Arabs surely hate us for our freedom...

Israel should lead investigation into attack on Gaza flotilla, says US

More guns and idiots...

http://seattletimes.nwsource.com/html/localnews/2011994648_accident31m.html

Can we stuff 2000 calories in one drink? Yes, we can! (via Reddit)

This is really, truly horrible: http://worldmysteries9.blogspot.com/2010/05/harmful-drinks-in-america.html

The worst-worst drink has 2,010 calories, 131 g fat (68 g saturated), 153 g sugars (this is the number of calories a human should consume in a day; and more fat than one probably should have in a week).

#4 on the list has 1,210 calories, 19 g fat (10 g saturated), and 240 g sugars! Yes, this is really 1/4 of a kg. In one "drink"!!!

Server 200x RAID tips and tricks

Since 2003, Windows Server was shipping with a very cool feature: software RAID.

Software RAID has two major advantages over hardware.

First, hardware RAID protects you against the disk failure, but not against the controller failure. The drive array contains proprietary disk allocation information that varies from manufacturer to manufacturer and controller to controller, so disks are not easily movable between different controllers. So when your controller fails - potentially, long time in the future, when the same boards are no longer available, - or a new release of OS stops supporting your older driver, you may be very much out of luck with your data.

Second, software RAID is considerably more flexible than hardware RAID. Hardware RAID operates on disks as atomic units - you RAID the whole disks together. Software RAID operates on volumes, and each volule can be configured with different level of redundancy. For example, you can have 2 constructs on two disks at the same time - an OS partition that is RAID-1 mirrored with an image on the second disk, and another volume that combines the rest of the space on two drives as a single span or RAID-0. The level of redundancy is selected per volume, not per disk.

Also, for those of us who like coding, software RAID has a very nice software interface (http://msdn.microsoft.com/en-us/library/bb986750(v=VS.85).aspx and its undocumented managed counterpart in Microsoft.Storage.Vds.dll) which allows one to code simple things like checking the health of the storage and send an email if something goes bad.

But what about performance? A while ago when we were designing Windows Home Server, we tested various hardware RAID implementations versus software RAID in Server 2003.

It turns out that both RAID-0 and RAID-1 exhibit very similar performance for both hardware and software solutions. If you think about what the system has to do (write the same data to two disks at the same time in the case of RAID-1) it quickly becomes obvious that hardware implementation does not really add anything over the software in this case: both can write the same data in two places at the same time with the same speed. Big surprise :-).

RAID-5 is a different beast though - there actually is a computation going on, and it is possible to build a specialized chip for doing vector XOR operations that would leave the general purpose x86 in the dust.

A much bigger problem also exists in the lack of integration between the formatting and the RAID code. When you format a RAID drive, the default allocation unit that the UI presents is very small.

Due to the way the software RAID is implemented, it leads to incredibly slow performance. On my relatively powerful system the writes clock at only 20-30MBps (this going to the drives that are supposed to sustain 3Gbps, or 300MBps transfers). Selecting a more reasonable allocation unit of 64k improves the write speed by a factor of four, to almost 120MBps.

The other performance problem that is format related is after creating a new RAID volume, the default behavior is that format and resync happen at the same time. I covered it in the previous blog post here: http://1-800-magic.blogspot.com/2010/05/solution-to-slow-formatting-puzzle.html.

In summary, here are the two very simple rules can make your RAID array much faster:
- Select the 64k as a default allocation unit when formatting the RAID-5 volume.
- When formatting any new RAID volume, use quick format first, wait for the volume to finish resyncing, then repeat with full format if you like (remember to keep the large allocation unit in the second format though!)

Happy RAIDing!

Solution to the slow formatting puzzle

A couple of weeks back I posted a puzzle about an experience that I just had with Server 2008 R2 software RAID subsystem: somehow the speed of formatting a new very large RAID-5 array was highly unpredictable. It advanced by a mere 12% a day for the 4 days and then suddenly sped up 4x and finished the last 50% of the formatting in just one day.

Furthermore, I expected precisely this behavior. Why?

This weirdness occurs because of the typical Microsoft phenomenon that was best expressed by an acquaintance of mine that works in Windows Mobile. He thought that the biggest difference between Microsoft's and Apple's approaches to development are these:

Microsoft attacks problems horizontally: it builds core system, then builds layers on top of it. When it's time to ship and something needs to get cut, it's the top of the stack that goes first - and more often than not it's the user experience.

Apple develops software vertically - it enables a user experience, from top to bottom. When they need to cut, they cut the entire experiences (and also maybe the bugs, judging from the fact that my iPhone seems to crash considerably more often than most of Windows Mobile phones I owned in the past). So for example, iPhone would ship with Bluetooth support just for the mono phone headset, but no stereo profile. But the experiences that are left are implemented completely, to the maximum possible level of usability.

Back to our RAID problem. The reason the formatting is so slow in the very beginning is because two things happen at the same time: RAID resync and the formatting. Resync gets kicked off immediately after the RAID system is built, and it simply ensures that the parity volume (in the case of RAID-5) has correct checksums, or the mirrored volume (in the case of RAID-1) has a correct copy of the primary volume.

The other thing that happens once you create RAID volume from the UI is, of course, formatting. One can select quick format which only creates a file system. However, most people probably prefer to run full format when a new drive is added, just to make sure that it is not full of bad sectors.

So we have two write operations that are going on - first one is creating the redundant information, the other is filling the disk with zeroes.

Writing to the disk is very fast these days, but the seek time has barely improved in the last decade, and because the two writes happen in different places on the disk, the whole thing is completely dominated by the disk head moving from one track to another. And with a seek time of 15ms you can only have ~70 of these per second.

Of course, if the format is in progress, there is absolutely no sense to do a resync at the same time - whatever redundancy data resync creates is going to be instantly overwritten by the format.

But there was no Steve Jobs standing over the devs' shoulder, and getting the (filesystem) format in sync with (block device) RAID required two different teams to do something together so it probably got punted. I am sure it's in in a readme somewhere... and now in at least one blog :-).

So to avoid unnecesary delays, select quick format option when adding the RAID volume, wait for the resync to finish, THEN format it again with the full format.

Formatting the large hard drive - continued

As mentioned in the previous post, one of my servers is formatting a 7.5TB RAID-5 volume.

During the first 24 hours, it has completed 13%.
During the next 24 hours, the progress was at 26%.
Next day, it was at 38%, and the day after (last night) it clocked at 50%.
However, this morning, 12 hours later, it was at 75%, and tonight I expect it to be completed.

So in the very last day the system's progress was the same as for the first 4 days. Moreover, I fully expected this to be the case. Can anyone guess why?

Formatting a large RAID-5 drive

Old hard drives - I have had them for 3 years - on my servers at home started to fail, so I had to replace one RAID-5 array. I bought 5 2TB drives and used software RAID-5 function in Windows Server 2008 R2 to create a monstrous 7.5TB (usable space) drive out of these 5 disks.

I only use soft RAID because the disks can be read by any Windows Server, whereas hardware RAID protects well against a drive failure, but if it is the controller that dies, you may be out of luck - who knows if this model would be even available a few years down the road, and most controllers use proprietary data formats, so the disk arrays are not portable between them.

Anyway, I built the array and starting formatting it. After the first 24 hours, the format was 13% complete. Next day (today) it is up to 26%.

Puzzle for the readers - how long do you think the format is going to take in total? Do not post the reasoning, just the number :-). I will post the final answer (and why) when the format completes.

Dictionary attacks on my home network

I am running Small Business Server 2003 at home. One service that it provides is sending daily reports about what happened to the system in the last 24 hours.

This report includes summaries of the event logs, and in the last several days the security logs were overflowing with logon audit failures caused by what looks like a distributed dictionary attack on my server.

The attack goes as follows: for a few hours someone (a script, really) is trying to connect to the system using various "likely" account names like aloha, admin, Administrator, master, root, randy, etc. The logon attempt for every user name repeats a few dozen times, most likely with different passwords, although security logs obviously don't show them, and then the user name changes.

After a few thousand attempts the attack subsides only to resume the next day from a different IP address.

Obviously, all the passwords used by our family are rather complex, and it is very unlikely that the thing will ever guess them, but it's discomforting nevertheless, so today I decided to get rid of the attackers altogether.

Small Business Server 2003 does not include standard Windows Firewall, instead it uses filters in its routing service to block unwelcome traffic.

The UI for this "firewall" of sorts can be accessed via Routing and Remote Access MMC available in the Administrative Tools menu. From there, one can expand IP Routing, the NAT/Basic Firewall, and double-clicking on the interface name brings up a properties dialog box with the "Inbound Filters" button.

I had no idea how to access it programmatically. Luckily, most of Windows networking management is scriptable through netsh, and netsh has a very nice property called dump which prints out a script that could be used to reproduce the current configuration.

So I created a rule with the fake firewall rule with an address that I could recognize, ran netsh dump, and searched the output for that ip address. As it turns out then, to add a firewall rule on Server 2003 with routing enabled, this is what needs to be done:

routing ip add filter name="Network Connection"
    filtertype=INPUT srcaddr=xx.xx.xx.xx srcmask=255.255.255.255
    dstaddr=0.0.0.0 dstmask=0.0.0.0 proto=ANY

where xx.xx.xx.xx is the placeholder for the ip address.

Poking around a bit more, it turned out that to list the existing rules (thus determining which IP addresses have already been blocked), one could do this:

routing ip show filter name="Network Connection"

The output is a table which can easily be picked apart with a regular expression:

private static Regex presentIps = new Regex(
    @"^\s*(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s+255.255.255.255\s+.*$",
    RegexOptions.Multiline);
private static HashSet<string> GetCurrentlyBlockedIps()
{
    string filters = NetSh(
        "routing ip show filter name=\"Network Connection\"");
    MatchCollection allMatches = presentIps.Matches(filters);
    HashSet<string> blockedIps = new HashSet<string>();
    foreach (Match m in allMatches)
    {
        Console.WriteLine("This ip is currently being blocked: {0}",
            m.Groups[1].Value);
        blockedIps.Add(m.Groups[1].Value);
    }
    return blockedIps;
}

NetSh() is the function that runs netsh.exe. It's a bit complex because it reads both STDOUT and STDERR. This has to be done asynchronously or else a read operation on STDOUT might block while STDERR overflows:

private delegate string ReaderDelegate();
private static string NetSh(string arguments)
{
    Process process = new Process();
    process.StartInfo.Arguments = arguments;
    process.StartInfo.CreateNoWindow = true;
    process.StartInfo.FileName = "netsh.exe";
    process.StartInfo.RedirectStandardError = true;
    process.StartInfo.RedirectStandardOutput = true;
    process.StartInfo.UseShellExecute = false;

    process.Start();

    ReaderDelegate stdoutReader =
        new ReaderDelegate(process.StandardOutput.ReadToEnd);
    ReaderDelegate stderrReader =
        new ReaderDelegate(process.StandardError.ReadToEnd);
    IAsyncResult stdoutResult = stdoutReader.BeginInvoke(null, null);
    IAsyncResult stderrResult = stderrReader.BeginInvoke(null, null);

    WaitHandle[] handles =
    {
        stdoutResult.AsyncWaitHandle,
        stderrResult.AsyncWaitHandle
    };

    if (!WaitHandle.WaitAll(handles))
        throw new Exception("netsh.exe was aborted");

    string stdout = stdoutReader.EndInvoke(stdoutResult);
    string stderr = stderrReader.EndInvoke(stderrResult);

    process.WaitForExit();

    if (!string.IsNullOrEmpty(stderr))
    {
        Console.Error.WriteLine(
            "Failed netsh {0}", process.StartInfo.Arguments);

        if (stdout != null)
            Console.Error.WriteLine("{0}", stdout);

        Console.Error.WriteLine("{0}", stderr);

        throw new Exception("netsh.exe failed");
    }

    process.Dispose();

    return stdout;
}

We're almost there. All I now need to do is to walk the security event log, picking out the IP address of the attacker out of the relevant entries:

using (EventLog ev = new EventLog("Security"))
{
    EventLogEntryCollection entries = ev.Entries;
    for (int index = entries.Count - 1; index >= 0; index--)
    {
        EventLogEntry ele = entries[index];
...
        if (ele.CategoryNumber != 2 || ele.InstanceId != 529 ||
            ele.EntryType != EventLogEntryType.FailureAudit)
            continue;

        if (ele.ReplacementStrings.Length < 11 ||
            knownGoodIps.IsMatch(ele.ReplacementStrings[11]))
            continue;

        string ip = ele.ReplacementStrings[11];
...

If the logon failure happens, say, more then 5 times in two minutes, we simply block out the whole ip address from accessing the server, as follows (you'll notice that I maintain a queue of events for all relevant ip addresses that allows me to detect the number of failures per time interval):

if (blockedIps.Contains(ip))
    continue;

LinkedList<DateTime> queue;
if (!trackedIps.ContainsKey(ip))
{
    queue = new LinkedList<DateTime>();
    queue.AddFirst(ele.TimeGenerated);
    trackedIps[ip] = queue;

    continue;
}

queue = trackedIps[ip];
queue.AddFirst(ele.TimeGenerated);

if (queue.Count < failEvents)
    continue;

while (queue.Count > failEvents)
    queue.RemoveLast();

TimeSpan period = queue.Last.Value - queue.First.Value;
if (period.Milliseconds > failPeriod)
    continue;

blockedIps.Add(ip);

NetSh("routing ip add filter name=\"Network Connection\" filtertype=INPUT srcaddr="
    + ip + " srcmask=255.255.255.255 dstaddr=0.0.0.0 dstmask=0.0.0.0 proto=ANY");

The rest is just accounting. Here's the full program for your enjoyment:

//-----------------------------------------------------------------------
// <copyright>
// Copyright (C) Sergey Solyanik.
//
// This file is subject to the terms and conditions of the Microsoft Public License (MS-PL).
// See http://www.microsoft.com/opensource/licenses.mspx#Ms-PL for more details.
// </copyright>
//----------------------------------------------------------------------- 
using System;
using System.Diagnostics;
using System.Collections.Generic;
using System.Configuration;
using System.Linq;
using System.Net.Mail;
using System.Text;
using System.Text.RegularExpressions;
using System.Threading;

namespace Defender
{
    /// <summary>
    /// This implements code that scans security event log and uses Windows Server 2003
    /// ip routing filters to block dictionary attacks.
    /// </summary>
    class Program
    {
        /// <summary>
        /// Regular expression that parses existing filters out of
        /// netsh ip show filter output.
        /// </summary>
        private static Regex presentIps = new Regex(
            @"^\s*(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s+255.255.255.255\s+.*$",
            RegexOptions.Multiline);

        /// <summary>
        /// Delegate used to read process output.
        /// </summary>
        /// <returns> STDOUT/ERR stream converted to a string. </returns>
        private delegate string ReaderDelegate();

        /// <summary>
        /// Runs netsh.exe with the given argument. Throws if there is an error, else
        /// returns the output as one string.
        /// </summary>
        /// <param name="arguments"> netsh.exe command arguments. </param>
        /// <returns> STDOUT converted to one string. </returns>
        private static string NetSh(string arguments)
        {
            Process process = new Process();
            process.StartInfo.Arguments = arguments;
            process.StartInfo.CreateNoWindow = true;
            process.StartInfo.FileName = "netsh.exe";
            process.StartInfo.RedirectStandardError = true;
            process.StartInfo.RedirectStandardOutput = true;
            process.StartInfo.UseShellExecute = false;

            process.Start();

            ReaderDelegate stdoutReader =
                new ReaderDelegate(process.StandardOutput.ReadToEnd);
            ReaderDelegate stderrReader =
                new ReaderDelegate(process.StandardError.ReadToEnd);
            IAsyncResult stdoutResult = stdoutReader.BeginInvoke(null, null);
            IAsyncResult stderrResult = stderrReader.BeginInvoke(null, null);

            WaitHandle[] handles =
            {
                stdoutResult.AsyncWaitHandle,
                stderrResult.AsyncWaitHandle
            };

            if (!WaitHandle.WaitAll(handles))
            {
                throw new Exception("netsh.exe was aborted");
            }

            string stdout = stdoutReader.EndInvoke(stdoutResult);
            string stderr = stderrReader.EndInvoke(stderrResult);

            process.WaitForExit();

            if (!string.IsNullOrEmpty(stderr))
            {
                Console.Error.WriteLine(
                    "Failed netsh {0}", process.StartInfo.Arguments);

                if (stdout != null)
                    Console.Error.WriteLine("{0}", stdout);

                Console.Error.WriteLine("{0}", stderr);

                throw new Exception("netsh.exe failed");
            }

            process.Dispose();

            return stdout;
        }

        /// <summary>
        /// Gets the collection of ips that are already blocked.
        /// </summary>
        /// <returns> HashSet of blocked ips. </returns>
        private static HashSet<string> GetCurrentlyBlockedIps()
        {
            string filters = NetSh(
                "routing ip show filter name=\"Network Connection\"");
            MatchCollection allMatches = presentIps.Matches(filters);
            HashSet<string> blockedIps = new HashSet<string>();
            foreach (Match m in allMatches)
            {
                Console.WriteLine("This ip is currently being blocked: {0}",
                    m.Groups[1].Value);
                blockedIps.Add(m.Groups[1].Value);
            }

            return blockedIps;
        }

        /// <summary>
        /// Convers a comma-separated list of "known good" ips into a filtering
        /// regular expression.
        /// </summary>
        /// <param name="ipList">Comma-separated list of known-good ips.</param>
        /// <returns>Regular expression that matches this list.</returns>
        private static Regex BuildRegexForIpList(string ipList)
        {
            return new Regex(
                "^(" + ipList.Replace(".", "\\.").Replace(',', '|') + ").*$");
        }

        /// <summary>
        /// Sends notification email.
        /// </summary>
        /// <param name="mailBody"> Body of the email. </param>
        private static void SendMail(StringBuilder mailBody)
        {
            MailMessage email = new MailMessage();
            email.To.Add(ConfigurationSettings.AppSettings["NotificationEmail"]);
            email.Subject = "New attack(s) detected. IPs blocked.";
            email.From = new MailAddress(
                ConfigurationSettings.AppSettings["FromEmail"]);
            email.Sender = new MailAddress(
                ConfigurationSettings.AppSettings["FromEmail"]);
            email.Body = mailBody.ToString();
            email.IsBodyHtml = false;

            SmtpClient client = new SmtpClient(
                ConfigurationSettings.AppSettings["SmtpServer"]);
            client.UseDefaultCredentials = true;
            client.EnableSsl = bool.Parse(
                ConfigurationSettings.AppSettings["UseSslForSmtp"]);

            client.Send(email);
        }

        /// <summary>
        /// Runs one round of periodic processing.
        /// </summary>
        /// <param name="lastProcessedIndex"> Previously seen event index. Processes
        /// all events that are newer than this. </param>
        /// <returns> The new watermark for event index. </returns>
        private static int Process(int lastProcessedIndex)
        {

            StringBuilder mailBody = new StringBuilder();

            int failEvents = int.Parse(
                ConfigurationSettings.AppSettings["LogonFailuresPerPeriod"]);
            int failPeriod = 1000 * 60 * int.Parse(
                ConfigurationSettings.AppSettings["FailurePeriodMinutes"]);
            Regex knownGoodIps = BuildRegexForIpList(
                ConfigurationSettings.AppSettings["KnownGoodIpList"]);

            Dictionary<string, LinkedList<DateTime>> trackedIps =
                new Dictionary<string, LinkedList<DateTime>>();

            HashSet<string> blockedIps = GetCurrentlyBlockedIps();
            
            Dictionary<string, List<string>> auditFailures =
                new Dictionary<string, List<string>>();

            HashSet<string> newlyBlockedIps = new HashSet<string>();

            using (EventLog ev = new EventLog("Security"))
            {
                EventLogEntryCollection entries = ev.Entries;

                for (int index = entries.Count - 1; index >= 0; index--)
                {
                    EventLogEntry ele = entries[index];
                    if (ele.Index < lastProcessedIndex)
                        break;

                    if (ele.CategoryNumber != 2 || ele.InstanceId != 529 ||
                        ele.EntryType != EventLogEntryType.FailureAudit)
                        continue;

                    if (ele.ReplacementStrings.Length < 11 ||
                        knownGoodIps.IsMatch(ele.ReplacementStrings[11]))
                        continue;

                    string ip = ele.ReplacementStrings[11];

                    if (!auditFailures.ContainsKey(ip))
                        auditFailures[ip] = new List<string>();

                    auditFailures[ip].Add(ele.Message);

                    if (blockedIps.Contains(ip))
                        continue;

                    LinkedList<DateTime> queue;
                    if (!trackedIps.ContainsKey(ip))
                    {
                        queue = new LinkedList<DateTime>();
                        queue.AddFirst(ele.TimeGenerated);
                        trackedIps[ip] = queue;

                        continue;
                    }

                    queue = trackedIps[ip];
                    queue.AddFirst(ele.TimeGenerated);

                    if (queue.Count < failEvents)
                        continue;

                    while (queue.Count > failEvents)
                        queue.RemoveLast();

                    TimeSpan period = queue.Last.Value - queue.First.Value;
                    if (period.Milliseconds > failPeriod)
                        continue;

                    string msg = string.Format(
                        "{0} Adding the following ip to blocked list: {1}",
                        DateTime.Now, ip);

                    blockedIps.Add(ip);
                    newlyBlockedIps.Add(ip);

                    NetSh("routing ip add filter name=\"Network Connection\" filtertype=INPUT srcaddr="
                        + ip + " srcmask=255.255.255.255 dstaddr=0.0.0.0 dstmask=0.0.0.0 proto=ANY");

                    Console.WriteLine(msg);
                    mailBody.Append(msg);
                    mailBody.Append("\r\n");
                }

                lastProcessedIndex = entries[entries.Count - 1].Index;
            }

            if (mailBody.Length > 0)
            {
                foreach (string ip in newlyBlockedIps)
                {
                    foreach (string message in auditFailures[ip])
                    {
                        mailBody.Append("-----\r\n");
                        mailBody.Append(message);
                    }
                }

                SendMail(mailBody);
            }

            return lastProcessedIndex;
        }

        /// <summary>
        /// Main entry point, does all the work.
        /// </summary>
        /// <param name="args"> Program arguments. Not used, all configuration is
        /// through app.config. </param>
        static void Main(string[] args)
        {
            int lastProcessedIndex = 0;
            for (; ; )
            {
                lastProcessedIndex = Process(lastProcessedIndex);

                GC.Collect();

                int sleep = int.Parse(
                    ConfigurationSettings.AppSettings["SleepIntervalMinutes"]);
                Console.WriteLine("Sleeping for {0} minutes @ {1}...",
                    sleep, lastProcessedIndex);
                Thread.Sleep(sleep * 60 * 1000);
            }
        }
    }
}

It also requires an app.config with the following settings:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    <add key="KnownGoodIpList" value="192.168.,131.107."/>
    <add key="LogonFailuresPerPeriod" value="5" />
    <add key="FailurePeriodMinutes" value="2" />
    <add key="SmtpServer" value="xxx" />
    <add key="NotificationEmail" value="yyy" />
    <add key="FromEmail" value="zzz" />
    <add key="UseSslForSmtp" value="true" />
    <add key="SleepIntervalMinutes" value="2"/>
  </appSettings>
</configuration>

The whole thing took barely 2 hours, and have stopped short two attacks just this evening!

J. K. Rowling on elections (via Reddit)

http://www.timesonline.co.uk/tol/comment/columnists/guest_contributors/article7096786.ece?print=yes

Printing and .NET

I needed to print a pattern to help mount a scope, which in turn made me learn .NET printing. As it turns out, it is really straightforward.

Here's an abbreviated guide.

First, add references to System.Drawing and System.Windows.Forms (the latter is for the printer selection dialog box).

Second, add the following usings:

using System.Drawing;
using System.Drawing.Printing;
using System.Windows.Forms;

Third, subclass the PrintDocument class and override OnPrintPage:

class PrintSomething : PrintDocument
{
    protected override void OnPrintPage(PrintPageEventArgs e)
    {
        ...
    }
}

The available paper space is defined by DefaultPageSettings.PrintableArea which is a rectangle with the coordinate system at (Left, Top) and extending down to (Right, Bottom). The units are in 1/100th of an inch, and the drawing coordinates are also in the same units. So for example, to draw a one inch circle in the middle of the page...

float middleX = (DefaultPageSettings.PrintableArea.Right +
    DefaultPageSettings.PrintableArea.Left) / 2.0F;
int middleY = (DefaultPageSettings.PrintableArea.Bottom +
    DefaultPageSettings.PrintableArea.Top) / 2.0F;
Pen p = new Pen(Color.Black, 4);
e.Graphics.DrawEllipse(p, middleX - 50.0F, middleY + 50.0F, 100.0F, 100.0F);

Finally, the following code displays the printer selecting dialog and renders the picture if user clicks OK.

static void Main(string[] args)
{
    PrintSomething lines = new PrintSomething();
    PrintDialog dlg = new PrintDialog();
    dlg.Document = lines;
    if (dlg.ShowDialog() == DialogResult.OK)
        lines.Print();
}

So here is the final program:

//-----------------------------------------------------------------------
// <copyright file="Program.cs" company="Sergey Solyanik">
// Copyright (C) Sergey Solyanik.
//
// This file is subject to the terms and conditions of the Microsoft Public License (MS-PL).
// See http://www.microsoft.com/opensource/licenses.mspx#Ms-PL for more details.
// </copyright>
//----------------------------------------------------------------------- 
using System;
using System.Drawing;
using System.Drawing.Printing;
using System.Windows.Forms;

/// <summary>
/// This code prints a target that can be used for scope alignment.
/// </summary>
class PrintTarget : PrintDocument
{
    /// <summary>
    /// Prints out a target.
    /// </summary>
    /// <param name="args"> Arguments array. Not used. </param>
    static void Main(string[] args)
    {
        PrintTarget lines = new PrintTarget();
        PrintDialog dlg = new PrintDialog();
        dlg.Document = lines;
        if (dlg.ShowDialog() == DialogResult.OK)
            lines.Print();
    }

    /// <summary>
    /// Renders the page.
    /// </summary>
    /// <param name="e"> Printer parameters and graphics. </param>
    protected override void OnPrintPage(PrintPageEventArgs e)
    {
        Graphics g = e.Graphics;
        Pen p1 = new Pen(Color.Black, 0.5F);
        Pen p2 = new Pen(Color.Black, 1);
        Pen p4 = new Pen(Color.Black, 4);

        float step = 100.0F / 16.0F;

        int middleX = (int)((DefaultPageSettings.PrintableArea.Right -
            DefaultPageSettings.PrintableArea.Left) / step) / 2;
        int boundX = ((int)((DefaultPageSettings.PrintableArea.Right -
            DefaultPageSettings.PrintableArea.Left) / 200.0F)) * 16;
        int middleY = (int)((DefaultPageSettings.PrintableArea.Bottom -
            DefaultPageSettings.PrintableArea.Top) / step) / 2;
        int boundY = ((int)((DefaultPageSettings.PrintableArea.Bottom -
            DefaultPageSettings.PrintableArea.Top) / 200.0F)) * 16;

        float top = DefaultPageSettings.PrintableArea.Top +
            (middleY - boundY) * step;
        float bottom = DefaultPageSettings.PrintableArea.Top +
            (middleY + boundY) * step;
        float left = DefaultPageSettings.PrintableArea.Left +
            (middleX - boundX) * step;
        float right = DefaultPageSettings.PrintableArea.Left +
            (middleX + boundX) * step;
        for (int i = 0; i < boundX; ++i)
        {
            Pen p;
            if (i % 16 == 0)
                p = p4;
            else if (i % 8 == 0)
                p = p2;
            else
                p = p1;

            g.DrawLine(p,
                DefaultPageSettings.PrintableArea.Left +
                (middleX + i) * step,
                top,
                DefaultPageSettings.PrintableArea.Left +
                (middleX + i) * step,
                bottom);

            g.DrawLine(p,
                DefaultPageSettings.PrintableArea.Left +
                (middleX - i) * step,
                top,
                DefaultPageSettings.PrintableArea.Left +
                (middleX - i) * step,
                bottom);
        }

        for (int i = 0; i < boundY; ++i)
        {
            Pen p;
            if (i % 16 == 0)
                p = p4;
            else if (i % 8 == 0)
                p = p2;
            else
                p = p1;

            g.DrawLine(p,
                left,
                DefaultPageSettings.PrintableArea.Top +
                (middleY + i) * step,
                right,
                DefaultPageSettings.PrintableArea.Top +
                (middleY + i) * step);

            g.DrawLine(p,
                left,
                DefaultPageSettings.PrintableArea.Top +
                (middleY - i) * step,
                right,
                DefaultPageSettings.PrintableArea.Top +
                (middleY - i) * step);
        }

        for (int i = 1; i < 4; ++i)
        {
            g.DrawEllipse(p4,
                DefaultPageSettings.PrintableArea.Left +
                (middleX - 8 * i) * step,
                DefaultPageSettings.PrintableArea.Top +
                (middleY - 8 * i) * step,
                16 * i * step, 16 * i * step);
        }
    }
}

And here's what it prints:

Hint: Microsoft XPS document writer is a great way to debug printer output without wasting paper.

South Dakota legislature

Idiocracy is alive and progressing. Here's a quote from a recent resolution just passed in South Dakota declaring global warming null and void. Among the clauses:

"That there are a variety of climatological, meteorological, astrological, thermological, cosmological, and ecological dynamics that can effect world weather phenomena and that the significance and interrelativity of these factors is largely speculative;"

(emphasis mine)

Note the "astrological" (!!!) and the fact that they don't even write proper English - in a resolution, no less (http://www.yourdictionary.com/grammar-rules/affect-effect-grammar.html).

For the Nth time, the global warming "sceptics" (you don't get to be a sceptic in an area where you have no clue whatsoever) have demonstrated their true face - a bunch of ignorant anti-science idiots.

A democracy without an educated demos does not work.

Here's the link...

http://legis.state.sd.us/sessions/2010/Bill.aspx?File=HCR1009P.htm

...and the screen shot, in case they correct it...

Like, C#

Yesterday I read absolutely hilarious post on Like, Python (http://www.staringispolite.com/likepython/) - a valleygirl/hillbilly dialect of Python. It was a big hit at work, and people immediately started asking me about the C# version.

You know our motto - we deliver!

So after 30 minutes of stitching together pieces from Malevich's Syntax Highlighter (to allow proper treatment of comments and string constants) (http://malevich.codeplex.com), and Scriptster (http://scriptster.codeplex.com), I present you with Like, C#.

Like, C# uses the same extended set of "keywords" as Like, Python, so we can write this code:

ohai

totally using System man;

uh class like Program
{
    ok static void Main(string[] args) bro
    {
        just Console.Write("yo! what's your name?");
        ok so like string name = Console.ReadLine();

        if actually (name.Equals(""))
            omg toootally just return;

        um yeah

        plz Console.WriteLine(like "Hi {0}, nice to meet you!", name);
    }
}

Full list of "keywords" is here:

• Valleygirl: omg, so, like, totally, right, toootally


• Frat guy: friggin, fuckin, dude, man, bro, broheim, broseph


• Internets: lol, rofl, teh, ohai, plz


• Snoop: yo, homey, homeboy, sup, dog, shit, girl, ma, biatch, ho, shiiit


• Local: wicked, hella, anyways


• Misc: just, hey, yeah, ok, um, uh, ah, actually, something

And here's the full source of Like, C# compiler if you want to adopt this new, rapidly growing language:

//-----------------------------------------------------------------------
// <copyright file="Program.cs" company="Sergey Solyanik">
// Copyright (C) Sergey Solyanik.
//
// This file is subject to the terms and conditions of the Microsoft Public License (MS-PL).
// See http://www.microsoft.com/opensource/licenses.mspx#Ms-PL for more details.
// </copyright>
//----------------------------------------------------------------------- 
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;

namespace LikeCSharp
{
    /// <summary>
    /// Implements Like, C# compiler.
    /// </summary>
    class Program
    {
        /// <summary>
        /// Extension keywords.
        /// </summary>
        private static string[] keywords = 
        {
            "omg", "so", "like", "totally", "right", "toootally",
            "friggin", "fuckin", "dude", "man", "bro", "broheim",
            "broseph", "lol", "rofl", "teh", "ohai", "plz",
            "yo", "homey", "homeboy", "sup", "dog", "shit", "girl",
            "ma", "biatch", "ho", "shiiit", "wicked", "hella", "anyways",
            "just", "hey", "yeah", "ok", "um", "uh", "ah", "actually",
            "something"
        };

        /// <summary>
        /// Main entry point.
        /// </summary>
        /// <param name="args"></param>
        static void Main(string[] args)
        {
            string runtimeEnvironmentPath =
                System.Runtime.InteropServices.RuntimeEnvironment.GetRuntimeDirectory();
            if (runtimeEnvironmentPath.EndsWith("\\"))
            {
                runtimeEnvironmentPath = runtimeEnvironmentPath.Substring(
                    0, runtimeEnvironmentPath.Length - 1);
            }

            string compilerExe = Path.Combine(runtimeEnvironmentPath, "csc.exe");

            // However, if it's 2.0, there is a compiler-only extension (3.5).
            // If 3.5 is indeed installed, use that.
            string baseFrameworkPath = Path.GetDirectoryName(runtimeEnvironmentPath);
            string runtimeName = Path.GetFileName(runtimeEnvironmentPath);
            if ("v2.0.50727".Equals(runtimeName))
            {
                string compilerExe35 = Path.Combine(
                    Path.Combine(baseFrameworkPath, "v3.5"), "csc.exe");
                if (File.Exists(compilerExe35))
                {
                    compilerExe = compilerExe35;
                }
            }

            StringBuilder sb = new StringBuilder();
            bool firstArg = true;
            HashSet<string> files = new HashSet<string>();

            string keywordRE = "\"|'|//|/\\*|#?[a-z_0-9]+";
            HashSet<string> keywordDictionary = new HashSet<string>(keywords);
            Regex keywordRegex = new Regex(
                keywordRE, RegexOptions.Compiled | RegexOptions.IgnoreCase);

            foreach (string arg in args)
            {
                if (firstArg)
                {
                    firstArg = false;
                }
                else
                {
                    sb.Append(' ');
                }

                string newArg = arg;
                if (arg.EndsWith(".lcs"))
                {
                    newArg = Preprocess(arg, keywordRegex, keywordDictionary);
                    files.Add(newArg);
                }

                if (newArg.Contains(' '))
                {
                    newArg = '"' + newArg + '"';
                }

                sb.Append(newArg);
            }

            Process compiler = new Process();
            compiler.StartInfo.UseShellExecute = false;
            compiler.StartInfo.FileName = compilerExe;
            compiler.StartInfo.Arguments = sb.ToString();

            compiler.Start();
            compiler.WaitForExit();

            Environment.ExitCode = compiler.ExitCode;

            foreach (string file in files)
            {
                File.Delete(file);
            }
        }

        /// <summary>
        /// Converts file from Like, C# to normal C#.
        /// </summary>
        /// <param name="fileName"> File name to process. </param>
        /// <param name="keywordRegex"> Regular expression that
        /// matches language elements. </param>
        /// <param name="keywordDictionary"> Like keywords. </param>
        /// <returns> Name of the temporary file. </returns>
        static string Preprocess(
            string fileName,
            Regex keywordRegex,
            HashSet<string> keywordDictionary)
        {
            string tempDir = Environment.GetEnvironmentVariable("TEMP");
            tempDir = Path.Combine(tempDir, "LIKECSHARP");
            if (!Directory.Exists(tempDir))
            {
                Directory.CreateDirectory(tempDir);
            }

            string target = Path.Combine(
                tempDir,
                Path.GetFileNameWithoutExtension(fileName) + ".cs");

            QuoteType quotes = QuoteType.None;
            bool inComments = false;

            StreamReader reader = new StreamReader(fileName);
            StreamWriter writer = new StreamWriter(target);

            for (; ; )
            {
                string s = reader.ReadLine();
                if (s == null)
                {
                    break;
                }

                StringBuilder encoded = new StringBuilder();

                int i = 0;
                while (i < s.Length)
                {
                    if (quotes == QuoteType.Double)
                    {
                        int endQuotes = s.IndexOf("\"", i);
                        if (endQuotes == -1)
                        {
                            // The entire string is in quotes.
                            // The tag will close after the loop.
                            encoded.Append(s.Substring(i));
                            break;
                        }

                        encoded.Append(s.Substring(i, endQuotes + 1 - i));
                        if ((endQuotes > 0 && s[endQuotes - 1] != '\\') ||
                            (endQuotes > 1 && s[endQuotes - 1] == '\\' &&
                            s[endQuotes - 2] == '\\'))
                        {
                            // This is not an escaped quote.
                            quotes = QuoteType.None;
                        }

                        i = endQuotes + 1;
                        continue;
                    }

                    if (quotes == QuoteType.Verbatim)
                    {
                        int endQuotes = s.IndexOf("\"", i);
                        if (endQuotes == -1)
                        {
                            // The entire string is in quotes.
                            // The tag will close after the loop.
                            encoded.Append(s.Substring(i));
                            break;
                        }

                        if (s.IndexOf("\"\"", endQuotes) == endQuotes)
                        {
                            // This is an escaped quote.
                            encoded.Append(s.Substring(i, endQuotes + 2 - i));
                            i = endQuotes + 2;
                            continue;
                        }

                        encoded.Append(s.Substring(i, endQuotes + 1 - i));
                        quotes = QuoteType.None;

                        i = endQuotes + 1;
                        continue;
                    }

                    if (quotes == QuoteType.Single)
                    {
                        int endQuotes = s.IndexOf('\'', i);
                        if (endQuotes == -1)
                        {
                            // The entire string is in quotes.
                            // The tag will close after the loop.
                            encoded.Append(s.Substring(i));
                            break;
                        }

                        encoded.Append(s.Substring(i, endQuotes + 1 - i));
                        if ((endQuotes > 0 && s[endQuotes - 1] != '\\') ||
                            (endQuotes > 1 && s[endQuotes - 1] == '\\' &&
                            s[endQuotes - 2] == '\\'))
                        {
                            // This is not an escaped quote.
                            quotes = QuoteType.None;
                        }

                        i = endQuotes + 1;
                        continue;
                    }

                    if (inComments)
                    {
                        int endComments = s.IndexOf("*/", i);
                        if (endComments == -1)
                        {
                            // The entire string is in comments.
                            // The tag will close after the loop.
                            encoded.Append(s.Substring(i));
                            break;
                        }

                        inComments = false;

                        encoded.Append(
                            s.Substring(i, endComments + 2 - i));
                        i = endComments + 2;
                        continue;
                    }

                    Match nextMatch = keywordRegex.Match(s, i);
                    if (!nextMatch.Success)
                    {
                        encoded.Append(s, i, s.Length - i);
                        break;
                    }

                    if (i != nextMatch.Index)
                    {
                        encoded.Append(s, i, nextMatch.Index - i);
                        i = nextMatch.Index;
                    }

                    string matched = nextMatch.Groups[0].Value;

                    if (matched.Equals("'"))
                    {
                        encoded.Append("'");
                        quotes = QuoteType.Single;
                        ++i;
                        continue;
                    }

                    if (matched.Equals("\""))
                    {
                        encoded.Append("\"");
                        quotes = QuoteType.Double;
                        ++i;
                        continue;
                    }

                    if (matched.Equals("@\""))
                    {
                        encoded.Append("@\"");
                        quotes = QuoteType.Verbatim;
                        i += 2;
                        continue;
                    }

                    if (matched.Equals("//"))
                    {
                        // The rest of the line is comments.
                        encoded.Append(s.Substring(i));
                        break;
                    }

                    if (matched.Equals("/*"))
                    {
                        // Comments start.
                        encoded.Append("/*");
                        inComments = true;
                        i += 2;
                        continue;
                    }

                    if (keywordDictionary.Contains(matched))
                    {
                        // Keyword. Eat it.
                        i += matched.Length;
                    }
                    else
                    {
                        encoded.Append(matched);
                        i += matched.Length;
                    }
                }

                writer.WriteLine(encoded.ToString());
            }

            reader.Close();
            writer.Close();

            return target;
        }

        /// <summary>
        /// Types of quoted string.
        /// </summary>
        enum QuoteType
        {
            None,
            Single,
            Double,
            Verbatim
        }
    }
}

William Buckley on General Franco

"General Franco is an authentic national hero. It is generally conceded that he above others had the combination of talents, the perseverance, and the sense of righteousness of his cause, that were required to wrest Spain from the hands of the visionaries, ideologues, Marxists and nihlistis that were imposing on her, in the thirties, a regime so grotesque as to do violence to the Spanish soul, to deny, even Spain's historical identity."

http://www.slate.com/id/2185301/pagenum/2

While we're at it, another quote, this time from Richard Nixon: "General Franco was a loyal friend and ally of the United States."

http://en.wikipedia.org/wiki/Francisco_Franco

When I am saying that in the US we're having a choice between fascists on the right, and the center-right party on the "left", people are thinking that I am exaggerating. I am not.

Article on mobile platforms...

...that I might have written, but this guy beat me to it :-).

http://blog.7touchgroup.com/2010/01/windows-mobile-iphone-android-marketplace-comparison/

Another pay vs. performance article

http://www.huffingtonpost.com/2009/12/31/high-ceo-pay-may-correlat_n_406578.html

Your faith is not strong enough...

http://www.ebonmusings.org/atheism/guestessays/religion101.html?

Sr Oracle needed at Bank of America

...presumably so that they could predict the bursting of the next bubble ahead of time.

30000 new troops for Afghanistan - yeeeeeeehaw!

Web applications!

2 browsers (IE and Firefox) running Ajax.org demo are maxing out T9300 on my laptop - a Core 2 Duo running at 2.5 GHz. This is plugged in.

Try it yourself:

http://www.ajax.org/public/presentation/tae/presentation2.html#home

Best pie chart ever - directly from Fox (via Reddit)

Modern capitalism

The top 5 execs at Lehman Brothers made $1B from 2000 to 2008. The top 5 execs at Bearn Stearns made $1.4B during the same period.

"The whole idea of capitalism is that the people provide the capital and the executives take care of it for us. In this case, the people provided the capital, and the executives took it."

http://crooksandliars.com/susie-madrak/lehman-bros-bear-stearns-ceos-walked

Victim of car accident is tragically not Glenn Beck

Victim In Fatal Car Accident Tragically Not Glenn Beck

Meet Britain's Dick Cheney!

The photo in this article is priceless...

http://www.guardian.co.uk/business/2009/oct/21/executive-pay-bonuses-goldmansachs

Matt Taibbi on naked short selling, or what killed Bear Stearns

Really awesome article on naked shorts. There are only a handful of news organizations who do real journalism these days, and, surprisingly, Rolling Stones seems to be one of these few. I am also very impressed with Bloomberg reporting recently - it looks like after Murdoch bought WSJ the best people went to Bloomberg.

http://www.rollingstone.com/politics/story/30481512/wall_streets_naked_swindle/print

Here's the really scary thing - we've lost the integrity of our financial markets.

The reason US was so successful in attracting capital was the transparency with which financial markets here used to operate, and strong regulations that ensured this transparency - for the most part, the legacy of Great Depression. However, over the last couple of decades these regulations have been weakened, and now we're back to the turn of the century stock swindles and off the books accounting (http://www.creditloan.com/blog/mark-to-market-accounting-changes-favor-banks/).

All this - coupled with the dollar fall - can mean only one thing: US will certainly lose its status as a financial capital of the world within the next decade.

In other news, marijuana legislation has been relaxed (http://finance.yahoo.com/news/AP-Newsbreak-New-medical-apf-4109207182.html?x=0&sec=topStories&pos=main&asset=&ccode=). So instead of worrying about markets, botched wars, and disappearing industrial base, the population can just get stoned! Finally, a government that knows how to rule...

Government rationing of health care

There were all these inane reports in the press about "Obama's suicide panels" and government rationing the health care - all widely covered.

Let's see what they say about this beauty - an insurance company says that they will only cover a woman if she gets sterilized - because she had a C-section in the past.

Under the hill

Check out where you can deliver a birthday present on 1-800-flowers.com!

Think pay for performance works? Think again...

http://www.ted.com/talks/dan_pink_on_motivation.html

This TED talk alludes to a number of social experiments where adding monetary incentive has significantly decreased the productivity of creative work.

Extremely enlightening, a must watch for every manager - and yet another proof that the claim that the Wall Streeters need to be paid exorbitant amounts of money to retain "the best and the brightest" is just a crapload of bull.

United States v. 8,800 Pounds, More or Less of Powdered White Egg Product

"United States v. 8,800 Pounds, More or Less of Powdered White Egg Product, et al., No. 07-3671, 2008 U.S. App. LEXIS 26098 (8th Cir. Dec. 24, 2008)(defendant’s importation of egg whites from Peru without first obtaining a pasteurization certificate from the Peruvian government and without obtaining authorization from the Food Safety Inspection Service violated the Egg Products Inspection Act, which could result in the condemnation and destruction of the egg whites; trial court’s granting of summary judgment upheld)."

It's hidden somewhere around the middle of the list on this page: http://www.calt.iastate.edu/aglawrl.html

I kid you not!

Big Pharma and research

I love it when they say that the pharmaceuticals are so expensive in the US because the companies need this money to finance research. In reality, if you look at a typical big pharma income statement, you will discover that only ~20% of its expenses are research - the rest is mostly sales and marketing.

But here's another proof beyond any reasonable doubt that what big pharma is all about is sales - its donations to political parties. Researchers giving money to repubs? Basically, the same people who say that Earth is 10,000 years old, and dinosaurs walked the planet alongside people, radiocarbon dating be damned?

This political contributions profile is more in line with what you expect from used car salespeople than the research and development organizations...

http://www.goodguide.com/contributions#sector=Pharmaceuticals



By comparison, here's how the software sector looks like:

http://www.goodguide.com/contributions#sector=Computer/Internet&sort=total&query=

Bravo, Apple! You have class!

Apple quits the U.S. Chamber of Commerce over its ‘frustrating’ global warming denialism.

http://thinkprogress.org/2009/10/05/apple-quits-chamber/

TFS 2010 will be much easier to install

I've spent a week trying to get TFS 2008 installed on my home infrastructure. It was the most painful piece of software that I've ever installed. And it looks like I was not the only one who felt that:

"Installing TFS has been a pain point for years. Although it’s gotten better, 2010 represents a quantum leap. The TFS installer now has 3 wizards: Basic, Standard and Advanced. The big innovation is the new “Basic” install wizard. It is a Next, Next, Next install experience that allows you to install and configure TFS in about 20 minutes or less (assuming .NET and SQL Express are already on your computer – a little longer if TFS has to install them for you). Both will already be there if you’ve installed VS 2010. The Basic wizard will install and configure IIS (if it’s not already there), install and configure SQL Express (if it’s not already there), and install and configure TFS. The only thing that really pains me is installing .NET 4.0 requires a reboot :(."

http://blogs.msdn.com/bharry/archive/2009/10/01/tfs-2010-for-sourcesafe-users.aspx