Kentucky, 2006

"The 2006 law organizing the state Office of Homeland Security lists its initial duty as "stressing the dependence on Almighty God as being vital to the security of the Commonwealth."

Specifically, Homeland Security is ordered to publicize God's benevolent protection in its reports, and it must post a plaque at the entrance to the state Emergency Operations Center with an 88-word statement that begins, "The safety and security of the Commonwealth cannot be achieved apart from reliance upon Almighty God.""

http://www.kentucky.com/210/story/608229.html

This is 2006. No kidding. Welcome to Jesusland!

It's Thanksgiving, not a Halloween! Why am I scared?

"In fact, in commercial and academic turkey-breeding programs, adult male turkeys, called toms, can reach 50 pounds at the tender age of five months, said John Anderson, a longtime turkey breeder at Ohio State University.

"We get 50 pounders at 20 weeks, but that's at the top edge of our normal distribution," Anderson said. "We've got some adult male-line birds that went over 80 pounds.""

http://blog.wired.com/wiredscience/2008/11/turkeytech.html

Organic Avocados

"I told my wife to stop this organic food malarkey,'' said Jeremy, a beleaguered hedge-fund manager, another man who fell for an extremely beautiful yet extravagant woman.

"She went ballistic. Organic Hass avocados cost £1.75 each and she wanted me to buy six of them! In the end, I just peeled off the labels that said they were certified organic and put them on ordinary avocados – she didn't notice the difference. I did the same with bananas…"

http://www.telegraph.co.uk/finance/financetopics/recession/3527803/Recession-When-the-money-goes-so-does-the-toxic-wife.html

US automakers vs. Honda

"You need to give people the freedom to spend and the freedom to make mistakes," says Takeo Fukui, Honda's 61-year-old president. "If management oversight is too strong, then it's difficult to innovate."

"There's not a company on earth that better understands the culture of engineering."

Very interesting article in Forbes about engineering-dominated Honda vs. marketing-dominated automakers in the US.

This is generally true beyond car manufacturing - at most US companies marketing and finance rules, technology and manufacturing get outsourced.

Even in hi-tech, there are fewer and fewer engineers at on near the top - the trend started during the .COM boom when the geekish software companies got "integrated" into society as a legitimate make money fast industry, and it never really ended.

How many VPs did you see that are "proud of the smart people who are making all these great products", but are decidedly (and pointedly) not one of them?

We're now reaping the "rewards" of the culture where it's not smart to be smart...

http://www.forbes.com/forbes/2006/0904/112.html

You know why the economy tanked? It's because we're not religulous enough!

Under Faux News aegis, Wall Street Journal opinion page is reaching the whole new level of ridiculous...

http://online.wsj.com/article/SB122714101083742715.html

Support jobs

This is ridiculously funny: http://www.quoted4truth.com/articles/I-wonder-if-he-liked-his-job.

Made me recall another tech support classic: http://dir.salon.com/story/tech/feature/2004/02/23/no_support/print.html

This is leadership...

JAL CEO cuts his salary to below that of his pilots.

http://www.cnn.com/video/#/video/business/2008/11/02/lah.japan.ceo.pay.cut.cnn.

I'd like to see Wagoner, Nardelli, and their ilk do the same before they get a dime of the bailout money...

Retards are out, smart people are in...

http://www.nytimes.com/2008/11/21/opinion/21brooks.html?_r=1

Maybe we have a hope...

RIP PC Magazine

http://www.paidcontent.org/entry/419-ziff-davis-to-close-print-pcmag-focus-on-online-still-looking-for-optio/

The last time I held it in my hands was some time in the late nineties, but it is an end of an era...

Alienation

"According to presidential historian Davis Logsdon of the University of Minnesota, some Americans might find it "alienating" to have a president who speaks English as if it were his first language."

http://www.huffingtonpost.com/andy-borowitz/obamas-use-of-complete-se_b_144642.html

Taking a private jet to beg for a bailout

http://abcnews.go.com/Blotter/WallStreet/story?id=6285739&page=1

Note also the salaries of these "leaders" in the article above.

Incidentally, none of these excesses existed before Reagan's tax cuts, and they do not exist today in Europe where taxes on super-high incomes are accordingly high.

Mekong Delta

In 1968 US has carried an Operation Speedy Express in the heavily populated Mekong river delta. The orders were to achieve the high body count at all costs.

That was achieved: 10899 enemy casualties were reported, with only 267 American losses.

The problem? Only 748 weapons recovered. Much of the rest were civilians.

http://www.thenation.com/doc/20081201/turse

Maybe in Ohio, but not in America!

"If it wasn't for Bush we would be out of street waving white flags
waiting for the salvation army trucks to bring us our daily meal.

I fear that with a liberal at helm, the only direction we'll be
steering for is Deep, Deadly Depression.

I am contemplating moving to Canada or somewhere in Europe. Push comes
to shove, I may even move to Mexico or Hawaii, as long as it's a
different country and I can get away with the depression that is
inevitable if a the government liberalizes everything."

http://finance.google.com/group/google.finance.358464/browse_thread/thread/5e9d0da4985a76c3?pli=1

Check out the reaction of Canadians down in the thread: "As a Canadian, i have to say, please don't come here. we don't need people like you!"

Exchange web services 2007

A few snippets from my weekly status database that show how to interact with EWS 2007.

First, you need to generate EWS 2007 proxy. In Visual Studio it is relatively easy: Project->Add service reference, enter the address of your exchange web server, for example, https://mail.microsoft.com/EWS/Exchange.asmx (you won't be able to use this one because it is obviously secured). Hit Go, then name the namespace and hit OK.

One problem with EWS 2007 is that it unfortunately requires user credentials - cleartext (they are sent using HTTPS, so the wire security is not a problem, but the user has to enter the password/user name, or the software needs to CryptProtectData the credentials and store them. Windows auth is not available.

See here for a snippet of code that stores the password SECURELY: http://1-800-magic.blogspot.com/2008/11/encrypting-data-in-net.html. Make sure you're running under real user account (not LocalService, and definitely not NetworkSerivce), otherwise anybody will be able to decrypt the password.

Once the proxy has been generated, and put in the namespace, and the namespace has been included in the file with "using" directive, we can start coding.

First, we need to bind to the service. Binding is required for all examples below:

const string EXCHANGE_URL = @"https://mail.mydomain.com/EWS/Exchange.asmx";
ExchangeServiceBinding binding = new ExchangeServiceBinding();
binding.Credentials = new NetworkCredential(User, Password, Domain);
binding.Url = EXCHANGE_URL;

Looking up a subfolder of inbox:

DistinguishedFolderIdType inbox = new DistinguishedFolderIdType();
inbox.Id = DistinguishedFolderIdNameType.inbox;
BaseFolderIdType[] parentFolder = new BaseFolderIdType[1];
parentFolder[0] = inbox;

FindFolderType folderRequest = new FindFolderType();
folderRequest.FolderShape = new FolderResponseShapeType();
folderRequest.FolderShape.BaseShape = DefaultShapeNamesType.AllProperties;
folderRequest.Traversal = FolderQueryTraversalType.Shallow;
folderRequest.ParentFolderIds = parentFolder;

FindFolderResponseType folderResponse = binding.FindFolder(folderRequest);
foreach (ResponseMessageType responseMessage in folderResponse.ResponseMessages.Items)
{
    FindFolderResponseMessageType findFolderResponseMessage =
        responseMessage as FindFolderResponseMessageType;
    if (findFolderResponseMessage.ResponseClass == ResponseClassType.Success)
    {
        BaseFolderType[] folders = findFolderResponseMessage.RootFolder.Folders;
        foreach (BaseFolderType folder in folders)
        {
            if (folder.DisplayName.Equals(folderWeAreLookingFor))
            {
               ...do stuff
            }
        }
    }
}

Here's how to get all email (we're only getting the ids of messages, the contents needs to be queried using a different request, covered later:

DistinguishedFolderIdType inbox = new DistinguishedFolderIdType();
inbox.Id = DistinguishedFolderIdNameType.inbox;
BaseFolderIdType[] parentFolder = new BaseFolderIdType[1];
parentFolder[0] = inbox;

FindItemType listRequest = new FindItemType();
listRequest.ItemShape = new ItemResponseShapeType();
listRequest.ItemShape.BaseShape = DefaultShapeNamesType.AllProperties;
listRequest.Traversal = ItemQueryTraversalType.Shallow;
listRequest.ParentFolderIds = parentFolder;

FindItemResponseType listResponse = binding.FindItem(listRequest);

List items = new List();

foreach (ResponseMessageType responseMessage in listResponse.ResponseMessages.Items)
{
    FindItemResponseMessageType findItemResponseMessage =
        responseMessage as FindItemResponseMessageType;
    if (findItemResponseMessage.ResponseClass == ResponseClassType.Success)
    {
        ArrayOfRealItemsType realItems =
            findItemResponseMessage.RootFolder.Item as ArrayOfRealItemsType;
        if (realItems != null && realItems.Items != null)
        {
            foreach (ItemType item in realItems.Items)
            {
                if (item is MessageType)
                {
                    MessageType message = item as MessageType;
                    items.Add(message.ItemId);
                }
            }
        }
    }
}

Now, suppose we accumulated all message ids that we want to get in the items list, (we presume that items.Count() > 0). We will now get all the messages as follows:

GetItemType getRequest = new GetItemType();
getRequest.ItemShape = new ItemResponseShapeType();
getRequest.ItemShape.BaseShape = DefaultShapeNamesType.AllProperties;
getRequest.ItemIds = items.ToArray();

SnippetsDataContext snippets = new SnippetsDataContext(Database);

GetItemResponseType getResponse = binding.GetItem(getRequest);
foreach (ResponseMessageType responseMessage in getResponse.ResponseMessages.Items)
{
    ItemInfoResponseMessageType getItemResponseMessage =
        responseMessage as ItemInfoResponseMessageType;
    if (getItemResponseMessage.ResponseClass == ResponseClassType.Success)
    {
        ArrayOfRealItemsType realItems =
            getItemResponseMessage.Items as ArrayOfRealItemsType;
        if (realItems != null && realItems.Items != null)
        {
            foreach (ItemType item in realItems.Items)
            {
                if (item is MessageType)
                {
                    MessageType message = item as MessageType;
                    // Message now contains everything, except for attachments.
                    ...
                }
            }
        }
    }
}

Suppose we want to move a message to a different folder. We need to find folder's id (see above for how to enumerate folders). If we do have this id, and the id of the message we're trying to move:

MoveItemType moveRequest = new MoveItemType();
moveRequest.ToFolderId = new TargetFolderIdType();
moveRequest.ToFolderId.Item = targetFolder;
moveRequest.ItemIds = new BaseItemIdType[1];
moveRequest.ItemIds[0] = messageId;
MoveItemResponseType moveResponse = binding.MoveItem(moveRequest);

Ok, enough moving stuff around. Let's send something!

ExchangeServiceBinding binding = new ExchangeServiceBinding();
binding.Credentials = new NetworkCredential(User, Password, Domain);
binding.Url = EXCHANGE_URL;

DistinguishedFolderIdType folder = new DistinguishedFolderIdType();
folder.Id = DistinguishedFolderIdNameType.sentitems;

TargetFolderIdType targetFolder = new TargetFolderIdType();
targetFolder.Item = folder;

CreateItemType createItem = new CreateItemType();
createItem.MessageDisposition = MessageDispositionType.SendAndSaveCopy;
createItem.MessageDispositionSpecified = true;
createItem.SavedItemFolderId = targetFolder;

MessageType message = new MessageType();

EmailAddressType recipient = new EmailAddressType();
recipient.EmailAddress = "email@domain.com";

message.ToRecipients = new EmailAddressType[1];
message.ToRecipients[0] = recipient;
message.Subject = "Hello, World!";
message.Sensitivity = SensitivityChoicesType.Normal;

message.Body = new BodyType();
message.Body.BodyType1 = BodyTypeType.Text;
message.Body.Value = "This is my first automated email!";

createItem.Items = new NonEmptyArrayOfAllItemsType();
createItem.Items.Items = new MessageType[1];
createItem.Items.Items[0] = message;
binding.CreateItem(createItem);

Many messages can be acted upon (moved, sent, deleted) by one command - all it takes is putting multiple messages or ids in Items.Items array.

Dealing with the web service is A LOT of code - even with the help of the proxy. Comare this to the Office interop, where most of the above tasks take 10-15 lines of code. http://1-800-magic.blogspot.com/2008/11/fun-with-outlook-and-net.html

More on EWS 2007 here: http://msdn.microsoft.com/en-us/library/bb418726.aspx

However stupid our Republican VP candidate might look...

...things are far, far worse in Russia. A couple of juicy quotes...

""Medvedev will serve his term and may be re-elected for another," says Edward Lozansky, a former dissident. "It is none of the British press's business anyway. Go and teach your British wives to cook cabbage soup.""
...

"Until last week, Svetlana, a young mother of two, held decidedly middle-class ambitions. An executive at a construction company, she was hoping to save enough to send her children to school in Britain and buy a new flat in an upmarket part of Moscow. But then, without warning, she was made redundant along with about 70 colleagues. She received no severance pay and was instead forced to sign a letter saying she had voluntarily resigned.
"They said that if I didn't sign then I could look forward to burying my own children," Svetlana says. "What kind of country do we live in? I thought we were close to becoming a civilised nation, but I've been forced to realise that that is an illusion.""

http://www.telegraph.co.uk/news/worldnews/europe/russia/3469652/Russias-crumbling-economy-provides-stiffest-test-yet-for-autocratic-leader.html

Sorting in C# with delegates

A delegate can be handy to avoid having to implement an IComparer class.

Suppose you need to sort an array of dates in descending order...

DateTime[] availableDates = allMailDates.ToArray();
Array.Sort(availableDates, delegate(DateTime d1, DateTime d2) {
    return Math.Sign(d2.ToBinary() - d1.ToBinary());
});

First encounter with T-SQL

I am building a little app for my team that collects weekly status emails and publishes it on a web site. Since I've worked on far lower level stuff most of my career (I only wrote the code that actually has UI at Google), it's fun and educating.

Using ASP.NET is quite fun actually - Microsoft REALLY nailed developer experience on this one. The API is logical, concise, well structured, and a pure joy to use. I started with having no clue whatsoever, and had a fully-functioning web app running in around two hours, and this included reading a book.

This story is about T-SQL, however. There came time where I wanted a stored procedure. On my website, there is an adminitrative user who can make changes in a database, and everybody else can do unlimited reading (anybody can see everybody's status mail), but only very limited types of writing (sign up to receive someone else's status updates in email, turn email reminders for her- or himself on and off, and join and depart a project).

So while everything other type of data access was done using LINQ, it seemed convenient to handle mutations by non-admin users via stored procedures. Specifically, I needed one right off the bat: any authenticated user - know by the alias - could sign up for status email for a person whose status was to be collected.

The task seemed quite obvious:

-- =============================================
-- Author:  Sergey Solyanik
-- Create date: 11/14/2008
-- Description: Adds a subscription
-- =============================================
CREATE PROCEDURE [dbo].[AddSubscriptionToPerson]
    @alias nvarchar(15),
    @personId int
AS
BEGIN
    DECLARE @subscriberId int
    
    SET @subscriberId = (SELECT SubscriberId
                         FROM Subscribers
                         WHERE SubscriberAlias = @alias)

    IF @subscriberId = NULL  -- BUGBUG: '=' does not work!
    BEGIN
        INSERT INTO Subscribers (SubscriberAlias) VALUES(@alias)
        SET @subscriberId = @@IDENTITY
    END

    IF NOT EXISTS(SELECT 1
                  FROM SubscriptionsPeople
                  WHERE SubscriberId = @subscriberId AND
                        PersonId = @personId)
    BEGIN
        INSERT INTO SubscriptionsPeople (PersonId, SubscriberId)
        VALUES(@personId, @subscriberId)
    END
END

Right? Wrong! If @subscriberId is NULL, the following statement:

IF @subscriberId = NULL
  SELECT 'IS NULL'
else
  SELECT 'IS NOT NULL'

prints out 'IS NOT NULL'! The correct way of tesing for NULL is not comparing with NULL, but using IS [NOT] NULL keyword:

IF @subscriberId IS NULL
  SELECT 'IS NULL'
else
  SELECT 'IS NOT NULL'

So the version of code that actually works is:

-- =============================================
-- Author:  Sergey Solyanik
-- Create date: 11/14/2008
-- Description: Adds a subscription
-- =============================================
CREATE PROCEDURE [dbo].[AddSubscriptionToPerson]
    @alias nvarchar(15),
    @personId int
AS
BEGIN
    DECLARE @subscriberId int
    
    SET @subscriberId = (SELECT SubscriberId
                         FROM Subscribers
                         WHERE SubscriberAlias = @alias)

    IF @subscriberId IS NULL
    BEGIN
        INSERT INTO Subscribers (SubscriberAlias) VALUES(@alias)
        SET @subscriberId = @@IDENTITY
    END

    IF NOT EXISTS(SELECT 1
                  FROM SubscriptionsPeople
                  WHERE SubscriberId = @subscriberId AND
                        PersonId = @personId)
    BEGIN
        INSERT INTO SubscriptionsPeople (PersonId, SubscriberId)
        VALUES(@personId, @subscriberId)
    END
END

Unfortuately, discovering this was not trivial. You see, IS [NOT] NULL is nowhere near IF in the documentation, and neither it is in the book I was reading. Also, EXISTS works if you put your SQL query that returns NULL in the IF itself, like so:

IF NOT EXISTS(SELECT SubscriberId
              FROM Subscribers
              WHERE SubscriberAlias = @alias)
THEN...

...but not like this:

SET @id = (SELECT SubscriberId
           FROM Subscribers
           WHERE SubscriberAlias = @alias)
IF EXISTS(@id)...

Well, what should we expect from a language with the syntax and semantics right out of 1950s. I haven't worked with anything like this since FORTRAN...

Redundancy

"There is little point in building a stored procedure just to run a set of T-SQL statements only once; conversely, a stored procedure is ideal for when you wish to run a set of T-SQL statements many times."

Or shall I repeat it?

"Beginning SQL Server 2008 for Developers", by Robin Dewson.

Michael Lewis on the Wall Street collapse

"I’d never taken an accounting course, never run a business, never even had savings of my own to manage. I stumbled into a job at Salomon Brothers in 1985 and stumbled out much richer three years later, and even though I wrote a book about the experience, the whole thing still strikes me as preposterous—which is one of the reasons the money was so easy to walk away from. I figured the situation was unsustainable. Sooner rather than later, someone was going to identify me, along with a lot of people more or less like me, as a fraud. Sooner rather than later, there would come a Great Reckoning when Wall Street would wake up and hundreds if not thousands of young people like me, who had no business making huge bets with other people’s money, would be expelled from finance."

http://www.portfolio.com/news-markets/national-news/portfolio/2008/11/11/The-End-of-Wall-Streets-Boom

If you haven't read the Liar's Poker, I strongly recommend... It's hilarious!

http://www.amazon.com/Liars-Poker-Rising-Through-Wreckage/dp/0140143459

GM in trouble?

Check out what they're paying their CEO...
http://www.reuters.com/article/businessNews/idUSN2534738420080425?feedType=RSS&feedName=businessNews

Encrypting data in .NET

Encrypting:

using System.Security.Cryptography;
...
string password;
string encryptedPassword = Convert.ToBase64String(
    ProtectedData.Protect(Encoding.UTF8.GetBytes(password), null,
    DataProtectionScope.CurrentUser));

Decrypting:

using System.Security.Cryptography;
...
string encryptedPassword;
string password = Encoding.UTF8.GetString(ProtectedData.Unprotect(
    Convert.FromBase64String(encryptedPassword), null,
    DataProtectionScope.CurrentUser))

Registry in .NET

Writing:

using Microsoft.Win32;
...
const string REGISTRY_KEY = "software\\yourcompany\\yourapp";
...
RegistryKey key = Registry.CurrentUser.CreateSubKey(REGISTRY_KEY);
key.SetValue("user", args[1]);
key.Close();

Reading:

using Microsoft.Win32;
...
const string REGISTRY_KEY = "software\\yourcompany\\yourapp";
...
string user = null;
RegistryKey key = Registry.CurrentUser.OpenSubKey(REGISTRY_KEY);
if (key != null)
{
    user = (string)key.GetValue("user");
    key.Close();
}

Fun with LINQ: database programming tasks in under 10 lines of code

Go to Project->New Item->Data LINQ to SQL classes. After you get the design surface, click on "Server Explorer" (a tab on the right), connect to a database, and drag over tables to the white space in the middle of the screen.

Save the resulting DBML file.

Let's say the DBML file is called Snippets.dbml, and it contains the table People (note that below LINQ class generator converted it to plural "Peoples" - a bit annoying, but... whatever).

Simplest query - get everything:

SnippetsDataContext snippets = new SnippetsDataContext();
var allPeople = from person in context.Peoples
                select person;
foreach(People person in allPeople)
{...}
snippets.Connection.Close();
snippets.Dispose();

A simple precondition:

SnippetsDataContext snippets = new SnippetsDataContext();
var activePeople = from person in context.Peoples
                   where person.IsActive
                   select person;
foreach(People person in activePeople)
{...}
snippets.Connection.Close();
snippets.Dispose();

A more complicated precondition:

SnippetsDataContext snippets = new SnippetsDataContext();
var activeMikes = from person in context.Peoples
                  where person.IsActive && person.Name.Contains("Mike")
                  select person;
foreach(People person in activeMikes)
{...}
snippets.Connection.Close();
snippets.Dispose();

Get only the data you need, with anonymous types:

SnippetsDataContext snippets = new SnippetsDataContext();
var peopleNamesAndIds = from person in context.Peoples
                        select new { person.PersonId, person.PersonName };
foreach(var p in peopleNamesAndIds)
{
   Console.WriteLine(p.PersonId + ": " + p.PersonName);
}
snippets.Connection.Close();
snippets.Dispose();

Modifying data:

SnippetsDataContext snippets = new SnippetsDataContext();
var renameMikesToJohns = from person in context.Peoples
                         where person.Name.Contains("Mike")
                         select person;
foreach(People person in renameMikesToJohns)
{
   person.PersonName = person.PersonName.Replace("Mike", "John");
}
context.SubmitChanges();
snippets.Connection.Close();
snippets.Dispose();

Adding a row:

SnippetsDataContext snippets = new SnippetsDataContext();
People person = new Person();
person.PersonName = "Bill Maher";
context.Peoples.InsertOnSubmit(person);

Fun with Outlook and .NET

You need to add reference to Microsoft.Office.Interop.Outlook.

Reading email in 11 lines of code:

using Microsoft.Office.Interop.Outlook;
...
Application Outlook = new Application();
NameSpace OutlookNs = Outlook.GetNamespace("MAPI");
OutlookNs.Logon(null, null, true, true);
Folder inbox =
    (Folder)Outlook.Session.GetDefaultFolder(OlDefaultFolders.olFolderInbox);
Folder destination = GetDestinationFolder();
foreach (object item in inbox.Items)
{
    if (item is MailItem)
    {
        MailItem mail = (MailItem)item;
        Console.Write(mail.SenderName + ": " + mail.Subject);
    }
}
OutlookNs.Logoff();

Sending email in 12 lines of code:

using Microsoft.Office.Interop.Outlook;
...
Application Outlook = new Application();
NameSpace OutlookNs = Outlook.GetNamespace("MAPI");
OutlookNs.Logon(null, null, true, true);
MailItem mail = (MailItem)OutlookNs.Application.CreateItem(OlItemType.olMailItem);
mail.BodyFormat = OlBodyFormat.olFormatPlain;
mail.Body = "Hello, world!";
mail.Subject = "Sent from C# code snippet!";
mail.Recipients.Add("alias");
mail.Recipients.ResolveAll();
mail.Send();
OutlookNs.Logoff();

Sarah, you’re no Hillary Clinton

“I know Hillary Clinton. Hillary Clinton is a friend of mine. And Sarah, you’re no Hillary Clinton.”

The McCain adviser who leaked that Palin did not know that Africa was not a country comes out of the closet...

http://www.eisenstadtgroup.com/2008/11/10/eisenstadt-the-source-for-sarah-palin-africa-leak-and-proud-of-it/

Reading the comments is spectacularly amusing. How did this country came to having an entire political party that viscerally resents intelligence?

----------------------
Update: likely a hoax. http://www.huffingtonpost.com/2008/11/10/martin-eisenstadt-non-exi_n_142785.html.

The comments are nevertheless hilarious.

Getting back your lost files with Vista

If there is one good thing about Vista compared to XP, it's the new storage stack, which is now on par with the server operating systems such as Windows Server 2003 and 2008.

I am still due to write a post about how to write image-based backup using VSS (volume snapshot services), but just now I used it to save a file that I thought I'd lost.

I was editing code in Visual Studio, hitting Ctl-S to save it every few modifications. Suddenly, a window popped us inviting me to save Program.cs to an alternate location. I cancelled out (first mistake), closed VS (second mistake), and went to the part of the tree where the file was. It was showing in the explorer, but I could not open it.

"Vista..." - I sighed, and rebooted the computer (in reality, I don't have a proof - or even reason to believe that it was Vista that ate the file - it is more likely a bug in Visual Studio that is to blame).

When it came back, the file was not there. Damn. Another reboot, but with "chkdsk /f c:" now to check the file system. No errors, and the file still not there. Luckily, I remembered about VSS and how it is now a part of the client operating system.

What is VSS, or "Volume Snapshot Services"?

It's a service in the operating system that periodically detects which clusters on the disk are used by the file system. Then, when any of these clusters is overwritten, before writing the new data, it copies the old information in an area of the disk reserved for it, and records it.

Then, the (read-only) image of the original (at the time of the snap) disk state can be reconstructed and mounted. So if the file system got corrupted, or a file overwritten or deleted, you can get your data back.

To do it, right-click on a file (if it's still present), or a folder, or a drive, and click on "Restore Previous Version..." menu item.



A dialog box will open inviting you to select from a bunch of snapshots that the system had taken for you at various times in the past:



Select the version, and the normal explorer window will open, pointing to the mounted image of your disk as it existed on the given date.



Find your file, and voila - you can copy it to where it should be.

US citizens are not allowed to boycott Israel?

http://www.axisoflogic.com/cgi-bin/exec/view.pl?archive=36&num=3915

WTF???

The next Depression

This promises to be a crisis of epic proportions.

People are comparing it with the Great Depression, but there is one huge difference: during the Depressions US still had it manufacturing base intact. It was a financial and social crisis, but the country had robust physical infrastructure to cope with it.

This is not the case today: US infrastructure is largely gone, and we're very quickly losing our R&D base.

I've realized how far this has gone when I took a trip to Taipei in 2003. For a large part of its history, Taiwan was one big factory. Products were designed in the United States and built here. By 2003 however manufacturing has moved on to the mainland China, and R&D moved in: we visited 5-6 companies that were building routers, and they were all purely R&D shops with manufacturing outsourced to China.

Quanta Computer in Taiwan is the largest laptop company in the world. You probably have not heard about it: the laptops that it designs (in Taiwan) and makes (in China) usually sell under Dell, SONY, and HP brand names. In 2006 it produced roughly 60% of all laptops made in the world. It also makes servers: about 40% of them.

Back in 2006 the joke that went around Microsoft was this: "Dell people think that they outsourced the manufacturing to Quanta, and Quanta people think that they outsourced marketing and distribution to Dell". Very true: a lot of what's left in the US is just that - marketing and distribution. We no longer make stuff - we just buy and sell it. With the money borrowed from Chinese.

The end of 90s were a brief respite: the pyramid scheme of .com boom had temporarily returned engineering to the top. Not for long: the enrollments into computer science programs are now below 1995 levels, US high school scores in math and science rank around 20th place in the world - below most European and a few Asian countries.

The universities churn out people who are largely incompetent. When I was interviewing graduates in China, it was immediately obvious that although there was a wide distribution in intellectual capabilities, they have all studied hard and came out of school with solid knowledge of computer science and math.

When I interview college graduates here, there is the same wide variety of IQ - and almost universal lack of basic training. At school they mostly had worked in large teams where there was one nerd that did everything, and everybody else managed process and collected customer requirements.

Our social protection net is in probably worst shape it's ever been since the New Deal. The unevenness of income distribution is as high as it's ever been: an average CEO makes 400 times the income of an average worker. This is compared to 20-40 range it's been throughout much of the history: the growth took off when Reagan lowered the upper tax bracked from 70% to 28% (it has since moved back a little to 36%).

The income distribution here is more similar to the Middle Ages than to any modern developed society.

The insane compensation packages at the top are the source of our current predicament: they incent a rationally-behaving CEO to maximize the short-term profit (after which he or she simply cashes out) at the expense of the long-term risk.

Meanwhile, the number of people lacking basic health insurance is as high as it's ever been, and the percentage of people who qualify for government assistance is as low as it's been as well, with state budgets weakened to the breaking point.

So now that we're in financial crisis, we don't have either manufacturing, or R&D capacity to lean on, nor health care or protection net to carry us through the bad times. To wit: even illegal immigration from Latin America is at its lowest point in decades.

We're in deep trouble.

So what do we do? The future depends on the new Democratic administration's willingness to take the reins, and do it vigorously.

They have the power to do a lot: strong majorities in Senate and House, and the Presidency. What they do not have is time: if the country is not on the right track in two years, if there's no improvement in the situation - beyond any reasonable doubt - they will lose the mandate, and the Obama government will be powerless.

Democrats should be quick, effective, and ruthless. Any talk about bipartisanship must end now: there's simply no time to reason with the Republican wackos in Congress - the moderates of that party are now largely gone anyway, and what's left are the crazies from the slave states, both geographically and mentally.

First and foremost, the Democrats need to take control of the media. They should push through legislation that breaks down enormous conglomerates that serve primarily corporate interests, the Rupert Murdoch epmire being first and foremost focus. Something akin to trust-busting efforts of Teddy Roosevelt, only directed at Faux News.

The discourse in the country must move away from infotainment that feeds the idiots the line that they are the greatest, smartest, most hardworking idiots in the best country in the world to actual information and informed deliberation. And yes, fewer ads :-)!

In parallel, they should break the lobbying industry. Now or never! The Deomcrats are at an enormous advantage today: with the strength of the grass root, they can survive (and win) without the lobbyists, unlike the Republicans. This will help the Democrats to solidify their power, and the government to start making the decisions that are actually good for the people, not industries and foreign countries.

Nationalize the health care. The elephant in the room that is never mentioned when the rising costs are discussed is that the lion's share of the money goes to the doctors. It's not to the malpractice insurance (<10%). Not the equipment, and not the drugs: the bottom quartile of doctors in this country make more money than the top quartile of the engineers. There's no reason for this other than the current broken health insurance system, which disintermediates the costs.

Fixing the health care is one of the few instances where the interests of the corporations and the people are identical (the fact that the companies have to pay the health care costs of their workers in the US puts them at a huge disadvantage compared to their competitors in Europe). The nationalized healthcare will give the voters immediate benefit, and the reason to vote for Democrats again in two years.

Invest in the education - big time. Literally, pour money into it. The teachers should have one of the best paying jobs in the country, and right now it's one of the worst. As a result, US's Sunday school education is better than the regular one, and we keep churning out religious fanatics. Better education correlates negatively with voting Republican, so investment here should help Democrats keep on their power.

Create massive programs for public works - and research investments. Energy independence, putting a man on Mars, rebuilding the physical infrastructure of the country - all at once. Massive spending is the only thing that can arrest the spread of the financial crisis, keep it from turning into multi-decade depression. Luckily, we're so behind in terms of progress, that the opportunities abound.

Finance these public works by devaluing the dollar and raising the taxes in the upper income bracket. The dollar os way too expensive, and it does not help: the paper should not stand in the way of the progress. Especially when much of the paper is held by foreing investors anyway :-). Cheap dollar will help revitalize manufacturing and R&D here in the US.

Higher taxes will restore the right set of incentives for the executive class: instead of get-rich-quick scheme, being a CEO will be what it was before here, and what it is now anywhere else in the developed world: a respectable, highly-paid job with high responsibilities.

Protect domestic manufacturing and R&D with tariffs. It's a myth that a country can live on moving the borrowed money from one pocket to another. To survive, we need to start - for once - making something useful again. And there's no way to do it without the base. Let's have the industrial revolution again!

Finally, investigate the abuses of the previous administration. Iraq, the torture, the wiretapping, the role of lobbyists in government must not be swept under the carpet: there's a wonderful opportunity to (1) get a lot of Republicans in jail where they belong, and (2) create and reinforce in the public mind the mental equivalency between the words "Republican" and a "crook". This association could last for a generation or two - it is very much in Democrats' interests to not pass on the opportunity.

Notes for the theory of computation class

http://www.cs.colorado.edu/~main/theory.pdf

Enough is enough...

Good riddance!

Mercer for President!

Lee L. Mercer Jr. Welcomes You!

My Campaign Theme

The United States Government must regulate government sleepers and government regulations authorized thought, ideas, acts, actions, rights, wrongs, controversies, facts, issues and circumstantial evidence through intelligence research, law research, law enforcement research and criminal law research implementing ROTC communications research innovating education national and international.

My Platform

My platform for President of the United States Of America is Criminal Law. It is developed from my Method of Education. I was ordered to create and or invent by the United States Army that is now intact regulating the United States Government protecting it through Military Intelligence Computerization Management a new Disipline I invented and the Administration of Criminal Law Laws across the board.

My Direction

The United States Government education will perfect the vindication of it’s government regulating it’s directions and resources requirement by the law only.


http://www.mercerforpresident2008.com/home.html

Beginning SQL Server 2008 for Developers: a book review

http://www.amazon.com/Beginning-SQL-Server-2008-Developers/dp/1590599586/ref=sr_1_1?ie=UTF8&s=books&qid=1225678034&sr=8-1

Short version: this book's information density could be much improved.

Long version: I picked this up because I am building a database app (a little snippet app which the devs on my team could use for status reports), and I needed to learn about SQL Server. I never used it before, so the book looked like a match: I am both a dev, and a database novice.

On the positive side, this book does provide information. On the negative side, the information it provides could be packed in quarter of the space. It is very, very verbose. While reading it, I could not shake off a feeling that the author was trying to fill (contractually?) pre-determined number of pages.

The book is actually very basic: it shows you how to create a database, but does not go into depth on any topic it discusses: best practices, performance considerations, and SQL 2008 specifics are some times mentioned, but never covered.

It does show you how to create a database (using UI and TSQL), add a table, set up basic relationships, back the database up, and add data: the minimum one must know to start using the database, but not enough to make one a database developer.

Lacking the depth (and breadth), the author compensates by sheer verbosity. Every dialog used is presented as a screen shot (no less than 25% of the book consists of these screen shots). Every parameter of a SQL statement is given a description, usually a perfunctory one:

• DESCRIPTION: A description of the backup (p.193)

Sometimes, it gets to ridiculous proportions. On page 191 there is a half-page template for the BACKUP DATABASE statement. In the next paragraph, the same half page of text is repeated again - with one line added to specify a file group parameter.

Here is its description of the SQL backup modes, page 188:

"The first possibility, full backup, is straightforward. Selecting the Full option ensures that the whole database will be backed up. All the tables, the data, and so on are backed up if we use this option. We also back up enough of the transaction log (transactions committed via code, but not physically applied to the relevant tables at the point of backup). This backup would be seen as any starting point for a restore from a database failure.

The second possibility is the differential backup, also known as an incremental backup. Use the Differential backup option when the circumstances are such that a full backupis either not required or not possible. This just performs a backup on data changed since the last backup. For example, we may take a full backup on the weekend and then a differential backup every night. When it comes to restoring, we would restore the weekend full backup, and then add to that the latest differential backup taken from that point. A word of warning here: if you do take a week's worth of differential backup, and you back up to magnetic tape rather than a hard drive, do keep at least enough different tapes for each differential backup. Therefore, use a complete backup when necessary and then in the interim, use a differential backup."

What this says in so many words is: full backup backs up the entire database (duh!), and differential backup backs up the data changed since the full backup. What it does not say is when is differential backup really should be used (other than "when appropriate"), how does the system guarantee the consistency of the data on backup (it does it by using VSS), etc.

How annoying all this really is depends on how the individual reader processes the information. I ended up in a mode where I was quickly scanning through the text in search of information. I lost a bit of time doing this, and I probably glossed over a thing or two that was important, and worth knowing. The book is still probably better than reading the documentation, but if I were buying it again, I would buy something else.

Classical Computer Science Texts

http://e7l3.org/ref/classics.html

Startups in the down economy and venture financing

I've recently read an article by Paul Graham about starting a company in bad economy (http://www.paulgraham.com/badeconomy.html). The gist of his position is that founders and the business idea are far more relevant to startup's success than sources of finances, and that a lot of formidable companies (Apple, Microsoft) were founded where when the economy was not exactly booming.

His position is logical of course, but it is interesting to note that a sound startup that is cash-positive on day one, or early enough, does not require venture capital. Neither Apple, not Microsoft had VCs as their shareholders when they went public, and the founders still owned vast majority of stock on IPO - something that happens very rarely today. I suspect - although I did not check - that the same is true for Sun, Oracle, Intel, SAP, and most other companies of the pre-dot-com generation.

And of course most of the crap VCs bring to market these days is companies without a revenue model (with the business based on attracting the eyeballs, not dollars), and they get sold either to the unsuspecting public, or a different company (i.e. the greater fool) before they make their first dime. No matter - one could always count that the greater fool would be found, and separated from their money.

The more entertaining read, however, was not the Graham's article itself, but the reaction it caused in the Silicon Valley "businessmen" (http://uncov.com/paul-graham-shut-your-face).

I was surprised by the level of vitriol, before realizing that the world of the 20-something founders must be crumbling: indeed, the immediate future is probably going to be tough for Intels and Microsofts of the world, but for the "eyeballs"-centered world of the Facebook apps vendors it will be nothing short of a mass extinction event.

People keep paying for the tools through the recessions, but the marketing budget - and with it, the Facebook ad revenue - gets cut first. Watch Google earnings for the next couple of quarters...

Anyway, Graham's post and the reactions to it are things of the not so distant past. The reason I remembered them was because today I read another article, this time by Thomas Friedman (http://www.nytimes.com/2008/10/26/opinion/26friedman.html), in which he argues for the government to play passive role in the banks in which it takes ownership. The question he poses is - would a government official give Larry and Sergey a loan to found Google.

As he often is on economic issues, Friedman is full of it. The whole reason we're having a crisis is because the banks that are part of the country's monetary system are also engaged in extremely risky speculative activities. Such as venture financing, derivatives trading, and so on. Specifically - because they are currently lending money to Larry and Sergey.

This was not always the case - before 1999 banks could not do any of these high risk activities, but it meant that the depositors' money - and the country's monetary system - were safe. Venture financing, complex financial instruments, insurance were handled by organization that had these functions - not banking - in their charter. All VCs could go out of business simultaneously, and nobody - outside the startup world - would have noticed.

Incidentally, this regulation framework was introduced during the Great Depression, as a reaction to activities that caused banks to fail during that era.

Gramm-Leach-Bliley Act of 1999 - heavily lobbied for by Greenspan, and written by McCain's former economic advisor - allowed banks to enter the businesses that were previously verbotten. As a side effect, it poured a lot of money into the .COM startup market, and later into the subprime mortgages, and thus precipitated the current collapse.

So of course government officials should not be deciding whether to lend money to a young startup - because a bank should never be lending money to uncollateralized ventures in the first place. This is the business of a venture financing firm, an angel investor, the founders themselves, but not a bank.

A bank should be - and for the most part of the recent history it was - an extension of Fed, and of course it is perfectly reasonable for a government official to sit on something so close to Fed and make decisions that impact the country's monetary system!

Wassup 2008

Titanic

An ad in Micronews (Microsoft's internal newspaper) circa 1998: "Titanic for sale. Tape 1 watched once. Tape 2 never watched."

Movies to see

Since I've been mostly critical of the movies I've seen recently, I figure the counterbalance post listing the movies I like is long overdue. Otherwise I am starting to sound like a crumudgeon.

So, in no particular order...

Fantasy

• Pan's Labyrinth (http://us.imdb.com/title/tt0457430/)


• Big Fish (http://us.imdb.com/title/tt0319061/)


• Stardust (http://us.imdb.com/title/tt0486655/)


• The 10th Kingdom (http://us.imdb.com/title/tt0207275/)


• Skazka Stranstviy (http://movies.nytimes.com/movie/158356/Skazka-Stranstviy/overview)


• Master I Margarita (http://www.imdb.com/title/tt0403783/)

Comedy

• Idiocracy (http://us.imdb.com/title/tt0387808/)


• Office Space (http://us.imdb.com/title/tt0151804/)


• Severance (http://us.imdb.com/title/tt0464196/)


• Fun with Dick and Jane 2006 (http://us.imdb.com/title/tt0369441/)


• Fun with Dick and Jane 1976 (http://us.imdb.com/title/tt0076059/)


• Get Smart (http://us.imdb.com/title/tt0425061/)


• Dr. Strangelove (http://us.imdb.com/title/tt0057012/)


• Addams Family (http://us.imdb.com/title/tt0101272/)


• Addams Family Values (http://us.imdb.com/title/tt0106220/)


• It's a Mad, Mad, Mad, Mad World! (http://us.imdb.com/title/tt0057193/)

Comedy/Drama/Romance - all in one

• Train Of Life (http://us.imdb.com/title/tt0170705/)


• Black Cat, White Cat (http://us.imdb.com/title/tt0118843/)


• Kramer vs. Kramer (http://us.imdb.com/title/tt0079417/)


• Rain Man (http://us.imdb.com/title/tt0095953/)


• The Graduate (http://us.imdb.com/title/tt0061722/)


• One Flew Over The Cuckoo's Nest (http://us.imdb.com/title/tt0073486/)


• Moscow Does Not Believe in Tears (http://us.imdb.com/title/tt0079579/)


• K-Pax (http://us.imdb.com/title/tt0272152/)


• Goya's Ghosts (http://us.imdb.com/title/tt0455957/)


• Life is Beautiful (http://us.imdb.com/title/tt0118799/)


• Shakespeare in Love (http://us.imdb.com/title/tt0138097/)


• The Shawshank Redemption (http://us.imdb.com/title/tt0111161/)


• Y Tu Mama Tambien (http://us.imdb.com/title/tt0245574/)

Animation

• Castle In The Sky (http://us.imdb.com/title/tt0092067/)


• Spirited Away (http://us.imdb.com/title/tt0245429/)


• Princess Mononoke (http://us.imdb.com/title/tt0119698/)


• Howl's Moving Castle (http://us.imdb.com/title/tt0347149/)


• Grave Of The Fireflies (http://us.imdb.com/title/tt0095327/)


• Corpse Bride (http://us.imdb.com/title/tt0121164/)


• Ice Age, 1 (http://us.imdb.com/title/tt0268380/) and 2 (http://us.imdb.com/title/tt0438097/)


• Tripping The Rift (http://us.imdb.com/title/tt0315081/)

Adventures

• V For Vendetta (http://us.imdb.com/title/tt0434409/)


• Kingdom of Heaven (http://us.imdb.com/title/tt0320661/)


• King Arthur (http://us.imdb.com/title/tt0349683/)


• Gladiator (http://us.imdb.com/title/tt0172495/)


• Troy (http://us.imdb.com/title/tt0332452/)

Horror

• Hostel (http://us.imdb.com/title/tt0450278/)


• Hostel 2 (http://us.imdb.com/title/tt0498353/)

War

• No Man's Land (http://us.imdb.com/title/tt0283509/)


• K19: tThe Widow Maker (http://us.imdb.com/title/tt0267626/


• Saving Private Ryan (http://us.imdb.com/title/tt0120815/)


• Lord of War (http://us.imdb.com/title/tt0399295/)


• Schindler's List (http://us.imdb.com/title/tt0108052/)


• The Quiet American (http://us.imdb.com/title/tt0258068/)


• All Quiet On the Western Front (http://www.imdb.com/title/tt0078753/)


• Stalingrad (http://www.imdb.com/title/tt0108211/)


• Das Boot (http://www.imdb.com/title/tt0082096/)

Epics

• Cleopatra (http://us.imdb.com/character/ch0026970/)


• Doctor Zhivago (http://us.imdb.com/title/tt0059113/)

Kurosawa

• Kagemusha (http://us.imdb.com/title/tt0080979/)


• Ran (http://us.imdb.com/title/tt0089881/)


• Rashomon (http://us.imdb.com/title/tt0042876/)


• Sanjuro (http://us.imdb.com/title/tt0056443/)


• Seven Samurai (http://us.imdb.com/title/tt0047478/)


• The Hidden Fortress (http://us.imdb.com/title/tt0051808/)


• Throne of Blood (http://us.imdb.com/title/tt0050613/)

Yojimbo (http://us.imdb.com/title/tt0055630/)

Sergio Leone

• The Good, The Bad, And The Ugly (http://us.imdb.com/title/tt0060196/)


• A Fistful Of Dollars (http://us.imdb.com/title/tt0058461/)


• For A Few Dollars More (http://us.imdb.com/title/tt0059578/)

Teleseries

• John Adams (http://us.imdb.com/title/tt0472027/)


• Rome (http://us.imdb.com/title/tt0384766/)

Russian Specifics (could be hard to understand if you were not born there before Perestroika)

• Irony of Fate (http://us.imdb.com/title/tt0073179/)


• Office Romance (http://en.wikipedia.org/wiki/Office_Romance_%28film%29)


• O Bednom Gusare Zamolvite Slovo (http://us.imdb.com/title/tt0081256/)

Religulous: A review

We watched Bill Maher's "Religulous" this Friday. The movie had a few entertaining moments but overall it was a little bit heavy on Bill's own persona. Maher is great at editorializing, but this was a two hour movie - and it felt like over half of it was about Bill himself.

I wish we could let his subjects talk more - I have a hunch that what they would have said then would be a lot funnier.

As it were, most of the interviews followed the following template:
Interviewee: <some insanely crazy stuff>
Bill Maher: "You can't seriously believe this crap..."
Interviewee: Yes I can...
Bill Maher: C'mon... You can't.

The one moment I really enjoyed was the interview with the senator from Arkansas, Mark Pryor, where he says that one doesn't have to pass an IQ test to get into the Senate. The absolutely priceless part is watching him right after he says it, and see how he slowly realizes what he'd just said, as the smile fades away from his face...


The other were a few quotes from Founders, to illustrate what they actually thought about Christianity (that the United States of America was founded as a "Christian nation" has long been the working theory of the right wing, and a platform of Texas Republican Party).

Here are the quotes:

Lighthouses are more useful than churches.
  -- Benjamin Franklin


Christianity is the most perverted system
that ever shone on man.
  -- Thomas Jefferson


This would be the best of all possible worlds,
if there were no religion in it.
  -- John Adams

Of course, the movie will not win any converts. The people who believe in the "talking snake" will probably never even see it. But as far as the entertainment value for the people who don't - I would say wait for the DVD.

When the Democrats are in power, their first action should be to break up Fox News...

AMEX Centurion

http://blogs.moneycentral.msn.com/smartspending/archive/2008/10/22/all-about-the-mysterious-black-card.aspx

So apparently there are people who, while paying $2500 yearly membership fee for a credit card, do care about the reward points accrued when they rent a limo...

"Let's hope we are all wealthy and retired by the time this house of cards falters...''

Oct. 22 (Bloomberg) -- Employees at Moody's Investors Service and Standard & Poor's privately questioned the value of some mortgage-backed securities that were given creditworthy ratings, saying they created a ``monster,'' according to e-mails released by a U.S. House panel.

http://www.bloomberg.com/apps/news?pid=20601087&sid=a2EMlP5s7iM0&refer=worldwide

Dear Red States...

"We've decided we're leaving. We intend to form our own country, and
we're taking the other Blue States with us."

http://www.craigslist.org/about/best/sfo/80714812.html

Interviewing for fun and profit

I've been interviewing a lot lately - starting the Windows Home Server project, helping build Servers and Tools division in China, serving on Seattle hiring committee at Google, and now building my new team. All this meant spending anywhere from 2-3 hours a week at Google to 7-8 hours a day in China on recruiting activities.

From Seattle to Lahore, from Google office in Fremont to Microsoft campus in Shanghai, the candidates that end up getting offers look very similar. This post is about how to be that kind of a candidate.

Getting a job at a good software development company is like being a genius - it is 1% inspiration, and 99% perspiration. This means a lot of work, vast majority of which must happen before the event. The good news is that this work is portable - once done, it opens a lot of doors, and puts you in a position where you choose where you want to work, not the other way around.

So without much ado, let's delve into the gory details.

Prework - long term

All interviewers look for basically the same thing - candidates that are "smart and get things done". There is no good way to determine one's intelligence, so we use an approximation. We ask the interviewees to solve complex algorithmic problems on the whiteboard. Luckily, unlike intelligence, this ability can be trained.

Unfortunately, most software development jobs, with very rare exceptions, do not provide on-the-job training for algorithmic skills. A colleague of mine says that there are two types of developers - people who can call APIs, and people who can build APIs. Vast majority of the jobs in this industry are about calling APIs. The most highly paid jobs, however, are about building them. Most teams at Microsoft, Google, and the like are looking for people of the second kind.

If you are not already working in the team that builds APIs, you need to learn to love computer science as science. Get interested in algorithms design. Learn about computer architecture. Start paying attention to how the functions you're calling are implemented.

Do you know what's actually inside the HashMap class you've been using all the time? What algorithm does STL use for sorting? Why? When is heap sort faster than the merge sort? What does a modern computer really do when reading a byte out of memory?

If you live near a good university, take a class every few years to keep connection with the scientific side of the software development.

In an earlier article (http://1-800-magic.blogspot.com/2007/12/recipe-for-getting-employed-by-google.html) I claimed that reading and doing the exercises in three books will get you a job in any good company. I still believe this to be the case - but you do need to actually read these books, and do the exercises. Just buying and putting them on the shelf will not work!

The "getting things done" part during the interview is usually proxied by having the candidate to write a lot of code on the white board. The better the code looks - syntactically correct, concise, professional - the more likely that the candidate has written a lot of code in his or her career, and therefore, got more things done.

One can see the difference when doing something as simple as copying a string:

while(*d++ = *s++) // Good!
  ;

vs.

 // Generates the same binary code,
// but too much text!
int i;
for(i = 0; s[i] != '\0'; ++i)
{
  d[i] = s[i];
}
d[i] = s[i];

(This is where STL with its "string s1 = s2;" really gets you :-).)

The only way to good code is to practice writing - a lot! Jump into the parts of your project that generates as much code as possible. If you're a developer, and you're not writing at least 20Kloc/year, your coding muscle does not get enough exercise. Get a coding project at home - write a piece of automation software for your light switches!

Prework - short term

The above suggestions are designed to get and keep you in shape as a software developer. They maintain the set of skills (AKA coding) that makes you employable. However, even great software engineers often do not get the job they want because there are other things that are required for a match.

I've been asking every person who I interviewed for various management positions at Microsoft what they are looking for when interviewing people. Every time the passion for technology was coming out very close to the top. What this means is that ideal candidate (1) wants to work in this general area, and (2) wants this job.

As an interviewee, you must have have plausible answers for both.

The best (and the only) proof that you want to work in this general area is to demonstrate that you invested time to research deeply the technologies/customers/competitive landscape/standards/other subcurrents that shape this part of the industry. Buzzwords are not enough - deep knowledge to an extent where one knows enough to form an educated opinion is required. I've seen many a qualified candidate being rejected because they gave standard, cookie-cutter answers to questions which an interviewer had considered too shallow.

SOAP is a great technology? Why? Because it uses XML? And why is that a plus?

The best (and the only) proof that you want this job is that you've spent time to learn about it. A lot of information is available publically. For example, if you're interviwing for a position of a program manager at Microsoft, it helps to know what this job really is.

Whenever I go on college recruiting trips for Microsoft, the most amusing thing is interviewing candidates for program management positions. A program manager's job does not actually involve management, nor it is any more senior compared to test and dev disciplines. Both of these aspects are described in detail at both the Microsoft recruiting web sites, as well as on the application for the college interviews. Yet 90% of the college candidates that I interviewed on campus assumed that (1) they will be managing people, and (2) right out of school they were entitled to do this.

If it's an internal transfer within the same company, more often than not most of the information about the project is available. I once had a case where a candidate knew more about the history of the project, the market, and the competition - without ever having worked on it - than I did. Needless to say, she got the job :-).

More often than not, recruiters and interviewers will go out of the way telling you directly what will be on the interview. An email that I've got from Google when I interviewed there listed all the areas where the interviews focused. Microsoft recruiters give out no less details. I am telling every candidate, before meeting them, about the areas where the interview will go. You have no idea how many people completely ignore the advice!

If, for example, it's written on the web site, in the recruiter email, and in the interview packet that THE INTERVIEW WILL INVOLVE WRITING CODE ON THE WHITE BOARD, and when you show up at the company and are asked to write code on the white board you say that your coding is "rusty", what outcome do you expect? It's a double-whammy: not only do you not match an explicit qualification for the position, you have wasted company's time and money by coming to the interview, because you apparently have not even bothered to read the description of the job!

Do prepare good, deep questions about things that you should want to know, but cannot. Avoid cliches - believe me, "Describe your typical day as a developer at X" does you no good. This question has been asked many times. Too many. The only reaction the interviewer has to a question like this - "Oh, here we go again..."

The day of the interview

Don't be late. It's unprofessional. Plus it cuts into the time that is available for you to solve the problem :-).

Don't use pseudocode, unless the problem requires it (many graph algorithms do). If you're a great coder, you should be writing in a real computer language with ease. Plus we warned you that (1) we're looking for great coders, and (2) you will be coding on a white board.

Avoid syntax errors when coding. Well-written, well-formatted code is classy and shows experience.

Do be VERY detail-oriented. Check and recheck what you're writing. Do not shrug off the "off-by-one" errors. If you're "always messing up some mundane detail" ((c) Office Space), you should not be in this trade. Messing up mundane details in the past had caused spaceships to miss their destination, to say nothing about expensive product recalls.

Do listen to what the interviewer says. This cannot be overemphasized! They are here to give you cues, to help you. Most of the time they genuinely want you to succeed. They know their problems. If an interviewer makes a suggestion, TAKE IT. (S)he knows!

Do test your code. After the code has been written, think about small, but exhaustive set of test inputs that would exercise all the code paths in your function, and elicit all the differences in behavior. For example if you're writing a binary search, test it on inputs with odd and even number of elements. An algorithm that takes an array should work well on inputs of size 0, 1, 2 and 3. Test the inputs that might cause overflows and out of bounds conditions.

Do step through the code as written, not as you think it should work. This is the most frequent error that I have been observing - a person would look at the data input and then reason through it with the algorithm (s)he had in mind - but not the one on the white board. Pretend you're a CPU instead. Write a sample on the board, create a column for every variable, and start iterating - mechanically.

Be humble. If you're saying "I am really good at X", this is a challenge to the interviewer. They will give you harder problems, give you less help, and treat the failure harsher because you behaved as an arrogant braggart. It's much better to underpromise and overdeliver than the other way around.

And last but not least, be enthusiastic. Ask questions. Offer suggestions. Demonstrate that YOU WANT THIS JOB.

Speaking of wanting the job.

It helps to set up multiple interviews when you're thinking about changing the job, and sequence them such that the more desireable jobs go last. Interviewing is a frame of mind, and it helps to put onself into the right state. But on the day you go to the interview - even if you aren't 100% sure this is the job you'll take in the end - on this day you have only one goal - to get this job. WANT IT!

Happy interviewing!
-----

By the way - we are hiring!

http://1-800-magic.blogspot.com/2008/07/looking-for-great-devs.html

Joe the Millionaire

The stupidity of McCain's "Joe the Plumber" dominated campaign - and the people who fall for it - is surreal.

Under Obama's plan, the taxes are raised for people with $250k AGI (adjusted gross income).

For the small business owner to pay him- or herself $250k in salary, the small business must bring at least as much in profit - assuming no reinvestment in the business growth, and no other share- or bondholders.

At a very conservative P/E of 10, this means that the business would be worth $2.5M.

With $2.5M in assets, that would make Joe the Plumber a multimillionaire - exactly the constituency to which Republican party caters. Well, that and of course the stupid people who take McCain's claims at the face value...

So... is this really better than Soviet occupation?

KABUL, Afghanistan — An appeals court sentenced a young Afghan journalist to 20 years in prison for blasphemy on Tuesday, overturning a death sentence ordered by a provincial court but raising further concerns of judicial propriety in the case.

The defendant, Sayed Parwiz Kambakhsh, 23, was a journalism student in the northern city of Mazar-i-Sharif and worked for a daily newspaper there. He was arrested last October and accused of printing and distributing an article from the Internet about Islam and women’s rights, on which he had written some comments about the Prophet Muhammad’s failings on that issue.

(the emphasis is mine)

http://www.nytimes.com/2008/10/22/world/asia/22afghan.html?hp

I guess... "freedom" is messy? ((C) Rumsfeld)

McCain and the witchcraft!

Surprised when McCain looks "confused and like an idiot"? It's the witchcraft, of course!



'The occultists are "weaving lazy 8's around McCain's mind to make him look confused and like an idiot". Bree K. said we need to break these curses off of him that are being sent from Kenya.'

http://www.injesus.com/index.php?module=message&task=view&MID=CB007FA2&GroupID=2A004N9G&label=&paging=all

How to build a server for your house

Five years ago there came a moment in my life where I realized that I was being overwhelmed by the amount of data in our household.

We have just bought a digital camera, and started accumulating tons of digital images that needed reliable store. There was also software on CDs that kept getting lost; various random bits of data - code, text, word documents - that spanned the last ten years, and occasionally required retrieving; and mp3 files.

Meanwhile, the number of computers that the household had kept increasing. My wife and I had laptops; my daughters had their own PCs. All were networked using a Windows CE-based router which was also a print, web, and a small file server.

After a brief attempt at using USB drives, I quickly came to conclusion that they are not the answer. There were several problems.

First, it is impossible to say if the data on a drive is good. There were bit errors introduced by copying. A bad sector in the middle of a file would suddenly make it unreadable, but you wouldn't know until you try. And of course occasionally, the whole drive would go bad. At least the later problem is easier to discover, if not fix.

Second, I ended up with lots of them, fast. Because I wanted to protect against hardware failures, I would usually create multiple copies of the data. Then I would forget where the data were, and how many copies I had. Most of the time I would fail to update all copies. Towards the end I started treating disks as a write-once media, creating a new backup every time I remembered to do it, on the theory that one of the copies would survive a disk/sector/bit error.

Finally, USB drives are slow. I don't know where they got the 480 megabit figure - I was getting 10 megabytes per second (100 megabits, if you account for the transport overhead), and even that only when the disk was directly connected to a PC, and there was only one disk. Plug in a couple in a USB hub, and the transfer quickly drops by a half.

So when Microsoft shipped the Small Business Server 2003 and gave a copy of it to every employee of the Windows team, I pounced on the opportunity, bought a second-hand workstation from one of the PC-Recycle shops that spawned around Microsoft, and built my first home server.

It turned out that building various servers for homes had quickly become my hobby. In the next five years I've tried almost every possible variant. From dirt cheap Fry's motherboard/CPU combinations to super-expensive Xeon/Supermicro performance monsters. From SBS 2003 to Server 2008. I've even helped start (as the very first developer, and then dev manager) the Windows Home Server project.

All in all, between home and work I've built perhaps 20 servers. I have 4 of them in my basement right now, totaling roughly 10TB of usable protected storage, doing my web site and email hosting.

The write-up below documents what I learned as a result.

The only caveat is that the software section is entirely Windows-based. Since I work at Microsoft, I can buy our own software for drastically discounted prices at the company store. If I didn't, perhaps there would be an incentive for me to look at what's available in the Linux world.

Or maybe not - one can buy SBS 2003 for around $300 these days, and this comes with Exchange, domain controller, file server, and everything else in one package conveniently wrapped in one installer. Would it be worth for me to learn Linux to avoid spending $300? I don't know. (http://1-800-magic.blogspot.com/2008/10/linux-vs-developers.html).

Anyway, let's get to the meat!

Hardware

There are three priorities that you should follow when building a home server.

First, it must be reliable.

Unlike a personal computer, a home server is always on, and is rebooted extremely infrequently. As a result, it is prone to weird errors that are usually invisible on a typical personal computer - such as spurious bit flips in memory, and during data transfers.

When my home servers were cheap Fry's motherboard/CPU combos, they would usually hang every month or so, and would produce a bit flip in data transfers - usually one per 100-200GB of files copied. It was then when I got in the habit of diffing the source and destination every time I copied files around - and in many cases I discovered that I was lucky I checked.

Therefore, a server motherboard and ECC memory are absolutely required. Say no to cheap desktop hardware - in most cases the motherboard should be in the $200+ range, and make sure it does take ECC memory before buying it.

http://www.newegg.com/Store/SubCategory.aspx?SubCategory=302&name=Server-Motherboards

Second, it must be able to accommodate many internal hard drives.

While you can add external drives, there are several problems with that. First, external drives usually cannot be combined in fault-tolerant arrays, which is the option you will most certainly want (there will be more discussion below). Second, they make the system fragile - they fall off, the cables spuriously disconnect. Third, they create an unsightly - and more importantly, hard to navigate - mess of cables and power supplies that makes the server impossible to move.

Definitely get a full tower case - not a mini- or mid-tower. Make sure that there are fans near the internal 3.5" hard drive bays, so the drives are properly cooled. Check the 5.25" bays - it would be the best if there was no railing in these slots. Smooth walls where the equipment is held in place only by the screws are the most versatile: many SATA enclosures do not have grooves and won't fit into multiple 5.25 bays without modifying the case.

Do buy the SATA enclosure that fits the case - you will want a minimum of 11 drives per server - and there are no cases with 11 internal drive bays.

For example: http://www.newegg.com/Product/Product.aspx?Item=N82E16817119404, or http://www.newegg.com/Product/Product.aspx?Item=N82E16816119014 (but note the absence of the grooves on the second one).

Also, hard drives are power-hungry, especially on boot when they spin up. The power consumption of my server (which has 11 drives) jumps to 500 watts on boot, before receding to around 270 watts in steady state. If 500 watts does not seem like much, consider that much of the power is consumed on the 12V rails, whereas much of the modern power supplies are optimized for the maximum wattage on the 5V rail - the video card.

A 750W PSU is barely enough for 9 drives - get a minimum of 850W for 11, and 1200W if you plan to have 16. Do pay attention that the enough power is available on the 12V rails - allow 40W/drive.

And do get a brand name, this tends to be one of the more important components for the reliability of the system. I used CoolMax and Thermaltake and they were quite reliable.

Third, the server should be QUIET and power-efficient.

These two things go together.

Drives generate a lot of heat, and heat takes many fans to dissipate. Pick out the quietest (12cm or above), yet powerful: if the temperature rises inside the case, the CPU fan (which is small, and therefore requires more RPM) tends to sound like a rocket engine. Even if the server is in the basement, you probably do not want it to be the most audibly prominent thing there. And forget the closet, unless it has proper ventilation :-).

Do get a CPU with a decent power management. Currently, 3000-series Xeons seem to have a decent tradeoff between the power consumption and the compute power.

Also keep in mind that a big server might dissipate ~250-300W of heat. This is equivalent to 1/2 of a portable oil heater, operated on low. And it's always on. Check the electricity prices in your neighborhood :-).

What is less important?
Rejoice - there ARE a few things you do NOT need to care about. They are usually the things you care most when building a desktop machine.

Unless you plan to run a virtual farm, CPU computing power is not important. But CPU I/O bandwidth is. Most desktop processors optimize for the former. Many server processors optimize for the later.

As of this writing, I would recommend Xeon 3xxx series. They are relatively cheap, have great power efficiency, and more than enough juice to run any Microsoft server OS. They do not support dual configuration - but two cores are plenty for serving files.

Obviously, you don't care about graphics - a built-in video adapter is available on most server motherboards, and it's more than enough.

More controversially, I do not recommend spending money on hardware-based RAID controllers.

(A short aside about RAID.

There are 3 most popular modes of combining multiple hard drives into one storage array for speed and/or reliability. All three can be accomplished in software - by storage driver doing the work - or in hardware - by the controller card doing the work.

RAID0 places half of the data on one hard drive, and one on another, in stripes. As a result, you are writing your data on two disks simultaneously and effectively at twice the speed. But if any one drive fails, all your data is gone - the probability of the array failure is twice the probability of a failure of a single drive. Because the task can be accomplished by scheduling two DMA transfers in parallel, the best hardware is not any faster than a software implementation.

RAID1 places the same copies of the data on both hard drives. If one fails, the other survives, and the array can be repaired by substituting the failed drive for a new. RAID1 arrays are very reliable - but you pay twice for protecting the data. Hardware and software speeds are the same for the same reason as above.

The best way to visualize RAID5 is to imagine a system of 9 drives where the first 8 store 1 bit of every byte, and the 9th stores the XOR of all the bits. If any of the drives fail, its contents can be regenerated from the other 7 and XORs (and if XOR drive fails, the mask can be regenerated again from the data). Note that excluding the cost of generating a mask, such a system could be 8 times faster in transferring the data. This is not how the RAID5 system really works - the data is striped, and XOR is computed from the corresponding stripes - but it's close enough. Real RAID5 arrays can consist of any number of disks greater than two - N-1 drives store the data, and Nth drive stores the XORs (again, it's more complicated in reality, but we'll skip the gory details).

RAID5 protects data for much less than RAID1, because of N drives only one is wasted. For this reason alone you will want to use RAID5 in your home server. Given a powerful CPU, a good software implementation could give RAID5 hardware a run for the money. As we will see below, the quality of software RAID5 stack in Windows Server varies.)

All versions of Windows Server support software RAID5. I am sure, Linux support it as well. Note that if your raid controller is cheaper than $400, chances are that its RAID5 implementation is in software, not hardware, anyway.

The most important consideration about hardware RAID is that it protects you from a disk failure, but not from the failure of the controller itself. Because the format of the data on hard disks managed by hardware RAID depends on the RAID card, the RAID controller cannot be substituted for a different model. So if your controller dies, so does your data if you can't get the exact replacement - which is very likely if it dies after a few years.

For this reason I prefer using software RAID - it is standard, it is hardware independent, and if your computer dies, you can put the disks in another computer and the array will be recognized.

I typically have 2 arrays of 5 disks each per server, plus one small (200GB) system disk.

Speaking of disks - more disks that are smaller have better reliability and better price/performance than very big expensive disks. I usually go for whatever is available around $100, which is typically two sizes below the current maximum.

For example, as of this writing the biggest available disk is 1.5T. 1T retails for ~$150, and 750GB drives can be had for ~90. 750GB would be what I'd use. For just below $1000 in drives, you'd get 6TB of usable protected space (10 drives in two 5-disk RAID5 arrays, 3TB of usable storage per array).

The memory. You don't need to stuff your computer with memory. For any reasonable home usage, 2-4GB is enough. For a corporate environment where hundreds of people may be using the system concurrently, this is obviously not true, but for home, it is really enough.

So in short, hardware wise, the rules are simple: big case, big power supply, server motherboard, 2-4GB of ECC memory, one midrange Xeon CPU, and simple, run-of-the-mill SATA controller card, with as many cheap disks as possible in software RAID5 arrays.

Software

Again, this only covers the Windows options. I do not know much about what's available on Linux.

Windows Server 2008

Pro:

• Works really well as a file server.


• RAID5 implementation is very good.


• Lots of useful data-management features.


• Fast.


• Secure.


• Available to current MSDN subscribers at 0 incremental cost. You keep your license key even if your subscription ends.

Con:

• If you don't work at MSFT, it's $800, plus CALs.

Windows Server 2003

Pro:

• Cheaper than Server 2008 - can be had for $500.

Con:

• Software RAID5 implementation is crappy. Expect ~15MBps throughput.


• $500 is still a lot of money.

Windows Home Server

Full disclosure: I was the first dev and then the dev manager for the v1 of WHS.

This is an interesting product, and its target market is, well, the homes in need of a server. It is relatively cheap ($140 in Newegg, although it is an OEM version, which means that once registered, it is tied to a motherboard). It comes with a great backup solution, a UI custom-made for home server scenarios, and a nice remote access portal.

Its weakness is the storage system.

It is a conceptual variant of RAID1 - you can have some of your files duplicated. If you have a lot of data though, you're paying twice for its protection - for me, it would not just be prohibitively expensive, but a physical impossibility of cramming so many disks into the server chassis. The recommended way to expand Windows Home Server is via external USB disks, but (see above) I do not recommend doing it because it makes the physical installation fragile.

And because the UI and all other subsystems in Windows Home Server are tied to its custom storage management solution, you can use native Server 2003 software RAID5, but you would then have to keep these disks on the side and leave them unconfigured in the Windows Home Server UI. You would not be able to put backups on them.

To do this, install Windows Home Server on a computer with a single big hard drive. Do not create any shares, and do not put any data on the shares created for the users. Then, AFTER the installation, add more disks, and configure them in RAID5 arrays by using native Server 2003 UI. Leave these disks unconfigured in Windows Home Server UI. You will also need to use native UI to share them out.

Another way to use Windows Home Server is by putting it on one big, hardware-managed RAID5 system. You do have to pay more for hardware and be mindful of possible controller failure, but given a substandard performance of 2003 software RAID, you will win on performance.

A relatively small disadvantage of Windows Home Server compared to other options is its inability to join the domain, or be a domain controller. This had to be done to justify the much lower price.

Small Business Server 2003

This is the product with potentially the best price/performance ratio that has ever come out of Microsoft. It is a full implementation of Server 2003 with Exchange server, Sharepoint, and a nice remote access portal, and you can buy a retail version of it for a mere $300 (go to www.pricegrabber.com; search for Small Business Server 2003 Standard; be careful - a lot of options are CAL packs, not the software itself; find the RETAIL version - OEM copies are only marginally cheaper, but tied to the motherboard once you register them).

It comes with the installer that sets it up as a domain controller, and configures its components. If a PC has two network interfaces, it can even be used as a router! It serves as a DHCP server, and has a DNS proxy, and even a VPN server built-in, so you can access your home network from the outside.

And yes, you do get a real domain for your home. Your user accounts and passwords are now centralized, and any user can log in to any machine in the household using the same name and the password. It's really cool!

In addition, you can host your own web site, your own email, share the calendar with the family members (Microsoft Outlook is included), etc. Highly recommended!

Two gotchas.

First, do not put any data on the system hard disk (the whole disk, not just the partition): the write cache on that disk is turned off, so accesses are SLOOOOOOOW. This is because of the Active Directory database.

Second, SBS is restricted to be a domain controller: it cannot join any other domain.

Processes

I use the following relatively simple rules to ensure that my data is healthy.

1. I ALWAYS diff source and destination on multi-hundred-gigabyte data transfers. Bit flips do happen when you're dealing with a lot of data. A single bit flip can render a ZIP file useless.


2. I keep all data on RAID5 arrays. I keep multiple copies of truly irreplaceable data, such as digital pictures, videos, and documents. There is one server for such data where two RAID5 arrays are mirrored by hand (and are periodically diffed).


3. Once every few months, I backup the irreplaceable data to a large drive and store it offsite.


4. Truth is always on servers. All other computers may periodically keep a cache, but all modifications are regularly transferred to servers.

If you've made it so far, congratulations! You can build servers for your home for fun and profit, and avoid all the mistakes I've made when building mine :-)!