Every year, researchers from the British Social Attitudes survey ask a representative sample of British people whether they regard themselves as belonging to any particular religion and, if so, to which one? When the survey first asked these questions in 1985, 63% of the respondents answered that they were Christians, compared with 34% who said they had no religion (the rest belonged to non-Christian religions).
http://www.guardian.co.uk/commentisfree/2010/dec/24/religion-respecting-the-minority?CMP=EMCGT_241210
Saturday, December 25, 2010
Sunday, December 19, 2010
The spirit of American revolution is now officially and irrevocably dead
People WANT to be duped by the government. Passionately so.
"The American public is highly critical of the recent release of confidential U.S. diplomatic cables on the WikiLeaks Web site and would support the arrest of WikiLeaks founder Julian Assange by U.S. authorities, a new Washington Post-ABC News poll finds.
Most of those polled - 68 percent - say the WikiLeaks' exposure of government documents about the State Department and U.S. diplomacy harms the public interest. Nearly as many - 59 percent - say the U.S. government should arrest Assange and charge him with a crime for releasing the diplomatic cables."
http://www.washingtonpost.com/wp-dyn/content/article/2010/12/14/AR2010121401650.html
Vladimir Putin, we are finally ready for your leadership...
"The American public is highly critical of the recent release of confidential U.S. diplomatic cables on the WikiLeaks Web site and would support the arrest of WikiLeaks founder Julian Assange by U.S. authorities, a new Washington Post-ABC News poll finds.
Most of those polled - 68 percent - say the WikiLeaks' exposure of government documents about the State Department and U.S. diplomacy harms the public interest. Nearly as many - 59 percent - say the U.S. government should arrest Assange and charge him with a crime for releasing the diplomatic cables."
http://www.washingtonpost.com/wp-dyn/content/article/2010/12/14/AR2010121401650.html
Vladimir Putin, we are finally ready for your leadership...
Saturday, December 18, 2010
If you ever doubted that GOP specifically targets morons for its electoral base...
...there should be no doubts now: the first target of the new Republican majority is... National Science Foundation.
http://scottaaronson.com/blog/?p=476
http://scottaaronson.com/blog/?p=476
Wednesday, December 8, 2010
Obama's record
http://www.reddit.com/r/politics/comments/ei2ka/olbermann_still_has_it_calls_obama_sellout/c18aby5
Obama was a sellout when he backed off on closing Guantanamo.
Obama was a sellout when he backed off of his promise to keep lobbyists out of his administration.
Obama was a sellout when he protected the Bush administration from prosecution for torture.
Obama was a sellout when he authorized the assassination of U.S. citizens abroad.
Obama was a sellout when he rescinded on his promise to not prosecute marijuana users in states where it is legal, and pushed for a 5 year prison term for a California-legal medical marijuana dispensary operator.
Obama was a sellout when he prosecuted child-soldier Omar Khadr using evidence gained through torture.
Obama was a sellout when he granted 27 waivers to oil companies drilling in the weeks following the Deepwater Horizon disaster.
Obama was a sellout when he fought for, and won, the right to deny habeas corpus to detainees.
He was a sellout when he blocked UN human rights investigations at Guantanamo.
He dropped charges against the CIA for destroying videotapes documenting torture of detainees.
He deported record number of undocumented immigrants.
He continued rendition of alleged terrorists to countries where they could be tortured.
He continued indefinitely detaining alleged terrorists, WITHOUT TRIAL.
He extended the PATRIOT Act, with no reforms.
He dramatically increased government secrecy, denying more Freedom of Information Act requests in 2009 than Bush did in 2008. So much for open government.
He cut a secret deal to kill the public option, while still campaigning on its behalf.
He defended Don't Ask Don't Tell from legal challenges.
He reaffirmed his opposition to same-sex marriage.
He granted waivers to 30 companies, including McDonalds, exempting them from health care reform.
He announced the single largest arms deal in history, of $60bil worth of arms, to Saudi Arabian dictatorship.
He gave permits to BP and other oil companies, exempting them from environmental protection laws.
He appointed Monsanto executive Michael Taylor to the FDA.
He appointed a former Monsanto lobbyist as Chief Agriculture Negotiator.
But NOW, he's a sellout, when he extends Bush's tax cuts? Oh no.
Obama has been a sellout since day one.
Obama was a sellout when he backed off on closing Guantanamo.
Obama was a sellout when he backed off of his promise to keep lobbyists out of his administration.
Obama was a sellout when he protected the Bush administration from prosecution for torture.
Obama was a sellout when he authorized the assassination of U.S. citizens abroad.
Obama was a sellout when he rescinded on his promise to not prosecute marijuana users in states where it is legal, and pushed for a 5 year prison term for a California-legal medical marijuana dispensary operator.
Obama was a sellout when he prosecuted child-soldier Omar Khadr using evidence gained through torture.
Obama was a sellout when he granted 27 waivers to oil companies drilling in the weeks following the Deepwater Horizon disaster.
Obama was a sellout when he fought for, and won, the right to deny habeas corpus to detainees.
He was a sellout when he blocked UN human rights investigations at Guantanamo.
He dropped charges against the CIA for destroying videotapes documenting torture of detainees.
He deported record number of undocumented immigrants.
He continued rendition of alleged terrorists to countries where they could be tortured.
He continued indefinitely detaining alleged terrorists, WITHOUT TRIAL.
He extended the PATRIOT Act, with no reforms.
He dramatically increased government secrecy, denying more Freedom of Information Act requests in 2009 than Bush did in 2008. So much for open government.
He cut a secret deal to kill the public option, while still campaigning on its behalf.
He defended Don't Ask Don't Tell from legal challenges.
He reaffirmed his opposition to same-sex marriage.
He granted waivers to 30 companies, including McDonalds, exempting them from health care reform.
He announced the single largest arms deal in history, of $60bil worth of arms, to Saudi Arabian dictatorship.
He gave permits to BP and other oil companies, exempting them from environmental protection laws.
He appointed Monsanto executive Michael Taylor to the FDA.
He appointed a former Monsanto lobbyist as Chief Agriculture Negotiator.
But NOW, he's a sellout, when he extends Bush's tax cuts? Oh no.
Obama has been a sellout since day one.
Monday, December 6, 2010
Sunday, November 28, 2010
Winows Phone 7
WP7 has hit the Microsoft campus on November 18. This was the date when the free phones the company gave to all the employees have arrived.
I've got one, too, and has been playing with it ever since. So far I cannot imagine going back to my iPhone 3GS.
The UI is slick and extremely responsive. In two weeks of use there was not a single case when the UI lagged. Not one frame of scrolling, zooming, or moving between apps. Not a trace of hesitation. It is also quite intuitive, everything you use often is at your fingertips when you need it.
Search is fully integrated into the experience. It works in the context of the currently running program - in maps it searches maps, in IE it searches the Internet, in Zune UI it searches the music, and in the Marketplace it searches the app store. It works well, too - the results are formatted for the phone, and the way local results are mixed in is uniquely useful for a mobile device.
The IE works great. Really, truly great. It feels faster than the iPhone's Safari browser. With the Samsung Focus' gigantic, bright screen it feels like a mini-tablet. When I bought the iPhone I noticed that I go to my laptop to browse less. With this device I go back to a real computer even less.
One of the reasons that I was awaiting WP7 was that the iPhone was syncing my calendar unreliably. Some of the appointments were transferred, and some were not. I have heard similar complaints from my peers who are also very heavy calendar users. Needless to say, this is the most critical part of the device for me, possibly even more important than the actual phone - I use it as a phone at most twice a day, but I use it as a calendar and email device at least once an hour.
Both calendar and email sync flawlessly on WP7. So does the rest of Office integration. I've spent a few hours reading a book formatted as a Word file, and, again, performance was fantastic and the document rendered flawlessly.
The device is rock solid. I have never - ever - had to reboot it for anything other than the change of the system locale.
Of course, as a v1 device, WP7 has a number of problems as well.
First, it has no way to upload custom ring tones to the device. Yes, this is actually true - the only ringtones that are available are ones preinstalled by the OEM. I have no idea how they managed to cut a feature that at this point is available anywhere else.
Second, the phone only supports a handful of mostly western European locales. For example, it has no support for entering Cyrillic, which, with me being in Russia right now is a big problem. Actually, this plus the absense of copy-paste makes entering an internet search query in any language other than English impossible. Given iPhone's completely universal keyboard that allows writing in any language with ease, this is a big disadvantage for anyone who speaks more than one language.
Third, the phone does not support - this is absolutely, amazingly stupid - user names that have spaces. So if your company has a "First Last" template for the user names, you are out of luck - Exchange sync will not work. Whoever tested the user input dialog - please hang your head in shame now.
Windows Phone 7 lacks the iPhone's breadth of apps, but this is actually an opportunity for developers, and the gap will probably be closed quickly.
Other than that - the device is actually great. I gave away my 3GS after playing with WP7 for less than a day, and 2 weeks in I did not regret that decision for a second. Very highly recommended!
I've got one, too, and has been playing with it ever since. So far I cannot imagine going back to my iPhone 3GS.
The UI is slick and extremely responsive. In two weeks of use there was not a single case when the UI lagged. Not one frame of scrolling, zooming, or moving between apps. Not a trace of hesitation. It is also quite intuitive, everything you use often is at your fingertips when you need it.
Search is fully integrated into the experience. It works in the context of the currently running program - in maps it searches maps, in IE it searches the Internet, in Zune UI it searches the music, and in the Marketplace it searches the app store. It works well, too - the results are formatted for the phone, and the way local results are mixed in is uniquely useful for a mobile device.
The IE works great. Really, truly great. It feels faster than the iPhone's Safari browser. With the Samsung Focus' gigantic, bright screen it feels like a mini-tablet. When I bought the iPhone I noticed that I go to my laptop to browse less. With this device I go back to a real computer even less.
One of the reasons that I was awaiting WP7 was that the iPhone was syncing my calendar unreliably. Some of the appointments were transferred, and some were not. I have heard similar complaints from my peers who are also very heavy calendar users. Needless to say, this is the most critical part of the device for me, possibly even more important than the actual phone - I use it as a phone at most twice a day, but I use it as a calendar and email device at least once an hour.
Both calendar and email sync flawlessly on WP7. So does the rest of Office integration. I've spent a few hours reading a book formatted as a Word file, and, again, performance was fantastic and the document rendered flawlessly.
The device is rock solid. I have never - ever - had to reboot it for anything other than the change of the system locale.
Of course, as a v1 device, WP7 has a number of problems as well.
First, it has no way to upload custom ring tones to the device. Yes, this is actually true - the only ringtones that are available are ones preinstalled by the OEM. I have no idea how they managed to cut a feature that at this point is available anywhere else.
Second, the phone only supports a handful of mostly western European locales. For example, it has no support for entering Cyrillic, which, with me being in Russia right now is a big problem. Actually, this plus the absense of copy-paste makes entering an internet search query in any language other than English impossible. Given iPhone's completely universal keyboard that allows writing in any language with ease, this is a big disadvantage for anyone who speaks more than one language.
Third, the phone does not support - this is absolutely, amazingly stupid - user names that have spaces. So if your company has a "First Last" template for the user names, you are out of luck - Exchange sync will not work. Whoever tested the user input dialog - please hang your head in shame now.
Windows Phone 7 lacks the iPhone's breadth of apps, but this is actually an opportunity for developers, and the gap will probably be closed quickly.
Other than that - the device is actually great. I gave away my 3GS after playing with WP7 for less than a day, and 2 weeks in I did not regret that decision for a second. Very highly recommended!
Thursday, November 25, 2010
Windows Phone 7 unlocking - NOT!
I am travelling to Russia, and tried to use the services of this company - windowsphoneunlock.com - to unlock my phone. They claim on the web site that it takes "up to 72 hours" to unlock the phone.
After waiting for 3 days, no response, I sent them an email, asking about the status. They responded that "some Windows Phone 7 devices take longer", and they will be clarifying this on their web page. However, they are confident that they will have the code in 24 hours.
Two more days go through, no code in the mail. I send them another query. This time, the response:
"Dear Sergey
Please accept our apologies with the length of time it is taking to acquire your unlock code. This is because it is with AT&T. It seems that they will not be giving out the unlock codes for several months after the phone has been registered, or until June 2011. We have just had your order returned to us as not found, not surprising as we are finding out no-one can get codes for Windows Phone 7 handsets on AT&T. However along with yours and other returns of not found came back one successful code return, which is peculiar. I don't know if this is a one-off or it means with some other techniques it might be possible.
I appreciate it has been a long time to wait, so at this point can you please advise if you want a full refund, or you want us to keep trying other techniques.
Many thanks for your patience,
windowsphoneunlock.com"
So, basically,
(1) They claimed that they could unlock Windows 7 phones, but they did not actually check it.
(2) The way they unlock phones is by simply asking for the code from AT&T on your behalf. Or, you can do it yourself and save the money. Of course, in either case AT&T does not supply the unlock code unless you own the phone for 90 days, as any customer service rep will tell you, which is why they couldn't "unlock" mine.
So stay away from these "services" and save yourself some time and money.
After waiting for 3 days, no response, I sent them an email, asking about the status. They responded that "some Windows Phone 7 devices take longer", and they will be clarifying this on their web page. However, they are confident that they will have the code in 24 hours.
Two more days go through, no code in the mail. I send them another query. This time, the response:
"Dear Sergey
Please accept our apologies with the length of time it is taking to acquire your unlock code. This is because it is with AT&T. It seems that they will not be giving out the unlock codes for several months after the phone has been registered, or until June 2011. We have just had your order returned to us as not found, not surprising as we are finding out no-one can get codes for Windows Phone 7 handsets on AT&T. However along with yours and other returns of not found came back one successful code return, which is peculiar. I don't know if this is a one-off or it means with some other techniques it might be possible.
I appreciate it has been a long time to wait, so at this point can you please advise if you want a full refund, or you want us to keep trying other techniques.
Many thanks for your patience,
windowsphoneunlock.com"
So, basically,
(1) They claimed that they could unlock Windows 7 phones, but they did not actually check it.
(2) The way they unlock phones is by simply asking for the code from AT&T on your behalf. Or, you can do it yourself and save the money. Of course, in either case AT&T does not supply the unlock code unless you own the phone for 90 days, as any customer service rep will tell you, which is why they couldn't "unlock" mine.
So stay away from these "services" and save yourself some time and money.
Wednesday, November 17, 2010
Tuesday, November 16, 2010
Trickle-up economics
With all the talk about tax cuts as a universal cure (for which of course there is exactly zero statistical evidence) that are prevalent in US media I wonder why no one is talking about much more obvious concept - a trickle-up economics, where the well-being of the population is certain to benefit the people at the top of the economic ladder.
After all, to earn more money companies need more people who can afford their products. The debt-financed prosperity of 1990s is perverse, but recent, evidence for this. By supporting policies that lead to growing inequality, the top 1% of earners are actually digging a hole for themselves. With the debt as a source of income nearly exhausted, the population will soon not be able to afford much, and the inevitable result will be collapsing incomes at the top as well...
After all, to earn more money companies need more people who can afford their products. The debt-financed prosperity of 1990s is perverse, but recent, evidence for this. By supporting policies that lead to growing inequality, the top 1% of earners are actually digging a hole for themselves. With the debt as a source of income nearly exhausted, the population will soon not be able to afford much, and the inevitable result will be collapsing incomes at the top as well...
Devices and PCs
The device wave seems to be sweeping the world. Apple's new tablet is selling like hot cakes, there are over a dozen of eBook readers on the market, and Android announces the support of a new device category every week.
I have been building the OS for devices for what is still the majority of my career at Microsoft, and had been a passionate advocate for and an avid user of various embedded electronics since 1997. I have shipped the first Microsoft's embedded product that sold for less than $100 (MN-700 residential gateway).
I have used video players, networked storage, terminal server clients, eBook readers, tablets, Internet terminals, portable music players, PDAs, car media players, portable displays, residential gateways, cameras, game boxes, etc, etc, etc. And, of course, smartphones, smartphones, smartphones, and more smartphones.
Over the years I have accumulated 4 large cardboard boxes full of devices in my basement. Almost all of them followed a very similar usage pattern: a new exciting device would come to market, I would get it and use it for about a year. Within a year though, rarely two, almost never three, the device would be obsolete. A new media format will become popular. A new display technology would call for higher video resolution. An upgrade to an OS would obsolete the device drivers. A company would go out of business and shut the web service down.
In the end, the destiny of all these gizmos was always the same - the brown box under the stairs. I think the longest I had embedded device was a couple of $15 wireless routers I got at Fry's 3-4 years ago, and I had a Sony eBook reader for 3 years. All the rest hit the box in at most 2 years.
I actually loved my SONY Reader. Despite the fact it took 10 minutes to boot. Despite the fact that there was no way to organize the collection in any meaningful way, so I always had to go through 66 pages of book titles to find the one I intended to read. Despite the fact that the battery life was nothing like advertized.
I even wrote a filter to automatically translate Project Gutenberg books to LRF format (http://1-800-magic.blogspot.com/2008/01/gutenberg-for-sony-pre-alpha.html).
What eventually killed the Reader was a multitude of factors. The hacked support for Russian script was no longer available for the most current firmware upgrade, so I had to stay on the old version. The new connection software no longer supports the old firmware. Finally, BookDesigner, a program that I used to translate RTF files into LRF to read them on the device does not work on newer operating systems.
At the same time, while playing with all these devices, I was slowly but surely - and mostly without realizing it - replacing them with PCs. Where devices lasted only a bit, PCs persisted.
A NAS device was replaced by three of server PCs first consisting of expensive case, good power supplies, cheap motherboards, and 30 disks. The cheap motherboards were quickly replaced by an expensive ones, but the rest was reused, and the machines are now in service 24/7 through the last 5 years. I have replaced a few disks, taking a couple of opportunities to upgrade - first going from 300 to 500GB, then to 1TB drives.
A couple of media center PCs that were in the house since 2005 went through a refresh this year. I have replaced motherboards, CPU, and memory ($400 per PC for 6-core Athlons, 8GB RAM, and top of the line ASUS boards), and upgraded the OS to Windows 7. I have reused the 500GB disks that were left over from the server upgrade.
A gaming PC that I have in my garage in front of a thread mill is in daily use since 2007, and it plays Halo 2 as well as it was on the first day of its life four and a half years ago. I have turned on the Xbox 360 next to it less than a dozen times over the same period of time. Last time only to find out that it had died, perhaps of loneliness.
Because of the super high level of commoditization, PC hardware is unmatched in price. Even the retail markup is different - for devices such as DVD players it usually is over 50%, whereas for PC hardware - thank you, Dell! - it is almost always below 15%. So you can buy a 4TB TerraStation for $800, or a pretty decent PC with 4 1TB hard drives which would in many cases cost much less.
And because PC hardware is interchangeable, when you decomission an old PC, you can almost always reuse at least some of the parts - a case, a power supply, a DVD drive - at a minimum, which makes your new build so much cheaper.
But of course the biggest advantage of the PC platform is that it is constantly evolving. A new codec that could kill a media player will be just a tiny upgrade to a Media Center PC. A Blu-ray is trivial to install without changing much of the existing functionality.
PCs do come with a cost - they could be hard to maintain. But there is a very simple solution - my PCs are all single function. A game PC is only used to play games. A Media Center is only used to watch movies. Servers are only used for storage and virtualization. There is a dedicated PC for Skype. I do software development and email on a VM that is specially built just for that.
As a result, there are usually no compatibility problems with software, and in fact configuration changes are very infrequent, so I spend very little time on my home infrastructure these days. Much less, in fact, than I used to spend trying to get all the devices to talk to each other :-).
I have been building the OS for devices for what is still the majority of my career at Microsoft, and had been a passionate advocate for and an avid user of various embedded electronics since 1997. I have shipped the first Microsoft's embedded product that sold for less than $100 (MN-700 residential gateway).
I have used video players, networked storage, terminal server clients, eBook readers, tablets, Internet terminals, portable music players, PDAs, car media players, portable displays, residential gateways, cameras, game boxes, etc, etc, etc. And, of course, smartphones, smartphones, smartphones, and more smartphones.
Over the years I have accumulated 4 large cardboard boxes full of devices in my basement. Almost all of them followed a very similar usage pattern: a new exciting device would come to market, I would get it and use it for about a year. Within a year though, rarely two, almost never three, the device would be obsolete. A new media format will become popular. A new display technology would call for higher video resolution. An upgrade to an OS would obsolete the device drivers. A company would go out of business and shut the web service down.
In the end, the destiny of all these gizmos was always the same - the brown box under the stairs. I think the longest I had embedded device was a couple of $15 wireless routers I got at Fry's 3-4 years ago, and I had a Sony eBook reader for 3 years. All the rest hit the box in at most 2 years.
I actually loved my SONY Reader. Despite the fact it took 10 minutes to boot. Despite the fact that there was no way to organize the collection in any meaningful way, so I always had to go through 66 pages of book titles to find the one I intended to read. Despite the fact that the battery life was nothing like advertized.
I even wrote a filter to automatically translate Project Gutenberg books to LRF format (http://1-800-magic.blogspot.com/2008/01/gutenberg-for-sony-pre-alpha.html).
What eventually killed the Reader was a multitude of factors. The hacked support for Russian script was no longer available for the most current firmware upgrade, so I had to stay on the old version. The new connection software no longer supports the old firmware. Finally, BookDesigner, a program that I used to translate RTF files into LRF to read them on the device does not work on newer operating systems.
At the same time, while playing with all these devices, I was slowly but surely - and mostly without realizing it - replacing them with PCs. Where devices lasted only a bit, PCs persisted.
A NAS device was replaced by three of server PCs first consisting of expensive case, good power supplies, cheap motherboards, and 30 disks. The cheap motherboards were quickly replaced by an expensive ones, but the rest was reused, and the machines are now in service 24/7 through the last 5 years. I have replaced a few disks, taking a couple of opportunities to upgrade - first going from 300 to 500GB, then to 1TB drives.
A couple of media center PCs that were in the house since 2005 went through a refresh this year. I have replaced motherboards, CPU, and memory ($400 per PC for 6-core Athlons, 8GB RAM, and top of the line ASUS boards), and upgraded the OS to Windows 7. I have reused the 500GB disks that were left over from the server upgrade.
A gaming PC that I have in my garage in front of a thread mill is in daily use since 2007, and it plays Halo 2 as well as it was on the first day of its life four and a half years ago. I have turned on the Xbox 360 next to it less than a dozen times over the same period of time. Last time only to find out that it had died, perhaps of loneliness.
Because of the super high level of commoditization, PC hardware is unmatched in price. Even the retail markup is different - for devices such as DVD players it usually is over 50%, whereas for PC hardware - thank you, Dell! - it is almost always below 15%. So you can buy a 4TB TerraStation for $800, or a pretty decent PC with 4 1TB hard drives which would in many cases cost much less.
And because PC hardware is interchangeable, when you decomission an old PC, you can almost always reuse at least some of the parts - a case, a power supply, a DVD drive - at a minimum, which makes your new build so much cheaper.
But of course the biggest advantage of the PC platform is that it is constantly evolving. A new codec that could kill a media player will be just a tiny upgrade to a Media Center PC. A Blu-ray is trivial to install without changing much of the existing functionality.
PCs do come with a cost - they could be hard to maintain. But there is a very simple solution - my PCs are all single function. A game PC is only used to play games. A Media Center is only used to watch movies. Servers are only used for storage and virtualization. There is a dedicated PC for Skype. I do software development and email on a VM that is specially built just for that.
As a result, there are usually no compatibility problems with software, and in fact configuration changes are very infrequent, so I spend very little time on my home infrastructure these days. Much less, in fact, than I used to spend trying to get all the devices to talk to each other :-).
Wednesday, November 10, 2010
We are screwed...
Here's the new (Republican) chairman of Energy and Commerce Committee. He says that Bible promises to not flood the Earth after Noah, therefore there will be no global warming. True story. This is the level of idiocy in Congress. I think I would rather live under Brezhnev again... Soviet government may have been evil, but it was not so monumentally stupid.
Tuesday, November 2, 2010
This is why Democrats need to abolish farm subsidies...
Among other things that they should have done, but didn't (http://1-800-magic.blogspot.com/2008/11/next-depression.html) because they are too corrupt and incompetent...
Monday, November 1, 2010
Couple of interesting quotes
"Egotism is the anesthetic that dulls the pain of stupidity."
-- Frank Leahy
"Competition whose motive is merely to compete, to drive some other fellow out, never carries very far. The competitor to be feared is one who never bothers about you at all, but goes on making his own business better all the time. Businesses that grow by development and improvement do not die. But when a business ceases to be creative, when it believes it has reached perfection and needs to do nothing but produce - no improvement, no development - it is done."
-- Henry Ford
-- Frank Leahy
"Competition whose motive is merely to compete, to drive some other fellow out, never carries very far. The competitor to be feared is one who never bothers about you at all, but goes on making his own business better all the time. Businesses that grow by development and improvement do not die. But when a business ceases to be creative, when it believes it has reached perfection and needs to do nothing but produce - no improvement, no development - it is done."
-- Henry Ford
Thursday, October 28, 2010
Windows disk management with .NET framework
Windows has a very powerful programming API that allows developers almost total control over the way storage is managed. There are 3 main API subsets:
Unfortunately, .NET framework does not ship with an official wrapper, so there are two ways to access VDS from managed code. VDS is exposed through COM, but has no automation support, so one could use .NET COM interop, but would have to declare the interface by hand. There is a good example of how to do it here: http://stackoverflow.com/questions/2755458/retrieving-virtual-disk-file-name-from-disk-number/2892042.
However, VDS is a relatively big API, and so doing the marshalling by hand is a challenging job. Of course, if you do this, the humanity will be eternally grateful (especially if you post the results to http://pinvoke.net/)!
For those of us who are lazy, the next best thing is to use the DLL that is present on Windows Server 2008 and above - Microsoft.Storage.Vds.dll. It lives in both GAC and SxS. In both cases the paths to the files are absolutely awful, and I will dare not utter them here. Instead, go to you Windows directory and do "dir /s Microsoft.Storage.Vds.dll". The resulting DLL you can then copy to your application and add a reference to it in your project.
Important note: the DLL is ONLY present in Server OS - not in consumer Windows - but the interop logic that it exposes will work on Windows 7 as well. You just have to carry the file around with your app (note of course that Microsoft does not grant you any rights to redistribute parts of Windows!) .
Another important note: to use VDS on a local machine, your application must have an admin token in its process. When debugging in Visual Studio, it helps to start VS "as Administrator". To ensure that the right privileges exist when running the application, add the manifest file as follows:
This means that a new disk cannot be made dynamic using this interface, and therefore it cannot be used to build new disk arrays. If you need this functionality, you arescrewed have to do COM interop by hand. You can still use this library with the reflector to extract much of the interop code. However, all the code needed to check the disk system health, and to replace a disk is present, which means that the library is very useful for many interesting scenarios. (Again, Microsoft does not give you a right to reverse-engineer parts of Windows to incorporate them into your application!)
With this in mind, on to managed VDS!
To use VDS, you have to first connect to VDS service. You can connect to a service both on your local machine as well as on a remote computer (providing that your user has admin rights on it:
The basic provider contains all the "normal", non-dynamic disks. To get dynamic and basic providers, do the following:
Here is how to print information on all the disks in a system:
Among other things, Disk.Heath is a property that allows you to ferret out disks that are missing or unhealthy. From the reflector:
Time for a complete example. The following program scans a disk system on a running computer, finds disks that went missing/unhealthy, and sends an email to a pre-defined account.
Program.cs:
- VDS (Virtual Disk Service) allows to programmatically manage disks, volumes, and access features of Windows software RAID implementation. (http://msdn.microsoft.com/en-us/library/bb986750(v=VS.85).aspx)
- VSS (Volume Shadow Copy Service) lets one create snapshots of the current disk state. This can be used for a backup (these snapshots are internally consistent, because VSS exposes - and many software products consume - a subscription interface that allows application to write their state to disk before the snapshot is taken), as well as to preserve the current state of the volume state in time, so it (or individual files) could be rolled back at a later date. (http://msdn.microsoft.com/en-us/library/bb968832(VS.85).aspx)
- Block device access through device IOCTL and direct I/O (http://1-800-magic.blogspot.com/2008/02/taking-disk-image-snapshots.html)
Unfortunately, .NET framework does not ship with an official wrapper, so there are two ways to access VDS from managed code. VDS is exposed through COM, but has no automation support, so one could use .NET COM interop, but would have to declare the interface by hand. There is a good example of how to do it here: http://stackoverflow.com/questions/2755458/retrieving-virtual-disk-file-name-from-disk-number/2892042.
However, VDS is a relatively big API, and so doing the marshalling by hand is a challenging job. Of course, if you do this, the humanity will be eternally grateful (especially if you post the results to http://pinvoke.net/)!
For those of us who are lazy, the next best thing is to use the DLL that is present on Windows Server 2008 and above - Microsoft.Storage.Vds.dll. It lives in both GAC and SxS. In both cases the paths to the files are absolutely awful, and I will dare not utter them here. Instead, go to you Windows directory and do "dir /s Microsoft.Storage.Vds.dll". The resulting DLL you can then copy to your application and add a reference to it in your project.
Important note: the DLL is ONLY present in Server OS - not in consumer Windows - but the interop logic that it exposes will work on Windows 7 as well. You just have to carry the file around with your app (note of course that Microsoft does not grant you any rights to redistribute parts of Windows!) .
Another important note: to use VDS on a local machine, your application must have an admin token in its process. When debugging in Visual Studio, it helps to start VS "as Administrator". To ensure that the right privileges exist when running the application, add the manifest file as follows:
<?xml version="1.0" encoding="utf-8"?>The most important note: The managed API lacks one member of IVdsPack interface - the function that is absolutely crucial to creating RAID arrays (IVdsPack::MigrateDisks). As you can see in reflector, the IVdsPack is not completely marshalled ("slot7" occupies the place where MigrateDisks should have been):
<asmv1:assembly manifestVersion="1.0" xmlns="urn:schemas-microsoft-com:asm.v1" xmlns:asmv1="urn:schemas-microsoft-com:asm.v1" xmlns:asmv2="urn:schemas-microsoft-com:asm.v2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<assemblyIdentity version="1.0.0.0" name="MyApplication.app"/>
<trustInfo xmlns="urn:schemas-microsoft-com:asm.v2">
<security>
<requestedPrivileges xmlns="urn:schemas-microsoft-com:asm.v3">
<requestedExecutionLevel level="requireAdministrator" uiAccess="false" />
</requestedPrivileges>
</security>
</trustInfo>
</asmv1:assembly>
This means that a new disk cannot be made dynamic using this interface, and therefore it cannot be used to build new disk arrays. If you need this functionality, you are
With this in mind, on to managed VDS!
To use VDS, you have to first connect to VDS service. You can connect to a service both on your local machine as well as on a remote computer (providing that your user has admin rights on it:
using Microsoft.Storage.Vds;The object hierarchy in VDS is as follows. The VDS object contains a list of providers, and a list of unallocated disks. Unallocated disks are the disks that have not been initialized. You can access them as follows:
...
ServiceLoader loader = new ServiceLoader();
Service vds = loader.LoadService("computername"); // or null
vds.WaitForServiceReady();
foreach (Disk d in vds.UnallocatedDisks)On most computers (excluding the very rare cases where hardware manufacturers exposed their custom storage systems through VDS, which can happen with SANs), there are two providers, a dynamic provider and basic provider. A dynamic provider contains dynamic disks - the disks that can contain RAID volumes (simple volumes, spanned volumes, RAID-0, RAID-1, and RAID-5). You make the disk dynamic in disk management applet:
{
Console.WriteLine("Found unused disk {0} {1} {2} {3} {4}",
d.FriendlyName, d.Name, d.DevicePath, d.Size, d.Health);
}
The basic provider contains all the "normal", non-dynamic disks. To get dynamic and basic providers, do the following:
private static SoftwareProvider GetDynamicProvider(Service vds)Provider contains a list of packs. There is usually only one pack in the dynamic provider, unless a set of dynamic disks initialized on a different computer was added, but not migrated (they appear as "foreign" disks in disk management applet). Basic disks are one per pack, so if there several basic disks in the system, there would be multiple packs in basic provider.
{
foreach (Provider p in vds.Providers)
{
if ((p.Flags & ProviderFlags.Dynamic) != 0)
return p as SoftwareProvider;
}
return null;
}
private static SoftwareProvider GetBasicProvider(Service vds)
{
foreach (Provider p in vds.Providers)
{
if ((p.Flags & (ProviderFlags.Dynamic |
ProviderFlags.InternalHardwareProvider)) == 0)
return p as SoftwareProvider;
}
return null;
}
Here is how to print information on all the disks in a system:
SoftwareProvider dynamicProvider = GetDynamicProvider(vds);
foreach (Pack p in dynamicProvider.Packs)
{
foreach (Disk d in p.Disks)
{
Console.WriteLine("Found dynamic disk {0} {1} {2} {3} {4}",
d.FriendlyName, d.Name, d.DevicePath, d.Size, d.Health);
}
}
SoftwareProvider basicProvider = GetBasicProvider(vds);
foreach (Pack p in basicProvider.Packs)
{
foreach (Disk d in p.Disks)
{
Console.WriteLine("Found basic disk {0} {1} {2} {3} {4}",
d.FriendlyName, d.Name, d.DevicePath, d.Size, d.Health);
}
}
foreach (Disk d in vds.UnallocatedDisks)
{
Console.WriteLine("Found unused disk {0} {1} {2} {3} {4}",
d.FriendlyName, d.Name, d.DevicePath, d.Size, d.Health);
}
Among other things, Disk.Heath is a property that allows you to ferret out disks that are missing or unhealthy. From the reflector:
public enum HealthThe pack also contains volumes, e.g.:
{
Unknown = 0,
Healthy = 1,
Rebuilding = 2,
Stale = 3,
Failing = 4,
FailingRedundancy = 5,
FailedRedundancy = 6,
FailedRedundancyFailing = 7,
Failed = 8,
}
foreach (Volume v in p.Volumes)And, of course, given a disk, it is easy to figure out which volumes are on this disk (in case of RAID arrays, of course, the volume might not be on just one disk of course). A Disk structure has a list of extents in it. An extent has a volume id, which can be translated into a Volume object:
{
Console.WriteLine("Found volume {0}: {1} {2} {3} {4}",
v.DriveLetter, v.Name, v.Type, v.Size, v.Health);
}
foreach (DiskExtent de in d.Extents)The reverse - finding out what disks a volume occupies - is only slightly more involved:
{
Volume v = vds.GetObject<Volume>(de.DiskId);
}
foreach (VolumePlex vp in v.Plexes)Another useful piece of functionality in managed VDS interface is ability to replace a dead disk easily by using Pack.BeginReplaceDisk.
{
foreach (DiskExtent de in vp.Extents)
{
Disk d = vds.GetObject<Disk>(de.DiskId);
}
}
Time for a complete example. The following program scans a disk system on a running computer, finds disks that went missing/unhealthy, and sends an email to a pre-defined account.
Program.cs:
//-----------------------------------------------------------------------app.config:
// <copyright>
// Copyright (C) Sergey Solyanik.
//
// This file is subject to the terms and conditions of the Microsoft Public License (MS-PL).
// See http://www.microsoft.com/opensource/licenses.mspx#Ms-PL for more details.
// </copyright>
//-----------------------------------------------------------------------
using System;
using System.Collections.Generic;
using System.Configuration;
using System.Linq;
using System.Net.Mail;
using System.Text;
using Microsoft.Storage.Vds;
namespace DiskMonitor
{
/// <summary>
/// Implements a simple disk monitor. The program is a periodic task
/// that looks for an unhealthy disk and sends an email if it finds one.
/// </summary>
class Program
{
/// <summary>
/// Main program.
/// </summary>
/// <param name="args"> Arguments from the command line. None expected. </param>
static void Main(string[] args)
{
ServiceLoader loader = new ServiceLoader();
Service vds = loader.LoadService(null);
vds.WaitForServiceReady();
StringBuilder sb = new StringBuilder();
SoftwareProvider dynamicProvider = GetDynamicProvider(vds);
foreach (Pack p in dynamicProvider.Packs)
{
foreach (Disk d in p.Disks)
{
if (d.Health != Health.Healthy && d.Health != Health.Rebuilding)
{
sb.Append(string.Format("Disk {0} has health status {1}\n",
d.Name, d.Health));
}
}
}
SoftwareProvider basicProvider = GetBasicProvider(vds);
foreach (Pack p in basicProvider.Packs)
{
foreach (Disk d in p.Disks)
{
if (d.Health != Health.Healthy && d.Health != Health.Rebuilding)
{
sb.Append(string.Format("Disk {0} has health status {1}\n",
d.Name, d.Health));
}
}
}
if (sb.Length != 0)
{
Console.Error.Write(sb);
}
else
{
Console.WriteLine("Everything is great!");
}
MailMessage m = new MailMessage();
m.To.Add(ConfigurationManager.AppSettings["NotificationEmail"]);
m.From = new MailAddress(
ConfigurationManager.AppSettings["FromEmail"]);
m.Sender = new MailAddress(
ConfigurationManager.AppSettings["FromEmail"]);
m.IsBodyHtml = false;
if (sb.Length != 0)
{
m.Subject = string.Format(
"Problems with disk array on {0}", Environment.MachineName);
m.Body = sb.ToString();
}
else if (bool.Parse(ConfigurationManager.AppSettings["VerboseMail"]))
{
m.Subject = string.Format(
"Checked out disk system on {0}: everything is healthy!",
Environment.MachineName);
m.Body = "Nothing to report.";
}
SmtpClient client = new SmtpClient(
ConfigurationManager.AppSettings["SmtpServer"]);
client.UseDefaultCredentials = true;
client.EnableSsl = bool.Parse(
ConfigurationManager.AppSettings["UseSslForSmtp"]);
client.Send(m);
}
/// <summary>
/// Finds the dynamic software provider.
/// </summary>
/// <param name="vds"> VDS service. </param>
/// <returns> Basic software provider or null of none exists. </returns>
private static SoftwareProvider GetDynamicProvider(Service vds)
{
foreach (Provider p in vds.Providers)
{
if ((p.Flags & ProviderFlags.Dynamic) != 0)
return p as SoftwareProvider;
}
return null;
}
/// <summary>
/// Finds the basic software provider.
/// </summary>
/// <param name="vds"> VDS service. </param>
/// <returns> Basic software provider or null of none exists. </returns>
private static SoftwareProvider GetBasicProvider(Service vds)
{
foreach (Provider p in vds.Providers)
{
if ((p.Flags & (ProviderFlags.Dynamic |
ProviderFlags.InternalHardwareProvider)) == 0)
return p as SoftwareProvider;
}
return null;
}
}
}
<?xml version="1.0" encoding="utf-8" ?>app.manifest:
<configuration>
<appSettings>
<add key="SmtpServer" value="smtp.server.domain.com" />
<add key="NotificationEmail" value="target@domain.com" />
<add key="FromEmail" value="user@domain.com" />
<add key="UseSslForSmtp" value="true" />
<add key="VerboseMail" value="true" />
</appSettings>
</configuration>
<?xml version="1.0" encoding="utf-8"?>
<asmv1:assembly manifestVersion="1.0" xmlns="urn:schemas-microsoft-com:asm.v1" xmlns:asmv1="urn:schemas-microsoft-com:asm.v1" xmlns:asmv2="urn:schemas-microsoft-com:asm.v2" xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance>
<assemblyIdentity version="1.0.0.0" name="DiskMonitor"/>
<trustInfo xmlns="urn:schemas-microsoft-com:asm.v2">
<security>
<requestedPrivileges xmlns="urn:schemas-microsoft-com:asm.v3">
<requestedExecutionLevel level="requireAdministrator" uiAccess="false" />
</requestedPrivileges>
</security>
</trustInfo>
</asmv1:assembly>
Tuesday, October 19, 2010
How much is mobile entertainment market worth?
With AAPL approaching $300B, it is worth more than Toyota, Daimler, and Honda combined.
In fact, here is the list of top automobile manufacturers (which apparently includes ZAP, the electric scooter company):
Together, these companies are worth a combined total of $453.7B, only 50% more than AAPL. Wow. Just wow...
In fact, here is the list of top automobile manufacturers (which apparently includes ZAP, the electric scooter company):
| Market Cap | |
| Toyota Motor Corporation (ADR) | 112.13 Bil |
| Daimler AG (USA) | 70.73 Bil |
| HONDA MOTOR CO., LTD. (ADR) | 66.31 Bil |
| Volkswagen AG (ADR) | 53.73 Bil |
| Ford Motor Company | 47.46 Bil |
| Nissan Motor Co., Ltd. (ADR) | 38.77 Bil |
| AB Volvo (ADR) | 28.26 Bil |
| Fiat S.p.A. (ADR) | 20.35 Bil |
| Tata Motors Limited (ADR) | 12.22 Bil |
| Tesla Motors Inc | 1.91 Bil |
| Sorl Auto Parts, Inc. | 172.59 Mil |
| Azure Dynamics Corp | 161.25 Mil |
| ZAP | 56.02 Mil |
| Zenn Motor Co Inc | 55.25 Mil |
| Tongxin International Ltd | 37.68 Mil |
Together, these companies are worth a combined total of $453.7B, only 50% more than AAPL. Wow. Just wow...
Thursday, September 30, 2010
David Patterson
David Patterson (of Computer Architecture: A Quantitative Approach) gave a lecture today at Microsoft about his ROC project.
Among other things, I learned two things:
...Peres's law
“If a problem has no solution, it may not be a problem, but a fact, not to be solved, but to be coped with over time”
— Shimon Peres
...and that Bing is now a better search engine than Google.
For the last several months I was using Bing and Google more or less interchangeably, but Google was always an authoritative source. So when a few hours after the lecture I wanted to quote the Peres's law to a co-worker, I went to Google first. I typed in the following query: "unsolvable problems Shimon Peres facts cope". To my surprise, the quote was nowhere to be found:
I was really surprised, and tried Bing. Both of the top two results pointed to the quote.
Among other things, I learned two things:
...Peres's law
“If a problem has no solution, it may not be a problem, but a fact, not to be solved, but to be coped with over time”
— Shimon Peres
...and that Bing is now a better search engine than Google.
For the last several months I was using Bing and Google more or less interchangeably, but Google was always an authoritative source. So when a few hours after the lecture I wanted to quote the Peres's law to a co-worker, I went to Google first. I typed in the following query: "unsolvable problems Shimon Peres facts cope". To my surprise, the quote was nowhere to be found:
I was really surprised, and tried Bing. Both of the top two results pointed to the quote.
Thursday, September 23, 2010
Taliban, v2
http://www.independent.co.uk/news/world/asia/how-the-cia-ran-a-secret-army-of-3000-assassins-2087039.html
Apparently CIA is financing an army in Afghanistan in exactly the same way they finances the anti-Soviet resistance which resulted in Osama Bin Ladin's movement. Smart people learn on other's mistakes. Dumb people learn on their own. And then there's a whole new level...
Apparently CIA is financing an army in Afghanistan in exactly the same way they finances the anti-Soviet resistance which resulted in Osama Bin Ladin's movement. Smart people learn on other's mistakes. Dumb people learn on their own. And then there's a whole new level...
Saturday, September 11, 2010
Republican Lobbyist Edwina Rogers Wraps Gifts in Money
http://www.youtube.com/watch?v=xXj-oQm-NbE
I have no further comments... But there was one of note on Reddit: "The poor can dig the wrapper out of the trash later. This is called trickle down economics."
Court dismisses torture lawsuit because it "might expose state secrets"
"While the alleged abuses occurred during the Bush administration, the ruling added a chapter to the Obama administration’s aggressive national security policies.
Its counterterrorism programs have in some ways departed from the expectations of change fostered by President Obama’s campaign rhetoric, which was often sharply critical of former President George W. Bush’s approach.
Among other policies, the Obama national security team has also authorized the C.I.A. to try to kill a United States citizen suspected of terrorism ties, blocked efforts by detainees in Afghanistan to bring habeas corpus lawsuits challenging the basis for their imprisonment without trial, and continued the C.I.A.’s so-called extraordinary rendition program of prisoner transfers — though the administration has forbidden torture and says it seeks assurances from other countries that detainees will not be mistreated."
http://www.nytimes.com/2010/09/09/us/09secrets.html?_r=4&hp=&adxnnl=1&pagewanted=all&adxnnlx=1284220860-JSOfEHxr+sgBSLlDU9nJSA
It looks like I am voting for a third party candidate in the next election season...
Its counterterrorism programs have in some ways departed from the expectations of change fostered by President Obama’s campaign rhetoric, which was often sharply critical of former President George W. Bush’s approach.
Among other policies, the Obama national security team has also authorized the C.I.A. to try to kill a United States citizen suspected of terrorism ties, blocked efforts by detainees in Afghanistan to bring habeas corpus lawsuits challenging the basis for their imprisonment without trial, and continued the C.I.A.’s so-called extraordinary rendition program of prisoner transfers — though the administration has forbidden torture and says it seeks assurances from other countries that detainees will not be mistreated."
http://www.nytimes.com/2010/09/09/us/09secrets.html?_r=4&hp=&adxnnl=1&pagewanted=all&adxnnlx=1284220860-JSOfEHxr+sgBSLlDU9nJSA
It looks like I am voting for a third party candidate in the next election season...
Tuesday, September 7, 2010
Friday, September 3, 2010
Monday, August 30, 2010
Israel: Your tax dollars at work
Looks like another terrorist is down.
http://www.guardian.co.uk/world/2005/nov/16/israel2?CMP=twt_gu
From the watchtower "It's a little girl. She's running defensively eastward."
From the operations room "Are we talking about a girl under the age of 10?"
Watchtower "A girl about 10, she's behind the embankment, scared to death."
Watchtower "I think that one of the positions took her out."
Captain R "I and another soldier ... are going in a little nearer, forward, to confirm the kill ... Receive a situation report. We fired and killed her ... I also confirmed the kill. Over."
http://www.guardian.co.uk/world/2005/nov/16/israel2?CMP=twt_gu
From the watchtower "It's a little girl. She's running defensively eastward."
From the operations room "Are we talking about a girl under the age of 10?"
Watchtower "A girl about 10, she's behind the embankment, scared to death."
Watchtower "I think that one of the positions took her out."
Captain R "I and another soldier ... are going in a little nearer, forward, to confirm the kill ... Receive a situation report. We fired and killed her ... I also confirmed the kill. Over."
How to make a sniper rifle. Table of contents.
Part 1: Introduction
http://www.preciseshooter.com/blog/MosinSniperPart1.aspx
Part 2: Buying a Mosin Nagant
http://www.preciseshooter.com/blog/MosinSniperPart2.aspx
Part 3: Cleaning a Mosin Nagant
http://www.preciseshooter.com/blog/MosinSniperPart3.aspx
Part 4: Disassembling a Mosin Nagant
http://www.preciseshooter.com/blog/MosinSniperPart4.aspx
Part 5: Accurizing a Mosin Nagant
http://www.preciseshooter.com/blog/MosinSniperPart5.aspx
Part 6: Selecting a scope mount
http://www.preciseshooter.com/blog/MosinSniperPart6.aspx
Part 7: Mounting a side rail on the Mosin Nagant
http://www.preciseshooter.com/blog/MosinSniperPart7.aspx
http://www.preciseshooter.com/blog/MosinSniperPart1.aspx
Part 2: Buying a Mosin Nagant
http://www.preciseshooter.com/blog/MosinSniperPart2.aspx
Part 3: Cleaning a Mosin Nagant
http://www.preciseshooter.com/blog/MosinSniperPart3.aspx
Part 4: Disassembling a Mosin Nagant
http://www.preciseshooter.com/blog/MosinSniperPart4.aspx
Part 5: Accurizing a Mosin Nagant
http://www.preciseshooter.com/blog/MosinSniperPart5.aspx
Part 6: Selecting a scope mount
http://www.preciseshooter.com/blog/MosinSniperPart6.aspx
Part 7: Mounting a side rail on the Mosin Nagant
http://www.preciseshooter.com/blog/MosinSniperPart7.aspx
Friday, August 27, 2010
Saturday, August 21, 2010
Tuesday, August 10, 2010
Monday, August 9, 2010
Sunday, August 8, 2010
Thursday, August 5, 2010
Thursday, July 22, 2010
Tuesday, July 20, 2010
World's most inefficient way to check for primeness (via Reddit)
http://www.noulakaz.net/weblog/2007/03/18/a-regular-expression-to-check-for-prime-numbers/
Fire your perl interpreter, feed this function 982451653, and watch your computer die a slow and painful death...
Fire your perl interpreter, feed this function 982451653, and watch your computer die a slow and painful death...
The only democracy in the Middle East (TM) (via Reddit)
A Palestinian man has been convicted of rape after having consensual sex with an Israeli woman who believed he was Jewish because he introduced himself as "Daniel".
http://www.telegraph.co.uk/news/worldnews/middleeast/israel/7901025/Palestinian-jailed-for-rape-after-claiming-to-be-Jewish.html
http://www.telegraph.co.uk/news/worldnews/middleeast/israel/7901025/Palestinian-jailed-for-rape-after-claiming-to-be-Jewish.html
Sunday, July 18, 2010
Inception
Watched the highly acclaimed "Inception" on Friday. Below is my take.
Short version: "Titanic" is better.
And I don't say this because I like "Titanic".
My two favorite stories about "Titanic" are as follows. When the movie was first released on VHS, the MicroNews - internal MS paper that back then was a print edition - published this classified ad: "Titanic on VHS. First tape watched once. Second tape never watched".
Second story: among Windows developers Intel's Itanium chip is known as "Itanic". Although this one has nothing to do with the movie...
Long version: If you consider "Matrix" brainy and captivating, "Inception" is for you!
It is extremely similar to "Matrix". Both movies take a trivial idea ("Matrix": reality is actually a computer generated dream; "Inception": reality is influenced by a human-induced shared dreams) and make a movie out of it by adding large number of superficialities.
In "Matrix" it's tough looking impeccably clothed men beating main heroes up in highly choreographed fight scenes. I guess it must be the Hollywood's idea of what's going on inside a computer. I suppose if I had majored in communications, I would have thought about computers like that, too.
In "Inception" it is impeccably "architected" dreamscapes where the heroes get to confront - and conquer! - their demons. I guess it must be the Hollywood's idea of what's going on inside a human brain. I suppose if I had majored in said communications, I would have thought about the mechanics of human brain like that as well.
Why do I think "Titanic" is better? There, if one abstracts from the plot, one still gets to enjoy the vistas. In "Inception", unfortunately, there is no way to abstract from the plot...
Short version: "Titanic" is better.
And I don't say this because I like "Titanic".
My two favorite stories about "Titanic" are as follows. When the movie was first released on VHS, the MicroNews - internal MS paper that back then was a print edition - published this classified ad: "Titanic on VHS. First tape watched once. Second tape never watched".
Second story: among Windows developers Intel's Itanium chip is known as "Itanic". Although this one has nothing to do with the movie...
Long version: If you consider "Matrix" brainy and captivating, "Inception" is for you!
It is extremely similar to "Matrix". Both movies take a trivial idea ("Matrix": reality is actually a computer generated dream; "Inception": reality is influenced by a human-induced shared dreams) and make a movie out of it by adding large number of superficialities.
In "Matrix" it's tough looking impeccably clothed men beating main heroes up in highly choreographed fight scenes. I guess it must be the Hollywood's idea of what's going on inside a computer. I suppose if I had majored in communications, I would have thought about computers like that, too.
In "Inception" it is impeccably "architected" dreamscapes where the heroes get to confront - and conquer! - their demons. I guess it must be the Hollywood's idea of what's going on inside a human brain. I suppose if I had majored in said communications, I would have thought about the mechanics of human brain like that as well.
Why do I think "Titanic" is better? There, if one abstracts from the plot, one still gets to enjoy the vistas. In "Inception", unfortunately, there is no way to abstract from the plot...
How to make a service in .NET
Here's a complete, self-contained way to build a system service using .NET. I was looking for a way to do it on the internets, but most of the examples rely on .NET template (which relies on designer, which is ugly) and don't have a way to install the service programmatically.
Without much ado, here's the code. All of it. Just replace ServiceMainThread with your code, and you're done. It even supports installing multiple instances of itself.
Without much ado, here's the code. All of it. Just replace ServiceMainThread with your code, and you're done. It even supports installing multiple instances of itself.
//-----------------------------------------------------------------------
// <copyright>
// Copyright (C) Sergey Solyanik.
//
// This file is subject to the terms and conditions of the Microsoft Public License (MS-PL).
// See http://www.microsoft.com/opensource/licenses.mspx#Ms-PL for more details.
// </copyright>
//-----------------------------------------------------------------------
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Runtime.InteropServices;
using System.ServiceProcess;
using System.Text;
using System.Threading;
namespace CSServiceHost
{
/// <summary>
/// Main service class.
/// </summary>
public class MyService : ServiceBase
{
/// <summary>
/// The thread that contains the execution path for the service.
/// </summary>
Thread runner;
/// <summary>
/// Event which gets signalled when the service stops.
/// </summary>
EventWaitHandle stop;
/// <summary>
/// Processes the start event for service.
/// </summary>
/// <param name="args"></param>
protected override void OnStart(string[] args)
{
stop = new EventWaitHandle(false, EventResetMode.ManualReset);
runner = new Thread(ServiceMainThread);
runner.Start();
}
/// <summary>
/// Processes the stop event for service.
/// </summary>
protected override void OnStop()
{
stop.Set();
runner.Join();
}
/// <summary>
/// The main entry point for the application.
/// </summary>
/// <param name="args"> Program arguments. See help.</param>
static void Main(string[] args)
{
if (args.Length > 0)
{
ProcessServiceCommand(args);
return;
}
ServiceBase[] ServicesToRun;
ServicesToRun = new ServiceBase[]
{
new MyService()
};
ServiceBase.Run(ServicesToRun);
}
/// <summary>
/// Executes service installation/uinstallation, or runs it as a process.
/// </summary>
/// <param name="args"> Program arguments.</param>
private static void ProcessServiceCommand(string[] args)
{
string exe = Assembly.GetExecutingAssembly().Location;
if ("/install".Equals(args[0], StringComparison.OrdinalIgnoreCase))
{
object[] assemblyAttributes =
Assembly.GetExecutingAssembly().GetCustomAttributes(false);
string instance =
(from a in assemblyAttributes
where a is AssemblyTitleAttribute
select ((AssemblyTitleAttribute)a).Title).SingleOrDefault();
string name =
(from a in assemblyAttributes
where a is AssemblyDescriptionAttribute
select ((AssemblyDescriptionAttribute)a).Description).SingleOrDefault();
string account = null;
string password = String.Empty;
for (int i = 1; i < args.Length; ++i)
{
if (args[i].StartsWith(
"/instance:", StringComparison.OrdinalIgnoreCase))
{
instance = args[i].Substring(10);
}
else if (args[i].StartsWith(
"/name:", StringComparison.OrdinalIgnoreCase))
{
name = args[i].Substring(6);
}
else if (args[i].StartsWith(
"/account:", StringComparison.OrdinalIgnoreCase))
{
account = args[i].Substring(9);
}
else if (args[i].StartsWith(
"/password:", StringComparison.OrdinalIgnoreCase))
{
password = args[i].Substring(10);
}
else
{
Console.Error.WriteLine("Could not parse: {0}", args[i]);
}
}
InstallService(exe, instance, name, account, password);
}
else if ("/uninstall".Equals(
args[0], StringComparison.OrdinalIgnoreCase))
{
object[] assemblyAttributes =
Assembly.GetExecutingAssembly().GetCustomAttributes(false);
string instance =
(from a in assemblyAttributes
where a is AssemblyTitleAttribute
select ((AssemblyTitleAttribute)a).Title).SingleOrDefault();
for (int i = 1; i < args.Length; ++i)
{
if (args[i].StartsWith(
"/instance:", StringComparison.OrdinalIgnoreCase))
{
instance = args[i].Substring(10);
}
else
{
Console.Error.WriteLine("Could not parse: {0}", args[i]);
}
}
UninstallService(instance);
}
else if ("/run".Equals(args[0], StringComparison.OrdinalIgnoreCase))
{
MyService service = new MyService();
service.OnStart(new string[0]);
Console.WriteLine("Service is running as a process.");
Console.WriteLine("Press <ENTER> to stop and exit.");
Console.ReadLine();
service.OnStop();
}
else
{
Console.WriteLine("To install service:");
Console.WriteLine(" {0} /install", exe);
Console.WriteLine(" [/instance:instance_name [/name:display_name]]");
Console.WriteLine(" [/account:account [/password:password]]");
Console.WriteLine("To uninstall service:");
Console.WriteLine(" {0} /uninstall [/instance:instance_name]", exe);
Console.WriteLine("To run as a regular process:");
Console.WriteLine(" {0} /run", exe);
}
}
/// <summary>
/// Installs service.
/// </summary>
/// <param name="exe"> Path to the executable. </param>
/// <param name="instance"> Name of the service instance. </param>
/// <param name="name"> Display name of the service. </param>
/// <param name="account"> Account name or NULL for LocalSystem. </param>
/// <param name="password"> Password or empty string if any of
/// the machine accounts. </param>
private static void InstallService(
string exe,
string instance,
string name,
string account,
string password)
{
IntPtr scm = Win32.OpenSCManager(
null, null, Win32.SC_MANAGER_CREATE_SERVICE);
if (scm.ToInt32() == 0)
{
Console.Error.WriteLine(
"Failed to open SCM (error {0}).", Win32.GetLastError());
return;
}
try
{
IntPtr service = Win32.CreateService(
scm,
instance,
name,
Win32.SERVICE_ALL_ACCESS,
Win32.SERVICE_WIN32_OWN_PROCESS,
Win32.SERVICE_AUTO_START,
Win32.SERVICE_ERROR_NORMAL,
exe,
null,
0,
null,
account,
password);
if (service.ToInt32() == 0)
{
Console.Error.WriteLine(
"Failed to create service (error {0}).",
Win32.GetLastError());
return;
}
try
{
if (Win32.StartService(service, 0, null) == 0)
{
Console.Error.WriteLine(
"Failed to start service (error {0}).",
Win32.GetLastError());
}
else
{
Console.WriteLine(
"Service installed successfully.");
}
}
finally
{
Win32.CloseServiceHandle(service);
}
}
finally
{
Win32.CloseServiceHandle(scm);
}
}
/// <summary>
/// Uninstalls service.
/// </summary>
/// <param name="instance"> Service instance. </param>
private static void UninstallService(string instance)
{
IntPtr scm = Win32.OpenSCManager(
null, null, Win32.SC_MANAGER_ALL_ACCESS);
if(scm.ToInt32() == 0)
{
Console.Error.WriteLine(
"Failed to open SCM (error {0}).",
Win32.GetLastError());
return;
}
try
{
IntPtr service = Win32.OpenService(
scm, instance, Win32.DELETE | Win32.SERVICE_STOP);
if (service.ToInt32() == 0)
{
Console.Error.WriteLine(
"Failed to open service (error {0}).",
Win32.GetLastError());
return;
}
try
{
Win32.SERVICE_STATUS stat;
if (0 == Win32.ControlService(
service, Win32.SERVICE_CONTROL_STOP, out stat))
{
Console.Error.WriteLine(
"Could not stop the service (error {0}).",
Win32.GetLastError());
}
while (Win32.QueryServiceStatus(service, out stat) != 0
&& stat.dwCurrentState != Win32.SERVICE_STOPPED)
{
Thread.Sleep(1000);
}
if (Win32.DeleteService(service) == 0)
{
Console.Error.WriteLine(
"Failed to delete service (error {0}).",
Win32.GetLastError());
}
else
{
Console.WriteLine(
"Service successfully uninstalled.");
}
}
finally
{
Win32.CloseServiceHandle(service);
}
}
finally
{
Win32.CloseServiceHandle(scm);
}
}
/// <summary>
/// The actual logic.
/// </summary>
private void ServiceMainThread()
{
for (; ; )
{
if (stop.WaitOne(10000))
{
break;
}
using (StreamWriter w =
new StreamWriter(@"c:\temp\testservice.txt", true))
w.WriteLine(
"Tick {0} {1}",
DateTime.Now,
Environment.UserName);
}
}
/// <summary>
/// Win32 thunks.
/// </summary>
private static class Win32
{
public const UInt32 SC_MANAGER_ALL_ACCESS = 0xF003F;
public const UInt32 SC_MANAGER_CREATE_SERVICE = 0x0002;
public const UInt32 SERVICE_WIN32_OWN_PROCESS = 0x00000010;
public const UInt32 SERVICE_AUTO_START = 0x00000002;
public const UInt32 SERVICE_ERROR_NORMAL = 0x00000001;
public const UInt32 STANDARD_RIGHTS_REQUIRED = 0xF0000;
public const UInt32 SERVICE_QUERY_CONFIG = 0x0001;
public const UInt32 SERVICE_CHANGE_CONFIG = 0x0002;
public const UInt32 SERVICE_QUERY_STATUS = 0x0004;
public const UInt32 SERVICE_ENUMERATE_DEPENDENTS = 0x0008;
public const UInt32 SERVICE_START = 0x0010;
public const UInt32 SERVICE_STOP = 0x0020;
public const UInt32 SERVICE_PAUSE_CONTINUE = 0x0040;
public const UInt32 SERVICE_INTERROGATE = 0x0080;
public const UInt32 SERVICE_USER_DEFINED_CONTROL = 0x0100;
public const UInt32 SERVICE_ALL_ACCESS = STANDARD_RIGHTS_REQUIRED |
SERVICE_QUERY_CONFIG | SERVICE_CHANGE_CONFIG |
SERVICE_QUERY_STATUS | SERVICE_ENUMERATE_DEPENDENTS |
SERVICE_START | SERVICE_STOP | SERVICE_PAUSE_CONTINUE |
SERVICE_INTERROGATE | SERVICE_USER_DEFINED_CONTROL;
public const UInt32 DELETE = 0x10000;
public const UInt32 SERVICE_CONTROL_STOP = 0x00000001;
public const UInt32 SERVICE_STOPPED = 0x00000001;
[StructLayout(LayoutKind.Sequential)]
public struct SERVICE_STATUS
{
public UInt32 dwServiceType;
public UInt32 dwCurrentState;
public UInt32 dwControlAccepted;
public UInt32 dwWin32ExitCode;
public UInt32 dwServiceSpecificExitCode;
public UInt32 dwCheckPoint;
public UInt32 dwWaitHint;
};
[DllImport("advapi32.dll")]
public static extern IntPtr OpenSCManager(
string lpMachineName,
string lpSCDB,
UInt32 scParameter);
[DllImport("advapi32.dll")]
public static extern IntPtr CreateService(
IntPtr SC_HANDLE,
string lpSvcName,
string lpDisplayName,
UInt32 dwDesiredAccess,
UInt32 dwServiceType,
UInt32 dwStartType,
UInt32 dwErrorControl,
string lpPathName,
string lpLoadOrderGroup,
int lpdwTagId,
string lpDependencies,
string lpServiceStartName,
string lpPassword);
[DllImport("advapi32.dll")]
public static extern void CloseServiceHandle(IntPtr SCHANDLE);
[DllImport("advapi32.dll")]
public static extern int StartService(
IntPtr SVHANDLE,
UInt32 dwNumServiceArgs,
string lpServiceArgVectors);
[DllImport("advapi32.dll", SetLastError = true)]
public static extern IntPtr OpenService(
IntPtr SCHANDLE,
string lpSvcName,
UInt32 dwNumServiceArgs);
[DllImport("advapi32.dll", SetLastError = true)]
public static extern int ControlService(
IntPtr SCHANDLE,
UInt32 dwControl,
[MarshalAs(UnmanagedType.Struct)]
out SERVICE_STATUS lpServiceStatus);
[DllImport("advapi32.dll", SetLastError = true)]
public static extern int QueryServiceStatus(
IntPtr SCHANDLE,
[MarshalAs(UnmanagedType.Struct)]
out SERVICE_STATUS lpServiceStatus);
[DllImport("advapi32.dll")]
public static extern int DeleteService(IntPtr SVHANDLE);
[DllImport("kernel32.dll")]
public static extern int GetLastError();
}
}
}
Thursday, July 8, 2010
Microsoft Mini
Microsoft Mini blog has a lot of angst today with new round of layoffs at Microsoft.
http://minimsft.blogspot.com/2010/07/kin-fusing-kin-clusion-to-kin-and-fy11.html
I tried to comment, but the Blogger was broken and wouldn't accept it. Since I already typed it up, I am posting it here instead...
I worked at Microsoft for 10 years, then went to Google, then back to Microsoft.
All companies have problems. Apple. Google. Microsoft. They are just different problems, and they look bigger when you are closer to them.
Yes, Microsoft is failing in consumer markets, always have, maybe always will. We don't get consumer. Vista (AKA Abby & Toby platform), Windows Mobile, Kin... Even XBox - it is successful because it was built more for a typical Microsoft employee than for a regular person, it's just MS people love the same kind of videogames that 14 years old males do :-). But if you look at kids or family games on XBox - total failure.
The problem is that we target some mythical "dumb" customer, we don't really know who that is, and we overshoot the level of dumbness by a wide margin. I worked on the first version of Windows Home Server and we had people on the team - not developers, obviously - who seriously tried to argue that our customers don't know what a file share is. I kid you not.
However, just like Microsoft doesn't get consumers, Google and Apple don't get the enterprise. I have participated in creation of a business product at Google, and the people around me did not understand basic concepts like the need for customer service, a refund process, or the like. The entire Google internal system is antithetic to schedule predictability and release stability that is required for a corporate product. People who say that Gmail and Google Docs somehow threaten Exchange and Office have obviously never used these products in a work setting for an extended period of time.
One thing that is going for Microsoft is the plentitude of cultures. We have Xbox team, and an Office team, and Windows team, and Bing, and all these organizations are as unlike each others as they can be. My advice to people, especially developers, who complain about politics, poor managers, boring products, etc - check out the career site! Plenty of teams are hiring, and there are tons of really fun places which will match your preferred style, values, or culture. Just keep moving until you find the right place for YOU. Trust me, it does exist. (By the way, I am hiring, too! If you dream in code and can implement a semaphore if I woke you up at 3am, and like a blend of "old Microsoft" and "new Google" cultures, look me up on career site!)
Finally, it is true that often leaders make companies/armies/countries great. But not always, and never alone, and certainly not in democratic societies :-). I don't think that success of Microsoft in the 90's is directly attributable to BillG and BillG alone, and the steam somehow magically went out of the company the day he left the building. Yes, we have plenty of people at the high places that probably should not have been there. So does Google, so does Apple, so does Oracle, Intel, ..., ..., ... These aren't the people who (most of) you work with, they aren't the people who you meet every day, and I'll let you in on a secret - they aren't the people who make YOUR product a success or a failure. YOU do. They can't affect the stock price much - YOU can, by shipping great products, and by making environment around YOU better, so it attracts more people like YOU.
So don't get consumed by paranoia and politics, focus on your job and your team, and everything else will follow.
And if not, as long as you do the above, you will still be very employable. At Google, at Amazon, or in my team :-).
http://minimsft.blogspot.com/2010/07/kin-fusing-kin-clusion-to-kin-and-fy11.html
I tried to comment, but the Blogger was broken and wouldn't accept it. Since I already typed it up, I am posting it here instead...
I worked at Microsoft for 10 years, then went to Google, then back to Microsoft.
All companies have problems. Apple. Google. Microsoft. They are just different problems, and they look bigger when you are closer to them.
Yes, Microsoft is failing in consumer markets, always have, maybe always will. We don't get consumer. Vista (AKA Abby & Toby platform), Windows Mobile, Kin... Even XBox - it is successful because it was built more for a typical Microsoft employee than for a regular person, it's just MS people love the same kind of videogames that 14 years old males do :-). But if you look at kids or family games on XBox - total failure.
The problem is that we target some mythical "dumb" customer, we don't really know who that is, and we overshoot the level of dumbness by a wide margin. I worked on the first version of Windows Home Server and we had people on the team - not developers, obviously - who seriously tried to argue that our customers don't know what a file share is. I kid you not.
However, just like Microsoft doesn't get consumers, Google and Apple don't get the enterprise. I have participated in creation of a business product at Google, and the people around me did not understand basic concepts like the need for customer service, a refund process, or the like. The entire Google internal system is antithetic to schedule predictability and release stability that is required for a corporate product. People who say that Gmail and Google Docs somehow threaten Exchange and Office have obviously never used these products in a work setting for an extended period of time.
One thing that is going for Microsoft is the plentitude of cultures. We have Xbox team, and an Office team, and Windows team, and Bing, and all these organizations are as unlike each others as they can be. My advice to people, especially developers, who complain about politics, poor managers, boring products, etc - check out the career site! Plenty of teams are hiring, and there are tons of really fun places which will match your preferred style, values, or culture. Just keep moving until you find the right place for YOU. Trust me, it does exist. (By the way, I am hiring, too! If you dream in code and can implement a semaphore if I woke you up at 3am, and like a blend of "old Microsoft" and "new Google" cultures, look me up on career site!)
Finally, it is true that often leaders make companies/armies/countries great. But not always, and never alone, and certainly not in democratic societies :-). I don't think that success of Microsoft in the 90's is directly attributable to BillG and BillG alone, and the steam somehow magically went out of the company the day he left the building. Yes, we have plenty of people at the high places that probably should not have been there. So does Google, so does Apple, so does Oracle, Intel, ..., ..., ... These aren't the people who (most of) you work with, they aren't the people who you meet every day, and I'll let you in on a secret - they aren't the people who make YOUR product a success or a failure. YOU do. They can't affect the stock price much - YOU can, by shipping great products, and by making environment around YOU better, so it attracts more people like YOU.
So don't get consumed by paranoia and politics, focus on your job and your team, and everything else will follow.
And if not, as long as you do the above, you will still be very employable. At Google, at Amazon, or in my team :-).
Sunday, July 4, 2010
Republican party, circa 1956
Republican Party Platform of 1956
http://www.presidency.ucsb.edu/ws/print.php?pid=25838
This is what DemocraticUnderground.com had to say about it:
"By these standards, modern Democrats have become Republicans, and modern Republicans have become batshit crazy. I think I’ve heard Kucinich mention Taft-Hartley, but you get a blank stare if you mention it to anybody under the age of 40. And to think that it used to be a topic of polite conversation among the political classes only 54 years ago."
This: http://journals.democraticunderground.com/eridani/449 has TL;DR.
Incidentally, I am hearing that the health care reform passed by Obama is very, very similar to the Republican counter-offer to the Clinton health plan.
http://www.presidency.ucsb.edu/ws/print.php?pid=25838
This is what DemocraticUnderground.com had to say about it:
"By these standards, modern Democrats have become Republicans, and modern Republicans have become batshit crazy. I think I’ve heard Kucinich mention Taft-Hartley, but you get a blank stare if you mention it to anybody under the age of 40. And to think that it used to be a topic of polite conversation among the political classes only 54 years ago."
This: http://journals.democraticunderground.com/eridani/449 has TL;DR.
Incidentally, I am hearing that the health care reform passed by Obama is very, very similar to the Republican counter-offer to the Clinton health plan.
Saturday, June 26, 2010
Microsoft by the numbers
"The reports of my death have been greatly exaggerated."
-- Mark Twain
http://www.businessinsider.com/microsoft-by-the-numbers-2010-6
-- Mark Twain
http://www.businessinsider.com/microsoft-by-the-numbers-2010-6
Tuesday, June 8, 2010
Friday, June 4, 2010
Terrorism...
Thirty years ago Chief of Staff Mordechai Gur observed that since 1948, "we have been fighting against a population that lives in villages and cities." As Israel's most prominent military analyst, Zeev Schiff, summarized his remarks, "the Israeli Army has always struck civilian populations, purposely and consciously...the Army, he said, has never distinguished civilian [from military] targets...[but] purposely attacked civilian targets."
http://www.chomsky.info/articles/20090119.htm
http://www.chomsky.info/articles/20090119.htm
Tuesday, June 1, 2010
Wednesday, May 19, 2010
Can we stuff 2000 calories in one drink? Yes, we can! (via Reddit)
This is really, truly horrible: http://worldmysteries9.blogspot.com/2010/05/harmful-drinks-in-america.html
The worst-worst drink has 2,010 calories, 131 g fat (68 g saturated), 153 g sugars (this is the number of calories a human should consume in a day; and more fat than one probably should have in a week).
#4 on the list has 1,210 calories, 19 g fat (10 g saturated), and 240 g sugars! Yes, this is really 1/4 of a kg. In one "drink"!!!
The worst-worst drink has 2,010 calories, 131 g fat (68 g saturated), 153 g sugars (this is the number of calories a human should consume in a day; and more fat than one probably should have in a week).
#4 on the list has 1,210 calories, 19 g fat (10 g saturated), and 240 g sugars! Yes, this is really 1/4 of a kg. In one "drink"!!!
Monday, May 17, 2010
Server 200x RAID tips and tricks
Since 2003, Windows Server was shipping with a very cool feature: software RAID.
Software RAID has two major advantages over hardware.
First, hardware RAID protects you against the disk failure, but not against the controller failure. The drive array contains proprietary disk allocation information that varies from manufacturer to manufacturer and controller to controller, so disks are not easily movable between different controllers. So when your controller fails - potentially, long time in the future, when the same boards are no longer available, - or a new release of OS stops supporting your older driver, you may be very much out of luck with your data.
Second, software RAID is considerably more flexible than hardware RAID. Hardware RAID operates on disks as atomic units - you RAID the whole disks together. Software RAID operates on volumes, and each volule can be configured with different level of redundancy. For example, you can have 2 constructs on two disks at the same time - an OS partition that is RAID-1 mirrored with an image on the second disk, and another volume that combines the rest of the space on two drives as a single span or RAID-0. The level of redundancy is selected per volume, not per disk.
Also, for those of us who like coding, software RAID has a very nice software interface (http://msdn.microsoft.com/en-us/library/bb986750(v=VS.85).aspx and its undocumented managed counterpart in Microsoft.Storage.Vds.dll) which allows one to code simple things like checking the health of the storage and send an email if something goes bad.
But what about performance? A while ago when we were designing Windows Home Server, we tested various hardware RAID implementations versus software RAID in Server 2003.
It turns out that both RAID-0 and RAID-1 exhibit very similar performance for both hardware and software solutions. If you think about what the system has to do (write the same data to two disks at the same time in the case of RAID-1) it quickly becomes obvious that hardware implementation does not really add anything over the software in this case: both can write the same data in two places at the same time with the same speed. Big surprise :-).
RAID-5 is a different beast though - there actually is a computation going on, and it is possible to build a specialized chip for doing vector XOR operations that would leave the general purpose x86 in the dust.
A much bigger problem also exists in the lack of integration between the formatting and the RAID code. When you format a RAID drive, the default allocation unit that the UI presents is very small.
Due to the way the software RAID is implemented, it leads to incredibly slow performance. On my relatively powerful system the writes clock at only 20-30MBps (this going to the drives that are supposed to sustain 3Gbps, or 300MBps transfers). Selecting a more reasonable allocation unit of 64k improves the write speed by a factor of four, to almost 120MBps.
The other performance problem that is format related is after creating a new RAID volume, the default behavior is that format and resync happen at the same time. I covered it in the previous blog post here: http://1-800-magic.blogspot.com/2010/05/solution-to-slow-formatting-puzzle.html.
In summary, here are the two very simple rules can make your RAID array much faster:
- Select the 64k as a default allocation unit when formatting the RAID-5 volume.
- When formatting any new RAID volume, use quick format first, wait for the volume to finish resyncing, then repeat with full format if you like (remember to keep the large allocation unit in the second format though!)
Happy RAIDing!
Software RAID has two major advantages over hardware.
First, hardware RAID protects you against the disk failure, but not against the controller failure. The drive array contains proprietary disk allocation information that varies from manufacturer to manufacturer and controller to controller, so disks are not easily movable between different controllers. So when your controller fails - potentially, long time in the future, when the same boards are no longer available, - or a new release of OS stops supporting your older driver, you may be very much out of luck with your data.
Second, software RAID is considerably more flexible than hardware RAID. Hardware RAID operates on disks as atomic units - you RAID the whole disks together. Software RAID operates on volumes, and each volule can be configured with different level of redundancy. For example, you can have 2 constructs on two disks at the same time - an OS partition that is RAID-1 mirrored with an image on the second disk, and another volume that combines the rest of the space on two drives as a single span or RAID-0. The level of redundancy is selected per volume, not per disk.
Also, for those of us who like coding, software RAID has a very nice software interface (http://msdn.microsoft.com/en-us/library/bb986750(v=VS.85).aspx and its undocumented managed counterpart in Microsoft.Storage.Vds.dll) which allows one to code simple things like checking the health of the storage and send an email if something goes bad.
But what about performance? A while ago when we were designing Windows Home Server, we tested various hardware RAID implementations versus software RAID in Server 2003.
It turns out that both RAID-0 and RAID-1 exhibit very similar performance for both hardware and software solutions. If you think about what the system has to do (write the same data to two disks at the same time in the case of RAID-1) it quickly becomes obvious that hardware implementation does not really add anything over the software in this case: both can write the same data in two places at the same time with the same speed. Big surprise :-).
RAID-5 is a different beast though - there actually is a computation going on, and it is possible to build a specialized chip for doing vector XOR operations that would leave the general purpose x86 in the dust.
A much bigger problem also exists in the lack of integration between the formatting and the RAID code. When you format a RAID drive, the default allocation unit that the UI presents is very small.
Due to the way the software RAID is implemented, it leads to incredibly slow performance. On my relatively powerful system the writes clock at only 20-30MBps (this going to the drives that are supposed to sustain 3Gbps, or 300MBps transfers). Selecting a more reasonable allocation unit of 64k improves the write speed by a factor of four, to almost 120MBps.
The other performance problem that is format related is after creating a new RAID volume, the default behavior is that format and resync happen at the same time. I covered it in the previous blog post here: http://1-800-magic.blogspot.com/2010/05/solution-to-slow-formatting-puzzle.html.
In summary, here are the two very simple rules can make your RAID array much faster:
- Select the 64k as a default allocation unit when formatting the RAID-5 volume.
- When formatting any new RAID volume, use quick format first, wait for the volume to finish resyncing, then repeat with full format if you like (remember to keep the large allocation unit in the second format though!)
Happy RAIDing!
Solution to the slow formatting puzzle
A couple of weeks back I posted a puzzle about an experience that I just had with Server 2008 R2 software RAID subsystem: somehow the speed of formatting a new very large RAID-5 array was highly unpredictable. It advanced by a mere 12% a day for the 4 days and then suddenly sped up 4x and finished the last 50% of the formatting in just one day.
Furthermore, I expected precisely this behavior. Why?
This weirdness occurs because of the typical Microsoft phenomenon that was best expressed by an acquaintance of mine that works in Windows Mobile. He thought that the biggest difference between Microsoft's and Apple's approaches to development are these:
Microsoft attacks problems horizontally: it builds core system, then builds layers on top of it. When it's time to ship and something needs to get cut, it's the top of the stack that goes first - and more often than not it's the user experience.
Apple develops software vertically - it enables a user experience, from top to bottom. When they need to cut, they cut the entire experiences (and also maybe the bugs, judging from the fact that my iPhone seems to crash considerably more often than most of Windows Mobile phones I owned in the past). So for example, iPhone would ship with Bluetooth support just for the mono phone headset, but no stereo profile. But the experiences that are left are implemented completely, to the maximum possible level of usability.
Back to our RAID problem. The reason the formatting is so slow in the very beginning is because two things happen at the same time: RAID resync and the formatting. Resync gets kicked off immediately after the RAID system is built, and it simply ensures that the parity volume (in the case of RAID-5) has correct checksums, or the mirrored volume (in the case of RAID-1) has a correct copy of the primary volume.
The other thing that happens once you create RAID volume from the UI is, of course, formatting. One can select quick format which only creates a file system. However, most people probably prefer to run full format when a new drive is added, just to make sure that it is not full of bad sectors.
So we have two write operations that are going on - first one is creating the redundant information, the other is filling the disk with zeroes.
Writing to the disk is very fast these days, but the seek time has barely improved in the last decade, and because the two writes happen in different places on the disk, the whole thing is completely dominated by the disk head moving from one track to another. And with a seek time of 15ms you can only have ~70 of these per second.
Of course, if the format is in progress, there is absolutely no sense to do a resync at the same time - whatever redundancy data resync creates is going to be instantly overwritten by the format.
But there was no Steve Jobs standing over the devs' shoulder, and getting the (filesystem) format in sync with (block device) RAID required two different teams to do something together so it probably got punted. I am sure it's in in a readme somewhere... and now in at least one blog :-).
So to avoid unnecesary delays, select quick format option when adding the RAID volume, wait for the resync to finish, THEN format it again with the full format.
Furthermore, I expected precisely this behavior. Why?
This weirdness occurs because of the typical Microsoft phenomenon that was best expressed by an acquaintance of mine that works in Windows Mobile. He thought that the biggest difference between Microsoft's and Apple's approaches to development are these:
Microsoft attacks problems horizontally: it builds core system, then builds layers on top of it. When it's time to ship and something needs to get cut, it's the top of the stack that goes first - and more often than not it's the user experience.
Apple develops software vertically - it enables a user experience, from top to bottom. When they need to cut, they cut the entire experiences (and also maybe the bugs, judging from the fact that my iPhone seems to crash considerably more often than most of Windows Mobile phones I owned in the past). So for example, iPhone would ship with Bluetooth support just for the mono phone headset, but no stereo profile. But the experiences that are left are implemented completely, to the maximum possible level of usability.
Back to our RAID problem. The reason the formatting is so slow in the very beginning is because two things happen at the same time: RAID resync and the formatting. Resync gets kicked off immediately after the RAID system is built, and it simply ensures that the parity volume (in the case of RAID-5) has correct checksums, or the mirrored volume (in the case of RAID-1) has a correct copy of the primary volume.
The other thing that happens once you create RAID volume from the UI is, of course, formatting. One can select quick format which only creates a file system. However, most people probably prefer to run full format when a new drive is added, just to make sure that it is not full of bad sectors.
So we have two write operations that are going on - first one is creating the redundant information, the other is filling the disk with zeroes.
Writing to the disk is very fast these days, but the seek time has barely improved in the last decade, and because the two writes happen in different places on the disk, the whole thing is completely dominated by the disk head moving from one track to another. And with a seek time of 15ms you can only have ~70 of these per second.
Of course, if the format is in progress, there is absolutely no sense to do a resync at the same time - whatever redundancy data resync creates is going to be instantly overwritten by the format.
But there was no Steve Jobs standing over the devs' shoulder, and getting the (filesystem) format in sync with (block device) RAID required two different teams to do something together so it probably got punted. I am sure it's in in a readme somewhere... and now in at least one blog :-).
So to avoid unnecesary delays, select quick format option when adding the RAID volume, wait for the resync to finish, THEN format it again with the full format.
Subscribe to:
Posts (Atom)
