Taking disk image snapshots - programmatically

A while ago I published a quick program to rip software CD images by opening CD device as a file for block access.

Today I will expand it to read (and restore) the images of a regular hard drive.

Technically speaking, just running the code from http://1-800-magic.blogspot.com/2008/02/backing-up-software-cds.html directly would work (sort of, we'll talk about an important gotcha later). But it will result in a very, very big file. Hard drives are huge these days, but mostly empty. So a lot of the backup image would be the empty space.

Luckily, Windows has a handy facility to figure out what blocks on the disks are actually used. Device IO control FSCTL_GET_VOLUME_BITMAP returns a bitmap where 1s correspond to used blocks, and 0s to the clusters that are empty.

Unluckily, this io control only works for NTFS. For FAT it returns the bitmap, but it does not start with the beginning of the disk - rather, with the end of the FAT tables as it seems.

Funnily enough, FSCTL_GET_VOLUME_BITMAP also reports the starting cluster where the bitmap is supposed to start. For FAT, it should have been non-zero - we could then backup everything before it, and then whatever is in the bitmap after, but no luck - it just happily returns zero, and the shifted bitmap.

There's surely a way to figure out the sizes of FATs, but since this filesystem is less and less common, and the drives formatted with FAT are small, so in the interests of simplicity I just backup the whole disk if the filesystem is not NTFS.

Another bit of interesting data is that your partition is actually bigger than your file system by one cluster. This cluster contains a copy of the the boot sector for the partition. The intent, I think, is to use it in disk recovery. We back it up for NTFS only, not for other file systems.

Alright, so here's the code. The usual things first. See this post for implementation of AtoI: http://1-800-magic.blogspot.com/2008/02/down-with-atoi.html.

#define UNICODE 1
#define _UNICODE 1

#include <windows.h>
#include <stdio.h>
#include <ctype.h>
#include <strsafe.h>
#include <assert.h>

#define ARRLEN(c) (sizeof(c)/sizeof(c[0]))

template<class IntType> bool AtoI(const WCHAR *sz, IntType *out) {
...}

GetDeviceName translates the disk number (0-N), a drive letter (A:), or a volume mount point (C:\, or c:\cdrom) to a name that Windows would understand as a block device.

bool GetDeviceName(WCHAR *szDevName,
                   size_t cchDevName,
                   const WCHAR *szInput) {
  unsigned int disk_no = 0;
  if (AtoI<unsigned int>(szInput, &disk_no)) {
    // It's a physical disk
    StringCchPrintfW(szDevName,
                     cchDevName,
                     L"\\\\.\\PHYSICALDRIVE%d", 
                     disk_no);
    return true;
  }

  WCHAR drive_letter = toupper(szInput[0]);
  if (drive_letter >= 'A' && 
      drive_letter <= 'Z' &&
      szInput[1] == ':' &&
      szInput[2] == '\0') {
    // It's a drive letter
    StringCchPrintfW(szDevName,
                     cchDevName,
                     L"\\\\.\\%c:",
                     drive_letter);
    return true;
  }

  WCHAR sz[_MAX_PATH];
  const WCHAR *p = wcsrchr(szInput, '\\');
  if (!p || p[1] != '\0') {
    // Mount point needs to end in backslash
    StringCchCopyW(sz, ARRLEN(sz), szInput);
    StringCchCatW(sz, ARRLEN(sz), L"\\");
    szInput = sz;
  }

  if (!GetVolumeNameForVolumeMountPointW(
      szInput, szDevName, cchDevName))
    return false;

  // This returns a name ending with '\'
  // but CreateFile does not take it, so
  // we need to get rid of it.
  WCHAR *q = wcsrchr(szDevName, '\\');
  if (q && q[1] == '\0')
    *q = '\0';

  return true;
}

Then opening is straightforward. Notice the sharing mode - this is part of the gotcha I will be talking about later. Also notice that I removed the FSCTL_ALLOW_EXTENDED_DASD_IO from here. This IO control allows reading and writing past the end of the volume as understood by the file system. Since we backup the overflow sector only for NTFS, I moved it to NTFS-specific place.

HANDLE OpenBlockDevice(const WCHAR *szDevName,
                       bool fWrite) {
  HANDLE h = CreateFile(szDevName,
    fWrite ? GENERIC_WRITE : GENERIC_READ,
    FILE_SHARE_WRITE, NULL, OPEN_EXISTING,
    FILE_FLAG_NO_BUFFERING |
        (fWrite ? FILE_FLAG_WRITE_THROUGH : 0),
    NULL);
  return h;
}

Now the fun part - let's get the volume bitmap. The idiosyncrasy of this function is that there's no way - at least not a way I know of - to get the size of the bitmap that we need to pre-allocate, first. If you give it the NULL input, as is customary with majority of Windows API, and expect the output to say how many bytes it needs, you will be disappointed - the function just errors out, and returns 0 in the bytes returned. Therefore, I do this ugly dance:

VOLUME_BITMAP_BUFFER *GetVolumeBitmap(
    HANDLE hDev, unsigned int *puiBitmapSize) {
  STARTING_LCN_INPUT_BUFFER sStartLcn;
  sStartLcn.StartingLcn.QuadPart = 0;

  DWORD dwBitmapSize = 0;
  DWORD dwAllocatedSize = 64 * 1024;
  VOLUME_BITMAP_BUFFER *pVolumeBitmap = NULL;
  for ( ; ; ) {
    pVolumeBitmap = (VOLUME_BITMAP_BUFFER *)
        LocalAlloc(LMEM_FIXED, dwAllocatedSize);

    BOOL ret = DeviceIoControl(hDev,
                      FSCTL_GET_VOLUME_BITMAP,
                      &sStartLcn,
                      sizeof(sStartLcn),
                      pVolumeBitmap,
                      dwAllocatedSize,
                      &dwBitmapSize,
                      NULL);
    if (ret) {
      *puiBitmapSize = dwBitmapSize;
      return pVolumeBitmap;
    }

    if (GetLastError() != ERROR_MORE_DATA)
      return NULL;

    dwAllocatedSize *= 2;
  }
}

I want to support both the case where we backup the entire volume as well as the case where we would only backup the used clusters (let's call it sparse backup). I do it by prepending the following header to the sparse backup file - so when I read it back, if the magic string is there, I know that I need to read the volume bitmap, and then write the data accordingly. If it's not there, the file is basically the image and I just blast the whole thing on the disk:. When I restore, I also need the sizes of the cluster and the sector, so I know how to interpret the bitmap. If I were somewhat less lazy, I would also care to check that on restore the sector size of the disk matches that of the backup.

#define SIG "Compressed Disk Image"

struct BackupHeader {
  char signature[sizeof(SIG)];
  unsigned int uiSectorSize;
  unsigned int uiClusterSize;
  unsigned int uiBitmapSize;
};

I put together a generic function that can transfer data from disk to file, and from file to disk according to the volume bitmap if it is given. The small gotcha here is that the size of the volume bitmap is in BITS. Storage API rarely follows standard Windows patters for some reason!

Notice that in the case where pVolumeBitmap is NULL, the function gets reduced to a very simple loop that just reads and writes the entire block device.

// This function assumes the ownership of
// pVolumeBitmap, i. e. it frees it.
// hFrom is the handle from which the data is read.
// hTo is the handle to which the data is written.
// hSeek is always the handle of the device - this
// is the pointer that is being moved to skip empty
// extents.
// For backup: hFrom = hSeek - disk handle
//             hTo - file handle
// For restore: hTo = hSeek - disk handle
//             hFrom - file handle
// If device is not an NTFS volume, pBackupData,
// pVolumeBitmap are both NULL.
DWORD Transfer(HANDLE hFrom,
               HANDLE hTo,
               HANDLE hSeek,
               BackupHeader *pBackupData,
               VOLUME_BITMAP_BUFFER *pVolumeBitmap,
               void *buffer,
               const unsigned int cbBufferSize,
               unsigned __int64 &ui64Bytes) {
  DWORD dwErr = ERROR_SUCCESS;

  // This is used to step through the bitmap
  // of used clusters.
  unsigned char bmpMask = 1;
  unsigned char *bmpIndex = NULL;
  int maxClustersPerRead = 0;
  unsigned __int64 maxClusters = 0;
  if (pVolumeBitmap) {
    bmpIndex = pVolumeBitmap->Buffer;
    maxClusters = pVolumeBitmap->BitmapSize.QuadPart;
    maxClustersPerRead = cbBufferSize /
        pBackupData->uiClusterSize;
  }

  unsigned __int64 cluster = 0;
  bool fTerm = false;

  while (!fTerm) {
    DWORD dwToMove = cbBufferSize;
    if (pVolumeBitmap) {
      // Skip empty. Note that at the end
      // of this loop cluster can be more
      // than maxCluster, because sometimes
      // we step in 8.
      while (cluster < maxClusters) {
        if (!*bmpIndex) {
          cluster += 8;

          ++bmpIndex;
          assert(bmpMask == 1);

          continue;
        }

        if ((*bmpIndex & bmpMask) == 0) {
          ++cluster;

          bmpMask <<= 1;
          if (! bmpMask) {
            bmpMask = 1;
            ++bmpIndex;
          }

          continue;
        }
        break;
      }

      dwToMove = 0;
      int numClusters = 0;
      while (cluster  + numClusters < maxClusters
          && numClusters < maxClustersPerRead
          && (*bmpIndex & bmpMask) != 0) {
        dwToMove += pBackupData->uiClusterSize;
        ++numClusters;

        bmpMask <<= 1;
        if (!bmpMask) {
          bmpMask = 1;
          ++bmpIndex;
        }
      }

      assert(dwToMove <= cbBufferSize);
      if (dwToMove == 0) { // The End?
        // The last sector on the volume
        // contains duplicate of the boot
        // sector, but is not part of the
        // volume map.
        dwToMove = pBackupData->uiSectorSize;

        // Since we might have skipped more
        // clusters than there were because
        // sometimes we count by 8-s, we reset
        // the current cluster count as well:
        cluster = maxClusters;

        // Convenient place to free the bitmap.
        LocalFree(pVolumeBitmap);
        pVolumeBitmap = NULL;
        fTerm = true;
      }

      unsigned __int64 offset = cluster *
        (unsigned __int64)pBackupData->uiClusterSize;

      if (!SetFilePointerEx(hSeek,
                            *(LARGE_INTEGER *)&offset,
                            NULL,
                            FILE_BEGIN)) {
        dwErr = GetLastError();
        wprintf(L"Seek error %d (0x%08x)\n",
            dwErr, dwErr);
      }

      cluster += numClusters;
    }

    DWORD dwRead = 0;
    if (!ReadFile(hFrom, buffer,
        dwToMove, &dwRead, NULL)) {
        dwErr = GetLastError();
        wprintf(L"\nRead error %d (0x%08x)\n",
            dwErr, dwErr);
        break;
    }

    if (dwRead == 0)
      break;

    DWORD dwWrit = 0;
    if (!WriteFile(hTo, buffer,
        dwRead, &dwWrit, NULL) ||
        dwWrit != dwRead) {
      dwErr = GetLastError();
      wprintf(L"\nWrite error %d (0x%08x)\n",
          dwErr, dwErr);
      break;
    }

    ui64Bytes += dwWrit;
    wprintf(L"%I64u\r", ui64Bytes);
  }

  return dwErr;
}

This makes read much shorter than what it would have been:

DWORD ReadToFile(HANDLE hDev,
                 HANDLE hFile,
                 void *buffer,
                 const unsigned int cbBufferSize,
                 unsigned __int64 &ui64BytesRead) {
  DWORD dwErr = ERROR_SUCCESS;

  unsigned int uiBitmapSize = 0;
  VOLUME_BITMAP_BUFFER *pVolumeBitmap = NULL;

  NTFS_VOLUME_DATA_BUFFER sVolumeData;
  DWORD dwRead = 0;
  // For NTFS volumes we can store just the
  // clusters that are used. We get the
  // bitmap where 1 means that the cluster
  // is used here.
  if (DeviceIoControl(hDev,
                      FSCTL_GET_NTFS_VOLUME_DATA,
                      NULL,
                      0,
                      &sVolumeData,
                      sizeof(sVolumeData),
                      &dwRead,
                      NULL)) {
    // For NTFS disable volume boundary checks by FS
    DWORD dwBytes = 0;
    DeviceIoControl(hDev,
                    FSCTL_ALLOW_EXTENDED_DASD_IO,
                    NULL,
                    0,
                    NULL,
                    0,
                    &dwBytes,
                    NULL);

    pVolumeBitmap = GetVolumeBitmap(hDev, &uiBitmapSize);
  }

  BackupHeader h;
  DWORD dwWrit = 0;
  if (pVolumeBitmap) {
    memcpy(h.signature, SIG, sizeof(SIG));
    h.uiBitmapSize = uiBitmapSize;
    h.uiSectorSize = sVolumeData.BytesPerSector;
    h.uiClusterSize = sVolumeData.BytesPerCluster;
    if (!WriteFile(hFile,
                   &h,
                   sizeof(h),
                   &dwWrit,
                   NULL)
        || dwWrit != sizeof(h)) {
      dwErr = GetLastError();
      wprintf(L"Write error %d (0x%08x)\n",
          dwErr, dwErr);
      return dwErr;
    }

    if (!WriteFile(hFile,
                   pVolumeBitmap,
                   uiBitmapSize,
                   &dwWrit,
                   NULL)
        || dwWrit != uiBitmapSize) {
      dwErr = GetLastError();
      wprintf(L"Write error %d (0x%08x)\n",
          dwErr, dwErr);
      return dwErr;
    }
  }

  return Transfer(hDev,
                  hFile,
                  hDev,
                  &h,
                  pVolumeBitmap,
                  buffer,
                  cbBufferSize,
                  ui64BytesRead);
}

And write, even shorter:

DWORD WriteFromFile(HANDLE hDev,
                 HANDLE hFile,
                 void *buffer,
                 const unsigned int cbBufferSize,
                 unsigned __int64 &ui64BytesWritten) {
  DWORD dwRead = 0;
  DeviceIoControl(hDev,
                  FSCTL_ALLOW_EXTENDED_DASD_IO,
                  NULL,
                  0,
                  NULL,
                  0,
                  &dwRead,
                  NULL);

  unsigned int uiBitmapSize = 0;
  VOLUME_BITMAP_BUFFER *pVolumeBitmap = NULL;

  BackupHeader h;

  if (ReadFile(hFile, &h, sizeof(h), &dwRead, NULL)
      && dwRead == sizeof(h)) {
    if (memcmp(h.signature, SIG, sizeof(SIG)) == 0) {
      pVolumeBitmap = (VOLUME_BITMAP_BUFFER *)
          LocalAlloc(LMEM_FIXED, h.uiBitmapSize);
      if (!ReadFile(hFile, pVolumeBitmap, h.uiBitmapSize,
          &dwRead, NULL) || dwRead != h.uiBitmapSize) {
        wprintf(L"Backup corrupted - could not read the "
                L"volume bitmap!\n");
        return ERROR_DATA_NOT_ACCEPTED;
      }
    } else
      SetFilePointer(hFile, 0, NULL, FILE_BEGIN);
  }

  DWORD dwErr = Transfer(hFile,
                         hDev,
                         hDev,
                         &h,
                         pVolumeBitmap,
                         buffer,
                         cbBufferSize,
                         ui64BytesWritten);

  DeviceIoControl(hDev,
                  IOCTL_DISK_UPDATE_PROPERTIES,
                  NULL,
                  0,
                  NULL,
                  0,
                  &dwRead,
                  NULL);

  return dwErr;
}

The main just parses the command line then:

int wmain(int argc, WCHAR **argv) {
  if (argc < 4) {
    wprintf(L"Usage: %s {disk|volume} cmd "
            L"args\n", argv[0]);
    wprintf(L"Where disk is physical number "
            L"of the drive, e. g. 0\n");
    wprintf(L"Volume is a drive letter or a "
            L"full path to \n");
    wprintf(L"the mount point, e. g. a: or "
            L"c:\\mount\\vol\n");
    wprintf(L"Commands:\n");
    wprintf(L"readto filename - read the "
            L"contents of the device into "
            L"a file\n");
    wprintf(L"writefrom filename - write the "
            L"contents of the file onto "
            L"the disk or volume\n");
    return 0;
  }

  UINT uiErrModeSav = SetErrorMode(
      SEM_FAILCRITICALERRORS);

  WCHAR szDevName[_MAX_PATH];
  if (!GetDeviceName(szDevName,
                     ARRLEN(szDevName),
                     argv[1])) {
    wprintf(L"Could not recognize %s\n", argv[1]);
    SetErrorMode(uiErrModeSav);
    return 1;
  }

  const int kBufferSize = 64 * 1024;
  void *buffer = VirtualAlloc(
      NULL,
      kBufferSize,
      MEM_COMMIT | MEM_RESERVE,
      PAGE_READWRITE);
  if (!buffer) {
    DWORD dwErr = GetLastError();
    wprintf (L"Could not reserve memory, "
             L"error %d (0x%08x)\n", dwErr, dwErr);
    SetErrorMode(uiErrModeSav);
    return 3;
  }

  DWORD dwTickStart = GetTickCount();
  unsigned __int64 ui64Bytes = 0;
  DWORD dwErr = ERROR_SUCCESS;
  if (_wcsicmp(argv[2], L"readto") == 0) {
    HANDLE hDev = OpenBlockDevice(szDevName, false);
    if (hDev != INVALID_HANDLE_VALUE) {
      HANDLE hFile = CreateFile(
          argv[3], GENERIC_WRITE, 0, NULL,
          CREATE_ALWAYS, 0, NULL);
      if (hFile != INVALID_HANDLE_VALUE) {
        wprintf(L"%s --> %s:\n", szDevName, argv[3]);
        dwErr = ReadToFile(hDev,
                           hFile,
                           buffer,
                           kBufferSize,
                           ui64Bytes);
        CloseHandle(hFile);
        if (dwErr != ERROR_SUCCESS)
          DeleteFile(argv[3]);
      } else {
        dwErr = GetLastError();
        wprintf (L"Could not open %s, error %d "
                 L"(0x%08x)\n", argv[3],
                 dwErr, dwErr);
      }
      CloseHandle(hDev);
    } else {
      dwErr = GetLastError();
      wprintf(L"Could not open %s, "
              L"error %d (0x%08x)\n",
              argv[1], dwErr, dwErr);
    }
  } else if (_wcsicmp(argv[2], L"writefrom") == 0) {
    HANDLE hDev = OpenBlockDevice(szDevName, true);
    if (hDev != INVALID_HANDLE_VALUE) {
      HANDLE hFile = CreateFile(
          argv[3], GENERIC_READ, 0, NULL,
          OPEN_EXISTING, 0, NULL);
      if (hFile != INVALID_HANDLE_VALUE) {
        wprintf(L"%s --> %s:\n", argv[3], szDevName);
        dwErr = WriteFromFile(hDev,
                              hFile,
                              buffer,
                              kBufferSize,
                              ui64Bytes);
        CloseHandle(hFile);
      } else {
        dwErr = GetLastError();
        wprintf (L"Could not open %s, error %d "
                 L"(0x%08x)\n", argv[3],
                 dwErr, dwErr);
      }
      CloseHandle(hDev);
    } else {
      dwErr = GetLastError();
      wprintf(L"Could not open %s, "
              L"error %d (0x%08x)\n",
              argv[1], dwErr, dwErr);
    }
  } else {
    wprintf (L"Command %s not recognized.\n",
             argv[2]);
    dwErr = ERROR_NOT_SUPPORTED;
  }

  if (dwErr == ERROR_SUCCESS &&
      ui64Bytes != 0) {
    wprintf(L"Transferred %I64u bytes in %d "
            L"seconds.\n", ui64Bytes,
            (GetTickCount() - dwTickStart) / 1000);
  }

  VirtualFree(buffer, 0, MEM_RELEASE);
  SetErrorMode(uiErrModeSav);

  return dwErr == ERROR_SUCCESS ? 0 : 4;
}

Now it WOULD be time to publish the binary and the source code in its intact form, except for one thing. This code works poorly if it is run against your system volume. The problem is that the system volume does not usually exist in the consistent state - it is being constantly written to, with part of the information already on disk, and another part in buffers.

Moreover, while the backup is in progress, another program can add data, thus creating an inconsistency between the disk and the NTFS descriptors (say, we'd read the disk, but not yet the MFT. Then the program would add a file to the area of the disk we have already written, and modify the MFT - we would've gotten an image where there's a file full of garbage).

The solution is called VSS, and I will cover it in another post. Hopefully, at that point we will have the code that would work well with the system volume. But even before that, I owe an article on testing it.

As it stands now, it can be used on secondary volumes (when you're sure nothing is writing on it), unprotected CDs and DVDs, or on a system volume if you're booting an alternative OS.