Virtualizing a desktop

I saw that Ross was a little curious about virtualization at the Smurf on Spreadsheets blog, and figured I’d share my reasons, some experiences, and methods for working with this tool. I might even post more on this, as VMWare has some really cool features that can make setting up testing environments very easy.

What is a virtual machine and why use one?

For anyone who doesn’t know what a virtual machine (VM) is, this is a pretty simplistic summary:

You run a “host” operating system, be it Windows XP, Vista, Linux, or whatever. You then install a virtualization software and create a “space” in which you can install a virtual machine. Into this space you install another (“guest”) operating system. It can be the same, or different if you like. Basically, it allows you to run another computer within a window on your machine. The guest virtual machine works just like a standalone PC. It boots up, shuts down, accesses the internet, etc.

There are a few reasons why I use a virtual machine for my desktop at work:

  • I had an old desktop with a ton of programs configured on it. Rather than have to reinstall everything, and source out programs whose install disks I may have lost, I can just run the old computer within a window on my new computer.
  • We use Novell Groupwise 6.5 for our email systems. It is not supported on Windows Vista. Again, virtualization allows me to still run this on XP.
  • It allows a clear separation between my work and personal use of the PC. I can turn off “work” at the end of the day and it is all contained within that space.

What does it take to run a Virtual Machine?

You basically need two things in order to run a VM: software and resources.

The main virtual machine softwares in use right now seem to be VMWare (which comes in a few flavours), or Microsoft’s Virtual PC. Personally, I’d rate VMWare Workstation about five years ahead of Virtual PC. Virtual PC is free, but you can download a trial version of VMWare Workstation, create your VM, then use it in VMWare’s free Player once your workstation trial expires. So cost shouldn’t be too much of a cost unless you feel the need to create new VM’s on a regular basis.

The other thing you need is resources. It should be pretty obvious that you’ll still need enough resources to run your host operating system, as well as the resources to run the guest operating system as well. So assuming that you want to run a Vista host you’ll need the minimum of 512MB of RAM. If you want to run a Vista guest, you’ll need another 512MB of RAM for the guest operating system. Now, this is a bare minimum, of course. You’re going to get way better performance the more resources you have. On my system, I’m running a Core 2 duo processor and 4GB of RAM, with one processor and 1.5GM of RAM dedicated to the guest. The rest is reserved for the host OS, in case I want to run 2 VM’s at the same time.

This is a pretty brief overview, but to put it simply, you should expect that you’ll get best performance if you add more oomph to your host system . You’ll also get better performance running a host OS that doesn’t consume all the resources… in other words… Vista makes a lousy host. ;)

Converting your existing PC to a VM

Now, due to my little disaster the other day, I had to rebuild the Windows XP virtual machine that I use for work. I figured that I would document how I did it, partly so I remember for next time, and partly for anyone else who is interested. Remember that I’m using VMWare, not Virtual PC, so the steps listed are geared towards VMWare.

  1. The first step was to download VMWare’s Converter program. This utility can be used from a couple of different angles, including allowing you to virtualize (image) your PC from your host PC. The route that worked best for me was to install the converter on the old PC and image it to a USB hard drive.
  2. I then took the image, copied it onto my new computer and booted it up in VMWare. Make sure when you do this, that you choose the settings you want for processors, RAM, etc… Try not to get too aggressive at this point. Things will be a bit slow in the guest until you tune it, and in my experience, putting in more processors and high RAM doesn’t seem to help too much.
  3. Next you’ll need to activate windows. Virtualization moves your machine into a hardware agnostic environment, but it does this by replacing all of your “hardware” with generic virtualized hardware. Because Windows activation keys are tied to your hardware, this massive shift requires an immediate reactivation before you can move on.
  4. Upon booting into windows, I immediately installed VM Tools. This is huge, since it improves performance of the guest OS, affecting the mouse, video, etc…
  5. If you want to be able to boot up your old PC as well as the new VM based on that machine, you’ll need to change the security identifier of one of the machines.  There is a tool call NewSID available from Microsoft TechNet to do exactly this.  It takes a while to run this, but can be well worth it to install.
  6. I then spent a huge amount of time uninstalling applications that I did not need any more. Many of these were legacy apps that I should have cleaned up on the old PC, others were utilities from the manufacturer of my old PC. I basically hacked and pruned out everything I could, leaving only the programs that I still needed. The reason? So I could compress the image to make it run more quickly and efficiently. As you can imagine, a few reboots were necessary in this process.
  7. After uninstalling all this stuff and rebooting, I ran my favourite registry cleaner (CCleaner) in the VM a few times. I made sure that I cleaned up all the legacy reg keys, as well as cleaned up all the temp files and other garbage that was left behind.
  8. Another reboot later, and it was time to do a full defragmentation inside the guest OS. This is an important step in cleaning up, as I had uninstalled a ton of stuff, so the drive was badly fragmented. I have my own tool of choice for this, but you could use the standard defragmenter built into the OS. Personally, I like to run a defragmenter twice, just to make sure it’s all good, but that is probably overkill.
  9. Once this was done, I shut down the VM and used VMWare’s tools to defragment the virtual disk file. Fragmentation in a VM can actually occur on 3 levels. Inside the guest OS itself, on the actual virtual disk file that contains the VM, and in the host operating systems disk as well.
  10. You may have guessed the next step already… I defragmented my host machine’s hard disk to make sure that everything was set up nicely to be as efficient as possible.

Now, at this point I went ahead and used it for a day. It worked okay, but I was having a couple of issues. The first was that the performance wasn’t all that I wanted out of it. The second was that I didn’t have my dual monitor support. So I set to fixing both of those.

  1. With the VM booted up, I decided to “shrink” the virtual hard disk. This is important, since my old PC had a 37.3GB hard drive. After all the other performance tuning and program removal, it didn’t need to be this large. So using the VM Tools from within the guest OS, I did the “shrink”. Essentially, this just scales the disk size down to only use what it needs, allowing it to dynamically grow later if it needs more space. My virtual disk decreased from 37.3 GB to 17.2GB. It now boots WAY faster as well. :)
  2. Finally, I upgraded the VM to Workstation v6 from Workstation v5. I’m not sure if this is a problem because I’m using an old version of the converter or not, but all the same it was necessary. v6 supports USB in the guest, as well as adds dual monitor support.

And that’s the end of it. It is a lot of work to set up, sure, but less so that installing a fresh copy of Windows and customizing it to the nth degree to get it just right.

Oh… and one final thing… this time I made a backup of the completed VM image. ;)

The value of a backup…

To quote my friend John, “The value of a backup is not in the backup.  It’s in the restore.”  I got to feel this first hand last night.
I’ve been having some performance issues over the last week with my wireless connection, and after doing a fair bit of reading, decided to do a laptop rebuild.  I’ve been meaning to do it for a while anyway, as I wanted to try the 64bit edition of Vista, and this seemed like a good time to take the plunge.  Armed with an excellent ThinkPad specific post at NotebookReview.com, I proceeded to set up for this process.  I made my restore disks, downloaded all the drivers and placed them all on a USB hard drive.  In addition, I also made a backup of my data, as suggested.

Now, I’ve lost count of how many times I’ve reinstalled a PC/laptop OS, and I’ve never had an issue.  I’ve used CD’s, DVD’s, hard drives and USB drives to back up and store my data during migrations.  In fact, the USB drive I used for this migration has already been used to do it previously.  This time, though, I did something that turned out to be… well… it seems to be leaning towards the stupid side of the brave vs stupid balance.

The drive is a Western Digital 250GB external USB hard drive.  It works very nicely, is compact, quiet, and has more than enough storage for what I need to do.  It also came with a little program on it, called WDSync.exe, to sync data between your computer and the hard drive.  Now seriously, why would you have that for any other reason than to create a backup?

I figured I’d give this thing a shot, and started using it to sync my data on Wednesday night.  For the next two days, I ran the software diligently to sync the data between my laptop and the drive, and it appeared to work really well.  Remembering the saying above, I also took it to my wife’s PC to see if I could get the data back.  I tested with a few files, copying them back to her machine, and it seemed to work.  I was thinking, “hey, this is pretty cool!” :)  No more need to wipe the drive and re-copy everything… I only needed to back up the change between the two, so it was way faster!

As an additional part of the back story, I should also mention that I run VMWare on this laptop, and actually run my work desktop as a Virtual Machine on a daily basis.  There are a few reasons for this, which I won’t go in to, but suffice it to say that the VM is used Monday through Friday, and is quite large.

So Friday night, I sat down, and synched the drive one last time before I wiped the laptop.  I watched carefully to make sure that the VMWare image that I use a work was synched.  19 huge vmdk files later, I was ready to begin.  As a final check, I just opened the synching software to make sure all the VMWare files were indeed listed, and they were.  I unplugged the USB drive, removed the Thinkpad’s hidden restore partition, rebooted with my MSDN Vista x64 DVD in the drive and set to work.

The reinstall went very smoothly.  I’ve got no complaints there.  The more times I reload Vista from scratch, the easier I find it to do.  Upon gaining a desktop again, I then installed all the correct drivers, and everything seemed good.   But then I tried to restore my data…

First off, you don’t appear to be able to sync from your backup to a new computer.  It gives you a cryptic message about being a new machine, and needing to do something which sounds ominously like deleting the backup set.  Not a chance was I doing that!  There is also an option to copy files to the PC, so I went that route.  Making sure everything was selected, I set it to go.  It went ripping through files, cheerily putting them in a deeply nested directory from where I told it too, until it hit my work PC VM.  I then received a nice little message about the USB drive being inaccessible.  Naturally, I told it to try again… and again… and again…  By this time, a cold sweat started forming… again… again… again… and then it started peeling off more files.  I started to relax until this process repeated itself.  I just kept telling it to try again and eventually it worked… or so I thought.

Upon checking my files, the software had failed to restore two main blocks of files (that I know of).  One was my entire work VM image.  That sucks.  The other set of files were a huge amount of work I did on the book stuff.  Fortunately that has all gone in now, and I have email backups of much of it, but still, it sucks.

I tried the restore again, and received the same issue.  I’m now pretty sure that the original synching process itself didn’t work right, although I never got so much as a warning.  You’d think that, just maybe, that would have been something important to mention to a user wouldn’t it?  Maybe a different message than “All files successfully synched.” (Or whatever it was.)  After all, isn’t the entire point of synching your files to make a backup of them?  And the entire point of a backup to be able to restore it to a different computer?  (Even if it is the same PC, but reformatted, it is essentially a different PC to the device!)

After further playing, I found out that a few of the snapshots in my main VMWare testing tree won’t load either.  I’m not impressed.

In hindsight, I should have stuck with just making a straight file backup to the USB drive, and not bothered with their software.  I should have.  I did make an effort to check the backup, but obviously didn’t do enough checking.  I  supposed I should have attempted to restore the entire thing to my wife’s PC, but it’s a little late to be doing that now.

As far as good news for me, at least the book, images and example files are all in to the publisher.  Of course, I wouldn’t have done this if they weren’t.  ;)

And as far as my work VM… the old PC I virtualized is still in my office, and has not been wiped out yet.  Using VMWare’s migration tool, I can easily re-image it,  it will just take some time.  I’m also fortunate in that I can access our Citrix server to get a working desktop while I’m waiting.  (I’m on a desktop as I do a fair bit of coding and testing apps, and I don’t want to risk bluescreening the Citrix server.

Overall, this is just going to cost me some time, and didn’t really lose anything critical, but it’s still VERY frustrating.  Particularly where I thought that I had done sufficient testing.

As an interesting point, I’d been talking with a colleague at work about backups of data for the “Road Warriors” of the organization, as well as synching data.  I see these as very similar issues, which are becoming more and more important as time goes on.  My issue sort of helps to demonstrate that point.  In my case, I have a file that is important to do my work, and it’s now gone and needs to be rebuilt.  In other cases, we may have people who take a file away, then need to load it back up to the server so others can work on it.

To my understanding, this is one of the issues that Microsoft Groove is supposed to solve.  For those who don’t know, it’s supposed to be a software to sync data over the inter/intra-net to a central server.  Every user would then be able to share files which get synched up when the network/internet is available.  For laptop users, this also offers a very convenient backup solution as well.  I may just have to start looking at this more seriously now…

Issues with Firefox 2.0.0.8 update?

I just installed the Firefox 2.0.0.8 security update, and now I’m getting 3 “Failure in Chrome Registration” error messages each time I start Firefox.  In addition, it’s disabled some of my addons (which I can only assume is what the errors are all about.  The addons that have been disabled for me are:

  • Google Toolbar
  • Fasterfox
  • IETab
  • ThinkVantage Password Manager (This is a Lenovo Thinkpad thing)

In addition, it has also caused the Littlefox theme to stop working.  Very frustrating.

Is anyone else seeing these kinds of issues?

PDFCreator and Vista Bug

As many of you know, I’m a big fan of the free PDFCreator utility since it allows us to automate PDF creation via VBA in Office 2003 and earlier.  One issue with it, however, is that it won’t run on Vista yet.

It looks like PDFForge has offered a $150 reward to anyone who can solve the bug.

Hopefully they can get this sorted, as it is a great little utility.  Even if you don’t want to automate the process, it’s a handy little tool.

Where I’ve been…

Some of you may have noticed that it’s been a bit quiet around here lately.

Shortly after the MVP Summit back in April, fellow Excel MVP Robert Martin invited me to co-author a book on customizing the Ribbon in Excel, Access and Word. Since that time, I’ve been spending virtually all of my free time working on this project, which means that I haven’t been seen too much in the forums or on my own site. I’m happy to report that we’re very close to the end of this project now, and that the book should be out early in December 2007/January 2008. You can find it at Wiley‘s site, among others. I’ve also noticed that pre-sales are being accepted at Amazon now. :)

It’s an All MVP team that’s been involved with the project, which is kind of cool too. The team consists of:

I’m hoping that this makes a useful read for (more than) a few people. It’s geared at fairly wide target audience; from beginners who know no code at all through experienced developers who have not been exposed to the Ribbon. We’ve packed it full of tables and practical examples, and discuss the full development cycle from creation through to deployment of the created tools in Excel, Access and Word.

One thing that was important to me is that Office developers like myself, those who migrated into code after learning their program, had a book geared towards them. Therefore this book is PURELY Office and VBA. There is no VB.Net/VSTO here. I’m certainly not saying that there is no need for a book like this geared towards the VB.Net/VSTO crowd. There is, but I’m leaving that for someone else at this stage, as I’m not experienced enough with .Net to be able to write it. I will probably get back into fooling with .Net again once this is done, though.
So that was the first thing that’s been keeping me busy… but there is more!

On September 18th, I received at promotion at work. Our General Manager stepped down, my boss accepted the role of Acting GM and I became Controller of Fairwinds. (For those who don’t know, a Controller is the person responsible for all of the accounting, financial statements, and generally making sure that the company is being run in a financially sound manner.) I have hired someone to take over some of my old tasks, and she started this past week. And then, as only Murphy could predict, one of my other staff was offered a great job at another company, so tendered her resignation Friday morning. Thankfully it is October, and our business cycle is slowing down at the moment!

All in all, the Controllership is a huge opportunity for me, and is already involving a lot of learning, which I am very happy about. I’ve been with the company for over 8 years now, and am looking forward to being part of the team that really pushes the company forward to some solid success. What this means to my posting abilities remains to be seen, of course. I am working longer hours than I used to, so that cuts in a bit already. My community contributions are an important part of my life though, and one that I miss greatly. I do have intentions of getting back to form in the next few weeks, as soon as the book is finally put to bed for good.

One funny thing about the promotion is that I celebrated by teaching a Beginning Excel course to a forestry company in the area the very next day. (Okay, it had been planned for a while, and some of you contributed to the ideas for it.) The course went very well, and I really enjoyed doing it.

Now, just because that isn’t enough to talk about, this article was published about me in the August/September issue of CMA Magazine. The magazine is the bi-monthly magazine that goes out to about 44,000 CMA’s and students in Canada. The interview was back in July, which is why it bears my old title at work, but still pretty darn cool. :)

And finally, I should also mention that I was re-awarded my MVP award for the 2007-2008 year. I’m very excited about that, as I really felt shy on public contributions from the summit forward. So a big thanks to Microsoft for that.

It’s been a busy fall for me so far… ;)