SharePoint Development Productivity and Virtualisation Technologies

In the near future, I’ll be discussing the results of the SharePoint Development productivity testing that I’ve been working on for some time. A key part of the background to that story is a choice to virtualise SharePoint, and within that, a choice of virtualisation technology. In this post I’ll be reviewing the problem in advance of a more detailed discussion of the productivity gains and losses with some of these technologies/approaches.

For clarity, I will quickly state the problem as I see it. SharePoint 2010 system requirements and practitioner mobility requirements are inherently at odds. What guidance exists for this unique problem space tends to regurgitate preferences/allegiances rather than comparing technologies and ratifying assumptions with real-world tests. At best, you get system performance indices for a single laptop model, but these results may vary when any hardware component is changed.

How can virtualisation improve system performance?

It doesn’t. People look to virtualisation to solve other problems. However, SharePoint 2010 performs differently in different virtualisation technologies, and the margins of these differences vary by hardware configuration. By all means, the advantages of virtualisation often make it a desirable choice, but these performance characteristics need to be accounted for, lest system performance losses negate the productivity improvements that virtualisation can introduce.

Why virtualise?

There are a number of advantages to virtual systems over physical systems. Many of these benefits can also be obtained with sufficiently mature systems management technologies and physical systems, but these benefits are often easier, quicker or less costly to implement through virtualisation. Some of the benefits include:

• Provisioning times for new SharePoint environments.
• Standardisation through cloned, network-isolated virtual machines.
• Account for volatility with snapshots.
• Standard builds per-project, to share with team members, reducing project initiation costs.
• Virtual appliances produced by Microsoft and third parties, such as the Information Worker Demo VM.
• Reduced hardware rebuilds by removing development tools and SharePoint from the host.

This list is by no means comprehensive. As I say, many of these benefits can be realised with scripting and/or management tools. This list is only intended to illustrate why it’s a powerful design option.

An overview of virtualisation and related technologies

Some example technologies by type:

• Type I Hypervisors
  • VMWare ESXi
  • Hyper-V
• Type II Hypervisors
  • Oracle VirtualBox
  • VMWare Workstation
• Infrastructure as a Service (IaaS)
  • Amazon EC2
  • Azure VM Role (forthcoming)
• Local Systems
  • Native Boot Windows 7 (virtual hard disk)
  • Citrix XenDesktop (VDI)

Note: Virtual PC was not included because it doesn’t support 64-bit guest operating systems. SharePoint 2010 only runs on 64-bit systems.

Some of the alleged benefits of these approaches:

• Type I Hypervisors
  • Better performance**
  • Good management options/tools
• Type II Hypervisors
  • Host Operating System
  • Easy to use
• Infrastructure as a Service (IaaS)
  • Pay as you go
  • Scalability
• Local Systems
  • Good performance
  • Simple to use

Some of the alleged drawbacks of these approaches:

• Type I Hypervisors
  • No Host Operating System***
  • Driver issues*
  • Complicated
• Type II Hypervisors
  • Historically poor performance**
  • Historically, less manageable (snapshots, import/export, etc)
• Infrastructure as a Service (IaaS)
  • Requires stable connectivity
  • Complicated
  • Pay-As-You-Go requires diligence
• Local Systems
  • Easy to damage
  • Slow to rebuild

*Hyper-V has driver issues on some newer laptops. These are most noticeable with graphics, although I have seen audio driver problems as well. Some of these driver issues may be fixed or alleviated in the SP1 Beta/RC for Windows Server 2008 R2.

**This performance bias is one of the things I will be examining in more detail in later posts.

***This is only “sort of” true for Hyper-V, which invokes a “parent partition”. This is a special type of virtual machine that fulfils a similar role to a host operating system, and is often referred to as such.

Why are “Local Systems” included?

I’ve lumped these in for two reasons. 1) They share some characteristics with the other virtualisation technologies, like running from virtual hard drives. 2) By virtue of being local systems, they fundamentally negate some of the benefits that are obtained through virtualisation. Cloning these machines is not an option if SharePoint is installed and configured. It will be necessary to invest in scripting environment provision in order to retain those productivity benefits. It happens that many people choose to take this scripting approach, but it’s worth pointing out that network isolation and cloning can achieve similar results through virtualisation, and this does not obtain with Local Systems.

What about shared, hosted development environments?

In this scenario I’m thinking of hosted development farms, where some or all members of a team use a single environment. Based on my subjective reading of the community, this option seems to be fading away. I think there are three reasons why.

1. Cost. Running development environments on proper infrastructure is expensive. Most components have been made redundant, the storage will be expensive if it performs well, the power/cooling costs are considerably more expensive than for laptops/desktops and you will need to pay people to manage the systems. Even when these costs are split across multiple developers, it’s still expensive unless resources are overcommitted, which negates productivity gains. It also tends to be more expensive to provision new environments and this process can often be an obstacle to business agility. In a nutshell, these are protections that are unnecessary for development environments. Redundancy and resilience at this level is overkill given the associated costs. The most important assets, such as code, standard images and project-specific builds can be protected separately.
2. Hive pollution. If these farms will support multiple projects, as they often do per the previous comments about provisioning, then these systems will inherently differ from the test/stage/UAT/production systems they should resemble. Core files in the hive can be altered from project-to-project, resulting in unexpected behaviour when moving code between these environments. This can seriously complicate troubleshooting and should be avoided.
3. Mobility. These farms aren’t terribly useful to people who are travelling or who are working on-site with restricted outbound connectivity.

All of this said, there are times when project-specific requirements may make shared farms a good option. It may be sensible to take another look for:

• Integration projects.
• Developing with large amounts of data.
• Projects with heavy infrastructure requirements, such as FAST.
  • Perhaps individual development environments can consume a shared FAST Service Application?

Generally speaking, I believe these resources should be provided only in these niche cases.

How is this different from IaaS?

The main differences are costs and capital. Cloud-based infrastructure services are fundamentally just virtualised hosting on an enormous scale. This scale lowers costs to a point where it may be affordable to deploy individual machines per-developer. Although in my analyses I found that IaaS would be more expensive than desktop workstations over three years, this still may be compelling when cash flow issues preclude significant one-time investment or credit flows are restricted. IaaS should also be kept in mind when specific projects require significant provisioning or investment for a short term, for instance testing in a large farm.

While providing a single cloud-based VM per-user solves the first two issues with shared development environments, mobility is still an issue. In many places, stable mobile broadband is flaky at best. Additionally, there are key architectural differences that need to be accounted for when working in the cloud, and on a Pay-As-You-Go basis. I address all of this in my series on SharePoint 2010 Infrastructure for Amazon EC2.

Which approach is best?

This is a high-level overview of the design constraints that limited my choices, before I plunged into a concrete performance review of the remaining technologies.

Local Systems
In my view, Local Systems are only a better choice if the supporting IT systems and processes are very mature and the performance benefits are clear and significant. For most development scenarios, that has yet to be proven. I’ve postponed this virtual to physical performance comparison for now, as the other benefits of virtualisation have ruled this approach out for us, but I hope to revisit it in the new year.

IaaS
IaaS has two key planning considerations. The first is fairly obvious. Outbound RDP Connectivity needs to be open whenever the systems are needed. I encourage people to consider this in some detail and pilot with many types of users before diving in. The second consideration is Pay-As-You-Go. While cloud providers often have an always-on option, it’s usually pretty pricey. The alternative is to find a mechanism to limit compute usage to when it is truly being used, without introducing usability problems. Management tools or scripting may be able to answer these problems, but no one should enter in to this process thinking it will be easy. This is not an easy option. For a more detailed consideration of these issues, refer to my series on EC2.

Type II Hypervisors
VMWare Workstation is the most mature desktop virtualisation product on the market, although in recent years VirtualBox has been gaining share. Choosing between these technologies for my tests was never going to be easy, but I reduced it to a few factors:

• I’ve never met a VirtualBox user that would complain about using VMWare but I can’t say that proposition is reversible. There are a lot of SharePoint practitioners with a strong preference for VMWare.
• VMWare Workstation has native interoperability with other VMWare assets. While VirtualBox supports the VMDK file format, it’s not quite the same thing.
• Both products are fairly inexpensive in the grand scheme of things.
• I had stability issues with VirtualBox circa version 3.14 that left a bad taste in my mouth.

Perhaps most importantly, I felt that the performance comparison of VMWare Workstation to Hyper-V would be the most valuable decision-making information.

Type I Hypervisors
Most Type I Hypervisors would not be suitable for desktop virtualisation because they don’t have a host operating system. While it would be possible to boot a guest OS and remote in to other Virtual Machines over internal networks, this is a complicated approach and the networking requirements would be enough to put off most developers. However, as mentioned above, Hyper-V is a notable pseudo-exception to this with its parent partition.

We’ve been using the Hyper-V role in Windows Server 2008 R2 for development for a little over a year now. While we have successfully capitalised on many of the productivity benefits of virtualisation through this approach, there are a few issues that have never been entirely satisfactory:

• Despite having the host OS, using Hyper-V is still complicated for non-Systems people – particularly the networking.
  • Work-around solutions for Wireless networking are fiddly.
  • Lack of self-contained NAT requires the use of Internet Connection Sharing in order to achieve network isolation, which some users struggle with.
• Lack of Sleep/Hibernate is painful for many users.
• Graphics performance is poor – particularly with large PowerPoint/Visio files, large images and video.
• Audio can also suffer during large file operations.
• Hyper-V is not ready for laptop power schemes.

Despite these niggles, we’ve continued to use Hyper-V while waiting for the forthcoming graphics/memory improvements in Windows Server 2008 R2 SP1. I would class these usability problems as significant inconveniences that sometimes manifest themselves in lost productivity – particularly with new users learning our approach.

New Problems in SharePoint 2010

Since we properly immersed ourselves in SharePoint 2010 development, negative reports about performance started to roll in. These proved hard to validate until a few months ago when my colleagues showed me first page load times after an IISRESET in excess of one minute. This was concrete and repeatable. The problem was more severe on some systems than others, but it was clearly a problem.

The performance tests I’ve been conducting have been an effort to pick apart these results in Hyper-V. Was this new in SharePoint 2010 or did it amplify something that was minor before? Do we get the same problems on different virtualisation technologies, in the cloud or is this a symptom of virtualisation itself? In my next post I’ll discuss the environments, the tests and the testing process.