Or… why it’s important to disable Host Time Synchronisation on a domain controller.
A few months ago I reminded myself of a major gotcha when planning a virtual infrastructure. Assume that you run more than one domain in more than one forest and that trusts are in place to authenticate users across those forests. This could be a development/test/staging environment, or as will no doubt be more common in the coming years, it could be a virtualised infrastructure.
Assume that Kerberos needs to operate across these domains for any number of purposes, from application authentication to Active Directory migration. If you want Kerberos to work, you’re going to need to synchronise time across these domains, as the clock synchronisation is used to time stamp tickets in order to prevent replay attacks. Time synchronisation is of course built in to the Windows domain infrastructure, and should support this nicely. Configuring the Windows Time service on a PDC emulator is a bit fiddly, but should be achievable for anyone who runs a multiple domain infrastructure. But what if it goes wrong?
I spent a few hours bashing my head against this utterly confounding problem until the obvious whacked me in the face. The problem:
- I would run w32tm /resync /rediscover and all would synchronise successfully, then about five seconds later the clock would revert to its former time, about 2 minutes and 45 seconds ahead of the recently received absolute value
- There were no error logs, just successful resynchronisation messages and then nothing to indicate why the clock reverted
- I set the registry keys to enable debug logging and this revealed nothing more than that the time of the events was shifting as you would expect
Then it hit me. The PDC emulator was a virtual machine and the host virtual server was in another domain/forest. That host server was failing time synchronisation with its DC, so I manually reset the host clock and voila! The PDC emulator was synchronised within seconds. Obviously there’s another task to find out why the host was failing synchronisation with its time server, but that’s totally beside the point.
The lesson: Obey the best practice guidance and turn off Host Time Synchronisation for virtual domain controllers. In Virtual Server 2005 R2 the setting is in the Virtual Machine Addition Properties.
And a follow-up note: once Host Time Synchronisation is disabled you will need to find a new source of reliable clock for the the VMs, as they don’t have a CMOS clock to rely on. You could use something as simple and free as Atomic Clock Sync. If you fail to do this the VMs will lose time when they are shut down.