A look inside Vista security

Microsoft Vista promises to be the most secure operating system that the company has ever shipped. It's the first one to be developed totally under the firm's Secure Development Lifecycle (the methodology it adopted after Bill froze development in 2002 and forced the company to teach its programmers how to code properly).

Vista was gutted of many of the features promised in the first public demo, back in October 2003, but some good security measures remain. Some of the Trusted Computing Platform Alliance's Trusted Computing Module (TPM), originally to be supported by Vista's Next Generation Secure Computing Base (NGSCB), still survives in the form of BitLocker, Vista's AES-based hard drive encryption system.

BitLocker, which can be backed up by a password held on a USB key for two-factor authentication, can also check the integrity of system files at start up. "We're well past the time when encryption of laptops should be routine," says Gartner analyst Jay Heiser. "So I applaud anything that Microsoft can do in that area."

Other enhancements include the elimination of the mandatory Graphical Identification and Authentication (GINA) logon system. Instead, it will be easier for companies to write their own logon environments for the operating system, making it simpler to integrate smart cards for two factor authentication, for example.

Group Policy has been improved with new policy settings, better awareness of where clients are in relation to the network when trying to enforce group policy, and a redesign of the ADM template system used to store group policy settings.

Look out for address space layout randomisation (ASLR), which will randomise executable code between 256 possible locations, making it harder for malicious code to hijack the executable code and adapt it. And heap buffer overflow protection will kill applications that try to stuff the buffer with data to make it leak into executable memory.

Claming down in admin mode

All of that looks good from here, but some of Microsoft's other security measures have been criticised. Take administrative account privileges. The company has tried to stop people running the operating system in admin mode for years, because this mode gives all processes privileged access to system resources, making it easier for malware to do damage. Users have largely ignored these pleas, not least because running in standard mode is inconvenient. Many developers tend to write their applications while running in admin mode, which makes it easy to overlook restrictions that their software will experience when running in standard mode. For example, trying to digitise film into Adobe Premiere across a FireWire link works fine in admin mode, but is not allowed in standard mode.

Microsoft has tried to circumnavigate the problem with User Access Control, (UAC), a system that lets standard mode accounts carry out administrative tasks by entering their credentials. It also introduces an approved mode for administrator accounts that requires administrators to grant consent before executing high-privilege actions.

Theoretically, this is little more complicated than Apple's current protection in OS X, which requires users to enter a password when performing certain actions. Nevertheless, it has infuriated some industry commentators. UAC could have significant implications for the Windows interface, warns Ken Munro, managing director of penetration testing company SecureTest. "One of the problems they'll have is that people using the Windows UI will get click-happy, as they did with Windows XP SP2, and start enabling things that they shouldn't be enabling, and not really understanding what they're clicking and accepting."

The same could go for the anti-phishing technology built into Internet Explorer 7. The technology, which checks web sites against a centrally maintained Microsoft registry of known scam sites, uses a colour-coded bar to warn users that they may be visiting a suspect online destination. With users displaying a worrying tendency to click through such warnings, it is unclear how effective such technologies can be -- but it is equally unclear what more Microsoft could do to help solve the problem.

At least it is tightening up browser security. It is calling the browser IE7+ when running on Vista, explains Microsoft technical security evangelist Steve Lamb, because it is running in a protected mode that is even more restricted than the standard user account. "If anything malicious was to happen in IE that would compromise the IE process, then because the process didn't have the privilege to do much on the system at all, nor would the malicious software."

Microsoft has also taken steps to protect the operating system kernel. The kernel has always been a crucial attack point for root kits, which are malicious programs designed to operate at a low level of the operating system, co-opting it and providing backdoor access to intruders. Because the kernel controls the OS, malware operating at this level can cope itself from detection by hiding itself from the operating system, for example. Patchguard is a technology that stops software from injecting unauthorised code into the kernel.

Kernel patch protection was actually implemented in the 64-bit versions of Windows XP and 2003 Server SP1, but it becomes more of an issue in Vista because of the operating system's built-in 64-bit support, and the growing uibiquity of 64-bit client hardware. Patchguard doesn't prevent kernel patching in 32-bit systems, and Microsoft documents suggest that in the Intel world, the technology only extends to Intel Extended Memory 64 systems - that is, 64-bit extensions to the existing IA32 architecture. According to Microsoft kernel security architects, the ia64 processor won't detect patching any part of the kernel. Only AMD64-based systems will do that.

But even when Patchguard does work, it may not be totally secure. At the Blackhat Security conference in July, researchers demonstrated a way to theoretically inject executable code into the kernel by increasing memory allocation to existing processes and forcing the system to page drivers to disk. Text in the paged files could then be updated with new code which would be executed when the driver files were loaded back into memory. This would enable hackers to inject Blue Pill, a hack for the hypervisor system that they claimed would be undetectable. However, it was only possible to implement Blue Pill when running in administrator mode.

"Microsoft wants to get everyone out of the kernel," says Vincent Weafer, director of development for security response at Symantec, who worries that blocking kernel access to third party security products makes it difficult for them to do their job. They need to monitor kernel-level functions to make systems more secure. "This is the core of our technology. We can protect users that way and have been doing so successfully for many years." He also argues that Symantec's kernel-level security technology is likely to be more secure than Microsoft's, which has not been concentrating on this area for as long. Rather than lock out everyone (good and bad), why not implement a trust system, he suggests? Microsoft engineers answer that there is no easy way to sort good companies from bad (but isn't that what application signing using strong certificates is for?)

Rivals taking a closer look

Symantec has also been busy dismantling the Vista network stack, which has been largely rewritten from the bottom up with more security in mind. After all, the original TCP/IP stack was bundled with the operating system just as Microsoft decided that there might just be some value in embracing the Web, but when it was still tinkering with an OS written without public networking in mind.

"Older network stacks have had a chance to be thrashed around and broken by people because they've been around for a period of time," says Munro. A Symantec paper on the new stack reveals that it was vulnerable to the LAND attack, an old exploit that hasn't been in the system since Windows 95." Other attacks also surfaced, but like LAND they were fixed in subsequent Vista builds. However, Symantec believes that rewriting the stack at this level is bound to have introduced other vulnerabilities. How many of them will be thrashed out before the final ship date remains to be seen. "Further research should be fruitful," says its paper on the subject.

Mehmood Khan, a Vista expert at Microsoft-focused IT consultancy EuroData, agrees: "If I was an IT manager doing a corporate rollout, and giving it to a number of users, I'd turn off three things straight away: the UAC, the Aero theme, and as much security as I can. Defender will come straight out, because that'll be controlled under group policy," he says. "My understanding is that these are the three main things that they're pushing."

The other thing that Khan is concerned about is the firewall. Rewritten, it protects both inbound and outbound traffic. He likes it, but worries that it will prove daunting for IT departments, who will find it difficult to open up ports for different applications. "Microsoft has really developed a hard firewall. It's very good. But it's just customising it so that the customer knows what ports should be open," he says, lamenting the lack of diagnostic information in the operating system to support these decisions. "You look at the log file and it's basically gibberish." The danger is that IT departments might turn off the firewall altogether and rely on group policy, he says, which if misconfigured could render the machine vulnerable when it's off the network.

Vista is a marked improvement over XP SP2, which itself was a vast departure from previous versions of XP. Microsoft has to be applauded for trying to fix its tarnished security image with the latest system, but we won't really know how well the burnishing has gone until the crackers get a chance to play with its off-the-shelf final release.

Danny Bradbury

Danny Bradbury has been a print journalist specialising in technology since 1989 and a freelance writer since 1994. He has written for national publications on both sides of the Atlantic and has won awards for his investigative cybersecurity journalism work and his arts and culture writing. 

Danny writes about many different technology issues for audiences ranging from consumers through to software developers and CIOs. He also ghostwrites articles for many C-suite business executives in the technology sector and has worked as a presenter for multiple webinars and podcasts.