Microsoft Windows Internals Manual

PUBLISHED BY
Microsoft Press
A Division of Microsoft Corporation
One Microsoft Way
Redmond, Washington 98052-6399
Copyright © 2005 by David Solomon, Mark Russinovich
All rights reserved. No part of the contents of this book may be reproduced or transmitted in any form or by
any means without the written permission of the publisher.
Library of Congress Control Number 2005921847
Printed and bound in the United States of America.
1 2 3 4 5 6 7 8 9 QWT 9 8 7 6 5 4
Distributed in Canada by H.B. Fenn and Company Ltd.
A CIP catalogue record for this book is available from the British Library.
Microsoft Press books are available through booksellers and distributors worldwide. For further information
about international editions, contact your local Microsoft Corporation office or contact Microsoft Press International directly at fax (425) 936-7329. Visit our Web site at www.microsoft.com/learning/. Send comments
to [email protected].
Microsoft, Active Desktop, Active Directory, ActiveX, DirectX, Microsoft Press, MSDN, MS-DOS, Outlook,
PowerPoint, Visual Basic, Visual C++, Visual Studio, Win32, Windows, Windows NT, Windows Server, and
WinFX are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or
other countries. Other product and company names mentioned herein may be the trademarks of their respective owners.
The example companies, organizations, products, domain names, e-mail addresses, logos, people, places, and
events depicted herein are fictitious. No association with any real company, organization, product, domain
name, e-mail address, logo, person, place, or event is intended or should be inferred.
This book expresses the author’s views and opinions. The information contained in this book is provided without any express, statutory, or implied warranties. Neither the authors, Microsoft Corporation, nor its resellers,
or distributors will be held liable for any damages caused or alleged to be caused either directly or indirectly
by this book.
Acquisitions Editors: Robin Van Steenburgh, Ben Ryan
Project Editor: Valerie Woolley
Development Editor: Sally Stickney
Copy Editor: Roger LeBlanc
Indexer: Lynn Armstrong
SubAssy Part No. X11-16607
Body Part No. X11-16608
ISBN: 0-7356-1917-4
Copyrighted material.
To Dave Cutler, father of the Windows kernel
Copyrighted material.
Copyrighted material.
Contents at a Glance
1
2
3
4
5
6
7
8
9
10
11
12
13
14
Concepts and Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
System Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
System Mechanisms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
Management Mechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
Startup and Shutdown . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
Processes, Threads, and Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
Memory Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485
I/O System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537
Storage Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 615
Cache Manager. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 655
File Systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 689
Networking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 787
Crash Dump Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 845
Copyrighted material.
Copyrighted material.
Table of Contents
Historical Perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix
Foreword. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii
Acknowledgments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xxv
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xxvii
1
Concepts and Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1
Windows Operating System Versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Foundation Concepts and Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Windows API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Services, Functions, and Routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Processes, Threads, and Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Virtual Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Kernel Mode vs. User Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Terminal Services and Multiple Sessions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Objects and Handles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Registry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Unicode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Digging into Windows Internals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Performance Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Windows Support Tools. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Windows Resource Kits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Kernel Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Platform Software Development Kit (SDK). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Device Driver Kit (DDK) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Sysinternals Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2
System Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Requirements and Design Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Operating System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
What do you think of this book?
We want to hear from you!
Copyrighted material.
Microsoft is interested in hearing your feedback about this publication so we can
continually improve our books and learning resources for you. To participate in a brief
online survey, please visit: www.microsoft.com/learning/booksurvey/
viii
Table of Contents
Architecture Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Portability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Symmetric Multiprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
Differences Between Client and Server Versions . . . . . . . . . . . . . . . . . . . . . . . . 47
Checked Build . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Key System Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Environment Subsystems and Subsystem DLLs . . . . . . . . . . . . . . . . . . . . . . . . . 53
Ntdll.dll . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
Executive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
Kernel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
Hardware Abstraction Layer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
Device Drivers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
System Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
3
System Mechanisms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
Trap Dispatching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
Interrupt Dispatching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
Exception Dispatching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
System Service Dispatching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
Object Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
Executive Objects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
Object Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
High-IRQL Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
Low-IRQL Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
System Worker Threads. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
Windows Global Flags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
Local Procedure Calls (LPCs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
Kernel Event Tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
Wow64. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
Wow64 Process Address Space Layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
System Calls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
Exception Dispatching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
User Callbacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
File System Redirection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
Copyrighted material.
Table of Contents
ix
Registry Redirection and Reflection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
I/O Control Requests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
16-Bit Installer Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
Printing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
Restrictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
4
Management Mechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
The Registry. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
Viewing and Changing the Registry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
Registry Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
Registry Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
Registry Logical Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
Troubleshooting Registry Problems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
Registry Internals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
Service Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
Service Accounts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
The Service Control Manager. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
Service Startup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
Startup Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
Accepting the Boot and Last Known Good . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
Service Failures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
Service Shutdown. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
Shared Service Processes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
Service Control Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
Windows Management Instrumentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
WMI Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
Providers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
The Common Information Model and
the Managed Object Format Language . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
The WMI Namespace. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
Class Association. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
WMI Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
WMI Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
Copyrighted material.
x
Table of Contents
5
Startup and Shutdown . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
Boot Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
x86 and x64 Preboot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
The x86/x64 Boot Sector and Ntldr . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
The IA64 Boot Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
Initializing the Kernel and Executive Subsystems . . . . . . . . . . . . . . . . . . . . . . . 266
Smss, Csrss, and Winlogon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
Images that Start Automatically . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
Troubleshooting Boot and Startup Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
Last Known Good . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
Safe Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
Recovery Console . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
Solving Common Boot Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
Shutdown . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
6
Processes, Threads, and Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
Process Internals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
Data Structures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
Kernel Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
Performance Counters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
Relevant Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298
Flow of CreateProcess . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
Stage 1: Opening the Image to Be Executed. . . . . . . . . . . . . . . . . . . . . . . . . . . 302
Stage 2: Creating the Windows Executive Process Object . . . . . . . . . . . . . . . 304
Stage 3: Creating the Initial Thread and Its Stack and Context . . . . . . . . . . . 308
Stage 4 : Notifying the Windows Subsystem about the New Process. . . . . . 309
Stage 5: Starting Execution of the Initial Thread. . . . . . . . . . . . . . . . . . . . . . . . 310
Stage 6: Performing Process Initialization in the Context
of the New Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310
Thread Internals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313
Data Structures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313
Kernel Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320
Performance Counters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
Relevant Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322
Birth of a Thread . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322
Examining Thread Activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
Copyrighted material.
Table of Contents
xi
Thread Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
Overview of Windows Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326
Priority Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327
Windows Scheduling APIs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330
Relevant Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331
Real-Time Priorities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
Thread States. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334
Dispatcher Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338
Quantum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340
Scheduling Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345
Context Switching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347
Idle Thread . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348
Priority Boosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348
Multiprocessor Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357
Multiprocessor Thread-Scheduling Algorithms. . . . . . . . . . . . . . . . . . . . . . . . . 366
Job Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
7
Memory Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
Introduction to the Memory Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
Memory Manager Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376
Internal Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377
Configuring the Memory Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378
Examining Memory Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378
Services the Memory Manager Provides . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382
Large and Small Pages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382
Reserving and Committing Pages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
Locking Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385
Allocation Granularity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385
Shared Memory and Mapped Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386
Protecting Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388
No Execute Page Protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390
Copy-on-Write . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392
Heap Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394
Address Windowing Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399
System Memory Pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
Configuring Pool Sizes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
Monitoring Pool Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404
Copyrighted material.
xii
Table of Contents
Look-Aside Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408
Driver Verifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409
Virtual Address Space Layouts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413
x86 User Address Space Layouts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
x86 System Address Space Layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
x86 Session Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418
System Page Table Entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421
64-Bit Address Space Layouts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422
Address Translation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425
x86 Virtual Address Translation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425
Translation Look-Aside Buffer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434
Physical Address Extension (PAE) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435
IA-64 Virtual Address Translation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437
x64 Virtual Address Translation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438
Page Fault Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439
Invalid PTEs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440
Prototype PTEs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
In-Paging I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443
Collided Page Faults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444
Page Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444
Virtual Address Descriptors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 448
Section Objects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450
Working Sets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457
Demand Paging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 458
Logical Prefetcher . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 458
Placement Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462
Working Set Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463
Balance Set Manager and Swapper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466
System Working Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467
Page Frame Number Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 469
Page List Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472
Modified Page Writer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475
PFN Data Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476
Low and High Memory Notification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483
Copyrighted material.
Table of Contents
8
xiii
Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485
Security System Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 488
Protecting Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492
Access Checks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493
Security Descriptors and Access Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 506
Account Rights and Privileges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516
Account Rights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517
Privileges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 518
Super Privileges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523
Security Auditing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524
Logon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 526
Winlogon Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 528
User Logon Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529
Software Restriction Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 533
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535
9
I/O System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537
I/O System Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537
The I/O Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 539
Typical I/O Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 540
Device Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 541
Types of Device Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 541
Structure of a Driver. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 548
Driver Objects and Device Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 550
Opening Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555
I/O Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 561
Types of I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 561
I/O Request Packets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 564
I/O Request to a Single-Layered Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 569
I/O Requests to Layered Drivers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 577
I/O Completion Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 585
Driver Verifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 589
The Plug and Play (PnP) Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 590
Level of Plug and Play Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 591
Driver Support for Plug and Play. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 592
Driver Loading, Initialization, and Installation . . . . . . . . . . . . . . . . . . . . . . . . . . 594
Driver Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 603
Copyrighted material.
xiv
Table of Contents
The Power Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 607
Power Manager Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 609
Driver Power Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 610
Driver Control of Device Power . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 613
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 613
10
Storage Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 615
Storage Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 615
Disk Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 616
Ntldr. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 616
Disk Class, Port, and Miniport Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 617
Disk Device Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 620
Partition Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 622
Volume Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 622
Basic Disks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 624
Dynamic Disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 626
Multipartition Volume Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 632
The Volume Namespace. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 638
Volume I/O Operations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 646
Virtual Disk Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 648
Volume Shadow Copy Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 649
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 654
11
Cache Manager. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 655
Key Features of the Cache Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 655
Single, Centralized System Cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 656
The Memory Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 656
Cache Coherency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 656
Virtual Block Caching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 658
Stream-Based Caching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 658
Recoverable File System Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 658
Cache Virtual Memory Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 660
Cache Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 662
LargeSystemCache. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 662
Cache Virtual Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 663
Cache Working Set Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 665
Cache Physical Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 667
Copyrighted material.
Table of Contents
xv
Cache Data Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 668
Systemwide Cache Data Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 669
Per-File Cache Data Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 670
File System Interfaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 674
Copying to and from the Cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 676
Caching with the Mapping and Pinning Interfaces . . . . . . . . . . . . . . . . . . . . . 677
Caching with the Direct Memory Access Interfaces . . . . . . . . . . . . . . . . . . . . . 678
Fast I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 679
Read Ahead and Write Behind . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 682
Intelligent Read-Ahead . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 682
Write-Back Caching and Lazy Writing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 683
Write Throttling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 686
System Threads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 687
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 688
12
File Systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 689
Windows File System Formats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 690
CDFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 690
UDF. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 691
FAT12, FAT16, and FAT32. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 691
NTFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 694
File System Driver Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 694
Local FSDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 695
Remote FSDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 696
File System Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 700
File System Filter Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 705
Troubleshooting File System Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 711
Filemon Basic vs. Advanced Modes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 711
Filemon Troubleshooting Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 712
NTFS Design Goals and Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 717
High-End File System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 717
Advanced Features of NTFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 719
NTFS File System Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 729
NTFS On-Disk Structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 732
Volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 732
Clusters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 732
Master File Table. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 733
Copyrighted material.
xvi
Table of Contents
File Reference Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 739
File Records. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 740
Filenames . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 742
Resident and Nonresident Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 744
Data Compression and Sparse Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 747
The Change Journal File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 752
Indexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 753
Object IDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 754
Quota Tracking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 755
Consolidated Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 756
Reparse Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 758
NTFS Recovery Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 758
Evolution of File System Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 759
Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 761
Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 767
NTFS Bad-Cluster Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 771
Encrypting File System Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 775
Encrypting a File for the First Time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 778
The Decryption Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 783
Backing Up Encrypted Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 784
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 785
13
Networking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 787
Windows Networking Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 787
The OSI Reference Model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 787
Windows Networking Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 789
Networking APIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 791
Windows Sockets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 791
Remote Procedure Call . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 798
Web Access APIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 803
Named Pipes and Mailslots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 804
NetBIOS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 811
Other Networking APIs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 813
Multiple Redirector Support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 815
Multiple Provider Router . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 816
Multiple UNC Provider . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 818
Copyrighted material.
Table of Contents
xvii
Name Resolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 820
Domain Name System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 820
Windows Internet Name Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 820
Protocol Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 821
TCP/IP Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 824
NDIS Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 828
Variations on the NDIS Miniport . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 832
Connection-Oriented NDIS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 832
Remote NDIS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 835
QOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 836
Binding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 838
Layered Network Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 839
Remote Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 839
Active Directory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 840
Network Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 841
File Replication Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 843
Distributed File System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 843
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 844
14
Crash Dump Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 845
Why Does Windows Crash? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 845
The Blue Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 846
Crash Dump Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 849
Crash Dump Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 852
Windows Error Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 853
Online Crash Analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 854
Basic Crash Dump Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 855
Notmyfault . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 855
Basic Crash Dump Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 856
Verbose Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 858
Using Crash Troubleshooting Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 860
Buffer Overrun and Special Pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 861
Code Overwrite and System Code Write Protection . . . . . . . . . . . . . . . . . . . . 863
Advanced Crash Dump Analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 864
Stack Trashes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 865
Hung or Unresponsive Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 866
When There Is No Crash Dump . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 869
Copyrighted material.
xviii
Table of Contents
Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 871
Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 895
What do you think of this book?
We want to hear from you!
Copyrighted material.
Microsoft is interested in hearing your feedback about this publication so we can
continually improve our books and learning resources for you. To participate in a brief
online survey, please visit: www.microsoft.com/learning/booksurvey/
Historical Perspective
Again I find myself indebted to David Solomon and Mark Russinovich for providing the
opportunity to write a few words about their newest edition to a series of books on Windows
Internals. It has been over three years since the last publication in this series, and this passage
of time has seen two major releases: a very significant update to the client system, and another
very significant update to the server system, currently being readied for shipment.
Two of the growing problems faced by the authors of a book such as this are tracing the implementation evolution of the Microsoft Windows NT system and documenting the way in
which feature implementation has changed in each version. To this end, the authors have
done a remarkable job of providing examples and explanations throughout the book.
(Left to right) David Solomon, David Cutler, and Mark Russinovich
I first met David Solomon when I was working at Digital Equipment Corporation on the VMS
operating system for VAX, and he was only 16. Since that time, he has been involved with
operating system development and teaching operating system internals. I met Mark Russinovich more recently but have been aware of his expertise in the area of operating systems for
some time. He has done some amazing work, such as his NTFS file system running on
Microsoft Windows 98 and his “live” Windows kernel debugger, which can be used to peer
into the Windows system while it is running.
The beginnings of Windows NT started in October 1988 with a set of goals to produce a portable system that addressed OS/2 compatibility, security, POSIX, multiprocessing, integrated
networking, and reliability. With the advent and huge success of Windows 3.0, the system
goals were soon changed to natively address Windows compatibility directly and move OS/2
compatibility to a subsystem.
Copyrighted material.
xix
xx
Historical Perspective
We originally thought we could produce the first Windows NT system in a little over two
years. It actually ended up taking us four and a half years for the first release in the summer of
1993, and that release supported the Intel i386, Intel i486, and the MIPS R4000 processors.
Six weeks later, we also introduced support for the Digital Alpha processors.
The first release of Windows NT was larger and slower than expected, so the next major push
was a project called Daytona, named after the speedway in Florida. The main goals for this
release were to reduce the size of the system, increase the speed of the system, and, of course,
to make it more reliable. Six months after the release of Windows NT 3.5 in the fall of 1994,
we released Windows NT 3.51, an updated version containing support for the IBM PowerPC
processor.
The goal for the next version of Windows NT was to update the user interface to be compatible with Windows 95 and to incorporate the Cairo technologies that had been under development at Microsoft for a couple of years. This system took two more years to develop and was
introduced in the summer of 1996 as Windows NT 4.0.
The following version of NT saw a name change to Windows 2000 and was the last system for
which the client and server systems were released at the same time. This version was built on
the same Windows NT technology as the previous versions and introduced significant new
features such as Active Directory. Windows 2000 took three and a half years to produce and
was the most tested and tuned version of Windows NT technology produced at the time. Windows 2000 was the culmination of over eleven years of development spanning implementations on four architectures.
At the end of Windows 2000 development, we embarked on an ambitious plan to implement
new versions of the client and server systems, which would include new enhanced consumer
features and improved server capabilities. As plans developed, it became clear that implementation of the server features would cause a lag in the implementation of the client features, and
therefore, the releases were split. In August of 2001, Windows XP Professional and Windows
XP Home Edition were released, and a little over a year later, in March of 2003, Microsoft Windows Server 2003 was released. In addition to the Intel x86 architecture, these systems contained support for the Intel IA-64, marking Windows NT’s first move to 64-bit processing.
This book is the definitive work on the internal structures and workings of Windows XP and
Windows Server 2003. In addition, it offers a glimpse into the future of Windows’ move to
64-bit computing by covering AMD's introduction of the x64 architecture (AMD64) in 2003
and Intel's announced support (EM64T) in February 2004. A fully supported x64 client and
server release is planned in the first half of 2005, and this book contains many insights into
the implementation details of the x64 system.
The x64 architecture is the beginning of a new era for Windows NT at a time when the x86
architecture is beginning to show signs of old age. This architecture offers 32-bit x86 compatibility at speed to protect legacy software investments, and provides 64-bit addressing capability to address the most ambitious of new applications. This will protect 32-bit software
Copyrighted material.
Historical Perspective
xxi
investments while providing Windows NT with a breath of new life well into the next decade
and beyond.
Although the Windows NT system has undergone several name changes over the past several
years, it remains entirely based on the original Windows NT code base. As time has marched
on and invention has thrived, the implementation of many internal features has changed significantly. The authors have done a laudable job of assimilating the details of the Windows NT
code base and its differing implementations from release to release and platform to platform,
and of producing examples and tools that help the reader understand how things work. Every
serious operating system developer should have a copy of this book on his or her desk.
David N. Cutler
Senior Distinguished Engineer
Microsoft Corporation
Copyrighted material.
Copyrighted material.
Foreword
Microsoft Windows has been a core
part of my life for 14 years. During
this time, over release after release,
the operating system has evolved in
breadth and depth. Producing
Windows is one of the most important and complex projects in the
world today. Easily 5,000 engineers
work on Windows releases. Across
virtually all cultures, Windows
users comprise an entire spectrum
from the largest mission-critical
business to theyoungest child.
Customers using Windows
demand constant improvements in virtually every aspect—from being able to run the largest
servers to being easy enough for a pre-schooler to use. Windows comes in many shapes and
sizes, from embedded editions to media center editions to data center editions. All these products use the same core Windows internals, which evolve and improve with each release.
This is the definitive book on the core Windows internals. If you want to learn how Windows
works internally, in the fastest way possible, then this is the book for you. Understanding all
the pieces of such a large product is a daunting task. But if you start at the core concepts of the
system and work out, the puzzle fits together a lot more easily. Just as Windows itself has
evolved, so has the comprehensive nature of this book, now in its fourth edition. For years, we
have used earlier editions of this material to train brand-new employees at Microsoft, so this
material is tried and true.
If you’re like me, you like to figure out how things really work. Reading “how to use” books or
“tips and tricks” has never been sufficient for me. If you understand how something works
internally, you know how to better use it, maximize performance and security, diagnose failures, and frankly, have more fun. If you’re like me and want to see Windows from the “inside
out,” then you’re starting in the right place.
David and Mark have done an outstanding job detailing the “inside” Windows technical story.
The tools that they highlight are a great resource for direct hands-on training and diagnostics
work. After you read this book, you'll have a much greater understanding of how the operating system fits together, the latest improvements made throughout the system, and how to get
the most from these improvements.
Copyrighted material.
xxiii
xxiv
Foreword
It has been quite a journey—one that is still underway. So, start reading and dive deep into one
of the most impressive operating systems ever created.
Jim Allchin
Group Vice President, Platforms
Microsoft Corporation
Copyrighted material.
Acknowledgments
First, special thanks to the following people:
■
Dave Cutler, Senior Distinguished Engineer and the original architect of Microsoft
Windows NT. Dave originally approved David Solomon's source code access and has
been supportive of his work to explain the internals of Windows NT through his training business as well as during the writing of Inside Windows NT, Second Edition, and
Inside Microsoft Windows 2000, Third Edition. Besides reviewing the chapter on processes and threads, Dave answered many questions on the kernel architecture of the
system and wrote a historical perspective for this edition.
■
Jim Allchin, our executive sponsor, for writing the Foreword to this book and championing our cause within Microsoft.
■
Rob Short, vice president, who made sure we had the resources we needed, as well as
access to the relevant people.
We also thank two developers in the Windows division for writing new content that was
incorporated into this edition:
■
Adrian Marinescu, who wrote the greatly expanded heap manager section in the memory management chapter.
■
Samer Arafeh, who wrote the description of Wow64.
Thanks to our friend Jeffrey Richter, for writing the “What about .NET and WinFX” sidebar in
Chapter 1 and for continuing to remind us over many dinners together of his view on how few
people should care about what we talk about in this book.
This book wouldn’t contain the depth of technical detail or the level of accuracy it has without the
review, input, and support of key members of the Microsoft Windows development team. Therefore, we want to thank the following people, who provided technical review and input to the book:
Murali Brahmadesam
Pat Hoffer
Daniel Pravat
Molly Brown
Anthony Jones
Dragos Sambotin
Duncan Bryce
Tom Jones
Jon Schwartz
Daniel Bucherer
Joseph Joy
Rob Short
Neal Christian
Shreeniwas Kelkar
Paul Sliwowicz
Neill Clift
Connie La Chasse
Chittur Subbaraman
Mike Danseglio
Mike Lai
Cristian Teodorescu
Joseph Davies
Paul Leach
Andre Vachon
Cenk Ergan
Gerald Maffeo
Landy Wang
Copyrighted material.
xxv
xxvi
Acknowledgments
Tom Fout
Aaron Margosis
Richard Ward
Nar Ganapathy
Iain McDonald
Brad Waters
David Golds
Kamen Moutafov
Bruce Worthington
Robert Gu
Adi Oltean
Mark Zbikowsk
Jeff Hamblin
Vince Orgovan
Khawar Zuberi
Others might have contributed by answering questions in the hallway or cafeteria and providing technical material—if we missed you, please forgive us!
Thanks also to Jamie Hanrahan of Azius Developer Training (www.azius.com), who coauthored with David the original Windows Internal Architecture class on which the second edition was based. Jamie, who has a real knack for explaining complicated concepts in a simple
and practical fashion, developed several of the explanations, diagrams, and figures.
Thanks to Dave Probert for hosting the share where review drafts were distributed to internal
Microsoft reviewers.
And thanks to Jonathan Sloves of AMD for arranging AMD64 test systems to be sent to us to
help us develop the 64-bit content and port some of the Sysinternals tools to x64.
Finally, we want to thank the following people from Microsoft Press for their contribution to
this book:
■
Robin van Steenburgh, acquisitions editor, for patiently working with us to complete
this project.
■
Sally Stickney, who, for a time, continued as our project editor, but later got drawn into
the management vortex. We missed working with you this time!
■
Valerie Woolley, who took over as project editor from Sally. You were great (and not as
rough on us as Sally was for the last two editions)!
■
Roger LeBlanc, who laboriously went through all our chapters to tighten up text, find
inconsistencies, and in general bring the manuscript up to the high standards of
Microsoft Press.
David Solomon and Mark Russinovich
September, 2004
Copyrighted material.
Introduction
Microsoft Windows Internals, Fourth Edition is intended for advanced computer professionals
(both developers and system administrators) who want to understand how the core components of the Microsoft Windows 2000, Windows XP, and Microsoft Windows Server 2003
operating systems work internally. With this knowledge, developers can better comprehend
the rationale behind design choices when building applications specific to the Windows
platform. Such knowledge can also help developers debug complex problems. System
administrators can benefit from this information as well, because understanding how the
operating system works “under the covers” facilitates understanding the performance behavior of the system and makes it easier to troubleshoot system problems when things go
wrong. After reading this book, you should have a better understanding of how Windows
works and why it behaves as it does.
Structure of the Book
The first two chapters (“Concepts and Tools” and “System Architecture”) lay the foundation
with terms and concepts used throughout the rest of the book. The next three chapters—“System Mechanisms,” “Management Mechanisms,” and “Startup and Shutdown”—describe key
underlying mechanisms in the system. The next eight chapters explain the core components
of the operating system: processes, threads, and jobs; memory management; security; the I/O
system; storage management; the cache manager; file systems; and networking. The last chapter covers crash dump analysis.
History of the Book
This is the fourth edition of a book that was originally called Inside Windows NT (Microsoft
Press, 1992), written by Helen Custer (prior to the initial release of Microsoft Windows NT 3.1).
Inside Windows NT was the first book ever published about Windows NT and provided key
insight into the architecture and design of the system. Inside Windows NT, Second Edition
(Microsoft Press, 1998) was written by David Solomon. It was updated to cover Windows NT 4.0
and had a greatly increased level of technical depth. Inside Microsoft Windows 2000, Third
Edition (Microsoft Press, 2000) was authored by David Solomon and Mark Russinovich. It
added many new topics such as startup and shutdown, service internals, registry internals,
file system drivers, and networking, as well as kernel changes in Windows 2000 such as the
Windows Driver Model (WDM), Plug and Play, power management, Windows Management
Instrumentation (WMI), encryption, the job object, and Terminal Services.
Copyrighted material.
xxvii
xxviii
Introduction
Fourth Edition Changes
This latest edition, now called Microsoft Windows Internals, Fourth Edition, has been updated
to cover the kernel changes made in Windows XP and Windows Server 2003, including
support for 64-bit systems. Hands-on experiments have been updated to reflect changes in
tools, and newly added experiments use new tools not available when the third edition was
written.
Since the level of kernel change from Windows 2000 to these versions was relatively small (as
compared to the changes between Windows NT 4.0 and Windows 2000), the vast majority of
this text applies to Windows 2000, Windows XP, and Windows Server 2003. Therefore,
unless explicitly stated, everything applies to all three versions.
Hands-On Experiments
Even without access to the source code, much can be gleaned about Windows internals from
available tools such as the kernel debugger. When a tool can be used to expose or demonstrate some aspect of Windows internal behavior, the steps for trying the tool yourself are
listed in “Experiment” boxes. These appear throughout the book, and we encourage you to try
these as you’re reading—seeing visible proof of how Windows works internally will make
much more of an impression on you than just reading about it will.
Topics Not Covered
Windows is a large and complex operating system. This book doesn’t cover everything relevant to Windows internals but instead focuses on the base system components. For example,
this book doesn’t describe COM+, the Windows distributed object-oriented programming
infrastructure, or the Microsoft .NET Framework, the foundation of the next generation of
managed code applications.
Because this is an internals book and not a user, programming, or system administration
book, it doesn’t describe how to use, program, or configure Windows.
A Warning and Caveat
Because this book describes undocumented behavior of the internal architecture and operation of the Windows operating system (such as internal kernel structures and functions), this
content is subject to change between releases. (External interfaces, such as the Windows API,
are not subject to incompatible changes.)
By “subject to change,” we don’t necessarily mean that details described in this book will
change between releases, but you can’t count on them not changing. Any software that uses
these undocumented interfaces might not work on future releases of Windows. Even worse,
Copyrighted material.
Introduction
xxix
software that runs in kernel mode (such as device drivers) and uses these undocumented
interfaces might experience a system crash when running on a newer release of Windows.
Support
Every effort has been made to ensure the accuracy of this book. Should you run into any problems or issues, please refer to the sources listed below.
From the Authors
This book isn’t perfect. No doubt it contains some inaccuracies, or possibly, we’ve omitted
some topics we should have covered. If you find anything you think is incorrect, or if you
believe we should have included material that isn’t here, please feel free to send e-mail to [email protected]. Updates and corrections will be posted on the page www.sysinternals.com/windowsinternals.
From Microsoft Press
Microsoft also provides corrections for books through the World Wide Web at the following
address:
http://www.microsoft.com/learning/support
To connect directly with the Microsoft Learning Knowledge Base and enter a query regarding
an issue you might have encountered, go to http://www.microsoft.com/learning/support/
search.asp.
In addition to sending feedback directly to the authors, if you have comments, questions, or
ideas regarding the presentation or use of this book, you can send them to Microsoft using
either of the following methods:
Postal Mail:
Microsoft Press
Attn: Windows Internals Editor
One Microsoft Way
Redmond, WA 98052-6399
E-mail:
[email protected]
Please note that product support isn’t offered through the above mail addresses. For support
information regarding Microsoft Windows, go to www.microsoft.com/windows. You can also
call Standard Support at (425) 635-7011 weekdays between 6 a.m. and 6 p.m. Pacific time, or
you can search Microsoft’s Support Online at support.microsoft.com/support.
Copyrighted material.
xxx
Introduction
System Requirements
To use the Microsoft Windows Server 2003 Resource Kit tools, eBooks, and other materials, you
need to meet the following minimum system requirements:
■
Microsoft Windows Server 2003 or Windows XP operating system
■
PC with 233-megahertz (MHz) or higher processor; 550-MHz or higher processor is recommended
■
128 MB of RAM; 256 MB or higher is recommended
■
1.5 to 2 GB of available hard disk space
■
Super VGA (800 x 600) or higher resolution video adapter and monitor
■
CD or DVD drive
■
Keyboard and Microsoft mouse or compatible pointing device
■
Adobe Acrobat or Adobe Reader
■
Internet connectivity for tools that are downloaded
Note
Resource Kit tools are written and tested in English only. Using these tools with a nonEnglish version of Windows might produce unpredictable results. Resource Kit tools are not
supported on 64-bit platforms.
An evaluation edition for Windows Server 2003 Enterprise Edition with Service Pack 1 will be
available on release of Service Pack 1. You can download the evaluation software from the
Microsoft Download Center at http://www.microsoft.com/downloads/. (Availability of software
on the Download Center is at the discretion of Microsoft Corporation and is subject to
change.) To use the evaluation, you need
■
133-MHz or higher processor; 733-MHz or higher processor is recommended for x86based PCs and Itanium-based PCs.
■
128 MB of RAM; 256 MB of RAM is recommended; 32 GB is recommended for x86based PCs (32-bit version) and 64 GB is recommended for Itanium-based PCs (64-bit
version).
■
1.5 to 2 GB of available hard disk space.
■
Super VGA (500 x 600) or higher resolution video adapter and monitor.
■
Keyboard and Microsoft mouse or compatible pointing device.
Note
Actual requirements, including Internet and network access and any related charges,
will vary based on your system configuration and the applications and features you choose to
install. Additional hard disk space may be required if you are installing over a network.
Copyrighted material.
Chapter 1
Concepts and Tools
In this chapter, we’ll introduce the key Microsoft Windows operating system concepts and
terms we’ll be using throughout this book, such as the Windows API, processes, threads, virtual memory, kernel mode and user mode, objects, handles, security, and the registry. We’ll
also introduce the tools that you can use to explore Windows internals, such as the kernel
debugger, the Performance tool, and key tools from www.sysinternals.com. In addition, we’ll
explain how you can use the Windows Device Driver Kit (DDK) and Platform Software Development Kit (SDK) as resources for finding further information on Windows internals.
Be sure that you understand everything in this chapter—the remainder of the book is written
assuming that you do.
Windows Operating System Versions
This book covers the three most recent versions of the Microsoft Windows operating system
based on the Windows NT code base: Windows 2000, Windows XP (32-bit and 64-bit versions), and Windows Server 2003 (32-bit and 64-bit versions). Unless specifically stated, the
text applies to all three versions. As background information, Table 1-1 lists the releases of the
Windows NT code base, their internal version number, and the external product name.
Table 1-1
Windows Operating System Releases
Product Name
Internal Version
Number
Release Date
Windows NT 3.1
3.1
July 1993
Windows NT 3.5
3.5
September 1994
Windows NT 3.51
3.51
May 1995
Windows NT 4.0
4.0
July 1996
Windows 2000
5.0
December 1999
Windows XP
5.1
August 2001
Windows Server 2003
5.2
March 2003
Copyrighted material.
1
2
Microsoft Windows Internals, Fourth Edition
Windows NT vs. Windows 95
From the initial announcement of Windows NT, Microsoft made it clear that it was to be
the long-term replacement for Windows 95 (and its subsequent releases, Windows 98
and Windows Millennium Edition). The following list highlights some architectural differences and advantages that Windows NT (and its subsequent releases) has over Windows 95 (and its subsequent releases):
■
Windows NT supports multiprocessor systems—Windows 95 doesn’t.
■
The Windows NT file system supports security (such as discretionary access control). The Windows 95 file system doesn’t.
■
Windows NT is fully a 32-bit (and now 64-bit) operating system—it contains no
16-bit code, other than support code for running 16-bit Windows applications.
Windows 95 contains a large amount of old 16-bit code from its predecessors,
Windows 3.1 and MS-DOS.
■
Windows NT is fully reentrant—significant parts of Windows 95 are nonreentrant
(mainly the 16-bit code taken from Windows 3.1). This nonreentrant code
includes the majority of the graphics and window management functions (GDI
and USER). When a 32-bit application on Windows 95 attempts to call a system
service implemented in nonreentrant 16-bit code, the application must first obtain
a system-wide lock (or mutex) to block other threads from entering the nonreentrant code base. And even worse, a 16-bit application holds this lock while running.
As a result, although the core of Windows 95 contains a preemptive 32-bit multithreaded scheduler, applications often run single threaded because so much of the
system is still implemented in nonreentrant code.
■
Windows NT provides an option to run 16-bit Windows applications in their own
address space—Windows 95 always runs 16-bit Windows applications in a shared
address space, in which they can corrupt (and hang) each other.
■
Process shared memory on Windows NT is visible only to the processes that are
mapping the same shared memory section. On Windows 95, all shared memory is
visible and writable from all processes. Thus, any process can write to and corrupt
shared memory being used by other cooperating processes.
■
Windows 95 has some critical operating system pages that are writable from user
mode, thus allowing a user application to corrupt or crash the system.
The one thing Windows 95 can do that Windows NT–based systems will never do is
run all older MS-DOS and Windows 3.1 applications (notably applications that require
direct hardware access) as well as 16-bit MS-DOS device drivers. Whereas 100 percent
compatibility with MS-DOS and Windows 3.1 was a mandatory goal for Windows 95,
the original goal for Windows NT was to run most existing 16-bit applications while preserving the integrity and reliability of the system.
Copyrighted material.
Chapter 1:
Concepts and Tools
3
Foundation Concepts and Terms
In the course of this book, we’ll be referring to some structures and concepts that might be
unfamiliar to some readers. In this section, we’ll define the terms we’ll be using throughout.
You should become familiar with them before proceeding to subsequent chapters.
Windows API
The Windows application programming interface (API) is the system programming interface
to the Microsoft Windows operating system family, including Windows 2000, Windows XP,
Windows Server 2003, Windows 95, Windows 98, Windows Millennium Edition (Me), and
Windows CE. Each operating system implements a different subset of the Windows API. Windows 95, Windows 98, Windows Me, and Windows CE are not addressed in this book.
Note
The Windows API is described in the Platform Software Development Kit (SDK) documentation. (See the section “Platform Software Development Kit (SDK)” later in this chapter.)
This documentation is available for free viewing online at msdn.microsoft.com. It is also
included with all subscription levels to the Microsoft Developer Network (MSDN), Microsoft’s
support program for developers. For more information, see msdn.microsoft.com. An excellent
description of how to program the Windows base API is Jeffrey Richter’s book Programming
Applications for Microsoft Windows (4th ed., Microsoft Press, 1999).
Prior to the introduction of 64-bit versions of Windows XP and Windows Server 2003, the
programming interface to the 32-bit version of the Windows operating systems was called the
Win32 API, to distinguish it from the original 16-bit Windows API, which was the programming interface to the original 16-bit versions of Windows. In this book, the term Windows API
refers to the 32-bit interface to Windows 2000 and both the 32-bit and 64-bit programming
interfaces to Windows XP and Windows Server 2003.
The Windows API consists of thousands of callable functions, which are divided into the following major categories:
■
Base Services
■
Component Services
■
User Interface Services
■
Graphics and Multimedia Services
■
Messaging and Collaboration
■
Networking
■
Web Services
This book focuses on the internals of the key base services, such as processes and threads,
memory management, I/O, and security.
Copyrighted material.
4
Microsoft Windows Internals, Fourth Edition
What About .NET and WinFX?
The .NET Framework consists of a library of classes called the Framework Class Library
(FCL) and a Common Language Runtime (CLR) that provides a managed code execution environment with features such as just-in-time compilation, type verification, garbage collection, and code access security. By offering these features, the CLR provides a
development environment that improves programmer productivity and reduces common programming errors. (For an excellent description of the .NET Framework and its
core architecture, see Applied Microsoft .NET Framework Programming by Jeffrey Richter.)
The CLR is implemented as a classic COM server whose code resides in a standard usermode Windows DLL. In fact, all components of the .NET Framework are implemented
as standard user-mode Windows DLLs layered over unmanaged Windows API functions. (None of the .NET Framework runs in kernel mode.) Figure 1-1 illustrates the
relationship of these components:
.NET/WinFX Application
(Standard User-Mode EXEs)
User mode
(managed code)
Framework Class Library Assemblies
(Standard User-Mode DLLs)
User mode
(unmanaged code)
CLR DLLs
(COM server)
Windows API DLLs
Kernel mode
Figure 1-1
Windows Kernel
Relationship of .NET Framework components
WinFX is “the new Windows API.” It is the evolution of the .NET Framework that ships
with Windows “Longhorn,” the next major release of Windows. It will also be installable on Windows XP and Windows Server 2003. WinFX provides the foundation for the
next generation of applications built for the Windows operating system.
Copyrighted material.
Chapter 1:
Concepts and Tools
5
History of the Win32 API
Interestingly, Win32 wasn’t slated to be the original programming interface to Microsoft
Windows NT. Because the Windows NT project started as a replacement for OS/2 version 2, the primary programming interface was the 32-bit OS/2 Presentation Manager
API. A year into the project, however, Microsoft Windows 3.0 hit the market and took
off. As a result, Microsoft changed direction and made Windows NT the future replacement for the Windows family of products as opposed to the replacement for OS/2. It
was at this juncture that the need to specify the Windows API arose—before this, the
Windows API existed only as a 16-bit interface.
Although the Windows API would introduce many new functions that hadn’t been available on Windows 3.1, Microsoft decided to make the new API compatible with the 16-bit
Windows API function names, semantics, and use of data types whenever possible to
ease the burden of porting existing 16-bit Windows applications to Windows NT. So
those of you who are looking at the Windows API for the first time and wondering why
many function names and interfaces seem inconsistent should keep in mind that one
reason for the inconsistency was to ensure that the Windows API is compatible with the
old 16-bit Windows API.
Services, Functions, and Routines
Several terms in the Windows user and programming documentation have different meanings in different contexts. For example, the word service can refer to a callable routine in the
operating system, a device driver, or a server process. The following list describes what certain
terms mean in this book:
■
Windows API functions
■
Native system services (or executive system services) The undocumented, underlying
services in the operating system that are callable from user mode. For example, NtCreateProcess is the internal system service the Windows CreateProcess function calls to create a new process. (For a definition of native functions, see the section “System Service
Dispatching” in Chapter 3.)
■
Kernel support functions (or routines)
Copyrighted material.
Documented, callable subroutines in the Windows API. Examples include CreateProcess, CreateFile, and GetMessage.
Subroutines inside the Windows operating system that can be called only from kernel mode (defined later in this chapter). For example, ExAllocatePool is the routine that device drivers call to allocate memory from the
Windows system heaps.
6
Microsoft Windows Internals, Fourth Edition
■
Windows services Processes started by the Windows service control manager.
(Although the registry defines Windows device drivers as “services,” we don’t refer to
them as such in this book.) For example, the Task Scheduler service runs in a usermode process that supports the at command (which is similar to the UNIX commands at or cron).
■
DLL (dynamic-link library)
A set of callable subroutines linked together as a binary file
that can be dynamically loaded by applications that use the subroutines. Examples
include Msvcrt.dll (the C run-time library) and Kernel32.dll (one of the Windows API
subsystem libraries). Windows user-mode components and applications use DLLs
extensively. The advantage DLLs provide over static libraries is that applications can
share DLLs, and Windows ensures that there is only one in-memory copy of a DLL’s
code among the applications that are referencing it.
Processes, Threads, and Jobs
Although programs and processes appear similar on the surface, they are fundamentally different. A program is a static sequence of instructions, whereas a process is a container for a set
of resources used when executing the instance of the program. At the highest level of abstraction, a Windows process comprises the following:
■
A private virtual address space, which is a set of virtual memory addresses that the process
can use
■
An executable program, which defines initial code and data and is mapped into the process’s virtual address space
■
A list of open handles to various system resources, such as semaphores, communication
ports, and files, that are accessible to all threads in the process
■
A security context called an access token that identifies the user, security groups, and
privileges associated with the process
■
A unique identifier called a process ID (internally called a client ID)
■
At least one thread of execution
Each process also points to its parent or creator process. However, if the parent exits, this
information is not updated. Therefore, it is possible for a process to point to a nonexistent parent. This is not a problem, as nothing relies on this information being present. The following
experiment illustrates this case.
Copyrighted material.
Chapter 1:
Concepts and Tools
EXPERIMENT: Viewing the Process Tree
One unique attribute about a process that most tools don’t display is the parent or creator process ID. You can retrieve this value with the Performance tool (or programmatically) by querying the Creating Process ID. The Tlist.exe tool (in the Windows
Debugging Tools) can show the process tree by using /t switch. Here’s an example of
output from tlist /t:
C:\>tlist /t
System Process (0)
System (2)
smss.exe (21)
csrss.exe (24)
winlogon.exe (35)
services.exe (41)
spoolss.exe (69)
llssrv.exe (94)
LOCATOR.EXE (96)
RpcSs.exe (112)
inetinfo.exe (128)
lsass.exe (44)
nddeagnt.exe (119)
explorer.exe (123) Program Manager
OSA.EXE (121)
WINWORD.EXE (117) Microsoft Word - msch02(s).doc
cmd.exe (72) Command Prompt - tlist /t
tlist.EXE (100)
The list indents each process to show its parent/child relationship. Processes whose parents aren’t alive are left-justified (as is Explorer.exe in the preceding example) because
even if a grandparent process exists, there’s no way to find that relationship. Windows
maintains only the creator process ID, not a link back to the creator of the creator, and
so forth.
To demonstrate the fact that Windows doesn’t keep track of more than just the parent
process ID, follow these steps:
1. Open a Command Prompt window.
2. Type start cmd (which starts a second Command Prompt).
3. Bring up Task Manager.
4. Switch to the second Command Prompt.
5. Type mspaint (which runs Microsoft Paint).
Copyrighted material.
7
8
Microsoft Windows Internals, Fourth Edition
6. Click the intermediate (second) Command Prompt window.
7. Type exit. (Notice that Paint remains.)
8. Switch to Task Manager.
9. Click the Applications tab.
10. Right-click on the Command Prompt task, and select Go To Process.
11. Click on the Cmd.exe process highlighted in gray.
12. Right-click on this process, and select End Process Tree.
13. Click Yes in the Task Manager Warning message box.
The first Command Prompt window will disappear, but you should still see the Paintbrush window because it was the grandchild of the Command Prompt process you terminated; and because the intermediate process (the parent of Paintbrush) was
terminated, there was no link between the parent and the grandchild.
A number of tools for viewing (and modifying) processes and process information are available. The following experiments illustrate the various views of process information you can
obtain with some of these tools. These tools are included within Windows itself and within
the Windows Support Tools, Windows Debugging Tools, Windows resource kits, the Platform SDK, and from www.sysinternals.com. Many of these tools show overlapping subsets of
the core process and thread information, sometimes identified by different names.
Probably the most widely used tool to examine process activity is Task Manager. (Interestingly, there is no such thing as a “task” in the Windows kernel, so Task Manager is really a tool
to manage processes.) The following experiment shows the difference between what Task
Manager lists as applications and processes.
EXPERIMENT: Viewing Process Information with Task Manager
The built-in Windows Task Manager provides a quick list of the processes running on
the system. You can start Task Manager in one of three ways: (1) press Ctrl+Shift+Esc,
(2) right-click on the taskbar and select Task Manager, or (3) press Ctrl+Alt+Delete and
click the Task Manager button. Once Task Manager has started, click the Processes tab
to see the list of running processes. Notice that processes are identified by the name of
the image of which they are an instance. Unlike some objects in Windows, processes
can’t be given global names. To display additional details, choose Select Columns from
the View menu and select additional columns to be added, as shown here:
Copyrighted material.
Chapter 1:
Concepts and Tools
Although what you see in the Task Manager Processes tab is clearly a list of processes, what
the Applications tab displays isn’t as obvious. The Applications tab lists the top-level visible
windows on all the desktops in the interactive window station. (By default, there are two
desktop objects—you can create more by using the Windows CreateDesktop function.) The
Status column indicates whether or not the thread that owns the window is in a Windows
message wait state. “Running” means the thread is waiting for windowing input; “Not
Responding” means the thread isn’t waiting for windowing input (for example, the thread
might be running or waiting for I/O or some Windows synchronization object).
From the Applications tab, you can match a task to the process that owns the thread that
owns the task window by right-clicking on the task name and choosing Go To Process.
Copyrighted material.
9
10
Microsoft Windows Internals, Fourth Edition
Process Explorer, from www.sysinternals.com, shows more details about processes and threads
than any other available tool, which is why you will see it used in a number of experiments
throughout the book. The following are some of the unique things that Process Explorer
shows or enables:
■
Full path name for the image being executed
■
Process security token (list of groups and privileges)
■
Highlighting to show changes in the process and thread list
■
List of services inside service-hosting processes, including display name and description
■
Processes that are part of a job and job details
■
Processes running .NET/WinFX applications and.NET-specific details (such as the list
of appdomains and CLR performance counters)
■
Start time for processes and threads
■
Complete list of memory mapped files (not just DLLs)
■
Ability to suspend a process
■
Ability to kill an individual thread
■
Easy identification of which processes were consuming the most CPU time over a period
of time (The Performance Tool can display process CPU utilization for a given set of processes, but it won’t automatically show processes created after the performance monitoring session has started.)
Process Explorer also provides easy access to information available through other tools from
one central place, such as:
■
Process tree (with ability to collapse parts of the tree)
■
Open handles in a process without prior setup (The Microsoft tools to show open handles require the setting of a systemwide flag and a reboot before they can be used.)
■
List of DLLs (and memory-mapped files) in a process
■
Thread activity within a process
■
User-mode thread stacks (including mapping of addresses to names using the debugging tools’ symbol engine)
■
Kernel-mode thread stacks for system threads (including mapping of addresses to
names using the debugging tools’ symbol engine)
■
Context switch delta (a better representation of CPU activity, as explained in Chapter 6)
■
Kernel memory (paged and nonpaged pool) limits (other tools show only current size)
An introductory experiment using Process Explorer follows.
Copyrighted material.
Chapter 1:
Concepts and Tools
EXPERIMENT: Viewing Process Details with Process Explorer
Download the latest version of Process Explorer from www.sysinternals.com and run it. The
first time you run it, you will receive a message that symbols are not currently configured.
If properly configured, Process Explorer can access symbol information to display the
symbolic name of the thread start function and functions on its call stack (available by
double-clicking on a process and clicking on the Threads tab). This is useful for identifying what threads are doing within a process. To access symbols, you must have the Debugging Tools installed (described later in this chapter). Then click on Options, choose
Configure Symbols, and fill in the appropriate Symbols path. For example:
In the preceding example, the on-demand symbol server is being used to access symbols
and a copy of the symbol files are being stored on the local machine in the c:\symbols
folder. For more information on configuring use of the symbol server, see http://
www.microsoft.com/whdc/ddk/debugging/symbols.mspx.
When Process Explorer starts, it shows by default the process list on the top half and the
open handles for the currently selected process on the bottom half. It also shows the
image description, company name, and full path if you hover the mouse pointer over the
process name.
Copyrighted material.
11
12
Microsoft Windows Internals, Fourth Edition
Here are a few steps to walk you through some basic capabilities of Process Explorer:
1. Turn off the lower pane by deselecting View, Show Lower Pane. (The lower pane
can show open handles or mapped DLLs and memory mapped files—these are
explored in Chapters 3 and 7.)
2. Notice that processes hosting services are highlighted by default in pink. Your own
processes are highlighted in blue. (These colors can be configured.)
3. Hover your mouse pointer over the image name for processes, and notice the full
path displayed by the ToolTip.
4. Click on View, Select Columns, and add the image path.
5. Sort on the process column, and notice the tree view disappears. (You can either
display tree view or sort by any of the columns shown.) Click again to sort from Z
to A. Then click again and the display returns to tree view.
6. Deselect View, Show Processes From All Users to show only your processes.
7. Go to Options, Difference Highlight Duration, and change the value to 5 seconds.
Then launch a new process (anything), and notice the new process highlighted in
green for 5 seconds. Exit this new process, and notice the process is highlighted in
red for 5 seconds before disappearing from the display. This can be useful to see
processes being created and exiting on your system.
8. Finally, double-click on a process and explore the various tabs available from the
process properties display. (These will be referenced in various experiments
throughout the book where the information being shown is being explained.)
A thread is the entity within a process that Windows schedules for execution. Without it, the
process’s program can’t run. A thread includes the following essential components:
■
The contents of a set of CPU registers representing the state of the processor.
■
Two stacks, one for the thread to use while executing in kernel mode and one for executing in user mode.
■
A private storage area called thread-local storage (TLS) for use by subsystems, run-time
libraries, and DLLs.
■
A unique identifier called a thread ID (also internally called a client ID—process IDs and
thread IDs are generated out of the same namespace, so they never overlap).
■
Threads sometimes have their own security context that is often used by multithreaded
server applications that impersonate the security context of the clients that they serve.
The volatile registers, stacks, and private storage area are called the thread’s context. Because
this information is different for each machine architecture that Windows runs on, this structure, by necessity, is architecture-specific. The Windows GetThreadContext function provides
access to this architecture-specific information (called the CONTEXT block).
Copyrighted material.
Chapter 1:
Concepts and Tools
13
Fibers vs. Threads
Fibers allow an application to schedule its own “threads” of execution rather than rely
on the priority-based scheduling mechanism built into Windows. Fibers are often called
“lightweight” threads, and in terms of scheduling, they’re invisible to the kernel because
they’re implemented in user mode in Kernel32.dll. To use fibers, a call is first made to
the Windows ConvertThreadToFiber function. This function converts the thread to a running fiber. Afterward, the newly converted fiber can create additional fibers with the CreateFiber function. (Each fiber can have its own set of fibers.) Unlike a thread, however, a
fiber doesn’t begin execution until it’s manually selected through a call to the
SwitchToFiber function. The new fiber runs until it exits or until it calls SwitchToFiber,
again selecting another fiber to run. For more information, see the Platform SDK documentation on fiber functions.
Although threads have their own execution context, every thread within a process shares the
process’s virtual address space (in addition to the rest of the resources belonging to the process), meaning that all the threads in a process can write to and read from each other’s memory. Threads cannot accidentally reference the address space of another process, however,
unless the other process makes available part of its private address space as a shared memory
section (called a file mapping object in the Windows API) or unless one process has the right to
open another process to use cross-process memory functions such as ReadProcessMemory and
WriteProcessMemory.
In addition to a private address space and one or more threads, each process has a security
identification and a list of open handles to objects such as files, shared memory sections, or
one of the synchronization objects such as mutexes, events, or semaphores, as illustrated in
Figure 1-2.
Access token
Process
object
Virtual address descriptors (VADs)
VAD
VAD
VAD
Handle table
Object
Object
Thread
Thread
Thread
...
Access token
Figure 1-2
Copyrighted material.
A process and its resources
14
Microsoft Windows Internals, Fourth Edition
Every process has a security context that is stored in an object called an access token. The process access token contains the security identification and credentials for the process. By
default, threads don’t have their own access token, but they can obtain one, thus allowing
individual threads to impersonate the security context of another process—including processes running on a remote Windows system—without affecting other threads in the process.
(See Chapter 8 for more details on process and thread security.)
The virtual address descriptors (VADs) are data structures that the memory manager uses to
keep track of the virtual addresses the process is using. These data structures are described in
more depth in Chapter 7.
Windows provides an extension to the process model called a job. A job object’s main function is to allow groups of processes to be managed and manipulated as a unit. A job object
allows control of certain attributes and provides limits for the process or processes associated
with the job. It also records basic accounting information for all processes associated with the
job and for all processes that were associated with the job but have since terminated. In some
ways, the job object compensates for the lack of a structured process tree in Windows—yet in
many ways it is more powerful than a UNIX-style process tree.
You’ll find out much more about the internal structure of jobs, processes and threads, the
mechanics of process and thread creation, and the thread-scheduling algorithms in Chapter 6.
Virtual Memory
Windows implements a virtual memory system based on a flat (linear) address space that provides each process with the illusion of having its own large, private address space. Virtual
memory provides a logical view of memory that might not correspond to its physical layout.
At run time, the memory manager, with assistance from hardware, translates, or maps, the virtual addresses into physical addresses, where the data is actually stored. By controlling the
protection and mapping, the operating system can ensure that individual processes don’t
bump into one another or overwrite operating system data. Figure 1-3 illustrates three virtually contiguous pages mapped to three discontiguous pages in physical memory.
Virtual memory
Physical memory
Figure 1-3 Mapping virtual memory to physical memory
Copyrighted material.
Chapter 1:
Concepts and Tools
15
Because most systems have much less physical memory than the total virtual memory in use
by the running processes, the memory manager transfers, or pages, some of the memory contents to disk. Paging data to disk frees physical memory so that it can be used for other processes or for the operating system itself. When a thread accesses a virtual address that has
been paged to disk, the virtual memory manager loads the information back into memory
from disk. Applications don’t have to be altered in any way to take advantage of paging
because hardware support enables the memory manager to page without the knowledge or
assistance of processes or threads.
The size of the virtual address space varies for each hardware platform. On 32-bit x86 systems, the total virtual address space has a theoretical maximum of 4 GB. By default, Windows
allocates half this address space (the lower half of the 4 GB virtual address space, from
x00000000 through x7FFFFFFF) to processes for their unique private storage and uses the
other half (the upper half, addresses x80000000 through xFFFFFFFF) for its own protected
operating system memory utilization. The mappings of the lower half change to reflect the virtual address space of the currently executing process, but the mappings of the upper half
always consist of the operating system’s virtual memory. Windows 2000 Advanced Server,
Windows 2000 Datacenter Server, Windows XP (SP2 and later), and Windows Server 2003
support boot-time options (the /3GB and /USERVA qualifiers in Boot.ini, described in Chapter 5) that give processes running specially marked programs (the large address space aware
flag must be set in the header of the executable image) the ability to use up to 3 GB of private
address space (leaving 1 GB for the operating system). This option allows applications such as
database servers to keep larger portions of a database in the process address space, thus
reducing the need to map subset views of the database. Figure 1-4 shows the two virtual
address space layouts supported by 32-bit Windows.
Default
3 GB user space
2 GB User
process space
3 GB User
process space
2 GB System
space
1 GB System
space
Figure 1-4
Address space layouts for 32-bit Windows
Although 3 GB is better than 2 GB, it’s still not enough virtual address space to map very large
(multigigabyte) databases. To address this need on 32-bit systems, Windows provides a mechanism called Address Windowing Extension (AWE), which allows a 32-bit application to allocate
up to 64 GB of physical memory and then map views, or windows, into its 2-GB virtual
address space. Although using AWE puts the burden of managing mappings of virtual to
Copyrighted material.
16
Microsoft Windows Internals, Fourth Edition
physical memory on the programmer, it does address the need of being able to directly access
more physical memory than can be mapped at any one time in a 32-bit process address space.
64-bit Windows provides a much larger address space for processes: 7152 GB on Itanium systems and 8192 GB on x64 systems. Figure 1-5 shows a simplified view of the 64-bit system
address space layouts. (For a detailed description, see Chapter 7.) Note that these sizes do not
represent the architectural limits for these platforms, but rather implementation limits in the
current versions of 64-bit Windows.
x64
Itanium
8192 GB
(8 TB) User
process space
7152 GB
(7 TB) User
process space
6657 GB
System space
6144 GB
System space
Figure 1-5
Address space layouts for 64-bit Windows
Details of the implementation of the memory manager, including how address translation
works and how Windows manages physical memory, are described in Chapter 7.
Kernel Mode vs. User Mode
To protect user applications from accessing and/or modifying critical operating system data,
Windows uses two processor access modes (even if the processor on which Windows is running
supports more than two): user mode and kernel mode. User application code runs in user
mode, whereas operating system code (such as system services and device drivers) runs in
kernel mode. Kernel mode refers to a mode of execution in a processor that grants access to
all system memory and all CPU instructions. By providing the operating system software with
a higher privilege level than the application software has, the processor provides a necessary
foundation for operating system designers to ensure that a misbehaving application can’t disrupt the stability of the system as a whole.
Note The architecture of the Intel x86 processor defines four privilege levels, or rings, to
protect system code and data from being overwritten either inadvertently or maliciously by
code of lesser privilege. Windows uses privilege level 0 (or ring 0) for kernel mode and privilege
level 3 (or ring 3) for user mode. The reason Windows uses only two levels is that some hardware architectures that were supported in the past (such as Compaq Alpha and Silicon Graphics MIPS) implemented only two privilege levels.
Copyrighted material.
Chapter 1:
Concepts and Tools
17
Although each Windows process has its own private memory space, the kernel-mode operating system and device driver code share a single virtual address space. Each page in virtual
memory is tagged as to what access mode the processor must be in to read and/or write the
page. Pages in system space can be accessed only from kernel mode, whereas all pages in the
user address space are accessible from user mode. Read-only pages (such as those that contain executable code) are not writable from any mode.
Windows doesn’t provide any protection to private read/write system memory being used by
components running in kernel mode. In other words, once in kernel mode, operating system
and device driver code has complete access to system space memory and can bypass Windows security to access objects. Because the bulk of the Windows operating system code runs
in kernel mode, it is vital that components that run in kernel mode be carefully designed and
tested to ensure that they don’t violate system security.
This lack of protection also emphasizes the need to take care when loading a third-party
device driver, because once in kernel mode the software has complete access to all operating
system data. This vulnerability was one of the reasons behind the driver-signing mechanism
introduced in Windows, which warns the user if an attempt is made to add an unauthorized
(unsigned) driver. (See Chapter 9 for more information on driver signing.) Also, a mechanism
called Driver Verifier helps device driver writers to find bugs (such as buffer overruns or memory leaks). Driver Verifier is also explained in Chapter 7.
As you’ll see in Chapter 2, user applications switch from user mode to kernel mode when they
make a system service call. For example, a Windows ReadFile function eventually needs to call
the internal Windows routine that actually handles reading data from a file. That routine,
because it accesses internal system data structures, must run in kernel mode. The transition
from user mode to kernel mode is accomplished by the use of a special processor instruction
that causes the processor to switch to kernel mode. The operating system traps this instruction, notices that a system service is being requested, validates the arguments the thread
passed to the system function, and then executes the internal function. Before returning control to the user thread, the processor mode is switched back to user mode. In this way, the
operating system protects itself and its data from perusal and modification by user processes.
Note A transition from user mode to kernel mode (and back) does not affect thread scheduling per se—a mode transition is not a context switch. Further details on system service dispatching are included in Chapter 3.
Copyrighted material.
18
Microsoft Windows Internals, Fourth Edition
Thus, it’s normal for a user thread to spend part of its time executing in user mode and part
in kernel mode. In fact, because the bulk of the graphics and windowing system also runs in
kernel mode, graphics-intensive applications spend more of their time in kernel mode than in
user mode. An easy way to test this is to run a graphics-intensive application such as Microsoft
Paint or Microsoft Pinball and watch the time split between user mode and kernel mode using
one of the performance counters listed in Table 1-2.
Table 1-2 Mode-Related Performance Counters
Object: Counter
Function
Processor: % Privileged Time
Percentage of time that an individual CPU (or all
CPUs) has run in kernel mode during a specified
interval
Processor: % User Time
Percentage of time that an individual CPU (or all
CPUs) has run in user mode during a specified
interval
Process: % Privileged Time
Percentage of time that the threads in a process
have run in kernel mode during a specified
interval
Process: % User Time
Percentage of time that the threads in a process
have run in user mode during a specified interval
Thread: % Privileged Time
Percentage of time that a thread has run in kernel mode during a specified interval
Thread: % User Time
Percentage of time that a thread has run in user
mode during a specified interval
EXPERIMENT: Viewing Thread Activity with QuickSlice
QuickSlice gives a quick, dynamic view of the proportions of system and kernel time
that each process currently running on your system is using. On line, the red part of the
bar shows the amount of CPU time spent in kernel mode, and the blue part shows the
user-mode time. (Although reproduced in the window below in black and white, the
bars in the online display are always red and blue.) The total of all bars shown in the
QuickSlice window should add up to 100 percent of CPU time. To run QuickSlice, click
the Start button, choose Run, and enter Qslice.exe (assuming the Windows 2000
resource kit is in your path). For example, try running a graphics-intensive application
such as Paint (Mspaint.exe). Open QuickSlice and Paint side by side, and draw squiggles in the Paint window. When you do so, you’ll see Mspaint.exe running in the QuickSlice window, as shown here:
Copyrighted material.
Chapter 1:
Concepts and Tools
For additional information about the threads in a process, you can also double-click on
a process (on either the process name or the colored bar). Here you can see the threads
within the process and the relative CPU time each thread uses (not across the system):
EXPERIMENT: Kernel Mode vs. User Mode
You can use the Performance tool to see how much time your system spends executing
in kernel mode vs. in user mode. Follow these steps:
1. Run the Performance tool by opening the Start menu and selecting Programs/
Administrative Tools/Performance.
2. Click the Add button (+) on the toolbar.
3. With the Processor performance object selected, click the % Privileged Time
counter and, while holding down the Ctrl key, click the % User Time counter.
4. Click Add, and then click Close.
5. Move the mouse rapidly back and forth. You should notice the % Privileged Time
line going up when you move the mouse around, reflecting the time spent servicing the mouse interrupts and the time spent in the graphics part of the windowing
system (which, as explained in Chapter 2, runs primarily as a device driver in kernel mode). (See Figure 1-6.)
Copyrighted material.
19
20
Microsoft Windows Internals, Fourth Edition
6. When you’re finished, click the New Counter Set button on the toolbar (or just
close the tool).
You can also quickly see this activity by using Task Manager. Just click the Performance
tab, and then select Show Kernel Times from the View menu. The CPU usage bar will
show user-mode time in green and kernel-mode time in red.
Figure 1-6
Performance tool showing time split between kernel mode and user mode
To see how the Performance tool itself uses kernel time and user time, run it again, but
add the individual Process counters % User Time and % Privileged Time for every process in the system:
1. If it’s not already running, run the Performance tool again. (If it is already running,
start with a blank display by pressing the New Counter Set button on the toolbar.)
2. Click the Add button (+) on the toolbar.
3. Change the Performance Object to Process.
4. Select the % Privileged Time and % User Time counters.
5. Select all processes in the Instance box (except the _Total process).
6. Click Add, and then click Close.
7. Move the mouse rapidly back and forth.
8. Press Ctrl+H to turn on highlighting mode. This highlights the currently selected
counter in white on Windows 2000 and black on Windows XP and Windows
Server 2003.
9. Scroll through the counters at the bottom of the display to identify the processes
whose threads were running when you moved the mouse, and note whether they
were running in user mode or kernel mode.
Copyrighted material.
Chapter 1:
Concepts and Tools
21
You should see the Performance tool process (by looking in the Instance column for the
mmc process) kernel-mode and user-mode time go up when you move the mouse
because it is executing application code in user mode and calling Windows functions
that run in kernel mode. You’ll also notice kernel-mode thread activity in a process
named csrss when you move the mouse. This activity occurs because the Windows subsystem’s kernel-mode raw input thread, which handles keyboard and mouse input, is
attached to this process. (See Chapter 2 for more information about system threads.)
Finally, the process named Idle that you see spending nearly 100 percent of its time in
kernel mode isn’t really a process—it’s a fake process used to account for idle CPU cycles.
As you can observe from the mode in which the threads in the Idle process run, when
Windows has nothing to do, it does it in kernel mode.
Terminal Services and Multiple Sessions
Terminal Services refers to the support in Windows for multiple interactive user sessions on a
single system. With Windows Terminal Services, a remote user can establish a session on
another machine, log in, and run applications on the server. The server transmits the graphical user interface to the client, and the client transmits the user’s input back to the server.
(This is different than X windows on UNIX systems, which permit running individual applications on a server system with the display remoted to the client, because the entire user session is remoted, not just a single application.)
The first login session at the physical console of the machine is considered the console session, or session zero. Additional sessions can be created through the use of the remote desktop connection program (Mstsc.exe) or on Windows XP systems through the use of fast user
switching (described later).
The capability to create a remote session is supported on Windows 2000 Server systems but
not Windows 2000 Professional. Windows XP Professional permits a single remote user to
connect to the machine, but if someone is logged in at the console, the workstation is locked
(that is, someone can be using the system either locally or remotely, but not at the same time).
Windows 2000 Server and Windows Server 2003 Standard Edition support two simultaneous remote connections. (This is to facilitate remote management—for example, use of management tools that require being logged in to the machine being managed.) Windows 2000
Advanced Server, Datacenter Server, Windows Server 2003 Enterprise Edition, and Datacenter Edition can support more than two sessions if appropriately licensed and configured as
a terminal server.
Although Windows XP Home and Professional editions do not support multiple remote desktop connections, they do support multiple sessions created locally through a feature called
fast user switching. (This feature is disabled on Windows XP Professional if the system joins
a domain.) When a user chooses to disconnect their session instead of log off (for example, by
Copyrighted material.
22
Microsoft Windows Internals, Fourth Edition
clicking Start, clicking Log Off, and choosing Switch User or by holding down the Windows
key and pressing “L”), the current session (that is, the processes running in that session and
all the sessionwide data structures that describe the session) remains in the system and the
system returns to the main logon screen. If a new user logs in, a new session is created.
For applications that want to be aware of running in a terminal server session, there are a set
of Windows APIs for programmatically detecting that as well as for controlling various aspects
of terminal services. (See the Platform SDK for details.)
Chapter 2 describes briefly how sessions are created and has some experiments showing how
to view session information with various tools, including the kernel debugger. The “Object
Manager” section in Chapter 3 describes how the system namespace for objects is instantiated
on a per-session basis and how applications that need to be aware of other instances of themselves on the same system can accomplish that. Finally, Chapter 7 covers how the memory
manager sets up and manages sessionwide data.
Objects and Handles
In the Windows operating system, an object is a single, run-time instance of a statically defined
object type. An object type comprises a system-defined data type, functions that operate on
instances of the data type, and a set of object attributes. If you write Windows applications,
you might encounter process, thread, file, and event objects, to name just a few examples.
These objects are based on lower-level objects that Windows creates and manages. In Windows, a process is an instance of the process object type, a file is an instance of the file object
type, and so on.
An object attribute is a field of data in an object that partially defines the object’s state. An
object of type process, for example, would have attributes that include the process ID, a base
scheduling priority, and a pointer to an access token object. Object methods, the means for
manipulating objects, usually read or change the object attributes. For example, the open
method for a process would accept a process identifier as input and return a pointer to the
object as output.
Note Although there is a parameter named ObjectAttributes that a caller supplies when creating an object using either the Windows API or native object services, that parameter
shouldn’t be confused with the more general meaning of the term as used in this book.
The most fundamental difference between an object and an ordinary data structure is that the
internal structure of an object is hidden. You must call an object service to get data out of an
object or to put data into it. You can’t directly read or change data inside an object. This difference separates the underlying implementation of the object from code that merely uses it, a
technique that allows object implementations to be changed easily over time.
Copyrighted material.
Chapter 1:
Concepts and Tools
23
Objects provide a convenient means for accomplishing the following four important operating system tasks:
■
Providing human-readable names for system resources
■
Sharing resources and data among processes
■
Protecting resources from unauthorized access
■
Reference tracking, which allows the system to know when an object is no longer in use
so that it can be automatically deallocated
Not all data structures in the Windows operating system are objects. Only data that needs to
be shared, protected, named, or made visible to user-mode programs (via system services) is
placed in objects. Structures used by only one component of the operating system to implement internal functions are not objects. Objects and handles (references to an instance of an
object) are discussed in more detail in Chapter 3.
Security
Windows was designed from the start to be secure and to meet the requirements of various
formal government and industry security ratings, such as the Common Criteria for Information Technology Security Evaluation (CCITSE) specification. Achieving a governmentapproved security rating allows an operating system to compete in that arena. Of course,
many of these required capabilities are advantageous features for any multiuser system.
The core security capabilities of Windows include: discretionary (need-to-know) protection
for all shareable system objects (such as files, directories, processes, threads, and so forth),
security auditing (for accountability of subjects, or users and the actions they initiate), password authentication at logon, and the prevention of one user from accessing uninitialized
resources (such as free memory or disk space) that another user has deallocated.
Windows has two forms of access control over objects. The first form—discretionary access
control—is the protection mechanism that most people think of when they think of operating
system security. It’s the method by which owners of objects (such as files or printers) grant or
deny access to others. When users log in, they are given a set of security credentials, or a security context. When they attempt to access objects, their security context is compared to the
access control list on the object they are trying to access to determine whether they have permission to perform the requested operation.
Privileged access control is necessary for those times when discretionary access control isn’t
enough. It’s a method of ensuring that someone can get to protected objects if the owner isn’t
available. For example, if an employee leaves a company, the administrator needs a way to gain
access to files that might have been accessible only to that employee. In that case, under Windows, the administrator can take ownership of the file so that you can manage its rights as
necessary.
Copyrighted material.
24
Microsoft Windows Internals, Fourth Edition
Security pervades the interface of the Windows API. The Windows subsystem implements
object-based security in the same way the operating system does; the Windows subsystem
protects shared Windows objects from unauthorized access by placing Windows security
descriptors on them. The first time an application tries to access a shared object, the Windows
subsystem verifies the application’s right to do so. If the security check succeeds, the Windows subsystem allows the application to proceed.
The Windows subsystem implements object security on a number of shared objects, some of
which were built on top of native Windows objects. The Windows objects include desktop
objects, window objects, menu objects, files, processes, threads, and several synchronization
objects.
For a comprehensive description of Windows security, see Chapter 8.
Registry
If you’ve worked at all with Windows operating systems, you’ve probably heard about or
looked at the registry. You can’t talk much about Windows internals without referring to the
registry because it’s the system database that contains the information required to boot and
configure the system, systemwide software settings that control the operation of Windows ,
the security database, and per-user configuration settings (such as which screen saver to use).
In addition, the registry is a window into in-memory volatile data, such as the current hardware state of the system (what device drivers are loaded, the resources they are using, and so
on) as well as the Windows performance counters. The performance counters, which aren’t
actually “in” the registry, are accessed through the registry functions. See Chapter 4 for more
on how performance counter information is accessed from the registry.
Although many Windows users and administrators will never need to look directly into the
registry (because you can view or change most configuration settings with standard administrative utilities), it is still a useful source of Windows internals information because it contains
many settings that affect system performance and behavior. (If you decide to directly change
registry settings, you must exercise extreme caution; any changes might adversely affect system performance or, worse, cause the system to fail to boot successfully.) You’ll find references
to individual registry keys throughout this book as they pertain to the component being
described. Most registry keys referred to in this book are under HKEY_LOCAL_MACHINE,
which we’ll abbreviate throughout as HKLM.
For further information on the registry and its internal structure, see Chapter 4.
Copyrighted material.
Chapter 1:
Concepts and Tools
25
Unicode
Windows differs from most other operating systems in that most internal text strings are
stored and processed as 16-bit-wide Unicode characters. Unicode is an international character
set standard that defines unique 16-bit values for most of the world’s known character sets.
(For more information about Unicode, see www.unicode.org as well as the programming documentation in the MSDN Library.)
Because many applications deal with 8-bit (single-byte) ANSI character strings, Windows
functions that accept string parameters have two entry points: a Unicode (wide, 16-bit) and an
ANSI (narrow, 8-bit) version. The Windows 95, Windows 98, and Windows Millennium Edition implementations of Windows don’t implement all the Unicode interfaces to all the Windows functions, so applications designed to run on one of these operating systems as well as
Windows typically use the narrow versions. If you call the narrow version of a Windows function, input string parameters are converted to Unicode before being processed by the system
and output parameters are converted from Unicode to ANSI before being returned to the
application. Thus, if you have an older service or piece of code that you need to run on Windows but this code is written using ANSI character text strings, Windows will convert the
ANSI characters into Unicode for its own use. However, Windows never converts the data
inside files—it’s up to the application to decide whether to store data as Unicode or as ANSI.
In previous editions of Windows, Asian and Middle East editions were a superset of the core
U.S. and European editions and contained additional Windows functions to handle more
complex text input and layout requirements (such as right-to-left text input). As of Windows
2000, all language editions contain the same Windows functions. Instead of having separate
language versions, Windows has a single worldwide binary so that a single installation can
support multiple languages (by adding various language packs). Applications can also take
advantage of Windows functions that allow single worldwide application binaries that can
support multiple languages.
Digging into Windows Internals
Although much of the information in this book is based on reading the Windows source code
and talking to the developers, you don’t have to take everything on faith. Many details about
the internals of Windows can be exposed and demonstrated by using a variety of available
tools, such as those that come with Windows, the Windows Support Tools, the Windows
resource kit tools, and the Windows debugging tools. These tool packages are briefly
described later in this section.
To encourage your exploration of Windows internals, we’ve included “Experiment” sidebars
throughout the book that describe steps you can take to examine a particular aspect of Windows internal behavior. (You already saw one of these sections earlier in this chapter.) We
encourage you to try these experiments so that you can see in action many of the internals
topics described in this book.
Copyrighted material.
26
Microsoft Windows Internals, Fourth Edition
Table 1-3 shows a list of the tools used in this book and where they come from.
Table 1-3 Tools for Viewing Windows Internals
Tool
Image Name
Origin
Startup Programs Viewer
AUTORUNS
www.sysinternals.com
Dependency Walker
DEPENDS
Support Tools, Platform SDK
DLL List
LISTDLLS
www.sysinternals.com
EFS Information Dumper
EFSDUMP
www.sysinternals.com*
File Monitor
FILEMON
www.sysinternals.com
Global Flags
GFLAGS
Support Tools
Handle Viewer
HANDLE
www.sysinternals.com
Junction tool
JUNCTION
www.sysinternals.com
Kernel debuggers
WINDBG, KD
Debugging tools, Platform SDK,
Windows DDK
Live Kernel Debugging
LIVEKD
www.sysinternals.com
Logon Sessions
LOGINSESSIONS
www.sysinternals.com
Object Viewer
WINOBJ
www.sysinternals.com
Open Handles
OH
Resource kits
Page Fault Monitor
PFMON
Support Tools, Resource kits,
Platform SDK
Pending File Moves
PENDMOVES
www.sysinternals.com
Performance tool
PERFMON.MSC
Windows built-in tool
PipeList tool
PIPELIST
www.sysinternals.com
Pool Monitor
POOLMON
Support Tools, Windows DDK
Process Explorer
PROCEXP
www.sysinternals.com
Get SID tool
PSGETSID
www.sysinternals.com
Process Statistics
PSTAT
Support Tools, Windows 2000
Resource kits, Platform SDK,
www.reskit.com
Process Viewer
PVIEWER (in the Support
Tools) or PVIEW (in the
Platform SDK)
Platform SDK
Quick Slice
QSLICE
Windows 2000 resource kits
Registry Monitor
REGMON
www.sysinternals.com
Service Control
SC
Windows XP, Platform SDK,
Windows 2000 resource kits
Task (Process) List
TLIST
Debugging tools
Task Manager
TASKMGR
Windows built-in tool
TDImon
TDIMON
www.sysinternals.com
Copyrighted material.
Chapter 1:
Concepts and Tools
27
Performance Tool
We’ll refer to the Performance tool found in the Administrative Tools folder on the Start menu
(or via Control Panel) throughout this book. The Performance tool has three functions: system monitoring, viewing performance counter logs, and setting alerts. For simplicity, when we
refer to the Performance tool, we are referring to the System Monitor function within the tool.
The Performance tool can provide more information about how your system is operating than
any other single utility. It includes hundreds of counters for various objects. For each major
topic described in this book, a table of the relevant Windows performance counters is
included.
The Performance tool contains a brief description for each counter. To see the descriptions,
select a counter in the Add Counter window and click the Explain button. Or open the Performance Counter Reference help file in the resource kit. For information on how to interpret
these counters to detect bottlenecks or plan capacity, see the section “Performance Monitoring” in the Windows 2000 Server Operations Guide, which is part of the Windows 2000 Server
Resource Kit. These chapters provide an excellent description to anyone seriously interested
in understanding Windows performance. For Windows XP and Windows Server 2003 , see
the Windows Server 2003 Resource Kit Performance Counters Reference documentation
(available online at www.microsoft.com).
Note that all the Windows performance counters are accessible programmatically. The section “HKEY_PERFORMANCE_DATA” in Chapter 4 has a brief description of the components
involved in retrieving performance counters through the Windows API.
Windows Support Tools
The Windows Support Tools consist of about 40 tools useful in administering and troubleshooting Windows systems. Many of these tools were formerly part of the Windows NT 4
resource kits.
You can install the Support Tools by running Setup.exe in the \Support\Tools folder on any
Windows product distribution media. For Windows 2000, the Support Tools are the same on
Windows 2000 Professional, Server, Advanced Server, and Datacenter Server. Windows XP
has its own version of the Support Tools, as does Windows Server 2003.
Windows Resource Kits
The Windows resource kits supplement the Support Tools, adding additional tools for system
administration and support. The Windows 2003 Resource Kit tools are freely downloadable
from www.microsoft.com (by searching for “resource kit tools”). They can be installed on Windows XP or Windows Server 2003.
There are two editions of the Windows 2000 resource kits: the Windows 2000 Professional
Resource Kit and the Windows 2000 Server Resource Kit. (Supplement 1 is the most recent
Copyrighted material.
28
Microsoft Windows Internals, Fourth Edition
version.) Although the latter kit is a superset of the former and can be installed on Windows
2000 Professional systems, none of the experiments in this book use the tools that are
included only with the Windows 2000 Server Resource Kit. Unlike the Windows Server 2003
Resource Kit, these tools are not freely downloadable. However, the Windows 2000 Server
Resource Kit is included with the MSDN and TechNet subscriptions.
Kernel Debugging
Kernel debugging means examining internal kernel data structures and/or stepping through
functions in the kernel. It is a useful way to investigate Windows internals because you can
display internal system information not available through any other tools and get a clearer
idea of code flows within the kernel.
Kernel debugging can be performed with a variety of tools: the Windows Debugging Tools
from Microsoft, LiveKD from www.sysinternals.com, or SoftIce from Compuware NuMega.
Before describing these tools, let’s examine a file that you’ll need to perform any type of kernel
debugging.
Symbols for Kernel Debugging
Symbol files contain the names of functions and variables. They are generated by the linker
and used debuggers to reference and display these names during a debug session. This information is not usually stored in the binary image because it is not needed to execute the code.
This means that binaries are smaller and faster. However, this means that when debugging,
you must make sure that the debugger can access the symbol files that are associated with the
images you are referencing during a debugging session.
To use any of the kernel debugging tools to examine internal Windows kernel data structures
(such as the process list, thread blocks, loaded driver list, memory usage information, and so
on), you must have the correct symbol files for at least the kernel image, Ntoskrnl.exe. (The
section “Architecture Overview” in Chapter 2 explains more about this file.) Symbol table files
must match the version of the image they were taken from. For example, if you install a
Windows Service Pack or hot fix, you must obtain the matching, updated symbol files for at
least the kernel image; otherwise, you’ll get a checksum error when you try to load them with
the kernel debugger.
While it is possible to download and install symbols for various versions of Windows,
updated symbols for hot fixes are not always available. The easiest solution to obtain the correct version of symbols for debugging is to use the Microsoft on-demand symbol server by
using a special syntax for the symbol path that you specify in the debugger. For example, the
following symbol path causes the debugging tools to load required symbols from the Internet
symbol server and keep a local copy in the c:\symbols folder:
srv*c:\symbols*http://msdl.microsoft.com/download/symbols
Copyrighted material.
Chapter 1:
Concepts and Tools
29
For detailed instructions on how to use the symbol server, see the Debugging Tools help file
or the Web page www.microsoft.com/whdc/ddk/debugging/symbols.mspx.
Windows Debugging Tools
The Windows Debugging Tools package contains advanced debugging tools used in this
book to explore Windows internals. You can find the latest version at www.microsoft.com/
whdc/ddk/debugging. These tools can be used to debug user-mode processes as well as the
kernel. (See the following sidebar.)
Note The Windows Debugging Tools are updated frequently and released independently of
Windows operating system versions, so check often for new versions.
User-Mode Debugging
The debugging tools can also be used to attach to a user-mode process and examine
and/or change process memory. There are two options when attaching to a process:
■
Invasive
■
Noninvasive
Unless specified otherwise, when you attach to a running process, the
DebugActiveProcess Windows function is used to establish a connection between
the debugger and the debugee. This permits examining and/or changing process
memory, setting breakpoints, and performing other debugging functions. In Windows 2000, when the debugger exits, the debugee process is killed. However, as of
Windows XP, you can detach a debugger without killing the target process.
With this option, the debugger simply opens the process with the
OpenProcess function. It does not attach to the process as a debugger. This allows
you to examine and/or change memory in the target process, but you cannot set
breakpoints. The advantage of this option is that you can exit the debugger on
Windows 2000 without killing the target process.
You can also open user-mode process dump files with the debugging tools. User mode
dump files are explained in Chapter 3 in the section on exception dispatching.
There are two primary variants of the Microsoft debuggers that can be used for kernel debugging: a command-line version (Kd.exe) and a graphical user interface (GUI) version
(Windbg.exe). Both provide the same set of commands, so which you choose is a matter of
personal preference. You can perform three types of kernel debugging with these tools:
■
Open a crash dump file created as a result of a Windows system crash. (See Chapter 14
for more information on crash dumps.)
■
Connect to a live, running system and examine the system state (or set breakpoints if
you’re debugging device driver code). This operation requires two computers—a target
Copyrighted material.
30
Microsoft Windows Internals, Fourth Edition
and a host. The target is the system being debugged, and the host is the system running
the debugger. The target system can be either local (connected to the host via a null
modem or IEEE 1394 cable) or remote (connected to the host via a modem). The target
system must be booted with the /DEBUG qualifier (either by pressing F8 during the
boot process and selecting Debug Mode or by adding a boot selection entry in Boot.ini).
■
For Windows XP and Windows Server 2003 systems, connect to the local system and
examine the system state. This is called local kernel debugging. To initiate local kernel
debugging, select the menu item File, select Kernel Debug, click on the Local tab, and
click OK. An example output screen is shown in Figure 1-7. Some kernel debugger commands do not work when used in local kernel debugging mode (such as viewing kernel
stacks and creating a memory dump with the .dump command). However, you can use
the free LiveKd tool from www.sysinternals.com in cases where the native local debugging
support does not work. (See the next section.)
Figure 1-7
Local kernel debugging
Once connected in kernel debugging mode, you can use one of the many debugger extension
commands (commands that begin with “!”) to display the contents of internal data structures
such as threads, processes, I/O request packets, and memory management information.
Throughout this book, the relevant kernel debugger commands and output are included as
they apply to each topic being discussed. In addition, the dt (display type) command can format over 400 kernel structures because the kernel symbol files for Windows 2000 Service
Pack 3, Windows XP, and Windows Server 2003 contain type information that the debugger
can use to format structures.
Copyrighted material.
Chapter 1:
Concepts and Tools
EXPERIMENT: Displaying Type Information for Kernel Structures
To display the list of kernel structures whose type information is included in the kernel
symbols, type dt nt!_* in the kernel debugger. A sample partial output is shown below:
lkd> dt nt!_*
nt!_LIST_ENTRY
nt!_LIST_ENTRY
nt!_IMAGE_NT_HEADERS
nt!_IMAGE_FILE_HEADER
nt!_IMAGE_OPTIONAL_HEADER
nt!_IMAGE_NT_HEADERS
nt!_LARGE_INTEGER
You can also use the dt command to search for specific structures by using its wildcard
lookup capability. For example, if you were looking for the structure name for an interrupt object, type dt nt!_*interrupt*:
lkd> dt nt!_*interrupt*
nt!_KINTERRUPT
nt!_KINTERRUPT_MODE
Then, you can use dt to format a specific structure as shown below:
lkd> dt nt!_kinterrupt
nt!_KINTERRUPT
+0x000 Type
:
+0x002 Size
:
+0x004 InterruptListEntry
+0x00c ServiceRoutine
:
+0x010 ServiceContext
:
+0x014 SpinLock
:
+0x018 TickCount
:
+0x01c ActualLock
:
+0x020 DispatchAddress :
+0x024 Vector
:
+0x028 Irql
:
+0x029 SynchronizeIrql :
+0x02a FloatingSave
:
+0x02b Connected
:
+0x02c Number
:
+0x02d ShareVector
:
+0x030 Mode
:
+0x034 ServiceCount
:
+0x038 DispatchCount
:
+0x03c DispatchCode
:
Copyrighted material.
Int2B
Int2B
: _LIST_ENTRY
Ptr32
Ptr32 Void
Uint4B
Uint4B
Ptr32 Uint4B
Ptr32
Uint4B
UChar
UChar
UChar
UChar
Char
UChar
_KINTERRUPT_MODE
Uint4B
Uint4B
[106] Uint4B
31
32
Microsoft Windows Internals, Fourth Edition
Note that dt does not show substructures (structures within structures) by default. To
recurse through substructures, use the “-r” switch. For example, using this switch to display the kernel interrupt object shows the format of the _LIST_ENTRY structure stored
at the InterruptListEntry field:
lkd> dt nt!_kinterrupt -r
nt!_KINTERRUPT
+0x000 Type
:
+0x002 Size
:
+0x004 InterruptListEntry
+0x000 Flink
+0x000 Flink
+0x004 Blink
+0x004 Blink
+0x000 Flink
+0x004 Blink
+0x00c ServiceRoutine
:
Int2B
Int2B
:
: Ptr32
: Ptr32
: Ptr32
: Ptr32
: Ptr32
: Ptr32
Ptr32
_LIST_ENTRY
_LIST_ENTRY
_LIST_ENTRY
_LIST_ENTRY
The Windows Debugging Tools help file explains how to set up and use the kernel debuggers.
Additional details on using the kernel debuggers that are aimed primarily at device driver
writers can be found in the Windows DDK documentation. There are also several useful
Knowledge Base articles on the kernel debugger. Search for “debugref” in the Windows
Knowledge Base (an online database of technical articles) on support.microsoft.com.
LiveKd Tool
LiveKd is a free tool from www.sysinternals.com that allows you to use the standard Microsoft kernel debuggers just described to examine the running system without requiring a second computer to act as the host (via a null modem cable). While the built-in support for local kernel
debugging works only on Windows XP and Windows Server 2003, LiveKd permits local kernel
debugging on Windows NT 4.0, Windows 2000, Windows XP, and Windows Server 2003.
You run LiveKd just as you would Windbg or Kd. LiveKd passes any command-line options
you specify to the debugger you select. By default, LiveKd runs the new command-line kernel
debugger (Kd). To run the GUI debugger (Windbg), specify the –w switch. To see the help
files on the switches for LiveKd, specify the –? switch.
LiveKd presents a simulated crash dump file to the debugger, so you can perform any operations in LiveKd that are supported on a crash dump. Because LiveKd is relying on physical
memory to back the simulated dump, the kernel debugger might run into situations in which
data structures are in the middle of being changed by the system and are inconsistent. Each
time the debugger is launched, it gets a snapshot of the system state, so if you want to refresh
the snapshot, quit the debugger (with the “q” command) and LiveKd will ask you whether
you want to start it again. If the debugger gets in a loop in printing output, press Ctrl+C to
interrupt the output, quit, and rerun it. If it hangs, press Ctrl+Break, which will terminate the
debugger process and ask you whether you want to run the debugger again.
Copyrighted material.
Chapter 1:
Concepts and Tools
33
SoftICE
Another debugging tool that doesn’t require two machines for live kernel debugging is a thirdparty kernel debugger called SoftICE, which you can buy from Compuware NuMega. (See
www.compuware.com for details.) SoftICE has essentially the same capabilities as the Windows
debugging tools, but it also supports stepping between user-mode and kernel-mode code. It
also supports the Microsoft kernel extension DLLs, so most of the commands we describe in
the book also work in SoftICE. Figure 1-8 shows the SoftICE user interface, which appears in
response to the SoftICE activation key (by default, Ctrl+D) as a window on the desktop of the
machine on which it’s running.
Figure 1-8
The SoftICE interface
Platform Software Development Kit (SDK)
The Platform SDK is part of the MSDN Professional and higher subscription levels, or it can be
downloaded for free from msdn.microsoft.com. It contains the documentation, C header files,
and libraries necessary to compile and link Windows applications. (Although Microsoft
Visual C++ comes with a copy of these header files, the versions contained in the Platform
SDK always match the latest version of the Windows operating systems, whereas the version
that comes with Visual C++ might be an older version that was current when Visual C++ was
released.) From an internals perspective, items of interest in the Platform SDK include the
Windows API header files (\Program Files\Microsoft SDK\Include) as well as several utilities
(Pfmon.exe, Pstat.exe, Pview.exe, Vadump.exe, and Winobj.exe). Some tools in the Platform
SDK also come with the Support Tools and Resource Kits. Finally, a few of these tools are also
shipped as example source code in both the Platform SDK and the MSDN Library.
Copyrighted material.
34
Microsoft Windows Internals, Fourth Edition
Device Driver Kit (DDK)
The Windows DDK is also shipped as part of the MSDN Professional (and higher) subscription levels, but unlike the Platform SDK, it is not available for free download (although you
can order the CD-ROM for a minimal cost). The Windows DDK documentation is included in
the MSDN Library.
Although the DDK is aimed at device-driver developers, it is an abundant source of Windowsinternals information. For example, while Chapter 9 describes the I/O system architecture,
driver model, and basic device driver data structures, it does not describe the individual kernel support functions in detail. The DDK documentation contains a comprehensive description of all the Windows kernel support functions and mechanisms used by device drivers in
both a tutorial and reference form.
Besides including the documentation, the DDK contains header files (in particular, Ntddk.h
and Wdm.h) that define key internal data structures and constants as well as interfaces to
many internal system routines. These files are useful when exploring Windows internal data
structures with the kernel debugger because although the general layout and content of these
structures are shown in this book, detailed field-level descriptions (such as size and data
types) are not. A number of these data structures (such as object dispatcher headers, wait
blocks, events, mutants, semaphores, and so on) are, however, fully described in the DDK.
So if you want to dig into the I/O system and driver model beyond what is presented in this
book, read the DDK documentation (especially the Kernel-Mode Driver Architecture Design
Guide and Reference manuals). Another excellent source is Programming the Microsoft
Windows Driver Model, Second Edition (Microsoft Press) by Walt Oney.
Sysinternals Tools
Many experiments in this book use freeware tools that you can download from www.sysinternals.com. Mark Russinovich, coauthor of this book, wrote most of these tools. The most popular tools include Process Explorer, Filemon, and Regmon. Note that many of these utilities
involve the installation and execution of kernel-mode device drivers and thus require administrator privileges.
Conclusion
In this chapter, you’ve been introduced to the key Windows technical concepts and terms that
will be used throughout the book. You’ve also had a glimpse of the many useful tools available
for digging into Windows internals. Now we’re ready to begin our exploration of the internal
design of the system, beginning with an overall view of the system architecture and its key
components.
Copyrighted material.
Chapter 2
System Architecture
Now that we’ve covered the terms, concepts, and tools you need to be familiar with, we’re
ready to start our exploration of the internal design goals and structure of the Microsoft Windows operating system. This chapter explains the overall architecture of the system—the key
components, how they interact with each other, and the context in which they run. To provide
a framework for understanding the internals of Windows, let’s first review the requirements
and goals that shaped the original design and specification of the system.
Requirements and Design Goals
The following requirements drove the specification of Windows NT back in 1989:
■
Provide a true 32-bit, preemptive, reentrant, virtual memory operating system
■
Run on multiple hardware architectures and platforms
■
Run and scale well on symmetric multiprocessing systems
■
Be a great distributed computing platform, both as a network client and as a server
■
Run most existing 16-bit MS-DOS and Microsoft Windows 3.1 applications
■
Meet government requirements for POSIX 1003.1 compliance
■
Meet government and industry requirements for operating system security
■
Be easily adaptable to the global market by supporting Unicode
To guide the thousands of decisions that had to be made to create a system that met these
requirements, the Windows NT design team adopted the following design goals at the beginning of the project:
■
Extensibility The code must be written to comfortably grow and change as market
requirements change.
■
Portability
■
Reliability and robustness The system should protect itself from both internal malfunction and external tampering. Applications should not be able to harm the operating system or other applications.
Copyrighted material.
The system must be able to run on multiple hardware architectures and
must be able to move with relative ease to new ones as market demands dictate.
35
36
Microsoft Windows Internals, Fourth Edition
■
Compatibility Although Windows NT should extend existing technology, its user interface and APIs should be compatible with older versions of Windows and with MS-DOS.
It should also interoperate well with other systems such as UNIX, OS/2, and NetWare.
■
Performance Within the constraints of the other design goals, the system should be as
fast and responsive as possible on each hardware platform.
As we explore the details of the internal structure and operation of Windows, you’ll see how
these original design goals and market requirements were woven successfully into the construction of the system. But before we start that exploration, let’s examine the overall design
model for Windows and compare it with other modern operating systems.
Operating System Model
In most multiuser operating systems, applications are separated from the operating system
itself—the operating system kernel code runs in a privileged processor mode (referred to as
kernel mode in this book), with access to system data and to the hardware; application code
runs in a nonprivileged processor mode (called user mode), with a limited set of interfaces
available, limited access to system data, and no direct access to hardware. When a user-mode
program calls a system service, the processor traps the call and then switches the calling
thread to kernel mode. When the system service completes, the operating system switches the
thread context back to user mode and allows the caller to continue.
Windows is similar to most UNIX systems in that it’s a monolithic operating system in the
sense that the bulk of the operating system and device driver code shares the same kernelmode protected memory space. This means that any operating system component or device
driver can potentially corrupt data being used by other operating system components.
Is Windows a Microkernel-Based System?
Although some claim it as such, Windows isn’t a microkernel-based operating system in
the classic definition of microkernels, where the principal operating system components (such as the memory manager, process manager, and I/O manager) run as separate processes in their own private address spaces, layered on a primitive set of services
the microkernel provides. For example, the Carnegie Mellon University Mach operating
system, a contemporary example of a microkernel architecture, implements a minimal
kernel that comprises thread scheduling, message passing, virtual memory, and device
drivers. Everything else, including various APIs, file systems, and networking, runs in
user mode. However, commercial implementations of the Mach microkernel operating
system typically run at least all file system, networking, and memory management code
in kernel mode. The reason is simple: the pure microkernel design is commercially
impractical because it’s too inefficient.
Copyrighted material.
Chapter 2:
System Architecture
37
Does the fact that so much of Windows runs in kernel mode mean that it’s more susceptible to crashes than a true microkernel operating system? Not at all. Consider the following scenario. Suppose the file system code of an operating system has a bug that
causes it to crash from time to time. In a traditional operating system, a bug in kernelmode code such as the memory manager or the file system would likely crash the entire
operating system. In a pure microkernel operating system, such components run in user
mode, so theoretically a bug would simply mean that the component’s process exits. But
in practical terms, the system would crash because recovering from the failure of such a
critical process would likely be impossible.
All these operating system components are, of course, fully protected from errant applications
because applications don’t have direct access to the code and data of the privileged part of the
operating system (although they can quickly call other kernel services). This protection is one
of the reasons that Windows has the reputation for being both robust and stable as an application server and as a workstation platform yet fast and nimble from the perspective of core
operating system services, such as virtual memory management, file I/O, networking, and file
and print sharing.
The kernel-mode components of Windows also embody basic object-oriented design principles. For example, they don’t in general reach into one another’s data structures to access
information maintained by individual components. Instead, they use formal interfaces to pass
parameters and access and/or modify data structures.
Despite its pervasive use of objects to represent shared system resources, Windows is not an
object-oriented system in the strict sense. Most of the operating system code is written in C for
portability and because C development tools are widely available. C doesn’t directly support
object-oriented constructs, such as dynamic binding of data types, polymorphic functions, or
class inheritance. Therefore, the C-based implementation of objects in Windows borrows
from, but doesn’t depend on, features of particular object-oriented languages.
Architecture Overview
With this brief overview of the design goals and packaging of Windows, let’s take a look at the
key system components that make up its architecture. A simplified version of this architecture
is shown in Figure 2-1. Keep in mind that this diagram is basic—it doesn’t show everything.
(For example, the networking components and the various types of device driver layering are
not shown.)
Copyrighted material.
38
Microsoft Windows Internals, Fourth Edition
System
support
processes
Service
processes
User
applications
Environment
subsystems
Subsystem DLLs
User mode
Kernel mode
Executive
Kernel
Device drivers
Windowing
and graphics
Hardware abstraction layer (HAL)
Figure 2-1
Simplified Windows architecture
In Figure 2-1, first notice the line dividing the user-mode and kernel-mode parts of the Windows operating system. The boxes above the line represent user-mode processes, and the
components below the line are kernel-mode operating system services. As mentioned in
Chapter 1, user-mode threads execute in a protected process address space (although while
they are executing in kernel mode, they have access to system space). Thus, system support
processes, service processes, user applications, and environment subsystems each have their
own private process address space.
The four basic types of user-mode processes are described as follows:
■
Fixed (or hardwired) system support processes, such as the logon process and the session
manager, that are not Windows services. (That is, they are not started by the service control manager. Chapter 4 describes services in detail.)
■
Service processes that host Windows services, such as the Task Scheduler and Spooler
services. Services generally have the requirement that they run independently of user
logons. Many Windows server applications, such as Microsoft SQL Server and Microsoft
Exchange Server, also include components that run as services.
■
User applications, which can be one of six types: Windows 32-bit, Windows 64-bit, Windows 3.1 16-bit, MS-DOS 16-bit, POSIX 32-bit, or OS/2 32-bit.
■
Environment subsystem server processes, which implement part of the support for the operating system environment, or personality presented to the user and programmer. Windows NT originally shipped with three environment subsystems: Windows, POSIX, and
OS/2. OS/2 was dropped as of Windows 2000. As of Windows XP, only the Windows
subsystem is shipped in the base product—an enhanced POSIX subsystem is available as
part of the free Services for Unix product.
In Figure 2-1, notice the “Subsystem DLLs” box below the “Service processes” and “User applications” boxes. Under Windows, user applications don’t call the native Windows operating system
services directly; rather, they go through one or more subsystem dynamic-link libraries (DLLs). The
Copyrighted material.
Chapter 2:
System Architecture
39
role of the subsystem DLLs is to translate a documented function into the appropriate internal
(and generally undocumented) Windows system service calls. This translation might or might
not involve sending a message to the environment subsystem process that is serving the user
application.
The kernel-mode components of Windows include the following:
■
The Windows executive contains the base operating system services, such as memory
management, process and thread management, security, I/O, networking, and interprocess communication.
■
The Windows kernel consists of low-level operating system functions, such as thread
scheduling, interrupt and exception dispatching, and multiprocessor synchronization.
It also provides a set of routines and basic objects that the rest of the executive uses to
implement higher-level constructs.
■
Device drivers include both hardware device drivers that translate user I/O function calls
into specific hardware device I/O requests as well as file system and network drivers.
■
The hardware abstraction layer (HAL) is a layer of code that isolates the kernel, device
drivers, and the rest of the Windows executive from platform-specific hardware differences (such as differences between motherboards).
■
The windowing and graphics system implements the graphical user interface (GUI) functions (better known as the Windows USER and GDI functions), such as dealing with
windows, user interface controls, and drawing.
Table 2-1 lists the filenames of the core Windows operating system components. (You’ll need
to know these filenames because we’ll be referring to some system files by name.) Each of
these components is covered in greater detail both later in this chapter and in the chapters
that follow.
Table 2-1
Core Windows System Files
Filename
Components
Ntoskrnl.exe
Executive and kernel
Ntkrnlpa.exe (32-bit systems only)
Executive and kernel with support for Physical Address
Extension (PAE), which allows addressing of up to 64 GB
of physical memory
Hal.dll
Hardware abstraction layer
Win32k.sys
Kernel-mode part of the Windows subsystem
Ntdll.dll
Internal support functions and system service dispatch
stubs to executive functions
Kernel32.dll, Advapi32.dll, User32.dll,
Gdi32.dll
Core Windows subsystem DLLs
Before we dig into the details of these system components, though, let’s examine how Windows achieves portability across multiple hardware architectures.
Copyrighted material.
40
Microsoft Windows Internals, Fourth Edition
Portability
Windows was designed to run on a variety of hardware architectures, including Intel-based CISC
systems as well as RISC systems. The initial release of Windows NT supported the x86 and MIPS
architecture. Support for the Digital Equipment Corporation (which was bought by Compaq,
who later merged with Hewlett Packard) Alpha AXP was added shortly thereafter. (Although
Alpha AXP was a 64-bit processor, Windows NT ran in 32-bit mode. During the development of
Windows 2000, a native 64-bit version was running on Alpha AXP, but this never was released.)
Support for a fourth processor architecture, the Motorola PowerPC, was added in Windows NT
3.51. Because of changing market demands, however, support for the MIPS and PowerPC architectures was dropped before development began on Windows 2000. Later, Compaq withdrew
support for the Alpha AXP architecture, resulting in Windows 2000 being supported only on the
x86 architecture. The most recent releases, Windows XP and Windows Server 2003, add support
for three 64-bit processor families: the Intel Itanium IA-64 family, the AMD x86-64 family, and the
Intel 64-bit Extension Technology (EM64T) for x86 (which is compatible with the AMD x86-64
architecture, although there are slight differences in instructions supported). The latter two processor families are called 64-bit extended systems and in this book are referred to as x64. The most
recent releases, Windows XP and Windows Server 2003, add support for three 64-bit processor
families: the Intel Itanium IA-64 family, the AMD64 family, and the Intel 64-bit Extension Technology (EM64T) for x86 (which is compatible with the AMD64 architecture, although there are
slight differences in instructions supported). (How Windows runs 32-bit applications on 64-bit
Windows is explained in Chapter 3.)
Windows achieves portability across hardware architectures and platforms in two primary ways:
■
Windows has a layered design, with low-level portions of the system that are processorarchitecture-specific or platform-specific isolated into separate modules so that upper
layers of the system can be shielded from the differences between architectures and
among hardware platforms. The two key components that provide operating system
portability are the kernel (contained in Ntoskrnl.exe) and the hardware abstraction
layer (or HAL, contained in Hal.dll). Both these components are described in more
detail later in this chapter. Functions that are architecture-specific (such as thread context switching and trap dispatching) are implemented in the kernel. Functions that can
differ among systems within the same architecture (for example, different motherboards) are implemented in the HAL. The only other component with a significant
amount of architecture-specific code is the memory manager, but even that is a small
amount compared to the system as a whole.
■
The vast majority of Windows is written in C, with some portions in C++. Assembly language is used only for those parts of the operating system that need to communicate
directly with system hardware (such as the interrupt trap handler) or that are extremely
performance-sensitive (such as context switching). Assembly language code exists not
only in the kernel and the HAL but also in a few other places within the core operating
system (such as the routines that implement interlocked instructions as well as one
module in the local procedure call facility), in the kernel-mode part of the Windows
Copyrighted material.
Chapter 2:
System Architecture
41
subsystem, and even in some user-mode libraries, such as the process startup code in
Ntdll.dll (a system library explained later in this chapter).
Symmetric Multiprocessing
Multitasking is the operating system technique for sharing a single processor among multiple
threads of execution. When a computer has more than one processor, however, it can execute
two threads simultaneously. Thus, whereas a multitasking operating system only appears to
execute multiple threads at the same time, a multiprocessing operating system actually does
it, executing one thread on each of its processors.
As mentioned at the beginning of this chapter, one of the key design goals for Windows was
that it had to run well on multiprocessor computer systems. Windows is a symmetric multiprocessing (SMP) operating system. There is no master processor—the operating system as well as
user threads can be scheduled to run on any processor. Also, all the processors share just one
memory space. This model contrasts with asymmetric multiprocessing (ASMP), in which the
operating system typically selects one processor to execute operating system kernel code
while other processors run only user code. The differences in the two multiprocessing models
are illustrated in Figure 2-2.
Symmetric
Asymmetric
Memory
Memory
Processor A
Processor B
Processor A
User
thread
Operating
system
User
thread
User
thread
User
thread
User
thread
User
thread
Operating
system
Operating
system
I/O devices
Figure 2-2
Processor B
User
thread
I/O devices
Symmetric vs. asymmetric multiprocessing
Windows XP and Windows Server 2003 support two new types of multiprocessor systems:
hyperthreading and NUMA (non-uniform memory architecture). These are briefly mentioned
in the following paragraphs. (For a complete detailed description of the scheduling support
for these systems, see the thread scheduling section in Chapter 6.)
Copyrighted material.
42
Microsoft Windows Internals, Fourth Edition
Hyperthreading is a technology introduced by Intel that provides many logical processors on
one physical processor. Each logical processor has its CPU state, but the execution engine and
onboard cache is shared. This permits one logical CPU to make progress while the other logical CPUs are busy (such as performing interrupt processing work, which prevents threads
from running on that logical processor). The scheduling algorithms as of Windows XP have
been enhanced to make optimal use of multiprocessor hyperthreaded machines, such as by
scheduling threads on an idle physical processor versus choosing an idle logical processor on
a physical processor whose other logical processors are busy.
In non-uniform memory architecture NUMA systems, processors are grouped in smaller units
called nodes. Each node has its own processors and memory and is connected to the larger
system through a cache-coherent interconnect bus. Windows on a NUMA system still runs as
an SMP system, in that all processors have access to all memory—it’s just that node-local memory is faster to reference than memory attached to other nodes. The system attempts to
improve performance by scheduling threads on processors that are in the same node as the
memory being used. It attempts to satisfy memory-allocation requests from within the node,
but will allocate memory from other nodes if necessary.
Although Windows was originally designed to support up to 32 processors, nothing inherent
in the multiprocessor design limits the number of processors to 32—that number is simply an
obvious and convenient limit because 32 processors can easily be represented as a bit mask
using a native 32-bit data type. In fact, the 64-bit versions of Windows support up to 64 processors, because the native size of a word on a 64-bit machine is 64 bits.
The actual number of supported processors depends on the edition of Windows being used.
(See tables 2-3 and 2-4.) This number is stored in the registry value HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\LicensedProcessors. (Keep in mind that tampering
with that data is a violation of the software license and modifying the registry to allow use of
more processors involves more than just changing this value.)
For performance reasons, there are separate uniprocessor and multiprocessor versions of the
kernel and HAL (and in the case of Windows 2000, a few other key system files). On Windows 2000, six system files (as explained in the following Note) are different on a multiprocessor system than on a uniprocessor system; on 32-bit Windows XP and Windows Server
2003 systems, only three are different. (See Table 2-2.) On 64-bit Windows systems, there is
no PAE kernel, so only the kernel and HAL vary from uniprocessor to multiprocessor systems.
At installation time, the appropriate files are selected and copied to the local \Windows\
System32 directory. To determine which files were copied, see the file \Windows\
Repair\Setup.log, which itemizes all the files that were copied to the local system disk and
where they came from off the distribution media.
Copyrighted material.
Chapter 2:
Table 2-2
System Architecture
43
Multiprocessor-Specific vs. Uniprocessor-Specific System Files
Name of File on System
Disk
Name of Uniprocessor Version on Name of Multiprocessor Version
Distribution Media
on Distribution Media
Ntoskrnl.exe
Ntoskrnl.exe
Ntkrnlmp.exe
Ntkrnlpa.exe (PAE kernel;
32-bit systems only)
Ntkrnlpa.exe in \Windows\<arch>\Driver.cab
Ntkrpamp.exe in \Windows\<arch>\Driver.cab
Hal.dll
Depends on system type (See the
list of HALs in Table 2-6.)
Depends on system type (See the
list of HALs in Table 2-6.)
Win32k.sys
\I386\UNIPROC\Win32k.sys
Win32k.sys in \I386\Driver.cab
Ntdll.dll
\I386\UNIPROC\Ntdll.dll
\I386\Ntdll.dll
Kernel32.dll
\I386\UNIPROC\Kernel32.dll
\I386\Kernel32.dll
Windows 2000 only
Note If you look in the \I386\UNIPROC folder in the Windows 2000 distribution tree, you’ll
see a file named Winsrv.dll. Although this file exists in a folder named UNIPROC, implying that
there is a uniprocessor version, in fact there is only one version of this image for both multiprocessor and uniprocessor systems. This folder has been removed in Windows XP and Windows Server 2003.
EXPERIMENT: Looking at Multiprocessor-Specific Support Files
on Windows 2000
You can see the files that are different for a 32-bit Windows 2000 multiprocessor system
by looking at the driver details for the Computer in Device Manager:
1. Open the System properties (either by selecting System from Control Panel or by
right-clicking the My Computer icon on your desktop and selecting Properties).
2. Click the Hardware tab.
3. Click Device Manager.
4. Expand the Computer object.
5. Double-click the child node underneath Computer.
6. Click the Driver tab.
7. Click Driver Details.
Copyrighted material.
44
Microsoft Windows Internals, Fourth Edition
You should see the following dialog box for a multiprocessor system:
The reason for having uniprocessor versions of these key system files is performance—multiprocessor synchronization is inherently more complex and time consuming than the use of a
single processor, so by having special uniprocessor versions of the key system files, this overhead is avoided on uniprocessor systems (which constitute the vast majority of systems running Windows).
Interestingly, although the uniprocessor and multiprocessor versions of Ntoskrnl are generated using conditionally compiled source code, the uniprocessor versions of Ntdll.dll and
Kernel32.dll for Windows 2000 are created by patching the x86 LOCK and UNLOCK instructions, which are used to synchronize multiple threads with no-operation (NOP) instructions
(which do nothing).
The rest of the system files that make up Windows (including all utilities, libraries, and device
drivers) have the same version on both uniprocessor and multiprocessor systems (that is,
they handle multiprocessor synchronization issues correctly). You should use this approach
on any software you build, whether it is a Windows application or a device driver—keep multiprocessor synchronization issues in mind when you design your software, and test the software on both uniprocessor and multiprocessor systems.
Copyrighted material.
Chapter 2:
System Architecture
EXPERIMENT: Checking Which Ntoskrnl Version You’re Running
In Windows 2000 and later, there is no utility to show which version of Ntoskrnl you are
running. However, an Event Log entry is written each time the system boots that does
record the type of kernel image that loaded (uniprocessor vs. multiprocessor and free vs.
checked), as shown in the following screen shot. (From the Start menu select Programs/
Administrative Tools/Event Viewer, select System Log, and double-click an Event Log
entry with an Event ID of 6009, indicating the entry was written at the system start.)
This Event Log entry doesn’t indicate whether you booted the PAE version of the kernel
image that supports more than 4 GB of physical memory (Ntkrnlpa.exe). However, you
can tell if you booted the PAE kernel by looking at the registry value HKLM\SYSTEM\CurrentControlSet\Control\SystemStartOptions. Also, if you boot the PAE kernel, the registry value HKLM\SYSTEM\CurrentControlSet\Control\Session
Manager\Memory Management\PhysicalAddressExtension is set to 1.
You can also determine whether you installed the multiprocessor version of Ntoskrnl
(or Ntkrnlpa) by examining the file properties: run Windows Explorer, right-click on
Ntoskrnl.exe in your \Windows\System32 folder, and select Properties. Then click on
Copyrighted material.
45
46
Microsoft Windows Internals, Fourth Edition
the Version tab, and select the Original Filename property—if you’re running the multiprocessor version, you’ll see the following dialog box:
Finally, as mentioned earlier, you can see exactly which kernel image and HAL were
selected at installation time by looking at the file \Windows\Repair\Setup.log.
Scalability
One of the key issues with multiprocessor systems is scalability. To run correctly on an SMP
system, operating system code must adhere to strict guidelines and rules. Resource contention and other performance issues are more complicated in multiprocessing systems than in
uniprocessor systems and must be accounted for in the system’s design. Windows incorporates several features that are crucial to its success as a multiprocessor operating system:
■
The ability to run operating system code on any available processor and on multiple
processors at the same time
■
Multiple threads of execution within a single process, each of which can execute simultaneously on different processors
■
Fine-grained synchronization within the kernel (such as spinlocks, queued spinlocks,
and pushlocks, described in Chapter 3) as well as within device drivers and server processes, which allows more components to run concurrently on multiple processors
■
Programming mechanisms such as I/O completion ports (described in Chapter 9) that
facilitate the efficient implementation of multithreaded server processes that can scale
well on multiprocessor systems.
Copyrighted material.
Chapter 2:
System Architecture
47
The scalability of the Windows kernel has evolved over time. For example, Windows Server
2003 has per-CPU scheduling queues, which permits thread scheduling decisions to occur in
parallel on multiple machines. Multiprocessor thread scheduling details are covered in Chapter 6. Further details on multiprocessor synchronization can be found in Chapter 3.
Differences Between Client and Server Versions
Windows ships in both client and server retail packages. In Windows 2000, the client version
is called Windows 2000 Professional. There are three Windows 2000 server versions: Windows 2000 Server, Advanced Server, and Datacenter Server.
There are six client versions of Windows XP: Windows XP Home Edition, Windows XP Professional, Windows XP Starter Edition, Windows XP Tablet PC Edition, Windows XP Media
Center Edition, and Windows XP Embedded. The latter three are supersets of Windows XP
Professional and are not described in detail in this book because they are all built on the same
core operating system as Windows XP Professional.
There are six variants of Windows Server 2003: Windows Server 2003 Web Edition, Standard
Edition, Small Business Server, Storage Server, Enterprise Edition, and Datacenter Edition.
These versions differ by:
■
The number of processors supported
■
The amount of physical memory supported
■
The number of concurrent network connections supported (For example, a maximum of
10 concurrent connections are allowed to the file and print services in the client version.)
■
Layered services that come with Server editions that don’t come with the Professional edition (for example, directory services, clustering, and multiuser Terminal Services support)
Table 2-3 summarizes the differences in memory and processor support for Windows 2000.
Table 2-4 lists the same information for Windows XP and Windows Server 2003. For a
detailed comparison chart of the different editions of Windows Server 2003, see http://
www.microsoft.com/windowsserver2003/evaluation/features/compareeditions.mspx.
Table 2-3
Differences Between Windows 2000 Professional and Server
Edition
Number of Processors
Supported
Physical Memory
Supported
Windows 2000 Professional
2
4 GB
Windows 2000 Server
4
4 GB
Windows 2000 Advanced Server
8
8 GB
Windows 2000 Datacenter Server
32
64 GB
Copyrighted material.
48
Microsoft Windows Internals, Fourth Edition
Table 2-4 Differences Between Windows XP and Windows Server 2003
Number of
Processors
Supported
(32-bit
edition)
Physical
Memory
Supported
(32-bit
edition)
Number of
Processors
Supported
(64-bit
edition)
Physical
Memory
Supported
(Itanium
editions)
Physical
Memory
Supported
(x64 editions)
Windows XP
Home Edition
1
4 GB
Not available
Not available
Not available
Windows XP
Professional
2
4 GB
2
16 GB
16 GB
Windows
Server 2003
Web Edition
2
2 GB
Not available
Not available
Not available
Windows
2
Server 2003
Small Business
Server
2 GB
Not available
Not available
Not available
Windows
Server 2003
Standard Edition
4 GB
Not available
Not available
Not available
32 GB
8
64 GB
64 GB
128 GB on
x64; 64 GB on
x86
64
512 GB
(1024 GB in
SP1)
Not available
4
Windows
8
Server 2003
Enterprise Edition
Windows
Server 2003
Datacenter
Edition
32
Although there are several client and server retail packages of the Windows operating system,
they share a common set of core system files, including the kernel image, Ntoskrnl.exe (and
the PAE version, Ntkrnlpa.exe); the HAL libraries; the device drivers; and the base system utilities and DLLs. These files are identical for all editions of Windows 2000.
Note
Windows XP was the first client release of the Windows NT code base to ship without
corresponding server versions. Instead, development continued on what became Windows
Server 2003 for over a year after the release of Windows XP. Therefore, the core system files are
not identical for Windows XP and Windows Server 2003. Even so, the differences are not major
(and in many cases, components were unchanged).
So if the kernel image for Windows 2000 Professional and Windows 2000 Server are identical
(and similar for Windows XP and Windows Server 2003), how does the system know which
edition is booted? By querying the registry values ProductType and ProductSuite under the
HKLM\ SYSTEM\CurrentControlSet\Control\ProductOptions key. ProductType is used to
Copyrighted material.
Chapter 2:
System Architecture
49
distinguish whether the system is a client system or a server system (of any flavor). The valid
values are listed in Table 2-5. The result is stored in the system global variable MmProductType,
which can be queried from a device driver using the kernel-mode support function MmIsThisAnNtAsSystem, documented in the Windows DDK.
Table 2-5
ProductType Registry Values
Edition of Windows
Value of ProductType
Windows 2000 Professional, Windows XP Professional,
Windows XP Home Edition
WinNT
Windows Server (domain controller)
LanmanNT
Windows Server (server only)
ServerNT
A different registry value, ProductSuite, distinguishes the various flavors of Windows Server
systems (Standard, Enterprise, Datacenter, and so on) as well as distinguishing a Windows XP
Home from a Windows XP Professional system.
If user programs need to determine which edition of Windows is running, they can call the
Windows VerifyVersionInfo function, documented in the Platform SDK. Device drivers can call
the kernel-mode function RtlGetVersion, documented in the Windows DDK.
So if the core files are essentially the same for the client and server versions, how do the systems differ in operation? In short, Server systems are by default optimized for system throughput as high-performance application servers, whereas the client version, although it has server
capabilities, is optimized for response time for interactive desktop use. For example, based on
the product type, several resource allocation decisions are made differently at system boot
time, such as the size and number of operating system heaps (or pools), the number of internal system worker threads, and the size of the system data cache. Also, run-time policy decisions, such as the way the memory manager trades off system and process memory demands,
differ between the server and client editions. Even some thread scheduling details have different default behavior in the two families (the default length of the time slice, or thread quantum—see Chapter 6 for details). Where there are significant operational differences in the two
products, these are highlighted in the pertinent chapters throughout the rest of this book.
Unless otherwise noted, everything in this book applies to both the client and server versions.
Checked Build
There is a special debug version of Windows 2000 Professional, Windows XP Professional,
and Windows Server 2003 called the checked build (available only with the MSDN Professional or higher subscription). It is a recompilation of the Windows source code with a compile-time flag defined called “DBG” (to cause compile time conditional debugging and tracing
code to be included). Also, to make it easier to understand the machine code, the post-processing of the Windows binaries to optimize code layout for faster execution is not performed.
(See the section “Performance-Optimized Code” in the Debugging Tools help file.)
Copyrighted material.
50
Microsoft Windows Internals, Fourth Edition
The checked build is provided primarily to aid device driver developers because it performs
more stringent error checking on kernel-mode functions called by device drivers or other system code. For example, if a driver (or some other piece of kernel-mode code) makes an invalid
call to a system function that is checking parameters (such as acquiring a spinlock at the wrong
interrupt level), the system will stop execution when the problem is detected rather than allow
some data structure to be corrupted and the system to possibly crash at a later time.
EXPERIMENT: Determining If You Are Running the Checked Build
There is no built-in tool to display whether you are running the checked build or the
retail build (called the free build). However, this information is available through the
“Debug” property of the Windows Management Instrumentation (WMI)
Win32_OperatingSystem class. The following sample Visual Basic script displays this
property:
strComputer = "."
Set objWMIService = GetObject("winmgmts:" _
& "{impersonationLevel=impersonate}!\\" & strComputer & "\root\cimv2")
Set colOperatingSystems = objWMIService.ExecQuery _
("SELECT * FROM Win32_OperatingSystem"S)
For Each objOperatingSystem in colOperatingSystems
Wscript.Echo "Caption: " & objOperatingSystem.Caption
Wscript.Echo "Debug: " & objOperatingSystem.Debug
Wscript.Echo "Version: " & objOperatingSystem.Version
Next
To try this, type in the preceding script and save it as file. The following is the output
from running the script:
C:\>cscript osversion.vbs
Microsoft (R) Windows Script Host Version 5.6
Copyright (C) Microsoft Corporation 1996-2001. All rights reserved.
Caption: Microsoft Windows XP Professional
Debug: False
Version: 5.1.2600
This system is not running the checked build, as the Debug flag shown here says False.
Much of the additional code in the checked-build binaries is a result of using the ASSERT
macro, which is defined in the DDK header file Ntddk.h and documented in the DDK documentation. This macro tests a condition (such as the validity of a data structure or parameter),
and if the expression evaluates to FALSE, the macro calls the kernel-mode function RtlAssert,
which calls DbgPrint to send the text of the debug message to a debug message buffer. If a kernel debugger is attached, this message is displayed automatically followed by a prompt asking
the user what to do about the assertion failure (breakpoint, ignore, terminate process, or terminate thread). If the system wasn’t booted with the kernel debugger (using the /DEBUG
Copyrighted material.
Chapter 2:
System Architecture
51
switch in Boot.ini) and no kernel debugger is currently attached, failure of an ASSERT test will
crash the system. For a list of ASSERT checks made by some of the kernel support routines,
see the section “Checked Build ASSERTs” in the Windows DDK documentation.
Note
On the checked build, if you compare Ntoskrnl.exe and Ntkrnlmp.exe or Ntkrnlpa.exe
and Ntkrpamp.exe, you’ll find that they are identical—they are all multiprocessor versions of
the same files. In other words, there is no debug uniprocessor version of the kernel images provided with the checked build.
The checked build is also useful for system administrators because of the additional detailed
informational tracing that can be enabled for certain components. (For detailed instructions,
see the Microsoft Knowledge Base Article number 314743 entitled HOWTO: Enable Verbose
Debug Tracing in Various Drivers and Subsystems.) This information output is sent to an internal
debug message buffer using the DbgPrint function referred to earlier. To view the debug messages, you can either attach a kernel debugger to the target system (which requires booting
the target system in debugging mode), use the !dbgprint command while performing local kernel debugging, or use the Dbgview.exe tool from www.sysinternals.com.
You don’t have to install the entire checked build to take advantage of the debug version of the
operating system. You can just copy the checked version of the kernel image (Ntoskrnl.exe)
and the appropriate HAL (Hal.dll) to a normal retail installation. The advantage of this
approach is that device drivers and other kernel code get the rigorous checking of the checked
build without having to run the slower debug versions of all components in the system. For
detailed instructions on how to do this, see the section “Installing Just the Checked Operating
System and HAL” in the Windows DDK documentation. Because Microsoft doesn’t supply a
checked build version of Windows 2000 Server, you can also apply this technique to run the
checked version of the kernel on a Windows 2000 Server system.
Finally, the checked build can also be useful for testing user-mode code only because the timing of the system is different. (This is because of the additional checking taking place within
the kernel and the fact that the components are compiled without optimizations.) Often, multithreaded synchronization bugs are related to specific timing conditions. By running your
tests on a system running the checked build (or at least the checked kernel and HAL), the fact
that the timing of the whole system is different might cause latent timing bugs to surface that
do not occur on a normal retail system.
Key System Components
Now that we’ve looked at the high-level architecture of Windows, let’s delve deeper into the
internal structure and the role each key operating system component plays. Figure 2-3 is a
more detailed and complete diagram of the core Windows system architecture and components than was shown earlier in the chapter (in Figure 2-2). Note that it still does not show all
components (networking in particular, which is explained in Chapter 13).
Copyrighted material.
52
Microsoft Windows Internals, Fourth Edition
The following sections elaborate on each major element of this diagram. Chapter 3 explains
the primary control mechanisms the system uses (such as the object manager, interrupts, and
so forth). Chapter 5 describes the process of starting and shutting down Windows, and Chapter 4 details management mechanisms such as the registry, service processes, and Windows
Management Instrumentation (WMI). Then the remaining chapters explore in even more
detail the internal structure and operation of key areas such as processes and threads, memory management, security, the I/O manager, storage management, the cache manager, the
Windows file system (NTFS), and networking.
System Processes
Services
Service
control mgr.
Environment
Subsystems
Windows
SvcHost.exe
LSASS
Task Manager
WinMgt.exe
OS/2
Explorer
SpoolSv.exe
Winlogon
Session
manager
Applications
User
application
Services.exe
POSIX
Windows DLLs
Subsystem DLLs
NTDLL.DLL
User mode
Kernel mode
System
threads
System Service Dispatcher
(Kernel mode callable interfaces)
Kernel
Hardware Abstraction Layer (HAL)
Hardware interfaces (buses, I/O devices, interrupts,
interval timers, DMA, memory cache control, etc.)
Figure 2-3
Copyrighted material.
Windows architecture
Local
Procedure
Call
Configuration
Mgr (registry)
Processes
&
Threads
VIrtual
Memory
Plug and
Play Mgr
Object
Mgr
File
System
Cache
Device &
File Sys.
Drivers
Security
Reference
Monitor
I/O Mgr
Windows
USER,
GDI
Graphics
drivers
Chapter 2:
System Architecture
53
Environment Subsystems and Subsystem DLLs
As shown in Figure 2-3, Windows originally had three environment subsystems: OS/2,
POSIX, and Windows. As stated earlier, the OS/2 subsystem was removed in Windows 2000.
Although the basic POSIX subsystem that originally shipped with Windows no longer ships
with the system as of Windows XP, a greatly enhanced version is available for free as part of
the Services for UNIX product.
As we’ll explain shortly, of the three, the Windows subsystem is special in that Windows can’t
run without it. (It owns the keyboard, mouse, and display, and it is required to be present even
on server systems with no interactive users logged in.) In fact, the other two subsystems are
configured to start on demand, whereas the Windows subsystem must always be running.
The subsystem startup information is stored under the registry key HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\SubSystems. Figure 2-4 shows the values under this key.
Figure 2-4
Registry Editor showing Windows startup information
The Required value lists the subsystems that load when the system boots. The value has two
strings: Windows and Debug. The Windows value contains the file specification of the Windows subsystem, Csrss.exe, which stands for Client/Server Run-Time Subsystem. (See the
Note later in this section.) Debug is blank (because it’s used for internal testing) and therefore
does nothing. The Optional value indicates that the OS/2 and POSIX subsystems will be
started on demand. The registry value Kmode contains the filename of the kernel-mode portion of the Windows subsystem, Win32k.sys (explained later in this chapter).
The role of an environment subsystem is to expose some subset of the base Windows executive system services to application programs. Each subsystem can provide access to different
subsets of the native services in Windows. That means that some things can be done from an
application built on one subsystem that can’t be done by an application built on another subsystem. For example, a Windows application can’t use the POSIX fork function.
Each executable image (.exe) is bound to one and only one subsystem. When an image is run,
the process creation code examines the subsystem type code in the image header so that it can
notify the proper subsystem of the new process. This type code is specified with the /SUBSYSTEM qualifier of the link command in Microsoft Visual C++ and can be viewed with the
Exetype tool in the Windows resource kits.
Copyrighted material.
54
Microsoft Windows Internals, Fourth Edition
Note
As a historical note, the reason the Windows subsystem process is called Csrss.exe is
that in the original design of Windows NT, all the subsystems were going to execute as threads
inside a single systemwide environment subsystem process. When the POSIX and OS/2 subsystems were removed and put in their own processes, the filename for the Windows subsystem process wasn’t changed.
Function calls can’t be mixed between subsystems. In other words, a POSIX application can
call only services exported by the POSIX subsystem, and a Windows application can call only
services exported by the Windows subsystem. As you’ll see later, this restriction is one reason
why the original POSIX subsystem, which implements a very limited set of functions (only
POSIX 1003.1), wasn’t a useful environment for porting UNIX applications.
As mentioned earlier, user applications don’t call Windows system services directly. Instead,
they go through one or more subsystem DLLs. These libraries export the documented interface that the programs linked to that subsystem can call. For example, the Windows subsystem DLLs (such as Kernel32.dll, Advapi32.dll, User32.dll, and Gdi32.dll) implement the
Windows API functions. The POSIX subsystem DLL (Psxdll.dll) implements the POSIX API
functions.
EXPERIMENT: Viewing the Image Subsystem Type
You can see the image subsystem type by using either the Exetype tool in the Windows
resource kits or the Dependency Walker tool (Depends.exe) in the Windows Support
Tools and Platform SDK. For example, notice the image types for two different Windows
images, Notepad.exe (the simple text editor) and Cmd.exe (the Windows command
prompt):
C:\>exetype \Windows\system32\notepad.exe
File "\Windows\system32\notepad.exe" is of the following type:
Windows NT
32 bit machine
Built for the Intel 80386 processor
Runs under the Windows GUI subsystem
C:\>exetype \Windows\system32\cmd.exe
File "\Windows\system32\cmd.exe" is of the following type:
Windows NT
32 bit machine
Built for the Intel 80386 processor
Runs under the Windows character-based subsystem
This shows that Notepad is a GUI program while Cmd is a console or character-based
program. And although the output of the Exetype tool implies there are two different
subsystems for GUI and character-based programs, there is just one Windows subsystem. Also, Windows isn’t supported on the Intel 386 processor (or the 486 for that
matter)—the text output by the Exetype program hasn’t been updated.
Copyrighted material.
Chapter 2:
System Architecture
55
When an application calls a function in a subsystem DLL, one of three things can occur:
■
The function is entirely implemented in user mode inside the subsystem DLL. In other
words, no message is sent to the environment subsystem process, and no Windows
executive system services are called. The function is performed in user mode, and the
results are returned to the caller. Examples of such functions include GetCurrentProcess
(which always returns -1, a value that is defined to refer to the current process in all process-related functions) and GetCurrentProcessId. (The process ID doesn’t change for a
running process, so this ID is retrieved from a cached location, thus avoiding the need to
call into the kernel.)
■
The function requires one or more calls to the Windows executive. For example, the
Windows ReadFile and WriteFile functions involve calling the underlying internal (and
undocumented) Windows I/O system services NtReadFile and NtWriteFile, respectively.
■
The function requires some work to be done in the environment subsystem process.
(The environment subsystem processes, running in user mode, are responsible for
maintaining the state of the client applications running under their control.) In this
case, a client/server request is made to the environment subsystem via a message sent to
the subsystem to perform some operation. The subsystem DLL then waits for a reply
before returning to the caller.
Some functions can be a combination of the second and third items just listed, such as the
Windows CreateProcess and CreateThread functions.
Although Windows was designed to support multiple, independent environment subsystems,
from a practical perspective, having each subsystem implement all the code to handle windowing and display I/O would result in a large amount of duplication of system functions
that, ultimately, would have negatively affected both system size and performance. Because
Windows was the primary subsystem, the Windows designers decided to locate these basic
functions there and have the other subsystems call on the Windows subsystem to perform
display I/O. Thus, the POSIX and OS/2 subsystems call services in the Windows subsystem
to perform display I/O. (In fact, if you examine the subsystem type for these images, you’ll see
that they are Windows executables.)
Let’s take a closer look at each of the environment subsystems.
Windows Subsystem
The Windows subsystem consists of the following major components:
■
Copyrighted material.
The environment subsystem process (Csrss.exe) contains support for:
❑
Console (text) windows
❑
Creating and deleting processes and threads
❑
Portions of the support for 16-bit virtual DOS machine (VDM) processes
56
Microsoft Windows Internals, Fourth Edition
❑
■
Other miscellaneous functions, such as GetTempFile, DefineDosDevice, ExitWindowsEx, and several natural language support functions
The kernel-mode device driver (Win32k.sys) contains:
❑
The window manager, which controls window displays; manages screen output;
collects input from keyboard, mouse, and other devices; and passes user messages
to applications.
❑
The Graphics Device Interface (GDI), which is a library of functions for graphics
output devices. It includes functions for line, text, and figure drawing and for
graphics manipulation.
■
Subsystem DLLs (such as Kernel32.dll, Advapi32.dll, User32.dll, and Gdi32.dll) translate documented Windows API functions into the appropriate and mostly undocumented kernel-mode system service calls to Ntoskrnl.exe and Win32k.sys.
■
Graphics device drivers are hardware-dependent graphics display drivers, printer drivers, and video miniport drivers.
Applications call the standard USER functions to create user interface controls, such as windows and buttons, on the display. The window manager communicates these requests to the
GDI, which passes them to the graphics device drivers, where they are formatted for the display device. A display driver is paired with a video miniport driver to complete video display
support.
The GDI provides a set of standard two-dimensional functions that let applications communicate with graphics devices without knowing anything about the devices. GDI functions mediate between applications and graphics devices such as display drivers and printer drivers. The
GDI interprets application requests for graphic output and sends the requests to graphics display drivers. It also provides a standard interface for applications to use varying graphics output devices. This interface enables application code to be independent of the hardware
devices and their drivers. The GDI tailors its messages to the capabilities of the device, often
dividing the request into manageable parts. For example, some devices can understand directions to draw an ellipse; others require the GDI to interpret the command as a series of pixels
placed at certain coordinates. For more information about the graphics and video driver architecture, see the “Design Guide” section of the book Graphics Drivers in the Windows DDK.
Prior to Windows NT 4, the window manager and graphics services were part of the usermode Windows subsystem process. In Windows NT 4, the bulk of the windowing and graphics code was moved from running in the context of the Windows subsystem process to a set
of callable services running in kernel mode (in the file Win32k.sys). The primary reason for
Copyrighted material.
Chapter 2:
System Architecture
57
this shift was to improve overall system performance. Having a separate server process that
contains the Windows graphics subsystem required multiple thread and process context
switches, which consumed considerable CPU cycles and memory resources even though the
original design was highly optimized.
For example, for each thread on the client side there was a dedicated, paired server thread in the
Windows subsystem process waiting on the client thread for requests. A special interprocess
communication facility called fast LPC was used to send messages between these threads. Unlike
normal thread context switches, transitions between paired threads via fast LPC don’t cause a
rescheduling event in the kernel, thereby enabling the server thread to run for the remaining
time slice of the client thread before having to take its turn in the kernel’s preemptive thread
scheduler. Moreover, shared memory buffers were used to allow fast passing of large data structures, such as bitmaps, and clients had direct but read-only access to key server data structures
to minimize the need for thread/process transitions between clients and the Windows server.
Also, GDI operations were (and still are) batched. Batching means that a series of graphics calls
by a Windows application aren’t “pushed” over to the server and drawn on the output device
until a GDI batching queue is filled. You can set the size of the queue by using the Windows
GdiSetBatchLimit function, and you can flush the queue at any time with GdiFlush. Conversely,
read-only properties and data structures of GDI, once they were obtained from the Windows
subsystem process, were cached on the client side for fast subsequent access.
Despite these optimizations, however, the overall system performance was still not adequate
for graphics-intensive applications. The obvious solution was to eliminate the need for the
additional threads and resulting context switches by moving the windowing and graphics system into kernel mode. Also, once applications have called into the window manager and the
GDI, those subsystems can access other Windows executive components directly without the
cost of user-mode or kernel-mode transitions. This direct access is especially important in the
case of the GDI calling through video drivers, a process that involves interaction with video
hardware at high frequencies and high bandwidths.
So, what remains in the user-mode process part of the Windows subsystem? All the drawing
and updating for console or text windows are handled by it because console applications have
no notion of repainting a window. It’s easy to see this activity—simply open a command
prompt and drag another window over it, and you’ll see the Windows subsystem consuming
CPU time as it repaints the console window. But other than console window support, only a
few Windows functions result in sending a message to the Windows subsystem process anymore: process and thread creation and termination, network drive letter mapping, and creation of temporary files. In general, a running Windows application won’t be causing many, if
any, context switches to the Windows subsystem process.
Copyrighted material.
58
Microsoft Windows Internals, Fourth Edition
Is Windows Less Stable with USER and GDI in Kernel Mode?
Some people wondered whether moving this much code into kernel mode would substantially affect system stability. The reason the impact on system stability has been minimal is that prior to Windows NT 4 (and this is still true today), a bug (such as an access
violation) in the user-mode Windows subsystem process (Csrss.exe) results in a system
crash because the Windows subsystem process was (and still is) a vital process to the
running of the system. Because it was the process that contained the data structures that
described the windows on the display, the death of that process would kill the user
interface. However, even a Windows system operating as a server, with no interactive
processes, can’t run without this process, because server processes might be using window messaging to drive the internal state of the application. With Windows, an access
violation in the same code now running in kernel mode simply crashes the system more
quickly, because exceptions in kernel mode result in a system crash.
There is, however, one additional theoretical danger that didn’t exist prior to moving the
windowing and graphics system into kernel mode. Because this body of code is now running in kernel mode, a bug (such as the use of a bad pointer) could result in corrupting kernel-mode protected data structures. Prior to Windows NT 4, such references would have
caused an access violation because kernel-mode pages aren’t writable from user mode. But
a system crash would have then resulted, as described earlier. With the code now running
in kernel mode, a bad pointer reference that caused a write operation to some kernel-mode
page might not immediately cause a system crash, but if it corrupted some data structure,
a crash would likely result soon after. There is a small chance, however, that such a reference
could corrupt a memory buffer (rather than a data structure), possibly resulting in returning corrupt data to a user program or writing bad data to the disk.
Another area of possible impact can come from the move of the graphics drivers into
kernel mode. Previously, some portions of a graphics driver ran within Csrss and others
ran in kernel mode. Now, the entire driver runs in kernel mode. Although Microsoft
doesn’t develop all the graphics device drivers supported in Windows, it does work
directly with hardware manufacturers to help ensure that they are able to produce reliable and efficient drivers. All drivers shipped with the system are submitted to the same
rigorous testing as other executive components.
Finally, it’s important to understand that this design (running the windowing and
graphics subsystem in kernel mode) is not fundamentally risky. It is identical to the
approaches many other device drivers use (for example, network card drivers and hard
disk drivers). All these drivers have been operating in kernel mode since the inception of
Windows NT with a high degree of reliability.
Some people speculated that the move of the window manager and the GDI into kernel
mode would hurt the preemptive multitasking capability of Windows. The theory was
that with all the additional Windows processing time spent in kernel mode, other
Copyrighted material.
Chapter 2:
System Architecture
threads would have less opportunity to be run preemptively. This view was based on a
misunderstanding of the Windows architecture. It is true that in many other nominally
preemptive operating systems, executing in kernel mode is never preempted by the
operating system scheduler—or is preempted only at a certain limited number of predefined points of kernel reentrancy. In Windows, however, threads running anywhere
in the executive are preempted and scheduled alongside threads running in user mode,
and all code within the executive is fully reentrant. Among other reasons, this capability
is necessary to achieve a high degree of system scalability on SMP hardware.
Another line of speculation was that SMP scaling would be hurt by this change. The theory went like this: Previously, an interaction between an application and the window
manager or the GDI involved two threads, one in the application and one in Csrss.exe.
Therefore, on an SMP system, the two threads could run in parallel, thus improving
throughput. This analysis shows a misunderstanding of how Windows NT technology
worked prior to Windows NT 4. In most cases, calls from a client application to the Windows subsystem process run synchronously; that is, the client thread entirely blocks
waiting on the server thread and begins to run again only when the server thread has
completed the call. Therefore, no parallelism on SMP hardware can ever be achieved.
This phenomenon is easily observable with a busy graphics application using the Performance tool on an SMP system. The observer will discover that on a two-processor system each processor is approximately 50 percent loaded, and it’s relatively easy to find
the single Csrss thread that is paired off with the busy application thread. Indeed,
because the two threads are fairly intimate with each other and sharing state, the processors’ caches must be flushed constantly to maintain coherency. This constant flushing is
the reason that with Windows NT 3.51 a single-threaded graphics application typically
runs slightly slower on an SMP machine than on a single processor system.
As a result, the changes in Windows NT 4 increased SMP throughput of applications
that make heavy use of the window manager and the GDI, especially when more than
one application thread is busy. When two application threads are busy on a two-processor Windows NT 3.51–based machine, a total of four threads (two in the application
plus two in Csrss) are battling for time on the two processors. Although only two are typically ready to run at any given time, the lack of a consistent pattern in which threads
run results in a loss of locality of reference and cache coherency. This loss occurs
because the busy threads are likely to get shuffled from one processor to another. In the
Windows NT 4 design, each of the two application threads essentially has its own processor, and the automatic thread affinity of Windows tends to run the same thread on
the same processor indefinitely, thus maximizing locality of reference and minimizing
the need to synchronize the private per-processor memory caches.
So in summary, moving the window manager and the GDI from user mode to kernel
mode has provided improved performance without any significant decrease in system
stability or reliability, even in the case of multiple sessions being created in a Terminal
Service enabled configuration.
Copyrighted material.
59
60
Microsoft Windows Internals, Fourth Edition
POSIX Subsystem
POSIX, an acronym loosely defined as “a portable operating system interface based on
UNIX,” refers to a collection of international standards for UNIX-style operating system interfaces. The POSIX standards encourage vendors implementing UNIX-style interfaces to make
them compatible so that programmers can move their applications easily from one system to
another.
Windows implements only one of the many POSIX standards, POSIX.1, formally known as
ISO/IEC 9945-1:1990 or IEEE POSIX standard 1003.1-1990. This standard was included primarily to meet U.S. government procurement requirements set in the mid-to-late 1980s that
mandated POSIX.1 compliance as specified in Federal Information Processing Standard
(FIPS) 151-2, developed by the National Institute of Standards and Technology. Windows NT
3.5, 3.51, and 4 have been formally tested and certified according to FIPS 151-2.
Because POSIX.1 compliance was a mandatory goal for Windows, the operating system was
designed to ensure that the required base system support was present to allow for the implementation of a POSIX.1 subsystem (such as the fork function, which is implemented in the
Windows executive, and the support for hard file links in the Windows file system). However,
because POSIX.1 defines a limited set of services (such as process control, interprocess communication, simple character cell I/O, and so on), the POSIX subsystem that comes with Windows 2000 isn’t a complete programming environment. And because applications can’t mix
calls between subsystems on Windows, by default, POSIX applications are limited to the strict
set of services defined in POSIX.1. This restriction means that a POSIX executable on Windows can’t create a thread or a window or use remote procedure calls (RPCs) or sockets.
To address this limitation, Microsoft provides a product called Windows Services for Unix,
which includes (as of version 3.5) an enhanced POSIX subsystem environment that provides
nearly 2000 UNIX functions and 300 UNIX-like tools and utilities. (See http://
www.microsoft.com/windows/sfu/default.asp for more information on Windows Services for
Unix.)
This enhanced POSIX subsystem assists in porting UNIX applications to Windows. However,
because the programs are still linked as POSIX executables, they cannot call Windows functions. To port UNIX applications to Windows and allow the use of Windows functions, you
can purchase UNIX-to-Windows porting packages, such as the MKS Toolkit products available from Mortice Kern Systems Inc. (www.mkssoftware.com). With this approach, a UNIX
application can be recompiled and relinked as a Windows executable and can slowly start to
integrate calls to native Windows functions.
Copyrighted material.
Chapter 2:
System Architecture
61
EXPERIMENT: Watching the POSIX Subsystem Start
The POSIX subsystem is configured by default to start the first time a POSIX executable
is run, so you can watch it start by running a POSIX program, such as one of the POSIX
utilities that comes with the Windows Services for Unix. (You can also find a small set of
POSIX utilities in the \Apps\POSIX folder on the Windows 2000 resource kit tools
media—they are not installed as part of the resource kit tools installation.) Follow these
steps to watch the POSIX subsystem start:
1. Start a command prompt.
2. Run Process Explorer and check that the POSIX subsystem isn’t already running
(that is, that there’s no Psxss.exe process on the system). Make sure Process
Explorer is displaying the process list in tree view (by pressing Ctrl+T).
3. Run a POSIX program, such as the C Shell or Korn Shell included with Windows
Services for Unix (or a POSIX tool from the Windows 2000 resource kit, such as
\Apps\POSIX\Ls.exe).
4. Go back to Process Explorer and notice the new Psxss.exe process that is a child of
Smss.exe (which, depending on your different highlight duration, might still be
highlighted as a new process on the display).
To compile and link a POSIX application in Windows requires the POSIX headers and libraries from the Platform SDK. POSIX executables are linked against the POSIX subsystem
library, Psxdll.dll. Because by default Windows is configured to start the POSIX subsystem on
demand, the first time you run a POSIX application, the POSIX subsystem process
(Psxss.exe) must be started. It remains running until the system reboots. (If you kill the
POSIX subsystem process, you won’t be able to run more POSIX applications until you
reboot.) The POSIX image itself isn’t run directly—instead, a special support image called
Posix.exe is launched, which in turn creates a child process to run the POSIX application.
OS/2 Subsystem
The OS/2 environment subsystem, like the built-in POSIX subsystem, is fairly limited in usefulness in that it supports only OS/2 1.2 16-bit character-based or video I/O (VIO) applications. Although Microsoft did sell a replacement OS/2 1.2 Presentation Manager subsystem
for Windows NT 4, it didn’t support OS/2 2.x (or later) applications (and it isn’t available for
Windows 2000 or later).
Also, because Windows doesn’t allow direct hardware access by user applications, OS/2 programs that contain I/O privilege segments that attempt to perform IN/OUT instructions (to
access some hardware device) as well as advanced video I/O (AVIO) aren’t supported. Applications that use the CLI/STI instructions are supported—but all the other OS/2 applications
Copyrighted material.
62
Microsoft Windows Internals, Fourth Edition
in the system and all the other threads in the OS/2 process issuing the CLI instructions are
suspended until an STI instruction is executed.
The 16-MB memory limitation on native OS/2 1.2 doesn’t apply to Windows—the OS/2 subsystem uses the 32-bit virtual address space of Windows to provide up to 512 MB of memory
to OS/2 1.2 applications, as illustrated in Figure 2-5.
2 GB
32-bit
16-bit
Win32 code and data
OS/2 client code and data
RTL code
.
.
.
High 32-bit user-mode area
Logical video buffer (LVB)
mapped to both 16-bit
application code and
32-bit OS/2 subsystem code
Tiled area (512 MB)
Heap area (used for 32-bit
structures that can be mapped into
16-bit application space)
16-bit DLLs and executables
16-bit application shared memory
16-bit application private memory
(DosAllocSec and so on)
Rtl heap and more
Figure 2-5
Low 32-bit user-mode area
0
OS/2 subsystem virtual memory layout
The tiled area is 512 MB of virtual address space that is reserved up front and then committed
or decommitted when 16-bit applications need segments. The OS/2 subsystem maintains a
local descriptor table (LDT) for each process, with shared memory segments at the same LDT
slot for all OS/2 processes.
As we’ll discuss in detail in Chapter 6, threads are the elements of a program that execute, and
as such they must be scheduled for processor time. Although Windows priority levels range
from 0 through 31, the 64 OS/2 priority levels (0 through 63) are mapped to Windows
dynamic priorities 1 through 15. OS/2 threads never receive Windows real-time priorities 16
through 31.
As with the POSIX subsystem, the OS/2 subsystem starts automatically the first time you activate a compatible OS/2 image. It remains running until the system is rebooted.
For more information on how Windows handles running POSIX and OS/2 applications, see
the section “Flow of CreateProcess” in Chapter 6.
Copyrighted material.
Chapter 2:
System Architecture
63
Ntdll.dll
Ntdll.dll is a special system support library primarily for the use of subsystem DLLs. It contains two types of functions:
■
System service dispatch stubs to Windows executive system services
■
Internal support functions used by subsystems, subsystem DLLs, and other native
images
The first group of functions provides the interface to the Windows executive system services
that can be called from user mode. There are more than 200 such functions, such as NtCreateFile, NtSetEvent, and so on. As noted earlier, most of the capabilities of these functions are
accessible through the Windows API. (A number are not, however, and are for use within the
operating system.)
For each of these functions, Ntdll contains an entry point with the same name. The code
inside the function contains the architecture-specific instruction that causes a transition into
kernel mode to invoke the system service dispatcher (explained in more detail in Chapter 3),
which after verifying some parameters, calls the actual kernel-mode system service that contains the real code inside Ntoskrnl.exe.
Ntdll also contains many support functions, such as the image loader (functions that start
with Ldr), the heap manager, and Windows subsystem process communication functions
(functions that start with Csr), as well as general run-time library routines (functions that start
with Rtl). It also contains the user-mode asynchronous procedure call (APC) dispatcher and
exception dispatcher. (APCs and exceptions are explained in Chapter 3.)
Executive
The Windows executive is the upper layer of Ntoskrnl.exe. (The kernel is the lower layer.) The
executive includes the following types of functions:
■
Functions that are exported and callable from user mode. These functions are called system services and are exported via Ntdll. Most of the services are accessible through the
Windows API or the APIs of another environment subsystem. A few services, however,
aren’t available through any documented subsystem function. (Examples include LPCs
and various query functions such as NtQueryInformationProcess, specialized functions
such as NtCreatePagingFile, and so on.)
■
Device driver functions that are called through the use of the DeviceIoControl function.
This provides a general interface from user mode to kernel mode to call functions in
device drivers that are not associated with a read or write.
■
Functions that can be called only from kernel mode that are exported and documented
in the Windows DDK or Windows Installable File System (IFS) Kit. (For information on
the Windows IFS Kit, go to http://www.microsoft.com/whdc/ddk/ifskit/default.mspx.)
Copyrighted material.
64
Microsoft Windows Internals, Fourth Edition
■
Functions that are exported and callable from kernel mode but are not documented in
the Windows DDK or IFS Kit (such as the functions called by the boot video driver,
which start with Inbv).
■
Functions that are defined as global symbols but are not exported. These include internal support functions called within Ntoskrnl, such as those that start with Iop (internal
I/O manager support functions) or Mi (internal memory management support functions).
■
Functions that are internal to a module that are not defined as global symbols.
The executive contains the following major components, each of which is covered in detail in
a subsequent chapter of this book:
■
The configuration manager (explained in Chapter 4) is responsible for implementing and
managing the system registry.
■
The process and thread manager (explained in Chapter 6) creates and terminates processes and threads. The underlying support for processes and threads is implemented
in the Windows kernel; the executive adds additional semantics and functions to these
lower-level objects.
■
The security reference monitor (or SRM, described in Chapter 8) enforces security policies
on the local computer. It guards operating system resources, performing run-time object
protection and auditing.
■
The I/O manager (explained in Chapter 9) implements device-independent I/O and is
responsible for dispatching to the appropriate device drivers for further processing.
■
The Plug and Play (PnP) manager (explained in Chapter 9) determines which drivers are
required to support a particular device and loads those drivers. It retrieves the hardware
resource requirements for each device during enumeration. Based on the resource
requirements of each device, the PnP manager assigns the appropriate hardware
resources such as I/O ports, IRQs, DMA channels, and memory locations. It is also
responsible for sending proper event notification for device changes (addition or
removal of a device) on the system.
■
The power manager (explained in Chapter 9) coordinates power events and generates
power management I/O notifications to device drivers. When the system is idle, the
power manager can be configured to reduce power consumption by putting the CPU to
sleep. Changes in power consumption by individual devices are handled by device drivers but are coordinated by the power manager.
■
The WDM Windows Management Instrumentation routines (explained in Chapter 4)
enable device drivers to publish performance and configuration information and receive
commands from the user-mode WMI service. Consumers of WMI information can be on
the local machine or remote across the network.
Copyrighted material.
Chapter 2:
System Architecture
65
■
The cache manager (explained in Chapter 11) improves the performance of file-based
I/O by causing recently referenced disk data to reside in main memory for quick access
(and by deferring disk writes by holding the updates in memory for a short time before
sending them to the disk). As you’ll see, it does this by using the memory manager’s
support for mapped files.
■
The memory manager (explained in Chapter 7) implements virtual memory, a memory
management scheme that provides a large, private address space for each process that
can exceed available physical memory. The memory manager also provides the underlying support for the cache manager.
■
The logical prefetcher (explained in Chapter 7) accelerates system and process startup by
optimizing the loading of data referenced during the startup of the system or a process.
In addition, the executive contains four main groups of support functions that are used by the
executive components just listed. About a third of these support functions are documented in
the DDK because device drivers also use them. These are the four categories of support functions:
■
The object manager, which creates, manages, and deletes Windows executive objects and
abstract data types that are used to represent operating system resources such as processes, threads, and the various synchronization objects. The object manager is
explained in Chapter 3.
■
The LPC facility (explained in Chapter 3) passes messages between a client process and
a server process on the same computer. LPC is a flexible, optimized version of remote
procedure call (RPC), an industry-standard communication facility for client and server
processes across a network.
■
A broad set of common run-time library functions, such as string processing, arithmetic
operations, data type conversion, and security structure processing.
■
Executive support routines, such as system memory allocation (paged and nonpaged
pool), interlocked memory access, as well as two special types of synchronization
objects: resources and fast mutexes.
Kernel
The kernel consists of a set of functions in Ntoskrnl.exe that provide fundamental mechanisms (such as thread scheduling and synchronization services) used by the executive components, as well as low-level hardware architecture-dependent support (such as interrupt and
exception dispatching), that are different on each processor architecture. The kernel code is
written primarily in C, with assembly code reserved for those tasks that require access to specialized processor instructions and registers not easily accessible from C.
Copyrighted material.
66
Microsoft Windows Internals, Fourth Edition
Like the various executive support functions mentioned in the preceding section, a number of
functions in the kernel are documented in the DDK (and can be found by searching for functions beginning with Ke) because they are needed to implement device drivers.
Kernel Objects
The kernel provides a low-level base of well-defined, predictable operating system primitives
and mechanisms that allow higher-level components of the executive to do what they need to
do. The kernel separates itself from the rest of the executive by implementing operating system mechanisms and avoiding policy making. It leaves nearly all policy decisions to the executive, with the exception of thread scheduling and dispatching, which the kernel implements.
Outside the kernel, the executive represents threads and other shareable resources as objects.
These objects require some policy overhead, such as object handles to manipulate them, security checks to protect them, and resource quotas to be deducted when they are created. This
overhead is eliminated in the kernel, which implements a set of simpler objects, called kernel
objects, that help the kernel control central processing and support the creation of executive
objects. Most executive-level objects encapsulate one or more kernel objects, incorporating
their kernel-defined attributes.
One set of kernel objects, called control objects, establishes semantics for controlling various
operating system functions. This set includes the APC object, the deferred procedure call (DPC)
object, and several objects the I/O manager uses, such as the interrupt object.
Another set of kernel objects, known as dispatcher objects, incorporates synchronization capabilities that alter or affect thread scheduling. The dispatcher objects include the kernel thread,
mutex (called mutant internally), event, kernel event pair, semaphore, timer, and waitable
timer. The executive uses kernel functions to create instances of kernel objects, to manipulate
them, and to construct the more complex objects it provides to user mode. Objects are
explained in more detail in Chapter 3, and processes and threads are described in Chapter 6.
Hardware Support
The other major job of the kernel is to abstract or isolate the executive and device drivers from
variations between the hardware architectures supported by Windows. This job includes handling variations in functions such as interrupt handling, exception dispatching, and multiprocessor synchronization.
Even for these hardware-related functions, the design of the kernel attempts to maximize the
amount of common code. The kernel supports a set of interfaces that are portable and semantically identical across architectures. Most of the code that implements this portable interface
is also identical across architectures.
Some of these interfaces are implemented differently on different architectures, however, or
some of the interfaces are partially implemented with architecture-specific code. These
Copyrighted material.
Chapter 2:
System Architecture
67
architecturally independent interfaces can be called on any machine, and the semantics of
the interface will be the same whether or not the code varies by architecture. Some kernel
interfaces (such as spinlock routines, which are described in Chapter 3) are actually implemented in the HAL (described in the next section) because their implementation can vary
for systems within the same architecture family.
The kernel also contains a small amount of code with x86-specific interfaces needed to support
old MS-DOS programs. These x86 interfaces aren’t portable in the sense that they can’t be
called on a machine based on any other architecture; they won’t be present. This x86-specific
code, for example, supports calls to manipulate global descriptor tables (GDTs) and LDTs,
hardware features of the x86.
Other examples of architecture-specific code in the kernel include the interface to provide
translation buffer and CPU cache support. This support requires different code for the different architectures because of the way caches are implemented.
Another example is context switching. Although at a high level the same algorithm is used for
thread selection and context switching (the context of the previous thread is saved, the context of the new thread is loaded, and the new thread is started), there are architectural differences among the implementations on different processors. Because the context is described
by the processor state (registers and so on), what is saved and loaded varies depending on the
architecture.
Hardware Abstraction Layer
As mentioned at the beginning of this chapter, one of the crucial elements of the Windows
design is its portability across a variety of hardware platforms. The hardware abstraction layer
(HAL) is a key part of making this portability possible. The HAL is a loadable kernel-mode
module (Hal.dll) that provides the low-level interface to the hardware platform on which
Windows is running. It hides hardware-dependent details such as I/O interfaces, interrupt
controllers, and multiprocessor communication mechanisms—any functions that are both
architecture-specific and machine-dependent.
So rather than access hardware directly, Windows internal components as well as user-written
device drivers maintain portability by calling the HAL routines when they need platformdependent information. For this reason, the HAL routines are documented in the Windows
DDK. To find out more about the HAL and its use by device drivers, refer to the DDK.
Although several HALs are included with Windows (as shown in Table 2-6), only one is chosen at installation time and copied to the system disk with the filename Hal.dll. (Other operating systems, such as VMS, select the equivalent of the HAL at system boot time.) Therefore,
you can’t assume that a system disk from one x86 installation will boot on a different processor if the HAL that supports the other processor is different.
Copyrighted material.
68
Microsoft Windows Internals, Fourth Edition
Table 2-6 List of x86 HALs in \Windows\Driver Cache\i386\Driver.cab
HAL File Name
Systems Supported
Hal.dll
Standard PCs
Halacpi.dll
Advanced Configuration and Power Interface (ACPI) PCs
Halapic.dll
Advanced Programmable Interrupt Controller (APIC) PCs
Halaacpi.dll
APIC ACPI PCs
Halmps.dll
Multiprocessor PCs
Halmacpi.dll
Multiprocessor ACPI PCs
Halborg.dll
Silicon Graphics Workstation (Windows 2000 only; platform no longer
marketed)
Halsp.dll
Compaq SystemPro (Windows XP only)
Note
As of Windows Server 2003, no vendor-specific HALs are shipped with the base sys-
tem.
EXPERIMENT: Viewing the Base HALs Included with Windows
To view the HALs included with Windows, open the file Driver.cab in the appropriate
architecture-specific folder underneath \Windows\Driver Cache. (For example, for x86
systems, the file name is \Windows\Driver Cache\i386\Driver.cab.) Scroll down to the
files beginning with “Hal” and you should see the files listed in Table 2-6.
EXPERIMENT: Determining Which HAL You’re Running
There are two ways to determine which HAL you’re running:
1. Open the file \Windows\Repair\Setup.log, search for Hal.dll, and look at the filename after the equals sign. This is the name of the HAL on the distribution media
extracted from Driver.cab.
2. In Device Manager (right-click on the My Computer icon on your desktop, select
Properties, click on the Hardware tab, and then click Device Manager), look at the
name of the “driver” under the Computer device type. For example, the following
screen shot is from a system running the ACPI HAL:
Copyrighted material.
Chapter 2:
System Architecture
69
EXPERIMENT: Viewing NTOSKRNL and HAL Image Dependencies
You can view the relationship of the kernel and HAL images by examining their export
and import tables using the Dependency Walker tool (Depends.exe), which is contained in the Windows Support Tools and the Platform SDK. To examine an image in
the Dependency Walker, select Open from the File menu to open the desired image file.
Here is a sample of output you can see by viewing the dependencies of Ntoskrnl using
this tool:
Notice that Ntoskrnl is linked against the HAL, which is in turn linked against Ntoskrnl.
(They both use functions in each other.) Ntoskrnl is also linked against Bootvid.dll, the
boot video driver that is used to implement the GUI startup screen. On Windows XP
and later, you will see an additional DLL, Kdcom.dll, in the list. This contains kernel
debugger infrastructure code that used to be part of Ntoskrnl.exe.
For a detailed description of the information displayed by this tool, see the Dependency
Walker help file (Depends.hlp).
Device Drivers
Although device drivers are explained in detail in Chapter 9, this section provides a brief overview of the types of drivers and explains how to list the drivers installed and loaded on your
system.
Copyrighted material.
70
Microsoft Windows Internals, Fourth Edition
Device drivers are loadable kernel-mode modules (typically ending in .sys) that interface
between the I/O manager and the relevant hardware. They run in kernel mode in one of three
contexts:
■
In the context of the user thread that initiated an I/O function
■
In the context of a kernel-mode system thread
■
As a result of an interrupt (and therefore not in the context of any particular process or
thread—whichever process or thread was current when the interrupt occurred)
As stated in the preceding section, device drivers in Windows don’t manipulate hardware
directly, but rather they call functions in the HAL to interface with the hardware. Drivers are
typically written in C (sometimes C++) and therefore, with proper use of HAL routines, can be
source code portable across the CPU architectures supported by Windows and binary portable within an architecture family.
There are several types of device drivers:
■
Hardware device drivers manipulate hardware (using the HAL) to write output to or
retrieve input from a physical device or network. There are many types of hardware
device drivers, such as bus drivers, human interface drivers, mass storage drivers, and so
on.
■
File system drivers are Windows drivers that accept file-oriented I/O requests and translate them into I/O requests bound for a particular device.
■
File system filter drivers, such as those that perform disk mirroring and encryption, intercept I/Os and perform some added-value processing before passing the I/O to the next
layer.
■
Network redirectors and servers are file system drivers that transmit file system I/O
requests to a machine on the network and receive such requests, respectively.
■
Protocol drivers implement a networking protocol such as TCP/IP, NetBEUI, and IPX/
SPX.
■
Kernel streaming filter drivers are chained together to perform signal processing on data
streams, such as recording or displaying audio and video.
Because installing a device driver is the only way to add user-written kernel-mode code to the
system, some programmers have written device drivers simply as a way to access internal
operating system functions or data structures that are not accessible from user mode (but that
are documented and supported in the DDK). For example, many of the utilities from www.sysinternals.com combine a Windows GUI application and a device driver that is used to gather
internal system state and call kernel-mode-only accessible functions not accessible from the
user-mode Windows API.
Copyrighted material.
Chapter 2:
System Architecture
71
Windows Driver Model (WDM)
Windows 2000 added support for Plug and Play, Power Options, and an extension to the
Windows NT driver model called the Windows Driver Model (WDM). Windows 2000 and
later can run legacy Windows NT 4 drivers, but because these don’t support Plug and Play
and Power Options, systems running these drivers will have reduced capabilities in these two
areas.
From the WDM perspective, there are three kinds of drivers:
■
A bus driver services a bus controller, adapter, bridge, or any device that has child
devices. Bus drivers are required drivers, and Microsoft generally provides them; each
type of bus (such as PCI, PCMCIA, and USB) on a system has one bus driver. Third parties can write bus drivers to provide support for new buses, such as VMEbus, Multibus,
and Futurebus.
■
A function driver is the main device driver and provides the operational interface for its
device. It is a required driver unless the device is used raw (an implementation in which
I/O is done by the bus driver and any bus filter drivers, such as SCSI PassThru). A function driver is by definition the driver that knows the most about a particular device, and
it is usually the only driver that accesses device-specific registers.
■
A filter driver is used to add functionality to a device (or existing driver) or to modify
I/O requests or responses from other drivers (and is often used to fix hardware that
provides incorrect information about its hardware resource requirements). Filter drivers are optional and can exist in any number, placed above or below a function driver
and above a bus driver. Usually, system original equipment manufacturers (OEMs) or
independent hardware vendors (IHVs) supply filter drivers.
In the WDM driver environment, no single driver controls all aspects of a device: a bus driver
is concerned with reporting the devices on its bus to the PnP manager, while a function driver
manipulates the device.
In most cases, lower-level filter drivers modify the behavior of device hardware. For example,
if a device reports to its bus driver that it requires four I/O ports when it actually requires 16
I/O ports, a lower-level device-specific function filter driver could intercept the list of hardware resources reported by the bus driver to the PnP manager, and update the count of I/O
ports.
Upper-level filter drivers usually provide added-value features for a device. For example, an
upper-level device filter driver for a keyboard can enforce additional security checks.
Interrupt processing is explained in Chapter 3. Further details about the I/O manager, WDM,
Plug and Play, and Power Options are included in Chapter 9.
Copyrighted material.
72
Microsoft Windows Internals, Fourth Edition
EXPERIMENT: Viewing the Installed Device Drivers
You can list the installed drivers by running Computer Management. (From the Start
menu, select Programs, Administrative Tools, and then Computer Management; or from
Control Panel, open Administrative Tools and select Computer Management.) From
within Computer Management, expand System Information and then Software Environment, and open Drivers. Here’s an example output of the list of installed drivers:
This window displays the list of device drivers defined in the registry, their type, and
their state (Running or Stopped). Device drivers and Windows service processes are
both defined in the same place: HKLM\ SYSTEM\CurrentControlSet\Services. However, they are distinguished by a type code—for example, type 1 is a kernel-mode device
driver. (For a complete list of the information stored in the registry for device drivers, see
Table 4-7.)
Alternatively, you list the currently loaded device drivers with the Drivers utility (Drivers.exe in the Windows 2000 resource kits) or the Pstat utility (Pstat.exe in the Windows XP Support Tools, Windows Server 2003 Support Tools, Windows 2000 resource
kits, and the Platform SDK). Here is a partial output from the Drivers utility:
C:\>drivers
ModuleName
Code
Data Bss
Paged
Init
LinkDate
---------------------------------------------------------------ntoskrnl.exe 429184 96896
0 775360 138880 Tue Dec 07 18:41:11
hal.dll 25856
6016
0
16160
10240 Tue Nov 02 20:14:22
BOOTVID.DLL
5664
2464
0
0
320 Wed Nov 03 20:24:33
ACPI.sys 92096
8960
0
43488
4448 Wed Nov 10 20:06:04
WMILIB.SYS
512
0
0
1152
192 Sat Sep 25 14:36:47
pci.sys 12704
1536
0
31264
4608 Wed Oct 27 19:11:08
isapnp.sys 14368
832
0
22944
2048 Sat Oct 02 16:00:35
compbatt.sys
2496
0
0
2880
1216 Fri Oct 22 18:32:49
BATTC.SYS
800
0
0
2976
704 Sun Oct 10 19:45:37
intelide.sys
1760
32
0
0
128 Thu Oct 28 19:20:03
PCIIDEX.SYS
4544
480
0
10944
1632 Wed Oct 27 19:02:19
pcmcia.sys 32800
8864
0
23680
6240 Fri Oct 29 19:20:08
ftdisk.sys
4640
32
0
95072
3392 Mon Nov 22 14:36:23
---------------------------------------------------------------Total 4363360 580320 0 3251424 432992
Copyrighted material.
1999
1999
1999
1999
1999
1999
1999
1999
1999
1999
1999
1999
1999
Chapter 2:
System Architecture
Each loaded kernel-mode component (Ntoskrnl, the HAL, as well as device drivers) is
shown, along with the sizes of the sections in each image.
The Pstat utility also shows the loaded driver list, but only after it first displays the process list and the threads in each process. Pstat includes one important piece of information that the Drivers utility doesn’t: the load address of the module in system space. As
we’ll explain later, this address is needed to map running system threads to the device
driver in which they exist.
Peering into Undocumented Interfaces
Examining the names of the exported or global symbols in key system images (such as
Ntoskrnl.exe, Hal.dll, or Ntdll.dll) can be enlightening—you can get an idea of the kinds
of things Windows can do versus what happens to be documented and supported
today. Of course, just because you know the names of these functions doesn’t mean that
you can or should call them—the interfaces are undocumented and are subject to
change. We suggest that you look at these functions purely to gain more insight into the
kinds of internal functions Windows performs, not to bypass supported interfaces.
For example, looking at the list of functions in Ntdll.dll gives you the list of all the system services that Windows provides to user-mode subsystem DLLs versus the subset
that each subsystem exposes. Although many of these functions map clearly to documented and supported Windows functions, several are not exposed via the Windows
API. (See the article “Inside the Native API” from www.sysinternals.com.)
Conversely, it’s also interesting to examine the imports of Windows subsystem DLLs
(such as Kernel32.dll or Advapi32.dll) and which functions they call in Ntdll.
Another interesting image to dump is Ntoskrnl.exe—although many of the exported routines that kernel-mode device drivers use are documented in the Windows DDK, quite
a few are not. You might also find it interesting to take a look at the import table for
Ntoskrnl and the HAL; this table shows the list of functions in the HAL that Ntoskrnl
uses and vice versa.
Copyrighted material.
73
74
Microsoft Windows Internals, Fourth Edition
Table 2-7 lists most of the commonly used function name prefixes for the executive components. Each of these major executive components also uses a variation of the prefix to
denote internal functions—either the first letter of the prefix followed by an i (for internal) or the full prefix followed by a p (for private). For example, Ki represents internal
kernel functions, and Psp refers to internal process support functions.
Table 2-7
Commonly Used Prefixes
Prefix
Component
Cc
Cache manager
Cm
Configuration manager
Ex
Executive support routines
FsRtl
File system driver run-time library
Hal
Hardware abstraction layer
Io
I/O manager
Ke
Kernel
Lpc
Local procedure call
Lsa
Local security authentication
Mm
Memory manager
Nt
Windows system services (most of which are exported as Windows
functions)
Ob
Object manager
Po
Power manager
Pp
PnP manager
Ps
Process support
Rtl
Run-time library
Se
Security
Wmi
Windows Management Instrumentation
Zw
Mirror entry point for system services (beginning with Nt) that sets previous
access mode to kernel, which eliminates parameter validation, because Nt
system services validate parameters only if previous access mode is user
You can decipher the names of these exported functions more easily if you understand
the naming convention for Windows system routines. The general format is:
<Prefix><Operation><Object>
In this format, Prefix is the internal component that exports the routine, Operation
tells what is being done to the object or resource, and Object identifies what is being
operated on.
For example, ExAllocatePoolWithTag is the executive support routine to allocate from a
paged or nonpaged pool. KeInitializeThread is the routine that allocates and sets up a
kernel thread object.
Copyrighted material.
Chapter 2:
System Architecture
75
System Processes
The following system processes appear on every Windows system. (Two of these—Idle and
System—are not full processes, as they are not running a user-mode executable.)
■
Idle process (contains one thread per CPU to account for idle CPU time)
■
System process (contains the majority of the kernel-mode system threads)
■
Session manager (Smss.exe)
■
Windows subsystem (Csrss.exe)
■
Logon process (Winlogon.exe)
■
Service control manager (Services.exe) and the child service processes it creates (such as
the system-supplied generic service host process, Svchost.exe)
■
Local security authentication server (Lsass.exe)
To understand the relationship of these processes, it is helpful to view the process “tree”—that
is, the parent/child relationship between processes. Seeing which process created each process helps to understand where each process comes from. Figure 2-6 is a partial screen snapshot of the process tree with comments put on the first few system processes. (Process
Explorer allows you to add a comment for individual processes and optionally display that as
a column on the display.)
Figure 2-6
Initial System Process Tree
The next sections explain the key system processes shown in Figure 2-6. Although these sections briefly indicate the order of process startup, Chapter 5 contains a detailed description of
the steps involved in booting and starting Windows.
Idle Process
The first process listed in Figure 2-6 is the system idle process. As we’ll explain in Chapter 6,
processes are identified by their image name. However, this process (as well as the process
Copyrighted material.
76
Microsoft Windows Internals, Fourth Edition
named System) isn’t running a real user-mode image (in that there is no “System Idle Process.exe” in the \Windows directory). In addition, the name shown for this process differs
from utility to utility (because of implementation details). Table 2-8 lists several of the names
given to the Idle process (process ID 0). The Idle process is explained in detail in Chapter 6.
Table 2-8 Names for Process ID 0 in Various Utilities
Utility
Name for Process ID 0
Task Manager
System Idle Process
Process Viewer (Pviewer.exe)
Idle
Process Status (Pstat.exe)
Idle Process
Process Explode (Pview.exe)
System Process
Task List (Tlist.exe)
System Process
QuickSlice (Qslice.exe)
Systemprocess
Now let’s look at system threads and the purpose of each of the system processes that are running real images.
Interrupts and DPCs
The two lines labeled Interrupts and DPCs represent time spent servicing interrupts and
deferred procedure calls. These mechanisms are explained in Chapter 3. Note that while Process Explorer displays these as entries in the process list, they are not processes. They are
shown because they account for CPU time not charged to any process. (For example, a system
with heavy interrupt activity will not appear as a process consuming CPU time.) Note that
Task Manager includes interrupt and DPC time in the system idle time. Thus a system with
heavy interrupt activity will appear to be idle when using Task Manager.
System Process and System Threads
The System process (process ID 8 in Windows 2000 and process ID 4 in Windows XP and
Windows Server 2003) is the home for a special kind of thread that runs only in kernel mode:
a kernel-mode system thread. System threads have all the attributes and contexts of regular usermode threads (such as a hardware context, priority, and so on) but are different in that they
run only in kernel-mode executing code loaded in system space, whether that is in Ntoskrnl.exe or in any other loaded device driver. In addition, system threads don’t have a user process address space and hence must allocate any dynamic storage from operating system
memory heaps, such as a paged or nonpaged pool.
System threads are created by the PsCreateSystemThread function (documented in the DDK),
which can be called only from kernel mode. Windows as well as various device drivers create
system threads during system initialization to perform operations that require thread context,
such as issuing and waiting for I/Os or other objects or polling a device. For example, the
memory manager uses system threads to implement such functions as writing dirty pages to
the page file or mapped files, swapping processes in and out of memory, and so forth. The kerCopyrighted material.
Chapter 2:
System Architecture
77
nel creates a system thread called the balance set manager that wakes up once per second to
possibly initiate various scheduling and memory management–related events. The cache
manager also uses system threads to implement both read-ahead and write-behind I/Os. The
file server device driver (Srv.sys) uses system threads to respond to network I/O requests for
file data on disk partitions shared to the network. Even the floppy driver has a system thread
to poll the floppy device. (Polling is more efficient in this case because an interrupt-driven
floppy driver consumes a large amount of system resources.) Further information on specific
system threads is included in the chapters in which the component is described.
By default, system threads are owned by the System process, but a device driver can create a
system thread in any process. For example, the Windows subsystem device driver
(Win32k.sys) creates system threads in the Windows subsystem process (Csrss.exe) so that
they can easily access data in the user-mode address space of that process.
When you’re troubleshooting or going through a system analysis, it’s useful to be able to map
the execution of individual system threads back to the driver or even to the subroutine that
contains the code. For example, on a heavily loaded file server, the System process will likely
be consuming considerable CPU time. But the knowledge that when the System process is
running “some system thread” is running isn’t enough to determine which device driver or
operating system component is running.
So if threads in the System process are running, first determine which ones are running (for
example, with the Performance tool). Once you find the thread (or threads) that is running,
look up in which driver the system thread began execution (which at least tells you which
driver likely created the thread) or examine the call stack (or at least the current address) of
the thread in question, which would indicate where the thread is currently executing.
Both of these techniques are illustrated in the following experiments.
EXPERIMENT: Identifying System Threads in the System Process
You can see that the threads inside the System process must be kernel-mode system
threads because the start address for each thread is greater than the start address of system space (which by default begins at 0x80000000, unless the system was booted with
the /3GB Boot.ini switch). Also, if you look at the CPU time for these threads, you’ll see
that those that have accumulated any CPU time have run only in kernel mode.
To find out which driver created the system thread, look up the start address of the thread
(which you can display with Pviewer.exe) and look for the driver whose base address is
closest to (but before) the start address of the thread. Both the Pstat utility (at the end of
its output) as well as the !drivers kernel debugger command list the base address of each
loaded device driver.
To quickly find the current address of the thread, use the !stacks 0 command in the kernel
debugger. Here is sample output from a live system (using LiveKd):
Copyrighted material.
78
Microsoft Windows Internals, Fourth Edition
kd> !stacks 0
Proc.Thread Thread
8.000004
8.00000c
8.000010
8.000014
8.000018
8.00001c
8.000020
8.000024
8.000028
8.00002c
8.000030
8.000034
8.000038
8.00003c
8.000040
8.000044
8.000048
8.00004c
8.000050
8.000054
8.000058
8.00005c
8.000060
8.000070
8.000074
8.00006c
8.000080
8.000084
8.00008c
8.00015c
8.000160
8.000178
8.0002d0
8.0002d4
8.000404
8.000430
8.00069c
8146edb0
8146e730
8146e4b0
8146d030
8146ddb0
8146db30
8146d8b0
8146d630
8146d3b0
8146c030
8146cdb0
8146b470
8146b1f0
8146a030
8146adb0
8146a5b0
8146a330
81461030
8143a770
81439730
81436c90
813d9170
813d8030
8139c850
8139c5d0
81384030
81333330
813330b0
81321db0
81205570
81204570
811fcdb0
811694f0
81168030
811002b0
810f4990
80993030
ThreadState
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
BLOCKED
READY
READY
Blocker
[System]
ntoskrnl!MmZeroPageThread+0x5f
?? Kernel stack not resident ??
ntoskrnl!ExpWorkerThread+0x73
?? Kernel stack not resident ??
ntoskrnl!ExpWorkerThread+0x73
ntoskrnl!ExpWorkerThread+0x73
ntoskrnl!ExpWorkerThread+0x73
ntoskrnl!ExpWorkerThread+0x73
ntoskrnl!ExpWorkerThread+0x73
ntoskrnl!ExpWorkerThread+0x73
ntoskrnl!ExpWorkerThreadBalanceManager+0x55
ntoskrnl!MiDereferenceSegmentThread+0x44
ntoskrnl!MiModifiedPageWriterWorker+0x31
ntoskrnl!KeBalanceSetManager+0x7e
ntoskrnl!KeSwapProcessOrStack+0x24
ntoskrnl!FsRtlWorkerThread+0x33
ntoskrnl!FsRtlWorkerThread+0x33
ACPI!ACPIWorker+0x46
ntoskrnl!MiMappedPageWriter+0x4d
dmio!voliod_loop+0x399
NDIS!ndisWorkerThread+0x22
ltmdmntt!WakeupTimerThread+0x27
ltmdmntt!WriteRegistryThread+0x1c
raspptp!MainPassiveLevelThread+0x78
raspptp!PacketWorkingThread+0xc0
rasacd!AcdNotificationRequestThread+0xd8
rdbss!RxpWorkerThreadDispatcher+0x6f
rdbss!RxSpinUpRequestsDispatcher+0x58
?? Kernel stack not resident ??
INO_FLTR+0x68bd
INO_FLTR+0x80e9
irda!RxThread+0xfa
?? Kernel stack not resident ??
?? Kernel stack not resident ??
rdbss!RxpWorkerThreadDispatcher+0x6f
parallel!ParallelThread+0x3e
rdbss!RxpWorkerThreadDispatcher+0x6f
The first column is the process ID and thread ID (in the form “process ID.thread ID”). The
second column is the current address of the thread. The third column indicates whether
the thread is in a wait state, ready state, or running state. (See Chapter 6 for a description
of thread states.) The last column is the top-most address on the thread’s stack. The information in this last column makes it easy to see which driver each thread started in. For the
threads in Ntoskrnl, the name of the function gives a further indication of what the thread
is doing.
However, if the thread running is one of the system worker threads (ExpWorkerThread),
you still don’t really know what the thread is doing because any device driver can submit
work to a system worker thread. Therefore, the only way to trace back worker thread activity is to set a breakpoint at ExQueueWorkItem. When you reach the breakpoint, type !dso
Copyrighted material.
Chapter 2:
System Architecture
work_queue_item esp+4. This command will dump the first argument to ExQueueWorkItem (a work queue structure), which in turn contains the address of the worker routine to be called in the context of the worker thread. Alternatively, you can look at the
caller by using the k command in the kernel debugger, which displays the current call
stack. The current call stack will show the driver that is queuing the work to the worker
thread (as opposed to the routine to be called from the worker thread).
EXPERIMENT: Mapping a System Thread to a Device Driver
In this experiment, we’ll see how to map CPU activity in the System process to the
responsible system thread (and the driver it falls in) generating the activity. This is
important because when the System process is running, you must go to the thread granularity to really understand what’s going on. For this experiment, we will generate system thread activity by generating file server activity on your machine. (The file server
driver, Srv.sys, creates system threads to handle inbound requests for file I/O. See
Chapter 13 for more information on this component.)
1. Open a command prompt.
2. Do a directory listing of your entire C drive using a network path to access your C
drive. For example, if your computer name is COMPUTER1, type “dir
\\computer1\c$ /s”. (The /s switch lists all subdirectories.)
3. Run Process Explorer, and double-click on the System process.
4. Click on the Threads tab.
5. Sort by the CSwitch Delta (context switch delta) column. You should see one or
more threads in Srv.sys running such as the following:
Copyrighted material.
79
80
Microsoft Windows Internals, Fourth Edition
If you see a system thread running and you are not sure what the driver is, press the Module button, which will bring up the file properties. Pressing the Module button while highlighting the thread in Srv.sys previously shown results in the following display:
Session Manager (Smss)
The Session Manager (\Windows\System32\Smss.exe) is the first user-mode process created
in the system. The kernel-mode system thread that performs the final phase of the initialization of the executive and kernel creates the actual Smss process.
The Session Manager is responsible for a number of important steps in starting Windows, such
as opening additional page files, performing delayed file rename and delete operations, and creating system environment variables. It also launches the subsystem processes (normally just
Csrss.exe) and the Winlogon process, which in turn creates the rest of the system processes.
Much of the configuration information in the registry that drives the initialization steps of
Smss can be found under HKLM\SYSTEM\CurrentControlSet\Control\Session Manager.
Some of these are explained in Chapter 5 in the section on Smss. (For a more complete
description of the keys and values, see the Registry Entries help file, Regentry.chm, in the
Windows 2000 resource kits.)
After performing these initialization steps, the main thread in Smss waits forever on the process handles to Csrss and Winlogon. If either of these processes terminates unexpectedly,
Smss crashes the system (using the crash code STATUS_SYSTEM_PROCESS_TERMINATED,
or 0xC000021A), because Windows relies on their existence. Meanwhile, Smss waits for
Copyrighted material.
Chapter 2:
System Architecture
81
requests to load subsystems, debug events, and requests to create new terminal server sessions. (For a description of terminal services, see the section “Terminal Services and Multiple
Sessions” in Chapter 1.)
Terminal Services session creation is performed by Smss. When a request comes in to Smss to
create a session, it first calls NtSetSystemInformation with a request to set up kernel-mode
session data structures. This in turn calls the internal memory manager function MmSessionCreate, which sets up the session virtual address space that will contain the session paged
pool and the per-session data structures allocated by the kernel-mode part of the Win32 subsystem (Win32k.sys) and other session-space device drivers. (See Chapter 7 for more details.)
Smss then creates an instance of Winlogon and Csrss for the session.
Winlogon, LSASS and Userinit
The Windows logon process (\Windows\System32\Winlogon.exe) handles interactive user
logons and logoffs. Winlogon is notified of a user logon request when the secure attention
sequence (SAS) keystroke combination is entered. The default SAS on Windows is the combination Ctrl+Alt+Delete. The reason for the SAS is to protect users from password-capture programs that simulate the logon process, because this keyboard sequence cannot be intercepted
by a user mode application.
The identification and authentication aspects of the logon process are implemented in a replaceable DLL named GINA (Graphical Identification and Authentication). The standard Windows
GINA, Msgina.dll, implements the default Windows logon interface. However, developers can
provide their own GINA DLL to implement other identification and authentication mechanisms
in place of the standard Windows username/password method (such as one based on a voice
print). In addition, Winlogon can load additional network provider DLLs that need to perform
secondary authentication. This capability allows multiple network providers to gather identification and authentication information all at one time during normal logon.
Once the username and password have been captured, they are sent to the local security authentication server process (\Windows\System32\Lsass.exe, described in Chapter 8) to be authenticated. LSASS calls the appropriate authentication package (implemented as a DLL) to perform the
actual verification, such as checking whether a password matches what is stored in the active
directory or the SAM (the part of the registry that contains the definition of the users and groups).
Upon a successful authentication, LSASS calls a function in the security reference monitor (for
example, NtCreateToken) to generate an access token object that contains the user’s security
profile. This access token is then used by Winlogon to create the initial process(es) in the
user’s session. The initial process(es) are stored in the registry value Userinit under the registry key HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon. (The
default is Userinit.exe, but there can be more than one image in the list.)
Userinit performs some initialization of the user environment (such as running the login script
and applying group policies) and then looks in the registry at the Shell value (under the same
Winlogon key referred to previously) and creates a process to run the system-defined shell (by
Copyrighted material.
82
Microsoft Windows Internals, Fourth Edition
default, Explorer.exe). Then Userinit exits. This is the reason Explorer.exe is shown with no parent—its parent has exited, and as explained earlier, tlist left-justifies processes whose parent isn’t
running. (Another way of looking at it is that Explorer is the grandchild of Winlogon.)
Winlogon is active not only during user logon and logoff but also whenever it intercepts the
SAS from the keyboard. For example, when you press Ctrl+Alt+Delete while logged in, the
Windows Security dialog box comes up, providing the options to log off, start the Task Manager, lock the workstation, shut down the system, and so forth. Winlogon is the process that
handles this interaction.
For a complete description of the steps involved in the logon process, see the section “Smss,
Csrss, and Winlogon” in Chapter 5. For more details on security authentication, see Chapter
8. For details on the callable functions that interface with LSASS (the functions that start with
Lsa), see the documentation in the Platform SDK.
Service Control Manager (SCM)
Recall from earlier in the chapter that “services” on Windows can refer either to a server process or to a device driver. This section deals with services that are user-mode processes. Services are like UNIX “daemon processes” or VMS “detached processes” in that they can be
configured to start automatically at system boot time without requiring an interactive logon.
They can also be started manually (such as by running the Services administrative tool or by
calling the Windows StartService function). Typically, services do not interact with the loggedon user, although there are special conditions when this is possible. (See Chapter 4.)
The service control manager is a special system process running the image \Windows\
System32\Services.exe that is responsible for starting, stopping, and interacting with service processes. Service programs are really just Windows images that call special Windows functions to
interact with the service control manager to perform such actions as registering the service’s successful startup, responding to status requests, or pausing or shutting down the service. Services
are defined in the registry under HKLM\SYSTEM\CurrentControlSet \Services. The resource kit
Registry Entries help file (Regentry.chm) documents the subkeys and values for services.
Keep in mind that services have three names: the process name you see running on the system, the internal name in the registry, and the display name shown in the Services administrative tool. (Not all services have a display name—if a service doesn’t have a display name, the
internal name is shown.) With Windows, services can also have a description field that further details what the service does.
To map a service process to the services contained in that process, use the tlist /s command. Note
that there isn’t always one-to-one mapping between service process and running services, however,
because some services share a process with other services. In the registry, the type code indicates
whether the service runs in its own process or shares a process with other services in the image.
A number of Windows components are implemented as services, such as the Spooler, Event
Log, Task Scheduler, and various networking components.
Copyrighted material.
Chapter 2:
System Architecture
EXPERIMENT: Listing Installed Services
To list the installed services, select Administrative Tools from Control Panel, and then
select Services. You should see output like this:
To see the detailed properties about a service, right-click on a service and select Properties. For example, here are the properties for the Print Spooler service (highlighted
in the previous figure):
Notice that the Path To Executable field identifies the program that contains this service. Remember that some services share a process with other services—mapping isn’t
always one to one.
Copyrighted material.
83
84
Microsoft Windows Internals, Fourth Edition
EXPERIMENT: Viewing Service Details Inside Service Processes
Process Explorer highlights processes hosting one service or more. (You can configure this
by selecting the Configure Highlighting entry in the Options menu.) If you double-click
on a service-hosting process, you will see a Services tab that lists the services inside the
process: the name of the registry key that defines the service, the display name seen by the
administrator, and the description text for that service (if present). For example, listing the
services in a Svchost.exe process on Windows XP running under the System account
looks like this:
For more details on services, see Chapter 4.
Conclusion
In this chapter, we’ve taken a broad look at the overall system architecture of Windows. We’ve
examined the key components of Windows and seen how they interrelate. In the next chapter, we’ll look in more detail at the core system mechanisms that these components are built
on, such as the object manager and synchronization.
Copyrighted material.
Chapter 3
System Mechanisms
Microsoft Windows provides several base mechanisms that kernel-mode components such as
the executive, the kernel, and device drivers use. This chapter explains the following system
mechanisms and describes how they are used:
■
Trap dispatching, including interrupts, deferred procedure calls (DPCs), asynchronous
procedure calls (APCs), exception dispatching, and system service dispatching
■
The executive object manager
■
Synchronization, including spinlocks, kernel dispatcher objects, and how waits are
implemented
■
System worker threads
■
Miscellaneous mechanisms such as Windows global flags
■
Local procedure calls (LPCs)
■
Kernel Event Tracing
■
Wow64
Trap Dispatching
Interrupts and exceptions are operating system conditions that divert the processor to code outside the normal flow of control. Either hardware or software can detect them. The term trap
refers to a processor’s mechanism for capturing an executing thread when an exception or an
interrupt occurs and transferring control to a fixed location in the operating system. In Windows, the processor transfers control to a trap handler, a function specific to a particular interrupt or exception. Figure 3-1 illustrates some of the conditions that activate trap handlers.
The kernel distinguishes between interrupts and exceptions in the following way. An interrupt
is an asynchronous event (one that can occur at any time) that is unrelated to what the processor is executing. Interrupts are generated primarily by I/O devices, processor clocks, or
timers, and they can be enabled (turned on) or disabled (turned off). An exception, in contrast,
is a synchronous condition that results from the execution of a particular instruction. Running a program a second time with the same data under the same conditions can reproduce
exceptions. Examples of exceptions include memory access violations, certain debugger
Copyrighted material.
85
86
Microsoft Windows Internals, Fourth Edition
instructions, and divide-by-zero errors. The kernel also regards system service calls as exceptions (although technically they’re system traps).
Trap handlers
Interrupt
Interrupt
service
routines
System service call
System
services
Hardware exceptions
Software exceptions
(Exception
frame)
Virtual address
exceptions
Figure 3-1
Exception
dispatcher
Exception
handlers
Virtual memory
manager’s
pager
Trap dispatching
Either hardware or software can generate exceptions and interrupts. For example, a bus error
exception is caused by a hardware problem, whereas a divide-by-zero exception is the result of
a software bug. Likewise, an I/O device can generate an interrupt, or the kernel itself can issue
a software interrupt (such as an APC or DPC, described later in this chapter).
When a hardware exception or interrupt is generated, the processor records enough machine
state on the kernel stack of the thread that’s interrupted so that it can return to that point in
the control flow and continue execution as if nothing had happened. If the thread was executing in user mode, Windows switches to the thread’s kernel-mode stack. Windows then creates a trap frame on the kernel stack of the interrupted thread into which it stores the
execution state of the thread. The trap frame is a subset of a thread’s complete context, and
you can view its definition by typing dt nt!_ktrap_frame in the kernel debugger. (Thread context is described in Chapter 6.) The kernel handles software interrupts either as part of hardware interrupt handling or synchronously when a thread invokes kernel functions related to
the software interrupt.
In most cases, the kernel installs front-end trap handling functions that perform general trap
handling tasks before and after transferring control to other functions that field the trap. For
example, if the condition was a device interrupt, a kernel hardware interrupt trap handler
transfers control to the interrupt service routine (ISR) that the device driver provided for the
interrupting device. If the condition was caused by a call to a system service, the general
Copyrighted material.
Chapter 3:
System Mechanisms
87
system service trap handler transfers control to the specified system service function in the
executive. The kernel also installs trap handlers for traps that it doesn’t expect to see or
doesn’t handle. These trap handlers typically execute the system function KeBugCheckEx,
which halts the computer when the kernel detects problematic or incorrect behavior that, if
left unchecked, could result in data corruption. (For more information on bug checks, see
Chapter 14.) The following sections describe interrupt, exception, and system service dispatching in greater detail.
Interrupt Dispatching
Hardware-generated interrupts typically originate from I/O devices that must notify the processor when they need service. Interrupt-driven devices allow the operating system to get the
maximum use out of the processor by overlapping central processing with I/O operations. A
thread starts an I/O transfer to or from a device and then can execute other useful work while
the device completes the transfer. When the device is finished, it interrupts the processor for
service. Pointing devices, printers, keyboards, disk drives, and network cards are generally
interrupt driven.
System software can also generate interrupts. For example, the kernel can issue a software
interrupt to initiate thread dispatching and to asynchronously break into the execution of a
thread. The kernel can also disable interrupts so that the processor isn’t interrupted, but it
does so only infrequently—at critical moments while it’s processing an interrupt or dispatching an exception, for example.
The kernel installs interrupt trap handlers to respond to device interrupts. Interrupt trap
handlers transfer control either to an external routine (the ISR) that handles the interrupt
or to an internal kernel routine that responds to the interrupt. Device drivers supply ISRs to
service device interrupts, and the kernel provides interrupt handling routines for other
types of interrupts.
In the following subsections, you’ll find out how the hardware notifies the processor of device
interrupts, the types of interrupts the kernel supports, the way device drivers interact with the
kernel (as a part of interrupt processing), and the software interrupts the kernel recognizes
(plus the kernel objects that are used to implement them).
Hardware Interrupt Processing
On the hardware platforms supported by Windows, external I/O interrupts come into one of
the lines on an interrupt controller. The controller in turn interrupts the processor on a single
line. Once the processor is interrupted, it queries the controller to get the interrupt request
(IRQ). The interrupt controller translates the IRQ to an interrupt number, uses this number
as an index into a structure called the interrupt dispatch table (IDT), and transfers control to
the appropriate interrupt dispatch routine. At system boot time, Windows fills in the IDT with
pointers to the kernel routines that handle each interrupt and exception.
Copyrighted material.
88
Microsoft Windows Internals, Fourth Edition
EXPERIMENT: Viewing the IDT
You can view the contents of the IDT, including information on what trap handlers Windows has assigned to interrupts (including exceptions and IRQs), using the !idt kernel
debugger command. The !idt command with no flags shows vectors that map to
addresses in modules other than Ntoskrnl.exe.
The following example shows what the output of the !idt command looks like:
kd> !idt
Dumping IDT:
30:
31:
34:
35:
806b14c0
8a39dc3c
8a436dd4
8a44ed74
38:
39:
3b:
806abe80
8a4a8abc
8a48d8c4
3c:
3e:
3f:
8a39d3ec
8a47264c
8a489b3c
hal!HalpClockInterrupt
i8042prt!I8042KeyboardInterruptService (KINTERRUPT 8a39dc00)
serial!SerialCIsrSw (KINTERRUPT 8a436d98)
NDIS!ndisMIsr (KINTERRUPT 8a44ed38)
portcls!CInterruptSync::Release+0x10 (KINTERRUPT 899c44a0)
hal!HalpProfileInterrupt
ACPI!ACPIInterruptServiceRoutine (KINTERRUPT 8a4a8a80)
pcmcia!PcmciaInterrupt (KINTERRUPT 8a48d888)
ohci1394!OhciIsr (KINTERRUPT 8a41da18)
VIDEOPRT!pVideoPortInterrupt (KINTERRUPT 8a1bc2c0)
USBPORT!USBPORT_InterruptService (KINTERRUPT 8a2302b8)
USBPORT!USBPORT_InterruptService (KINTERRUPT 8a0b8008)
USBPORT!USBPORT_InterruptService (KINTERRUPT 8a170008)
USBPORT!USBPORT_InterruptService (KINTERRUPT 8a258380)
NDIS!ndisMIsr (KINTERRUPT 8a0e0430)
i8042prt!I8042MouseInterruptService (KINTERRUPT 8a39d3b0)
atapi!IdePortInterrupt (KINTERRUPT 8a472610)
atapi!IdePortInterrupt (KINTERRUPT 8a489b00)
On the system used to provide the output for this experiment, the keyboard device
driver’s (I8042prt.sys) keyboard ISR is at interrupt number 0x3C and several devices—
including the video adapter, PCMCIA bus, USB and IEEE 1394 ports, and network
adapter—share interrupt 0x3B.
Windows maps hardware IRQs to interrupt numbers in the IDT, and the system also uses the
IDT to configure trap handlers for exceptions. For example, the x86 and x64 exception number for a page fault (an exception that occurs when a thread attempts to access a page of virtual memory that isn’t defined or present) is 0xe. Thus, entry 0xe in the IDT points to the
system’s page fault handler. Although the architectures supported by Windows allow up to
256 IDT entries, the number of IRQs a particular machine can support is determined by the
design of the interrupt controller the machine uses.
Copyrighted material.
Chapter 3:
System Mechanisms
89
Each processor has a separate IDT so that different processors can run different ISRs, if appropriate. For example, in a multiprocessor system, each processor receives the clock interrupt,
but only one processor updates the system clock in response to this interrupt. All the processors, however, use the interrupt to measure thread quantum and to initiate rescheduling when
a thread’s quantum ends. Similarly, some system configurations might require that a particular processor handle certain device interrupts.
x86 Interrupt Controllers
Most x86 systems rely on either the i8259A Programmable Interrupt Controller (PIC) or a
variant of the i82489 Advanced Programmable Interrupt Controller (APIC); the majority of
new computers include an APIC. The PIC standard originates with the original IBM PC. PICs
work only with uniprocessor systems and have 15 interrupt lines. APICs and SAPICs (discussed shortly) work with multiprocessor systems and have 256 interrupt lines. Intel and
other companies have defined the Multiprocessor Specification (MP Specification), a design
standard for x86 multiprocessor systems that centers on the use of APIC. To provide compatibility with uniprocessor operating systems and boot code that starts a multiprocessor system
in uniprocessor mode, APICs support a PIC compatibility mode with 15 interrupts and delivery of interrupts to only the primary processor. Figure 3-2 depicts the APIC architecture. The
APIC actually consists of several components: an I/O APIC that receives interrupts from
devices, local APICs that receive interrupts from the I/O APIC on a private APIC bus and that
interrupt the CPU they are associated with, and an i8259A-compatible interrupt controller
that translates APIC input into PIC-equivalent signals. The I/O APIC is responsible for implementing interrupt routing algorithms—which are software-selectable (the hardware abstraction layer, or HAL, makes the selection on Windows)—that both balance the device interrupt
load across processors and attempt to take advantage of locality, delivering device interrupts
to the same processor that has just fielded a previous interrupt of the same type.
CPU 0
CPU 1
Local
APIC
Local
APIC
Device
interrupts
Figure 3-2
Copyrighted material.
I/O
APIC
x86 APIC architecture
i8259Aequivalent
PIC
90
Microsoft Windows Internals, Fourth Edition
x64 Interrupt Controllers
Because the x64 architecture is compatible with x86 operating systems, x64 systems must
provide the same interrupt controllers as does the x86. A significant difference, however, is
that the x64 versions of Windows will not run on systems that do not have an APIC and they
use the APIC for interrupt control.
IA64 Interrupt Controllers
The IA64 architecture relies on the Streamlined Advanced Programmable Interrupt Controller
(SAPIC), which is an evolution of the APIC. A major difference between the APIC and SAPIC
architectures is that the I/O APICs on an APIC system deliver interrupts to local APICs over a
private APIC bus, whereas on a SAPIC system interrupts traverse the I/O and system bus for
faster delivery. Another difference is that interrupt routing and load balancing is handled by
the APIC bus on an APIC system, but a SAPIC system, which doesn’t have a private APIC bus,
requires that the support be programmed into the firmware. Even if load balancing and routing are present in the firmware, Windows does not take advantage of it; instead, it statically
assigns interrupts to processors in a round-robin manner.
EXPERIMENT: Viewing the PIC and APIC
You can view the configuration of the PIC on a uniprocessor and the APIC on a multiprocessor by using the !pic and !apic kernel debugger commands, respectively. (You
can’t use LiveKd for this experiment because LiveKd can’t access hardware.) Here’s the
output of the !pic command on a uniprocessor. (Note that the !pic command doesn’t
work if your system is using an APIC HAL.)
lkd> !pic
----- IRQ Number ----- 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
Physically in service: . . . . . . . . . . . . . . . .
Physically masked:
. . . Y . . Y Y . . Y . . Y . .
Physically requested:
. . . . . . . . . . . . . . . .
Level Triggered:
. . . . . Y . . . Y . Y . . . .
Here’s the output of the !apic command on a system running with the MPS HAL. The
“0:” prefix for the debugger prompt indicates that commands are running on processor
0, so this is the I/O APIC for processor 0:
lkd> !apic
Apic @ fffe0000 ID:0 (40010) LogDesc:01000000 DestFmt:ffffffff
TimeCnt: 0bebc200clk SpurVec:3f FaultVec:e3 error:0
Ipi Cmd: 0004001f Vec:1F FixedDel
Dest=Self
edg high
Timer..: 000300fd Vec:FD FixedDel
Dest=Self
edg high
Linti0.: 0001003f Vec:3F FixedDel
Dest=Self
edg high
Linti1.: 000184ff Vec:FF NMI
Dest=Self
lvl high
TMR: 61, 82, 91-92, B1
IRR:
ISR:
Copyrighted material.
TPR 20
masked
masked
masked
Chapter 3:
System Mechanisms
91
The following output is for the !ioapic command, which displays the configuration of the
I/O APIC, the interrupt controller component connected to devices:
0: kd> !ioapic
IoApic @ ffd02000
Inti00.: 000100ff
Inti01.: 00000962
Inti02.: 000100ff
Inti03.: 00000971
Inti04.: 000100ff
Inti05.: 00000961
Inti06.: 00010982
Inti07.: 000100ff
Inti08.: 000008d1
Inti09.: 000100ff
Inti0A.: 000100ff
Inti0B.: 000100ff
Inti0C.: 00000972
Inti0D.: 000100ff
Inti0E.: 00000992
Inti0F.: 000100ff
Inti10.: 000100ff
Inti11.: 000100ff
Inti12.: 000100ff
Inti13.: 000100ff
Inti14.: 0000a9a3
Inti15.: 0000a993
Inti16.: 000100ff
Inti17.: 000100ff
ID:8 (11) Arb:0
Vec:FF FixedDel
Vec:62 LowestDl
Vec:FF FixedDel
Vec:71 LowestDl
Vec:FF FixedDel
Vec:61 LowestDl
Vec:82 LowestDl
Vec:FF FixedDel
Vec:D1 FixedDel
Vec:FF FixedDel
Vec:FF FixedDel
Vec:FF FixedDel
Vec:72 LowestDl
Vec:FF FixedDel
Vec:92 LowestDl
Vec:FF FixedDel
Vec:FF FixedDel
Vec:FF FixedDel
Vec:FF FixedDel
Vec:FF FixedDel
Vec:A3 LowestDl
Vec:93 LowestDl
Vec:FF FixedDel
Vec:FF FixedDel
PhysDest:00
Lg:03000000
PhysDest:00
Lg:03000000
PhysDest:00
Lg:03000000
Lg:02000000
PhysDest:00
Lg:01000000
PhysDest:00
PhysDest:00
PhysDest:00
Lg:03000000
PhysDest:00
Lg:03000000
PhysDest:00
PhysDest:00
PhysDest:00
PhysDest:00
PhysDest:00
Lg:03000000
Lg:03000000
PhysDest:00
PhysDest:00
edg
edg
edg
edg
edg
edg
edg
edg
edg
edg
edg
edg
edg
edg
edg
edg
edg
edg
edg
edg
lvl
lvl
edg
edg
masked
masked
masked
masked
masked
masked
masked
masked
masked
masked
masked
masked
masked
masked
masked
masked
Software Interrupt Request Levels (IRQLs)
Although interrupt controllers perform a level of interrupt prioritization, Windows imposes
its own interrupt priority scheme known as interrupt request levels (IRQLs). The kernel represents IRQLs internally as a number from 0 through 31 on x86 and from 0 to 15 on x64 and
IA64, with higher numbers representing higher-priority interrupts. Although the kernel
defines the standard set of IRQLs for software interrupts, the HAL maps hardware-interrupt
numbers to the IRQLs. Figure 3-3 shows IRQLs defined for the x86 architecture, and Figure
3-4 shows IRQLs for the x64 and IA64 architectures.
Note
SYNCH_LEVEL, which multiprocessor versions of the kernel use to protect access to
per-processor processor control blocks (PRCB), is not shown in the charts because its value varies across different versions of Windows. See Chapter 6 for a description of SYNCH_LEVEL and
its possible values.
Copyrighted material.
92
Microsoft Windows Internals, Fourth Edition
High
31
30
Power fail
29
Inter-processor interrupt
28
Clock
27
Profile
26
Device n
Hardware interrupts
• • •
3
Device 1
2
DPC/dispatch
1
APC
0
Passive
Figure 3-3
Software interrupts
Normal thread execution
x86 interrupt request levels (IRQLs)
IA64
15
High/Profile
High/Profile/Power
14
Inter-processor interrupt/Power
Inter-processor interrupt
13
Clock
Clock
12
Synch (Srv 2003)
Synch (MP only)
11
Device n
Device n
4
…
Device 1
…
x64
3
Device 1
Correctable Machine Check
2
Dispatch/DPC
Dispatch/DPC & Synch (UP only)
1
APC
APC
0
Passive/Low
Passive/Low
Figure 3-4
x64 and IA64 interrupt request levels (IRQLs)
Interrupts are serviced in priority order, and a higher-priority interrupt preempts the servicing
of a lower-priority interrupt. When a high-priority interrupt occurs, the processor saves the
interrupted thread’s state and invokes the trap dispatchers associated with the interrupt. The
trap dispatcher raises the IRQL and calls the interrupt’s service routine. After the service routine executes, the interrupt dispatcher lowers the processor’s IRQL to where it was before the
interrupt occurred and then loads the saved machine state. The interrupted thread resumes
executing where it left off. When the kernel lowers the IRQL, lower-priority interrupts that
were masked might materialize. If this happens, the kernel repeats the process to handle the
new interrupts.
IRQL priority levels have a completely different meaning than thread-scheduling priorities
(which are described in Chapter 6). A scheduling priority is an attribute of a thread, whereas
Copyrighted material.
Chapter 3:
System Mechanisms
93
an IRQL is an attribute of an interrupt source, such as a keyboard or a mouse. In addition,
each processor has an IRQL setting that changes as operating system code executes.
Each processor’s IRQL setting determines which interrupts that processor can receive. IRQLs
are also used to synchronize access to kernel-mode data structures. (You’ll find out more
about synchronization later in this chapter.) As a kernel-mode thread runs, it raises or lowers
the processor’s IRQL either directly by calling KeRaiseIrql and KeLowerIrql or, more commonly, indirectly via calls to functions that acquire kernel synchronization objects. As Figure
3-5 illustrates, interrupts from a source with an IRQL above the current level interrupt the
processor, whereas interrupts from sources with IRQLs equal to or below the current level are
masked until an executing thread lowers the IRQL.
IRQL setting
High
Power fail
Processor A
IRQL = Clock
Inter-processor interrupt
Clock
Profile
Device n
• • •
Interrupts masked on
Processor A
Processor B
Device 1
DPC/dispatch
APC
IRQL = DPC/dispatch
Interrupts masked on
Processor B
Passive
Figure 3-5
Masking interrupts
Because accessing a PIC is a relatively slow operation, HALs that use a PIC implement a performance optimization, called lazy IRQL, that avoids PIC accesses. When the IRQL is raised,
the HAL notes the new IRQL internally instead of changing the interrupt mask. If a lower-priority interrupt subsequently occurs, the HAL sets the interrupt mask to the settings appropriate for the first interrupt and postpones the lower-priority interrupt until the IRQL is lowered.
Thus, if no lower-priority interrupts occur while the IRQL is raised, the HAL doesn’t need to
modify the PIC.
A kernel-mode thread raises and lowers the IRQL of the processor on which it’s running,
depending on what it’s trying to do. For example, when an interrupt occurs, the trap handler
(or perhaps the processor) raises the processor’s IRQL to the assigned IRQL of the interrupt
source. This elevation masks all interrupts at and below that IRQL (on that processor only),
which ensures that the processor servicing the interrupt isn’t waylaid by an interrupt at the
same or a lower level. The masked interrupts are either handled by another processor or held
Copyrighted material.
94
Microsoft Windows Internals, Fourth Edition
back until the IRQL drops. Therefore, all components of the system, including the kernel and
device drivers, attempt to keep the IRQL at passive level (sometimes called low level). They do
this because device drivers can respond to hardware interrupts in a timelier manner if the
IRQL isn’t kept unnecessarily elevated for long periods.
Note
An exception to the rule that raising the IRQL blocks interrupts of that level and lower
relates to APC_LEVEL interrupts. If a thread raises the IRQL to APC_LEVEL and then is rescheduled because of a DISPATCH_LEVEL interrupt, the system might deliver an APC_LEVEL interrupt
to the newly scheduled thread. Thus, APC_LEVEL can be considered a thread-local rather than
processor-wide IRQL.
EXPERIMENT: Viewing the IRQL
If you are running the kernel debugger on Windows Server 2003, you can view a processor’s IRQL with the !irql debugger command:
kd> !irql
Debugger saved IRQL for processor 0x0 -- 0 (LOW_LEVEL)
Note that there is a field called IRQL in a data structure called the processor control region
(PCR) and its extension the processor control block (PRCB), which contain information
about the state of each processor in the system, such as the current IRQL, a pointer to
the hardware IDT, the currently running thread, and the next thread selected to run.
The kernel and the HAL use this information to perform architecture-specific and
machine-specific actions. Portions of the PCR and PRCB structures are defined publicly
in the Windows Device Driver Kit (DDK) header file Ntddk.h, so examine that file if you
want a complete definition of these structures.
You can view the contents of the PCR with the kernel debugger by using the !pcr command:
kd> !pcr
PCR Processor 0 @ffdff000
NtTib.ExceptionList:
NtTib.StackBase:
NtTib.StackLimit:
NtTib.SubSystemTib:
NtTib.Version:
NtTib.UserPointer:
NtTib.SelfTib:
Copyrighted material.
f8effc68
f8effdf0
f8efd000
00000000
00000000
00000000
7ffde000
Chapter 3:
SelfPcr:
Prcb:
Irql:
IRR:
IDR:
InterruptMode:
IDT:
GDT:
TSS:
System Mechanisms
95
ffdff000
ffdff120
00000000
00000000
ffff28e8
00000000
80036400
80036000
802b5000
CurrentThread: 81638020
NextThread: 00000000
IdleThread: 8046bdf0
Unfortunately, Windows does not maintain the Irql field on systems that do not use lazy
IRQL, so on most systems the field will always be 0.
Because changing a processor’s IRQL has such a significant effect on system operation, the
change can be made only in kernel mode—user-mode threads can’t change the processor’s
IRQL. This means that a processor’s IRQL is always at passive level when it’s executing usermode code. Only when the processor is executing kernel-mode code can the IRQL be higher.
Each interrupt level has a specific purpose. For example, the kernel issues an interprocessor
interrupt (IPI) to request that another processor perform an action, such as dispatching a particular thread for execution or updating its translation look-aside buffer cache. The system
clock generates an interrupt at regular intervals, and the kernel responds by updating the
clock and measuring thread execution time. If a hardware platform supports two clocks, the
kernel adds another clock interrupt level to measure performance. The HAL provides a number
of interrupt levels for use by interrupt-driven devices; the exact number varies with the processor and system configuration. The kernel uses software interrupts (described later in this chapter) to initiate thread scheduling and to asynchronously break into a thread’s execution.
Mapping Interrupts to IRQLs IRQL levels aren’t the same as the interrupt requests (IRQs)
defined by interrupt controllers—the architectures on which Windows runs don’t implement
the concept of IRQLs in hardware. So how does Windows determine what IRQL to assign to
an interrupt? The answer lies in the HAL. In Windows, a type of device driver called a bus
driver determines the presence of devices on its bus (PCI, USB, and so on) and what interrupts can be assigned to a device. The bus driver reports this information to the Plug and Play
manager, which decides, after taking into account the acceptable interrupt assignments for all
other devices, which interrupt will be assigned to each device. Then it calls the HAL function
HalpGetSystemInterruptVector, which maps interrupts to IRQLs.
Copyrighted material.
96
Microsoft Windows Internals, Fourth Edition
The algorithm for assignment differs for the various HALs that Windows includes. On uniprocessor x86 systems, the HAL performs a straightforward translation: the IRQL of a given interrupt vector is calculated by subtracting the interrupt vector from 27. Thus, if a device uses
interrupt vector 5, its ISR executes at IRQL 22. On an x86 multiprocessor system, the mapping isn’t as simple. APICs support over 200 interrupt vectors, so there aren’t enough IRQLs
for a one-to-one correspondence. The multiprocessor HAL therefore assigns IRQLs to interrupt vectors in a round-robin manner, cycling through the device IRQL (DIRQL) range. As a
result, on an x86 multiprocessor system there’s no easy way for you to predict or to know
what IRQL Windows assigns to APIC IRQs. Finally, on x64 and IA64 systems, the HAL computes the IRQL for a given IRQ by dividing the interrupt vector assigned to the IRQ by 16.
Predefined IRQLs Let’s take a closer look at the use of the predefined IRQLs, starting from
the highest level shown in Figure 3-5:
■
The kernel uses high level only when it’s halting the system in KeBugCheckEx and masking out all interrupts.
■
Power fail level originated in the original Microsoft Windows NT design documents,
which specified the behavior of system power failure code, but this IRQL has never been
used.
■
Inter-processor interrupt level is used to request another processor to perform an action,
such as queue a DISPATCH_LEVEL interrupt to schedule a particular thread for execution, updating the processor’s translation look-aside buffer (TLB) cache, system shutdown, or system crash.
■
Clock level is used for the system’s clock, which the kernel uses to track the time of day
as well as to measure and allot CPU time to threads.
■
The system’s real-time clock uses profile level when kernel profiling, a performance measurement mechanism, is enabled. When kernel profiling is active, the kernel’s profiling
trap handler records the address of the code that was executing when the interrupt
occurred. A table of address samples is constructed over time that tools can extract and
analyze. You can download Kernrate, a kernel profiling tool that you can use to configure and view profiling-generated statistics, from http://www.microsoft.com/whdc/system/sysperf/krview.mspx. See the Kernrate experiment for more information on using
this tool.
■
The device IRQLs are used to prioritize device interrupts. (See the previous section for
how hardware interrupt levels are mapped to IRQLs.)
■
DPC/dispatch-level and APC-level interrupts are software interrupts that the kernel and
device drivers generate. (DPCs and APCs are explained in more detail later in this chapter.)
■
The lowest IRQL, passive level, isn’t really an interrupt level at all; it’s the setting at which
normal thread execution takes place and all interrupts are allowed to occur.
Copyrighted material.
Chapter 3:
System Mechanisms
EXPERIMENT: Using Kernel Profiler to Profile Execution
You can use the Kernel Profiler tool to enable the system profiling timer, collect samples
of the code that is executing when the timer fires, and display a summary showing the
frequency distribution across image files and functions. It can be used to track CPU
usage consumed by individual processes and/or time spent in kernel mode independent of processes (for example, interrupt service routines). Kernel profiling is useful
when you want to obtain a breakdown of where the system is spending time.
In its simplest form, Kernrate samples where time has been spent in each kernel module
(for example, Ntoskrnl, drivers, and so on). For example, after installing the Krview
package referred to previously, try performing the following steps:
1. Open a command prompt.
2. Type cd c:\program files\krview\kernrates.
3. Type dir. (You will see kernrate images for each platform.)
4. Run the image that matches your platform (with no arguments or switches). For
example, Kernrate_i386_XP.exe is the image for Windows XP running on an x86
system.
5. While Kernrate is running, go perform some other activity on the system. For example, run Windows Media Player and play some music, run a graphics-intensive game,
or perform network activity such as doing a directory of a remote network share.
6. Press Ctrl+C to stop Kernrate. This causes Kernrate to display the statistics from
the sampling period.
In the sample partial output from Kernrate, Windows Media Player was running, playing a track from a CD.
C:\Program Files\KrView\Kernrates>Kernrate_i386_XP.exe
/==============================\
<
KERNRATE LOG
>
\==============================/
Date: 2004/05/13
Time: 9:48:28
Machine Name: BIGDAVID
Number of Processors: 1
PROCESSOR_ARCHITECTURE: x86
PROCESSOR_LEVEL: 6
Kernrate User-Specified Command Line:
Kernrate_i386_XP.exe
***> Press ctrl-c to finish collecting profile data
===> Finished Collecting Data, Starting to Process Results
------------Overall Summary:-------------P0
K 0:00:03.234 (11.7%) U 0:00:08.352 (30.2%) I 0:00:16.093 (58.1%)
DPC 0:00:01.772 ( 6.4%) Interrupt 0:00:00.350 ( 1.3%)
Copyrighted material.
97
98
Microsoft Windows Internals, Fourth Edition
Interrupts= 52899, Interrupt Rate= 1911/
sec.Time
7315 hits, 19531 events per hit -------Module
Hits
msec %Total Events/Sec
gv3
4735
27679
64 %
3341135
smwdm
872
27679
11 %
615305
win32k
764
27679
10 %
539097
ntoskrnl
739
27679
10 %
521457
hal
124
27679
1 %
87497
The overall summary shows that the system spent 11.7 percent of the time in kernel
mode, 30.2 percent in user mode, 58.1 percent idle, 6.4 percent at DPC level, and 1.3
percent at interrupt level. The module with the highest hit rate was GV3.SYS, the processor driver for the Pentium M Geyserville family. It is used for performance collection,
which is why it is first. The module with the second highest hit rate was Smwdm.sys, the
audio driver for the sound card on the machine used for the test. This makes sense
because the major activity going on in the system was Windows Media Player sending
sound I/O to the sound driver.
If you have symbols available, you can zoom in on individual modules and see the time
spent by function name. For example, profiling the system while dragging a window
around the screen rapidly resulted in the following (partial) output:
C:\Program Files\KrView\Kernrates>Kernrate_i386_XP.exe -z ntoskrnl -z win32k
/==============================\
<
KERNRATE LOG
>
\==============================/
Date: 2004/05/13
Time: 10:26:55
Time
4087 hits, 19531 events per hit -------Module
Hits
msec %Total Events/Sec
win32k
1649
10424
40 %
3089660
ati2dvag
1269
10424
31 %
2377670
ntoskrnl
794
10424
19 %
1487683
gv3
162
10424
3 %
303532
----- Zoomed module win32k.sys (Bucket size = 16 bytes, Rounding Down) ------Module
Hits
msec %Total Events/Sec
EngPaint
328
10424
19 %
614559
EngLpkInstalled
302
10424
18 %
565844
----- Zoomed module ntoskrnl.exe (Bucket size = 16 bytes, Rounding Down) ----Module
Hits
msec %Total Events/Sec
KiDispatchInterrupt
243
10424
26 %
455298
ZwYieldExecution
50
10424
5 %
93682
InterlockedDecrement
39
10424
4 %
73072
The module with the highest hit rate was Win32k.sys, the windowing system driver.
Second on the list was the video driver. These results make sense because the main
activity in the system was drawing on the screen. Note in the zoomed display for
Win32k.sys, the function with the highest hit was EngPaint, the main GDI function to
paint on the screen.
Copyrighted material.
Chapter 3:
System Mechanisms
99
One important restriction on code running at DPC/dispatch level or above is that it can’t wait
for an object if doing so would necessitate the scheduler to select another thread to execute,
which is an illegal operation because the scheduler synchronizes its data structures at DPC/
dispatch level and cannot therefore be invoked to perform a reschedule. Another restriction is
that only nonpaged memory can be accessed at IRQL DPC/dispatch level or higher. This rule
is actually a side-effect of the first restriction because attempting to access memory that isn’t
resident results in a page fault. When a page fault occurs, the memory manager initiates a disk
I/O and then needs to wait for the file system driver to read the page in from disk. This wait
would in turn require the scheduler to perform a context switch (perhaps to the idle thread if
no user thread is waiting to run), thus violating the rule that the scheduler can’t be invoked
(because the IRQL is still DPC/dispatch level or higher at the time of the disk read). If either
of these two restrictions is violated, the system crashes with an
IRQL_NOT_LESS_OR_EQUAL crash code. (See Chapter 4 for a thorough discussion of system crashes.) Violating these restrictions is a common bug in device drivers. The Windows
Driver Verifier, explained in the section “Driver Verifier” in Chapter 7, has an option you can
set to assist in finding this particular type of bug.
Interrupt Objects The kernel provides a portable mechanism—a kernel control object
called an interrupt object—that allows device drivers to register ISRs for their devices. An interrupt object contains all the information the kernel needs to associate a device ISR with a particular level of interrupt, including the address of the ISR, the IRQL at which the device
interrupts, and the entry in the kernel’s IDT with which the ISR should be associated. When
an interrupt object is initialized, a few instructions of assembly language code, called the dispatch code, are copied from an interrupt handling template, KiInterruptTemplate, and stored
in the object. When an interrupt occurs, this code is executed.
This interrupt-object resident code calls the real interrupt dispatcher, which is typically either
the kernel’s KiInterruptDispatch or KiChainedDispatch routine, passing it a pointer to the interrupt object. KiInterruptDispatch is the routine used for interrupt vectors for which only one
interrupt object is registered, and KiChainedDispatch is for vectors shared among multiple
interrupt objects. The interrupt object contains information this second dispatcher routine
needs to locate and properly call the ISR the device driver provides. The interrupt object also
stores the IRQL associated with the interrupt so that KiInterruptDispatch or KiChainedDispatch can raise the IRQL to the correct level before calling the ISR and then lower the IRQL
after the ISR has returned. This two-step process is required because there’s no way to pass a
pointer to the interrupt object (or any other argument for that matter) on the initial dispatch
because the initial dispatch is done by hardware. On a multiprocessor system, the kernel allocates and initializes an interrupt object for each CPU, enabling the local APIC on that CPU to
accept the particular interrupt. Figure 3-6 shows typical interrupt control flow for interrupts
associated with interrupt objects.
Copyrighted material.
100
Microsoft Windows Internals, Fourth Edition
JP
00 3
93 8N
8 F
0
2
3
Peripheral Device
Controller
CPU Interrupt
Controller
n
CPU Interrupt
Dispatch Table
ISR Address
Spinlock
Dispatch
Code
Interrupt
Object
Read from device
Raise IRQL
Grab Spinlock
Drop Spinlock
Request DPC
Lower IRQL
KiInterruptDispatch
Figure 3-6
AcknowledgeInterrupt
Driver ISR
Typical interrupt control flow
EXPERIMENT: Examining Interrupt Internals
Using the kernel debugger, you can view details of an interrupt object, including its
IRQL, ISR address, and custom interrupt dispatching code. First, execute the !idt command and locate the entry that includes a reference to I8042KeyboardInterruptService,
the ISR routine for the PS2 keyboard device:
31:
8a39dc3c i8042prt!I8042KeyboardInterruptService (KINTERRUPT 8a39dc00)
To view the contents of the interrupt object associated with the interrupt, execute dt
nt!_kinterrupt with the address following KINTERRUPT:
kd> dt nt!_kinterrupt 8a39dc00
nt!_KINTERRUPT
+0x000 Type
: 22
+0x002 Size
: 484
+0x004 InterruptListEntry : _LIST_ENTRY [ 0x8a39dc04 - 0x8a39dc04 ]
+0x00c ServiceRoutine
: 0xba7e74a2
i8042prt!I8042KeyboardInterruptService+0
+0x010 ServiceContext
: 0x8a067898
+0x014 SpinLock
: 0
+0x018 TickCount
: 0xffffffff
+0x01c ActualLock
: 0x8a067958 -> 0
+0x020 DispatchAddress : 0x80531140
nt!KiInterruptDispatch+0
+0x024 Vector
: 0x31
+0x028 Irql
: 0x1a ’’
+0x029 SynchronizeIrql : 0x1a ’’
+0x02a FloatingSave
: 0 ’’
Copyrighted material.
Chapter 3:
+0x02b
+0x02c
+0x02d
+0x030
+0x034
+0x038
+0x03c
Connected
Number
ShareVector
Mode
ServiceCount
DispatchCount
DispatchCode
:
:
:
:
:
:
:
System Mechanisms
101
0x1 ’’
0 ’’
0 ’’
1 ( Latched )
0
0xffffffff
[106] 0x56535554
In this example, the IRQL Windows assigned to the interrupt is 0x1a (which is 26 in
decimal). Because this output is from a uniprocessor x86 system, we calculate that the
IRQ is 1, because IRQLs on x86 uniprocessors are calculated by subtracting the IRQ
from 27. We can verify this by opening the Device Manager (on the Hardware tab in the
System applet in the Control Panel), locating the PS/2 keyboard device, and viewing its
resource assignments, as shown in the following figure.
On a multiprocessor x86, the IRQ will be essentially randomly assigned, and on an x64
or IA64 system you will see that the IRQ is the interrupt vector number (0x31—49 decimal—in this example) divided by 16.
The ISR’s address for the interrupt object is stored in the ServiceRoutine field (which is
what !idt displays in its output), and the interrupt code that actually executes when an
interrupt occurs is stored in the DispatchCode array at the end of the interrupt object.
The interrupt code stored there is programmed to build the trap frame on the stack and
then call the function stored in the DispatchAddress field (KiInterruptDispatch in the
example), passing it a pointer to the interrupt object.
Copyrighted material.
102
Microsoft Windows Internals, Fourth Edition
Windows and Real-Time Processing
Deadline requirements, either hard or soft, characterize real-time environments. Hard
real-time systems (for example, a nuclear power plant control system) have deadlines
that the system must meet to avoid catastrophic failures such as loss of equipment or
life. Soft real-time systems (for example, a car’s fuel-economy optimization system) have
deadlines that the system can miss, but timeliness is still a desirable trait. In real-time
systems, computers have sensor input devices and control output devices. The designer
of a real-time computer system must know worst-case delays between the time an input
device generates an interrupt and the time the device’s driver can control the output
device to respond. This worst-case analysis must take into account the delays the operating system introduces as well as the delays the application and device drivers impose.
Because Windows doesn’t prioritize device IRQs in any controllable way and user-level
applications execute only when a processor’s IRQL is at passive level, Windows isn’t
always suitable as a real-time operating system. The system’s devices and device drivers—not Windows—ultimately determine the worst-case delay. This factor becomes a
problem when the real-time system’s designer uses off-the-shelf hardware. The designer
can have difficulty determining how long every off-the-shelf device’s ISR or DPC might
take in the worst case. Even after testing, the designer can’t guarantee that a special case
in a live system won’t cause the system to miss an important deadline. Furthermore, the
sum of all the delays a system’s DPCs and ISRs can introduce usually far exceeds the tolerance of a time-sensitive system.
Although many types of embedded systems (for example, printers and automotive computers) have real-time requirements, Windows XP Embedded doesn’t have real-time
characteristics. It is simply a version of Windows XP that makes it possible, using system
designer technology that Microsoft licensed from VenturCom, to produce small-footprint versions of Windows XP suitable for running on devices with limited resources.
For example, a device that has no networking capability would omit all the Windows XP
components related to networking, including network management tools and adapter
and protocol stack device drivers.
Still, there are third-party vendors that supply real-time kernels for Windows. The
approach these vendors take is to embed their real-time kernel in a custom HAL and to
have Windows run as a task in the real-time operating system. The task running Windows serves as the user interface to the system and has a lower priority than the tasks
responsible for managing the device. See VenturCom’s Web site, www.venturcom.com, for
an example of a third-party real-time kernel extension for Windows.
Associating an ISR with a particular level of interrupt is called connecting an interrupt object,
and dissociating an ISR from an IDT entry is called disconnecting an interrupt object. These
Copyrighted material.
Chapter 3:
System Mechanisms
103
operations, accomplished by calling the kernel functions IoConnectInterrupt and IoDisconnectInterrupt, allow a device driver to “turn on” an ISR when the driver is loaded into the system and to “turn off” the ISR if the driver is unloaded.
Using the interrupt object to register an ISR prevents device drivers from fiddling directly
with interrupt hardware (which differs among processor architectures) and from needing to
know any details about the IDT. This kernel feature aids in creating portable device drivers
because it eliminates the need to code in assembly language or to reflect processor differences
in device drivers.
Interrupt objects provide other benefits as well. By using the interrupt object, the kernel
can synchronize the execution of the ISR with other parts of a device driver that might share
data with the ISR. (See Chapter 9 for more information about how device drivers respond to
interrupts.)
Furthermore, interrupt objects allow the kernel to easily call more than one ISR for any interrupt level. If multiple device drivers create interrupt objects and connect them to the same
IDT entry, the interrupt dispatcher calls each routine when an interrupt occurs at the specified interrupt line. This capability allows the kernel to easily support “daisy-chain” configurations, in which several devices share the same interrupt line. The chain breaks when one of
the ISRs claims ownership for the interrupt by returning a status to the interrupt dispatcher.
If multiple devices sharing the same interrupt require service at the same time, devices not
acknowledged by their ISRs will interrupt the system again once the interrupt dispatcher has
lowered the IRQL. Chaining is permitted only if all the device drivers wanting to use the same
interrupt indicate to the kernel that they can share the interrupt; if they can’t, the Plug and
Play manager reorganizes their interrupt assignments to ensure that it honors the sharing
requirements of each. If the interrupt vector is shared, the interrupt object invokes KiChainedDispatch, which will invoke the ISRs of each registered interrupt object in turn until one of
them claims the interrupt or all have been executed. In the earlier sample !idt output, vector
0x3b is connected to several chained interrupt objects.
Software Interrupts
Although hardware generates most interrupts, the Windows kernel also generates software
interrupts for a variety of tasks, including these:
■
Initiating thread dispatching
■
Non-time-critical interrupt processing
■
Handling timer expiration
■
Asynchronously executing a procedure in the context of a particular thread
■
Supporting asynchronous I/O operations
These tasks are described in the following subsections.
Copyrighted material.
104
Microsoft Windows Internals, Fourth Edition
Dispatch or Deferred Procedure Call (DPC) Interrupts When a thread can no longer continue executing, perhaps because it has terminated or because it voluntarily enters a wait state,
the kernel calls the dispatcher directly to effect an immediate context switch. Sometimes, however, the kernel detects that rescheduling should occur when it is deep within many layers of
code. In this situation, the kernel requests dispatching but defers its occurrence until it completes its current activity. Using a DPC software interrupt is a convenient way to achieve this
delay.
The kernel always raises the processor’s IRQL to DPC/dispatch level or above when it needs
to synchronize access to shared kernel structures. This disables additional software interrupts
and thread dispatching. When the kernel detects that dispatching should occur, it requests a
DPC/dispatch-level interrupt; but because the IRQL is at or above that level, the processor
holds the interrupt in check. When the kernel completes its current activity, it sees that it’s
going to lower the IRQL below DPC/dispatch level and checks to see whether any dispatch
interrupts are pending. If there are, the IRQL drops to DPC/dispatch level and the dispatch
interrupts are processed. Activating the thread dispatcher by using a software interrupt is a
way to defer dispatching until conditions are right. However, Windows uses software interrupts to defer other types of processing as well.
In addition to thread dispatching, the kernel also processes deferred procedure calls (DPCs)
at this IRQL. A DPC is a function that performs a system task—a task that is less time-critical
than the current one. The functions are called deferred because they might not execute immediately.
DPCs provide the operating system with the capability to generate an interrupt and execute a
system function in kernel mode. The kernel uses DPCs to process timer expiration (and
release threads waiting for the timers) and to reschedule the processor after a thread’s quantum expires. Device drivers use DPCs to complete I/O requests. To provide timely service for
hardware interrupts, Windows—with the cooperation of device drivers—attempts to keep the
IRQL below device IRQL levels. One way that this goal is achieved is for device driver ISRs to
perform the minimal work necessary to acknowledge their device, save volatile interrupt state,
and defer data transfer or other less time-critical interrupt processing activity for execution in
a DPC at DPC/dispatch IRQL. (See Chapter 9 for more information on DPCs and the I/O system.)
A DPC is represented by a DPC object, a kernel control object that is not visible to user-mode
programs but is visible to device drivers and other system code. The most important piece of
information the DPC object contains is the address of the system function that the kernel will
call when it processes the DPC interrupt. DPC routines that are waiting to execute are stored
in kernel-managed queues, one per processor, called DPC queues. To request a DPC, system
code calls the kernel to initialize a DPC object and then places it in a DPC queue.
By default, the kernel places DPC objects at the end of the DPC queue of the processor on
which the DPC was requested (typically the processor on which the ISR executed). A device
Copyrighted material.
Chapter 3:
System Mechanisms
105
driver can override this behavior, however, by specifying a DPC priority (low, medium, or
high, where medium is the default) and by targeting the DPC at a particular processor. A DPC
aimed at a specific CPU is known as a targeted DPC. If the DPC has a low or medium priority,
the kernel places the DPC object at the end of the queue; if the DPC has a high priority, the
kernel inserts the DPC object at the front of the queue.
When the processor’s IRQL is about to drop from an IRQL of DPC/dispatch level or higher to
a lower IRQL (APC or passive level), the kernel processes DPCs. Windows ensures that the
IRQL remains at DPC/dispatch level and pulls DPC objects off the current processor’s queue
until the queue is empty (that is, the kernel “drains” the queue), calling each DPC function in
turn. Only when the queue is empty will the kernel let the IRQL drop below DPC/dispatch
level and let regular thread execution continue. DPC processing is depicted in Figure 3-7.
DPC
IRQL setting
1 A timer expires, and the kernel
table
queues a DPC that will release
High
any threads waiting on the
timer. The kernel then
Power failure
requests a software interrupt.
• • •
2 When the IRQL drops below
DPC/dispatch level, a DPC
interrupt occurs.
DPC
DPC queue
DPC/dispatch
APC
Passive
3 After the DPC interrupt,
control transfers to the
(thread) dispatcher.
Dispatcher
DPC
4 The dispatcher executes each DPC routine
in the DPC queue, emptying the queue as
it proceeds. If required, the dispatcher also
reschedules the processor.
Figure 3-7
Delivering a DPC
DPC priorities can affect system behavior another way. The kernel usually initiates DPC queue
draining with a DPC/dispatch-level interrupt. The kernel generates such an interrupt only if
the DPC is directed at the processor the ISR is requested on and the DPC has a high or
medium priority. If the DPC has a low priority, the kernel requests the interrupt only if the
number of outstanding DPC requests for the processor rises above a threshold or if the number of DPCs requested on the processor within a time window is low. If a DPC is targeted at a
CPU different from the one on which the ISR is running and the DPC’s priority is high, the
kernel immediately signals the target CPU (by sending it a dispatch IPI) to drain its DPC
queue. If the priority is medium or low, the number of DPCs queued on the target processor
must exceed a threshold for the kernel to trigger a DPC/dispatch interrupt. The system idle
thread also drains the DPC queue for the processor it runs on. Although DPC targeting and
Copyrighted material.
106
Microsoft Windows Internals, Fourth Edition
priority levels are flexible, device drivers rarely need to change the default behavior of their
DPC objects. Table 3-1 summarizes the situations that initiate DPC queue draining.
Table 3-1 DPC Interrupt Generation Rules
DPC Priority
DPC Targeted at ISR’s Processor
DPC Targeted at Another Processor
Low
DPC queue length exceeds maximum DPC queue length or DPC
request rate is less than minimum
DPC request rate
DPC queue length exceeds maximum
DPC queue length or System is idle
Medium
Always
DPC queue length exceeds maximum
DPC queue length or System is idle
High
Always
Always
Because user-mode threads execute at low IRQL, the chances are good that a DPC will interrupt the execution of an ordinary user’s thread. DPC routines execute without regard to what
thread is running, meaning that when a DPC routine runs, it can’t assume what process
address space is currently mapped. DPC routines can call kernel functions, but they can’t call
system services, generate page faults, or create or wait for dispatcher objects (explained later
in this chapter). They can, however, access nonpaged system memory addresses, because system address space is always mapped regardless of what the current process is.
DPCs are provided primarily for device drivers, but the kernel uses them too. The kernel most
frequently uses a DPC to handle quantum expiration. At every tick of the system clock, an
interrupt occurs at clock IRQL. The clock interrupt handler (running at clock IRQL) updates
the system time and then decrements a counter that tracks how long the current thread has
run. When the counter reaches 0, the thread’s time quantum has expired and the kernel
might need to reschedule the processor, a lower-priority task that should be done at DPC/dispatch IRQL. The clock interrupt handler queues a DPC to initiate thread dispatching and then
finishes its work and lowers the processor’s IRQL. Because the DPC interrupt has a lower priority than do device interrupts, any pending device interrupts that surface before the clock
interrupt completes are handled before the DPC interrupt occurs.
EXPERIMENT: Monitoring Interrupt and DPC Activity
You can use Process Explorer to monitor interrupt and DPC activity by adding the
Context Switch Delta column and watching the Interrupt and DPC processes. These
are not real processes, but they are shown as processes for convenience and therefore
do not incur context switches. Process Explorer’s context switch count for these
pseudo processes reflects the number of occurrences of each within the previous
refresh interval. You can stimulate interrupt and DPC activity by moving the mouse
quickly around the screen.
Copyrighted material.
Chapter 3:
System Mechanisms
107
You can also trace the execution of specific interrupt service routines and deferred procedure calls with the built-in event tracing support (described later in this chapter) in
Windows XP Service Pack 2 and Windows Server 2003 Service Pack 1 and later.
1. Start capturing events by typing the following command:
tracelog -start -f kernel.etl -b 64 -UsePerfCounter eflag 8 0x307 0x4084 0 0 0 0 0 0
2. Stop capturing events by typing:
tracelog -stop to stop logging.
3. Generate reports for the event capture by typing:
tracerpt kernel.etl -df -o -report
This will generate two files: workload.txt and dumpfile.csv.
4. Open “workload.txt” and you will see summaries of the time spent in ISRs and
DPCs by each driver type.
5. Open the file “dumpfile.csv” created in step 4; search for lines with “DPC” or “ISR”
in the second value. For example, the following three lines from a dumpfile.csv
generated using the above commands show a timer DPC, a DPC, and an ISR:
PerfInfo,
TimerDPC, 0xFFFFFFFF,
127383953645422825,
0, 127383953645421500, 0xFB03A385, 0, 0
PerfInfo,
DPC, 0xFFFFFFFF,
127383953645424040,
0, 127383953645421394, 0x804DC87D, 0, 0
PerfInfo,
ISR, 0xFFFFFFFF,
127383953645470903,
0, 127383953645468696, 0xFB48D5E0, 0, 0, 0
0,
0,
0,
Doing an “ln” command in the kernel debugger on the start address in each event
record (the eighth value on each line) shows the name of the function that executed the
DPC or ISR:
lkd> ln 0xFB03A385
(fb03a385)
rdbss!RxTimerDispatch
| (fb03a41e)
rdbss!RxpWorkerThreadDispatcher
lkd> ln 0x804DC87D
(804dc87d)
nt!KiTimerExpiration
| (804dc93b)
nt!KeSetTimerEx
lkd> ln 0xFB48D5E0
(fb48d5e0)
atapi!IdePortInterrupt
| (fb48d622)
atapi!IdeCheckEmptyChannel
The first is a DPC for a timer expiration for a timer queued by the file system redirector
client driver. The second is a DPC for a generic timer expiration. The third address is the
address of the ISR for the ATAPI port driver. For more information, see http://
www.microsoft.com/whdc/driver/perform/mmdrv.mspx.
Copyrighted material.
108
Microsoft Windows Internals, Fourth Edition
Asynchronous Procedure Call (APC) Interrupts Asynchronous procedure calls (APCs)
provide a way for user programs and system code to execute in the context of a particular user
thread (and hence a particular process address space). Because APCs are queued to execute in
the context of a particular thread and run at an IRQL less than DPC/dispatch level, they don’t
operate under the same restrictions as a DPC. An APC routine can acquire resources (objects),
wait for object handles, incur page faults, and call system services.
APCs are described by a kernel control object, called an APC object. APCs waiting to execute
reside in a kernel-managed APC queue. Unlike the DPC queue, which is systemwide, the APC
queue is thread-specific—each thread has its own APC queue. When asked to queue an APC,
the kernel inserts it into the queue belonging to the thread that will execute the APC routine.
The kernel, in turn, requests a software interrupt at APC level, and when the thread eventually
begins running, it executes the APC.
There are two kinds of APCs: kernel mode and user mode. Kernel-mode APCs don’t require
“permission” from a target thread to run in that thread’s context, while user-mode APCs do.
Kernel-mode APCs interrupt a thread and execute a procedure without the thread’s intervention or consent. There are also two types of kernel-mode APCs: normal and special. A thread
can disable both types by raising the IRQL to APC_LEVEL or by calling KeEnterGuardedRegion, which was introduced in Windows Server 2003. KeEnterGuardedRegionThread disables
APC delivery by setting the SpecialApcDisable field in the calling thread’s KTHREAD structure (described further in Chapter 6). A thread can disable normal APCs only by calling KeEnterCriticalRegion, which sets the KernelApcDisable field in the thread’s KTHREAD structure.
The executive uses kernel-mode APCs to perform operating system work that must be completed within the address space (in the context) of a particular thread. It can use special kernel-mode APCs to direct a thread to stop executing an interruptible system service, for
example, or to record the results of an asynchronous I/O operation in a thread’s address
space. Environment subsystems use special kernel-mode APCs to make a thread suspend or
terminate itself or to get or set its user-mode execution context. The POSIX subsystem uses
kernel-mode APCs to emulate the delivery of POSIX signals to POSIX processes.
Device drivers also use kernel-mode APCs. For example, if an I/O operation is initiated and a
thread goes into a wait state, another thread in another process can be scheduled to run.
When the device finishes transferring data, the I/O system must somehow get back into the
context of the thread that initiated the I/O so that it can copy the results of the I/O operation
to the buffer in the address space of the process containing that thread. The I/O system uses
a special kernel-mode APC to perform this action. (The use of APCs in the I/O system is discussed in more detail in Chapter 9.)
Copyrighted material.
Chapter 3:
System Mechanisms
109
Several Windows APIs, such as ReadFileEx, WriteFileEx, and QueueUserAPC, use user-mode
APCs. For example, the ReadFileEx and WriteFileEx functions allow the caller to specify a completion routine to be called when the I/O operation finishes. The I/O completion is implemented by queueing an APC to the thread that issued the I/O. However, the callback to the
completion routine doesn’t necessarily take place when the APC is queued because user-mode
APCs are delivered to a thread only when it’s in an alertable wait state. A thread can enter a wait
state either by waiting for an object handle and specifying that its wait is alertable (with the
Windows WaitForMultipleObjectsEx function) or by testing directly whether it has a pending
APC (using SleepEx). In both cases, if a user-mode APC is pending, the kernel interrupts
(alerts) the thread, transfers control to the APC routine, and resumes the thread’s execution
when the APC routine completes. Unlike kernel-mode APCs, which execute at APC level, usermode APCs execute at passive level.
APC delivery can reorder the wait queues—the lists of which threads are waiting for what, and
in what order they are waiting. (Wait resolution is described in the section “Low-IRQL Synchronization” later in this chapter.) If the thread is in a wait state when an APC is delivered,
after the APC routine completes, the wait is reissued or reexecuted. If the wait still isn’t
resolved, the thread returns to the wait state, but now it will be at the end of the list of objects
it’s waiting for. For example, because APCs are used to suspend a thread from execution, if the
thread is waiting for any objects, its wait will be removed until the thread is resumed, after
which that thread will be at the end of the list of threads waiting to access the objects it was
waiting for.
Exception Dispatching
In contrast to interrupts, which can occur at any time, exceptions are conditions that result
directly from the execution of the program that is running. Windows introduced a facility
known as structured exception handling, which allows applications to gain control when exceptions occur. The application can then fix the condition and return to the place the exception
occurred, unwind the stack (thus terminating execution of the subroutine that raised the
exception), or declare back to the system that the exception isn’t recognized and the system
should continue searching for an exception handler that might process the exception. This
section assumes you’re familiar with the basic concepts behind Windows structured exception handling—if you’re not, you should read the overview in the Windows API reference documentation on the Platform SDK or chapters 23 through 25 in Jeffrey Richter’s book
Programming Applications for Microsoft Windows (Fourth Edition, Microsoft Press, 2000) before
proceeding. Keep in mind that although exception handling is made accessible through language extensions (for example, the __try construct in Microsoft Visual C++), it is a system
mechanism and hence isn’t language-specific. Other examples of consumers of Windows
exception handling include C++ and Java exceptions.
Copyrighted material.
110
Microsoft Windows Internals, Fourth Edition
On the x86, all exceptions have predefined interrupt numbers that directly correspond to the
entry in the IDT that points to the trap handler for a particular exception. Table 3-2 shows
x86-defined exceptions and their assigned interrupt numbers. Because the first entries of the
IDT are used for exceptions, hardware interrupts are assigned entries later in the table, as
mentioned earlier.
Table 3-2 x86 Exceptions and Their Interrupt Numbers
Interrupt Number
Exception
0
Divide Error
1
DEBUG TRAP
2
NMI/NPX Error
3
Breakpoint
4
Overflow
5
BOUND/Print Screen
6
Invalid Opcode
7
NPX Not Available
8
Double Exception
9
NPX Segment Overrun
A
Invalid Task State Segment (TSS)
B
Segment Not Present
C
Stack Fault
D
General Protection
E
Page Fault
F
Intel Reserved
10
Floating Point
11
Alignment Check
All exceptions, except those simple enough to be resolved by the trap handler, are serviced by
a kernel module called the exception dispatcher. The exception dispatcher’s job is to find an
exception handler that can “dispose of” the exception. Examples of architecture-independent
exceptions that the kernel defines include memory access violations, integer divide-by-zero,
integer overflow, floating-point exceptions, and debugger breakpoints. For a complete list of
architecture-independent exceptions, consult the Windows API reference documentation.
The kernel traps and handles some of these exceptions transparently to user programs. For
example, encountering a breakpoint while executing a program being debugged generates an
exception, which the kernel handles by calling the debugger. The kernel handles certain other
exceptions by returning an unsuccessful status code to the caller.
A few exceptions are allowed to filter back, untouched, to user mode. For example, a memory
access violation or an arithmetic overflow generates an exception that the operating system
doesn’t handle. An environment subsystem can establish frame-based exception handlers to
Copyrighted material.
Chapter 3:
System Mechanisms
111
deal with these exceptions. The term frame-based refers to an exception handler’s association
with a particular procedure activation. When a procedure is invoked, a stack frame representing that activation of the procedure is pushed onto the stack. A stack frame can have one or
more exception handlers associated with it, each of which protects a particular block of code
in the source program. When an exception occurs, the kernel searches for an exception handler associated with the current stack frame. If none exists, the kernel searches for an exception handler associated with the previous stack frame, and so on, until it finds a frame-based
exception handler. If no exception handler is found, the kernel calls its own default exception
handlers.
When an exception occurs, whether it is explicitly raised by software or implicitly raised by
hardware, a chain of events begins in the kernel. The CPU hardware transfers control to the
kernel trap handler, which creates a trap frame (as it does when an interrupt occurs). The trap
frame allows the system to resume where it left off if the exception is resolved. The trap handler also creates an exception record that contains the reason for the exception and other pertinent information.
If the exception occurred in kernel mode, the exception dispatcher simply calls a routine to
locate a frame-based exception handler that will handle the exception. Because unhandled
kernel-mode exceptions are considered fatal operating system errors, you can assume that the
dispatcher always finds an exception handler.
If the exception occurred in user mode, the exception dispatcher does something more elaborate. As you’ll see in Chapter 6, the Windows subsystem has a debugger port and an exception port to receive notification of user-mode exceptions in Windows processes. The kernel
uses these in its default exception handling, as illustrated in Figure 3-8.
Debugger
port
Debugger
(first chance)
Trap
handler
Frame-based
handlers
Exception
record
Exception
dispatcher
Function call
LPC
Figure 3-8
Copyrighted material.
Dispatching an exception
Debugger
port
Debugger
(second chance)
Exception
port
Environment
subsystem
Kernel default
handler
112
Microsoft Windows Internals, Fourth Edition
Debugger breakpoints are common sources of exceptions. Therefore, the first action the
exception dispatcher takes is to see whether the process that incurred the exception has an
associated debugger process. If it does and the system is Windows 2000, the exception dispatcher sends the first-chance debug message via an LPC to the debugger port associated with
the process that incurred the exception. The LPC message is sent to the session manager process, which then dispatches it to the appropriate debugger process. On Windows XP and Windows Server 2003, the exception dispatcher sends a debugger object message to the debug
object associated with the process (which internally the system refers to as a port).
If the process has no debugger process attached, or if the debugger doesn’t handle the exception, the exception dispatcher switches into user mode, copies the trap frame to the user stack
formatted as a CONTEXT data structure (documented in the Platform SDK), and calls a routine to find a frame-based exception handler. If none is found, or if none handles the exception, the exception dispatcher switches back into kernel mode and calls the debugger again to
allow the user to do more debugging. (This is called the second-chance notification.)
If the debugger isn’t running and no frame-based handlers are found, the kernel sends a message to the exception port associated with the thread’s process. This exception port, if one
exists, was registered by the environment subsystem that controls this thread. The exception
port gives the environment subsystem, which presumably is listening at the port, the opportunity to translate the exception into an environment-specific signal or exception. CSRSS (Client/Server Run-Time Subsystem) simply presents a message box notifying the user of the fault
and terminates the process, and when POSIX gets a message from the kernel that one of its
threads generated an exception, the POSIX subsystem sends a POSIX-style signal to the
thread that caused the exception. However, if the kernel progresses this far in processing the
exception and the subsystem doesn’t handle the exception, the kernel executes a default
exception handler that simply terminates the process whose thread caused the exception.
Unhandled Exceptions
All Windows threads have an exception handler declared at the top of the stack that processes
unhandled exceptions. This exception handler is declared in the internal Windows start-ofprocess or start-of-thread function. The start-of-process function runs when the first thread in a
process begins execution. It calls the main entry point in the image. The start-of-thread function runs when a user creates additional threads. It calls the user-supplied thread start routine
specified in the CreateThread call.
Copyrighted material.
Chapter 3:
System Mechanisms
113
EXPERIMENT: Viewing the Real User Start Address for Windows
Threads
The fact that each Windows thread begins execution in a system-supplied function (and
not the user-supplied function) explains why the start address for thread 0 is the same
for every Windows process in the system (and why the start addresses for secondary
threads are also the same). The start address for thread 0 in Windows processes is the
Windows start-of-process function; the start address for any other threads would be the
Windows start-of-thread function. To see the user-supplied function address, use the
Tlist utility in the Windows Support Tools. Type tlist process-name or tlist process-id
to get the detailed process output that includes this information. For example, compare
the thread start addresses for the Windows Explorer process as reported by Pstat (in the
Platform SDK) and Tlist:
C:\> pstat
§
pid:3f8 pri: 8 Hnd: 329 Pf: 80043 Ws:
tid pri Ctx Swtch StrtAddr
User Time
7c
9
16442 77E878C1 0:00:01.241
42c 11
157888 77E92C50 0:00:07.110
44c
8
6357 77E92C50 0:00:00.070
1cc
8
3318 77E92C50 0:00:00.030
§
4620K explorer.exe
Kernel Time State
0:00:01.251 Wait:UserRequest
0:00:34.309 Wait:UserRequest
0:00:00.140 Wait:UserRequest
0:00:00.070 Wait:DelayExecution
C:\> tlist explorer
1016 explorer.exe
Program Manager
CWD:
C:\
CmdLine: Explorer.exe
VirtualSize:
25348 KB
PeakVirtualSize:
31052 KB
WorkingSetSize: 1804 KB
PeakWorkingSetSize: 3276 KB
NumberOfThreads: 4
149 Win32StartAddr:0x01009dbd LastErr:0x0000007e State:Waiting
86 Win32StartAddr:0x77c5d4a5 LastErr:0x00000000 State:Waiting
62 Win32StartAddr:0x00000977 LastErr:0x00000000 State:Waiting
179 Win32StartAddr:0x0100d8d4 LastErr:0x00000002 State:Waiting
The start address of thread 0 reported by Pstat is the internal Windows start-of-process
function; the start addresses for threads 1 through 3 are the internal Windows start-ofthread functions. Tlist, on the other hand, shows the user-supplied Windows start
address (the user function called by the internal Windows start function).
Copyrighted material.
114
Microsoft Windows Internals, Fourth Edition
Because most threads in Windows processes start at one of the system-supplied wrapper functions, Process Explorer, when displaying the start address of threads in a process, skips the initial call frame that represents the wrapper function and instead shows
the second frame on the stack. For example, notice the thread start address of a process
running Notepad.exe:
Process Explorer does display the complete call hierarchy when it displays the call stack.
Notice the following results when the Stack button is clicked:
Line 12 in the preceding figure is the first frame on the stack—the start of the process
wrapper. The second frame (line 11) is the main entry point into Notepad.exe.
Copyrighted material.
Chapter 3:
System Mechanisms
115
The generic code for these internal start functions is shown here:
void Win32StartOfProcess(
LPTHREAD_START_ROUTINE lpStartAddr,
LPVOID lpvThreadParm){
__try {
DWORD dwThreadExitCode = lpStartAddr(lpvThreadParm);
ExitThread(dwThreadExitCode);
} __except(UnhandledExceptionFilter(
GetExceptionInformation())) {
ExitProcess(GetExceptionCode());
}
}
Notice that the Windows unhandled exception filter is called if the thread has an exception
that it doesn’t handle. The purpose of this function is to provide the system-defined behavior
for what to do when an exception is not handled, which is based on the contents of the
HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug registry key. There
are two important values: Auto and Debugger. Auto tells the unhandled exception filter
whether to automatically run the debugger or ask the user what to do. By default, it is set to 1,
which means that it will launch the debugger automatically. However, installing development
tools such as Visual Studio changes this to 0. The Debugger value is a string that points to the
path of the debugger executable to run in the case of an unhandled exception.
The default debugger is \Windows\System32\Drwtsn32.exe (Dr. Watson), which isn’t really
a debugger but rather a postmortem tool that captures the state of the application “crash” and
records it in a log file (Drwtsn32.log) and a process crash dump file (User.dmp), both found
by default in the \Documents And Settings\All Users\Documents\DrWatson folder. To see
(or modify) the configuration for Dr. Watson, run it interactively—it displays a window with
the current settings, as shown in Figure 3-9.
Figure 3-9
Copyrighted material.
Windows 2000 Dr. Watson default settings
116
Microsoft Windows Internals, Fourth Edition
The log file contains basic information such as the exception code, the name of the image that
failed, a list of loaded DLLs, and a stack and instruction trace for the thread that incurred the
exception. For a detailed description of the contents of the log file, run Dr. Watson and click
the Help button shown in Figure 3-9.
The crash dump file contains the private pages in the process at the time of the exception.
(The file doesn’t include code pages from EXEs or DLLs.) This file can be opened by WinDbg,
the Windows debugger that comes with the Debugging Tools package, or by Visual Studio
2003 and later. Because the User.dmp file is overwritten each time a process crashes, unless
you rename or copy the file after each process crash, you’ll have only the latest one on your
system.
On Windows 2000 Professional systems, visual notification is turned on by default. The message box shown in Figure 3-10 is displayed by Dr. Watson after it generates the crash dump
and records information in its log file.
Figure 3-10
Windows 2000 Dr. Watson error message
The Dr. Watson process remains until the message box is dismissed, which is why on Windows 2000 Server systems visual notification is turned off by default. This default is used
because if a server application fails, there is usually nobody at the console to see it and dismiss
the message box. Instead, server applications should log errors to the Windows event log.
On Windows 2000, if the Auto value is set to zero, the message box shown in Figure 3-11 is
displayed.
Figure 3-11
Windows 2000 Unhandled exception message
If the OK button is clicked, the process exits. If Cancel is clicked, the system defined debugger
process (specified by the Debugger’s value in the registry path referred to earlier) is launched.
Copyrighted material.
Chapter 3:
System Mechanisms
117
EXPERIMENT: Unhandled Exceptions
To see a sample Dr. Watson log file, download and run the program Accvio.exe, which
you can download from www.sysinternals.com/windowsinternals.shtml. This program
generates a memory access violation by attempting to write to address 0, which is always
an invalid address in Windows processes. (See Table 7-6 in Chapter 7.)
1. Run the Registry Editor, and locate HKLM\SOFTWARE\ Microsoft\Windows
NT\CurrentVersion\AeDebug.
2. If the Debugger value is “drwtsn32 -p %ld -e %ld –g”, your system is set up to run
Dr. Watson as the default debugger. Proceed to step 4.
3. If the value of Debugger was not set up to run Drwtsn32.exe, you can still test
Dr. Watson by temporarily installing it and then restoring your previous debugger settings:
a. Save the current value somewhere (for example, in a Notepad file or in the
current paste buffer).
b. Select Run from the taskbar Start menu, and then type drwtsn32 –i. (This
initializes the Debugger field to run Dr. Watson.)
3. Run the test program Accvio.exe.
4. You should see one of the message boxes described earlier (depending on which
version of Windows you are running).
5. If you have the default Dr. Watson settings, you should now be able to examine the
log file and dump file in the dump file directory. To see the configuration settings
for Dr. Watson, run drwtsn32 with no additional arguments. (Select Run from the
Start menu, and then type drwtsn32.)
6. Alternatively, in the list of Application Errors displayed by Dr. Watson, click on the
last entry and then click the View button—the portion of the Dr. Watson log file
containing the details of the access violation from Accvio.exe will be displayed.
(For details on the log file format, open the help in Dr. Watson and select Dr. Watson Log File Overview.)
7. If the original value of Debugger wasn’t the default Dr. Watson settings, restore the
saved value from step 1.
As another experiment, try changing the value of Debugger to another program, such as
Notepad.exe (Notepad editor) or Sol.exe (Solitaire). Rerun Accvio.exe, and notice that
whatever program is specified in the Debugger value is run—that is, there’s no validation
that the program defined in Debugger is actually a debugger. Make sure you restore your
registry settings. (As noted in step 3b, to reset to the system default Dr. Watson settings,
type drwtsn32 –i in the Run dialog box or at a command prompt.)
Copyrighted material.
118
Microsoft Windows Internals, Fourth Edition
Windows Error Reporting
Windows XP and Windows Server 2003 have a new, more sophisticated error-reporting
mechanism called Windows Error Reporting that automates the submission of both usermode process crashes as well as kernel-mode system crashes. (For a description of how this
applies to system crashes, see Chapter 14).
Windows Error Reporting can be configured by going to My Computer, selecting Properties,
Advanced, and then Error Reporting (which brings up the dialog box shown in Figure 3-12)
or by local or domain group policy settings under System, Error Reporting. These settings are
stored in the registry under the key HKLM\Software\Microsoft\PCHealth\ErrorReporting.
Figure 3-12
Error Reporting Configuration dialog box
When an unhandled exception is caught by the unhandled exception filter (described in the
previous section), an initial check is made to see whether or not to initiate Windows Error
Reporting. If the registry value HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug\Auto is set to zero or the Debugger string contains the text “Drwtsn32”, the
unhandled exception filter loads \Windows\System32\Faultrep.dll into the failing process
and calls its ReportFault function. ReportFault then checks the error-reporting configuration
stored under HKLM\Software\Microsoft\PCHealth\ErrorReporting to see whether this process crash should be reported, and if so, how. In the normal case, ReportFault creates a process
running \Windows\System32\Dwwin.exe, which displays a message box announcing the process crash along with an option to submit the error report to Microsoft as seen in Figure 3-13.
Copyrighted material.
Chapter 3:
Figure 3-13
System Mechanisms
119
Windows Error Reporting dialog box
If the Send Error Report button is clicked, the error report (a minidump and a text file with
details on the DLL version numbers loaded in the process) is sent to Microsoft’s online crash
analysis server, Watson.Microsoft.com. (Unlike kernel mode system crashes, in this situation
there is no way to find out whether a solution is available at the time of the report submission.)
Then the unhandled exception filter creates a process to run the system-defined debugger (normally Drwtsn32.exe), which by default creates its own dump file and log entry. Unlike Windows
2000, the dump file is a minidump, not a full dump. So, in the case where a full process memory
dump is needed to debug a failing application, you can change the configuration of Dr. Watson
by running it with no command-line arguments as described in the previous section.
In environments where systems are not connected to the Internet or where the administrator
wants to control which error reports are submitted to Microsoft, the destination for the error
report can be configured to be an internal file server. Microsoft provides to qualified customers a tool set called Corporate Error Reporting that understands the directory structure created by Windows Error Reporting and provides the administrator with the option to take
selective error reports and submit them to Microsoft. (For more information, see http://
www.microsoft.com/resources/satech/cer.)
System Service Dispatching
As Figure 3-1 illustrated, the kernel’s trap handlers dispatch interrupts, exceptions, and system service calls. In the preceding sections, you’ve seen how interrupt and exception handling
work; in this section, you’ll learn about system services. A system service dispatch is triggered
as a result of executing an instruction assigned to system service dispatching. The instruction
that Windows uses for system service dispatching depends on the processor on which it’s executing.
32-Bit System Service Dispatching
On x86 processors prior to the Pentium II, Windows uses the int 0x2e instruction (46) decimal, which results in a trap. Windows fills in entry 46 in the IDT to point to the system service
Copyrighted material.
120
Microsoft Windows Internals, Fourth Edition
dispatcher. (Refer to Table 3-1.) The trap causes the executing thread to transition into kernel
mode and enter the system service dispatcher. A numeric argument passed in the EAX processor register indicates the system service number being requested. The EBX register points to
the list of parameters the caller passes to the system service.
On x86 Pentium II processors and higher, Windows uses the special sysenter instruction,
which Intel defined specifically for fast system service dispatches. To support the instruction,
Windows stores at boot time the address of the kernel’s system service dispatcher routine in
a register associated with the instruction. The execution of the instruction causes the change
to kernel-mode and execution of the system service dispatcher. The system service number is
passed in the EAX processor register, and the EDX register points to the list of caller arguments. To return to user-mode, the system service dispatcher usually executes the sysexit
instruction. (In some cases, like when the single-step flag is enabled on the processor, the system service dispatcher uses the iretd instruction instead.)
On K6 and higher 32-bit AMD processors, Windows uses the special syscall instruction, which
functions similar to the x86 sysenter instruction, with Windows configuring a syscall-associated processor register with the address of the kernel’s system service dispatcher. The system
call number is passed in the EAX register, and the stack stores the caller arguments. After completing the dispatch, the kernel executes the sysret instruction.
At boot time, Windows detects the type of processor on which it’s executing and sets up the
appropriate system call code to be used. The system service code for NtReadFile in user mode
looks like this:
ntdll!NtReadFile:
77f5bfa8 b8b7000000
77f5bfad ba0003fe7f
77f5bfb2 ffd2
77f5bfb4 c22400
mov
mov
call
ret
eax,0xb7
edx,0x7ffe0300
edx
0x24
The system service number is 0xb7 (183 in decimal) and the call instruction executes the system service dispatch code set up by the kernel, which in this example is at address
0x7ffe0300. Because this was taken from a Pentium M, it uses sysenter:
SharedUserData!SystemCallStub:
7ffe0300 8bd4
mov
edx,esp
7ffe0302 0f34
sysenter
7ffe0304 c3
ret
64-Bit System Service Dispatching
On the x64 architecture, Windows uses the syscall instruction, which functions like the AMD
K6’s syscall instruction, for system service dispatching, passing the system call number in the
EAX register, the first four parameters in registers, and any parameters beyond those four on
the stack:
Copyrighted material.
Chapter 3:
ntdll!NtReadFile:
00000000`77f9fc60
00000000`77f9fc63
00000000`77f9fc68
00000000`77f9fc6a
4c8bd1
b8bf000000
0f05
c3
System Mechanisms
121
mov
r10,rcx
mov
eax,0xbf
syscall
ret
On the IA64 architecture, Windows uses the epc (Enter Privileged Mode) instruction. The first
eight system call arguments are passed in registers, and the rest are passed on the stack.
Kernel-Mode System Service Dispatching
As Figure 3-14 illustrates, the kernel uses this argument to locate the system service information in the system service dispatch table. This table is similar to the interrupt dispatch table
described earlier in the chapter except that each entry contains a pointer to a system service
rather than to an interrupt handling routine.
Note
System service numbers can change between service packs—Microsoft occasionally
adds or removes system services, and the system service numbers are generated automatically
as part of a kernel compile.
User mode
Kernel mode
System
service call
System service
dispatch table
System
service
dispatcher
0
1
2
3
System service 2
• • •
n
Figure 3-14
System service exceptions
The system service dispatcher, KiSystemService, copies the caller’s arguments from the thread’s
user-mode stack to its kernel-mode stack (so that the user can’t change the arguments as the
kernel is accessing them), and then executes the system service. If the arguments passed to a
system service point to buffers in user space, these buffers must be probed for accessibility
before kernel-mode code can copy data to or from them.
As you’ll see in Chapter 6, each thread has a pointer to its system service table. Windows has
two built-in system service tables, but up to four are supported. The system service dispatcher
determines which table contains the requested service by interpreting a 2-bit field in the 32-
Copyrighted material.
122
Microsoft Windows Internals, Fourth Edition
bit system service number as a table index. The low 12 bits of the system service number serve
as the index into the table specified by the table index. The fields are shown in Figure 3-15.
Table Index
Index into table
31
13 11
System service number
0
0
0
Native API
Native API
1
1
Unused
Win32k.sys API
2
2
IIS Spud Driver
3
KeServiceDescriptorTable
Figure 3-15
IIS Spud Driver
3
KeServiceDescriptorTableShadow
System service number to system service translation
Service Descriptor Tables
A primary default array table, KeServiceDescriptorTable, defines the core executive system services implemented in Ntosrknl.exe. The other table array, KeServiceDescriptorTableShadow,
includes the Windows USER and GDI services implemented in the kernel-mode part of the
Windows subsystem, Win32k.sys. The first time a Windows thread calls a Windows USER or
GDI service, the address of the thread’s system service table is changed to point to a table that
includes the Windows USER and GDI services. The KeAddSystemServiceTable function allows
Win32k.sys and other device drivers to add system service tables. If you install Internet Information Services (IIS) on Windows 2000, its support driver (Spud.sys) upon loading defines
an additional service table, leaving only one left for definition by third parties. With the exception of the Win32k.sys service table, a service table added with KeAddSystemServiceTable is
copied into both the KeServiceDescriptorTable array and the KeServiceDescriptorTableShadow
array. Windows supports the addition of only two system service tables beyond the core and
Win32 tables.
Copyrighted material.
Chapter 3:
System Mechanisms
123
Note
Windows Server 2003 service pack 1 and higher does not support adding additional
system service tables beyond that added by Win32k.sys, so adding system service tables is not
a way to extend the functionality of those systems.
The system service dispatch instructions for Windows executive services exist in the system
library Ntdll.dll. Subsystem DLLs call functions in Ntdll to implement their documented
functions. The exception is Windows USER and GDI functions, in which the system service
dispatch instructions are implemented directly in User32.dll and Gdi32.dll—there is no
Ntdll.dll involved. These two cases are shown in Figure 3-16.
Windows USER and
GDI APIs
Windows kernel APIs
Windows
application
Call WriteFile(...)
WriteFile in
Kernel32.dll
Call NtWriteFile
Return to caller
Windowsspecific
NtWriteFile in
Ntdll.dll
Int 2E
Return to caller
Used by all
subsystems
Application
Call USER or
GDI service(...)
Gdi32.dll
or User32.dll
Int 2E
Return to caller
Windowsspecific
User mode
Kernel mode
Software interrupt
Software interrupt
KiSystemService
in Ntoskrnl.exe
Call NtWriteFile
Dismiss interrupt
KiSystemService in
Ntoskrnl.exe
Call Windows
routine
Dismiss interrupt
NtWriteFile in
Ntoskrnl.exe
Do the operation
Return to caller
Service entry point
in Win32k.sys
Do the operation
Return to caller
Figure 3-16
System service dispatching
As shown in Figure 3-16, the Windows WriteFile function in Kernel32.dll calls the NtWriteFile
function in Ntdll.dll, which in turn executes the appropriate instruction to cause a system service trap, passing the system service number representing NtWriteFile. The system service dispatcher (function KiSystemService in Ntoskrnl.exe) then calls the real NtWriteFile to process
Copyrighted material.
124
Microsoft Windows Internals, Fourth Edition
the I/O request. For Windows USER and GDI functions, the system service dispatch calls
functions in the loadable kernel-mode part of the Windows subsystem, Win32k.sys.
EXPERIMENT: Viewing System Service Activity
You can monitor system service activity by watching the System Calls/Sec performance
counter in the System object. Run the Performance tool, and in chart view, click the Add
button to add a counter to the chart; select the System object, select the System Calls/
Sec counter, and then click the Add button to add the counter to the chart.
Object Manager
As mentioned in Chapter 2, Windows implements an object model to provide consistent and
secure access to the various internal services implemented in the executive. This section
describes the Windows object manager, the executive component responsible for creating,
deleting, protecting, and tracking objects. The object manager centralizes resource control
operations that otherwise would be scattered throughout the operating system. It was
designed to meet the goals listed on later in the chapter.
EXPERIMENT: Exploring the Object Manager
Throughout this section, you’ll find experiments that show you how to peer into the
object manager database. These experiments use the following tools, which you should
become familiar with if you aren’t already:
■
Winobj (available from www.sysinternals.com) displays the internal object manager’s namespace. There is also a version of Winobj in the Platform SDK (in \Program Files\Microsoft Platform SDK\Bin\Winnt\Winobj.exe), but the Winobj
from www.sysinternals.com displays more accurate information about objects (such
as the reference count, the number of open handles, security descriptors, and so
forth).
■
Process Explorer and Handle from www.sysinternals.com (introduced in
Chapter 1) displays the open handles for a process.
■
Oh.exe (available in Windows resource kits) displays the open handles for a process, but it requires a global flag to be set in order to operate.
■
The Openfiles /query command (in Windows XP and Windows Server 2003) displays the open handles for a process, but it requires a global flag to be set in order
to operate.
■
The kernel debugger !handle command displays the open handles for a process.
Copyrighted material.
Chapter 3:
System Mechanisms
125
The object viewer provides a way to traverse the namespace that the object manager
maintains. (As we’ll explain later, not all objects have names.) Try running the WinObj
object manager utility from www.sysinternals.com and examining the layout, shown here:
As noted previously, both the OH utility and the Openfiles /query command require that
a Windows global flag called maintain objects list be enabled. (See the “Windows Global
Flags” section later in this chapter for more details about global flags.) OH will set the
flag if it is not set. If you type Openfiles /Local, it will tell you whether the flag is
enabled. You can enable it with the Openfiles /Local ON command. In either case, you
must reboot the system for the setting to take effect. Neither Process Explorer nor Handle from www.sysinternals.com require object tracking to be turned on because they use
a device driver to obtain the information.
The object manager was designed to meet the following goals:
■
Provide a common, uniform mechanism for using system resources
■
Isolate object protection to one location in the operating system so that C2 security compliance can be achieved
■
Provide a mechanism to charge processes for their use of objects so that limits can be
placed on the usage of system resources
■
Establish an object-naming scheme that can readily incorporate existing objects, such as
the devices, files, and directories of a file system, or other independent collections of
objects
Copyrighted material.
126
Microsoft Windows Internals, Fourth Edition
■
Support the requirements of various operating system environments, such as the ability
of a process to inherit resources from a parent process (needed by Windows and POSIX)
and the ability to create case-sensitive filenames (needed by POSIX)
■
Establish uniform rules for object retention (that is, for keeping an object available until
all processes have finished using it)
Internally, Windows has two kinds of objects: executive objects and kernel objects. Executive
objects are objects implemented by various components of the executive (such as the process
manager, memory manager, I/O subsystem, and so on). Kernel objects are a more primitive
set of objects implemented by the Windows kernel. These objects are not visible to user-mode
code but are created and used only within the executive. Kernel objects provide fundamental
capabilities, such as synchronization, on which executive objects are built. Thus, many executive objects contain (encapsulate) one or more kernel objects, as shown in Figure 3-17.
Details about the structure of kernel objects and how they are used to implement synchronization are given later in this chapter. In the remainder of this section, we’ll focus on how the
object manager works and on the structure of executive objects, handles, and handle tables.
Here we’ll just briefly describe how objects are involved in implementing Windows security
access checking; we’ll cover this topic thoroughly in Chapter 8.
Owned by the
object manager
Name
HandleCount
ReferenceCount
Type
Owned by the
kernel
Kernel object
Owned by the
executive
Executive object
Figure 3-17
Executive objects that contain kernel objects
Executive Objects
Each Windows environment subsystem projects to its applications a different image of the
operating system. The executive objects and object services are primitives that the environment subsystems use to construct their own versions of objects and other resources.
Copyrighted material.
Chapter 3:
System Mechanisms
127
Executive objects are typically created either by an environment subsystem on behalf of a user
application or by various components of the operating system as part of their normal operation.
For example, to create a file, a Windows application calls the Windows CreateFile function, implemented in the Windows subsystem DLL Kernel32.dll. After some validation and initialization,
CreateFile in turn calls the native Windows service NtCreateFile to create an executive file object.
The set of objects an environment subsystem supplies to its applications might be larger or
smaller than the set the executive provides. The Windows subsystem uses executive objects to
export its own set of objects, many of which correspond directly to executive objects. For
example, the Windows mutexes and semaphores are directly based on executive objects
(which are in turn based on corresponding kernel objects). In addition, the Windows subsystem supplies named pipes and mailslots, resources that are based on executive file objects.
Some subsystems, such as POSIX, don’t support objects as objects at all. The POSIX subsystem uses executive objects and services as the basis for presenting POSIX-style processes,
pipes, and other resources to its applications.
Table 3-3 lists the primary objects the executive provides and briefly describes what they represent. You can find further details on executive objects in the chapters that describe the
related executive components (or in the case of executive objects directly exported to Windows, in the Windows API reference documentation).
Note
The executive implements a total of 27 object types in Windows 2000 and 29 on Windows XP and Windows Server 2003. (These newer Windows versions add the DebugObject and
KeyedEvent objects.) Many of these objects are for use only by the executive component that
defines them and are not directly accessible by Windows APIs. Examples of these objects
include Driver, Device, and EventPair.
Table 3-3
Executive Objects Exposed to the Windows API
Object Type
Represents
Symbolic link
A mechanism for referring to an object name indirectly.
Process
The virtual address space and control information necessary for the execution
of a set of thread objects.
Thread
An executable entity within a process.
Job
A collection of processes manageable as a single entity through the job.
Section
A region of shared memory (known as a file mapping object in Windows).
File
An instance of an opened file or an I/O device.
Access token
The security profile (security ID, user rights, and so on) of a process or a thread.
Event
An object with a persistent state (signaled or not signaled) that can be used for
synchronization or notification.
Semaphore
A counter that provides a resource gate by allowing some maximum number of
threads to access the resources protected by the semaphore.
Mutex*
A synchronization mechanism used to serialize access to a resource.
Timer
A mechanism to notify a thread when a fixed period of time elapses.
Copyrighted material.
128
Microsoft Windows Internals, Fourth Edition
Table 3-3 Executive Objects Exposed to the Windows API
Object Type
Represents
IoCompletion
A method for threads to enqueue and dequeue notifications of the completion
of I/O operations (known as an I/O completion port in the Windows API).
Key
A mechanism to refer to data in the registry. Although keys appear in the
object manager namespace, they are managed by the configuration manager,
in a way similar to that in which file objects are managed by file system drivers.
Zero or more key values are associated with a key object; key values contain
data about the key.
WindowStation
An object that contains a clipboard, a set of global atoms, and a group of desktop objects.
Desktop
An object contained within a window station. A desktop has a logical display
surface and contains windows, menus, and hooks.
Note Externally in the Windows API, mutants are called mutexes. Internally, the kernel
object that underlies mutexes is called a mutant.
Object Structure
As shown in Figure 3-18, each object has an object header and an object body. The object manager controls the object headers, and the owning executive components control the object
bodies of the object types they create. In addition, each object header points to the list of processes that have the object open and to a special object called the type object that contains
information common to each instance of the object.
Process
1
Object name
Object directory
Security descriptor
Object header Quota charges
Open handle count
Open handles list
Object type
Reference count
Object body Object-specific data
Figure 3-18
Copyrighted material.
Structure of an object
Process
2
Process
3
Type object
Type name
Pool type
Default quota charges
Access types
Generic access rights mapping
Synchronizable? (Y/N)
Methods:
Open, close, delete,
parse, security,
query name
Chapter 3:
System Mechanisms
129
Object Headers and Bodies
The object manager uses the data stored in an object’s header to manage objects without
regard to their type. Table 3-4 briefly describes the object header attributes.
Table 3-4
Standard Object Header Attributes
Attribute
Purpose
Object name
Makes an object visible to other processes for sharing
Object directory
Provides a hierarchical structure in which to store object names
Security descriptor
Determines who can use the object and what they can do with it (Note: it
might be null for objects without a name.)
Quota charges
Lists the resource charges levied against a process when it opens a handle
to the object
Open handle count
Counts the number of times a handle has been opened to the object
Open handles list
Points to the list of processes that have opened handles to the object (not
present for all objects)
Object type
Points to a type object that contains attributes common to objects of this
type
Reference count
Counts the number of times a kernel-mode component has referenced
the address of the object
In addition to an object header, each object has an object body whose format and contents are
unique to its object type; all objects of the same type share the same object body format. By
creating an object type and supplying services for it, an executive component can control the
manipulation of data in all object bodies of that type.
The object manager provides a small set of generic services that operate on the attributes
stored in an object’s header and can be used on objects of any type (although some generic
services don’t make sense for certain objects). These generic services, some of which the Windows subsystem makes available to Windows applications, are listed in Table 3-5.
Although these generic object services are supported for all object types, each object has its
own create, open, and query services. For example, the I/O system implements a create file
service for its file objects, and the process manager implements a create process service for its
process objects. Although a single create object service could have been implemented, such a
routine would have been quite complicated, because the set of parameters required to initialize a file object, for example, differs markedly from that required to initialize a process object.
Also, the object manager would have incurred additional processing overhead each time a
thread called an object service to determine the type of object the handle referred to and to
call the appropriate version of the service. For these reasons and others, the create, open, and
query services are implemented separately for each object type.
Copyrighted material.
130
Microsoft Windows Internals, Fourth Edition
Table 3-5 Generic Object Services
Service
Purpose
Close
Closes a handle to an object
Duplicate
Shares an object by duplicating a handle and giving it to another
process
Query object
Gets information about an object’s standard attributes
Query security
Gets an object’s security descriptor
Set security
Changes the protection on an object
Wait for a single object
Synchronizes a thread’s execution with one object
Wait for multiple objects
Synchronizes a thread’s execution with multiple objects
Type Objects
Object headers contain data that is common to all objects but that can take on different values
for each instance of an object. For example, each object has a unique name and can have a
unique security descriptor. However, objects also contain some data that remains constant for
all objects of a particular type. For example, you can select from a set of access rights specific
to a type of object when you open a handle to objects of that type. The executive supplies terminate and suspend access (among others) for thread objects and read, write, append, and
delete access (among others) for file objects. Another example of an object-type-specific
attribute is synchronization, which is described shortly.
To conserve memory, the object manager stores these static, object-type-specific attributes
once when creating a new object type. It uses an object of its own, a type object, to record this
data. As Figure 3-19 illustrates, if the object-tracking debug flag (described in the “Windows
Global Flags” section later in this chapter) is set, a type object also links together all objects of
the same type (in this case the Process type), allowing the object manager to find and enumerate them, if necessary.
Process
type
object
Process
Object 1
Process
Object 2
Figure 3-19
Copyrighted material.
Process
Object 3
Process
Object 4
Process objects and the process type object
Chapter 3:
System Mechanisms
131
EXPERIMENT: Viewing Object Headers and Type Objects
You can see the list of type objects declared to the object manager with the Winobj tool
from www.sysinternals.com. After running Winobj, open the \ObjectTypes directory, as
shown here:
You can look at the process object type data structure in the kernel debugger by first
identifying a process object with the !process command:
kd> !process 0 0
**** NT ACTIVE PROCESS DUMP ****
PROCESS 8a4ce668 SessionId: none Cid: 0004
DirBase: 00039000 ObjectTable: e1001c88
Image: System
Peb: 00000000 ParentCid: 0000
HandleCount: 474.
Then execute the !object command with the process object address as the argument:
kd> !object 8a4ce668
Object: 8a4ce668 Type: (8a4ceca0) Process
ObjectHeader: 8a4ce650
HandleCount: 2 PointerCount: 89
Notice that the object header starts 0x18 (24 decimal) bytes prior to the start of the
object body. You can view the object header with this command:
kd> dt _object_header 8a4ce650
nt!_OBJECT_HEADER
+0x000 PointerCount
: 79
+0x004 HandleCount
: 2
+0x004 NextToFree
: 0x00000002
+0x008 Type
: 0x8a4ceca0
+0x00c NameInfoOffset
: 0 ’’
+0x00d HandleInfoOffset : 0 ’’
+0x00e QuotaInfoOffset : 0 ’’
Copyrighted material.
132
Microsoft Windows Internals, Fourth Edition
+0x00f
+0x010
+0x010
+0x014
+0x018
Flags
: 0x22 ’"‘
ObjectCreateInfo : 0x80545620
QuotaBlockCharged : 0x80545620
SecurityDescriptor : 0xe10001dc
Body
: _QUAD
Now look at the object type data structure by obtaining its address from the Type field
of the object header data structure:
kd> dt _object_type 8a4ceca0
ntdll!_OBJECT_TYPE
+0x000 Mutex
: _ERESOURCE
+0x038 TypeList
: _LIST_ENTRY [ 0x8a4cecd8 - 0x8a4cecd8 ]
+0x040 Name
: _UNICODE_STRING “Process"
+0x048 DefaultObject
: (null)
+0x04c Index
: 5
+0x050 TotalNumberOfObjects : 0x30
+0x054 TotalNumberOfHandles : 0x1b4
+0x058 HighWaterNumberOfObjects : 0x3f
+0x05c HighWaterNumberOfHandles : 0x1b8
+0x060 TypeInfo
: _OBJECT_TYPE_INITIALIZER
+0x0ac Key
: 0x636f7250
+0x0b0 ObjectLocks
: [4] _ERESOURCE
The output shows that the object type structure includes the name of the object type,
tracks the total number of active objects of that type, and tracks the peak number of
handles and objects of that type. The TypeInfo field stores the pointer to the data structure that stores attributes common to all objects of the object type as well as pointers to
the object type’s methods:
kd> dt _object_type_initializer 8a4ceca0+60
ntdll!_OBJECT_TYPE_INITIALIZER
+0x000 Length
: 0x4c
+0x002 UseDefaultObject : 0 ’’
+0x003 CaseInsensitive : 0 ’’
+0x004 InvalidAttributes : 0xb0
+0x008 GenericMapping
: _GENERIC_MAPPING
+0x018 ValidAccessMask : 0x1f0fff
+0x01c SecurityRequired : 0x1 ’’
+0x01d MaintainHandleCount : 0 ’’
+0x01e MaintainTypeList : 0 ’’
+0x020 PoolType
: 0 ( NonPagedPool )
+0x024 DefaultPagedPoolCharge : 0x1000
+0x028 DefaultNonPagedPoolCharge : 0x288
+0x02c DumpProcedure
: (null)
+0x030 OpenProcedure
: (null)
+0x034 CloseProcedure
: (null)
+0x038 DeleteProcedure : 0x805abe6e
nt!PspProcessDelete+0
+0x03c ParseProcedure
: (null)
+0x040 SecurityProcedure : 0x805cf682
nt!SeDefaultObjectMethod+0
+0x044 QueryNameProcedure : (null)
+0x048 OkayToCloseProcedure : (null)
Copyrighted material.
Chapter 3:
System Mechanisms
133
Type objects can’t be manipulated from user mode because the object manager supplies no
services for them. However, some of the attributes they define are visible through certain
native services and through Windows API routines. The attributes stored in the type objects
are described in Table 3-6.
Table 3-6
Type Object Attributes
Attribute
Purpose
Type name
The name for objects of this type (“process,” “event,” “port,”
and so on)
Pool type
Indicates whether objects of this type should be allocated from
paged or nonpaged memory
Default quota charges
Default paged and nonpaged pool values to charge to process
quotas
Access types
The types of access a thread can request when opening a handle to an object of this type (“read,” “write,” “terminate,” “suspend,” and so on)
Generic access rights mapping
A mapping between the four generic access rights (read, write,
execute, and all) to the type-specific access rights
Synchronization
Indicates whether a thread can wait for objects of this type
Methods
One or more routines that the object manager calls automatically at certain points in an object’s lifetime
Synchronization, one of the attributes visible to Windows applications, refers to a thread’s
ability to synchronize its execution by waiting for an object to change from one state to
another. A thread can synchronize with executive job, process, thread, file, event, semaphore,
mutex, and timer objects. Other executive objects don’t support synchronization. An object’s
ability to support synchronization is based on whether the object contains an embedded dispatcher object, a kernel object that is covered in the section “Low-IRQL Synchronization” later
in this chapter.
Object Methods
The last attribute in Table 3-6, methods, comprises a set of internal routines that are similar to
C++ constructors and destructors—that is, routines that are automatically called when an
object is created or destroyed. The object manager extends this idea by calling an object
method in other situations as well, such as when someone opens or closes a handle to an
object or when someone attempts to change the protection on an object. Some object types
specify methods, whereas others don’t, depending on how the object type is to be used.
When an executive component creates a new object type, it can register one or more methods
with the object manager. Thereafter, the object manager calls the methods at well-defined
points in the lifetime of objects of that type, usually when an object is created, deleted, or modified in some way. The methods that the object manager supports are listed in Table 3-7.
Copyrighted material.
134
Microsoft Windows Internals, Fourth Edition
Table 3-7 Object Methods
Method
When Method Is Called
Open
When an object handle is opened
Close
When an object handle is closed
Delete
Before the object manager deletes an object
Query name
When a thread requests the name of an object, such as a file, that exists in a
secondary object namespace
Parse
When the object manager is searching for an object name that exists in a
secondary object namespace
Security
When a process reads or changes the protection of an object, such as a file, that
exists in a secondary object namespace
The object manager calls the open method whenever it creates a handle to an object, which it
does when an object is created or opened. However, only one object type, the Windowstation,
defines an open method. The Windowstation object type requires an open method so that
Win32k.sys can share a piece of memory with the process that serves as a desktop-related
memory pool.
An example of the use of a close method occurs in the I/O system. The I/O manager registers
a close method for the file object type, and the object manager calls the close method each
time it closes a file object handle. This close method checks whether the process that is closing the file handle owns any outstanding locks on the file and, if so, removes them. Checking
for file locks isn’t something the object manager itself could or should do.
The object manager calls a delete method, if one is registered, before it deletes a temporary
object from memory. The memory manager, for example, registers a delete method for the section object type that frees the physical pages being used by the section. It also verifies that any
internal data structures the memory manager has allocated for a section are deleted before the
section object is deleted. Once again, the object manager can’t do this work because it knows
nothing about the internal workings of the memory manager. Delete methods for other types
of objects perform similar functions.
The parse method (and similarly, the query name method) allows the object manager to relinquish control of finding an object to a secondary object manager if it finds an object that exists
outside the object manager namespace. When the object manager looks up an object name, it
suspends its search when it encounters an object in the path that has an associated parse
method. The object manager calls the parse method, passing to it the remainder of the object
name it is looking for. There are two namespaces in Windows in addition to the object manager’s: the registry namespace, which the configuration manager implements, and the file system namespace, which the I/O manager implements with the aid of file system drivers. (See
Chapter 5 for more information on the configuration manager and Chapter 9 for more about
the I/O manager and file system drivers.)
Copyrighted material.
Chapter 3:
System Mechanisms
135
For example, when a process opens a handle to the object named \Device\Floppy0\
docs\resume.doc, the object manager traverses its name tree until it reaches the device object
named Floppy0. It sees that a parse method is associated with this object, and it calls the
method, passing to it the rest of the object name it was searching for—in this case, the string
\docs\resume.doc. The parse method for device objects is an I/O routine because the I/O
manager defines the device object type and registers a parse method for it. The I/O manager’s
parse routine takes the name string and passes it to the appropriate file system, which finds
the file on the disk and opens it.
The security method, which the I/O system also uses, is similar to the parse method. It is
called whenever a thread tries to query or change the security information protecting a file.
This information is different for files than for other objects because security information is
stored in the file itself rather than in memory. The I/O system, therefore, must be called to
find the security information and read or change it.
Object Handles and the Process Handle Table
When a process creates or opens an object by name, it receives a handle that represents its
access to the object. Referring to an object by its handle is faster than using its name because
the object manager can skip the name lookup and find the object directly. Processes can also
acquire handles to objects by inheriting handles at process creation time (if the creator specifies the inherit handle flag on the CreateProcess call and the handle was marked as inheritable,
either at the time it was created or afterward by using the Windows SetHandleInformation
function) or by receiving a duplicated handle from another process. (See the Windows
DuplicateHandle function.)
All user-mode processes must own a handle to an object before their threads can use the
object. Using handles to manipulate system resources isn’t a new idea. C and Pascal (and
other language) run-time libraries, for example, return handles to opened files. Handles serve
as indirect pointers to system resources; this indirection keeps application programs from fiddling directly with system data structures.
Note
Executive components and device drivers can access objects directly because they are
running in kernel mode and therefore have access to the object structures in system memory.
However, they must declare their usage of the object by incrementing the reference count so
that the object won’t be deallocated while it’s still being used. (See the section “Object Retention” later in this chapter for more details.)
Object handles provide additional benefits. First, except for what they refer to, there is no difference between a file handle, an event handle, and a process handle. This similarity provides a consistent interface to reference objects, regardless of their type. Second, the object manager has the
exclusive right to create handles and to locate an object that a handle refers to. This means that
the object manager can scrutinize every user-mode action that affects an object to see whether
the security profile of the caller allows the operation requested on the object in question.
Copyrighted material.
136
Microsoft Windows Internals, Fourth Edition
EXPERIMENT: Viewing Open Handles
Run Process Explorer, and make sure the lower pane is enabled and configured to show
open handles. (Click on View, Lower Pane View, and then Handles). Then open a command prompt and view the handle table for the new Cmd.exe process. You should see
an open file handle to the current directory. For example, assuming the current directory is C:\, Process Explorer shows the following:
If you then change the current directory with the CD command, you will see in Process
Explorer that the handle to the previous current directory is closed and a new handle is
opened to the new current directory. The previous handle is highlighted briefly in red,
and the new handle is highlighted in green. The duration of the highlight can be
adjusted by clicking Options and then Difference Highlight Duration.
Process Explorer’s differences highlighting feature makes it easy to see changes in the
handle table. For example, if a process is leaking handles, viewing the handle table with
Process Explorer can quickly show what handle or handles are being opened but not
closed. This information can assist the programmer to find the handle leak.
Copyrighted material.
Chapter 3:
System Mechanisms
137
You can also display the open handle table by using the command line Handle tool from
www.sysinternals.com. For example, note the following partial output of Handle examining the handle table for a Cmd.exe process before and after changing the directory:
C:\>handle -p cmd.exe
Handle v2.2
Copyright (C) 1997-2004 Mark Russinovich
Sysinternals - www.sysinternals.com
-----------------------------------------------------------cmd.exe pid: 3184 BIGDAVID\dsolomon
b0: File
C:\
C:\>cd windows
C:\WINDOWS>handle -p cmd.exe
.
.
cmd.exe pid: 3184 BIGDAVID\dsolomon
b4: File
C:\WINDOWS
An object handle is an index into a process-specific handle table, pointed to by the executive
process (EPROCESS) block (described in Chapter 6). The first handle index is 4, the second
8, and so on. A process’s handle table contains pointers to all the objects that the process has
opened a handle to. Handle tables are implemented as a three-level scheme, similar to the way
that the x86 memory management unit implements virtual-to-physical address translation,
giving a maximum of more than 16,000,000 handles per process. (See Chapter 7 for details
about memory management in x86 systems.)
In Windows 2000, when a process is created, the object manager allocates the top level of the
handle table, which contains pointers to the middle-level tables; the middle level, which contains the first array of pointers to subhandle tables; and the lowest level, which contains the
first subhandle table. Figure 3-20 illustrates the Windows 2000 handle table architecture. In
Windows 2000, the object manager treats the low 24 bits of an object handle’s value as three
8-bit fields that index into each of the three levels in the handle table. In Windows XP and
Windows Server 2003, only the lowest level handle table is allocated on process creation—the
other levels are created as needed. In Windows 2000, the subhandle table consists of 255
usable entries. In Windows XP and Windows Server 2003, the subhandle table consists of as
many entries as will fit in a page minus one entry that is used for handle auditing. For example, for x86 systems a page is 4096 bytes, divided by the size of a handle table entry (8 bytes),
which is 512, minus 1, which is a total of 511 entries in the lowest level handle table. In Windows XP and Windows Server 2003, the mid-level handle table contains a full page of pointers
to subhandle tables, so the number of subhandle tables depends on the size of the page and
the size of a pointer for the platform.
Copyrighted material.
138
Microsoft Windows Internals, Fourth Edition
Process
0
0
0
Handle
table
255
Middle-level
pointers
255
255
Subhandle
table
Top-level
pointers
Figure 3-20
Windows 2000 process handle table architecture
EXPERIMENT: Creating the Maximum Number of Handles
The test program Testlimit from www.sysinternals.com/windowsinternals.shtml has an
option to open handles to an object until it cannot open any more handles. You can use
this to see how many handles can be created in a single process on your system. Because
handle tables are allocated from paged pool, you might run out of paged pool before you
hit the maximum number of handles that can be created in a single process. To see how
many handles you can create on your system, follow these steps:
1. Download the Testlimit zip file from the link just mentioned, and unzip it into a
directory.
2. Run Process Explorer, and click View and then System Information. Notice the
current and maximum size of paged pool. (To display the maximum pool size values, Process Explorer must be configured properly to access the symbols for the
kernel image, Ntoskrnl.exe.) Leave this system information display running so
that you can see pool utilization when you run the Testlimit program.
3. Open a command prompt.
4. Run the Testlimit program with the “-h” switch (do this by typing testlimit –h).
When Testlimit fails to open a new handle, it will display the total number of handles it was able to create. If the number is less than approximately 16 million, you
are probably running out of paged pool before hitting the theoretical per-process
handle limit.
5. Close the command-prompt window; doing this will kill the Testlimit process,
thus closing all the open handles.
Copyrighted material.
Chapter 3:
System Mechanisms
139
As shown in Figure 3-21, on x86 systems, each handle entry consists of a structure with two
32-bit members: a pointer to the object (with flags), and the granted access mask. On 64-bit
systems, a handle table entry is 12 bytes long: a 64-bit pointer to the object header and a 32bit access mask. (Access masks are described in Chapter 8.)
On Windows 2000, the first 32-bit member contains both a pointer to the object header and
four flags. Because object headers are always 8-byte aligned, the low-order 3 bits of this field
are free for use as flags. An entry’s high bit is used as a lock. When the object manager translates a handle to an object pointer, it locks the handle entry while the translation is in
progress. Because all objects are located in the system address space, the high bit of the object
pointer is set. (The addresses are guaranteed to be higher than 0x80000000 even on systems
with the /3GB boot switch.) Thus, the object manager can keep the high bit clear when a handle table entry is unlocked and, in the process of locking the entry, set the bit and obtain the
object’s correct pointer value. The object manager needs to lock a process’s entire handle
table, using a handle table lock associated with each process, only when the process creates a
new handle or closes an existing handle. In Windows XP and Windows Server 2003, the lock
bit is the low-order bit of the object pointer. The flag that was stored in this low-order bit in
Windows 2000 is now stored in an unused bit in the access mask.
Audit on close
Inheritable
Lock
Protect from close
Pointer to object header
A I P
Access mask
32 bits
Figure 3-21
Structure of a handle table entry
The first flag indicates whether the caller is allowed to close this handle. The second flag is the
inheritance designation—that is, it indicates whether processes created by this process will get
a copy of this handle in their handle tables. As already noted, handle inheritance can be specified on handle creation or later with the SetHandleInformation function. (This flag can also be
specified with the Windows SetHandleInformation function.) The third flag indicates whether
closing the object should generate an audit message. (This flag isn’t exposed to Windows—the
object manager uses it internally.)
System components and device drivers often need to open handles to objects that user-mode
applications shouldn’t have access to. This is done by creating handles in the kernel handle
table (referenced internally with the name ObpKernelHandleTable). The handles in this table
are accessible only from kernel mode and in any process context. This means that a kernelmode function can reference the handle in any process context with no performance impact.
The object manager recognizes references to handles from the kernel handle table when the
Copyrighted material.
140
Microsoft Windows Internals, Fourth Edition
high bit of the handle is set—that is, when references to kernel-handle-table handles have values greater than 0x80000000. On Windows 2000, the kernel-handle table is an independent
handle table, but on Windows XP and Windows Server 2003 the kernel-handle table also
serves as the handle table for the System process.
EXPERIMENT: Viewing the Handle Table with the Kernel
Debugger
The !handle command in the kernel debugger takes three arguments:
!handle <handle index> <flags> <processid>
The handle index identifies the handle entry in the handle table. (Zero means display all
handles.) The first handle is index 4, the second 8, and so on. For example, typing !handle 4 will show the first handle for the current process.
The flags you can specify are a bitmask, where bit 0 means display only the information
in the handle entry, bit 1 means display free handles (not just used handles), and bit 2
means display information about the object that the handle refers to. The following
command displays full details about the handle table for process ID 0x408:
kd> !handle 0 7 408
processor number 0
Searching for Process with Cid == 408
PROCESS 865f0790 SessionId: 0 Cid: 0408 Peb: 7ffdf000 ParentCid: 01dc
DirBase: 04fd3000 ObjectTable: 856ca888 TableSize: 21.
Image: i386kd.exe
Handle Table at e2125000 with 21 Entries in use
0000: free handle
0004: Object: e20da2e0 GrantedAccess: 000f001f
Object: e20da2e0 Type: (81491b80) Section
ObjectHeader: e20da2c8
HandleCount: 1 PointerCount: 1
0008: Object: 80b13330 GrantedAccess: 00100003
Object: 80b13330 Type: (81495100) Event
ObjectHeader: 80b13318
HandleCount: 1 PointerCount: 1
Object Security
When you open a file, you must specify whether you intend to read or to write. If you try to
write to a file that is opened for read access, you get an error. Likewise, in the executive, when
a process creates an object or opens a handle to an existing object, the process must specify a
set of desired access rights—that is, what it wants to do with the object. It can request either a set
of standard access rights (such as read, write, and execute) that apply to all object types or
Copyrighted material.
Chapter 3:
System Mechanisms
141
specific access rights that vary depending on the object type. For example, the process can
request delete access or append access to a file object. Similarly, it might require the ability to
suspend or terminate a thread object.
When a process opens a handle to an object, the object manager calls the security reference
monitor, the kernel-mode portion of the security system, sending it the process’s set of desired
access rights. The security reference monitor checks whether the object’s security descriptor
permits the type of access the process is requesting. If it does, the reference monitor returns a
set of granted access rights that the process is allowed, and the object manager stores them in
the object handle it creates. How the security system determines who gets access to which
objects is explored in Chapter 8.
Thereafter, whenever the process’s threads use the handle, the object manager can quickly
check whether the set of granted access rights stored in the handle corresponds to the usage
implied by the object service the threads have called. For example, if the caller asked for read
access to a section object but then calls a service to write to it, the service fails.
Object Retention
There are two types of objects: temporary and permanent. Most objects are temporary—that is,
they remain while they are in use and are freed when they are no longer needed. Permanent
objects remain until they are explicitly freed. Because most objects are temporary, the rest of this
section describes how the object manager implements object retention—that is, retaining temporary objects only as long as they are in use and then deleting them. Because all user-mode processes that access an object must first open a handle to it, the object manager can easily track
how many of these processes, and even which ones, are using an object. Tracking these handles
represents one part in implementing retention. The object manager implements object retention
in two phases. The first phase is called name retention, and it is controlled by the number of open
handles to an object that exist. Every time a process opens a handle to an object, the object manager increments the open handle counter in the object’s header. As processes finish using the
object and close their handles to it, the object manager decrements the open handle counter.
When the counter drops to 0, the object manager deletes the object’s name from its global
namespace. This deletion prevents new processes from opening a handle to the object.
The second phase of object retention is to stop retaining the objects themselves (that is, to
delete them) when they are no longer in use. Because operating system code usually accesses
objects by using pointers instead of handles, the object manager must also record how many
object pointers it has dispensed to operating system processes. It increments a reference count
for an object each time it gives out a pointer to the object; when kernel-mode components finish using the pointer, they call the object manager to decrement the object’s reference count.
The system also increments the reference count when it increments the handle count, and
likewise decrements the reference count when the handle count decrements, because a handle is also a reference to the object that must be tracked. (For further details on object retention, see the DDK documentation on the functions ObReferenceObjectByPointer and
ObDereferenceObject.)
Copyrighted material.
142
Microsoft Windows Internals, Fourth Edition
Figure 3-22 illustrates two event objects that are in use. Process A has the first event open. Process B has both events open. In addition, the first event is being referenced by some kernelmode structure; thus, the reference count is 3. So even if processes A and B closed their handles to the first event object, it would continue to exist because its reference count is 1. However, when process B closes its handle to the second event object, the object would be
deallocated.
Process A
System space
Handles
Event object
Handle table
HandleCount=2
ReferenceCount=3
Index
Other structure
DuplicateHandle
Process B
Event object
Handle table
HandleCount=1
ReferenceCount=1
Figure 3-22
Handles and reference counts
So even after an object’s open handle counter reaches 0, the object’s reference count might
remain positive, indicating that the operating system is still using the object. Ultimately, when
the reference count drops to 0, the object manager deletes the object from memory.
Because of the way object retention works, an application can ensure that an object and its
name remain in memory simply by keeping a handle open to the object. Programmers who
write applications that contain two or more cooperating processes need not be concerned
that one process might delete an object before the other process has finished using it. In addition, closing an application’s object handles won’t cause an object to be deleted if the operating system is still using it. For example, one process might create a second process to execute
a program in the background; it then immediately closes its handle to the process. Because
the operating system needs the second process to run the program, it maintains a reference to
its process object. Only when the background program finishes executing does the object
manager decrement the second process’s reference count and then delete it.
Copyrighted material.
Chapter 3:
System Mechanisms
143
Resource Accounting
Resource accounting, like object retention, is closely related to the use of object handles. A
positive open handle count indicates that some process is using that resource. It also indicates
that some process is being charged for the memory the object occupies. When an object’s handle count and reference count drop to 0, the process that was using the object should no
longer be charged for it.
Many operating systems use a quota system to limit processes’ access to system resources.
However, the types of quotas imposed on processes are sometimes diverse and complicated,
and the code to track the quotas is spread throughout the operating system. For example, in
some operating systems, an I/O component might record and limit the number of files a process can open, whereas a memory component might impose a limit on the amount of memory
a process’s threads can allocate. A process component might limit users to some maximum
number of new processes they can create or a maximum number of threads within a process.
Each of these limits is tracked and enforced in different parts of the operating system.
In contrast, the Windows object manager provides a central facility for resource accounting.
Each object header contains an attribute called quota charges that records how much the
object manager subtracts from a process’s allotted paged and/or nonpaged pool quota when
a thread in the process opens a handle to the object.
Each process on Windows points to a quota structure that records the limits and current values
for nonpaged pool, paged pool, and page file usage. (Type dt nt!_EPROCESS_QUOTA_ENTRY
in the kernel debugger to see the format of this structure.) These quotas default to 0 (no limit)
but can be specified by modifying registry values. (See NonPagedPoolQuota, PagedPoolQuota, and
PagingFileQuota under HKLM\System\CurrentControlSet\Session Manager\Memory Management.) Note that all the processes in an interactive session share the same quota block (and
there’s no documented way to create processes with their own quota blocks).
Object Names
An important consideration in creating a multitude of objects is the need to devise a successful system for keeping track of them. The object manager requires the following information
to help you do so:
■
A way to distinguish one object from another
■
A method for finding and retrieving a particular object
The first requirement is served by allowing names to be assigned to objects. This is an extension of what most operating systems provide—the ability to name selected resources, files,
pipes, or a block of shared memory, for example. The executive, in contrast, allows any
resource represented by an object to have a name. The second requirement, finding and
retrieving an object, is also satisfied by object names. If the object manager stores objects by
name, it can find an object by looking up its name.
Copyrighted material.
144
Microsoft Windows Internals, Fourth Edition
Object names also satisfy a third requirement, which is to allow processes to share objects.
The executive’s object namespace is a global one, visible to all processes in the system. One
process can create an object and place its name in the global namespace, and a second process
can open a handle to the object by specifying the object’s name. If an object isn’t meant to be
shared in this way, its creator doesn’t need to give it a name.
To increase efficiency, the object manager doesn’t look up an object’s name each time someone uses the object. Instead, it looks up a name under only two circumstances. The first is
when a process creates a named object: the object manager looks up the name to verify that it
doesn’t already exist before storing the new name in the global namespace. The second is
when a process opens a handle to a named object: the object manager looks up the name,
finds the object, and then returns an object handle to the caller; thereafter, the caller uses the
handle to refer to the object. When looking up a name, the object manager allows the caller to
select either a case-sensitive or a case-insensitive search, a feature that supports POSIX and
other environments that use case-sensitive filenames.
Where the names of objects are stored depends on the object type. Table 3-8 lists the standard
object directories found on all Windows systems and what types of objects have their names
stored there. Of the directories listed, only \BaseNamedObjects and \GLOBAL?? (\?? on Windows 2000) are visible to user programs (see the Session Namespace section later in this
chapter for more information).
Because the base kernel objects such as mutexes, events, semaphores, waitable timers, and
sections have their names stored in a single object directory, no two of these objects can have
the same name, even if they are of a different type. This restriction emphasizes the need to
choose names carefully so that they don’t collide with other names (for example, prefix
names with your company and product name).
Table 3-8 Standard Object Directories
Directory
Types of Object Names Stored
\GLOBAL?? (\?? in Windows
2000)
MS-DOS device names (\DosDevices is a symbolic link to this
directory.)
\BaseNamedObjects
Mutexes, events, semaphores, waitable timers, and section
objects
\Callback
Callback objects
\Device
Device objects
\Driver
Driver objects
\FileSystem
File system driver objects and file system recognizer device
objects
\KnownDlls
Section names and path for known DLLs (DLLs mapped by the
system at startup time)
\Nls
Section names for mapped national language support tables
\ObjectTypes
Names of types of objects
\RPC Control
Port objects used by remote procedure calls (RPCs)
Copyrighted material.
Chapter 3:
Table 3-8
Directory
System Mechanisms
145
Standard Object Directories
Types of Object Names Stored
\Security
Names of objects specific to the security subsystem
\Windows
Windows subsystem ports and window stations
Object names are global to a single computer (or to all processors on a multiprocessor computer), but they’re not visible across a network. However, the object manager’s parse method
makes it possible to access named objects that exist on other computers. For example, the
I/O manager, which supplies file object services, extends the functions of the object manager to remote files. When asked to open a remote file object, the object manager calls a
parse method, which allows the I/O manager to intercept the request and deliver it to a network redirector, a driver that accesses files across the network. Server code on the remote
Windows system calls the object manager and the I/O manager on that system to find the file
object and return the information back across the network.
EXPERIMENT: Looking at the Base Named Objects
You can see the list of base objects that have names with the Winobj tool from www.sysinternals.com. Run Winobj.exe and click on \BaseNamedObjects, as shown here:
The named objects are shown on the right. The icons indicate the object type.
Copyrighted material.
■
Mutexes are indicated with a stop sign.
■
Sections (Windows file mapping objects) are shown as memory chips.
■
Events are shown as exclamation points.
■
Semaphores are indicated with an icon that resembles a traffic signal.
■
Symbolic links have icons that are curved arrows.
146
Microsoft Windows Internals, Fourth Edition
Object directories The object directory object is the object manager’s means for supporting this hierarchical naming structure. This object is analogous to a file system directory and
contains the names of other objects, possibly even other object directories. The object directory object maintains enough information to translate these object names into pointers to the
objects themselves. The object manager uses the pointers to construct the object handles that
it returns to user-mode callers. Both kernel-mode code (including executive components and
device drivers) and user-mode code (such as subsystems) can create object directories in
which to store objects. For example, the I/O manager creates an object directory named
\Device, which contains the names of objects representing I/O devices.
Symbolic links In certain file systems (on NTFS and some UNIX systems, for example), a
symbolic link lets a user create a filename or a directory name that, when used, is translated by
the operating system into a different file or directory name. Using a symbolic link is a simple
method for allowing users to indirectly share a file or the contents of a directory, creating a
cross-link between different directories in the ordinarily hierarchical directory structure.
The object manager implements an object called a symbolic link object, which performs a similar function for object names in its object namespace. A symbolic link can occur anywhere
within an object name string. When a caller refers to a symbolic link object’s name, the object
manager traverses its object namespace until it reaches the symbolic link object. It looks
inside the symbolic link and finds a string that it substitutes for the symbolic link name. It
then restarts its name lookup.
One place in which the executive uses symbolic link objects is in translating MS-DOS-style
device names into Windows internal device names. In Windows, a user refers to floppy and
hard disk drives using the names A:, B:, C:, and so on and serial ports as COM1, COM2, and
so on. The Windows subsystem makes these symbolic link objects protected, global data by
placing them in the object manager namespace under the \?? object directory on Windows
2000 and the \Global?? directory on Windows XP and Windows Server 2003.
Session Namespace
Windows NT was originally written with the assumption that only one user would log on to
the system interactively and that the system would run only one instance of any interactive
application. The addition of Windows Terminal Services in Windows 2000 Server and fast
user switching in Windows XP changed these assumptions, thus requiring changes to the
object manager namespace model to support multiple users. (For a basic description of terminal services and sessions, see Chapter 1.)
Copyrighted material.
Chapter 3:
System Mechanisms
147
A user logged on to the console session has access to the global namespace, a namespace that
serves as the first instance of the namespace. Additional sessions are given a session-private
view of the namespace known as a local namespace. The parts of the namespace that are localized for each session include \DosDevices, \Windows, and \BaseNamedObjects. Making separate copies of the same parts of the namespace is known as instancing the namespace.
Instancing \DosDevices makes it possible for each user to have different network drive letters
and Windows objects such as serial ports. On Windows 2000, the global \DosDevices directory is named \?? and is the directory to which the \DosDevices symbolic link points, and
local \DosDevices directories are identified by the session id for the terminal server session.
On Windows XP and later, the global \DosDevices directory is named \Global?? and is the
directory to which \DosDevices points, and local \DosDevices directories are identified by
the logon session ID.
The \Windows directory is where Win32k.sys creates the interactive window station,
\WinSta0. A Terminal Services environment can support multiple interactive users, but each
user needs an individual version of WinSta0 to preserve the illusion that he or she is accessing
the predefined interactive window station in Windows. Finally, applications and the system
create shared objects in \BaseNamedObjects, including events, mutexes, and memory sections. If two users are running an application that creates a named object, each user session
must have a private version of the object so that the two instances of the application don’t
interfere with one another by accessing the same object.
The object manager implements a local namespace by creating the private versions of the
three directories mentioned under a directory associated with the user’s session under \Sessions\X (where X is the session identifier). When a Windows application in remote session
two creates a named event, for example, the object manager transparently redirects the
object’s name from \BaseNamedObjects to \Sessions\2\BaseNamedObjects.
All object manager functions related to namespace management are aware of the instanced
directories and participate in providing the illusion that nonconsole sessions use the same
namespace as the console session. Windows subsystem DLLs prefix names passed by Windows applications that reference objects in \DosDevices with \?? (for example, C:\Windows
becomes \??\C:\Windows). When the object manager sees the special \?? prefix, the steps it
takes depends on the version of Windows, but it always relies on a field named DeviceMap in
the executive process object (EPROCESS, which is described further in Chapter 6) that points
to a data structure shared by other processes in the same session. The DosDevicesDirectory
field of the DeviceMap structure points at the object manager directory that represents the
process’s local \DosDevices. The target directory varies depending on the system:
■
Copyrighted material.
If the system is Windows 2000 and Terminal Services are not installed, the DosDevicesDirectory field of the DeviceMap structure of the process points at the \?? directory
because there are no local namespaces.
148
Microsoft Windows Internals, Fourth Edition
■
If the system is Windows 2000 and Terminal Services are installed, when a new session
becomes active the system copies all the objects from the global \?? directory into the
session’s local \Devices directory and the DosDevicesDirectory field of the DeviceMap
structure points at the local directory.
■
On Windows XP and Windows Server 2003, the system does not make copies of global
objects in the local DosDevices directories. When the object manager sees a reference to
\??, it locates the process’s local \DosDevices by using the DosDevicesDirectory field of
the DeviceMap. If the object manager doesn’t find the object in that directory, it checks
the DeviceMap field of the directory object, and if it’s valid it looks for the object in the
directory pointed to by the GlobalDosDevicesDirectory field of the DeviceMap structure, which is always \Global??.
Under certain circumstances, applications that are Terminal Services–aware need to access
objects in the console session even if the application is running in a remote session. The application might want to do this to synchronize with instances of itself running in other remote
sessions or with the console session. For these cases, the object manager provides the special
override “\Global” that an application can prefix to any object name to access the global
namespace. For example, an application in session two opening an object named \Global\ApplicationInitialized is directed to \BasedNamedObjects\ApplicationInitialized instead
of \Sessions\2\BaseNamedObjects\ApplicationInitialized.
On Windows XP and Windows Server 2003, an application that wants to access an object in
the global \DosDevices directory does not need to use the \Global prefix as long as the object
doesn’t exist in its local \DosDevices directory. This is because the object manager will automatically look in the global directory for the object if it doesn’t find it in the local directory.
However, an application running on Windows 2000 with Terminal Services must always specify the \Global prefix to access objects in the global \DosDevices directory.
EXPERIMENT: Viewing Namespace Instancing
You can see the object manager instance of the namespace by creating a session other
than the console session and then viewing the handle table for a process in that session.
On Windows XP Home Edition or on a Windows XP Professional system that is not a
member of a domain, disconnect the console session (by clicking Start, clicking Log Off,
and choosing Disconnect and Switch User, or by pressing the Windows key + L) and
logging in to a new account. If you have a Windows 2000 Server, Advanced Server, or
Datacenter Server system, run the Terminal Services client, connect to the server, and
log in.
Copyrighted material.
Chapter 3:
System Mechanisms
149
Once you are logged in to the new session, run Winobj.exe from www.sysinternals.com
and click on the \Sessions directory. You’ll see a subdirectory with a numeric name for
each active remote session. If you open one of these directories, you’ll see subdirectories
named \DosDevices, \Windows, and \BaseNamedObjects, which are the local
namespace subdirectories of the session. The following screen shot shows a local
namespace:
Next run Process Explorer and select a process in the new session (such as
Explorer.exe), and then view the handle table (by clicking View, Lower Pane View, and
then Handles). You should see a handle to \Windows\Windowstations\WinSta0
underneath \Sessions\n, where n is the session id. Objects with global names will
appear under \Sessions\n\BaseNamedObjects.
Synchronization
The concept of mutual exclusion is a crucial one in operating systems development. It refers to
the guarantee that one, and only one, thread can access a particular resource at a time. Mutual
exclusion is necessary when a resource doesn’t lend itself to shared access or when sharing
would result in an unpredictable outcome. For example, if two threads copy a file to a printer
port at the same time, their output could be interspersed. Similarly, if one thread reads a memory location while another one writes to it, the first thread will receive unpredictable data. In
general, writable resources can’t be shared without restrictions, whereas resources that aren’t
subject to modification can be shared. Figure 3-23 illustrates what happens when two threads
running on different processors both write data to a circular queue.
Copyrighted material.
150
Microsoft Windows Internals, Fourth Edition
Time
Processor B
Processor A
Get queue tail
Insert data at current location
• • •
• • •
Get queue tail
• • •
Increment tail pointer
• • •
Insert data at current location /*ERROR*/
Increment tail pointer
• • •
Figure 3-23
Incorrect sharing of memory
Because the second thread got the value of the queue tail pointer before the first thread had
finished updating it, the second thread inserted its data into the same location that the first
thread had used, overwriting data and leaving one queue location empty. Even though this
figure illustrates what could happen on a multiprocessor system, the same error could occur
on a single-processor system if the operating system were to perform a context switch to the
second thread before the first thread updated the queue tail pointer.
Sections of code that access a nonshareable resource are called critical sections. To ensure correct code, only one thread at a time can execute in a critical section. While one thread is writing to a file, updating a database, or modifying a shared variable, no other thread can be
allowed to access the same resource. The pseudocode shown in Figure 3-23 is a critical section
that incorrectly accesses a shared data structure without mutual exclusion.
The issue of mutual exclusion, although important for all operating systems, is especially
important (and intricate) for a tightly coupled, symmetric multiprocessing (SMP) operating system such as Windows, in which the same system code runs simultaneously on more than one
processor, sharing certain data structures stored in global memory. In Windows, it is the kernel’s job to provide mechanisms that system code can use to prevent two threads from modifying the same structure at the same time. The kernel provides mutual-exclusion primitives
that it and the rest of the executive use to synchronize their access to global data structures.
Because the scheduler synchronizes access to its data structures at DPC/Dispatch level IRQL,
the kernel and executive cannot rely on synchronization mechanisms that would result in a
page fault or reschedule operation to synchronize access to data structures when the IRQL is
DPC/Dispatch level or higher (levels known as an elevated or high IRQL). In the following sections, you’ll find out how the kernel and executive uses mutual exclusion to protect its global
Copyrighted material.
Chapter 3:
System Mechanisms
151
data structures when the IRQL is high and what mutual-exclusion and synchronization mechanisms the kernel and executive use when the IRQL is low (below DPC/Dispatch level).
High-IRQL Synchronization
At various stages during its execution, the kernel must guarantee that one, and only one, processor at a time is executing within a critical section. Kernel critical sections are the code segments that modify a global data structure such as the kernel’s dispatcher database or its DPC
queue. The operating system can’t function correctly unless the kernel can guarantee that
threads access these data structures in a mutually exclusive manner.
The biggest area of concern is interrupts. For example, the kernel might be updating a global
data structure when an interrupt occurs whose interrupt-handling routine also modifies the
structure. Simple single-processor operating systems sometimes prevent such a scenario by
disabling all interrupts each time they access global data, but the Windows kernel has a more
sophisticated solution. Before using a global resource, the kernel temporarily masks those
interrupts whose interrupt handlers also use the resource. It does so by raising the processor’s IRQL to the highest level used by any potential interrupt source that accesses the global
data. For example, an interrupt at DPC/dispatch level causes the dispatcher, which uses the
dispatcher database, to run. Therefore, any other part of the kernel that uses the dispatcher
database raises the IRQL to DPC/dispatch level, masking DPC/dispatch-level interrupts
before using the dispatcher database.
This strategy is fine for a single-processor system, but it’s inadequate for a multiprocessor configuration. Raising the IRQL on one processor doesn’t prevent an interrupt from occurring on
another processor. The kernel also needs to guarantee mutually exclusive access across several processors.
Interlocked Operations
The simplest form of synchronization mechanisms rely on hardware support for multiprocessor-safe manipulating integer values and for performing comparisons. They include functions
such as InterlockedIncrement, InterlockedDecrement, InterlockedExchange, and InterlockedCompareExchange. The InterlockedDecrement function, for example, uses the x86 lock instruction prefix (for example, lock xadd) to lock the multiprocessor bus during the subtraction
operation so that another processor that’s also modifying the memory location being decremented won’t be able to modify between the decrement’s read of the original value and write of
the decremented value. This form of basic synchronization is used by the kernel and drivers.
Copyrighted material.
152
Microsoft Windows Internals, Fourth Edition
Spinlocks
The mechanism the kernel uses to achieve multiprocessor mutual exclusion is called a spinlock. A spinlock is a locking primitive associated with a global data structure, such as the DPC
queue shown in Figure 3-24.
Processor A
Processor B
• • •
• • •
Do
Try to acquire
DPC queue
spinlock
Until SUCCESS
Begin
Remove DPC from queue
End
Do
Try to acquire
DPC queue
spinlock
Until SUCCESS
Spinlock
DPC
Release DPC queue spinlock
DPC
DPC queue
Begin
Add DPC from queue
End
Release DPC queue spinlock
Critical section
Figure 3-24
Using a spinlock
Before entering either critical section shown in the figure, the kernel must acquire the spinlock associated with the protected DPC queue. If the spinlock isn’t free, the kernel keeps trying to acquire the lock until it succeeds. The spinlock gets its name from the fact that the
kernel (and thus, the processor) is held in limbo, “spinning,” until it gets the lock.
Spinlocks, like the data structures they protect, reside in global memory. The code to acquire
and release a spinlock is written in assembly language for speed and to exploit whatever locking mechanism the underlying processor architecture provides. On many architectures, spinlocks are implemented with a hardware-supported test-and-set operation, which tests the
value of a lock variable and acquires the lock in one atomic instruction. Testing and acquiring
the lock in one instruction prevents a second thread from grabbing the lock between the time
when the first thread tests the variable and the time when it acquires the lock.
All kernel-mode spinlocks in Windows have an associated IRQL that is always at DPC/dispatch level or higher. Thus, when a thread is trying to acquire a spinlock, all other activity at
the spinlock’s IRQL or lower ceases on that processor. Because thread dispatching happens at
DPC/dispatch level, a thread that holds a spinlock is never preempted because the IRQL
masks the dispatching mechanisms. This masking allows code executing a critical section protected by a spinlock to continue executing so that it will release the lock quickly. The kernel
uses spinlocks with great care, minimizing the number of instructions it executes while it
holds a spinlock.
Copyrighted material.
Chapter 3:
System Mechanisms
153
Note
Because the IRQL is an effective synchronization mechanism on uniprocessors, the
spinlock acquisition and release functions of uniprocessor HALs don’t implement spinlocks—
they simply raise and lower the IRQL.
The kernel makes spinlocks available to other parts of the executive through a set of kernel
functions, including KeAcquireSpinlock and KeReleaseSpinlock. Device drivers, for example,
require spinlocks in order to guarantee that device registers and other global data structures
are accessed by only one part of a device driver (and from only one processor) at a time. Spinlocks are not for use by user programs—user programs should use the objects described in the
next section.
Kernel spinlocks carry with them restrictions for code that uses them. Because spinlocks
always have an IRQL of DPC/dispatch level or higher, as explained earlier, code holding a
spinlock will crash the system if it attempts to make the scheduler perform a dispatch operation or if it causes a page fault.
Queued Spinlocks
A special type of spinlock called a queued spinlock is used in some circumstances instead of a
standard spinlock. A queued spinlock is a form of spinlock that scales better on multiprocessors than a standard spinlock. In general, Windows will use only standard spinlocks when it
expects there to be low contention for the lock.
A queued spinlock work like this: When a processor wants to acquire a queued spinlock that
is currently held, it places its identifier in a queue associated with the spinlock. When the processor that’s holding the spinlock releases it, it hands the lock over to the first processor identified in the queue. In the meantime, a processor waiting for a busy spinlock checks the status
not of the spinlock itself but of a per-processor flag that the processor ahead of it in the queue
sets to indicate that the waiting processor’s turn has arrived.
The fact that queued spinlocks result in spinning on per-processor flags rather than global
spinlocks has two effects. The first is that the multiprocessor’s bus isn’t as heavily trafficked by
interprocessor synchronization. The second is that instead of a random processor in a waiting
group acquiring a spinlock, the queued spinlock enforces first-in, first-out (FIFO) ordering to
the lock. FIFO ordering means more consistent performance across processors accessing the
same locks.
Windows defines a number of global queued spinlocks by storing pointers to them in an array
contained in each processor’s processer control region (PCR). A global spinlock can be acquired
by calling KeAcquireQueuedSpinlock with the index into the PCR array at which the pointer to
the spinlock is stored. The number of global spinlocks has grown in each release of the operating system, and the table of index definitions for them is published in the DDK header file
Ntddk.h.
Copyrighted material.
154
Microsoft Windows Internals, Fourth Edition
EXPERIMENT: Viewing Global Queued Spinlocks
You can view the state of the global queued spinlocks (the ones pointed to by the
queued spinlock array in each processor’s PCR) by using the !qlock kernel debugger
command. This command is meaningful only on a multiprocessor system because uniprocessor HALs don’t implement spinlocks. In the following example, taken from a
Windows 2000 system, the dispatcher database queued spinlock is held by processor 1,
and the other queued spinlocks are not acquired. (The dispatcher database is described
in Chapter 6.)
kd> !qlocks
Key: O = Owner, 1-n = Wait order, blank = not owned/waiting, C = Corrupt
Lock Name
KE
KE
MM
MM
CC
-
0
Processor Number
1 2 3 4 5 6 7
8
9 10 11 12 13 14 15
Dispatcher
O
Context Swap
PFN
System Space
Vacb CC – Master
Instack Queued Spinlocks
In addition to using static queued spinlocks that are globally defined, Windows XP and Windows Server 2003 kernels support dynamically allocated queued spinlocks with the KeAcquireInStackQueuedSpinlock and KeReleaseInStackQueuedSpinlock functions. Several
components—including the cache manager, executive pool manager, and NTFS—take advantage of these types of locks, and the functions are documented in the DDK for use by thirdparty driver writers.
KeAcquireInStackQueuedSpinlock takes a pointer to a spinlock data structure and a spin lock
queue handle. The spin lock handle is actually a data structure in which the kernel stores
information about the lock’s status, including the lock’s ownership and the queue of processors that might be waiting for the lock to become available.
Executive Interlocked Operations
The kernel supplies a number of simple synchronization functions constructed on spinlocks
for more advanced operations, such as adding and removing entries from singly and doubly
linked lists. Examples include ExInterlockedPopEntryList and ExInterlockedPushEntryList for
singly linked lists, and ExInterlockedInsertHeadList and ExInterlockedRemoveHeadList for doubly linked lists. All these functions require a standard spinlock as a parameter and are used
throughout the kernel and device drivers.
Copyrighted material.
Chapter 3:
System Mechanisms
155
Low-IRQL Synchronization
Executive software outside the kernel also needs to synchronize access to global data structures in a multiprocessor environment. For example, the memory manager has only one page
frame database, which it accesses as a global data structure, and device drivers need to ensure
that they can gain exclusive access to their devices. By calling kernel functions, the executive
can create a spinlock, acquire it, and release it.
Spinlocks only partially fill the executive’s needs for synchronization mechanisms, however.
Because waiting for a spinlock literally stalls a processor, spinlocks can be used only under
the following strictly limited circumstances:
■
The protected resource must be accessed quickly and without complicated interactions
with other code.
■
The critical section code can’t be paged out of memory, can’t make references to pageable data, can’t call external procedures (including system services), and can’t generate
interrupts or exceptions.
These restrictions are confining and can’t be met under all circumstances. Furthermore, the
executive needs to perform other types of synchronization in addition to mutual exclusion,
and it must also provide synchronization mechanisms to user mode.
There are several additional synchronization mechanisms for use when spinlocks are not suitable:
■
Kernel Dispatcher Objects
■
Fast Mutexes and Guarded Mutexes
■
Push Locks
■
Executive Resources
Table 3-9 serves as a reference that compares and contrasts the capabilities of these mechanisms and their interaction with kernel-mode APC delivery.
Table 3-9
Kernel Synchronization Mechanisms
Exposed for
Use by
Device
Drivers
Disables Normal Disables Special Supports
Kernel-Mode
Kernel-Mode
Recursive
APCs
APCs
Acquisition
Supports
Shared and
Exclusive
Acquisition
Kernel Dispatcher
Mutexes
Yes
Yes
No
Yes
No
Kernel Dispatcher
Semaphores
Yes
No
No
No
No
Fast Mutexes
Yes
Yes
Yes
No
No
Guarded Mutexes
No
Yes
Yes
No
No
Push Locks
No
No
No
No
Yes
Executive Resources
Yes
Yes
No
Yes
Yes
Copyrighted material.
156
Microsoft Windows Internals, Fourth Edition
Kernel Dispatcher Objects
The kernel furnishes additional synchronization mechanisms to the executive in the form of
kernel objects, known collectively as dispatcher objects. The user-visible synchronization
objects acquire their synchronization capabilities from these kernel dispatcher objects. Each
user-visible object that supports synchronization encapsulates at least one kernel dispatcher
object. The executive’s synchronization semantics are visible to Windows programmers
through the WaitForSingleObject and WaitForMultipleObjects functions, which the Windows
subsystem implements by calling analogous system services the object manager supplies. A
thread in a Windows application can synchronize with a Windows process, thread, event,
semaphore, mutex, waitable timer, I/O completion port, or file object.
One other type of executive synchronization object worth noting is called executive resources.
Executive resources provide both exclusive access (like a mutex) as well as shared read access
(multiple readers sharing read-only access to a structure). However, they’re available only to
kernel-mode code and thus aren’t accessible from the Windows API. Executive resources are
not dispatcher objects but rather data structures allocated directly from nonpaged pool that
have their own specialized services to initialize, lock, release, query, and wait for them. The
executive resource structure is defined in Ntddk.h, and the executive support routines are
documented in the DDK reference documentation.
The remaining subsections describe the implementation details of waiting for dispatcher
objects.
Waiting for Dispatcher Objects A thread can synchronize with a dispatcher object by waiting for the object’s handle. Doing so causes the kernel to suspend the thread and change its
dispatcher state from running to waiting, as shown in Figure 3-25. The kernel removes the
thread from the dispatcher ready queue and no longer considers it for execution.
Note Figure 3-25 is a process state transition diagram with focus on the ready, waiting, and
running states (the states related to waiting for objects). The other states are described in
Chapter 6.
At any given moment, a synchronization object is in one of two states: either the signaled state
or the nonsignaled state. A thread can’t resume its execution until the kernel changes its dispatcher state from waiting to ready. This change occurs when the dispatcher object whose
handle the thread is waiting for also undergoes a state change, from the nonsignaled state to
the signaled state (when a thread sets an event object, for example). To synchronize with an
object, a thread calls one of the wait system services the object manager supplies, passing a
handle to the object it wants to synchronize with. The thread can wait for one or several
objects and can also specify that its wait should be canceled if it hasn’t ended within a certain
amount of time. Whenever the kernel sets an object to the signaled state, the kernel’s KiWaitTest function checks to see whether any threads are waiting for the object and not also waiting
Copyrighted material.
Chapter 3:
System Mechanisms
157
for other objects to become signaled. If there are, the kernel releases one or more of the
threads from their waiting state so that they can continue executing.
Initialized
Set object to
signaled state
Terminated
Ready
Waiting
Thread waits
on an object
handle
Transition
Running
Figure 3-25
Standby
Waiting for a dispatcher object
The following example of setting an event illustrates how synchronization interacts with
thread dispatching:
■
A user-mode thread waits for an event object’s handle.
■
The kernel changes the thread’s scheduling state from ready to waiting and then adds
the thread to a list of threads waiting for the event.
■
Another thread sets the event.
■
The kernel marches down the list of threads waiting for the event. If a thread’s conditions for waiting are satisfied (see Note below), the kernel changes the thread’s state
from waiting to ready. If it is a variable-priority thread, the kernel might also boost its
execution priority.
■
Because a new thread has become ready to execute, the dispatcher reschedules. If it
finds a running thread with a priority lower than that of the newly ready thread, it preempts the lower-priority thread and issues a software interrupt to initiate a context
switch to the higher-priority thread.
■
If no processor can be preempted, the dispatcher places the ready thread in the dispatcher ready queue to be scheduled later.
Note
Copyrighted material.
Some threads might be waiting for more than one object, so they continue waiting.
158
Microsoft Windows Internals, Fourth Edition
What Signals an Object The signaled state is defined differently for different objects. A
thread object is in the nonsignaled state during its lifetime and is set to the signaled state by
the kernel when the thread terminates. Similarly, the kernel sets a process object to the signaled state when the process’s last thread terminates. In contrast, the timer object, like an
alarm, is set to “go off” at a certain time. When its time expires, the kernel sets the timer object
to the signaled state.
When choosing a synchronization mechanism, a program must take into account the rules
governing the behavior of different synchronization objects. Whether a thread’s wait ends
when an object is set to the signaled state varies with the type of object the thread is waiting
for, as Table 3-10 illustrates.
Table 3-10
Definitions of the Signaled State
Object Type
Set to Signaled State When
Effect on Waiting Threads
Process
Last thread terminates
All released
Thread
Thread terminates
All released
File
I/O operation completes
All released
Debug Object
Debug message is queued to
the object
All released
Event (notification type)
Thread sets the event
All released
Event (synchronization type)
Thread sets the event
One thread released; event
object reset
Keyed Event
Thread sets event with a key
Thread waiting for key and
which is of same process as signaler is released
Semaphore
Semaphore count drops by 1
One thread released
Timer (notification type)
Set time arrives or time interval All released
expires
Timer (synchronization type)
Set time arrives or time interval One thread released
expires
Mutex
Thread releases the mutex
One thread released
File
I/O completes
All threads released
Queue
Item is placed on queue
One thread released
When an object is set to the signaled state, waiting threads are generally released from their
wait states immediately. Some of the kernel dispatcher objects and the system events that
induce their state changes are shown in Figure 3-26.
Copyrighted material.
Chapter 3:
Dispatcher object
System Mechanisms
Effect of signaled state
on waiting threads
System events and
resulting state change
Owning thread
releases the mutex.
Mutex (kernelNonsignaled
mode use only)
Signaled
Kernel resumes one
waiting thread.
Signaled
Kernel resumes one
waiting thread.
Resumed thread
acquires the mutex.
Owning thread or other
thread releases the mutex.
Mutex (exported to
Nonsignaled
user mode)
Resumed thread
acquires the mutex.
One thread releases the
semaphore, freeing a resource.
Semaphore Nonsignaled
Signaled
Kernel resumes one
or more waiting threads.
A thread acquires the semaphore.
More resources are not available.
A thread sets the event.
Event Nonsignaled
Signaled
Kernel resumes one
or more waiting threads.
Signaled
Kernel resumes waiting
dedicated thread.
Signaled
Kernel resumes all
waiting threads.
Signaled
Kernel resumes all
waiting threads.
Kernel resumes one
or more threads.
Dedicated thread sets
one event in the event pair.
Event pair Nonsignaled
Kernel resumes the
other dedicated thread.
Timer expires.
Timer Nonsignaled
A thread (re)initializes
the timer.
Thread terminates.
Thread Nonsignaled
A thread reinitializes
the thread object.
Figure 3-26
Copyrighted material.
Selected kernel dispatcher objects
159
160
Microsoft Windows Internals, Fourth Edition
For example, a notification event object (called a manual reset event in the Windows API) is
used to announce the occurrence of some event. When the event object is set to the signaled
state, all threads waiting for the event are released. The exception is any thread that is waiting
for more than one object at a time; such a thread might be required to continue waiting until
additional objects reach the signaled state.
In contrast to an event object, a mutex object has ownership associated with it. It is used to
gain mutually exclusive access to a resource, and only one thread at a time can hold the
mutex. When the mutex object becomes free, the kernel sets it to the signaled state and then
selects one waiting thread to execute. The thread selected by the kernel acquires the mutex
object, and all other threads continue waiting.
Keyed Events and Critical Sections
A synchronization object new to Windows XP, called a keyed event, bears special mention because of the role it plays in helping processes deal with low-memory situations
when using critical sections. A keyed event, which is not documented, allows a thread to
specify a “key” for which it waits, where the thread wakes when another thread of the
same process signals the event with the same key.
Windows processes often use critical section functions, EnterCriticalSection and LeaveCriticalSection, to synchronize thread access to resources private to the process. These
functions have advantages over direct use of mutex objects because if there is no contention they do not make a transition to kernel mode. If there is contention, EnterCriticalSection dynamically allocates an event object and the thread wanting to acquire the
critical section waits for the thread that owns the critical section to signal it in LeaveCriticalSection.
EnterCriticalSection uses a global keyed event named CritSecOutOfMemoryEvent (in the
\Kernel directory of the object manager namespace) when the allocation of the event
object for the critical section fails because system memory is low. If EnterCriticalSection
has to use CritSecOutOfMemoryEvent instead of a standard event, a thread waiting for the
critical section uses the address of the critical section as the key. This allows the critical
section functions to operate properly even when memory is temporarily low.
This brief discussion wasn’t meant to enumerate all the reasons and applications for using the
various executive objects but rather to list their basic functionality and synchronization
behavior. For information on how to put these objects to use in Windows programs, see the
Windows reference documentation on synchronization objects or Jeffrey Richter’s Programming Applications for Microsoft Windows.
Copyrighted material.
Chapter 3:
System Mechanisms
161
Data Structures Two data structures are key to tracking who is waiting for what: dispatcher
headers and wait blocks. Both these structures are publicly defined in the DDK include file
Ntddk.h. The definitions are reproduced here for convenience:
typedef struct _DISPATCHER_HEADER {
UCHAR Type;
UCHAR Absolute;
UCHAR Size;
UCHAR Inserted;
LONG SignalState;
LIST_ENTRY WaitListHead;
} DISPATCHER_HEADER;
typedef struct _KWAIT_BLOCK {
LIST_ENTRY WaitListEntry;
struct _KTHREAD *RESTRICTED_POINTER Thread;
PVOID Object;
struct _KWAIT_BLOCK *RESTRICTED_POINTER NextWaitBlock;
USHORT WaitKey;
USHORT WaitType;
} KWAIT_BLOCK, *PKWAIT_BLOCK, *RESTRICTED_POINTER PRKWAIT_BLOCK;
The dispatcher header contains the object type, signaled state, and a list of the threads waiting
for that object. The wait block represents a thread waiting for an object. Each thread that is in
a wait state has a list of the wait blocks that represent the objects the thread is waiting for.
Each dispatcher object has a list of the wait blocks that represent which threads are waiting for
the object. This list is kept so that when a dispatcher object is signaled, the kernel can quickly
determine who is waiting for that object. The wait block has a pointer to the object being
waited for, a pointer to the thread waiting for the object, and a pointer to the next wait block
(if the thread is waiting for more than one object). It also records the type of wait (any or all)
as well as the position of that entry in the array of handles passed by the thread on the WaitForMultipleObjects call (position zero if the thread was waiting for only one object).
Figure 3-27 shows the relationship of dispatcher objects to wait blocks to threads. In this
example, thread 1 is waiting for object B, and thread 2 is waiting for objects A and B. If object
A is signaled, the kernel will see that because thread 2 is also waiting for another object,
thread 2 can’t be readied for execution. On the other hand, if object B is signaled, the kernel
can ready thread 1 for execution right away because it isn’t waiting for any other objects.
Copyrighted material.
162
Microsoft Windows Internals, Fourth Edition
Thread objects
Thread 1
Thread 2
Wait block list
Wait block list
Dispatcher objects
Size
Object A
Type
State
Wait blocks
Wait list head
List entry
Object-typespecific data
Thread
Object
Key
Type
Next link
Size
Type
State
Object B
Thread 2 wait block
Wait list head
List entry
List entry
Object-typespecific data
Thread
Thread
Object
Key
Figure 3-27
Type
Object
Key
Type
Next link
Next link
Thread 1 wait block
Thread 2 wait block
Wait data structures
EXPERIMENT: Looking at Wait Queues
Although many process viewer utilities indicate whether a thread is in a wait state (and
if so, they also indicate what kind of wait), you can see the list of objects a thread is waiting for only with the kernel debugger !thread command. For example, the following
excerpt from the output of a !process command shows that the thread is waiting for an
event object:
kd> !process
§
THREAD 8a12a328 Cid 0bb8.0d50 Teb: 7ffdd000 Win32Thread: e7c9aeb0 WAIT
: (WrUserRequest) UserMode Non-Alertable
8a21bf58 SynchronizationEvent
Copyrighted material.
Chapter 3:
System Mechanisms
163
You can use the dt command to interpret the dispatcher header of the object like this:
kd> dt nt!_dispatcher_header
nt!_DISPATCHER_HEADER
+0x000 Type
:
+0x001 Absolute
:
+0x002 Size
:
+0x003 Inserted
:
+0x004 SignalState
:
+0x008 WaitListHead
:
8a21bf58
0x1 ’’
0 ’’
0x4 ’’
0 ’’
0
_LIST_ENTRY [ 0x8a12a398 - 0x8a12a398 ]
From this, we can ascertain that no other threads are waiting for this event object
because the wait list head forward and backward pointers point to the same location (a
single wait block). Dumping the wait block (at address 0x8a12a398) yields the following:
kd> dt nt!_kwait_block 0x8a12a398
nt!_KWAIT_BLOCK
+0x000 WaitListEntry
: _LIST_ENTRY [ 0x8a21bf60 - 0x8a21bf60 ]
+0x008 Thread
: 0x8a12a328
+0x00c Object
: 0x8a21bf58
+0x010 NextWaitBlock
: 0x8a12a398
+0x014 WaitKey
: 0
+0x016 WaitType
: 1
If the wait list had more than one entry, you could execute the same command on the
second pointer value in the WaitListEntry field of each wait block (by executing !thread
on the thread pointer in the wait block) to traverse the list and see what other threads
are waiting for the object.
Fast Mutexes and Guarded Mutexes
Fast mutexes, which are also known as executive mutexes, usually offer better performance
than mutex objects because, although they are built on dispatcher event objects, they avoid
waiting for the event object (and therefore the spinlocks on which an event object is based) if
there’s no contention for the fast mutex. This gives the fast mutex especially good performance in a multiprocessor environment. Fast mutexes are used widely throughout the kernel
and device drivers.
However, fast mutexes are suitable only when normal kernel-mode APC (described earlier in
this chapter) delivery can be disabled. The executive defines two functions for acquiring
them: ExAcquireFastMutex and ExAcquireFastMutexUnsafe. The former function blocks all APC
delivery by raising the IRQL of the processor to APC_LEVEL and the latter expects to be
called with normal kernel-mode APC delivery disabled, which can be done by raising the
IRQL to APC level or by calling KeEnterCriticalRegion. Another limitation of fast mutexes is
that they can’t be acquired recursively like mutex objects can.
Copyrighted material.
164
Microsoft Windows Internals, Fourth Edition
Guarded mutexes are new to Windows Server 2003 and are essentially the same as fast
mutexes (although they use a different synchronization object, the KGATE, internally). They
are acquired with the KeAcquireGuardedMutex function, but instead of disabling APCs by calling KeEnterCriticalRegion, which disables only normal kernel-mode APCs, it disables all kernel-mode APC delivery by calling KeEnterGuardedRegion. They are not exposed for use outside
of the kernel and are used primarily by the memory manager, which uses them to protect global operations such as creating paging files, deleting certain types of shared memory sections,
and performing paged pool expansion. (See Chapter 7 for more information on the memory
manager.)
Executive Resources
Executive resources are a synchronization mechanism that supports shared and exclusive
access, and like fast mutexes, require that normal kernel-mode APC delivery be disabled
before they are acquired. They are also built on dispatcher objects that are only used when
there is contention. Executive resources are used throughout the system, especially in file-system drivers.
Threads waiting to acquire a resource for shared access wait for a semaphore associated with
the resource, and threads waiting to acquire a resource for exclusive access wait for an event.
A semaphore with unlimited count is used for shared waiters because they can all be woken
and granted access to the resource when an exclusive holder releases the resource simply by
signaling the semaphore. When a thread waits for exclusive access of a resource that is currently owned, it waits on a synchronization event object because only one of the waiters will
wake when the event is signaled.
Because of the flexibility that shared and exclusive access offers, there are a number of functions for acquiring resources: ExAcquireResourceSharedLite, ExAcquireResourceExclusiveLite,
ExAcquireSharedStarveExclusive, ExAcquireWaitForExclusive, and ExTryToAcquireResourceExclusiveLite. These functions are documented in the DDK.
EXPERIMENT: Listing Acquired Executive Resources
The kernel debugger !locks command searches paged pool for executive resource objects
and dumps their state. By default, the command lists only executive resources that are
currently owned, but the –d option will list all executive resources. Here is partial output
of the command:
lkd> !locks
**** DUMP OF ALL RESOURCE OBJECTS ****
KD: Scanning for held locks.
Resource @ nt!MmSystemWsLock (0x805439a0)
Exclusively owned
Contention Count = 123
Threads: 89b36020-01<*>
KD: Scanning for held locks...........................................................
Copyrighted material.
Chapter 3:
System Mechanisms
165
......................................................................................
..............................
Resource @ 0x89da1a68
Shared 1 owning threads
Threads: 8a4cb533-01<*> *** Actual Thread 8a4cb530
Note that the contention count, which is extracted from the resource structure, records
the number of times threads have tried to acquire the resource and had to wait because
it was already owned.
You can examine the details of a specific resource object, including the thread that owns
the resource and any threads that are waiting for the resource, by specifying the –v
switch and the address of the resource:
lkd> !locks -v 0x805439a0
Resource @ nt!MmSystemWsLock (0x805439a0)
Contention Count = 123
Threads: 89b36020-01<*>
Exclusively owned
THREAD 89b36020 Cid 0e98.0bd4 Teb: 7ffd9000 Win32Thread: e2bcc538 RUNNING on pr
ocessor 0
Not impersonating
DeviceMap
e1df7d18
Owning Process
8999d020
Wait Start TickCount
492582
Elapsed Ticks: 15
Context Switch Count
532
LargeStack
UserTime
00:00:01.0462
KernelTime
00:00:00.0320
Start Address 0x77e7d342
Win32 Start Address 0x0101f1d0
Stack Init a9d20000 Current a9d1fd44 Base a9d20000 Limit a9d1d000 Call 0
Priority 11 BasePriority 8 PriorityDecrement 2 DecrementCount 16
Unable to get context for thread running on processor 0, HRESULT 0x80004001
Push Locks
Push locks, which were introduced in Windows XP, are another optimized synchronization
mechanism built on the event object (and in Windows Server 2003 they are built on the internal KGATE synchronization object), and like fast mutexes, they wait for an event object only
when there’s contention on the lock. They offer advantages over the fast mutex in that they
can be acquired in shared or exclusive mode. They are not documented or exported by the
kernel and are therefore reserved for use by the operating system.
There are two types of push locks: normal and cache aware. Normal push locks require only
the size of a pointer in storage (4 bytes on 32-bit systems and 8 bytes on 64-bit systems).
When a thread acquires a normal push lock, the push lock code marks the push lock as
owned if it is not currently owned. If the push lock is owned exclusively or the thread wants
to acquire the thread exclusively and the push lock is owned on a shared basis, the thread alloCopyrighted material.
166
Microsoft Windows Internals, Fourth Edition
cates a wait block on the thread’s stack, initializes an event object in the wait block, and adds
the wait block to the wait list associated with the push lock. When a thread releases a push
lock, the thread wakes a waiter, if any are present, by signaling the event in the waiter’s wait
block.
A cache-aware push lock layers on the basic push lock by allocating a push lock for each processor in the system and associating it with the cache-aware push lock. When a thread wants
to acquire a cache-aware push lock for shared access, it simply acquires the push lock allocated for its current processor in shared mode; to acquire a cache-aware push lock exclusively,
it acquires the push lock for each processor in exclusive mode.
Areas where push locks are used include the object manager, where they protect global object
manager data structures and object security descriptors, and the memory manager, where
they protect AWE data structures.
Deadlock Detection with Driver Verifier
A deadlock is a synchronization issue resulting from two threads or processors holding
resources that the other wants and neither will yield what it has. This situation might
result in system or process hangs. Driver Verifier, described in Chapter 7 and Chapter 9,
has an option to check for deadlocks involving spinlocks, fast mutexes, and mutexes.
For information on when to enable Driver Verifier to help resolve system hangs, see
Chapter 14.
System Worker Threads
During system initialization, Windows creates several threads in the System process, called
system worker threads, that exist solely to perform work on behalf of other threads. In many
cases, threads executing at DPC/dispatch level need to execute functions that can be performed only at a lower IRQL. For example, a DPC routine, which executes in an arbitrary
thread context (because DPC execution can usurp any thread in the system) at DPC/dispatch
level IRQL, might need to access paged pool or wait for a dispatcher object used to synchronize execution with an application thread. Because a DPC routine can’t lower the IRQL, it
must pass such processing to a thread that executes at an IRQL below DPC/dispatch level.
Some device drivers and executive components create their own threads dedicated to processing work at passive level; however, most use system worker threads instead, which avoids the
unnecessary scheduling and memory overhead associated with having additional threads in
the system. A device driver or an executive component requests a system worker thread’s services by calling the executive functions ExQueueWorkItem or IoQueueWorkItem. These functions place a work item on a queue dispatcher object where the threads look for work. (Queue
Copyrighted material.
Chapter 3:
System Mechanisms
167
dispatcher objects are described in more detail in the section “I/O Completion Ports” in Chapter 9.) Work items include a pointer to a routine and a parameter that the thread passes to the
routine when it processes the work item. The routine is implemented by the device driver or
executive component that requires passive-level execution.
For example, a DPC routine that must wait for a dispatcher object can initialize a work item
that points to the routine in the driver that waits for the dispatcher object, and perhaps points
to a pointer to the object. At some stage, a system worker thread will remove the work item
from its queue and execute the driver’s routine. When the driver’s routine finishes, the system
worker thread checks to see whether there are more work items to process. If there aren’t any
more, the system worker thread blocks until a work item is placed on the queue. The DPC
routine might or might not have finished executing when the system worker thread processes
its work item. (On a uniprocessor system, a DPC routine always finishes executing before its
work item is processed because thread scheduling doesn’t take place when the IRQL is at
DPC/dispatch level.)
There are three types of system worker threads:
■
Delayed worker threads execute at priority 12, process work items that aren’t considered
time-critical, and can have their stack paged out to a paging file while they wait for work
items.
■
Critical worker threads execute at priority 13, process time-critical work items, and on
Windows Server systems, have their stacks present in physical memory at all times.
■
A single hypercritical worker thread executes at priority 15 and also keeps its stack in
memory. The process manager uses the hypercritical work item to execute the thread
“reaper” function that frees terminated threads.
The number of delayed and critical worker threads created by the executive’s ExpWorkerInitialization function, which is called early in the boot process, depends on the amount of memory present on the system and whether the system is a server. Table 3-11 shows the initial
number of threads created on different system configurations. You can specify that ExpInitializeWorker create up to 16 additional delayed and 16 additional critical worker threads with the
AdditionalDelayedWorkerThreads and AdditionalCriticalWorkerThreads values under the
registry key HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Executive.
Table 3-11
Initial Number of System Worker Threads
Windows 2000
Windows 2000
Server
Windows XP and
Windows Server 2003
Delayed
3
3
7
Critical
5
10
5
Hypercritical
1
1
1
Copyrighted material.
168
Microsoft Windows Internals, Fourth Edition
The executive tries to match the number of critical worker threads with changing workloads
as the system executes. Once every second, the executive function ExpWorkerThreadBalanceManager determines whether it should create a new critical worker thread. The critical worker
threads that are created by ExpWorkerThreadBalanceManager are called dynamic worker
threads, and all the following conditions must be satisfied before such a thread is created:
■
Work items exist in the critical work queue.
■
The number of inactive critical worker threads (ones that are either blocked waiting for
work items or that have blocked on dispatcher objects while executing a work routine)
must be less than the number of processors on the system.
■
There are fewer than 16 dynamic worker threads.
Dynamic worker threads exit after 10 minutes of inactivity. Thus, when the workload dictates,
the executive can create up to 16 dynamic worker threads.
EXPERIMENT: Listing System Worker Threads
You can use the !exqueue kernel debugger command to see a listing of system worker
threads classified by their type:
kd> !exqueue
Dumping ExWorkerQueue: 8046A5C0
**** Critical WorkQueue( current = 0 maximum = 1 )
THREAD 818a2d40 Cid 8.c
Teb: 00000000 Win32Thread:
THREAD 818a2ac0 Cid 8.10 Teb: 00000000 Win32Thread:
THREAD 818a2840 Cid 8.14 Teb: 00000000 Win32Thread:
THREAD 818a25c0 Cid 8.18 Teb: 00000000 Win32Thread:
THREAD 818a2340 Cid 8.1c Teb: 00000000 Win32Thread:
**** Delayed WorkQueue( current
THREAD 818a20c0 Cid 8.20 Teb:
THREAD 818a1020 Cid 8.24 Teb:
THREAD 818a1da0 Cid 8.28 Teb:
00000000
00000000
00000000
00000000
00000000
WAIT
WAIT
WAIT
WAIT
WAIT
= 0 maximum = 1 )
00000000 Win32Thread: 00000000 WAIT
00000000 Win32Thread: 00000000 WAIT
00000000 Win32Thread: 00000000 WAIT
**** HyperCritical WorkQueue( current = 0 maximum = 1 )
THREAD 818a1b20 Cid 8.2c Teb: 00000000 Win32Thread: 00000000 WAIT
Windows Global Flags
Windows has a set of flags stored in a systemwide global variable named NtGlobalFlag that
enable various internal debugging, tracing, and validation support in the operating system.
The system variable NtGlobalFlag is initialized from the registry key HKLM\SYSTEM\CurrentControlSet\Control\Session Manager in the value GlobalFlag at system boot time. By
Copyrighted material.
Chapter 3:
System Mechanisms
169
default, this registry value is 0, so it’s likely that on your systems, you’re not using any global
flags. In addition, each image has a set of global flags that also turn on internal tracing and validation code (although the bit layout of these flags is entirely different than the systemwide
global flags). These flags aren’t documented or supported for customer use, but they can be
useful tools for exploring the internal operation of Windows.
Fortunately, the Platform SDK and the debugging tools contain a utility named Gflags.exe
that allows you to view and change the system global flags (either in the registry or in the running system) as well as image global flags. Gflags has both a command-line and a GUI interface. To see the command-line flags, type gflags /?. If you run the utility without any switches,
the dialog box shown in Figure 3-28 is displayed.
Figure 3-28
Setting system debugging options with Gflags
You can toggle between the settings in the registry (by clicking System Registry) and the current value of the variable in system memory (by clicking Kernel Mode). You must click the
Apply button to make the changes. (You’ll exit if you click the OK button.) Although you can
change flag settings on a running system, most flags require a reboot to take effect, and there’s
no documentation on which flags do and which don’t require rebooting. So when in doubt,
reboot after changing a global flag.
The Image File Options choice requires that you fill in the filename of a valid executable
image. This option is used to change a set of global flags that apply to an individual image
(rather than to the whole system). In Figure 3-29, notice that the flags are different than the
operating system ones shown in Figure 3-28.
Copyrighted material.
170
Microsoft Windows Internals, Fourth Edition
Figure 3-29
Setting image global flags with Gflags
EXPERIMENT: Enabling Image Loader Tracing and Viewing
NtGlobalFlag
To see an example of the detailed tracing information you can obtain by setting global
flags, try running Gflags on a system booted with the kernel debugger that is connected
to a host system running Kd or Windbg.
As an example, try enabling the Show Loader Snaps flag. To do this, choose Kernel
Mode, select the Show Loader Snaps check box, and click the Apply button. Then run
an image on this machine, and in the kernel debugger you’ll see volumes of output like
the following:
LDR: PID: 0xb8 started - ’notepad’
LDR: NEW PROCESS
Image Path: C:\Windows\system32\notepad.exe (notepad.exe)
Current Directory: C:\ddk\bin
Search Path: C:\Windows\System32;C:\Windows\system;C:\Windows
LDR: notepad.exe bound to comdlg32.dll
LDR: ntdll.dll used by comdlg32.dll
LDR: Snapping imports for comdlg32.dll from ntdll.dll
§
LDR: KERNEL32.dll loaded. - Calling init routine at 77f01000
LDR: RPCRT4.dll loaded. - Calling init routine at 77e1b6d5
LDR: ADVAPI32.dll loaded. - Calling init routine at 77dc1000
LDR: USER32.dll loaded. - Calling init routine at 77e78037
Copyrighted material.
Chapter 3:
System Mechanisms
171
You can use the !gflags and !gflag kernel debugger commands to view the state of the
NtGlobalFlag kernel variable. The !gflags command lists all the flags, indicating which
ones are enabled, whereas !gflag reports only the flags that are enabled.
kd> !gflags
NT!NtGlobalFlag 0x4400
STOP_ON_EXCEPTION
DEBUG_INITIAL_COMMAND
HEAP_ENABLE_TAIL_CHECK
HEAP_VALIDATE_PARAMETERS
*POOL_ENABLE_TAGGING
USER_STACK_TRACE_DB
*MAINTAIN_OBJECT_TYPELIST
ENABLE_CSRDEBUG
DISABLE_PAGE_KERNEL_STACKS
ENABLE_CLOSE_EXCEPTIONS
ENABLE_HANDLE_TYPE_TAGGING
DEBUG_INITIAL_COMMAND_EX
SHOW_LDR_SNAPS
STOP_ON_HUNG_GUI
HEAP_ENABLE_FREE_CHECK
HEAP_VALIDATE_ALL
HEAP_ENABLE_TAGGING
KERNEL_STACK_TRACE_DB
HEAP_ENABLE_TAG_BY_DLL
ENABLE_KDEBUG_SYMBOL_LOAD
HEAP_DISABLE_COALESCING
ENABLE_EXCEPTION_LOGGING
HEAP_PAGE_ALLOCS
DISABLE_DBGPRINT
kd> !gflag
NtGlobalFlag at 8046a164 Current NtGlobalFlag contents: 0x00004400
ptg - Enable pool tagging
otl - Maintain a list of objects for each type
Local Procedure Calls (LPCs)
A local procedure call (LPC) is an interprocess communication facility for high-speed message
passing. It is not directly available through the Windows API; it is an internal mechanism
available only to Windows operating system components. Here are some examples of where
LPCs are used:
■
Windows applications that use remote procedure calls (RPCs), a documented API, indirectly use LPCs when they specify local-RPC, a form of RPC used to communicate
between processes on the same system.
■
A few Windows APIs result in sending messages to the Windows subsystem process.
■
Winlogon uses LPCs to communicate with the local security authentication server process, LSASS.
■
The security reference monitor (an executive component explained in Chapter 8) uses
LPCs to communicate with the LSASS process.
Copyrighted material.
172
Microsoft Windows Internals, Fourth Edition
EXPERIMENT: Viewing LPC Port Objects
You can see named LPC port objects with the Winobj tool from www.sysinternals.com.
Run Winobj.exe and select the root directory. A plug icon identifies the port objects, as
shown here:
To see the LPC port objects used by RPC, select the \RPC Control directory, as shown
here:
You can also view LPC port objects by using the !lpc kernel debugger command. The
command accepts parameters that direct it to show LPC ports, LPC messages, and
threads that are waiting or sending LPC messages. To view the LSASS authentication
port (the port that Winlogon sends logon requests to), first obtain a list of the ports on
the system:
Copyrighted material.
Chapter 3:
System Mechanisms
kd> !lpc
Usage:
!lpc
- Display this help
!lpc message [MessageId] - Display the message with a
given ID and all related
information
If MessageId is not
specified, dump all messages
!lpc port [PortAddress] - Display the port information
!lpc scan PortAddress
- Search this port and any
connected port
!lpc thread [ThreadAddr] - Search the thread in rundown
port queues and display the
port info
If ThreadAddr is missing,
display all threads marked
as doing some lpc operations
kd> !lpc port
Scanning 206 objects
1 Port: 0xe1360320 Connection: 0xe1360320
Communication: 0x00000000 ’SeRmCommandPort’
1 Port: 0xe136bc20 Connection: 0xe136bc20
Communication: 0x00000000 ’SmApiPort’
1 Port: 0xe133ba80 Connection: 0xe133ba80
Communication: 0x00000000 ’DbgSsApiPort’
1 Port: 0xe13606e0 Connection: 0xe13606e0
Communication: 0x00000000 ’DbgUiApiPort’
§
1 Port: 0xe205f040 Connection: 0xe205f040
Communication: 0x00000000 ’LsaAuthenticationPort’
§
Locate the port named LsaAuthenticationPort in the output and then examine it by
passing its address to the !lpc command, as shown in the following code segment.
kd> !lpc port 0xe205f040
Server connection port e205f040 Name: LsaAuthenticationPort
Handles: 1
References: 37
Server process : ff7d56c0 (lsass.exe)
Queue semaphore : ff7bfcc8
Semaphore state 0 (0x0)
The message queue is empty
The LpcDataInfoChainHead queue is empty
Copyrighted material.
173
174
Microsoft Windows Internals, Fourth Edition
Typically, LPCs are used between a server process and one or more client processes of that
server. An LPC connection can be established between two user-mode processes or between a
kernel-mode component and a user-mode process. For example, as noted in Chapter 2, Windows processes send occasional messages to the Windows subsystem by using LPCs. Also,
some system processes use LPCs to communicate, such as Winlogon and Lsass. An example
of a kernel-mode component using an LPC to talk to a user process is the communication
between the security reference monitor and the LSASS process.
LPCs are designed to allow three methods of exchanging messages:
■
A message that is shorter than 256 bytes can be sent by calling the LPC with a buffer
containing the message. This message is then copied from the address space of the sending process into system address space, and from there to the address space of the receiving process.
■
If a client and a server want to exchange more than 256 bytes of data, they can choose to
use a shared section to which both are mapped. The sender places message data in the
shared section and then sends a small message to the receiver with pointers to where the
data is to be found in the shared section.
■
When a server wants to read or write larger amounts of data than will fit in a shared section, data can be directly read from or written to a client’s address space. The LPC component supplies two functions that a server can use to accomplish this. A message sent
by the first function is used to synchronize the message passing.
An LPC exports a single executive object called the port object to maintain the state needed for
communication. Although an LPC uses a single object type, it has several kinds of ports:
■
A named port that is a server connection request point. Clients
can connect to the server by connecting to this port.
■
Server communication port An unnamed port a server uses to communicate with a particular client. The server has one such port per active client.
■
Client communication port An unnamed port a particular client thread uses to commu-
Server connection port
nicate with a particular server.
■
Unnamed communication port
An unnamed port created for use by two threads in the
same process.
LPCs are typically used as follows: A server creates a named server connection port object. A
client makes a connect request to this port. If the request is granted, two new unnamed ports,
a client communication port and a server communication port, are created. The client gets a
handle to the client communication port, and the server gets a handle to the server communication port. The client and the server will then use these new ports for their communication.
A completed connection between a client and a server is shown in Figure 3-30.
Copyrighted material.
Chapter 3:
Client address
space
Kernel address space
System Mechanisms
175
Server address
space
Connection port
Message
queue
Client process
Server process
Handle
Handle
Client
communication
port
Server
communication
port
Client view
of section
Handle
Server view
of section
Shared
section
Figure 3-30
Use of LPC ports
Kernel Event Tracing
Various components of the Windows kernel and several core device drivers are instrumented
to record trace data of their operation for use in system troubleshooting. They rely on a common infrastructure in the kernel that provides trace data to the user-mode Event Tracing for
Windows (ETW) facility. An application that uses ETW falls into one or more of three categories:
■
Controller
■
Provider
■
Consumer
A controller starts and stops logging sessions and manages buffer pools.
A provider defines GUIDs (globally unique identifiers) for the event classes it
can produce traces for and registers them with ETW. The provider accepts commands
from a controller for starting and stopping traces of the event classes for which it’s
responsible.
A consumer selects one or more trace sessions for which it wants to read
trace data. They can receive the events in buffers in real-time or in log files.
Windows Server systems include several built-in providers in user mode, including ones for
Active Directory, Kerberos, and Netlogon. ETW defines a logging session with the name NT
Kernel Logger (also known as the kernel logger) for use by the kernel and core drivers. The
provider for the NT Kernel Logger is implemented by the Windows Management Instrumen-
Copyrighted material.
176
Microsoft Windows Internals, Fourth Edition
tation (WMI) device driver (driver name Wmixwdm), which is part of Ntoskrnl.exe. (See the
WMI section in Chapter 5 for more information on WMI.) Besides serving as the core of the
kernel logger, the driver manages user-mode ETW event class registration.
The WMI driver exports I/O control interfaces for use by the ETW routines in user mode and
the device drivers that provide traces data for the kernel logger. (See Chapter 9 for more information on I/O control commands.) It also implements functions for use by the components
in Ntoskrnl.exe kernel mode that produce trace output.
When a controller in user mode enables the kernel logger, the ETW library, which is implemented in \Windows\System32\Ntdll.dll, sends an I/O control request to the WMI driver
telling it which event classes the controller wants to start tracing. If file logging is configured
(as opposed to in-memory logging to a buffer), the WMI driver creates a system thread in the
system process that creates a log file. When the WMI driver receives trace events from the
enabled trace sources, it records them to a buffer. If it was started, the file logging thread
wakes up once per second to dump the contents of the buffers to the log file.
Trace records generated for the kernel logger have a standard ETW trace event header, which
records timestamp, process, and thread IDs, as well as information on what class of event the
record corresponds to. Event classes can provide additional data specific to their events. For
example, disk event class trace records indicate the operation type (read or write), disk number at which the operation is directed, and sector offset and length of the operation.
The trace classes that can be enabled for the kernel logger and the component that generates
each class include:
■
Disk I/O
■
File I/O File system drivers
■
Hardware Configuration
Disk class driver
Plug and play manager (See Chapter 9 for information on the
Plug and Play Manager.)
■
Image Load/Unload
■
Page Faults
■
Process Create/Delete Process manager (See Chapter 6 for more information on the
The system image loader in the kernel
Memory manager (See Chapter 7 for more information on page faults.)
process manager.)
■
Thread Create/Delete Process manager
■
Registry Activity Configuration manager (See “The Registry” section in Chapter 4 for
more information on the configuration manager.)
■
TCP/UDP Activity TCP/IP driver
You can find more information on ETW and the kernel logger, including sample code for controllers and consumers, in the Platform SDK.
Copyrighted material.
Chapter 3:
System Mechanisms
177
EXPERIMENT: Tracing TCP/IP Activity with the Kernel Logger
To enable the kernel logger and have it generate a log file of TCP/IP activity, follow these
steps:
1. Run the Performance Tool, and select the Performance Logs And Alerts node.
2. Select Trace Logs, and then select New Log Settings from the Action menu.
3. When prompted, enter a name for the settings (for example, experiment).
4. On the dialog box that opens, select the Events Logged By System Provider option
and then deselect everything except the Network TCP/IP option.
5. In the Run As edit box, enter the Administrator account name and set the password to match it.
6. Dismiss the dialog box, and generate network activity by opening a browser and
visiting a Web site.
7. Select the trace log you created in the trace log node, and select Stop from the
Action menu.
8. Open a command prompt, and change to the C:\Perflogs directory (or the directory into which you specified that the trace log file be stored).
9. If you are running Windows XP or Windows Server 20003, run Tracerpt (located
in the \Windows\System32 directory) and pass it the name of the trace log file. If
you are running Windows 2000, download and run Tracedmp from the Windows
2000 Resource Kit. Both tools generate two files: dumpfile.csv and summary.txt.
Copyrighted material.
178
Microsoft Windows Internals, Fourth Edition
10. Open dumpfile.csv in Microsoft Excel or in a text editor. You should see TCP and/
or UDP trace records like the following:
TcpIp
Recv 0xFFFFFFFF
1.27E+17
0
0
4
88
192.168.001.101
192.168.001.108 4608
0
0
0
TcpIp Send 0xFFFFFFFF
1.27E+17
0
0
4
76
192.168.001.101
192.168.001.108 4608
0
0
0
TcpIp
Recv 0xFFFFFFFF
1.27E+17
0
0
4
88
192.168.001.101
192.168.001.108 4608
0
0
0
TcpIp Send 0xFFFFFFFF
1.27E+17
0
0
4
76
192.168.001.101
192.168.001.108 4608
0
0
0
TcpIp
Recv 0xFFFFFFFF
1.27E+17
0
0
4
88
192.168.001.101
192.168.001.108 4608
0
0
0
TcpIp Send 0xFFFFFFFF
1.27E+17
0
0
4
76
192.168.001.101
192.168.001.108 4608
0
0
0
TcpIp
Recv 0xFFFFFFFF
1.27E+17
0
0
4
88
192.168.001.101
192.168.001.108 4608
0
0
0
TcpIp Send 0xFFFFFFFF
1.27E+17
0
0
4
76
192.168.001.101
192.168.001.108 4608
0
0
0
Wow64
Wow64 (Win32 emulation on 64-bit Windows) refers to the software that permit the execution
of 32-bit x86 applications on 64-bit Windows. It is implemented as a set of user-mode Dlls:
■
Wow64.dll: Manages process and thread creation, hooks exception dispatching and
base system calls exported by Ntoskrnl.exe. It also implements file system redirection
and registry redirection and reflection.
■
Wow64Cpu.dll: Manages the 32-bit CPU context of each running thread inside Wow64,
and provides processor architecture-specific support for switching CPU mode from 32bit to 64-bit and vice versa.
■
Wow64Win.dll: Intercepts the GUI system calls exported by Win32k.sys.
The relationship of these Dlls is shown in Figure 3-31.
32-bit EXE, DLLs
32-bit Ntdll.dll
Wow64cpu.dll
Wow64.dll
Wow64win.dll
64-bit Ntdll.dll
Ntoskrnl.exe
Figure 3-31
Copyrighted material.
Win32k.sys
Wow64 architecture
Chapter 3:
System Mechanisms
179
Wow64 Process Address Space Layout
Wow64 processes may run with 2 GB or 4 GB of virtual space. If the image header has the
large address aware flag set, then the memory manager will reserve the user mode address
space above the 4 GB boundary through the end of the user mode boundary. If the image is
not marked large address space aware, the memory manager will reserve the user mode
address space above 2 GB. (For more information on large address space support, see the section “x86 User Address Space Layouts” in Chapter 7.)
System Calls
Wow64 hooks all the code paths where 32-bit code would transition to the native 64-bit system or when the native system needs to call into 32-bit user mode code. During process creation, the process manager maps into the process address space the native 64-bit Ntdll.dll.
When the loader initialization is called, it inspects the image header and if it is 32-bit x86, it
loads Wow64.dll. Wow64 then maps in the 32-bit Ntdll.dll (stored in the \Windows\System32\Syswow64 directory). Wow64 then sets up the startup context inside Ntdll,
switches the CPU mode to 32-bits, and starts executing the 32-bit loader. From this point
onward, execution continues as if the process is running on a native 32-bit system.
Special 32-bit versions of Ntdll.dll, User32.dll, and Gdi32.dll are located in the \Windows\System32\Syswow64 folder. These call into Wow64 rather than issuing the native 32-bit
system call instruction. Wow64 transitions to native 64-bit mode, captures the parameters associated with the system call (converting 32-bit pointers to 64-bit pointers), and issues the corresponding native 64-bit system call. When the native system call returns, Wow64 converts any
output parameters if necessary from 64-bit to 32-bit formats before returning to 32-bit mode.
Exception Dispatching
Wow64 hooks exception dispatching through ntdll’s KiUserExceptionDispatcher. Whenever
the 64-bit kernel is about to dispatch an exception to a Wow64 process, Wow64 captures the
native exception and context record in user mode and then prepares a 32-bit exception and
context record and dispatches it the same way the native 32-bit kernel would do.
User Callbacks
Wow64 intercepts all callbacks from the kernel into user mode. Wow64 treats such calls as
system calls; however, the data conversion is done in the reverse order: input parameters are
converted from 64-bits to 32-bits and output parameters are converted when the callback
returns from 32-bit to 64-bit.
Copyrighted material.
180
Microsoft Windows Internals, Fourth Edition
File System Redirection
To maintain application compatibility and to reduce the effort of porting applications from
Win32 to 64-bit Windows, system directory names were kept the same. Therefore, the \Windows\System32 contains native 64-bit images. Wow64, as it hooks all the system calls, translates all the path-related APIs, and replaces the path name of the \Windows\System32 folder
with \Windows\System32\Syswow64. Wow64 also redirects \Windows\System32\Ime to
\Windows\System32\IME (x86) to help 32-bit application compatibility on 64-bit systems
with Far East languages installed. Also, 32-bit programs are installed in \Program Files (x86),
while 64-bit programs go in the normal \Program Files folder.
There are a few subdirectories of \Windows\System32 which, for compatibility reasons, are
exempted from being redirected such that accesses to them made by 32-bit applications actually access the real one. These directories include:
■
%windir%\system32\drivers\etc
■
%windir%\system32\spool
■
%windir%\system32\catroot2
■
%windir%\system32\logfiles
Finally, Wow64 provides a mechanism to disable the file system redirection built into Wow64
on a per-thread basis using the Wow64EnableWow64FsRedirection function, available on Windows Server 2003 and later.
Registry Redirection and Reflection
Applications and components store their configuration data in the registry. Components usually write their configuration data in the registry when they are registered during installation.
If the same component is installed and registered both as a 32-bit binary and a 64-bit binary,
then the last component being registered will override the registration of the previous component as they both write to the same location in the registry.
To help solve this problem transparently without introducing any code changes to 32-bit components, the registry is split into two portions: Native and Wow64. By default, 32-bit components access the 32-bit view, and 64-bit components access the 64-bit view. This provides a
safe execution environment for 32-bit and 64-bit components and separates the 32-bit application state from the 64-bit one if it exists.
To implement this, Wow64 intercepts all the system calls that open registry keys and re-translates the key path to point it to the Wow64 view of the registry. Wow64 splits the registry at
these points:
■
HKLM\Software
■
HKEY_CLASSES_ROOT
Copyrighted material.
Chapter 3:
■
System Mechanisms
181
HKEY_CURRENT_USER\Software\Classes
Under each of these keys, Wow64 creates a key called Wow6432Node. Under this key is
stored 32-bit configuration information. All other portions of the registry are shared between
32-bit and 64-bit applications (e.g., HKLM\System).
For applications that need to explicitly specify a registry key for a certain view, the following
flags on the RegOpenKeyEx and RegCreateKeyEx permit this:
■
KEY_WOW64_64KEY – explicitly opens a 64-bit key from either a 32-bit or 64-bit
application
■
KEY_WOW64_32KEY – explicitly opens a 32-bit key from either a 32-bit or 64-bit
application
To enable interoperability through 32-bit and 64-bit COM components, Wow64 mirrors certain portions of the registry when updated in one view to the other. It does this by intercepting updates to any of the reflected keys, and mirrors the changes intelligently to the other view
of the registry. The list of reflected keys is:
■
HKEY_LOCAL_MACHINE\Software\Classes
■
HKEY_LOCAL_MACHINE\Software\Ole
■
HKEY_LOCAL_MACHINE\Software\Rpc
■
HKEY_LOCAL_MACHINE\Software\COM3
■
HKEY_LOCAL_MACHINE\Software\EventSystem
Reflection of HKLM\Software\Classes\CLSID is intelligent; only LocalServer32 CLSIDs are
reflected because they run out of process, thus they can be COM-activated by 32-bit or 64-bit
applications. However, InProcServer32 CLSIDs are not reflected because 32-bit COM DLLs can’t
be loaded in a 64-bit process and likewise 64-bit COM DLLs can’t be loaded in a 32-bit process.
When reflecting a key/value, the registry reflector marks the key so that it understands that it
has been created by the reflector. This is to help the deletion case when deleting a key that has
been reflected; thus the reflector will be able to tell if it needs to delete the reflected key if it has
been written by the reflector.
I/O Control Requests
Besides normal read and write operations, applications can communicate with some device
drivers through device I/O control functions using the Windows DeviceIoControlFile API. The
application may specify an input and/or output buffer along with the call. If the buffer contains pointer-dependent data, and the process sending the control request is a Wow64 process, then the view of the input and/or output structure is different between the 32-bit
application and the 64-bit driver, since pointers are 4 bytes for 32-bit applications and 8 bytes
Copyrighted material.
182
Microsoft Windows Internals, Fourth Edition
for 64-bit applications. In this case, the kernel driver is expected to convert the associated
pointer-dependent structures. Drivers can call the IoIs32bitProcess function to detect if an I/O
request originated from a Wow64 process or not.
16-Bit Installer Applications
Wow64 doesn’t support running 16-bit applications. However, since many application installers are 16-bit programs, Wow64 has special case code to make references to certain well
known 16-bit installers work. These installers include:
■
Microsoft ACME Setup version: 2.6, 3.0, 3.01, and 3.1.
■
InstallShield version 5.x (where x is any minor version number).
Whenever a 16-bit process is about to be created using CreateProcess() API, Ntvdm64.dll is
loaded and control is transferred to it to inspect whether the 16-bit executable is one of the
supported installers. If it is, another CreateProcess is issued to launch a 32-bit version of the
installer with the same command line arguments.
Printing
32-bit printer drivers cannot be used on 64-bit Windows. Print drivers must be ported to
native 64-bit versions. However, since printer drivers run in the user mode address space of
the requesting process, and since only native 64-bit printer drivers are supported on 64-bit
Windows, a special mechanism is needed to support printing from 32-bit processes. This is
done by redirecting all printing functions to Splwow64.exe, the Wow64 RPC print server.
Since Splwow64 is a 64-bit process, it can load 64-bit printer drivers.
Restrictions
Wow64 does not support the execution of 16-bit applications (this is supported on 32-bit versions of Windows) or the loading of 32-bit kernel mode device drivers (they must be ported
to native 64-bits). Wow64 processes can only load 32-bit DLLs and can’t load native 64-bit
Dlls. Likewise, native 64-bit processes can’t load 32-bit DLLs.
In addition to the above, due to page size differences, Wow64 on IA-64 systems does not support the ReadFileScatter, WriteFileGather, GetWriteWatch, or Address Window Extension
(AWE) functions. Also, hardware acceleration through DirectX is not available (software emulation is provided for Wow64 processes).
Conclusion
In this chapter, we’ve examined the key base system mechanisms on which the Windows
executive is built. In the next chapter, we’ll look at three important mechanisms involved with
the management infrastructure of Windows: the registry, services, and Windows Management
Instrumentations (WMI).
Copyrighted material.
Chapter 4
Management Mechanisms
This chapter describes three fundamental mechanisms in Microsoft Windows that are critical
to the management and configuration of the system:
■
The registry
■
Services
■
Windows Management Instrumentation
The Registry
The registry plays a key role in the configuration and control of Windows systems. It is the
repository for both systemwide and per-user settings. Although most people think of the registry as static data stored on the hard disk, as you’ll see in this section, the registry is also a
window into various in-memory structures maintained by the Windows executive and kernel.
This section isn’t meant to be a complete reference to the contents of the Windows registry.
That kind of in-depth information is documented in the “Technical Reference to the Windows
2000 Registry” help file in the Windows 2000 resource kits (Regentry.chm), and for Windows
XP and Windows Server 2003 that information can be found online as part of the Windows
Server 2003 Deployment Kit at http://www.microsoft.com/windowsserver2003/techinfo/reskit/
deploykit.mspx.
We’ll start by providing you with an overview of the registry structure, a discussion of the data
types it supports, and a brief tour of the key information Windows maintains in the registry.
Then we’ll look inside the internals of the configuration manager, the executive component
responsible for implementing the registry database. Among the topics we’ll cover are the internal on-disk structure of the registry, how Windows retrieves configuration information when
an application requests it, and what measures are employed to protect this critical system
database.
Viewing and Changing the Registry
In general, you should never have to edit the registry directly: application and system settings
stored in the registry that might require manual changes should have a corresponding user
interface to control their modification. However, as you’ve already seen a number of times in
Copyrighted material.
183
184
Microsoft Windows Internals, Fourth Edition
this book, some advanced and debug settings have no editing user interface. Therefore, a
number of tools are included with Windows that enable you to view and modify the registry.
Windows 2000 comes with two tools for editing the registry—Regedit.exe and Regedt32.exe—
whereas Windows XP and Windows Server 2003 have only Regedit.exe. The reason is that the
Windows 2000 version of Regedit, which has flexible searching, importing, and exporting
capabilities, was ported from Windows 98 and therefore does not support editing or viewing
registry security or registry data types not defined on Windows 98. Windows 2000 includes
Regedt32 because although it doesn’t have as powerful a search feature or support importing
and exporting, it was written to run only on Windows 2000 and so it supports security and
Windows 2000–specific data types. The Regedit included with Windows XP and Windows
Server 2003 includes security editing and knowledge of all registry data types, and thus obviates the need for Regedt32.
There are also a number of command-line registry tools. Reg.exe, for instance, which is
included in Windows XP and Windows Server 2003 and available in the Windows 2000 Support Tools, has the ability to import, export, back up, and restore keys, as well as to compare,
modify, and delete keys and values.
Registry Usage
There are three principal times that configuration data is read:
■
During the boot process, the system reads settings that specify what device drivers to
load and how various subsystems—such as the memory manager and process manager—
configure themselves and tune system behavior.
■
During login, Explorer and other Windows components read per-user preferences from
the registry, including network drive-letter mappings, desktop wallpaper, screen saver,
menu behavior, and icon placement.
■
During their startup, applications read systemwide settings, such as a list of optionally
installed components and licensing data, as well as per-user settings that might include
menu and toolbar placement and a list of most-recently accessed documents.
However, the registry can be read at other times as well, such as in response to a modification
of a registry value or key. Some applications monitor their configuration settings in the registry and read updated settings when they see a change. In general, however, on an idle system
there should be no registry activity.
The registry is commonly modified in the following cases:
■
Although not a modification, the registry’s initial structure and many default settings are
defined by a prototype version of the registry that ships on the Windows setup media
that is copied onto a new installation.
■
Application setup utilities create default application settings and settings that reflect
installation configuration choices.
Copyrighted material.
Chapter 4:
Management Mechanisms
185
■
During the installation of a device driver, the Plug and Play system creates settings in the
registry that tell the I/O manager how to start the driver and creates other settings that
configure the driver’s operation. (See Chapter 9 for more information on how device
drivers are installed.)
■
When you change application or system settings through user interfaces, the changes
are often stored in the registry.
Note
Sadly, some applications poll the registry looking for changes when they should be
using the registry’s RegNotifyChangeKey function, which puts a thread to sleep until a change
occurs to the area of the registry in which they’re interested.
Registry Data Types
The registry is a database whose structure is similar to that of a disk volume. The registry contains keys, which are similar to a disk’s directories, and values, which are comparable to files on
a disk. A key is a container that can consist of other keys (subkeys) or values. Values, on the
other hand, store data. Top-level keys are root keys. Throughout this section, we’ll use the
words subkey and key interchangeably. (Only root keys are not subkeys.)
Both keys and values borrow their naming convention from the file system. Thus, you can
uniquely identify a value with the name mark, which is stored in a key called trade, with the
name trade\mark. One exception to this naming scheme is each key’s unnamed value. The
two Registry Editor utilities, Regedit and Regedt32, display these values differently: Regedit
displays the unnamed value as (Default); Regedt32 uses <No Name>.
Values store different kinds of data and can be one of the 15 types listed in Table 4-1. The majority
of registry values are REG_DWORD, REG_BINARY, or REG_SZ. Values of type REG_DWORD
can store numbers or Booleans (on/off values); REG_BINARY values can store numbers larger
than 32 bits or raw data such as encrypted passwords; REG_SZ values store strings (Unicode, of
course) that can represent elements such as names, filenames, paths, and types.
Table 4-1
Registry Value Types
Value Type
Description
REG_NONE
No value type.
REG_SZ
Fixed-length Unicode string.
REG_EXPAND_SZ
Variable-length Unicode string that can have embedded environment variables.
REG_BINARY
Arbitrary-length binary data.
REG_DWORD
32-bit number.
REG_DWORD_LITTLE_ENDIAN
32-bit number, with low byte first. This is equivalent
to REG_DWORD.
REG_DWORD_BIG_ENDIAN
32-bit number, with high byte first.
REG_LINK
Unicode symbolic link.
Copyrighted material.
186
Microsoft Windows Internals, Fourth Edition
Table 4-1 Registry Value Types
Value Type
Description
REG_MULTI_SZ
Array of Unicode NULL-terminated strings.
REG_RESOURCE_LIST
Hardware resource description.
REG_FULL_RESOURCE_DESCRIPTOR
Hardware resource description.
REG_RESOURCE_REQUIREMENTS_LIST
Resource requirements.
REG_QWORD
64-bit number.
REG_QWORD_LITTLE_ENDIAN
64-bit number, with low byte first. This is equivalent
to REG_QWORD.
REG_QWORD_BIG_ENDIAN
64-bit number, with high byte first.
The REG_LINK type is particularly interesting because it lets a key transparently point to
another key or value. When you traverse the registry through a link, the path searching continues at the target of the link. For example, if \Root1\Link has a REG_LINK value of
\Root2\RegKey, and RegKey contains the value RegValue, two paths identify RegValue:
\Root1\Link\RegValue and \Root2\RegKey\RegValue. As explained in the next section,
Windows prominently uses registry links: three of the six registry root keys are links to subkeys within the three nonlink root keys. Links aren’t saved; they must be dynamically created
after each reboot.
Registry Logical Structure
You can chart the organization of the registry via the data stored within it. There are six root
keys (and you can’t add new root keys or delete existing ones) that store information, as
shown in Table 4-2.
Table 4-2 The Six Root Keys
Root Key
Description
HKEY_CURRENT_USER
Stores data associated with the currently logged-on user
HKEY_USERS
Stores information about all the accounts on the machine
HKEY_CLASSES_ROOT
Stores file association and Component Object Model (COM)
object registration information
HKEY_LOCAL_MACHINE
Stores system-related information
HKEY_PERFORMANCE_DATA
Stores performance information
HKEY_CURRENT_CONFIG
Stores some information about the current hardware profile
Why do root-key names begin with an H? Because the root-key names represent Windows
handles (H) to keys (KEY). As mentioned in Chapter 1, HKLM is an abbreviation used for
HKEY_LOCAL_MACHINE. Table 4-3 lists all the root keys and their abbreviations. The following sections explain in detail the contents and purpose of each of these six root keys.
Again, see the “Technical Reference to the Windows 2000 Registry” help file in the Windows
2000 resource kits or the registry section of the Windows Server 2003 Deployment Kit for
details on the contents of these keys.
Copyrighted material.
Chapter 4:
Table 4-3
Management Mechanisms
187
Registry Root Keys
AbbreviaDescription
tion
Root Key
Link
HKEY_CURRENT_
USER
HKCU
Points to the user profile
of the currently logged-on
user
HKEY_USERS
HKU
Contains subkeys for all loaded Not a link
user profiles
HKEY_CLASSES_
ROOT
HKCR
Contains file association
and COM registration
information
HKLM\SOFTWARE\Classes
HKEY_LOCAL_
MACHINE
HKLM
Placeholder—contains other
keys
Not a link
HKEY_CURRENT_
CONFIG
HKCC
Current hardware profile
HKLM\SYSTEM\CurrentControlSet\Hardware Profiles\
Current
HKEY_PERFORMANCE_DATA
HKPD
Performance counters
Not a link
Subkey under HKEY_USERS
corresponding to currently
logged-on user
HKEY_CURRENT_USER
The HKCU root key contains data regarding the preferences and software configuration of the
locally logged-on user. It points to the currently logged-on user’s user profile, located on the
hard disk at \Documents and Settings\<username>\Ntuser.dat. (See the section “Registry
Internals” later in this chapter to find out how root keys are mapped to files on the hard disk.)
Whenever a user profile is loaded (such as at logon time or when a service process runs under
the context of a specific username), HKCU is created as a link to the user’s key under
HKEY_USERS. Table 4-4 lists some of the subkeys under HKCU.
Table 4-4
HKEY_CURRENT_USER Subkeys
Subkey
Description
AppEvents
Sound/event associations
Console
Command window settings (for example, width, height, and
colors)
Control Panel
Screen saver, desktop scheme, keyboard, and mouse settings
as well as accessibility and regional settings
Environment
Environment variable definitions
Keyboard Layout
Keyboard layout setting (for example, U.S. or U.K.)
Network
Network drive mappings and settings
Printers
Printer connection settings
Software
User-specific software preferences
UNICODE Program Groups
User-specific start menu group definitions
Windows 3.1 Migration Status
File status data for systems that upgrade from Windows 3.x to
Windows 2000 and higher
Copyrighted material.
188
Microsoft Windows Internals, Fourth Edition
HKEY_USERS
HKU contains a subkey for each loaded user profile and user class registration database on
the system. It also contains a subkey named HKU\.DEFAULT that is linked to the profile for
the system (which is used by processes running under the local system account and is
described in more detail in the section “Services” later in this chapter). This is the profile used
by Winlogon, for example, so that changes to the desktop background settings in that profile
will be implemented on the logon screen. When a user logs on to a system for the first time
and her account does not depend on a roaming domain profile (that is, the user’s profile is
obtained from a central network location at the direction of a domain controller), the system
creates a profile for her account that’s based on the profile stored in C:\Documents and Settings\Default User.
The location under which the system stores profiles is defined by the registry value
HKLM\Software\Microsoft\Windows NT\CurrentVersion\ProfileList\ProfilesDirectory,
which is by default set to %SystemDrive%\Documents and Settings. The ProfileList key also
stores the list of profiles present on a system. Information for each profile resides under a subkey that has a name reflecting the Security Identifier (SID) of the account to which the profile
corresponds. (See Chapter 8 for more information on SIDs.) Data stored in a profile’s key
includes the time of the last load of the profile in the ProfileLoadTimeLow and ProfileLoadTimeHigh values, the binary representation of the account SID in the Sid value, and the path to the
profile’s on-disk hive (which is described later in this chapter in the “Hives” section) in the
ProfileImagePath directory. Windows XP and Windows Server 2003 show the list of profiles
stored on a system in the User Profiles management dialog box, shown in Figure 4-1, that you
access by clicking Settings in the User Profiles section of the Advanced Tab on the System
Control Panel applet.
Figure 4-1
Copyrighted material.
The User Profiles management dialog box
Chapter 4:
Management Mechanisms
189
EXPERIMENT: Watching Profile Loading and Unloading
You can see a profile load into the registry and then unload by using the Runas command to launch a process in an account that’s not currently logged on to the machine.
While the new process is running, run Regedit and note the loaded profile key under
HKEY_USERS. After terminating the process, perform a refresh in Regedit by pressing
the F5 key and the profile should no longer be present.
HKEY_CLASSES_ROOT
HKCR consists of two types of information: file extension associations and COM class registrations. A key exists for every registered filename extension. Most keys contain a REG_SZ
value that points to another key in HKCR containing the association information for the class
of files that extension represents. For example, HKCR\.xls would point to information on
Microsoft Excel files in a key such as HKCU\Excel.Sheet.8. Other keys contain configuration
details for COM objects registered on the system.
The data under HKEY_CLASSES_ROOT comes from two sources:
■
The per-user class registration data in HKCU\SOFTWARE\Classes (mapped to the file
on hard disk \Documents and Settings\<username>\Local Settings\Application
Data\Microsoft\Windows\Usrclass.dat)
■
Systemwide class registration data in HKLM\SOFTWARE\Classes
The reason that there is a separation of per-user registration data from systemwide registration
data is so that roaming profiles can contain these customizations. It also closes a security hole:
a nonprivileged user cannot change or delete keys in the systemwide version
HKEY_CLASSES_ROOT, and thus cannot affect the operation of applications on the system.
Nonprivileged users and applications can read systemwide data and can add new keys and
values to systemwide data (which are mirrored in their per-user data), but they can modify
existing keys and values in their private data only.
HKEY_LOCAL_MACHINE
HKLM is the root key that contains all the systemwide configuration subkeys: HARDWARE,
SAM, SECURITY, SOFTWARE, and SYSTEM.
The HKLM\HARDWARE subkey maintains descriptions of the system’s hardware and all
hardware device-to-driver mappings. The Device Manager tool (which is available by running
System from Control Panel, clicking the Hardware tab, and then clicking Device Manager) lets
you view registry hardware information that it obtains by simply reading values out of the
HARDWARE key.
Copyrighted material.
190
Microsoft Windows Internals, Fourth Edition
EXPERIMENT: Fun with the Hardware Key
You can fool your coworkers or friends into thinking that you have the latest and greatest
processor by modifying the value of the ProcessorNameString value under HKLM\HARDWARE\DESCRIPTION\System\CentralProcessor\0. The System applet of the control
panel displays the ProcessorNameString value on the General page. Changes you make to
other values in that key, such as the ~MHz, do not have any affect on what the System
applet displays, however, because the system caches many of the values for use by functions that applications use to query the system’s processor capabilities.
HKLM\SAM holds local account and group information, such as user passwords, group definitions, and domain associations. Windows Server systems that are operating as domain controllers store domain accounts and groups in Active Directory, a database that stores
domainwide settings and information. (Active Directory isn’t described in this book.) By
default, the security descriptor on the SAM key is configured so that even the administrator
account doesn’t have access.
HKLM\SECURITY stores systemwide security policies and user-rights assignments.
HKLM\SAM is linked into the SECURITY subkey under HKLM\SECURITY\SAM. By default,
you can’t view the contents of HKLM\SECURITY or HKLM\SAM\SAM because the security
settings of those keys allow access only by the system account. (System accounts are discussed in greater detail later in this chapter.) You can change the security descriptor to allow
read access to administrators, or you can use PsExec to run Regedit in the local system
account (as shown in the related experiment for how to do that) if you want to peer inside.
However, that glimpse won’t be very revealing because the data is undocumented and the
passwords are encrypted with one-way mapping—that is, you can’t determine a password
from its encrypted form.
HKLM\SOFTWARE is where Windows stores systemwide configuration information not
needed to boot the system. Also, third-party applications store their systemwide settings here,
such as paths to application files and directories, and licensing and expiration date information.
HKLM\SYSTEM contains the systemwide configuration information needed to boot the system, such as which device drivers to load and which services to start. Because this information
is critical to starting the system, Windows also maintains a copy of part of this information,
called the last known good control set, under this key. The maintenance of a copy allows an administrator to select a previously working control set in the case that configuration changes made to
the current control set prevent the system from booting. For details on when Windows declares
the current control set “good,” see the section “Accepting the Boot and Last Known Good.”
HKEY_CURRENT_CONFIG
HKEY_CURRENT_CONFIG is just a link to the current hardware profile, stored under
HKLM\SYSTEM\CurrentControlSet\Hardware Profiles\Current. Hardware profiles allow
Copyrighted material.
Chapter 4:
Management Mechanisms
191
the administrator to configure variations to the base system driver settings. Although the
underlying profile might change from boot to boot, applications can always reference the currently active profile through this key. Hardware profile management is managed through the
Hardware Profiles dialog box that you access by clicking Settings in the Hardware Profiles section on the Hardware page of the Control Panel’s System applet. During the boot process,
Ntldr will prompt you to specify which profile it should use if there is more than one.
HKEY_PERFORMANCE_DATA
The registry is the mechanism to access performance counter values on Windows, whether
those are from operating system components or server applications. One of the side benefits
of providing access to the performance counters via the registry is that remote performance
monitoring works “for free” because the registry is easily accessible remotely through the normal registry APIs.
You can access the registry performance counter information directly by opening a special key
named HKEY_PERFORMANCE_DATA and querying values beneath it. You won’t find this
key by looking in the Registry Editor; this key is available only programmatically through the
Windows registry functions, such as RegQueryValueEx. Performance information isn’t actually
stored in the registry; the registry functions use this key to locate the information from performance data providers.
You can also access performance counter information by using the Performance Data Helper
(PDH) functions available through the Performance Data Helper API (Pdh.dll). Figure 4-2
shows the components involved in accessing performance counter information.
Performancemonitoring
applications
Custom
application A
Custom
application B
Performance
tool
Pdh.dll
Programming
interfaces
RegQueryValueEx
High-performance provider interface
Advapi32.dll
System
performance
DLL
Figure 4-2
Copyrighted material.
Windows Management Instrumentation
Perf Lib
Registry DLL provider
Performance
extension
DLL
Registry performance counter architecture
HighHighperformance
Highperformance
data
provider
performance
data
provider
object
data
provider
object
object
192
Microsoft Windows Internals, Fourth Edition
Troubleshooting Registry Problems
Because the system and applications depend so heavily on configuration settings to guide
their behavior, system and application failures can result from changing registry data or security. When the system or an application fails to read settings that it assumes it will always be
able to access, it can misbehave by crashing, displaying error messages that hide the root
cause, or by not executing with limited functionality. It’s virtually impossible to know what
registry keys or values are misconfigured without understanding how the system or the application that’s failing is accessing the registry. In such situations, the Regmon utility from
www.sysinternals.com might provide the answer.
Regmon lets you monitor registry activity as it occurs. For each registry access, Regmon shows
you the process that performed the access and the time, type, and result of the access. This
information is useful for seeing how applications and the system rely on the registry, discovering where applications and the system store configuration settings and troubleshooting
problems related to applications having missing registry keys or values. Regmon includes
advanced filtering and highlighting so that you can zoom in on activity related to specific keys
or values, or to the activity of particular processes.
Regmon Internals
Regmon relies on a device driver that it extracts from its executable image at run time and
then starts. Its first execution requires that the account running it have the Load Driver privilege as well as the Debug privilege; subsequent executions in the same boot session require
only the Debug privilege because once loaded, the driver remains resident.
There are actually three drivers stored within the Regmon executable: one for use on
Windows 95, Windows 98, and Windows Millennium; one for Windows NT, Windows 2000,
and Windows XP; and another for use on Windows Server 2003. The reason that there is a
driver specific to Windows Server 2003 is that on Windows NT, Windows 2000, and Windows XP the only way for a driver to monitor all registry activity is through system-call hooking and because on Windows Server 2003 a driver can use the registry callback mechanism to
monitor registry activity. (Windows 95, Windows 98, and Windows Millennium support a
different registry monitoring mechanism.)
Recall from the “System Service Dispatching” section of Chapter 3 that system service function addresses are stored in a system service dispatch table in the kernel. A driver can hook a
system service by saving the address of a function from the array and replacing the array entry
with the address of its hook function. After performing theses steps, any invocations of the
hooked system service get diverted to the hooking driver’s function, which can examine or modify the parameters to the function and, optionally, execute the original system service function.
If it calls the original function, the driver can also examine the result of the operation and examine data the function returns, such as data associated with registry values. Figure 4-3 shows how
Regmon intercepts registry functions in kernel mode.
Copyrighted material.
Chapter 4:
System service call
5 The Regmon GUI
periodically obtains
monitored data from
the driver.
System service array
System service
dispatcher
2 Windows system
service dispatcher
looks up the system
service function
address, which
Regmon has replaced
with the address
of its hook function.
Figure 4-3
193
Regmon GUI
Application
1 Application
executes
registry-related
system service
call.
Management Mechanisms
User mode
Kernel mode
Regmon
driver
3 Regmon
calls its
system service
hook function.
Registry
System Service
4 Regmon
invokes the
original
function.
Regmon’s use of system service hooking
The registry callback mechanism was introduced in Windows XP; however, Regmon still uses
system call hooking when run on Windows XP because the callback mechanism on Windows
XP does not report all registry activity. When a driver uses the callback mechanism, it registers
a callback function with the configuration manager. The configuration manager executes the
driver’s callback functions at certain points during the execution of registry system services so
that the driver has full visibility and control over registry accesses. Antivirus products that
scan registry data for viruses or prevent unauthorized processes from modifying the registry
are other users of the callback mechanism.
EXPERIMENT: Viewing Registry Activity on an Idle System
Because the registry implements the RegNotifyChangeKey function that applications can
use to request notification of registry changes without polling for them, when you Regmon on a system that’s idle you should not see repetitive accesses to the same registry
keys or values. Any such activity identifies a poorly written application that unnecessarily negatively affects a system’s overall performance.
Run Regmon, and after several seconds examine the output log to see whether you can
spot polling behavior. Right-click on an output line associated with polling, and choose
Process Properties from the context menu to view details about the process performing
the activity.
Copyrighted material.
194
Microsoft Windows Internals, Fourth Edition
EXPERIMENT: Using Regmon to Locate Application Registry
Settings
In some troubleshooting scenarios, you might need to determine where in the registry
the system or an application stores particular settings. This experiment has you use Regmon to discover the location of Notepad’s settings. Notepad, like most Windows applications, saves user preferences—such as word-wrap mode, font and font size, and
window position—across executions. By having Regmon watching when Notepad reads
or writes its settings, you can identify the registry key in which the settings are stored.
Here are the steps for doing this:
1. Have Notepad save a setting that you can easily search for in a Regmon trace. You
can do this by running Notepad, setting the font to Times New Roman, and then
exiting Notepad.
2. Run Regmon. Open the highlighting filter dialog box and enter notepad.exe in
the Include filter. This will have Regmon log only activity that has notepad.exe in
either the Process or Path columns.
3. Run Notepad again, and after it has launched stop Regmon’s event capture by toggling Capture Events in the Regmon File menu.
4. Scroll to the top line of the resultant log and select it.
5. Press Ctrl+F to open a Find dialog box, and search for times new. Regmon should
highlight a line like the one shown in the following graphic that represents Notepad reading the font value from the Registry. Other operations in the immediate
vicinity should relate to other Notepad settings.
6. Finally, double-click the highlighted line. Regmon will execute Regedit (if it’s not
already running) and cause it to navigate to and select the Notepad referenced registry value.
Copyrighted material.
Chapter 4:
Management Mechanisms
195
Regmon Troubleshooting Techniques
Two basic Regmon troubleshooting techniques are effective for discovering the cause of registry-related application or system problems:
■
Look at the last thing in the Regmon trace that the application did before it failed. This
action might point to the problem.
■
Compare a Regmon trace of the failing application with a trace from a working system.
To follow the first approach, run Regmon and then run the application. At the point the failure
occurs, go back to Regmon and stop the logging (by pressing Ctrl+E). Then go to the end of
the log and find the last operations performed by the application before it failed (or crashed,
hung, or whatever). Starting with the last line, work your way backward, examining the files,
registry keys, or both that were referenced—often this will help pinpoint the problem.
Use the second approach when the application fails on one system but works on another.
Capture a Regmon trace of the application on the working and failing systems, and save the
output to a log file. Then open the good and bad log files with Microsoft Excel (accepting the
defaults on the Import wizard), and delete the first three columns. (If you don’t delete the first
three columns, the comparison will show every line as different because the first three columns contain information that is different from run to run, such as the time and the process
ID.) Finally, compare the resulting log files. (You can do this by using WinDiff, which on Windows XP is included in the free support tools on the Windows XP CD, and for Windows 2000
it is included in the Resource Kit.)
Entries in a Regmon trace that have values of “NOTFOUND” or “ACCESS DENIED” in the
Result column are ones that you should investigate. NOTFOUND is reported when an application attempts to read from a registry key or value that doesn’t exist. In many cases, a missing
key or value is innocuous because a process that fails to read a setting from the registry simply
falls back on default values. In some cases, however, applications expect to find values for
which there is no default and will fail if they are missing.
Access-denied errors are a common source of registry-related application failures and occur
when an application doesn’t have permission to access a key the way that it wants. Applications that do not validate registry operation results or perform proper error recovery will fail.
A common result string that might appear suspicious is BUFROVERFLOW. It does not indicate a buffer-overflow exploit in the application that receives it. Instead, it’s used by the configuration manager to inform an application that the buffer it specified to store a registry value
is too small to hold the value. Application developers often take advantage of this behavior to
determine how large a buffer to allocate to store a value. They first perform a registry query
with a 0-length buffer that returns a buffer-overflow error and the length of the data it
attempted to read. The application then allocates a buffer of the indicated size and rereads the
value. You should therefore see operations that return BUFROVERFLOW repeat with a successful result.
Copyrighted material.
196
Microsoft Windows Internals, Fourth Edition
In one example of Regmon being used to troubleshoot a real problem, it saved a user from
doing a complete reinstall of his Windows XP system. The symptom was that Internet
Explorer would hang on startup if the user did not first manually dial the Internet connection.
This Internet connection was set as the default connection for the system, so starting Internet
Explorer should have caused an automatic dial-up to the Internet (because Internet Explorer
was set to display a default home page upon startup).
An examination of a Regmon log of Internet Explorer startup activity, going backward from
the point in the log where Internet Explorer hung, showed a query to a key under
HKCU\Software\Microsoft\RAS Phonebook. The user reported that he had previously uninstalled the dialer program associated with the key and manually created the dial-up connection. Because the dial-up connection name did not match that of the uninstalled dialer
program, it appeared that the key had not been deleted by the dialer’s uninstall program and
that it was causing Internet Explorer to hang. After the key was deleted, Internet Explorer
functioned as expected.
Logging Activity in Unprivileged Accounts or During Logon/Logoff
A common application-failure scenario is that an application works when run in an account
that has Administrative group membership but not when run in the account of an unprivileged user. As described earlier, executing Regmon requires security privileges that are not
normally assigned to standard user accounts, but you can capture a trace of applications executing in the logon session of an unprivileged user by using the Runas command to execute
Regmon in an administrative account.
If a registry problem relates to account logon or logoff, you’ll also have to take special steps to be
able to use Regmon to capture a trace of those phases of a logon session. Applications that are run
in the local system account are not terminated when a user logs off, and you can take advantage
of that fact to have Regmon run through a logoff and subsequent logon. You can launch Regmon
in the local system account either by using the At command that’s built into Windows and specifying the /interactive flag, or by using the PsExec utility from www.sysinternals.com, like this:
psexec –i –s –d c:\regmon.exe
The -i switch directs PsExec to have Regmon's window appear on the interactive console, the
-s switch has PsExec run Regmon in the local system account, and the -d switch has PsExec
launch Regmon and exit without waiting for Regmon to terminate. When you execute this
command, the instance of Regmon that executes will survive logoff and reappear on the desktop when you log back on, having captured the registry activity of both actions.
Another way to monitor registry activity during the logon, logoff, boot, or shut down process
is to use the Regmon log boot feature, which you can enable by selecting Log Boot in the
Options menu. The next time you boot the system, the Regmon device driver logs registry
activity from early in the boot to the \Windows\Regmon.log. It will continue logging to that
file until disk space runs out, the system shuts down, or you run Regmon. A log file storing a
registry trace of startup, logon, logoff, and shut down on a Windows XP system will typically
be between 50 and 150 MB in size.
Copyrighted material.
Chapter 4:
Management Mechanisms
197
Registry Internals
In this section, you’ll find out how the configuration manager—the executive subsystem that
implements the registry—organizes the registry’s on-disk files. We’ll examine how the configuration manager manages the registry as applications and other operating system components read and change registry keys and values. We’ll also discuss the mechanisms by which
the configuration manager tries to ensure that the registry is always in a recoverable state,
even if the system crashes while the registry is being modified.
Hives
On disk, the registry isn’t simply one large file but rather a set of discrete files called hives. Each
hive contains a registry tree, which has a key that serves as the root or starting point of the tree.
Subkeys and their values reside beneath the root. You might think that the root keys displayed
by the Registry Editor tools correlate to the root keys in the hives, but such is not the case. Table
4-5 lists registry hives and their on-disk filenames. The pathnames of all hives except for user
profiles are coded into the configuration manager. As the configuration manager loads hives,
including system profiles, it notes each hive’s path in the values under the HKLM\SYSTEM\CurrentControlSet\Control\hivelist subkey, removing the path if the hive is unloaded.
(User profiles are unloaded when not referenced.) It creates the root keys, linking these hives
together to build the registry structure you’re familiar with and that the Registry Editor displays.
Table 4-5
On-Disk Files Corresponding to Paths in the Registry
Hive Registry Path
Hive File Path
HKEY_LOCAL_MACHINE\SYSTEM
\Windows\System32\Config\System
HKEY_LOCAL_MACHINE\SAM
\Windows\System32\Config\Sam
HKEY_LOCAL_MACHINE\SECURITY
\Windows\System32\Config\Security
HKEY_LOCAL_MACHINE\SOFTWARE
\Windows\System32\Config\Software
HKEY_LOCAL_MACHINE\HARDWARE
Volatile hive
HKEY_LOCAL_MACHINE\SYSTEM\Clone
Volatile hive (on Windows 2000 only)
HKEY_USERS\<security ID of username>
\Documents and Settings\<username>\Ntuser.dat
HKEY_USERS\<security ID of username>_Classes
\Documents and Settings\<username>\Local
Settings\Application Data\Microsoft\Windows\
Usrclass.dat
HKEY_USERS\.DEFAULT
\Windows\System32\Config\Default
You’ll notice that some of the hives listed in Table 4-5 are volatile and don’t have associated
files. The system creates and manages these hives entirely in memory; the hives are therefore
temporary. The system creates volatile hives every time it boots. An example of a volatile hive
is the HKLM\HARDWARE hive, which stores information about physical devices and the
devices’ assigned resources. Resource assignment and hardware detection occur every time
the system boots, so not storing this data on disk is logical.
Copyrighted material.
198
Microsoft Windows Internals, Fourth Edition
EXPERIMENT: Manually Loading and Unloading Hives
Regedt32 on Windows 2000 and Regedit on Windows XP and Windows Server 2003
have the ability to load hives that you can access through its File menu. This capability
can be useful in troubleshooting scenarios where you want to view or edit a hive from an
unbootable system or a backup medium. In this experiment, you’ll use Regedt32 (if
you’re running Windows 2000) or Regedit (if you’re running Windows XP and Windows Server 2003) to load a version of the HKLM\SYSTEM hive that Windows Setup
creates and stores in \Windows\Repair during the install process.
1. Hives can be loaded only underneath HKLM or HKU, so open Regedit or
Regedt32, select HKLM, and choose Load Hive from the Regedit File menu or the
Regedt32 Registry menu.
2. Navigate to the \Windows\Repair directory in the Load Hive dialog box, select
System.bak, and open it. When prompted, enter Test as the name of the key under
which it will load.
3. Open the newly created HKLM\Test key, and explore the contents of the hive.
4. Open HKLM\System\CurrentControlSet\Control\Hivelist, and locate the entry
\Registry\Machine\Test, which demonstrates how the configuration manager
lists loaded hives in the HiveList key.
5. Select HKLM\Test, and choose Unload Hive from the Regedit File menu or the
Regedt32 Registry menu to unload the hive.
Hive Size Limits
In some cases, hive sizes are limited. For example, Windows places a limit on the size of the
HKLM\SYSTEM hive. It does so because Ntldr reads the entire HKLM\SYSTEM hive into
physical memory near the start of the boot process when virtual memory paging is not
enabled. Ntldr also loads Ntoskrnl and boot device drivers into physical memory, so it must
constrain the amount of physical memory assigned to HKLM\SYSTEM. (See Chapter 6 for
more information on the role Ntldr plays during the startup process.) On Windows 2000,
Ntldr places a fixed upper limit on its size of 12 MB, but on Windows XP and Windows Server
2003 it is more flexible, allowing the hive to be up to 200 MB or one fourth the amount of
physical memory on the system, whichever is lower.
On Windows 2000, there is also a limit on the combined sizes of all loaded registry hives.
Windows 2000 uses a type of kernel memory called paged pool to hold registry hives in memory, and therefore, the total amount of loaded registry data is constrained by the amount of
paged pool that’s available. The amount of paged pool the memory manager creates during its
initialization is based on a number of factors, such as the amount of physical memory on the
system. On a system where the memory manager creates the largest amount of paged pool
Copyrighted material.
Chapter 4:
Management Mechanisms
199
possible, the registry size limit is 376 MB. Because a system will not operate smoothly if there
is not enough paged pool left over for other uses, Windows 2000 won’t let registry data grow
to more than 80 percent of paged pool and also honors a user-configurable registry quota if
it’s less than that amount. Click the Change button in the Virtual Memory section of the Performance Options dialog box that you reach on the Advanced page of the Control Panel’s System applet to view or modify the registry quota setting, which you can see in Figure 4-4.
Figure 4-4
Windows 2000 registry quota setting
The upper limit on the total size of loaded registry hives can create a limit on the number of
concurrently logged-in users on a Windows 2000 system running Terminal Services, because
each user’s profile contributes to the loaded hive size. On Windows XP and Windows Server
2003, the configuration manager therefore does not use paged pool and instead relies on the
memory manager’s memory-mapping functions to map into system memory only the portions of registry hives that it’s accessing at any given point in time. There is no registry quota
on Windows XP or Windows Server 2003, and the total size of loaded hives does not constrain the scalability of Terminal Services.
EXPERIMENT: Looking at Hive Handles
The configuration manager opens hives by using the kernel handle table (described in
Chapter 3) so that it can access hives from any process context. Using the kernel handle
table is an efficient alternative to approaches that involve using drivers or executive components to access from the system process only handles that must be protected from
user processes. You can use the Process Explorer utility, available from www.sysinternals.com, to see the hive handles. On Windows 2000, the object manager reports kernel
Copyrighted material.
200
Microsoft Windows Internals, Fourth Edition
handle table handles as being opened in the System Idle process, and on Windows XP
and Windows Server 2003 it reports them as being opened in the System process. Select
the appropriate process for the Windows version that you are running, and select Handles from the Lower Pane View menu entry in the View menu. Sort by handle type, and
scroll until you see the hive files, as shown in the following graphic.
A special type of key known as a symbolic link makes it possible for the configuration manager
to link hives to organize the registry. A symbolic link is a key that redirects the configuration
manager to another key. Thus, the key HKLM\SAM is a symbolic link to the key at the root of
the SAM hive.
Hive Structure
The configuration manager logically divides a hive into allocation units called blocks in much
the same way that a file system divides a disk into clusters. By definition, the registry block
size is 4096 bytes (4 KB). When new data expands a hive, the hive always expands in blockgranular increments. The first block of a hive is the base block. The base block includes global
information about the hive, including a signature—regf—that identifies the file as a hive,
updated sequence numbers, a time stamp that shows the last time a write operation was initiated on the hive, the hive format version number, a checksum, and the hive file’s internal filename (for example, \Device\HarddiskVolume1\WINDOWS\SYSTEM32\CONFIG\SAM).
We’ll clarify the significance of the updated sequence numbers and time stamp when we
describe how data is written to a hive file. The hive format version number specifies the data
format within the hive. The configuration manager uses hive format version 1.3 on Windows
2000. On Windows XP and Windows Server 2003, it uses format version 1.3 for all hives
except for System and Software for roaming profile compatibility with Windows 2000. For
System and Software hives, it uses version 1.5 because of the new format’s optimizations for
large values and searching.
Windows organizes the registry data that a hive stores in containers called cells. A cell can hold
a key, a value, a security descriptor, a list of subkeys, or a list of key values. A field at the
Copyrighted material.
Chapter 4:
Management Mechanisms
201
beginning of a cell’s data describes the data’s type. Table 4-6 describes each cell data type in
detail. A cell’s header is a field that specifies the cell’s size. When a cell joins a hive and the
hive must expand to contain the cell, the system creates an allocation unit called a bin. A bin
is the size of the new cell rounded up to the next block boundary. The system considers any
space between the end of the cell and the end of the bin to be free space that it can allocate to
other cells. Bins also have headers that contain a signature, hbin, and a field that records the
offset into the hive file of the bin and the bin’s size.
Table 4-6
Cell Data Types
Data Type
Description
Key cell
A cell that contains a registry key, also called a key node. A key cell
contains a signature (kn for a key, kl for a symbolic link), the time
stamp of the most recent update to the key, the cell index of the
key’s parent key cell, the cell index of the subkey-list cell that identifies the key’s subkeys, a cell index for the key’s security descriptor
cell, a cell index for a string key that specifies the class name of the
key, and the name of the key (for example, CurrentControlSet).
Value cell
A cell that contains information about a key’s value. This cell includes a signature (kv), the value’s type (for example, REG_ DWORD
or REG_BINARY), and the value’s name (for example, Boot-Execute).
A value cell also contains the cell index of the cell that contains the
value’s data.
Subkey-list cell
A cell composed of a list of cell indexes for key cells that are all subkeys of a common parent key.
Value-list cell
A cell composed of a list of cell indexes for value cells that are all values of a common parent key.
Security-descriptor cell
A cell that contains a security descriptor. Security-descriptor cells include a signature (ks) at the head of the cell and a reference count
that records the number of key nodes that share the security descriptor. Multiple key cells can share security-descriptor cells.
By using bins, instead of cells, to track active parts of the registry, Windows minimizes some
management chores. For example, the system usually allocates and deallocates bins less frequently than it does cells, which lets the configuration manager manage memory more efficiently. When the configuration manager reads a registry hive into memory, it can choose to
read only bins that contain cells (that is, active bins) and to ignore empty bins. When the system adds and deletes cells in a hive, the hive can contain empty bins interspersed with active
bins. This situation is similar to disk fragmentation, which occurs when the system creates
and deletes files on the disk. When a bin becomes empty, the configuration manager joins to
the empty bin any adjacent empty bins to form as large a contiguous empty bin as possible.
The configuration manager also joins adjacent deleted cells to form larger free cells. (The configuration manager shrinks a hive only when bins at the end of the hive become free. You can
compact the registry by backing it up and restoring it using the Windows RegSaveKey and
RegReplaceKey functions, which are used by the Windows Backup utility.)
Copyrighted material.
202
Microsoft Windows Internals, Fourth Edition
The links that create the structure of a hive are called cell indexes. A cell index is the offset of a
cell into the hive file. Thus, a cell index is like a pointer from one cell to another cell that the
configuration manager interprets relative to the start of a hive. For example, as you saw in
Table 4-6, a cell that describes a key contains a field specifying the cell index of its parent key;
a cell index for a subkey specifies the cell that describes the subkeys that are subordinate to
the specified subkey. A subkey-list cell contains a list of cell indexes that refer to the subkey’s
key cells. Therefore, if you want to locate, for example, the key cell of subkey A, whose parent
is key B, you must first locate the cell containing key B’s subkey list using the subkey-list cell
index in key B’s cell. Then you locate each of key B’s subkey cells by using the list of cell
indexes in the subkey-list cell. For each subkey cell, you check to see whether the subkey’s
name, which a key cell stores, matches the one you want to locate, in this case, subkey A.
The distinction between cells, bins, and blocks can be confusing, so let’s look at an example of
a simple registry hive layout to help clarify the differences. The sample registry hive file in Figure 4-5 contains a base block and two bins. The first bin is empty, and the second bin contains
several cells. Logically, the hive has only two keys: the root key Root, and a subkey of Root,
Sub Key. Root has two values, Val 1 and Val 2. A subkey-list cell locates the root key’s subkey,
and a value-list cell locates the root key’s values. The free spaces in the second bin are empty
cells. Figure 4-5 doesn’t show the security cells for the two keys, which would be present in a
hive.
Block boundaries
Base block
Empty bin
Root
Val 1
Sub
Val 2
Key
Bin 1
Key cell (key node)
Value cell
Value-list cell
Figure 4-5
Subkey-list cell
Free space
Bin 2
Internal structure of a registry hive
Figure 4-6 shows an example of the Disk Probe utility (Dskprobe.exe) examining the first bin
in a SYSTEM hive. Notice the bin’s signature, hbin, at the top right side of the image. Look
beneath the bin signature and you’ll see the signature nk. This signature is the signature of a
key cell (kn). The signature displays backward because of the way x86 computers store data.
The cell is the SYSTEM hive’s root cell, which the configuration manager has named internally
$$$PROTO.HIV, as specified by the name that follows the nk signature.
Copyrighted material.
Chapter 4:
Management Mechanisms
203
Bin signature
Key cell signature
Name
Figure 4-6
Binary contents of first bin in the SYSTEM hive
To optimize searches for both values and subkeys, the configuration manager sorts subkey-list
cells alphabetically. The configuration manager can then perform a binary search when it
looks for a subkey within a list of subkeys. The configuration manager examines the subkey in
the middle of the list, and if the name of the subkey the configuration manager is looking for
is alphabetically before the name of the middle subkey, the configuration manager knows that
the subkey is in the first half of the subkey list; otherwise, the subkey is in the second half of
the subkey list. This splitting process continues until the configuration manager locates the
subkey or finds no match. Value-list cells aren’t sorted, however, so new values are always
added to the end of the list.
Cell Maps
The configuration manager doesn’t access a hive’s image on disk every time a registry access
occurs. Windows 2000 keeps a version of every hive in the kernel’s address space. When a
hive initializes, the configuration manager determines the size of the hive file, allocates
enough memory from the kernel’s paged pool to store it, and reads the hive file into memory.
(For more information on paged pool, see Chapter 7.) Because all loaded registry hives are
read into paged pool, that registry data is typically the largest consumer of the paged pool in
Windows 2000. (To check paged pool allocation, use the Poolmon utility, described in the
“Experiment: Monitoring Pool Usage” sidebar in Chapter 7.)
In Windows XP and Windows Server 2003, the configuration manager maps portions of a hive
into memory as it needs to access them. It uses the cache manager’s file mapping functions to
map in 16-KB views into the hive files. (See Chapter 10 for more information on the cache
manager.) To prevent hive mapping from consuming all the cache manager’s address range, the
Copyrighted material.
204
Microsoft Windows Internals, Fourth Edition
configuration manager tries to keep no more than 256 views of a hive mapped at any given point
in time by unmapping least-recently used (LRU) views when it reaches that limit. The configuration manager still uses the paged pool to store various data structures (including the LRU list
of views), but its use of the paged pool is a fraction of what it is in Windows 2000.
Note
On Windows XP and Windows Server 2003, the configuration manager will store a
block in the paged pool instead of mapping it if the block exceeds 256 KB in size.
If hives never grew, the configuration manager could perform all its registry management on
the in-memory version of a hive as if the hive were a file. Given a cell index, the configuration
manager could calculate the location in memory of a cell simply by adding the cell index,
which is a hive file offset, to the base of the in-memory hive image. Early in the system boot,
this process is exactly what Ntldr does with the SYSTEM hive: Ntldr reads the entire SYSTEM
hive into memory as a read-only hive and adds the cell indexes to the base of the in-memory
hive image to locate cells. Unfortunately, hives grow as they take on new keys and values,
which means the system must allocate paged pool memory to store the new bins that contain
added keys and values. Thus, the paged pool that keeps the registry data in memory isn’t necessarily contiguous.
EXPERIMENT: Viewing Hive Paged Pool Usage
There are no administrative-level tools that show you the amount of paged pool that registry hives, including user profiles, are consuming on Windows 2000. However, the !reg
dumppool kernel debugger command shows you not only how many pages of the paged
pool each loaded hive consumes but also how many of the pages store volatile and nonvolatile data. The command prints the total hive memory usage at the end of the output.
(The command shows only the last 32 characters of a hive’s name.)
kd> !reg dumppool
dumping hive at e20d66a8 (a\Microsoft\Windows\UsrClass.dat)
Stable Length = 1000
1/1 pages present
Volatile Length = 0
dumping hive at e215ee88 (ettings\Administrator\ntuser.dat)
Stable Length = f2000
242/242 pages present
Volatile Length = 2000
2/2 pages present
dumping hive at e13fa188 (\SystemRoot\System32\Config\SAM)
Stable Length = 5000
5/5 pages present
Volatile Length = 0
...
Copyrighted material.
Chapter 4:
Management Mechanisms
205
EXPERIMENT: Viewing Hive Memory Usage
In Windows XP and Windows Server 2003, you can view statistics on hive memory
usage, including its stable (on-disk) size and nonvolatile size, the number of active
views, and the number of views that are locked into memory, using the !reg hivelist command (note that the line output wraps):
-----------------------------------------------------------------------------------------------------------| HiveAddr |Stable Length|Stable Map|Volatile Length|Volatile Map|MappedViews|PinnedVi
ews|U(Cnt)| BaseBlock | FileName
-----------------------------------------------------------------------------------------------------------| e22f8b68 |
5000 | e22f8bc4 |
1000
| e22f8ca0 |
2 |
0 |
0| e2353000 | \Microsoft
\Windows\UsrClass.dat
| e28c3008 |
3fe000 | e1e84000 |
c000
| e28c3140 |
116 |
0 |
0| e1e48000 | ttings\Adm
inistrator\ntuser.dat
| e23ec008 |
1000 | e23ec064 |
0
| 00000000 |
1 |
0 |
0| e23ee000 | \Microsoft
\Windows\UsrClass.dat
| e23ed760 |
37000 | e23ed7bc |
1000
| e23ed898 |
14 |
0 |
0| e23ef000 | ettings\Lo
calService\ntuser.dat
...
In the preceding output, the Administrator account’s profile hive (the full path of which,
\Documents and Settings\Administrator\ntuser.dat, is truncated in the output) has 116
mapped views and is approximately 4 MB in size (0x3f000 in decimal). The !reg viewlist
command will dump the mapped views of the hive you specify. Here’s the output of that
command when executed for the UsrClass.dat hive that was printed as the first hive of
the !reg hivelist command’s output:
kd> !reg viewlist e22f8b68
0
Pinned Views ; PinViewListHead = e22f8da0 e22f8da0
2 Mapped Views ; LRUViewListHead = e1cf4448 e1c5d440
------------------------------------------------------------------------------------------------------------| ViewAddr |FileOffset|
Size
|ViewAddress|
Bcb
|
LRUViewList
|
PinV
iewList
| UseCount |
------------------------------------------------------------------------------------------------------------| e1cf4448 |
0 |
4000 | c9a40000 | 8a4bb0e9 | e1c5d440 e22f8d98 | e1cf445
0 e1cf4450 |
0 |
| e1c5d440 |
4000 |
2000 | c9a44000 | 8a4bb0e9 | e22f8d98 e1cf4448 | e1c5d44
8 e1c5d448 |
0 |
-------------------------------------------------------------------------------------------------------------
Copyrighted material.
206
Microsoft Windows Internals, Fourth Edition
The output shows the addresses of the two views that the hivelist command reported for
the hive in the ViewAddress column. Using the debugger’s db command to dump the
contents of memory at the address of the first view reveals that it maps the base block of
the hive, recognizable with its regf signature:
kd> db c9a40000
c9a40000 72 65
c9a40010 3d 40
c9a40020 01 00
c9a40030 5c 00
c9a40040 66 00
c9a40050 77 00
c9a40060 61 00
c9a40070 00 00
67
c4
00
4d
74
73
73
00
66
01
00
00
00
00
00
00
d5
01
20
69
5c
5c
73
00
01
00
00
00
00
00
00
00
00
00
00
63
57
55
2e
00
00-d5
00-03
00-00
00-72
00-69
00-73
00-64
00-00
01
00
50
00
00
00
00
00
00
00
00
6f
6e
72
61
00
00
00
00
00
00
00
00
00
cc
00
01
73
64
43
74
00
20
00
00
00
00
00
00
00
43
00
00
6f
6f
6c
00
00
c7
00
00
00
00
00
00
00
regf......... C.
=@..............
.... ....P......
\.M.i.c.r.o.s.o.
f.t.\.W.i.n.d.o.
w.s.\.U.s.r.C.l.
a.s.s...d.a.t...
................
To deal with noncontiguous memory addresses referencing hive data in memory, the configuration manager adopts a strategy similar to what the Windows memory manager uses to
map virtual memory addresses to physical memory addresses. The configuration manager
employs a two-level scheme, which Figure 4-7 illustrates, that takes as input a cell index (that
is, a hive file offset) and returns as output both the address in memory of the block the cell
index resides in and the address in memory of the block the cell resides in. Remember that a
bin can contain one or more blocks and that hives grow in bins, so Windows always represents a bin with a contiguous region of memory. Therefore, all blocks within a bin occur
within the same cache manager view (in Windows XP and Windows Server 2003) or portion
of a paged pool (in Windows 2000).
Cell index
Directory index
Byte offset
Table index
32
0
Hive’s cell map
directory
0
Cell map table
Target block
0
Cell
1023
511
Hive cell map directory pointer
Figure 4-7
Structure of a cell index
To implement the mapping, the configuration manager divides a cell index logically into
fields, in the same way that the memory manager divides a virtual address into fields. Windows interprets a cell index’s first field as an index into a hive’s cell map directory. The cell
Copyrighted material.
Chapter 4:
Management Mechanisms
207
map directory contains 1024 entries, each of which refers to a cell map table that contains 512
map entries. An entry in this cell map table is specified by the second field in the cell index.
That entry locates the bin and block memory addresses of the cell. In Windows XP and Windows Server 2003, not all bins are necessarily mapped into memory, and if a cell lookup yields
an address of 0, the configuration manager maps the bin into memory, unmapping another
on the mapping LRU list it maintains, if necessary.
In the final step of the translation process, the configuration manager interprets the last field
of the cell index as an offset into the identified block to precisely locate a cell in memory.
When a hive initializes, the configuration manager dynamically creates the mapping tables,
designating a map entry for each block in the hive, and it adds and deletes tables from the cell
directory as the changing size of the hive requires.
The Registry Namespace and Operation
The configuration manager defines a key object object type to integrate the registry’s namespace
with the kernel’s general namespace. The configuration manager inserts a key object named Registry into the root of the Windows namespace, which serves as the entry point to the registry.
Regedit shows key names in the form HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet,
but the Windows subsystem translates such names into their object namespace form (for example, \Registry\Machine\System\CurrentControlSet). When the Windows object manager
parses this name, it encounters the key object by the name of Registry first and hands the rest of
the name to the configuration manager. The configuration manager takes over the name parsing, looking through its internal hive tree to find the desired key or value. Before we describe the
flow of control for a typical registry operation, we need to discuss key objects and key control
blocks. Whenever an application opens or creates a registry key, the object manager gives a handle with which to reference the key to the application. The handle corresponds to a key object
that the configuration manager allocates with the help of the object manager. By using the object
manager’s object support, the configuration manager takes advantage of the security and reference-counting functionality that the object manager provides.
For each open registry key, the configuration manager also allocates a key control block. A key
control block stores the full pathname of the key, includes the cell index of the key node that
the control block refers to, and contains a flag that notes whether the configuration manager
needs to delete the key cell that the key control block refers to when the last handle for the key
closes. Windows places all key control blocks into a hash table to enable quick searches for
existing key control blocks by name. A key object points to its corresponding key control
block, so if two applications open the same registry key, each will receive a key object, and
both key objects will point to a common key control block.
When an application opens an existing registry key, the flow of control starts with the application specifying the name of the key in a registry API that invokes the object manager’s nameparsing routine. The object manager, upon encountering the configuration manager’s registry
key object in the namespace, hands the pathname to the configuration manager. The configu-
Copyrighted material.
208
Microsoft Windows Internals, Fourth Edition
ration manager uses the in-memory hive data structures to search through keys and subkeys
to find the specified key. If the configuration manager finds the key cell, the configuration
manager searches the key control block tree to determine whether the key is open (by the
same or another application). The search routine is optimized to always start from the closest
ancestor with a key control block already opened. For example, if an application opens \Registry\Machine\Key1\Subkey2, and \Registry\Machine is already opened, the parse routine
uses the key control block of \Registry\Machine as a starting point. If the key is open, the configuration manager increments the existing key control block’s reference count. If the key isn’t
open, the configuration manager allocates a new key control block and inserts it into the tree.
Then the configuration manager allocates a key object, points the key object at the key control
block, and returns control to the object manager, which returns a handle to the application.
When an application creates a new registry key, the configuration manager first finds the key
cell for the new key’s parent. The configuration manager then searches the list of free cells for
the hive in which the new key will reside to determine whether cells exist that are large
enough to hold the new key cell. If there aren't any free cells large enough, the configuration
manager allocates a new bin and uses it for the cell, placing any space at the end of the bin on
the free cell list. The new key cell fills with pertinent information—including the key’s name—
and the configuration manager adds the key cell to the subkey list of the parent key’s subkeylist cell. Finally, the system stores the cell index of the parent cell in the new subkey’s key cell.
The configuration manager uses a key control block’s reference count to determine when to
delete the key control block. When all the handles that refer to a key in a key control block
close, the reference count becomes 0, which denotes that the key control block is no longer
necessary. If an application that calls an API to delete the key sets the delete flag, the configuration manager can delete the associated key from the key’s hive because it knows that no
application is keeping the key open.
EXPERIMENT: Viewing Key Control Blocks
You can use the kernel debugger to list all the key control blocks allocated on a system
with the command !reg openkeys. Alternatively, if you want to view the key control block
for a particular open key, use !reg findkcb:
kd> !reg findkcb \registry\machine\software\microsoft
Found KCB = e1034d40 :: \REGISTRY\MACHINE\SOFTWARE\MICROSOFT
You can then examine a reported key control block with the !reg kcb command:
kd> !reg kcb e1034d40
Key
RefCount
Flags
Copyrighted material.
: \REGISTRY\MACHINE\SOFTWARE\MICROSOFT
: 1f
: CompressedName, Stable
Chapter 4:
ExtFlags
:
Parent
:
KeyHive
:
KeyCell
:
TotalLevels
:
DelayedCloseIndex:
MaxNameLen
:
MaxValueNameLen :
MaxValueDataLen :
LastWriteTime
:
KeyBodyListHead :
SubKeyCount
:
ValueCache.Count :
KCBLock
:
KeyLock
:
Management Mechanisms
209
0xe1997368
0xe1c8a768
0x64e598 [cell index]
4
2048
0x3c
0x0
0x0
0x 1c42501:0x7eb6d470
0xe1034d70 0xe1034d70
137
0
0xe1034d40
0xe1034d40
The Flags field indicates that the name is stored in compressed form and the SubKeyCount field shows that the key has 137 subkeys.
Stable Storage
To make sure that a nonvolatile registry hive (one with an on-disk file) is always in a recoverable state, the configuration manager uses log hives. Each nonvolatile hive has an associated
log hive, which is a hidden file with the same base name as the hive and a .log extension. For
example, if you look in your \Windows\System32\Config directory (and you have the Show
Hidden Files And Folders folder option selected), you’ll see System.log, Sam.log, and other
.log files. When a hive initializes, the configuration manager allocates a bit array in which each
bit represents a 512-byte portion, or sector, of the hive. This array is called the dirty sector array
because an on bit in the array means that the system has modified the corresponding sector in
the hive in memory and must write the sector back to the hive file. (An off bit means that the
corresponding sector is up to date with the in-memory hive’s contents.)
When the creation of a new key or value or the modification of an existing key or value takes
place, the configuration manager notes the sectors of the hive that change in the hive’s dirty
sector array. Then the configuration manager schedules a lazy write operation, or a hive sync.
The hive lazy writer system thread wakes up 5 seconds after the request to synchronize the
hive and writes dirty hive sectors for all hives from memory to the hive files on disk. Thus, the
system flushes, at the same time, all the registry modifications that take place between the
time a hive sync is requested and the time the hive sync occurs. When a hive sync takes place,
the next hive sync will occur no sooner than 5 seconds later.
Note
On Windows Server 2003, you can change the default 5-second delay the hive lazy
writer thread uses up by setting the registry value HKLM\System\CurrentControlSet\Session
Manager\Configuration Manager\RegistryLazyFlushInterval.
Copyrighted material.
210
Microsoft Windows Internals, Fourth Edition
If the lazy writer simply wrote all a hive’s dirty sectors to the hive file and the system crashed
in midoperation, the hive file would be in an inconsistent (corrupted) and unrecoverable
state. To prevent such an occurrence, the lazy writer first dumps the hive’s dirty sector array
and all the dirty sectors to the hive’s log file, increasing the log file’s size if necessary. The lazy
writer then updates a sequence number in the hive’s base block and writes the dirty sectors to
the hive. When the lazy writer is finished, it updates a second sequence number in the base
block. Thus, if the system crashes during the write operations to the hive, at the next reboot
the configuration manager will notice that the two sequence numbers in the hive’s base block
don’t match. The configuration manager can update the hive with the dirty sectors in the
hive’s log file to roll the hive forward. The hive is then up to date and consistent.
To further protect the integrity of the crucial SYSTEM hive in Windows 2000, the configuration
manager maintains a mirror of the SYSTEM hive on disk. If you look at the nonhidden files in a
Windows 2000 \Windows\System32\Config directory, you’ll see System.alt. System.alt is the
alternate hive. Whenever a hive sync flushes dirty sectors to the SYSTEM hive, the hive sync also
updates the System.alt hive. If the configuration manager detects that the SYSTEM hive is corrupt when the system boots, the configuration manager attempts to load the hive’s alternate. If
that hive is usable, it then uses that alternate to update the original SYSTEM hive.
Windows XP and Windows Server 2003 do not maintain a System.alt hive because NTLDR on
those versions of Windows knows how to process the System.log file to bring up to date a
System hive that’s become inconsistent during a shut down or crash. Windows Server 2003 has
other enhancements for tolerating corruption of the registry. Prior to Windows Server 2003, the
configuration manager crashes the system if it reads a base block, bin, or cell that contains data
that fails basic consistency checks. The configuration manager in Windows Server 2003 is more
tolerant of such problems, and if the corruption isn’t too severe, it will reinitialize corrupted data
structures, possibly deleting subkeys in the process, and continue operation. If it has to resort to
self-healing operation, it pops up a system error dialog box notifying the user.
Note
When you look at the hidden files on \Windows\System32\Config, you’ll also see a file
named System.sav. System.Sav is the version of the SYSTEM hive that served as the initial copy
of the System hive and is what Windows Setup copied from the install media.
Registry Optimizations
The configuration manager makes a few noteworthy performance optimizations. First, virtually every registry key has a security descriptor that protects access to the key. Storing a
unique security-descriptor copy for every key in a hive would be highly inefficient, however,
because the same security settings often apply to entire subtrees of the registry. When the system applies security to a key in Windows 2000, the configuration manager first checks the
security descriptors associated with the key’s parent key and then checks all the parent’s subkeys. If any of those security descriptors match the security descriptor the system is applying
to the key, the configuration manager simply shares the existing descriptors with the key,
Copyrighted material.
Chapter 4:
Management Mechanisms
211
employing a reference count to track how many keys share the same descriptor. In Windows
XP and Windows Server 2003, the configuration manager checks a pool of the unique security descriptors used within the same hive as the key to which new security is being applied,
and it shares any existing descriptor for the key, ensuring that there is at most one copy of
every unique security descriptor in a hive.
The configuration manager also optimizes the way it stores key and value names in a hive.
Although the registry is fully Unicode-capable and specifies all names using the Unicode convention, if a name contains only ASCII characters, the configuration manager stores the name
in ASCII form in the hive. When the configuration manager reads the name (such as when
performing name lookups), it converts the name into Unicode form in memory. Storing the
name in ASCII form can significantly reduce the size of a hive.
To minimize memory usage, key control blocks don’t store full key registry pathnames.
Instead, they reference only a key’s name. For example, a key control block that refers to \Registry\System\Control would refer to the name Control rather than to the full path. A further
memory optimization is that the configuration manager uses key name control blocks to store
key names, and all key control blocks for keys with the same name share the same key name
control block. To optimize performance, the configuration manager stores the key control
block names in a hash table for quick lookups.
To provide fast access to key control blocks, the configuration manager stores frequently
accessed key control blocks in the cache table, which is configured as a hash table. When the
configuration manager needs to look up a key control block, it first checks the cache table.
Finally, the configuration manager has another cache, the delayed close table, that stores key
control blocks that applications close, so that an application can quickly reopen a key it has
recently closed. The configuration manager removes the oldest key control blocks from the
delayed close table as it adds the most recently closed blocks to the table.
Services
Almost every operating system has a mechanism to start processes at system startup time that
provide services not tied to an interactive user. In Windows, such processes are called services
or Windows services, because they rely on the Windows API to interact with the system. Services are similar to UNIX daemon processes and often implement the server side of client/
server applications. An example of a Windows service might be a Web server because it must
be running regardless of whether anyone is logged on to the computer and it must start running when the system starts so that an administrator doesn’t have to remember, or even be
present, to start it.
Windows services consist of three components: a service application, a service control program (SCP), and the service control manager (SCM). First, we’ll describe service applications,
service accounts, and the operations of the SCM. Then we’ll explain how auto-start services
are started during the system boot. We’ll also cover the steps the SCM takes when a service
fails during its startup and the way the SCM shuts down services.
Copyrighted material.
212
Microsoft Windows Internals, Fourth Edition
Service Applications
Service applications, such as Web servers, consist of at least one executable that runs as a Windows service. A user wanting to start, stop, or configure a service uses an SCP. Although Windows supplies built-in SCPs that provide general start, stop, pause, and continue functionality,
some service applications include their own SCP that allows administrators to specify configuration settings particular to the service they manage.
Service applications are simply Windows executables (GUI or console) with additional code
to receive commands from the SCM as well as to communicate the application’s status back to
the SCM. Because most services don’t have a user interface, they are built as console programs.
When you install an application that includes a service, the application’s setup program must
register the service with the system. To register the service, the setup program calls the Windows CreateService function, a services-related function implemented in Advapi32.dll (\Windows\System32\Advapi32.dll). Advapi32, the “Advanced API” DLL, implements all the
client-side SCM APIs.
When a setup program registers a service by calling CreateService, a message is sent to the SCM
on the machine where the service will reside. The SCM then creates a registry key for the service under HKLM\SYSTEM\CurrentControlSet\Services. The Services key is the nonvolatile
representation of the SCM’s database. The individual keys for each service define the path of
the executable image that contains the service as well as parameters and configuration
options.
After creating a service, an installation or management application can start the service via the
StartService function. Because some service-based applications also must initialize during the
boot process to function, it’s not unusual for a setup program to register a service as an autostart service, ask the user to reboot the system to complete an installation, and let the SCM
start the service as the system boots.
When a program calls CreateService, it must specify a number of parameters describing the service’s characteristics. The characteristics include the service’s type (whether it’s a service that
runs in its own process rather than a service that shares a process with other services), the
location of the service’s executable image file, an optional display name, an optional account
name and password used to start the service in a particular account’s security context, a start
type that indicates whether the service starts automatically when the system boots or manually under the direction of an SCP, an error code that indicates how the system should react if
the service detects an error when starting, and, if the service starts automatically, optional
information that specifies when the service starts relative to other services.
The SCM stores each characteristic as a value in the service’s registry key. Figure 4-8 shows an
example of a service registry key.
Copyrighted material.
Chapter 4:
Figure 4-8
Management Mechanisms
213
Example of a service registry key
Table 4-7 lists all the service characteristics, many of which also apply to device drivers. (Not
every characteristic applies to every type of service or device driver.) If a service needs to store
configuration information that is private to the service, the convention is to create a subkey
named Parameters under its service key and then store the configuration information in values
under that subkey. The service then can retrieve the values by using standard registry functions.
Note
The SCM does not access a service’s Parameters subkey until the service is deleted, at
which time the SCM deletes the service’s entire key, including subkeys like Parameters.
Table 4-7
Service and Driver Registry Parameters
Value Setting
Value Name
Value Setting Description
Start
SERVICE_BOOT_START (0)
Ntldr or Osloader preloads the driver
so that it is in memory during the
boot. These drivers are initialized just
prior to SERVICE_ SYSTEM_START
drivers.
SERVICE_SYSTEM_START (1)
The driver loads and initializes
during kernel initialization after
SERVICE_ BOOT_START drivers have
initialized.
SERVICE_AUTO_START (2)
The SCM starts the driver or service
after the SCM process, Services.exe,
starts.
SERVICE_DEMAND_START (3)
The SCM starts the driver or service
on demand.
Copyrighted material.
214
Microsoft Windows Internals, Fourth Edition
Table 4-7 Service and Driver Registry Parameters
Value Setting
Value Name
Value Setting Description
SERVICE_DISABLED (4)
The driver or service doesn’t load or
initialize.
SERVICE_ERROR_IGNORE (0)
Any error the driver or service returns
is ignored and no warning is logged
or displayed.
SERVICE_ERROR_NORMAL (1)
If the driver or service reports an
error, a warning displays.
SERVICE_ERROR_SEVERE (2)
If the driver or service returns an
error and last known good isn’t
being used, reboot into last known
good; otherwise, continue the boot.
SERVICE_ERROR_CRITICAL (3)
If the driver or service returns an
error and last known good isn’t
being used, reboot into last known
good; otherwise, stop the boot with
a blue screen crash.
SERVICE_KERNEL_DRIVER (1)
Device driver.
SERVICE_FILE_SYSTEM_DRIVER (2)
Kernel-mode file system driver.
SERVICE_ADAPTER (4)
Obsolete.
SERVICE_RECOGNIZER_DRIVER (8)
File system recognizer driver.
SERVICE_WIN32_OWN_PROCESS
(16)
The service runs in a process that
hosts only one service.
SERVICE_WIN32_SHARE_PROCESS
(32)
The service runs in a process that
hosts multiple services.
SERVICE_INTERACTIVE_PROCESS
(256)
The service is allowed to display
windows on the console and receive
user input.
Group
Group name
The driver or service initializes when
its group is initialized.
Tag
Tag number
The specified location in a group
initialization order. This parameter
doesn’t apply to services.
ImagePath
Path to service or driver executable
file
If ImagePath isn’t specified, the I/O
manager looks for drivers in
\Windows\System32\Drivers and the
SCM uses Windows functions that
search for the image using the PATH
environment variable.
DependOnGroup
Group name
The driver or service won’t load
unless a driver or service from the
specified group loads.
ErrorControl
Type
Copyrighted material.
Chapter 4:
Table 4-7
Management Mechanisms
215
Service and Driver Registry Parameters
Value Setting
Value Name
Value Setting Description
DependOnService
Service name
The service won’t load until after the
specified service loads. This parameter doesn’t apply to device drivers
other than those with a start type of
SERVICE_AUTO_START.
ObjectName
Usually LocalSystem, but can be an
Specifies the account in which the
account name, such as .\Administra- service will run. If ObjectName isn’t
tor
specified, LocalSystem is the account
used. This parameter doesn’t apply
to device drivers.
DisplayName
Name of service
The service application shows
services by this name. If no name is
specified, the name of the service’s
registry key becomes its name.
Description
Description of service
Up to 32767-byte description of the
service.
FailureActions
Description of actions the SCM
Failure actions include restarting the
should take when service process ex- service process, rebooting the system,
its unexpectedly
and running a specified program. This
value doesn’t apply to drivers.
FailureCommand
Program command line
The SCM reads this value only if FailureActions specifies that a program
should execute upon service failure.
This value doesn’t apply to drivers.
Security
Security descriptor
This value contains the security descriptor that defines who has what
access to the service object created
internally by the SCM.
Notice that Type values include three that apply to device drivers: device driver, file system
driver, and file system recognizer. These are used by Windows device drivers, which also store
their parameters as registry data in the Services registry key. The SCM is responsible for starting
drivers with a Start value of SERVICE_AUTO_START or SERVICE_DEMAND_START, so it’s
natural for the SCM database to include drivers. Services use the other types, SERVICE_
WIN32_OWN_PROCESS and SERVICE_WIN32_SHARE_PROCESS, which are mutually
exclusive. An executable that hosts more than one service specifies the SERVICE_WIN32_
SHARE_PROCESS type. An advantage to having a process run more than one service is that the
system resources that would otherwise be required to run them in distinct processes are saved.
A potential disadvantage is that if one of the services of a collection running in the same process
causes an error that terminates the process, all the services of that process terminate. Also,
another limitation is that all the services must run under the same account.
Copyrighted material.
216
Microsoft Windows Internals, Fourth Edition
When the SCM starts a service process, the process immediately invokes the StartServiceCtrlDispatcher function. StartServiceCtrlDispatcher accepts a list of entry points into services, one
entry point for each service in the process. Each entry point is identified by the name of the
service the entry point corresponds to. After making a named pipe communications connection
to the SCM, StartServiceCtrlDispatcher sits in a loop waiting for commands to come through the
pipe from the SCM. The SCM sends a service-start command each time it starts a service the process owns. For each start command it receives, the StartServiceCtrlDispatcher function creates a
thread, called a service thread, to invoke the starting service’s entry point and implement the
command loop for the service. StartServiceCtrlDispatcher waits indefinitely for commands from
the SCM and returns control to the process’s main function only when all the process’s services
have stopped, allowing the service process to clean up resources before exiting.
A service entry point’s first action is to call the RegisterServiceCtrlHandler function. This function receives and stores a pointer to a function, called the control handler, which the service
implements to handle various commands it receives from the SCM. RegisterServiceCtrlHandler
doesn’t communicate with the SCM, but it stores the function in local process memory for the
StartServiceCtrlDispatcher function. The service entry point continues initializing the service,
which can include allocating memory, creating communications end points, and reading private configuration data from the registry. A convention most services follow is to store their
parameters under a subkey of their service registry key, named Parameters. While the entry
point is initializing the service, it might periodically send status messages, using the SetServiceStatus function, to the SCM indicating how the service’s startup is progressing. After the entry
point finishes initialization, a service thread usually sits in a loop waiting for requests from client applications. For example, a Web server would initialize a TCP listen socket and wait for
inbound HTTP connection requests.
A service process’s main thread, which executes in the StartServiceCtrlDispatcher function,
receives SCM commands directed at services in the process and invokes the target service’s
control handler function (stored by RegisterServiceCtrlHandler). SCM commands include stop,
pause, resume, interrogate, and shutdown, or application-defined commands. Figure 4-9
shows the internal organization of a service process. Pictured are the two threads that make
up a process hosting one service: the main thread and the service thread.
Copyrighted material.
Chapter 4:
Main thread
Main
Management Mechanisms
217
Service thread
1
RegisterServiceCtrlHandler
Pipe to
SCM
3
StartServiceCtrlDispatcher
2
Initialize
3
Service control handler
Process client requests
4
1.
2.
3.
4.
StartServiceCtrlDispatcher launches service thread.
Service thread registers control handler.
StartServiceCtrlDispatcher calls handlers in response to SCM commands.
Service thread processes client requests.
Figure 4-9
Connections to
service clients
Inside a service process
SrvAny Tool
If you have a program that you want to run as a service, you need to modify the startup
code to conform to the requirements for services outlined in this section. If you don’t
have the source code, you can use the SrvAny tool in the Windows resource kits. SrvAny
enables you to run any application as a service. It reads the path of the service file that it
must load from the Parameters subkey of the service’s registry key. When SrvAny starts,
it notifies the SCM that it is hosting a particular service, and when it receives a start command, it launches the service executable as a child process. The child process receives a
copy of the SrvAny process’s access token and a reference to the same window station,
so the executable runs within the same security account and with the same interactivity
setting as you specified when configuring the SrvAny process. SrvAny services don’t
have the share-process Type value, so each application you install as a service with
SrvAny runs in a separate process with a different instance of the SrvAny host program.
Service Accounts
The security context of a service is an important consideration for service developers as well
as for system administrators because it dictates what resources the process can access. Unless
a service installation program or administrator specifies otherwise, most services run in the
security context of the local system account (displayed sometimes as SYSTEM and other times
as LocalSystem). Windows XP introduced two variants on the local system account, the network service and local service accounts. The new accounts have fewer capabilities than the local
system account from a security standpoint, and any built-in Windows service that does not
Copyrighted material.
218
Microsoft Windows Internals, Fourth Edition
require the power of the local system account runs in the appropriate alternate service
account. The following subsections describe the special characteristics of these accounts.
The Local System Account
The local system account is the same account in which core Windows user-mode operating system components run, including the Session Manager (\Windows\System32\Smss.exe), the
Windows subsystem process (Csrss.exe), the local security authority subsystem (\Windows\
System32\Lsass.exe), and the Winlogon process (\Windows\System32\Winlogon.exe).
From a security perspective, the local system account is extremely powerful—more powerful
than any local or domain account when it comes to security ability on a local system. This
account has the following characteristics:
■
It is a member of the local administrators group. Table 4-8 shows the groups to which
the local system account belongs. (See Chapter 8 for information on how group membership is used in object access checks.)
■
It has the right to enable virtually every privilege (even privileges not normally granted
to the local administrator account, such as creating security tokens). See Table 4-9 for
the list of privileges assigned to the local system account. (Chapter 8 describes the use
of each privilege.)
■
Most files and registry keys grant full access to the local system account. (Even if they
don’t grant full access, a process running under the local system account can exercise
the take-ownership privilege to gain access.)
■
Processes running under the local system account run with the default user profile
(HKU\.DEFAULT). Therefore, they can’t access configuration information stored in the
user profiles of other accounts.
■
When a system is a member of a Windows domain, the local system account includes
the machine security identifier (SID) for the computer on which a service process is running. Therefore, a service running in the local system account will be automatically
authenticated on other machines in the same forest by using its computer account. (A
forest is a grouping of domains.)
■
Unless the machine account is specifically granted access to resources (such as network
shares, named pipes, and so on), a process can access network resources that allow null
sessions—that is, connections that require no credentials. You can specify the shares and
pipes on a particular computer that permit null sessions in the NullSessionPipes and
NullSessionShares registry values under HKLM\SYSTEM\CurrentControlSet\Services\lanmanserver\parameters.
Copyrighted material.
Chapter 4:
Table 4-8
Management Mechanisms
Service Account Group Membership
Local System
Network Service
Local Service
Everyone
Everyone
Everyone
Authenticated Users
Authenticated Users
Authenticated Users
Administrators
Users
Users
Local
Local
Network Service
Local Service
Service
Service
Table 4-9
219
Service Account Privileges
Local System
Network Service
Local Service
SeAssignPrimaryToken
Privilege
SeAssignPrimaryToken
Privilege
SeAssignPrimaryTokenPrivilege
SeAuditPrivilege
SeAuditPrivilege
SeChangeNotifyPrivilege
SeBackupPrivilege
SeChangeNotifyPrivilege
SeIncreaseQuotaPrivilege
SeChangeNotifyPrivilege
SeIncreaseQuotaPrivilege
SeCreateGlobalPrivilege
Privileges assigned to the
Everyone, Authenticated
Users, and Users groups
Privileges assigned to the Everyone, Authenticated Users, and
Users groups
SeCreatePagefilePrivilege
SeCreatePermanentPrivilege
SeAuditPrivilege
SeCreateTokenPrivilege*
SeDebugPrivilege
SeImpersonatePrivilege
SeIncreaseBasePriorityPrivilege
SeIncreaseQuotaPrivilege
SeLoadDriverPrivilege
SeLockMemoryPrivilege
SeManageVolumePrivilege
SeProfileSingleProcessPrivilege
SeRestorePrivilege
SeSecurityPrivilege
SeShutdownPrivilege
SeSystemEnvironmentPrivilege
SeSystemTimePrivilege
SeTakeOwnershipPrivilege
SeTcbPrivilege
SeUndockPrivilege
* The local system account on Windows Server 2003 does not include this privilege.
Copyrighted material.
220
Microsoft Windows Internals, Fourth Edition
The Network Service Account
The network service account is intended for use by services that wish to authenticate to other
machines on the network using the computer account, as does the local system account, but
do not have the need for membership in the administrators group or the use of many of the
privileges assigned to the local system account. Because the network service account does not
belong to the administrators group, services running in the network service account by
default have access to far fewer registry keys and file system folders and files than the services
running the local system account. Further, the assignment of few privileges limits the scope of
a compromised network service process. For example, a process running in the network service account cannot load a device driver or open arbitrary processes.
Another difference between the network service and local system accounts is that processes
running in the network service account use the network service account’s profile. The registry
component of the network service profile loads under HKU\S-1-5-20, and the files and directories that make up the component reside in \Documents and Settings\NetworkService.
A service that runs in the network service account in Windows XP and Windows Server
2003 is the DNS client, which is responsible for resolving DNS names and for locating
domain controllers.
The Local Service Account
The local service account is virtually identical to the network service account with the important difference that it can access only network resources that allow anonymous access. Table
4-9 shows that it has the same privileges as the local service account, and Table 4-8 shows that
it belongs to the same groups with the exception that it belongs to the Network Service group
instead of the Local Service group. The profile used by processes running in the local service
loads into HKU\S-1-5-19 and is stored in \Documents and Settings\LocalService.
Examples of services that Windows XP and Windows Server 2003 run in the local service
account include the Remote Registry Service that allows remote access to the local system’s
registry, the Alerter service that receives network-broadcast administrative alerts messages,
and the LmHosts service that performs NetBIOS name resolution.
Running Services in Alternate Accounts
Because of the restrictions just outlined, some services need to run with the security credentials of a user account. You can configure a service to run in an alternate account when the
service is created or by specifying an account and password that the service should run
under with the Windows Services MMC snap-in. In the Services snap-in, right-click on a service and select Properties, click the Log On tab, and select the This Account option, as
shown in Figure 4-10.
Copyrighted material.
Chapter 4:
Figure 4-10
Management Mechanisms
221
Service account settings
Interactive Services
Another restriction for services running under the local system, local service, and network service accounts is that they can’t (without using a special flag on the MessageBox function, discussed in a moment) display dialog boxes or windows on the interactive user’s desktop. This
limitation isn’t the direct result of running under these accounts but rather a consequence of
the way the Windows subsystem assigns service processes to window stations.
The Windows subsystem associates every Windows process with a window station. A window
station contains desktops, and desktops contain windows. Only one window station can be
visible on a console and receive user mouse and keyboard input. In a Terminal Services environment, one window station per session is visible, but services all run as part of the console
session. Windows names the visible window station WinSta0, and all interactive processes
access WinSta0.
Unless otherwise directed, the Windows subsystem associates services running in the local
system account with a nonvisible window station named Service-0x0-3e7$ that all noninteractive services share. The number in the name, 3e7, represents the logon session identifier
Lsass assigns to the logon session the SCM uses for noninteractive services running in the
local system account.
Services configured to run under a user account (that is, not the local system account) are run
in a different nonvisible window station named with the LSASS logon identifier assigned for
the service’s logon session. Figure 4-11 shows a sample display from the Winobj tool, available from www.sysinternals.com, viewing the object manager directory in which Windows
Copyrighted material.
222
Microsoft Windows Internals, Fourth Edition
places window station objects. Visible are the interactive window station (WinSta0), the noninteractive system service window station (Service-0x0-3e7$), and a noninteractive window
station assigned to a service process logged on as a user (Service-0x0-6368f$).
Figure 4-11
List of window stations
Regardless of whether services are running in a user account, the local system account, or the
local or network service accounts, services that aren’t running on the visible window station
can’t receive input from a user or display windows on the console. In fact, if a service were to
pop up a normal dialog box on the window station, the service would appear hung because
no user would be able to see the dialog box, which of course would prevent the user from providing keyboard or mouse input to dismiss it and allow the service to continue executing.
(The one exception is if the special flag MB_SERVICE_NOTIFICATION or
MB_DEFAULT_DESKTOP_ ONLY is set on the MessageBox call—if
MB_SERVICE_NOTIFICATION is specified, the message box will always be displayed on the
interactive window station, even if the service wasn’t configured with permission to interact
with the user; if MB_DEFAULT_DESKTOP_ONLY is specified, the message box is displayed
on the default desktop of the interactive window station.)
In rare cases, a service can have a valid reason to interact with the user via dialog boxes or windows. To configure a service with the right to interact with the user, the SERVICE_
INTERACTIVE_PROCESS modifier must be present in the service’s registry key’s Type
parameter. (Note that services configured to run under a user account can’t be marked as
interactive.) When the SCM starts a service marked as interactive, it launches the service’s process in the local system account’s security context but connects the service with WinSta0
instead of the noninteractive service window station. This connection to WinSta0 allows the
service to display dialog boxes and windows on the console and allows those windows to
respond to user input.
Copyrighted material.
Chapter 4:
Management Mechanisms
223
Note
Microsoft discourages running interactive services, especially in the local system
account, because of the inherent security vulnerability it creates. Windows presented by an interactive service are susceptible to the receipt of windows messages that a malicious process running on the desktop of an unprivileged user can use to cause buffer overflows in the service
process and subvert the service process to elevate the security privileges of the malicious process.
The Service Control Manager
The SCM’s executable file is \Windows\System32\Services.exe, and like most service processes, it runs as a Windows console program. The Winlogon process starts the SCM early
during the system boot. (Refer to Chapter 5 for details on the boot process.) The SCM’s
startup function, SvcCtrlMain, orchestrates the launching of services that are configured for
automatic startup. SvcCtrlMain executes shortly after the screen switches to a blank desktop
but generally before Winlogon has loaded the graphical identification and authentication
interface (GINA) that presents a logon dialog box.
SvcCtrlMain first creates a synchronization event named SvcCtrlEvent_ A3752DX that it initializes as nonsignaled. Only after the SCM completes steps necessary to prepare it to receive
commands from SCPs does the SCM set the event to a signaled state. The function that an
SCP uses to establish a dialog with the SCM is OpenSCManager. OpenSCManager prevents an
SCP from trying to contact the SCM before the SCM has initialized by waiting for
SvcCtrlEvent_A3752DX to become signaled.
Next, SvcCtrlMain gets down to business and calls ScCreateServiceDB, the function that builds
the SCM’s internal service database. ScCreateServiceDB reads and stores the contents of
HKLM\SYSTEM\CurrentControlSet\Control\ServiceGroupOrder\List, a REG_MULTI_SZ
value that lists the names and order of the defined service groups. A service’s registry key contains an optional Group value if that service or device driver needs to control its startup ordering with respect to services from other groups. For example, the Windows networking stack
is built from the bottom up, so networking services must specify Group values that place them
later in the startup sequence than networking device drivers. SCM internally creates a group
list that preserves the ordering of the groups it reads from the registry. Groups include (but
are not limited to) NDIS, TDI, Primary Disk, Keyboard Port, and Keyboard Class. Add-on and
third-party applications can even define their own groups and add them to the list. Microsoft
Transaction Server, for example, adds a group named MS Transactions.
ScCreateServiceDB then scans the contents of HKLM\SYSTEM\CurrentControlSet\Services,
creating an entry in the service database for each key it encounters. A database entry includes
all the service-related parameters defined for a service as well as fields that track the service’s
status. The SCM adds entries for device drivers as well as for services because the SCM starts
services and drivers marked as auto-start and detects startup failures for drivers marked bootstart and system-start. It also provides a means for applications to query the status of drivers.
Copyrighted material.
224
Microsoft Windows Internals, Fourth Edition
The I/O manager loads drivers marked boot-start and system-start before any user-mode processes execute, and therefore any drivers having these start types load before the SCM starts.
ScCreateServiceDB reads a service’s Group value to determine its membership in a group and
associates this value with the group’s entry in the group list created earlier. The function also
reads and records in the database the service’s group and service dependencies by querying
its DependOnGroup and DependOnService registry values. Figure 4-12 shows how the SCM
organizes the service entry and group order lists. Notice that the service list is alphabetically
sorted. The reason this list is sorted alphabetically is that the SCM creates the list from the Services registry key, and Windows stores registry keys alphabetically.
Service database
Group order list
Group1
Group2
Group3
Service entry list
Service1
Service2
Type
Start
DependOnGroup
DependOnService
Status
Group
…
Figure 4-12
Type
Start
DependOnGroup
DependOnService
Status
Group
…
Service3
Type
Start
DependOnGroup
DependOnService
Status
Group
…
Organization of a service database
During service startup, the SCM might need to call on LSASS (for example, to log on a service
in a user account), so the SCM waits for LSASS to signal the LSA_RPC_SERVER_ACTIVE synchronization event, which it does when it finishes initializing. Winlogon also starts the LSASS
process, so the initialization of LSASS is concurrent with that of the SCM, and the order in
which LSASS and the SCM complete initialization can vary. Then SvcCtrlMain calls ScGetBootAndSystemDriverState to scan the service database looking for boot-start and system-start
device driver entries. ScGetBootAndSystemDriverState determines whether or not a driver successfully started by looking up its name in the object manager namespace directory named
\Driver. When a device driver successfully loads, the I/O manager inserts the driver’s object in
the namespace under this directory, so if its name isn’t present, it hasn’t loaded. Figure 4-13
shows Winobj displaying the contents of the Driver directory. If a driver isn’t loaded, the SCM
looks for its name in the list of drivers returned by the PnP_DeviceList function. PnP_DeviceList
supplies the drivers included in the system’s current hardware profile. SvcCtrlMain notes the
names of drivers that haven’t started and that are part of the current profile in a list named
ScFailedDrivers.
Before starting the auto-start services, the SCM performs a few more steps. It creates its remote
procedure call (RPC) named pipe, which is named \Pipe\Ntsvcs, and then RPC launches a
Copyrighted material.
Chapter 4:
Management Mechanisms
225
thread to listen on the pipe for incoming messages from SCPs. The SCM then signals its initialization-complete event, SvcCtrlEvent_A3752DX. Registering a console application shutdown event handler and registering with the Windows subsystem process via
RegisterServiceProcess prepares the SCM for system shutdown.
Figure 4-13
List of driver objects
Network Drive Letters
In addition to its role as an interface to services, the SCM has another totally unrelated
responsibility: it notifies GUI applications in a system whenever the system creates or
deletes a network drive-letter connection. The SCM waits for the Multiple Provider
Router (MPR) to signal a named event, \BaseNamedObjects\ScNetDrvMsg, which MPR
signals whenever an application assigns a drive letter to a remote network share or
deletes a remote-share drive-letter assignment. (See Chapter 13 for more information on
MPR.) When MPR signals the event, the SCM calls the GetDriveType Windows function
to query the list of connected network drive letters. If the list changes across the event
signal, the SCM sends a Windows broadcast message of type WM_DEVICECHANGE.
The SCM uses either DBT_ DEVICEREMOVECOMPLETE or DBT_DEVICEARRIVAL as
the message’s subtype. This message is primarily intended for Windows Explorer so
that it can update any open My Computer windows to show the presence or absence of
a network drive letter.
Service Startup
SvcCtrlMain invokes the SCM function ScAutoStartServices to start all services that have a Start
value designating auto-start. ScAutoStartServices also starts auto-start device drivers. To avoid
confusion, you should assume that the term services means services and drivers unless indiCopyrighted material.
226
Microsoft Windows Internals, Fourth Edition
cated otherwise. The algorithm in ScAutoStartServices for starting services in the correct order
proceeds in phases, whereby a phase corresponds to a group and phases proceed in the
sequence defined by the group ordering stored in the HKLM\SYSTEM\CurrentControlSet\Control\ServiceGroupOrder\List registry value. The List value, shown in Figure 4-14,
includes the names of groups in the order that the SCM should start them. Thus, assigning a
service to a group has no effect other than to fine-tune its startup with respect to other services
belonging to different groups.
Figure 4-14
ServiceGroupOrder registry key
When a phase starts, ScAutoStartServices marks all the service entries belonging to the phase’s
group for startup. Then ScAutoStartServices loops through the marked services seeing whether
it can start each one. Part of the check it makes consists of determining whether the service
has a dependency on another group, as specified by the existence of the DependOnGroup
value in the service’s registry key. If a dependency exists, the group on which the service is
dependent must have already initialized, and at least one service of that group must have successfully started. If the service depends on a group that starts later than the service’s group in
the group startup sequence, the SCM notes a “circular dependency” error for the service. If
ScAutoStartServices is considering a Windows service or an auto-start device driver, it next
checks to see whether the service depends on one or more other services, and if so, if those
services have already started. Service dependencies are indicated with the DependOnService
registry value in a service’s registry key. If a service depends on other services that belong to
groups that come later in the ServiceGroupOrder\List, the SCM also generates a “circular
dependency” error and doesn’t start the service. If the service depends on any services from
the same group that haven’t yet started, the service is skipped.
When the dependencies of a service have been satisfied, ScAutoStartServices makes a final
check to see whether the service is part of the current boot configuration before starting the
service. When the system is booted in safe mode, the SCM ensures that the service is either
identified by name or by group in the appropriate safe boot registry key. There are two safe
boot keys, Minimal and Network, under HKLM\SYSTEM\CurrentControlSet\Control\SafeBoot, and the one that the SCM checks depends on what safe mode the user booted. If the
user chose Safe Mode or Safe Mode With Command Prompt at the special boot menu (which
you can access by pressing F8 when prompted in the boot process), the SCM references the
Copyrighted material.
Chapter 4:
Management Mechanisms
227
Minimal key; if the user chose Safe Mode With Networking, the SCM refers to Network. The
existence of a string value named Option under the SafeBoot key indicates not only that the
system booted in safe mode but also the type of safe mode the user selected. For more information about safe boots, see the section “Safe Mode” in Chapter 5.
Once the SCM decides to start a service, it calls ScStartService, which takes different steps for
services than for device drivers. When ScStartService starts a Windows service, it first determines the name of the file that runs the service’s process by reading the ImagePath value from
the service’s registry key. It then examines the service’s Type value, and if that value is
SERVICE_WINDOWS_SHARE_PROCESS (0x20), the SCM ensures that the process the service runs in, if already started, is logged on using the same account as specified for the service
being started. A service’s ObjectName registry value stores the user account in which the service should run. A service with no ObjectName or an ObjectName of LocalSystem runs in the
local system account.
The SCM verifies that the service’s process hasn’t already been started in a different account
by checking to see whether the service’s ImagePath value has an entry in an internal SCM
database called the image database. If the image database doesn’t have an entry for the ImagePath value, the SCM creates one. When the SCM creates a new entry, it stores the logon
account name used for the service and the data from the service’s ImagePath value. The SCM
requires services to have an ImagePath value. If a service doesn’t have an ImagePath value, the
SCM reports an error stating that it couldn’t find the service’s path and isn’t able to start the
service. If the SCM locates an existing image database entry with matching ImagePath data,
the SCM ensures that the user account information for the service it’s starting is the same as
the information stored in the database entry—a process can be logged on as only one account,
so the SCM reports an error when a service specifies a different account name than another
service that has already started in the same process.
The SCM calls ScLogonAndStartImage to log on a service if the service’s configuration specifies
and to start the service’s process. The SCM logs on services that don’t run in the system
account by calling the LSASS function LsaLogonUser. LsaLogonUser normally requires a password, but the SCM indicates to LSASS that the password is stored as a service’s LSASS “secret”
under the key HKLM\SECURITY\Policy\Secrets in the registry. (Keep in mind that the contents of the SECURITY aren’t typically visible because its default security settings permit
access only from the system account.) When the SCM calls LsaLogonUser, it specifies a service
logon as the logon type, so LSASS looks up the password in the Secrets subkey that has a
name in the form _SC_<service name>.
The SCM directs LSASS to store a logon password as a secret using the LsaStorePrivateData
function when an SCP configures a service’s logon information. When a logon is successful,
LsaLogonUser returns a handle to an access token to the caller. Windows uses access tokens to
represent a user’s security context, and the SCM later associates the access token with the process that implements the service.
Copyrighted material.
228
Microsoft Windows Internals, Fourth Edition
After a successful logon, the SCM loads the account’s profile information, if it’s not already
loaded, by calling the UserEnv DLL’s (\Windows\System32\Userenv.dll) LoadUserProfile
function. The value HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList\<user profile key>\ProfileImagePath contains the location on disk of a registry hive that
LoadUserProfile loads into the registry, making the information in the hive the
HKEY_CURRENT_USER key for the service.
An interactive service must open the WinSta0 window station, but before ScLogonAndStartImage allows an interactive service to access WinSta0 it checks to see whether the value
HKLM\SYSTEM\CurrentControlSet\Control\Windows\NoInteractiveServices is set.
Administrators set this value to prevent services marked as interactive from displaying windows on the console. This option is desirable in unattended server environments in which no
user is present to respond to popups from interactive services.
As its next step, ScLogonAndStartImage proceeds to launch the service’s process, if the process
hasn’t already been started (for another service, for example). The SCM starts the process in
a suspended state with the CreateProcessAsUser Windows function. The SCM next creates a
named pipe through which it communicates with the service process, and it assigns the pipe
the name \Pipe\Net\NtControlPipeX, where X is a number that increments each time the
SCM creates a pipe. The SCM resumes the service process via the ResumeThread function and
waits for the service to connect to its SCM pipe. If it exists, the registry value HKLM\SYSTEM\CurrentControlSet\Control\ServicesPipeTimeout determines the length of time that
the SCM waits for a service to call StartServiceCtrlDispatcher and connect before it gives up, terminates the process, and concludes that the service failed to start. If ServicesPipeTimeout
doesn’t exist, the SCM uses a default timeout of 30 seconds. The SCM uses the same timeout
value for all its service communications.
When a service connects to the SCM through the pipe, the SCM sends the service a start command. If the service fails to respond positively to the start command within the timeout
period, the SCM gives up and moves on to start the next service. When a service doesn’t
respond to a start request, the SCM doesn’t terminate the process, as it does when a service
doesn’t call StartServiceCtrlDispatcher within the timeout; instead, it notes an error in the system Event Log that indicates the service failed to start in a timely manner.
If the service the SCM starts with a call to ScStartService has a Type registry value of
SERVICE_KERNEL_DRIVER or SERVICE_FILE_SYSTEM_ DRIVER, the service is really a
device driver, and so ScStartService calls ScLoadDeviceDriver to load the driver. ScLoadDeviceDriver enables the load driver security privilege for the SCM process and then invokes the kernel service NtLoadDriver, passing in the data in the ImagePath value of the driver’s registry
key. Unlike services, drivers don’t need to specify an ImagePath value, and if the value is
absent, the SCM builds an image path by appending the driver’s name to the string \Windows\System32\Drivers\.
Copyrighted material.
Chapter 4:
Management Mechanisms
229
ScAutoStartServices continues looping through the services belonging to a group until all the
services have either started or generated dependency errors. This looping is the SCM’s way of
automatically ordering services within a group according to their DependOnService dependencies. The SCM will start the services that other services depend on in earlier loops, skipping the dependent services until subsequent loops. Note that the SCM ignores Tag values for
Windows services, which you might come across in subkeys under the HKLM\SYSTEM\
CurrentControlSet\Services key; the I/O manager honors Tag values to order device driver
startup within a group for boot and system-start drivers.
Once the SCM completes phases for all the groups listed in the ServiceGroupOrder\List
value, it performs a phase for services belonging to groups not listed in the value and a final
phase for services without a group. When it’s finished starting all auto-start services and drivers, the SCM signals the event \BaseNamedObjects\SC_AutoStartComplete.
Startup Errors
If a driver or a service reports an error in response to the SCM’s startup command, the ErrorControl value of the service’s registry key determines how the SCM reacts. If the ErrorControl
value is SERVICE_ERROR_IGNORE (0) or the ErrorControl value isn’t specified, the SCM
simply ignores the error and continues processing service startups. If the ErrorControl value
is SERVICE_ERROR_NORMAL (1), the SCM writes an event to the system Event Log that
says, “The <service name> service failed to start due to the following error:”. The SCM includes
the textual representation of the Windows error code that the service returned to the SCM as
the reason for the startup failure in the Event Log record. Figure 4-15 shows the Event Log
entry that reports a service startup error.
Figure 4-15
Copyrighted material.
Service startup failure Event Log entry
230
Microsoft Windows Internals, Fourth Edition
If a service with an ErrorControl value of SERVICE_ERROR_SEVERE (2) or
SERVICE_ERROR_CRITICAL (3) reports a startup error, the SCM logs a record to the Event
Log and then calls the internal function ScRevertToLastKnownGood. This function switches the
system’s registry configuration to a version, named last known good, with which the system
last booted successfully. Then it restarts the system using the NtShutdownSystem system service, which is implemented in the executive. If the system is already booting with the last
known good configuration, the system just reboots.
Accepting the Boot and Last Known Good
Besides starting services, the system charges the SCM with determining when the system’s
registry configuration, HKLM\SYSTEM\CurrentControlSet, should be saved as the last
known good control set. The CurrentControlSet key contains the Services key as a subkey, so
CurrentControlSet includes the registry representation of the SCM database. It also contains
the Control key, which stores many kernel-mode and user-mode subsystem configuration settings. By default, a successful boot consists of a successful startup of auto-start services and a
successful user logon. A boot fails if the system halts because a device driver crashes the system during the boot or if an auto-start service with an ErrorControl value of
SERVICE_ERROR_SEVERE or SERVICE_ERROR_CRITICAL reports a startup error.
The SCM obviously knows when it has completed a successful startup of the auto-start services, but Winlogon (\Windows\System32\Winlogon.exe) must notify it when there is a successful logon. Winlogon invokes the NotifyBootConfigStatus function when a user logs on, and
NotifyBootConfigStatus sends a message to the SCM. Following the successful start of the autostart services or the receipt of the message from NotifyBootConfigStatus (whichever comes
last), the SCM calls the system function NtInitializeRegistry to save the current registry startup
configuration.
Third-party software developers can supersede Winlogon’s definition of a successful logon
with their own definition. For example, a system running Microsoft SQL Server might not
consider a boot successful until after SQL Server is able to accept and process transactions.
Developers impose their definition of a successful boot by writing a boot-verification program
and installing the program by pointing to its location on disk with the value stored in the registry key HKLM\SYSTEM\CurrentControlSet\Control\BootVerificationProgram. In addition, a boot-verification program’s installation must disable Winlogon’s call to
NotifyBootConfigStatus by setting HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon\ReportBootOk to 0. When a boot-verification program is installed, the SCM
launches it after finishing auto-start services and waits for the program’s call to NotifyBootConfigStatus before saving the last known good control set.
Windows maintains several copies of CurrentControlSet, and CurrentControlSet is really a
symbolic registry link that points to one of the copies. The control sets have names in the form
HKLM\SYSTEM\ControlSetnnn, where nnn is a number such as 001 or 002. The HKLM\SYSTEM\Select key contains values that identify the role of each control set. For example, if CurCopyrighted material.
Chapter 4:
Management Mechanisms
231
rentControlSet points to ControlSet001, the Current value under Select has a value of 1. The
LastKnownGood value under Select contains the number of the last known good control set,
which is the control set last used to boot successfully. Another value that might be on your
system under the Select key is Failed, which points to the last control set for which the boot
was deemed unsuccessful and aborted in favor of an attempt at booting with the last known
good control set. Figure 4-16 displays a system’s control sets and Select values.
Figure 4-16
Control set selection key
NtInitializeRegistry takes the contents of the last known good control set and synchronizes it
with that of the CurrentControlSet key’s tree. If this was the system’s first successful boot, the
last known good won’t exist and the system will create a new control set for it. If the last
known good tree exists, the system simply updates it with differences between it and CurrentControlSet.
Last known good is helpful in situations in which a change to CurrentControlSet, such as the
modification of a system performance-tuning value under HKLM\SYSTEM\Control or the
addition of a service or device driver, causes the subsequent boot to fail. Users can press F8
early in the boot process to bring up a menu that lets them direct the boot to use the last
known good control set, rolling the system’s registry configuration back to the way it was the
last time the system booted successfully. Chapter 5 describes in more detail the use of Last
Known Good and other recovery mechanisms for troubleshooting system startup problems.
Service Failures
A service can have optional FailureActions and FailureCommand values in its registry key that
the SCM records during the service’s startup. The SCM registers with the system so that the
system signals the SCM when a service process exits. When a service process terminates unexpectedly, the SCM determines which services ran in the process and takes the recovery steps
specified by their failure-related registry values.
Actions that a service can configure for the SCM include restarting the service, running a program, and rebooting the computer. Furthermore, a service can specify the failure actions that
Copyrighted material.
232
Microsoft Windows Internals, Fourth Edition
take place the first time the service process fails, the second time, and subsequent times, and
it can indicate a delay period that the SCM waits before restarting the service if the service asks
to be restarted. The service failure action of the IIS Admin Service results in the SCM running
the IISReset application, which performs cleanup work and then restarts the service. You can
easily manage the recovery actions for a service with the Recovery tab of the service’s Properties dialog box in the Services MMC snap-in, as shown in Figure 4-17.
Figure 4-17
Service recovery options
Service Shutdown
When Winlogon calls the Windows ExitWindowsEx function, ExitWindowsEx sends a message
to Csrss, the Windows subsystem process, to invoke Csrss’s shutdown routine. Csrss loops
through the active processes and notifies them that the system is shutting down. For every
system process except the SCM, Csrss waits up to the number of seconds specified by
HKU\.DEFAULT\Control Panel\Desktop\WaitToKillAppTimeout (which defaults to 20 seconds) for the process to exit before moving on to the next process. When Csrss encounters
the SCM process, it also notifies it that the system is shutting down but employs a timeout
specific to the SCM. Csrss recognizes the SCM using the process ID Csrss saved when the
SCM registered with Csrss using the RegisterServicesProcess function during system initialization. The SCM’s timeout differs from that of other processes because Csrss knows that the
SCM communicates with services that need to perform cleanup when they shut down, and so
an administrator might need to tune only the SCM’s timeout. The SCM’s timeout value resides
in the HKLM\SYSTEM\CurrentControlSet\Control\WaitToKillServiceTimeout registry
value, and it defaults to 20 seconds.
Copyrighted material.
Chapter 4:
Management Mechanisms
233
The SCM’s shutdown handler is responsible for sending shutdown notifications to all the services that requested shutdown notification when they initialized with the SCM. The SCM
function ScShutdownAllServices loops through the SCM services database searching for services desiring shutdown notification and sends each one a shutdown command. For each service to which it sends a shutdown command, the SCM records the value of the service’s wait
hint, a value that a service also specifies when it registers with the SCM. The SCM keeps track
of the largest wait hint it receives. After sending the shutdown messages, the SCM waits either
until one of the services it notified of shutdown exits or until the time specified by the largest
wait hint passes.
If the wait hint expires without a service exiting, the SCM determines whether one or more of
the services it was waiting on to exit have sent a message to the SCM telling the SCM that the
service is progressing in its shutdown process. If at least one service made progress, the SCM
waits again for the duration of the wait hint. The SCM continues executing this wait loop until
either all the services have exited or none of the services upon which it’s waiting has notified
it of progress within the wait hint timeout period.
While the SCM is busy telling services to shut down and waiting for them to exit, Csrss waits
for the SCM to exit. If Csrss’s wait ends without the SCM having exited (the WaitToKillServiceTimeout time expires), Csrss simply moves on, continuing the shutdown process. Thus,
services that fail to shut down in a timely manner are simply left running, along with the SCM,
as the system shuts down. Unfortunately, there’s no way for administrators to know whether
they should raise the WaitToKillServiceTimeout value on systems where services aren’t getting a chance to shut down completely before the system shuts down. See “Shutdown” in
Chapter 5 for more information on the shutdown process.
Shared Service Processes
Running every service in its own process instead of having services share a process whenever possible wastes system resources. However, sharing processes means that if any of the
services in the process has a bug that causes the process to exit, all the services in that process terminate.
Of the Windows built-in services, some run in their own process and some share a process
with other services. For example, the SCM process hosts the Event Log service and the usermode Plug and Play service, and the LSASS process contains security-related services—such as
the Security Accounts Manager (SamSs) service, the Net Logon (Netlogon) service, and the
IPSec Policy Agent (PolicyAgent) service.
There is also a “generic” process named Service Host (SvcHost - \Windows\System32\Svchost.exe) to contain multiple services. Multiple instances of SvcHost can
be running in different processes. Services that run in SvcHost processes include Telephony
Copyrighted material.
234
Microsoft Windows Internals, Fourth Edition
(TapiSrv), Remote Procedure Call (RpcSs), and Remote Access Connection Manager (RasMan). Windows implements services that run in SvcHost as DLLs and includes an ImagePath
definition of the form “%SystemRoot%\System32\svchost.exe -k netsvcs” in the service’s registry key. The service’s registry key must also have a registry value named ServiceDll under a
Parameters subkey that points to the service’s DLL file.
All services that share a common SvcHost process specify the same parameter (“-k netsvcs” in
the example in the preceding paragraph) so that they have a single entry in the SCM’s image
database. When the SCM encounters the first service that has a SvcHost ImagePath with a particular parameter during service startup, it creates a new image database entry and launches a
SvcHost process with the parameter. The new SvcHost process takes the parameter and looks
for a value having the same name as the parameter under HKLM\SOFTWARE\Microsoft\
Windows NT\CurrentVersion\Svchost. SvcHost reads the contents of the value, interpreting
it as a list of service names, and notifies the SCM that it’s hosting those services when SvcHost
registers with the SCM. Figure 4-18 presents an example Svchost registry key that shows that
a SvcHost process started with the “-k netsvcs” parameter is prepared to host a number of different network-related services.
Figure 4-18
Svchost registry key
When the SCM encounters a SvcHost service during service startup with an ImagePath
matching an entry it already has in the image database, it doesn’t launch a second process but
instead just sends a start command for the service to the SvcHost it already started for that
ImagePath value. The existing SvcHost process reads the ServiceDll parameter in the service’s
registry key and loads the DLL into its process to start the service.
Copyrighted material.
Chapter 4:
Management Mechanisms
235
EXPERIMENT: Viewing Services Running Inside Processes
The Process Explorer utility that you can download from www.sysinternals.com shows
detailed information about the services running with processes. Run Process Explorer
and view Services tabs on the process properties dialog box for the following processes:
Services.exe, Lsass.exe, and Svchost.exe. Several instances of SvcHost will be running
on your system, and you can see the account in which each is running by adding the
Username column to the Process Explorer display or by looking at the Username field
on the Image tab of a process’s Process Properties dialog box. The following figure
shows the list of services running within a SvcHost executing in the local service
account:
The information displayed includes the service name, display name, and service description, if it has one, which Process Explorer shows beneath the service list when you select
a service.
You can also use the tlist.exe tool from the Windows Support Tools or Tasklist, which
ships with Windows XP and Windows Server 2003, to view the list of services running
within processes from a command prompt. The syntax to see services with Tlist is:
tlist /s
The syntax for tasklist is:
tasklist /svc
Note that these utilities do not show service display names or descriptions, only service
names.
Copyrighted material.
236
Microsoft Windows Internals, Fourth Edition
Service Control Programs
Service control programs are standard Windows applications that use XSCM service management functions, including CreateService, OpenService, StartService, ControlService, QueryServiceStatus, and DeleteService. To use the SCM functions, an SCP must first open a communications
channel to the SCM by calling the OpenSCManager function. At the time of the open call, the
SCP must specify what types of actions it wants to perform. For example, if an SCP simply
wants to enumerate and display the services present in the SCM’s database, it requests enumerate-service access in its call to OpenSCManager. During its initialization, the SCM creates
an internal object that represents the SCM database and uses the Windows security functions
to protect the object with a security descriptor that specifies what accounts can open the
object with what access permissions. For example, the security descriptor indicates that the
Authenticated Users group can open the SCM object with enumerate-service access. However,
only administrators can open the object with the access required to create or delete a service.
As it does for the SCM database, the SCM implements security for services themselves. When
an SCP creates a service by using the CreateService function, it specifies a security descriptor
that the SCM associates internally with the service’s entry in the service database. The SCM
stores the security descriptor in the service’s registry key as the Security value, and it reads
that value when it scans the registry’s Services key during initialization so that the security settings persist across reboots. In the same way that an SCP must specify what types of access it
wants to the SCM database in its call to OpenSCManager, an SCP must tell the SCM what
access it wants to a service in a call to OpenService. Accesses that an SCP can request include
the ability to query a service’s status and to configure, stop, and start a service.
The SCP you’re probably most familiar with is the Services MMC snap-in that’s included in
Windows, which resides in \Windows\System32\Filemgmt.dll. Windows XP and Windows
Server 2003 include Sc.exe (Service Controller tool), a command-line service control program
that’s available for Windows 2000 in the Windows 2000 resource kits.
SCPs sometimes layer service policy on top of what the SCM implements. A good example is
the timeout that the Services MMC snap-in implements when a service is started manually.
The snap-in presents a progress bar that represents the progress of a service’s startup.
Whereas the SCM waits indefinitely for a service to respond to a start command, the Services
snap-in waits only 2 minutes before the progress bar reaches 100 percent and the snap-in
announces that the service didn’t start in a timely manner. Services indirectly interact with
SCPs by setting their configuration status to reflect their progress as they respond to SCM
commands such as the start command. SCPs query the status with the QueryServiceStatus
function. They can tell when a service actively updates the status versus when a service
appears to be hung, and the SCM can take appropriate actions in notifying a user about what
the service is doing.
Copyrighted material.
Chapter 4:
Management Mechanisms
237
Windows Management Instrumentation
Windows NT has always had integrated performance and system-event monitoring tools.
Applications and the system typically use the Event Manager to report errors and diagnostic
messages. The Event Viewer utility lets administrators view event output from either the local
computer or another computer on the network. Similarly, the performance counter mechanism lets applications and operating system components report performance-related statistics
to performance-monitoring applications such as the Performance Monitor.
Although the Windows NT 4 event-monitoring and performance-monitoring features met
their design goals, they had limitations. For example, the programming interfaces differ from
one another, and this variation increases the complexity of applications that use both event
and performance monitoring to collect data. Perhaps the biggest drawback to the monitoring
facilities in Windows NT 4 is that they have little or no extensibility and that neither event logging nor performance data collection provides the two-way interaction necessary in a management API. Applications must provide data in predefined formats. The Performance API
provides no way for an application to receive notification of performance-related events, and
applications that request notification of Event Manager events can’t restrict notification to specific event types or sources. Finally, clients of either collection facility can’t communicate with
event-data or performance-data providers through the Event Manager or Performance API.
To address these limitations as well as to provide management capabilities for other types of
data sources, Windows has a new management mechanism, Windows Management Instrumentation (WMI). WMI is an implementation of Web-Based Enterprise Management
(WBEM), a standard that the Distributed Management Task Force (DMTF—an industry consortium) defines. The WBEM standard encompasses the design of an extensible enterprise
data-collection and data-management facility that has the flexibility and extensibility required
to manage local and remote systems that comprise arbitrary components. WMI support was
added to Windows NT 4 in Service Pack 4. It is also supported in Windows 95 OSR2, Windows 98 and Windows Millennium. Although most of this section applies to all the Windows
platforms that support WMI, implementation details are specific to Windows 2000, Windows
XP, and Windows Server 2003.
WMI Architecture
WMI consists of four main components, as shown in Figure 4-19: management applications,
WMI infrastructure, providers, and managed objects. Management applications are Windows applications that access and display or process data that the applications obtain about
managed objects. A simple example of a management application is a Performance tool
replacement that relies on WMI rather than the Performance API to obtain performance
information. A more complex example is an enterprise-management tool that lets administrators perform automated inventories of the software and hardware configuration of every
computer in their enterprise.
Copyrighted material.
238
Microsoft Windows Internals, Fourth Edition
Database
application
Web browser
ODBC
ActiveX
controls
C/C++
application
Management
applications
Windows Management API
COM/DCOM
CIM repository
CIM Object Manager (CIMOM)
WMI infrastructure
COM/DCOM
SNMP provider
Windows provider
Registry
provider
Providers
SNMP objects
Windows objects
Registry
objects
Managed objects
Figure 4-19
WMI architecture
Developers typically must target management applications to collect data from and manage
specific objects. An object might represent one component, such as a network adapter device,
or a collection of components, such as a computer. (The computer object might contain the
network adapter object.) Providers need to define and export the representation of the objects
that management applications are interested in. For example, the vendor of a network adapter
might want to add adapter-specific properties to the network adapter WMI support that Windows includes, querying and setting the adapter’s state and behavior as the management
applications direct. In some cases (for example, for device drivers), Microsoft supplies a provider that has its own API to help developers leverage the provider’s implementation for their
own managed objects with minimal coding effort.
The WMI infrastructure, the heart of which is the Common Information Model (CIM) Object
Manager (CIMOM), is the glue that binds management applications and providers. (CIM is
described later in this chapter.) The infrastructure also serves as the object-class store and, in
many cases, as the storage manager for persistent object properties. WMI implements the
store, or repository, as an on-disk database named the CIMOM Object Repository. As part of
its infrastructure, WMI supports several APIs through which management applications access
object data and providers supply data and class definitions.
Copyrighted material.
Chapter 4:
Management Mechanisms
239
Windows programs use the WMI COM API, the primary management API, to directly interact
with WMI. Other APIs layer on top of the COM API and include an Open Database Connectivity (ODBC) adapter for the Microsoft Access database application. A database developer
uses the WMI ODBC adapter to embed references to object data in the developer’s database.
Then the developer can easily generate reports with database queries that contain WMI-based
data. WMI ActiveX controls support another layered API. Web developers use the ActiveX
controls to construct Web-based interfaces to WMI data. Another management API is the
WMI scripting API, for use in script-based applications and Microsoft Visual Basic programs.
WMI scripting support exists for all Microsoft programming language technologies.
As they are for management applications, WMI COM interfaces constitute the primary API for
providers. However, unlike management applications, which are COM clients, providers are
COM or Distributed COM (DCOM) servers (that is, the providers implement COM objects
that WMI interacts with). Possible embodiments of a WMI provider include DLLs that load
into WMI’s manager process or stand-alone Windows applications or Windows services.
Microsoft includes a number of built-in providers that present data from well-known sources,
such as the Performance API, the registry, the Event Manager, Active Directory, SNMP, and
Windows Driver Model (WDM) device drivers. The WMI SDK lets developers develop thirdparty WMI providers.
Providers
At the core of WBEM is the DMTF-designed CIM specification. The CIM specifies how management systems represent, from a systems management perspective, anything from a computer to an application or device on a computer. Provider developers use the CIM to represent
the components that make up the parts of an application for which the developers want to
enable management. Developers use the Managed Object Format (MOF) language to implement a CIM representation.
In addition to defining classes that represent objects, a provider must interface WMI to the
objects. WMI classifies providers according to the interface features the providers supply.
Table 4-10 lists WMI provider classifications. Note that a provider can implement one or more
features; therefore, a provider can be, for example, both a class and an event provider. To clarify the feature definitions in Table 4-10, let’s look at a provider that implements several of
those features. The Event Log provider supports several objects, including an Event Log Computer, an Event Log Record, and an Event Log File. The Event Log is an Instance provider
because it can define multiple instances for several of its classes. One class for which the Event
Log provider defines multiple instances is the Event Log File class (Win32_NTEventlogFile);
the Event Log provider defines an instance of this class for each of the system’s event logs
(that is, System Event Log, Application Event Log, and Security Event Log).
Copyrighted material.
240
Microsoft Windows Internals, Fourth Edition
Table 4-10
Provider Classifications
Classification
Description
Class
Can supply, modify, delete, and enumerate a provider-specific class. Can
also support query processing. Active Directory is a rare example of a service that is a class provider.
Instance
Can supply, modify, delete, and enumerate instances of system and provider-specific classes. An instance represents a managed object. Can also support query processing.
Property
Can supply and modify individual object property values.
Method
Supplies methods for a provider-specific class.
Event
Generates event notifications.
Event consumer
Maps a physical consumer to a logical consumer to support event notification.
The Event Log provider defines the instance data and lets management applications enumerate the records. To let management applications use WMI to back up and restore the Event
Log files, the Event Log provider implements backup and restore methods for Event Log File
objects. Doing so makes the Event Log provider a Method provider. Finally, a management
application can register to receive notification whenever a new record writes to one of the
Event Logs. Thus, the Event Log provider serves as an Event provider when it uses WMI event
notification to tell WMI that Event Log records have arrived.
The Common Information Model and the Managed Object Format
Language
The CIM follows in the steps of object-oriented languages such as C++ and Java, in which a
modeler designs representations as classes. Working with classes lets developers use the
powerful modeling techniques of inheritance and composition. Subclasses can inherit the
attributes of a parent class, and they can add their own characteristics and override the characteristics they inherit from the parent class. A class that inherits properties from another
class derives from that class. Classes also compose: a developer can build a class that includes
other classes.
The DMTF provides multiple classes as part of the WBEM standard. These classes are CIM’s
basic language and represent objects that apply to all areas of management. The classes are
part of the CIM core model. An example of a core class is CIM_ManagedSystemElement. This
class contains a few basic properties that identify physical components such as hardware
devices, and logical components such as processes and files. The properties include a caption,
description, installation date, and status. Thus, the CIM_LogicalElement and
CIM_PhysicalElement classes inherit the attributes of the CIM_ManagedSystemElement
class. These two classes are also part of the CIM core model. The WBEM standard calls these
classes abstract classes because they exist solely as classes that other classes inherit (that is, no
Copyrighted material.
Chapter 4:
Management Mechanisms
241
object instances of an abstract class exist). You can therefore think of abstract classes as templates that define properties for use in other classes.
A second category of classes represents objects that are specific to management areas but
independent of a particular implementation. These classes constitute the common model and
are considered an extension of the core model. An example of a common-model class is the
CIM_FileSystem class, which inherits the attributes of CIM_LogicalElement. Because virtually
every operating system—including Windows, Linux, and other varieties of UNIX—rely on filesystem-based structured storage, the CIM_FileSystem class is an appropriate constituent of
the common model.
The final class category, the extended model, comprises technology-specific additions to the
common model. Windows defines a large set of these classes to represent objects specific to
the Windows environment. Because all operating systems store data in files, the CIM common
model includes the CIM_LogicalFile class. The CIM_DataFile class inherits the
CIM_LogicalFile class, and Windows adds the Win32_PageFile and Win32_ShortcutFile file
classes for those Windows file types.
The Event Log provider makes extensive use of inheritance. Figure 4-20 shows a view of the
WMI CIM Studio, a class browser that ships with the WMI Administrative Tools that you can
obtain from the Microsoft download center at the Microsoft Web site. You can see where the
Event Log provider relies on inheritance in the provider’s Win32_NTEventlogFile class,
which derives from CIM_DataFile. Event Log files are data files that have additional Event
Log–specific attributes such as a log file name (LogfileName) and a count of the number of
records that the file contains (NumberOfRecords). The tree that the class browser shows
reveals that Win32_NTEventlogFile is based on several levels of inheritance, in which
CIM_DataFile derives from CIM_LogicalFile, which derives from CIM_LogicalElement, and
CIM_LogicalElement derives from CIM_ManagedSystemElement.
As stated earlier, WMI provider developers write their classes in the MOF language. The following output shows the definition of the Event Log provider’s Win32_NTEventlogFile,
which is selected in Figure 4-20. Notice the correlation between the properties that the right
panel in Figure 4-20 lists and those properties’ definitions in the MOF file below. CIM Studio
uses yellow arrows to tag those properties that a class inherits. Thus, you don’t see those properties specified in Win32_NTEventlogFile’s definition.
dynamic: ToInstance, provider(“MS_NT_EVENTLOG_PROVIDER”), Locale(1033), UUID(“{8502C57B5FBB-11D2-AAC1-006008C78BC7}”)]
class Win32_NTEventlogFile : CIM_DataFile
{
[read] string LogfileName;
[read, write] uint32 MaxFileSize;
[read] uint32 NumberOfRecords;
[read, volatile, ValueMap{"0", “1..365", “4294967295"}] string OverWritePolicy;
[read, write, Units(“Days”), Range(“0-365 | 4294967295”)] uint32 OverwriteOutDated;
[read] string Sources[];
Copyrighted material.
242
Microsoft Windows Internals, Fourth Edition
[implemented, Privileges{"SeSecurityPrivilege",
“SeBackupPrivilege"}] uint32 ClearEventlog([in] string ArchiveFileName);
[implemented, Privileges{"SeSecurityPrivilege",
“SeBackupPrivilege"}] uint32 BackupEventlog([in] string ArchiveFileName);
};
Figure 4-20
WMI CIM Studio
One term worth reviewing is dynamic, which is a descriptive designator for the
Win32_NTEventlogFile class that the MOF file in the preceding output shows. Dynamic
means that the WMI infrastructure asks the WMI provider for the values of properties associated with an object of that class whenever a management application queries the object’s
properties. A static class is one in the WMI repository; the WMI infrastructure refers to the
repository to obtain the values instead of asking a provider for the values. Because updating
the repository is a relatively expensive operation, dynamic providers are more efficient for
objects that have properties that change frequently.
Copyrighted material.
Chapter 4:
Management Mechanisms
243
EXPERIMENT: Viewing the MOF Definitions of WMI Classes
You can view the MOF definition for any WMI class by using the WbemTest tool that
comes with Windows. In this experiment, we’ll look at the MOF definition for the
Win32_NTEventLogFile class:
1. Run Wbemtest from the Start menu’s Run dialog box.
2. Click the Connect button, change the Namespace to root\cimv2, and connect.
3. Select Enum Classes, select the Recursive option button, and then click OK.
4. Find Win32_NTEventLogFile in the list classes, and double-click it to see its class
properties.
5. Click the Show MOF button to open a window that displays the MOF text.
After constructing classes in MOF, WMI developers can supply the class definitions to WMI in
several ways. WDM provider developers compile a MOF file into a binary MOF (BMF) file—a
more compact binary representation than a MOF file—and give the BMF files to the WDM
infrastructure. Another way is for the provider to compile the MOF and use WMI COM APIs
to give the definitions to the WMI infrastructure. Finally, a provider can use the MOF Compiler (Mofcomp.exe) tool to give the WMI infrastructure a classes-compiled representation
directly.
The WMI Namespace
Classes define the properties of objects, and objects are class instances on a system. WMI uses
a namespace that contains several subnamespaces that WMI arranges hierarchically to organize objects. A management application must connect to a namespace before the application
can access objects within the namespace.
WMI names the namespace root directory root. All WMI installations have four predefined
namespaces that reside beneath root: CIMV2, Default, Security, and WMI. Some of these
namespaces have other namespaces within them. For example, CIMV2 includes the Applications and ms_409 namespaces as subnamespaces. Providers sometimes define their own
namespaces; you can see the WMI namespace (which the Windows device driver WMI provider defines) beneath root in Windows.
EXPERIMENT: Viewing WMI Namespaces
You can see what namespaces are defined on a system with WMI CIM Studio. WMI
CIM Studio presents a connection dialog box when you run it that includes a
namespace browsing button to the right of the namespace edit box. Opening the
browser and selecting a namespace has WMI CIM Studio connect to that namespace.
Copyrighted material.
244
Microsoft Windows Internals, Fourth Edition
Windows Server 2003 defines over a dozen namespaces beneath root, some of which
are visible here:
Unlike a file system namespace, which comprises a hierarchy of directories and files, a WMI
namespace is only one level deep. Instead of using names as a file system does, WMI uses
object properties that it defines as keys to identify the objects. Management applications specify class names with key names to locate specific objects within a namespace. Thus, each
instance of a class must be uniquely identifiable by its key values. For example, the Event Log
provider uses the Win32_NTLogEvent class to represent records in an Event Log. This class
has two keys: Logfile, a string; and RecordNumber, an unsigned integer. A management application that queries WMI for instances of Event Log records obtains them from the provider
key pairs that identify records. The application refers to a record using the syntax that you see
in this sample object pathname:
\\DARYL\root\CIMV2:Win32_NTLogEvent.Logfile="Application",
RecordNumber="1”
The first component in the name (\\DARYL) identifies the computer on which the object is
located, and the second component (\root\CIMV2) is the namespace in which the object
resides. The class name follows the colon, and key names and their associated values follow
the period. A comma separates the key values.
WMI provides interfaces that let applications enumerate all the objects in a particular class or
to make queries that return instances of a class that match a query criteria.
Class Association
Many object types are related to one another in some way. For example, a computer object has
a processor, software, an operating system, active processes, and so on. WMI lets providers
construct an association class to represent a logical connection between two different classes.
Association classes associate one class with another, so the classes have only two properties: a
class name and the Ref modifier. The following output shows an association in which the
Copyrighted material.
Chapter 4:
Management Mechanisms
245
Event Log provider’s MOF file associates the Win32_NTLogEvent class with the
Win32_ComputerSystem class. Given an object, a management application can query associated objects. In this way, a provider defines a hierarchy of objects.
[dynamic: ToInstance, provider("MS_NT_EVENTLOG_PROVIDER"): ToInstance, EnumPrivileges{"SeSec
urityPrivilege"}: ToSubClass, Locale(1033): ToInstance, UUID("{8502C57F-5FBB-11D2-AAC1006008C78BC7}"): ToInstance, Association: DisableOverride ToInstance ToSubClass]
class Win32_NTLogEventComputer
{
[key, read: ToSubClass] Win32_ComputerSystem ref Computer;
[key, read: ToSubClass] Win32_NTLogEvent ref Record;
};
Figure 4-21 shows the WMI Object Browser (another tool that the WMI Administrative Tools
includes) displaying the contents of the CIMV2 namespace. Windows system components
typically place their objects within the CIMV2 namespace. The Object Browser first locates
the Win32_ComputerSystem object instance MR-XEON, which is the object that represents
the computer. Then the Object Browser obtains the objects associated with
Win32_ComputerSystem and displays them beneath MR-XEON. The Object Browser user
interface displays association objects with a double-arrow folder icon. The associated class
type’s objects display beneath the folder.
Figure 4-21
WMI Object Browser
You can see in the Object Browser that the Event Log provider’s association class
Win32_NTLogEventComputer is beneath MR-XEON and that numerous instances of the
Win32_NTLogEvent class exist. Refer to the preceding output to verify that the MOF file
defines the Win32_NTLogEventComputer class to associate the Win32_ComputerSystem
class with the Win32_NTLogEvent class. Selecting an instance of Win32_NTLogEvent in the
Object Browser reveals that class’s properties under the Properties tab in the right-hand pane.
Microsoft intended the Object Browser to help WMI developers examine their objects, but a
management application would perform the same operations and display properties or collected information more intelligibly.
Copyrighted material.
246
Microsoft Windows Internals, Fourth Edition
EXPERIMENT: Using WMI Scripts to Manage Systems
A powerful aspect of WMI is its support for scripting languages. Microsoft has generated
hundreds of scripts that perform common administrative tasks for managing user
accounts, files, the registry, processes, and hardware devices. While some scripts ship in
the Windows Resource Kits, the Microsoft TechNet Scripting Center Web site serves as
the central location for Microsoft scripts. Using a script from the scripting center is as
easy as copying its text from your Internet browser, storing it in a file with with a .vbs
extension, and running it with the command cscript script.vbs, where “script” is the
name you gave the script. Cscript is the command-line interface to Windows Script Host
(WSH).
Here’s a sample TechNet script that registers to receive events when Win32_Process
object instances are created, which occurs whenever a process starts, and prints a line
with the name of the process that the object represents:
strComputer = "."
Set objWMIService = GetObject("winmgmts:" _
& "{impersonationLevel=impersonate}!\\" & strComputer & "\root\cimv2")
Set colMonitoredProcesses = objWMIService. _
ExecNotificationQuery("select * from __instancecreationevent " _
& " within 1 where TargetInstance isa 'Win32_Process'")
i = 0
Do While i = 0
Set objLatestProcess = colMonitoredProcesses.NextEvent
Wscript.Echo objLatestProcess.TargetInstance.Name
Loop
The line that invokes ExecNotificationQuery does so with a parameter that includes a
“select” statement, which highlights WMI’s support for a read-only subset of the ANSI
standard Structured Query Language (SQL), known as WQL, to provide a flexible way
for WMI consumers to specify the information that they want to extract from WMI providers. Running the sample script with Cscript and then starting Notepad results in the
following output:
C:\>cscript monproc.vbs
Microsoft (R) Windows Script Host Version 5.6
Copyright (C) Microsoft Corporation 1996-2001. All rights reserved.
NOTEPAD.EXE
Copyrighted material.
Chapter 4:
Management Mechanisms
247
WMI Implementation
In Windows 2000, the WMI service is implemented in \Windows\System32\Winmgmt.exe,
which the Windows SCM starts the first time a management application or WMI provider
tries to access WMI APIs. In Windows XP and Windows Server 2003, the WMI service runs in
a shared Svchost process that executes in the local system account.
In Windows 2000, WMI loads providers as in-process DCOM servers that execute within the
Winmgmt service process. If a provider bug crashes the WMI process, the WMI service exits and
then restarts in response to the next WMI request. Because the WMI service shares its Svchost
process with several other services that would also exit if a WMI provider bug caused the process to exit, in Windows XP and Windows Server 2003, WMI loads providers into the Wmiprvse.exe provider-hosting process. Wmiprvse.exe launches as a child of the RPC service
process. WMI executes Wmiprvse in the local system, local service, or network service accounts,
depending on the value of the HostingModel property of the WMI Win32Provider object
instance that represents the provider implementation. A Wmiprvse process exits after the provider is removed from the cache, one minute following the last provider request it receives.
EXPERIMENT: Viewing Wmiprvse Creation
You can see Wmiprvse being created by running Process Explorer from www.sysinternals.com and executing Wmic. A Wmiprvse process will appear beneath the Svchost process that hosts the RPC service. If Process Explorer job highlighting is enabled, it will
appear with the job highlight color because, to prevent a runaway provider from consuming all virtual memory resources on a system, Wmiprvse executes in a job object
that limits the number of child processes it can create and the amount of virtual memory
each process and all the processes of the job can allocate. (See Chapter 6 for more information on job objects.)
Copyrighted material.
248
Microsoft Windows Internals, Fourth Edition
Most WMI components reside by default in \Windows\System32 and \Windows\System32\
Wbem, including Windows MOF files, built-in provider DLLs, and management application
WMI DLLs. Look in the \Windows\System32\Wbem directory, and you’ll find Ntevt.mof,
the Event Log provider MOF file. You’ll also find Ntevt.dll, the Event Log provider’s DLL,
which the WMI service uses.
Directories beneath \Windows\System32\Wbem store the repository, log files, and thirdparty MOF files. WMI implements the repository—named the CIMOM repository—using a
proprietary version of the Microsoft JET database engine. In Windows 2000, the database file
stores in \Windows\System32\Wbem\Repository\Cim.rep; in Windows XP and Windows
Server 2003, the database resides in \Windows\System32\Wbem\Repository\Fs.
WMI honors numerous registry settings (including various internal performance parameters
such as CIMOM backup locations and intervals in Windows 2000) that the service’s
HKLM\SOFTWARE\Microsoft\WBEM\CIMOM registry key stores.
Device drivers use special interfaces to provide data to and accept commands—called the WMI
System Control commands—from WMI. These interfaces are part of the WDM, which is
explained in Chapter 9. Because the interfaces are cross-platform, they fall under the
\root\WMI namespace.
WMIC
Windows XP and Windows Server 2003 include Wmic.exe, a utility that allows you to
interact with WMI from a WMI-aware command-line shell. All WMI objects and their
properties, including their methods, are accessible through the shell, which makes
WMIC an advanced systems management console.
WMI Security
WMI implements security at the namespace level. If a management application successfully
connects to a namespace, the application can view and access the properties of all the objects
in that namespace. An administrator can use the WMI Control application to control which
users can access a namespace. To start the WMI Control application, from the Start menu,
select Programs, Administrative Tools, Computer Management. Next, open the Services And
Applications branch. Right-click WMI Control, and select Properties to launch the WMI Control Properties dialog box, which Figure 4-22 shows. To configure security for namespaces,
click the Security tab, select the namespace, and click Security. The other tabs in the WMI
Control Properties dialog box let you modify the performance and backup settings that the
registry stores.
Copyrighted material.
Chapter 4:
Figure 4-22
Management Mechanisms
249
WMI security properties
Conclusion
So far, we’ve examined the overall structure of Windows and the core system mechanisms on
which the structure is built, and core management mechanisms. With this foundation laid,
we're ready to explore the boot process and the individual executive components in more
detail.
Copyrighted material.
Copyrighted material.
Chapter 5
Startup and Shutdown
In this chapter, we’ll describe the steps required to boot Microsoft Windows and the options
that can affect system startup. Understanding the details of the boot process will help you
diagnose problems that can arise during a boot. Then we’ll explain the kinds of things that
can go wrong during the boot process and how to resolve them. Finally, we’ll explain what
occurs on an orderly system shutdown.
Boot Process
In describing the Windows boot process, we’ll start with the installation of Windows and proceed through the execution of boot support files. Device drivers are a crucial part of the boot
process, so we’ll explain the way that they control the point in the boot process at which they
load and initialize. Then we’ll describe how the executive subsystems initialize and how the
kernel launches the user-mode portion of Windows by starting the Session Manager process
(Smss.exe), the Windows subsystem, and the logon process (Winlogon). Along the way, we’ll
highlight the points at which various text appears on the screen to help you correlate the internal process with what you see when you watch Windows boot.
The early phases of the boot process differ significantly on x86 and x64 systems versus IA64
systems. The next sections describe the portions of the boot process specific to x86 and x64
and follow with a section describing the IA64-specific portions of the boot process.
x86 and x64 Preboot
The Windows boot process doesn’t begin when you power on your computer or press the
reset button. It begins when you install Windows on your computer. At some point during the
execution of the Windows Setup program, the system’s primary hard disk is prepared with
code that takes part in the boot process. Before we get into what this code does, let’s look at
how and where Windows places the code on a disk. Since the early days of MS-DOS, a standard has existed on x86 systems for the way physical hard disks are divided into volumes.
Microsoft operating systems split hard disks into discrete areas known as partitions and use
file systems (such as FAT and NTFS) to format each partition into a volume. A hard disk can
Copyrighted material.
251
252
Microsoft Windows Internals, Fourth Edition
contain up to four primary partitions. Because this apportioning scheme would limit a disk
to four volumes, a special partition type, called an extended partition, further allocates up to
four additional partitions within each primary partition. Extended partitions can contain
extended partitions, which can contain extended partitions, and so on, making the number
of volumes an operating system can place on a disk effectively infinite. Figure 5-1 shows an
example of a hard disk layout, and Table 5-1 summarizes the files involved in the x86 and
x64 boot process. (You can learn more about Windows partitioning in Chapter 10, which
covers storage management.)
Table 5-1 x86 and x64 Boot Process Components
Component
Processor Execution
Responsibilities
Master Boot Record (MBR)
code
16-bit real mode
Reads and loads partition boot sectors.
Boot sector
16-bit real mode
Reads the root directory to load Ntldr.
Ntldr
16-bit real mode and
Reads Boot.ini, presents boot menu, and
32-bit or 64-bit protected loads Ntoskrnl.exe, Bootvid.dll, Hal.dll,
mode; turns on paging
and boot-start device drivers. If a 32-bit
installation is booted, switches to 32-bit
protected mode; if a 64-bit installation is
booted, switches to 64-bit long mode.
Ntdetect.com
16-bit real mode
Performs hardware detection for Ntldr.
Ntbootdd.sys
Protected mode
Device driver used for disk I/O on SCSI
and Advanced Technology Attachment
(ATA) systems where the BIOS is not
used.
Ntoskrnl.exe
Protected mode with
paging
Initializes executive subsystems and
boot and system-start device drivers,
prepares the system for running native
applications, and runs Smss.exe.
Hal.dll
Protected mode with
paging
Kernel-mode DLL that interfaces
Ntoksnrl and drivers to the hardware.
Smss
Native application
Loads Windows subsystem, including
Win32k.sys and Csrss.exe, and starts
Winlogon process.
Winlogon
Native application
Starts the service control manager
(SCM), starts the Local Security Subsystem (LSASS), and presents interactive
logon dialog box.
Service control manager
(SCM)
Native application
Loads and initializes auto-start device
drivers and Windows services.
Copyrighted material.
Chapter 5:
Startup and Shutdown
253
Boot code
1
2
3
4
Partition table
Partitions within an
extended partition
Boot partition
Partition 1
MBR
Figure 5-1
Partition 2
Boot sector
Partition 3
(Extended)
Partition 4
Extended partition boot record
Sample hard disk layout
Physical disks are addressed in units known as sectors. A hard disk sector on an IBM-compatible PC is typically 512 bytes. Utilities that prepare hard disks for the definition of volumes,
including the MS-DOS Fdisk utility or the Windows Setup program, write a sector of data
called a Master Boot Record (MBR) to the first sector on a hard disk. (MBR partitioning is
described in Chapter 10.) The MBR includes a fixed amount of space that contains executable
instructions (called boot code) and a table (called a partition table) with four entries that define
the locations of the primary partitions on the disk. When an IBM-compatible computer boots,
the first code it executes is called the BIOS, which is encoded into the computer’s ROM. The
BIOS selects a boot device, reads that device’s MBR into memory, and transfers control to the
code in the MBR.
The MBRs written by Microsoft partitioning tools, such as the one integrated into Windows
Setup and the Disk Management MMC snap-in, go through a similar process of reading and
transferring control. First, an MBR’s code scans the primary partition table until it locates a
partition containing a flag that signals the partition is bootable. When the MBR finds at least
one such flag, it reads the first sector from the flagged partition into memory and transfers
control to code within the partition. This type of partition is called a boot partition, and the
first sector of such a partition is called a boot sector. The volume defined for the boot partition
is called the system volume.
Copyrighted material.
254
Microsoft Windows Internals, Fourth Edition
Operating systems generally write boot sectors to disk without a user’s involvement. For
example, when Windows Setup writes the MBR to a hard disk, it also writes a boot sector to
the first bootable partition of the disk. You might have created a MS-DOS boot sector during
the installation of MS-DOS, Windows Me, Windows 98, or Windows 95. Windows Setup
checks to see whether the boot sector it will overwrite with a Windows boot sector is a valid
MS-DOS boot sector. If it is, Windows Setup copies the boot sector’s contents to a file named
Bootsect.dos in the root directory of the partition.
Before writing to a partition’s boot sector, Windows Setup ensures that the partition is formatted with a file system that Windows supports (FAT, FAT32, or NTFS) by formatting the boot
partition (and any other partition) with a file system type you specify. If partitions are already
formatted, you can instruct Setup to skip this step. After Setup formats the boot partition,
Setup copies the files Windows uses to the boot partition (the system volume), including two
files that are part of the boot sequence, Ntldr and Ntdetect.com.
Another of Setup’s roles is to create a boot menu file, Boot.ini, in the root directory of the system volume. This file contains options for starting the version of Windows that Setup installs
and any preexisting Windows installations. If Bootsect.dos contains a valid MS-DOS boot sector, one of the entries Boot.ini creates is to boot into MS-DOS. The following output shows a
sample Boot.ini file from a dual-boot computer on which MS-DOS is installed before Windows XP:
[boot loader]
timeout=30
default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
[operating systems]
multi(0)disk(0)rdisk(0)partition(1) \WINDOWS="Microsoft Windows XP Professional" /fastdetect
C:\="Microsoft Windows"
You’ll notice in the sample file that the path to the Windows directory is specified in a special
syntax that conforms to the Advanced RISC Computing (ARC) naming convention. There are
three variants to the syntax used by Windows. The first, shown in the preceding code sample,
is the multi() syntax, which instructs Windows to use BIOS INT 13 functions to load system
files. Thus, the multi() syntax is present when the disk on which the boot volume is located
has a controller that provides INT-13 support. The multi() syntax follows this format:
multi(W)disk(X)rdisk(Y)partition(Z)
W is the disk controller number (also known as the ordinal number) and is typically 0. X is
always 0 in the multi() syntax. Y specifies the physical hard disk attached to controller W. For
ATA controllers, this number is typically between 0 and 3. For SCSI controllers, the number is
typically between 0 and 15. Z indicates the partition number on the physical disk that corresponds to the boot volume. The first partition is assigned the number 1.
Copyrighted material.
Chapter 5:
Startup and Shutdown
255
The scsi() ARC syntax informs Windows that it should rely on disk I/O services provided by
Ntbootdd.sys (described shortly) to access the files on the boot volume. The format of the
scsi() syntax is:
scsi(W)disk(X)rdisk(Y)partition(Z)
In this syntax, W is the controller number, and X is the physical hard disk attached to the controller and is typically between 0 and 15. Y specifies the SCSI logical unit number (LUN) of
the disk that contains the boot volume and is typically 0. Finally, Z is the partition that corresponds to the boot volume with numbering starting at 1.
The final syntax used by Windows is the signature() syntax. This instructs Windows to locate
the disk with the signature that matches the first value in parentheses, regardless of the controller number associated with the disk and to use Ntbootdd.sys to access the boot volume. A
disk signature is a globally unique identifier (GUID) that Windows Setup extracts from information in the MBR and writes to the disk. The signature() syntax is as follows:
signature(V)disk(X)rdisk(Y)partition(Z)
V is a 32-bit hexadecimal disk signature that identifies the disk. X is the physical hard disk
with the specific signature, and it can be attached to any controller on the system. Y is always
0, and Z is the partition number on which the boot volume is located.
Windows uses the signature() syntax in the following cases:
■
The boot volume is larger than 7.8 GB in size, and BIOS extended INT-13 functions
(those used to access parts of a disk beyond 7.8 GB) cannot access the entire volume.
■
The BIOS does not support extended INT-13 functions.
The x86/x64 Boot Sector and Ntldr
Setup must know the partition format before it writes a boot sector because the contents of
the boot sector vary depending on the format. For example, if the boot partition is a FAT partition, Windows writes code to the boot sector that understands the FAT file system. But if the
partition is in NTFS format, Windows writes NTFS-capable code. The role of the boot-sector
code is to give Windows information about the structure and format of a volume and to read
in the Ntldr file from the root directory of the volume. Thus, the boot-sector code contains just
enough read-only file system code to accomplish this task. After the boot-sector code loads
Ntldr into memory, it transfers control to Ntldr’s entry point. If the boot-sector code can’t find
Ntldr in the volume’s root directory, it displays the error message “BOOT: Couldn’t find
NTLDRP” if the boot file system is FAT or “NTLDR is missing” if the file system is NTFS.
Copyrighted material.
256
Microsoft Windows Internals, Fourth Edition
Ntldr begins its existence while a system is executing in an x86 operating mode called real mode.
In real mode, no virtual-to-physical translation of memory addresses occurs, which means that
programs that use the memory addresses interpret them as physical addresses and that only the
first 1 MB of the computer’s physical memory is accessible. Simple MS-DOS programs execute
in a real-mode environment. However, the first action Ntldr takes is to switch the system to protected mode. Still no virtual-to-physical translation occurs at this point in the boot process, but a
full 32 bits of memory becomes accessible. After the system is in protected mode, Ntldr can
access all of physical memory. After creating enough page tables to make memory below 16 MB
accessible with paging turned on, Ntldr enables paging. Protected mode with paging enabled is
the mode in which Windows executes in normal operation.
After Ntldr enables paging, it is fully operational. However, it still relies on functions supplied
by the boot code to access IDE-based system and boot disks as well as the display. The bootcode functions briefly switch off paging and switch the processor back to a mode in which services provided by the BIOS can be executed. If the disk containing the boot volume is SCSIbased and is not accessible using BIOS firmware support, Ntldr loads a file named
Ntbootdd.sys and uses it instead of the boot-code functions for disk access. Ntbootdd.sys is a
copy of the SCSI miniport driver that Windows uses when its fully operation to access the
boot disk. (See Chapter 10 for more information on disk drivers.) Ntldr next reads the
Boot.ini file from the root directory using built-in file system code. Like the boot sector’s code,
Ntldr contains read-only NTFS and FAT code; unlike the boot sector’s code, however, Ntldr’s
file system code can read subdirectories.
Ntldr next clears the screen. If there is a valid Hiberfil.sys file in the root of the system volume,
it shortcuts the boot process by reading the contents of the file into memory and transferring
control to code in the kernel that resumes a hibernated system. That code is responsible for
restarting drivers that were active when the system was shut down. Hiberfil.sys will be valid
only if the last time the computer was shut down it was hibernated. (See the section “The
Power Manager” in Chapter 11 for information on hibernation.)
If there is more than one boot-selection entry in Boot.ini, it presents the user with the boot-selection menu. (If there is only one entry, Ntldr bypasses the menu and proceeds to displaying the
startup progress bar.) Selection entries in Boot.ini direct Ntldr to the partition on which the
Windows system directory (typically \Windows) of the selected installation resides. This partition might be the same as the boot partition, or it might be another primary partition.
If the Boot.ini entry refers to an MS-DOS installation (that is, by referring to C:\ as the system
partition), Ntldr reads the contents of the Bootsect.dos file into memory, switches back to 16bit real mode, and calls the MBR code in Bootsect.dos. This action causes the Bootsect.dos
code to execute as if the MBR had read the code from disk. Code in Bootsect.dos continues an
MS-DOS-specific boot, such as is used to boot Microsoft Windows Me, Windows 98, or Windows 95 on a computer on which these operating systems are installed with Windows.
Copyrighted material.
Chapter 5:
Startup and Shutdown
257
Entries in Boot.ini can include optional arguments that Ntldr and other components involved
in the boot process interpret. Table 5-2 contains a complete list of these options and their
effects. The Bootcfg.exe tool, introduced in Windows XP, provides a convenient interface to
setting a number of the switches. Any options that are included on the Boot.ini save to the
Registry value HKLM\System\CurrentControlSet\Control\SystemStartOptions.
Table 5-2
Boot Options
Boot Qualifier
Meaning
/3GB
Increases the size of the user process address space from 2 GB to
3 GB (and therefore reduces the size of system space from 2 GB
to 1 GB). Giving virtual-memory-intensive applications such as
database servers a larger address space can improve their performance. For an application to take advantage of this feature,
however, two additional conditions must be met: the system
must be running Windows XP, Windows Server 2003, Windows
2000 Advanced Server, or Datacenter Server; and the application
.exe must be flagged as a 3-GB-aware application (applies to 32bit systems only). (See the section “Address Space Layout” in
Chapter 7 for more information.)
/BASEVIDEO
Causes Windows to use the standard VGA display driver for GUImode operations.
/BAUDRATE=
Enables kernel-mode debugging, and specifies an override for
the default baud rate (19200) at which a remote kernel debugger host will connect. Example: /BAUDRATE=115200.
/BOOTLOG
Causes Windows to write a log of the boot to the file %SystemRoot%\Ntbtlog.txt.
/BOOTLOGO
Use this switch to have Windows XP or Windows Server 2003 display an installable splash screen instead of the standard splash
screen. First, create a 16-color (any 16 colors) 640x480 bitmap and
save it in the Windows directory with the name Boot.bmp. Then
add "/bootlogo /noguiboot" to the boot.ini selection.
/BREAK
Causes the hardware abstraction layer (HAL) to stop at a breakpoint at HAL initialization. The first thing the Windows kernel does
when it initializes is to initialize the HAL, so this breakpoint is the
earliest one possible. The HAL will wait indefinitely at the breakpoint until a kernel-debugger connection is made. If the switch is
used without the /DEBUG switch, the system will elicit a blue
screen with a STOP code of 0x00000078 (PHASE0_ EXCEPTION).
/BURNMEMORY=
Specifies an amount of memory Windows can’t use (similar to
the /MAXMEM switch). The value is specified in megabytes.
Example: /BURNMEMORY=128 would indicate that Windows
can’t use 128 MB of the total physical memory on the machine.
Copyrighted material.
258
Microsoft Windows Internals, Fourth Edition
Table 5-2
Boot Options
Boot Qualifier
Meaning
/CHANNEL=
Used in conjunction with /DEBUGPORT=1394 to specify the IEEE
1394 channel through which kernel debugging communications
will flow. This can be any number between 0 and 62 and defaults
to 0 if not set.
/CLKLVL
Causes the standard x86 multiprocessor HAL (Halmps.dll) to
configure itself for a level-sensitive system clock rather than an
edge-triggered clock. Level-sensitive and edge-triggered are
terms used to describe hardware interrupt types.
/CMDCONS
Passed when booting into the Recovery Console (described later
in this chapter).
/CRASHDEBUG
Causes the kernel debugger to be loaded when the system
boots, but to remain inactive unless a crash occurs. This allows
the serial port that the kernel debugger would use to be available for use by the system until the system crashes (vs. /DEBUG,
which causes the kernel debugger to use the serial port for the
life of the system session).
/DEBUG
Enables kernel-mode debugging.
/DEBUGPORT=
Enables kernel-mode debugging, and specifies an override for the
default serial (usually COM2 on systems with at least two serial
ports) to which a remote kernel-debugger host is connected.
Windows XP and Windows Server 2003 also support debugging
through accepted IEEE 1394 ports. Examples:
/DEBUGPORT=COM2, /DEBUGPORT=1394.
/EXECUTE
Disables no-execute protection. See the /NOEXECUTE switch for
more information.
/FASTDETECT
Default boot option for Windows. Replaces the Windows NT 4
switch /NOSERIALMICE. The reason the qualifier exists (vs. just
having NTDETECT perform this operation by default) is so that
NTDETECT can support booting Windows NT 4. Windows Plug
and Play device drivers perform detection of parallel and serial
devices, but Windows NT 4 expects NTDETECT to perform the
detection. Thus, specifying /FASTDETECT causes NTDETECT to
skip parallel and serial device enumeration (actions that are not
required when booting Windows), whereas omitting the switch
causes NTDETECT to perform this enumeration (which is required for booting Windows NT 4).
/INTAFFINITY
Directs the standard x86 multiprocessor HAL (Halmps.dll) to set
interrupt affinities such that only the highest numbered processor will receive interrupts. Without the switch, the HAL defaults
to its normal behavior of letting all processors receive interrupts.
Copyrighted material.
Chapter 5:
Table 5-2
Startup and Shutdown
259
Boot Options
Boot Qualifier
Meaning
/KERNEL= /HAL=
Enables you to override Ntldr’s default filename for the kernel
image (Ntoskrnl.exe) and/or the HAL (Hal.dll). These options are
useful for alternating between a checked kernel environment
and a free (retail) kernel environment or even to manually select
a different HAL. If you want to boot a checked environment that
consists solely of the checked kernel and HAL, which is typically
all that is needed to test drivers, follow these steps on a system
installed with the free build:
1. Copy the checked versions of the kernel images from the
checked build CD to your \Windows\System32 directory,
giving the images different names than the default. For
example, if you’re on a uniprocessor, copy Ntoskrnl.exe to
Ntoschk.exe and Ntkrnlpa.exe to Ntoschkpa.exe. If you’re
on a multiprocessor, copy Ntkrnlmp.exe to Ntoschk.exe
and Ntkrpamp.exe to Ntoschkpa.exe. The kernel filename
must be an 8.3-style short name.
2. Copy the checked version of the appropriate HAL needed
for your system from \I386\Driver.cab on the checked
build CD to your \Windows\System32 directory, naming it
Halchk.dll. To determine which HAL to copy, open \Windows\Repair\Setup.log and search for Hal.dll; you’ll find a
line like \WINDOWS\system32\ hal.dll="halacpi.dll","1d8a1". The name immediately to the right of the
equals sign is the name of the HAL you should copy. The
HAL filename must be an 8.3-style short name.
3. Make a copy of the default line in the system’s Boot.ini file.
4. In the string description of the boot selection, add something that indicates that the new selection will be for a
checked build environment (for example, “Windows XP
Professional Checked”).
5. Add the following to the end of the new selection’s line:
/KERNEL=NTOSCHK.EXE /HAL= HALCHK.DLL
Now when the selection menu appears during the boot process,
you can select the new entry to boot a checked environment or
select the entry you were using to boot the free build.
/LASTKNOWNGOOD
Causes the system to boot as if the LastKnownGood boot option
was selected.
/MAXMEM=
Limits Windows to ignore (not use) physical memory beyond the
amount indicated. The number is interpreted in megabytes. Example: /MAXMEM=32 would limit the system to using the first
32 MB of physical memory even if more were present.
Copyrighted material.
260
Microsoft Windows Internals, Fourth Edition
Table 5-2 Boot Options
Boot Qualifier
Meaning
/MAXPROCSPERCLUSTER=
For the standard x86 multiprocessor HAL (Halmps.dll), forces
cluster-mode Advanced Programmable Interrupt Controller
(APIC) addressing (not supported on systems with an 82489DX
external APIC interrupt controller).
/MININT
This option is used by Windows PE (Preinstallation Environment)
and causes the Configuration Manager to load the Registry
SYSTEM hive as a volatile hive such that changes made to it in
memory are not saved back to the hive image.
/NODEBUG
Prevents kernel-mode debugging from being initialized. Overrides the specification of any of the three debug-related switches,
/DEBUG, /DEBUGPORT, and /BAUDRATE.
/NOEXECUTE
This option is available only on 32-bit versions of Windows when
running on AMD64 processors and only when PAE (explained
further in the /PAE switch entry) is also enabled. It enables noexecute protection, which results in the Memory Manager marking pages containing data as no-execute so that they cannot be
executed as code. This can be useful for preventing malicious
code from exploiting buffer overflow bugs with unexpected program input in order to execute arbitrary code. No-execute protection is always enabled on 64-bit versions of Windows on
AMD64 processors. There are 4 modifiers that can be specified
to the /NOEXECUTE switch:
=OPTIN,=OPTOUT,=ALWAYSON,=ALWAYSOFF. See Chapter 7
for a description of their behavior.
/NOGUIBOOT
Instructs Windows not to initialize the VGA video driver
responsible for presenting bitmapped graphics during the
boot process. The driver is used to display boot progress information, so disabling it will disable the ability of Windows to
show this information.
/NOLOWMEM
Requires that the /PAE switch be present and that the system have
more than 4 GB of physical memory. If these conditions are met,
the PAE-enabled version of the Windows kernel, Ntkrnlpa.exe,
won’t use the first 4 GB of physical memory. Instead, it will load all
applications and device drivers, and allocate all memory pools,
from above that boundary. This switch is useful only to test
device-driver compatibility with large memory systems.
/NOPAE
Forces Ntldr to load the non–Physical Address Extension (PAE)
version of the Windows kernel, even if the system is detected as
supporting x86 PAEs and has more than 4 GB of physical memory.
/NOSERIALMICE=[COMx |
COMx,y,z...]
Obsolete Windows NT 4 qualifier—replaced by the absence of the
/FASTDETECT switch. Disables serial mouse detection of the
specified COM ports. This switch was used if you had a device
other than a mouse attached to a serial port during the startup
sequence. Using /NOSERIALMICE without specifying a COM port
disables serial mouse detection on all COM ports. See Microsoft
Knowledge Base article Q131976 for more information.
Copyrighted material.
Chapter 5:
Table 5-2
Startup and Shutdown
261
Boot Options
Boot Qualifier
Meaning
/NUMPROC=
Specifies the number of CPUs that can be used on a multiprocessor system. Example: /NUMPROC=2 on a four-way system
will prevent Windows from using two of the four processors.
/ONECPU
Causes Windows to use only one CPU on a multiprocessor
system.
/PAE
Causes Ntldr to load Ntkrnlpa.exe, which is the version of the x86
kernel that is able to take advantage of x86 PAEs. The PAE
version of the kernel presents 64-bit physical addresses to device
drivers, so this switch is helpful for testing device driver support
for large memory systems.
/PCILOCK
Stops Windows from dynamically assigning IO/IRQ resources to
PCI devices and leaves the devices configured by the BIOS. See
Microsoft Knowledge Base article Q148501 for more information.
/RDPATH=
Specifies the path to a System Disk Image (SDI) file, which can be
on the network, that the system will use to boot from. Often
used in conjunction with the /RDIMAGEOFFSET= flag to indicate
to NTLDR where in the file the system image starts.
/REDIRECT
Introduced with Windows Server 2003. Used to cause Windows to
enable Emergency Management Services (EMS), which reports
boot information and accepts system management commands
through a serial port. Specify serial port and baudrate used in
conjunction with EMS with redirect= and redirectbaudrate= lines
in the [boot loader] section of the Boot.ini file.
/SAFEBOOT:
Specifies options for a safe mode boot. You should never have
to specify this option manually because Ntldr specifies it for
you when you use the F8 menu to perform a safe mode boot.
(A safe mode boot is a boot in which Windows loads only
drivers and services that are specified by name or group under
the Minimal or Network registry keys under HKLM\SYSTEM\
CurrentControlSet\Control\SafeBoot.) Following the colon in
the option, you must specify one of three additional switches:
MINIMAL, NETWORK, or DSREPAIR. The MINIMAL and
NETWORK flags correspond to safe mode boot with no network
and safe mode boot with network support, respectively. The
DSREPAIR (Directory Services Repair) switch causes Windows to
boot into a mode in which the Active Directory directory service
is offline and the active directory database unopened. This allows
an administrator to perform diagnostic, repair, or restore functions on the database. An additional option you can append is
(ALTERNATESHELL), which tells Windows to use the program
specified by the HKLM\SYSTEM\CurrentControlSet\SafeBoot\
AlternateShell value as the graphical shell rather than to use the
default, which is Windows Explorer.
Copyrighted material.
262
Microsoft Windows Internals, Fourth Edition
Table 5-2 Boot Options
Boot Qualifier
Meaning
/SCSIORDINAL:
Directs Windows to the SCSI ID of the controller. (Adding a new
SCSI device to a system with an on-board SCSI controller can
cause the controller’s SCSI ID to change.) See Microsoft
Knowledge Base article Q103625 for more information.
/SDIBOOT=
Used in Windows XP Embedded systems to have Windows boot
from a RAM disk image stored in the specified System Disk
Image (SDI) file.
/SOS
Causes Windows to list the device drivers marked to load at boot
time and then to display the system version number (including
the build number), amount of physical memory, and number of
processors.
/TIMERES=
Sets the resolution of the system timer on the standard x86 multiprocessor HAL (Halmps.dll). The argument is a number interpreted in hundreds of nanoseconds, but the rate is set to the
closest resolution the HAL supports that isn’t larger than the one
requested. The HAL supports the following resolutions:
Hundreds of nanoseconds
97660.98
390633.90
Milliseconds (ms)
195322.00
781257.80
The default resolution is 7.8 ms. The system timer resolution
affects the resolution of waitable timers. Example:
/TIMERES=21000 would set the timer to a resolution of 2.0 ms.
/USERVA=
This switch is supported only on Windows XP and Windows
Server 2003. Like the /3GB switch, this switch gives applications
a larger address space. Specify the amount in MB between 2048
and 3072. This switch has the same application requirements as
the /3GB switch and requires that the /3GB switch also be
present (applies to 32-bit systems only).
/WIN95
Directs Ntldr to boot the Consumer Windows boot sector stored
in Bootsect.w40. This switch is pertinent only on a triple-boot
system that has MS-DOS, Consumer Windows, and Windows
installed. See Microsoft Knowledge Base article Q157992 for
more information.
/WIN95DOS
Directs Ntldr to boot the Consumer Windows boot sector stored
in Bootsect.w40. This switch is pertinent only on a triple-boot
system that has MS-DOS, Consumer Windows, and Windows
installed. See Microsoft Knowledge Base article Q157992 for
more information.
/YEAR=
Instructs the Windows core time function to ignore the year that
the computer’s real-time clock reports and instead use the one
indicated. Thus, the year used in the switch affects every piece of
software on the system, including the Windows kernel. Example:
/YEAR=2001. (This switch was created to assist in Y2K testing.)
Copyrighted material.
Chapter 5:
Startup and Shutdown
263
If the user doesn’t select an entry from the selection menu within the timeout period the
Boot.ini file specifies, Ntldr chooses the default selection, which is the top-most entry in
boot.ini with a path matching the path specified in the “default=” line. Once the boot selection
has been made, Ntldr loads and executes Ntdetect.com, a 16-bit real-mode program that uses
a system’s BIOS to query the computer for basic device and configuration information. This
information includes the following:
■
The time and date information stored in the system’s CMOS (nonvolatile memory)
■
The types of buses (for example, ISA, PCI, EISA, Micro Channel Architecture [MCA]) on
the system and identifiers for devices attached to the buses
■
The number, size, and type of disk drives on the system
■
The types of mouse input devices connected to the system
■
The number and type of parallel ports configured on the system
■
The types of video adapters present on the system
This information is gathered into internal data structures that will be stored under the
HKLM\HARDWARE\DESCRIPTION registry key later in the boot.
On Windows 2000, Ntldr then clears the screen and displays the “Starting Windows”
progress bar. This progress bar remains empty until Ntldr begins loading boot drivers. (See
step 5 in the following list.) Below the progress bar is the message, “For troubleshooting and
advanced startup options for Windows, press F8.” If the user presses F8, the advanced boot
menu is presented, which allows the user to select such options as booting from last known
good, safe mode, debug mode, and so on. On Windows XP and Windows Server 2003, Ntldr
presents a logo splash screen instead of a progress bar.
If Ntldr is running on an x64 system and the kernel specified by the entry selected in the boot
menu is for x64, Ntldr switches the processor to long mode, where the native word size is 64bits. Next, Ntldr begins loading the files from the boot volume needed to start the kernel initialization. The boot volume is the volume that corresponds to the partition on which the system directory (usually \Windows) of the installation being booted is located. The steps Ntldr
follows here include:
1. Loads the appropriate kernel and HAL images (Ntoskrnl.exe and Hal.dll by default). If
Ntldr fails to load either of these files, it prints the message “Windows could not start
because the following file was missing or corrupt”, followed by the name of the file.
2. Reads in the SYSTEM registry hive, \Windows\System32\Config\System, so that it can
determine which device drivers need to be loaded to accomplish the boot. (A hive is a file
that contains a registry subtree. You’ll find more details about the registry in Chapter 4)
3. Scans the in-memory SYSTEM registry hive and locates all the boot device drivers. Boot
device drivers are drivers necessary to boot the system. These drivers are indicated in
the registry by a start value of SERVICE_BOOT_START (0). Every device driver has a
registry subkey under HKLM\SYSTEM\CurrentControlSet\Services. For example,
Copyrighted material.
264
Microsoft Windows Internals, Fourth Edition
Services has a subkey named Dmio for the Logical Disk Manager driver, which you can
see in Figure 5-2. (For a detailed description of the Services registry entries, see the section “Services” in Chapter 4)
Figure 5-2
Logical Disk Manager driver service settings
4. Adds the file system driver that’s responsible for implementing the code for the type of
partition (FAT, FAT32, or NTFS) on which the installation directory resides to the list of
boot drivers to load. Ntldr must load this driver at this time; if it didn’t, the kernel would
require the drivers to load themselves, a requirement that would introduce a circular
dependency.
5. Loads the boot drivers, which should only be drivers that, like the file system driver for
the boot volume, would introduce a circular dependency if the kernel was required to
load them. To indicate the progress of the loading, Ntldr updates a progress bar displayed below the text “Starting Windows”. The progress bar moves for each driver
loaded. (It assumes there are 80 boot device drivers—each successful load moves the
progress bar by 1.25 percent.) If the /SOS switch is specified in the Boot.ini selection,
Ntldr doesn’t display the progress bar but instead displays the filenames of each boot
driver. Keep in mind that the drivers are loaded but not initialized at this time—they initialize later in the boot sequence.
6. Prepares CPU registers for the execution of Ntoskrnl.exe.
This action is the end of Ntldr’s role in the boot process. At this point, Ntldr calls the main
function in Ntoskrnl.exe to perform the rest of the system initialization.
The IA64 Boot Process
Table 5-3 lists the files involved in the IA64 boot process. IA64 systems conform to the Extensible Firmware Interface (EFI) specification as defined by Intel. An EFI-compliant system has
firmware that runs boot loader code that’s been programmed into the system’s nonvolatile
RAM (NVRAM) by Windows Setup. The boot code reads the IA64-equivalent of the x86 and
x64 Boot.ini contents, which are also stored in NVRAM. Both Microsoft EFI tools runnable in
the EFI console and Bootcfg.exe, a tool included with Windows, allow for modification of the
NVRAM boot selections and switches.
Copyrighted material.
Chapter 5:
Startup and Shutdown
265
Hardware detection occurs next, where the boot loader uses EFI interfaces to determine the
number and type of the following devices:
■
Network adapters
■
Video adapters
■
Keyboards
■
Disk controllers
■
Storage devices
Just as Ntldr does on x86 and x64 systems, the boot loader then presents a menu of boot
selections with an optional timeout. Once a boot selection is made, the loader navigates to the
subdirectory on the EFI System partition corresponding to the selection and loads several
other files required to continue the boot: Fpswa.efi and Ia64ldr.efi. The EFI specification
requires that the system have a partition designated as the EFI System partition that is formatted with the FAT file system and that is between 100 MB and 1 GB in size or up to one percent
of the size of the disk, and each Windows installation has a subdirectory on the EFI System
partition under EFI\Microsoft. The first installation is assigned the folder Winnt50, the second Winnt50.1, and each subsequent installation has a unique index number following the
period in the folder name. Ia64ldr.efi is responsible for loading Ntoskrnl.exe, Hal.dll, and the
boot-start drivers, after which the boot proceeds through the same steps as for x86 and x64.
Table 5-3
IA64 Boot Process Components
Component
Location
Responsibilities
Fpswa.efi
EFI\Microsoft\Winnt50.x on
the EFI System partition
A file that contains support for EFI
floating-point operations
Ia64ldr.efi
EFI\Microsoft\Winnt50.x on
the EFI System partition
Loads Ntoskrnl.exe, Hal.dll, and boot
drivers
Ntoskrnl.exe
\Windows\System32
Initializes executive subsystems and
boot and system-start device drivers, prepares the system for running
native applications, and runs
Smss.exe
Hal.dll
\Windows\System32
Kernel-mode DLL that interfaces
Ntoksnrl and drivers to the hardware
Service control manager
(SCM)
\Windows\System32
Loads and initializes auto-start device drivers and Windows services
Smss
\Windows\System32
Loads Windows subsystem, including Win32k.sys and Csrss.exe, and
starts Winlogon process
Winlogon
\Windows\System32
Starts the service control manager
(SCM), the Local Security Authority
Subsystem (LSASS), and presents the
interactive logon dialog box
Copyrighted material.
266
Microsoft Windows Internals, Fourth Edition
Initializing the Kernel and Executive Subsystems
When Ntldr calls Ntoskrnl, it passes a data structure that contains a copy of the line in
Boot.ini that represents the selected menu option for this boot, a pointer to the memory tables
Ntldr generated to describe the physical memory on the system, a pointer to the in-memory
copy of the HARDWARE and SYSTEM registry hives, and a pointer to the list of boot drivers
Ntldr loaded.
Ntoskrnl then begins the first of its two-phase initialization process, called phase 0 and phase
1. Most executive subsystems have an initialization function that takes a parameter that identifies which phase is executing.
During phase 0, interrupts are disabled. The purpose of this phase is to build the rudimentary
structures required to allow the services needed in phase 1 to be invoked. Ntoskrnl’s main
function calls KiSystemStartup, which in turn calls HalInitializeProcessor and KiInitializeKernel
for each CPU. KiInitializeKernel, if running on the boot CPU, performs systemwide kernel initialization, such as initializing internal listheads and other data structures that all CPUs share.
Each instance of KiInitializeKernel then calls the function responsible for orchestrating phase
0, ExpInitializeExecutive.
ExpInitializeExecutive starts by calling the HAL function HalInitSystem, which gives the HAL a
chance to gain system control before Windows performs significant further initialization. One
responsibility of HalInitSystem is to prepare the system interrupt controller of each CPU for
interrupts and to configure the interval clock timer interrupt, which is used for CPU time
accounting. (See the section “Quantum Accounting” in Chapter 6 for more on CPU time
accounting.)
Only on the boot processor does ExpInitializeExecutive perform initialization other than calling HalInitSystem. When HalInitSystem returns control, ExpInitializeExecutive on the boot CPU
proceeds by processing the /BURNMEMORY Boot.ini switch (if the switch is present in the
line from the Boot.ini file that corresponds to the menu selection the user made when choosing which installation to boot) and discarding the amount of memory the switch specifies.
Next, ExpInitializeExecutive calls the phase 0 initialization routines for the memory manager,
object manager, security reference monitor, process manager, and Plug and Play manager.
These components perform the following initialization steps:
1. The memory manager constructs page tables and internal data structures that are necessary to provide basic memory services. The memory manager also builds and reserves
an area for the system file cache and creates memory areas for the paged and nonpaged
pools. The other executive subsystems, the kernel, and the device drivers use these two
memory pools for allocating their data structures.
2. During the object manager initialization, the objects that are necessary to construct the
object manager namespace are defined so that other subsystems can insert objects into
it. A handle table is created so that resource tracking can begin.
Copyrighted material.
Chapter 5:
Startup and Shutdown
267
3. The security reference monitor initializes the token type object and then uses the object
to create and prepare the first local system account token for assignment to the initial
process. (See Chapter 8 for a description of the local system account.)
4. The process manager performs most of its initialization in phase 0, defining the process
and thread object types and setting up lists to track active processes and threads. The
process manager also creates a process object for the initial process and names it Idle. As
its last step, the process manager creates the System process and a system thread to execute the routine Phase1Initialization. This thread doesn’t start running right away
because interrupts are still disabled.
5. The Plug and Play manager’s phase 0 initialization then takes place, which involves simply initializing an executive resource used to synchronize bus resources.
When control returns to the KiInitializeKernel function on each processor, control proceeds to
the Idle loop, which then causes the system thread created in step 4 of the previous process
description to begin executing phase 1. (Secondary processors wait to begin their initialization until step 5 of phase 1, described in the following list.) Phase 1 consists of the following
steps. The boot splash screen of Windows 2000 systems includes a progress bar, and the
steps at which the progress bar on the screen is updated are included in this list:
1. HalInitSystem is called to prepare the system to accept interrupts from devices and to
enable interrupts.
2. The boot video driver (\Windows\System32\Bootvid.dll) is called, which in turn displays the Windows startup screen. (On Windows XP and Windows Server 2003 systems, the driver presents the same graphic that Ntldr placed on the screen earlier in the
boot.)
3. The power manager’s initialization is called.
4. The system time is initialized (by calling HalQueryRealTimeClock) and then stored as the
time the system booted.
5. On a multiprocessor system, the remaining processors are initialized and execution
starts.
6. The progress bar is set to 5 percent.
7. The object manager creates the namespace root directory (\), \ObjectTypes directory,
and the DOS device name mapping directory (\?? on Windows 2000, and \Global?? on
Windows XP and Windows Server 2003). It then creates the \DosDevices symbolic link
that points at the DOS device name mapping directory.
8. The executive is called to create the executive object types, including semaphore, mutex,
event, and timer.
9. The kernel initializes scheduler (dispatcher) data structures and the system service dispatch table.
Copyrighted material.
268
Microsoft Windows Internals, Fourth Edition
10. The security reference monitor creates the \Security directory in the object manager
namespace and initializes auditing data structures if auditing is enabled.
11. The progress bar is set to 10 percent.
12. The memory manager is called to create the section object and the memory manager’s
system worker threads (which are explained in Chapter 7).
13. National language support (NLS) tables are mapped into system space.
14. Ntdll.dll is mapped into the system address space.
15. The cache manager initializes the file system cache data structures and creates its worker
threads.
16. The configuration manager creates the \Registry key object in the object manager
namespace and copies the initial registry data passed by Ntldr into the HARDWARE and
SYSTEM hives.
17. Global file system driver data structures are initialized.
18. The Plug and Play manager calls the Plug and Play BIOS.
19. The progress bar is set to 20 percent.
20. The local procedure call (LPC) subsystem initializes the LPC port type object.
21. If the system was booted with boot logging (/BOOTLOG), the boot log file is initialized.
22. The progress bar is set to 25 percent.
23. The I/O manager initialization now takes place. This stage is a complex phase of system
startup that accounts for 50 percent of the “progress” reported in the progress bar. The
I/O manager considers each successful driver load to be another 2 percent of progress
for the boot. (If there are more than 25 drivers to load, the progress bar stops at 75 percent.)
The I/O manager first initializes various internal structures and creates the driver and
device object types. It then calls the Plug and Play manager, power manager, and HAL to
begin the various stages of dynamic device enumeration and initialization. (Because this
process is complex and specific to the I/O system, we’ll save the details for Chapter 9.)
Then the Windows Management Instrumentation (WMI) subsystem is initialized,
which provides WMI support for device drivers. (See the section “Windows Management Instrumentation” in Chapter 4 for more information.) Next, all the boot-start drivers are called to perform their driver-specific initialization, and the system-start device
drivers are loaded and initialized. (Details on the processing of the driver load control
information on the registry are also covered in Chapter 9.) Finally, the MS-DOS device
names are created as symbolic links in the object manager’s namespace.
24. The progress bar is set to 75 percent.
Copyrighted material.
Chapter 5:
Startup and Shutdown
269
25. If the computer is booting in safe mode, this fact is recorded in the registry.
26. Unless explicitly disabled in the registry, paging of kernel-mode code (in Ntoskrnl and
drivers) is enabled.
27. The progress bar is set to 80 percent.
28. The power manager is called to initialize various power management structures.
29. The progress bar is set to 85 percent.
30. The security reference monitor is called to create the Command Server Thread that communicates with Lsass. (See the section “Security System Components” in Chapter 8 for
more on how security is enforced in Windows.)
31. The progress bar is set to 90 percent.
32. The last step is to create the Session Manager subsystem (Smss) process (introduced in
Chapter 2). Smss is responsible for creating the user-mode environment that provides
the visible interface to Windows—its initialization steps are covered in the next section.
33. The progress bar is (finally) set to 100%.
As a final step before considering the executive and kernel initialization complete, the phase 1
initialization thread waits for the handle to the Session Manager process with a timeout value
of 5 seconds. If the Session Manager process exits before the 5 seconds elapse, the system
crashes itself with a SESSION5_ INITIALIZATION_FAILED bug check code.
If the 5-second wait times out (that is, if 5 seconds elapse), the Session Manager is assumed to
have started successfully, and the phase 1 initialization function calls the memory manager’s
zero page thread function (explained in Chapter 7). Thus, this system thread becomes the
zero page thread for the remainder of the life of the system.
Smss, Csrss, and Winlogon
Smss is like any other user-mode process except for two differences: First, Windows considers
Smss a trusted part of the operating system. Second, Smss is a native application. Because it’s
a trusted operating system component, Smss can perform actions few other processes can
perform, such as creating security tokens. Because it’s a native application, Smss doesn’t use
Windows APIs—it uses only core executive APIs known collectively as the Windows native
API. Smss doesn’t use the Windows APIs because the Windows subsystem isn’t executing
when Smss launches. In fact, one of Smss’s first tasks is to start the Windows subsystem.
Smss then calls the configuration manager executive subsystem to finish initializing the registry, fleshing the registry out to include all its keys. The configuration manager is programmed
to know where the core registry hives are stored on disk (excluding hives corresponding to
user profiles), and it records the paths to the hives it loads in the HKLM\SYSTEM\CurrentControlSet\Control\hivelist key.
Copyrighted material.
270
Microsoft Windows Internals, Fourth Edition
The main thread of Smss performs the following initialization steps:
1. Creates an LPC port object (\SmApiPort) and two threads to wait for client requests
(such as to load a new subsystem or create a session).
2. Defines the symbolic links for MS-DOS device names (such as COM1 and LPT1).
3. If Terminal Services is installed, creates the \Sessions directory in the object manager’s
namespace (for multiple sessions).
4. Runs any programs defined in HKLM\SYSTEM\CurrentControlSet\Control\Session
Manager\BootExecute. Typically, this value contains one command to run Autochk (the
boot-time version of Chkdsk).
5. Performs delayed file rename and delete operations as directed by HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\PendingFileRenameOperations
and HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\PendingFileRenameOperations2.
6. Opens known DLLs, and creates section objects for them in the \Knowndlls directory
of the Object Manager namespace. The list of DLLs considered known is located in
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\KnownDLLs, and the path to the directory in which the DLLs are located is stored
in the Dlldirectory value of the key. See Chapter 6 for information on how the Known
DLLs sections are used during DLL loading.
7. Creates additional paging files. Paging file configuration is stored under
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\PagingFiles.
8. Initializes the registry. The configuration manager fleshes out the registry by loading the
registry hives for the HKLM\SAM, HKLM\SECURITY, and HKLM\SOFTWARE keys.
Although HKLM\SYSTEM\CurrentControlSet\Control\hivelist locates the hive files on
disk, the configuration manager is coded to look for them in \Windows\System32\Config.
9. Creates system environment variables that are defined in HKLM\System\CurrentControlSet\Session Manager\Environment.
10. Loads the kernel-mode part of the Windows subsystem (Win32k.sys). Smss determines
the location of Win32k.sys and other components it loads by looking for their paths in
HKLM\SYSTEM\CurrentControlSet\Control\Session Manager. The initialization code
in Win32k.sys uses the video driver to switch the screen to the resolution defined by the
default profile, so this is the point at which the screen changes from the VGA mode the
boot video driver uses to the default resolution chosen for the system.
11. Starts the subsystem processes, including Csrss. (As noted in Chapter 2, on Windows
2000 the POSIX and OS/2 subsystems are defined to start on demand.)
Copyrighted material.
Chapter 5:
Startup and Shutdown
271
12. Starts the logon process (Winlogon). The startup steps of Winlogon are described
shortly.
13. Creates LPC ports for debug event messages (DbgSsApiPort and DbgUiApiPort) and
threads to listen on those ports.
Pending File Rename Operations
The fact that executable images and DLLs are memory-mapped when they are used
makes it impossible to update core system files after Windows has finished booting.
The MoveFileEx Windows API has an option to specify that a file move be delayed
until the next boot. Service Packs and hotfixes that must update in-use memorymapped files install replacement files onto a system in temporary locations and use the
MoveFileEx API to have them replace otherwise in-use files. When used with that option,
MoveFileEx simply records commands in the PendingFileRenameOperations and
PendingFileRenameOperations2 values under HKLM\SYSTEM\CurrentControlSet\
Control\Session Manager. These registry values are of type MULTI_SZ, where each
operation is specified in pairs of file names: the first file name is the source location,
and the second is the target location. Delete operations use an empty string as their
target path. You can use the Pendmoves utility from www.sysinternals.com to view registered delayed rename and delete commands.
After performing these initialization steps, the main thread in Smss waits forever for the process handles to Csrss and Winlogon. If either of these processes terminates unexpectedly,
Smss crashes the system, because Windows relies on their existence. (In Windows XP and
later, if Csrss exits for any reason, the kernel crashes the system, not the Smss.)
Winlogon then performs its startup steps, such as creating the initial window station and
desktop objects. If a DLL is specified in HKLM\Software\Microsoft\Windows NT\Current
Version\WinLogon\GinaDLL, Winlogon uses that DLL as the GINA; otherwise, it uses the
Microsoft default GINA, Msgina (\Windows\System32\Msgina.dll), which displays the standard Windows logon dialog box. Winlogon then creates the service control manager (SCM)
process (\Windows\System32\Services.exe), which loads all services and device drivers
marked for auto-start, and the local security authentication subsystem (Lsass) process (\Windows\System32\Lsass.exe). (For more details on the startup sequence for Winlogon and
Lsass, see the section “Winlogon Initialization” in Chapter 8.)
After the SCM initializes the auto-start services and drivers and a user has successfully logged
on at the console, the SCM deems the boot successful. The registry last known good control
set (as indicated by HKLM\SYSTEM\Select\LastKnownGood) is updated to match \CurrentControlSet.
Copyrighted material.
272
Microsoft Windows Internals, Fourth Edition
Note Because noninteractive servers might never have an interactive logon, they might not
get LastKnownGood updated to reflect the control set used for a successful boot.
You can override the definition of a successful boot by setting HKLM\Software\Microsoft\Windows NT\CurrentVersion\Winlogon\ReportBootOk to 0, writing a custom boot verification
program that calls the NotifyBootConfigStatus Windows API when a boot is successful, and
entering the path to the verification program in HKLM\System\CurrentControlSet\Control\BootVerificationProgram.
After launching the SCM, Winlogon waits for an interactive logon notification from the GINA.
When it receives a logon and validates the logon (a process for which you can find more information in the section “User Logon Steps” in Chapter 8), Winlogon loads the registry hive from
the profile of the user logging in and maps it to HKCU. It then sets the user’s environment variables that are stored in HKCU\Environment and notifies the Winlogon notification packages
registered in HKLM\Software\Microsoft\Windows NT\CurrentVersion\Winlogon\Notify
that a logon has occurred.
Winlogon next tells the GINA to start the shell. In response to this request, Msgina
launches the executable or executables specified in HKLM\Software\Microsoft\Windows
NT\CurrentVersion\WinLogon\Userinit (with multiple executables separated by commas) that by default points at \Windows\System32\Userinit.exe. Userinit.exe performs the
following steps:
1. Processes the user scripts specified in HKCU\Software\Policies\Microsoft\Windows\System\Scripts and the machine logon scripts in HKLM\Software\Policies\Microsoft\Windows\System\Scripts. (Because machine scripts run after user
scripts, they can override user settings.)
2. If group policy specifies a user profile quota, starts \Windows\System32\Proquota.exe
to enforce the quota for the current user.
3. Launches the comma-separated shell or shells specified in HKCU\Software\Microsoft\Windows NT\CurrentVersion\Winlogon\Shell. If that value doesn’t
exist, Userinit.exe launches the shell or shell specified in HKLM\Software\Microsoft\Windows NT\CurrentVersion\Winlogon\Shell, which is by default
Explorer.exe.
Winlogon then notifies registered network providers that a user has logged in. The Microsoft
network provider, Multiple Provider Router (\Windows\System32\Mpr.dll), restores the
user’s persistent drive letter and printer mappings stored in HCU\Network and HKCU\
Printers, respectively. Figure 5-3 shows the process tree as seen in Process Explorer during a
login before Userinit has exited.
Copyrighted material.
Chapter 5:
Figure 5-3
Startup and Shutdown
273
Process tree during logon
Images that Start Automatically
In addition to the Userinit and Shell registry values in Winlogon’s key, there are many other registry locations and directories that default system components check and process for automatic
process startup during the boot and logon process. The Msconfig utility (included in Windows
XP and Windows Server 2003 in \Windows\System32\Msconfig.exe) displays the images configured by several of the locations. Sysinternals’ Autoruns tool, which you can download from
www.sysinternals.com and that is shown in Figure 5-4, examines more locations than Msconfig and
displays more information about the images configured to automatically run. By default, Autoruns shows only the locations that are configured to automatically execute at least one image, but
checking the Include Empty Locations entry in the View menu causes Autoruns to show all the
locations it inspects. The View menu also has selections to direct Autoruns to display information
about other types of autostarting images, such as Windows services and Explorer add-ons.
Figure 5-4
Copyrighted material.
The Autoruns tool available from www.sysinternals.com
274
Microsoft Windows Internals, Fourth Edition
EXPERIMENT: Autoruns
Many users are unaware of how many programs execute as part of their logon. Original
equipment manufacturers (OEMs) often configure their systems with add-on utilities
that execute in the background using registry values or file system directories processed
for automatic execution and so are not normally visible. See what programs are configured to start automatically on your computer by running the Autoruns utility from
www.sysinternals.com. Compare the list shown in Autoruns with that shown in Msconfig
(available on Windows XP and Windows Server 2003), and identify any differences.
Then ensure that you understand the purpose of each program.
Troubleshooting Boot and Startup Problems
This section presents approaches to solving problems that can occur during the Windows
startup process as a result of hard disk corruption, file corruption, missing files, and thirdparty driver bugs. First, we describe three Windows boot-problem recovery modes: last
known good, safe mode, and the Recovery Console. Then we present common boot problems, their causes, and approaches to solving them. The solutions refer to last known good,
safe mode, the Recovery Console, and other tools that ship with Windows.
Last Known Good
Last known good (LKG) is a useful mechanism for getting a system that crashes during the
boot process back to a bootable state. Because the system’s configuration settings are stored
in HKLM\System\CurrentControlSet\Control and driver and service configuration is stored
in HKLM\System\CurrentControlSet\Services, changes to these parts of the registry can render a system unbootable. For example, if you install a device driver that has a bug that crashes
the system during the boot, you can press the F8 key during the boot and select last known
good from the resulting menu. The system marks the control set that it was using to boot the
system as failed by setting the Failed value of HKLM\System\Select and then changes
HKLM\System\Select\Current to the value stored in HKLM\System\Select\LastKnownGood. It also updates the symbolic link HKLM\System\CurrentControlSet to point at the
LastKnownGood control set. Because the new driver’s key is not present in the Services subkey of the LastKnownGood control set, the system will boot successfully.
Safe Mode
Perhaps the most common reason Windows systems become unbootable is that a device
driver crashes the machine during the boot sequence. Because software or hardware configurations can change over time, latent bugs can surface in drivers at any time. Windows
offers a way for an administrator to attack the problem: booting in safe mode. Safe mode is a
concept Windows borrows from Consumer Windows—a boot configuration that consists of
Copyrighted material.
Chapter 5:
Startup and Shutdown
275
the minimal set of device drivers and services. By relying on only the drivers and services
that are necessary for booting, Windows avoids loading third-party and other nonessential
drivers that might crash.
When Windows boots, you press the F8 key to enter a special boot menu that contains the
safe-mode boot options. You typically choose from three safe-mode variations: Safe Mode, Safe
Mode With Networking, and Safe Mode With Command Prompt. Standard safe mode comprises the minimum number of device drivers and services necessary to boot successfully.
Networking-enabled safe mode adds network drivers and services to the drivers and services
that standard safe mode includes. Finally, safe mode with command prompt is identical to
standard safe mode except that Windows runs the command prompt application (Cmd.exe)
instead of Windows Explorer as the shell when the system enables GUI mode.
Windows includes a fourth safe mode-Directory Services Restore mode—which is different
from the standard and networking—enabled safe modes. You use Directory Services Restore
mode to boot the system into a mode where the Active Directory directory service of a domain
controller is offline and unopened. This allows you to perform repair operations on the database or restore it from backup media. All drivers and services, with the exception of the Active
Directory service, load during a Directory Services Restore mode boot. In cases where you
can’t log in a system because of Active Directory database corruption, this mode enables you
to repair the corruption.
Driver Loading in Safe Mode
How does Windows know which device drivers and services are part of standard and networking-enabled safe mode? The answer lies in the HKLM\SYSTEM\CurrentControlSet\
Control\SafeBoot registry key. This key contains the Minimal and Network subkeys. Each
subkey contains more subkeys that specify the names of device drivers or services or of
groups of drivers. For example, the vga.sys subkey identifies the VGA display device driver
that the startup configuration includes. The VGA display driver provides basic graphics services for any PC-compatible display adapter. The system uses this driver as the safe-mode display driver in lieu of a driver that might take advantage of an adapter’s advanced hardware
features but that might also prevent the system from booting. Each subkey under the SafeBoot
key has a default value that describes what the subkey identifies; the vga.sys subkey’s default
value is “Driver”.
The Boot file system subkey has as its default value “Driver Group”. When developers design
a device driver’s installation script, they can specify that the device driver belongs to a driver
group. The driver groups that a system defines are listed in the List value of the HKLM\SYSTEM\CurrentControlSet\Control\ServiceGroupOrder key. A developer specifies a driver as a
member of a group to indicate to Windows at what point during the boot process the driver
should start. The ServiceGroupOrder key’s primary purpose is to define the order in which
driver groups load; some driver types must load either before or after other driver types. The
Group value beneath a driver’s configuration registry key associates the driver with a group.
Copyrighted material.
276
Microsoft Windows Internals, Fourth Edition
Driver and service configuration keys reside beneath HKLM\SYSTEM\CurrentControlSet\
Services. If you look under this key, you’ll find the VgaSave key for the VGA display device
driver, which you can see in the registry is a member of the Video Save group. Any file system
drivers that Windows requires for access to the Windows system drive are in the Boot file system group. If the system drive is NTFS, the NTFS driver is part of this group. (The value of
Group under the Ntfs key is Boot file system.) Otherwise, the Fastfat file system driver (which
supports FAT12, FAT16, and FAT32 drives in Windows) is part of this group. Other file system drivers are part of the File system group, which the standard and networking-enabled
safe-mode configurations also include.
When you boot into a safe-mode configuration, the boot loader (Ntldr) passes an associated
switch to the kernel (Ntoskrnl.exe) as a command-line parameter, along with any switches
you’ve specified in the Boot.ini file for the installation you’re booting. If you boot into any safe
mode, Ntldr passes the /SAFEBOOT: switch. Ntldr appends one or more additional strings to
/SAFEBOOT:, depending on which type of safe mode you select. For standard safe mode,
Ntldr appends MINIMAL, and for networking-enabled safe mode, it adds NETWORK. Ntldr
adds MINIMAL(ALTERNATESHELL) for safe mode with command prompt and DSREPAIR
for Directory Services Restore mode.
The Windows kernel scans boot parameters in search of the safe-mode switches early during the
boot and sets the internal variable InitSafeBootMode to a value that reflects the switches the kernel finds. The kernel writes the InitSafeBootMode value to the registry value HKLM\SYSTEM\
CurrentControlSet\Control\SafeBoot\Option\OptionValue so that user-mode components, such as the SCM, can determine what boot mode the system is in. In addition, if the
system is booting safe mode with command prompt, the kernel sets the HKLM\SYSTEM\
CurrentControlSet\Control\SafeBoot\Option\UseAlternateShell value to 1. The kernel records
the parameters that Ntldr passes to it in the value HKLM\SYSTEM\CurrentControlSet\
Control\SystemStartOptions.
When the I/O manager kernel subsystem loads device drivers that HKLM\SYSTEM\CurrentControlSet\Services specifies, the I/O manager executes the function IopLoadDriver. When
the Plug and Play manager detects a new device and wants to dynamically load the device
driver for the detected device, the Plug and Play manager executes the function IopCallDriverAddDevice. Both these functions call the function IopSafeBootDriverLoad before they load the
driver in question. IopSafeBootDriverLoad checks the value of InitSafeBootMode and determines whether the driver should load. For example, if the system boots in standard safe
mode, IopSafeBootDriverLoad looks for the driver’s group, if the driver has one, under the Minimal subkey. If IopSafeBootDriverLoad finds the driver’s group listed, IopSafeBootDriverLoad
indicates to its caller that the driver can load. Otherwise, IopSafeBootDriverLoad looks for the
driver’s name under the Minimal subkey. If the driver’s name is listed as a subkey, the driver
can load. If IopSafeBootDriverLoad can’t find the driver group or driver name subkeys, the
driver can’t load. If the system boots in networking-enabled safe mode, IopSafeBootDriverLoad
performs the searches on the Network subkey. If the system doesn’t boot in safe mode, IopSafeBootDriverLoad lets all drivers load.
Copyrighted material.
Chapter 5:
Startup and Shutdown
277
An exception loophole exists regarding the drivers that safe mode excludes from a boot: Ntldr,
rather than the kernel, loads any drivers with a Start value of 0 in their registry key, which
specifies loading the drivers at boot time. Ntldr doesn’t check the SafeBoot registry key
because it assumes that any driver with a Start value of 0 is required for the system to boot successfully. Because Ntldr doesn’t check the SafeBoot registry key to identify which drivers to
load, Ntldr therefore loads all boot-start drivers (and later Ntoskrnl starts them).
Safe-Mode-Aware User Programs
When the service control manager (SCM) user-mode component (which Services.exe implements) initializes during the boot process, the SCM checks the value of HKLM\SYSTEM\
CurrentControlSet\Control\SafeBoot\Option\OptionValue to determine whether the system
is performing a safe mode boot. If so, the SCM mirrors the actions of IopSafeBootDriverLoad.
Although the SCM processes the services listed under HKLM\SYSTEM\CurrentControlSet\
Services, it loads only services that the appropriate safe-mode subkey specifies by name. You can
find more information on the SCM initialization process in the section “Services” in Chapter 4.
Userinit (\Windows\System32\Userinit.exe) is another user-mode component that needs to
know whether the system is booting in safe mode. Userinit, the component that initializes a
user’s environment when the user logs on, checks HKLM\SYSTEM\CurrentControlSet\Control\SafeBoot\Option\UseAlternateShell. If this value is set, Userinit runs the program specified as the user’s shell in the value HKLM\SYSTEM\CurrentControlSet\Control\SafeBoot\
AlternateShell rather than executing Explorer.exe. Windows writes the program name Cmd.exe
to the AlternateShell value during installation, making the Windows command prompt the
default shell for safe mode with command prompt. Even though command prompt is the shell,
you can type Explorer.exe at the command prompt to start Windows Explorer, and you can run
any other GUI program from the command prompt as well.
How does an application determine whether the system is booting in safe mode? By calling
the Windows GetSystemMetrics(SM_CLEANBOOT) function. Batch scripts that need to perform certain operations when the system boots in safe mode look for the SAFEBOOT_
OPTION environment variable because the system defines this environment variable only
when booting in safe mode.
Boot Logging in Safe Mode
When you direct the system to boot into safe mode, Ntldr hands the string specified by the
/BOOTLOG option to the Windows kernel as a parameter, together with the parameter that
requests safe mode. When the kernel initializes, it checks for the presence of the boot log
parameter, whether or not any safe-mode parameter is present. If the kernel detects a boot log
string, the kernel records the action the kernel takes on every device driver it considers for
loading. For example, if IopSafeBootDriverLoad tells the I/O manager not to load a driver, the
Copyrighted material.
278
Microsoft Windows Internals, Fourth Edition
I/O manager calls IopBootLog to record that the driver wasn’t loaded. Likewise, after IopLoadDriver successfully loads a driver that is part of the safe-mode configuration, IopLoadDriver
calls IopBootLog to record that the driver loaded. You can examine boot logs to see which
device drivers are part of a boot configuration.
Because the kernel wants to avoid modifying the disk until Chkdsk executes, late in the boot
process, IopBootLog can’t simply dump messages into a log file. Instead, IopBootLog records
messages in the HKLM\SYSTEM\CurrentControlSet\BootLog registry value. As the first
user-mode component to load during a boot, the Session Manager (\Windows\System32\
Smss.exe) executes Chkdsk to ensure the system drives’ consistency and then completes registry initialization by executing the NtInitializeRegistry system call. The kernel takes this action
as a cue that it can safely open a log file on the disk, which it does, invoking the function IopCopyBootLogRegistryToFile. This function creates the file Ntbtlog.txt in the Windows system
directory (\Windows by default) and copies the contents of the BootLog registry value to the
file. IopCopyBootLogRegistryToFile also sets a flag for IopBootLog that lets IopBootLog know that
writing directly to the log file, rather than recording messages in the registry, is now OK. The
following output shows the partial contents of a sample boot log:
Service Pack 1 3 30 2004 14:05:21.500
Loaded driver \WINDOWS\system32\ntoskrnl.exe
Loaded driver \WINDOWS\system32\hal.dll
Loaded driver \WINDOWS\system32\KDCOM.DLL
Loaded driver \WINDOWS\system32\BOOTVID.dll
Loaded driver ACPI.sys
Loaded driver \WINDOWS\System32\DRIVERS\WMILIB.SYS
Loaded driver pci.sys
Loaded driver isapnp.sys
Loaded driver intelide.sys
Loaded driver \WINDOWS\System32\DRIVERS\PCIIDEX.SYS
Loaded driver MountMgr.sys
Loaded driver ftdisk.sys
Loaded driver dmload.sys
Loaded driver dmio.sys Microsoft (R) Windows 2000 (R) Version 5.0 (Build 2195)
2 11 2000 10:53:27.500
Loaded driver \WINNT\System32\ntoskrnl.exe
Loaded driver \WINNT\System32\hal.dll
Loaded driver \WINNT\System32\BOOTVID.DLL
Loaded driver ACPI.sys
Loaded driver \WINNT\System32\DRIVERS\WMILIB.SYS
Loaded driver pci.sys
Loaded driver isapnp.sys
Loaded driver compbatt.sys
Loaded driver \WINNT\System32\DRIVERS\BATTC.SYS
Loaded driver intelide.sys
Loaded driver \WINNT\System32\DRIVERS\PCIIDEX.SYS
Loaded driver pcmcia.sys
Loaded driver ftdisk.sys
Loaded driver Diskperf.sys
Loaded driver dmload.sys
Loaded driver dmio.sys
Copyrighted material.
Chapter 5:
§
Did not load
Did not load
Did not load
l Devices
Did not load
Did not load
§
Startup and Shutdown
279
driver \SystemRoot\System32\Drivers\lbrtfdc.SYS
driver \SystemRoot\System32\Drivers\Sfloppy.SYS
driver \SystemRoot\System32\Drivers\i2omgmt.SYSDid not load driver Media Contro
driver Communications Port
driver Audio Codecs
Recovery Console
Safe mode is a satisfactory fallback for systems that become unbootable because a device
driver crashes during the boot sequence, but in some situations a safe-mode boot won’t help
the system boot. For example, if a driver that prevents the system from booting is a member
of a Safe group, safe-mode boots will fail. Another example of a situation in which safe mode
won’t help the system boot is when a third-party driver, such as a virus scanner driver, that
loads at the boot prevents the system from booting. (Boot-start drivers load whether or not
the system is in safe mode.) Other situations in which safe-mode boots will fail are when a system module or critical device driver file that is part of a safe-mode configuration becomes corrupt or when the system drive’s Master Boot Record (MBR) is damaged. You can get around
these problems by using the Windows Recovery Console. The Recovery Console allows you
to boot into a limited command-line shell from the Windows CD or boot disks to repair an
installation without having to boot the installation.
When you boot a system from the Windows CD or boot disks, you eventually see a screen that
gives you the choice of either installing Windows or repairing an existing installation. If you
choose to repair an installation, the system prompts you to insert the Windows CD (if it isn’t
already loaded in the system’s CD drive) and then to choose among two repair options: to
start the Recovery Console or to initiate the emergency repair process. If you press the F10 key
at the Setup Welcome screen, you bypass the menu options and take a shortcut directly to the
Recovery Console.
When you start the Recovery Console, it gives you a list of Windows NT and Windows installations to choose from that it compiled when it scanned the computer’s hard disks. After you
make a selection, the system prompts you to enter the Administrator account password to log
on to the installation as the administrator. If you successfully log on, the system puts you into
a command shell that is similar to an MS-DOS environment. The command set is flexible and
lets you perform simple file operations (such as copy, rename, and delete), enable and disable
services and drivers, and even repair MBRs and boot records. However, the Recovery Console
won’t let you access directories other than root directories, the system directory of the installation you logged on to, or directories on removable drives such as CDs and 3.5-inch floppy
disks unless local security policy settings stored in the SECURITY hive of the Registry of the
installation into which you log in permit it. This prohibition provides a certain level of security
for data that an administrator might not usually be able to access. You can override this restriction by using the Local Security Policy editor (secpol.msc) to configure the Recovery Console
settings in the Security Options folder of Local Policies when the system is booted normally.
Copyrighted material.
280
Microsoft Windows Internals, Fourth Edition
The Recovery Console uses the native Windows system call interface to perform file I/O to
support commands such as Cd, Rename, and Move. The Enable and Disable commands, which
let you change the startup modes of device drivers and services, work differently. For example,
when you tell the Recovery Console that you want to disable a device driver, it reaches into the
installation’s Services key and manipulates the Start value of the specified driver’s key, changing the value to SERVICE_DISABLED. The next time the installation boots, that device driver
won’t load. (The Recovery Console also loads the SYSTEM hive [\Windows\System32\Config\System] for the installation you log on to. This hive contains the information stored in the
HKLM\SYSTEM\CurrentControlSet\Services registry key.)
When you boot from the Windows CD or the boot disks, by the time the system gives you the
choice to install or repair Windows, the CD has booted a copy of the Windows kernel, including all necessary supporting device drivers (for example, NTFS or FAT drivers, SCSI drivers, a
video driver). On x86 systems, the Txtsetup.sif file in the I386 directory of the Windows CD
guides the boot from the CD; the file contains directives that identify which files need to load
and where the files are located on the CD. Just as when you boot Windows from a hard disk,
the first user-mode program the kernel executes is Session Manager (Smss.exe), located in the
I386\System32 folder. The Session Manager that Windows Setup uses differs from the standard-installation Session Manager. The former component presents you with the menus that
let you install or repair Windows and the menu that asks you what type of repair you want to
perform. If you’re installing Windows, Session Manager is the component that guides you
through choosing a partition to install to and copies files to the hard disk.
When you run the Recovery Console, Session Manager loads and starts two device drivers
that implement the Recovery Console: Spcmdcon.sys and Setupdd.sys. Spcmdcon.sys presents
an interactive command prompt and performs high-level command processing. Setupdd.sys is a
support driver that gives Spcmdcon.sys a set of functions that let Spcmdcon.sys manage disk
partitions, load registry hives, and display and manage video output. Setupdd.sys also communicates with disk drivers to manage disk partitions and uses basic video support built into the
Windows kernel to display messages on the screen.
When you choose an installation to log on to and the Recovery Console accepts your password, the Recovery Console must validate your logon attempt, even though the installation’s
Windows security subsystem isn’t up and running. Thus, the Recovery Console alone must
determine whether your password matches the system’s Administrator account. The Recovery
Console’s first step in this process is to use Setupdd.sys to load the installation’s Security
Accounts Manager (SAM) registry hive, which stores password information, from the hard
disk. The SAM hive resides in \Windows\System32\Config\Sam. After loading the hive, the
Recovery Console locates the system key in the installation’s registry and uses the system key
to decrypt the in-memory copy of the SAM. SAM hive encryption is a feature introduced in
Windows NT 4 Service Pack 3 that adds protection against MS-DOS-based password snoopers who try to read passwords directly out of a hive file.
Copyrighted material.
Chapter 5:
Startup and Shutdown
281
Next, the Recovery Console (Spcmdcon.sys) locates the Administrator account password in
the SAM, and in the final authentication step, the Recovery Console uses the MD5 hash algorithm—the same algorithm that the Windows logon process uses—to hash the password
entered and compares the hash against the hashed password that the SAM stores. If the Recovery Console finds a match, the system considers you logged on. If the Recovery Console
doesn’t find a match, the system denies you access to the Recovery Console.
Solving Common Boot Problems
This section describes problems that can occur during the boot process, describing their
symptoms, causes, and approaches to solving them. To help you locate a problem that you
might encounter, they are organized according to the place in the boot at which they occur.
MBR Corruption
■
Symptoms
■
Cause The MBR can become corrupt because of hard-disk errors, disk corruption as a
result of a driver bug while Windows is running, or intentional scrambling as a result of
a virus.
■
Resolution Boot into the Recovery Console and execute the fixmbr command. This
command replaces the executable code in the MBR. Unfortunately, it does not repair the
partition table. The only way to restore a damaged partition table is to restore it from a
backup copy or to use a third-party disk-corruption repair tool.
A system that has Master Boot Record (MBR) corruption will execute the
BIOS power-on self test (POST), display BIOS version information or OEM branding,
switch to a black screen, and then hang. Depending on the type of corruption the MBR
has experienced, you might see one of the following messages: “Invalid Partition Table,”
“Error Loading Operating System,” or “Missing Operating System.”
Boot Sector Corruption
■
Symptoms
■
Cause The MBR can become corrupt because of hard disk errors, disk corruption as a
result of a driver bug while Windows is running, or intentional scrambling as a result of
a virus.
■
Resolution Boot into the Recovery Console and execute the fixboot command. This
command rewrites the boot sector of the volume that you specify. You should execute
the command on both the system and boot volumes if they are different.
Copyrighted material.
Boot sector corruption can look like MBR corruption where the system
hangs after BIOS POST at a black screen, or you might see the messages “A disk read
error occurred,” “NTLDR is missing,” or “NTLDR is compressed” displayed in a black
screen.
282
Microsoft Windows Internals, Fourth Edition
Boot.ini Misconfiguration
■
Symptom After BIOS POST, you’ll see a message that begins “Windows could not start
because of a computer disk hardware configuration problem,” “Could not read from
selected boot disk,” or “Check boot path and disk hardware.”
■
Cause The Boot.ini file has been deleted, is corrupted, or no longer references the boot
volume because the addition of a partition has changed the Advanced RISC Computing
(ARC) name of the volume.
■
Boot into the Recovery Console, and execute the “bootcfg /rebuild”. This
command has the Recovery Console scan each volume looking for Windows installations. When it discovers an installation, it asks you whether it should add it to Boot.ini
as a boot option and what name it should display for the installation in the boot menu.
Resolution
System File Corruption
■
Symptoms
■
Causes The volume on which a system file is located is corrupt or one or more system
files have been deleted or become corrupt.
■
Resolution
There are several ways the corruption of system files—which include executables, drivers, or DLLs—can manifest. One way is with a message on a black screen after
BIOS POST that says, “Windows could not start because the following file is missing or
corrupt,” followed by the name of a file and a request to re-install the file. Another way
is with a blue screen crash during the boot with the text, “STOP: 0xC0000135 {Unable
to Locate Component}.”
Boot into the Recovery Console, and execute the chkdsk command. Chkdsk
will attempt to repair volume corruption. If Chkdsk does not report any problems,
obtain a backup copy of the system file in question. One place to check is in the \Windows\System32\DllCache directory, in which Windows places copies of many system
files for access by Windows File Protection. (See the “Windows File Protection” sidebar.)
If you cannot find a copy of the file there, see if you can locate a copy from another system in the network. Note that the backup file must be from the same Service Pack or hot
fix as the file that you are replacing.
In some cases, multiple system files are deleted or become corrupt, so the repair process can
involve multiple reboots and boot failures as you repair the files one by one. If you believe the
system file corruption to be extensive, you should consider restoring the system from a
backup image, such as one generated by Automated System Recovery (ASR). When you run
Windows Backup (located in the System folder under Accessories in the Start menu), you can
generate an ASR backup image, which includes all the files on the system and boot volumes,
plus a floppy disk on which it stores information about the system’s disks and volumes. To
restore a system from an ASR, back up boot from the Windows setup media and press F2
when prompted.
If you do not have a backup from which to restore, a last resort is to execute a Windows
repair install: boot from the Windows setup media, and follow the wizard as if you were
Copyrighted material.
Chapter 5:
Startup and Shutdown
283
going to perform a new installation. The wizard will ask you whether you want to perform
a repair or fresh install. When you tell it that you want to repair, Setup reinstalls all system
files, leaving your application data and registry settings intact.
Windows File Protection
In addition to its role as the interactive logon interface and Session Manager, Winlogon
also implements Windows File Protection (WFP). WFP, which is implemented in the two
DLLs \Windows\System32\Sfc.dll and \Windows\System32\Sfc_os.dll, monitors several directories for changes to key drivers, executables, and DLLs, including most subdirectories under \Windows, using the native API version of ReadDirectoryChangesW. When
WFP sees that a change has occurred to a system file listed in \Windows\System32\
Sfcfiles.Dll (and you can use the Strings utility from www.sysinternals.com to see the files
listed in Sfcfiles.dll), it checks to see whether the file is digitally signed by Microsoft (a
process for which you can find more information in the “Driver Installation” section of
Chapter 9). If the file is digitally signed by Microsoft, WFP allows the change and copies
the file to the WFP backup directory. By default, the backup directory is \Windows\
System32\DllCache, although that can be overridden by defining the Registry value
HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\
Winlogon\SFCDllCacheDir. Hot fixes and service packs always install Microsoftsigned system files.
If the file modification doesn’t result in a file that isn’t Microsoft-signed, WFP replaces
the modification with the backup version of the file from the DLLCache subdirectory. If
Winlogon can’t find a backup version in that directory, it checks in the network install
path if the system was installed using a network install or in the setup media (prompting for insertion) if the install was from local media.
System Hive Corruption
■
Symptoms
■
Causes
■
Resolution
Copyrighted material.
If the System registry hive (which is discussed along with hive files in the
“Registry” section of Chapter 4) is missing or corrupted, NTLDR will display the message, “Windows could not start because the following file is missing or corrupt:
\WINDOWS\SYSTEM32\CONFIG\SYSTEM,” on a black screen after the BIOS POST.
The System registry hive, which contains configuration information necessary
for the system to boot, has become corrupt or has been deleted.
Boot into the Recovery Console, and execute the chkdsk command on the
boot volume to correct any volume corruption. If the problem is not corrected, obtain a
backup of the System registry hive. If you have made ASR backups of the system or have
used the Windows Backup utility to make backups of system state (an option in the
backup UI), copies of the registry hives from the most recent backup are stored in
\Windows\Repair, so copy the file named System to \Windows\System32\Config.
284
Microsoft Windows Internals, Fourth Edition
If you’re running Windows XP and System Restore is enabled (System Restore is discussed in
Chapter 12) , you can often obtain a more recent backup of the registry hives, including the
System hive, from the most recent restore point. However, you may not be able to access the
directory in which restores points are stored, \System Volume Information, from within the
Recovery Console. Windows XP Service Pack 1 Versions of the Recovery Console allow access
to that directory, but older versions do not unless the system’s local security policy allows it.
You can override the restriction if necessary by using the Local Security Policy Editor to
change Recovery Console settings, as described earlier. You can also use third-party tools to
gain access to other directories. If you can access the restore point directories, you can follow
these steps to get at their registry hives:
1. Navigate to the directory whose name begins with “_restore” under the \System Volume
Information directory of the boot volume.
2. Locate the RP subdirectory with the highest number as its suffix (for example, “RP173”).
3. Copy the file named _REGISTRY_MACHINE_SYSTEM from the snapshot subdirectory
to \Windows\System32\Config\System.
4. Reboot.
Another option is to try and repair the corruption using the Microsoft ChkReg tool. The tool
attempts to automatically repair registry corruption, and it works by running off of the
Windows XP setup floppy disks. You can find the tool and instructions on how to use it at
http://www.microsoft.com/downloads/details.aspx?displaylang=en&familyid=56d3c201-2c684de8-9229-ca494362419c.
If you haven’t made backups, don’t have access to restore points, and the ChkReg tool doesn’t
fix the corruption (or the hives are missing), you can use the copy of the System hive stored
in \Windows\Repair as a last resort. Windows Setup makes a copy of the System hive after it
completes an installation, so you will lose system configuration changes and device driver
installations made since then.
Post–Splash Screen Crash or Hang
■
Symptoms
■
Causes These problems are almost always a result of a bug in a device driver, but they
can sometimes be the result of corruption of a registry hive other than the System hive.
■
You can take several steps to try and correct the problem. The first thing
you should try is the last known good configuration. Last known good (LKG), which is
described earlier in this chapter and in the “Services” section of Chapter 4, consists of
the registry control set that was last used to boot the system successfully. Because a control set includes core system configuration and the device driver and services
Problems that occur after the Windows splash screen displays, the desktop
appears, or you log in fall into this category and can appear as a blue screen crash or a
hang, where the entire system is frozen or the mouse cursor tracks the mouse but the
system is otherwise unresponsive.
Resolution
Copyrighted material.
Chapter 5:
Startup and Shutdown
285
registration database, using a version that does not reflect changes or newly installed
drivers or services might avoid the source of the problem. You access last known good
by pressing the F8 key early in the boot process to access the same menu from which
you can boot into safe mode.
As stated earlier in the chapter, when you boot into LKG, the system saves the control set that
you are avoiding and labels it as the failed control set. You can leverage the failed control set in
cases where LKG makes a system bootable to determine what was causing the system to fail to
boot by exporting the contents of the current control set of the successful boot and the failed
control set to .reg files. You do this by using the Regedit’s export functionality, which you
access under the File menu (or under the Registry menu if you are running Windows 2000):
1. Run Regedit, and select HKLM\System\CurrentControlSet.
2. Select Export from the File menu, and save to a file named good.reg.
3. Open HKLM\System\Select, read the value of Failed, and select the subkey named
HKLM\System\ControlXXX, where XXX is the value of Failed.
4. Export the contents of the control set to bad.reg.
5. Use Wordpad (which is found under Accessories in the Start menu) to globally replace
all instances of “CurrentControlSet” in good.reg with “ControlSet”.
6. Use Wordpad to change all instances of “ControlXXX” (replacing XXX with the value of
the Failed control set) in bad.reg with “ControlSet”.
7. Run Windiff from the Support Tools, and compare the two files.
The differences between a failed control set and a good one can be numerous, so you should
focus your examination on changes beneath the Control subkey as well as under the Parameters subkeys of drivers and services registered in the Services subkey. Ignore changes made to
Enum subkeys of driver registry keys in the Services branch of the control set.
If the problem you’re experiencing is caused by a driver or service that was present on the system since before the last successful boot, LKG will not make the system bootable. Similarly, if
a problematic configuration setting changed outside the control set or was made before the
last successful boot, LKG will not help. In those cases, the next option to try is safe mode
(described earlier in this section). If the system boots successfully in safe mode and you know
that particular driver was causing the normal boot to fail, you can disable the driver by using
the Device Manager (accessible from the Hardware tab of the System Control Panel applet).
To do so, select the driver in question and choose Disable from the Action menu. If you’re running Windows XP or Windows Server 2003, you recently updated the driver, and believe that
the update introduced a bug, you can choose to roll back the driver to its previous version
instead, also with the Device Manager. To restore a driver to its previous version, double-click
on the driver to open its properties dialog box and press Roll Back Driver on the Drivers tab.
On Windows XP systems with System Restore enabled, an option when LKG fails is to roll
back all system state (as defined by System Restore) to a previous point in time. Safe mode
Copyrighted material.
286
Microsoft Windows Internals, Fourth Edition
detects the existence of restore points, and when they are present it will ask you whether you
want to log in to the installation to perform a manual diagnosis and repair or launch the System Restore Wizard. Using System Restore to make a system bootable again is attractive when
you know the cause of a problem and want the repair to be automatic or when you don’t know
the cause but do not want to invest time to determine the cause.
If System Restore is not an option or you want to determine the cause of a crash during the
normal boot and the system boots successfully in safe mode, attempt to obtain a boot log
from the unsuccessful boot by pressing F8 to access the special boot menu and choosing the
boot logging option. As described earlier in this chapter, Session Manager (\Windows\
System32\Smss.exe) saves a log of the boot that includes a record of device drivers that the
system loaded and chose not to load to \Windows\ntbtlog.txt, so you’ll obtain a boot log if
the crash or hang occurs after Session Manager initializes. When you reboot into safe mode,
the system appends new entries to the existing boot log. Extract the portions of the log file
that refer to the failed attempt, and safe mode boots into separate files. Strip out lines that contain the text “Did not load driver”, and then compare them with a text comparison tool such
as Windiff. One by one, disable the drivers that loaded during the normal boot but not in the
safe-mode boot until the system boots successfully again. (Then re-enable the drivers that
were not responsible for the problem.)
If you cannot obtain a boot log from the normal boot (for instance, because the system is
crashing before Session Manager initializes), if the system also crashes during the safe-mode
boot, or if a comparison of boot logs from the normal and safe-mode boots do not reveal any
significant differences (for example, when the driver that’s crashing the normal boot starts
after Session Manager initializes), the next tool to try is the Driver Verifier combined with
crash dump analysis. (See Chapter 14 for more information on both these topics.)
Shutdown
If someone is logged on and a process initiates a shutdown by calling the Windows ExitWindowsEx function, a message is sent to Csrss instructing it to perform the shutdown. Csrss in
turn impersonates the caller and sends a Windows message to a hidden window owned by
Winlogon, telling it to perform a system shutdown. Winlogon then impersonates the currently logged-on user (who might or might not have the same security context as the user who
initiated the system shutdown) and calls ExitWindowsEx with some special internal flags.
Again, this call causes a message to be sent to Csrss requesting a system shutdown.
This time, Csrss sees that the request is from Winlogon and loops through all the processes in
the logon session of the interactive user (again, not the user who requested a shutdown) in
reverse order of their shutdown level. A process can specify a shutdown level, which indicates
to the system when they want to exit with respect to other processes, by calling SetProcessShutdownParameters. Valid shutdown levels are in the range 0 through 1023, and the default level
is 640. Explorer, for example, sets its shutdown level to 2 and Task Manager specifies 1. For each
process that owns a top-level window, Csrss sends the WM_QUERYENDSESSION message to
Copyrighted material.
Chapter 5:
Startup and Shutdown
287
each thread in the process that has a Windows message loop. If the thread returns TRUE, the
system shutdown can proceed. Csrss then sends the WM_ENDSESSION Windows message to
the thread to request it to exit. Csrss waits the number of seconds defined in HKCU\Control
Panel\Desktop\HungAppTimeout for the thread to exit. (The default is 5000 milliseconds.)
If the thread doesn’t exit before the timeout, Csrss displays the hung-program dialog box
shown in Figure 5-5. (You can disable this dialog box by changing the registry value
HKCU\Control Panel\Desktop\AutoEndTasks to 1.) This dialog box indicates that a program isn’t shutting down in a timely manner and gives the user a choice of either killing the
process or aborting the shutdown. (There is no timeout on this dialog box, which means that
a shutdown request could wait forever at this point.)
Figure 5-5
Hung program dialog box
If the thread does exit before the timeout, Csrss continues sending the WM_QUERYENDSESSION/WM_ENDSESSION message pairs to the other threads in the process that own windows. Once all the threads that own windows in the process have exited, Csrss terminates the
process and goes on to the next process in the interactive session.
EXPERIMENT: Witnessing the HungAppTimeout
You can see the use of the HungAppTimeout registry value by running Notepad, entering text into its editor, and then logging off. After the amount of time specified by the
HungAppTimeout registry value has expired, Csrss.exe presents a dialog box that asks
you whether or not you want to end the Notepad process, which has not exited because
it’s waiting for you to tell it whether or not to save the entered text to a file. If you press
the Cancel button on the dialog box, Csrss.exe aborts the shutdown.
If Csrss finds a console application, it invokes the console control handler by sending the
CTRL_LOGOFF_EVENT event. (Only service processes receive the CTRL_SHUTDOWN_
EVENT event on shutdown.) If the handler returns FALSE, Csrss kills the process. If the
handler returns TRUE or doesn’t respond by the number of seconds defined by HKCU\Control Panel\Desktop\WaitToKillAppTimeout (the default is 20,000 milliseconds), Csrss displays the hung-program dialog box shown in Figure 5-5.
Copyrighted material.
288
Microsoft Windows Internals, Fourth Edition
Next, Winlogon calls ExitWindowsEx to have Csrss terminate any COM processes that are part
of the interactive user’s session.
At this point, all the processes in the interactive user’s session have been terminated.
Winlogon calls ExitWindowsEx again, but this time in the system process context, which again
sends a message to Csrss, which looks at all the processes belonging to the system context
and performs and sends the WM_ QUERYENDSESSION/WM_ENDSESSION messages to
GUI threads (as before). Instead of sending CTRL_LOGOFF_EVENT, however, it sends
CTRL_ SHUTDOWN_EVENT to console applications that have registered control handlers.
Note that the SCM is a console program that does register a control handler. When it receives
the shutdown request, it in turn sends the service shutdown control message to all services
that registered for shutdown notification. For more details on service shutdown (such as the
shutdown timeout Csrss uses for the SCM), see the “Services” section in Chapter 4.
Although Csrss performs the same timeouts as when it was terminating the user processes, it
doesn’t display any dialog boxes and doesn’t kill any processes. (The registry values for the
system process timeouts are taken from the default user profile.) These timeouts simply allow
system processes a chance to clean up and exit before the system shuts down. Therefore,
many system processes are in fact still running when the system shuts down, such as Smss,
Winlogon, the SCM, and Lsass.
Once Csrss has finished its pass notifying system processes that the system is shutting down,
Winlogon finishes the shutdown process by calling the executive subsystem function NtShutdownSystem. This function calls the function NtSetSystemPowerState to orchestrate the shutdown of drivers and the rest of the executive subsystems (Plug and Play manager, power
manager, executive, I/O manager, configuration manager, and memory manager).
For example, NtSetSystemPowerState calls the I/O manager to send shutdown I/O packets to
all device drivers that have requested shutdown notification. This action gives device drivers a
chance to perform any special processing their device might require before Windows exits.
The configuration manager flushes any modified registry data to disk, and the memory manager writes all modified pages containing file data back to their respective files. If the option to
clear the paging file at shutdown is enabled, the memory manager clears the paging file at this
time. The I/O manager is called a second time to inform the file system drivers that the system
is shutting down. System shutdown ends in the power manager. The action the power manager takes depends on whether the user specified a shutdown, a reboot, or a power down.
Conclusion
In this chapter, we’ve examined the detailed steps involved in starting and shutting down
Windows (both normally and in error cases). So far, we’ve examined the overall structure of
Windows and the core system mechanisms that get the system going, keep it running, and
eventually shut it down. With this foundation laid, we’re ready to explore the individual executive components in more detail, starting with processes and threads.
Copyrighted material.
Chapter 6
Processes, Threads, and Jobs
In this chapter, we’ll explain the data structures and algorithms that deal with processes,
threads, and jobs in Microsoft Windows. The first section focuses on the internal structures
that make up a process. The second section outlines the steps involved in creating a process
(and its initial thread). The internals of threads and thread scheduling are then described. The
chapter concludes with a description of the job object.
Where relevant performance counters or kernel variables exist, they are mentioned. Although
this book isn’t a Windows programming book, the pertinent process, thread, and job Windows functions are listed so that you can pursue additional information on their use.
Because processes and threads touch so many components in Windows, a number of terms
and data structures (such as working sets, objects and handles, system memory heaps, and so
on) are referred to in this chapter but are explained in detail elsewhere in the book. To fully
understand this chapter, you need to be familiar with the terms and concepts explained in
chapters 1 and 2, such as the difference between a process and a thread, the Windows virtual
address space layout, and the difference between user mode and kernel mode.
Process Internals
This section describes the key Windows process data structures. Also listed are key kernel
variables, performance counters, and functions and tools that relate to processes.
Data Structures
Each Windows process is represented by an executive process (EPROCESS) block. Besides
containing many attributes relating to a process, an EPROCESS block contains and points to
a number of other related data structures. For example, each process has one or more threads
represented by executive thread (ETHREAD) blocks. (Thread data structures are explained in
the section “Thread Internals” later in this chapter.) The EPROCESS block and its related data
structures exist in system space, with the exception of the process environment block (PEB),
which exists in the process address space (because it contains information that is modified by
user-mode code).
Copyrighted material.
289
290
Microsoft Windows Internals, Fourth Edition
In addition to the EPROCESS block, the Windows subsystem process (Csrss) maintains a parallel structure for each Windows process that executes a Windows program. Also, the kernelmode part of the Windows subsystem (Win32k.sys) has a per-process data structure that is
created the first time a thread calls a Windows USER or GDI function that is implemented in
kernel mode.
Figure 6-1 is a simplified diagram of the process and thread data structures. Each data structure shown in the figure is described in detail in this chapter.
Process
environment
block
Thread
environment
block
Process address space
System address space
Process
block
Windows process block
Handle table
Thread
block
Figure 6-1
…
Data structures associated with processes and threads
First let’s focus on the process block. (We’ll get to the thread block in the section “Thread
Internals” later in the chapter.) Figure 6-2 shows the key fields in an EPROCESS block.
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
291
Kernel process block (or PCB)
Process ID
Parent process ID
Exit status
Create and exit times
PsActiveProcessHead
Active process link
EPROCESS
Quota block
Memory management information
Exception port
Debugger port
Primary access token
Handle table
Device map
Process environment block
Image filename
Image base address
Process priority class
Windows process block
Job object
Figure 6-2
Structure of an executive process block
EXPERIMENT: Displaying the Format of an EPROCESS Block
For a list of the fields that make up an EPROCESS block and their offsets in hexadecimal, type dt _eprocess in the kernel debugger. (See Chapter 1 for more information on
the kernel debugger and how to perform kernel debugging on the local system.) The
output (truncated for the sake of space) looks like this:
lkd> dt _eprocess
nt!_EPROCESS
+0x000 Pcb
: _KPROCESS
+0x06c ProcessLock
: _EX_PUSH_LOCK
+0x070 CreateTime
: _LARGE_INTEGER
+0x078 ExitTime
: _LARGE_INTEGER
+0x080 RundownProtect
: _EX_RUNDOWN_REF
+0x084 UniqueProcessId : Ptr32 Void
+0x088 ActiveProcessLinks : _LIST_ENTRY
+0x090 QuotaUsage
: [3] Uint4B
+0x09c QuotaPeak
: [3] Uint4B
+0x0a8 CommitCharge
: Uint4B
+0x0ac PeakVirtualSize : Uint4B
+0x0b0 VirtualSize
: Uint4B
+0x0b4 SessionProcessLinks : _LIST_ENTRY
+0x0bc DebugPort
: Ptr32 Void
+0x0c0 ExceptionPort
: Ptr32 Void
+0x0c4 ObjectTable
: Ptr32 _HANDLE_TABLE
Copyrighted material.
292
Microsoft Windows Internals, Fourth Edition
+0x0c8
+0x0cc
+0x0ec
+0x0f0
+0x110
+0x114
+0x118
Token
: _EX_FAST_REF
WorkingSetLock
: _FAST_MUTEX
WorkingSetPage
: Uint4B
AddressCreationLock : _FAST_MUTEX
HyperSpaceLock
: Uint4B
ForkInProgress
: Ptr32 _ETHREAD
HardwareTrigger : Uint4B
Note that the first field (Pcb) is actually a substructure, the kernel process block (KPROCESS), which is where scheduling-related information is stored. To display the format of
the kernel process block, type dt_kprocess:
lkd> dt _kprocess
nt!_KPROCESS
+0x000 Header
:
+0x010 ProfileListHead :
+0x018 DirectoryTableBase
+0x020 LdtDescriptor
:
+0x028 Int21Descriptor :
+0x030 IopmOffset
:
+0x032 Iopl
:
+0x033 Unused
:
+0x034 ActiveProcessors :
+0x038 KernelTime
:
+0x03c UserTime
:
+0x040 ReadyListHead
:
+0x048 SwapListEntry
:
+0x04c VdmTrapcHandler :
+0x050 ThreadListHead
:
+0x058 ProcessLock
:
+0x05c Affinity
:
+0x060 StackCount
:
+0x062 BasePriority
:
+0x063 ThreadQuantum
:
+0x064 AutoAlignment
:
+0x065 State
:
+0x066 ThreadSeed
:
+0x067 DisableBoost
:
+0x068 PowerState
:
+0x069 DisableQuantum
:
+0x06a IdealNode
:
+0x06b Spare
:
_DISPATCHER_HEADER
_LIST_ENTRY
: [2] Uint4B
_KGDTENTRY
_KIDTENTRY
Uint2B
UChar
UChar
Uint4B
Uint4B
Uint4B
_LIST_ENTRY
_SINGLE_LIST_ENTRY
Ptr32 Void
_LIST_ENTRY
Uint4B
Uint4B
Uint2B
Char
Char
UChar
UChar
UChar
UChar
UChar
UChar
UChar
UChar
An alternate way to see the KPROCESS (and other substructures in the EPROCESS) is
to use the recursion (-r) switch of the dt command. For example, typing dt _eprocess –
r1 will recurse and display all substructures one level deep.
The dt command shows the format of a process block, not its contents. To show an
instance of an actual process, you can specify the address of an EPROCESS structure as
an argument to the dt command. You can get the address of all the EPROCESS blocks in
the system by using the !process 0 0 command. An annotated example of the output from
this command is included later in this chapter.
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
293
Table 6-1 explains some of the fields in the preceding experiment in more detail and includes
references to other places in the book where you can find more information about them. As
we’ve said before and will no doubt say again, processes and threads are such an integral part
of Windows that it’s impossible to talk about them without referring to many other parts of
the system. To keep the length of this chapter manageable, however, we’ve covered those
related subjects (such as memory management, security, objects, and handles) elsewhere.
Table 6-1
Contents of the EPROCESS Block
Element
Purpose
Additional Reference
Kernel process (KPROCESS)
block
Common dispatcher object header,
pointer to the process page directory,
list of kernel thread (KTHREAD)
blocks belonging to the process, default base priority, quantum, affinity
mask, and total kernel and user time
for the threads in the process.
Thread Scheduling
(Chapter 6)
Process identification
Unique process ID, creating process
ID, name of image being run, window station process is running on.
Quota block
Limits on nonpaged pool, paged
pool, and page file usage plus current and peak process nonpaged
and paged pool usage. (Note: Several processes can share this structure:
all the system processes point to the
single systemwide default quota
block; all the processes in the interactive session share a single quota
block that Winlogon sets up.)
Virtual address descriptors
(VADs)
Series of data structures that describes the status of the portions of
the address space that exist in the
process.
Virtual Address
Descriptors
(Chapter 7)
Working set information
Pointer to working set list (MMWSL
structure); current, peak, minimum,
and maximum working set size; last
trim time; page fault count; memory
priority; outswap flags; page fault
history.
Working Sets
(Chapter 7)
Virtual memory information
Current and peak virtual size, page
file usage, hardware page table entry for process page directory.
Chapter 7
Exception local procedure call
(LPC) port
Interprocess communication channel to which the process manager
sends a message when one of the
process’s threads causes an exception.
Exception Dispatching
(Chapter 3)
Copyrighted material.
294
Microsoft Windows Internals, Fourth Edition
Table 6-1
Contents of the EPROCESS Block
Element
Purpose
Additional Reference
Debugging LPC port
Interprocess communication channel to which the process manager
sends a message when one of the
process’s threads causes a debug
event.
Local Procedure Calls
(LPCs) (Chapter 3)
Access token (ACCESS_TOKEN)
Executive object describing the security profile of this process.
Chapter 8
Handle table
Address of per-process handle table.
Object Handles and the
Process Handle Table
(Chapter 3)
Device map
Address of object directory to resolve device name references in
(supports multiple users).
Object Names
(Chapter 3)
Process environment block
(PEB)
Image information (base address,
version numbers, module list), process heap information, and threadlocal storage utilization. (Note: The
pointers to the process heaps start
at the first byte after the PEB.)
Chapter 6
Windows subsystem process
block (W32PROCESS)
Process details needed by the kernel-mode component of the Windows subsystem.
The kernel process (KPROCESS) block, which is part of the EPROCESS block, and the process environment block (PEB), which is pointed to by the EPROCESS block, contain additional details about the process object. The KPROCESS block (which is sometimes called the
PCB, or process control block) is illustrated in Figure 6-3. It contains the basic information
that the Windows kernel needs to schedule threads. (Page directories are covered in
Chapter 7, and kernel thread blocks are described in more detail later in this chapter.)
The PEB, which lives in the user process address space, contains information needed by the
image loader, the heap manager, and other Windows system DLLs that need to modify it from
user mode. (The EPROCESS and KPROCESS blocks are accessible only from kernel mode.)
The basic structure of the PEB is illustrated in Figure 6-4 and is explained in more detail later
in this chapter.
Copyrighted material.
Processes, Threads, and Jobs
Chapter 6:
Dispatcher header
Process page directory
Kernel time
User time
Inswap/Outswap list entry
…
KTHREAD
Process spinlock
Processor affinity
Resident kernel stack count
Process base priority
Default thread quantum
Process state
Thread seed
Disable boost flag
Figure 6-3
Structure of the executive process block
Image base address
Module list
Thread-local storage data
Code page data
Critical section timeout
Number of heaps
Heap size information
Process heap
GDI shared handle table
Operating system version number information
Image version information
Image process affinity mask
Figure 6-4
Copyrighted material.
Fields of the process environment block
295
296
Microsoft Windows Internals, Fourth Edition
EXPERIMENT: Examining the PEB
You can dump the PEB structure with the !peb command in the kernel debugger. To get
the address of the PEB, use the !process command as follows:
lkd> !process
PROCESS 8575f030 SessionId: 0 Cid: 08d0
Peb: 7ffdf000
DirBase: 1a81b000 ObjectTable: e12bd418 HandleCount:
Image: windbg.exe
ParentCid: 0360
66.
Then specify that address to the !peb command as follows:
lkd> !peb 7ffdf000
PEB at 7ffdf000
InheritedAddressSpace:
No
ReadImageFileExecOptions: No
BeingDebugged:
No
ImageBaseAddress:
01000000
Ldr
00181e90
Ldr.Initialized:
Yes
Ldr.InInitializationOrderModuleList: 00181f28 . 00183188
Ldr.InLoadOrderModuleList:
00181ec0 . 00183178
Ldr.InMemoryOrderModuleList:
00181ec8 . 00183180
Base TimeStamp
Module
1000000 40478dbd Mar 04 15:12:45 2004 C:\Program Files\Debugging Tools for
Windows\windbg.exe
77f50000 3eb1b41a May 01 19:56:10 2003 C:\WINDOWS\System32\ntdll.dll
77e60000 3d6dfa28 Aug 29 06:40:40 2002 C:\WINDOWS\system32\kernel32.dll
2000000 40476db2 Mar 04 12:56:02 2004 C:\Program Files\Debugging Tools for
Windows\dbgeng.dll
.
SubSystemData:
00000000
ProcessHeap:
00080000
ProcessParameters: 00020000
WindowTitle: 'C:\Documents and Settings\All Users\Start Menu\Programs\Debugging
Tools for Windows\WinDbg.lnk'
ImageFile:
'C:\Program Files\Debugging Tools for Windows\windbg.exe'
CommandLine: '"C:\Program Files\Debugging Tools for Windows\windbg.exe" '
DllPath:
'C:\Program Files\Debugging Tools for Windows;C:\WINDOWS\System32;C:
\WINDOWS\system;C:\WINDOWS;.;C:\Program Files\Windows Resource Kits\Tools\;C:\WINDOWS\
system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files\Support Tools\;c:\sysint
;C:\Program Files\ATI Technologies\ATI Control Panel;C:\Program Files\Resource Kit\;C:
\PROGRA~1\CA\Common\SCANEN~1;C:\PROGRA~1\CA\eTrust\ANTIVI~1;C:\Program Files\Common
Files\Roxio Shared\DLLShared;C:\SFU\common\'
Environment: 00010000
=::=::\
ALLUSERSPROFILE=C:\Documents and Settings\All Users
APPDATA=C:\Documents and Settings\dsolomon\Application Data
.
.
.
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
297
Kernel Variables
A few key kernel global variables that relate to processes are listed in Table 6-2. These variables
are referred to later in the chapter, when the steps in creating a process are described.
Table 6-2
Process-Related Kernel Variables
Variable
Type
Description
PsActiveProcessHead
Queue header
List head of process blocks
PsIdleProcess
EPROCESS
Idle process block
PsInitialSystemProcess
Pointer to EPROCESS
Pointer to the process block
of the initial system process
that contains the system
threads
PspCreateProcessNotifyRoutine
Array of pointers
Array of pointers to routines
to be called on process creation and deletion (maximum of eight)
PspCreateProcessNotifyRoutineCount
DWORD
Count of registered process
notification routines
PspLoadImageNotifyRoutine
Array of pointers
Array of pointers to routines
to be called on image load
PspLoadImageNotifyRoutineCount
DWORD
Count of registered imageload notification routines
PspCidTable
Pointer to
HANDLE_TABLE
Handle table for process
and thread client IDs
Performance Counters
Windows maintains a number of counters with which you can track the processes running on
your system; you can retrieve these counters programmatically or view them with the Performance tool. Table 6-3 lists the performance counters relevant to processes (except for memory management and I/O-related counters, which are described in Chapters 7 and 9,
respectively).
Table 6-3
Process-Related Performance Counters
Object: Counter
Function
Process: % Privileged Time
Describes the percentage of time that the threads in the process
have run in kernel mode during a specified interval.
Process: % Processor Time
Describes the percentage of CPU time that the threads in the
process have used during a specified interval. This count is the
sum of % Privileged Time and % User Time.
Process: % User Time
Describes the percentage of time that the threads in the process
have run in user mode during a specified interval.
Copyrighted material.
298
Microsoft Windows Internals, Fourth Edition
Table 6-3 Process-Related Performance Counters
Object: Counter
Function
Process: Elapsed Time
Describes the total elapsed time in seconds since this process was
created.
Process: ID Process
Returns the process ID. This ID applies only while the process exists because process IDs are reused.
Process: Creating Process ID
Returns the process ID of the creating process. This value isn’t updated if the creating process exits.
Process: Thread Count
Returns the number of threads in the process.
Process: Handle Count
Returns the number of handles open in the process.
Relevant Functions
For reference purposes, some of the Windows functions that apply to processes are described
in Table 6-4. For further information, consult the Windows API documentation in the MSDN
Library.
Table 6-4 Process-Related Functions
Function
Description
CreateProcess
Creates a new process and thread using the caller’s security
identification
CreateProcessAsUser
Creates a new process and thread with the specified alternate
security token
CreateProcessWithLogonW
Creates a new process and thread to run under the credentials
of the specified username and password
CreateProcessWithTokenW
Creates a new process and thread with the specified alternate
security token, with additional options such as allowing the user
profile to be loaded
OpenProcess
Returns a handle to the specified process object
ExitProcess
Ends a process, and notifies all attached DLLs
TerminateProcess
Ends a process without notifying the DLLs
FlushInstructionCache
Empties the specified process’s instruction cache
GetProcessTimes
Obtains a process’s timing information, describing how much
time the process has spent in user and kernel mode
GetExitCodeProcess
Returns the exit code for a process, indicating how and why the
process shut down
GetCommandLine
Returns a pointer to the command-line string passed to the current process
GetCurrentProcess
Returns a pseudo handle for the current process
GetCurrentProcessId
Returns the ID of the current process
Copyrighted material.
Chapter 6:
Table 6-4
Processes, Threads, and Jobs
299
Process-Related Functions
Function
Description
GetProcessVersion
Returns the major and minor versions of the Windows version on
which the specified process expects to run
GetStartupInfo
Returns the contents of the STARTUPINFO structure specified
during CreateProcess
GetEnvironmentStrings
Returns the address of the environment block
GetEnvironmentVariable
Returns a specific environment variable
Get/SetProcessShutdownParame- Defines the shutdown priority and number of retries for the curters
rent process
GetGuiResources
Returns a count of User and GDI handles
EXPERIMENT: Using the Kernel Debugger !process Command
The kernel debugger !process command displays a subset of the information in an EPROCESS block. This output is arranged in two parts for each process. First you see the information about the process, as shown here (when you don’t specify a process address or
ID, !process lists information for the active process on the current CPU):
lkd> !process
PROCESS 8575f030 SessionId: 0 Cid: 08d0
Peb: 7ffdf000 ParentCid: 0360
DirBase: 1a81b000 ObjectTable: e12bd418 HandleCount: 65.
Image: windbg.exe
VadRoot 857f05e0 Vads 71 Clone 0 Private 1152. Modified 98. Locked 1.
DeviceMap e1e96c88
Token
e1f5b8a8
ElapsedTime
1:23:06.0219
UserTime
0:00:11.0897
KernelTime
0:00:07.0450
QuotaPoolUsage[PagedPool]
38068
QuotaPoolUsage[NonPagedPool]
2840
Working Set Sizes (now,min,max) (2552, 50, 345) (10208KB, 200KB, 1380KB)
PeakWorkingSetSize
2715
VirtualSize
41 Mb
PeakVirtualSize
41 Mb
PageFaultCount
3658
MemoryPriority
BACKGROUND
BasePriority
8
CommitCharge
1566
After the basic process output comes a list of the threads in the process. That output is
explained in the “Experiment: Using the Kernel Debugger !thread Command” section
later in the chapter. Other commands that display process information include !handle,
which dumps the process handle table (which is described in more detail in the section
“Object Handles and the Process Handle Table” in Chapter 3). Process and thread security structures are described in Chapter 8.
Copyrighted material.
300
Microsoft Windows Internals, Fourth Edition
Flow of CreateProcess
So far in this chapter, you’ve seen the structures that are part of a process and the API functions with which you (and the operating system) can manipulate processes. You’ve also found
out how you can use tools to view how processes interact with your system. But how did those
processes come into being, and how do they exit once they’ve fulfilled their purpose? In the
following sections, you’ll discover how a Windows process comes to life.
A Windows process is created when an application calls one of the process creation functions,
such as CreateProcess, CreateProcessAsUser, CreateProcessWithTokenW, or CreateProcessWithLogonW. Creating a Windows process consists of several stages carried out in three parts of
the operating system: the Windows client-side library Kernel32.dll, the Windows executive,
and the Windows subsystem process (Csrss). Because of the multiple environment subsystem
architecture of Windows, creating a Windows executive process object (which other subsystems
can use) is separated from the work involved in creating a Windows process. So, although the
following description of the flow of the Windows CreateProcess function is complicated, keep in
mind that part of the work is specific to the semantics added by the Windows subsystem as
opposed to the core work needed to create a Windows executive process object.
The following list summarizes the main stages of creating a process with the Windows CreateProcess function. The operations performed in each stage are described in detail in the subsequent sections.
Note
Many steps of CreateProcess are related to the setup of the process virtual address
space and therefore refer to many memory management terms and structures that are defined
in Chapter 7.
1. Open the image file (.exe) to be executed inside the process.
2. Create the Windows executive process object.
3. Create the initial thread (stack, context, and Windows executive thread object).
4. Notify the Windows subsystem of the new process so that it can set up for the new process and thread.
5. Start execution of the initial thread (unless the CREATE_ SUSPENDED flag was specified).
6. In the context of the new process and thread, complete the initialization of the address
space (such as load required DLLs) and begin execution of the program.
Figure 6-5 shows an overview of the stages Windows follows to create a process.
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
301
Creating process
Stage 1
Open EXE and
create section
object
Stage 2
Create
Windows
process object
Stage 3
Create
Windows
thread object
Stage 4
Stage 5
Figure 6-5
Notify
Windows
subsystem
Windows subsystem
Set up for new
process and
thread
New process
Start execution
of the initial
thread
Final
process/image
initialization
Return
to caller!
Start execution
at entry point
to image
Stage 6
The main stages of process creation
Before opening the executable image to run, CreateProcess performs the following steps:
■
In CreateProcess, the priority class for the new process is specified as independent bits in
the CreationFlags parameter. Thus, you can specify more than one priority class for a single CreateProcess call. Windows resolves the question of which priority class to assign to
the process by choosing the lowest-priority class set.
■
If no priority class is specified for the new process, the priority class defaults to Normal
unless the priority class of the process that created it is Idle or Below Normal, in which
case the priority class of the new process will have the same priority as the creating class.
■
If a Real-time priority class is specified for the new process and the process’s caller
doesn’t have the Increase Scheduling Priority privilege, the High priority class is used
instead. In other words, CreateProcess doesn’t fail just because the caller has insufficient
privileges to create the process in the Real-time priority class; the new process just won’t
have as high a priority as Real-time.
■
All windows are associated with desktops, the graphical representation of a workspace.
If no desktop is specified in CreateProcess, the process is associated with the caller’s current desktop.
Copyrighted material.
302
Microsoft Windows Internals, Fourth Edition
Stage 1: Opening the Image to Be Executed
As illustrated in Figure 6-6, the first stage in CreateProcess is to find the appropriate Windows
image that will run the executable file specified by the caller and to create a section object to
later map it into the address space of the new process. If no image name is specified, the first
token of the command line (defined to be the first part of the command-line string ending
with a space or tab that is a valid file specification) is used as the image filename.
On Windows XP and Windows Server 2003, CreateProcess checks whether software restriction policies on the machine prevent the image from being run. (See Chapter 8 for a complete
description of software restriction policies.)
If the executable file specified is a Windows .exe, it is used directly. If it’s not a Windows .exe
(for example, if it’s an MS-DOS, Win16, or a POSIX application), CreateProcess goes through a
series of steps to find a Windows support image to run it. This process is necessary because
non-Windows applications aren’t run directly—Windows instead uses one of a few special
support images that in turn are responsible for actually running the non-Windows program.
For example, if you attempt to run a POSIX application, CreateProcess identifies it as such and
changes the image to be run on the Windows executable file Posix.exe. If you attempt to run
an MS-DOS or a Win16 executable, the image to be run becomes the Windows executable
Ntvdm.exe. In short, you can’t directly create a process that is not a Windows process. If Windows can’t find a way to resolve the activated image as a Windows process (as shown in Table
6-5), CreateProcess fails.
Run Cmd.exe
Run Ntvdm.exe
MS-DOS .bat
or .cmd
Win16
Use .exe directly
Windows
What kind of
application is it?
OS/2 1.x
Run Os2.exe
Figure 6-6
Copyrighted material.
POSIX
Run Posix.exe
MS-DOS .exe,
.com, or .pif
Run Ntvdm.exe
Choosing a Windows image to activate
Chapter 6:
Table 6-5
Processes, Threads, and Jobs
303
Decision Tree for Stage 1 of CreateProcess
If the image is a/an
And this will happen
This image will run
POSIX executable file
Posix.exe
CreateProcess restarts Stage 1.
MS-DOS application with an .exe,
a .com, or a .pif extension
Ntvdm.exe
CreateProcess restarts Stage 1.
Win16 application
Ntvdm.exe
Command procedure (application with Cmd.exe
a .bat or a .cmd extension)
CreateProcess restarts Stage 1.
CreateProcess restarts Stage 1.
Specifically, the decision tree that CreateProcess goes through to run an image is as follows:
■
If the image is an MS-DOS application with an .exe, a .com, or a .pif extension, a message
is sent to the Windows subsystem to check whether an MS-DOS support process
(Ntvdm.exe, specified in the registry value HKLM\SYSTEM\CurrentControlSet\Control\WOW\ cmdline) has already been created for this session. If a support process has
been created, it is used to run the MS-DOS application. (The Windows subsystem sends
the message to the VDM [Virtual DOS Machine] process to run the new image.) Then
CreateProcess returns. If a support process hasn’t been created, the image to be run
changes to Ntvdm.exe and CreateProcess restarts at Stage 1.
■
If the file to run has a .bat or a .cmd extension, the image to be run becomes Cmd.exe,
the Windows command prompt, and CreateProcess restarts at Stage 1. (The name of the
batch file is passed as the first parameter to Cmd.exe.)
■
If the image is a Win16 (Windows 3.1) executable, CreateProcess must decide whether a
new VDM process must be created to run it or whether it should use the default sessionwide shared VDM process (which might not yet have been created). The CreateProcess
flags CREATE_SEPARATE_WOW_VDM and CREATE_SHARED_WOW_VDM control
this decision. If these flags aren’t specified, the registry value HKLM\SYSTEM\CurrentControlSet\Control\WOW\ DefaultSeparateVDM dictates the default behavior. If the
application is to be run in a separate VDM, the image to be run changes to the value of
HKLM\SYSTEM\CurrentControlSet\Control\WOW\wowcmdline and CreateProcess
restarts at Stage 1. Otherwise, the Windows subsystem sends a message to see whether
the shared VDM process exists and can be used. (If the VDM process is running on a different desktop or isn’t running under the same security as the caller, it can’t be used and
a new VDM process must be created.) If a shared VDM process can be used, the Windows subsystem sends a message to it to run the new image and CreateProcess returns. If
the VDM process hasn’t yet been created (or if it exists but can’t be used), the image to
be run changes to the VDM support image and CreateProcess restarts at Stage 1.
Copyrighted material.
304
Microsoft Windows Internals, Fourth Edition
At this point, CreateProcess has successfully opened a valid Windows executable file and
created a section object for it. The object isn’t mapped into memory yet, but it is open. Just
because a section object has been successfully created doesn’t mean that the file is a valid
Windows image, however; it could be a DLL or a POSIX executable. If the file is a POSIX
executable, the image to be run changes to Posix.exe and CreateProcess restarts from the beginning of Stage 1. If the file is a DLL, CreateProcess fails.
Now that CreateProcess has found a valid Windows executable image, it looks in the registry
under HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution
Options to see whether a subkey with the filename and extension of the executable image
(but without the directory and path information—for example, Image.exe) exists there. If it
does, CreateProcess looks for a value named Debugger for that key. If this is present, the image
to be run becomes the string in that value and CreateProcess restarts at Stage 1.
Tip
You can take advantage of this CreateProcess behavior and debug the startup code of
Windows service processes before they start rather than attach the debugger after starting the
service, which doesn’t allow you to debug the startup code.
Stage 2: Creating the Windows Executive Process Object
At this point, CreateProcess has opened a valid Windows executable file and created a section
object to map it into the new process address space. Next it creates a Windows executive process object to run the image by calling the internal system function NtCreateProcess. Creating
the executive process object (which is done by the creating thread) involves the following substages:
■
Setting up the EPROCESS block
■
Creating the initial process address space
■
Initializing the kernel process block (KPROCESS)
■
Concluding the setup of the process address space (which includes initializing the
working set list and virtual address space descriptors and mapping the image into
address space)
■
Setting up the PEB
■
Completing the setup of the executive process object
Note
The only time there won’t be a parent process is during system initialization. After that
point, a parent process is always required to provide a security context for the new process.
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
305
Stage 2A: Setting Up the EPROCESS Block
This substage involves nine steps:
1. Allocate and initialize the Windows EPROCESS block.
2. Inherit the process affinity mask from the parent process.
3. The process minimum and maximum working set size are set to the values of PsMinimumWorkingSet and PsMaximumWorkingSet, respectively.
4. Set the new process’s quota block to the address of its parent process’s quota block, and
increment the reference count for the parent’s quota block.
5. Inherit the Windows device name space (including the definition of drive letters, COM
ports, and so on).
6. Store the parent process’s process ID in the InheritedFromUniqueProcessId field in the
new process object.
7. Create the process’s primary access token (a duplicate of its parent’s primary token).
New processes inherit the security profile of their parents. If the CreateProcessAsUser
function is being used to specify a different access token for the new process, the token
is then changed appropriately.
8. The process handle table is initialized. If the inherit handles flag is set for the parent process, any inheritable handles are copied from the parent’s object handle table into the
new process. (For more information about object handle tables, see Chapter 3.)
9. Set the new process’s exit status to STATUS_PENDING.
Stage 2B: Creating the Initial Process Address Space
The initial process address space consists of the following pages:
■
Page directory (and it’s possible there’ll be more than one for systems with page tables
more than two levels, such as x86 systems in PAE mode or 64-bit systems)
■
Hyperspace page
■
Working set list
To create these three pages, the following steps are taken:
1. Page table entries are created in the appropriate page tables to map the initial pages.
The number of pages is deducted from the kernel variable MmTotalCommittedPages and
added to MmProcessCommit.
2. The systemwide default process minimum working set size (PsMinimumWorkingSet) is
deducted from MmResidentAvailablePages.
3. The page table pages for the nonpaged portion of system space and the system cache are
mapped into the process.
Copyrighted material.
306
Microsoft Windows Internals, Fourth Edition
Stage 2C: Creating the Kernel Process Block
The next stage of CreateProcess is the initialization of the KPROCESS block, which contains a
pointer to a list of kernel threads. (The kernel has no knowledge of handles, so it bypasses the
object table.) The kernel process block also points to the process’s page table directory (which
is used to keep track of the process’s virtual address space), the total time the process’s
threads have executed, the process’s default base-scheduling priority (which starts as Normal,
or 8, unless the parent process was set to Idle or Below Normal, in which case the setting is
inherited), the default processor affinity for the threads in the process, and the initial value of
the process default quantum (which is described in more detail in the “Thread Scheduling”
section later in the chapter), which is taken from the value of PspForegroundQuantum[0], the
first entry in the systemwide quantum array.
Note
The default initial quantum differs between Windows client and server systems. For more
information on thread quantums, turn to their discussion in the section “Thread Scheduling.”
Stage 2D: Concluding the Setup of the Process Address Space
Setting up the address space for a new process is somewhat complicated, so let’s look at what’s
involved one step at a time. To get the most out of this section, you should have some familiarity
with the internals of the Windows memory manager, which are described in Chapter 7.
■
The virtual memory manager sets the value of the process’s last trim time to the current
time. The working set manager (which runs in the context of the balance set manager
system thread) uses this value to determine when to initiate working set trimming.
■
The memory manager initializes the process’s working set list—page faults can now be
taken.
■
The section (created when the image file was opened) is now mapped into the new process’s address space, and the process section base address is set to the base address of
the image.
■
Ntdll.dll is mapped into the process.
■
The systemwide national language support (NLS) tables are mapped into the process’s
address space.
Note
POSIX processes clone the address space of their parents, so they don’t have to go
through these steps to create a new address space. In the case of POSIX applications, the new
process’s section base address is set to that of its parent process and the parent’s PEB is cloned
for the new process.
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
307
Stage 2E: Setting Up the PEB
CreateProcess allocates a page for the PEB and initializes a number of fields, which are
described in Table 6-6.
Table 6-6
Initial Values of the Fields of the PEB
Field
Initial Value
ImageBaseAddress
Base address of section
NumberOfProcessors
KeNumberProcessors kernel variable
NtGlobalFlag
NtGlobalFlag kernel variable
CriticalSectionTimeout
MmCriticalSectionTimeout kernel variable
HeapSegmentReserve
MmHeapSegmentReserve kernel variable
HeapSegmentCommit
MmHeapSegmentCommit kernel variable
HeapDeCommitTotalFreeThreshold
MmHeapDeCommitTotalFreeThreshold kernel variable
HeapDeCommitFreeBlockThreshold
MmHeapDeCommitFreeBlockThreshold kernel variable
NumberOfHeaps
0
MaximumNumberOfHeaps
(Size of a page - size of a PEB) / 4
ProcessHeaps
First byte after PEB
OSMajorVersion
NtMajorVersion kernel variable
OSMinorVersion
NtMinorVersion kernel variable
OSBuildNumber
NtBuildNumber kernel variable & 0x3FFF
OSPlatformId
2
If the image file specifies explicit Windows version values, this information replaces the initial
values shown in Table 6-6. The mapping from image version information fields to PEB fields
is described in Table 6-7.
Table 6-7
Windows Replacements for Initial PEB Values
Field Name
Value Taken from Image Header
OSMajorVersion
OptionalHeader.Win32VersionValue & 0xFF
OSMinorVersion
(OptionalHeader.Win32VersionValue >> 8) & 0xFF
OSBuildNumber
(OptionalHeader.Win32VersionValue >> 16) & 0x3FFF
OSPlatformId
(OptionalHeader.Win32VersionValue >> 30) ^ 0x2
Stage 2F: Completing the Setup of the Executive Process Object
Before the handle to the new process can be returned, a few final setup steps must be completed:
1. If systemwide auditing of processes is enabled (either as a result of local policy settings
or group policy settings from a domain controller), the process’s creation is written to
the Security event log.
Copyrighted material.
308
Microsoft Windows Internals, Fourth Edition
2. If the parent process was contained in a job, the new process is added to the job. (Jobs
are described at the end of this chapter.)
3. If the image header characteristic’s IMAGE_FILE_UP_SYSTEM_ ONLY flag is set (indicating that the image can run only on a uniprocessor system), a single CPU is chosen for
all the threads in this new process to run on. This choosing process is done by simply
cycling through the available processors—each time this type of image is run, the next
processor is used. In this way, these types of images are spread out across the processors
evenly.
4. If the image specifies an explicit processor affinity mask (for example, a field in the configuration header), this value is copied to the PEB and later set as the default process
affinity mask.
5. CreateProcess inserts the new process block at the end of the Windows list of active processes (PsActiveProcessHead).
6. The process’s creation time is set, the handle to the new process is returned to the caller
(CreateProcess in Kernel32.dll).
Stage 3: Creating the Initial Thread and Its Stack and Context
At this point, the Windows executive process object is completely set up. It still has no thread,
however, so it can’t do anything yet. Before the thread can be created, it needs a stack and a
context in which to run, so these are set up now. The stack size for the initial thread is taken
from the image—there’s no way to specify another size.
Now the initial thread can be created, which is done by calling NtCreateThread. The thread
parameter (which can’t be specified in CreateProcess but can be specified in CreateThread) is
the address of the PEB. This parameter will be used by the initialization code that runs in the
context of this new thread (as described in Stage 6). However, the thread won’t do anything
yet—it is created in a suspended state and isn’t resumed until the process is completely initialized (as described in Stage 5). NtCreateThread calls PspCreateThread (a function also used to
create system threads) and performs the following steps:
1. The thread count in the process object is incremented.
2. An executive thread block (ETHREAD) is created and initialized.
3. A thread ID is generated for the new thread.
4. The TEB is set up in the user-mode address space of the process.
5. The user-mode thread start address is stored in the ETHREAD. For Windows threads, this
is the system-supplied thread startup function in Kernel32.dll (BaseProcessStart for the
first thread in a process and BaseThreadStart for additional threads). The user’s specified
Windows start address is stored in the ETHREAD block in a different location so that the
system-supplied thread startup function can call the user-specified startup function.
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
309
6. KeInitThread is called to set up the KTHREAD block. The thread’s initial and current base
priorities are set to the process’s base priority, and its affinity and quantum are set to that
of the process. This function also sets the initial thread ideal processor. (See the section
“Ideal and Last Processor” for a description of how this is chosen.) KeInitThread next allocates a kernel stack for the thread and initializes the machine-dependent hardware context for the thread, including the context, trap, and exception frames. The thread’s context
is set up so that the thread will start in kernel mode in KiThreadStartup. Finally, KeInitThread sets the thread’s state to Initialized and returns to PspCreateThread.
7. Any registered systemwide thread creation notification routines are called.
8. The thread’s access token is set to point to the process access token, and an access check
is made to determine whether the caller has the right to create the thread. This check
will always succeed if you’re creating a thread in the local process, but it might fail if
you’re using CreateRemoteThread to create a thread in another process and the process
creating the thread doesn’t have the debug privilege enabled.
9. Finally, the thread is readied for execution.
Stage 4: Notifying the Windows Subsystem about the New Process
If software restriction policies dictate, a restricted token is created for the new process. At this
point, all the necessary executive process and thread objects have been created. Kernel32.dll
next sends a message to the Windows subsystem so that it can set up for the new process and
thread. The message includes the following information:
■
Process and thread handles
■
Entries in the creation flags
■
ID of the process’s creator
■
Flag indicating whether the process belongs to a Windows application (so that Csrss
can determine whether or not to show the startup cursor)
The Windows subsystem performs the following steps when it receives this message:
1. CreateProcess duplicates a handle for the process and thread. In this step, the usage
count of the process and the thread is incremented from 1 (which was set at creation
time) to 2.
2. If a process priority class isn’t specified, CreateProcess sets it according to the algorithm
described earlier in this section.
3. The Csrss process block is allocated.
4. The new process’s exception port is set to be the general function port for the Windows
subsystem so that the Windows subsystem will receive a message when an exception
occurs in the process. (For further information on exception handling, see Chapter 3.)
Copyrighted material.
310
Microsoft Windows Internals, Fourth Edition
5. If the process is being debugged (that is, if it is attached to a debugger process), the process debug port is set to the Windows subsystem’s general function port. This setting
ensures that Windows will send debug events that occur in the new process (such as
thread creation and deletion, exceptions, and so on) as messages to the Windows subsystem so that it can then dispatch the events to the process that is acting as the new
process’s debugger.
6. The Csrss thread block is allocated and initialized.
7. CreateProcess inserts the thread in the list of threads for the process.
8. The count of processes in this session is incremented.
9. The process shutdown level is set to 0x280 (the default process shutdown level—see SetProcessShutdownParameters in the MSDN Library documentation for more information).
10. The new process block is inserted into the list of Windows subsystemwide processes.
11. The per-process data structure used by the kernel-mode part of the Windows subsystem
(W32PROCESS structure) is allocated and initialized.
12. The application start cursor is displayed. This cursor is the familiar arrow with an hourglass attached—the way that Windows says to the user, “I’m starting something, but you
can use the cursor in the meantime.” If the process doesn’t make a GUI call after 2 seconds, the cursor reverts to the standard pointer. If the process does make a GUI call in
the allotted time, CreateProcess waits 5 seconds for the application to show a window.
After that time, CreateProcess will reset the cursor again.
Stage 5: Starting Execution of the Initial Thread
At this point, the process environment has been determined, resources for its threads to use
have been allocated, the process has a thread, and the Windows subsystem knows about the
new process. Unless the caller specified the CREATE_ SUSPENDED flag, the initial thread is
now resumed so that it can start running and perform the remainder of the process initialization work that occurs in the context of the new process (Stage 6).
Stage 6: Performing Process Initialization in the Context of the New
Process
The new thread begins life running the kernel-mode thread startup routine KiThreadStartup.
KiThreadStartup lowers the thread’s IRQL level from DPC/dispatch level to APC level and
then calls the system initial thread routine, PspUserThreadStartup. The user-specified thread
start address is passed as a parameter to this routine.
On Windows 2000, PspUserThreadStartup first enables working set expansion. If the process
being created is a debuggee, all threads in the process are suspended. (Threads might have been
created during Stage 3.) A create process message is then sent to the process’s debug port
(which is the Windows subsystem function port, because this is a Windows process) so that the
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
311
subsystem can deliver the process startup debug event (CREATE_PROCESS_DEBUG_INFO) to
the appropriate debugger process. PspUserThreadStartup then waits for the Windows subsystem
to get the reply from the debugger (via the ContinueDebugEvent function). When the Windows
subsystem replies, all the threads are resumed.
On Windows XP and Windows Server 2003, PspUserThreadStartup checks whether application prefetching is enabled on the system and, if so, calls the logical prefetcher to process the
prefetch instruction file (if it exists) and prefetch pages referenced during the first 10 seconds
the process started last time. (For details on the prefetcher, see Chapter 3.) Finally, PspUserThreadStartup queues a user-mode APC to run the image loader initialization routine (LdrInitializeThunk in Ntdll.dll). The APC will be delivered when the thread attempts to return to
user mode.
When PspUserThreadStartup returns to KiThreadStartup, it returns from kernel mode, the APC is
delivered, and LdrInitializeThunk is called. The LdrInitializeThunk routine initializes the loader,
heap manager, NLS tables, thread-local storage (TLS) array, and critical section structures. It
then loads any required DLLs and calls the DLL entry points with the DLL_PROCESS_
ATTACH function code. (See the sidebar “Side-by-Side Assemblies” for a description of a
mechanism introduced in Windows XP to address DLL versioning problems.)
Finally, the image begins execution in user mode when the loader initialization returns to the
user mode APC dispatcher, which then calls the thread’s start function that was pushed on
the user stack when the user APC was delivered.
Side-by-Side Assemblies
A problem that has long plagued Windows users is “DLL hell.” You enter DLL hell when
you install an application that replaces one or more core system DLLs, such as those for
common controls, the Microsoft Visual Basic runtime, or MFC. Application installation
programs make these replacements to ensure that the application runs properly, but at
the same time, updated DLLs might have incompatibilities with other already-installed
applications.
Windows 2000 partly addressed DLL hell by preventing the modification of core
system DLLs with the Windows File Protection feature, and by allowing applications to
use private copies of these core DLLs. To use a private copy of a DLL instead of the one
in the system directory, an application’s installation must include a file named
Application.exe.local (where Application is the name of the application’s executable),
which directs the loader to first look for DLLs in that directory. This type of DLL redirection avoids application/DLL incompatibility problems, but it does so at the expense of
sharing DLLs, which is one of the points of DLLs in the first place. In addition, any DLLs
that are loaded from the list of KnownDLLs (DLLs that are permanently mapped into
memory) or that are loaded by those DLLs cannot be redirected using this mechanism.
Copyrighted material.
312
Microsoft Windows Internals, Fourth Edition
To further address application and DLL compatibility while allowing sharing, Windows
XP introduces shared assemblies. An assembly consists of a group of resources, including DLLs, and an XML manifest file that describes the assembly and its contents. An
application references an assembly through the existence of its own XML manifest. The
manifest can be a file in the application’s installation directory that has the same name
as the application with “.manifest” appended (for example, application.exe.manifest), or
it can be linked into the application as a resource. The manifest describes the application
and its dependence on assemblies.
There are two types of assemblies: private and shared. The difference between the two is
that shared assemblies are digitally signed so that corruption or modification of their
contents can be detected. In addition, shared assemblies are stored under the \Windows\Winsxs directory, whereas private assemblies are stored in an application’s installation directory. Thus, shared assemblies also have an associated catalog file (.cat) that
contains its digital signature information. Shared assemblies can be “side-by-side”
assemblies because multiple versions of a DLL can reside on a system simultaneously,
with applications dependent on a particular version of a DLL always using that particular version.
An assembly’s manifest file typically has a name that includes the name of the assembly,
version information, some text that represents a unique signature, and the extension
“.manifest”. The manifests are stored in \Windows\Winsxs\Manifests, and the rest of
the assembly’s resources are stored in subdirectories of \Windows\Winsxs that have
the same name as the corresponding manifest files, with the exception of the trailing
.manifest extension.
An example of a shared assembly is version 6 of the Windows common controls DLL,
comctl32.dll, which is new to Windows XP. Its manifest file is named \Windows\Winsxs\Manifest\x86_Microsoft.Windows.CommonControls_6595b64144ccf1df_6.0.0.0_x-ww_1382d70a.manifest. It has an associated
catalog file (which is the same name with the .cat extension) and a subdirectory of Winsxs that includes comctl32.dll.
Version 6 of Comctl32.dll includes integration with Windows XP themes, and because
applications not written with themes-support in mind might not appear correctly with
the new DLL, it’s available only to applications that explicitly reference the shared
assembly containing it—the version of Comctl32.dll installed in \Windows\System32 is
an instance of version 5.x, which is not theme aware. When an application loads, the
loader looks for the application’s manifest, and if one exists, loads the DLLs from the
assemblies specified. DLLs not included in assemblies referenced in the manifest are
loaded in the traditional way. Legacy applications, therefore, link against the version in
\Windows\System32, whereas theme-aware applications can specify the new version in
their manifest.
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
313
You can see the effect of a manifest that directs the system to use the new common control library on Windows XP by running the User State Migration Wizard (\Windows\System32\Usmt\Migwiz.exe) with and without its manifest file:
1. Run it, and notice the Windows XP themes on the buttons in the wizard.
2. Open the Migwiz.exe.manifest file in Notepad, and locate the inclusion of the version 6 common control library.
3. Rename the Migwiz.exe.manifest to Migwiz.exe.manifest.bak.
4. Rerun the wizard, and notice the unthemed buttons.
5. Restore the manifest file to its original name.
A final advantage that shared assemblies have is that a publisher can issue a publisher
configuration, which can redirect all applications that use a particular assembly to use
an updated version. Publishers would do this if they were preserving backward compatibility while addressing bugs. Ultimately, however, because of the flexibility inherent in
the assembly model, an application could decide to override the new setting and continue to use an older version.
Thread Internals
Now that we’ve dissected processes, let’s turn our attention to the structure of a thread.
Unless explicitly stated otherwise, you can assume that anything in this section applies to
both user-mode threads and kernel-mode system threads (which are described in Chapter 2).
Data Structures
At the operating-system level, a Windows thread is represented by an executive thread
(ETHREAD) block, which is illustrated in Figure 6-7. The ETHREAD block and the structures
it points to exist in the system address space, with the exception of the thread environment
block (TEB), which exists in the process address space. In addition, the Windows subsystem
process (Csrss) maintains a parallel structure for each thread created in a Windows process.
Also, for threads that have called a Windows subsystem USER or GDI function, the kernelmode portion of the Windows subsystem (Win32k.sys) maintains a per-thread data structure
(called the W32THREAD structure) that the ETHREAD block points to.
Copyrighted material.
314
Microsoft Windows Internals, Fourth Edition
TEB
KTHREAD
Create and exit times
Process ID
EPROCESS
Thread start address
Access token
Impersonation information
LPC message information
Timer information
Pending I/O requests
Figure 6-7
Structure of the executive thread block
Most of the fields illustrated in Figure 6-7 are self-explanatory. The first field is the kernel
thread (KTHREAD) block. Following that are the thread identification information, the process identification information (including a pointer to the owning process so that its environment information can be accessed), security information in the form of a pointer to the access
token and impersonation information, and finally, fields relating to LPC messages and pending I/O requests. As you can see in Table 6-8, some of these key fields are covered in more
detail elsewhere in this book. For more details on the internal structure of an ETHREAD
block, you can use the kernel debugger dt command to display the format of the structure.
Table 6-8 Key Contents of the Executive Thread Block
Element
Description
Additional Reference
KTHREAD
See Table 6-9.
Thread time
Thread create and exit time
information.
Process identification
Process ID and pointer to
EPROCESS block of the process
that the thread belongs to.
Start address
Address of thread start routine.
Impersonation information
Access token and impersonation
level (if the thread is impersonating a client).
Chapter 8
LPC information
Message ID that the thread is
waiting for and address of
message.
Local procedure calls
(Chapter 3)
I/O information
List of pending I/O request
packets (IRPs).
I/O system (Chapter 9)
Let’s take a closer look at two of the key thread data structures referred to in the preceding
text: the KTHREAD block and the TEB. The KTHREAD block contains the information that
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
315
the Windows kernel needs to access to perform thread scheduling and synchronization on
behalf of running threads. Its layout is illustrated in Figure 6-8.
Dispatcher header
Total user time
Total kernel time
Kernel stack information
System service table
Thread-scheduling information
Trap frame
Thread-local storage array
Synchronization information
List of pending APCs
Timer block and wait block
List of objects thread is waiting on
TEB
Figure 6-8
Structure of the kernel thread block
The key fields of the KTHREAD block are described briefly in Table 6-9.
Table 6-9
Key Contents of the KTHREAD Block
Element
Description
Additional Reference
Dispatcher header
Because the thread is an object that
can be waited on, it starts with a standard kernel dispatcher object header.
Kernel Dispatcher objects
(Chapter 3)
Execution time
Total user and kernel CPU time.
Pointer to kernel stack
information
Base and upper address of the kernel
stack.
Memory management
(Chapter 7)
Pointer to system service
table
Each thread starts out with this field
service table pointing to the main
system service table
(KeServiceDescriptorTable). When a
thread first calls a Windows GUI service, its system service table is changed
to one that includes the GDI and USER
services in Win32k.sys.
System Service Dispatching (Chapter 3)
Scheduling information
Base and current priority, quantum, affinity mask, ideal processor, scheduling
state, freeze count, and suspend count.
Thread Scheduling
Wait blocks
The thread block contains four built-in
wait blocks so that wait blocks don’t
have to be allocated and initialized
each time the thread waits for something. (One wait block is dedicated to
timers.)
Synchronization
(Chapter 3)
Copyrighted material.
316
Microsoft Windows Internals, Fourth Edition
Table 6-9 Key Contents of the KTHREAD Block
Element
Description
Additional Reference
Wait information
List of objects the thread is waiting for,
wait reason, and time at which the
thread entered the wait state.
Synchronization
(Chapter 3)
Mutant list
List of mutant objects the thread owns.
Synchronization
(Chapter 3)
APC queues
List of pending user-mode and kernelmode APCs, and alertable flag.
Aynchronous Procedure
Call (APC) Interrrupts
(Chapter 3)
Timer block
Built-in timer block (also a corresponding wait block).
Queue list
Pointer to queue object that the
thread is associated with.
Pointer to TEB
Thread ID, TLS information, PEB pointer, and GDI and OpenGL information.
Synchronization
(Chapter 3)
EXPERIMENT: Displaying ETHREAD and KTHREAD Structures
The ETHREAD and KTHREAD structures can be displayed with the dt command in the
kernel debugger. The following output shows the format of an ETHREAD:
lkd> dt nt!_ethread
nt!_ETHREAD
+0x000 Tcb
: _KTHREAD
+0x1c0 CreateTime
: _LARGE_INTEGER
+0x1c0 NestedFaultCount : Pos 0, 2 Bits
+0x1c0 ApcNeeded
: Pos 2, 1 Bit
+0x1c8 ExitTime
: _LARGE_INTEGER
+0x1c8 LpcReplyChain
: _LIST_ENTRY
+0x1c8 KeyedWaitChain
: _LIST_ENTRY
+0x1d0 ExitStatus
: Int4B
+0x1d0 OfsChain
: Ptr32 Void
+0x1d4 PostBlockList
: _LIST_ENTRY
+0x1dc TerminationPort : Ptr32 _TERMINATION_PORT
+0x1dc ReaperLink
: Ptr32 _ETHREAD
+0x1dc KeyedWaitValue
: Ptr32 Void
+0x1e0 ActiveTimerListLock : Uint4B
+0x1e4 ActiveTimerListHead : _LIST_ENTRY
+0x1ec Cid
: _CLIENT_ID
+0x1f4 LpcReplySemaphore : _KSEMAPHORE
+0x1f4 KeyedWaitSemaphore : _KSEMAPHORE
+0x208 LpcReplyMessage : Ptr32 Void
+0x208 LpcWaitingOnPort : Ptr32 Void
+0x20c ImpersonationInfo : Ptr32 _PS_IMPERSONATION_INFORMATION
+0x210 IrpList
: _LIST_ENTRY
+0x218 TopLevelIrp
: Uint4B
+0x21c DeviceToVerify
: Ptr32 _DEVICE_OBJECT
+0x220 ThreadsProcess
: Ptr32 _EPROCESS
Copyrighted material.
Chapter 6:
+0x224
+0x228
+0x228
+0x22c
+0x234
+0x238
+0x23c
+0x240
+0x244
+0x248
+0x248
+0x248
+0x248
+0x248
+0x248
+0x248
+0x248
+0x248
+0x248
+0x24c
+0x24c
+0x24c
+0x24c
+0x250
+0x250
+0x250
+0x250
+0x254
+0x255
Processes, Threads, and Jobs
StartAddress
: Ptr32 Void
Win32StartAddress : Ptr32 Void
LpcReceivedMessageId : Uint4B
ThreadListEntry : _LIST_ENTRY
RundownProtect
: _EX_RUNDOWN_REF
ThreadLock
: _EX_PUSH_LOCK
LpcReplyMessageId : Uint4B
ReadClusterSize : Uint4B
GrantedAccess
: Uint4B
CrossThreadFlags : Uint4B
Terminated
: Pos 0, 1 Bit
DeadThread
: Pos 1, 1 Bit
HideFromDebugger : Pos 2, 1 Bit
ActiveImpersonationInfo : Pos 3, 1 Bit
SystemThread
: Pos 4, 1 Bit
HardErrorsAreDisabled : Pos 5, 1 Bit
BreakOnTermination : Pos 6, 1 Bit
SkipCreationMsg : Pos 7, 1 Bit
SkipTerminationMsg : Pos 8, 1 Bit
SameThreadPassiveFlags : Uint4B
ActiveExWorker
: Pos 0, 1 Bit
ExWorkerCanWaitUser : Pos 1, 1 Bit
MemoryMaker
: Pos 2, 1 Bit
SameThreadApcFlags : Uint4B
LpcReceivedMsgIdValid : Pos 0, 1 Bit
LpcExitThreadCalled : Pos 1, 1 Bit
AddressSpaceOwner : Pos 2, 1 Bit
ForwardClusterOnly : UChar
DisablePageFaultClustering : UChar
The KTHREAD can be displayed with a similar command:
lkd> dt nt!_kthread
nt!_KTHREAD
+0x000 Header
+0x010 MutantListHead
+0x018 InitialStack
+0x01c StackLimit
+0x020 Teb
+0x024 TlsArray
+0x028 KernelStack
+0x02c DebugActive
+0x02d State
+0x02e Alerted
+0x030 Iopl
+0x031 NpxState
+0x032 Saturation
+0x033 Priority
+0x034 ApcState
+0x04c ContextSwitches
+0x050 IdleSwapBlock
+0x051 Spare0
+0x054 WaitStatus
Copyrighted material.
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
_DISPATCHER_HEADER
_LIST_ENTRY
Ptr32 Void
Ptr32 Void
Ptr32 Void
Ptr32 Void
Ptr32 Void
UChar
UChar
[2] UChar
UChar
UChar
Char
Char
_KAPC_STATE
Uint4B
UChar
[3] UChar
Int4B
317
318
Microsoft Windows Internals, Fourth Edition
EXPERIMENT: Using the Kernel Debugger !thread Command
The kernel debugger !thread command dumps a subset of the information in the thread
data structures. Some key elements of the information the kernel debugger displays
can’t be displayed by any utility: internal structure addresses; priority details; stack
information; the pending I/O request list; and, for threads in a wait state, the list of
objects the thread is waiting for.
To display thread information, use either the !process command (which displays all the
thread blocks after displaying the process block) or the !thread command to dump a
specific thread. The output of the thread information, along with some annotations of
key fields, is shown here:
Address of
ETHREAD
Thread ID
Address of thread
environment block
THREAD 83160f0 Cid: 9f.3dTeb: 7ffdc000 Win32Thread:
WAIT: (WrUserRequest) UserMode Non-Alertable
808e9d60 SynchronizationEvent
e153d2c8
Thread state
Objects being waited on
Not imersonating
Owning Process 81b44880
Address of EPROCESS for owning process
953945
Wait Time (seconds)
2697
LargeStack
Context Switch Count
UserTime
0:00:00.0289
Actual thread
KernelTime
0:00:04.0644
start address
Start Address kernal32!BaseProcessStart (0x77e8f268)
Address of user thread function
Win32 Start Address 0x020d9d98
Stack Init f7818000 Current f7817bb0 Base f7818000 Limit f7812000 Call 0
Priority 14 BasePriority 9 PriorityDecrement 6 DecrementCount 13
Kernal stack not resident.
ChildEBP RetAddr
Args to Child
Priority
information
F7817bb0 8008f430 00000001 00000000 00000000 ntoskrnl!KiSwapThreadExit
F7817c50 de0119ec 00000001 00000000 00000000 ntoskrnl!KeWaitForSingleObject+0x2a0
F7817cc0 de0123f4 00000001 00000000 00000000 win32k!xxxSleepThread+0x23c
F7817d10 de01f2f0 00000001 00000000 00000000 win32k!xxxInternalGetMessage+0x504
F7817d80 800bab58 00000001 00000000 00000000 win32k!NtUserGetMessage+0x58
F7817df0 77d887d0 00000001 00000000 00000000 ntoskrnl!KiSystemServiceEndAddress+0x4
0012fef0 00000000 00000001 00000000 00000000 user32!GetMessageW+0x30
Stack dump
EXPERIMENT: Viewing Thread Information
The following output is the detailed display of a process produced by using the Tlist utility in the Windows Debugging Tools. Notice that the thread list shows the
“Win32StartAddress.” This is the address passed to the CreateThread function by the
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
319
application. All the other utilities, except Process Explorer, that show the thread start
address show the actual start address (a function in Kernel32.dll), not the applicationspecified start address.
C:\> tlist winword
155 WINWORD.EXE
Document1 - Microsoft Word
CWD:
C:\book\
CmdLine: "C:\Program Files\Microsoft Office\Office\WINWORD.EXE"
VirtualSize:
64448 KB
PeakVirtualSize:
106748 KB
WorkingSetSize: 1104 KB
PeakWorkingSetSize: 6776 KB
NumberOfThreads: 2
156 Win32StartAddr:0x5032cfdb LastErr:0x00000000 State:Waiting
167 Win32StartAddr:0x00022982 LastErr:0x00000000 State:Waiting
0x50000000 WINWORD.EXE
5.0.2163.1 shp 0x77f60000 ntdll.dll
5.0.2191.1 shp 0x77f00000 KERNEL32.dll
§
list of DLLs loaded in process
The TEB, illustrated in Figure 6-9, is the only data structure explained in this section that
exists in the process address space (as opposed to the system space).
The TEB stores context information for the image loader and various Windows DLLs. Because
these components run in user mode, they need a data structure writable from user mode.
That’s why this structure exists in the process address space instead of in the system space,
where it would be writable only from kernel mode. You can find the address of the TEB with
the kernel debugger !thread command.
Exception list
Stack base
Stack limit
Subsystem thread information block (TIB)
Fiber information
Thread ID
Active RPC handle
PEB
LastError value
Count of owned critical sections
Current locale
User32 client information
GDI32 information
OpenGL information
TLS array
Winsock data
Figure 6-9
Copyrighted material.
Fields of the thread environment block
320
Microsoft Windows Internals, Fourth Edition
EXPERIMENT: Examining the TEB
You can dump the TEB structure with the !teb command in the kernel debugger. The
output looks like this:
kd> !teb
TEB at 7ffde000
ExceptionList:
StackBase:
StackLimit:
SubSystemTib:
FiberData:
ArbitraryUserPointer:
Self:
EnvironmentPointer:
ClientId:
RpcHandle:
Tls Storage:
PEB Address:
LastErrorValue:
LastStatusValue:
Count Owned Locks:
0006b540
00070000
00065000
00000000
00001e00
00000000
7ffde000
00000000
00000254 . 000007ac
00000000
00000000
7ffdf000
2
c0000034
0
HardErrorMode:
0
Kernel Variables
As with processes, a number of Windows kernel variables control how threads run. Table 6-10
shows the kernel-mode kernel variables that relate to threads.
Table 6-10
Thread-Related Kernel Variables
Variable
Type
Description
PspCreateThreadNotifyRoutine
Array of pointers
Array of pointers to routines to
be called on during thread creation and deletion (maximum of
eight).
PspCreateThreadNotifyRoutineCount
DWORD
Count of registered thread-notification routines.
PspCreateProcessNotifyRoutine
Array of pointers
Array of pointers to routines to
be called on during process creation and deletion (maximum of
eight).
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
321
Performance Counters
Most of the key information in the thread data structures is exported as performance
counters, which are listed in Table 6-11. You can extract much information about the internals
of a thread just by using the Performance tool in Windows.
Table 6-11
Thread-Related Performance Counters
Object: Counter
Function
Process: Priority Base
Returns the current base priority of the process. This is the
starting priority for threads created within this process.
Thread: % Privileged Time
Describes the percentage of time that the thread has run in
kernel mode during a specified interval.
Thread: % Processor Time
Describes the percentage of CPU time that the thread has
used during a specified interval. This count is the sum of %
Privileged Time and % User Time.
Thread: % User Time
Describes the percentage of time that the thread has run in
user mode during a specified interval.
Thread: Context Switches/Sec
Returns the number of context switches per second that the
system is executing.
Thread: Elapsed Time
Returns the amount of CPU time (in seconds) that the thread
has consumed.
Thread: ID Process
Returns the process ID of the thread’s process. This ID is valid
only during the process’s lifetime because process IDs are reused.
Thread: ID Thread
Returns the thread’s thread ID. This ID is valid only during the
thread’s lifetime because thread IDs are reused.
Thread: Priority Base
Returns the thread’s current base priority. This number might
be different from the thread’s starting base priority.
Thread: Priority Current
Returns the thread’s current dynamic priority.
Thread: Start Address
Returns the thread’s starting virtual address (Note: This address will be the same for most threads.)
Thread: Thread State
Returns a value from 0 through 7 relating to the current state
of the thread.
Thread: Thread Wait Reason
Returns a value from 0 through 19 relating to the reason why
the thread is in a wait state.
Copyrighted material.
322
Microsoft Windows Internals, Fourth Edition
Relevant Functions
Table 6-12 shows the Windows functions for creating and manipulating threads. This table
doesn’t include functions that have to do with thread scheduling and priorities—those are
included in the section “Thread Scheduling” later in this chapter.
Table 6-12
Windows Thread Functions
Function
Description
CreateThread
Creates a new thread
CreateRemoteThread
Creates a thread in another process
OpenThread
Opens an existing thread
ExitThread
Ends execution of a thread normally
TerminateThread
Terminates a thread
GetExitCodeThread
Gets another thread’s exit code
GetThreadTimes
Returns timing information for a thread
GetCurrentProcess
Returns a pseudo handle for the current thread
GetCurrentProcessId
Returns the thread ID of the current thread
GetThreadId
Returns the thread ID of the specified thread
Get/SetThreadContext
Returns or changes a thread’s CPU registers
GetThreadSelectorEntry
Returns another thread’s descriptor table entry (applies only to x86
systems)
Birth of a Thread
A thread’s life cycle starts when a program creates a new thread. The request filters down to
the Windows executive, where the process manager allocates space for a thread object and
calls the kernel to initialize the kernel thread block. The steps in the following list are taken
inside the Windows CreateThread function in Kernel32.dll to create a Windows thread.
1. CreateThread creates a user-mode stack for the thread in the process’s address space.
2. CreateThread initializes the thread’s hardware context (CPU architecture–specific). (For
further information on the thread context block, see the Windows API reference documentation on the CONTEXT structure.)
3. NtCreateThread is called to create the executive thread object in the suspended state. For
a description of the steps performed by this function, see the description of Stage 3 and
Stage 6 in the section “Flow of CreateProcess.”
4. CreateThread notifies the Windows subsystem about the new thread, and the subsystem
does some setup work for the new thread.
5. The thread handle and the thread ID (generated during step 3) are returned to the
caller.
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
323
6. Unless the caller created the thread with the CREATE_SUSPENDED flag set, the thread
is now resumed so that it can be scheduled for execution. When the thread starts running, it executes the steps described in the earlier section “Stage 6: Performing Process
Initialization in the Context of the New Process” before calling the actual user’s specified
start address.
Examining Thread Activity
Besides the Performance tool, several other tools expose various elements of the state of Windows threads. (The tools that show thread-scheduling information are listed in the section
“Thread Scheduling.”) These tools are itemized in Figure 6-10.
Note
To display thread details with Tlist, you must type tlist xxx, where xxx is a process
image name or window title. (Wildcards are supported.)
Object
Perfmon Pviewer Pstat
Qslice
Tlist
KD
Process
!thread Explorer Pslist
Thread ID
Actual start address
Win32 start address
Current address
Number of context switches
Total user time
Total privileged time
Elapsed time
Thread state
Reason for wait state
Last error
Percentage of CPU time
Percentage of user time
Percentage of privileged time
Address of TEB
Address of ETHREAD
Objects waiting on
Figure 6-10
Thread-related tools and their functions
Process Explorer provides easy access to thread activity within a process. This is especially important if you are trying to determine why a process is running that is hosting multiple services (such
as Svchost.exe, Dllhost.exe, Inetinfo.exe, or the System process) or why a process is hung.
Copyrighted material.
324
Microsoft Windows Internals, Fourth Edition
To view the threads in a process, select a process and open the process properties (doubleclick on the process or click on the Process, Properties menu item). Then click on the Threads
tab. This tab shows a list of the threads in the process. For each thread it shows the percentage
of CPU consumed (based on the refresh interval configured), the number of context switches
to the thread, and the thread start address. You can sort by any of these three columns.
New threads that are created are highlighted in green, and threads that exit are highlighted in
red. (The highlight duration can be configured with the Options, Configure Highlighting
menu item.) This might be helpful to discover unnecessary thread creation occurring in a process. (In general, threads should be created at process startup, not every time a request is processed inside a process.)
As you select each thread in the list, Process Explorer displays the thread ID, start time, state,
CPU time counters, number of context switches, and the base and current priority. There is a
Kill button, which will terminate an individual thread, but this should be used with extreme
care.
The context switch delta represents the number of times that thread began running in
between the refreshes configured for Process Explorer. It provides a different way to determine thread activity than using the percentage of CPU consumed. In some ways it is better
because many threads run for such a short amount of time that they are seldom (if ever) the
currently running thread when the interval clock timer interrupt occurs, and therefore, are
not charged for their CPU time. For example, if you add the context switch delta column to
the process display and sort by that column, you will see processes that have threads running
but that also have a CPU time percentage of zero (or very small).
The thread start address is displayed in the form “module!function”, where module is the name
of the .exe or .dll. The function name relies on access to symbol files for the module. (See
“Experiment: Viewing Process Details with Process Explorer” in Chapter 1.) If you are unsure
what the module is, press the Module button. This opens an Explorer file properties window
for the module containing the thread’s start address (for example, the .exe or .dll).
Note For threads created by the Windows CreateThread function, Process Explorer displays
the function passed to CreateThread, not the actual thread start function. That is because all
Windows threads start at a common process or thread startup wrapper function (BaseProcessStart or BaseThreadStart in Kernel32.dll). If Process Explorer showed the actual start
address, most threads in processes would appear to have started at the same address, which
would not be helpful in trying to understand what code the thread was executing.
However, the thread start address displayed might not be enough information to pinpoint what
the thread is doing and which component within the process is responsible for the CPU consumed by the thread. This is especially true if the thread start address is a generic startup
function (for example, if the function name does not indicate what the thread is actually doing).
In this case, examining the thread stack might answer the question. To view the stack for a
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
325
thread, double-click on the thread of interest (or select it and click the Stack button). Process
Explorer displays the thread’s stack (both user and kernel, if the thread was in kernel mode).
Note
While the user-mode debuggers (Windbg, Ntsd, and Cdb) permit you to attach to a
process and display the user stack for a thread, Process Explorer shows both the user and kernel stack in one easy click of a button. You can also examine user and kernel thread stacks using
Livekd from www.sysinternals.com. However, it is more difficult to use. Note that running
Windbg in local kernel debugging mode, which is supported only on Windows XP or Windows
Server 2003, does not show thread stacks.
Viewing the thread stack can also help you determine why a process is hung. As an example,
on one system, Microsoft PowerPoint was hanging for one minute on startup. To determine
why it was hung, after starting PowerPoint, Process Explorer was used to examine the thread
stack of the one thread in the process. The result is shown in Figure 6-11.
Figure 6-11
Hung Thread Stack in PowerPoint
This thread stack shows that PowerPoint (line 10) called a function in Mso.dll (the central
Microsoft Office Dll), which called the OpenPrinterW function in Winspool.drv (a Dll used to
connect to printers). Winspool.drv then dispatched to a function OpenPrinterRPC, which then
called a function in the RPC runtime Dll, indicating it was sending the request to a remote
printer. So, without having to understand the internals of PowerPoint, the module and function names displayed on the thread stack indicate that the thread was waiting to connect to a
network printer. On this particular system, there was a network printer that was not responding, which explained the delay starting PowerPoint. (Microsoft Office applications connect to
all configured printers at process startup.) The connection to that printer was deleted from
the user’s system, and the problem went away.
Thread Scheduling
This section describes the Windows scheduling policies and algorithms. The first subsection
provides a condensed description of how scheduling works on Windows and a definition of
key terms. Then Windows priority levels are described from both the Windows API and the
Windows kernel points of view. After a review of the relevant Windows functions and
Copyrighted material.
326
Microsoft Windows Internals, Fourth Edition
_Windows utilities and tools that relate to scheduling, the detailed data structures and algorithms that make up the Windows scheduling system are presented, with uniprocessor systems examined first and then multiprocessor systems.
Overview of Windows Scheduling
Windows implements a priority-driven, preemptive scheduling system—the highest-priority
runnable (ready) thread always runs, with the caveat that the thread chosen to run might be
limited by the processors on which the thread is allowed to run, a phenomenon called processor affinity. By default, threads can run on any available processor, but you can alter processor
affinity by using one of the Windows scheduling functions listed in Table 6-13 (shown later in
the chapter) or by setting an affinity mask in the image header.
EXPERIMENT: Viewing Ready Threads
You can view the list of ready threads with the kernel debugger !ready command. This
command displays the thread or list of threads that are ready to run at each priority
level. In the following example, two threads are ready to run at priority 10 and six at priority 8. Because this output was generated using LiveKd on a uniprocessor system, the
current thread will always be the kernel debugger (Kd or WinDbg).
kd> !ready 1
Ready Threads at priority 10
THREAD 810de030 Cid 490.4a8
THREAD 81110030 Cid 490.48c
Ready Threads at priority 8
THREAD 811fe790 Cid 23c.274
THREAD 810bec70 Cid 23c.50c
THREAD 8003a950 Cid 23c.550
THREAD 85ac2db0 Cid 23c.5e4
THREAD 827318d0 Cid 514.560
THREAD 8117adb0 Cid 2d4.338
Teb: 7ffd9000
Teb: 7ffde000
Win32Thread: e297e008 READY
Win32Thread: e29425a8 READY
Teb:
Teb:
Teb:
Teb:
Teb:
Teb:
Win32Thread:
Win32Thread:
Win32Thread:
Win32Thread:
Win32Thread:
Win32Thread:
7ffdb000
7ffd9000
7ffda000
7ffd8000
7ffd9000
7ffaf000
e258cda8
e2ccf748
e29a7ae8
e297a9e8
00000000
00000000
READY
READY
READY
READY
READY
READY
When a thread is selected to run, it runs for an amount of time called a quantum. A quantum
is the length of time a thread is allowed to run before another thread at the same priority level
(or higher, which can occur on a multiprocessor system) is given a turn to run. Quantum values can vary from system to system and process to process for any of three reasons: system
configuration settings (long or short quantums), foreground/background status of the process, or use of the job object to alter the quantum. (Quantums are described in more detail in
the “Quantum” section later in the chapter.) A thread might not get to complete its quantum,
however. Because Windows implements a preemptive scheduler, if another thread with a
higher priority becomes ready to run, the currently running thread might be preempted
before finishing its time slice. In fact, a thread can be selected to run next and be preempted
before even beginning its quantum!
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
327
The Windows scheduling code is implemented in the kernel. There’s no single “scheduler”
module or routine, however—the code is spread throughout the kernel in which schedulingrelated events occur. The routines that perform these duties are collectively called the kernel’s
dispatcher. The following events might require thread dispatching:
■
A thread becomes ready to execute—for example, a thread has been newly created or has
just been released from the wait state.
■
A thread leaves the running state because its time quantum ends, it terminates, it yields
execution, or it enters a wait state.
■
A thread’s priority changes, either because of a system service call or because Windows
itself changes the priority value.
■
A thread’s processor affinity changes so that it will no longer run on the processor on
which it was running.
At each of these junctions, Windows must determine which thread should run next. When
Windows selects a new thread to run, it performs a context switch to it. A context switch is the
procedure of saving the volatile machine state associated with a running thread, loading
another thread’s volatile state, and starting the new thread’s execution.
As already noted, Windows schedules at the thread granularity. This approach makes sense
when you consider that processes don’t run but only provide resources and a context in
which their threads run. Because scheduling decisions are made strictly on a thread basis, no
consideration is given to what process the thread belongs to. For example, if process A has 10
runnable threads, process B has 2 runnable threads, and all 12 threads are at the same priority, each thread would theoretically receive one-twelfth of the CPU time—Windows wouldn’t
give 50 percent of the CPU to process A and 50 percent to process B.
Priority Levels
To understand the thread-scheduling algorithms, you must first understand the priority levels
that Windows uses. As illustrated in Figure 6-12, internally, Windows uses 32 priority levels,
ranging from 0 through 31. These values divide up as follows:
■
Sixteen real-time levels (16 through 31)
■
Fifteen variable levels (1 through 15)
■
One system level (0), reserved for the zero page thread
Copyrighted material.
328
Microsoft Windows Internals, Fourth Edition
31
16 real-time levels
16
15
15 variable levels
1
0
Figure 6-12
1 system level
(Zero page thread, one per system)
Thread priority levels
Thread priority levels are assigned from two different perspectives: those of the Windows API
and those of the Windows kernel. The Windows API first organizes processes by the priority
class to which they are assigned at creation (Real-time, High, Above Normal, Normal, Below
Normal, and Idle) and then by the relative priority of the individual threads within those processes (Time-critical, Highest, Above-normal, Normal, Below-normal, Lowest, and Idle).
In the Windows API, each thread has a base priority that is a function of its process priority
class and its relative thread priority. The mapping from Windows priority to internal Windows numeric priority is shown in Figure 6-13.
Whereas a process has only a single base priority value, each thread has two priority values:
current and base. Scheduling decisions are made based on the current priority. As explained
in the following section on priority boosting, the system under certain circumstances
increases the priority of threads in the dynamic range (1 through 15) for brief periods. Windows never adjusts the priority of threads in the real-time range (16 through 31), so they
always have the same base and current priority.
A thread’s initial base priority is inherited from the process base priority. A process, by default,
inherits its base priority from the process that created it. This behavior can be overridden on
the CreateProcess function or by using the command-line START command. A process priority
can also be changed after being created by using the SetPriorityClass function or various tools
that expose that function such as Task Manager and Process Explorer (by right-clicking on the
process and choosing a new priority class). For example, you can lower the priority of a CPUintensive process so that it does not interfere with normal system activities. Changing the priority of a process changes the thread priorities up or down, but their relative settings remain
the same. It usually doesn’t make sense, however, to change individual thread priorities within
a process, because unless you wrote the program or have the source code, you don’t really
know what the individual threads are doing, and changing their relative importance might
cause the program not to behave in the intended fashion.
Copyrighted material.
Chapter 6:
Real-time
time critical
Processes, Threads, and Jobs
329
31
Real-time
Real-time
Levels 16–31
Real-time idle
Dynamic time
critical
24
High
16
15
Above
Normal
13
Normal
10
Dynamic
Levels 1–15
Below
Normal
8
Idle
6
4
Dynamic idle
Figure 6-13
1
0
Used for zero page thread—not available to Win32 applications
Mapping of Windows kernel priorities to the Windows API
Normally, the process base priority (and therefore the starting thread base priority) will
default to the value at the middle of each process priority range (24, 13, 10, 8, 6, or 4). However, some Windows system processes (such as the Session Manager, service controller, and
local security authentication server) have a base process priority slightly higher than the
default for the Normal class (8). This higher default value ensures that the threads in these
processes will all start at a higher priority than the default value of 8. These system processes
use an internal system call (NtSetInformationProcess) to set its process base priority to a
numeric value other than the normal default starting base priority.
Copyrighted material.
330
Microsoft Windows Internals, Fourth Edition
Windows Scheduling APIs
The Windows API functions that relate to thread scheduling are listed in Table 6-13. (For
more information, see the Windows API reference documentation.)
Table 6-13
Scheduling-Related APIs and Their Functions
API
Function
Suspend/ResumeThread
Suspends or resumes a paused thread from execution.
Get/SetPriorityClass
Returns or sets a process’s priority class (base priority).
Get/SetThreadPriority
Returns or sets a thread’s priority (relative to its process base
priority).
Get/SetProcessAffinityMask
Returns or sets a process’s affinity mask.
SetThreadAffinityMask
Sets a thread’s affinity mask (must be a subset of the process’s affinity mask) for a particular set of processors, restricting it to running on those processors.
SetInformationJobObject
Sets attributes for a job; some of the attributes affect scheduling, such as affinity and priority. (See the “Job Objects” section later in the chapter for a description of the job object.)
GetLogicalProcessorInformation
Returns details about processor hardware configuration (for
hyperthreaded and NUMA systems).
Get/SetThreadPriorityBoost
Returns or sets the ability for Windows to boost the priority
of a thread temporarily. (This ability applies only to threads
in the dynamic range.)
SetThreadIdealProcessor
Establishes a preferred processor for a particular thread, but
doesn’t restrict the thread to that processor.
Get/SetProcessPriorityBoost
Returns or sets the default priority boost control state of the
current process. (This function is used to set the thread priority boost control state when a thread is created.)
SwitchToThread
Yields execution to another thread (at priority 1 or higher)
that is ready to run on the current processor.
Sleep
Puts the current thread into a wait state for a specified time
interval (figured in milliseconds [msec]). A zero value relinquishes the rest of the thread’s quantum.
SleepEx
Causes the current thread to go into a wait state until either
an I/O completion callback is completed, an APC is queued
to the thread, or the specified time interval ends.
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
331
Relevant Tools
The following table lists the tools related to thread scheduling. You can change (and view) the
base process priority with a number of different tools, such as Task Manager, Process
Explorer, Pview, or Pviewer. Note that you can kill individual threads in a process with Process
Explorer. This should be done, of course, with extreme care.
You can view individual thread priorities with the Performance tool, Process Explorer, Pslist,
Pview, Pviewer, and Pstat. While it might be useful to increase or lower the priority of a process, it typically does not make sense to adjust individual thread priorities within a process
because only a person who thoroughly understands the program would understand the relative importance of the threads within the process.
Object
Taskman Perfmon Pviewer
Pview
Pstat
KD
Process
!thread Explorer
Process priority class
Process base priority
Thread base priority
Thread current priority
The only way to specify a starting priority class for a process is with the start command in the
Windows command prompt. If you want to have a program start every time with a specific priority, you can define a shortcut to use the start command by beginning the command with
cmd /c. This runs the command prompt, executes the command on the command line, and
terminates the command prompt. For example, to run Notepad in the low-process priority,
the shortcut would be cmd /c start /low notepad.exe.
EXPERIMENT: Examining and Specifying Process and Thread
Priorities
Try the following experiment:
1. From the command prompt, type start /realtime notepad. Notepad should open.
2. Run either Process Explorer or the Process Viewer utility in the Support Tools
(Pviewer.exe), and select Notepad.exe from the list of processes, as shown here.
Notice that the dynamic priority of the thread in Notepad is 24. This matches the
real-time value shown in Figure 6-13.
Copyrighted material.
332
Microsoft Windows Internals, Fourth Edition
3. Task Manager can show you similar information. Press Ctrl+Shift+Esc to start Task
Manager, and go to the Processes tab. Right-click on the Notepad.exe process, and
select the Set Priority option. You can see that Notepad’s process priority class is
Realtime, as shown in the following dialog box.
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
333
Windows System Resource Manager
Windows Server 2003, Enterprise Edition and Windows Server 2003, Datacenter Edition include an optionally installable component called Windows System Resource
Manager (WSRM). It permits the administrator to configure policies that specify CPU
utilization, affinity settings, and memory limits (both physical and virtual) for processes.
In addition, WSRM can generate resource utilization reports that can be used for
accounting and verification of service-level agreements with users.
Policies can be applied for specific applications (by matching the name of the image
with or without specific command-line arguments), users, or groups. The policies can
be scheduled to take effect at certain periods or can be enabled all the time.
After you have set a resource-allocation policy to manage specific processes, the WSRM
service monitors CPU consumption of managed processes and adjusts process base priorities when those processes do not meet their target CPU allocations.
The physical memory limitation uses the function SetProcessWorkingSetSizeEx to set a
hard-working set maximum. The virtual memory limit is implemented by the service
checking the private virtual memory consumed by the processes. (See Chapter 7 for an
explanation of these memory limits.) If this limit is exceeded, WSRM can be configured
to either kill the processes or write an entry to the event log. This behavior could be
used to detect a process with a memory leak before it consumes all the available committed virtual memory on the system. Note that WSRM memory limits do not apply to
Address Windowing Extensions (AWE) memory, large page memory, or kernel memory
(nonpaged or paged pool).
Real-Time Priorities
You can raise or lower thread priorities within the dynamic range in any application; however,
you must have the increase scheduling priority privilege to enter the real-time range. Be aware
that many important Windows kernel-mode system threads run in the real-time priority
range, so if threads spend excessive time running in this range, they might block critical system functions (such as in the memory manager, cache manager, or other device drivers).
Note
As illustrated in the following figure showing the x86 Interrupt Request Levels (IRQLs),
although Windows has a set of priorities called real-time, they are not real-time in the common
definition of the term. This is because Windows doesn’t provide true real-time operating system facilities, such as guaranteed interrupt latency or a way for threads to obtain a guaranteed
execution time. For more information, see the sidebar “Windows and Real-Time Processing” in
Chapter 3 as well as the MSDN Library article “Real-Time Systems and Microsoft Windows NT.”
Copyrighted material.
334
Microsoft Windows Internals, Fourth Edition
Interrupt Levels vs. Priority Levels
As illustrated in the following figure, threads normally run at IRQL 0 or 1. (For a
description of how Windows uses interrupt levels, see Chapter 3.) User-mode threads
always run at IRQL 0. Because of this, no user-mode thread, regardless of its priority,
blocks hardware interrupts (although high-priority real-time threads can block the execution of important system threads). Only kernel-mode APCs execute at IRQL 1
because they interrupt the execution of a thread. (For more information on APCs, see
Chapter 3.) Threads running in kernel mode can raise IRQL to higher levels, though—
for example, while executing a system call that involves thread dispatching.
IRQLs
31
Thread priorities 0–31
High
30
Power fail
29
Inter-processor interrupt
28
Clock
27
Profile
26
Device n
3
Device 1
2
DPC/dispatch
1
APC
0
Passive
Hardware interrupts
Software interrupts
Thread States
Before you can comprehend the thread-scheduling algorithms, you need to understand the
various execution states that a thread can be in. Figure 6-14 illustrates the state transitions for
threads on Windows 2000 and Windows XP. (The numeric values shown represent the value
of the thread state performance counter.) More details on what happens at each transition are
included later in this section.
Copyrighted material.
Chapter 6:
Init (0)
preempt
Processes, Threads, and Jobs
Standby (3)
Ready (1)
335
preemption,
quantum end
Running (2)
voluntary
switch
Transition (6)
Figure 6-14
Waiting (5)
Terminate (4)
Thread states on Windows 2000 and Windows XP
The thread states are as follows:
■
Ready
■
Standby A thread in the standby state has been selected to run next on a particular processor. When the correct conditions exist, the dispatcher performs a context switch to
this thread. Only one thread can be in the standby state for each processor on the system. Note that a thread can be preempted out of the standby state before it ever executes
(if, for example, a higher priority thread becomes runnable before the standby thread
begins execution).
■
Running
■
Waiting
Copyrighted material.
A thread in the ready state is waiting to execute. When looking for a thread to
execute, the dispatcher considers only the pool of threads in the ready state.
Once the dispatcher performs a context switch to a thread, the thread enters
the running state and executes. The thread’s execution continues until its quantum
ends (and another thread at the same priority is ready to run), it is preempted by a
higher priority thread, it terminates, it yields execution, or it voluntarily enters the wait
state.
A thread can enter the wait state in several ways: a thread can voluntarily wait
for an object to synchronize its execution, the operating system can wait on the thread’s
behalf (such as to resolve a paging I/O), or an environment subsystem can direct the
thread to suspend itself. When the thread’s wait ends, depending on the priority, the
thread either begins running immediately or is moved back to the ready state.
336
Microsoft Windows Internals, Fourth Edition
■
Transition A thread enters the transition state if it is ready for execution but its kernel
stack is paged out of memory. Once its kernel stack is brought back into memory, the
thread enters the ready state.
■
Terminated
■
Initialized This state is used internally while a thread is being created.
When a thread finishes executing, it enters the terminated state. Once the
thread is terminated, the executive thread block (the data structure in nonpaged pool
that describes the thread) might or might not be deallocated. (The object manager sets
policy regarding when to delete the object.)
EXPERIMENT: Thread-Scheduling State Changes
You can watch thread-scheduling state changes with the Performance tool in Windows.
This utility can be useful when you’re debugging a multithreaded application if you’re
unsure about the state of the threads running in the process. To watch thread-scheduling state changes by using the Performance tool, follow these steps:
1. Run the Microsoft Notepad utility (Notepad.exe).
2. Start the Performance tool by selecting Programs from the Start menu and then
selecting Performance from the Adminstrative Tools menu.
3. Select chart view if you’re in some other view.
4. Right-click on the graph, and choose Properties.
5. Click the Graph tab, and change the chart vertical scale maximum to 7. (As you’ll
see from the explanation text for the performance counter, thread states are numbered from 0 through 7.) Click OK.
6. Click the Add button on the toolbar to bring up the Add Counters dialog box.
7. Select the Thread performance object, and then select the Thread State counter.
Click the Explain button to see the definition of the values:
8. In the Instances box, scroll down until you see the Notepad process (notepad/0);
select it, and click the Add button.
9. Scroll back up in the Instances box to the Mmc process (the Microsoft Management Console process running the System Monitor), select all the threads (mmc/
0, mmc/1, and so on), and add them to the chart by clicking the Add button.
Before you click Add, you should see something like the following dialog box.
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
337
10. Now close the Add Counters dialog box by clicking Close.
11. You should see the state of the Notepad thread (the very top line in the following
figure) as a 5, which, as shown in the explanation text you saw under step 5, represents the waiting state (because the thread is waiting for GUI input):
12. Notice that one thread in the Mmc process (running the Performance tool snapin) is in the running state (number 2). This is the thread that’s querying the thread
states, so it’s always displayed in the running state.
13. You’ll never see Notepad in the running state (unless you’re on a multiprocessor
system) because Mmc is always in the running state when it gathers the state of
the threads you’re monitoring.
Copyrighted material.
338
Microsoft Windows Internals, Fourth Edition
The state diagram for threads on Windows Server 2003 is shown in Figure 6-15. Notice the
new state called deferred ready. This state is used for threads that have been selected to run on
a specific processor but have not yet been scheduled. This new state exists so that the kernel
can minimize the amount of time the systemwide lock on the scheduling database is held.
(This process is explained further in the section “Multiprocessor Dispatcher Database.”)
Ready (1)
Init (0)
preempt
Standby (3)
preemption,
quantum end
Deferred
Ready (7)
Running (2)
voluntary
switch
Transition (6)
Figure 6-15
Waiting (5)
Terminate (4)
Thread states on Windows Server 2003
Dispatcher Database
To make thread-scheduling decisions, the kernel maintains a set of data structures known collectively as the dispatcher database, illustrated in Figure 6-16. The dispatcher database keeps
track of which threads are waiting to execute and which processors are executing which
threads.
Note
The dispatcher database on a uniprocessor system has the same structure as on multiprocessor Windows 2000 and Windows XP systems, but is different on Windows Server 2003
systems. These differences, as well as the differences in the way Windows selects threads to run
on multiprocessor systems, are explained in the section “Multiprocessor Systems.”
Copyrighted material.
Chapter 6:
Process
Processes, Threads, and Jobs
339
Process
Thread 1
Thread 2
Thread 3
Thread 4
Ready queues
31
0
Ready summary
31
Figure 6-16
0
Dispatcher database (uniprocessor and Windows 2000/XP multiprocessor)
The dispatcher ready queues (KiDispatcherReadyListHead) contain the threads that are in the
ready state, waiting to be scheduled for execution. There is one queue for each of the 32 priority levels. To speed up the selection of which thread to run or preempt, Windows maintains
a 32-bit bit mask called the ready summary (KiReadySummary). Each bit set indicates one or
more threads in the ready queue for that priority level. (Bit 0 represents priority 0, and so on.)
Table 6-15 lists the kernel-mode kernel variables that are related to thread scheduling on uniprocessor systems.
Table 6-14
Thread-Scheduling Kernel Variables
Variable
Type
Description
KiReadySummary
Bitmask (32 bits)
Bitmask of priority levels that have
one or more ready threads
KiDispatcherReadyListHead
Array of 32 list entries
List heads for the 32 ready queues
On uniprocessor systems, the dispatcher database is synchronized by raising IRQL to DPC/
dispatch level and SYNCH_LEVEL (both of which are defined as level 2). (For an explanation
of interrupt priority levels, see the “Trap Dispatching” section in Chapter 3.) Raising IRQL in
Copyrighted material.
340
Microsoft Windows Internals, Fourth Edition
this way prevents other threads from interrupting thread dispatching because threads normally
run at IRQL 0 or 1. On multiprocessor systems, more is required than raising IRQL because
each processor can, at the same time, raise to the same IRQL and attempt to operate on the dispatcher database. How Windows synchronizes access to the dispatcher database on multiprocessor systems is explained in the “Multiprocessor Systems” section later in the chapter.
Quantum
As mentioned earlier in the chapter, a quantum is the amount of time a thread gets to run
before Windows checks to see whether another thread at the same priority is waiting to run.
If a thread completes its quantum and there are no other threads at its priority, Windows permits the thread to run for another quantum.
On Windows 2000 Professional and Windows XP, threads run by default for 2 clock intervals;
on Windows Server systems, by default, a thread runs for 12 clock intervals. (We’ll explain
how you can change these values later.) The rationale for the longer default value on server
systems is to minimize context switching. By having a longer quantum, server applications
that wake up as the result of a client request have a better chance of completing the request
and going back into a wait state before their quantum ends.
The length of the clock interval varies according to the hardware platform. The frequency of
the clock interrupts is up to the HAL, not the kernel. For example, the clock interval for most
x86 uniprocessors is about 10 milliseconds and for most x86 multiprocessors it is about 15
milliseconds. (The actual clock rate is not exactly a round number of milliseconds—see the following experiment for a way to check the actual clock interval.)
EXPERIMENT: Determining the Clock Interval Frequency
The Windows GetSystemTimeAdjustment function returns the clock interval. To determine the clock interval, download and run the Clockres program from www.sysinternals.com. Here’s the output from a uniprocessor x86 system:
C:\>clockres
ClockRes - View the system clock resolution
By Mark Russinovich
SysInternals - www.sysinternals.com
The system clock interval is 10.014400 ms
Quantum Accounting
Each process has a quantum value in the kernel process block. This value is used when giving
a thread a new quantum. As a thread runs, its quantum is reduced at each clock interval. If
there is no remaining thread quantum, the quantum end processing is triggered. If there is
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
341
another thread at the same priority waiting to run, a context switch occurs to the next thread
in the ready queue. Note that when the clock interrupt interrupts a DPC or another interrupt
that was in progress, the thread that was in the running state has its quantum deducted, even
if it hadn’t been running for a full clock interval. If this was not done and device interrupts or
DPCs occurred right before the clock interval timer interrupts, threads might not ever get
their quantum reduced.
Internally, the quantum value is stored as a multiple of three times the number of clock ticks.
This means that on Windows 2000 and Windows XP systems, threads, by default, have a
quantum value of 6 (2 * 3), and Windows Server systems have a quantum value of 36 (12 * 3).
Each time the clock interrupts, the clock-interrupt routine deducts a fixed value (3) from the
thread quantum.
The reason quantum is stored internally in terms of a multiple of 3 quantum units per clock
tick rather than as single units is to allow for partial quantum decay on wait completion.
When a thread that has a current priority less than 16 and a base priority less than 14 executes
a wait function (such as WaitForSingleObject or WaitForMultipleObjects) that is satisfied immediately (for example, without having to wait), its quantum is reduced by 1 quantum unit. In
this way, threads that wait will eventually expire their quantum.
In the case where a wait is not satisfied immediately, threads below priority 16 also have their
quantum reduced by 1 unit (except if the thread is waking up to execute a kernel APC because
the wait code will charge the quantum after the wait is actually satisfied). However, before
doing the reduction, if the thread is at priority 14 or above, its quantum is reset to a full turn.
This is also done for threads running at less than 14 if they are not running with a special priority boost (such as is done for foreground processes or in the case of CPU starvation) and if
they are receiving a priority boost as a result of the unwait operation. (Priority boosting is
explained in the next section.)
This partial decay addresses the case in which a thread enters a wait state before the clock
interval timer fires. If this adjustment were not made, it would be possible for threads never to
have their quantums reduced. For example, if a thread ran, entered a wait state, ran again, and
entered another wait state but was never the currently running thread when the clock interval
timer fired, it would never have its quantum charged for the time it was running. Note that this
does not apply to zero timeout waits, but only wait operations that do not require waiting
because all the wait conditions are fulfilled at the time of the wait.
Controlling the Quantum
You can change the thread quantum for all processes, but you can choose only one of two settings: short (2 clock ticks, the default for client machines) or long (12 clock ticks, the default
for server systems).
Copyrighted material.
342
Microsoft Windows Internals, Fourth Edition
Note
By using the job object on a system running with long quantums, you can select other
quantum values for the processes in the job. For more information on the job object, see the
“Job Objects” section later in the chapter.
To change this on Windows 2000, right click My Computer, Properties (or go to Control Panel
and open the System settings applet), click the Advanced tab, and then click the Performance
Options button. The dialog box you will see is shown in Figure 6-17.
Figure 6-17
Quantum Configuration on Windows 2000
To change this on Windows XP and Windows Server 2003, right click My Computer, Properties, click the Advanced tab, click the Settings button in the Performance section, and finally
click the Advanced tab. The dialog box displayed is slightly different for Windows XP and
Windows Server 2003. These are shown in Figure 6-18.
Figure 6-18
Copyrighted material.
Quantum Configuration on Windows XP and Windows Server 2003
Chapter 6:
Processes, Threads, and Jobs
343
The Programs setting (called “Applications” on Windows 2000) designates the use of short,
variable quantums—the default for Windows 2000 Professional and Windows XP. If you
install Terminal Services on Windows Server systems and configure the server as an application server, this setting is selected so that the users on the terminal server will have the same
quantum settings that would normally be set on a desktop or client system. You might also
select this manually if you were running Windows Server as your desktop operating system.
The Background Services option designates the use of long, fixed quantums—the default for
Windows Server systems. The only reason why you might select this on a workstation system
is if you were using the workstation as a server system.
One additional difference between the Programs and Background Services settings is the
effect they have on the quantum of the threads in the foreground process. This is explained in
the next section.
Quantum Boosting
Prior to Windows NT 4.0, when a window was brought into the foreground on a workstation
or client system, all the threads in the foreground process (the process that owns the thread
that owns the window that’s in focus) received a priority boost of 2. This priority boost
remained in effect while any thread in the process owned the foreground window. The problem with this approach was that if you started a long-running, CPU-intensive process (such as
a spreadsheet recalculation) and then switched to another CPU-intensive process (such as a
computer-aided design tool, graphics editor, or a game), the process now running in the background would get little or no CPU time because the foreground process would have its
threads boosted by 2 (assuming the base priority of the threads in both processes are the
same) while it remained in the foreground.
This default behavior was changed as of Windows NT 4.0 Workstation to instead triple the
quantum of the threads in the foreground process. Thus, threads in the foreground process
run with a quantum of 6 clock ticks, whereas threads in other processes have the default
workstation quantum of 2 clock ticks. In this way, when you switch away from a CPU-intensive process, the new foreground process will get proportionally more of the CPU, because
when its threads run they will have a longer turn that background threads (again, assuming
the thread priorities are the same in both the foreground and background processes).
Note that this adjustment of quantums applies only to processes with a priority higher than
Idle on systems configured to Programs (or Applications, in Windows 2000) in the Performance Options settings described in the previous section. Thread quantums are not changed
for the foreground process on systems configured to Background Services (the default on
Windows Server systems).
Copyrighted material.
344
Microsoft Windows Internals, Fourth Edition
Quantum Settings Registry Value
The user interface to control quantum settings described earlier modify the registry value
HKLM\SYSTEM\CurrentControlSet\Control\PriorityControl\Win32PrioritySeparation. In
addition to specifying the relative length of thread quantums (short or long), this registry
value also defines whether or not threads in the foreground process should have their quantums boosted (and if so, the amount of the boost). This value consists of 6 bits divided into
the three 2-bit fields shown in Figure 6-19.
4
Short vs.
Long
Figure 6-19
2
Variable vs.
Fixed
0
Foreground
Quantum Boost
Fields of the Win32PrioritySeparation registry value
The fields shown in Figure 6-19 can be defined as follows:
■
Short vs. Long A setting of 1 specifies long, and 2 specifies short. A setting of 0 or 3
indicates that the default will be used (short for Windows 2000 Professional and Windows XP, long for Windows Server systems).
■
Variable vs. Fixed A setting of 1 means to vary the quantum for the foreground process,
and 2 means that quantum values don’t change for foreground processes. A setting of 0
or 3 means that the default (which is variable for Windows 2000 Professional and Windows XP and fixed for Windows Server systems) will be used.
■
This field (stored in the kernel variable PsPrioritySeparation) must have a value of 0, 1, or 2. (A setting of 3 is invalid and treated as 2.) It is used
as an index into a three-element byte array named PspForegroundQuantum to obtain the
quantum for the threads in the foreground process. The quantum for threads in background processes is taken from the first entry in this quantum table. Table 6-16 shows
the possible settings for PspForegroundQuantum.
Foreground Quantum Boost
Table 6-15
Quantum Values
Short
Long
Variable
6
12
18
12
24
36
Fixed
18
18
18
36
36
36
Note that when you’re using the Performance Options dialog box described earlier, you can
choose from only two combinations: short quantums with foreground quantums tripled, or
long quantums with no quantum changes for foreground threads. However, you can select
other combinations by modifying the Win32PrioritySeparation registry value directly.
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
345
Scheduling Scenarios
Windows bases the question of “Who gets the CPU?” on thread priority; but how does this
approach work in practice? The following sections illustrate just how priority-driven preemptive multitasking works on the thread level.
Voluntary Switch
First a thread might voluntarily relinquish use of the processor by entering a wait state on
some object (such as an event, a mutex, a semaphore, an I/O completion port, a process, a
thread, a window message, and so on) by calling one of the Windows wait functions (such as
WaitForSingleObject or WaitForMultipleObjects). Waiting for objects is described in more detail
in Chapter 3.
Figure 6-20 illustrates a thread entering a wait state and Windows selecting a new thread to
run.
Priority
Running
Ready
20
19
18
17
16
15
14
To wait state
Figure 6-20
Voluntary switching
In Figure 6-20, the top block (thread) is voluntarily relinquishing the processor so that the
next thread in the ready queue can run (as represented by the halo it has when in the Running
column). Although it might appear from this figure that the relinquishing thread’s priority is
being reduced, it’s not—it’s just being moved to the wait queue of the objects the thread is
waiting for. What about any remaining quantum for the thread? The quantum value isn’t reset
when a thread enters a wait state—in fact, as explained earlier, when the wait is satisfied, the
thread’s quantum value is decremented by 1 quantum unit, equivalent to one-third of a clock
interval (except for threads running at priority 14 or higher, which have their quantum reset
after a wait to a full turn).
Copyrighted material.
346
Microsoft Windows Internals, Fourth Edition
Preemption
In this scheduling scenario, a lower-priority thread is preempted when a higher-priority thread
becomes ready to run. This situation might occur for a couple of reasons:
■
A higher-priority thread’s wait completes. (The event that the other thread was waiting
for has occurred.)
■
A thread priority is increased or decreased.
In either of these cases, Windows must determine whether the currently running thread
should still continue to run or whether it should be preempted to allow a higher-priority
thread to run.
Note Threads running in user mode can preempt threads running in kernel mode—the mode
in which the thread is running doesn’t matter. The thread priority is the determining factor.
When a thread is preempted, it is put at the head of the ready queue for the priority it was running at. Figure 6-21 illustrates this situation.
Priority
Running
18
Ready
From wait state
17
16
15
14
13
Figure 6-21
Preemptive thread scheduling
In Figure 6-21, a thread with priority 18 emerges from a wait state and repossesses the CPU,
causing the thread that had been running (at priority 16) to be bumped to the head of the
ready queue. Notice that the bumped thread isn’t going to the end of the queue but to the
beginning; when the preempting thread has finished running, the bumped thread can complete its quantum.
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
347
Quantum End
When the running thread exhausts its CPU quantum, Windows must determine whether the
thread’s priority should be decremented and then whether another thread should be scheduled on the processor.
If the thread priority is reduced, Windows looks for a more appropriate thread to schedule.
(For example, a more appropriate thread would be a thread in a ready queue with a higher priority than the new priority for the currently running thread.) If the thread priority isn’t
reduced and there are other threads in the ready queue at the same priority level, Windows
selects the next thread in the ready queue at that same priority level and moves the previously
running thread to the tail of that queue (giving it a new quantum value and changing its state
from running to ready). This case is illustrated in Figure 6-22. If no other thread of the same
priority is ready to run, the thread gets to run for another quantum.
Priority
Running
Ready
15
14
13
12
11
Figure 6-22
Quantum end thread scheduling
Termination
When a thread finishes running (either because it returned from its main routine, called ExitThread, or was killed with TerminateThread), it moves from the running state to the terminated
state. If there are no handles open on the thread object, the thread is removed from the process thread list and the associated data structures are deallocated and released.
Context Switching
A thread’s context and the procedure for context switching vary depending on the processor’s
architecture. A typical context switch requires saving and reloading the following data:
■
Instruction pointer
■
User and kernel stack pointers
■
A pointer to the address space in which the thread runs (the process’s page table directory)
Copyrighted material.
348
Microsoft Windows Internals, Fourth Edition
The kernel saves this information from the old thread by pushing it onto the current (old
thread’s) kernel-mode stack, updating the stack pointer, and saving the stack pointer in the
old thread’s KTHREAD block. The kernel stack pointer is then set to the new thread’s kernel
stack, and the new thread’s context is loaded. If the new thread is in a different process, it loads
the address of its page table directory into a special processor register so that its address space
is available. (See the description of address translation in Chapter 7.) If a kernel APC that needs
to be delivered is pending, an interrupt at IRQL 1 is requested. Otherwise, control passes to the
new thread’s restored instruction pointer and the new thread resumes execution.
Idle Thread
When no runnable thread exists on a CPU, Windows dispatches the per-CPU idle thread.
Each CPU is allotted one idle thread because on a multiprocessor system one CPU can be executing a thread while other CPUs might have no threads to execute.
Various Windows process viewer utilities report the idle process using different names. Task
Manager and Process Explorer call it “System Idle Process,” Process Viewer reports it as “Idle,”
Pstat calls it “Idle Process,” Process Explode and Tlist call it “System Process,” and Qslice calls
it “SystemProcess.” Windows reports the priority of the idle thread as 0. In reality, however,
the idle threads don’t have a priority level because they run only when there are no real
threads to run. (Remember, only one thread per Windows system is actually running at priority 0—the zero page thread, explained in Chapter 7.)
The idle loop runs at DPC/dispatch level, polling for work to do, such as delivering deferred
procedure calls (DPCs) or looking for threads to dispatch to. Although some details of the
flow vary between architectures, the basic flow of control of the idle thread is as follows:
1. Enables and disables interrupts (allowing any pending interrupts to be delivered).
2. Checks whether any DPCs (described in Chapter 3) are pending on the processor. If
DPCs are pending, clears the pending software interrupt and delivers them.
3. Checks whether a thread has been selected to run next on the processor, and if so, dispatches that thread.
4. Calls the HAL processor idle routine (in case any power management functions need to
be performed).
In Windows Server 2003, the idle thread also scans for threads waiting to run on other processors. (This is explained in the upcoming multiprocessor scheduling section.)
Priority Boosts
In five cases, Windows can boost (increase) the current priority value of threads:
■
On completion of I/O operations
■
After waiting for executive events or semaphores
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
349
■
After threads in the foreground process complete a wait operation
■
When GUI threads wake up because of windowing activity
■
When a thread that’s ready to run hasn’t been running for some time (CPU starvation)
The intent of these adjustments is to improve overall system throughput and responsiveness
as well as resolve potentially unfair scheduling scenarios. Like any scheduling algorithms,
however, these adjustments aren’t perfect, and they might not benefit all applications.
Note Windows never boosts the priority of threads in the real-time range (16 through 31).
Therefore, scheduling is always predictable with respect to other threads in the real-time
range. Windows assumes that if you’re using the real-time thread priorities, you know what
you’re doing.
Priority Boosting after I/O Completion
Windows gives temporary priority boosts upon completion of certain I/O operations so that
threads that were waiting for an I/O will have more of a chance to run right away and process
whatever was being waited for. Recall that 1 quantum unit is deducted from the thread’s remaining quantum when it wakes up so that I/O bound threads aren’t unfairly favored. Although
you’ll find recommended boost values in the DDK header files (by searching for “#define IO” in
Wdm.h or Ntddk.h), the actual value for the boost is up to the device driver. (These values are
listed in Table 6-17.) It is the device driver that specifies the boost when it completes an I/O
request on its call to the kernel function IoCompleteRequest. In Table 6-17, notice that I/O
requests to devices that warrant better responsiveness have higher boost values.
Table 6-16
Recommended Boost Values
Device
Boost
Disk, CD-ROM, parallel, video
1
Network, mailslot, named pipe, serial
2
Keyboard, mouse
6
Sound
8
The boost is always applied to a thread’s base priority, not its current priority. As illustrated in
Figure 6-23, after the boost is applied, the thread gets to run for one quantum at the elevated
priority level. After the thread has completed its quantum, it decays one priority level and then
runs another quantum. This cycle continues until the thread’s priority level has decayed back
to its base priority. A thread with a higher priority can still preempt the boosted thread, but
the interrupted thread gets to finish its time slice at the boosted priority level before it decays
to the next lower priority.
Copyrighted material.
350
Microsoft Windows Internals, Fourth Edition
Quantum
Priority
Priority decay at
quantum end
Boost upon
wait complete
Preempt
(before quantum end)
Base
priority
Run
Wait
Run
Round-robin at
base priority
Run
Time
Figure 6-23
Priority boosting and decay
As noted earlier, these boosts apply only to threads in the dynamic priority range (0 through
15). No matter how large the boost is, the thread will never be boosted beyond level 15 into
the real-time priority range. In other words, a priority 14 thread that receives a boost of 5 will
go up to priority 15. A priority 15 thread that receives a boost will remain at priority 15.
Boosts after Waiting for Events and Semaphores
When a thread that was waiting for an executive event or a semaphore object has its wait satisfied (because of a call to the function SetEvent, PulseEvent, or ReleaseSemaphore), it receives a
boost of 1. (See the value for EVENT_ INCREMENT and SEMAPHORE_INCREMENT in the
DDK header files.) Threads that wait for events and semaphores warrant a boost for the same
reason that threads that wait for I/O operations do—threads that block on events are requesting
CPU cycles less frequently than CPU-bound threads. This adjustment helps balance the scales.
This boost operates the same as the boost that occurs after I/O completion, as described in
the previous section:
■
The boost is always applied to the base priority (not the current priority).
■
The priority will never be boosted over 15.
■
The thread gets to run at the elevated priority for its remaining quantum (as described
earlier, quantums are reduced by 1 when threads exit a wait) before decaying one priority level at a time until it reaches its original base priority.
A special boost is applied to threads that are awoken as a result of setting an event with the special functions NtSetEventBoostPriority (used in Ntdll.dll for critical sections) and KeSetEventBoostPriority (used for executive resources and push locks). If a thread waiting for an event is
woken up as a result of the special event boost function and its priority is 13 or below, it will
have its priority boosted to be the setting thread’s priority plus one. If its quantum is less than 4
quantum units, it is set to 4 quantum units. This boost is removed at quantum end.
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
351
Priority Boosts for Foreground Threads after Waits
Whenever a thread in the foreground process completes a wait operation on a kernel object,
the kernel function KiUnwaitThread boosts its current (not base) priority by the current value
of PsPrioritySeparation. (The windowing system is responsible for determining which process
is considered to be in the foreground.) As described in the section on quantum controls,
PsPrioritySeparation reflects the quantum-table index used to select quantums for the threads
of foreground applications. However, in this case, it is being used as a priority boost value.
The reason for this boost is to improve the responsiveness of interactive applications—by giving the foreground application a small boost when it completes a wait, it has a better chance
of running right away, especially when other processes at the same base priority might be running in the background.
Unlike other types of boosting, this boost applies to all Windows systems, and you can’t disable this boost, even if you’ve disabled priority boosting using the Windows SetThreadPriorityBoost function.
EXPERIMENT: Watching Foreground Priority Boosts and Decays
Using the CPU Stress tool (in the resource kit and the Platform SDK), you can watch priority boosts in action. Take the following steps:
1. Open the System utility in Control Panel (or right-click My Computer and select
Properties), click the Advanced tab, and click the Performance Options button.
Select the Applications option. This causes PsPrioritySeparation to get a value of 2.
2. Run Cpustres.exe.
3. Run the Windows NT 4 Performance Monitor (Perfmon4.exe in the Windows
2000 resource kits). This older version of the Performance tool is needed for this
experiment because it can query performance counter values at a frequency faster
than the Windows Performance tool (which has a maximum interval of once per
second).
4. Click the Add Counter toolbar button (or press Ctrl+I) to bring up the Add To
Chart dialog box.
5. Select the Thread object, and then select the Priority Current counter.
6. In the Instance box, scroll down the list until you see the cpustres process. Select
the second thread (thread 1). (The first thread is the GUI thread.) You should see
something like this:
Copyrighted material.
352
Microsoft Windows Internals, Fourth Edition
7. Click the Add button, and then click the Done button.
8. Select Chart from the Options menu. Change the Vertical Maximum to 16 and the
Interval to 0.010, as follows, and click OK:
9. Now bring the Cpustres process to the foreground. You should see the priority of
the Cpustres thread being boosted by 2 and then decaying back to the base priority as follows:
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
353
10. The reason Cpustres receives a boost of 2 periodically is because the thread you’re
monitoring is sleeping about 75 percent of the time and then waking up—the
boost is applied when the thread wakes up. To see the thread get boosted more frequently, increase the Activity level from Low to Medium to Busy. If you set the
Activity level to Maximum, you won’t see any boosts because Maximum in
Cpustres puts the thread into an infinite loop. Therefore, the thread doesn’t
invoke any wait functions and therefore doesn’t receive any boosts.
11. When you’ve finished, exit Performance Monitor and CPU Stress.
Priority Boosts after GUI Threads Wake Up
Threads that own windows receive an additional boost of 2 when they wake up because of windowing activity, such as the arrival of window messages. The windowing system (Win32k.sys)
applies this boost when it calls KeSetEvent to set an event used to wake up a GUI thread. The reason for this boost is similar to the previous one—to favor interactive applications.
EXPERIMENT: Watching Priority Boosts on GUI Threads
You can also see the windowing system apply its boost of 2 for GUI threads that wake
up to process window messages by monitoring the current priority of a GUI application
and moving the mouse across the window. Just follow these steps:
1. Open the System utility in Control Panel (or right-click My Computer and select
Properties), click the Advanced tab, and click the Performance Options button. If
you’re running Windows XP or Windows Server 2003 select the Advanced tab
and ensure that the Programs option is selected; if you’re running Windows 2000
ensure that the Applications option is selected. This causes PsPrioritySeparation to
get a value of 2.
2. Run Notepad from the Start menu by selecting Programs/Accessories/Notepad.
3. Run the Windows NT 4 Performance Monitor (Perfmon4.exe in the Windows
2000 resource kits). This older version of the Performance tool is needed for this
experiment because it can query performance counter values at a faster frequency.
(The Windows Performance tool has a maximum interval of once per second.)
4. Click the Add Counter toolbar button (or press Ctrl+I) to bring up the Add To
Chart dialog box.
5. Select the Thread object, and then select the Priority Current counter.
6. In the Instance box, scroll down the list until you see Notepad thread 0. Click it,
click the Add button, and then click the Done button.
7. As in the previous experiment, select Chart from the Options menu. Change the
Vertical Maximum to 16 and the Interval to 0.010, and click OK.
Copyrighted material.
354
Microsoft Windows Internals, Fourth Edition
8. You should see the priority of thread 0 in Notepad at 8, 9, or 10. Because Notepad
entered a wait state shortly after it received the boost of 2 that threads in the foreground process receive, it might not yet have decayed from 10 to 9 and then to 8.
9. With Performance Monitor in the foreground, move the mouse across the Notepad window. (Make both windows visible on the desktop.) You’ll see that the priority sometimes remains at 10 and sometimes at 9, for the reasons just explained.
(The reason you won’t likely catch Notepad at 8 is that it runs so little after receiving the GUI thread boost of 2 that it never experiences more than one priority
level of decay before waking up again because of additional windowing activity
and receiving the boost of 2 again.)
10. Now bring Notepad to the foreground. You should see the priority rise to 12 and
remain there (or drop to 11, because it might experience the normal priority decay
that occurs for boosted threads on the quantum end) because the thread is receiving two boosts: the boost of 2 applied to GUI threads when they wake up to process windowing input and an additional boost of 2 because Notepad is in the
foreground.
11. If you then move the mouse over Notepad (while it’s still in the foreground), you
might see the priority drop to 11 (or maybe even 10) as it experiences the priority
decay that normally occurs on boosted threads as they complete quantums. However, the boost of 2 that is applied because it’s the foreground process remains as
long as Notepad remains in the foreground.
12. When you’ve finished, exit Performance Monitor and Notepad.
Priority Boosts for CPU Starvation
Imagine the following situation: you have a priority 7 thread that’s running, preventing a priority 4 thread from ever receiving CPU time; however, a priority 11 thread is waiting for some
resource that the priority 4 thread has locked. But because the priority 7 thread in the middle
is eating up all the CPU time, the priority 4 thread will never run long enough to finish whatever it’s doing and release the resource blocking the priority 11 thread. What does Windows
do to address this situation? Once per second, the balance set manager (a system thread that
exists primarily to perform memory management functions and is described in more detail in
Chapter 7) scans the ready queues for any threads that have been in the ready state (that is,
haven’t run) for approximately 4 seconds. If it finds such a thread, the balance set manager
boosts the thread’s priority to 15. On Windows 2000 and Windows XP, the thread quantum
is set to twice the process quantum. On Windows Server 2003, the quantum is set to 4 quantum units. Once the quantum is expired, the thread’s priority decays immediately to its original base priority. If the thread wasn’t finished and a higher priority thread is ready to run, the
decayed thread will return to the ready queue, where it again becomes eligible for another
boost if it remains there for another 4 seconds.
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
355
The balance set manager doesn’t actually scan all ready threads every time it runs. To minimize
the CPU time it uses, it scans only 16 ready threads; if there are more threads at that priority
level, it remembers where it left off and picks up again on the next pass. Also, it will boost only
10 threads per pass—if it finds 10 threads meriting this particular boost (which would indicate
an unusually busy system), it stops the scan at that point and picks up again on the next pass.
Will this algorithm always solve the priority inversion issue? No—it’s not perfect by any
means. But over time, CPU-starved threads should get enough CPU time to finish whatever
processing they were doing and reenter a wait state.
EXPERIMENT: Watching Priority Boosts for CPU Starvation
Using the CPU Stress tool (in the resource kit and the Platform SDK), you can watch priority boosts in action. In this experiment, we’ll see CPU usage change when a thread’s
priority is boosted. Take the following steps:
1. Run Cpustres.exe. Change the activity level of the active thread (by default, Thread
1) from Low to Maximum. Change the thread priority from Normal to Below Normal. The screen should look like this:
2. Run the Windows NT 4 Performance Monitor (Perfmon4.exe in the Windows
2000 resource kits). Again, you need the older version for this experiment because
it can query performance counter values at a frequency faster than once per second.
3. Click the Add Counter toolbar button (or press Ctrl+I) to bring up the Add To
Chart dialog box.
4. Select the Thread object, and then select the % Processor Time counter.
Copyrighted material.
356
Microsoft Windows Internals, Fourth Edition
5. In the Instance box, scroll down the list until you see the cpustres process. Select
the second thread (thread 1). (The first thread is the GUI thread.) You should see
something like this:
6. Click the Add button, and then click the Done button.
7. Raise the priority of Performance Monitor to real-time by running Task Manager,
clicking the Processes tab, and selecting the Perfmon4.exe process. Right-click the
process, select Set Priority, and then select Realtime. (If you receive a Task Manager Warning message box warning you of system instability, click the Yes button.)
8. Run another copy of CPU Stress. In this copy, change the activity level of Thread 1
from Low to Maximum.
9. Now switch back to Performance Monitor. You should see CPU activity every 4 or
so seconds because the thread is boosted to priority 15.
When you’ve finished, exit Performance Monitor and the two copies of CPU Stress.
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
357
EXPERIMENT: “Listening” to Priority Boosting
To “hear” the effect of priority boosting for CPU starvation, perform the following steps
on a system with a sound card:
1. Run Windows Media Player (or some other audio playback program), and begin
playing some audio content.
2. Run Cpustres from the Windows 2000 resource kits, and set the activity level of
thread 1 to maximum.
3. Raise the priority of thread 1 from Normal to Time Critical.
4. You should hear the music playback stop as the compute-bound thread begins
consuming all available CPU time.
5. Every so often, you should hear bits of sound as the starved thread in the audio
playback process gets boosted to 15 and runs enough to send more data to the
sound card.
6. Stop Cpustres and Windows Media Player.
Multiprocessor Systems
On a uniprocessor system, scheduling is relatively simple: the highest priority thread that
wants to run is always running. On a multiprocessor system, it is more complex, as Windows
attempts to schedule threads on the most optimal processor for the thread, taking into
account the thread’s preferred and previous processors, as well as the configuration of the
multiprocessor system. Therefore, while Windows attempts to schedule the highest priority
runnable threads on all available CPUs, it only guarantees to be running the (single) highest
priority thread somewhere.
Before we describe the specific algorithms used to choose which threads run where and when,
let’s examine the additional information Windows maintains to track thread and processor
state on multiprocessor systems and the two new types of multiprocessor systems supported
by Windows (hyperthreaded and NUMA).
Multiprocessor Dispatcher Database
As explained in the “Dispatcher Database” section earlier in the chapter, the dispatcher database refers to the information maintained by the kernel to perform thread scheduling. As
shown in Figure 6-16, on multiprocessor Windows 2000 and Windows XP systems, the ready
queues and ready summary have the same structure as they do on uniprocessor systems. In
addition to the ready queues and the ready summary, Windows maintains two bitmasks that
track the state of the processors on the system. (How these bitmasks are used is explained in
the upcoming section “Multiprocessor Thread-Scheduling Algorithms”.) Following are the
two bitmasks that Windows maintains:
Copyrighted material.
358
Microsoft Windows Internals, Fourth Edition
■
The active processor mask (KeActiveProcessors), which has a bit set for each usable processor on the system (This might be less than the number of actual processors if the licensing limits of the version of Windows running supports less than the number of available
physical processors.)
■
The idle summary (KiIdleSummary), in which each set bit represents an idle processor
Whereas on uniprocessor systems, the dispatcher database is locked by raising IRQL (to DPC/
dispatch level on Windows 2000 and Windows XP and to both DPC/dispatch level and Synch
level on Windows Server 2003), on multiprocessor systems more is required, because each processor could, at the same time, raise IRQL and attempt to operate on the dispatcher database.
(This is true for any systemwide structure accessed from high IRQL.) (See Chapter 3 for a general description of kernel synchronization and spinlocks.) On Windows 2000 and Windows XP,
two kernel spinlocks are used to synchronize access to thread dispatching: the dispatcher spinlock (KiDispatcherLock) and the context swap spinlock (KiContextSwapLock). The former is held
while changes are made to structures that might affect which thread should run. The latter is
held after the decision is made but during the thread context swap operation itself.
To improve scalability including thread dispatching concurrency, Windows Server 2003 multiprocessor systems have per-processor dispatcher ready queues, as illustrated in Figure 6-24.
In this way, on Windows Server 2003, each CPU can check its own ready queues for the next
thread to run without having to lock the systemwide ready queues.
Process
Process
Thread 1
Thread 2
Thread 3
Thread 4
CPU 0
ready queues
CPU 1
ready queues
31
31
0
0
Ready summary
Ready summary
31
0
31
Deferred
ready queue
Figure 6-24
Copyrighted material.
Windows Server 2003 multiprocessor dispatcher database
0
Deferred
ready queue
Chapter 6:
Processes, Threads, and Jobs
359
The per-processor ready queues, as well as the per-processor ready summary are part of the
processor control block (PRCB) structure. (To see the fields in the PRCB, type dt nt!_prcb in
the Kernel Debugger.) Because on a multiprocessor system one processor might need to modify another processor’s per-CPU scheduling data structures (such as inserting a thread that
would like to run on a certain processor), these structures are synchronized by using a new
per-PRCB queued spinlock, which is held at IRQL SYNCH_LEVEL. (See Table 6-18 for the various values of SYNCH_LEVEL). Thus, thread selection can occur while locking only an individual processor’s PRCB, in contrast to doing this on Windows 2000 and Windows XP, where
the systemwide dispatcher spinlock had to be held.
Table 6-17
IRQL SYNCH_LEVEL on Multiprocessor Systems
Windows Version
IRQL
Windows 2000
2
Windows XP on x86
2
Windows Server 2003 on x86
27
Windows XP on x64
12
Windows XP on IA-64
2
Windows Server 2003 on x64 & IA-64
12
There is also a per-CPU list of threads in the deferred ready state. These represent threads that
are ready to run but have not yet been readied for execution; the actual ready operation has
been deferred to a more appropriate time. Because each processor manipulates only its own
per-processor deferred ready list, this list is not synchronized by the PRCB spinlock. The
deferred ready thread list is processed before exiting the thread dispatcher, before performing
a context switch, and after processing a DPC. Threads on the deferred ready list are either dispatched immediately or are moved to the per-process ready queue for their priority level.
Note that the systemwide dispatcher spinlock still exists and is used on Windows Server
2003, but it is held only for the time needed to modify systemwide state that might affect
which thread runs next. For example, changes to synchronization objects (mutexes, events,
and semaphores) and their wait queues require holding the dispatcher lock to prevent more
than one processor from changing the state of such objects (and the consequential action of
possibly readying threads for execution). Other examples include changing the priority of a
thread, timer expiration, and swapping of thread kernel stacks.
Finally, synchronization of thread context switching has also been improved on Windows
Server 20003, as it is now synchronized by using a per-thread spinlock, whereas in Windows
2000 and Windows XP context switching was synchronized by holding a systemwide context
swap spinlock.
Copyrighted material.
360
Microsoft Windows Internals, Fourth Edition
Hyperthreaded Systems
As described in the “Symmetric Multiprocessing” section in Chapter 2, Windows XP and Windows Server 2003 support hyperthreaded multiprocessor systems in two primary ways:
1. Logical processors do not count against physical processor licensing limits. For example, Windows XP Home Edition, which has a licensed processor limit of 1, will use both
logical processors on a single processor hyperthreaded system.
2. When choosing a processor for a thread, if there is a physical processor with all logical
processors idle, a logical processor from that physical processor will be selected, as
opposed to choosing an idle logical processor on a physical processor that has another
logical processor running a thread.
EXPERIMENT: Viewing Hyperthreading Information
You can examine the information Windows maintains for hyperthreaded processors
using the !smt command in the kernel debugger. The following output is from a dual
processor hyperthreaded Xeon system (four logical processors):
lkd> !smt
SMT Summary:
-----------KeActiveProcessors:
KiIdleSummary:
No PRCB
Set Master
0 ffdff120 Master
1 f771f120 Master
2 f7727120 ffdff120
3 f772f120 f771f120
****----------------------------***---------------------------SMT Set
*-*-----------------------------*-*---------------------------*-*-----------------------------*-*----------------------------
(0000000f)
(0000000e)
(00000005)
(0000000a)
(00000005)
(0000000a)
#LP IAID
2 00
2 06
2 01
2 07
Number of licensed physical processors: 2
Logical processor 0 and 1 are on separate physical processors (as indicated by the term
“Master”).
NUMA Systems
Another type of multiprocessor system supported by Windows XP and Windows Server 2003
are those with nonuniform memory access (NUMA) architectures. In a NUMA system, processors are grouped together in smaller units called nodes. Each node has its own processors and
memory, and is connected to the larger system through a cache-coherent interconnect bus.
These systems are called “nonuniform” because each node has its own local high-speed memory. While any processor in any node can access all of memory, node-local memory is much
faster to access.
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
361
The kernel maintains information about each node in a NUMA system in a data structure
called KNODE. The kernel variable KeNodeBlock is an array of pointers to the KNODE structures for each node. The format of the KNODE structure can be shown using the dt command
in the kernel debugger, as shown here:
lkd> dt nt!_knode
nt!_KNODE
+0x000 ProcessorMask
: Uint4B
+0x004 Color
: Uint4B
+0x008 MmShiftedColor
: Uint4B
+0x00c FreeCount
: [2] Uint4B
+0x018 DeadStackList
: _SLIST_HEADER
+0x020 PfnDereferenceSListHead : _SLIST_HEADER
+0x028 PfnDeferredList : Ptr32 _SINGLE_LIST_ENTRY
+0x02c Seed
: UChar
+0x02d Flags
: _flags
EXPERIMENT: Viewing NUMA Information
You can examine the information Windows maintains for each node in a NUMA system
using the !numa command in the kernel debugger. The following partial output is from
a 32-processor NUMA system by NEC with 4 processors per node:
21: kd> !numa
NUMA Summary:
-----------Number of NUMA nodes
Number of Processors
MmAvailablePages
KeActiveProcessors
----- (00000000ffffffff)
:
:
:
:
8
32
0x00F70D2C
********************************---------------------------
NODE 0 (E00000008428AE00):
ProcessorMask
: ****----------------------------------------------------------Color
: 0x00000000
MmShiftedColor
: 0x00000000
Seed
: 0x00000000
Zeroed Page Count: 0x00000000001CF330
Free Page Count : 0x0000000000000000
NODE 1 (E00001597A9A2200):
ProcessorMask
: ----****------------------------------------------------------Color
: 0x00000001
MmShiftedColor
: 0x00000040
Seed
: 0x00000006
Zeroed Page Count: 0x00000000001F77A0
Free Page Count : 0x0000000000000004
Copyrighted material.
362
Microsoft Windows Internals, Fourth Edition
The following partial output is from a 64-processor NUMA system from Hewlett Packard with 4 processors per node:
26: kd> !numa
NUMA Summary:
-----------Number of NUMA nodes
Number of Processors
MmAvailablePages
KeActiveProcessors
***** (ffffffffffffffff)
:
:
:
:
16
64
0x03F55E67
***********************************************************
NODE 0 (E000000084261900):
ProcessorMask
: ****----------------------------------------------------------Color
: 0x00000000
MmShiftedColor
: 0x00000000
Seed
: 0x00000001
Zeroed Page Count: 0x00000000003F4430
Free Page Count : 0x0000000000000000
NODE 1 (E0000145FF992200):
ProcessorMask
: ----****------------------------------------------------------Color
: 0x00000001
MmShiftedColor
: 0x00000040
Seed
: 0x00000007
Zeroed Page Count: 0x00000000003ED59A
Free Page Count : 0x0000000000000000
Applications that want to gain the most performance out of NUMA systems can set the affinity
mask to restrict a process to the processors in a specific node. This information can be
obtained using the functions listed in Table 6-18. Functions that can alter thread affinity are
listed in Table 6-13.
Table 6-18
NUMA-Related Functions
Function
Description
GetNumaHighestNodeNumber
Retrieves the node that currently has the highest number.
GetNumaNodeProcessorMask
Retrieves the processor mask for the specified node.
GetNumaProcessorNode
Retrieves the node number for the specified processor.
How the scheduling algorithms take into account NUMA systems will be covered in the
upcoming section “Multiprocessor Thread-Scheduling Algorithms” (and the optimizations in
the memory manager to take advantage of node-local memory are covered in Chapter 7).
Affinity
Each thread has an affinity mask that specifies the processors on which the thread is allowed
to run. The thread affinity mask is inherited from the process affinity mask. By default, all proCopyrighted material.
Chapter 6:
Processes, Threads, and Jobs
363
cesses (and therefore all threads) begin with an affinity mask that is equal to the set of active
processors on the system—in other words, the system is free to schedule all threads on any
available processor.
However, to optimize throughput and/or partition workloads to a specific set of processors,
applications can choose to change the affinity mask for a thread. This can be done at several
levels:
■
Calling the SetThreadAffinityMask function to set the affinity for an individual thread
■
Calling the SetProcessAffinityMask function to set the affinity for all the threads in a process. Task Manager and Process Explorer provide a GUI interface to this function if you
right-click a process and choose Set Affinity. The Psexec tool (from www.sysinternals.com) provides a command-line interface to this function. (See the –a switch.)
■
By making a process a member of a job that has a jobwide affinity mask set using the SetInformationJobObject function (Jobs are described in the upcoming “Job Objects” section.)
■
By specifying an affinity mask in the image header using, for example, the Imagecfg tool
in the Windows 2000 Server Resource Kit Supplement 1 (For more information on the
detailed format of Windows images, see the article “Portable Executable and Common
Object File Format Specification” in the MSDN Library.)
You can also set the “uniprocessor” flag for an image (using the Imagecfg –u switch). If this
flag is set, the system chooses a single processor at process creation time and assigns that as
the process affinity mask, starting with the first processor and then going round-robin across
all the processors. For example, on a dual-processor system, the first time you run an image
marked as uniprocessor, it is assigned to CPU 0; the second time, CPU 1; the third time, CPU
0; the fourth time, CPU 1; and so on. This flag can be useful as a temporary workaround for
programs that have multithreaded synchronization bugs that, as a result of race conditions,
surface on multiprocessor systems but that don’t occur on uniprocessor systems. (This has
actually saved the authors of this book on two different occasions.)
EXPERIMENT: Viewing and Changing Process Affinity
In this experiment, you will modify the affinity settings for a process and see that process affinity is inherited by new processes:
1. Run the Command Prompt (cmd.exe).
2. Run Task Manager or Process Explorer, and find the cmd.exe process in the process list.
3. Right-click the process, and select Affinity. A list of processors should be displayed.
For example, on a dual-processor system you will see this:
Copyrighted material.
364
Microsoft Windows Internals, Fourth Edition
4. Select a subset of the available processors on the system, and press OK. The process’s threads are now restricted to run on the processors you just selected.
5. Now run Notepad.exe from the Command Prompt (by typing Notepad.exe).
6. Go back to Task Manager or Process Explorer and find the new Notepad process.
Right-click it, and choose Affinity. You should see the same list of processors you
chose for the Command Prompt process. This is because processes inherit their
affinity settings from their parent.
EXPERIMENT: Changing the Image Affinity
In this experiment (which requires access to a multiprocessor system), you will change
the affinity mask of a program to force it to run on the first processor:
1. Make a copy of Cpustres.exe from the Windows 2000 resource kits. For example,
assuming there is a c:\temp folder on your system, from the command prompt,
type the following:
copy c:\program files\resource kit\cpustres.exe c:\temp\cpustres.exe
2. Set the image affinity mask to force the process’s threads to run on CPU 0 by typing the following in the command prompt (assuming that the path to the resource
kit tools is in your path):
imagecfg –a 1 c:\temp\cpustres.exe
3. Now run the modified Cpustres from the c:\temp folder.
4. Enable two worker threads, and set the activity level for both threads to Maximum
(not Busy). The Cpustres screen should look like this:
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
365
5. Find the Cpustres process in Process Explorer or Task Manager, right click, and
choose Set Affinity. The affinity settings should show the process bound to CPU 0.
6. Examine the systemwide CPU usage by clicking Show, System Information (if running Process Explorer) or by clicking the Performance tab (if running Task Manager). Assuming there are no other compute-bound processes, you should see the
total percentage of CPU time consumed, approximately 1/# CPUs (for example,
50% on a dual-CPU system, 25% on a four-CPU system), because the two threads
in Cpustres are forced to run on a single processor, leaving the other processor(s)
idle.
7. Finally, change the affinity mask of the Cpustres process to permit it to run on all
CPUs. Go back and examine the systemwide CPU usage. You should see 100% on
a dual-CPU system, 50% on a four-CPU system, and so forth.
Windows won’t move a running thread that could run on a different processor from one CPU
to a second processor to permit a thread with an affinity for the first processor to run on the
first processor. For example, consider this scenario: CPU 0 is running a priority 8 thread that
can run on any processor, and CPU 1 is running a priority 4 thread that can run on any processor. A priority 6 thread that can run on only CPU 0 becomes ready. What happens? Windows won’t move the priority 8 thread from CPU 0 to CPU 1 (preempting the priority 4
thread) so that the priority 6 thread can run; the priority 6 thread has to wait.
Therefore, changing the affinity mask for a process or a thread can result in threads getting
less CPU time than they normally would, as Windows is restricted from running the thread
on certain processors. Therefore, setting affinity should be done with extreme care—in most
cases, it is optimal to let Windows decide which threads run where.
Copyrighted material.
366
Microsoft Windows Internals, Fourth Edition
Ideal and Last Processor
Each thread has two CPU numbers stored in the kernel thread block:
■
Ideal processor, or the preferred processor that this thread should run on
■
Last processor, or the processor on which the thread last ran
The ideal processor for a thread is chosen when a thread is created using a seed in the process
block. The seed is incremented each time a thread is created so that the ideal processor for
each new thread in the process will rotate through the available processors on the system. For
example, the first thread in the first process on the system is assigned an ideal processor of 0.
The second thread in that process is assigned an ideal processor of 1. However, the next process in the system has its first thread’s ideal processor set to 1, the second to 2, and so on. In
that way, the threads within each process are spread evenly across the processors.
Note that this assumes the threads within a process are doing an equal amount of work. This
is typically not the case in a multithreaded process, which normally has one or more housekeeping threads and then a number of worker threads. Therefore, a multithreaded application
that wants to take full advantage of the platform might find it advantageous to specify the
ideal processor numbers for its threads by using the SetThreadIdealProcessor function.
On hyperthreaded systems, the next ideal processor is the first logical processor on the next
physical processor. For example, on a dual-processor hyperthreaded system with four logical
processors, if the ideal processor for the first thread is assigned to logical processor 0, the second thread would be assigned to logical processor 2, the third thread to logical processor 1,
the fourth thread to logical process 3, and so forth. In this way, the threads are spread evenly
across the physical processors.
On NUMA systems, when a process is created, an ideal node for the process is selected. The
first process is assigned to node 0, the second process to node 1, and so on. Then, the ideal
processors for the threads in the process are chosen from the process’s ideal node. The ideal
processor for the first thread in a process is assigned to the first processor in the node. As
additional threads are created in processes with the same ideal node, the next processor is
used for the next thread’s ideal processor, and so on.
Multiprocessor Thread-Scheduling Algorithms
Now that we’ve described the types of multiprocessor systems supported by Windows as well
as the thread affinity and ideal processor settings, we’re ready to examine how this information is used to determine which threads run where. There are two basic decisions to describe:
■
Choosing a processor for a thread that wants to run
■
Choosing a thread on a processor that needs something to do
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
367
Choosing a Processor for a Thread When There Are Idle Processors
When a thread becomes ready to run, Windows first tries to schedule the thread to run on an
idle processor. If there is a choice of idle processors, preference is given first to the thread’s
ideal processor, then to the thread’s previous processor, and then to the currently executing
processor (that is, the CPU on which the scheduling code is running).
On Windows 2000, if none of these CPUs are idle, the first idle processor the thread can run
on is selected by scanning the idle processor mask from highest to lowest CPU number.
On Windows XP and Windows Server 2003, the idle processor selection is more sophisticated.
First, the idle processor set is set to the idle processors that the thread’s affinity mask permits it
to run on. If the system is NUMA and there are idle CPUs in the node containing the thread’s
ideal processor, the list of idle processors is reduced to that set. If this eliminates all idle processors, the reduction is not done. Next, if the system is running hyperthreaded processors and
there is a physical processor with all logical processors idle, the list of idle processors is reduced
to that set. If that results in an empty set of processors, the reduction is not done.
If the current processor (the processor trying to determine what to do with the thread that
wants to run) is in the remaining idle processor set, the thread is scheduled on it. If the current processor is not in the remaining set of idle processors, it is a hyperthreaded system, and
there is an idle logical processor on the physical processor containing the ideal processor for
the thread, the idle processors are reduced to that set. If not, the system checks whether there
are any idle logical processors on the physical processor containing the thread’s previous processor. If that set is nonzero, the idle processors are reduced to that list.
In the set of idle processors remaining, any CPUs in a sleep state are eliminated from consideration. (Again, this reduction is not performed if that would eliminate all possible processors.) Finally, the lowest numbered CPU in the remaining set is selected as the processor to
run the thread on.
Regardless of the Windows version, once a processor has been selected for the thread to run
on, that thread is put in the Standby state and the idle processor’s PRCB is updated to point
to this thread. When the idle loop on that processor runs, it will see that a thread has been
selected to run and will dispatch that thread.
Choosing a Processor for a Thread When There Are No Idle Processors
If there are no idle processors when a thread wants to run, Windows compares the priority of
the thread running (or the one in the standby state) on the thread’s ideal processor to determine whether it should preempt that thread. On Windows 2000, a thread’s affinity mask can
exclude the ideal processor. (This condition is not allowed as of Windows XP.) If that is the
case, Windows 2000 selects the thread’s previous processor. If that processor is not in the
thread’s affinity mask, the highest processor number that the thread can run on is selected.
Copyrighted material.
368
Microsoft Windows Internals, Fourth Edition
If the thread’s ideal processor already has a thread selected to run next (waiting in the standby
state to be scheduled) and that thread’s priority is less than the priority of the thread being
readied for execution, the new thread preempts that first thread out of the standby state and
becomes the next thread for that CPU. If there is already a thread running on that CPU, Windows checks whether the priority of the currently running thread is less than the thread being
readied for execution. If so, the currently running thread is marked to be preempted and Windows queues an interprocessor interrupt to the target processor to preempt the currently running thread in favor of this new thread.
Note
Windows doesn’t look at the priority of the current and next threads on all the CPUs—
just on the one CPU selected as just described. If no thread can be preempted on that one CPU,
the new thread is put in the ready queue for its priority level, where it awaits its turn to get
scheduled. Therefore, Windows does not guarantee to be running all the highest priority
threads, but it will always run the highest priority thread.
If the ready thread cannot be run right away, it is moved into the ready state where it awaits its
turn to run. Note that in Windows Server 2003, threads are always put on their ideal processor’s per-processor ready queues.
Selecting a Thread to Run on a Specific CPU (Windows 2000 and
Windows XP)
In several cases (such as when a thread enters a wait state, lowers its priority, changes its
affinity, or delays or yields execution), Windows must find a new thread to run on the CPU
that the currently executing thread is running on. As described earlier, on a single-processor
system, Windows simply picks the first thread in the highest-priority nonempty ready
queue. On a multiprocessor system, however, Windows 2000 and Windows XP don’t simply pick the first thread in the ready queue. Instead, they look for a thread in the highest-priority nonempty read queue that meets one of the following conditions:
■
Ran last on the specified processor
■
Has its ideal processor set to the specified processor
■
Has been ready to run for longer than 3 clock ticks
■
Has a priority greater than or equal to 24
Threads that don’t have the specified processor in their hard affinity mask are skipped, obviously. If there are no threads that meet one of these conditions, Windows picks the thread at
the head of the ready queue it began searching from.
Why does it matter which processor a thread was last running on? As usual, the answer is
speed—giving preference to the last processor a thread executed on maximizes the chances
that thread data remains in the secondary cache of the processor in question.
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
369
Selecting a Thread to Run on a Specific CPU (Windows Server 2003)
Because each processor in Windows Server 2003 has its own list of threads waiting to run on
that processor, when a thread finishes running, the processor can simply check its per-processor ready queue for the next thread to run. If the per-processor ready queues are empty, the
idle thread for that processor is scheduled. The idle thread then begins scanning other processor’s ready queues for threads it can run. Note that on NUMA systems, the idle thread first
looks at processors on its node before looking at other nodes’ processors.
Job Objects
A job object is a nameable, securable, shareable kernel object that allows control of one or
more processes as a group. A job object’s basic function is to allow groups of processes to be
managed and manipulated as a unit. A process can be a member of only one job object. By
default, its association with the job object can’t be broken and all processes created by the
process and its descendents are associated with the same job object as well. The job object
also records basic accounting information for all processes associated with the job and for all
processes that were associated with the job but have since terminated. Table 6-20 lists the
Windows functions to create and manipulate job objects.
Table 6-19
Windows API Functions for Jobs
Function
Description
CreateJobObject
Creates a job object (with an optional name)
OpenJobObject
Opens an existing job object by name
AssignProcessToJobObject
Adds a process to a job
TerminateJobObject
Terminates all processes in a job
SetInformationJobObject
Sets limits
QueryInformationJobObject
Retrieves information about the job, such as CPU time, page
fault count, number of processes, list of process IDs, quotas or
limits, and security limits
The following are some of the CPU-related and memory-related limits you can specify for a
job:
■
Maximum number of active processes Limits the number of concurrently existing
processes in the job.
■
Jobwide user-mode CPU time limit Limits the maximum amount of user-mode CPU
time that the processes in the job can consume (including processes that have run and
exited). Once this limit is reached, by default all the processes in the job will be terminated with an error code and no new processes can be created in the job (unless the
limit is reset). The job object is signaled, so any threads waiting for the job will be
released. You can change this default behavior with a call to EndOfJobTimeAction.
Copyrighted material.
370
Microsoft Windows Internals, Fourth Edition
■
Per-process user-mode CPU time limit Allows each process in the job to accumulate
only a fixed maximum amount of user-mode CPU time. When the maximum is reached,
the process terminates (with no chance to clean up).
■
Job scheduling class Sets the length of the time slice (or quantum) for threads in processes in the job. This setting applies only to systems running with long, fixed quantums (the default for Windows Server systems). The value of the job-scheduling class
determines the quantum as shown here:
Scheduling Class
Quantum Units
0
6
1
12
2
18
3
24
4
30
5
36
6
42
7
48
8
54
9
Infinite if real-time; 60 otherwise
■
Job processor affinity Sets the processor affinity mask for each process in the job. (Individual threads can alter their affinity to any subset of the job affinity, but processes can’t
alter their process affinity setting.)
■
Job process priority class Sets the priority class for each process in the job. Threads
can’t increase their priority relative to the class (as they normally can). Attempts to
increase thread priority are ignored. (No error is returned on calls to SetThreadPriority,
but the increase doesn’t occur.)
■
Default working set minimum and maximum Defines the specified working set minimum and maximum for each process in the job. (This setting isn’t jobwide—each process has its own working set with the same minimum and maximum values.)
■
Process and job committed virtual memory limit Defines the maximum amount of
virtual address space that can be committed by either a single process or the entire job.
Jobs can also be set to queue an entry to an I/O completion port object, which other threads
might be waiting for, with the Windows GetQueuedCompletionStatus function.
You can also place security limits on processes in a job. You can set a job so that each process
runs under the same jobwide access token. You can then create a job to restrict processes from
impersonating or creating processes that have access tokens that contain the local administrator’s group. In addition, you can apply security filters so that when threads in processes contained in a job impersonate client threads, certain privileges and security IDs (SIDs) can be
eliminated from the impersonation token.
Copyrighted material.
Chapter 6:
Processes, Threads, and Jobs
371
Finally, you can also place user-interface limits on processes in a job. Such limits include being
able to restrict processes from opening handles to windows owned by threads outside the job,
reading and/or writing to the clipboard, and changing the many user-interface system parameters via the Windows SystemParametersInfo function.
Windows 2000 Datacenter Server has a tool called the Process Control Manager that allows
an administrator to define job objects, the various quotas and limits that can be specified for
a job, and which processes, if run, should be added to the job. A service component monitors
process activity and adds the specified processes to the jobs. Note that this tool is no longer
shipped with Windows Server 2003 Datacenter Edition, but will remain on the system if a
Windows 2000 Datacenter Server is upgraded to Windows Server 2003 Datacenter Edition.
EXPERIMENT: Viewing the Job Object
You can view named job objects with the Performance tool. (See the Job Object and Job
Object Details performance objects.) You can view unnamed jobs with the kernel
debugger !job or dt nt!_ejob commands.
To see whether a process is associated with a job, you can use the kernel debugger
!process command, or on Windows XP and Windows Server 2003, Process Explorer.
Follow these steps to create and view an unnamed job object:
1. From the command prompt, use the runas command to create a process running
the command prompt (Cmd.exe). For example, type runas /user:<domain>\<
username> cmd. You’ll be prompted for your password. Enter your password, and
a command prompt window will appear. The Windows service that executes
runas commands creates an unnamed job to contain all processes (so that it can
terminate these processes at logoff time).
2. From the command prompt, run Notepad.exe.
3. Then run Process Explorer and notice that the Cmd.exe and Notepad.exe processes are highlighted as part of a job. (You can configure the colors used to highlight processes that are members of a job by clicking Options, Configure
Highlighting.) Here is a screen shot showing these two processes:
4. Double-click either the Cmd.exe or Notepad.exe process to bring up the process
properties. You will see a Job tab on the process properties dialog box.
Copyrighted material.
372
Microsoft Windows Internals, Fourth Edition
5. Click the Job tab to view the details about the job. In this case, there are no quotas
associated with the job, but there are two member processes:
6. Now run the kernel debugger on the live system (either WinDbg in local kernel
debugging mode or LiveKd if you are on Windows 2000), display the process list
with !process, and find the recently created process running Cmd.exe. Then display the process block by using !process <process ID>, find the address of the job
object, and finally display the job object with the !job command. Here’s some partial debugger output of these commands on a live system:
lkd> !process 0 0
**** NT ACTIVE PROCESS DUMP ****
.
.
PROCESS 8567b758 SessionId: 0 Cid: 0fc4
Peb: 7ffdf000
DirBase: 1b3fb000 ObjectTable: e18dd7d0 HandleCount:
Image: cmd.exe
PROCESS 856561a0 SessionId: 0 Cid: 0d70
Peb: 7ffdf000
DirBase: 2e341000 ObjectTable: e19437c8 HandleCount:
Image: notepad.exe
lkd> !process 0fc4
Searching for Process with Cid == fc4
PROCESS 8567b758 SessionId: 0 Cid: 0fc4
Peb: 7ffdf000
DirBase: 1b3fb000 ObjectTable: e18dd7d0 HandleCount:
Image: cmd.exe
BasePriority
8
.
.
Job
85557988
lkd> !job 85557988
Job at 85557988
TotalPageFaultCount
TotalProcesses
ActiveProcesses
TotalTerminatedProcesses
Copyrighted material.
0
2
2
0
ParentCid: 00b0
19.
ParentCid: 0fc4
16.
ParentCid: 00b0
19.
Chapter 6:
LimitFlags
MinimumWorkingSetSize
MaximumWorkingSetSize
ActiveProcessLimit
PriorityClass
UIRestrictionsClass
SecurityLimitFlags
Token
Processes, Threads, and Jobs
0
0
0
0
0
0
0
00000000
7. Finally, use the dt command to display the job object and notice the additional
fields shown about the job:
lkd> dt nt!_ejob 85557988
nt!_EJOB
+0x000 Event
: _KEVENT
+0x010 JobLinks
: _LIST_ENTRY [ 0x805455c8 - 0x85797888 ]
+0x018 ProcessListHead : _LIST_ENTRY [ 0x8567b8dc - 0x85656324 ]
+0x020 JobLock
: _ERESOURCE
+0x058 TotalUserTime
: _LARGE_INTEGER 0x0
+0x060 TotalKernelTime : _LARGE_INTEGER 0x0
+0x068 ThisPeriodTotalUserTime : _LARGE_INTEGER 0x0
+0x070 ThisPeriodTotalKernelTime : _LARGE_INTEGER 0x0
+0x078 TotalPageFaultCount : 0
+0x07c TotalProcesses
: 2
+0x080 ActiveProcesses : 2
+0x084 TotalTerminatedProcesses : 0
+0x088 PerProcessUserTimeLimit : _LARGE_INTEGER 0x0
+0x090 PerJobUserTimeLimit : _LARGE_INTEGER 0x0
+0x098 LimitFlags
: 0
+0x09c MinimumWorkingSetSize : 0
+0x0a0 MaximumWorkingSetSize : 0
+0x0a4 ActiveProcessLimit : 0
+0x0a8 Affinity
: 0
+0x0ac PriorityClass
: 0 ’’
+0x0b0 UIRestrictionsClass : 0
+0x0b4 SecurityLimitFlags : 0
+0x0b8 Token
: (null)
+0x0bc Filter
: (null)
+0x0c0 EndOfJobTimeAction : 0
+0x0c4 CompletionPort
: 0x8619d8c0
+0x0c8 CompletionKey
: (null)
+0x0cc SessionId
: 0
+0x0d0 SchedulingClass : 5
+0x0d8 ReadOperationCount : 0
+0x0e0 WriteOperationCount : 0
+0x0e8 OtherOperationCount : 0
+0x0f0 ReadTransferCount : 0
+0x0f8 WriteTransferCount : 0
+0x100 OtherTransferCount : 0
+0x108 IoInfo
: _IO_COUNTERS
+0x138 ProcessMemoryLimit : 0
+0x13c JobMemoryLimit
: 0
+0x140 PeakProcessMemoryUsed : 0x256
+0x144 PeakJobMemoryUsed : 0x1f6
+0x148 CurrentJobMemoryUsed : 0x1f6
+0x14c MemoryLimitsLock : _FAST_MUTEX
+0x16c JobSetLinks
: _LIST_ENTRY [ 0x85557af4 - 0x85557af4 ]
+0x174 MemberLevel
: 0
+0x178 JobFlags
: 0
Copyrighted material.
373
374
Microsoft Windows Internals, Fourth Edition
Conclusion
In this chapter, we’ve examined the structure of processes and threads and jobs, seen how they
are created, and looked at how Windows decides which threads should run and for how long.
Many references in this chapter are to topics related to memory management. Because threads
run inside processes and processes in large part define an address space, the next logical topic
is how Windows performs virtual and physical memory management—the subjects of Chapter 7.
Copyrighted material.
Chapter 7
Memory Management
In this chapter, you’ll learn how Microsoft Windows implements virtual memory and how it
manages the subset of virtual memory kept in physical memory. We’ll also describe the internal structure and components that make up the memory manager, including key data structures and algorithms. Before examining these mechanisms, let’s review the basic services
provided by the memory manager and key concepts such as reserved memory versus committed memory and shared memory.
Introduction to the Memory Manager
By default, the virtual size of a process on 32-bit Windows is 2 GB. If the image is marked specially as large address space aware, and the system is booted with a special switch (described
later in this chapter), a 32-bit process can grow to 3 GB on 32-bit Windows and up to 4 GB on
64-bit Windows. The process virtual address space size on 64-bit Windows is 7152 GB on IA64 systems and 8192 GB on x64 systems. (This value could be increased in future releases.)
As you saw in Chapter 2 (specifically in Table 2-4), the maximum amount of physical memory
supported by Windows ranges from 2 GB to 1024 GB, depending on which version and edition of Windows you are running. Because the virtual address space might be larger or smaller
than the physical memory on the machine, the memory manager has two primary tasks:
■
Translating, or mapping, a process’s virtual address space into physical memory so that
when a thread running in the context of that process reads or writes to the virtual
address space, the correct physical address is referenced. (The subset of a process’s virtual address space that is physically resident is called the working set. Working sets are
described in more detail later in this chapter.)
■
Paging some of the contents of memory to disk when it becomes overcommitted—that
is, when running threads or system code try to use more physical memory than is currently available—and bringing the contents back into physical memory when needed.
In addition to providing virtual memory management, the memory manager provides a core
set of services on which the various Windows environment subsystems are built. These services include memory mapped files (internally called section objects), copy-on-write memory, and support for applications using large, sparse address spaces. In addition, the
Copyrighted material.
375
376
Microsoft Windows Internals, Fourth Edition
memory manager provides a way for a process to allocate and use larger amounts of physical memory than can be mapped into the process virtual address space (for example, on 32bit systems with more than 4 GB of physical memory). This is explained in the section
“Address Windowing Extensions” later in this chapter.
Memory Manager Components
The memory manager is part of the Windows executive and therefore exists in the file
Ntoskrnl.exe. No parts of the memory manager exist in the HAL. The memory manager consists of the following components:
■
A set of executive system services for allocating, deallocating, and managing virtual
memory, most of which are exposed through the Windows API or kernel-mode device
driver interfaces
■
A translation-not-valid and access fault trap handler for resolving hardware-detected
memory management exceptions and making virtual pages resident on behalf of a process
■
Several key components that run in the context of six different kernel-mode system
threads:
❑
The working set manager (priority 16), which the balance set manager (a system
thread that the kernel creates) calls once per second as well as when free memory
falls below a certain threshold, drives the overall memory management policies,
such as working set trimming, aging, and modified page writing.
❑
The process/stack swapper (priority 23) performs both process and kernel thread
stack inswapping and outswapping. The balance set manager and the threadscheduling code in the kernel awaken this thread when an inswap or outswap
operation needs to take place.
❑
The modified page writer (priority 17) writes dirty pages on the modified list back
to the appropriate paging files. This thread is awakened when the size of the modified list needs to be reduced. (See the section “Modified Page Writer” to find out
how you can change this default value.)
❑
The mapped page writer (priority 17) writes dirty pages in mapped files to disk. It is
awakened when the size of the modified list needs to be reduced or if pages for
mapped files have been on the modified list for more than 5 minutes. This second
modified page writer thread is necessary because it can generate page faults that
result in requests for free pages. If there were no free pages and there was only one
modified page writer thread, the system could deadlock waiting for free pages.
Copyrighted material.
Chapter 7:
Memory Management
377
❑
The dereference segment thread (priority 18) is responsible for cache reduction as
well as page file growth and shrinkage. (For example, if there is no virtual address
space for paged pool growth, this thread trims the page cache so that the paged
pool used to anchor it can be freed for reuse.)
❑
The zero page thread (priority 0) zeroes out pages on the free list so that a cache of
zero pages is available to satisfy future demand-zero page faults. (Memory zeroing
in some cases is done by a faster function called MiZeroInParallel. See the note in
the section “Page List Dynamics.”)
Each of these components is covered in more detail later in the chapter.
Internal Synchronization
Like all other components of the Windows executive, the memory manager is fully reentrant
and supports simultaneous execution on multiprocessor systems—that is, it allows two
threads to acquire resources in such a way that they don’t corrupt each other’s data. To
accomplish the goal of being fully reentrant, the memory manager uses several different internal synchronization mechanisms to control access to its own internal data structures, such as
spinlocks and executive resources. (Synchronization objects are discussed in Chapter 3.)
Systemwide resources to which the memory manager must synchronize access include the page
frame number (PFN) database (controlled by a spinlock), section objects and the system working set (controlled by pushlocks), and page file creation (controlled by a mutex). In Windows
XP and Windows Server 2003, a number of these locks have been either removed completely or
optimized, resulting in much less contention. For example, in Windows 2000, spinlocks were
used to synchronize changes to system address space and memory commitment; however to
improve scalability, these spinlocks have been removed as of Windows XP. Per-process memory
management data structures that require synchronization include the working set lock (held
while changes are being made to the working set list) and the address space lock (held whenever the address space is being changed). Working set synchronization in Windows 2000 was
implemented as a mutex; in Windows XP and later, a pushlock is used, improving parallelism
and scalability, because pushlocks support both shared and exclusive access.
Other operations that no longer involve acquiring locks include charging nonpaged and
paged pool quotas, charging commitment of pages, and allocating and mapping physical
memory allocated through the address windowing extensions (AWE) functions. Also, the
lock that synchronizes access to the structures that describe physical memory (the PFN database) is now acquired less and when acquired, held for less time. These changes translate into
greater parallelism and scalability on multiprocessor systems because the number of times
the memory manager might have to block while another CPU is making a change to a global
structure has been reduced or eliminated.
Copyrighted material.
378
Microsoft Windows Internals, Fourth Edition
Configuring the Memory Manager
Like most of Windows, the memory manager attempts to automatically provide optimal system performance for varying workloads on systems of varying sizes and types. While there
are a limited number of registry values you can add and/or modify under the key HKLM\
SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management to override
some of these default performance calculations, in general, the memory manager’s default
computations will be sufficient for the majority of workloads.
Many of the thresholds and limits that control memory manager policy decisions are computed at system boot time on the basis of memory size and product type. (Windows 2000 Professional and Windows XP Professional and Home editions are optimized for desktop
interactive use, and Windows Server systems are optimized for running server applications.)
These values are stored in various kernel variables and later used by the memory manager. To
find some of these, search for global variables in Ntoskrnl.exe that have names beginning with
Mm that contain the word “maximum” or “minimum.” (See the experiment “Peering into
Undocumented Interfaces” in Chapter 2.)
Warning
Although you’ll find references to some of these variables, you shouldn’t change
them. Windows has been tested to operate properly with the current possible permutations of
these values that can be computed. Changing the value of these kernel variables on a running
system can result in unpredictable system behavior, including system hangs or even crashes.
Examining Memory Usage
The Memory and Process performance counter objects provide access to most of the details
about system and process memory utilization. Throughout the chapter, we’ll include references to specific performance counters that contain information related to the component
being described.
Besides the Performance tool, a number of tools in the Windows Support Tools and Windows
resource kits display different subsets of memory usage information. We’ve included relevant
examples and experiments throughout the chapter. One word of caution, however—different
utilities use varying and sometimes inconsistent or confusing names when displaying memory information. The following experiment illustrates this point. (We’ll explain the terms used
in this example in subsequent sections.)
EXPERIMENT: Viewing System Memory Information
The Performance tab in the Windows Task Manager, shown in the following screen shot
from Windows XP, displays basic system memory information. This information is a
subset of the detailed memory information available through the performance counters.
Copyrighted material.
Chapter 7:
Memory Management
379
Actual physical memory
on machine
Total size of standby,
free, and zero lists
Standby list + size of
system working set
Total of next two values
Paged pool virtual size
Nonpaged pool
physical size
Both Pmon.exe (in the Windows Support Tools) and Pstat.exe (in the Platform SDK)
display system and process memory information. The annotations in the following output from Pstat explain the information reported. (For an explanation of the commit
total and limit, see Table 7-15.)
Total physical
memory
Memory:
Total of standby,
free, zero lists
Total of all process
working sets (not
the total process
memory usage,
which would double
count shared pages)
64692K Avail: 3676K TotalWs:
62836K
Resident
system
code
InRam Kernel: 3788K
Commit: 111848K/85808K Limit: 182400K Peak: 113732K
Pool N: 2344K
System working
set size (not just
the size of the file
cache)
Ws
20032
Resident
paged
pool
P: 9604K
P:14448K
Nonpaged pool
physical size
User Time
Kernel Time
Faults Commit Pri Hnd Thd Pid
194378
0:00:00.000
8:03:55.734
16
1
0
0
0
2
0:00:00.000
0:01:24.421
32
1601
36
8
140
27
0:00:00.140
0:00:00.468
200
2008
164
11
35
6
20 smss.exe
0:00:00.640
0:00:02.843
716
8625
1588
13
145
9
30 csrss.exe
Paged
pool
virtual
size
Name
File Cache
0 Idle Process
2 System
To see the specific usage of paged and nonpaged pool, use the Poolmon utility,
described in the “Monitoring Pool Usage” section.
Copyrighted material.
380
Microsoft Windows Internals, Fourth Edition
Finally, the !vm command in the kernel debugger shows the basic memory management
information available through the memory-related performance counters. This command can be useful if you’re looking at a crash dump or hung system. Here’s an example
of its output:
kd> !vm
*** Virtual Memory Usage ***
Physical Memory:
32620
( 130480
Page File: \??\C:\pagefile.sys
Current:
204800Kb Free Space:
Minimum:
204800Kb Maximum:
Available Pages:
3604
(
14416
ResAvail Pages:
24004
(
96016
Modified Pages:
768
(
3072
NonPagedPool Usage: 1436
(
5744
NonPagedPool Max:
12940
(
51760
PagedPool 0 Usage:
6817
(
27268
PagedPool 1 Usage:
982
(
3928
PagedPool 2 Usage:
984
(
3936
PagedPool Usage:
8783
(
35132
PagedPool Maximum: 26624
( 106496
Shared Commit:
1361
(
5444
Special Pool:
0
(
0
Free System PTEs: 189291
( 757164
Shared Process:
3165
(
12660
PagedPool Commit:
8783
(
35132
Driver Commit:
1098
(
4392
Committed pages:
45113
( 180452
Commit limit:
79556
( 318224
Total Private:
IEXPLORE.EXE
svchost.exe
WINWORD.EXE
POWERPNT.EXE
Acrobat.exe
winlogon.exe
explorer.exe
livekd.exe
hh.exe
30536
3028
2128
1971
1905
1761
1361
1300
1015
960
(
(
(
(
(
(
(
(
(
(
122144
12112
8512
7884
7620
7044
5444
5200
4060
3840
Kb)
101052Kb
204800Kb
Kb)
Kb)
Kb)
Kb)
Kb)
Kb)
Kb)
Kb)
Kb)
Kb)
Kb)
Kb)
Kb)
Kb)
Kb)
Kb)
Kb)
Kb)
Kb)
Kb)
Kb)
Kb)
Kb)
Kb)
Kb)
Kb)
Kb)
Kb)
EXPERIMENT: Accounting for Physical Memory Use
By combining information available from performance counters with output from kernel debugger commands, you can come close to accounting for physical memory usage
on a machine running Windows. To examine the memory usage information available
through performance counters, run the Performance tool and add the counters to view
the following information. (You’ll see the results more easily if you change the vertical
scale maximum on the graph to 1000.)
Copyrighted material.
Chapter 7:
Memory Management
381
■
To view this information, select the Process performance object and the Working Set counter for the _Total process instance. This
number will be larger than the actual total process memory utilization because
shared pages are counted in each process working set. To get a more accurate picture of process memory utilization, subtract free memory (available bytes), operating system memory used (nonpaged pool, resident paged pool, and resident
operating system and driver code), and the size of the modified list from the total
physical memory on the machine. What you’re left with is the memory being used
by processes. Comparing this value against the total process working set size as
reported by the Performance tool gives you some indication of the amount of sharing occurring between processes. Although examining process physical memory
usage is interesting, of more concern is the private committed virtual memory
usage by processes, because memory leaks show up as an increasing private virtual size, not an increasing working set size. At some point, the memory manager
will stop the process from growing in physical size, though it can continue to grow
in its virtual size until the systemwide commit limit—the maximum amount of private committed virtual memory available on the system—is reached (or less if the
process is a member of a job with a job-wide or process virtual memory limit). For
more information, see the section “Page Files.”
■
To view this information, select the Memory processor object and the Cache Bytes counter. As explained in the section “System Working Set,” the total system working set size includes more than just the cache size—
it includes the subset of paged pool, pageable operating system code, and pageable driver code that is resident and in the system working set.
■
Total process working set size
Total system working set size
Size of nonpaged pool
View this information by adding the Memory: Pool Non-
paged Bytes counter.
■
View the sizes of these lists by adding the
Memory: Available Bytes counter. (Use the !memusage kernel debugger command
to get the size of each of the lists separately.)
Size of the free, zero, and standby lists
Your graph or report now contains a representation of all of physical memory except for
two components that you can’t obtain from a performance counter:
■
Nonpaged operating system and driver code
■
The modified and modified-no-write paging lists
Although you can easily obtain the size of both the modified and modified-no-write lists
by using the kernel debugger !memusage command, there’s no easy way to get the size of
the nonpaged operating system and driver code.
Copyrighted material.
382
Microsoft Windows Internals, Fourth Edition
Services the Memory Manager Provides
The memory manager provides a set of system services to allocate and free virtual memory,
share memory between processes, map files into memory, flush virtual pages to disk, retrieve
information about a range of virtual pages, change the protection of virtual pages, and lock the
virtual pages into memory.
Like other Windows executive services, the memory management services allow their caller to
supply a process handle, indicating the particular process whose virtual memory is to be
manipulated. The caller can thus manipulate either its own memory or (with the proper permissions) the memory of another process. For example, if a process creates a child process, by
default it has the right to manipulate the child process’s virtual memory. Thereafter, the parent process can allocate, deallocate, read, and write memory on behalf of the child process by
calling virtual memory services and passing a handle to the child process as an argument. This
feature is used by subsystems to manage the memory of their client processes, and it is also
key for implementing debuggers because debuggers must be able to read and write to the
memory of the process being debugged.
Most of these services are exposed through the Windows API. The Windows API has three
groups of functions for managing memory in applications: page granularity virtual memory
functions (Virtualxxx), memory-mapped file functions (CreateFileMapping, MapViewOfFile),
and heap functions (Heapxxx and the older interfaces Localxxx and Globalxxx). (We’ll
describe the heap manager later in this section.)
The memory manager also provides a number of services, such as allocating and deallocating
physical memory and locking pages in physical memory for direct memory access (DMA)
transfers, to other kernel-mode components inside the executive as well as to device drivers.
These functions begin with the prefix Mm. In addition, though not strictly part of the memory
manager, executive support routines that begin with Ex are used to allocate and deallocate
from the system heaps (paged and nonpaged pool) as well as to manipulate look-aside lists.
We’ll touch on these topics later in this chapter, in the section “System Memory Pools.”
Although we’ll be referring to Windows functions and kernel-mode memory management
and memory allocation routines provided for device drivers, we won’t cover the interface and
programming details but rather the internal operations of these functions. Refer to the Platform Software Development Kit (SDK) and Device Driver Kit (DDK) documentation on
MSDN for a complete description of the available functions and their interfaces.
Large and Small Pages
The virtual address space is divided into units called pages. That is because the hardware memory management unit translates virtual to physical addresses at the granularity of a page. Hence,
a page is the smallest unit of protection at the hardware level. (The various page protection
options are described in the section “Protecting Memory.”) There are two page sizes: small and
large. The actual sizes vary based on hardware architecture, and they are listed in Table 7-1.
Copyrighted material.
Chapter 7:
Table 7-1
Memory Management
383
Page Sizes
Architecture
Small Page Size
Large Page Size
x86
4 KB
4 MB (2 MB on PAE systems)
x64
4 KB
2 MB
IA64
8 KB
16 MB
The advantage of large pages is speed of address translation for references to other data within
the large page. This advantage exists because the first reference to any byte within a large page
will cause the hardware’s translation look-aside buffer (or TLB, which is described in the section “Translation Look-Aside Buffer”) to have in its cache the information necessary to translate references to any other byte within the large page. If small pages are used, more TLB
entries are needed for the same range of virtual addresses, thus increasing recycling of entries
as new virtual addresses require translation. This, in turn, means having to go back to the page
table structures when references are made to virtual addresses outside the scope of a small
page whose translation has been cached. The TLB is a very small cache, and thus large pages
make better use of this limited resource.
To take advantage of large pages, on systems considered to have enough memory (see Table 7-2
for the minimums), Windows maps with large pages the core operating system images
(Ntoskrnl.exe and Hal.dll) as well as core operating system data (such as the initial part of
nonpaged pool and the data structures that describe the state of each physical memory page).
Windows also automatically maps I/O space requests (calls by device drivers to MmMapIoSpace) with large pages if the request is of satisfactory large page length and alignment.
Lastly, Windows also allows applications to map their images, private memory and pagefilebacked sections with large pages. (See the MEM_LARGE_PAGE flag on the VirtualAlloc function.) You can also specify other device drivers to be mapped with large pages by adding a
multistring registry value HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\LargePageDrivers and specifying the names of
the drivers as separately null terminated strings.
Table 7-2
Large Page Minimums
Operating System
Minimum Memory to Use Large Pages
Windows 2000
>127 MB
Windows XP, Windows Server 2003
>255 MB
One side-effect of large pages is that because each large page must be mapped with a single
protection (because hardware memory protection is on a per-page basis), if a large page contains both read-only code and read/write data, the page must be marked as read/write, which
means that the code will be writable. This means device drivers or other kernel mode code
could, as a result of a bug, modify what is supposed to be read-only operating system or driver
code without causing a memory access violation. However, if small pages are used to map the
kernel, the read-only portions of NTOSKRNL.EXE and HAL.DLL will be mapped as read-only
pages. Although this reduces efficiency of address translation, if a device driver (or other kerCopyrighted material.
384
Microsoft Windows Internals, Fourth Edition
nel mode code) attempts to modify a read-only part of the operating system, the system will
crash immediately with the finger pointing at the offending instruction, as opposed to allowing the corruption to occur and later the system crashing (in a harder-to-diagnose way) when
some other component trips over their corrupted data. If you suspect you are experiencing
kernel code corruptions, enable Driver Verifier (described later in this chapter)—this will disable the use of large pages.
Reserving and Committing Pages
Pages in a process address space are free, reserved, or committed. Applications can first reserve
address space and then commit pages in that address space. Or they can reserve and commit
in the same function call. These services are exposed through the Windows VirtualAlloc and
VirtualAllocEx functions.
Reserved address space is simply a way for a thread to reserve a range of virtual addresses for
future use. Attempting to access reserved memory results in an access violation because the
page isn’t mapped to any storage that can resolve the reference.
Committed pages are pages that, when accessed, ultimately translate to valid pages in physical
memory. Committed pages are either private and not shareable or mapped to a view of a section (which might or might not be mapped by other processes). Sections are described in two
upcoming sections: “Shared Memory and Mapped Files” and “Section Objects.”
If the pages are private to the process and have never been accessed before, they are created at
the time of first access as zero-initialized pages (or demand zero). Private committed pages can
later be automatically written to the paging file by the operating system if memory demands dictate. Committed pages that are private are inaccessible to any other process unless they’re
accessed using cross-process memory functions, such as ReadProcessMemory or WriteProcessMemory. If committed pages are mapped to a portion of a mapped file, they might need to be
brought in from disk when accessed unless they’ve already been read earlier, either by the process accessing the page or by another process that had the same file mapped and had previously
accessed the page. (See the section “Shared Memory and Mapped Files” later in this chapter.)
Pages are written to disk through normal modified page writing as pages are moved from the
process working set to the modified list and ultimately to disk. (Working sets and the modified list are explained later in this chapter.) Mapped file pages can also be written back to disk
as a result of an explicit call to FlushViewOfFile.
You can decommit pages and/or release address space with the VirtualFree or VirtualFreeEx
function. The difference between decommittal and release is similar to the difference between
reservation and committal—decommitted memory is still reserved, but released memory is
neither committed nor reserved. (It’s freed.)
Using the two-step process of reserving and committing memory can reduce memory usage
by deferring committing pages until needed but keeping the convenience of virtual contiguity.
Copyrighted material.
Chapter 7:
Memory Management
385
Reserving memory is a relatively fast and inexpensive operation under Windows because it
doesn’t consume any committed pages (a precious system resource) or process page file
quota (a limit on the number of committed pages a process can consume—not necessarily
page file space). All that needs to be updated or constructed is the relatively small internal
data structures that represent the state of the process address space. (We’ll explain these data
structures, called virtual address descriptors, or VADs, later in the chapter.)
Reserving and then committing memory is useful for applications that need a potentially large
virtually contiguous memory buffer; rather than committing pages for the entire region, the
address space can be reserved and then committed later when needed. A utilization of this
technique in the operating system is the user-mode stack for each thread. When a thread is
created, a stack is reserved. (1 MB is the default; you can override this size with the CreateThread
function call or on an imagewide basis by using the /STACK linker flag.) By default, the initial
page in the stack is committed and the next page is marked as a guard page, which isn’t committed, that traps references beyond the end of the committed portion of the stack and
expands it.
Locking Memory
In general, it’s better to let the memory manager decide which pages remain in physical memory. However, there might be special circumstances where this is necessary. Pages can be
locked in memory in two ways:
■
Windows applications can call the VirtualLock function to lock pages in their process
working set. The number of pages a process can lock can’t exceed its minimum working
set size minus eight pages. Therefore, if a process needs to lock more pages, it can
increase its working set minimum with the SetProcessWorkingSetSize function (referred
to in the section “Working Set Management”).
■
Device drivers can call the kernel-mode functions MmProbeAndLockPages, MmLockPagableCodeSection, MmLockPagableDataSection, or MmLockPagableSectionByHandle. Pages
locked using this mechanism remain in memory until explicitly unlocked. Although no
quota is imposed on the number of pages a driver can lock in memory, a driver can’t
lock more pages than the resident available page count will allow.
Allocation Granularity
Windows aligns each region of reserved process address space to begin on an integral boundary defined by the value of the system allocation granularity, which can be retrieved from the
Windows GetSystemInfo function. Currently, this value is 64 KB. This size was chosen so that
if support were added for future processors with large page sizes (for example, up to 64 KB)
or virtually indexed caches that require systemwide physical-to-virtual page alignment, the
risk of requiring changes to applications that made assumptions about allocation alignment
would be reduced. (Windows kernel-mode code isn’t subject to the same restrictions; it can
reserve memory on a single-page granularity.)
Copyrighted material.
386
Microsoft Windows Internals, Fourth Edition
Finally, when a region of address space is reserved, Windows ensures that the size and base of
the region is a multiple of the system page size, whatever that might be. For example, because
x86 systems use 4-KB pages, if you tried to reserve a region of memory 18 KB in size, the actual
amount reserved on an x86 system would be 20 KB. If you specified a base address of 3 KB for
an 18-KB region, the actual amount reserved would be 24 KB.
Shared Memory and Mapped Files
As is true with most modern operating systems, Windows provides a mechanism to share
memory among processes and the operating system. Shared memory can be defined as memory that is visible to more than one process or that is present in more than one process virtual
address space. For example, if two processes use the same DLL, it would make sense to load
the referenced code pages for that DLL into physical memory only once and share those pages
between all processes that map the DLL, as illustrated in Figure 7-1.
Page 1
Original data
Page 2
Original data
Page 3
Process
address
space
Figure 7-1
Physical memory
Process
address
space
Sharing memory between processes
Each process would still maintain its private memory areas in which to store private data, but
the program instructions and unmodified data pages could be shared without harm. As we’ll
explain later, this kind of sharing happens automatically because the code pages in executable
images are mapped as execute-only and writable pages are mapped copy-on-write. (See the
section “Copy-on-Write” for more information.)
The underlying primitives in the memory manager used to implement shared memory are
called section objects, which are called file mapping objects in the Windows API. The internal
structure and implementation of section objects are described in the section “Section Objects”
later in this chapter.
This fundamental primitive in the memory manager is used to map virtual addresses, whether
in main memory, in the page file, or in some other file that an application wants to access as
if it were in memory. A section can be opened by one process or by many; in other words, section objects don’t necessarily equate to shared memory.
A section object can be connected to an open file on disk (called a mapped file) or to committed memory (to provide shared memory). Sections mapped to committed memory are called
Copyrighted material.
Chapter 7:
Memory Management
387
page file backed sections because the pages are written to the paging file if memory demands
dictate. (Because Windows can run with no paging file, page file backed sections might in fact
be “backed” only by physical memory.) As with any other page that is made visible to user
mode (such as private committed pages), shared committed pages are always zero-filled when
they are first accessed.
To create a section object, call the Windows CreateFileMapping function, specifying the file
handle to map it to (or INVALID_HANDLE_VALUE for a page file backed section), and
optionally a name and security descriptor. If the section has a name, other processes can open
it with OpenFileMapping. Or you can grant access to section objects through handle inheritance (by specifying that the handle be inheritable when opening or creating the handle) or
handle duplication (by using DuplicateHandle). Device drivers can also manipulate section
objects with the ZwOpenSection, ZwMapViewOfSection, and ZwUnmapViewOfSection functions.
A section object can refer to files that are much larger than can fit in the address space of a process. (If the paging file backs a section object, sufficient space must exist in the paging file to
contain it.) To access a very large section object, a process can map only the portion of the section object that it requires (called a view of the section) by calling the MapViewOfFile function
and then specifying the range to map. Mapping views permits processes to conserve address
space because only the views of the section object needed at the time must be mapped into
memory.
Windows applications can use mapped files to conveniently perform I/O to files by simply
making them appear in their address space. User applications aren’t the only consumers of
section objects: the image loader uses section objects to map executable images, DLLs, and
device drivers into memory, and the cache manager uses them to access data in cached files.
(For information on how the cache manager integrates with the memory manager, see Chapter 11.) How shared memory sections are implemented, both in terms of address translation
and the internal data structures, is explained later in this chapter.
EXPERIMENT: Viewing Memory Mapped Files
You can list the memory mapped files in a process by using Process Explorer from Sysinternals. To view the memory mapped files by using Process Explorer, configure the
lower pane to show the DLL view. (Click on View, Lower Pane View, DLLs.) Note that
this is more than just a list of DLLs—it represents all memory mapped files in the process address space. Some of these are DLLs, one is the image file (EXE) being run, and
additional entries might represent memory mapped data files. For example, the following display from Process Explorer shows a Microsoft PowerPoint process that has memory mapped the PowerPoint file being edited into its address space:
Copyrighted material.
388
Microsoft Windows Internals, Fourth Edition
You can also search for memory mapped files by clicking on Find, DLL. This can be useful
when trying to determine which process(es) are using a DLL that you are trying to replace.
Finally, comparing the list of DLLs loaded in a process with another instance of the
same program running on another system might help point to DLL configuration
issues, such as the wrong version of a DLL getting loaded in a process. This problem is
known affectionately as “DLL hell.”
Protecting Memory
As explained in Chapter 1, Windows provides memory protection so that no user process can
inadvertently or deliberately corrupt the address space of another process or the operating
system itself. Windows provides this protection in four primary ways.
First, all systemwide data structures and memory pools used by kernel-mode system components can be accessed only while in kernel mode—user-mode threads can’t access these pages.
If they attempt to do so, the hardware generates a fault, which in turn the memory manager
reports to the thread as an access violation.
Note In contrast, Microsoft Windows 95, Microsoft Windows 98, and Microsoft Windows Millennium Edition have some pages in system address space that are writable from user mode, thus
allowing an errant application to corrupt key system data structures and crash the system.
Second, each process has a separate, private address space, protected from being accessed by
any thread belonging to another process. The only exceptions are if the process decides to
share pages with other processes or if another process has virtual memory read or write access
to the process object and thus can use the ReadProcessMemory or WriteProcessMemory functions. Each time a thread references an address, the virtual memory hardware, in concert with
the memory manager, intervenes and translates the virtual address into a physical one. By
controlling how virtual addresses are translated, Windows can ensure that threads running in
one process don’t inappropriately access a page belonging to another process.
Copyrighted material.
Chapter 7:
Memory Management
389
Third, in addition to the implicit protection virtual-to-physical address translation offers, all
processors supported by Windows provide some form of hardware-controlled memory protection (such as read/write, read-only, and so on); the exact details of such protection vary
according to the processor. For example, code pages in the address space of a process are
marked read-only and are thus protected from modification by user threads.
Table 7-3 lists the memory protection options defined in the Windows API. (See the VirtualProtect, VirtualProtectEx, VirtualQuery, and VirtualQueryEx functions.)
Table 7-3
Memory Protection Options Defined in the Windows API
Attribute
Description
PAGE_NOACCESS
Any attempt to read from, write to, or execute code in this region
causes an access violation.
PAGE_READONLY
Any attempt to write to (and on processors with no execute support, execute code in) memory causes an access violation, but
reads are permitted.
PAGE_READWRITE
The page is readable and writable, but not executable.
PAGE_EXECUTE*
Any attempt to write to code in memory in this region causes an
access violation, but execution (and read on all existing processors) is permitted.
PAGE_EXECUTE_READ*
Any attempt to write to code in memory in this region causes an
access violation, but executes and reads are permitted.
PAGE_EXECUTE_READWRITE*
The page is readable, writable, and executable—no action will
cause an access violation.
PAGE_WRITECOPY
Any attempt to write to memory in this region causes the system
to give the process a private copy of the page. On processors
with no execute support, attempts to execute code in memory in
this region cause an access violation.
PAGE_EXECUTE_WRITECOPY
Any attempt to write to memory in this region causes the system to
give the process a private copy of the page. Reading and executing
code in this region is permitted. (No copy is made in this case.)
PAGE_GUARD
Any attempt to read from or write to a guard page raises an
EXCEPTION_GUARD_PAGE exception and turns off the guard
page status. Guard pages thus act as a one-shot alarm. Note that
this flag can be specified with any of the page protections listed
in this table except PAGE_NOACCESS.
PAGE_NOCACHE
Use physical memory that is not cached. This is not recommended for general usage. It is useful for device drivers—for example,
mapping a video frame buffer with no caching.
PAGE_WRITECOMBINE
Enables write-combined memory accesses. When enabled, the
processor might cache memory write requests to optimize performance. For example, if multiple writes are made to the same
address, only the most recent write might occur.
* No execute protection is supported by Windows XP Service Pack 2 and Windows Server 2003 Service Pack 1 and
later on processors that have the necessary hardware support (for example, the x64, IA-64, and future x86 processors). On earlier versions of Windows and on processors that do not support no execute protection, all page permissions allow execution. The next section contains a more complete description of no execute protection.
Copyrighted material.
390
Microsoft Windows Internals, Fourth Edition
And finally, shared memory section objects have standard Windows access-control lists
(ACLs) that are checked when processes attempt to open them, thus limiting access of shared
memory to those processes with the proper rights. Security also comes into play when a
thread creates a section to contain a mapped file. To create the section, the thread must have
at least read access to the underlying file object or the operation will fail.
Once a thread has successfully opened a handle to a section, its actions are still subject to the
memory manager and the hardware-based page protections described earlier. A thread can
change the page-level protection on virtual pages in a section if the change doesn’t violate the
permissions in the ACL for that section object. For example, the memory manager allows a
thread to change the pages of a read-only section to have copy-on-write access but not to have
read/write access. The copy-on-write access is permitted because it has no effect on other processes sharing the data.
These four primary memory protection mechanisms are part of the reason that Windows is a
robust, reliable operating system that is impervious to and resilient to application errors.
No Execute Page Protection
Although the Windows memory management API has always had page protection bits
defined in the programming interface that allow specification of whether or not pages can
contain executable code, it is only as of Windows XP Service Pack 2 and Windows Server
2003 Service Pack 1 that this capability is supported on processors that have hardware “no
execute” protection, including all AMD64 processors (including AMD Athlon64 and AMD
Opteron) and certain exclusively 32-bit AMD processors (selected AMD Sempron processors—
details are in AMD's product literature), Intel IA-64, and Intel Pentium 4 and Xeon processors
with Intel Extended Memory 64 Technology (EM64T).
No execute page protection (also referred to as data execution prevention, or DEP) means an
attempt to transfer control to an instruction in a page marked as “no execute” will generate an
access fault. This can prevent certain types of viruses from exploiting bugs in the system that
permit the execution of code placed in a data page. If an attempt is made in kernel mode to
execute code in a page marked as no execute, the system will crash with the ATTEMPTED_
EXECUTE_OF_NOEXECUTE_MEMORY bugcheck code. If this occurs in user mode, a
STATUS_ACCESS_VIOLATION (0xc0000005) exception is delivered to the thread attempting the illegal reference. If a process allocates memory that needs to be executable, it must
explicitly mark such pages by specifying the PAGE_EXECUTE, PAGE_EXECUTE_READ,
PAGE_EXECUTE_READWRITE, or PAGE_EXECUTE_WRITECOPY flags on the page granularity memory allocation functions.
On 64-bit versions of Windows, execution protection is always applied to all 64-bit programs
and device drivers and cannot be disabled. Execution protection for 32-bit programs depends
on system configuration settings, described shortly. On 64-bit Windows, execution protection
is applied to thread stacks (both user and kernel mode), user mode pages not specifically
marked as executable, kernel paged pool, and kernel session pool (for a description of kernel
memory pools, see the section “System Memory Pools”). However, on 32-bit Windows,
Copyrighted material.
Chapter 7:
Memory Management
391
execution protection is only applied to thread stacks and user mode pages, not to paged
pool and session pool. Also, when execution protection is enabled on 32-bit Windows, the
system automatically boots in PAE mode (automatically selecting the PAE kernel, \Windows\
System32\Ntkrnlpa.exe). For a description of PAE, see the section “Physical Address Extension (PAE).”
The application of execution protection for 32-bit programs depends on the Boot.ini /NOEXECUTE= switch. The settings can be changed by going to the Data Execution Prevention tab
under My Computer, Properties, Advanced, Performance Settings. (See Figure 7-2.) When you
configure no execute protection with the DEP settings dialog box, Boot.ini is modified to add the
appropriate /NOEXECUTE Boot.ini switch. For a list of the variations of the switch and how they
correspond to the DEP settings tab, see Table 7-4. 32-bit applications that are excluded from
execution protection are listed as registry values under the key HKLM\Software\Microsoft\
Windows NT\CurrentVersion\AppCompatFlags\Layers with the value name being the full path
of the executable and the data set to “DisableNXShowUI”.
Figure 7-2
Data Execution Protection settings
On Windows XP (both 64-bit and 32-bit versions), execution protection for 32-bit applications is configured by default to apply only to core Windows operating system executables
(/NOEXECUTE=OPTIN) so as not to break 32-bit applications that might rely on being
able to execute code in pages not specifically marked as executable. On Windows Server 2003
systems, execution protection for 32-bit applications is configured by default to apply to all
32-bit programs (/NOEXECUTE=OPTOUT).
Note
To obtain a complete list of which programs are protected, install the Windows Application Compatibility Toolkit (downloadable from www.microsoft.com) and run the Compatibility Administrator Tool. Click on System Database, Applications, and Windows Components
and, on the right-hand pane, the list of protected executables will be shown.
Copyrighted material.
392
Microsoft Windows Internals, Fourth Edition
Table 7-4 Boot.ini/NOEXECUTE Switch
Boot.ini Switch
Option in DEP Settings
Dialog Box
Meaning
/NOEXECUTE=OPTIN
Turn on DEP for necessary
Windows programs and services only.
Enables DEP for core
Windows system images.
/NOEXECUTE=OPTOUT
Turn on DEP for all programs
and services except those that
I select.
Enables DEP for all executables except those specified.
/NOEXECUTE=ALWAYSON
(No GUI interface to select
this option)
Enables DEP for all components with no ability to exclude certain applications.
/NOEXECUTE=ALWAYSOFF
(No GUI interface to select
this option)
Disables DEP (not recommended).
Software Data Execution Prevention
Because most processors running Windows these days do not support hardware “no
execute” protection, Windows XP Service Pack 2 and Windows Server 2003 Service
Pack 1 and later support limited software data execution prevention (DEP). One aspect
of software DEP reduces exploits of the exception handling mechanism in Windows.
(See Chapter 3 for a description of structured exception handling.) If the program’s
image files are built with safe structured exception handling (a new feature in the
Microsoft Visual C++ 2003 compiler), before an exception is dispatched, the system verifies that the exception handler is registered in the function table located within the
image file. If the program’s image files are not built with safe structured exception handling, software DEP ensures that before an exception is dispatched, the exception handler is located within a memory region marked as executable.
Copy-on-Write
Copy-on-write page protection is an optimization the memory manager uses to conserve
physical memory. When a process maps a copy-on-write view of a section object that contains read/write pages, instead of making a process private copy at the time the view is
mapped (as the Hewlett Packard OpenVMS operating system does), the memory manager
defers making a copy of the pages until the page is written to. All modern UNIX systems use
this technique as well. For example, as shown in Figure 7-3, two processes are sharing three
pages, each marked copy-on-write, but neither of the two processes has attempted to modify
any data on the pages.
Copyrighted material.
Chapter 7:
Memory Management
393
Page 1
Original data
Page 2
Original data
Page 3
Process
address
space
Figure 7-3
Physical memory
Process
address
space
The “before” of copy-on-write
If a thread in either process writes to a page, a memory management fault is generated. The
memory manager sees that the write is to a copy-on-write page, so instead of reporting the
fault as an access violation, it allocates a new read/write page in physical memory, copies the
contents of the original page to the new page, updates the corresponding page-mapping information (explained later in this chapter) in this process to point to the new location, and dismisses the exception, thus causing the instruction that generated the fault to be reexecuted.
This time, the write operation succeeds, but as shown in Figure 7-4, the newly copied page is
now private to the process that did the writing and isn’t visible to the other processes still
sharing the copy-on-write page. Each new process that writes to that same shared page will
also get its own private copy.
Page 1
Original data
Page 2
Modified data
Copy of page 2
Process
address
space
Page 3
Process
address
space
Figure 7-4
Physical memory
The “after” of copy-on-write
One application of copy-on-write is to implement breakpoint support in debuggers. For example, by default, code pages start out as execute-only. If a programmer sets a breakpoint while
debugging a program, however, the debugger must add a breakpoint instruction to the code.
It does this by first changing the protection on the page to PAGE_EXECUTE_READWRITE
and then changing the instruction stream. Because the code page is part of a mapped section,
the memory manager creates a private copy for the process with the breakpoint set, while
other processes continue using the unmodified code page.
Copy-on-write is one example of an evaluation technique known as lazy evaluation that the memory manager uses as often as possible. Lazy-evaluation algorithms avoid performing an expensive
operation until absolutely required—if the operation is never required, no time is wasted on it.
Copyrighted material.
394
Microsoft Windows Internals, Fourth Edition
The POSIX subsystem takes advantage of copy-on-write to implement the fork function. Typically, when a UNIX application calls the fork function to create another process, the first thing
that the new process does is call the exec function to reinitialize the address space with an executable program. Instead of copying the entire address space on fork, the new process shares
the pages in the parent process by marking them copy-on-write. If the child writes to the data,
a process private copy is made. If not, the two processes continue sharing and no copying
takes place. One way or the other, the memory manager copies only the pages the process
tries to write to rather than the entire address space.
To examine the rate of copy-on-write faults, see the performance counter Memory: Write
Copies/Sec.
Heap Manager
Many applications allocate smaller blocks than the 64-KB minimum allocation granularity
possible using page granularity functions such as VirtualAlloc. Allocating such a large area for
relatively small allocations is not optimal from the memory usage and performance standpoint. To address this need, Windows provides a component called the heap manager, which
manages allocations inside larger memory areas reserved using the page granularity memory
allocation functions. The allocation granularity in the heap manager is relatively small: 8 bytes
on 32-bit systems and 16 bytes on 64-bit systems. The heap manager has been designed to
optimize memory usage and performance in the case of these smaller allocations.
The heap manager exists in two places: Ntdll.dll and Ntoskrnl.exe. The subsystem APIs (such
as the Windows heap APIs) call the functions in Ntdll, and various executive components and
device drivers call the functions in Ntoskrnl. Its native interfaces (prefixed with Rtl) are available only for use in internal Windows components or kernel mode device drivers. The documented Windows API interface to the heap (prefixed with Heap) are thin functions that call
the native functions in Ntdll.dll. In addition, legacy APIs (prefixed with either Local or Global)
are provided to support older Windows applications. The most common Windows heap functions are:
■
HeapCreate or HeapDestroy—Creates or deletes, respectively, a heap. The initial reserved
and committed size can be specified at creation.
■
HeapAlloc—Allocates a heap block.
■
HeapFree—Frees a block previously allocated with HeapAlloc.
■
HeapReAlloc—Changes the size of an existing allocation (grows or shrinks an existing
block).
■
HeapLock or HeapUnlock—Controls mutual exclusion to the heap operations.
■
HeapWalk—Enumerates the entries and regions in a heap.
Copyrighted material.
Chapter 7:
Memory Management
395
Types of Heaps
Each process has at least one heap: the default process heap. The default heap is created at
process startup and is never deleted during the process’s lifetime. It defaults to 1 MB in size,
but it can be bigger by specifying a starting size in the image file by using the /HEAP linker
flag. This size is just the initial reserve, however—it will expand automatically as needed. (You
can also specify the initial committed size in the image file.)
The default heap can be explicitly used by a program or implicitly used by some Windows
internal functions. An application can query the default process heap by making a call to the
Windows function GetProcessHeap. Processes can also create additional private heaps with the
HeapCreate function. When a process no longer needs a private heap, it can recover the virtual
address space by calling HeapDestroy. An array with all heaps is maintained in each process,
and a thread can query them with the Windows function GetProcessHeaps.
A heap can manage allocations either in large memory regions reserved from the memory
manager via VirtualAlloc or from memory mapped file objects mapped in the process address
space. The latter approach is rarely used in practice, but it’s suitable for scenarios where the
content of the blocks need to be shared between two processes or between a kernel mode and
a user mode component. If a heap is built on top of a memory mapped file region, certain constraints apply with respect to the component that can call heap functions. First, the internal
heap structures use pointers, and therefore do not allow relocation to different addresses. Second, the synchronization across multiple processes or between a kernel component and a
user process is not supported by the heap functions. Also, in the case of a shared heap
between user and kernel mode, the user mode mapping should be read-only to prevent user
mode code from corrupting the heap internal structures, which would result in a system
crash.
Heap Manager Structure
As shown in Figure 7-5, the heap manager is structured in two layers: an optional front-end
layer and the core heap. The core heap handles the basic functionality and is mostly common
across the user and kernel mode heap implementations. The core functionality includes the
management of blocks inside segments, the management of the segments, policies for extending the heap, committing and decommitting memory, and management of the large blocks.
For user mode heaps only, an optional front-end heap layer can exist on top of the existing
core functionality. There are two types of front-end layers: look-aside lists and the Low Fragmentation Heap (or LFH, which is available in Windows XP and later), both of which are
described later in this section. Only one front-end layer can be used for one heap at one time.
Copyrighted material.
396
Microsoft Windows Internals, Fourth Edition
Application
Windows heap APIs
(HeapAlloc, HeapFree, LocalAlloc, GlobalAlloc etc.)
Front-end heap layer
(optional)
Heap manager
Core heap layer
Memory manager
Figure 7-5
Heap manager layers
Heap Synchronization
The heap manager supports concurrent access from multiple threads by default. However, if a
process is single threaded or uses an external mechanism for synchronization, it can tell the
heap manager to avoid the overhead of synchronization by specifying HEAP_NO_SERIALIZE
either at heap creation or on a per-allocation basis.
A process can also lock the entire heap and prevent other threads from performing heap operations for operations that would require consistent states across multiple heap calls. For
instance, enumerating the heap blocks in a heap with the Windows function HeapWalk
requires locking the heap if multiple threads can perform heap operations simultaneously.
If heap synchronization is enabled, there is one lock per heap that protects all internal heap
structures. In heavily multithreaded applications (especially when running on multiprocessor
systems), the heap lock might become a significant contention point. In that case, performance might be improved by enabling the front-end heap, described in an upcoming section.
Look-Aside Lists
Look-aside lists are single linked lists that allow elementary operations such as “push to the
list” or “pop from the list” in a last in, first out (LIFO) order with nonblocking algorithms. A
simplified version of these data structures is also available to Windows applications through
the functions InterlockedPopEntrySList or InterlockedPushEntrySList. There are 128 look-aside
Copyrighted material.
Chapter 7:
Memory Management
397
lists per heap, which handle allocations up to 1 KB on 32-bit platforms and up to 2 KB on 64bit platforms.
Look-aside lists provide a significant performance improvement over normal heap allocations
because multiple threads can concurrently perform allocation and deallocation operations
without acquiring the heap global lock. Also, cache locality is optimized by using a LIFO
ordering model and by accessing fewer internal data files in each heap operation.
The heap manager maintains the number of blocks in each look-aside list and some counters
that help tune the usage of each list independently. If a thread allocates a block of a size that
does not exist in the corresponding look-aside list, the heap manager will forward the call to
the core heap manager to complete the operation. The heap manager will also update in this
case an internal counter of misses at allocations, which will later be used in tuning decisions.
The heap manager creates look-aside lists automatically when a heap is created, as long as no
debugging options are enabled and the heap is expandable. Some applications might have compatibility issues as a result of the heap manager’s use of look-aside lists. In this case, these legacy
applications can be made to run properly by specifying the DisableHeapLookaside flag in the
image file execution options for that application. (Image file execution options can be specified
using the Imagecfg.exe tool in the Windows 2000 Server Resource Kit, supplement 1.)
The Low Fragmentation Heap
Many applications running in Windows have relatively small heap memory usage (usually
less than one megabyte). For this class of applications, the heap manager’s best-fit policy
helps keep a low memory footprint for each process. However, this strategy does not scale for
large processes and multiprocessor machines. In these cases, memory available for heap usage
might be reduced as a result of heap fragmentation. Performance can suffer in scenarios where
only certain sizes are often used concurrently from different threads scheduled to run on different processors. This is because several processors need to modify the same memory location (for example, the head of the look-aside list for that particular size) at the same time, thus
invalidating the corresponding cache line for the other processors.
The Low Fragmentation Heap (LFH) addresses these issues using the core heap manager and
look-aside lists. Unlike the look-aside lists that are used as front-end heaps by default if other
heap settings are allowing it, LFH is turned on only if an application calls the HeapSetInformation function. For large heaps, a significant percentage of allocations is generally grouped in a
relatively small number of buckets of certain sizes. The allocation strategy used by LFH is to
optimize the usage for these patterns by efficiently handling same-size blocks.
To address scalability, LFH expands the frequently accessed internal structures to a number
of slots that is two times larger than the current number of processors on the machine. The
assignment of threads to these slots is done by an LFH component called the affinity manager.
Initially, LFH starts using the first slot for heap allocations; however, if a contention is detected
at accessing some internal data, LFH switches the current thread to use a different slot. FurCopyrighted material.
398
Microsoft Windows Internals, Fourth Edition
ther contentions will spread threads on more slots. These slots are controlled for each size
bucket, also to improve locality and minimize the overall memory consumption.
Heap Debugging Features
The heap manager includes several features to help detect bugs by using the following heap
functions:
■
Enable tail checking
■
Enable free checking A free block is filled with a pattern, which is checked at various
The end of each block carries a signature, which is checked when
the block is released. If a buffer overrun destroyed the signature entirely or partially, the
heap will report this error.
points when the heap manager needs to access the block (such as at removal from the
free list to allocate the block). If the process continued to write to the block after freeing
it, the heap manager will detect changes in the pattern and the error will be reported.
■
Parameter checking This function consists of extensive checking of the parameters
passed to the heap functions.
■
Heap validation
■
Heap tagging and stack traces support
The entire heap is validated at each heap call.
This function supports specifying tags for allocation and/or captures user mode stack traces for the heap calls to help narrow the possible causes of a heap error.
The first three options are enabled by default if the loader detects that a process is started
under the control of a debugger. (A debugger can override this behavior and turn off these features.) The heap debugging features can be specified for an executable image by setting various debugging flags in the image header using the gflags tool. (See the section “Windows
Global Flags” in Chapter 3.) Or, heap debugging options can be enabled using the !heap command in the standard Windows debuggers. (See the debugger help for more information.)
Enabling heap debugger options affects all heaps in the process. Also, if any of the heap debug
options are enabled, the front-end heap will be disabled automatically and the core heap will
be used (with the required debugging options enabled). The front-end heaps are also not used
for heaps that are not expandable (because of the extra overhead added to the existing heap
structures) or for heaps that do not allow serialization.
Pageheap
Because the tail and free checking options described in the preceding sections might be discovering corruptions that occurred well before the problem was detected, an additional heap
debugging tool, called pageheap, is provided that directs all or part of the heap calls to a different heap manager. Pageheap is part of the Windows Application Compatibility Toolkit, which
can be downloaded from www.microsoft.com. The pageheap places allocations at the end of
pages so that if a buffer overrun occurs, it will cause an access violation, making it easier to
Copyrighted material.
Chapter 7:
Memory Management
399
detect the offending code. Optionally, pageheap allows placing the blocks at the beginning of
the pages to detect buffer underrun problems. (This is a rare occurrence.) The pageheap also
can protect freed pages against any access to detect references to heap blocks after they have
been freed.
Note that using the pageheap can result in running out of address space because of the significant overhead added for small allocations. Also, performance can suffer as of result of the
increase of references to demand zero pages, loss of locality, and additional overhead caused by
frequent calls to validate heap structures. A process can reduce the impact by specifying that the
pageheap be used only for blocks of certain sizes, address ranges, and/or originating DLLs.
Note For more information on pageheap, see article 286470 in the Microsoft Knowledge
Base (http://support.microsoft.com).
Address Windowing Extensions
Although the 32-bit version of Windows can support up to 128 GB of physical memory (as
shown in Table 2-4), each 32-bit user process has by default only a 2-GB virtual address space.
(This can be configured up to 3 GB when using the /3GB and /USERVA Boot.ini switches,
described in the upcoming section “x86 User Address Space Layouts.”) To allow a 32-bit process to allocate and access more physical memory than can be represented in its limited
address space, Windows provides a set of functions called Address Windowing Extensions
(AWE). For example, on a Windows 2000 Advanced Server system with 8 GB of physical
memory, a database server application could use AWE to allocate and use perhaps 6 GB of
memory as a database cache.
Allocating and using memory via the AWE functions is done in three steps:
1. Allocating the physical memory to be used
2. Creating a region of virtual address space to act as a window to map views of the physical memory
3. Mapping views of the physical memory into the window
To allocate physical memory, an application calls the Windows function AllocateUserPhysicalPages. (This function requires the Lock Pages In Memory user right.) The application then
uses the Windows VirtualAlloc function with the MEM_PHYSICAL flag to create a window in
the private portion of the process’s address space that is mapped to some or all of the physical
memory previously allocated. The AWE-allocated memory can then be used with nearly all
the Windows APIs. (For example, the Microsoft DirectX functions can’t use AWE memory.)
If an application creates a 256-MB window in its address space and allocates 4 GB of physical
memory (on a system with more than 4 GB of physical memory), the application can use the
MapUserPhysicalPages or MapUserPhysicalPagesScatter Windows functions to access any
Copyrighted material.
400
Microsoft Windows Internals, Fourth Edition
portion of the physical memory by mapping the memory into the 256-MB window. The size
of the application’s virtual address space window determines the amount of physical memory
that the application can access with a given mapping. Figure 7-6 shows an AWE window in a
server application address space mapped to a portion of physical memory previously allocated by AllocateUserPhysicalPages.
4 GB
64 GB
System
address
space
2 GB
User
address
space
AWE window
AWE memory
0
Server application
address space
0
Physical memory
Figure 7-6
Using AWE to map physical memory
The AWE functions exist on all editions of Windows and are usable regardless of how much
physical memory a system has. However, AWE is most useful on systems with more than 2 GB
of physical memory because it’s the only way for a 32-bit process to directly use more than 2
GB of memory. Another use is for security purposes: because AWE memory is never paged
out, the data in AWE memory could never have a copy in the paging file that someone could
examine by rebooting into an alternate operating system.
Finally, there are some restrictions on memory allocated and mapped by the AWE functions:
■
Pages can’t be shared between processes.
■
The same physical page can’t be mapped to more than one virtual address in the same
process.
■
On older versions of Windows, page protection is limited to read/write. In Windows
Server 2003 Service Pack 1 and later, no access and read-only are supported.
For a description of the page table data structures used to map memory on systems with more
than 4 GB of physical memory, see the section “Physical Address Extension (PAE).”
Copyrighted material.
Chapter 7:
Memory Management
401
System Memory Pools
At system initialization, the memory manager creates two types of dynamically sized memory
pools that the kernel-mode components use to allocate system memory:
■
Nonpaged pool
■
A region of virtual memory in system space that can be paged in and out of
the system. Device drivers that don’t need to access the memory from DPC/dispatch
level or above can use paged pool. It is accessible from any process context.
Consists of ranges of system virtual addresses that are guaranteed to
reside in physical memory at all times and thus can be accessed at any time (from any
IRQL level and from any process context) without incurring a page fault. One of the reasons nonpaged pool is required is because of the rule described in Chapter 2: page faults
can’t be satisfied at DPC/dispatch level or above.
Paged pool
Both memory pools are located in the system part of the address space and are mapped in the
virtual address space of every process. (In Table 7-8, you’ll find out where in the system memory they start.) The executive provides routines to allocate and deallocate from these pools;
for information on these routines, see the functions that start with ExAllocatePool in the Windows DDK documentation.
Uniprocessor systems have three paged pools; multiprocessor systems have five. Having more
than one paged pool reduces the frequency of system code blocking on simultaneous calls to
pool routines. Both nonpaged and paged pool start at an initial size based on the amount of
physical memory on the system and then grow, if necessary, up to a maximum size computed
at system boot time.
Note Future releases of Windows might support dynamic pool sizes, which means the maximum size will no longer apply. Therefore, applications and device drivers should not assume
that the pool size maximum is a fixed value for each system.
Configuring Pool Sizes
You can override the initial size of these pools by changing the values NonPagedPoolSize and
PagedPoolSize in the registry key HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management from 0 (which causes the system to compute the size) to the size
desired in bytes. You can’t, however, go beyond the maximum pool sizes listed in Table 7-5. A
value of 0xFFFFFFFF for PagedPoolSize indicates that the largest possible size is selected, which
means allowing a larger paged pool at the expense of system page table entries (PTEs).
Table 7-5
Maximum Pool Sizes
Pool Type
Maximum on 32-Bit Systems
Maximum on 64-Bit Systems
Nonpaged
256 MB (128 MB if booted /3GB)
128 GB
Paged
491.875 MB (Windows 2000 and Windows XP); 128 GB
650 MB (Windows Server 2003)
Copyrighted material.
402
Microsoft Windows Internals, Fourth Edition
The computed sizes are stored in four kernel variables, three of which are exposed as performance counters. These variables and counters, as well as the two registry keys that can alter
the sizes, are listed in Table 7-6.
Table 7-6
System Pool Size Variables and Performance Counters
Kernel Variable
Performance
Counter
Registry Key to Override
Description
MmSizeOfNonPagedPoolInBytes
Memory: Pool
Nonpaged Bytes
Not applicable
Current size of
nonpaged pool
MmMaximumNonPagedPoolInBytes
Not available
HKLM\SYSTEM\CurrentControlSet\
Control\Session Manager\Memory
Management\NonPagedPoolSize
Maximum size of
nonpaged pool
Not available
Memory: Pool
Paged Bytes
Not applicable
Current virtual
size of paged
pool
MmPagedPoolPage
(number of pages)
Memory: Pool
Paged Resident
Bytes
Not applicable
Current physical
(resident) size of
paged pool
MmSizeOfPagedPoolInBytes
Not available
HKLM\SYSTEM\CurrentControlSet\
Control\Session Manager\
Memory Management\PagedPoolSize
Maximum (virtual) size of paged
pool
EXPERIMENT: Determining the Maximum Pool Sizes
Because paged and nonpaged pool represent a critical system resource, it is important to
know when you’re nearing the maximum size computed for your system so that you can
determine whether you need to override the default maximum with the appropriate registry values. The pool-size performance counters report only the current size, however,
not the maximum size. So you don’t know when you’re reaching the limit until you’ve
exhausted pool. (As noted earlier, future versions of Windows might support dynamically sized pools. Therefore, the need to check the pool maximums might no longer be
necessary in the future.)
You can obtain the pool maximums by using either Process Explorer or live kernel
debugging (explained in Chapter 1). To view pool maximums with Process Explorer,
click on View, System Information. The pool maximums are displayed in the Kernel
Memory section as shown below:
Copyrighted material.
Chapter 7:
Memory Management
403
Note that for Process Explorer to retrieve this information, it must have access to the
symbols for the kernel running on your system. (For a description of how to configure
Process Explorer to use symbols, see the experiment “Viewing Process Details with Process Explorer” in Chapter 1.)
To view the same information by using the kernel debugger, you can use the !vm command as shown below:
lkd> !vm
*** Virtual Memory Usage ***
Physical Memory:
261980
( 1047920 Kb)
Page File: \??\C:\pagefile.sys
Current:
1024000Kb Free Space:
862236Kb
Minimum:
1024000Kb Maximum:
1024000Kb
Available Pages:
96077
( 384308 Kb)
ResAvail Pages:
194722
( 778888 Kb)
Locked IO Pages:
89
(
356 Kb)
Free System PTEs:
175933
( 703732 Kb)
Free NP PTEs:
31832
( 127328 Kb)
Free Special NP:
0
(
0 Kb)
Modified Pages:
529
(
2116 Kb)
Modified PF Pages:
529
(
2116 Kb)
NonPagedPool Usage:
5617
(
22468 Kb)
NonPagedPool Max:
52741
( 210964 Kb)
PagedPool 0 Usage:
6582
(
26328 Kb)
PagedPool 1 Usage:
3009
(
12036 Kb)
PagedPool 2 Usage:
3044
(
12176 Kb)
PagedPool Usage:
12635
(
50540 Kb)
PagedPool Maximum:
40960
( 163840 Kb)
Copyrighted material.
404
Microsoft Windows Internals, Fourth Edition
On this system, nonpaged and paged pool were far from their maximums. You can also
examine the values of the kernel variables listed in Table 7-6:
kd> dd mmmaximumnonpagedpoolinbytes l1
8047f620 0328c000
kd> ? 328c000
Evaluate expression: 53002240 = 0328c000
kd> dd mmsizeofpagedpoolinbytes l1
80470a98 06800000
kd> ? 6800000
Evaluate expression: 109051904 = 06800000
From this example, you can see that the maximum size of nonpaged pool is 53,002,240
bytes (approximately 50 MB) and the maximum size of paged pool is 109,051,904 bytes
(104 MB). On the test system we used for this example, current nonpaged pool usage
was 5.5 MB and paged pool usage was 34 MB, so both pools were far from full.
Monitoring Pool Usage
The Memory performance counter object has separate counters for the size of nonpaged pool
and paged pool (both virtual and physical). In addition, the Poolmon utility (in the Windows
Support Tools) allows you to monitor the detailed usage of nonpaged and paged pool. To do
so, you must have the internal Enable Pool Tagging option enabled. Pool tagging is turned on
by default as of Windows Server 2003 (in fact, it cannot be turned off) and in all checked
builds of Windows. To enable pool tagging, run the Gflags utility in the Windows Support
Tools, Platform SDK, or DDK, and select Enable Pool Tagging, as shown here:
Copyrighted material.
Chapter 7:
Memory Management
405
Then click Apply, and reboot the system. After the system reboots, run Poolmon; you should
see a display like this one:
The highlighted lines represent changes to the display. (You can disable the highlighting feature by typing l while running Poolmon. Type l again to reenable highlighting.) Type ? while
Poolmon is running to bring up its help screen. You can configure which pools you want to
monitor (paged, nonpaged, or both) and the sort order. Also, the command-line options are
shown, which allow you to monitor specific structures (or everything but one structure type).
For example, the command poolmon -iCM will monitor only structures of type CM (the configuration manager, which manages the registry). The columns have the meanings shown in
Table 7-7.
Table 7-7
Poolmon Columns
Column
Explanation
Tag
Four-byte tag given to the pool allocation
Type
Pool type (paged or nonpaged pool)
Allocs
Count of all allocations (The number in parentheses shows the difference in the
Allocs column since the last update.)
Frees
Count of all Frees (The number in parentheses shows the difference in the Frees
column since the last update.)
Diff
Count of Allocs minus Frees
Bytes
Total bytes consumed by this structure type (The number in parentheses shows
the difference in the Bytes column since the last update.)
Per Alloc
Size in bytes of a single instance of this structure type
In this example, CM structures (the configuration manager, or registry) are taking up the
most space in paged pool, and MmSt structures (a memory management–related structure
used for mapped files) are taking up the most space in nonpaged pool.
For a description of the meaning of the pool tags used by Windows, see the file \Program
Files\Debugging Tools for Windows\Triage\Pooltag.txt. (This file is installed as part of the
Windows Debugging Tools described in Chapter 1.) Because third-party device driver pool
tags are not listed in this file, you can use the “-c” switch on the version of Poolmon that comes
with the Windows Server 2003 Device Driver Kit (DDK) to generate a local pool tag file
Copyrighted material.
406
Microsoft Windows Internals, Fourth Edition
(Localtag.txt). This file will contain pool tags used by drivers found on your system, including
third-party drivers. (Note that if a device driver binary has been deleted after it was loaded, its
pool tags will not be recognized.)
Alternatively, you can search the device drivers on your system for a pool tag by using the
Strings.exe tool from www.sysinternals.com. For example, the command
strings \windows\system32\drivers\*.sys > findstr /i "abcd"
will display drivers that contain the string “abcd”. Note that device drivers do not necessarily
have to be located in \Windows\System32\Drivers—they can be in any folder. To list the full
path of all loaded drivers, click Start, click Run, and type Msinfo32. Then click on Software
Environment and System Drivers. As already noted, if a device driver has been loaded and
then deleted from the system, it will not be listed here.
An alternative to view pool usage by device driver is to enable the pool tracking feature of
Driver Verifier, explained later in this chapter. While this makes the mapping from pool tag to
device driver unnecessary, it does require a reboot (to enable Driver Verifier on the desired
drivers). After rebooting with pool tracking enabled, you can either run the graphical Driver
Verifier Manager (\Windows\System32\Verifier.exe) or use the Verifier /Log command to
send the pool usage information to a file.
Finally, if you are looking at a crash dump, you can view pool usage with the kernel debugger
!poolused command. The command !poolused 2 shows nonpaged pool usage sorted by structure tag using the most amount of pool. The command !poolused 4 lists paged pool usage,
again sorted by structure tag using the most amount of pool. The following example shows
the partial output from these two commands:
lkd> !poolused 2
Sorting by NonPaged Pool Consumed
Pool Used:
NonPaged
Paged
Tag
Allocs
Used
Allocs
Used
MmCm
386 2237440
0
0
ary: nt!mm
File
12544 2121232
0
0
Devi
385 1487184
0
0
Ntfr
19163 1226888
0
0
kd> !poolused 4
Sorting by Paged Pool Consumed
Pool Used:
NonPaged
Tag
Allocs
Used
CM
11
704
Gh 5
0
0
MmSt
0
0
Ntff
10
2240
Gla1
1
128
Ttfd
0
0
Copyrighted material.
Paged
Allocs
Used
5146 22311904
727 2523648
968 1975872
790
682560
383
600544
265
545440
Calls made to MmAllocateContiguousMemory , Bin
File objects
Device objects
ERESOURCE , Binary: ntfs.sys
Chapter 7:
Memory Management
407
EXPERIMENT: Troubleshooting a Pool Leak
In this experiment, you will fix a real paged pool leak on your system so that you can put
to use the techniques described in the previous section to track down the leak. The leak
will be generated by the NotMyFault tool, which you can download from www.sysinternals.com/windowsinternals.shtml. (Note that this tool is not listed on the main Sysinternals tools list.) When you run NotMyFault.exe, it loads a device driver Myfault.sys and
presents the following dialog box:
1. Click the Leak Pool button. This causes NotMyFault to begin sending requests to
the Myfault device driver to allocate paged pool. (Do not click the Do Bug button
or you will experience a system crash; this button is used in Chapter 14 to demonstrate various types of crashes.) NotMyFault will continue to do this until you click
the Stop Leaking button. Note that the paged pool is not released even when you
close the program; the pool is permanently leaked until you reboot the system.
However, because you will leak pool only for a short period, this should not cause
any problems on your system.
2. While the pool is leaking, first open Task Manager and click on the Performance
tab. You should notice Paged Pool climbing. You can also check this with Process
Explorer’s System Information display. (Click on Show and then System Information.)
3. To determine the pool tag that is leaking, run Poolmon and press the “b” key to
sort by the number of bytes. Press “p” twice so that Poolmon is showing only
paged pool. You should notice the pool tag “Leak” climbing to the top of the list.
(Poolmon shows changes to pool allocations by highlighting the lines that
change.)
Copyrighted material.
408
Microsoft Windows Internals, Fourth Edition
4. Now press the “Stop Leaking” button so that you don’t exhaust paged pool on
your system.
5. Using the technique described in the previous section, run Strings (from www.sysinternals.com) to look for driver binaries that contain the pool tag “Leak”:
Strings \windows\system32\drivers\*.sys
|
findstr Leak
This should display a match on the file Myfault.sys, thus confirming it as the driver
using the “Leak” pool tag.
Look-Aside Lists
Windows also provides a fast memory allocation mechanism called look-aside lists. The basic
difference between pools and look-aside lists is that while general pool allocations can vary in
size, a look-aside list contains only fixed-sized blocks. Although the general pools are more
flexible in terms of what they can supply, look-aside lists are faster because they don’t use any
spinlocks and also because the system doesn’t have to search for free memory that fits a varying size allocation.
Executive components and device drivers can create look-aside lists that match the size of frequently allocated data structures by using the ExInitializeNPagedLookasideList and ExInitializePagedLookasideList functions (documented in the DDK). To minimize the overhead of
multiprocessor synchronization, several executive subsystems (such as the I/O manager,
cache manager, and object manager) create separate look-aside lists for each processor for
their frequently accessed data structures. The executive also creates a general per-processor
paged and nonpaged look-aside list for small allocations (256 bytes or less).
If a look-aside list is empty (as it is when it’s first created), the system must allocate from paged
or nonpaged pool. But if it contains a freed structure, the allocation can be satisfied very
quickly. (The list grows as structures are returned to it.) The pool allocation routines automatically tune the number of freed buffers that look-aside lists store according to how often a
device driver or executive subsystem allocates from the list—the more frequent the allocations,
the more buffers are stored on a list. Look-aside lists are automatically reduced in size if they
aren’t being allocated from. (This check happens once per second when the balance set manager system thread wakes up and calls the function KiAdjustLookasideDepth.)
EXPERIMENT: Viewing the System Look-Aside Lists
You can display the contents and sizes of the various system look-aside lists with the
kernel debugger !lookaside command. The following excerpt is from the output of this
command:
Copyrighted material.
Chapter 7:
Memory Management
409
kd> !lookaside
Lookaside "nt!IopSmallIrpLookasideList" @ 804758a0 "Irps"
Type
=
0000 NonPagedPool
Current Depth =
3
Max Depth =
4
Size
=
148
Max Alloc =
592
AllocateMisses =
32
FreeMisses =
9
TotalAllocates =
52
TotalFrees =
32
Hit Rate
=
38% Hit Rate
=
71%
Lookaside "nt!IopLargeIrpLookasideList" @ 804756a0 "Irpl"
Type
=
0000 NonPagedPool
Current Depth =
4
Max Depth =
4
Size
=
436
Max Alloc =
1744
AllocateMisses =
2623
FreeMisses =
2443
TotalAllocates =
7039
TotalFrees =
6863
Hit Rate
=
62% Hit Rate
=
64%
"
Lookaside "nt!IopMdlLookasideList" @ 80475740 "Mdl "
Type
=
0000 NonPagedPool
Current Depth =
3
Max Depth =
4
Size
=
120
Max Alloc =
480
AllocateMisses =
7017
FreeMisses =
1824
TotalAllocates =
10901
TotalFrees =
5711
Hit Rate
=
35% Hit Rate
=
68%
§
Total NonPaged currently allocated for above lists = 4200
Total NonPaged potential for above lists
= 6144
Total Paged currently allocated for above lists
= 5136
Total Paged potential for above lists
= 12032
Driver Verifier
Driver Verifier is a mechanism that can be used to help find and isolate commonly found bugs
in device driver or other kernel-mode system code. Microsoft uses Driver Verifier to check its
own device drivers as well as all device drivers that vendors submit for Hardware Compatibility List (HCL) testing. Doing so ensures that the drivers on the HCL are compatible with Windows and free from common driver errors. (Although not described in this book, there is also
a corresponding Application Verifier tool that has resulted in quality improvements for usermode code in Windows.)
Driver Verifier consists of support in several system components: the memory manager, I/O
manager, and the HAL all have driver verification options that can be enabled. This section
describes the memory management–related verification options Driver Verifier provides. The
options related to device drivers are described in Chapter 9.
Driver Verifier Configuration and Initialization
To configure Driver Verifier and view statistics about its operation, run the Driver Verifier
Manager (\Windows\System32\Verifier.exe). (The Windows 2000 version is shown in
Figure 7-7.) It displays several tabbed pages. You use the Settings tab to specify which device
Copyrighted material.
410
Microsoft Windows Internals, Fourth Edition
drivers you want to verify (including an option to Verify All Drivers) and what types of verification you want performed.
Figure 7-7
The Driver Verifier Manager in Windows 2000
In Windows XP and Windows Server 2003, Verifier.exe has a wizard-style interface, as shown
in Figure 7-8.
Figure 7-8
The Driver Verifier Manager in Windows XP and Windows Server 2003
You can also enable and disable Driver Verifier as well as display current settings by using its
command line interface. Type “verifier /?” to see the switches.
The settings are stored in the registry under HKLM\SYSTEM\CurrentControlSet\Control\
Session Manager\Memory Management. The value VerifyDriverLevel contains a bitmask that
represents the verification types enabled. The VerifyDrivers value contains the names of the
drivers to validate. (These values won’t exist in the registry until you select drivers to verify in
the Driver Verifier Manager.) If you choose to verify all drivers, VerifyDrivers is set to an asterisk (*) character. Depending on the settings you have made, you might need to reboot the system for the selected verification to occur.
Copyrighted material.
Chapter 7:
Memory Management
411
Early in the boot process, the memory manager reads the Driver Verifier registry values to
determine which drivers to verify and which Driver Verifier options you enabled. (Note that if
you boot in safe mode, any Driver Verifier settings are ignored.) Subsequently, if you’ve
selected at least one driver for verification, the kernel checks the name of every device driver
it loads into memory against the list of drivers you’ve selected for verification. For every device
driver that appears in both places, the kernel invokes the MiApplyDriverVerifier function,
which replaces the driver’s references to a number of kernel functions with references to
Driver Verifier–equivalent versions of those functions. For example, ExAllocatePool is replaced
with a call to VerifierAllocatePool. The windowing system driver also makes similar changes to
use Driver Verifier–equivalent functions.
Now that we’ve reviewed how to set up Driver Verifier, let’s examine the four memory-related
verification options that can be applied to device drivers: Special Pool, Pool Tracking, Force
IRQL Checking, and Low Resources Simulation.
Special Pool The Special Pool option causes the pool allocation routines to bracket pool
allocations with an invalid page so that references before or after the allocation will result in a
kernel-mode access violation, thus crashing the system with the finger pointed at the buggy
driver. Special pool also causes some additional validation checks to be performed when a
driver allocates or frees memory.
When special pool is enabled, the pool allocation routines allocate a region of kernel memory
for Driver Verifier to use. Driver Verifier redirects memory allocation requests that drivers
under verification make to the special pool area rather than to the standard kernel-mode
memory pools. When a device driver allocates memory from special pool, Driver Verifier
rounds up the allocation to an even-page boundary. Because Driver Verifier brackets the allocated page with invalid pages, if a device driver attempts to read or write past the end of the
buffer, the driver will access an invalid page and the memory manager will raise a kernel-mode
access violation.
Figure 7-9 shows an example of the special pool buffer that Driver Verifier allocates to a device
driver when Driver Verifier checks for overrun errors.
Page 0
Invalid page
Page 1
Random signature
Driver buffer
Page 2
Invalid page
Figure 7-9
Copyrighted material.
Layout of special pool allocations
412
Microsoft Windows Internals, Fourth Edition
By default, Driver Verifier performs overrun detection. It does this by placing the buffer that
the device driver uses at the end of the allocated page and fills the beginning of the page
with a random pattern. Although the Driver Verifier Manager doesn’t let you specify underrun detection, you can set this type of detection manually by adding the DWORD registry
value HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\MemoryManagement\PoolTagOverruns and setting it to 0 (or by running the Gflags utility and selecting the Verify
Start option instead of the default option Verify End). When Windows enforces underrun detection, Driver Verifier allocates the driver’s buffer at the beginning of the page rather than at the end.
The overrun-detection configuration includes some measure of underrun detection as well.
When the driver frees its buffer to return the memory to Driver Verifier, Driver Verifier ensures
that the pattern preceding the buffer hasn’t changed. If the pattern is modified, the device
driver has underrun the buffer and written to memory outside the buffer.
Special pool allocations also check to ensure that the processor IRQL at the time of an allocation and deallocation is legal. This check catches an error that some device drivers make: allocating pageable memory from an IRQL at DPC/dispatch level or above.
You can also configure special pool manually by adding the DWORD registry value
HKLM\SYSTEMCurrentControlSet\Control\Session Manager\Memory Management\PoolTag, which represents the allocation tags the system uses for special pool. Thus,
even if Driver Verifier isn’t configured to verify a particular device driver, if the tag the driver
associates with the memory it allocates matches what is specified in the PoolTag registry
value, the pool allocation routines will allocate the memory from special pool. If you set the
value of PoolTag to 0x0000002a or the wildcard (*), all memory that drivers allocate is from
special pool, provided there’s enough virtual and physical memory. (The drivers will revert to
allocating from regular pool if there aren’t enough free pages—bounding exists, but each allocation uses two pages.)
Pool Tracking If pool tracking is enabled, the memory manager checks at driver unload time
whether the driver freed all the memory allocations it made. If it didn’t, it crashes the system,
indicating the buggy driver. Driver Verifier also shows general pool statistics on the Driver Verifier Manager’s Pool Tracking tab. You can also use the !verifier kernel debugger command. This
command shows more information than Driver Verifier and is useful to driver writers.
Force IRQL Checking One of the most common device driver bugs occurs when a driver
accesses pageable data or code when the processor on which the device driver is executing is
at an elevated IRQL. As explained in Chapter 3, the memory manager can’t service a page fault
when the IRQL is DPC/dispatch level or above. The system often doesn’t detect instances of
a device driver accessing pageable data when the processor is executing at a high IRQL level
because the pageable data being accessed happens to be physically resident at the time. At
other times, however, the data might be paged out, which results in a system crash with the
stop code IRQL_NOT_LESS_OR_EQUAL (that is, the IRQL wasn’t less than or equal to the
level required for the operation attempted—in this case, accessing pageable memory).
Copyrighted material.
Chapter 7:
Memory Management
413
Although testing device drivers for this kind of bug is usually difficult, Driver Verifier makes it
easy. If you select the Force IRQL Checking option, Driver Verifier forces all kernel-mode pageable code and data out of the system working set whenever a device driver under verification
raises the IRQL. The internal function that does this is MmTrimAllSystemPagableMemory. With
this setting enabled, whenever a device driver under verification accesses pageable memory
when the IRQL is elevated, the system instantly detects the violation and the resulting system
crash identifies the faulty driver.
Low Resources Simulation Enabling Low Resources Simulation causes Driver Verifier to
randomly fail memory allocations that verified device drivers perform. In the past, developers
wrote many device drivers under the assumption that kernel memory would always be available and that if memory ran out, the device driver didn’t have to worry about it because the
system would crash anyway. However, because low-memory conditions can occur temporarily, it’s important that device drivers properly handle allocation failures that indicate kernel
memory is exhausted.
Beginning 7 minutes after the system boots—which is enough time to get past the critical initialization period in which a low-memory condition might prevent a device driver from loading—Driver Verifier starts randomly failing allocation calls for device drivers it is verifying. If a
driver doesn’t correctly handle allocation failures, this will likely show up as a system crash.
Driver Verifier is a valuable addition to the arsenal of verification and debugging tools available to device driver writers. Many device drivers that first ran with Driver Verifier had bugs
that Driver Verifier was able to expose. Thus, Driver Verifier has resulted in an overall improvement in the quality of all kernel-mode code running in Windows.
Virtual Address Space Layouts
This section describes the components in the user and system address space, followed by the
specific layouts on 32-bit and 64-bit systems. This information helps you to understand the
limits on process and system virtual memory on both platforms.
Three main types of data are mapped into the virtual address space in Windows: per-process
private code and data, sessionwide code and data, and systemwide code and data.
As explained in Chapter 1, each process has a private address space that cannot be accessed
by other processes (unless they have permission to open the process for read or write access).
Threads within the process can never access virtual addresses outside this private address
space, unless they map to shared memory sections and/or use the cross-process memory
functions that allow access to another process’s address space. The information that describes
the process virtual address space, called page tables, is described in the section on address
translation. Page tables are marked as kernel-mode-only accessible pages so that user threads
in a process cannot modify their own address space layout.
Copyrighted material.
414
Microsoft Windows Internals, Fourth Edition
On systems with multiple sessions (Windows 2000 Server with Terminal Services installed,
Windows XP, and Windows Server 2003), session space contains information global to each
session. (For a description of sessions, see Chapter 2.) A session consists of the processes and
other system objects (such as the window station, desktops, and windows) that represent a
single user’s workstation logon session. Each session has a session-specific paged pool area
used by the kernel-mode portion of the Windows subsystem (Win32k.sys) to allocate session-private GUI data structures. In addition, each session has its own copy of the Windows
subsystem process (Csrss.exe) and logon process (Winlogon.exe). The session manager process (Smss.exe) is responsible for creating new sessions, which includes loading a session-private copy of Win32k.sys, creating the session-private object manager namespace, and creating
the session-specific instances of the Csrss and Winlogon processes.
To virtualize sessions, all sessionwide data structures are mapped into a region of system
space called session space. When a process is created, this range of addresses is mapped to the
pages appropriate to the session that the process belongs to. The size of the space for mapped
views in session space can be configured by modifying registry values under HKLM\System\CurrentControlSet\Control\Session Manager\Memory Management. (For 32-bit systems, these registry values are ignored if the system is booted /3GB.)
Finally, system space contains global operating system code and data structures visible to each
process. System space consists of the following components:
■
System code
■
System mapped views Used to map Win32k.sys, the loadable kernel-mode part of the
Windows subsystem, as well as kernel-mode graphics drivers it uses. (See Chapter 2 for
more information on Win32k.sys.)
■
Hyperspace A special region used to map the process working set list and to tempo-
Contains the operating system image, HAL, and device drivers used to
boot the system.
rarily map other physical pages for operations such as zeroing a page on the free list
(when the zero list is empty and a zero page is needed), invalidating page table entries in
other page tables (such as when a page is removed from the standby list), and process
creation to set up a new process’s address space.
■
System working set list The working set list data structures that describe the system
working set.*
■
System cache
■
Paged pool
Virtual address space used to map files open in the system cache. (See
Chapter 11 for detailed information about the cache manager.)
Copyrighted material.
Pageable system memory heap.
Chapter 7:
Memory Management
415
■
System page table entries (PTEs) Pool of system PTEs used to map system pages such
as I/O space, kernel stacks, and memory descriptor lists. You can see how many system
PTEs are available by examining the value of the Memory: Free System Page Table
Entries counter in the Performance tool.
■
Nonpaged pool
■
Nonpageable system memory heap, usually existing in two parts—one
in the lower end of system space and one in the upper end.
Crash dump information
Reserved to record information about the state of a system
crash.
■
HAL usage
System memory reserved for HAL-specific structures.
Note
*Internally, the system working set is called the system cache working set. This term is
misleading, however, because it includes not only the system cache but also the paged pool,
pageable system code and data, and pageable driver code and data.
Now that we’ve described the basic components of the virtual address space in Windows, let’s
examine the specific layout on the x86, IA-64, and x64 platforms.
x86 User Address Space Layouts
By default, each user process on the 32-bit version of Windows can have up to a 2-GB private
address space; the operating system takes the remaining 2 GB. Windows 2000 Advanced
Server, Windows 2000 Datacenter Server, Windows XP Service Pack 2 and later, and Windows Server 2003 (all versions) support a boot-time option (the /3GB switch in Boot.ini) that
allows 3-GB user address spaces. Windows XP and Windows Server 2003 support an additional switch (/USERVA), which allows specifying a user address space size between 2 GB and
3 GB. (The value is in number of megabytes.) These two possible address space layouts are
shown in Figure 7-10.
The ability for a 32-bit process to grow beyond 2 GB was added as a short-term solution to
accommodate the need for applications such as database servers to keep more data in memory than could be done with a 2-GB address space. The AWE functions described earlier in
this chapter provide a much better solution to the need for accessing more data than can fit in
the limited 2-GB (or 3-GB) process address space.
Copyrighted material.
416
Microsoft Windows Internals, Fourth Edition
00000000
Application code
Global variables
Per-thread stacks
DLL code
3-GB user space
7FFFFFFF
80000000
Kernel and executive
HAL
Boot drivers
C0000000
BFFFFFFF
C0000000
Process page tables
Hyperspace
C0800000
1-GB system space
System cache
Paged pool
Nonpaged pool
FFFFFFFF
Figure 7-10
FFFFFFFF
x86 virtual address space layouts
For a process to grow beyond 2 GB of address space, the image file must have the
IMAGE_FILE_LARGE_ADDRESS_AWARE flag set in the image header. Otherwise, Windows
reserves the additional address space for that process so that the application won’t see virtual
addresses greater than 0x7FFFFFFF. (This is to avoid breaking applications that might break
with such addresses.) You set this flag by specifying the linker flag /LARGEADDRESSAWARE
when building the executable. This flag has no effect when running the application on a system with a ">