Mac OS X and iOS Internals To the Apple's Core

An in-depth look into Mac OS X and iOS kernels

Powering Macs, iPhones, iPads and more, OS X and iOS are becoming ubiquitous. When it comes to documentation, however, much of them are shrouded in mystery. Cocoa and Carbon, the application frameworks, are neatly described, but system programmers find the rest lacking. This indispensable guide illuminates the darkest corners of those systems, starting with an architectural overview, then drilling all the way to the core.
* Provides you with a top down view of OS X and iOS
* Walks you through the phases of system startup--both Mac (EFi) and mobile (iBoot)
* Explains how processes, threads, virtual memory, and filesystems are maintained
* Covers the security architecture
* Reviews the internal Apis used by the system--BSD and Mach
* Dissects the kernel, XNU, into its sub components: Mach, the BSD Layer, and I/o kit, and explains each in detail
* Explains the inner workings of device drivers

From architecture to implementation, this book is essential reading if you want to get serious about the internal workings of Mac OS X and iOS.

Peeling Apple's Operating Systems

System–level developers, kernel hackers, and intrigued Apple–lytes, take heed: This book lets you explore the nooks and crannies of Mac OS X and iOS, delving into the architecture of both systems, and picking up where the frameworks (and documentation) leave off. It offers clear, detailed explanation of the inner workings of Apple's systems, including proprietary APIs, most of which are documented for the first time.

As you traverse the architecture, moving from user to kernel mode, each layer and component is unraveled with annotated code samples and hands–on experiments, comparing and contrasting its implementation in both OSes. Topics include:

• The boot process: Mac's EFI, iOS's iBoot, and kernel startup
• Processes, threads, and virtual memory management
• Debugging and profiling, using DTrace, ptrace, and hidden system calls
• The system level APIs: POSIX calls, Mach traps, and OS X/iOS proprietary
• File system architecture, including VFS and HFS+
• Network architecture: from sockets to interfaces, and network drivers
• Kernel extensions, drivers, and a discussion of I/O Kit architecture
• Security features, Mandatory Access Control, and iOS jail

The companion web site (http://www.newosxbook.com) includes sample programs, freely downloadable tools, updated references, and bonus additions.

Wrox Professional guides are planned and written by working programmers to meet the real–world needs of programmers, developers, and IT professionals. Focused and relevant, they address the issues technology professionals face every day. They provide examples, practical solutions, and expert education in new technologies, all designed to help programmers do a better job.

wrox.com

Programmer Forums

Join our Programmer to Programmer forums to ask and answer programming questions about this book, join discussions on the hottest topics in the industry, and connect with fellow programmers from around the world.

Code Downloads

Take advantage of free code samples from this book, as well as code samples from hundreds of other books, all ready to use.

Read More

Find articles, ebooks, sample chapters and tables of contents for hundreds of books, and more reference resources on programming topics that matter to you.

INTRODUCTION xxv

PART I: FOR POWER USERS

CHAPTER 1: DARWINISM: THE EVOLUTION OF OS X 3

The Pre–Darwin Era: Mac OS Classic 3

The Prodigal Son: NeXTSTEP 4

Enter: OS X 4

OS X Versions, to Date 5

10.0 Cheetah and the First Foray 5

10.1 Puma a Stronger Feline, but . . . 6

10.2 Jaguar Getting Better 6

10.3 Panther and Safari 6

10.4 Tiger and Intel Transition 6

10.5 Leopard and UNIX 7

10.6 Snow Leopard 7

10.7 Lion 8

10.8 Mountain Lion 9

iOS OS X Goes Mobile 10

1.x Heavenly and the First iPhone 11

2.x App Store, 3G and Corporate Features 11

3.x Farewell, 1st gen, Hello iPad 11

4.x iPhone 4, Apple TV, and the iPad 2 11

5.x To the iPhone 4S and Beyond 12

iOS vs. OS X 12

The Future of OS X 15

Summary 16

References 16

CHAPTER 2: E PLURIBUS UNUM: ARCHITECTURE OF OS X AND IOS 17

OS X Architectural Overview 17

The User Experience Layer 19

Aqua 19

Quicklook 20

Spotlight 21

Darwin The UNIX Core 22

The Shell 22

The File System 23

UNIX System Directories 24

OS X Specifi c Directories 25

iOS File System Idiosyncrasies 25

Interlude: Bundles 26

Applications and Apps 26

Info.plist 28

Resources 30

NIB Files 30

Internationalization with .lproj Files 31

Icons (.icns) 31

CodeResources 31

Frameworks 34

Framework Bundle Format 34

List of OS X and iOS Public Frameworks 37

Libraries 44

Other Application Types 46

System Calls 48

POSIX 48

Mach System Calls 48

A High–Level View of XNU 51

Mach 51

The BSD Layer 51

libkern 52

I/O Kit 52

Summary 52

References 53

CHAPTER 3: ON THE SHOULDERS OF GIANTS: OS X

AND IOS TECHNOLOGIES 55

BSD Heirlooms 55

sysctl 56

kqueues 57

Auditing (OS X) 59

Mandatory Access Control 62

OS X– and iOS–Specifi c Technologies 65

User and Group Management (OS X) 65

System Confi guration 67

Logging 69

Apple Events and AppleScript 72

FSEvents 74

Notifi cations 78

Additional APIs of interest 79

OS X and iOS Security Mechanisms 79

Code Signing 80

Compartmentalization (Sandboxing) 81

Entitlements: Making the Sandbox Tighter Still 83

Enforcing the Sandbox 89

Summary 90

References 90

CHAPTER 4: PARTS OF THE PROCESS: MACH–O,

PROCESS, AND THREAD INTERNALS 91

A Nomenclature Refresher 91

Processes and Threads 91

The Process Lifecycle 92

UNIX Signals 95

Executables 98

Universal Binaries 99

Mach–O Binaries 102

Load Commands 106

Dynamic Libraries 111

Launch–Time Loading of Libraries 111

Runtime Loading of Libraries 122

dyld Features 124

Process Address Space 130

The Process Entry Point 130

Address Space Layout Randomization 131

32–Bit (Intel) 132

64–Bit 132

32–Bit (iOS) 133

Experiment: Using vmmap(1) to Peek Inside a Process s

Address Space 135

Process Memory Allocation (User Mode) 138

Heap Allocations 139

Virtual Memory The sysadmin Perspective 140

Threads 143

Unraveling Threads 143

References 146

CHAPTER 5: NON SEQUITUR:

PROCESS TRACING AND DEBUGGING 147

DTrace 147

The D Language 147

dtruss 150

How DTrace Works 152

Other Profi ling mechanisms 154

The Decline and Fall of CHUD 154

AppleProfi leFamily: The Heir Apparent 155

Process Information 156

sysctl 156

proc—info 156

Process and System Snapshots 159

system—profi ler(8) 159

sysdiagnose(1) 159

allmemory(1) 160

stackshot(1) 160

The stack—snapshot System Call 162

kdebug 165

kdebug–based Utilities 165

kdebug codes 166

Writing kdebug messages 168

Reading kdebug messages 169

Application Crashes 170

Application Hangs and Sampling 173

Memory Corruption Bugs 174

Memory Leaks 176

heap(1) 177

leaks(1) 177

malloc—history(1) 178

Standard UNIX Tools 178

Process listing with ps(1) 179

System–Wide View with top(1) 179

File Diagnostics with lsof(1) and fuser(1) 180

Using GDB 181

GDB Darwin Extensions 181

GDB on iOS 182

LLDB 182

Summary 182

References and Further Reading 182

CHAPTER 6: ALONE IN THE DARK:

THE BOOT PROCESS: EFI AND IBOOT 183

Traditional Forms of Boot 183

EFI Demystifi ed 185

Basic Concepts of EFI 186

The EFI Services 188

NVRAM Variables 192

OS X and boot.efi 194

Flow of boot.efi 195

Booting the Kernel 201

Kernel Callbacks into EFI 203

Boot.efi Changes in Lion 204

Boot Camp 204

Count Your Blessings 204

Experiment: Running EFI Programs on a Mac 206

iOS and iBoot 210

Precursor: The Boot ROM 210

Normal Boot 211

Recovery Mode 212

Device Firmware Update (DFU) Mode 213

Downgrade and Replay Attacks 213

Installation Images 214

OS X Installation Process 214

iOS File System Images (.ipsw) 219

Summary 225

References and Further Reading 225

CHAPTER 7: THE ALPHA AND THE OMEGA LAUNCHD 227

launchd 227

Starting launchd 227

System–Wide Versus Per–User launchd 228

Daemons and Agents 229

The Many Faces of launchd 229

Lists of LaunchDaemons 241

GUI Shells 246

Finder (OS X) 247

SpringBoard (iOS) 248

XPC (Lion and iOS) 253

Summary 257

References and Further Reading 258

PART II: THE KERNEL

CHAPTER 8: SOME ASSEMBLY REQUIRED:

KERNEL ARCHITECTURES 261

Kernel Basics 261

Kernel Architectures 262

User Mode versus Kernel Mode 266

Intel Architecture Rings 266

ARM Architecture: CPSR 267

Kernel/User Transition Mechanisms 268

Trap Handlers on Intel 269

Voluntary kernel transition 278

System Call Processing 283

POSIX/BSD System calls 284

Mach Traps 287

Machine Dependent Calls 292

Diagnostic calls 292

XNU and hardware abstraction 295

Summary 297

References 297

CHAPTER 9: FROM THE CRADLE TO THE GRAVE

KERNEL BOOT AND PANICS 299

The XNU Sources 299

Getting the Sources 299

Making XNU 300

One Kernel, Multiple Architectures 302

The XNU Source Tree 305

Booting XNU 308

The Bird s Eye View 309

OS X: vstart 310

iOS: start 310

[i386|arm]—init 311

i386—init—slave() 313

machine—startup 314

kernel—bootstrap 314

kernel—bootstrap—thread 318

bsd—init 320

bsdinit—task 325

Sleeping and Waking Up 328

Boot Arguments 329

Kernel Debugging 332

Don t Panic 333

Implementation of Panic 334

Panic Reports 336

Summary 340

References 341

CHAPTER 10: THE MEDIUM IS THE MESSAGE: MACH PRIMITIVES 343

Introducing: Mach 344

The Mach Design Philosophy 344

Mach Design Goals 345

Mach Messages 346

Simple Messages 346

Complex messages 347

Sending Messages 348

Ports 349

The Mach Interface Generator (MIG) 351

IPC, in Depth 357

Behind the Scenes of Message Passing 359

Synchronization Primitives 360

Lock Group Objects 361

Mutex Object 362

Read–Write Lock Object 363

Spinlock Object 364

Semaphore Object 364

Lock Set Object 366