README.md

   1 What is XNU?
   2 ===========
   3
   4 XNU kernel is part of the Darwin operating system for use in macOS and iOS operating systems. XNU is an acronym for X is Not Unix.
   5 XNU is a hybrid kernel combining the Mach kernel developed at Carnegie Mellon University with components from FreeBSD and a C++ API for writing drivers called IOKit.
   6 XNU runs on x86_64 for both single processor and multi-processor configurations.
   7
   8 XNU Source Tree
   9 ===============
  10
  11   * `config` - configurations for exported apis for supported architecture and platform
  12   * `SETUP` - Basic set of tools used for configuring the kernel, versioning and kextsymbol management.
  13   * `EXTERNAL_HEADERS` - Headers sourced from other projects to avoid dependency cycles when building. These headers should be regularly synced when source is updated.
  14   * `libkern` - C++ IOKit library code for handling of drivers and kexts.
  15   * `libsa` -  kernel bootstrap code for startup
  16   * `libsyscall` - syscall library interface for userspace programs
  17   * `libkdd` - source for user library for parsing kernel data like kernel chunked data.
  18   * `makedefs` - top level rules and defines for kernel build.
  19   * `osfmk` - Mach kernel based subsystems
  20   * `pexpert` - Platform specific code like interrupt handling, atomics etc.
  21   * `security` - Mandatory Access Check policy interfaces and related implementation.
  22   * `bsd` - BSD subsystems code
  23   * `tools` - A set of utilities for testing, debugging and profiling kernel.
  24
  25 How to build XNU
  26 ================
  27
  28 Building `DEVELOPMENT` kernel
  29 -----------------------------
  30
  31 The xnu make system can build kernel based on `KERNEL_CONFIGS` & `ARCH_CONFIGS` variables as arguments.
  32 Here is the syntax:
  33
  34     make SDKROOT=<sdkroot> ARCH_CONFIGS=<arch> KERNEL_CONFIGS=<variant>
  35
  36 Where:
  37
  38   * \<sdkroot>: path to macOS SDK on disk. (defaults to `/`)
  39   * \<variant>: can be `debug`, `development`, `release`, `profile` and configures compilation flags and asserts throughout kernel code.
  40   * \<arch>   : can be valid arch to build for. (E.g. `X86_64`)
  41
  42 To build a kernel for the same architecture as running OS, just type
  43
  44     $ make
  45     $ make SDKROOT=macosx.internal
  46
  47 Additionally, there is support for configuring architectures through `ARCH_CONFIGS` and kernel configurations with `KERNEL_CONFIGS`.
  48
  49     $ make SDKROOT=macosx.internal ARCH_CONFIGS=X86_64 KERNEL_CONFIGS=DEVELOPMENT
  50     $ make SDKROOT=macosx.internal ARCH_CONFIGS=X86_64 KERNEL_CONFIGS="RELEASE DEVELOPMENT DEBUG"
  51
  52
  53 Note:
  54   * By default, architecture is set to the build machine architecture, and the default kernel
  55     config is set to build for DEVELOPMENT.
  56
  57
  58 This will also create a bootable image, kernel.[config],  and a kernel binary
  59 with symbols, kernel.[config].unstripped.
  60
  61 To intall the kernel into a DSTROOT, use the `install_kernels` target:
  62
  63     $ make install_kernels DSTROOT=/tmp/xnu-dst
  64
  65 Hint:
  66 For a more satisfying kernel debugging experience, with access to all
  67 local variables and arguments, but without all the extra check of the
  68 DEBUG kernel, add something like:
  69         CFLAGS_DEVELOPMENTARM64="-O0 -g -DKERNEL_STACK_MULTIPLIER=2"
  70         CXXFLAGS_DEVELOPMENTARM64="-O0 -g -DKERNEL_STACK_MULTIPLIER=2"
  71 to your make command.
  72 Replace DEVELOPMENT and ARM64 with the appropriate build and platform.
  73
  74
  75   * To build with RELEASE kernel configuration
  76
  77         make KERNEL_CONFIGS=RELEASE SDKROOT=/path/to/SDK
  78
  79
  80 Building FAT kernel binary
  81 --------------------------
  82
  83 Define architectures in your environment or when running a make command.
  84
  85     $ make ARCH_CONFIGS="X86_64" exporthdrs all
  86
  87 Other makefile options
  88 ----------------------
  89
  90  * $ make MAKEJOBS=-j8    # this will use 8 processes during the build. The default is 2x the number of active CPUS.
  91  * $ make -j8             # the standard command-line option is also accepted
  92  * $ make -w              # trace recursive make invocations. Useful in combination with VERBOSE=YES
  93  * $ make BUILD_LTO=0      # build without LLVM Link Time Optimization
  94  * $ make REMOTEBUILD=user@remotehost # perform build on remote host
  95  * $ make BUILD_JSON_COMPILATION_DATABASE=1 # Build Clang JSON Compilation Database
  96
  97 The XNU build system can optionally output color-formatted build output. To enable this, you can either
  98 set the `XNU_LOGCOLORS` environment variable to `y`, or you can pass `LOGCOLORS=y` to the make command.
  99
 100
 101 Debug information formats
 102 =========================
 103
 104 By default, a DWARF debug information repository is created during the install phase; this is a "bundle" named kernel.development.\<variant>.dSYM
 105 To select the older STABS debug information format (where debug information is embedded in the kernel.development.unstripped image), set the BUILD_STABS environment variable.
 106
 107     $ export BUILD_STABS=1
 108     $ make
 109
 110
 111 Building KernelCaches
 112 =====================
 113
 114 To test the xnu kernel, you need to build a kernelcache that links the kexts and
 115 kernel together into a single bootable image.
 116 To build a kernelcache you can use the following mechanisms:
 117
 118   * Using automatic kernelcache generation with `kextd`.
 119     The kextd daemon keeps watching for changing in `/System/Library/Extensions` directory.
 120     So you can setup new kernel as
 121
 122         $ cp BUILD/obj/DEVELOPMENT/X86_64/kernel.development /System/Library/Kernels/
 123         $ touch /System/Library/Extensions
 124         $ ps -e | grep kextd
 125
 126   * Manually invoking `kextcache` to build new kernelcache.
 127
 128         $ kextcache -q -z -a x86_64 -l -n -c /var/tmp/kernelcache.test -K /var/tmp/kernel.test /System/Library/Extensions
 129
 130
 131
 132 Running KernelCache on Target machine
 133 =====================================
 134
 135 The development kernel and iBoot supports configuring boot arguments so that we can safely boot into test kernel and, if things go wrong, safely fall back to previously used kernelcache.
 136 Following are the steps to get such a setup:
 137
 138   1. Create kernel cache using the kextcache command as `/kernelcache.test`
 139   2. Copy exiting boot configurations to alternate file
 140
 141          $ cp /Library/Preferences/SystemConfiguration/com.apple.Boot.plist /next_boot.plist
 142
 143   3. Update the kernelcache and boot-args for your setup
 144
 145          $ plutil -insert "Kernel Cache" -string "kernelcache.test" /next_boot.plist
 146          $ plutil -replace "Kernel Flags" -string "debug=0x144 -v kernelsuffix=test " /next_boot.plist
 147
 148   4. Copy the new config to `/Library/Preferences/SystemConfiguration/`
 149
 150          $ cp /next_boot.plist /Library/Preferences/SystemConfiguration/boot.plist
 151
 152   5. Bless the volume with new configs.
 153
 154          $ sudo -n bless  --mount / --setBoot --nextonly --options "config=boot"
 155
 156      The `--nextonly` flag specifies that use the `boot.plist` configs only for one boot.
 157      So if the kernel panic's you can easily power reboot and recover back to original kernel.
 158
 159
 160
 161
 162 Creating tags and cscope
 163 ========================
 164
 165 Set up your build environment and from the top directory, run:
 166
 167     $ make tags     # this will build ctags and etags on a case-sensitive volume, only ctags on case-insensitive
 168     $ make TAGS     # this will build etags
 169     $ make cscope   # this will build cscope database
 170
 171
 172 How to install a new header file from XNU
 173 =========================================
 174
 175 To install IOKit headers, see additional comments in [iokit/IOKit/Makefile]().
 176
 177 XNU installs header files at the following locations -
 178
 179     a. $(DSTROOT)/System/Library/Frameworks/Kernel.framework/Headers
 180     b. $(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders
 181     c. $(DSTROOT)/usr/include/
 182     d. $(DSTROOT)/System/DriverKit/usr/include/
 183     e. $(DSTROOT)/System/Library/Frameworks/System.framework/PrivateHeaders
 184
 185 `Kernel.framework` is used by kernel extensions.\
 186 The `System.framework` and `/usr/include` are used by user level applications. \
 187 `/System/DriverKit/usr/include` is used by userspace drivers. \
 188 The header files in framework's `PrivateHeaders` are only available for ** Apple Internal Development **.
 189
 190 The directory containing the header file should have a Makefile that
 191 creates the list of files that should be installed at different locations.
 192 If you are adding the first header file in a directory, you will need to
 193 create Makefile similar to `xnu/bsd/sys/Makefile`.
 194
 195 Add your header file to the correct file list depending on where you want
 196 to install it. The default locations where the header files are installed
 197 from each file list are -
 198
 199     a. `DATAFILES` : To make header file available in user level -
 200        `$(DSTROOT)/usr/include`
 201
 202     b. `DRIVERKIT_DATAFILES` : To make header file available to DriverKit userspace drivers -
 203        `$(DSTROOT)/System/DriverKit/usr/include`
 204
 205     c. `PRIVATE_DATAFILES` : To make header file available to Apple internal in
 206        user level -
 207        `$(DSTROOT)/System/Library/Frameworks/System.framework/PrivateHeaders`
 208
 209     d. `KERNELFILES` : To make header file available in kernel level -
 210        `$(DSTROOT)/System/Library/Frameworks/Kernel.framework/Headers`
 211        `$(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders`
 212
 213     e. `PRIVATE_KERNELFILES` : To make header file available to Apple internal
 214        for kernel extensions -
 215        `$(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders`
 216
 217 The Makefile combines the file lists mentioned above into different
 218 install lists which are used by build system to install the header files. There
 219 are two types of install lists: machine-dependent and machine-independent.
 220 These lists are indicated by the presence of `MD` and `MI` in the build
 221 setting, respectively. If your header is architecture-specific, then you should
 222 use a machine-dependent install list (e.g. `INSTALL_MD_LIST`). If your header
 223 should be installed for all architectures, then you should use a
 224 machine-independent install list (e.g. `INSTALL_MI_LIST`).
 225
 226 If the install list that you are interested does not exist, create it
 227 by adding the appropriate file lists.  The default install lists, its
 228 member file lists and their default location are described below -
 229
 230     a. `INSTALL_MI_LIST` : Installs header file to a location that is available to everyone in user level.
 231         Locations -
 232            $(DSTROOT)/usr/include
 233        Definition -
 234            INSTALL_MI_LIST = ${DATAFILES}
 235
 236     b. `INSTALL_DRIVERKIT_MI_LIST` : Installs header file to a location that is
 237         available to DriverKit userspace drivers.
 238         Locations -
 239            $(DSTROOT)/System/DriverKit/usr/include
 240        Definition -
 241            INSTALL_DRIVERKIT_MI_LIST = ${DRIVERKIT_DATAFILES}
 242
 243     c.  `INSTALL_MI_LCL_LIST` : Installs header file to a location that is available
 244        for Apple internal in user level.
 245        Locations -
 246            $(DSTROOT)/System/Library/Frameworks/System.framework/PrivateHeaders
 247        Definition -
 248            INSTALL_MI_LCL_LIST = ${PRIVATE_DATAFILES}
 249
 250     d. `INSTALL_KF_MI_LIST` : Installs header file to location that is available
 251        to everyone for kernel extensions.
 252        Locations -
 253             $(DSTROOT)/System/Library/Frameworks/Kernel.framework/Headers
 254        Definition -
 255             INSTALL_KF_MI_LIST = ${KERNELFILES}
 256
 257     e. `INSTALL_KF_MI_LCL_LIST` : Installs header file to location that is
 258        available for Apple internal for kernel extensions.
 259        Locations -
 260             $(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders
 261        Definition -
 262             INSTALL_KF_MI_LCL_LIST = ${KERNELFILES} ${PRIVATE_KERNELFILES}
 263
 264     f. `EXPORT_MI_LIST` : Exports header file to all of xnu (bsd/, osfmk/, etc.)
 265        for compilation only. Does not install anything into the SDK.
 266        Definition -
 267             EXPORT_MI_LIST = ${KERNELFILES} ${PRIVATE_KERNELFILES}
 268
 269     g. `INSTALL_MODULEMAP_INCDIR_MI_LIST` : Installs module map file to a
 270        location that is available to everyone in user level, installing at the
 271        root of INCDIR.
 272        Locations -
 273            $(DSTROOT)/usr/include
 274        Definition -
 275            INSTALL_MODULEMAP_INCDIR_MI_LIST = ${MODULEMAP_INCDIR_FILES}
 276
 277 If you want to install the header file in a sub-directory of the paths
 278 described in (1), specify the directory name using two variables
 279 `INSTALL_MI_DIR` and `EXPORT_MI_DIR` as follows -
 280
 281     INSTALL_MI_DIR = dirname
 282     EXPORT_MI_DIR = dirname
 283
 284 A single header file can exist at different locations using the steps
 285 mentioned above.  However it might not be desirable to make all the code
 286 in the header file available at all the locations.  For example, you
 287 want to export a function only to kernel level but not user level.
 288
 289  You can use C language's pre-processor directive (#ifdef, #endif, #ifndef)
 290  to control the text generated before a header file is installed.  The kernel
 291  only includes the code if the conditional macro is TRUE and strips out
 292  code for FALSE conditions from the header file.
 293
 294  Some pre-defined macros and their descriptions are -
 295
 296     a. `PRIVATE` : If defined, enclosed definitions are considered System
 297         Private Interfaces. These are visible within xnu and
 298         exposed in user/kernel headers installed within the AppleInternal
 299         "PrivateHeaders" sections of the System and Kernel frameworks.
 300     b. `KERNEL_PRIVATE` : If defined, enclosed code is available to all of xnu
 301         kernel and Apple internal kernel extensions and omitted from user
 302         headers.
 303     c. `BSD_KERNEL_PRIVATE` : If defined, enclosed code is visible exclusively
 304         within the xnu/bsd module.
 305     d. `MACH_KERNEL_PRIVATE`: If defined, enclosed code is visible exclusively
 306         within the xnu/osfmk module.
 307     e. `XNU_KERNEL_PRIVATE`: If defined, enclosed code is visible exclusively
 308         within xnu.
 309     f. `KERNEL` :  If defined, enclosed code is available within xnu and kernel
 310        extensions and is not visible in user level header files.  Only the
 311        header files installed in following paths will have the code -
 312
 313             $(DSTROOT)/System/Library/Frameworks/Kernel.framework/Headers
 314             $(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders
 315     g. `DRIVERKIT`: If defined, enclosed code is visible exclusively in the
 316     DriverKit SDK headers used by userspace drivers.
 317
 318 Conditional compilation
 319 =======================
 320
 321 `xnu` offers the following mechanisms for conditionally compiling code:
 322
 323     a. *CPU Characteristics* If the code you are guarding has specific
 324     characterstics that will vary only based on the CPU architecture being
 325     targeted, use this option. Prefer checking for features of the
 326     architecture (e.g. `__LP64__`, `__LITTLE_ENDIAN__`, etc.).
 327     b. *New Features* If the code you are guarding, when taken together,
 328     implements a feature, you should define a new feature in `config/MASTER`
 329     and use the resulting `CONFIG` preprocessor token (e.g. for a feature
 330     named `config_virtual_memory`, check for `#if CONFIG_VIRTUAL_MEMORY`).
 331     This practice ensures that existing features may be brought to other
 332     platforms by simply changing a feature switch.
 333     c. *Existing Features* You can use existing features if your code is
 334     strongly tied to them (e.g. use `SECURE_KERNEL` if your code implements
 335     new functionality that is exclusively relevant to the trusted kernel and
 336     updates the definition/understanding of what being a trusted kernel means).
 337
 338 It is recommended that you avoid compiling based on the target platform. `xnu`
 339 does not define the platform macros from `TargetConditionals.h`
 340 (`TARGET_OS_OSX`, `TARGET_OS_IOS`, etc.).
 341
 342
 343 There is a deprecated `TARGET_OS_EMBEDDED` macro, but this should be avoided
 344 as it is in general too broad a definition for most functionality.
 345 Please refer to TargetConditionals.h for a full picture.
 346
 347 How to add a new syscall
 348 ========================
 349
 350
 351
 352
 353 Testing the kernel
 354 ==================
 355
 356 XNU kernel has multiple mechanisms for testing.
 357
 358   * Assertions - The DEVELOPMENT and DEBUG kernel configs are compiled with assertions enabled. This allows developers to easily
 359     test invariants and conditions.
 360
 361   * XNU Power On Self Tests (`XNUPOST`): The XNUPOST config allows for building the kernel with basic set of test functions
 362     that are run before first user space process is launched. Since XNU is hybrid between MACH and BSD, we have two locations where
 363     tests can be added.
 364
 365         xnu/osfmk/tests/     # For testing mach based kernel structures and apis.
 366         bsd/tests/           # For testing BSD interfaces.
 367     Please follow the documentation at [osfmk/tests/README.md](osfmk/tests/README.md)
 368
 369   * User level tests: The `tools/tests/` directory holds all the tests that verify syscalls and other features of the xnu kernel.
 370     The make target `xnu_tests` can be used to build all the tests supported.
 371
 372         $ make RC_ProjectName=xnu_tests SDKROOT=/path/to/SDK
 373
 374     These tests are individual programs that can be run from Terminal and report tests status by means of std posix exit codes (0 -> success) and/or stdout.
 375     Please read detailed documentation in [tools/tests/unit_tests/README.md](tools/tests/unit_tests/README.md)
 376
 377
 378 Kernel data descriptors
 379 =======================
 380
 381 XNU uses different data formats for passing data in its api. The most standard way is using syscall arguments. But for complex data
 382 it often relies of sending memory saved by C structs. This packaged data transport mechanism is fragile and leads to broken interfaces
 383 between user space programs and kernel apis. `libkdd` directory holds user space library that can parse custom data provided by the
 384 same version of kernel. The kernel chunked data format is described in detail at [libkdd/README.md](libkdd/README.md).
 385
 386
 387 Debugging the kernel
 388 ====================
 389
 390 The xnu kernel supports debugging with a remote kernel debugging protocol (kdp). Please refer documentation at [technical note] [TN2063]
 391 By default the kernel is setup to reboot on a panic. To debug a live kernel, the kdp server is setup to listen for UDP connections
 392 over ethernet. For machines without ethernet port, this behavior can be altered with use of kernel boot-args. Following are some
 393 common options.
 394
 395   * `debug=0x144` - setups debug variables to start kdp debugserver on panic
 396   * `-v` - print kernel logs on screen. By default XNU only shows grey screen with boot art.
 397   * `kdp_match_name=en1` - Override default port selection for kdp. Supported for ethernet, thunderbolt and serial debugging.
 398
 399 To debug a panic'ed kernel, use llvm debugger (lldb) along with unstripped symbol rich kernel binary.
 400
 401     sh$ lldb kernel.development.unstripped
 402
 403 And then you can connect to panic'ed machine with `kdp_remote [ip addr]` or `gdb_remote [hostip : port]` commands.
 404
 405 Each kernel is packaged with kernel specific debug scripts as part of the build process. For security reasons these special commands
 406 and scripts do not get loaded automatically when lldb is connected to machine. Please add the following setting to your `~/.lldbinit`
 407 if you wish to always load these macros.
 408
 409     settings set target.load-script-from-symbol-file true
 410
 411 The `tools/lldbmacros` directory contains the source for each of these commands. Please follow the [README.md](tools/lldbmacros/README.md)
 412 for detailed explanation of commands and their usage.
 413
 414 [TN2118]: https://developer.apple.com/library/mac/technotes/tn2004/tn2118.html#//apple_ref/doc/uid/DTS10003352 "Kernel Core Dumps"
 415 [TN2063]: https://developer.apple.com/library/mac/technotes/tn2063/_index.html "Understanding and Debugging Kernel Panics"
 416 [Kernel Programming Guide]: https://developer.apple.com/library/mac/documentation/Darwin/Conceptual/KernelProgramming/build/build.html#//apple_ref/doc/uid/TP30000905-CH221-BABDGEGF