projects / apple / xnu.git / blame
| Commit | Line | Data |
|---|---|---|
| 39037602 A |
1 | Kernel Data Descriptors |
| 2 | ======================= | |
| 3 | ||
| 4 | This project allows for dynamic data to be passed from the kernel to userspace tools without binding them to particular version of | |
| 5 | struct definition. The `libkdd` library provides convenient API for parsing and interpreting `kernel chunked data`. | |
| 6 | ||
| 7 | The libkdd APIs are defined in [kdd.h](./kdd.h) | |
| 8 | ||
| 9 | The `KCDATA` format | |
| 10 | =================== | |
| 11 | ||
| 12 | The format for data is setup in a generic format as follows | |
| 13 | ||
| 14 | Layout of data structure | |
| 15 | ------------------------ | |
| 16 | ||
| 17 | | 8 - bytes | | |
| 18 | |---------------------------| ------ offset = 00 | |
| 19 | | type = MAGIC | LENGTH | # BEGIN Header | |
| 20 | | 0 | | |
| 21 | |---------------------------| ------ offset = 16 | |
| 22 | | type | size | # chunk header | |
| 23 | | flags | | |
| 24 | |---------------------------| ------ offset = 32 | |
| 25 | | data | # arbitrary data (len=16) | |
| 26 | |___________data____________| | |
| 27 | |---------------------------| ------ offset = 48 | |
| 28 | | type | size | # chunk header | |
| 29 | | flags | | |
| 30 | |---------------------------| ------ offset = 64 | |
| 31 | | data | # arbitrary data (len=32) | |
| 32 | | data | | |
| 33 | | data | | |
| 34 | |___________data____________| | |
| 35 | |---------------------------| ------ offset = 96 | |
| 36 | | type = END | size=0 | # chunk header | |
| 37 | | 0 | | |
| 38 | ||
| 39 | ||
| 40 | The type field describes what kind of data is passed. For example type = `TASK_CRASHINFO_UUID` means the following data is a uuid. | |
| 41 | These types need to be defined in task_corpses.h for easy consumption by userspace inspection tools. | |
| 42 | ||
| 43 | Some range of types is reserved for special types like ints, longs etc. A cool new functionality made possible with this | |
| 44 | extensible data format is that kernel can decide to put more information as required without requiring user space tools to | |
| 45 | re-compile to be compatible. The case of `rusage` struct versions could be introduced without breaking existing tools. | |
| 46 | ||
| 47 | Feature description: Generic data with description | |
| 48 | ------------------- | |
| 49 | Further more generic data with description is very much possible now. For example | |
| 50 | ||
| 51 | - kcdata_add_uint64_with_description(cdatainfo, 0x700, "NUM MACH PORTS"); | |
| 52 | - and more functions that allow adding description. | |
| 53 | ||
| 54 | The userspace tools can then look at the description and print the data even if they are not compiled with knowledge of the field apriori. | |
| 55 | ||
| 56 | Example data: | |
| 57 | 0000 57 f1 ad de 00 00 00 00 00 00 00 00 00 00 00 00 W............... | |
| 58 | 0010 01 00 00 00 00 00 00 00 30 00 00 00 00 00 00 00 ........0....... | |
| 59 | 0020 50 49 44 00 00 00 00 00 00 00 00 00 00 00 00 00 PID............. | |
| 60 | 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ | |
| 61 | 0040 9c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ | |
| 62 | 0050 01 00 00 00 00 00 00 00 30 00 00 00 00 00 00 00 ........0....... | |
| 63 | 0060 50 41 52 45 4e 54 20 50 49 44 00 00 00 00 00 00 PARENT PID...... | |
| 64 | 0070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ | |
| 65 | 0080 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ | |
| 66 | 0090 ed 58 91 f1 | |
| 67 | ||
| 68 | ||
| 69 | Feature description: Container markers for compound data | |
| 70 | ------------------ | |
| 71 | ||
| 72 | If a given kernel data type is complex and requires adding multiple optional fields inside a container | |
| 73 | object for a consumer to understand arbitrary data, we package it using container markers. | |
| 74 | ||
| 75 | For example, the stackshot code gathers information and describes the state of a given task with respect | |
| 76 | to many subsystems. It includes data such as io stats, vm counters, process names/flags and syscall counts. | |
| 77 | ||
| 78 | kcdata_add_container_marker(kcdata_p, KCDATA_TYPE_CONTAINER_BEGIN, STACKSHOT_KCCONTAINER_TASK, task_uniqueid); | |
| 79 | // add multiple data, or add_<type>_with_description()s here | |
| 80 | ||
| 81 | kcdata_add_container_marker(kcdata_p, KCDATA_TYPE_CONTAINER_END, STACKSHOT_KCCONTAINER_TASK, task_uniqueid); | |
| 82 | ||
| 83 | ||
| 84 | Feature description: Custom Data formats on demand | |
| 85 | -------------------- | |
| 86 | ||
| 87 | With the self describing nature of format, the kernel provider can describe a data type (uniquely identified by a number) and use | |
| 88 | it in the buffer for sending data. The consumer can parse the type information and have knowledge of describing incoming data. | |
| 89 | Following is an example of how we can describe a kernel specific struct sample_disk_io_stats in buffer. | |
| 90 | ||
| 91 | struct sample_disk_io_stats { | |
| 92 | uint64_t disk_reads_count; | |
| 93 | uint64_t disk_reads_size; | |
| 94 | uint64_t io_priority_count[4]; | |
| 95 | uint64_t io_priority_size; | |
| 96 | } __attribute__ ((packed)); | |
| 97 | ||
| 98 | ||
| 99 | struct kcdata_subtype_descriptor disk_io_stats_def[] = { | |
| 100 | {KCS_SUBTYPE_FLAGS_NONE, KC_ST_UINT64, 0 * sizeof(uint64_t), sizeof(uint64_t), "disk_reads_count"}, | |
| 101 | {KCS_SUBTYPE_FLAGS_NONE, KC_ST_UINT64, 1 * sizeof(uint64_t), sizeof(uint64_t), "disk_reads_size"}, | |
| 102 | {KCS_SUBTYPE_FLAGS_ARRAY, KC_ST_UINT64, 2 * sizeof(uint64_t), KCS_SUBTYPE_PACK_SIZE(4, sizeof(uint64_t)), "io_priority_count"}, | |
| 103 | {KCS_SUBTYPE_FLAGS_ARRAY, KC_ST_UINT64, (2 + 4) * sizeof(uint64_t), sizeof(uint64_t), "io_priority_size"}, | |
| 104 | }; | |
| 105 | ||
| 106 | Now you can add this custom type definition into the buffer as | |
| 107 | kcdata_add_type_definition(kcdata_p, KCTYPE_SAMPLE_DISK_IO_STATS, "sample_disk_io_stats", | |
| 108 | &disk_io_stats_def[0], sizeof(disk_io_stats_def)/sizeof(struct kcdata_subtype_descriptor)); | |
| 109 |