1 // -*- mode: cpp; mode: fold -*-
4 \brief pkgCache - Structure definitions for the cache file
6 The goal of the cache file is two fold:
7 Firstly to speed loading and processing of the package file array and
8 secondly to reduce memory consumption of the package file array.
10 The implementation is aimed at an environment with many primary package
11 files, for instance someone that has a Package file for their CD-ROM, a
12 Package file for the latest version of the distribution on the CD-ROM and a
13 package file for the development version. Always present is the information
14 contained in the status file which might be considered a separate package
17 Please understand, this is designed as a <b>Cache file</b> it is not meant to be
18 used on any system other than the one it was created for. It is not meant to
19 be authoritative either, i.e. if a system crash or software failure occurs it
20 must be perfectly acceptable for the cache file to be in an inconsistent
21 state. Furthermore at any time the cache file may be erased without losing
24 Also the structures and storage layout is optimized for use by the APT
25 and may not be suitable for all purposes. However it should be possible
26 to extend it with associate cache files that contain other information.
28 To keep memory use down the cache file only contains often used fields and
29 fields that are inexpensive to store, the Package file has a full list of
30 fields. Also the client may assume that all items are perfectly valid and
31 need not perform checks against their correctness. Removal of information
32 from the cache is possible, but blanks will be left in the file, and
33 unused strings will also be present. The recommended implementation is to
34 simply rebuild the cache each time any of the data files change. It is
35 possible to add a new package file to the cache without any negative side
38 <b>Note on Pointer access</b>
39 Clients should always use the CacheIterators classes for access to the
40 cache and the data in it. They also provide a simple STL-like method for
41 traversing the links of the datastructure.
43 Every item in every structure is stored as the index to that structure.
44 What this means is that once the files is mmaped every data access has to
45 go through a fix up stage to get a real memory pointer. This is done
46 by taking the index, multiplying it by the type size and then adding
47 it to the start address of the memory block. This sounds complex, but
48 in C it is a single array dereference. Because all items are aligned to
49 their size and indexes are stored as multiples of the size of the structure
50 the format is immediately portable to all possible architectures - BUT the
51 generated files are -NOT-.
53 This scheme allows code like this to be written:
55 void *Map = mmap(...);
56 Package *PkgList = (Package *)Map;
57 Header *Head = (Header *)Map;
58 char *Strings = (char *)Map;
59 cout << (Strings + PkgList[Head->HashTable[0]]->Name) << endl;
61 Notice the lack of casting or multiplication. The net result is to return
62 the name of the first package in the first hash bucket, without error
65 The generator uses allocation pools to group similarly sized structures in
66 large blocks to eliminate any alignment overhead. The generator also
67 assures that no structures overlap and all indexes are unique. Although
68 at first glance it may seem like there is the potential for two structures
69 to exist at the same point the generator never allows this to happen.
70 (See the discussion of free space pools)
72 See \ref pkgcachegen.h for more information about generating cache structures. */
74 #ifndef PKGLIB_PKGCACHE_H
75 #define PKGLIB_PKGCACHE_H
80 #include <apt-pkg/mmap.h>
84 class pkgVersioningSystem
;
85 class pkgCache
/*{{{*/
88 // Cache element predeclarations
102 template<typename Str
, typename Itr
> class Iterator
;
109 class PkgFileIterator
;
110 class VerFileIterator
;
111 class DescFileIterator
;
115 // These are all the constants used in the cache structures
117 // WARNING - if you change these lists you must also edit
118 // the stringification in pkgcache.cc and also consider whether
119 // the cache file will become incompatible.
122 enum DepType
{Depends
=1,PreDepends
=2,Suggests
=3,Recommends
=4,
123 Conflicts
=5,Replaces
=6,Obsoletes
=7,DpkgBreaks
=8,Enhances
=9};
124 /** \brief available compare operators
126 The lower 4 bits are used to indicate what operator is being specified and
127 the upper 4 bits are flags. OR indicates that the next package is
128 or'd with the current package. */
129 enum DepCompareOp
{Or
=0x10,NoOp
=0,LessEq
=0x1,GreaterEq
=0x2,Less
=0x3,
130 Greater
=0x4,Equals
=0x5,NotEquals
=0x6};
135 /** \brief priority of a package version
137 Zero is used for unparsable or absent Priority fields. */
138 enum VerPriority
{Important
=1,Required
=2,Standard
=3,Optional
=4,Extra
=5};
139 enum PkgSelectedState
{Unknown
=0,Install
=1,Hold
=2,DeInstall
=3,Purge
=4};
140 enum PkgInstState
{Ok
=0,ReInstReq
=1,HoldInst
=2,HoldReInstReq
=3};
141 enum PkgCurrentState
{NotInstalled
=0,UnPacked
=1,HalfConfigured
=2,
142 HalfInstalled
=4,ConfigFiles
=5,Installed
=6,
143 TriggersAwaited
=7,TriggersPending
=8};
148 enum PkgFlags
{Auto
=(1<<0),Essential
=(1<<3),Important
=(1<<4)};
149 enum PkgFFlags
{NotSource
=(1<<0),NotAutomatic
=(1<<1),ButAutomaticUpgrades
=(1<<2)};
154 // Memory mapped cache file
158 unsigned long sHash(const string
&S
) const;
159 unsigned long sHash(const char *S
) const;
163 // Pointers to the arrays of items
169 PackageFile
*PkgFileP
;
174 StringItem
*StringItemP
;
177 virtual bool ReMap(bool const &Errorchecks
= true);
178 inline bool Sync() {return Map
.Sync();};
179 inline MMap
&GetMap() {return Map
;};
180 inline void *DataEnd() {return ((unsigned char *)Map
.Data()) + Map
.Size();};
182 // String hashing function (512 range)
183 inline unsigned long Hash(const string
&S
) const {return sHash(S
);};
184 inline unsigned long Hash(const char *S
) const {return sHash(S
);};
186 // Useful transformation things
187 const char *Priority(unsigned char Priority
);
190 GrpIterator
FindGrp(const string
&Name
);
191 PkgIterator
FindPkg(const string
&Name
);
192 PkgIterator
FindPkg(const string
&Name
, const string
&Arch
);
194 Header
&Head() {return *HeaderP
;};
195 inline GrpIterator
GrpBegin();
196 inline GrpIterator
GrpEnd();
197 inline PkgIterator
PkgBegin();
198 inline PkgIterator
PkgEnd();
199 inline PkgFileIterator
FileBegin();
200 inline PkgFileIterator
FileEnd();
202 inline bool MultiArchCache() const { return MultiArchEnabled
; };
204 // Make me a function
205 pkgVersioningSystem
*VS
;
208 static const char *CompTypeDeb(unsigned char Comp
);
209 static const char *CompType(unsigned char Comp
);
210 static const char *DepType(unsigned char Dep
);
212 pkgCache(MMap
*Map
,bool DoMap
= true);
213 virtual ~pkgCache() {};
216 bool MultiArchEnabled
;
217 PkgIterator
SingleArchFindPkg(const string
&Name
);
220 // Header structure /*{{{*/
221 struct pkgCache::Header
223 /** \brief Signature information
225 This must contain the hex value 0x98FE76DC which is designed to
226 verify that the system loading the image has the same byte order
227 and byte size as the system saving the image */
228 unsigned long Signature
;
229 /** These contain the version of the cache file */
232 /** \brief indicates if the cache should be erased
234 Dirty is true if the cache file was opened for reading, the client
235 expects to have written things to it and have not fully synced it.
236 The file should be erased and rebuilt if it is true. */
239 /** \brief Size of structure values
241 All *Sz variables contains the sizeof() that particular structure.
242 It is used as an extra consistency check on the structure of the file.
244 If any of the size values do not exactly match what the client expects
245 then the client should refuse the load the file. */
246 unsigned short HeaderSz
;
247 unsigned short GroupSz
;
248 unsigned short PackageSz
;
249 unsigned short PackageFileSz
;
250 unsigned short VersionSz
;
251 unsigned short DescriptionSz
;
252 unsigned short DependencySz
;
253 unsigned short ProvidesSz
;
254 unsigned short VerFileSz
;
255 unsigned short DescFileSz
;
257 /** \brief Structure counts
259 These indicate the number of each structure contained in the cache.
260 PackageCount is especially useful for generating user state structures.
261 See Package::Id for more info. */
262 unsigned long GroupCount
;
263 unsigned long PackageCount
;
264 unsigned long VersionCount
;
265 unsigned long DescriptionCount
;
266 unsigned long DependsCount
;
267 unsigned long PackageFileCount
;
268 unsigned long VerFileCount
;
269 unsigned long DescFileCount
;
270 unsigned long ProvidesCount
;
272 /** \brief index of the first PackageFile structure
274 The PackageFile structures are singly linked lists that represent
275 all package files that have been merged into the cache. */
277 /** \brief index of the first StringItem structure
279 The cache contains a list of all the unique strings (StringItems).
280 The parser reads this list into memory so it can match strings
282 map_ptrloc StringList
;
283 /** \brief String representing the version system used */
284 map_ptrloc VerSysName
;
285 /** \brief Architecture(s) the cache was built against */
286 map_ptrloc Architecture
;
287 /** \brief The maximum size of a raw entry from the original Package file */
288 unsigned long MaxVerFileSize
;
289 /** \brief The maximum size of a raw entry from the original Translation file */
290 unsigned long MaxDescFileSize
;
292 /** \brief The Pool structures manage the allocation pools that the generator uses
294 Start indicates the first byte of the pool, Count is the number of objects
295 remaining in the pool and ItemSize is the structure size (alignment factor)
296 of the pool. An ItemSize of 0 indicates the pool is empty. There should be
297 the same number of pools as there are structure types. The generator
298 stores this information so future additions can make use of any unused pool
300 DynamicMMap::Pool Pools
[9];
302 /** \brief hash tables providing rapid group/package name lookup
304 Each group/package name is inserted into the hash table using pkgCache::Hash(const &string)
305 By iterating over each entry in the hash table it is possible to iterate over
306 the entire list of packages. Hash Collisions are handled with a singly linked
307 list of packages based at the hash item. The linked list contains only
308 packages that match the hashing function.
309 In the PkgHashTable is it possible that multiple packages have the same name -
310 these packages are stored as a sequence in the list.
312 Beware: The Hashmethod assumes that the hash table sizes are equal */
313 map_ptrloc PkgHashTable
[2*1048];
314 map_ptrloc GrpHashTable
[2*1048];
316 bool CheckSizes(Header
&Against
) const;
320 // Group structure /*{{{*/
321 /** \brief groups architecture depending packages together
323 On or more packages with the same name form a group, so we have
324 a simple way to access a package built for different architectures
325 Group exists in a singly linked list of group records starting at
326 the hash index of the name in the pkgCache::Header::GrpHashTable */
327 struct pkgCache::Group
329 /** \brief Name of the group */
330 map_ptrloc Name
; // StringItem
333 /** \brief Link to the first package which belongs to the group */
334 map_ptrloc FirstPackage
; // Package
335 /** \brief Link to the last package which belongs to the group */
336 map_ptrloc LastPackage
; // Package
337 /** \brief Link to the next Group */
338 map_ptrloc Next
; // Group
339 /** \brief unique sequel ID */
344 // Package structure /*{{{*/
345 /** \brief contains information for a single unique package
347 There can be any number of versions of a given package.
348 Package exists in a singly linked list of package records starting at
349 the hash index of the name in the pkgCache::Header::PkgHashTable
351 A package can be created for every architecture so package names are
352 not unique, but it is garanteed that packages with the same name
353 are sequencel ordered in the list. Packages with the same name can be
354 accessed with the Group.
356 struct pkgCache::Package
358 /** \brief Name of the package */
359 map_ptrloc Name
; // StringItem
360 /** \brief Architecture of the package */
361 map_ptrloc Arch
; // StringItem
362 /** \brief Base of a singly linked list of versions
364 Each structure represents a unique version of the package.
365 The version structures contain links into PackageFile and the
366 original text file as well as detailed information about the size
367 and dependencies of the specific package. In this way multiple
368 versions of a package can be cleanly handled by the system.
369 Furthermore, this linked list is guaranteed to be sorted
370 from Highest version to lowest version with no duplicate entries. */
371 map_ptrloc VersionList
; // Version
372 /** \brief index to the installed version */
373 map_ptrloc CurrentVer
; // Version
374 /** \brief indicates the deduced section
376 Should be the index to the string "Unknown" or to the section
377 of the last parsed item. */
378 map_ptrloc Section
; // StringItem
379 /** \brief index of the group this package belongs to */
380 map_ptrloc Group
; // Group the Package belongs to
383 /** \brief Link to the next package in the same bucket */
384 map_ptrloc NextPackage
; // Package
385 /** \brief List of all dependencies on this package */
386 map_ptrloc RevDepends
; // Dependency
387 /** \brief List of all "packages" this package provide */
388 map_ptrloc ProvidesList
; // Provides
390 // Install/Remove/Purge etc
391 /** \brief state that the user wishes the package to be in */
392 unsigned char SelectedState
; // What
393 /** \brief installation state of the package
395 This should be "ok" but in case the installation failed
396 it will be different.
398 unsigned char InstState
; // Flags
399 /** \brief indicates if the package is installed */
400 unsigned char CurrentState
; // State
402 /** \brief unique sequel ID
404 ID is a unique value from 0 to Header->PackageCount assigned by the generator.
405 This allows clients to create an array of size PackageCount and use it to store
406 state information for the package map. For instance the status file emitter uses
407 this to track which packages have been emitted already. */
409 /** \brief some useful indicators of the package's state */
413 // Package File structure /*{{{*/
414 /** \brief stores information about the files used to generate the cache
416 Package files are referenced by Version structures to be able to know
417 after the generation still from which Packages file includes this Version
418 as we need this information later on e.g. for pinning. */
419 struct pkgCache::PackageFile
421 /** \brief physical disk file that this PackageFile represents */
422 map_ptrloc FileName
; // StringItem
423 /** \brief the release information
425 Please see the files document for a description of what the
426 release information means. */
427 map_ptrloc Archive
; // StringItem
428 map_ptrloc Codename
; // StringItem
429 map_ptrloc Component
; // StringItem
430 map_ptrloc Version
; // StringItem
431 map_ptrloc Origin
; // StringItem
432 map_ptrloc Label
; // StringItem
433 map_ptrloc Architecture
; // StringItem
434 /** \brief The site the index file was fetched from */
435 map_ptrloc Site
; // StringItem
436 /** \brief indicates what sort of index file this is
438 @TODO enumerate at least the possible indexes */
439 map_ptrloc IndexType
; // StringItem
440 /** \brief Size of the file
442 Used together with the modification time as a
443 simple check to ensure that the Packages
444 file has not been altered since Cache generation. */
446 /** \brief Modification time for the file */
449 /* @TODO document PackageFile::Flags */
453 /** \brief Link to the next PackageFile in the Cache */
454 map_ptrloc NextFile
; // PackageFile
455 /** \brief unique sequel ID */
459 // VerFile structure /*{{{*/
460 /** \brief associates a version with a PackageFile
462 This allows a full description of all Versions in all files
463 (and hence all sources) under consideration. */
464 struct pkgCache::VerFile
466 /** \brief index of the package file that this version was found in */
467 map_ptrloc File
; // PackageFile
468 /** \brief next step in the linked list */
469 map_ptrloc NextFile
; // PkgVerFile
470 /** \brief position in the package file */
471 map_ptrloc Offset
; // File offset
472 /* @TODO document pkgCache::VerFile::Size */
476 // DescFile structure /*{{{*/
477 /** \brief associates a description with a Translation file */
478 struct pkgCache::DescFile
480 /** \brief index of the file that this description was found in */
481 map_ptrloc File
; // PackageFile
482 /** \brief next step in the linked list */
483 map_ptrloc NextFile
; // PkgVerFile
484 /** \brief position in the file */
485 map_ptrloc Offset
; // File offset
486 /* @TODO document pkgCache::DescFile::Size */
490 // Version structure /*{{{*/
491 /** \brief information for a single version of a package
493 The version list is always sorted from highest version to lowest
494 version by the generator. Equal version numbers are either merged
495 or handled as separate versions based on the Hash value. */
496 struct pkgCache::Version
498 /** \brief complete version string */
499 map_ptrloc VerStr
; // StringItem
500 /** \brief section this version is filled in */
501 map_ptrloc Section
; // StringItem
502 /** \brief stores the MultiArch capabilities of this version
504 None is the default and doesn't trigger special behaviour,
505 Foreign means that this version can fulfill dependencies even
506 if it is built for another architecture as the requester.
507 Same indicates that builds for different architectures can
508 be co-installed on the system and All is the marker for a
509 version with the Architecture: all. */
510 enum {None
, All
, Foreign
, Same
, Allowed
} MultiArch
;
512 /** \brief references all the PackageFile's that this version came from
514 FileList can be used to determine what distribution(s) the Version
515 applies to. If FileList is 0 then this is a blank version.
516 The structure should also have a 0 in all other fields excluding
517 pkgCache::Version::VerStr and Possibly pkgCache::Version::NextVer. */
518 map_ptrloc FileList
; // VerFile
519 /** \brief next (lower or equal) version in the linked list */
520 map_ptrloc NextVer
; // Version
521 /** \brief next description in the linked list */
522 map_ptrloc DescriptionList
; // Description
523 /** \brief base of the dependency list */
524 map_ptrloc DependsList
; // Dependency
525 /** \brief links to the owning package
527 This allows reverse dependencies to determine the package */
528 map_ptrloc ParentPkg
; // Package
529 /** \brief list of pkgCache::Provides */
530 map_ptrloc ProvidesList
; // Provides
532 /** \brief archive size for this version
534 For Debian this is the size of the .deb file. */
535 unsigned long long Size
; // These are the .deb size
536 /** \brief uncompressed size for this version */
537 unsigned long long InstalledSize
;
538 /** \brief characteristic value representing this version
540 No two packages in existence should have the same VerStr
541 and Hash with different contents. */
543 /** \brief unique sequel ID */
545 /** \brief parsed priority value */
546 unsigned char Priority
;
549 // Description structure /*{{{*/
550 /** \brief datamember of a linked list of available description for a version */
551 struct pkgCache::Description
553 /** \brief Language code of this description (translation)
555 If the value has a 0 length then this is read using the Package
556 file else the Translation-CODE file is used. */
557 map_ptrloc language_code
; // StringItem
558 /** \brief MD5sum of the original description
560 Used to map Translations of a description to a version
561 and to check that the Translation is up-to-date. */
562 map_ptrloc md5sum
; // StringItem
564 /* @TODO document pkgCache::Description::FileList */
565 map_ptrloc FileList
; // DescFile
566 /** \brief next translation for this description */
567 map_ptrloc NextDesc
; // Description
568 /** \brief the text is a description of this package */
569 map_ptrloc ParentPkg
; // Package
571 /** \brief unique sequel ID */
575 // Dependency structure /*{{{*/
576 /** \brief information for a single dependency record
578 The records are split up like this to ease processing by the client.
579 The base of the linked list is pkgCache::Version::DependsList.
580 All forms of dependencies are recorded here including Depends,
581 Recommends, Suggests, Enhances, Conflicts, Replaces and Breaks. */
582 struct pkgCache::Dependency
584 /** \brief string of the version the dependency is applied against */
585 map_ptrloc Version
; // StringItem
586 /** \brief index of the package this depends applies to
588 The generator will - if the package does not already exist -
589 create a blank (no version records) package. */
590 map_ptrloc Package
; // Package
591 /** \brief next dependency of this version */
592 map_ptrloc NextDepends
; // Dependency
593 /** \brief next reverse dependency of this package */
594 map_ptrloc NextRevDepends
; // Dependency
595 /** \brief version of the package which has the reverse depends */
596 map_ptrloc ParentVer
; // Version
598 /** \brief unique sequel ID */
600 /** \brief Dependency type - Depends, Recommends, Conflicts, etc */
602 /** \brief comparison operator specified on the depends line
604 If the high bit is set then it is a logical OR with the previous record. */
605 unsigned char CompareOp
;
608 // Provides structure /*{{{*/
609 /** \brief handles virtual packages
611 When a Provides: line is encountered a new provides record is added
612 associating the package with a virtual package name.
613 The provides structures are linked off the package structures.
614 This simplifies the analysis of dependencies and other aspects A provides
615 refers to a specific version of a specific package, not all versions need to
616 provide that provides.*/
617 struct pkgCache::Provides
619 /** \brief index of the package providing this */
620 map_ptrloc ParentPkg
; // Package
621 /** \brief index of the version this provide line applies to */
622 map_ptrloc Version
; // Version
623 /** \brief version in the provides line (if any)
625 This version allows dependencies to depend on specific versions of a
626 Provides, as well as allowing Provides to override existing packages.
627 This is experimental. Note that Debian doesn't allow versioned provides */
628 map_ptrloc ProvideVersion
; // StringItem
629 /** \brief next provides (based of package) */
630 map_ptrloc NextProvides
; // Provides
631 /** \brief next provides (based of version) */
632 map_ptrloc NextPkgProv
; // Provides
635 // StringItem structure /*{{{*/
636 /** \brief used for generating single instances of strings
638 Some things like Section Name are are useful to have as unique tags.
639 It is part of a linked list based at pkgCache::Header::StringList
641 All strings are simply inlined any place in the file that is natural
642 for the writer. The client should make no assumptions about the positioning
643 of strings. All StringItems should be null-terminated. */
644 struct pkgCache::StringItem
646 /** \brief string this refers to */
647 map_ptrloc String
; // StringItem
648 /** \brief Next link in the chain */
649 map_ptrloc NextItem
; // StringItem
652 #include <apt-pkg/cacheiterators.h>
654 inline pkgCache::GrpIterator
pkgCache::GrpBegin()
655 {return GrpIterator(*this);};
656 inline pkgCache::GrpIterator
pkgCache::GrpEnd()
657 {return GrpIterator(*this,GrpP
);};
658 inline pkgCache::PkgIterator
pkgCache::PkgBegin()
659 {return PkgIterator(*this);};
660 inline pkgCache::PkgIterator
pkgCache::PkgEnd()
661 {return PkgIterator(*this,PkgP
);};
662 inline pkgCache::PkgFileIterator
pkgCache::FileBegin()
663 {return PkgFileIterator(*this,PkgFileP
+ HeaderP
->FileList
);};
664 inline pkgCache::PkgFileIterator
pkgCache::FileEnd()
665 {return PkgFileIterator(*this,PkgFileP
);};
667 // Oh I wish for Real Name Space Support
668 class pkgCache::Namespace
/*{{{*/
671 typedef pkgCache::GrpIterator GrpIterator
;
672 typedef pkgCache::PkgIterator PkgIterator
;
673 typedef pkgCache::VerIterator VerIterator
;
674 typedef pkgCache::DescIterator DescIterator
;
675 typedef pkgCache::DepIterator DepIterator
;
676 typedef pkgCache::PrvIterator PrvIterator
;
677 typedef pkgCache::PkgFileIterator PkgFileIterator
;
678 typedef pkgCache::VerFileIterator VerFileIterator
;
679 typedef pkgCache::Version Version
;
680 typedef pkgCache::Description Description
;
681 typedef pkgCache::Package Package
;
682 typedef pkgCache::Header Header
;
683 typedef pkgCache::Dep Dep
;
684 typedef pkgCache::Flag Flag
;