1 // -*- mode: cpp; mode: fold -*-
4 \brief pkgCache - Structure definitions for the cache file
6 The goal of the cache file is two fold:
7 Firstly to speed loading and processing of the package file array and
8 secondly to reduce memory consumption of the package file array.
10 The implementation is aimed at an environment with many primary package
11 files, for instance someone that has a Package file for their CD-ROM, a
12 Package file for the latest version of the distribution on the CD-ROM and a
13 package file for the development version. Always present is the information
14 contained in the status file which might be considered a separate package
17 Please understand, this is designed as a <b>Cache file</b> it is not meant to be
18 used on any system other than the one it was created for. It is not meant to
19 be authoritative either, i.e. if a system crash or software failure occurs it
20 must be perfectly acceptable for the cache file to be in an inconsistent
21 state. Furthermore at any time the cache file may be erased without losing
24 Also the structures and storage layout is optimized for use by the APT
25 and may not be suitable for all purposes. However it should be possible
26 to extend it with associate cache files that contain other information.
28 To keep memory use down the cache file only contains often used fields and
29 fields that are inexpensive to store, the Package file has a full list of
30 fields. Also the client may assume that all items are perfectly valid and
31 need not perform checks against their correctness. Removal of information
32 from the cache is possible, but blanks will be left in the file, and
33 unused strings will also be present. The recommended implementation is to
34 simply rebuild the cache each time any of the data files change. It is
35 possible to add a new package file to the cache without any negative side
38 <b>Note on Pointer access</b>
39 Clients should always use the CacheIterators classes for access to the
40 cache and the data in it. They also provide a simple STL-like method for
41 traversing the links of the datastructure.
43 Every item in every structure is stored as the index to that structure.
44 What this means is that once the files is mmaped every data access has to
45 go through a fix up stage to get a real memory pointer. This is done
46 by taking the index, multiplying it by the type size and then adding
47 it to the start address of the memory block. This sounds complex, but
48 in C it is a single array dereference. Because all items are aligned to
49 their size and indexes are stored as multiples of the size of the structure
50 the format is immediately portable to all possible architectures - BUT the
51 generated files are -NOT-.
53 This scheme allows code like this to be written:
55 void *Map = mmap(...);
56 Package *PkgList = (Package *)Map;
57 Header *Head = (Header *)Map;
58 char *Strings = (char *)Map;
59 cout << (Strings + PkgList[Head->HashTable[0]]->Name) << endl;
61 Notice the lack of casting or multiplication. The net result is to return
62 the name of the first package in the first hash bucket, without error
65 The generator uses allocation pools to group similarly sized structures in
66 large blocks to eliminate any alignment overhead. The generator also
67 assures that no structures overlap and all indexes are unique. Although
68 at first glance it may seem like there is the potential for two structures
69 to exist at the same point the generator never allows this to happen.
70 (See the discussion of free space pools)
72 See \ref pkgcachegen.h for more information about generating cache structures. */
74 #ifndef PKGLIB_PKGCACHE_H
75 #define PKGLIB_PKGCACHE_H
77 #include <apt-pkg/mmap.h>
78 #include <apt-pkg/macros.h>
83 #ifndef APT_8_CLEANER_HEADERS
87 class pkgVersioningSystem
;
88 class pkgCache
/*{{{*/
91 // Cache element predeclarations
105 template<typename Str
, typename Itr
> class Iterator
;
112 class PkgFileIterator
;
113 class VerFileIterator
;
114 class DescFileIterator
;
118 // These are all the constants used in the cache structures
120 // WARNING - if you change these lists you must also edit
121 // the stringification in pkgcache.cc and also consider whether
122 // the cache file will become incompatible.
125 enum DepType
{Depends
=1,PreDepends
=2,Suggests
=3,Recommends
=4,
126 Conflicts
=5,Replaces
=6,Obsoletes
=7,DpkgBreaks
=8,Enhances
=9};
127 /** \brief available compare operators
129 The lower 4 bits are used to indicate what operator is being specified and
130 the upper 4 bits are flags. OR indicates that the next package is
131 or'd with the current package. */
132 enum DepCompareOp
{Or
=0x10,NoOp
=0,LessEq
=0x1,GreaterEq
=0x2,Less
=0x3,
133 Greater
=0x4,Equals
=0x5,NotEquals
=0x6};
138 /** \brief priority of a package version
140 Zero is used for unparsable or absent Priority fields. */
141 enum VerPriority
{Required
=1,Important
=2,Standard
=3,Optional
=4,Extra
=5};
142 enum PkgSelectedState
{Unknown
=0,Install
=1,Hold
=2,DeInstall
=3,Purge
=4};
143 enum PkgInstState
{Ok
=0,ReInstReq
=1,HoldInst
=2,HoldReInstReq
=3};
144 enum PkgCurrentState
{NotInstalled
=0,UnPacked
=1,HalfConfigured
=2,
145 HalfInstalled
=4,ConfigFiles
=5,Installed
=6,
146 TriggersAwaited
=7,TriggersPending
=8};
151 enum PkgFlags
{Auto
=(1<<0),Essential
=(1<<3),Important
=(1<<4)};
152 enum PkgFFlags
{NotSource
=(1<<0),NotAutomatic
=(1<<1),ButAutomaticUpgrades
=(1<<2)};
157 // Memory mapped cache file
158 std::string CacheFile
;
161 unsigned long sHash(const std::string
&S
) const APT_PURE
;
162 unsigned long sHash(const char *S
) const APT_PURE
;
166 // Pointers to the arrays of items
172 PackageFile
*PkgFileP
;
177 StringItem
*StringItemP
;
180 virtual bool ReMap(bool const &Errorchecks
= true);
181 inline bool Sync() {return Map
.Sync();}
182 inline MMap
&GetMap() {return Map
;}
183 inline void *DataEnd() {return ((unsigned char *)Map
.Data()) + Map
.Size();}
185 // String hashing function (512 range)
186 inline unsigned long Hash(const std::string
&S
) const {return sHash(S
);}
187 inline unsigned long Hash(const char *S
) const {return sHash(S
);}
189 // Useful transformation things
190 const char *Priority(unsigned char Priority
);
193 GrpIterator
FindGrp(const std::string
&Name
);
194 PkgIterator
FindPkg(const std::string
&Name
);
195 PkgIterator
FindPkg(const std::string
&Name
, const std::string
&Arch
);
197 Header
&Head() {return *HeaderP
;}
198 inline GrpIterator
GrpBegin();
199 inline GrpIterator
GrpEnd();
200 inline PkgIterator
PkgBegin();
201 inline PkgIterator
PkgEnd();
202 inline PkgFileIterator
FileBegin();
203 inline PkgFileIterator
FileEnd();
205 inline bool MultiArchCache() const { return MultiArchEnabled
; }
206 inline char const * NativeArch();
208 // Make me a function
209 pkgVersioningSystem
*VS
;
212 static const char *CompTypeDeb(unsigned char Comp
) APT_CONST
;
213 static const char *CompType(unsigned char Comp
) APT_CONST
;
214 static const char *DepType(unsigned char Dep
);
216 pkgCache(MMap
*Map
,bool DoMap
= true);
217 virtual ~pkgCache() {}
220 bool MultiArchEnabled
;
221 PkgIterator
SingleArchFindPkg(const std::string
&Name
);
224 // Header structure /*{{{*/
225 struct pkgCache::Header
227 /** \brief Signature information
229 This must contain the hex value 0x98FE76DC which is designed to
230 verify that the system loading the image has the same byte order
231 and byte size as the system saving the image */
232 unsigned long Signature
;
233 /** These contain the version of the cache file */
236 /** \brief indicates if the cache should be erased
238 Dirty is true if the cache file was opened for reading, the client
239 expects to have written things to it and have not fully synced it.
240 The file should be erased and rebuilt if it is true. */
243 /** \brief Size of structure values
245 All *Sz variables contains the sizeof() that particular structure.
246 It is used as an extra consistency check on the structure of the file.
248 If any of the size values do not exactly match what the client expects
249 then the client should refuse the load the file. */
250 unsigned short HeaderSz
;
251 unsigned short GroupSz
;
252 unsigned short PackageSz
;
253 unsigned short PackageFileSz
;
254 unsigned short VersionSz
;
255 unsigned short DescriptionSz
;
256 unsigned short DependencySz
;
257 unsigned short ProvidesSz
;
258 unsigned short VerFileSz
;
259 unsigned short DescFileSz
;
261 /** \brief Structure counts
263 These indicate the number of each structure contained in the cache.
264 PackageCount is especially useful for generating user state structures.
265 See Package::Id for more info. */
266 unsigned long GroupCount
;
267 unsigned long PackageCount
;
268 unsigned long VersionCount
;
269 unsigned long DescriptionCount
;
270 unsigned long DependsCount
;
271 unsigned long PackageFileCount
;
272 unsigned long VerFileCount
;
273 unsigned long DescFileCount
;
274 unsigned long ProvidesCount
;
276 /** \brief index of the first PackageFile structure
278 The PackageFile structures are singly linked lists that represent
279 all package files that have been merged into the cache. */
281 /** \brief index of the first StringItem structure
283 The cache contains a list of all the unique strings (StringItems).
284 The parser reads this list into memory so it can match strings
286 map_ptrloc StringList
;
287 /** \brief String representing the version system used */
288 map_ptrloc VerSysName
;
289 /** \brief native architecture the cache was built against */
290 map_ptrloc Architecture
;
291 /** \brief all architectures the cache was built against */
292 map_ptrloc Architectures
;
293 /** \brief The maximum size of a raw entry from the original Package file */
294 unsigned long MaxVerFileSize
;
295 /** \brief The maximum size of a raw entry from the original Translation file */
296 unsigned long MaxDescFileSize
;
298 /** \brief The Pool structures manage the allocation pools that the generator uses
300 Start indicates the first byte of the pool, Count is the number of objects
301 remaining in the pool and ItemSize is the structure size (alignment factor)
302 of the pool. An ItemSize of 0 indicates the pool is empty. There should be
303 the same number of pools as there are structure types. The generator
304 stores this information so future additions can make use of any unused pool
306 DynamicMMap::Pool Pools
[9];
308 /** \brief hash tables providing rapid group/package name lookup
310 Each group/package name is inserted into the hash table using pkgCache::Hash(const &string)
311 By iterating over each entry in the hash table it is possible to iterate over
312 the entire list of packages. Hash Collisions are handled with a singly linked
313 list of packages based at the hash item. The linked list contains only
314 packages that match the hashing function.
315 In the PkgHashTable is it possible that multiple packages have the same name -
316 these packages are stored as a sequence in the list.
318 Beware: The Hashmethod assumes that the hash table sizes are equal */
319 map_ptrloc PkgHashTable
[2*1048];
320 map_ptrloc GrpHashTable
[2*1048];
322 /** \brief Size of the complete cache file */
323 unsigned long CacheFileSize
;
325 bool CheckSizes(Header
&Against
) const APT_PURE
;
329 // Group structure /*{{{*/
330 /** \brief groups architecture depending packages together
332 On or more packages with the same name form a group, so we have
333 a simple way to access a package built for different architectures
334 Group exists in a singly linked list of group records starting at
335 the hash index of the name in the pkgCache::Header::GrpHashTable */
336 struct pkgCache::Group
338 /** \brief Name of the group */
339 map_ptrloc Name
; // StringItem
342 /** \brief Link to the first package which belongs to the group */
343 map_ptrloc FirstPackage
; // Package
344 /** \brief Link to the last package which belongs to the group */
345 map_ptrloc LastPackage
; // Package
346 /** \brief Link to the next Group */
347 map_ptrloc Next
; // Group
348 /** \brief unique sequel ID */
353 // Package structure /*{{{*/
354 /** \brief contains information for a single unique package
356 There can be any number of versions of a given package.
357 Package exists in a singly linked list of package records starting at
358 the hash index of the name in the pkgCache::Header::PkgHashTable
360 A package can be created for every architecture so package names are
361 not unique, but it is guaranteed that packages with the same name
362 are sequencel ordered in the list. Packages with the same name can be
363 accessed with the Group.
365 struct pkgCache::Package
367 /** \brief Name of the package */
368 map_ptrloc Name
; // StringItem
369 /** \brief Architecture of the package */
370 map_ptrloc Arch
; // StringItem
371 /** \brief Base of a singly linked list of versions
373 Each structure represents a unique version of the package.
374 The version structures contain links into PackageFile and the
375 original text file as well as detailed information about the size
376 and dependencies of the specific package. In this way multiple
377 versions of a package can be cleanly handled by the system.
378 Furthermore, this linked list is guaranteed to be sorted
379 from Highest version to lowest version with no duplicate entries. */
380 map_ptrloc VersionList
; // Version
381 /** \brief index to the installed version */
382 map_ptrloc CurrentVer
; // Version
383 /** \brief indicates the deduced section
385 Should be the index to the string "Unknown" or to the section
386 of the last parsed item. */
387 map_ptrloc Section
; // StringItem
388 /** \brief index of the group this package belongs to */
389 map_ptrloc Group
; // Group the Package belongs to
392 /** \brief Link to the next package in the same bucket */
393 map_ptrloc NextPackage
; // Package
394 /** \brief List of all dependencies on this package */
395 map_ptrloc RevDepends
; // Dependency
396 /** \brief List of all "packages" this package provide */
397 map_ptrloc ProvidesList
; // Provides
399 // Install/Remove/Purge etc
400 /** \brief state that the user wishes the package to be in */
401 unsigned char SelectedState
; // What
402 /** \brief installation state of the package
404 This should be "ok" but in case the installation failed
405 it will be different.
407 unsigned char InstState
; // Flags
408 /** \brief indicates if the package is installed */
409 unsigned char CurrentState
; // State
411 /** \brief unique sequel ID
413 ID is a unique value from 0 to Header->PackageCount assigned by the generator.
414 This allows clients to create an array of size PackageCount and use it to store
415 state information for the package map. For instance the status file emitter uses
416 this to track which packages have been emitted already. */
418 /** \brief some useful indicators of the package's state */
422 // Package File structure /*{{{*/
423 /** \brief stores information about the files used to generate the cache
425 Package files are referenced by Version structures to be able to know
426 after the generation still from which Packages file includes this Version
427 as we need this information later on e.g. for pinning. */
428 struct pkgCache::PackageFile
430 /** \brief physical disk file that this PackageFile represents */
431 map_ptrloc FileName
; // StringItem
432 /** \brief the release information
434 Please see the files document for a description of what the
435 release information means. */
436 map_ptrloc Archive
; // StringItem
437 map_ptrloc Codename
; // StringItem
438 map_ptrloc Component
; // StringItem
439 map_ptrloc Version
; // StringItem
440 map_ptrloc Origin
; // StringItem
441 map_ptrloc Label
; // StringItem
442 map_ptrloc Architecture
; // StringItem
443 /** \brief The site the index file was fetched from */
444 map_ptrloc Site
; // StringItem
445 /** \brief indicates what sort of index file this is
447 @TODO enumerate at least the possible indexes */
448 map_ptrloc IndexType
; // StringItem
449 /** \brief Size of the file
451 Used together with the modification time as a
452 simple check to ensure that the Packages
453 file has not been altered since Cache generation. */
455 /** \brief Modification time for the file */
458 /** @TODO document PackageFile::Flags */
462 /** \brief Link to the next PackageFile in the Cache */
463 map_ptrloc NextFile
; // PackageFile
464 /** \brief unique sequel ID */
468 // VerFile structure /*{{{*/
469 /** \brief associates a version with a PackageFile
471 This allows a full description of all Versions in all files
472 (and hence all sources) under consideration. */
473 struct pkgCache::VerFile
475 /** \brief index of the package file that this version was found in */
476 map_ptrloc File
; // PackageFile
477 /** \brief next step in the linked list */
478 map_ptrloc NextFile
; // PkgVerFile
479 /** \brief position in the package file */
480 map_ptrloc Offset
; // File offset
481 /** @TODO document pkgCache::VerFile::Size */
485 // DescFile structure /*{{{*/
486 /** \brief associates a description with a Translation file */
487 struct pkgCache::DescFile
489 /** \brief index of the file that this description was found in */
490 map_ptrloc File
; // PackageFile
491 /** \brief next step in the linked list */
492 map_ptrloc NextFile
; // PkgVerFile
493 /** \brief position in the file */
494 map_ptrloc Offset
; // File offset
495 /** @TODO document pkgCache::DescFile::Size */
499 // Version structure /*{{{*/
500 /** \brief information for a single version of a package
502 The version list is always sorted from highest version to lowest
503 version by the generator. Equal version numbers are either merged
504 or handled as separate versions based on the Hash value. */
505 struct pkgCache::Version
507 /** \brief complete version string */
508 map_ptrloc VerStr
; // StringItem
509 /** \brief section this version is filled in */
510 map_ptrloc Section
; // StringItem
512 /** \brief Multi-Arch capabilities of a package version */
513 enum VerMultiArch
{ None
= 0, /*!< is the default and doesn't trigger special behaviour */
514 All
= (1<<0), /*!< will cause that Ver.Arch() will report "all" */
515 Foreign
= (1<<1), /*!< can satisfy dependencies in another architecture */
516 Same
= (1<<2), /*!< can be co-installed with itself from other architectures */
517 Allowed
= (1<<3), /*!< other packages are allowed to depend on thispkg:any */
518 AllForeign
= All
| Foreign
,
519 AllAllowed
= All
| Allowed
};
520 /** \brief stores the MultiArch capabilities of this version
522 Flags used are defined in pkgCache::Version::VerMultiArch
524 unsigned char MultiArch
;
526 /** \brief references all the PackageFile's that this version came from
528 FileList can be used to determine what distribution(s) the Version
529 applies to. If FileList is 0 then this is a blank version.
530 The structure should also have a 0 in all other fields excluding
531 pkgCache::Version::VerStr and Possibly pkgCache::Version::NextVer. */
532 map_ptrloc FileList
; // VerFile
533 /** \brief next (lower or equal) version in the linked list */
534 map_ptrloc NextVer
; // Version
535 /** \brief next description in the linked list */
536 map_ptrloc DescriptionList
; // Description
537 /** \brief base of the dependency list */
538 map_ptrloc DependsList
; // Dependency
539 /** \brief links to the owning package
541 This allows reverse dependencies to determine the package */
542 map_ptrloc ParentPkg
; // Package
543 /** \brief list of pkgCache::Provides */
544 map_ptrloc ProvidesList
; // Provides
546 /** \brief archive size for this version
548 For Debian this is the size of the .deb file. */
549 unsigned long long Size
; // These are the .deb size
550 /** \brief uncompressed size for this version */
551 unsigned long long InstalledSize
;
552 /** \brief characteristic value representing this version
554 No two packages in existence should have the same VerStr
555 and Hash with different contents. */
557 /** \brief unique sequel ID */
559 /** \brief parsed priority value */
560 unsigned char Priority
;
563 // Description structure /*{{{*/
564 /** \brief datamember of a linked list of available description for a version */
565 struct pkgCache::Description
567 /** \brief Language code of this description (translation)
569 If the value has a 0 length then this is read using the Package
570 file else the Translation-CODE file is used. */
571 map_ptrloc language_code
; // StringItem
572 /** \brief MD5sum of the original description
574 Used to map Translations of a description to a version
575 and to check that the Translation is up-to-date. */
576 map_ptrloc md5sum
; // StringItem
578 /** @TODO document pkgCache::Description::FileList */
579 map_ptrloc FileList
; // DescFile
580 /** \brief next translation for this description */
581 map_ptrloc NextDesc
; // Description
582 /** \brief the text is a description of this package */
583 map_ptrloc ParentPkg
; // Package
585 /** \brief unique sequel ID */
589 // Dependency structure /*{{{*/
590 /** \brief information for a single dependency record
592 The records are split up like this to ease processing by the client.
593 The base of the linked list is pkgCache::Version::DependsList.
594 All forms of dependencies are recorded here including Depends,
595 Recommends, Suggests, Enhances, Conflicts, Replaces and Breaks. */
596 struct pkgCache::Dependency
598 /** \brief string of the version the dependency is applied against */
599 map_ptrloc Version
; // StringItem
600 /** \brief index of the package this depends applies to
602 The generator will - if the package does not already exist -
603 create a blank (no version records) package. */
604 map_ptrloc Package
; // Package
605 /** \brief next dependency of this version */
606 map_ptrloc NextDepends
; // Dependency
607 /** \brief next reverse dependency of this package */
608 map_ptrloc NextRevDepends
; // Dependency
609 /** \brief version of the package which has the reverse depends */
610 map_ptrloc ParentVer
; // Version
612 /** \brief unique sequel ID */
614 /** \brief Dependency type - Depends, Recommends, Conflicts, etc */
616 /** \brief comparison operator specified on the depends line
618 If the high bit is set then it is a logical OR with the previous record. */
619 unsigned char CompareOp
;
622 // Provides structure /*{{{*/
623 /** \brief handles virtual packages
625 When a Provides: line is encountered a new provides record is added
626 associating the package with a virtual package name.
627 The provides structures are linked off the package structures.
628 This simplifies the analysis of dependencies and other aspects A provides
629 refers to a specific version of a specific package, not all versions need to
630 provide that provides.*/
631 struct pkgCache::Provides
633 /** \brief index of the package providing this */
634 map_ptrloc ParentPkg
; // Package
635 /** \brief index of the version this provide line applies to */
636 map_ptrloc Version
; // Version
637 /** \brief version in the provides line (if any)
639 This version allows dependencies to depend on specific versions of a
640 Provides, as well as allowing Provides to override existing packages.
641 This is experimental. Note that Debian doesn't allow versioned provides */
642 map_ptrloc ProvideVersion
; // StringItem
643 /** \brief next provides (based of package) */
644 map_ptrloc NextProvides
; // Provides
645 /** \brief next provides (based of version) */
646 map_ptrloc NextPkgProv
; // Provides
649 // StringItem structure /*{{{*/
650 /** \brief used for generating single instances of strings
652 Some things like Section Name are are useful to have as unique tags.
653 It is part of a linked list based at pkgCache::Header::StringList
655 All strings are simply inlined any place in the file that is natural
656 for the writer. The client should make no assumptions about the positioning
657 of strings. All StringItems should be null-terminated. */
658 struct pkgCache::StringItem
660 /** \brief string this refers to */
661 map_ptrloc String
; // StringItem
662 /** \brief Next link in the chain */
663 map_ptrloc NextItem
; // StringItem
668 inline char const * pkgCache::NativeArch()
669 { return StrP
+ HeaderP
->Architecture
; }
671 #include <apt-pkg/cacheiterators.h>
673 inline pkgCache::GrpIterator
pkgCache::GrpBegin()
674 {return GrpIterator(*this);}
675 inline pkgCache::GrpIterator
pkgCache::GrpEnd()
676 {return GrpIterator(*this,GrpP
);}
677 inline pkgCache::PkgIterator
pkgCache::PkgBegin()
678 {return PkgIterator(*this);}
679 inline pkgCache::PkgIterator
pkgCache::PkgEnd()
680 {return PkgIterator(*this,PkgP
);}
681 inline pkgCache::PkgFileIterator
pkgCache::FileBegin()
682 {return PkgFileIterator(*this,PkgFileP
+ HeaderP
->FileList
);}
683 inline pkgCache::PkgFileIterator
pkgCache::FileEnd()
684 {return PkgFileIterator(*this,PkgFileP
);}
686 // Oh I wish for Real Name Space Support
687 class pkgCache::Namespace
/*{{{*/
690 typedef pkgCache::GrpIterator GrpIterator
;
691 typedef pkgCache::PkgIterator PkgIterator
;
692 typedef pkgCache::VerIterator VerIterator
;
693 typedef pkgCache::DescIterator DescIterator
;
694 typedef pkgCache::DepIterator DepIterator
;
695 typedef pkgCache::PrvIterator PrvIterator
;
696 typedef pkgCache::PkgFileIterator PkgFileIterator
;
697 typedef pkgCache::VerFileIterator VerFileIterator
;
698 typedef pkgCache::Version Version
;
699 typedef pkgCache::Description Description
;
700 typedef pkgCache::Package Package
;
701 typedef pkgCache::Header Header
;
702 typedef pkgCache::Dep Dep
;
703 typedef pkgCache::Flag Flag
;