1 // -*- mode: cpp; mode: fold -*-
4 \brief pkgCache - Structure definitions for the cache file
6 The goal of the cache file is two fold:
7 Firstly to speed loading and processing of the package file array and
8 secondly to reduce memory consumption of the package file array.
10 The implementation is aimed at an environment with many primary package
11 files, for instance someone that has a Package file for their CD-ROM, a
12 Package file for the latest version of the distribution on the CD-ROM and a
13 package file for the development version. Always present is the information
14 contained in the status file which might be considered a separate package
17 Please understand, this is designed as a <b>Cache file</b> it is not meant to be
18 used on any system other than the one it was created for. It is not meant to
19 be authoritative either, i.e. if a system crash or software failure occurs it
20 must be perfectly acceptable for the cache file to be in an inconsistent
21 state. Furthermore at any time the cache file may be erased without losing
24 Also the structures and storage layout is optimized for use by the APT
25 and may not be suitable for all purposes. However it should be possible
26 to extend it with associate cache files that contain other information.
28 To keep memory use down the cache file only contains often used fields and
29 fields that are inexpensive to store, the Package file has a full list of
30 fields. Also the client may assume that all items are perfectly valid and
31 need not perform checks against their correctness. Removal of information
32 from the cache is possible, but blanks will be left in the file, and
33 unused strings will also be present. The recommended implementation is to
34 simply rebuild the cache each time any of the data files change. It is
35 possible to add a new package file to the cache without any negative side
38 <b>Note on Pointer access</b>
39 Clients should always use the CacheIterators classes for access to the
40 cache and the data in it. They also provide a simple STL-like method for
41 traversing the links of the datastructure.
43 Every item in every structure is stored as the index to that structure.
44 What this means is that once the files is mmaped every data access has to
45 go through a fix up stage to get a real memory pointer. This is done
46 by taking the index, multiplying it by the type size and then adding
47 it to the start address of the memory block. This sounds complex, but
48 in C it is a single array dereference. Because all items are aligned to
49 their size and indexes are stored as multiples of the size of the structure
50 the format is immediately portable to all possible architectures - BUT the
51 generated files are -NOT-.
53 This scheme allows code like this to be written:
55 void *Map = mmap(...);
56 Package *PkgList = (Package *)Map;
57 Header *Head = (Header *)Map;
58 char *Strings = (char *)Map;
59 cout << (Strings + PkgList[Head->HashTable[0]]->Name) << endl;
61 Notice the lack of casting or multiplication. The net result is to return
62 the name of the first package in the first hash bucket, without error
65 The generator uses allocation pools to group similarly sized structures in
66 large blocks to eliminate any alignment overhead. The generator also
67 assures that no structures overlap and all indexes are unique. Although
68 at first glance it may seem like there is the potential for two structures
69 to exist at the same point the generator never allows this to happen.
70 (See the discussion of free space pools)
72 See \ref pkgcachegen.h for more information about generating cache structures. */
74 #ifndef PKGLIB_PKGCACHE_H
75 #define PKGLIB_PKGCACHE_H
79 #include <apt-pkg/mmap.h>
81 #ifndef APT_8_CLEANER_HEADERS
85 class pkgVersioningSystem
;
86 class pkgCache
/*{{{*/
89 // Cache element predeclarations
103 template<typename Str
, typename Itr
> class Iterator
;
110 class PkgFileIterator
;
111 class VerFileIterator
;
112 class DescFileIterator
;
116 // These are all the constants used in the cache structures
118 // WARNING - if you change these lists you must also edit
119 // the stringification in pkgcache.cc and also consider whether
120 // the cache file will become incompatible.
123 enum DepType
{Depends
=1,PreDepends
=2,Suggests
=3,Recommends
=4,
124 Conflicts
=5,Replaces
=6,Obsoletes
=7,DpkgBreaks
=8,Enhances
=9};
125 /** \brief available compare operators
127 The lower 4 bits are used to indicate what operator is being specified and
128 the upper 4 bits are flags. OR indicates that the next package is
129 or'd with the current package. */
130 enum DepCompareOp
{Or
=0x10,NoOp
=0,LessEq
=0x1,GreaterEq
=0x2,Less
=0x3,
131 Greater
=0x4,Equals
=0x5,NotEquals
=0x6};
136 /** \brief priority of a package version
138 Zero is used for unparsable or absent Priority fields. */
139 enum VerPriority
{Important
=1,Required
=2,Standard
=3,Optional
=4,Extra
=5};
140 enum PkgSelectedState
{Unknown
=0,Install
=1,Hold
=2,DeInstall
=3,Purge
=4};
141 enum PkgInstState
{Ok
=0,ReInstReq
=1,HoldInst
=2,HoldReInstReq
=3};
142 enum PkgCurrentState
{NotInstalled
=0,UnPacked
=1,HalfConfigured
=2,
143 HalfInstalled
=4,ConfigFiles
=5,Installed
=6,
144 TriggersAwaited
=7,TriggersPending
=8};
149 enum PkgFlags
{Auto
=(1<<0),Essential
=(1<<3),Important
=(1<<4)};
150 enum PkgFFlags
{NotSource
=(1<<0),NotAutomatic
=(1<<1),ButAutomaticUpgrades
=(1<<2)};
155 // Memory mapped cache file
156 std::string CacheFile
;
159 unsigned long sHash(const std::string
&S
) const;
160 unsigned long sHash(const char *S
) const;
164 // Pointers to the arrays of items
170 PackageFile
*PkgFileP
;
175 StringItem
*StringItemP
;
178 virtual bool ReMap(bool const &Errorchecks
= true);
179 inline bool Sync() {return Map
.Sync();};
180 inline MMap
&GetMap() {return Map
;};
181 inline void *DataEnd() {return ((unsigned char *)Map
.Data()) + Map
.Size();};
183 // String hashing function (512 range)
184 inline unsigned long Hash(const std::string
&S
) const {return sHash(S
);};
185 inline unsigned long Hash(const char *S
) const {return sHash(S
);};
187 // Useful transformation things
188 const char *Priority(unsigned char Priority
);
191 GrpIterator
FindGrp(const std::string
&Name
);
192 PkgIterator
FindPkg(const std::string
&Name
);
193 PkgIterator
FindPkg(const std::string
&Name
, const std::string
&Arch
);
195 Header
&Head() {return *HeaderP
;};
196 inline GrpIterator
GrpBegin();
197 inline GrpIterator
GrpEnd();
198 inline PkgIterator
PkgBegin();
199 inline PkgIterator
PkgEnd();
200 inline PkgFileIterator
FileBegin();
201 inline PkgFileIterator
FileEnd();
203 inline bool MultiArchCache() const { return MultiArchEnabled
; };
204 inline char const * const NativeArch() const;
206 // Make me a function
207 pkgVersioningSystem
*VS
;
210 static const char *CompTypeDeb(unsigned char Comp
);
211 static const char *CompType(unsigned char Comp
);
212 static const char *DepType(unsigned char Dep
);
214 pkgCache(MMap
*Map
,bool DoMap
= true);
215 virtual ~pkgCache() {};
218 bool MultiArchEnabled
;
219 PkgIterator
SingleArchFindPkg(const std::string
&Name
);
222 // Header structure /*{{{*/
223 struct pkgCache::Header
225 /** \brief Signature information
227 This must contain the hex value 0x98FE76DC which is designed to
228 verify that the system loading the image has the same byte order
229 and byte size as the system saving the image */
230 unsigned long Signature
;
231 /** These contain the version of the cache file */
234 /** \brief indicates if the cache should be erased
236 Dirty is true if the cache file was opened for reading, the client
237 expects to have written things to it and have not fully synced it.
238 The file should be erased and rebuilt if it is true. */
241 /** \brief Size of structure values
243 All *Sz variables contains the sizeof() that particular structure.
244 It is used as an extra consistency check on the structure of the file.
246 If any of the size values do not exactly match what the client expects
247 then the client should refuse the load the file. */
248 unsigned short HeaderSz
;
249 unsigned short GroupSz
;
250 unsigned short PackageSz
;
251 unsigned short PackageFileSz
;
252 unsigned short VersionSz
;
253 unsigned short DescriptionSz
;
254 unsigned short DependencySz
;
255 unsigned short ProvidesSz
;
256 unsigned short VerFileSz
;
257 unsigned short DescFileSz
;
259 /** \brief Structure counts
261 These indicate the number of each structure contained in the cache.
262 PackageCount is especially useful for generating user state structures.
263 See Package::Id for more info. */
264 unsigned long GroupCount
;
265 unsigned long PackageCount
;
266 unsigned long VersionCount
;
267 unsigned long DescriptionCount
;
268 unsigned long DependsCount
;
269 unsigned long PackageFileCount
;
270 unsigned long VerFileCount
;
271 unsigned long DescFileCount
;
272 unsigned long ProvidesCount
;
274 /** \brief index of the first PackageFile structure
276 The PackageFile structures are singly linked lists that represent
277 all package files that have been merged into the cache. */
279 /** \brief index of the first StringItem structure
281 The cache contains a list of all the unique strings (StringItems).
282 The parser reads this list into memory so it can match strings
284 map_ptrloc StringList
;
285 /** \brief String representing the version system used */
286 map_ptrloc VerSysName
;
287 /** \brief Architecture(s) the cache was built against */
288 map_ptrloc Architecture
;
289 /** \brief The maximum size of a raw entry from the original Package file */
290 unsigned long MaxVerFileSize
;
291 /** \brief The maximum size of a raw entry from the original Translation file */
292 unsigned long MaxDescFileSize
;
294 /** \brief The Pool structures manage the allocation pools that the generator uses
296 Start indicates the first byte of the pool, Count is the number of objects
297 remaining in the pool and ItemSize is the structure size (alignment factor)
298 of the pool. An ItemSize of 0 indicates the pool is empty. There should be
299 the same number of pools as there are structure types. The generator
300 stores this information so future additions can make use of any unused pool
302 DynamicMMap::Pool Pools
[9];
304 /** \brief hash tables providing rapid group/package name lookup
306 Each group/package name is inserted into the hash table using pkgCache::Hash(const &string)
307 By iterating over each entry in the hash table it is possible to iterate over
308 the entire list of packages. Hash Collisions are handled with a singly linked
309 list of packages based at the hash item. The linked list contains only
310 packages that match the hashing function.
311 In the PkgHashTable is it possible that multiple packages have the same name -
312 these packages are stored as a sequence in the list.
314 Beware: The Hashmethod assumes that the hash table sizes are equal */
315 map_ptrloc PkgHashTable
[2*1048];
316 map_ptrloc GrpHashTable
[2*1048];
318 /** \brief Size of the complete cache file */
319 unsigned long CacheFileSize
;
321 bool CheckSizes(Header
&Against
) const;
325 // Group structure /*{{{*/
326 /** \brief groups architecture depending packages together
328 On or more packages with the same name form a group, so we have
329 a simple way to access a package built for different architectures
330 Group exists in a singly linked list of group records starting at
331 the hash index of the name in the pkgCache::Header::GrpHashTable */
332 struct pkgCache::Group
334 /** \brief Name of the group */
335 map_ptrloc Name
; // StringItem
338 /** \brief Link to the first package which belongs to the group */
339 map_ptrloc FirstPackage
; // Package
340 /** \brief Link to the last package which belongs to the group */
341 map_ptrloc LastPackage
; // Package
342 /** \brief Link to the next Group */
343 map_ptrloc Next
; // Group
344 /** \brief unique sequel ID */
349 // Package structure /*{{{*/
350 /** \brief contains information for a single unique package
352 There can be any number of versions of a given package.
353 Package exists in a singly linked list of package records starting at
354 the hash index of the name in the pkgCache::Header::PkgHashTable
356 A package can be created for every architecture so package names are
357 not unique, but it is guaranteed that packages with the same name
358 are sequencel ordered in the list. Packages with the same name can be
359 accessed with the Group.
361 struct pkgCache::Package
363 /** \brief Name of the package */
364 map_ptrloc Name
; // StringItem
365 /** \brief Architecture of the package */
366 map_ptrloc Arch
; // StringItem
367 /** \brief Base of a singly linked list of versions
369 Each structure represents a unique version of the package.
370 The version structures contain links into PackageFile and the
371 original text file as well as detailed information about the size
372 and dependencies of the specific package. In this way multiple
373 versions of a package can be cleanly handled by the system.
374 Furthermore, this linked list is guaranteed to be sorted
375 from Highest version to lowest version with no duplicate entries. */
376 map_ptrloc VersionList
; // Version
377 /** \brief index to the installed version */
378 map_ptrloc CurrentVer
; // Version
379 /** \brief indicates the deduced section
381 Should be the index to the string "Unknown" or to the section
382 of the last parsed item. */
383 map_ptrloc Section
; // StringItem
384 /** \brief index of the group this package belongs to */
385 map_ptrloc Group
; // Group the Package belongs to
388 /** \brief Link to the next package in the same bucket */
389 map_ptrloc NextPackage
; // Package
390 /** \brief List of all dependencies on this package */
391 map_ptrloc RevDepends
; // Dependency
392 /** \brief List of all "packages" this package provide */
393 map_ptrloc ProvidesList
; // Provides
395 // Install/Remove/Purge etc
396 /** \brief state that the user wishes the package to be in */
397 unsigned char SelectedState
; // What
398 /** \brief installation state of the package
400 This should be "ok" but in case the installation failed
401 it will be different.
403 unsigned char InstState
; // Flags
404 /** \brief indicates if the package is installed */
405 unsigned char CurrentState
; // State
407 /** \brief unique sequel ID
409 ID is a unique value from 0 to Header->PackageCount assigned by the generator.
410 This allows clients to create an array of size PackageCount and use it to store
411 state information for the package map. For instance the status file emitter uses
412 this to track which packages have been emitted already. */
414 /** \brief some useful indicators of the package's state */
418 // Package File structure /*{{{*/
419 /** \brief stores information about the files used to generate the cache
421 Package files are referenced by Version structures to be able to know
422 after the generation still from which Packages file includes this Version
423 as we need this information later on e.g. for pinning. */
424 struct pkgCache::PackageFile
426 /** \brief physical disk file that this PackageFile represents */
427 map_ptrloc FileName
; // StringItem
428 /** \brief the release information
430 Please see the files document for a description of what the
431 release information means. */
432 map_ptrloc Archive
; // StringItem
433 map_ptrloc Codename
; // StringItem
434 map_ptrloc Component
; // StringItem
435 map_ptrloc Version
; // StringItem
436 map_ptrloc Origin
; // StringItem
437 map_ptrloc Label
; // StringItem
438 map_ptrloc Architecture
; // StringItem
439 /** \brief The site the index file was fetched from */
440 map_ptrloc Site
; // StringItem
441 /** \brief indicates what sort of index file this is
443 @TODO enumerate at least the possible indexes */
444 map_ptrloc IndexType
; // StringItem
445 /** \brief Size of the file
447 Used together with the modification time as a
448 simple check to ensure that the Packages
449 file has not been altered since Cache generation. */
451 /** \brief Modification time for the file */
454 /** @TODO document PackageFile::Flags */
458 /** \brief Link to the next PackageFile in the Cache */
459 map_ptrloc NextFile
; // PackageFile
460 /** \brief unique sequel ID */
464 // VerFile structure /*{{{*/
465 /** \brief associates a version with a PackageFile
467 This allows a full description of all Versions in all files
468 (and hence all sources) under consideration. */
469 struct pkgCache::VerFile
471 /** \brief index of the package file that this version was found in */
472 map_ptrloc File
; // PackageFile
473 /** \brief next step in the linked list */
474 map_ptrloc NextFile
; // PkgVerFile
475 /** \brief position in the package file */
476 map_ptrloc Offset
; // File offset
477 /** @TODO document pkgCache::VerFile::Size */
481 // DescFile structure /*{{{*/
482 /** \brief associates a description with a Translation file */
483 struct pkgCache::DescFile
485 /** \brief index of the file that this description was found in */
486 map_ptrloc File
; // PackageFile
487 /** \brief next step in the linked list */
488 map_ptrloc NextFile
; // PkgVerFile
489 /** \brief position in the file */
490 map_ptrloc Offset
; // File offset
491 /** @TODO document pkgCache::DescFile::Size */
495 // Version structure /*{{{*/
496 /** \brief information for a single version of a package
498 The version list is always sorted from highest version to lowest
499 version by the generator. Equal version numbers are either merged
500 or handled as separate versions based on the Hash value. */
501 struct pkgCache::Version
503 /** \brief complete version string */
504 map_ptrloc VerStr
; // StringItem
505 /** \brief section this version is filled in */
506 map_ptrloc Section
; // StringItem
508 /** \brief Multi-Arch capabilities of a package version */
509 enum VerMultiArch
{ None
= 0, /*!< is the default and doesn't trigger special behaviour */
510 All
= (1<<0), /*!< will cause that Ver.Arch() will report "all" */
511 Foreign
= (1<<1), /*!< can satisfy dependencies in another architecture */
512 Same
= (1<<2), /*!< can be co-installed with itself from other architectures */
513 Allowed
= (1<<3), /*!< other packages are allowed to depend on thispkg:any */
514 AllForeign
= All
| Foreign
,
515 AllAllowed
= All
| Allowed
};
516 /** \brief stores the MultiArch capabilities of this version
518 Flags used are defined in pkgCache::Version::VerMultiArch
520 unsigned char MultiArch
;
522 /** \brief references all the PackageFile's that this version came from
524 FileList can be used to determine what distribution(s) the Version
525 applies to. If FileList is 0 then this is a blank version.
526 The structure should also have a 0 in all other fields excluding
527 pkgCache::Version::VerStr and Possibly pkgCache::Version::NextVer. */
528 map_ptrloc FileList
; // VerFile
529 /** \brief next (lower or equal) version in the linked list */
530 map_ptrloc NextVer
; // Version
531 /** \brief next description in the linked list */
532 map_ptrloc DescriptionList
; // Description
533 /** \brief base of the dependency list */
534 map_ptrloc DependsList
; // Dependency
535 /** \brief links to the owning package
537 This allows reverse dependencies to determine the package */
538 map_ptrloc ParentPkg
; // Package
539 /** \brief list of pkgCache::Provides */
540 map_ptrloc ProvidesList
; // Provides
542 /** \brief archive size for this version
544 For Debian this is the size of the .deb file. */
545 unsigned long long Size
; // These are the .deb size
546 /** \brief uncompressed size for this version */
547 unsigned long long InstalledSize
;
548 /** \brief characteristic value representing this version
550 No two packages in existence should have the same VerStr
551 and Hash with different contents. */
553 /** \brief unique sequel ID */
555 /** \brief parsed priority value */
556 unsigned char Priority
;
559 // Description structure /*{{{*/
560 /** \brief datamember of a linked list of available description for a version */
561 struct pkgCache::Description
563 /** \brief Language code of this description (translation)
565 If the value has a 0 length then this is read using the Package
566 file else the Translation-CODE file is used. */
567 map_ptrloc language_code
; // StringItem
568 /** \brief MD5sum of the original description
570 Used to map Translations of a description to a version
571 and to check that the Translation is up-to-date. */
572 map_ptrloc md5sum
; // StringItem
574 /** @TODO document pkgCache::Description::FileList */
575 map_ptrloc FileList
; // DescFile
576 /** \brief next translation for this description */
577 map_ptrloc NextDesc
; // Description
578 /** \brief the text is a description of this package */
579 map_ptrloc ParentPkg
; // Package
581 /** \brief unique sequel ID */
585 // Dependency structure /*{{{*/
586 /** \brief information for a single dependency record
588 The records are split up like this to ease processing by the client.
589 The base of the linked list is pkgCache::Version::DependsList.
590 All forms of dependencies are recorded here including Depends,
591 Recommends, Suggests, Enhances, Conflicts, Replaces and Breaks. */
592 struct pkgCache::Dependency
594 /** \brief string of the version the dependency is applied against */
595 map_ptrloc Version
; // StringItem
596 /** \brief index of the package this depends applies to
598 The generator will - if the package does not already exist -
599 create a blank (no version records) package. */
600 map_ptrloc Package
; // Package
601 /** \brief next dependency of this version */
602 map_ptrloc NextDepends
; // Dependency
603 /** \brief next reverse dependency of this package */
604 map_ptrloc NextRevDepends
; // Dependency
605 /** \brief version of the package which has the reverse depends */
606 map_ptrloc ParentVer
; // Version
608 /** \brief unique sequel ID */
610 /** \brief Dependency type - Depends, Recommends, Conflicts, etc */
612 /** \brief comparison operator specified on the depends line
614 If the high bit is set then it is a logical OR with the previous record. */
615 unsigned char CompareOp
;
618 // Provides structure /*{{{*/
619 /** \brief handles virtual packages
621 When a Provides: line is encountered a new provides record is added
622 associating the package with a virtual package name.
623 The provides structures are linked off the package structures.
624 This simplifies the analysis of dependencies and other aspects A provides
625 refers to a specific version of a specific package, not all versions need to
626 provide that provides.*/
627 struct pkgCache::Provides
629 /** \brief index of the package providing this */
630 map_ptrloc ParentPkg
; // Package
631 /** \brief index of the version this provide line applies to */
632 map_ptrloc Version
; // Version
633 /** \brief version in the provides line (if any)
635 This version allows dependencies to depend on specific versions of a
636 Provides, as well as allowing Provides to override existing packages.
637 This is experimental. Note that Debian doesn't allow versioned provides */
638 map_ptrloc ProvideVersion
; // StringItem
639 /** \brief next provides (based of package) */
640 map_ptrloc NextProvides
; // Provides
641 /** \brief next provides (based of version) */
642 map_ptrloc NextPkgProv
; // Provides
645 // StringItem structure /*{{{*/
646 /** \brief used for generating single instances of strings
648 Some things like Section Name are are useful to have as unique tags.
649 It is part of a linked list based at pkgCache::Header::StringList
651 All strings are simply inlined any place in the file that is natural
652 for the writer. The client should make no assumptions about the positioning
653 of strings. All StringItems should be null-terminated. */
654 struct pkgCache::StringItem
656 /** \brief string this refers to */
657 map_ptrloc String
; // StringItem
658 /** \brief Next link in the chain */
659 map_ptrloc NextItem
; // StringItem
664 inline char const * const pkgCache::NativeArch() const
665 { return StrP
+ HeaderP
->Architecture
; };
667 #include <apt-pkg/cacheiterators.h>
669 inline pkgCache::GrpIterator
pkgCache::GrpBegin()
670 {return GrpIterator(*this);};
671 inline pkgCache::GrpIterator
pkgCache::GrpEnd()
672 {return GrpIterator(*this,GrpP
);};
673 inline pkgCache::PkgIterator
pkgCache::PkgBegin()
674 {return PkgIterator(*this);};
675 inline pkgCache::PkgIterator
pkgCache::PkgEnd()
676 {return PkgIterator(*this,PkgP
);};
677 inline pkgCache::PkgFileIterator
pkgCache::FileBegin()
678 {return PkgFileIterator(*this,PkgFileP
+ HeaderP
->FileList
);};
679 inline pkgCache::PkgFileIterator
pkgCache::FileEnd()
680 {return PkgFileIterator(*this,PkgFileP
);};
682 // Oh I wish for Real Name Space Support
683 class pkgCache::Namespace
/*{{{*/
686 typedef pkgCache::GrpIterator GrpIterator
;
687 typedef pkgCache::PkgIterator PkgIterator
;
688 typedef pkgCache::VerIterator VerIterator
;
689 typedef pkgCache::DescIterator DescIterator
;
690 typedef pkgCache::DepIterator DepIterator
;
691 typedef pkgCache::PrvIterator PrvIterator
;
692 typedef pkgCache::PkgFileIterator PkgFileIterator
;
693 typedef pkgCache::VerFileIterator VerFileIterator
;
694 typedef pkgCache::Version Version
;
695 typedef pkgCache::Description Description
;
696 typedef pkgCache::Package Package
;
697 typedef pkgCache::Header Header
;
698 typedef pkgCache::Dep Dep
;
699 typedef pkgCache::Flag Flag
;