1 // -*- mode: cpp; mode: fold -*-
4 \brief pkgCache - Structure definitions for the cache file
6 The goal of the cache file is two fold:
7 Firstly to speed loading and processing of the package file array and
8 secondly to reduce memory consumption of the package file array.
10 The implementation is aimed at an environment with many primary package
11 files, for instance someone that has a Package file for their CD-ROM, a
12 Package file for the latest version of the distribution on the CD-ROM and a
13 package file for the development version. Always present is the information
14 contained in the status file which might be considered a separate package
17 Please understand, this is designed as a <b>Cache file</b> it is not meant to be
18 used on any system other than the one it was created for. It is not meant to
19 be authoritative either, i.e. if a system crash or software failure occurs it
20 must be perfectly acceptable for the cache file to be in an inconsistent
21 state. Furthermore at any time the cache file may be erased without losing
24 Also the structures and storage layout is optimized for use by the APT
25 and may not be suitable for all purposes. However it should be possible
26 to extend it with associate cache files that contain other information.
28 To keep memory use down the cache file only contains often used fields and
29 fields that are inexpensive to store, the Package file has a full list of
30 fields. Also the client may assume that all items are perfectly valid and
31 need not perform checks against their correctness. Removal of information
32 from the cache is possible, but blanks will be left in the file, and
33 unused strings will also be present. The recommended implementation is to
34 simply rebuild the cache each time any of the data files change. It is
35 possible to add a new package file to the cache without any negative side
38 <b>Note on Pointer access</b>
39 Clients should always use the CacheIterators classes for access to the
40 cache and the data in it. They also provide a simple STL-like method for
41 traversing the links of the datastructure.
43 Every item in every structure is stored as the index to that structure.
44 What this means is that once the files is mmaped every data access has to
45 go through a fix up stage to get a real memory pointer. This is done
46 by taking the index, multiplying it by the type size and then adding
47 it to the start address of the memory block. This sounds complex, but
48 in C it is a single array dereference. Because all items are aligned to
49 their size and indexes are stored as multiples of the size of the structure
50 the format is immediately portable to all possible architectures - BUT the
51 generated files are -NOT-.
53 This scheme allows code like this to be written:
55 void *Map = mmap(...);
56 Package *PkgList = (Package *)Map;
57 Header *Head = (Header *)Map;
58 char *Strings = (char *)Map;
59 cout << (Strings + PkgList[Head->HashTable[0]]->Name) << endl;
61 Notice the lack of casting or multiplication. The net result is to return
62 the name of the first package in the first hash bucket, without error
65 The generator uses allocation pools to group similarly sized structures in
66 large blocks to eliminate any alignment overhead. The generator also
67 assures that no structures overlap and all indexes are unique. Although
68 at first glance it may seem like there is the potential for two structures
69 to exist at the same point the generator never allows this to happen.
70 (See the discussion of free space pools)
72 See \ref pkgcachegen.h for more information about generating cache structures. */
74 #ifndef PKGLIB_PKGCACHE_H
75 #define PKGLIB_PKGCACHE_H
80 #include <apt-pkg/mmap.h>
84 class pkgVersioningSystem
;
85 class pkgCache
/*{{{*/
88 // Cache element predeclarations
102 template<typename Str
, typename Itr
> class Iterator
;
109 class PkgFileIterator
;
110 class VerFileIterator
;
111 class DescFileIterator
;
115 // These are all the constants used in the cache structures
117 // WARNING - if you change these lists you must also edit
118 // the stringification in pkgcache.cc and also consider whether
119 // the cache file will become incompatible.
122 enum DepType
{Depends
=1,PreDepends
=2,Suggests
=3,Recommends
=4,
123 Conflicts
=5,Replaces
=6,Obsoletes
=7,DpkgBreaks
=8,Enhances
=9};
124 /** \brief available compare operators
126 The lower 4 bits are used to indicate what operator is being specified and
127 the upper 4 bits are flags. OR indicates that the next package is
128 or'd with the current package. */
129 enum DepCompareOp
{Or
=0x10,NoOp
=0,LessEq
=0x1,GreaterEq
=0x2,Less
=0x3,
130 Greater
=0x4,Equals
=0x5,NotEquals
=0x6};
135 /** \brief priority of a package version
137 Zero is used for unparsable or absent Priority fields. */
138 enum VerPriority
{Important
=1,Required
=2,Standard
=3,Optional
=4,Extra
=5};
139 enum PkgSelectedState
{Unknown
=0,Install
=1,Hold
=2,DeInstall
=3,Purge
=4};
140 enum PkgInstState
{Ok
=0,ReInstReq
=1,HoldInst
=2,HoldReInstReq
=3};
141 enum PkgCurrentState
{NotInstalled
=0,UnPacked
=1,HalfConfigured
=2,
142 HalfInstalled
=4,ConfigFiles
=5,Installed
=6,
143 TriggersAwaited
=7,TriggersPending
=8};
148 enum PkgFlags
{Auto
=(1<<0),Essential
=(1<<3),Important
=(1<<4)};
149 enum PkgFFlags
{NotSource
=(1<<0),NotAutomatic
=(1<<1),ButAutomaticUpgrades
=(1<<2)};
154 // Memory mapped cache file
158 unsigned long sHash(const string
&S
) const;
159 unsigned long sHash(const char *S
) const;
163 // Pointers to the arrays of items
169 PackageFile
*PkgFileP
;
174 StringItem
*StringItemP
;
177 virtual bool ReMap(bool const &Errorchecks
= true);
178 inline bool Sync() {return Map
.Sync();};
179 inline MMap
&GetMap() {return Map
;};
180 inline void *DataEnd() {return ((unsigned char *)Map
.Data()) + Map
.Size();};
182 // String hashing function (512 range)
183 inline unsigned long Hash(const string
&S
) const {return sHash(S
);};
184 inline unsigned long Hash(const char *S
) const {return sHash(S
);};
186 // Useful transformation things
187 const char *Priority(unsigned char Priority
);
190 GrpIterator
FindGrp(const string
&Name
);
191 PkgIterator
FindPkg(const string
&Name
);
192 PkgIterator
FindPkg(const string
&Name
, const string
&Arch
);
194 Header
&Head() {return *HeaderP
;};
195 inline GrpIterator
GrpBegin();
196 inline GrpIterator
GrpEnd();
197 inline PkgIterator
PkgBegin();
198 inline PkgIterator
PkgEnd();
199 inline PkgFileIterator
FileBegin();
200 inline PkgFileIterator
FileEnd();
202 inline bool MultiArchCache() const { return MultiArchEnabled
; };
204 // Make me a function
205 pkgVersioningSystem
*VS
;
208 static const char *CompTypeDeb(unsigned char Comp
);
209 static const char *CompType(unsigned char Comp
);
210 static const char *DepType(unsigned char Dep
);
212 pkgCache(MMap
*Map
,bool DoMap
= true);
213 virtual ~pkgCache() {};
216 bool MultiArchEnabled
;
217 PkgIterator
SingleArchFindPkg(const string
&Name
);
218 inline char const * const NativeArch() const;
221 // Header structure /*{{{*/
222 struct pkgCache::Header
224 /** \brief Signature information
226 This must contain the hex value 0x98FE76DC which is designed to
227 verify that the system loading the image has the same byte order
228 and byte size as the system saving the image */
229 unsigned long Signature
;
230 /** These contain the version of the cache file */
233 /** \brief indicates if the cache should be erased
235 Dirty is true if the cache file was opened for reading, the client
236 expects to have written things to it and have not fully synced it.
237 The file should be erased and rebuilt if it is true. */
240 /** \brief Size of structure values
242 All *Sz variables contains the sizeof() that particular structure.
243 It is used as an extra consistency check on the structure of the file.
245 If any of the size values do not exactly match what the client expects
246 then the client should refuse the load the file. */
247 unsigned short HeaderSz
;
248 unsigned short GroupSz
;
249 unsigned short PackageSz
;
250 unsigned short PackageFileSz
;
251 unsigned short VersionSz
;
252 unsigned short DescriptionSz
;
253 unsigned short DependencySz
;
254 unsigned short ProvidesSz
;
255 unsigned short VerFileSz
;
256 unsigned short DescFileSz
;
258 /** \brief Structure counts
260 These indicate the number of each structure contained in the cache.
261 PackageCount is especially useful for generating user state structures.
262 See Package::Id for more info. */
263 unsigned long GroupCount
;
264 unsigned long PackageCount
;
265 unsigned long VersionCount
;
266 unsigned long DescriptionCount
;
267 unsigned long DependsCount
;
268 unsigned long PackageFileCount
;
269 unsigned long VerFileCount
;
270 unsigned long DescFileCount
;
271 unsigned long ProvidesCount
;
273 /** \brief index of the first PackageFile structure
275 The PackageFile structures are singly linked lists that represent
276 all package files that have been merged into the cache. */
278 /** \brief index of the first StringItem structure
280 The cache contains a list of all the unique strings (StringItems).
281 The parser reads this list into memory so it can match strings
283 map_ptrloc StringList
;
284 /** \brief String representing the version system used */
285 map_ptrloc VerSysName
;
286 /** \brief Architecture(s) the cache was built against */
287 map_ptrloc Architecture
;
288 /** \brief The maximum size of a raw entry from the original Package file */
289 unsigned long MaxVerFileSize
;
290 /** \brief The maximum size of a raw entry from the original Translation file */
291 unsigned long MaxDescFileSize
;
293 /** \brief The Pool structures manage the allocation pools that the generator uses
295 Start indicates the first byte of the pool, Count is the number of objects
296 remaining in the pool and ItemSize is the structure size (alignment factor)
297 of the pool. An ItemSize of 0 indicates the pool is empty. There should be
298 the same number of pools as there are structure types. The generator
299 stores this information so future additions can make use of any unused pool
301 DynamicMMap::Pool Pools
[9];
303 /** \brief hash tables providing rapid group/package name lookup
305 Each group/package name is inserted into the hash table using pkgCache::Hash(const &string)
306 By iterating over each entry in the hash table it is possible to iterate over
307 the entire list of packages. Hash Collisions are handled with a singly linked
308 list of packages based at the hash item. The linked list contains only
309 packages that match the hashing function.
310 In the PkgHashTable is it possible that multiple packages have the same name -
311 these packages are stored as a sequence in the list.
313 Beware: The Hashmethod assumes that the hash table sizes are equal */
314 map_ptrloc PkgHashTable
[2*1048];
315 map_ptrloc GrpHashTable
[2*1048];
317 bool CheckSizes(Header
&Against
) const;
321 // Group structure /*{{{*/
322 /** \brief groups architecture depending packages together
324 On or more packages with the same name form a group, so we have
325 a simple way to access a package built for different architectures
326 Group exists in a singly linked list of group records starting at
327 the hash index of the name in the pkgCache::Header::GrpHashTable */
328 struct pkgCache::Group
330 /** \brief Name of the group */
331 map_ptrloc Name
; // StringItem
334 /** \brief Link to the first package which belongs to the group */
335 map_ptrloc FirstPackage
; // Package
336 /** \brief Link to the last package which belongs to the group */
337 map_ptrloc LastPackage
; // Package
338 /** \brief Link to the next Group */
339 map_ptrloc Next
; // Group
340 /** \brief unique sequel ID */
345 // Package structure /*{{{*/
346 /** \brief contains information for a single unique package
348 There can be any number of versions of a given package.
349 Package exists in a singly linked list of package records starting at
350 the hash index of the name in the pkgCache::Header::PkgHashTable
352 A package can be created for every architecture so package names are
353 not unique, but it is garanteed that packages with the same name
354 are sequencel ordered in the list. Packages with the same name can be
355 accessed with the Group.
357 struct pkgCache::Package
359 /** \brief Name of the package */
360 map_ptrloc Name
; // StringItem
361 /** \brief Architecture of the package */
362 map_ptrloc Arch
; // StringItem
363 /** \brief Base of a singly linked list of versions
365 Each structure represents a unique version of the package.
366 The version structures contain links into PackageFile and the
367 original text file as well as detailed information about the size
368 and dependencies of the specific package. In this way multiple
369 versions of a package can be cleanly handled by the system.
370 Furthermore, this linked list is guaranteed to be sorted
371 from Highest version to lowest version with no duplicate entries. */
372 map_ptrloc VersionList
; // Version
373 /** \brief index to the installed version */
374 map_ptrloc CurrentVer
; // Version
375 /** \brief indicates the deduced section
377 Should be the index to the string "Unknown" or to the section
378 of the last parsed item. */
379 map_ptrloc Section
; // StringItem
380 /** \brief index of the group this package belongs to */
381 map_ptrloc Group
; // Group the Package belongs to
384 /** \brief Link to the next package in the same bucket */
385 map_ptrloc NextPackage
; // Package
386 /** \brief List of all dependencies on this package */
387 map_ptrloc RevDepends
; // Dependency
388 /** \brief List of all "packages" this package provide */
389 map_ptrloc ProvidesList
; // Provides
391 // Install/Remove/Purge etc
392 /** \brief state that the user wishes the package to be in */
393 unsigned char SelectedState
; // What
394 /** \brief installation state of the package
396 This should be "ok" but in case the installation failed
397 it will be different.
399 unsigned char InstState
; // Flags
400 /** \brief indicates if the package is installed */
401 unsigned char CurrentState
; // State
403 /** \brief unique sequel ID
405 ID is a unique value from 0 to Header->PackageCount assigned by the generator.
406 This allows clients to create an array of size PackageCount and use it to store
407 state information for the package map. For instance the status file emitter uses
408 this to track which packages have been emitted already. */
410 /** \brief some useful indicators of the package's state */
414 // Package File structure /*{{{*/
415 /** \brief stores information about the files used to generate the cache
417 Package files are referenced by Version structures to be able to know
418 after the generation still from which Packages file includes this Version
419 as we need this information later on e.g. for pinning. */
420 struct pkgCache::PackageFile
422 /** \brief physical disk file that this PackageFile represents */
423 map_ptrloc FileName
; // StringItem
424 /** \brief the release information
426 Please see the files document for a description of what the
427 release information means. */
428 map_ptrloc Archive
; // StringItem
429 map_ptrloc Codename
; // StringItem
430 map_ptrloc Component
; // StringItem
431 map_ptrloc Version
; // StringItem
432 map_ptrloc Origin
; // StringItem
433 map_ptrloc Label
; // StringItem
434 map_ptrloc Architecture
; // StringItem
435 /** \brief The site the index file was fetched from */
436 map_ptrloc Site
; // StringItem
437 /** \brief indicates what sort of index file this is
439 @TODO enumerate at least the possible indexes */
440 map_ptrloc IndexType
; // StringItem
441 /** \brief Size of the file
443 Used together with the modification time as a
444 simple check to ensure that the Packages
445 file has not been altered since Cache generation. */
447 /** \brief Modification time for the file */
450 /* @TODO document PackageFile::Flags */
454 /** \brief Link to the next PackageFile in the Cache */
455 map_ptrloc NextFile
; // PackageFile
456 /** \brief unique sequel ID */
460 // VerFile structure /*{{{*/
461 /** \brief associates a version with a PackageFile
463 This allows a full description of all Versions in all files
464 (and hence all sources) under consideration. */
465 struct pkgCache::VerFile
467 /** \brief index of the package file that this version was found in */
468 map_ptrloc File
; // PackageFile
469 /** \brief next step in the linked list */
470 map_ptrloc NextFile
; // PkgVerFile
471 /** \brief position in the package file */
472 map_ptrloc Offset
; // File offset
473 /* @TODO document pkgCache::VerFile::Size */
477 // DescFile structure /*{{{*/
478 /** \brief associates a description with a Translation file */
479 struct pkgCache::DescFile
481 /** \brief index of the file that this description was found in */
482 map_ptrloc File
; // PackageFile
483 /** \brief next step in the linked list */
484 map_ptrloc NextFile
; // PkgVerFile
485 /** \brief position in the file */
486 map_ptrloc Offset
; // File offset
487 /* @TODO document pkgCache::DescFile::Size */
491 // Version structure /*{{{*/
492 /** \brief information for a single version of a package
494 The version list is always sorted from highest version to lowest
495 version by the generator. Equal version numbers are either merged
496 or handled as separate versions based on the Hash value. */
497 struct pkgCache::Version
499 /** \brief complete version string */
500 map_ptrloc VerStr
; // StringItem
501 /** \brief section this version is filled in */
502 map_ptrloc Section
; // StringItem
503 /** \brief stores the MultiArch capabilities of this version
505 None is the default and doesn't trigger special behaviour,
506 Foreign means that this version can fulfill dependencies even
507 if it is built for another architecture as the requester.
508 Same indicates that builds for different architectures can
509 be co-installed on the system */
510 /* FIXME: A bitflag would be better with the next abibreak… */
511 enum {None
, All
, Foreign
, Same
, Allowed
, AllForeign
, AllAllowed
} MultiArch
;
513 /** \brief references all the PackageFile's that this version came from
515 FileList can be used to determine what distribution(s) the Version
516 applies to. If FileList is 0 then this is a blank version.
517 The structure should also have a 0 in all other fields excluding
518 pkgCache::Version::VerStr and Possibly pkgCache::Version::NextVer. */
519 map_ptrloc FileList
; // VerFile
520 /** \brief next (lower or equal) version in the linked list */
521 map_ptrloc NextVer
; // Version
522 /** \brief next description in the linked list */
523 map_ptrloc DescriptionList
; // Description
524 /** \brief base of the dependency list */
525 map_ptrloc DependsList
; // Dependency
526 /** \brief links to the owning package
528 This allows reverse dependencies to determine the package */
529 map_ptrloc ParentPkg
; // Package
530 /** \brief list of pkgCache::Provides */
531 map_ptrloc ProvidesList
; // Provides
533 /** \brief archive size for this version
535 For Debian this is the size of the .deb file. */
536 unsigned long long Size
; // These are the .deb size
537 /** \brief uncompressed size for this version */
538 unsigned long long InstalledSize
;
539 /** \brief characteristic value representing this version
541 No two packages in existence should have the same VerStr
542 and Hash with different contents. */
544 /** \brief unique sequel ID */
546 /** \brief parsed priority value */
547 unsigned char Priority
;
550 // Description structure /*{{{*/
551 /** \brief datamember of a linked list of available description for a version */
552 struct pkgCache::Description
554 /** \brief Language code of this description (translation)
556 If the value has a 0 length then this is read using the Package
557 file else the Translation-CODE file is used. */
558 map_ptrloc language_code
; // StringItem
559 /** \brief MD5sum of the original description
561 Used to map Translations of a description to a version
562 and to check that the Translation is up-to-date. */
563 map_ptrloc md5sum
; // StringItem
565 /* @TODO document pkgCache::Description::FileList */
566 map_ptrloc FileList
; // DescFile
567 /** \brief next translation for this description */
568 map_ptrloc NextDesc
; // Description
569 /** \brief the text is a description of this package */
570 map_ptrloc ParentPkg
; // Package
572 /** \brief unique sequel ID */
576 // Dependency structure /*{{{*/
577 /** \brief information for a single dependency record
579 The records are split up like this to ease processing by the client.
580 The base of the linked list is pkgCache::Version::DependsList.
581 All forms of dependencies are recorded here including Depends,
582 Recommends, Suggests, Enhances, Conflicts, Replaces and Breaks. */
583 struct pkgCache::Dependency
585 /** \brief string of the version the dependency is applied against */
586 map_ptrloc Version
; // StringItem
587 /** \brief index of the package this depends applies to
589 The generator will - if the package does not already exist -
590 create a blank (no version records) package. */
591 map_ptrloc Package
; // Package
592 /** \brief next dependency of this version */
593 map_ptrloc NextDepends
; // Dependency
594 /** \brief next reverse dependency of this package */
595 map_ptrloc NextRevDepends
; // Dependency
596 /** \brief version of the package which has the reverse depends */
597 map_ptrloc ParentVer
; // Version
599 /** \brief unique sequel ID */
601 /** \brief Dependency type - Depends, Recommends, Conflicts, etc */
603 /** \brief comparison operator specified on the depends line
605 If the high bit is set then it is a logical OR with the previous record. */
606 unsigned char CompareOp
;
609 // Provides structure /*{{{*/
610 /** \brief handles virtual packages
612 When a Provides: line is encountered a new provides record is added
613 associating the package with a virtual package name.
614 The provides structures are linked off the package structures.
615 This simplifies the analysis of dependencies and other aspects A provides
616 refers to a specific version of a specific package, not all versions need to
617 provide that provides.*/
618 struct pkgCache::Provides
620 /** \brief index of the package providing this */
621 map_ptrloc ParentPkg
; // Package
622 /** \brief index of the version this provide line applies to */
623 map_ptrloc Version
; // Version
624 /** \brief version in the provides line (if any)
626 This version allows dependencies to depend on specific versions of a
627 Provides, as well as allowing Provides to override existing packages.
628 This is experimental. Note that Debian doesn't allow versioned provides */
629 map_ptrloc ProvideVersion
; // StringItem
630 /** \brief next provides (based of package) */
631 map_ptrloc NextProvides
; // Provides
632 /** \brief next provides (based of version) */
633 map_ptrloc NextPkgProv
; // Provides
636 // StringItem structure /*{{{*/
637 /** \brief used for generating single instances of strings
639 Some things like Section Name are are useful to have as unique tags.
640 It is part of a linked list based at pkgCache::Header::StringList
642 All strings are simply inlined any place in the file that is natural
643 for the writer. The client should make no assumptions about the positioning
644 of strings. All StringItems should be null-terminated. */
645 struct pkgCache::StringItem
647 /** \brief string this refers to */
648 map_ptrloc String
; // StringItem
649 /** \brief Next link in the chain */
650 map_ptrloc NextItem
; // StringItem
655 inline char const * const pkgCache::NativeArch() const
656 { return StrP
+ HeaderP
->Architecture
; };
658 #include <apt-pkg/cacheiterators.h>
660 inline pkgCache::GrpIterator
pkgCache::GrpBegin()
661 {return GrpIterator(*this);};
662 inline pkgCache::GrpIterator
pkgCache::GrpEnd()
663 {return GrpIterator(*this,GrpP
);};
664 inline pkgCache::PkgIterator
pkgCache::PkgBegin()
665 {return PkgIterator(*this);};
666 inline pkgCache::PkgIterator
pkgCache::PkgEnd()
667 {return PkgIterator(*this,PkgP
);};
668 inline pkgCache::PkgFileIterator
pkgCache::FileBegin()
669 {return PkgFileIterator(*this,PkgFileP
+ HeaderP
->FileList
);};
670 inline pkgCache::PkgFileIterator
pkgCache::FileEnd()
671 {return PkgFileIterator(*this,PkgFileP
);};
673 // Oh I wish for Real Name Space Support
674 class pkgCache::Namespace
/*{{{*/
677 typedef pkgCache::GrpIterator GrpIterator
;
678 typedef pkgCache::PkgIterator PkgIterator
;
679 typedef pkgCache::VerIterator VerIterator
;
680 typedef pkgCache::DescIterator DescIterator
;
681 typedef pkgCache::DepIterator DepIterator
;
682 typedef pkgCache::PrvIterator PrvIterator
;
683 typedef pkgCache::PkgFileIterator PkgFileIterator
;
684 typedef pkgCache::VerFileIterator VerFileIterator
;
685 typedef pkgCache::Version Version
;
686 typedef pkgCache::Description Description
;
687 typedef pkgCache::Package Package
;
688 typedef pkgCache::Header Header
;
689 typedef pkgCache::Dep Dep
;
690 typedef pkgCache::Flag Flag
;