doc/files.sgml

   1 <!-- -*- mode: sgml; mode: fold -*- -->
   2 <!doctype debiandoc  PUBLIC  "-//DebianDoc//DTD DebianDoc//EN">
   3 <book>
   4 <title>APT Files</title>
   5
   6 <author>Jason Gunthorpe <email>jgg@debian.org</email></author>
   7 <version>$Id: files.sgml,v 1.10 2002/09/15 23:14:34 jgg Exp $</version>
   8
   9 <abstract>
  10 This document describes the complete implementation and format of the
  11 installed APT directory structure. It also serves as guide to how APT
  12 views the Debian archive.
  13 </abstract>
  14
  15 <copyright>
  16 Copyright &copy; Jason Gunthorpe, 1998-1999.
  17 <p>
  18 "APT" and this document are free software; you can redistribute them and/or
  19 modify them under the terms of the GNU General Public License as published
  20 by the Free Software Foundation; either version 2 of the License, or (at your
  21 option) any later version.
  22
  23 <p>
  24 For more details, on Debian GNU/Linux systems, see the file
  25 /usr/doc/copyright/GPL for the full license.
  26 </copyright>
  27
  28 <toc sect>
  29
  30 <chapt>Introduction
  31 <!-- General                                                           {{{ -->
  32 <!-- ===================================================================== -->
  33 <sect>General
  34
  35 <p>
  36 This document serves two purposes. The first is to document the installed
  37 directory structure and the format and purpose of each file. The second
  38 purpose is to document how APT views the Debian archive and deals with
  39 multiple package files.
  40
  41 <p>
  42 The var directory structure is as follows:
  43 <example>
  44   /var/lib/apt/
  45                   lists/
  46                        partial/
  47                   xstatus
  48                   userstatus
  49                   cdroms.list
  50   /var/cache/apt/
  51                   pkgcache.bin
  52                   srcpkgcache.bin
  53                   archives/
  54                           partial/
  55   /etc/apt/
  56            sources.list
  57            apt.conf
  58   /usr/lib/apt/
  59                 methods/
  60                        cdrom
  61                        ftp
  62                        http
  63                        file
  64                        gzip
  65                        copy
  66 </example>
  67
  68 <p>
  69 As is specified in the FHS 2.1 /var/lib/apt is used for application
  70 data that is not expected to be user modified. /var/cache/apt is used
  71 for regeneratable data and is where the package cache and downloaded .debs
  72 go.
  73 </sect>
  74                                                                   <!-- }}} -->
  75
  76 <chapt>Files
  77 <!-- Distribution Source List                                          {{{ -->
  78 <!-- ===================================================================== -->
  79 <sect>Distribution Source list (sources.list)
  80
  81 <p>
  82 The distribution source list is used to locate archives of the debian
  83 distribution. It is designed to support any number of active sources and to
  84 support a mix of source media. The file lists one source per line, with the
  85 fastest source listed first. The format of each line is:
  86
  87 <p>
  88 <var>type uri args</var>
  89
  90 <p>
  91 The first item, <var>type</var>, indicates the format for the remainder
  92 of the line. It is designed to indicate the structure of the distribution
  93 the line is talking about. Currently the only defined value is <em>deb</em>
  94 which indicates a standard debian archive with a dists dir.
  95
  96 <sect1>The deb Type
  97    <p>
  98    The <em>deb</em> type is to be a typical two level debian distributions,
  99    dist/<var>distribution</var>/<var>component</var>. Typically distribution
 100    is one of stable, unstable or frozen while component is one of main,
 101    contrib, non-free or non-us. The format for the deb line is as follows:
 102
 103    <p>
 104    deb <var>uri</var> <var>distribution</var> <var>component</var>
 105    [<var>component</var> ...]
 106
 107    <p>
 108    <var>uri</var> for the <em>deb</em> type must specify the base of the
 109    debian distribution. APT will automatically generate the proper longer
 110    URIs to get the information it needs. <var>distribution</var> can specify
 111    an exact path, in this case the components must be omitted and
 112    <var>distribution</var> must end in a slash.
 113
 114    <p>
 115    Since only one distribution can be specified per deb line it may be
 116    necessary to list a number of deb lines for the same URI. APT will
 117    sort the URI list after it has generated a complete set to allow
 118    connection reuse. It is important to order things in the sourcelist
 119    from most preferred to least preferred (fastest to slowest).
 120 </sect1>
 121
 122 <sect1>URI specification
 123 <p>
 124 URIs in the source list support a large number of access schemes.
 125
 126 <taglist>
 127 <tag>cdrom<item>
 128    The cdrom scheme is special in that If Modified Since queries are never
 129    performed and that APT knows how to match a cdrom to the name it
 130    was given when first inserted. APT also knows all of the possible
 131    mount points the cdrom drives and that the user should be prompted
 132    to insert a CD if it cannot be found. The path is relative to an
 133    arbitrary mount point (of APT's choosing) and must not start with a
 134    slash. The first pathname component is the given name and is purely
 135    descriptive and of the users choice. However, if a file in the root of
 136    the cdrom is called '.disk/info' its contents will be used instead of
 137    prompting. The name serves as a tag for the cdrom and should be unique.
 138    <example>
 139    cdrom:Debian 1.3/debian
 140    </example>
 141
 142 <tag>http<item>
 143    This scheme specifies a HTTP server for the debian archive. HTTP is preferred
 144    over FTP because If Modified Since queries against the Package file are
 145    possible as well as deep pipelining and resume capabilities.
 146    <example>
 147    http://www.debian.org/archive
 148    </example>
 149
 150 <tag>ftp<item>
 151    This scheme specifies a FTP connection to the server. FTP is limited because
 152    there is no support for IMS and is hard to proxy over firewalls.
 153    <example>
 154    ftp://ftp.debian.org/debian
 155    </example>
 156
 157 <tag>file<item>
 158    The file scheme allows an arbitrary directory in the file system to be
 159    considered as a debian archive. This is useful for NFS mounts and
 160    local mirrors/archives.
 161    <example>
 162    file:/var/debian
 163    </example>
 164
 165 <tag>smb<item>
 166    A possible future expansion may be to have direct support for smb (Samba
 167    servers).
 168    <example>
 169    smb://ftp.kernel.org/pub/mirrors/debian
 170    </example>
 171 </taglist>
 172 </sect1>
 173
 174 <sect1>Hashing the URI
 175 <p>
 176 All permanent information acquired from any of the sources is stored in the
 177 lists directory. Thus, there must be a way to relate the filename in the
 178 lists directory to a line in the sourcelist. To simplify things this is
 179 done by quoting the URI and treating _'s as quoteable characters and
 180 converting / to _. The URI spec says this is done by converting a
 181 sensitive character into %xx where xx is the hexadecimal representation
 182 from the ASCII character set. Examples:
 183
 184 <example>
 185 http://www.debian.org/archive/dists/stable/binary-i386/Packages
 186 /var/lib/apt/lists/www.debian.org_archive_dists_stable_binary-i386_Packages
 187
 188 cdrom:Debian 1.3/debian/Packages
 189 /var/lib/apt/info/Debian%201.3_debian_Packages
 190 </example>
 191
 192 <p>
 193 The other alternative that was considered was to use a deep directory
 194 structure but this poses two problems, it makes it very difficult to prune
 195 directories back when sources are no longer used and complicates the handling
 196 of the partial directory. This gives a very simple way to deal with all
 197 of the situations that can arise. Also note that the same rules described in
 198 the <em>Archive Directory</> section regarding the partial sub dir apply
 199 here as well.
 200 </sect1>
 201
 202 </sect>
 203                                                                   <!-- }}} -->
 204 <!-- Extra Status                                                      {{{ -->
 205 <!-- ===================================================================== -->
 206 <sect>Extra Status File (xstatus)
 207
 208 <p>
 209 The extra status file serves the same purpose as the normal dpkg status file
 210 (/var/lib/dpkg/status) except that it stores information unique to apt.
 211 This includes the autoflag, target distribution and version and any other
 212 unique features that come up over time. It duplicates nothing from the normal
 213 dpkg status file.  Please see other APT documentation for a discussion
 214 of the exact internal behaviour of these fields. The Package field is
 215 placed directly before the new fields to indicate which package they
 216 apply to. The new fields are as follows:
 217
 218 <taglist>
 219 <tag>X-Auto<item>
 220    The Auto flag can be Yes or No and controls whether the package is in
 221    auto mode.
 222
 223 <tag>X-TargetDist<item>
 224    The TargetDist item indicates which distribution versions are offered for
 225    installation from. It should be stable, unstable or frozen.
 226
 227 <tag>X-TargetVersion<item>
 228    The target version item is set if the user selects a specific version, it
 229    overrides the TargetDist selection if both are present.
 230 </taglist>
 231 </sect>
 232                                                                   <!-- }}} -->
 233 <!-- Binary Package Cache                                              {{{ -->
 234 <!-- ===================================================================== -->
 235 <sect>Binary Package Cache (pkgcache.bin)
 236
 237 <p>
 238 Please see cache.sgml for a complete description of what this file is. The
 239 cache file is updated whenever the contents of the lists directory changes.
 240 If the cache is erased, corrupted or of a non-matching version it will
 241 be automatically rebuilt by all of the tools that need it.
 242 <em>srcpkgcache.bin</> contains a cache of all of the package files in the
 243 source list. This allows regeneration of the cache when the status files
 244 change to use a prebuilt version for greater speed.
 245 </sect>
 246                                                                   <!-- }}} -->
 247 <!-- Downloads Directory                                               {{{ -->
 248 <!-- ===================================================================== -->
 249 <sect>Downloads Directory (archives)
 250
 251 <p>
 252 The archives directory is where all downloaded .deb archives go. When the
 253 file transfer is initiated the deb is placed in partial. Once the file
 254 is fully downloaded and its MD5 hash and size are verified it is moved
 255 from partial into archives/. Any files found in archives/ can be assumed
 256 to be verified.
 257
 258 <p>
 259 No directory structure is transfered from the receiving site and all .deb
 260 file names conform to debian conventions. No short (msdos) filename should
 261 be placed in archives. If the need arises .debs should be unpacked, scanned
 262 and renamed to their correct internal names. This is mostly to prevent
 263 file name conflicts but other programs may depend on this if convenient.
 264 A conforming .deb is one of the form, name_version_arch.deb. Our archive
 265 scripts do not handle epochs, but they are necessary and should be re-inserted.
 266 If necessary _'s and :'s in the fields should be quoted using the % convention.
 267 It must be possible to extract all 3 fields by examining the file name.
 268 Downloaded .debs must be found in one of the package lists with an exact
 269 name + version match..
 270 </sect>
 271                                                                   <!-- }}} -->
 272 <!-- The Methods Directory                                             {{{ -->
 273 <!-- ===================================================================== -->
 274 <sect> The Methods Directory (/usr/lib/apt/methods)
 275
 276 <p>
 277 The Methods directory is more fully described in the APT Methods interface
 278 document.
 279 </sect>
 280                                                                   <!-- }}} -->
 281 <!-- The Mirror List                                                   {{{ -->
 282 <!-- ===================================================================== -->
 283 <sect> The Mirror List
 284
 285 <p>
 286 The mirror list is stored on the primary debian web server (www.debian.org)
 287 and contains a machine readable list of all known debian mirrors. It's
 288 format and style mirror the Package file.
 289
 290 <taglist>
 291 <tag>Site<item>
 292 This is the proper host name of the site. It should not be a host within
 293 debian.org and generally cnames should be avoided here.
 294
 295 <tag>Aliases<item>
 296 These list any commonly used aliases for the site. This field is used to make
 297 sure that a site is not added twice.
 298
 299 <tag>Type<item>
 300 This field can either be <em>Push-Primary</> or <em>leaf</>.
 301 <em>Push-Primary</> are authorized top level mirrors of the archive, all
 302 other mirrors are leaf.
 303
 304 <tag>Archive-[access]<item>
 305 The Archive field gives the path(s) to the debian archive. [access]
 306 specifies the access method and may be one of ftp, http, rsync, nfs, or
 307 smb. For many of the types it is possible to prefix the path with :###
 308 indicating that an alternate port should be used. Generally paths
 309 start with a / and end with a /, rsync is an exception in that the
 310 first directory component is not a path but a label.
 311
 312 <tag>WWW-[access]<item>
 313 The WWW field gives the path(s) to the debian web site.
 314
 315 <tag>CDImage-[access]<item>
 316 The WWW field gives the path(s) to the debian CD-ROM images
 317
 318 <tag>Incoming-[access]<item>
 319 The Incoming field gives the path(s) to a mirror of the debian incoming
 320 directory.
 321
 322 <tag>nonUS-[access]<item>
 323 The nonUS field gives the path(s) to a mirror of the non-US distribution.
 324
 325 <tag>Maintainer<item>
 326 This is the email address of the maintainer of the mirror.
 327
 328 <tag>Location<item>
 329 Location gives the general geographical region the mirror is in.
 330
 331 <tag>Sponsor<item>
 332 The Sponsor field indicates who owns the mirror and a URL to a web page
 333 describing the organization.
 334
 335 <tag>Comment<item>
 336 General free-form text.
 337
 338 </taglist>
 339
 340 <p>
 341 Some form of network measurement will have to be used to gauge performance
 342 of each of the mirrors. This will be discussed later, initial versions
 343 will use the first found URI.
 344 </sect>
 345                                                                   <!-- }}} -->
 346 <!-- The Release File                                                  {{{ -->
 347 <!-- ===================================================================== -->
 348 <sect> The Release File
 349
 350 <p>
 351 This file plays and important role in how APT presents the archive to the
 352 user. Its main purpose is to present a descriptive name for the source
 353 of each version of each package. It also is used to detect when new versions
 354 of debian are released. It augments the package file it is associated with
 355 by providing meta information about the entire archive which the Packages
 356 file describes.
 357
 358 <p>
 359 The full name of the distribution for presentation to the user is formed
 360 as 'label version archive', with a possible extended name being
 361 'label version archive component'.
 362
 363 <p>
 364 The file is formed as the package file (RFC-822) with the following tags
 365 defined:
 366
 367 <taglist>
 368 <tag>Archive<item>
 369 This is the common name we give our archives, such as <em>stable</> or
 370 <em>unstable</>.
 371
 372 <tag>Component<item>
 373 Refers to the sub-component of the archive, <em>main</>, <em>contrib</>
 374 etc. Component may be omitted if there are no components for this archive.
 375
 376 <tag>Version<item>
 377 This is a version string with the same properties as in the Packages file.
 378 It represents the release level of the archive.
 379
 380 <tag>Origin<item>
 381 This specifies who is providing this archive. In the case of Debian the
 382 string will read 'Debian'. Other providers may use their own string
 383
 384 <tag>Label<item>
 385 This carries the encompassing name of the distribution. For Debian proper
 386 this field reads 'Debian'. For derived distributions it should contain their
 387 proper name.
 388
 389 <tag>Architecture<item>
 390 When the archive has packages for a single architecture then the Architecture
 391 is listed here. If a mixed set of systems are represented then this should
 392 contain the keyword <em>mixed</em>.
 393
 394 <tag>NotAutomatic<item>
 395 A Yes/No flag indicating that the archive is extremely unstable and its
 396 version's should never be automatically selected. This is to be used by
 397 experimental.
 398
 399 <tag>Description<item>
 400 Description is used to describe the release. For instance experimental would
 401 contain a warning that the packages have problems.
 402 </taglist>
 403
 404 <p>
 405 The location of the Release file in the archive is very important, it must
 406 be located in the same location as the packages file so that it can be
 407 located in all situations. The following is an example for the current stable
 408 release, 1.3.1r6
 409
 410 <example>
 411 Archive: stable
 412 Component: main
 413 Version: 1.3.1r6
 414 Origin: Debian
 415 Label: Debian
 416 Architecture: i386
 417 </example>
 418
 419 This is an example of experimental,
 420 <example>
 421 Archive: experimental
 422 Version: 0
 423 Origin: Debian
 424 Label: Debian
 425 Architecture: mixed
 426 NotAutomatic: Yes
 427 </example>
 428
 429 And unstable,
 430 <example>
 431 Archive: unstable
 432 Component: main
 433 Version: 2.1
 434 Origin: Debian
 435 Label: Debian
 436 Architecture: i386
 437 </example>
 438
 439 </sect>
 440                                                                   <!-- }}} -->
 441
 442 </book>