doc/apt-ftparchive.1.sgml

   1 <!-- -*- mode: sgml; mode: fold -*- -->
   2 <!doctype refentry PUBLIC "-//OASIS//DTD DocBook V3.1//EN" [
   3
   4 <!ENTITY % aptent SYSTEM "apt.ent">
   5 %aptent;
   6
   7 ]>
   8
   9 <refentry>
  10  &apt-docinfo;
  11
  12  <refmeta>
  13    <refentrytitle>apt-ftparchive</>
  14    <manvolnum>1</>
  15  </refmeta>
  16
  17  <!-- Man page title -->
  18  <refnamediv>
  19     <refname>apt-ftparchive</>
  20     <refpurpose>Utility to generate index files</>
  21  </refnamediv>
  22
  23  <!-- Arguments -->
  24  <refsynopsisdiv>
  25    <cmdsynopsis>
  26       <command>apt-ftparchive</>
  27       <arg><option>-hvdsq</></arg>
  28       <arg><option>--md5</></arg>
  29       <arg><option>--delink</></arg>
  30       <arg><option>--readonly</></arg>
  31       <arg><option>--contents</></arg>
  32       <arg><option>-o=<replaceable/config string/</></arg>
  33       <arg><option>-c=<replaceable/file/</></arg>
  34       <group choice=req>
  35          <arg>packages<arg choice="plain" rep="repeat"><replaceable>path</replaceable></arg><arg><replaceable>override</replaceable><arg><replaceable>pathprefix</replaceable></arg></arg></arg>
  36          <arg>sources<arg choice="plain" rep="repeat"><replaceable>path</replaceable></arg><arg><replaceable>override</replaceable><arg><replaceable>pathprefix</replaceable></arg></arg></arg>
  37          <arg>contents <arg choice="plain"><replaceable>path</replaceable></arg></arg>
  38          <arg>generate <arg choice="plain"><replaceable>config-file</replaceable></arg> <arg choice="plain" rep="repeat"><replaceable>section</replaceable></arg></arg>
  39          <arg>clean <arg choice="plain"><replaceable>config-file</replaceable></arg></arg>
  40       </group>
  41    </cmdsynopsis>
  42  </refsynopsisdiv>
  43
  44  <RefSect1><Title>Description</>
  45    <para>
  46    <command/apt-ftparchive/ is the command line tool that generates the index
  47    files that APT uses to access a distribution source. The index files should
  48    be generated on the origin site based on the content of that site.
  49
  50    <para>
  51    <command/apt-ftparchive/ is a superset of the &dpkg-scanpackages; program,
  52    incorporating it's entire functionality via the <literal/directory/ command.
  53    It also contains a contents file generator, <literal/contents/, and an
  54    elaborate means to 'script' the generation process for a complete
  55    archive.
  56
  57    <para>
  58    Internally <command/apt-ftparchive/ can make use of binary databases to
  59    cache the contents of a .deb file and it does not rely on any external
  60    programs aside from &gzip;. When doing a full generate it automatically
  61    performs file-change checks and builds the desired compressed output files.
  62
  63    <para>
  64    Unless the <option/-h/, or <option/--help/ option is given one of the
  65    commands below must be present.
  66
  67    <VariableList>
  68      <VarListEntry><term>packages</term>
  69      <ListItem><Para>
  70      The packages command generates a package file from a directory tree. It
  71      takes the given directory and recursively searches it for .deb files,
  72      emitting a package record to stdout for each. This command is
  73      approximately equivalent to &dpkg-scanpackages;.
  74      <para>
  75      The option <option/--db/ can be used to specify a binary caching DB.
  76      </VarListEntry>
  77
  78      <VarListEntry><term>sources</term>
  79      <ListItem><Para>
  80      The <literal/sources/ command generates a source index file from a directory tree.
  81      It takes the given directory and recursively searches it for .dsc files,
  82      emitting a source record to stdout for each. This command is approximately
  83      equivalent to &dpkg-scansources;.
  84      <para>
  85      If an override file is specified then a source override file will be
  86      looked for with an extension of .src. The --source-override option can be
  87      used to change the source override file that will be used.
  88      </VarListEntry>
  89
  90      <VarListEntry><term>contents</term>
  91      <ListItem><Para>
  92      The <literal/contents/ command generates a contents file from a directory tree. It
  93      takes the given directory and recursively searches it for .deb files,
  94      and reads the file list from each file. It then sorts and writes to stdout
  95      the list of files matched to packages. Directories are not written to
  96      the output. If multiple packages own the same file then each package is
  97      separated by a comma in the output.
  98      <para>
  99      The option <option/--db/ can be used to specify a binary caching DB.
 100      </VarListEntry>
 101
 102      <VarListEntry><term>generate</term>
 103      <ListItem><Para>
 104      The <literal/generate/ command is designed to be runnable from a cron script and
 105      builds indexes according to the given config file. The config language
 106      provides a flexible means of specifying which index files are built from
 107      which directories, as well as providing a simple means of maintaining the
 108      required settings.
 109      </VarListEntry>
 110
 111      <VarListEntry><term>clean</term>
 112      <ListItem><Para>
 113      The <literal/clean/ command tidies the databases used by the given
 114      configuration file by removing any records that are no longer necessary.
 115      </VarListEntry>
 116    </VariableList>
 117
 118  </RefSect1>
 119
 120  <RefSect1><Title>The Generate Configuration</>
 121    <para>
 122    The <literal/generate/ command uses a configuration file to describe the
 123    archives that are going to be generated. It follows the typical ISC
 124    configuration format as seen in ISC tools like bind 8 and dhcpd.
 125    &apt-conf; contains a decsription of the syntax. Note that the generate
 126    configuration is parsed in sectional manner, but &apt-conf; is parsed in a
 127    tree manner. This only effects how the scope tag is handled.
 128
 129    <para>
 130    The generate configuration has 4 separate sections, each decribed below.
 131
 132    <refsect2><title>Dir Section</>
 133      <Para>
 134      The <literal/Dir/ section defines the standard directories needed to
 135      locate the files required during the generation process. These
 136      directories are prepended to certain relative paths defined in later
 137      sections to produce a complete an absolute path.
 138      <VariableList>
 139       <VarListEntry><term>ArchiveDir</term>
 140       <ListItem><Para>
 141       Specifies the root of the FTP archive, in a standard
 142       Debian configuration this is the directory that contains the
 143       <filename/ls-LR/, and dist nodes.
 144       </VarListEntry>
 145
 146       <VarListEntry><term>OverrideDir</term>
 147       <ListItem><Para>
 148       Specifies the location of the override files.
 149       </VarListEntry>
 150
 151       <VarListEntry><term>CacheDir</term>
 152       <ListItem><Para>
 153       Specifies the location of the cache files
 154       </VarListEntry>
 155
 156       <VarListEntry><term>FileListDir</term>
 157       <ListItem><Para>
 158       Specifies the location of the file list files,
 159       if the <literal/FileList/ setting is used below.
 160       </VarListEntry>
 161      </VariableList>
 162    </refsect2>
 163
 164    <refsect2><title>Default Section</>
 165      <para>
 166      The <literal/Default/ section specifies default values, and settings
 167      that control the operation of the generator. Other sections may override
 168      these defaults with a per-section setting.
 169      <VariableList>
 170       <VarListEntry><term>Packages::Compress</term>
 171       <ListItem><Para>
 172       Sets the default compression schemes to use
 173       for the Package index files. It is a string that contains a space
 174       separated list of at least one of: '.' (no compression), 'gzip' and
 175       'bzip2'. The default for all compression schemes is '. gzip'.
 176       </VarListEntry>
 177
 178       <VarListEntry><term>Packages::Extensions</term>
 179       <ListItem><Para>
 180       Sets the default list of file extensions that are package files.
 181       This defaults to '.deb'.
 182       </VarListEntry>
 183
 184       <VarListEntry><term>Sources::Compress</term>
 185       <ListItem><Para>
 186       This is similar to <literal/Packages::Compress/
 187       except that it controls the compression for the Sources files.
 188       </VarListEntry>
 189
 190       <VarListEntry><term>Sources::Extensions</term>
 191       <ListItem><Para>
 192       Sets the default list of file extensions that are source files.
 193       This defaults to '.dsc'.
 194       </VarListEntry>
 195
 196       <VarListEntry><term>Contents::Compress</term>
 197       <ListItem><Para>
 198       This is similar to <literal/Packages::Compress/
 199       except that it controls the compression for the Contents files.
 200       </VarListEntry>
 201
 202       <VarListEntry><term>DeLinkLimit</term>
 203       <ListItem><Para>
 204       Specifies the number of kilobytes to delink (and
 205       replace with hard links) per run. This is used in conjunction with the
 206       per-section <literal/External-Links/ setting.
 207       </VarListEntry>
 208
 209       <VarListEntry><term>FileMode</term>
 210       <ListItem><Para>
 211       Specifies the mode of all created index files. It
 212       defaults to 0644. All index files are set to this mode with no regard
 213       to the umask.
 214       </VarListEntry>
 215      </VariableList>
 216    </refsect2>
 217
 218    <refsect2><title>TreeDefault Section</>
 219      <para>
 220      Sets defaults specific to <literal/Tree/ sections. All of these
 221      variables are substitution variables and have the strings $(DIST),
 222      $(SECTION) and $(ARCH) replaced with their respective values.
 223
 224      <VariableList>
 225       <VarListEntry><term>MaxContentsChange</term>
 226       <ListItem><Para>
 227       Sets  the number of kilobytes of contents
 228       files that are generated each day. The contents files are round-robined
 229       so that over several days they will all be rebuilt.
 230       </VarListEntry>
 231
 232       <VarListEntry><term>ContentsAge</term>
 233       <ListItem><Para>
 234       Controls the number of days a contents file is allowed
 235       to be checked without changing. If this limit is passed the mtime of the
 236       contents file is updated. This case can occur if the package file is
 237       changed in such a way that does not result in a new contents file
 238       [overried edit for instance]. A hold off is allowed in hopes that new
 239       .debs will be installed, requiring a new file anyhow. The default is 10,
 240       the units are in days.
 241       </VarListEntry>
 242
 243       <VarListEntry><term>Directory</term>
 244       <ListItem><Para>
 245       Sets the top of the .deb directory tree. Defaults to
 246       <filename>$(DIST)/$(SECTION)/binary-$(ARCH)/</>
 247       </VarListEntry>
 248
 249       <VarListEntry><term>Packages</term>
 250       <ListItem><Para>
 251       Sets the output Packages file. Defaults to
 252       <filename>$(DIST)/$(SECTION)/binary-$(ARCH)/Packages</>
 253       </VarListEntry>
 254
 255       <VarListEntry><term>Sources</term>
 256       <ListItem><Para>
 257       Sets the output Packages file. Defaults to
 258       <filename>$(DIST)/$(SECTION)/source/Sources</>
 259       </VarListEntry>
 260
 261       <VarListEntry><term>InternalPrefix</term>
 262       <ListItem><Para>
 263       Sets the path prefix that causes a symlink to be
 264       considered an internal link instead of an external link. Defaults to
 265       <filename>$(DIST)/$(SECTION)/</>
 266       </VarListEntry>
 267
 268       <VarListEntry><term>Contents</term>
 269       <ListItem><Para>
 270       Sets the output Contents file. Defaults to
 271       <filename>$(DIST)/Contents-$(ARCH)</>. If this setting causes multiple
 272       Packages files to map onto a single Contents file (such as the default)
 273       then <command/apt-ftparchive/ will integrate those package files
 274       together automatically.
 275       </VarListEntry>
 276
 277       <VarListEntry><term>Contents::Header</term>
 278       <ListItem><Para>
 279       Sets header file to prepend to the contents output.
 280       </VarListEntry>
 281
 282       <VarListEntry><term>BinCacheDB</term>
 283       <ListItem><Para>
 284       Sets the binary cache database to use for this
 285       section. Multiple sections can share the same database.
 286       </VarListEntry>
 287
 288       <VarListEntry><term>FileList</term>
 289       <ListItem><Para>
 290       Specifies that instead of walking the directory tree
 291       that <command/apt-ftparchive/ should read the list of files from the given
 292       file. Relative files names are prefixed with the archive directory.
 293       </VarListEntry>
 294
 295       <VarListEntry><term>SourceFileList</term>
 296       <ListItem><Para>
 297       Specifies that instead of walking the directory tree
 298       that <command/apt-ftparchive/ should read the list of files from the given
 299       file. Relative files names are prefixed with the archive directory.
 300       This is used when processing source indexs.
 301       </VarListEntry>
 302      </VariableList>
 303    </refsect2>
 304
 305    <refsect2><title>Tree Section</>
 306      <para>
 307      The <literal/Tree/ section defines a standard Debian file tree which
 308      consists of a base directory, then multiple sections in that base
 309      directory and finally multiple Architectures in each section. The exact
 310      pathing used is defined by the <literal/Directory/ substitution variable.
 311      <para>
 312      The <literal/Tree/ section takes a scope tag which sets the
 313      <literal/$(DIST)/ variable and defines the root of the tree
 314      (the path is prefixed by <literal/ArchiveDir/).
 315      Typically this is a setting such as <filename>dists/woody</>.
 316      <para>
 317      All of the settings defined in the <literal/TreeDefault/ section can be
 318      use in a <literal/Tree/ section as well as three new variables.
 319      <para>
 320      When processing a <literal/Tree/ section <command/apt-ftparchive/
 321      performs an operation similar to:
 322 <informalexample><programlisting>
 323 for i in Sections do
 324    for j in Architectures do
 325       Generate for DIST=scope SECTION=i ARCH=j
 326 </programlisting></informalexample>
 327
 328      <VariableList>
 329       <VarListEntry><term>Sections</term>
 330       <ListItem><Para>
 331       This is a space separated list of sections which appear
 332       under the distribution, typically this is something like
 333       <literal/main contrib non-free/.
 334       </VarListEntry>
 335
 336       <VarListEntry><term>Architectures</term>
 337       <ListItem><Para>
 338       This is a space separated list of all the
 339       architectures that appear under seach section. The special architecture
 340       'source' is used to indicate that this tree has a source archive.
 341       </VarListEntry>
 342
 343       <VarListEntry><term>BinOverride</term>
 344       <ListItem><Para>
 345       Sets the binary override file. The override file
 346       contains section, priority and maintainer address information.
 347       </VarListEntry>
 348
 349       <VarListEntry><term>SrcOverride</term>
 350       <ListItem><Para>
 351       Sets the source override file. The override file
 352       contains section information.
 353       </VarListEntry>
 354      </VariableList>
 355    </refsect2>
 356
 357    <refsect2><title>BinDirectory Section</>
 358      <para>
 359      The <literal/bindirectory/ section defines a binary directory tree
 360      with no special structure. The scope tag specifies the location of
 361      the binary directory and the settings are similar to the <literal/Tree/
 362      section with no substitution variables or
 363      <literal>Section</><literal>Architecture</> settings.
 364      <VariableList>
 365       <VarListEntry><term>Packages</term>
 366       <ListItem><Para>
 367       Sets the Packages file output.
 368       </VarListEntry>
 369
 370       <VarListEntry><term>SrcPackages</term>
 371       <ListItem><Para>
 372       Sets the Sources file output. At least one of
 373       <literal/Packages/ or <literal/SrcPackages/ is required.
 374       </VarListEntry>
 375
 376       <VarListEntry><term>Contents</term>
 377       <ListItem><Para>
 378       Sets the Contents file output. (Optional)
 379       </VarListEntry>
 380
 381       <VarListEntry><term>Binoverride</term>
 382       <ListItem><Para>
 383       Sets the binary override file.
 384       </VarListEntry>
 385
 386       <VarListEntry><term>SrcOverride</term>
 387       <ListItem><Para>
 388       Sets the source override file.
 389       </VarListEntry>
 390
 391       <VarListEntry><term>BinCacheDB</term>
 392       <ListItem><Para>
 393       Sets the cache DB.
 394       </VarListEntry>
 395
 396       <VarListEntry><term>PathPrefix</term>
 397       <ListItem><Para>
 398       Appends a path to all the output paths.
 399       </VarListEntry>
 400
 401       <VarListEntry><term>FileList, SourceFileList</term>
 402       <ListItem><Para>
 403       Specifies the file list file.
 404       </VarListEntry>
 405      </VariableList>
 406    </refsect2>
 407  </RefSect1>
 408
 409  <RefSect1><Title>The Binary Override File</>
 410    <para>
 411    The binary override file is fully compatible with &dpkg-scanpackages;. It
 412    contains 4 fields sperated by spaces. The first field is the package name,
 413    the second is the priority to force that package to, the third is the
 414    the section to force that package to and the final field is the maintainer
 415    permutation field.
 416    <para>
 417    The general form of the maintainer field is:
 418    <literallayout>old [// oldn]* => new</literallayout>
 419    or simply,
 420    <literallayout>new</literallayout>
 421    The first form allows a double-slash separated list of old email addresses
 422    to be specified. If any of those are found then new is substituted for the
 423    maintainer field. The second form unconditionally substitutes the
 424    maintainer field.
 425  </RefSect1>
 426
 427  <RefSect1><title>The Source Override File</>
 428    <para>
 429    The source override file is fully compatible with &dpkg-scansources;. It
 430    contains 2 fields separated by spaces. The first fields is the source
 431    package name, the second is the section to assign it.
 432  </RefSect1>
 433
 434  <RefSect1><Title>Options</>
 435    &apt-cmdblurb;
 436
 437    <VariableList>
 438      <VarListEntry><term><option/--md5/</>
 439      <ListItem><Para>
 440      Generate MD5 sums. This defaults to on, when turned off the generated
 441      index files will not have MD5Sum fields where possible.
 442      Configuration Item: <literal/APT::FTPArchive::MD5/.
 443      </VarListEntry>
 444
 445      <VarListEntry><term><option/-d/</><term><option/--db/</>
 446      <ListItem><Para>
 447      Use a binary caching DB. This has no effect on the generate command.
 448      Configuration Item: <literal/APT::FTPArchive::DB/.
 449      </VarListEntry>
 450
 451      <VarListEntry><term><option/-q/</><term><option/--quiet/</>
 452      <ListItem><Para>
 453      Quiet; produces output suitable for logging, omitting progress indicators.
 454      More q's will produce more quiet up to a maximum of 2. You can also use
 455      <option/-q=#/ to set the quiet level, overriding the configuration file.
 456      Configuration Item: <literal/quiet/.
 457      </VarListEntry>
 458
 459      <VarListEntry><term><option/--delink/</>
 460      <ListItem><Para>
 461      Perform Delinking. If the <literal/External-Links/ setting is used then
 462      this option actually enables delinking of the files. It defaults to on and
 463      can be turned off with <option/--no-delink/.
 464      Configuration Item: <literal/APT::FTPArchive::DeLinkAct/.
 465      </VarListEntry>
 466
 467      <VarListEntry><term><option/--contents/</>
 468      <ListItem><Para>
 469      Perform contents generation. When this option is set and package indexes
 470      are being generated with a cache DB then the file listing will also be
 471      extracted and stored in the DB for later use. When using the generate
 472      command this option also allows the creation of any Contents files. The
 473      default is on.
 474      Configuration Item: <literal/APT::FTPArchive::Contents/.
 475      </VarListEntry>
 476
 477      <VarListEntry><term><option/-s/</><term><option/--source-override/</>
 478      <ListItem><Para>
 479      Select the source override file to use with the <literal/sources/ command.
 480      Configuration Item: <literal/APT::FTPArchive::SourceOverride/.
 481      </VarListEntry>
 482
 483      <VarListEntry><term><option/--readonly/</>
 484      <ListItem><Para>
 485      Make the caching databases read only.
 486      Configuration Item: <literal/APT::FTPArchive::ReadOnlyDB/.
 487      </VarListEntry>
 488
 489      &apt-commonoptions;
 490
 491    </VariableList>
 492  </RefSect1>
 493
 494  <RefSect1><Title>See Also</>
 495    <para>
 496    &apt-conf;
 497  </RefSect1>
 498
 499  <RefSect1><Title>Diagnostics</>
 500    <para>
 501    <command/apt-ftparchive/ returns zero on normal operation, decimal 100 on error.
 502  </RefSect1>
 503
 504  &manbugs;
 505  &manauthor;
 506
 507 </refentry>