doc/apt-ftparchive.1.sgml

   1 <!-- -*- mode: sgml; mode: fold -*- -->
   2 <!doctype refentry PUBLIC "-//OASIS//DTD DocBook V3.1//EN" [
   3
   4 <!ENTITY % aptent SYSTEM "apt.ent">
   5 %aptent;
   6
   7 ]>
   8
   9 <refentry>
  10  &apt-docinfo;
  11
  12  <refmeta>
  13    <refentrytitle>apt-ftparchive</>
  14    <manvolnum>1</>
  15  </refmeta>
  16
  17  <!-- Man page title -->
  18  <refnamediv>
  19     <refname>apt-ftparchive</>
  20     <refpurpose>Utility to generate index files</>
  21  </refnamediv>
  22
  23  <!-- Arguments -->
  24  <refsynopsisdiv>
  25    <cmdsynopsis>
  26       <command>apt-ftparchive</>
  27       <arg><option>-hvdsq</></arg>
  28       <arg><option>--md5</></arg>
  29       <arg><option>--delink</></arg>
  30       <arg><option>--readonly</></arg>
  31       <arg><option>--contents</></arg>
  32       <arg><option>-o=<replaceable/config string/</></arg>
  33       <arg><option>-c=<replaceable/file/</></arg>
  34       <group choice=req>
  35          <arg>packages<arg choice="plain" rep="repeat"><replaceable>path</replaceable></arg><arg><replaceable>override</replaceable><arg><replaceable>pathprefix</replaceable></arg></arg></arg>
  36          <arg>sources<arg choice="plain" rep="repeat"><replaceable>path</replaceable></arg><arg><replaceable>override</replaceable><arg><replaceable>pathprefix</replaceable></arg></arg></arg>
  37          <arg>contents <arg choice="plain"><replaceable>path</replaceable></arg></arg>
  38          <arg>generate <arg choice="plain"><replaceable>config-file</replaceable></arg> <arg choice="plain" rep="repeat"><replaceable>section</replaceable></arg></arg>
  39          <arg>clean <arg choice="plain"><replaceable>config-file</replaceable></arg></arg>
  40       </group>
  41    </cmdsynopsis>
  42  </refsynopsisdiv>
  43
  44  <RefSect1><Title>Description</>
  45    <para>
  46    <command/apt-ftparchive/ is the command line tool that generates the index
  47    files that APT uses to access a distribution source. The index files should
  48    be generated on the origin site based on the content of that site.
  49
  50    <para>
  51    <command/apt-ftparchive/ is a superset of the &dpkg-scanpackages; program,
  52    incorporating it's entire functionality via the <literal/directory/ command.
  53    It also contains a contents file generator, <literal/contents/, and an
  54    elaborate means to 'script' the generation process for a complete
  55    archive.
  56
  57    <para>
  58    Internally <command/apt-ftparchive/ can make use of binary databases to
  59    cache the contents of a .deb file and it does not rely on any external
  60    programs aside from &gzip;. When doing a full generate it automatically
  61    performs file-change checks and builds the desired compressed output files.
  62
  63    <para>
  64    Unless the <option/-h/, or <option/--help/ option is given one of the
  65    commands below must be present.
  66
  67    <VariableList>
  68      <VarListEntry><term>packages</term>
  69      <ListItem><Para>
  70      The packages command generates a package file from a directory tree. It
  71      takes the given directory and recursively searches it for .deb files,
  72      emitting a package record to stdout for each. This command is
  73      approximately equivalent to &dpkg-scanpackages;.
  74      <para>
  75      The option <option/--db/ can be used to specify a binary caching DB.
  76      </VarListEntry>
  77
  78      <VarListEntry><term>sources</term>
  79      <ListItem><Para>
  80      The <literal/sources/ command generates a source index file from a directory tree.
  81      It takes the given directory and recursively searches it for .dsc files,
  82      emitting a source record to stdout for each. This command is approximately
  83      equivalent to &dpkg-scansources;.
  84      <para>
  85      If an override file is specified then a source override file will be
  86      looked for with an extension of .src. The --source-override option can be
  87      used to change the source override file that will be used.
  88      </VarListEntry>
  89
  90      <VarListEntry><term>contents</term>
  91      <ListItem><Para>
  92      The <literal/contents/ command generates a contents file from a directory tree. It
  93      takes the given directory and recursively searches it for .deb files,
  94      and reads the file list from each file. It then sorts and writes to stdout
  95      the list of files matched to packages. Directories are not written to
  96      the output. If multiple packages own the same file then each package is
  97      separated by a comma in the output.
  98      <para>
  99      The option <option/--db/ can be used to specify a binary caching DB.
 100      </VarListEntry>
 101
 102      <VarListEntry><term>generate</term>
 103      <ListItem><Para>
 104      The <literal/generate/ command is designed to be runnable from a cron script and
 105      builds indexes according to the given config file. The config language
 106      provides a flexible means of specifying which index files are built from
 107      which directories, as well as providing a simple means of maintaining the
 108      required settings.
 109      </VarListEntry>
 110
 111      <VarListEntry><term>clean</term>
 112      <ListItem><Para>
 113      The <literal/clean/ command tidies the databases used by the given
 114      configuration file by removing any records that are no longer necessary.
 115      </VarListEntry>
 116    </VariableList>
 117
 118  </RefSect1>
 119
 120  <RefSect1><Title>The Generate Configuration</>
 121    <para>
 122    The <literal/generate/ command uses a configuration file to describe the
 123    archives that are going to be generated. It follows the typical ISC
 124    configuration format as seen in ISC tools like bind 8 and dhcpd.
 125    &apt-conf; contains a description of the syntax. Note that the generate
 126    configuration is parsed in sectional manner, but &apt-conf; is parsed in a
 127    tree manner. This only effects how the scope tag is handled.
 128
 129    <para>
 130    The generate configuration has 4 separate sections, each decribed below.
 131
 132    <refsect2><title>Dir Section</>
 133      <Para>
 134      The <literal/Dir/ section defines the standard directories needed to
 135      locate the files required during the generation process. These
 136      directories are prepended to certain relative paths defined in later
 137      sections to produce a complete an absolute path.
 138      <VariableList>
 139       <VarListEntry><term>ArchiveDir</term>
 140       <ListItem><Para>
 141       Specifies the root of the FTP archive, in a standard
 142       Debian configuration this is the directory that contains the
 143       <filename/ls-LR/, and dist nodes.
 144       </VarListEntry>
 145
 146       <VarListEntry><term>OverrideDir</term>
 147       <ListItem><Para>
 148       Specifies the location of the override files.
 149       </VarListEntry>
 150
 151       <VarListEntry><term>CacheDir</term>
 152       <ListItem><Para>
 153       Specifies the location of the cache files
 154       </VarListEntry>
 155
 156       <VarListEntry><term>FileListDir</term>
 157       <ListItem><Para>
 158       Specifies the location of the file list files,
 159       if the <literal/FileList/ setting is used below.
 160       </VarListEntry>
 161      </VariableList>
 162    </refsect2>
 163
 164    <refsect2><title>Default Section</>
 165      <para>
 166      The <literal/Default/ section specifies default values, and settings
 167      that control the operation of the generator. Other sections may override
 168      these defaults with a per-section setting.
 169      <VariableList>
 170       <VarListEntry><term>Packages::Compress</term>
 171       <ListItem><Para>
 172       Sets the default compression schemes to use
 173       for the Package index files. It is a string that contains a space
 174       separated list of at least one of: '.' (no compression), 'gzip' and
 175       'bzip2'. The default for all compression schemes is '. gzip'.
 176       </VarListEntry>
 177
 178       <VarListEntry><term>Packages::Extensions</term>
 179       <ListItem><Para>
 180       Sets the default list of file extensions that are package files.
 181       This defaults to '.deb'.
 182       </VarListEntry>
 183
 184       <VarListEntry><term>Sources::Compress</term>
 185       <ListItem><Para>
 186       This is similar to <literal/Packages::Compress/
 187       except that it controls the compression for the Sources files.
 188       </VarListEntry>
 189
 190       <VarListEntry><term>Sources::Extensions</term>
 191       <ListItem><Para>
 192       Sets the default list of file extensions that are source files.
 193       This defaults to '.dsc'.
 194       </VarListEntry>
 195
 196       <VarListEntry><term>Contents::Compress</term>
 197       <ListItem><Para>
 198       This is similar to <literal/Packages::Compress/
 199       except that it controls the compression for the Contents files.
 200       </VarListEntry>
 201
 202       <VarListEntry><term>DeLinkLimit</term>
 203       <ListItem><Para>
 204       Specifies the number of kilobytes to delink (and
 205       replace with hard links) per run. This is used in conjunction with the
 206       per-section <literal/External-Links/ setting.
 207       </VarListEntry>
 208
 209       <VarListEntry><term>FileMode</term>
 210       <ListItem><Para>
 211       Specifies the mode of all created index files. It
 212       defaults to 0644. All index files are set to this mode with no regard
 213       to the umask.
 214       </VarListEntry>
 215      </VariableList>
 216    </refsect2>
 217
 218    <refsect2><title>TreeDefault Section</>
 219      <para>
 220      Sets defaults specific to <literal/Tree/ sections. All of these
 221      variables are substitution variables and have the strings $(DIST),
 222      $(SECTION) and $(ARCH) replaced with their respective values.
 223
 224      <VariableList>
 225       <VarListEntry><term>MaxContentsChange</term>
 226       <ListItem><Para>
 227       Sets  the number of kilobytes of contents
 228       files that are generated each day. The contents files are round-robined
 229       so that over several days they will all be rebuilt.
 230       </VarListEntry>
 231
 232       <VarListEntry><term>ContentsAge</term>
 233       <ListItem><Para>
 234       Controls the number of days a contents file is allowed
 235       to be checked without changing. If this limit is passed the mtime of the
 236       contents file is updated. This case can occur if the package file is
 237       changed in such a way that does not result in a new contents file
 238       [overried edit for instance]. A hold off is allowed in hopes that new
 239       .debs will be installed, requiring a new file anyhow. The default is 10,
 240       the units are in days.
 241       </VarListEntry>
 242
 243       <VarListEntry><term>Directory</term>
 244       <ListItem><Para>
 245       Sets the top of the .deb directory tree. Defaults to
 246       <filename>$(DIST)/$(SECTION)/binary-$(ARCH)/</>
 247       </VarListEntry>
 248
 249       <VarListEntry><term>Packages</term>
 250       <ListItem><Para>
 251       Sets the output Packages file. Defaults to
 252       <filename>$(DIST)/$(SECTION)/binary-$(ARCH)/Packages</>
 253       </VarListEntry>
 254
 255       <VarListEntry><term>Sources</term>
 256       <ListItem><Para>
 257       Sets the output Packages file. Defaults to
 258       <filename>$(DIST)/$(SECTION)/source/Sources</>
 259       </VarListEntry>
 260
 261       <VarListEntry><term>InternalPrefix</term>
 262       <ListItem><Para>
 263       Sets the path prefix that causes a symlink to be
 264       considered an internal link instead of an external link. Defaults to
 265       <filename>$(DIST)/$(SECTION)/</>
 266       </VarListEntry>
 267
 268       <VarListEntry><term>Contents</term>
 269       <ListItem><Para>
 270       Sets the output Contents file. Defaults to
 271       <filename>$(DIST)/Contents-$(ARCH)</>. If this setting causes multiple
 272       Packages files to map onto a single Contents file (such as the default)
 273       then <command/apt-ftparchive/ will integrate those package files
 274       together automatically.
 275       </VarListEntry>
 276
 277       <VarListEntry><term>Contents::Header</term>
 278       <ListItem><Para>
 279       Sets header file to prepend to the contents output.
 280       </VarListEntry>
 281
 282       <VarListEntry><term>BinCacheDB</term>
 283       <ListItem><Para>
 284       Sets the binary cache database to use for this
 285       section. Multiple sections can share the same database.
 286       </VarListEntry>
 287
 288       <VarListEntry><term>FileList</term>
 289       <ListItem><Para>
 290       Specifies that instead of walking the directory tree,
 291       <command/apt-ftparchive/ should read the list of files from the given
 292       file. Relative files names are prefixed with the archive directory.
 293       </VarListEntry>
 294
 295       <VarListEntry><term>SourceFileList</term>
 296       <ListItem><Para>
 297       Specifies that instead of walking the directory tree,
 298       <command/apt-ftparchive/ should read the list of files from the given
 299       file. Relative files names are prefixed with the archive directory.
 300       This is used when processing source indexs.
 301       </VarListEntry>
 302      </VariableList>
 303    </refsect2>
 304
 305    <refsect2><title>Tree Section</>
 306      <para>
 307      The <literal/Tree/ section defines a standard Debian file tree which
 308      consists of a base directory, then multiple sections in that base
 309      directory and finally multiple Architectures in each section. The exact
 310      pathing used is defined by the <literal/Directory/ substitution variable.
 311      <para>
 312      The <literal/Tree/ section takes a scope tag which sets the
 313      <literal/$(DIST)/ variable and defines the root of the tree
 314      (the path is prefixed by <literal/ArchiveDir/).
 315      Typically this is a setting such as <filename>dists/woody</>.
 316      <para>
 317      All of the settings defined in the <literal/TreeDefault/ section can be
 318      use in a <literal/Tree/ section as well as three new variables.
 319      <para>
 320      When processing a <literal/Tree/ section <command/apt-ftparchive/
 321      performs an operation similar to:
 322 <informalexample><programlisting>
 323 for i in Sections do
 324    for j in Architectures do
 325       Generate for DIST=scope SECTION=i ARCH=j
 326 </programlisting></informalexample>
 327
 328      <VariableList>
 329       <VarListEntry><term>Sections</term>
 330       <ListItem><Para>
 331       This is a space separated list of sections which appear
 332       under the distribution, typically this is something like
 333       <literal/main contrib non-free/.
 334       </VarListEntry>
 335
 336       <VarListEntry><term>Architectures</term>
 337       <ListItem><Para>
 338       This is a space separated list of all the
 339       architectures that appear under seach section. The special architecture
 340       'source' is used to indicate that this tree has a source archive.
 341       </VarListEntry>
 342
 343       <VarListEntry><term>BinOverride</term>
 344       <ListItem><Para>
 345       Sets the binary override file. The override file
 346       contains section, priority and maintainer address information.
 347       </VarListEntry>
 348
 349       <VarListEntry><term>SrcOverride</term>
 350       <ListItem><Para>
 351       Sets the source override file. The override file
 352       contains section information.
 353       </VarListEntry>
 354
 355       <VarListEntry><term>ExtraOverride</term>
 356       <ListItem><Para>
 357       Sets the binary extra override file.
 358       </VarListEntry>
 359
 360       <VarListEntry><term>SrcExtraOverride</term>
 361       <ListItem><Para>
 362       Sets the source extra override file.
 363       </VarListEntry>
 364      </VariableList>
 365    </refsect2>
 366
 367    <refsect2><title>BinDirectory Section</>
 368      <para>
 369      The <literal/bindirectory/ section defines a binary directory tree
 370      with no special structure. The scope tag specifies the location of
 371      the binary directory and the settings are similar to the <literal/Tree/
 372      section with no substitution variables or
 373      <literal>Section</><literal>Architecture</> settings.
 374      <VariableList>
 375       <VarListEntry><term>Packages</term>
 376       <ListItem><Para>
 377       Sets the Packages file output.
 378       </VarListEntry>
 379
 380       <VarListEntry><term>SrcPackages</term>
 381       <ListItem><Para>
 382       Sets the Sources file output. At least one of
 383       <literal/Packages/ or <literal/SrcPackages/ is required.
 384       </VarListEntry>
 385
 386       <VarListEntry><term>Contents</term>
 387       <ListItem><Para>
 388       Sets the Contents file output. (Optional)
 389       </VarListEntry>
 390
 391       <VarListEntry><term>BinOverride</term>
 392       <ListItem><Para>
 393       Sets the binary override file.
 394       </VarListEntry>
 395
 396       <VarListEntry><term>SrcOverride</term>
 397       <ListItem><Para>
 398       Sets the source override file.
 399       </VarListEntry>
 400
 401       <VarListEntry><term>ExtraOverride</term>
 402       <ListItem><Para>
 403       Sets the binary extra override file.
 404       </VarListEntry>
 405
 406       <VarListEntry><term>SrcExtraOverride</term>
 407       <ListItem><Para>
 408       Sets the source extra override file.
 409       </VarListEntry>
 410
 411       <VarListEntry><term>BinCacheDB</term>
 412       <ListItem><Para>
 413       Sets the cache DB.
 414       </VarListEntry>
 415
 416       <VarListEntry><term>PathPrefix</term>
 417       <ListItem><Para>
 418       Appends a path to all the output paths.
 419       </VarListEntry>
 420
 421       <VarListEntry><term>FileList, SourceFileList</term>
 422       <ListItem><Para>
 423       Specifies the file list file.
 424       </VarListEntry>
 425      </VariableList>
 426    </refsect2>
 427  </RefSect1>
 428
 429  <RefSect1><Title>The Binary Override File</>
 430    <para>
 431    The binary override file is fully compatible with &dpkg-scanpackages;. It
 432    contains 4 fields separated by spaces. The first field is the package name,
 433    the second is the priority to force that package to, the third is the
 434    the section to force that package to and the final field is the maintainer
 435    permutation field.
 436    <para>
 437    The general form of the maintainer field is:
 438    <literallayout>old [// oldn]* => new</literallayout>
 439    or simply,
 440    <literallayout>new</literallayout>
 441    The first form allows a double-slash separated list of old email addresses
 442    to be specified. If any of those are found then new is substituted for the
 443    maintainer field. The second form unconditionally substitutes the
 444    maintainer field.
 445  </RefSect1>
 446
 447  <RefSect1><title>The Source Override File</>
 448    <para>
 449    The source override file is fully compatible with &dpkg-scansources;. It
 450    contains 2 fields separated by spaces. The first fields is the source
 451    package name, the second is the section to assign it.
 452  </RefSect1>
 453
 454  <RefSect1><title>The Extra Override File</>
 455    <para>
 456    The extra override file allows any arbitary tag to be added or replaced
 457    in the output. It has 3 columns, the first is the package, the second is
 458    the tag and the remainder of the line is the new value.
 459  </RefSect1>
 460
 461  <RefSect1><Title>Options</>
 462    &apt-cmdblurb;
 463
 464    <VariableList>
 465      <VarListEntry><term><option/--md5/</>
 466      <ListItem><Para>
 467      Generate MD5 sums. This defaults to on, when turned off the generated
 468      index files will not have MD5Sum fields where possible.
 469      Configuration Item: <literal/APT::FTPArchive::MD5/.
 470      </VarListEntry>
 471
 472      <VarListEntry><term><option/-d/</><term><option/--db/</>
 473      <ListItem><Para>
 474      Use a binary caching DB. This has no effect on the generate command.
 475      Configuration Item: <literal/APT::FTPArchive::DB/.
 476      </VarListEntry>
 477
 478      <VarListEntry><term><option/-q/</><term><option/--quiet/</>
 479      <ListItem><Para>
 480      Quiet; produces output suitable for logging, omitting progress indicators.
 481      More q's will produce more quiet up to a maximum of 2. You can also use
 482      <option/-q=#/ to set the quiet level, overriding the configuration file.
 483      Configuration Item: <literal/quiet/.
 484      </VarListEntry>
 485
 486      <VarListEntry><term><option/--delink/</>
 487      <ListItem><Para>
 488      Perform Delinking. If the <literal/External-Links/ setting is used then
 489      this option actually enables delinking of the files. It defaults to on and
 490      can be turned off with <option/--no-delink/.
 491      Configuration Item: <literal/APT::FTPArchive::DeLinkAct/.
 492      </VarListEntry>
 493
 494      <VarListEntry><term><option/--contents/</>
 495      <ListItem><Para>
 496      Perform contents generation. When this option is set and package indexes
 497      are being generated with a cache DB then the file listing will also be
 498      extracted and stored in the DB for later use. When using the generate
 499      command this option also allows the creation of any Contents files. The
 500      default is on.
 501      Configuration Item: <literal/APT::FTPArchive::Contents/.
 502      </VarListEntry>
 503
 504      <VarListEntry><term><option/-s/</><term><option/--source-override/</>
 505      <ListItem><Para>
 506      Select the source override file to use with the <literal/sources/ command.
 507      Configuration Item: <literal/APT::FTPArchive::SourceOverride/.
 508      </VarListEntry>
 509
 510      <VarListEntry><term><option/--readonly/</>
 511      <ListItem><Para>
 512      Make the caching databases read only.
 513      Configuration Item: <literal/APT::FTPArchive::ReadOnlyDB/.
 514      </VarListEntry>
 515
 516      &apt-commonoptions;
 517
 518    </VariableList>
 519  </RefSect1>
 520
 521  <RefSect1><Title>See Also</>
 522    <para>
 523    &apt-conf;
 524  </RefSect1>
 525
 526  <RefSect1><Title>Diagnostics</>
 527    <para>
 528    <command/apt-ftparchive/ returns zero on normal operation, decimal 100 on error.
 529  </RefSect1>
 530
 531  &manbugs;
 532  &manauthor;
 533
 534 </refentry>