doc/apt-ftparchive.1.sgml

   1 <!-- -*- mode: sgml; mode: fold -*- -->
   2 <!doctype refentry PUBLIC "-//OASIS//DTD DocBook V3.1//EN" [
   3
   4 <!ENTITY % aptent SYSTEM "apt.ent">
   5 %aptent;
   6
   7 ]>
   8
   9 <refentry>
  10  &apt-docinfo;
  11
  12  <refmeta>
  13    <refentrytitle>apt-ftparchive</>
  14    <manvolnum>1</>
  15  </refmeta>
  16
  17  <!-- Man page title -->
  18  <refnamediv>
  19     <refname>apt-ftparchive</>
  20     <refpurpose>Utility to generate index files</>
  21  </refnamediv>
  22
  23  <!-- Arguments -->
  24  <refsynopsisdiv>
  25    <cmdsynopsis>
  26       <command>apt-ftparchive</>
  27       <arg><option>-hvdsq</></arg>
  28       <arg><option>--md5</></arg>
  29       <arg><option>--delink</></arg>
  30       <arg><option>--readonly</></arg>
  31       <arg><option>--contents</></arg>
  32       <arg><option>-o=<replaceable/config string/</></arg>
  33       <arg><option>-c=<replaceable/file/</></arg>
  34       <group choice=req>
  35          <arg>packages<arg choice="plain" rep="repeat"><replaceable>path</replaceable></arg><arg><replaceable>override</replaceable><arg><replaceable>pathprefix</replaceable></arg></arg></arg>
  36          <arg>sources<arg choice="plain" rep="repeat"><replaceable>path</replaceable></arg><arg><replaceable>override</replaceable><arg><replaceable>pathprefix</replaceable></arg></arg></arg>
  37          <arg>contents <arg choice="plain"><replaceable>path</replaceable></arg></arg>
  38          <arg>release <arg choice="plain"><replaceable>path</replaceable></arg></arg>
  39          <arg>generate <arg choice="plain"><replaceable>config-file</replaceable></arg> <arg choice="plain" rep="repeat"><replaceable>section</replaceable></arg></arg>
  40          <arg>clean <arg choice="plain"><replaceable>config-file</replaceable></arg></arg>
  41       </group>
  42    </cmdsynopsis>
  43  </refsynopsisdiv>
  44
  45  <RefSect1><Title>Description</>
  46    <para>
  47    <command/apt-ftparchive/ is the command line tool that generates the index
  48    files that APT uses to access a distribution source. The index files should
  49    be generated on the origin site based on the content of that site.
  50
  51    <para>
  52    <command/apt-ftparchive/ is a superset of the &dpkg-scanpackages; program,
  53    incorporating its entire functionality via the <literal/packages/ command.
  54    It also contains a contents file generator, <literal/contents/, and an
  55    elaborate means to 'script' the generation process for a complete
  56    archive.
  57
  58    <para>
  59    Internally <command/apt-ftparchive/ can make use of binary databases to
  60    cache the contents of a .deb file and it does not rely on any external
  61    programs aside from &gzip;. When doing a full generate it automatically
  62    performs file-change checks and builds the desired compressed output files.
  63
  64    <para>
  65    Unless the <option/-h/, or <option/--help/ option is given one of the
  66    commands below must be present.
  67
  68    <VariableList>
  69      <VarListEntry><term>packages</term>
  70      <ListItem><Para>
  71      The packages command generates a package file from a directory tree. It
  72      takes the given directory and recursively searches it for .deb files,
  73      emitting a package record to stdout for each. This command is
  74      approximately equivalent to &dpkg-scanpackages;.
  75      <para>
  76      The option <option/--db/ can be used to specify a binary caching DB.
  77      </VarListEntry>
  78
  79      <VarListEntry><term>sources</term>
  80      <ListItem><Para>
  81      The <literal/sources/ command generates a source index file from a directory tree.
  82      It takes the given directory and recursively searches it for .dsc files,
  83      emitting a source record to stdout for each. This command is approximately
  84      equivalent to &dpkg-scansources;.
  85      <para>
  86      If an override file is specified then a source override file will be
  87      looked for with an extension of .src. The --source-override option can be
  88      used to change the source override file that will be used.
  89      </VarListEntry>
  90
  91      <VarListEntry><term>contents</term>
  92      <ListItem><Para>
  93      The <literal/contents/ command generates a contents file from a directory tree. It
  94      takes the given directory and recursively searches it for .deb files,
  95      and reads the file list from each file. It then sorts and writes to stdout
  96      the list of files matched to packages. Directories are not written to
  97      the output. If multiple packages own the same file then each package is
  98      separated by a comma in the output.
  99      <para>
 100      The option <option/--db/ can be used to specify a binary caching DB.
 101      </VarListEntry>
 102
 103      <VarListEntry><term>release</term>
 104      <ListItem><Para>
 105      The <literal/release/ command generates a Release file from a
 106      directory tree. It recursively searches the given directory for
 107      Packages, Packages.gz, Packages.bz2, Sources, Sources.gz,
 108      Sources.bz2, Release and md5sum.txt files.  It then writes to
 109      stdout a Release file containing an MD5 digest and SHA1 digest
 110      for each file.
 111      <para>
 112      Values for the additional metadata fields in the Release file are
 113      taken from the corresponding variables under
 114      <literal/APT::FTPArchive::Release/,
 115      e.g. <literal/APT::FTPArchive::Release::Origin/.  The supported fields
 116      are: <literal/Origin/, <literal/Label/, <literal/Suite/,
 117      <literal/Version/, <literal/Codename/, <literal/Date/,
 118      <literal/Architectures/, <literal/Components/, <literal/Description/.
 119
 120      </VarListEntry>
 121
 122      <VarListEntry><term>generate</term>
 123      <ListItem><Para>
 124      The <literal/generate/ command is designed to be runnable from a cron script and
 125      builds indexes according to the given config file. The config language
 126      provides a flexible means of specifying which index files are built from
 127      which directories, as well as providing a simple means of maintaining the
 128      required settings.
 129      </VarListEntry>
 130
 131      <VarListEntry><term>clean</term>
 132      <ListItem><Para>
 133      The <literal/clean/ command tidies the databases used by the given
 134      configuration file by removing any records that are no longer necessary.
 135      </VarListEntry>
 136    </VariableList>
 137
 138  </RefSect1>
 139
 140  <RefSect1><Title>The Generate Configuration</>
 141    <para>
 142    The <literal/generate/ command uses a configuration file to describe the
 143    archives that are going to be generated. It follows the typical ISC
 144    configuration format as seen in ISC tools like bind 8 and dhcpd.
 145    &apt-conf; contains a description of the syntax. Note that the generate
 146    configuration is parsed in sectional manner, but &apt-conf; is parsed in a
 147    tree manner. This only effects how the scope tag is handled.
 148
 149    <para>
 150    The generate configuration has 4 separate sections, each described below.
 151
 152    <refsect2><title>Dir Section</>
 153      <Para>
 154      The <literal/Dir/ section defines the standard directories needed to
 155      locate the files required during the generation process. These
 156      directories are prepended to certain relative paths defined in later
 157      sections to produce a complete an absolute path.
 158      <VariableList>
 159       <VarListEntry><term>ArchiveDir</term>
 160       <ListItem><Para>
 161       Specifies the root of the FTP archive, in a standard
 162       Debian configuration this is the directory that contains the
 163       <filename/ls-LR/, and dist nodes.
 164       </VarListEntry>
 165
 166       <VarListEntry><term>OverrideDir</term>
 167       <ListItem><Para>
 168       Specifies the location of the override files.
 169       </VarListEntry>
 170
 171       <VarListEntry><term>CacheDir</term>
 172       <ListItem><Para>
 173       Specifies the location of the cache files
 174       </VarListEntry>
 175
 176       <VarListEntry><term>FileListDir</term>
 177       <ListItem><Para>
 178       Specifies the location of the file list files,
 179       if the <literal/FileList/ setting is used below.
 180       </VarListEntry>
 181      </VariableList>
 182    </refsect2>
 183
 184    <refsect2><title>Default Section</>
 185      <para>
 186      The <literal/Default/ section specifies default values, and settings
 187      that control the operation of the generator. Other sections may override
 188      these defaults with a per-section setting.
 189      <VariableList>
 190       <VarListEntry><term>Packages::Compress</term>
 191       <ListItem><Para>
 192       Sets the default compression schemes to use
 193       for the Package index files. It is a string that contains a space
 194       separated list of at least one of: '.' (no compression), 'gzip' and
 195       'bzip2'. The default for all compression schemes is '. gzip'.
 196       </VarListEntry>
 197
 198       <VarListEntry><term>Packages::Extensions</term>
 199       <ListItem><Para>
 200       Sets the default list of file extensions that are package files.
 201       This defaults to '.deb'.
 202       </VarListEntry>
 203
 204       <VarListEntry><term>Sources::Compress</term>
 205       <ListItem><Para>
 206       This is similar to <literal/Packages::Compress/
 207       except that it controls the compression for the Sources files.
 208       </VarListEntry>
 209
 210       <VarListEntry><term>Sources::Extensions</term>
 211       <ListItem><Para>
 212       Sets the default list of file extensions that are source files.
 213       This defaults to '.dsc'.
 214       </VarListEntry>
 215
 216       <VarListEntry><term>Contents::Compress</term>
 217       <ListItem><Para>
 218       This is similar to <literal/Packages::Compress/
 219       except that it controls the compression for the Contents files.
 220       </VarListEntry>
 221
 222       <VarListEntry><term>DeLinkLimit</term>
 223       <ListItem><Para>
 224       Specifies the number of kilobytes to delink (and
 225       replace with hard links) per run. This is used in conjunction with the
 226       per-section <literal/External-Links/ setting.
 227       </VarListEntry>
 228
 229       <VarListEntry><term>FileMode</term>
 230       <ListItem><Para>
 231       Specifies the mode of all created index files. It
 232       defaults to 0644. All index files are set to this mode with no regard
 233       to the umask.
 234       </VarListEntry>
 235      </VariableList>
 236    </refsect2>
 237
 238    <refsect2><title>TreeDefault Section</>
 239      <para>
 240      Sets defaults specific to <literal/Tree/ sections. All of these
 241      variables are substitution variables and have the strings $(DIST),
 242      $(SECTION) and $(ARCH) replaced with their respective values.
 243
 244      <VariableList>
 245       <VarListEntry><term>MaxContentsChange</term>
 246       <ListItem><Para>
 247       Sets  the number of kilobytes of contents
 248       files that are generated each day. The contents files are round-robined
 249       so that over several days they will all be rebuilt.
 250       </VarListEntry>
 251
 252       <VarListEntry><term>ContentsAge</term>
 253       <ListItem><Para>
 254       Controls the number of days a contents file is allowed
 255       to be checked without changing. If this limit is passed the mtime of the
 256       contents file is updated. This case can occur if the package file is
 257       changed in such a way that does not result in a new contents file
 258       [override edit for instance]. A hold off is allowed in hopes that new
 259       .debs will be installed, requiring a new file anyhow. The default is 10,
 260       the units are in days.
 261       </VarListEntry>
 262
 263       <VarListEntry><term>Directory</term>
 264       <ListItem><Para>
 265       Sets the top of the .deb directory tree. Defaults to
 266       <filename>$(DIST)/$(SECTION)/binary-$(ARCH)/</>
 267       </VarListEntry>
 268
 269       <VarListEntry><term>SrcDirectory</term>
 270       <ListItem><Para>
 271       Sets the top of the source package directory tree. Defaults to
 272       <filename>$(DIST)/$(SECTION)/source/</>
 273       </VarListEntry>
 274
 275       <VarListEntry><term>Packages</term>
 276       <ListItem><Para>
 277       Sets the output Packages file. Defaults to
 278       <filename>$(DIST)/$(SECTION)/binary-$(ARCH)/Packages</>
 279       </VarListEntry>
 280
 281       <VarListEntry><term>Sources</term>
 282       <ListItem><Para>
 283       Sets the output Packages file. Defaults to
 284       <filename>$(DIST)/$(SECTION)/source/Sources</>
 285       </VarListEntry>
 286
 287       <VarListEntry><term>InternalPrefix</term>
 288       <ListItem><Para>
 289       Sets the path prefix that causes a symlink to be
 290       considered an internal link instead of an external link. Defaults to
 291       <filename>$(DIST)/$(SECTION)/</>
 292       </VarListEntry>
 293
 294       <VarListEntry><term>Contents</term>
 295       <ListItem><Para>
 296       Sets the output Contents file. Defaults to
 297       <filename>$(DIST)/Contents-$(ARCH)</>. If this setting causes multiple
 298       Packages files to map onto a single Contents file (such as the default)
 299       then <command/apt-ftparchive/ will integrate those package files
 300       together automatically.
 301       </VarListEntry>
 302
 303       <VarListEntry><term>Contents::Header</term>
 304       <ListItem><Para>
 305       Sets header file to prepend to the contents output.
 306       </VarListEntry>
 307
 308       <VarListEntry><term>BinCacheDB</term>
 309       <ListItem><Para>
 310       Sets the binary cache database to use for this
 311       section. Multiple sections can share the same database.
 312       </VarListEntry>
 313
 314       <VarListEntry><term>FileList</term>
 315       <ListItem><Para>
 316       Specifies that instead of walking the directory tree,
 317       <command/apt-ftparchive/ should read the list of files from the given
 318       file. Relative files names are prefixed with the archive directory.
 319       </VarListEntry>
 320
 321       <VarListEntry><term>SourceFileList</term>
 322       <ListItem><Para>
 323       Specifies that instead of walking the directory tree,
 324       <command/apt-ftparchive/ should read the list of files from the given
 325       file. Relative files names are prefixed with the archive directory.
 326       This is used when processing source indexs.
 327       </VarListEntry>
 328      </VariableList>
 329    </refsect2>
 330
 331    <refsect2><title>Tree Section</>
 332      <para>
 333      The <literal/Tree/ section defines a standard Debian file tree which
 334      consists of a base directory, then multiple sections in that base
 335      directory and finally multiple Architectures in each section. The exact
 336      pathing used is defined by the <literal/Directory/ substitution variable.
 337      <para>
 338      The <literal/Tree/ section takes a scope tag which sets the
 339      <literal/$(DIST)/ variable and defines the root of the tree
 340      (the path is prefixed by <literal/ArchiveDir/).
 341      Typically this is a setting such as <filename>dists/woody</>.
 342      <para>
 343      All of the settings defined in the <literal/TreeDefault/ section can be
 344      use in a <literal/Tree/ section as well as three new variables.
 345      <para>
 346      When processing a <literal/Tree/ section <command/apt-ftparchive/
 347      performs an operation similar to:
 348 <informalexample><programlisting>
 349 for i in Sections do
 350    for j in Architectures do
 351       Generate for DIST=scope SECTION=i ARCH=j
 352 </programlisting></informalexample>
 353
 354      <VariableList>
 355       <VarListEntry><term>Sections</term>
 356       <ListItem><Para>
 357       This is a space separated list of sections which appear
 358       under the distribution, typically this is something like
 359       <literal/main contrib non-free/.
 360       </VarListEntry>
 361
 362       <VarListEntry><term>Architectures</term>
 363       <ListItem><Para>
 364       This is a space separated list of all the
 365       architectures that appear under search section. The special architecture
 366       'source' is used to indicate that this tree has a source archive.
 367       </VarListEntry>
 368
 369       <VarListEntry><term>BinOverride</term>
 370       <ListItem><Para>
 371       Sets the binary override file. The override file
 372       contains section, priority and maintainer address information.
 373       </VarListEntry>
 374
 375       <VarListEntry><term>SrcOverride</term>
 376       <ListItem><Para>
 377       Sets the source override file. The override file
 378       contains section information.
 379       </VarListEntry>
 380
 381       <VarListEntry><term>ExtraOverride</term>
 382       <ListItem><Para>
 383       Sets the binary extra override file.
 384       </VarListEntry>
 385
 386       <VarListEntry><term>SrcExtraOverride</term>
 387       <ListItem><Para>
 388       Sets the source extra override file.
 389       </VarListEntry>
 390      </VariableList>
 391    </refsect2>
 392
 393    <refsect2><title>BinDirectory Section</>
 394      <para>
 395      The <literal/bindirectory/ section defines a binary directory tree
 396      with no special structure. The scope tag specifies the location of
 397      the binary directory and the settings are similar to the <literal/Tree/
 398      section with no substitution variables or
 399      <literal>Section</><literal>Architecture</> settings.
 400      <VariableList>
 401       <VarListEntry><term>Packages</term>
 402       <ListItem><Para>
 403       Sets the Packages file output.
 404       </VarListEntry>
 405
 406       <VarListEntry><term>SrcPackages</term>
 407       <ListItem><Para>
 408       Sets the Sources file output. At least one of
 409       <literal/Packages/ or <literal/SrcPackages/ is required.
 410       </VarListEntry>
 411
 412       <VarListEntry><term>Contents</term>
 413       <ListItem><Para>
 414       Sets the Contents file output. (Optional)
 415       </VarListEntry>
 416
 417       <VarListEntry><term>BinOverride</term>
 418       <ListItem><Para>
 419       Sets the binary override file.
 420       </VarListEntry>
 421
 422       <VarListEntry><term>SrcOverride</term>
 423       <ListItem><Para>
 424       Sets the source override file.
 425       </VarListEntry>
 426
 427       <VarListEntry><term>ExtraOverride</term>
 428       <ListItem><Para>
 429       Sets the binary extra override file.
 430       </VarListEntry>
 431
 432       <VarListEntry><term>SrcExtraOverride</term>
 433       <ListItem><Para>
 434       Sets the source extra override file.
 435       </VarListEntry>
 436
 437       <VarListEntry><term>BinCacheDB</term>
 438       <ListItem><Para>
 439       Sets the cache DB.
 440       </VarListEntry>
 441
 442       <VarListEntry><term>PathPrefix</term>
 443       <ListItem><Para>
 444       Appends a path to all the output paths.
 445       </VarListEntry>
 446
 447       <VarListEntry><term>FileList, SourceFileList</term>
 448       <ListItem><Para>
 449       Specifies the file list file.
 450       </VarListEntry>
 451      </VariableList>
 452    </refsect2>
 453  </RefSect1>
 454
 455  <RefSect1><Title>The Binary Override File</>
 456    <para>
 457    The binary override file is fully compatible with &dpkg-scanpackages;. It
 458    contains 4 fields separated by spaces. The first field is the package name,
 459    the second is the priority to force that package to, the third is the
 460    the section to force that package to and the final field is the maintainer
 461    permutation field.
 462    <para>
 463    The general form of the maintainer field is:
 464    <literallayout>old [// oldn]* => new</literallayout>
 465    or simply,
 466    <literallayout>new</literallayout>
 467    The first form allows a double-slash separated list of old email addresses
 468    to be specified. If any of those are found then new is substituted for the
 469    maintainer field. The second form unconditionally substitutes the
 470    maintainer field.
 471  </RefSect1>
 472
 473  <RefSect1><title>The Source Override File</>
 474    <para>
 475    The source override file is fully compatible with &dpkg-scansources;. It
 476    contains 2 fields separated by spaces. The first fields is the source
 477    package name, the second is the section to assign it.
 478  </RefSect1>
 479
 480  <RefSect1><title>The Extra Override File</>
 481    <para>
 482    The extra override file allows any arbitrary tag to be added or replaced
 483    in the output. It has 3 columns, the first is the package, the second is
 484    the tag and the remainder of the line is the new value.
 485  </RefSect1>
 486
 487  <RefSect1><Title>Options</>
 488    &apt-cmdblurb;
 489
 490    <VariableList>
 491      <VarListEntry><term><option/--md5/</>
 492      <ListItem><Para>
 493      Generate MD5 sums. This defaults to on, when turned off the generated
 494      index files will not have MD5Sum fields where possible.
 495      Configuration Item: <literal/APT::FTPArchive::MD5/.
 496      </VarListEntry>
 497
 498      <VarListEntry><term><option/-d/</><term><option/--db/</>
 499      <ListItem><Para>
 500      Use a binary caching DB. This has no effect on the generate command.
 501      Configuration Item: <literal/APT::FTPArchive::DB/.
 502      </VarListEntry>
 503
 504      <VarListEntry><term><option/-q/</><term><option/--quiet/</>
 505      <ListItem><Para>
 506      Quiet; produces output suitable for logging, omitting progress indicators.
 507      More q's will produce more quiet up to a maximum of 2. You can also use
 508      <option/-q=#/ to set the quiet level, overriding the configuration file.
 509      Configuration Item: <literal/quiet/.
 510      </VarListEntry>
 511
 512      <VarListEntry><term><option/--delink/</>
 513      <ListItem><Para>
 514      Perform Delinking. If the <literal/External-Links/ setting is used then
 515      this option actually enables delinking of the files. It defaults to on and
 516      can be turned off with <option/--no-delink/.
 517      Configuration Item: <literal/APT::FTPArchive::DeLinkAct/.
 518      </VarListEntry>
 519
 520      <VarListEntry><term><option/--contents/</>
 521      <ListItem><Para>
 522      Perform contents generation. When this option is set and package indexes
 523      are being generated with a cache DB then the file listing will also be
 524      extracted and stored in the DB for later use. When using the generate
 525      command this option also allows the creation of any Contents files. The
 526      default is on.
 527      Configuration Item: <literal/APT::FTPArchive::Contents/.
 528      </VarListEntry>
 529
 530      <VarListEntry><term><option/-s/</><term><option/--source-override/</>
 531      <ListItem><Para>
 532      Select the source override file to use with the <literal/sources/ command.
 533      Configuration Item: <literal/APT::FTPArchive::SourceOverride/.
 534      </VarListEntry>
 535
 536      <VarListEntry><term><option/--readonly/</>
 537      <ListItem><Para>
 538      Make the caching databases read only.
 539      Configuration Item: <literal/APT::FTPArchive::ReadOnlyDB/.
 540      </VarListEntry>
 541
 542      &apt-commonoptions;
 543
 544    </VariableList>
 545  </RefSect1>
 546
 547 <RefSect1><Title>Examples</>
 548
 549 <para>To create a compressed Packages file for a directory containing
 550 binary packages (.deb):
 551
 552 <programlisting
 553 <command/apt-ftparchive/ packages <replaceable/directory/ | <command/gzip/ > <filename/Packages.gz/
 554 </programlisting>
 555
 556 </RefSect1>
 557
 558  <RefSect1><Title>See Also</>
 559    <para>
 560    &apt-conf;
 561  </RefSect1>
 562
 563  <RefSect1><Title>Diagnostics</>
 564    <para>
 565    <command/apt-ftparchive/ returns zero on normal operation, decimal 100 on error.
 566  </RefSect1>
 567
 568  &manbugs;
 569  &manauthor;
 570
 571 </refentry>