1 <?xml version=
"1.0" encoding=
"utf-8" standalone=
"no"?>
2 <!DOCTYPE refentry PUBLIC
"-//OASIS//DTD DocBook XML V4.5//EN"
3 "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
4 <!ENTITY % aptent SYSTEM
"apt.ent"> %aptent;
5 <!ENTITY % aptverbatiment SYSTEM
"apt-verbatim.ent"> %aptverbatiment;
6 <!ENTITY % aptvendor SYSTEM
"apt-vendor.ent"> %aptvendor;
12 &apt-author.jgunthorpe;
16 <!-- The last update date -->
17 <date>2016-
11-
25T00:
00:
00Z
</date>
21 <refentrytitle>apt-ftparchive
</refentrytitle>
22 <manvolnum>1</manvolnum>
23 <refmiscinfo class=
"manual">APT
</refmiscinfo>
26 <!-- Man page title -->
28 <refname>apt-ftparchive
</refname>
29 <refpurpose>Utility to generate index files
</refpurpose>
32 &synopsis-command-apt-ftparchive;
34 <refsect1><title>Description
</title>
35 <para><command>apt-ftparchive
</command> is the command line tool that generates the index
36 files that APT uses to access a distribution source. The index files should
37 be generated on the origin site based on the content of that site.
</para>
39 <para><command>apt-ftparchive
</command> is a superset of the &dpkg-scanpackages; program,
40 incorporating its entire functionality via the
<literal>packages
</literal> command.
41 It also contains a contents file generator,
<literal>contents
</literal>, and an
42 elaborate means to 'script' the generation process for a complete
45 <para>Internally
<command>apt-ftparchive
</command> can make use of binary databases to
46 cache the contents of a .deb file and it does not rely on any external
47 programs aside from
&gzip;. When doing a full generate it automatically
48 performs file-change checks and builds the desired compressed output files.
</para>
50 <para>Unless the
<option>-h
</option>, or
<option>--help
</option> option is given,
51 one of the commands below must be present.
</para>
54 <varlistentry><term><option>packages
</option></term>
56 The packages command generates a package file from a directory tree. It
57 takes the given directory and recursively searches it for .deb files,
58 emitting a package record to stdout for each. This command is
59 approximately equivalent to &dpkg-scanpackages;.
</para>
61 <para>The option
<option>--db
</option> can be used to specify a binary caching DB.
</para></listitem>
64 <varlistentry><term><option>sources
</option></term>
66 The
<literal>sources
</literal> command generates a source index file from a directory tree.
67 It takes the given directory and recursively searches it for .dsc files,
68 emitting a source record to stdout for each. This command is approximately
69 equivalent to &dpkg-scansources;.
</para>
71 If an override file is specified then a source override file will be
72 looked for with an extension of .src. The --source-override option can be
73 used to change the source override file that will be used.
</para></listitem>
76 <varlistentry><term><option>contents
</option></term>
78 The
<literal>contents
</literal> command generates a contents file from a directory tree. It
79 takes the given directory and recursively searches it for .deb files,
80 and reads the file list from each file. It then sorts and writes to stdout
81 the list of files matched to packages. Directories are not written to
82 the output. If multiple packages own the same file then each package is
83 separated by a comma in the output.
</para>
85 The option
<option>--db
</option> can be used to specify a binary caching DB.
</para></listitem>
88 <varlistentry><term><option>release
</option></term>
90 The
<literal>release
</literal> command generates a Release file from a
91 directory tree. It recursively searches the given directory for
92 uncompressed and compressed
<filename>Packages
</filename>,
93 <filename>Sources
</filename>,
<filename>Contents
</filename>,
94 <filename>Components
</filename> and
<filename>icons
</filename> files as
95 well as
<filename>Release
</filename>,
<filename>Index
</filename> and
96 <filename>md5sum.txt
</filename> files by default
97 (
<literal>APT::FTPArchive::Release::Default-Patterns
</literal>).
98 Additional filename patterns can be added by listing them in
99 <literal>APT::FTPArchive::Release::Patterns
</literal>. It then writes to
100 stdout a
<filename>Release
</filename> file containing (by default) an MD5,
101 SHA1, SHA256 and SHA512 digest for each file.
</para>
103 Values for the additional metadata fields in the Release file are
104 taken from the corresponding variables under
105 <literal>APT::FTPArchive::Release
</literal>,
106 e.g.
<literal>APT::FTPArchive::Release::Origin
</literal>. The supported fields
107 are
<literal>Origin
</literal>,
<literal>Label
</literal>,
<literal>Suite
</literal>,
108 <literal>Version
</literal>,
<literal>Codename
</literal>,
<literal>Date
</literal>,
109 <literal>NotAutomatic
</literal>,
<literal>ButAutomaticUpgrades
</literal>,
110 <literal>Acquire-By-Hash
</literal>,
<literal>Valid-Until
</literal>,
111 <literal>Signed-By
</literal>,
<literal>Architectures
</literal>,
112 <literal>Components
</literal> and
<literal>Description
</literal>.
</para></listitem>
116 <varlistentry><term><option>generate
</option></term>
118 The
<literal>generate
</literal> command is designed to be runnable from a cron script and
119 builds indexes according to the given config file. The config language
120 provides a flexible means of specifying which index files are built from
121 which directories, as well as providing a simple means of maintaining the
122 required settings.
</para></listitem>
125 <varlistentry><term><option>clean
</option></term>
127 The
<literal>clean
</literal> command tidies the databases used by the given
128 configuration file by removing any records that are no longer necessary.
</para></listitem>
133 <refsect1><title>The Generate Configuration
</title>
135 The
<literal>generate
</literal> command uses a configuration file to describe the
136 archives that are going to be generated. It follows the typical ISC
137 configuration format as seen in ISC tools like bind
8 and dhcpd.
138 &apt-conf; contains a description of the syntax. Note that the generate
139 configuration is parsed in sectional manner, but &apt-conf; is parsed in a
140 tree manner. This only effects how the scope tag is handled.
</para>
143 The generate configuration has four separate sections, each described below.
</para>
145 <refsect2><title><literal>Dir
</literal> Section
</title>
147 The
<literal>Dir
</literal> section defines the standard directories needed to
148 locate the files required during the generation process. These
149 directories are prepended certain relative paths defined in later
150 sections to produce a complete an absolute path.
</para>
152 <varlistentry><term><option>ArchiveDir
</option></term>
154 Specifies the root of the FTP archive, in a standard
155 Debian configuration this is the directory that contains the
156 <filename>ls-LR
</filename> and dist nodes.
</para></listitem>
159 <varlistentry><term><option>OverrideDir
</option></term>
161 Specifies the location of the override files.
</para></listitem>
164 <varlistentry><term><option>CacheDir
</option></term>
166 Specifies the location of the cache files.
</para></listitem>
169 <varlistentry><term><option>FileListDir
</option></term>
171 Specifies the location of the file list files,
172 if the
<literal>FileList
</literal> setting is used below.
</para></listitem>
177 <refsect2><title><literal>Default
</literal> Section
</title>
179 The
<literal>Default
</literal> section specifies default values, and settings
180 that control the operation of the generator. Other sections may override
181 these defaults with a per-section setting.
</para>
183 <varlistentry><term><option>Packages::Compress
</option></term>
185 Sets the default compression schemes to use
186 for the package index files. It is a string that contains a space
187 separated list of at least one of the compressors configured via the
188 <option>APT::Compressor
</option> configuration scope.
189 The default for all compression schemes is '. gzip'.
</para></listitem>
192 <varlistentry><term><option>Packages::Extensions
</option></term>
194 Sets the default list of file extensions that are package files.
195 This defaults to '.deb'.
</para></listitem>
198 <varlistentry><term><option>Sources::Compress
</option></term>
200 This is similar to
<literal>Packages::Compress
</literal>
201 except that it controls the compression for the Sources files.
</para></listitem>
204 <varlistentry><term><option>Sources::Extensions
</option></term>
206 Sets the default list of file extensions that are source files.
207 This defaults to '.dsc'.
</para></listitem>
210 <varlistentry><term><option>Contents::Compress
</option></term>
212 This is similar to
<literal>Packages::Compress
</literal>
213 except that it controls the compression for the Contents files.
</para></listitem>
216 <varlistentry><term><option>Translation::Compress
</option></term>
218 This is similar to
<literal>Packages::Compress
</literal>
219 except that it controls the compression for the Translation-en master file.
</para></listitem>
222 <varlistentry><term><option>DeLinkLimit
</option></term>
224 Specifies the number of kilobytes to delink (and
225 replace with hard links) per run. This is used in conjunction with the
226 per-section
<literal>External-Links
</literal> setting.
</para></listitem>
229 <varlistentry><term><option>FileMode
</option></term>
231 Specifies the mode of all created index files. It
232 defaults to
0644. All index files are set to this mode with no regard
233 to the umask.
</para></listitem>
236 <varlistentry><term><option>LongDescription
</option></term>
238 Specifies whether long descriptions should be included in the
<filename>Packages
</filename> file or split
239 out into a master
<filename>Translation-en
</filename> file.
</para></listitem>
244 <refsect2><title><literal>TreeDefault
</literal> Section
</title>
246 Sets defaults specific to
<literal>Tree
</literal> sections. All of these
247 variables are substitution variables and have the strings $(DIST),
248 $(SECTION) and $(ARCH) replaced with their respective values.
</para>
251 <varlistentry><term><option>MaxContentsChange
</option></term>
253 Sets the number of kilobytes of contents
254 files that are generated each day. The contents files are round-robined
255 so that over several days they will all be rebuilt.
</para></listitem>
258 <varlistentry><term><option>ContentsAge
</option></term>
260 Controls the number of days a contents file is allowed
261 to be checked without changing. If this limit is passed the mtime of the
262 contents file is updated. This case can occur if the package file is
263 changed in such a way that does not result in a new contents file
264 [override edit for instance]. A hold off is allowed in hopes that new
265 .debs will be installed, requiring a new file anyhow. The default is
10,
266 the units are in days.
</para></listitem>
269 <varlistentry><term><option>Directory
</option></term>
271 Sets the top of the .deb directory tree. Defaults to
272 <filename>$(DIST)/$(SECTION)/binary-$(ARCH)/
</filename></para></listitem>
275 <varlistentry><term><option>SrcDirectory
</option></term>
277 Sets the top of the source package directory tree. Defaults to
278 <filename>$(DIST)/$(SECTION)/source/
</filename></para></listitem>
281 <varlistentry><term><option>Packages
</option></term>
283 Sets the output Packages file. Defaults to
284 <filename>$(DIST)/$(SECTION)/binary-$(ARCH)/Packages
</filename></para></listitem>
287 <varlistentry><term><option>Sources
</option></term>
289 Sets the output Sources file. Defaults to
290 <filename>$(DIST)/$(SECTION)/source/Sources
</filename></para></listitem>
293 <varlistentry><term><option>Translation
</option></term>
295 Sets the output Translation-en master file with the long descriptions if they
296 should be not included in the Packages file. Defaults to
297 <filename>$(DIST)/$(SECTION)/i18n/Translation-en
</filename></para></listitem>
300 <varlistentry><term><option>InternalPrefix
</option></term>
302 Sets the path prefix that causes a symlink to be
303 considered an internal link instead of an external link. Defaults to
304 <filename>$(DIST)/$(SECTION)/
</filename></para></listitem>
307 <varlistentry><term><option>Contents
</option></term>
309 Sets the output Contents file. Defaults to
310 <filename>$(DIST)/$(SECTION)/Contents-$(ARCH)
</filename>. If this setting causes multiple
311 Packages files to map onto a single Contents file (as is the default)
312 then
<command>apt-ftparchive
</command> will integrate those package files
313 together automatically.
</para></listitem>
316 <varlistentry><term><option>Contents::Header
</option></term>
318 Sets header file to prepend to the contents output.
</para></listitem>
321 <varlistentry><term><option>BinCacheDB
</option></term>
323 Sets the binary cache database to use for this
324 section. Multiple sections can share the same database.
</para></listitem>
327 <varlistentry><term><option>FileList
</option></term>
329 Specifies that instead of walking the directory tree,
330 <command>apt-ftparchive
</command> should read the list of files from the given
331 file. Relative files names are prefixed with the archive directory.
</para></listitem>
334 <varlistentry><term><option>SourceFileList
</option></term>
336 Specifies that instead of walking the directory tree,
337 <command>apt-ftparchive
</command> should read the list of files from the given
338 file. Relative files names are prefixed with the archive directory.
339 This is used when processing source indexes.
</para></listitem>
344 <refsect2><title><literal>Tree
</literal> Section
</title>
346 The
<literal>Tree
</literal> section defines a standard Debian file tree which
347 consists of a base directory, then multiple sections in that base
348 directory and finally multiple Architectures in each section. The exact
349 pathing used is defined by the
<literal>Directory
</literal> substitution variable.
</para>
351 The
<literal>Tree
</literal> section takes a scope tag which sets the
352 <literal>$(DIST)
</literal> variable and defines the root of the tree
353 (the path is prefixed by
<literal>ArchiveDir
</literal>).
354 Typically this is a setting such as
<filename>dists/&debian-stable-codename;
</filename>.
</para>
356 All of the settings defined in the
<literal>TreeDefault
</literal> section can be
357 used in a
<literal>Tree
</literal> section as well as three new variables.
</para>
359 When processing a
<literal>Tree
</literal> section
<command>apt-ftparchive
</command>
360 performs an operation similar to:
363 for j in Architectures do
364 Generate for DIST=scope SECTION=i ARCH=j
365 </programlisting></para>
368 <varlistentry><term><option>Sections
</option></term>
370 This is a space separated list of sections which appear
371 under the distribution; typically this is something like
372 <literal>main contrib non-free
</literal></para></listitem>
375 <varlistentry><term><option>Architectures
</option></term>
377 This is a space separated list of all the architectures that appear under
378 search section. The special architecture 'source' is used to indicate
379 that this tree has a source archive. The architecture 'all' signals that
380 architecture specific files like
<filename>Packages
</filename> should not
381 include information about architecture
<literal>all
</literal> packages in
382 all files as they will be available in a dedicated file.
386 <varlistentry><term><option>LongDescription
</option></term>
388 Specifies whether long descriptions should be included in the
<filename>Packages
</filename> file or split
389 out into a master
<filename>Translation-en
</filename> file.
</para></listitem>
392 <varlistentry><term><option>BinOverride
</option></term>
394 Sets the binary override file. The override file
395 contains section, priority and maintainer address information.
</para></listitem>
398 <varlistentry><term><option>SrcOverride
</option></term>
400 Sets the source override file. The override file
401 contains section information.
</para></listitem>
404 <varlistentry><term><option>ExtraOverride
</option></term>
406 Sets the binary extra override file.
</para></listitem>
409 <varlistentry><term><option>SrcExtraOverride
</option></term>
411 Sets the source extra override file.
</para></listitem>
416 <refsect2><title><literal>BinDirectory
</literal> Section
</title>
418 The
<literal>bindirectory
</literal> section defines a binary directory tree
419 with no special structure. The scope tag specifies the location of
420 the binary directory and the settings are similar to the
<literal>Tree
</literal>
421 section with no substitution variables or
422 <literal>Section
</literal><literal>Architecture
</literal> settings.
</para>
424 <varlistentry><term><option>Packages
</option></term>
426 Sets the Packages file output.
</para></listitem>
429 <varlistentry><term><option>Sources
</option></term>
431 Sets the Sources file output. At least one of
432 <literal>Packages
</literal> or
<literal>Sources
</literal> is required.
</para></listitem>
435 <varlistentry><term><option>Contents
</option></term>
437 Sets the Contents file output (optional).
</para></listitem>
440 <varlistentry><term><option>BinOverride
</option></term>
442 Sets the binary override file.
</para></listitem>
445 <varlistentry><term><option>SrcOverride
</option></term>
447 Sets the source override file.
</para></listitem>
450 <varlistentry><term><option>ExtraOverride
</option></term>
452 Sets the binary extra override file.
</para></listitem>
455 <varlistentry><term><option>SrcExtraOverride
</option></term>
457 Sets the source extra override file.
</para></listitem>
460 <varlistentry><term><option>BinCacheDB
</option></term>
462 Sets the cache DB.
</para></listitem>
465 <varlistentry><term><option>PathPrefix
</option></term>
467 Appends a path to all the output paths.
</para></listitem>
470 <varlistentry><term><option>FileList
</option></term><term><option>SourceFileList
</option></term>
472 Specifies the file list file.
</para></listitem>
479 <refsect1><title>The Binary Override File
</title>
480 <para>The binary override file is fully compatible with &dpkg-scanpackages;. It
481 contains four fields separated by spaces. The first field is the package name,
482 the second is the priority to force that package to, the third is
483 the section to force that package to and the final field is the maintainer
484 permutation field.
</para>
485 <para>The general form of the maintainer field is:
486 <literallayout>old [// oldn]* =
> new
</literallayout>
488 <literallayout>new
</literallayout>
489 The first form allows a double-slash separated list of old email addresses
490 to be specified. If any of those are found then new is substituted for the
491 maintainer field. The second form unconditionally substitutes the
492 maintainer field.
</para>
496 <refsect1><title>The Source Override File
</title>
498 The source override file is fully compatible with &dpkg-scansources;. It
499 contains two fields separated by spaces. The first field is the source
500 package name, the second is the section to assign it.
</para>
503 <refsect1><title>The Extra Override File
</title>
505 The extra override file allows any arbitrary tag to be added or replaced
506 in the output. It has three columns, the first is the package, the second is
507 the tag and the remainder of the line is the new value.
</para>
510 <refsect1><title>options
</title>
515 <term><option>--md5
</option></term>
516 <term><option>--sha1
</option></term>
517 <term><option>--sha256
</option></term>
518 <term><option>--sha512
</option></term>
520 Generate the given checksum. These options default to on, when turned off the generated
521 index files will not have the checksum fields where possible.
522 Configuration Items:
<literal>APT::FTPArchive::
<replaceable>Checksum
</replaceable></literal> and
523 <literal>APT::FTPArchive::
<replaceable>Index
</replaceable>::
<replaceable>Checksum
</replaceable></literal> where
524 <literal><replaceable>Index
</replaceable></literal> can be
<literal>Packages
</literal>,
<literal>Sources
</literal> or
525 <literal>Release
</literal> and
<literal><replaceable>Checksum
</replaceable></literal> can be
<literal>MD5
</literal>,
526 <literal>SHA1
</literal>,
<literal>SHA256
</literal> or
<literal>SHA512
</literal>.
</para></listitem>
529 <varlistentry><term><option>-d
</option></term><term><option>--db
</option></term>
531 Use a binary caching DB. This has no effect on the generate command.
532 Configuration Item:
<literal>APT::FTPArchive::DB
</literal>.
</para></listitem>
535 <varlistentry><term><option>-q
</option></term><term><option>--quiet
</option></term>
537 Quiet; produces output suitable for logging, omitting progress indicators.
538 More q's will produce more quiet up to a maximum of
2. You can also use
539 <option>-q=#
</option> to set the quiet level, overriding the configuration file.
540 Configuration Item:
<literal>quiet
</literal>.
</para></listitem>
543 <varlistentry><term><option>--delink
</option></term>
545 Perform Delinking. If the
<literal>External-Links
</literal> setting is used then
546 this option actually enables delinking of the files. It defaults to on and
547 can be turned off with
<option>--no-delink
</option>.
548 Configuration Item:
<literal>APT::FTPArchive::DeLinkAct
</literal>.
</para></listitem>
551 <varlistentry><term><option>--contents
</option></term>
553 Perform contents generation. When this option is set and package indexes
554 are being generated with a cache DB then the file listing will also be
555 extracted and stored in the DB for later use. When using the generate
556 command this option also allows the creation of any Contents files. The
558 Configuration Item:
<literal>APT::FTPArchive::Contents
</literal>.
</para></listitem>
561 <varlistentry><term><option>-s
</option></term><term><option>--source-override
</option></term>
563 Select the source override file to use with the
<literal>sources
</literal> command.
564 Configuration Item:
<literal>APT::FTPArchive::SourceOverride
</literal>.
</para></listitem>
567 <varlistentry><term><option>--readonly
</option></term>
569 Make the caching databases read only.
570 Configuration Item:
<literal>APT::FTPArchive::ReadOnlyDB
</literal>.
</para></listitem>
573 <varlistentry><term><option>-a
</option></term><term><option>--arch
</option></term>
574 <listitem><para>Accept in the
<literal>packages
</literal> and
<literal>contents
</literal>
575 commands only package files matching
<literal>*_arch.deb
</literal> or
576 <literal>*_all.deb
</literal> instead of all package files in the given path.
577 Configuration Item:
<literal>APT::FTPArchive::Architecture
</literal>.
</para></listitem>
580 <varlistentry><term><option>APT::FTPArchive::AlwaysStat
</option></term>
582 &apt-ftparchive; caches as much as possible of metadata in a cachedb. If packages
583 are recompiled and/or republished with the same version again, this will lead to problems
584 as the now outdated cached metadata like size and checksums will be used. With this option
585 enabled this will no longer happen as it will be checked if the file was changed.
586 Note that this option is set to "
<literal>false
</literal>" by default as it is not recommend
587 to upload multiply versions/builds of a package with the same versionnumber, so in theory
588 nobody will have these problems and therefore all these extra checks are useless.
592 <varlistentry><term><option>APT::FTPArchive::LongDescription</option></term>
594 This configuration option defaults to "<literal>true
</literal>" and should only be set to
595 <literal>"false
"</literal> if the Archive generated with &apt-ftparchive; also provides
596 <filename>Translation</filename> files. Note that the <filename>Translation-en</filename>
597 master file can only be created in the generate command.
606 <refsect1><title>Examples</title>
608 <para>To create a compressed Packages file for a directory containing
609 binary packages (.deb):
612 <command>apt-ftparchive</command> packages <replaceable>directory</replaceable> | <command>gzip</command> > <filename>Packages.gz</filename>
613 </programlisting></para>
617 <refsect1><title>See Also</title>
618 <para>&apt-conf;</para>
621 <refsect1><title>Diagnostics</title>
622 <para><command>apt-ftparchive</command> returns zero on normal operation, decimal 100 on error.</para>