1 <?xml version=
"1.0" encoding=
"utf-8" standalone=
"no"?>
2 <!DOCTYPE refentry PUBLIC
"-//OASIS//DTD DocBook XML V4.2//EN"
3 "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" [
5 <!ENTITY % aptent SYSTEM
"apt.ent">
13 &apt-author.jgunthorpe;
17 <!-- The last update date -->
18 <date>17 August
2009</date>
22 <refentrytitle>apt-ftparchive
</refentrytitle>
23 <manvolnum>1</manvolnum>
24 <refmiscinfo class=
"manual">APT
</refmiscinfo>
27 <!-- Man page title -->
29 <refname>apt-ftparchive
</refname>
30 <refpurpose>Utility to generate index files
</refpurpose>
36 <command>apt-ftparchive
</command>
37 <arg><option>-hvdsq
</option></arg>
38 <arg><option>--md5
</option></arg>
39 <arg><option>--delink
</option></arg>
40 <arg><option>--readonly
</option></arg>
41 <arg><option>--contents
</option></arg>
42 <arg><option>--arch
<replaceable>architecture
</replaceable></option></arg>
43 <arg><option>-o
<replaceable>config
</replaceable>=
<replaceable>string
</replaceable></option></arg>
44 <arg><option>-c=
<replaceable>file
</replaceable></option></arg>
46 <arg>packages
<arg choice=
"plain" rep=
"repeat"><replaceable>path
</replaceable></arg><arg><replaceable>override
</replaceable><arg><replaceable>pathprefix
</replaceable></arg></arg></arg>
47 <arg>sources
<arg choice=
"plain" rep=
"repeat"><replaceable>path
</replaceable></arg><arg><replaceable>override
</replaceable><arg><replaceable>pathprefix
</replaceable></arg></arg></arg>
48 <arg>contents
<arg choice=
"plain"><replaceable>path
</replaceable></arg></arg>
49 <arg>release
<arg choice=
"plain"><replaceable>path
</replaceable></arg></arg>
50 <arg>generate
<arg choice=
"plain"><replaceable>config-file
</replaceable></arg> <arg choice=
"plain" rep=
"repeat"><replaceable>section
</replaceable></arg></arg>
51 <arg>clean
<arg choice=
"plain"><replaceable>config-file
</replaceable></arg></arg>
56 <refsect1><title>Description
</title>
57 <para><command>apt-ftparchive
</command> is the command line tool that generates the index
58 files that APT uses to access a distribution source. The index files should
59 be generated on the origin site based on the content of that site.
</para>
61 <para><command>apt-ftparchive
</command> is a superset of the &dpkg-scanpackages; program,
62 incorporating its entire functionality via the
<literal>packages
</literal> command.
63 It also contains a contents file generator,
<literal>contents
</literal>, and an
64 elaborate means to 'script' the generation process for a complete
67 <para>Internally
<command>apt-ftparchive
</command> can make use of binary databases to
68 cache the contents of a .deb file and it does not rely on any external
69 programs aside from
&gzip;. When doing a full generate it automatically
70 performs file-change checks and builds the desired compressed output files.
</para>
72 <para>Unless the
<option>-h
</option>, or
<option>--help
</option> option is given one of the
73 commands below must be present.
</para>
76 <varlistentry><term>packages
</term>
78 The packages command generates a package file from a directory tree. It
79 takes the given directory and recursively searches it for .deb files,
80 emitting a package record to stdout for each. This command is
81 approximately equivalent to &dpkg-scanpackages;.
</para>
83 <para>The option
<option>--db
</option> can be used to specify a binary caching DB.
</para></listitem>
86 <varlistentry><term>sources
</term>
88 The
<literal>sources
</literal> command generates a source index file from a directory tree.
89 It takes the given directory and recursively searches it for .dsc files,
90 emitting a source record to stdout for each. This command is approximately
91 equivalent to &dpkg-scansources;.
</para>
93 If an override file is specified then a source override file will be
94 looked for with an extension of .src. The --source-override option can be
95 used to change the source override file that will be used.
</para></listitem>
98 <varlistentry><term>contents
</term>
100 The
<literal>contents
</literal> command generates a contents file from a directory tree. It
101 takes the given directory and recursively searches it for .deb files,
102 and reads the file list from each file. It then sorts and writes to stdout
103 the list of files matched to packages. Directories are not written to
104 the output. If multiple packages own the same file then each package is
105 separated by a comma in the output.
</para>
107 The option
<option>--db
</option> can be used to specify a binary caching DB.
</para></listitem>
110 <varlistentry><term>release
</term>
112 The
<literal>release
</literal> command generates a Release file from a
113 directory tree. It recursively searches the given directory for
114 Packages, Packages.gz, Packages.bz2, Sources, Sources.gz,
115 Sources.bz2, Release and md5sum.txt files. It then writes to
116 stdout a Release file containing an MD5 digest and SHA1 digest
117 for each file.
</para>
119 Values for the additional metadata fields in the Release file are
120 taken from the corresponding variables under
121 <literal>APT::FTPArchive::Release
</literal>,
122 e.g.
<literal>APT::FTPArchive::Release::Origin
</literal>. The supported fields
123 are:
<literal>Origin
</literal>,
<literal>Label
</literal>,
<literal>Suite
</literal>,
124 <literal>Version
</literal>,
<literal>Codename
</literal>,
<literal>Date
</literal>,
125 <literal>Valid-Until
</literal>,
<literal>Architectures
</literal>,
126 <literal>Components
</literal>,
<literal>Description
</literal>.
</para></listitem>
130 <varlistentry><term>generate
</term>
132 The
<literal>generate
</literal> command is designed to be runnable from a cron script and
133 builds indexes according to the given config file. The config language
134 provides a flexible means of specifying which index files are built from
135 which directories, as well as providing a simple means of maintaining the
136 required settings.
</para></listitem>
139 <varlistentry><term>clean
</term>
141 The
<literal>clean
</literal> command tidies the databases used by the given
142 configuration file by removing any records that are no longer necessary.
</para></listitem>
147 <refsect1><title>The Generate Configuration
</title>
149 The
<literal>generate
</literal> command uses a configuration file to describe the
150 archives that are going to be generated. It follows the typical ISC
151 configuration format as seen in ISC tools like bind
8 and dhcpd.
152 &apt-conf; contains a description of the syntax. Note that the generate
153 configuration is parsed in sectional manner, but &apt-conf; is parsed in a
154 tree manner. This only effects how the scope tag is handled.
</para>
157 The generate configuration has
4 separate sections, each described below.
</para>
159 <refsect2><title>Dir Section
</title>
161 The
<literal>Dir
</literal> section defines the standard directories needed to
162 locate the files required during the generation process. These
163 directories are prepended certain relative paths defined in later
164 sections to produce a complete an absolute path.
</para>
166 <varlistentry><term>ArchiveDir
</term>
168 Specifies the root of the FTP archive, in a standard
169 Debian configuration this is the directory that contains the
170 <filename>ls-LR
</filename> and dist nodes.
</para></listitem>
173 <varlistentry><term>OverrideDir
</term>
175 Specifies the location of the override files.
</para></listitem>
178 <varlistentry><term>CacheDir
</term>
180 Specifies the location of the cache files
</para></listitem>
183 <varlistentry><term>FileListDir
</term>
185 Specifies the location of the file list files,
186 if the
<literal>FileList
</literal> setting is used below.
</para></listitem>
191 <refsect2><title>Default Section
</title>
193 The
<literal>Default
</literal> section specifies default values, and settings
194 that control the operation of the generator. Other sections may override
195 these defaults with a per-section setting.
</para>
197 <varlistentry><term>Packages::Compress
</term>
199 Sets the default compression schemes to use
200 for the Package index files. It is a string that contains a space
201 separated list of at least one of: '.' (no compression), 'gzip' and
202 'bzip2'. The default for all compression schemes is '. gzip'.
</para></listitem>
205 <varlistentry><term>Packages::Extensions
</term>
207 Sets the default list of file extensions that are package files.
208 This defaults to '.deb'.
</para></listitem>
211 <varlistentry><term>Sources::Compress
</term>
213 This is similar to
<literal>Packages::Compress
</literal>
214 except that it controls the compression for the Sources files.
</para></listitem>
217 <varlistentry><term>Sources::Extensions
</term>
219 Sets the default list of file extensions that are source files.
220 This defaults to '.dsc'.
</para></listitem>
223 <varlistentry><term>Contents::Compress
</term>
225 This is similar to
<literal>Packages::Compress
</literal>
226 except that it controls the compression for the Contents files.
</para></listitem>
229 <varlistentry><term>Translation::Compress
</term>
231 This is similar to
<literal>Packages::Compress
</literal>
232 except that it controls the compression for the Translation-en master file.
</para></listitem>
235 <varlistentry><term>DeLinkLimit
</term>
237 Specifies the number of kilobytes to delink (and
238 replace with hard links) per run. This is used in conjunction with the
239 per-section
<literal>External-Links
</literal> setting.
</para></listitem>
242 <varlistentry><term>FileMode
</term>
244 Specifies the mode of all created index files. It
245 defaults to
0644. All index files are set to this mode with no regard
246 to the umask.
</para></listitem>
249 <varlistentry><term>LongDescription
</term>
251 Sets if long descriptions should be included in the Packages file or split
252 out into a master Translation-en file.
</para></listitem>
257 <refsect2><title>TreeDefault Section
</title>
259 Sets defaults specific to
<literal>Tree
</literal> sections. All of these
260 variables are substitution variables and have the strings $(DIST),
261 $(SECTION) and $(ARCH) replaced with their respective values.
</para>
264 <varlistentry><term>MaxContentsChange
</term>
266 Sets the number of kilobytes of contents
267 files that are generated each day. The contents files are round-robined
268 so that over several days they will all be rebuilt.
</para></listitem>
271 <varlistentry><term>ContentsAge
</term>
273 Controls the number of days a contents file is allowed
274 to be checked without changing. If this limit is passed the mtime of the
275 contents file is updated. This case can occur if the package file is
276 changed in such a way that does not result in a new contents file
277 [override edit for instance]. A hold off is allowed in hopes that new
278 .debs will be installed, requiring a new file anyhow. The default is
10,
279 the units are in days.
</para></listitem>
282 <varlistentry><term>Directory
</term>
284 Sets the top of the .deb directory tree. Defaults to
285 <filename>$(DIST)/$(SECTION)/binary-$(ARCH)/
</filename></para></listitem>
288 <varlistentry><term>SrcDirectory
</term>
290 Sets the top of the source package directory tree. Defaults to
291 <filename>$(DIST)/$(SECTION)/source/
</filename></para></listitem>
294 <varlistentry><term>Packages
</term>
296 Sets the output Packages file. Defaults to
297 <filename>$(DIST)/$(SECTION)/binary-$(ARCH)/Packages
</filename></para></listitem>
300 <varlistentry><term>Sources
</term>
302 Sets the output Sources file. Defaults to
303 <filename>$(DIST)/$(SECTION)/source/Sources
</filename></para></listitem>
306 <varlistentry><term>Translation
</term>
308 Set the output Translation-en master file with the long descriptions if they
309 should be not included in the Packages file. Defaults to
310 <filename>$(DIST)/$(SECTION)/i18n/Translation-en
</filename></para></listitem>
313 <varlistentry><term>InternalPrefix
</term>
315 Sets the path prefix that causes a symlink to be
316 considered an internal link instead of an external link. Defaults to
317 <filename>$(DIST)/$(SECTION)/
</filename></para></listitem>
320 <varlistentry><term>Contents
</term>
322 Sets the output Contents file. Defaults to
323 <filename>$(DIST)/Contents-$(ARCH)
</filename>. If this setting causes multiple
324 Packages files to map onto a single Contents file (such as the default)
325 then
<command>apt-ftparchive
</command> will integrate those package files
326 together automatically.
</para></listitem>
329 <varlistentry><term>Contents::Header
</term>
331 Sets header file to prepend to the contents output.
</para></listitem>
334 <varlistentry><term>BinCacheDB
</term>
336 Sets the binary cache database to use for this
337 section. Multiple sections can share the same database.
</para></listitem>
340 <varlistentry><term>FileList
</term>
342 Specifies that instead of walking the directory tree,
343 <command>apt-ftparchive
</command> should read the list of files from the given
344 file. Relative files names are prefixed with the archive directory.
</para></listitem>
347 <varlistentry><term>SourceFileList
</term>
349 Specifies that instead of walking the directory tree,
350 <command>apt-ftparchive
</command> should read the list of files from the given
351 file. Relative files names are prefixed with the archive directory.
352 This is used when processing source indexes.
</para></listitem>
357 <refsect2><title>Tree Section
</title>
359 The
<literal>Tree
</literal> section defines a standard Debian file tree which
360 consists of a base directory, then multiple sections in that base
361 directory and finally multiple Architectures in each section. The exact
362 pathing used is defined by the
<literal>Directory
</literal> substitution variable.
</para>
364 The
<literal>Tree
</literal> section takes a scope tag which sets the
365 <literal>$(DIST)
</literal> variable and defines the root of the tree
366 (the path is prefixed by
<literal>ArchiveDir
</literal>).
367 Typically this is a setting such as
<filename>dists/&stable-codename;
</filename>.
</para>
369 All of the settings defined in the
<literal>TreeDefault
</literal> section can be
370 use in a
<literal>Tree
</literal> section as well as three new variables.
</para>
372 When processing a
<literal>Tree
</literal> section
<command>apt-ftparchive
</command>
373 performs an operation similar to:
376 for j in Architectures do
377 Generate for DIST=scope SECTION=i ARCH=j
378 </programlisting></para>
381 <varlistentry><term>Sections
</term>
383 This is a space separated list of sections which appear
384 under the distribution, typically this is something like
385 <literal>main contrib non-free
</literal></para></listitem>
388 <varlistentry><term>Architectures
</term>
390 This is a space separated list of all the
391 architectures that appear under search section. The special architecture
392 'source' is used to indicate that this tree has a source archive.
</para></listitem>
395 <varlistentry><term>LongDescription
</term>
397 Sets if long descriptions should be included in the Packages file or split
398 out into a master Translation-en file.
</para></listitem>
401 <varlistentry><term>BinOverride
</term>
403 Sets the binary override file. The override file
404 contains section, priority and maintainer address information.
</para></listitem>
407 <varlistentry><term>SrcOverride
</term>
409 Sets the source override file. The override file
410 contains section information.
</para></listitem>
413 <varlistentry><term>ExtraOverride
</term>
415 Sets the binary extra override file.
</para></listitem>
418 <varlistentry><term>SrcExtraOverride
</term>
420 Sets the source extra override file.
</para></listitem>
425 <refsect2><title>BinDirectory Section
</title>
427 The
<literal>bindirectory
</literal> section defines a binary directory tree
428 with no special structure. The scope tag specifies the location of
429 the binary directory and the settings are similar to the
<literal>Tree
</literal>
430 section with no substitution variables or
431 <literal>Section
</literal><literal>Architecture
</literal> settings.
</para>
433 <varlistentry><term>Packages
</term>
435 Sets the Packages file output.
</para></listitem>
438 <varlistentry><term>Sources
</term>
440 Sets the Sources file output. At least one of
441 <literal>Packages
</literal> or
<literal>Sources
</literal> is required.
</para></listitem>
444 <varlistentry><term>Contents
</term>
446 Sets the Contents file output. (optional)
</para></listitem>
449 <varlistentry><term>BinOverride
</term>
451 Sets the binary override file.
</para></listitem>
454 <varlistentry><term>SrcOverride
</term>
456 Sets the source override file.
</para></listitem>
459 <varlistentry><term>ExtraOverride
</term>
461 Sets the binary extra override file.
</para></listitem>
464 <varlistentry><term>SrcExtraOverride
</term>
466 Sets the source extra override file.
</para></listitem>
469 <varlistentry><term>BinCacheDB
</term>
471 Sets the cache DB.
</para></listitem>
474 <varlistentry><term>PathPrefix
</term>
476 Appends a path to all the output paths.
</para></listitem>
479 <varlistentry><term>FileList, SourceFileList
</term>
481 Specifies the file list file.
</para></listitem>
488 <refsect1><title>The Binary Override File
</title>
489 <para>The binary override file is fully compatible with &dpkg-scanpackages;. It
490 contains
4 fields separated by spaces. The first field is the package name,
491 the second is the priority to force that package to, the third is the
492 the section to force that package to and the final field is the maintainer
493 permutation field.
</para>
494 <para>The general form of the maintainer field is:
495 <literallayout>old [// oldn]* =
> new
</literallayout>
497 <literallayout>new
</literallayout>
498 The first form allows a double-slash separated list of old email addresses
499 to be specified. If any of those are found then new is substituted for the
500 maintainer field. The second form unconditionally substitutes the
501 maintainer field.
</para>
505 <refsect1><title>The Source Override File
</title>
507 The source override file is fully compatible with &dpkg-scansources;. It
508 contains
2 fields separated by spaces. The first fields is the source
509 package name, the second is the section to assign it.
</para>
512 <refsect1><title>The Extra Override File
</title>
514 The extra override file allows any arbitrary tag to be added or replaced
515 in the output. It has
3 columns, the first is the package, the second is
516 the tag and the remainder of the line is the new value.
</para>
519 <refsect1><title>options
</title>
523 <varlistentry><term><option>--md5
</option></term>
525 Generate MD5 sums. This defaults to on, when turned off the generated
526 index files will not have MD5Sum fields where possible.
527 Configuration Item:
<literal>APT::FTPArchive::MD5
</literal></para></listitem>
530 <varlistentry><term><option>-d
</option></term><term><option>--db
</option></term>
532 Use a binary caching DB. This has no effect on the generate command.
533 Configuration Item:
<literal>APT::FTPArchive::DB
</literal>.
</para></listitem>
536 <varlistentry><term><option>-q
</option></term><term><option>--quiet
</option></term>
538 Quiet; produces output suitable for logging, omitting progress indicators.
539 More q's will produce more quiet up to a maximum of
2. You can also use
540 <option>-q=#
</option> to set the quiet level, overriding the configuration file.
541 Configuration Item:
<literal>quiet
</literal>.
</para></listitem>
544 <varlistentry><term><option>--delink
</option></term>
546 Perform Delinking. If the
<literal>External-Links
</literal> setting is used then
547 this option actually enables delinking of the files. It defaults to on and
548 can be turned off with
<option>--no-delink
</option>.
549 Configuration Item:
<literal>APT::FTPArchive::DeLinkAct
</literal>.
</para></listitem>
552 <varlistentry><term><option>--contents
</option></term>
554 Perform contents generation. When this option is set and package indexes
555 are being generated with a cache DB then the file listing will also be
556 extracted and stored in the DB for later use. When using the generate
557 command this option also allows the creation of any Contents files. The
559 Configuration Item:
<literal>APT::FTPArchive::Contents
</literal>.
</para></listitem>
562 <varlistentry><term><option>-s
</option></term><term><option>--source-override
</option></term>
564 Select the source override file to use with the
<literal>sources
</literal> command.
565 Configuration Item:
<literal>APT::FTPArchive::SourceOverride
</literal>.
</para></listitem>
568 <varlistentry><term><option>--readonly
</option></term>
570 Make the caching databases read only.
571 Configuration Item:
<literal>APT::FTPArchive::ReadOnlyDB
</literal>.
</para></listitem>
574 <varlistentry><term><option>-a
</option></term><term><option>--arch
</option></term>
575 <listitem><para>Accept in the
<literal>packages
</literal> and
<literal>contents
</literal>
576 commands only package files matching
<literal>*_arch.deb
</literal> or
577 <literal>*_all.deb
</literal> instead of all package files in the given path.
578 Configuration Item:
<literal>APT::FTPArchive::Architecture
</literal>.
</para></listitem>
581 <varlistentry><term><option>APT::FTPArchive::AlwaysStat
</option></term>
583 &apt-ftparchive; caches as much as possible of metadata in a cachedb. If packages
584 are recompiled and/or republished with the same version again, this will lead to problems
585 as the now outdated cached metadata like size and checksums will be used. With this option
586 enabled this will no longer happen as it will be checked if the file was changed.
587 Note that this option is set to "
<literal>false
</literal>" by default as it is not recommend
588 to upload multiply versions/builds of a package with the same versionnumber, so in theory
589 nobody will have these problems and therefore all these extra checks are useless.
593 <varlistentry><term><option>APT::FTPArchive::LongDescription</option></term>
595 This configuration option defaults to "<literal>true
</literal>" and should only be set to
596 <literal>"false
"</literal> if the Archive generated with &apt-ftparchive; also provides
597 <filename>Translation</filename> files. Note that the <filename>Translation-en</filename>
598 master file can only be created in the generate command.
607 <refsect1><title>Examples</title>
609 <para>To create a compressed Packages file for a directory containing
610 binary packages (.deb):
613 <command>apt-ftparchive</command> packages <replaceable>directory</replaceable> | <command>gzip</command> > <filename>Packages.gz</filename>
614 </programlisting></para>
618 <refsect1><title>See Also</title>
619 <para>&apt-conf;</para>
622 <refsect1><title>Diagnostics</title>
623 <para><command>apt-ftparchive</command> returns zero on normal operation, decimal 100 on error.</para>