configureable acquire targets to download additional files

[apt.git] / doc / acquire-additional-files.txt

# Acquire additional files in 'update' operations

The download and verification of data from multiple sources in different
compression formats, with partial downloads and patches is an involved
process which is hard to implement correctly and securely.

APT frontends share the code and binaries to make this happen in libapt
with the Acquire system, supported by helpers shipped in the apt package
itself and additional transports in individual packages like
apt-transport-https.

For its own operation libapt needs or can make use of Packages, Sources
and Translation-* files, which it will acquire by default, but
a repository might contain more data files (e.g.  Contents) a frontend
might want to use and would therefore need to be downloaded as well
(e.g. apt-file).

This file describes the configuration scheme such a frontend can use to
instruct the Acquire system to download those additional files.

Note that you can't download files from other sources. It must be files
in the same repository and below the Release file. The Release file must
also contain hashes for the file itself as well as for the compressed
file if wanted, otherwise a download isn't even tried!


# The Configuration Stanza

The Acquire system uses the same configuration settings to implement the
files it downloads by default. These settings are the default, but if
they would be written in a configuration file the configuration
instructing the Acquire system to download the Packages files would look
like this (see also apt.conf(5) manpage for configuration file syntax):

	APT::Acquire::Targets::deb::Packages {
		URI "$(COMPONENT)/binary-$(ARCHITECTURE)/Packages";
		ShortDescription "Packages";
		Description "$(SITE) $(RELEASE)/$(COMPONENT) $(ARCHITECTURE) Packages";

		flatURI "Packages";
		flatDescription "$(SITE) $(RELEASE) Packages";

		Optional "false";
	};

All files which should be downloaded (nicknamed 'Targets') are mentioned
below the APT::Acquire::Targets scope. 'deb' is here the type of the
sources.list entry the file should be acquired for. The only other
supported value is hence 'deb-src'. Beware: You can't specify multiple
types here and you can't download the same URI for multiple types!

After the type you can pick any valid and unique string which preferable
refers to the file it downloads (In the example we picked 'Packages').
This string is never shown or used.

All targets have three main properties you can define:
* URI: The identifier of the file to be downloaded as used in the
  Release file.  It is also the relative location of the file from the
  Release file.  You can neither download from a different server
  entirely (absolute URI) nor should you try to access directories above
  the Release file (e.g. "../../").
* ShortDescription: Very short string intended to be displayed to the
  user e.g.  while reporting progress. apt will e.g. use this string in
  the last line to indicate progress of e.g. the download of a specific
  item.
* Description: A preferable human understandable and readable identifier
  of which file is acquired exactly. Mainly used for progress reporting
  and error messages. apt will e.g. use this string in the Get/Hit/Err
  progress lines.

Additional optional properties:
* flat{URI,Description}: APT supports two types of repositories:
  dists-style repositories which are the default and by far the most
  common which are named after the fact that the files are in an
  elaborated directory structure.  In contrast a flat-style repositories
  lumps all files together in one directory.  Support for these flat
  repositories exists mainly for legacy purposes only.  It is therefore
  recommend to not set these values.
* Optional: The default value is 'true' and should be kept at this
  value.  If enabled the acquire system will skip the download if the
  file isn't mentioned in the Release file. Otherwise this is treated as
  a hard error and the update process fails.


Note that the acquire system will automatically choose to download
a compressed file if it is available and uncompress it for you, just as
it will also use pdiff patching if provided by the repository and
enabled by the user. You only have to ensure that the Release file
contains the information about the compressed files/pdiffs to make this
happen. NO properties have to be set to enable this.

# More examples

The stanzas for Translation-* files as well as for Sources files would
look like this:

APT::Acquire::Targets {
	deb::Translations {
		URI "$(COMPONENT)/i18n/Translation-$(LANGUAGE)";
		ShortDescription "Translation-$(LANGUAGE)";
		Description "$(SITE) $(RELEASE)/$(COMPONENT) Translation-$(LANGUAGE)";

		flatURI "$(LANGUAGE)";
		flatDescription "$(SITE) $(RELEASE) Translation-$(LANGUAGE)";
	};

	deb-src::Sources {
		URI "$(COMPONENT)/source/Sources";
		ShortDescription "Sources";
		Description "$(SITE) $(RELEASE)/$(COMPONENT) Sources";

		flatURI "Sources";
		flatDescription "$(SITE) $(RELEASE) Sources";

		Optional "false";
	};
};

# Substitution variables

As seen in the examples, properties can contain placeholders filled in
by the acquire system. The following variables are known; note that
unknown variables have no default value nor are they touched: They are
printed literally.

* $(SITE): An identifier of the site we access, e.g.
  "http://example.org/".
* $(RELEASE): This is usually an archive- or codename, e.g. "stable" or
  "stretch".  Note that flat-style repositories do not have a archive-
  or codename per-se, so the value might very well be just "/" or so.
* $(COMPONENT): as given in the sources.list, e.g. "main", "non-free" or
  "universe".  Note that flat-style repositories again do not really
  have a meaningful value here.
* $(LANGUAGE): Values are all entries (expect "none") of configuration
  option Acquire::Languages, e.g. "en", "de" or "de_AT".

These values are defined both for 'deb' as well as 'deb-src' targets.
'deb' targets additional have the variable:

* $(ARCHITECTURE): Values are all entries of configuration option
  APT::Architectures (potentially modified by sources.list options),
  e.g. "amd64", "i386" or "armel".