doc/design.sgml

   1 <!doctype debiandoc  PUBLIC  "-//DebianDoc//DTD DebianDoc//EN">
   2 <debiandoc>
   3   <book>
   4     <titlepag>
   5       <title> The APT project design document</title>
   6       <author>
   7         <name>Manoj Srivastava</name>
   8         <email>srivasta@debian.org</email>
   9       </author>
  10       <version>$Id: design.sgml,v 1.4 2003/02/12 15:05:45 doogie Exp $</version>
  11       <abstract>
  12         This document is an overview of the specifications and design
  13         goals of the APT project. It also attempts to give a broad
  14         description of the implementation as well.
  15       </abstract>
  16       <copyright>
  17         <copyrightsummary>Copyright &copy;1997 Manoj Srivastava
  18         </copyrightsummary>
  19         <p>
  20           APT, including this document, is free software; you may
  21           redistribute it and/or modify it under the terms of the GNU
  22           General Public License as published by the Free Software
  23           Foundation; either version 2, or (at your option) any later
  24           version.</p>
  25         <p>
  26           This is distributed in the hope that it will be useful, but
  27           <em>without any warranty</em>; without even the implied
  28           warranty of merchantability or fitness for a particular
  29           purpose.  See the GNU General Public License for more
  30           details.</p>
  31
  32         <p>
  33           You should have received a copy of the GNU General Public
  34           License with your Debian system, in
  35           <tt>/usr/share/common-licenses/GPL</tt>, or with the
  36           <prgn/debiandoc-sgml/ source package as the file
  37           <tt>COPYING</tt>.  If not, write to the Free Software
  38           Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139,
  39           USA.</p>
  40       </copyright>
  41     </titlepag>
  42     <chapt id="introduction">
  43       <heading>Introduction</heading>
  44       <p>APT is supposed to be a replacement for dselect, and not a
  45         replacement for dpkg. However, since addition functionality
  46         has been required for APT, and given the fact that this is
  47         very closely related to dpkg, it is not unreasonable to expect
  48         that additional functionality in the underlying dpkg would
  49         also be requested.</p>
  50
  51       <p> Diety/dselect are the first introduction that people have to
  52         Debian, and unfortunately this first impression contributes
  53         greatly to the public perception of the distribution. It is
  54         imperative that this be a showcase for Debian, rather than
  55         frighten novices away (which has been an accusation often
  56         levelled at the current system)</p>
  57     </chapt>
  58     <chapt>
  59       <heading>Requirements</heading>
  60       <p>
  61         <enumlist compact="compact">
  62           <item>
  63             <p>
  64               APT should be a replacement for dselect. Therefore it
  65               should have all the functionality that dselect has
  66               currently. This is the primary means of interaction
  67               between the user and the package management system, and
  68               it should be able to handle all tasks involved in
  69               installing, upgrading, and routine management without
  70               having the users take recourse to the underlying
  71               management system.</p>
  72           </item>
  73           <item>
  74             <p>
  75               It should be easier to use and less confusing for novice
  76               users. The primary stimulus for the creation of APT
  77               was the perceived intractability, complexity, and
  78               non-intuitive behavior of the existing user interface,
  79               and as such, human factors must be a primary mandate of
  80               APT.</p>
  81           </item>
  82           <item>
  83             <p>
  84               It should be able to group packages more flexibly, and
  85               possibly allow operations based on a group. One should
  86               be able to select, or deselect, a coherent group of
  87               related packages simultaneously, allowing one to add,
  88               remove, or upgrade functionality to a machine as one
  89               step.
  90             </p>
  91           </item>
  92           <item>
  93             <p>
  94               This would allow APT to handle <em>standard
  95                 installations</em>, namely, one could then install a
  96               set of packages to enable a machine to fulfill specific
  97               tasks.  Define a few standard installations, and which
  98               packages are included therein. The packages should be
  99               internally consistent.</p>
 100           </item>
 101           <item>
 102             <p>
 103               Make use of a keywords field in package headers; provide
 104               a standard list of keywords for people to use. This
 105               could be the underpinning to allow the previous two
 106               requirements to work (though the developers are not
 107               constrained to implement the previous requirements using
 108               keywords)
 109             </p>
 110           </item>
 111           <item>
 112             <p>
 113               Use dependencies, conflicts, and reverse dependencies to
 114               properly order packages for installation and
 115               removal. This has been a complaint in the past that the
 116               installation methods do not really understand
 117               dependencies, causing the upgrade process to break, or
 118               allowing the removal of packages that left the system in
 119               an untenable state by breaking the dependencies on
 120               packages that were dependent on the package being
 121               removed. A special emphasis is placed on handling
 122               pre-dependencies correctly; the target of a
 123               predependency has to be fully configured before
 124               attempting to install the pre-dependent package. Also,
 125               <em>configure immediately</em> requests mentioned below
 126               should be handled.</p>
 127           </item>
 128           <item>
 129             <p>
 130               Handle replacement of a package providing a virtual
 131               package with another (for example, it has been very
 132               difficult replacing <prgn>sendmail</prgn> with
 133               <prgn>smail</prgn>, or vice versa), making sure that the
 134               dependencies are still satisfied. </p>
 135           </item>
 136           <item>
 137             <p>
 138               Handle source lists for updates from multiple
 139               sources. APT should also be able to handle diverse
 140               methods of acquiring new packages; local filesystem,
 141               mountable CD-ROM drives, FTP accessible repositories are
 142               some of the methods that come to mind.  Also, the source
 143               lists can be separated into categories, such as main,
 144               contrib, non-us, non-local, non-free, my-very-own,
 145               etc. APT should be set up to retrieve the Packages
 146               files from these multiple source lists, as well as
 147               retrieving the packages themselves. </p>
 148           </item>
 149           <item>
 150             <p>
 151               Handle base of source and acquire all Packages files
 152               underneath.  (possibly select based on architecture),
 153               this should be a simple extension of the previous
 154               requirement.</p>
 155           </item>
 156           <item>
 157             <p>
 158               Handle remote installation (to be implemented maybe in a
 159               future version, it still needs to be designed). This
 160               would ease the burden of maintaining multiple Debian
 161               machines on a site. In the authors opinion this is a
 162               killer difference for the distribution, though it may be
 163               too hard a problem to be implemented with the initial
 164               version of APT. However, some thought must be given to
 165               this to enable APT to retain hooks for future
 166               functionality, or at least to refrain from methods that
 167               may preclude remote activity. It is desirable that
 168               adding remote installation not require a redesign of
 169               APT from the ground up.</p>
 170           </item>
 171           <item>
 172             <p>
 173               Be scalable. Dselect worked a lot better with 400
 174               packages, but at last count the number of packages was
 175               around twelve hundred and climbing. This also requires
 176               APT to pay attention to the needs of small machines
 177               which are low on memory (though this requirement shall
 178               diminish as we move towards bigger machines, it would
 179               still be nice if Debian worked on all old machines where
 180               Linux itself would work).</p>
 181           </item>
 182           <item>
 183             <p>
 184               Handle install immediately requests. Some packages, like
 185               watchdog, are required to be working for the stability
 186               of the machine itself. There are others which may be
 187               required for the correct functioning of a production
 188               machine, or which are mission critical
 189               applications. APT should, in these cases, upgrade the
 190               packages with minimal downtime; allowing these packages
 191               to be one of potentially hundreds of packages being
 192               upgraded concurrently may not satisfy the requirements
 193               of the package or the site. (Watchdog, for example, if
 194               not restarted quickly, may cause the machine to reboot
 195               in the midst of installation, which may cause havoc on
 196               the machine)</p>
 197           </item>
 198         </enumlist>
 199       </p>
 200     </chapt>
 201     <chapt>
 202       <heading>Procedural description</heading>
 203       <p><taglist>
 204           <tag>Set Options</tag>
 205           <item>
 206             <p>
 207               This process handles setting of user or
 208               site options, and configuration of all aspects of
 209               APT. It allows the user to set the location and order
 210               of package sources, allowing them to set up source list
 211               details, like ftp site locations, passwords,
 212               etc. Display options may also be set.</p>
 213           </item>
 214           <tag>Updates</tag>
 215           <item>
 216             <p>
 217               Build a list of available packages, using
 218               source lists or a base location and trawling for
 219               Packages files (needs to be aware of architecture). This
 220               may involve finding and retrieving Packages files,
 221               storing them locally for efficiency, and parsing the
 222               data for later use. This would entail contacting various
 223               underlying access modules (ftp, cdrom mounts, etc) Use a
 224               backing store for speed. This may also require
 225               downloading the actual package files locally for
 226               speed.</p>
 227           </item>
 228           <tag>Local status</tag>
 229           <item>
 230             <p>
 231               Build up a list of packages already
 232               installed. This requires reading and writing the local??
 233               status file. For remote installation, this should
 234               probably use similar mechanisms as the Packages file
 235               retrieval does. Use the backing store for speed. One
 236               should consider multiple backing stores, one for each
 237               machine.
 238             </p>
 239           </item>
 240           <tag>Relationship determination</tag>
 241           <item>
 242             <p>
 243               Determine forward and reverse dependencies. All known
 244               dependency fields should be acted upon, since it is
 245               fairly cheap to do so. Update the backing store with
 246               this information.</p>
 247           </item>
 248           <tag>Selection</tag>
 249           <item>
 250             <p>
 251               Present the data to the user. Look at Behan Webster's
 252               documentation for the user interface procedures. (Note:
 253               In the authors opinion deletions and reverse
 254               dependencies should also be presented to the user, in a
 255               strictly symmetric fashion; this may make it easier to
 256               prevent a package being removed that breaks
 257               dependencies)
 258             </p>
 259           </item>
 260           <tag>Ordering of package installations and configuration </tag>
 261           <item>
 262             <p>
 263               Build a list of events. Simple topological sorting gives
 264               order of packages in dependency order. At certain points
 265               in this ordering, predependencies/immediate configure
 266               directives cause an break in normal ordering. We need to
 267               insert the uninstall/purge directive in the stream
 268               (default: as early as possible).</p>
 269           </item>
 270           <tag>Action</tag>
 271           <item>
 272             <p>
 273               Take the order of installations and removals and build
 274               up a stream of events to send to the packaging system
 275               (dpkg). Execute the list of events if successful. Do not
 276               partially install packages and leave system in broken
 277               state. Go to The Selection step as needed.</p>
 278           </item>
 279
 280         </taglist>
 281       </p>
 282     </chapt>
 283     <chapt>
 284       <heading>Modules and interfaces</heading>
 285       <p><taglist>
 286           <tag>The user interface module</tag>
 287           <item>
 288             <p> Look at Behan Webster's documentation.</p>
 289           </item>
 290           <tag>Widget set</tag>
 291           <item>
 292             <p>
 293               Related closely to above Could some one present design
 294               decisions of the widget set here?</p>
 295           </item>
 296           <tag>pdate Module</tag>
 297           <item>
 298             <p>
 299               Distinct versions of the same package are recorded
 300               separately, but if multiple Packages files contain the
 301               same version of a package, then only the first one is
 302               recorded. For this reason, the least expensive update
 303               source should be listed first (local file system is
 304               better than a remote ftp site)</p>
 305             <p>
 306               This module should interact with the user interface
 307               module to set and change configuration parameters for
 308               the modules listed below. It needs to record that
 309               information in an on disk data file, to be read on
 310               future invocations. </p>
 311             <p><enumlist>
 312                 <item>
 313                   <p>FTP methods</p>
 314                 </item>
 315                 <item>
 316                   <p>mount and file traversal module(s)?</p>
 317                 </item>
 318                 <item>
 319                   <p>Other methods ???</p>
 320                 </item>
 321               </enumlist>
 322             </p>
 323           </item>
 324           <tag>Status file parser/generator</tag>
 325           <item>
 326             <p>
 327               The status file records the current state of the system,
 328               listing the packages installed, etc. The status file is
 329               also one method of communicating with dpkg, since it is
 330               perfectly permissible for the user to use APT to
 331               request packages be updated, put others on hold, mark
 332               other for removal, etc, and then run <tt>dpkg
 333                 -BORGiE</tt> on a file system.</p>
 334           </item>
 335           <tag>Package file parser/generator</tag>
 336           <item>
 337             <p>
 338               Related to above. Handle multiple Packages files, from
 339               different sources. Each package contains a link back to
 340               the packages file structure that contains details about
 341               the origin of the data. </p>
 342           </item>
 343           <tag>Dependency module</tag>
 344           <item>
 345             <p><list>
 346                 <item>
 347                   <p>dependency/conflict determination and linking</p>
 348                 </item>
 349                 <item>
 350                   <p>reverse dependency generator. Maybe merged with above</p>
 351                 </item>
 352               </list>
 353             </p>
 354           </item>
 355           <tag>Package ordering Module</tag>
 356           <item>
 357             <p>Create an ordering of the actions to be taken.</p>
 358           </item>
 359           <tag>Event generator</tag>
 360           <item>
 361             <p>module to interact with dpkg</p>
 362           </item>
 363         </taglist>
 364     </chapt>
 365     <chapt>
 366       <heading>Data flow and conversions analysis.</heading>
 367       <p>
 368         <example>
 369                                                           ____________
 370                                                        __\|ftp modules|
 371                                                       /  /|___________|
 372                                     _ ____________   /     ________________
 373                                     |    update   | /     |mount/local file|
 374         |==========================>|   module    |/_____\|  traversals    |
 375         |                           |_____________|      /|________________|
 376         |                             ^        ^
 377         |                             |        |               ______________
 378   ______|_______    _ _____ ______    |   _____v________      \|            |
 379  |Configuration |   |configuration|   |   |Packages Files|  ===|Status file |
 380  |  module      |<=>|    data     |   |   |______________| /  /|____________|
 381  |______________|   |_____________|   |        ^          /
 382          ^                            |        |         /
 383          |                            | _______v_______|/_
 384          |                            | |              |    ________________
 385          |                            | |              |/_\|   Dependency  |
 386          |                            | |backing store |\ /|     Module    |
 387          |                            | |______________|  _|_______________|
 388          |                             \       ^          /|       ^
 389          |                              \      |         /         |
 390          |                              _\|____v_______|/__    ____v_______
 391          |_____________________________\| User interaction|    |    dpkg   |
 392                                        /|_________________|<==>|  Invoker  |
 393                                                                |___________|
 394
 395         </example>
 396       <p> dpkg also interacts with status and available files.</p>
 397
 398
 399       <p>
 400         The backing store and the associated data structures are the
 401         core of APT. All modules essentially revolve around the
 402         backing store, feeding it data, adding and manipulating links
 403         and relationships between data in the backing store, allowing
 404         the user to interact with and modify the data in the backing
 405         store, and finally writing it out as the status file and
 406         possibly issuing directives to dpkg.</p>
 407
 408       <p>The other focal point for APT is the user interface.</p>
 409     </chapt>
 410   </book>
 411 </debiandoc>