The implementation was needlessly complex and flawed through preventing
positive dependencies from gaining points like they did before these
commits making library transitions harder instead of simpler. It worked
out anyhow most of the time out of pure 'luck' (and other ways of
gaining points) or got miss attributed to being a temporary hick-up.
Your mileage may vary, but don't worry: There is more than one way to
do it, but our one size fits all is not a bigger hammer, but an entire
roundhouse kick! So brace yourself for the tl;dr: The limit is gone.*
Beware: This fixes also the problem that a double newline is
unconditionally added 'later' which is an overcommitment in case
the dsc filesize is limit-2 <= x <= limit.
* limited to numbers fitting into an unsigned long long.
Michael Vogt [Tue, 6 Jan 2015 09:54:24 +0000 (10:54 +0100)]
Add regression test for the previous commit
The issue was that https.cc never called URIStart(), one way to
detect this is that no download progress is generated without
this call. The test now checks for this and as a side-effect will
also ensure that we do not break download progress reporting and
Acquire::{http,https}::Dl-Limit accidently.
Michael Vogt [Mon, 5 Jan 2015 09:27:53 +0000 (10:27 +0100)]
Fix missing URIStart() for https downloads
Add a explicit ReceivedData to HttpsMethod that indicates when
we got data from the connection so that we can send URISTart()
to the parent.
This is needed because URIStart got moved in f9b4f12d from
the progress_callback to write_data() and it only checks for
Res.Size. In the old code if progress_callback is called by
libcurl (and sets Res.Size) before write_data is called then
URIStart() is never send. Making this a explicit ReceivedData
variable fixes this issue.
This results in an extra image package being removed from
APT::NeverAutoRemove, losing the intended effect of keeping the {current,
previous, latest} set of images installed.
Requiring a “.” in the package name tightens the matched package names
to those that are installing a specific version of the image, thus
eliding the meta-packages.
pass-through stdin fd instead of content if not a terminal
Commit 299aea924ccef428219ed6f1a026c122678429e6 fixes the problem of
not logging terminal in case stdin & stdout are not a terminal. The
problem is that we are then trying to pass-through stdin content by
reading from the apt-process stdin and writing it to the stdin of the
child (dpkg), which works great for users who can control themselves,
but pipes and co are a bit less forgiving causing us to pass everything
to the first child process, which if the sending part of the pipe is
e.g. 'yes' we will never see the end of it (as the pipe is full at some
point and further writing blocks).
There is a simple solution for that of course: If stdin isn't a terminal,
we us the apt-process stdin as stdin for the child directly (We don't do
this if it is a terminal to be able to save the typed input in the log).
always run 'dpkg --configure -a' at the end of our dpkg callings
dpkg checks now for dependencies before running triggers, so that
packages can now end up in trigger states (especially those we are not
touching at all with our calls) after apt is done running.
The solution to this is trivial: Just tell dpkg to configure everything
after we have (supposely) configured everything already. In the worst
case this means dpkg will have to run a bunch of triggers, usually it
will just do nothing though.
The code to make this happen was already available, so we just flip a
config option here to cause it to be run. This way we can keep
pretending that triggers are an implementation detail of dpkg.
--triggers-only would supposely work as well, but --configure is more
robust in regards to future changes to dpkg and something we will
hopefully make use of in future versions anyway (as it was planed at the
time this and related options were implemented).
Note that dpkg currently has a workaround implemented to allow upgrades
to jessie to be clean, so that the test works before and after. Also
note that test (compared to the one in the bug) drops the await test as
its is considered a loop by dpkg now.
If we have no controlling terminal opening a terminal will make this
terminal our controller, which is a serious problem if this happens to
be the pseudo terminal we created to run dpkg in as we will close this
terminal at the end hanging ourself up in the process…
The offending open is the one we do to have at least one slave fd open
all the time, but for good measure, we apply the flag also to the slave
fd opening in the child process as we set the controlling terminal
explicitely here.
This is a regression from 150bdc9ca5d656f9fba94d37c5f4f183b02bd746 with
the slight twist that this usecase was silently broken before in that it
wasn't logging the output in term.log (as a pseudo terminal wasn't
created).
Real webservers (like apache) actually send an error page with a 416
response, but our client didn't expect it leaving the page on the socket
to be parsed as response for the next request (http) or as file content
(https), which isn't what we want at all… Symptom is a "Bad header line"
as html usually doesn't parse that well to an http-header.
This manifests itself e.g. if we have a complete file (or larger) in
partial/ which isn't discarded by If-Range as the server doesn't support
it (or it is just newer, think: mirror rotation).
It is a sort-of regression of 78c72d0ce22e00b194251445aae306df357d5c1a,
which removed the filesize - 1 trick, but this had its own problems…
To properly test this our webserver gains the ability to reply with
transfer-encoding: chunked as most real webservers will use it to send
the dynamically generated error pages.
(The tests and their binary helpers had to be slightly modified to
apply, but the patch to fix the issue itself is unchanged.)
James McCoy [Fri, 28 Nov 2014 13:21:06 +0000 (14:21 +0100)]
support long keyids in "apt-key del" instead of ignoring them
apt-key given a long keyid reports just "OK" all the time, but doesn't
delete the mentioned key as it doesn't find the key.
Note: In debian/experimental this was closed with 29f1b977100aeb6d6ebd38923eeb7a623e264ffe which just added the testcase
as the rewrite of apt-key had fixed this as well.
We run dpkg on its own pty, so we can log its output and have our own
output around it (like the progress bar), while also allowing debconf
and configfile prompts to happen.
In commit 223ae57d468fdcac451209a095047a07a5698212 we changed to
constantly reopening the slave for kfreebsd. This has the sideeffect
though that in some cases slave and master will lose their connection on
linux, so that no output is passed along anymore. We fix this by having
always an fd referencing the slave open (linux), but we don't use it
(kfreebsd).
Failing to get our PTY up and running has many (bad) consequences
including (not limited to, nor all at ones or in any case) garbled ouput,
no output, no logging, a (partial) mixture of the previous items, …
This commit is therefore also reshuffling quiet a bit of the creation
code to get especially the output part up and running on linux and the
logging for kfreebsd.
Note that the testcase tries to cover some cases, but this is an
interactivity issue so only interactive usage can really be a good test.
reenable support for -s (and co) in apt-get source
The conversion to accept only relevant options for commands has
forgotten another one, so adding it again even through the usecase might
very well be equally good served by --print-uris.
A version belongs to a section and has hence a section member of its
own. A package on the other hand can have multiple versions from
different sections. This was "solved" by using the section which was
parsed first as order of sources.list defines, but that is obviously a
horribly unpredictable thing.
Users are way better of with the Section() as returned by the version
they are dealing with. It is likely the same for all versions of a
package, but in the few cases it isn't, it is important (like packages
moving from main/* to contrib/* or into oldlibs …).
Backport of 7a66977 which actually instantly removes the member.
Collect all hashes we can get from the source record and put them into a
HashStringList so that 'apt-get source' can use it instead of using
always the MD5sum.
We therefore also deprecate the MD5 struct member in favor of the list.
While at it, the parsing of the Files is enhanced so that records which
miss "Files" (aka MD5 checksums) are still searched for other checksums
as they include just as much data, just not with a nice and catchy name.
This is a cherry-pick of 1262d35 with some dirty tricks to preserve ABI.
APT supports more than just one HashString and even allows to enforce
the usage of a specific hash. This class is intended to help with
storage and passing around of the HashStrings.
The cherry-pick here the un-const-ification of HashType() compared to f4c3850ea335545e297504941dc8c7a8f1c83358. The point of this commit is
adding infrastructure for the next one. All by itself, it just adds new
symbols.
Remove long obsolete (hold, hold-reinstreq, removal-failed) or just
wrong (post-inst-failed vs postinst-failed) values, that have been
autoconverted by dpkg at run-time to their new equivalents, so there
should not be any such instance in any recent system (removal-failed
since dpkg 1.1.4 in Apr 1996, hold and hold-reinstreq since dpkg
1.2.0 in May 1996). dpkg even stopped doing the mapping in 1.15.4
and 1.15.8 respectively.
At the same time sort the list in the same order as they appear in
the dpkg code.
Guillem Jover [Thu, 2 Oct 2014 15:48:13 +0000 (17:48 +0200)]
apt-get: Create the temporary downloaded changelog inside tmpdir
The code is creating a secure temporary directory, but then creates
the changelog alongside the tmpdir in the same base directory. This
defeats the secure tmpdir creation, making the filename predictable.
Inject a '/' between the tmpdir and the changelog filename.
Michael Vogt [Sun, 21 Sep 2014 19:40:10 +0000 (21:40 +0200)]
Ensure that iTFRewritePackageOrder is "MD5sum" to match apt-ftparchive
The iTFRewritePackageOrder is used in indexcopy to copy and normalize
cdrom Packages files. This change will ensure that there is no
"normalization" that changes MD5sum -> MD5Sum which alters the hash
of the Packages file on disk (oh the irony).
Michael Vogt [Sun, 21 Sep 2014 19:23:04 +0000 (21:23 +0200)]
Fix regression for cdrom: sources from latest security update
Skip a reverify for cdrom: sources. The reverify step is actually
harmful here because the apt-cdrom add code uses the indexcopy.cc
which will "normalize" the Packages file from the cdrom when it
writes it to the local disk. This leads to changing the "MD5sum"
field (notice the lower case "s") on the cdrom Packages file to
a "MD5Sum" field on the local file in /var/lib/apt/lists. Which
of course alters the hash and makes apt fail to reverify the file.
Michael Vogt [Fri, 19 Sep 2014 14:41:55 +0000 (16:41 +0200)]
Fix regression when copy: is used for a relative path
When we do a ReverifyAfterIMS() we use the copy: method to
verify the hashes again. If the user uses -o Dir=./something/relative
this fails because we use the URI class in copy.cc that strips
away the leading relative part. By not using URI this is fixed.
Michael Vogt [Wed, 17 Sep 2014 12:57:05 +0000 (14:57 +0200)]
Fix regression for file:/// uris from CVE-2014-0487
Do not run ReverifyAfterIMS() for local file URIs as this will
causes apt to mess around in the file:/// uri space. This is
wrong in itself, but it will also cause a incorrect verification
failure when the archive and the lists directory are on different
partitions as rename().
Michael Vogt [Tue, 16 Sep 2014 18:23:43 +0000 (20:23 +0200)]
SECURITY UPDATE for CVE-2014-{0488,0487,0489}
incorrect invalidating of unauthenticated data (CVE-2014-0488)
incorect verification of 304 reply (CVE-2014-0487)
incorrect verification of Acquire::Gzip indexes (CVE-2014-0489)
Builds, runs and generates everything needed to have a coverage report
at the end for apt. The report isn't perfect as most childs apt forks do
not have a regular exit and so data is never written for them, which
results in e.g. most methods to have zero coverage reported.
Most pagers are nice and default to running non-interactively if they
aren't connected to a terminal and we relied on that. On ci.debian.net
the configured pager is printing a header out of nowhere though, so if
we are printing to a non-terminal we call "cat" instead.
In the rework we also "remove" the dependency on sensible-utils in sofar
as we call some alternatives if calling the utils fail.
This seems to be the last problem preventing a "PASS" status on
ci.debian.net, so we close the associated bugreport.
rework PTY magic to fix stair-stepping on kfreebsd
A pty slave we have got from openpty can only be used for one dpkg
child, if we give it to a second child on kfreebsd setting TIOCSCTTY
fails causing the output to be stair-stepped from now on.
By switching the code to creating a master and opening a new slave in
the child for each child we can fix this glitch, so that at least the
master remains stable.
APT treats upgrades like installs and dpkg is very similar in this, but
prints still a slightly different processing message indicating that it
is really an upgrade which we hadn't parsed so far, but this wasn't
really visible as we quickly moved on to a 'known' state.
More problematic was the reinstall case as apt hadn't recognized this
for the package name detection, so that reinstalls had no progress since
we introduced MultiArch.
Commit cbcdd3ee9d86379d1b3a44e41ae8b17dc23111d0 removes the space at the
end of the debfile name dpkg send to us and we previously had included
in the pmerror message we printed on the statusfd.
Instead of trying to inspect /proc and the fds inside we use "test -t 1"
instead as this is available and working on kfreebsd as well – not that
something breaks if we wouldn't, but we like color.
Using 'kfreebsd' here makes the test fail on a kfreebsd system
(obviously), so we just use something totally madeup in the hope that
this is less like to conflict in the future.
No reason in and of by itself at the moment, but prepares for the goal
of having 'apt search' and 'apt-cache search' using the same code now
that they at least support the same stuff. The 'apt' code is just a
multitude slower at the moment…
The method already deals with a format string, but had an else path
doing a hardcoded format as well. This is changed now to use the same
code for both - the format in the second case is still fixed though.
Michael Vogt [Fri, 5 Sep 2014 10:50:15 +0000 (12:50 +0200)]
Ensure we have a Policy in CacheFile.BuildDepCache()
This partly reverts d059cc2 and fixes bug #753297 in a more
general way by ensuring that CacheFile.BuildDepCache() builds
a pkgPolicy if there isn't one already.
Michael Vogt [Fri, 5 Sep 2014 10:03:28 +0000 (12:03 +0200)]
Fix incorrect upgradable listing in "apt list" (thanks to Michael Musenbrock)
The "apt list" command was using only the pkgDepCache but not the
pkgPolicy to figure out if a package is upgradable. This lead to
incorrect display of upgradable package when the user used the
policy to pin-down packages. Thanks to Michael Musenbrock for the
initial patch.
Make Packages & Sources generation optional, during Generate call
refactor a bit, extract code out of Generate() into
DoGenerate{PackagesAndSources,Contents}, add new
APT::FTPArchive::ContentsOnly option to allow skipping the generation
of Package/Source files (if they are generated e.g. by some db outside
of apt-ftparchives control)
Michael Vogt [Tue, 2 Sep 2014 15:06:52 +0000 (17:06 +0200)]
Use heap to allocate PatternMatch to avoid potential stack overflow
When apt-cache search with many args (> 130) is given the allocation
of PatternMatch on the stack may fail resulting in a segmentation
fault. By using the heap the max size is much bigger and we also
get a bad_alloc expection instead of a segfault (which we can catch
*if* this ever becomes a pratical problem). No test for the crash
as its not reproducable with the MALLOC_ settings in framework.
Michael Vogt [Tue, 2 Sep 2014 15:24:24 +0000 (17:24 +0200)]
* apt-pkg/deb/dpkgpm.cc:
- update string matching for dpkg I/O errors. (LP: #1363257)
- properly parse the dpkg status line so that package name is properly set
and an apport report is created. Thanks to Anders Kaseorg for the patch.
(LP: #1353171)
Michael Vogt [Tue, 2 Sep 2014 13:50:19 +0000 (15:50 +0200)]
Make Proxy-Auto-Detect check for each host
When doing Acquire::http{,s}::Proxy-Auto-Detect, run the auto-detect
command for each host instead of only once. This should make using
"proxy" from libproxy-tools feasible which can then be used for PAC
style or other proxy configurations.
projects / apt.git / log
