Make Packages & Sources generation optional, during Generate call
refactor a bit, extract code out of Generate() into
DoGenerate{PackagesAndSources,Contents}, add new
APT::FTPArchive::ContentsOnly option to allow skipping the generation
of Package/Source files (if they are generated e.g. by some db outside
of apt-ftparchives control)
Michael Vogt [Tue, 2 Sep 2014 15:06:52 +0000 (17:06 +0200)]
Use heap to allocate PatternMatch to avoid potential stack overflow
When apt-cache search with many args (> 130) is given the allocation
of PatternMatch on the stack may fail resulting in a segmentation
fault. By using the heap the max size is much bigger and we also
get a bad_alloc expection instead of a segfault (which we can catch
*if* this ever becomes a pratical problem). No test for the crash
as its not reproducable with the MALLOC_ settings in framework.
Michael Vogt [Tue, 2 Sep 2014 15:24:24 +0000 (17:24 +0200)]
* apt-pkg/deb/dpkgpm.cc:
- update string matching for dpkg I/O errors. (LP: #1363257)
- properly parse the dpkg status line so that package name is properly set
and an apport report is created. Thanks to Anders Kaseorg for the patch.
(LP: #1353171)
Michael Vogt [Tue, 2 Sep 2014 13:50:19 +0000 (15:50 +0200)]
Make Proxy-Auto-Detect check for each host
When doing Acquire::http{,s}::Proxy-Auto-Detect, run the auto-detect
command for each host instead of only once. This should make using
"proxy" from libproxy-tools feasible which can then be used for PAC
style or other proxy configurations.
Warren He [Fri, 29 Aug 2014 09:15:30 +0000 (11:15 +0200)]
initialize iPolicyBrokenCount in DepCache::Update
All other counters are correctly initialized here, expect this one. The
practical effect is low as in apt we usually just do "!= 0" checks, but
only correct counters are good counters.
APT supported versioned provides for a long while in an attempt to get
it working with rpm. While this support is old, we can be relatively
sure that it works as versioned provides are used internally to make
Multi-Arch:foreign work.
Previous versions of apt will print a warning indicating that the
versioned provides is ignored, so that something which "Provides: foo (=
2)" doesn't provide anything.
Note that dpkg does allow only a equals-relation in the provides line
as anything else is deemed too complex. apt doesn't support anything
else either and such a support would require potentially big changes.
add dpkg::source-options for dpkg-source invocation
dpkg-source can be told to enforce signature checks with
--require-valid-signature, but while this isn't feasible as default for
Debian itself at the moment, a local admin should be able to use it.
This commit also fixes the size limit on the construction of the command
being called for dpkg-source and dpkg-buildpackage.
Not all are needed for all files at the moment, but the new docbook
building hadn't available some of the entities it used as the files
weren't correctly copied around in all cases and having the same across
the bord makes working with all of them a little easier.
Our integration tests need some additional dependencies to run and
function correctly, but while multiple places run them, there is no need
to also specify the these dependencies in multiple places.
Michael Vogt [Tue, 29 Jul 2014 13:01:13 +0000 (15:01 +0200)]
Fix SmartConfigure to ignore ordering of packages that are already valid
With the change of SmartConfigure() in git commit 42d51f the ordering
code was trying to re-order dependencies, even when at this point in
time this was not needed. Now it will first check all targets of the
given dependency and only if there is not a good one try to reorder
and unpack/configure as needed.
Michael Vogt [Wed, 16 Jul 2014 11:57:50 +0000 (13:57 +0200)]
StringToBool: only act if the entire string is consumed by strtol()
StringToBool uses strtol() internally to check if the argument is
a number. This function stops when it does not find any more numbers.
So a string like "0ad" (which is a valid packagename) is interpreted
as a "0". The code now checks that the entire string is consumed
not just a part of it. Thanks to Johannes Schauer for raising this
issue.
Michael Vogt [Tue, 8 Jul 2014 13:11:14 +0000 (15:11 +0200)]
Only allow "apt-get build-dep path" when path starts with ./ or /
This avoid the subtle problem that someone might have a directory
with the same package name as the build-depends he/she is trying
to fetch. Also print a note that the specific file/dir is used.
The behaviour of echo "\tA\t" differs between dash/zsh which interprets
the \t as tab and bash which prints it literally. Similar things happen
for other escape sequences – without the -e flag.
Switching to printf makes this more painless^Wportable, so that the
tests are also working correctly with bash as sh.
(commit message by committer, patch otherwise unmodified)
A call to UniqFindTagWrite can trigger the need for a bigger mmap, which
is usually done by moving it, but with this move all pointers into it
become invalid (and have to be remapped). The compiler calculates the
pointer before the execution of the call though, so it tries to store
the returned value at the old location, resulting in a segfault.
We solve this by use of a temprorary variable as we did in the other
instances of this problem before.
It still doesn't reflect the size the cache has on the disk compared to
what is given as total size (90 vs 103 MB), but by counting all structs
in we are at least a bit closer to the reality.
A version belongs to a section and has hence a section member of its
own. A package on the other hand can have multiple versions from
different sections. This was "solved" by using the section which was
parsed first as order of sources.list defines, but that is obviously a
horribly unpredictable thing.
We therefore directly remove this struct member to free some space and
mark the access method as deprecated, which is told to return the
section of the 'newest' known version, which is at least predictable,
but possible not what it returned before – but nobody knows.
Users are way better of with the Section() as returned by the version
they are dealing with. It is likely the same for all versions of a
package, but in the few cases it isn't, it is important (like packages
moving from main/* to contrib/* or into oldlibs …).
We had a wild mixture of (unsigned) int, long and long long here without
much sense, so this commit adds a few typedefs to get some sense in the
typesystem and ensures that a ID isn't sometimes computed as int, stored
as long and compared with a long long… as this could potentially bite us
later on as the size of the archive only increases over time.
increase hashtable size for packages/groups by factor 5
It also makes the size configureable, so it can be adapted in the future
without the need for an abi break - and even by users…
The increase was long overdue as it gives a >10% decrease in runtime of
e.g. 'apt-get check -s'. Some (useless) benchmark with 69933 groups and
187796 packages without a pre-built cache:
time apt-get check -so APT::Cache-HashTableSize=1 → 20m
time apt-get check -so APT::Cache-HashTableSize=1000 → 6,41s
time apt-get check -so APT::Cache-HashTableSize=2000 → 5,64s (old)
time apt-get check -so APT::Cache-HashTableSize=3000 → 5,30s
time apt-get check -so APT::Cache-HashTableSize=5000 → 5,08s
time apt-get check -so APT::Cache-HashTableSize=6000 → 5,05s
time apt-get check -so APT::Cache-HashTableSize=7000 → 5,02s
time apt-get check -so APT::Cache-HashTableSize=8000 → 5,00s
time apt-get check -so APT::Cache-HashTableSize=9000 → 4,98s
time apt-get check -so APT::Cache-HashTableSize=10000 → 4,96s (new)
time apt-get check -so APT::Cache-HashTableSize=15000 → 4,90s
time apt-get check -so APT::Cache-HashTableSize=20000 → 4,86s
time apt-get check -so APT::Cache-HashTableSize=30000 → 4,77s
time apt-get check -so APT::Cache-HashTableSize=40000 → 4,74s
time apt-get check -so APT::Cache-HashTableSize=50000 → 4,73s
time apt-get check -so APT::Cache-HashTableSize=60000 → 4,71s
The gap increases further for operations which have more package
lookups. Factor 5 was chosen as higher values do not provide any
really significant timing advantage anymore compared to the memory
increase in my testing and there is always the possibility to increase
it now if that changes. (also most users will not have 3 releases and
4 architectures in the cache, so theirs will be much smaller and faster).
The name suggests that it is supposed to substitute a variable with a
value, but we tend to use it in a more liberal replace_all() fashion,
but this breaks if either of the parameters is empty or more importantly
if two "variable" occurrences follow each other directly.
don't send pkg from an unknown architecture via EDSP
APT's cache can include packages from architectures dpkg has no
knowledge about and can therefore not be installed for e.g. to allow
easy lookups. There is no point in telling external solvers about them
though and some of them might even be really talkative about ignoring
them if we do.
projects / apt.git / log
