antirez [Mon, 21 May 2012 14:50:05 +0000 (16:50 +0200)]
Use comments to split aof.c into sections.
This makes the code more readable, it is still not the case to split the
file itself into three different files, but the logical separation
improves the readability especially since new commits are going to
introduce an additional section.
antirez [Sun, 20 May 2012 21:47:45 +0000 (23:47 +0200)]
TODO file removed.
The list of things to do is since long time in two places:
1) Github issues.
2) I've a private TOOD list of random ideas, what makes sense is later
moved to github issues. So github is anyway the true source of things to
do.
antirez [Wed, 16 May 2012 10:22:29 +0000 (12:22 +0200)]
Deleted jemalloc.orig from /deps.
In the commit upgrading jemalloc to version 3.0.0 I added the old
version of Jemalloc in the 'jemalloc.orig' directory for an error.
This commit removes the not useful version of jemalloc.
antirez [Mon, 14 May 2012 15:35:51 +0000 (17:35 +0200)]
Added time.h include in redis-cli.
redis-cli.c uses the time() function to seed the PRNG, but time.h was
not included. This was not noticed since sys/time.h is included and was
enough in most systems (but not correct). With Ubuntu 12.04 GCC
generates a warning that made us aware of the issue.
antirez [Mon, 14 May 2012 14:04:41 +0000 (16:04 +0200)]
activeExpireCycle(): better precision in max time used.
activeExpireCycle() can consume no more than a few milliseconds per
iteration. This commit improves the precision of the check for the time
elapsed in two ways:
1) We check every 16 iterations instead of the main loop instead of 256.
2) We reset iterations at the start of the function and not every time
we switch to the next database, so the check is correctly performed
every 16 iterations.
A previous commit introduced REDIS_HZ define that changes the frequency
of calls to the serverCron() Redis function. This commit improves
different related things:
1) Software watchdog: now the minimal period can be set according to
REDIS_HZ. The minimal period is two times the timer period, that is:
(1000/REDIS_HZ)*2 milliseconds
2) The incremental rehashing is now performed in the expires dictionary
as well.
3) The activeExpireCycle() function was improved in different ways:
- Now it checks if it already used too much time using microseconds
instead of milliseconds for better precision.
- The time limit is now calculated correctly, in the previous version
the division was performed before of the multiplication resulting in
a timelimit of 0 if HZ was big enough.
- Databases with less than 1% of buckets fill in the hash table are
skipped, because getting random keys is too expensive in this
condition.
4) tryResizeHashTables() is now called at every timer call, we need to
match the number of calls we do to the expired keys colleciton cycle.
antirez [Sun, 13 May 2012 14:40:29 +0000 (16:40 +0200)]
Redis timer interrupt frequency configurable as REDIS_HZ.
Redis uses a function called serverCron() that is very similar to the
timer interrupt of an operating system. This function is used to handle
a number of asynchronous things, like active expired keys collection,
clients timeouts, update of statistics, things related to the cluster
and replication, triggering of BGSAVE and AOF rewrite process, and so
forth.
In the past the timer was called 1 time per second. At some point it was
raised to 10 times per second, but it still was fixed and could not be
changed even at compile time, because different functions called from
serverCron() assumed a given fixed frequency.
This commmit makes the frequency configurable, so that it is simpler to
pick a good tradeoff between overhead of this function (that is usually
very small) and the responsiveness of Redis during a few critical
circumstances where a lot of work is done inside the timer.
An example of such a critical condition is mass-expire of a lot of keys
in the same second. Up to a given percentage of CPU time is used to
perform expired keys collection per expire cylce. Now changing the
REDIS_HZ macro it is possible to do less work but more times per second
in order to block the server for less time.
If this patch will work well in our tests it will enter Redis 2.6-final.
antirez [Fri, 11 May 2012 17:17:31 +0000 (19:17 +0200)]
More incremental active expired keys collection process.
If a large amonut of keys are all expiring about at the same time, the
"active" expired keys collection cycle used to block as far as the
percentage of already expired keys was >= 25% of the total population of
keys with an expire set.
This could block the server even for many seconds in order to reclaim
memory ASAP. The new algorithm uses at max a small amount of
milliseconds per cycle, even if this means reclaiming the memory less
promptly it also means a more responsive server.
Compare integers in ziplist regardless of encoding
Because of the introduction of new integer encoding types for ziplists
in the 2.6 tree, the same integer value may have a different encoding in
different versions of the ziplist implementation. This means that the
encoding can NOT be used as a fast path in comparing integers.
antirez [Wed, 2 May 2012 20:41:50 +0000 (22:41 +0200)]
syncio.c read / write functions reworked for correctness and performance.
The new implementation start reading / writing before blocking with
aeWait(), likely the descriptor can accept writes or has buffered data
inside and we can go faster, otherwise we get an error and wait.
This change has effects on speed but also on correctness: on socket
errors when we perform non blocking connect(2) write is performed ASAP
and the error is returned ASAP before waiting.
So the practical effect is that now a Redis slave is more available if it
can not connect to the master, previously the slave continued to block on
syncWrite() trying to send SYNC, and serving commands very slowly.
antirez [Wed, 2 May 2012 15:14:45 +0000 (17:14 +0200)]
Use specific error if master is down and slave-serve-stale-data is set to no.
We used to reply -ERR ... message ..., now the reply is
instead -MASTERDOWN ... message ... so that it can be distinguished
easily by the other error conditions.
Every matched key in a KEYS call is checked for expiration. When the key
is set to expire, the call to `getExpire` will assert that the key also
exists in the main dictionary. This in turn causes a rehashing step to
be executed. Rehashing a dictionary when there is an iterator active may
result in the iterator emitting duplicate entries, or not emitting some
entries at all. By using a safe iterator, the rehash step is omitted.
Set LUA_MASKCOUNT hook more selectively. Fixes issue #480.
An user reported a crash with Redis scripting (see issue #480 on
github), inspection of the kindly provided strack trace showed that
server.lua_caller was probably set to NULL. The stack trace also slowed
that the call to the hook was originating from a point where we just
used to set/get a few global variables in the Lua state.
What was happening is that we did not set the timeout hook selectively
only when the user script was called. Now we set it more selectively,
specifically only in the context of the lua_pcall() call, and make sure
to remove the hook when the call returns. Otherwise the hook can get
called in random contexts every time we do something with the Lua
state.
Re-introduce -g -rdynamic -ggdb when linking, fixing strack traces.
A previous commit removed -g -rdynamic -ggdb as LDFLAGS, not allowing
Redis to produce a stack trace wth symbol names on crash.
This commit fixes the issue.
The main reason is that otherwise it is completely pointless that we do
a lot of efforts to print the stack trace on crash, and the content of
the stack and registers as well. Using an alternate stack broken this
feature completely.
Redis test: More reliable BRPOPLPUSH replication test.
Now it uses the new wait_for_condition testing primitive.
Also wait_for_condition implementation was fixed in this commit to properly
escape the expr command and its argument.
Redis test: scripting EVALSHA replication test more reliable.
A new primitive wait_for_condition was introduced in the scripting
engine that makes waiting for events simpler, so that it is simpler to
write tests that are more resistant to timing issues.
1) One integer "immediate" encoding that can encode from 0 to 12 in the
encoding byte itself.
2) One 8 bit signed integer encoding that can encode 8 bit signed small
integers in a single byte.
The idea is to exploit all the not used bits we have around in a
backward compatible way.
Add a 24bit integer to ziplists to save one byte for ints that can
fit in 24 bits (thanks to antirez for catching and solving the two's compliment
bug).
Fix and refactoring of code used to get registers on crash.
This fixes compilation on FreeBSD (and possibly other systems) by
not using ucontext_t at all if HAVE_BACKTRACE is not defined.
Also the ifdefs to get the registers are modified to explicitly test for the
operating system in the first level, and the arch in the second level
of nesting.
1) Up to SLOWLOG_ENTRY_MAX_ARGV arguments are logged.
2) Up to SLOWLOG_ENTRY_MAX_STRING bytes per argument are logged.
3) slowlog-max-len is set to 128 by default (was 1024).
The number of remaining arguments / bytes is logged in the entry
so that the user can understand better the nature of the logged command.
just fieldobj itself as sentinel of the fact a field object is used or
not, instead of using the filed length, that may be confusing both for
people and for the compiler emitting a warning.
lookupKeyByPattern() used by SORT GET/BY rewritten. Fixes issue #460.
lookupKeyByPattern() was implemented with a trick to speedup the lookup
process allocating two fake Redis obejcts on the stack. However now that
we propagate expires to the slave as DEL operations the lookup of the
key may result into a call to expireIfNeeded() having the stack
allocated object as argument, that may in turn use it to create the
protocol to send to the slave. But since this fake obejcts are
inherently read-only this is a problem.
As a side effect of this fix there are no longer size limits in the
pattern to be used with GET/BY option of SORT.
See https://github.com/antirez/redis/issues/460 for bug details.
This reflects that REDIS_*FLAGS will only be used for compilation of
Redis and not for its dependencies. Similarly, that FINAL_*FLAGS are
composed of other variables and holds the options that are finally
passed to the compiler and linker.
projects / redis.git / log
