[redis.git] / doc / ExpireCommand.html


<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
    <head>
        <link type="text/css" rel="stylesheet" href="style.css" />
    </head>
    <body>
        <div id="page">
        
            <div id='header'>
            <a href="index.html">
            <img style="border:none" alt="Redis Documentation" src="redis.png">
            </a>
            </div>
        
            <div id="pagecontent">
                <div class="index">
<!-- This is a (PRE) block.  Make sure it's left aligned or your toc title will be off. -->
<b>ExpireCommand: Contents</b><br>&nbsp;&nbsp;<a href="#EXPIRE _key_ _seconds_">EXPIRE _key_ _seconds_</a><br>&nbsp;&nbsp;<a href="#EXPIREAT _key_ _unixtime_ (Redis &gt;">EXPIREAT _key_ _unixtime_ (Redis &gt;</a><br>&nbsp;&nbsp;<a href="#PERSIST _key_ (Redis &gt;">PERSIST _key_ (Redis &gt;</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#How the expire is removed from a key">How the expire is removed from a key</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Restrictions with write operations against volatile keys">Restrictions with write operations against volatile keys</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Restrictions for write operations with volatile keys as sources">Restrictions for write operations with volatile keys as sources</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Setting the timeout again on already volatile keys">Setting the timeout again on already volatile keys</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Enhanced Lazy Expiration algorithm">Enhanced Lazy Expiration algorithm</a><br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Version 1.0">Version 1.0</a><br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Version 1.1">Version 1.1</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Return value">Return value</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#FAQ: Can you explain better why Redis &lt; 2.1.3 deletes keys with an EXPIRE on write operations?">FAQ: Can you explain better why Redis &lt; 2.1.3 deletes keys with an EXPIRE on write operations?</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#FAQ: How this limitations were solved in Redis versions &gt; 2.1.3?">FAQ: How this limitations were solved in Redis versions &gt; 2.1.3?</a>
                </div>
                
                <h1 class="wikiname">ExpireCommand</h1>

                <div class="summary">
                    
                </div>

                <div class="narrow">
                    &iuml;&raquo;&iquest;#sidebar <a href="GenericCommandsSidebar.html">GenericCommandsSidebar</a><h1><a name="EXPIRE _key_ _seconds_">EXPIRE _key_ _seconds_</a></h1>
<h1><a name="EXPIREAT _key_ _unixtime_ (Redis &gt;">EXPIREAT _key_ _unixtime_ (Redis &gt;</a></h1> 1.1)=
<h1><a name="PERSIST _key_ (Redis &gt;">PERSIST _key_ (Redis &gt;</a></h1> 2.1.3) =
<i>Time complexity: O(1)</i><blockquote>Set a timeout on the specified key. After the timeout the key will beautomatically deleted by the server. A key with an associated timeout issaid to be <i>volatile</i> in Redis terminology.</blockquote>
<blockquote>Voltile keys are stored on disk like the other keys, the timeout is persistenttoo like all the other aspects of the dataset. Saving a dataset containingexpires and stopping the server does not stop the flow of time as Redisstores on disk the time when the key will no longer be available as Unixtime, and not the remaining seconds.</blockquote>
<blockquote>EXPIREAT works exctly like EXPIRE but instead to get the number of secondsrepresenting the Time To Live of the key as a second argument (that is arelative way of specifing the TTL), it takes an absolute one in the form ofa UNIX timestamp (Number of seconds elapsed since 1 Gen 1970).</blockquote>
<blockquote>EXPIREAT was introduced in order to implement <a href="AppendOnlyFileHowto.html">the Append Only File persistence mode</a>so that EXPIRE commands are automatically translated into EXPIREAT commands for the append only file. Of course EXPIREAT can alsoused by programmers that need a way to simply specify that a given key should expire at a given time in the future.</blockquote>
<blockquote>Since Redis 2.1.3 you can update the value of the timeout of a key alreadyhaving an expire set. It is also possible to undo the expire at allturning the key into a normal key using the PERSIST command.</blockquote>
<h2><a name="How the expire is removed from a key">How the expire is removed from a key</a></h2><blockquote>When the key is set to a new value using the SET command, or when a keyis destroied via DEL, the timeout is removed from the key.</blockquote>
<h2><a name="Restrictions with write operations against volatile keys">Restrictions with write operations against volatile keys</a></h2><blockquote>IMPORTANT: Since Redis 2.1.3 or greater, there are no restrictions aboutthe operations you can perform against volatile keys, however older versionsof Redis, including the current stable version 2.0.0, has the followinglimitations:</blockquote>
<blockquote>Write operations like LPUSH, LSET and every other command that has theeffect of modifying the value stored at a volatile key have a special semantic:basically a volatile key is destroyed when it is target of a write operation.See for example the following usage pattern:</blockquote>
<pre class="codeblock python" name="code">
% ./redis-cli lpush mylist foobar /Users/antirez/hack/redis
OK
% ./redis-cli lpush mylist hello  /Users/antirez/hack/redis
OK
% ./redis-cli expire mylist 10000 /Users/antirez/hack/redis
1
% ./redis-cli lpush mylist newelement
OK
% ./redis-cli lrange mylist 0 -1  /Users/antirez/hack/redis
1. newelement
</pre><blockquote>What happened here is that LPUSH against the key with a timeout set deletedthe key before to perform the operation. There is so a simple rule, writeoperations against volatile keys will destroy the key before to perform theoperation. Why Redis uses this behavior? In order to retain an importantproperty: a server that receives a given number of commands in the samesequence will end with the same dataset in memory. Without the delete-on-writesemantic what happens is that the state of the server depends on the timethe commands were issued. This is not a desirable property in a distributed databasethat supports replication.</blockquote>
<h2><a name="Restrictions for write operations with volatile keys as sources">Restrictions for write operations with volatile keys as sources</a></h2>Even when the volatile key is not modified as part of a write operation, if it is 
read in a composite write operation (such as SINTERSTORE) it will be cleared at the 
start of the operation.  This is done to avoid concurrency issues in replication.  
Imagine a key that is about to expire and the composite operation is run against it. 
On a slave node, this key might already be expired, which leaves you with a 
desync in your dataset.<h2><a name="Setting the timeout again on already volatile keys">Setting the timeout again on already volatile keys</a></h2><blockquote>Trying to call EXPIRE against a key that already has an associated timeoutwill not change the timeout of the key, but will just return 0. If insteadthe key does not have a timeout associated the timeout will be set and EXPIREwill return 1.</blockquote>
<h2><a name="Enhanced Lazy Expiration algorithm">Enhanced Lazy Expiration algorithm</a></h2><blockquote>Redis does not constantly monitor keys that are going to be expired.Keys are expired simply when some client tries to access a key, andthe key is found to be timed out.</blockquote>
<blockquote>Of course this is not enough as there are expired keys that will neverbe accessed again. This keys should be expired anyway, so once everysecond Redis test a few keys at random among keys with an  expire set.All the keys that are already expired are deleted from the keyspace. </blockquote>
<h3><a name="Version 1.0">Version 1.0</a></h3><blockquote>Each time a fixed number of keys where tested (100 by default). So ifyou had a client setting keys with a very short expire faster than 100for second the memory continued to grow. When you stopped to insertnew keys the memory started to be freed, 100 keys every second in thebest conditions. Under a peak Redis continues to use more and more RAMeven if most keys are expired in each sweep.</blockquote>
<h3><a name="Version 1.1">Version 1.1</a></h3><blockquote>Each time Redis:</blockquote>
<ol><li> Tests 100 random keys from expired keys set.</li><li> Deletes all the keys found expired.</li><li> If more than 25 keys were expired, it start again from 1.</li></ol>
<blockquote>This is a trivial probabilistic algorithm, basically the assumption isthat our sample is representative of the whole key space,and we continue to expire until the percentage of keys that are likelyto be expired is under 25%</blockquote>
<blockquote>This means that at any given moment the maximum amount of keys alreadyexpired that are using memory is at max equal to max setting operations per second divided by 4.</blockquote>
<h2><a name="Return value">Return value</a></h2><a href="ReplyTypes.html">Integer reply</a>, specifically:<br/><br/><pre class="codeblock python python" name="code">
1: the timeout was set.
0: the timeout was not set since the key already has an associated timeout
   (this may happen only in Redis versions &lt; 2.1.3, Redis &gt;= 2.1.3 will
   happily update the timeout), or the key does not exist.
</pre><h2><a name="FAQ: Can you explain better why Redis &lt; 2.1.3 deletes keys with an EXPIRE on write operations?">FAQ: Can you explain better why Redis &lt; 2.1.3 deletes keys with an EXPIRE on write operations?</a></h2>
Ok let's start with the problem:
<pre class="codeblock python python python" name="code">
redis&gt; set a 100
OK
redis&gt; expire a 360
(integer) 1
redis&gt; incr a
(integer) 1
</pre>
I set a key to the value of 100, then set an expire of 360 seconds, and then incremented the key (before the 360 timeout expired of course). The obvious result would be: 101, instead the key is set to the value of 1. Why?
There is a very important reason involving the Append Only File and Replication. Let's rework a bit our example adding the notion of time to the mix:
<pre class="codeblock python python python python" name="code">
SET a 100
EXPIRE a 5
... wait 10 seconds ...
INCR a
</pre>
Imagine a Redis version that does not implement the &quot;Delete keys with an expire set on write operation&quot; semantic.
Running the above example with the 10 seconds pause will lead to 'a' being set to the value of 1, as it no longer exists when INCR is called 10 seconds later.<br/><br/>Instead if we drop the 10 seconds pause, the result is that 'a' is set to 101.<br/><br/>And in the practice timing changes! For instance the client may wait 10 seconds before INCR, but the sequence written in the Append Only File (and later replayed-back as fast as possible when Redis is restarted) will not have the pause. Even if we add a timestamp in the AOF, when the time difference is smaller than our timer resolution, we have a race condition.<br/><br/>The same happens with master-slave replication. Again, consider the example above: the client will use the same sequence of commands without the 10 seconds pause, but the replication link will slow down for a few seconds due to a network problem. Result? The master will contain 'a' set to 101, the slave 'a' set to 1.<br/><br/>The only way to avoid this but at the same time have reliable non time dependent timeouts on keys is to destroy volatile keys when a write operation is attempted against it.<br/><br/>After all Redis is one of the rare fully persistent databases that will give you EXPIRE. This comes to a cost :)<h2><a name="FAQ: How this limitations were solved in Redis versions &gt; 2.1.3?">FAQ: How this limitations were solved in Redis versions &gt; 2.1.3?</a></h2>Since Redis 2.1.3 there are no longer restrictions in the use you can do of write commands against volatile keys, still the replication and AOF file are guaranteed to be fully consistent.<br/><br/>In order to obtain a correct behavior without sacrificing consistency now when a key expires, a DEL operation is synthesized in both the AOF file and against all the attached slaves. This way the expiration process is centralized in the master instance, and there is no longer a chance of consistency errors.<br/><br/>However while the slaves while connected to a master will not expire keys independently, they'll still take the full state of the expires existing in the dataset, so when a slave is elected to a master it will be able to expire the keys independently, fully acting as a master.
                </div>
        
            </div>
        </div>
    </body>
</html>
Commit	Line	Data
8fb13ce8	1
	2	<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
	3	<html>
	4	<head>
	5	<link type="text/css" rel="stylesheet" href="style.css" />
	6	</head>
	7	<body>
	8	<div id="page">
	9
	10	<div id='header'>
	11	<a href="index.html">
	12	<img style="border:none" alt="Redis Documentation" src="redis.png">
	13	</a>
	14	</div>
	15
	16	<div id="pagecontent">
	17	<div class="index">
	18	<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
b9f7e9e6	19	<b>ExpireCommand: Contents</b><br>  <a href="#EXPIRE _key_ _seconds_">EXPIRE _key_ _seconds_</a><br>  <a href="#EXPIREAT _key_ _unixtime_ (Redis >">EXPIREAT _key_ _unixtime_ (Redis ></a><br>  <a href="#PERSIST _key_ (Redis >">PERSIST _key_ (Redis ></a><br>    <a href="#How the expire is removed from a key">How the expire is removed from a key</a><br>    <a href="#Restrictions with write operations against volatile keys">Restrictions with write operations against volatile keys</a><br>    <a href="#Restrictions for write operations with volatile keys as sources">Restrictions for write operations with volatile keys as sources</a><br>    <a href="#Setting the timeout again on already volatile keys">Setting the timeout again on already volatile keys</a><br>    <a href="#Enhanced Lazy Expiration algorithm">Enhanced Lazy Expiration algorithm</a><br>      <a href="#Version 1.0">Version 1.0</a><br>      <a href="#Version 1.1">Version 1.1</a><br>    <a href="#Return value">Return value</a><br>    <a href="#FAQ: Can you explain better why Redis < 2.1.3 deletes keys with an EXPIRE on write operations?">FAQ: Can you explain better why Redis < 2.1.3 deletes keys with an EXPIRE on write operations?</a><br>    <a href="#FAQ: How this limitations were solved in Redis versions > 2.1.3?">FAQ: How this limitations were solved in Redis versions > 2.1.3?</a>
8fb13ce8	20	</div>
	21
	22	<h1 class="wikiname">ExpireCommand</h1>
	23
	24	<div class="summary">
	25
	26	</div>
	27
	28	<div class="narrow">
	29	ï»¿#sidebar <a href="GenericCommandsSidebar.html">GenericCommandsSidebar</a><h1><a name="EXPIRE _key_ _seconds_">EXPIRE _key_ _seconds_</a></h1>
	30	<h1><a name="EXPIREAT _key_ _unixtime_ (Redis >">EXPIREAT _key_ _unixtime_ (Redis ></a></h1> 1.1)=
b9f7e9e6	31	<h1><a name="PERSIST _key_ (Redis >">PERSIST _key_ (Redis ></a></h1> 2.1.3) =
b9f7e9e6	32	<i>Time complexity: O(1)</i><blockquote>Set a timeout on the specified key. After the timeout the key will beautomatically deleted by the server. A key with an associated timeout issaid to be <i>volatile</i> in Redis terminology.</blockquote>
abe18d0e	33	<blockquote>Voltile keys are stored on disk like the other keys, the timeout is persistenttoo like all the other aspects of the dataset. Saving a dataset containingexpires and stopping the server does not stop the flow of time as Redisstores on disk the time when the key will no longer be available as Unixtime, and not the remaining seconds.</blockquote>
8fb13ce8	34	<blockquote>EXPIREAT works exctly like EXPIRE but instead to get the number of secondsrepresenting the Time To Live of the key as a second argument (that is arelative way of specifing the TTL), it takes an absolute one in the form ofa UNIX timestamp (Number of seconds elapsed since 1 Gen 1970).</blockquote>
abe18d0e	35	<blockquote>EXPIREAT was introduced in order to implement <a href="AppendOnlyFileHowto.html">the Append Only File persistence mode</a>so that EXPIRE commands are automatically translated into EXPIREAT commands for the append only file. Of course EXPIREAT can alsoused by programmers that need a way to simply specify that a given key should expire at a given time in the future.</blockquote>
	36	<blockquote>Since Redis 2.1.3 you can update the value of the timeout of a key alreadyhaving an expire set. It is also possible to undo the expire at allturning the key into a normal key using the PERSIST command.</blockquote>
	37	<h2><a name="How the expire is removed from a key">How the expire is removed from a key</a></h2><blockquote>When the key is set to a new value using the SET command, or when a keyis destroied via DEL, the timeout is removed from the key.</blockquote>
	38	<h2><a name="Restrictions with write operations against volatile keys">Restrictions with write operations against volatile keys</a></h2><blockquote>IMPORTANT: Since Redis 2.1.3 or greater, there are no restrictions aboutthe operations you can perform against volatile keys, however older versionsof Redis, including the current stable version 2.0.0, has the followinglimitations:</blockquote>
	39	<blockquote>Write operations like LPUSH, LSET and every other command that has theeffect of modifying the value stored at a volatile key have a special semantic:basically a volatile key is destroyed when it is target of a write operation.See for example the following usage pattern:</blockquote>
8fb13ce8	40	<pre class="codeblock python" name="code">
	41	% ./redis-cli lpush mylist foobar /Users/antirez/hack/redis
	42	OK
	43	% ./redis-cli lpush mylist hello /Users/antirez/hack/redis
	44	OK
	45	% ./redis-cli expire mylist 10000 /Users/antirez/hack/redis
	46	1
	47	% ./redis-cli lpush mylist newelement
	48	OK
	49	% ./redis-cli lrange mylist 0 -1 /Users/antirez/hack/redis
	50	1. newelement
abe18d0e	51	</pre><blockquote>What happened here is that LPUSH against the key with a timeout set deletedthe key before to perform the operation. There is so a simple rule, writeoperations against volatile keys will destroy the key before to perform theoperation. Why Redis uses this behavior? In order to retain an importantproperty: a server that receives a given number of commands in the samesequence will end with the same dataset in memory. Without the delete-on-writesemantic what happens is that the state of the server depends on the timethe commands were issued. This is not a desirable property in a distributed databasethat supports replication.</blockquote>
	52	<h2><a name="Restrictions for write operations with volatile keys as sources">Restrictions for write operations with volatile keys as sources</a></h2>Even when the volatile key is not modified as part of a write operation, if it is
	53	read in a composite write operation (such as SINTERSTORE) it will be cleared at the
	54	start of the operation. This is done to avoid concurrency issues in replication.
	55	Imagine a key that is about to expire and the composite operation is run against it.
	56	On a slave node, this key might already be expired, which leaves you with a
	57	desync in your dataset.<h2><a name="Setting the timeout again on already volatile keys">Setting the timeout again on already volatile keys</a></h2><blockquote>Trying to call EXPIRE against a key that already has an associated timeoutwill not change the timeout of the key, but will just return 0. If insteadthe key does not have a timeout associated the timeout will be set and EXPIREwill return 1.</blockquote>
8fb13ce8	58	<h2><a name="Enhanced Lazy Expiration algorithm">Enhanced Lazy Expiration algorithm</a></h2><blockquote>Redis does not constantly monitor keys that are going to be expired.Keys are expired simply when some client tries to access a key, andthe key is found to be timed out.</blockquote>
	59	<blockquote>Of course this is not enough as there are expired keys that will neverbe accessed again. This keys should be expired anyway, so once everysecond Redis test a few keys at random among keys with an expire set.All the keys that are already expired are deleted from the keyspace. </blockquote>
	60	<h3><a name="Version 1.0">Version 1.0</a></h3><blockquote>Each time a fixed number of keys where tested (100 by default). So ifyou had a client setting keys with a very short expire faster than 100for second the memory continued to grow. When you stopped to insertnew keys the memory started to be freed, 100 keys every second in thebest conditions. Under a peak Redis continues to use more and more RAMeven if most keys are expired in each sweep.</blockquote>
	61	<h3><a name="Version 1.1">Version 1.1</a></h3><blockquote>Each time Redis:</blockquote>
	62	<ol><li> Tests 100 random keys from expired keys set.</li><li> Deletes all the keys found expired.</li><li> If more than 25 keys were expired, it start again from 1.</li></ol>
	63	<blockquote>This is a trivial probabilistic algorithm, basically the assumption isthat our sample is representative of the whole key space,and we continue to expire until the percentage of keys that are likelyto be expired is under 25%</blockquote>
	64	<blockquote>This means that at any given moment the maximum amount of keys alreadyexpired that are using memory is at max equal to max setting operations per second divided by 4.</blockquote>
	65	<h2><a name="Return value">Return value</a></h2><a href="ReplyTypes.html">Integer reply</a>, specifically:<br/><br/><pre class="codeblock python python" name="code">
	66	1: the timeout was set.
abe18d0e	67	0: the timeout was not set since the key already has an associated timeout
	68	(this may happen only in Redis versions < 2.1.3, Redis >= 2.1.3 will
	69	happily update the timeout), or the key does not exist.
	70	</pre><h2><a name="FAQ: Can you explain better why Redis < 2.1.3 deletes keys with an EXPIRE on write operations?">FAQ: Can you explain better why Redis < 2.1.3 deletes keys with an EXPIRE on write operations?</a></h2>
ab72b483	71	Ok let's start with the problem:
	72	<pre class="codeblock python python python" name="code">
	73	redis> set a 100
	74	OK
	75	redis> expire a 360
	76	(integer) 1
	77	redis> incr a
	78	(integer) 1
8fb13ce8	79	</pre>
ab72b483	80	I set a key to the value of 100, then set an expire of 360 seconds, and then incremented the key (before the 360 timeout expired of course). The obvious result would be: 101, instead the key is set to the value of 1. Why?
b9f7e9e6	81	There is a very important reason involving the Append Only File and Replication. Let's rework a bit our example adding the notion of time to the mix:
ab72b483	82	<pre class="codeblock python python python python" name="code">
	83	SET a 100
	84	EXPIRE a 5
	85	... wait 10 seconds ...
	86	INCR a
	87	</pre>
	88	Imagine a Redis version that does not implement the "Delete keys with an expire set on write operation" semantic.
abe18d0e	89	Running the above example with the 10 seconds pause will lead to 'a' being set to the value of 1, as it no longer exists when INCR is called 10 seconds later.<br/><br/>Instead if we drop the 10 seconds pause, the result is that 'a' is set to 101.<br/><br/>And in the practice timing changes! For instance the client may wait 10 seconds before INCR, but the sequence written in the Append Only File (and later replayed-back as fast as possible when Redis is restarted) will not have the pause. Even if we add a timestamp in the AOF, when the time difference is smaller than our timer resolution, we have a race condition.<br/><br/>The same happens with master-slave replication. Again, consider the example above: the client will use the same sequence of commands without the 10 seconds pause, but the replication link will slow down for a few seconds due to a network problem. Result? The master will contain 'a' set to 101, the slave 'a' set to 1.<br/><br/>The only way to avoid this but at the same time have reliable non time dependent timeouts on keys is to destroy volatile keys when a write operation is attempted against it.<br/><br/>After all Redis is one of the rare fully persistent databases that will give you EXPIRE. This comes to a cost :)<h2><a name="FAQ: How this limitations were solved in Redis versions > 2.1.3?">FAQ: How this limitations were solved in Redis versions > 2.1.3?</a></h2>Since Redis 2.1.3 there are no longer restrictions in the use you can do of write commands against volatile keys, still the replication and AOF file are guaranteed to be fully consistent.<br/><br/>In order to obtain a correct behavior without sacrificing consistency now when a key expires, a DEL operation is synthesized in both the AOF file and against all the attached slaves. This way the expiration process is centralized in the master instance, and there is no longer a chance of consistency errors.<br/><br/>However while the slaves while connected to a master will not expire keys independently, they'll still take the full state of the expires existing in the dataset, so when a slave is elected to a master it will be able to expire the keys independently, fully acting as a master.
8fb13ce8	90	</div>
	91
	92	</div>
	93	</div>
	94	</body>
	95	</html>
	96