- = A fifteen minutes introduction to Redis data types =
As you already probably know Redis is not a plain key-value store, actually it is a
data structures server, supporting different kind of values. That is, you can't just set strings as values of keys. All the following data types are supported as values:
- Binary-safe strings.
- Lists of binary-safe strings.
- Sets of binary-safe strings, that are collection of unique unsorted elements. You can think at this as a Ruby hash where all the keys are set to the 'true' value.
- Sorted sets, similar to Sets but where every element is associated to a floating number score. The elements are taken sorted by score. You can think at this as Ruby hashes where the key is the element and the value is the score, but where elements are always taken in order without requiring a sorting operation.
+ #sidebar
RedisGuides
+
As you already probably know Redis is not a plain key-value store, actually it is a
data structures server, supporting different kind of values. That is, you can't just set strings as values of keys. All the following data types are supported as values:
- Binary-safe strings.
- Lists of binary-safe strings.
- Sets of binary-safe strings, that are collection of unique unsorted elements. You can think at this as a Ruby hash where all the keys are set to the 'true' value.
- Sorted sets, similar to Sets but where every element is associated to a floating number score. The elements are taken sorted by score. You can think at this as Ruby hashes where the key is the element and the value is the score, but where elements are always taken in order without requiring a sorting operation.
It's not always trivial to grasp how this data types work and what to use in order to solve a given problem from the
Redis command reference, so this document is a crash course to Redis data types and their most used patterns.
For all the examples we'll use the
redis-cli utility, that's a simple but handy command line utility to issue commands against the Redis server.
Before to start talking about the different kind of values supported by Redis it is better to start saying that keys are not binary safe strings in Redis, but just strings not containing a space or a newline character. For instance "foo" or "123456789" or "foo_bar" are valid keys, while "hello world" or "hello\n" are not.
Actually there is nothing inside the Redis internals preventing the use of binary keys, it's just a matter of protocol, and actually the new protocol introduced with Redis 1.2 (1.2 betas are 1.1.x) in order to implement commands like MSET, is totally binary safe. Still for now consider this as an hard limit as the database is only tested with "normal" keys.
A few other rules about keys:
- Too long keys are not a good idea, for instance a key of 1024 bytes is not a good idea not only memory-wise, but also because the lookup of the key in the dataset may require several costly key-comparisons.
- Too short keys are not a good idea. There is no point in writing "u:1000:pwd" as key if you can write instead "user:1000:password", the latter is more readable and the added space is very little compared to the space used by the key object itself.
- Try to stick with a schema. For instance "object-type:id:field" can be a nice idea, like in "user:1000:password". I like to use dots for multi-words fields, like in "comment:1234:reply.to".
This is the simplest Redis type. If you use only this type, Redis will be something like a memcached server with persistence.
Let's play a bit with the string type:
$ ./redis-cli set mykey "my binary safe value"
@@ -42,7 +43,7 @@ $ ./redis-cli incr counter
(integer) 102
$ ./redis-cli incrby counter 10
(integer) 112
-
The
INCR command parses the string value as an integer, increments it by one, and finally sets the obtained value as the new string value. There are other similar commands like
INCRBY,
DECR and
DECRBY. Actually internally it's always the same command, acting in a slightly different way.
What means that INCR is atomic? That even multiple clients issuing INCR against the same key will never incur into a race condition. For instance it can't never happen that client 1 read "10", client 2 read "10" at the same time, both increment to 11, and set the new value of 11. The final value will always be of 12 ad the read-increment-set operation is performed while all the other clients are not executing a command at the same time.
Another interesting operation on string is the
GETSET command, that does just what its name suggests: Set a key to a new value, returning the old value, as result. Why this is useful? Example: you have a system that increments a Redis key using the
INCR command every time your web site receives a new visit. You want to collect this information one time every hour, without loosing a single key. You can GETSET the key assigning it the new value of "0" and reading the old value back.
To explain the List data type it's better to start with a little of theory, as the term
List is often used in an improper way by information technology folks. For instance "Python Lists" are not what the name may suggest (Linked Lists), but them are actually Arrays (the same data type is called Array in Ruby actually).
From a very general point of view a List is just a sequence of ordered elements: 10,20,1,2,3 is a list, but when a list of items is implemented using an Array and when instead a
Linked List is used for the implementation, the properties change a lot.
Redis lists are implemented via Linked Lists, this means that even if you have million of elements inside a list, the operation of adding a new element in the head or in the tail of the list is performed
in constant time. Adding a new element with the
LPOP command to the head of a ten elements list is the same speed as adding an element to the head of a 10 million elements list.
What's the downside? That accessing an element
by index is very fast in lists implemented with an Array and not so fast in lists implemented by linked lists.
Redis Lists are implemented with linked lists because for a database system is crucial to be able to add elements to a very long list in a very fast way. Another strong advantage is, as you'll see in a moment, that Redis Lists can be taken at constant length in constant time.
The
LPUSH command add a new element into a list, on the left (on head), while the
RPUSH command add a new element into alist, ot the right (on tail). Finally the
LRANGE command extract ranges of elements from lists:
+
The
INCR command parses the string value as an integer, increments it by one, and finally sets the obtained value as the new string value. There are other similar commands like
INCRBY,
DECR and
DECRBY. Actually internally it's always the same command, acting in a slightly different way.
What means that INCR is atomic? That even multiple clients issuing INCR against the same key will never incur into a race condition. For instance it can't never happen that client 1 read "10", client 2 read "10" at the same time, both increment to 11, and set the new value of 11. The final value will always be of 12 ad the read-increment-set operation is performed while all the other clients are not executing a command at the same time.
Another interesting operation on string is the
GETSET command, that does just what its name suggests: Set a key to a new value, returning the old value, as result. Why this is useful? Example: you have a system that increments a Redis key using the
INCR command every time your web site receives a new visit. You want to collect this information one time every hour, without loosing a single key. You can GETSET the key assigning it the new value of "0" and reading the old value back.
To explain the List data type it's better to start with a little of theory, as the term
List is often used in an improper way by information technology folks. For instance "Python Lists" are not what the name may suggest (Linked Lists), but them are actually Arrays (the same data type is called Array in Ruby actually).
From a very general point of view a List is just a sequence of ordered elements: 10,20,1,2,3 is a list, but when a list of items is implemented using an Array and when instead a
Linked List is used for the implementation, the properties change a lot.
Redis lists are implemented via Linked Lists, this means that even if you have million of elements inside a list, the operation of adding a new element in the head or in the tail of the list is performed
in constant time. Adding a new element with the
LPUSH command to the head of a ten elements list is the same speed as adding an element to the head of a 10 million elements list.
What's the downside? That accessing an element
by index is very fast in lists implemented with an Array and not so fast in lists implemented by linked lists.
Redis Lists are implemented with linked lists because for a database system is crucial to be able to add elements to a very long list in a very fast way. Another strong advantage is, as you'll see in a moment, that Redis Lists can be taken at constant length in constant time.
The
LPUSH command add a new element into a list, on the left (on head), while the
RPUSH command add a new element into alist, ot the right (on tail). Finally the
LRANGE command extract ranges of elements from lists:
$ ./redis-cli rpush messages "Hello how are you?"
OK
$ ./redis-cli rpush messages "Fine thanks. I'm having fun with Redis"
@@ -142,6 +143,7 @@ $ ./redis-cli zrangebyscore hackers -inf 1950
$ ./redis-cli zremrangebyscore hackers 1940 1960
(integer) 2
ZREMRANGEBYSCORE is not the best command name, but it can be very useful, and returns the number of removed elements.
For the last time, back to the Reddit example. Now we have a decent plan to populate a sorted set in order to generate the home page. A sorted set can contain all the news that are not older than a few days (we remove old entries from time to time using ZREMRANGEBYSCORE). A background job gets all the elements from this sorted set, get the user votes and the time of the news, and compute the score to populate the
reddit.home.page sorted set with the news IDs and associated scores. To show the home page we have just to perform a blazingly fast call to ZRANGE.
From time to time we'll remove too old news from the
reddit.home.page sorted set as well in order for our system to work always against a limited set of news.
Just a final note before to finish this tutorial. Sorted sets scores can be updated at any time. Just calling again ZADD against an element already included in the sorted set will update its score (and position) in O(log(N)), so sorted sets are suitable even when there are tons of updates.
This tutorial is in no way complete, this is just the basics to get started with Redis, read the
Command Reference to discover a lot more.
Thanks for reading. Salvatore.
+