From d158dc28f6617fd517172e04d3f46ce53d88013e Mon Sep 17 00:00:00 2001 From: antirez Date: Mon, 3 Jan 2011 10:58:59 +0100 Subject: [PATCH] comments on top of dscache.c updated --- src/dscache.c | 81 ++++++++++++--------------------------------------- 1 file changed, 18 insertions(+), 63 deletions(-) diff --git a/src/dscache.c b/src/dscache.c index 9c842188..6bfcac1d 100644 --- a/src/dscache.c +++ b/src/dscache.c @@ -23,76 +23,29 @@ * longer relevant with the new design. Here as a checklist to see if * some old ideas still apply. * - * - The WATCH helper will be used to signal the cache system - * we need to flush a given key/dbid into disk, adding this key/dbid - * pair into a server.ds_cache_dirty linked list AND hash table (so that we - * don't add the same thing multiple times). - * - * - cron() checks if there are elements on this list. When there are things - * to flush, we create an IO Job for the I/O thread. - * NOTE: We disalbe object sharing when server.ds_enabled == 1 so objects - * that are referenced an IO job for flushing on disk are marked as - * o->storage == REDIS_DS_SAVING. - * - * - This is what we do on key lookup: - * 1) The key already exists in memory. object->storage == REDIS_DS_MEMORY - * or it is object->storage == REDIS_DS_DIRTY: - * We don't do nothing special, lookup, return value object pointer. - * 2) The key is in memory but object->storage == REDIS_DS_SAVING. - * When this happens we block waiting for the I/O thread to process - * this object. Then continue. - * 3) The key is not in memory. We block to load the key from disk. - * Of course the key may not be present at all on the disk store as well, - * in such case we just detect this condition and continue, returning - * NULL from lookup. - * - * - Preloading of needed keys: - * 1) As it was done with VM, also with this new system we try preloading - * keys a client is going to use. We block the client, load keys - * using the I/O thread, unblock the client. Same code as VM more or less. - * - * - Reclaiming memory. - * In cron() we detect our memory limit was reached. What we - * do is deleting keys that are REDIS_DS_MEMORY, using LRU. - * - * If this is not enough to return again under the memory limits we also - * start to flush keys that need to be synched on disk synchronously, - * removing it from the memory. We do this blocking as memory limit is a - * much "harder" barrirer in the new design. - * - * - IO thread operations are no longer stopped for sync loading/saving of - * things. When a key is found to be in the process of being saved - * we simply wait for the IO thread to end its work. - * - * Otherwise if there is to load a key without any IO thread operation - * just started it is blocking-loaded in the lookup function. - * * - What happens when an object is destroyed? * - * If o->storage == REDIS_DS_MEMORY then we simply destory the object. - * If o->storage == REDIS_DS_DIRTY we can still remove the object. It had - * changes not flushed on disk, but is being removed so - * who cares. - * if o->storage == REDIS_DS_SAVING then the object is being saved so - * it is impossible that its refcount == 1, must be at - * least two. When the object is saved the storage will - * be set back to DS_MEMORY. - * - * - What happens when keys are deleted? + * If the object is destroyed since semantically it was deleted or + * replaced with something new, we don't care if there was a SAVE + * job pending for it. Anyway when the IO JOb will be created we'll get + * the pointer of the current value. * - * We simply schedule a key flush operation as usually, but when the - * IO thread will be created the object pointer will be set to NULL - * so the IO thread will know that the work to do is to delete the key - * from the disk store. + * If the object is already a REDIS_IO_SAVEINPROG object, then it is + * impossible that we get a decrRefCount() that will reach refcount of zero + * since the object is both in the dataset and in the io job entry. * * - What happens with MULTI/EXEC? * - * Good question. + * Good question. Without some kind of versioning with a global counter + * it is not possible to have trasactions on disk, but they are still + * useful since from the point of view of memory and client bugs it is + * a protection anyway. Also it's useful for WATCH. * - * - If dsSet() fails on the write thread log the error and reschedule the - * key for flush. + * Btw there is to check what happens when WATCH gets combined to keys + * that gets removed from the object cache. Should be save but better + * to check. * - * - Check why INCR will not update the LRU info for the object. + * - Check if/why INCR will not update the LRU info for the object. * * - Fix/Check the following race condition: a key gets a DEL so there is * a write operation scheduled against this key. Later the same key will @@ -114,10 +67,12 @@ * not marked as cacheKeyDoesNotExist(), otherwise, again, we can load * data from disk that should instead be deleted. * - * - dsSet() use rename(2) in order to avoid corruptions. + * - dsSet() should use rename(2) in order to avoid corruptions. * * - Don't add a LOAD if there is already a LOADINPROGRESS, or is this * impossible since anyway the io_keys stuff will work as lock? + * + * - Serialize special encoded things in a raw form. */ /* Virtual Memory is composed mainly of two subsystems: -- 2.45.2