How to determine the prefix in a sublevel

[kldavis4]

I am working on a redis level implementation where I need to know which part of the key parameter passed in to _put() is the sublevel prefix and what is the key without the prefix. So for this example:

const db = Mylevel()
const sub1 = db.sublevel('sub1')
const sub2 = sub1.sublevel('sub2')

await sub2.put('foo', 'bar')

So with the default separator, my _put implementation would get a key like !sub1!!sub2!foo. How can my _put implementation know that foo is the key in the sublevel and the prefix is !sub1!!sub2! (and is my reasoning so far correct)?

I can think of two ways of dealing with this. Override AbstractSublevel and pass the separator in where needed and then use that to extract the prefix from the key. Alternatively, again overriding AbstractSublevel, keep track of the name of each sublevel and pass it down as part of the options.

It seems like neither of these is particularly clean so wondering if there is any better way to do this.

[vweevers]

I need to know which part of the key parameter passed in to _put() is the sublevel prefix

Why do you need this? For context.

[vweevers]

Somewhat relevant: #61

[kldavis4]

thanks for your response.

I'm using redisdown as a guide for implementation. If you look at https://github.com/hmalphettes/redisdown/blob/master/index.js#L167 the put function does this:

RedisDown.prototype.__appendPutCmd = function (commandList, key, value, prefix) {
    var resolvedPrefix = this.__getPrefix(prefix);
    key = cleanKey(key);
    commandList.push(['hset', resolvedPrefix + ':h', key, value === undefined ? '' : value]);
    commandList.push(['zadd', resolvedPrefix + ':z', 0, key]);
    return commandList;
};

So the prefix provides a namespace.

[vweevers]

That stems from old level-sublevel conventions: hmalphettes/redisdown#23. Since then, sublevels have become a builtin feature of abstract-level, and batch operations don't have a prefix option (we do have a sublevel option which is conceptually similar, but not as loose).

I would have argued against using the name "prefix" in that PR though, because it's not just doing sublevels: it allows mixing in entirely separate databases (that have a different location). That's outside the scope of "official" sublevels. I should also note that they will be more strict in abstract-level v2:

If a sublevel [option] is provided as an option to db.batch(), that sublevel must now be a descendant of db:

const colorIndex = indexes.sublevel('colors')
const flavorIndex = indexes.sublevel('flavors')

// No longer works because colorIndex isn't a descendant of flavorIndex
flavorIndex.batch([{ type: 'del', key: 'blue', sublevel: colorIndex }])

// OK
indexes.batch([{ type: 'del', key: 'blue', sublevel: colorIndex }])

// OK
db.batch([{ type: 'del', key: 'blue', sublevel: colorIndex }])

If you're looking to implement a more limited subset of prefix features than redisdown, then let's define that. Might change the story. In general, I would let abstract-sublevel take care of the sublevels for you, unless there's a significant performance benefit to handling it in Redis.

[kldavis4]

thanks for the comment @vweevers and I think I'm following...

I would very much prefer it if my level implementation is unaware of the prefix used for the key in the sub-level. I've written three different abstract level implementations now (one in sql and two KV stores) and it has been pretty straightforward.

What I am tripping over is that if I use the same approach as redisdown for sorting keys (sorted set), query / mutation performance will be proportional to the number of items in the level and not the number of items in the smaller sublevel. It looks like redisdown restricts the sorted set key to the sublevel prefix, which I think means query / mutations mutation performance is proportional to the number of items in the specific sublevel (not the level). Is that reasoning correct? Maybe this is premature / over optimization on my part, but I'm also not sure why some sort of namespace metadata in the level commands would be a bad design.