Add custom comparators

[matthew-mcallister]

Closes #1180.

The diff is large due to having to add a type parameter C and update constraints in numerous places, but the logic changes are not large.

This trait adds the Comparator and Equivalator traits, which generalize the Comparable and Equivalent traits by taking a self parameter in addition to the two keys to compare, making it possible to change the comparison function used without changing the key type.

Summary

• Adds trait Comparator<L, R> and trait Equivalator<L, R>.
• Adds struct BasicComparator, which is a zero-sized type that implements Comparator using the Comparable trait, which itself falls back on Borrow and Ord.
• Adds a type parameter C to SkipList, SkipMap, and SkipSet.
  • C is defaulted to OrdComparator. This preserves the existing behavior of functions like get(), insert(), etc.
• Adds SkipList::with_comparator(), SkipSet::with_comparator(), and SkipMap::with_comparator() constructors.
  • SkipList::new(), SkipMap::new(), and SkipSet::new() always use C = OrdComparator. It would be nice if it worked for any C that implements Default, but that would break type inference. This is exactly the reason std::collections::HashMap::new() always uses S = RandomState for its defaulted parameter even if S implements Default.
• Adds a test using a wrapper around Box<dyn Fn(&[u8], &[u8]) -> Ordering> as the comparator.

Hashing

Because the goal of these traits is to support totally different comparison operations from the default for the underlying key type, Equivalator explicitly does not require two equivalent keys to have the same hash. If someone, someday wants to support Equivalator on their hash map, they will have to generalize the Hash trait as well.

Safety

My one minor question is whether unsafe impl<Q, R, K, V, C> Send for RefRange<'_, Q, R, K, V, C> should require C: Sync. I don't think so because it doesn't require K or V to be sync, but it was enough to make me second-guess myself.

[taiki-e]

cc @al8n

[matthew-mcallister]

Restored the Equivalent and Comparable traits. Now there is no change in behavior compared to master unless you explicitly define your own comparator type.

[matthew-mcallister]

It's been a while since I circled back to this, but is there a sentiment as to whether to merge?

[agavra]

This looks good to me, though I admit I'm a little sketched out by the concept of breaking the hash/eq dependency definition with Equivalator trait...

Would be good to get some committers reviewing this as well! Excited for this feature.

[agavra]

It would be good to spell out the motivations for this. It is always possible for the user to implement some funky transform of their keys before they insert into a set/map and when retrieving data as a workaround for not having this. (e.g. in your test, I think a much better implementation if the user wanted that is to just set.insert(key.length()) instead of doing something convoluted with the eq function). That being said, you could still have a comparator that sorts based on key.length and considers two keys equivalent for a sort order if they have the same length.

On the other hand, if we introduce this, it would be impossible to leverage traditional hash structures as optimizations in the future (e.g. add a bloom filter on top of a skiplist) because two elements could be equivalent under this equivilator but have different hashes.

It generally seems dangerous to me to break the standard hash/eq dependency that is implicitly used by so many data structures.

[matthew-mcallister]

It would be good to spell out the motivations for this.

Agreed. My example was pretty contrived; I can provide more realistic motivation.

Though I will remark that set.insert(key.length()) would produce a totally different program as you'd lose access to the first key inserted. Even skip_map.insert(key.length(), key) is not equivalent since that would give you the last key inserted instead of the first.

It generally seems dangerous to me to break the standard hash/eq dependency that is implicitly used by so many data structures.

Well, this doesn't "break" the Hash/Eq dependency since nothing changes if you stick to the standard library traits. But I think the documentation here may be confusing because hashing was out of scope for this PR, so there is no indication of how you would handle hashing when you need it. In reality, hashing is very easy to accommodate; it just requires another "Hash"-ator trait to go alongside Comparator and Equivalator. This is another concept that has been in the C++ standard library for decades.

[agavra]

Got it! If there's a good reference implementation in the standard C++ library that gives me a lot more confidence in the approach. Thanks for the explanation.

[agavra]

Suggested change

	/// sorted by key.
	/// sorted by the comparator's ordering of the key.

[matthew-mcallister]

Should also update SkipSet and change all method docs to use the term "equivalent" (instead of equal).

[agavra]

probably makes sense to add tests for SkipMap as well

[matthew-mcallister]

Sorry it took me an eternity to circle back round to this. I got busy and I'm bad at multitasking.

I pushed a commit that addresses the last round of feedback with respect to comments and tests/examples. Are there any maintainers that want to provide feedback and give a disposition as to whether there is a desire to merge this? Or whether we should perhaps take a different approach, such as spinning out the traits into a new crate and adding a hashing trait for DB folks who want to support this with their Bloom filters?