This issue is to organize and track progress on abstracting TiKV over generic storage
engines (previous issue: #4184). There are many tasks to be done and contributions are welcome.

In scope: the process of abstracting the entire RocksDB engine API into the engine_traits and engine_rocks crates, decoupling TiKV and its test suite from RocksDB., and prototyping new engine bindings. Out of scope: refactoring engine_traits to better support with non-RocksDB engines.

The engine_traits crate docs describe the design and
migration process.

The issue is updated to contain a checklist of tasks that definitely need to
happen, along with name of who is working on it. When you decide to take a task,
say so on the issue before attempting it so that someone else doesn't duplicate
your work.

It's hard to know in what order these need to happen, nor exactly how to
complete each of them, as each tends to present new challenges. Instead I have
documented here, in the engine_traits crate docs, and in a status report, my experiences so far and suggestions for how to proceed.

The most important thing to keep in mind is:

Make small changes: small commits, small PRs. When you find that something else
needs to change first, stop what you are doing, and make that change, and submit
that PR. Don't let your patches grow out of control. Don't be afraid to throw away
your work and try something else.

Task list (updated 2021/01/10)

• launch tikv-server with PanicEngine
  • Make it possible to create a tikv-server with a non-rocks KV engine #10528
• eliminate direct use of engine_rocks in raftstore (brson)
  • blocked on Make PerfContext abstract and use it in raftstore #9243
• Extract storage::kv to a crate
  • Extract tikv::storage::kv to its own crate #9508
• Remove table properties abstractions
  • Remove TableProperties abstractions from engine_traits #10068
• Remove engine_rocks from tikv_kv
  • Remove engine_rocks dependency from tikv_kv #9609
• Refactor storage::kv
  • Refactor storage::kv to reduce abstraction duplication #9480
• eliminate direct use of engine_rocks in tikv crate
  • lots of small abstraction leaks
  • (blocked) Convert more of tikv crate to engine traits #8980
  • (related) Refactor storage::kv to reduce abstraction duplication #9480
• figure out how to construct all engines and configure tikv to use them
  • Move engine_rocks::config to a new engine_config crate #9242
  • need to solve previous problems first
  • this will probably involve an engines_ctor crate that knows about every engine,
    based on code already in the engine_test crate that constructs multiple engines
  • to start with, non-rocks engines will probably need to use the current rocks-oriented
    configuration options on a best-effort basis, just to get tikv running
• modify test cases to use engine_test instead of engine_rocks
• Write a engine_traits_tests test suite that is parameterized over implementations
• Create engine_traits bindings for agatedb
  • Create engine_trait bindings for agatedb #9844
  • brson started
• Write tests for advanced features in engine_traits_tests
• Remove unnecessary dependencies from engine_traits
  • there are several here at the wrong abstraction level, e.g. protobufs
  • this is just cleanup, not necessary functionality
• Clean up WriteBatch / Mutable traits
  • engine_traits: merge Mutable trait with WriteBatch #9481