ti-chi-bot
added
the
size/XXL
label
Jan 27, 2021
ti-chi-bot
removed
the
needs-rebase
label
Jan 28, 2021Merged
ti-chi-bot pushed a commit that referenced this pull request
Feb 5, 2021
### What problem does this PR solve? In principle this util file shouldn't belong exclusively to external_storage and instead should be shareable among other crates. In practice I needed to make this change for my cloud library refactoring [PR that is WIP](#9576): cloud code depends on this and external_storage will depend on cloud code. That PR is getting very large and I am trying to break out some separate changes from it. We already have a future.rs file and perhaps the two of these should be re-organized appropriately. However, I am already working on massive refactoring so would like to leave that for someone else to do. /cc @kennytm @yiwu-arbug @BusyJay ### What is changed and how it works? move external_storage/util to tikv_util/stream ### Related changes - #9576 still WIP will rebase to use this commit. ### Check List <!--REMOVE the items that are not applicable--> Tests - No code changes, so I am relying on the compiler and test suite Side effects None ### Release note None
ti-chi-bot
removed
the
needs-rebase
label
Feb 7, 2021Greg Weber and others added 10 commits
February 8, 2021 15:03add a new create encryption/export Signed-off-by: Greg Weber <greg@pingcap.com> Signed-off-by: Greg Weber <greg@gregweber.info>
Cloud providers should import TiKV functionality from here. Currently it only re-exports crates. In the future it may provide some general cloud adapters. Signed-off-by: Greg Weber <greg@pingcap.com>
ti-chi-bot pushed a commit that referenced this pull request
Feb 24, 2021### What problem does this PR solve? We are organizing the cloud code in a separate crate. This will facilitate the cloud team being able to own the code and modifying builds to include or exclude cloud providers or in the future dynamically link them. This is the encryption portion split off from this cloud re-organization PR: #9576 After this PR is merged, I will create a second PR that organizes the external_storage code. ### What is changed and how it works? * AWS code is moved from the encryption crate into the cloud/aws crate * To further separate out the cloud crate, the encryption crate depends on the `KmsProvider` trait that it exposes. A few types (DataKeyPair, EncryptedKey) are duplicated in the cloud encryption crate to avoid any dependency. * components/encryption/export is added to avoid cyclical dependencies. Cloud constructors of encryption traits are imported here and then re-exported along with the encryption crate for end users of encryption * rusoto_util code is moved to aws/util. This is currently duplicated because external_storage still depends on rusoto_util. The next PR will organize the external_storage code and remove the rusoto_util crate. /cc @yiwu-arbug ### Related changes This is the encryption portion split off from this cloud re-organization PR: #9576 A second PR will be based off of that to organize the external_storage code. ### Check List Tests - Unit test: new unit tests covering the functionality - Integration test: no new integration tests - Manual test: basic test of running with encryption and rotating keys Compatibility - New cloud error codes are used for AWS KMS operations. - Programs built from the TiKV lib (tikv-server, etc) now import the encryption_export crate instead of encryption directly. ### Release note Move KMS cloud provider code out of encryption and into separate cloud provider crates.
Merged
5 tasks