GitLab

## Background Offline transfer (&8985) currently supports `aws` and `s3_compatible` providers via `Import::Clients::ObjectStorage` (fog-aws). A self-managed customer migrating namespaces to GitLab.com requested GCS support with [Application Default Credentials](https://docs.cloud.google.com/docs/authentication/application-default-credentials) (ADC) for the export side. The import on GitLab.com would use explicit credentials (HMAC keys via `s3_compatible`, or a service account JSON key via a new native `gcs` provider). `fog-google ~> 1.29` is already in the Gemfile and supports `google_application_default: true` (used by consolidated object storage today). ## Questions to answer ### Interim path (unblock the customer now) 1. Does the GCS S3-interop path (`s3_compatible` + `endpoint: https://storage.googleapis.com` + [HMAC keys](https://cloud.google.com/storage/docs/authentication/hmackeys)) work end-to-end today? Verify multipart upload (`multipart_chunk_size: 100.megabytes`) and streaming download against GCS with a multi-GB export. 2. Are `offline_transfer_imports` and `allow_s3_compatible_storage_for_offline_transfer` enabled on GitLab.com? If not, what's needed to enable them for this customer? 3. Is the customer's GCP org policy compatible with HMAC key creation? ### Native GCS provider 4. Verify fog-google behavior for our client's operations: `head_bucket`/`request_url` equivalents, listing pagination (`max_results` vs `max_keys`), error classes (`Google::Apis::Error` vs `Fog::Errors::Error`/`Excon::Error`), and large-object upload/download performance. 5. Decide the credentials schema for a `gcs` provider: `google_project` + (`google_json_key_string` | `google_application_default`), and how to relax the `object_storage_credentials` presence/JSON-schema validation for the ADC case. 6. Decide client architecture: provider-specific adapters behind the existing `Import::Clients::ObjectStorage` interface vs conditionals. ### ADC security 7. Confirm ADC must be export-side, self-managed/Dedicated only, gated behind an admin application setting (mirroring `allow_s3_compatible_storage_for_offline_transfer`). On GitLab.com, ADC would resolve to GitLab's own infra service account and must be rejected. Get AppSec input. ## Outcome - Create implementation issues for GCS support (expected: model/schema + API params, object storage client GCS adapter, ADC admin setting + gating, docs). - Decide when to schedule the work (Experiment or Beta or GA?)

Read the original on gitlab.com ↗