Storage Transfer Service

Transfer service for on-premises data produces transfer logs so that you can verify the results of your transfer jobs and offers an option to monitor progress of transfer jobs via Cloud Monitoring. Storage Transfer Service could be configured to deliver Pub/Sub notification on transfer completion.

Storage Transfer Service uses metadata available from the source storage system, such as checksums and file sizes, to ensure that data written to Cloud Storage is the same data read from the source. In addition to data integrity provided by the TLS protocol, the Storage Transfer Service for on-premises data calculates a CRC32C checksum over each file that it copies as the file is being read.

We use TLS encryption for HTTPs connections, through the public internet and private connections, and support transfers to Cloud Storage buckets protected by VPC Service Controls. The only exception to TLS encryption is if you specify an HTTP URL for a URL list transfer. If you are using interconnect, you can get another layer of security by accessing a private API endpoint.

You can schedule one-time transfer operations or recurring transfer operations. When transferring data from other cloud providers, you can schedule periodic synchronization with advanced filters based on file creation dates, file-names, and the times of day you prefer to import data. There is no manual intervention needed, and the results are recorded in the Cloud Console.

We’ve incorporated controls into Storage Transfer Service for on-premises data to avoid disruption of your day-to-day business operations. Configure a bandwidth limit for your Google Cloud project to limit the rate at which on-premises agents will copy files. The bandwidth limit is shared across all transfer jobs in your project.

By default all transfers begin by evaluating the data present at the source and destination. This process determines which source objects are new, updated, or deleted since the last transfer. With this initial step, you are able to minimize the amount of data that needs to be sent, use bandwidth effectively, and ensure transfers run quickly.

You can use includePrefix and excludePrefix when creating transfers to limit what objects Storage Transfer Service operates on. We also support modification-time-based data filtering options.

When transferring data from on-premises locations, getting better performance is as easy as bringing up more Storage Transfer Service agents. No transfer config changes or re-submissions required. As soon as the agents are up, they will start performing transfer work.

We make reliability work out of the box. If some of your agents fail, the remaining ones will pick up their work. If all of your agents fail, no problem—as soon as you bring them back up, transfers will pick up where they had left off. No special recovery or retry logic required.

When using Storage Transfer Service for on-premises data, we charge at a per-GB rate. When transferring data from other cloud providers, Storage Transfer Service is free of charge. After transferring data, you are charged for data stored on Cloud Storage as documented in Cloud Storage pricing, and external cloud providers' costs may also apply while you use Storage Transfer Service.

You can use Storage Transfer Service to transfer data from a list of public data locations to a Cloud Storage bucket.