Storage
Cloud Storage
Cloud Storage provides globally unified, scalable, and highly durable object storage. You can store files in Cloud Storage without having to worry about running out of space, or managing your own filesystems.
Cloud Storage buckets can be configured to store files in a single region, dual-region, or multi-region.
Location | Use Case |
Region | Use a region to help optimize latency and network bandwidth for data consumers, such as analytics pipelines, that are grouped in the same region. |
Dual-Region | Use a dual-region when you want similar performance advantages as regions, but also want the higher availability that comes with being geo-redundant. |
Multi-Region | Use a multi-region when you want to serve content to data consumers that are outside of the Google network and distributed across large geographic areas, or when you want the higher availability that comes with being geo-redundant. |
See Cloud Storage Bucket locations for more information.
Cloud Storage buckets can also be configured with a different Storage Class, for different access patterns to optimize cost.
Storage Class | Use Case |
Standard | Standard Storage is best for data that is frequently accessed ("hot" data) and/or stored for only brief periods of time. |
Nearline | Nearline Storage is a low-cost, highly durable storage service for storing infrequently accessed data. Nearline Storage is a better choice than Standard Storage in scenarios where slightly lower availability, a 30-day minimum storage duration. |
Coldline | Coldline Storage is a very-low-cost, highly durable storage service for storing infrequently accessed data. Coldline Storage is a better choice than Standard Storage or Nearline Storage in scenarios where slightly lower availability, a 90-day minimum storage duration. |
Archive | Archive Storage is the lowest-cost, highly durable storage service for data archiving, online backup, and disaster recovery. Unlike the "coldest" storage services offered by other Cloud providers, your data is available within milliseconds, not hours or days. |
See Cloud Storage Classes for more information.
Enable API
Create a Bucket
A bucket is the top-level directory that you can add additional files and sub-directories into. A bucket name must be globally unique. For example, create a bucket with the same name as your Google Cloud project.
Copy a File
Delete a File
Spring Resource
With Spring Cloud GCP, you can use Spring Resource to perform store and retrieve file from Cloud Storage.
Dependency
Add the Spring Cloud GCP Storage starter:
Configuration
There is no explicit configuration required if you use the automatic authentication and project ID detection. I.e., if you already logged in locally with gcloud
command line, then it'll automatically access buckets that you have access to.
Notice that there is no explicit configuration for username/password. Cloud Storage authentication uses the GCP credential (either your user credential, or Service Account credential), and authorization is configured via Identity Access Management (IAM).
Storage Client
The starter automatically creates a pre-configured Storage
bean that provides raw-access to Google Cloud Storage.
Resource URI
You can address a Cloud Storage file by using the resource URI prefixed with gs://
. The fully qualified URI is of the form: gs://project-id/path/to/file
.
Read a file
You can open the Resource
using ApplicationContext
. Then read the content from InputStream
. You must close
the stream when you are done (or wrap with try-with-resource since it's auto-closeable).
Write a file
You can open the Resource
using ApplicationContext
. Then cast the Resource
to a WritableResource
, and then use the OutputStream
to write the content. Lastly, you must close
the stream in order for the file to write (or wrap with try-with-resource since it's auto-closeable).
Samples
Spring Integration
You can channel adapters for Google Cloud Storage to read and write files to Google Cloud Storage through MessageChannels
.
Dependency
Add both the Spring Cloud GCP Storage starter, and Spring Integration File component.
Inbound Channel Adapter
Inbound File Synchronizer
File-based integration with files typically requires polling a directory that contains the new files. In Spring Integration, this is configured through an InboundFileSynchronizer
. Use GcsInboundFileSyncronizer
to create a MessageSource
and adapt it to a MessageChannel
.
The files are temporarily stored in a directory in the local file system.
Streaming Message Source
For most use cases, you should use the streaming message source, which does not require files to be stored in the file system.
Outbound Channel Adapter
The outbound channel adapter allows files to be written to Google Cloud Storage. When it receives a Message
containing a payload of type File
, it writes that file to the Google Cloud Storage bucket specified in the adapter.
Samples
Last updated