Python

Version History (0.6.0)
Report an Issue

Storage#

Cloud Storage in 10 seconds#

Install the library#

The source code for the library (and demo code) lives on GitHub, You can install the library quickly with pip:

$ pip install gcloud

Run the demo#

In order to run the demo, you need to have registred an actual gcloud project and so you’ll need to provide some environment variables to facilitate authentication to your project:

  • GCLOUD_TESTS_PROJECT_ID: Developers Console project ID (e.g. bamboo-shift-455).
  • GCLOUD_TESTS_DATASET_ID: The name of the dataset your tests connect to. This is typically the same as GCLOUD_TESTS_PROJECT_ID.
  • GOOGLE_APPLICATION_CREDENTIALS: The path to a JSON key file; see regression/app_credentials.json.sample as an example. Such a file can be downloaded directly from the developer’s console by clicking “Generate new JSON key”. See private key docs for more details.

Run the example script included in the package:

$ python -m gcloud.storage.demo

And that’s it! You should be walking through a demonstration of using gcloud.storage to read and write data to Google Cloud Storage.

Try it yourself#

You can interact with a demo dataset in a Python interactive shell.

Start by importing the demo module and instantiating the demo connection:

>>> from gcloud.storage import demo
>>> connection = demo.get_connection()

Once you have the connection, you can create buckets and blobs:

>>> from gcloud import storage
>>> storage.list_buckets(connection)
[<Bucket: ...>, ...]
>>> bucket = storage.create_bucket('my-new-bucket', connection=connection)
>>> print bucket
<Bucket: my-new-bucket>
>>> blob = storage.Blob('my-test-file.txt', bucket=bucket)
>>> print blob
<Blob: my-new-bucket, my-test-file.txt>
>>> blob = blob.upload_from_string('this is test content!')
>>> print blob.download_as_string()
'this is test content!'
>>> print bucket.list_blobs()
[<Blob: my-new-bucket, my-test-file.txt>]
>>> blob.delete()
>>> bucket.delete()

Note

The get_connection method is just a shortcut for:

>>> from gcloud import storage
>>> from gcloud.storage import demo
>>> connection = storage.get_connection(demo.PROJECT_ID)

gcloud.storage#

Shortcut methods for getting set up with Google Cloud Storage.

You’ll typically use these to get started with the API:

>>> from gcloud import storage
>>> bucket = storage.get_bucket('bucket-id-here')
>>> # Then do other things...
>>> blob = bucket.get_blob('/remote/path/to/file.txt')
>>> print blob.download_as_string()
>>> blob.upload_from_string('New contents!')
>>> bucket.upload_file('/remote/path/storage.txt', '/local/path.txt')

The main concepts with this API are:

gcloud.storage.get_connection()[source]#

Shortcut method to establish a connection to Cloud Storage.

Use this if you are going to access several buckets with the same set of credentials:

>>> from gcloud import storage
>>> connection = storage.get_connection()
>>> bucket1 = storage.get_bucket('bucket1', connection=connection)
>>> bucket2 = storage.get_bucket('bucket2', connection=connection)
Return type:gcloud.storage.connection.Connection
Returns:A connection defined with the proper credentials.
gcloud.storage.get_default_connection()[source]#

Get default connection.

Return type:gcloud.storage.connection.Connection or NoneType
Returns:The default connection if one has been set.
gcloud.storage.get_default_bucket()[source]#

Get default bucket.

Return type:gcloud.storage.bucket.Bucket or NoneType
Returns:The default bucket if one has been set.
gcloud.storage.set_default_connection(connection=None)[source]#

Set default connection either explicitly or implicitly as fall-back.

Parameters:connection (gcloud.storage.connection.Connection) – A connection provided to be the default.
gcloud.storage.set_default_bucket(bucket=None)[source]#

Set default bucket either explicitly or implicitly as fall-back.

In implicit case, currently only supports enviroment variable but will support App Engine, Compute Engine and other environments in the future.

In the implicit case, relies on an implicit connection in addition to the implicit bucket name.

Local environment variable used is: - GCLOUD_BUCKET_NAME

Parameters:bucket (gcloud.storage.bucket.Bucket) – Optional. The bucket to use as default.
gcloud.storage.set_defaults(bucket=None, project=None, connection=None)[source]#

Set defaults either explicitly or implicitly as fall-back.

Uses the arguments to call the individual default methods.

Parameters:

Connections#

Create / interact with gcloud storage connections.

class gcloud.storage.connection.Connection(credentials=None, http=None)[source]#

Bases: gcloud.connection.JSONConnection

A connection to Google Cloud Storage via the JSON REST API.

API_BASE_URL = 'https://www.googleapis.com'#

The base of the API call URL.

API_URL_TEMPLATE = '{api_base_url}/storage/{api_version}{path}'#

A template for the URL of a particular API call.

API_VERSION = 'v1'#

The version of the API, used in building the API call’s URL.

gcloud.storage.connection.SCOPE = ('https://www.googleapis.com/auth/devstorage.full_control', 'https://www.googleapis.com/auth/devstorage.read_only', 'https://www.googleapis.com/auth/devstorage.read_write')#

The scopes required for authenticating as a Cloud Storage consumer.

Iterators#

Iterators for paging through API responses.

These iterators simplify the process of paging through API responses where the response is a list of results with a nextPageToken.

To make an iterator work, just override the get_items_from_response method so that given a response (containing a page of results) it parses those results into an iterable of the actual objects you want:

class MyIterator(Iterator):
  def get_items_from_response(self, response):
    items = response.get('items', [])
    for item in items:
      my_item = MyItemClass(other_arg=True)
      my_item._set_properties(item)
      yield my_item

You then can use this to get all the results from a resource:

>>> iterator = MyIterator(...)
>>> list(iterator)  # Convert to a list (consumes all values).

Or you can walk your way through items and call off the search early if you find what you’re looking for (resulting in possibly fewer requests):

>>> for item in MyIterator(...):
>>>   print item.name
>>>   if not item.is_valid:
>>>     break
class gcloud.storage.iterator.Iterator(connection, path, extra_params=None)[source]#

Bases: object

A generic class for iterating through Cloud Storage list responses.

Parameters:
PAGE_TOKEN = 'pageToken'#
RESERVED_PARAMS = frozenset(['pageToken'])#
get_items_from_response(response)[source]#

Factory method called while iterating. This should be overriden.

This method should be overridden by a subclass. It should accept the API response of a request for the next page of items, and return a list (or other iterable) of items.

Typically this method will construct a Bucket or a Blob from the page of results in the response.

Parameters:response (dict) – The response of asking for the next page of items.
Return type:iterable
Returns:Items that the iterator should yield.
get_next_page_response()[source]#

Requests the next page from the path provided.

Return type:dict
Returns:The parsed JSON response of the next page’s contents.
get_query_params()[source]#

Getter for query parameters for the next request.

Return type:dict
Returns:A dictionary of query parameters.
has_next_page()[source]#

Determines whether or not this iterator has more pages.

Return type:boolean
Returns:Whether the iterator has more pages or not.
reset()[source]#

Resets the iterator to the beginning.