Oracle Public Cloud offers Storage services in the form of Block Storage, Object Storage and Archive Storage. Block Storage is primarily used by Virtual Machines, Database, Java and other services that require reading from and writing to filesystems. Block Storage optimizes storage for IOPS and offers POSIX compliant file system. Object Storage storage data in a flat hierarchy of containers. Each object contains an unique ID, metadata and data. There are many ways of accessing Object storage in Oracle Cloud. Some of these are listed below:
- REST API
- Java Library
- File Transfer Manager (FTM) API and CLI
- Storage Cloud Software Distribution
- Web Console and Third-party software.
- Amazon S3 API Compatible Clients
In spite of large number of methods and tools available to interact with Object Storage in Oracle Cloud, I managed to find a need to write my own Python based API 🙂 My initial requirement was to have an ability to bulk delete some old backup files in a container. I could do it using cURL, but that was too cumbersome as it involves getting an authentication token, remember the URL, generate a file with name of objects to delete and then make the API call ! And hence the need and justification to write a python based API wrapping REST call using Requests module. Once I had the bulk delete functionality working, I could see the benefit of having other functions included in this API set as it seamlessly integrates with any Python script. Note that this is NOT Oracle official Python SDK. Check out Oracle Documentation for Storage Cloud Service Java SDK, CLI and RESTful API.
It is developed using Python 2.7.13 on Mac OS X El Capitan (10.11.6) and tested on Mac OS X El Capitan and Oracle Linux 7.2. It should work on other Operating Systems with Python 2.7 and dependent packages installed.
Brief description of this API / utility is below. The code and a detailed README is available from my github repository.
Currently it supports the following operations:
- LISTING of containers and objects
- CREATE new containers
- UPLOAD new objects / UPDATE existing objects
- DELETE a single object
- DELETE all objects in a container (BULK DELETE)
- DOWNLOAD an objects to local machine
Also worth noting the following limitations and workarounds:
- DELETE operationDoes not include – “multipart-manifest=delete” option
Impact – For static large objects, only manifest file is deleted but not the segment files.
Workaround – Pass segment object name in a separate call to opc-storage tool Or use Java based CLI
- UPLOAD operationUploading multiple objects from an archive is not supported.
Doesn’t support objects larger than 5GB.
- DOWNLOAD operationDownload of multipart object is not supported
If you are building a project using Python to interact with Oracle Cloud, you may find it quite handy. You can get the code and detailed README from opc-storage github repository. Feel free to download it, modify it as per your need and contribute if you build a new functionality on top of this existing codebase. Thank you for reading !