In many analytical applications, the performance of queries in BigQuery are better than expected. Typically these queries scan a large number of rows, apply an analytical function and return the result to the calling program in a matter of seconds. However, even in an analytical application, you will often have a combination of analytical queries […]
Ingesting data from transactional systems to data warehouses or data lakes for analysis is one of the commonly used patterns. On Google Cloud, we have seen a variety of data pipelines to bring data from transactional systems/databases to Google Cloud Storage and/or BigQuery. Some of these patterns are Batch Pipeline where data is pushed from […]
Cloud Dataflow offers a unique feature called Dataflow Template to allow building customizable and reusable pipelines. Additionally, it is a much better way to segregate the development, test and production process of creating and running a data pipeline using Apache Beam. The documentation covers plenty of details about templates (classic and flex) as well as […]
While adopting cloud computing, one of the key asks especially within regulated industries (e.g. Finance, Healthcare, Government, Telco etc.) is the ability to have full control over the data. In my experience with Financial Services teams, full control over data usually translates to Controlling the location of data processing and data storage systems Controlling the […]
Cloud native data warehouse products have become quite popular in last few years primarily due to low administration overhead. Some of these products are closer to being an Infrastructure service i.e. developer needs to define CPU, Memory or Storage while others are platform service. In this space, Google’s BigQuery is a pure Serverless offering where […]
In this blogpost, I will share my learning about JSON processing capabilities of Oracle database. My dataset contains 1.5 million tweets collected over 2 days for a given set of tokens (more about the tokens later in this post). And IMO it represents a good size of real world dataset for experiments rather than having […]
Oracle Cloud Database Service offers an excellent option for databases requiring extremely high disk IO performance using locally attached NVMe storage in HighIO and DenseIO compute shapes. However, there has been some discussion / doubts around using NVMe storage for databases. The cause of concern was persistence of data on NVMe devices across reboot of […]
Oracle SQLcl (Command Line interface) offers many developer friendly features. The detail about this utility and a complete list of features can be found at it’s documentation site . Among these, I find ‘alias’ and ‘repeat’ particularly useful as it allows me to easily build adhoc monitoring utilities that we so often need. And I […]
Oracle Public Cloud offers Storage services in the form of Block Storage, Object Storage and Archive Storage. Block Storage is primarily used by Virtual Machines, Database, Java and other services that require reading from and writing to filesystems. Block Storage optimizes storage for IOPS and offers POSIX compliant file system. Object Storage storage data in a flat […]
In this blog post, I will be discussing backing up Oracle Database to storage in Oracle Public Cloud mainly around why you will consider such an option and how to setup and run some simple backup and restore operation. Typically an Oracle database backup using Recovery Manager (RMAN) involves one of the following: Backup to disk […]
Partly Cloudy 