GoldenGate Transactional Data Management 9.0

Organisations have so much data today that they are struggling to make the best use of it. Data warehouses, business intelligence (BI), data mining, data consolidation - these are complex, time consuming and resource hungry tasks. The tools available are rarely off the shelf and until 2000, anyone braving this market would expect to pay substantial sums of money for software tools.

When Microsoft released SQL Server 2000 with support for data warehouses and put BI at the heart of its data story, the market was rocked. Other database vendors, Oracle, Sybase and IBM soon followed suite and released more accessible tools at much lower prices. It seemed that the days of high cost tools were over.

The reality soon began to hit home. Manipulating gigabytes (GB), terrabytes (TB) even petabyes (PB) of data requires tools that are capable of doing complex data manipulation at very high speed. While the database vendors have continued to attack this market, the data tools vendors have continued to dominate the market.

GoldenGate is one such tool vendor whose Transactional Data Management v9 (TDM9) was recently released.

TDM9 is sold as part of a package which includes consultancy. Customers are provided with documentation and software on CDs but this is not something you can just buy off the shelf. There are several deployment models for TDM9 and the price you pay depends on the model that you adopt.

Installation of TDM9 is simple. Unzip the software into its directory, copy in the licence file and run the GoldenGate Command Line Interface (GGCLI) to configure and setup the software. For those who do not want to work with the CLI, there is a GUI tool - GoldenGate Director - that allows you to manage your deployments.

Director is a Java based tool and you will need to download the latest Java Virtual Machine from the Sun website in order to run Director. In order to use Director you need to login to it and users are assigned roles. Unfortunately, Director does not integrate to any Director Service or Domain logon database, instead using its own separate login and password database.

Before installing GoldenGate you need to think through what you want to do with the data and spend time planning your architecture. The first run through of any GoldenGate solution has to run against the source database, after that it uses the transaction log. This significantly reduces the load on the database server.

In a nutshell, GoldenGate takes data from the transaction log, transforms it and sends it to a data pump. That pump transmits the data to a receiving data pump which then passes the data to a transformation engine. Further operations are carried out on the data, after which it is inserted into the target database.

It sounds simple and in essence it is. The complexity that often occurs depends on what you want to do with the data and which of the various GoldenGate models you choose to use.

Only when you look at the process in detail, do you get any real idea of the complexity of the software and what it is doing.

Capture: This is the first step. It takes the transaction log and selects the data you want. Only data that has been committed and is therefore a completed transaction will be used. You can then do a series of operations on that data to ensure that you are only selecting the data you really need rather than send the entire contents of the transaction log.