Sep 12

Written by: host
9/12/2010 12:00 AM 

More and more applications will be dismantled into services. Services that only require part of the data to function properly. Thus the databases linked with the applications will also tend to become smaller. But where does the data stay?

Next to that we see that companies operate in an environment in which they communicate more and more near real-time with many parties. Partners, suppliers, customers, government etcetera. This triggers projects to set up bus architectures to exchange data based on a publish and subscribe mechanism.

And of course companies continue to set up a BI architecture to support decision making processes.

A good example of this is the way municipalities are addressing this. We conducted a survey in the south of the Netherlands and discovered that all big municipalities are setting up a bus architecture to enable near real-time information exchange with partners (police, youth care, fire department, tax departments, central government, property etcetera). The data model to support this is called RSGB. This data model has been defined by central government.
Next to that all these municipalities are setting up separate datawarehouse environments collecting the same data again for reporting purposes. In most cases there are even two different project teams that hardly communicate with each other.

In our view these projects are two of a kind and should not be treated separately. We suggest to set up an integration platform or information hub (on purpose I avoid the term data warehouse for it is contaminated and is easily understood as only serving reporting requirements).
This integration platform distinguishes different layers of which the two most important ones are the source layer and the business layer.

Source layer
This layer collects the data from the source systems and adds attributes thus enabling to build up history. At this level only semantic integration can be available when business keys in different sources are the same. What is very important is that the data is not altered yet to support traceability. An important requirement in the light of new legislation (Basel II, Solvency II, USGAAP, IFRS etc).

This source layer contains:
1. A minimum data model required for information exchange purposes with all partners. We call this the communication model. An example could be the RSGB data model as described above;
2. An extended data model containing the other attributes to support operational, tactical and strategical (management) reporting.

Business layer
Part of the data in the source layer needs modification/alteration to meet certain information exchange and reporting requirements. Therefore (part of) the data is transformed and stored in the business layer. Since data marts are derived typically from the source and business layer, having precalculated data in the business layer helps in preventing having to calculate the same data again and again.
The distinction between the source and business layer is purely a logical one for they both could perfectly reside in one technical database schema.

Tags: