Datastage data Architecture

Reading Time: 4 minutes

In general, we have 4  Architecture

• Simple
• ODS  – Operations Data source
• NDS – Normal  Data source
• Integrity  

 we can implement this Architecture either in Inmon Model or Kimball Model

Top to bottom                                Bottom to top

Universally  –   8 Architecture

Inmon – Simple, ODS,  NDS, Integrity

Kimbell – Simple, ODS,  NDS, Integrity.

Simple

SRC —- WH

ODS

• Inmon Models are preferred when Requirements are very clear.
• Kim bell Model is preferred when   Requirements are not clear.  
• For the project, we deal with

1. ODS –à80% ODS Model
2. NDS

WHY ODS?

Sol:- when  Business Requirements are defined, click/customer requests the team  what we want exactly

.     

Project Architecture depends on no. of operational and Analysis Reports.

• More priority is given to operational and Analysis Reports
• Are we have more operational Reports we use ODS.
• Less operational Reports and more Analytical Reports, we use NDS.
• In NDS, we can’t See ODS, But in ODS, we can find warehouse and Data marts (NDS).
• ODS gets refreshed every 10 seconds.
• ODS dim Table will always be type 1 (No History)

Architecture

Data Acquisition 

Pull   You get  (take) the data

Push he sends the data

DRA:-  Data Repository Area

CSA:- Common Staging Ares

DRA:- Will get only files  –    Memory which is accessed by different departments.

Data Query:-   When the  data is being such  what the data should contain

Quality Gates:-  Requirements .

Every file should have a specific file name:-

Script 

To validate the files according to Quality  Gates.

Loading

Loading, Stage, and warehouse is a Single Data stage.

LoadingStaging  ODS  Warehousing  Box size of 100 Tear loading  

10 TB  loading

30 TB Staging

60 TB Warehouse.  

Dumping the data from DRA to loading à a copy of data

When we load Data to Target data, We maintain all details, that is sequence no;