Runtime Architecture

Reading Time: 3 minutes

Features of Data Stage

  1. Any to Any
  2. Platform Independent
  3. Node Configuration
  4. Portion Parallelism
  5. Pipeline Parallelism

Any to Any

Reads the data from any Source and loads it to any Target.

Any SRC    ↔   Any Target  

Platform Independent

Designed for one O.S, can be executed

  – – >Platform generally can be either Software or Hardware.

Platform Independent
  • – > In the Data stage, Platform is w. r. t Hardware.

Hardware environment 

  • Uni processing Environment

  Hard disk à CPU – – > RAM  

  • Symmetric Multi-Processing: – (SMP)
Hard Disk

can have 32–64 CPU that is Hard disk with multiple CPU‘S

  Massively Parallel processing:-  (MPP)

MPP
  • Collection of different SMPS

  Node Configuration

  • The best feature of the Data stage
  • It is a technique of creating logical CPUs

  Node – – > logical CPU (or) instance of (physical) CPU

àIt is an S/W which will Create virtual CPU’S

  • Data Stage is Executed on logical CPU’S
  • TO run a job in the Data stage, WE require at least 1 Node.

EX:- ETL 

UNI Process

Hard disk – – > CPU – – > RAM

  • To access 1000 records, it takes 10 mins.

SMP SMP

  • To access 1000 records, with 4 CPU’S it takes 2.5 min

Node config:

  • Uni Processing – – > Virtual SMF
Node Configuration

S is not using the max. capabilities of CPU, So Node config. is an S/W Which drives into different Nodes. That is Boost up the Capabilities & Energy level of CPU  

Partition parallelism

– – > Horizontal Combining

  – – > Combining primary rows with Secondary rows w. r. t  Key column values

Partition Parallelism

Pipeline Parallelism

Simultaneously doing the extraction of Transforming and loading jobs.

Leave a comment