Data transformation and movement is the process by which source data is selected, converted, and mapped to the format required by targeted systems. The process manipulates data to bring it into compliance with business, domain, and integrity rules and with other data in the target environment. Transformation can take some of the following forms:
AggregationConsolidating or summarizing data values into a single value. Collecting daily sales data to be aggregated to the weekly level is a common example of aggregation.
Basic conversionEnsuring that data types are correctly converted and mapped from source to target columns.
CleansingResolving inconsistencies and fixing the anomalies in source data.
DerivationTransforming data from multiple sources by using a complex business rule or algorithm.
EnrichmentCombining data from internal or external sources to provide additional meaning to the data.
NormalizingReducing the amount of redundant and potentially duplicated data.CombiningThe process of combining data from multiple sources via parallel Lookup, Join, or Merge operations.
PivotingConverting records in an input stream to many records in the appropriate table in the data warehouse or data mart.
SortingGrouping related records and sequencing data based on data or string values.