Data transformation and movement is the process by which source data is selected, converted, and mapped to the format required by targeted systems. The process manipulates data to bring it into compliance with business, domain, and integrity rules and with other data in the target environment. Transformation can take some of the following forms:
Consolidating or summarizing data values into a single value. Collecting daily sales data to be aggregated to the weekly level is a common example of aggregation.
Ensuring that data types are correctly converted and mapped from source to target columns.
Resolving inconsistencies and fixing the anomalies in source data.
Transforming data from multiple sources by using a complex business rule or algorithm.
Combining data from internal or external sources to provide additional meaning to the data.
Reducing the amount of redundant and potentially duplicated data.
The process of combining data from multiple sources via parallel Lookup, Join, or Merge operations.
Converting records in an input stream to many records in the appropriate table in the data warehouse or data mart.
Grouping related records and sequencing data based on data or string values.