Hash Partitioning - DataStage скачать в хорошем качестве

Hash Partitioning - DataStage

In hash partitioning method, Input records are grouped based on certain fields and the groups are randomly distributed across the processing nodes (or partitions). The fields which are used to partition the data are called as hash key fields. The hash partitioner will make sure that every record belonging to a certain hash key field values are available in the same processing node (or partition). This method of partitioning is particularly useful when we use remove duplicate stage, sort stage, or aggregator stage in DataStage jobs. The hash partitioning requires at least one column to be define as has key (primary key field). Also, it can have multiple secondary key columns. In the diagram, the column City in the input dataset has been chosen as the hash partition key. As you can see, the records with City values “Chennai” and “Pune” are sent to first partition (processing node 1) and the records with City values “Mumbai” and “Kolkata” are sent to the second partition (or processing node 2). You may observe that, Node 1 has a smaller number of records compared to that of node 2. Hence, Node 2 processor will have to do a lot more processing work than that of node 1. This is the disadvantage of hash partitioning method: - It creates uneven sized partitions.