In September 1998 Coglin Mill spent two weeks in the IBM Teraplex Integration Center in Rochester Minnesota testing performance improvements in the latest series of Northstar processors running under OS/400 V4R3 and using RODIN V2R2, all the very latest hardware, operating system, database and data warehouse technology from IBM and Coglin Mill.

The results were amazing. 

Using an 8 way join (under program control) to select, extract, validate, transform and load a data warehouse table with 200 million rows, RODIN was able to complete the load in just over 3 hours on a new Northstar 12 way processor, achieving a rate of 65 million rows per hour.  

It is important to note this was not just a simple SQL copy / load with effectively no processing logic or database activity.  Each of the 200 million source rows accessed 7 other reference tables to execute the very sophisticated logic used to process and load the data.

Each row consisted of 50 columns (sourced from all the 8 input tables and / or derived from logic) and resulted in a total output record length of 500 bytes.  The target table therefore was 100GB giving a load rate of approximately 33GB per hour on a single AS/400.

Given these load rates and given the new maximum disk storage capabilities of these machines (2TB officially but 17.5TB unofficially) it is highly unlikely that any customer, no matter how large, will need to cluster a number of AS/400s together in an MPP configuration to address their scaling requirements. 

However it is nice to know that if they do need to implement across multiple AS/400s, up to 32 x 12 way machines can be installed in a single configuration giving up to 384 high performance Northstar processors, 1.28TB of memory and 67TB (560TB unofficially) of on-line disk storage. 

Imagine loading your data warehouse at better than 2 billion rows or 1TB per hour !!!  Try doing that on your Teradata or on Oracle.