Archives of the TeradataForum
Message Posted: Sat, 22 Feb 2003 @ 17:05:35 GMT
| Subj: | | Re: Can it be any faster |
| |
| From: | | rmhsmd |
Consider Teradata, the LAN, the Fastload client platform CPU, and the source speed. Any of these may be the effective bottleneck
for the acquisition phase. It is often possible to make it faster--the approach depends on what the bottleneck is.
Teradata performance is relatively well documented in the Capacity Planning Reports. Depending on the hardware Teradata has been
measured from 5-8 MB/Second per node for 100 and 500 byte records for the acquisition phase.
fwiw, in the FDL acquisition phase each session is a direct connection to a different AMP (PEs uninvolved). The client FDL
process builds and sends 64K buffers to these--thus this is balanced across the AMPs. Each amp computes each row's hash and does
buffered redistribution to its target amp. Teradata and the client connection are scalable. pre-Teradata Warehouse Builder Fastload
client is a single process that reads a single input using asynch non-blocking I/O on the Teradata sessions. Additional client SMP
exploitation is possible via enabling 'parallel CLI' (the TCP/IP send/receive is done in separate threads that interchange info with
Fastload via shared memory). TWB can scale the processing on an SMP client and read multiple input streams as well.
| 
