Comparison of time to append rows, flat file vs. Datablitz

The graph below compares the time it took to insert rows in two different ways: The data were gathered from several runs made at ATL which inserted 10, 1,000, and 10,000 rows. The rows did not have indices. The code used to do the row additions was excerpted from the Warrior system and modified for the purposes of the test. The test platform was a Sun Ultra 1 with 64 Mb of memory.



In general DataBlitz is significantly faster, inserting rows in about 40% of the time required for the flat file. The DataBlitz inserts were done using the relational API, the slowest of the three DataBlitz APIs. The data do not factor out processing unrelated to the row insertion. Since the insertions were done using an excerpt from a complex application, the time is probably significant, and is larger proportionately for the DataBlitz runs. In particular, there was a fixed initialization overhead. This is evident in the raw numerical data for DataBlitz, since the insertion time per row drops as the number of rows inserted goes up (modulo the anomalies noted in the following paragraph).

In three cases, DataBlitz is slower or comparable to the flat file strategy. A possible explanation for this is that the default table preallocation was exceeded. When this happens, a larger area is allocated, and the preexisting data are copied into the new area. This additional overhead made the run considerably slower. This would not, however, occur in an operational system in which the tables were preallocated at an adequate size.

Raw Data

The graph below is a log/log plot of the raw data, how long it took, in milliseconds, to do 10, 1,000, and 10,000 row inserts in several runs for the two different ways.



The raw numerical data appear below.

Number flat file DataBlitz flat file DataBlitz
of inserts time [ms] time [ms] ms/insert ms/insert
100 307 352 3.07 3.52
1000 3002 1736 3.00 1.74
10000 29633 10431 2.96 1.04
100 429 95 4.29 0.95
1000 2949 4521 2.95 4.52
10000 30015 14849 3.00 1.48
100 288 176 2,88 1.76
1000 2869 1168 2.87 1.17
10000 29820 37286 2.98 3.73
100 292 91 2.92 0.91
1000 3267 822 3.27 0.82
10000 30307 8481 3.03 0.85