Choosing Hardware (CPU, HDD) for Vertica nodes

I'm not a hardware expert so please forgive me if I make false assumptions :) Vertica's documentation recommends min 20 MB/s IO throughput (reads and writes) per physical core in full duplex. That means if I have a machine with 12 physical cores and 12 hyperthreaded cores I would need to reach 240 Mb/s read and write simultaneously. A typical 15K RPM drive would do ~150 MB/s sequential read/write. Obviously it seems I need a RAID of some sort. This would mean that I need at least two drives to reach 300 MB/s capacity. If I understand correctly this then needs to be multiplied in order to be able to read AND write 300 MB/s simultaneously. So 4 drives would be needed to reach a total capacity of 600 MB/s. And possibly to use RAID10. Is my thinking correct here?


  • Žilvinas_ŠaltysŽilvinas_Šaltys Registered User
    Also what does the recommended minimum actually mean? Why is there a recommended minimum?
  • Hi Žilvinas, The minimum on disk IO isn't a hard minimum -- what it means is, if you have less than that, you probably spent too much money on CPU cores; your machine will be spending most of its time waiting on the hard disk. (This is, of course, workload-dependent; but I've found it to hold true for typical workloads.) Regarding your math -- Yes, that would be 240 Mb/s single duplex; but that's not 600Mb/s full duplex, 240 * 2 = 480Mb/s. Which is hopefully more attainable. To get that performance, there's no particular need to use 15k RPM drives. The main advantage of those drives is that they have significantly better seek performance. However, Vertica (unlike many other databases) mostly does large sequential reads. So you can get similar performance out of a slightly larger number of slower (and much cheaper) drives. (Alternatively, you could use SSDs. Some of our customers do, and are quite happy with them. A single good SSD will give you 480Mb/s. But there are other issues with SSDs; not the least of which that a single drive is not very large.) Alternatively, you could buy a less-fancy CPU. Recall, Vertica is a cluster database. You could start with a basic server with dual quad-core processors, but buy a bunch more of them, and achieve the same performance that way. (Or even more even slower machines than that -- <>) Adam
  • brian_Doughertybrian_Dougherty Registered User

    I had a similar question. we are currently running a 4 node DL380p cluster with each node containing 2 sockets @ 8 cores or 16 total cores.  16 cores @ 20Mbps = 320 Mb/s. however with the p420i controller and 22 drives in the RAID set we measure with vioperf over 1Gbps. 

    From all the testing we have performed I am wondering whether a better price/perf config would be to move from (4) 2u DL380p to (8) two-socket 1u DL360p - the net cost is the same and the CPU count doubles. The only issue is that the DL360p will probably only deliver 300-350 Mb/s for each node.

    However, the data per node should be half and therefore the io throughput requirement less.

    any thoughts would be appreciated


  • Hi Brian,

    I haven't personally tested those particular configurations, so I can't say for sure...  I would guess that the 8-node configuration could be somewhat faster.  But, as always, it depends on the specific specs of the machines and on your query workload.  Whenever someone totally makes up an answer without having ever used the hardware in question, take it with a large grain of salt :-)

    Hopefully someone else here has some experience that they can share.

    One thing to consider -- as you add nodes, you tend to stress the network more.  So make sure you're not sacrificing any network performance in order to add the extra capacity.  With 8 nodes, you can usually still have them all on the same network switch, which is preferred.

  • brian_Doughertybrian_Dougherty Registered User
    Thanks Adam. I agree with your caution.  we will test this config differential in our labs.

    I understand about the network. we are taking a look at 1Gb/s versus 10Gb/s network differentials now across different workloads. We package and sell pre-built vertica clusters so we are always evaluating different price/performance configuration points.

    I pose the possibility because from our testing, running 15-20 concurrent queries against 1-10 billion row tables using the DL380P as configured, it appears we are leaving some excess i/o capability at the table. In other words, we have rarely driven a sustainable 1Gb/s i/o throughput on the node. The most significant i/o load has been on what appears to be sort spills (writing). The vertica i/o has been very judicious especially relative to other databases we test and sell (oracle and greenplum).

    thanks again brian

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file