Failed to create a vertica cluster with 8+ nodes
I am creating a Vertica cluster on OpenStack (Kilo) through Trove. It is possible to create a 2-node, 4-node or 6-node Vertica cluster but it is failed to create 8-node Vertica cluster (formed by 8 Nova instances).
I have 4 physical server (1 controller node, 3 compute nodes), each of them has 8 physical CPU and 128 RAM. For creating Vertica cluster, the flavor includes 2 VCPUs, 4 GB RAM, 15 GB root disk, 2 GB swap disk.
As OpenStack allows CPU over-committing 16:1 and RAM over-committing 1.5:1, therefore 512 vcpu and 768 GB memory could be used in the environment. (At most 8 vcpu is assigned to each instance).
After creating the Vertica cluster, I found that the status of Trove instances of the cluster is ERROR. Then, I use /opt/vertica/bin/admintools to view the status of the database cluster state which is UP. I login to the database, and try the database with '\d' or '\dn' but it stuck with no any responds.
I cannot find any hints in vertica.log, install.log and adminTools-dbadmin.log. No content can be found in ErrorReport.txt among all 8 nodes of Vertica cluster. Then, I try to use Vertica analytics tools like vcpuperf, vioperf and vnetperf. Only 1 node of the 8-node Vertica cluster can run vnetperf, other 7 nodes stuck with the command as follows:
The results of Vertica analytic tools (as mentioned only 1 node of the 8-node Vertica cluster can run vnetperf)
CPU Time: 8.880000s
This machine's high load time: 86 microseconds.
This machine's low load time: 135 microseconds.
2017-07-23 17:45:51.201 1110 ERROR trove.guestagent.datastore.experimental.vertica.service [-] Failed to get database status.
2017-07-23 17:45:51.201 1110 TRACE trove.guestagent.datastore.experimental.vertica.service Stdout: u'\nERROR: /opt/vertica/config/admintools.conf does not exist.\nThis file is created by the installer upon successful completion.\nYou must successfully run /opt/vertica/sbin/install_vertica before running admintools.\n\n'
Can anyone give suggestions on the above conditions?