Step: "Waiting for 1 of 3 site" Taking too much time.

Is it normal the step "Waiting for 1 of 3 sites" stay blocked? I can't pass this step. What it trying to do? My Last log msg is: May 20 00:43:05 root@15.185.115.21: ['(echo \'#!/usr/local/bin/python\' && echo \'\' && echo \'import socket\' && echo \'\' && echo \'import time\' && echo \'\' && echo \'attempt=0\' && echo \'done=False\' && echo \'while not done and attempt < 5:\' && echo \' attempt = attempt+1\' && echo \' try:\' && echo \' s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)\' && echo \' s.connect(("15.185.115.21", 48656))\' && echo \' s.send("ECHO FROM 15.185.115.21")\' && echo \' starttime = time.time()\' && echo \' bytes_to_receive = 1024*10240\' && echo \' while(bytes_to_receive > 0):\' && echo \' data = s.recv(bytes_to_receive)\' && echo \' bytes_to_receive = bytes_to_receive - len(data)\' && echo \' endtime = time.time()\' && echo \' s.close()\' && echo \' sec = endtime - starttime\' && echo \' print 1024*10240, "bytes received from 15.185.115.21 in", sec, "secs, throughput", 1024*10240*8/(1024*1024*sec), "Mbps"\' && echo \' done = True\' && echo \' except socket.error, (errno, errmsg):\' && echo \' print "(Attempt #%d) Host 15.185.115.21 could not send a message to 15.185.115.21: " %(attempt) + str(errmsg)\' && echo \' time.sleep(1)\' && echo \' except Exception, mye:\' && echo \' print "(Attempt #%d) Host 15.185.115.21 could not send a message to 15.185.115.21: " %(attempt) + str(mye)\' && echo \' except ValueError, vye:\' && echo \' print "(Attempt #%d) Host 15.185.115.21 could not send a message to 15.185.115.21: " %(attempt) + str(vye)\' && echo \'if not done:\' && echo \' print "Host 15.185.115.21 could not send a message to 15.185.115.21: " + str(errmsg)\' && echo \'\' && echo \'attempt=0\' && echo \'done=False\' && echo \'while not done and attempt < 5:\' && echo \' attempt = attempt+1\' && echo \' try:\' && echo \' s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)\' && echo \' s.connect(("15.185.113.21", 48656))\' && echo \' s.send("ECHO FROM 15.185.115.21")\' && echo \' starttime = time.time()\' && echo \' bytes_to_receive = 1024*10240\' && echo \' while(bytes_to_receive > 0):\' && echo \' data = s.recv(bytes_to_receive)\' && echo \' bytes_to_receive = bytes_to_receive - len(data)\' && echo \' endtime = time.time()\' && echo \' s.close()\' && echo \' sec = endtime - starttime\' && echo \' print 1024*10240, "bytes received from 15.185.113.21 in", sec, "secs, throughput", 1024*10240*8/(1024*1024*sec), "Mbps"\' && echo \' done = True\' && echo \' except socket.error, (errno, errmsg):\' && echo \' print "(Attempt #%d) Host 15.185.115.21 could not send a message to 15.185.113.21: " %(attempt) + str(errmsg)\' && echo \' time.sleep(1)\' && echo \' except Exception, mye:\' && echo \' print "(Attempt #%d) Host 15.185.115.21 could not send a message to 15.185.113.21: " %(attempt) + str(mye)\' && echo \' except ValueError, vye:\' && echo \' print "(Attempt #%d) Host 15.185.115.21 could not send a message to 15.185.113.21: " %(attempt) + str(vye)\' && echo \'if not done:\' && echo \' print "Host 15.185.115.21 could not send a message to 15.185.113.21: " + str(errmsg)\' && echo \'\' && echo \'attempt=0\' && echo \'done=False\' && echo \'while not done and attempt < 5:\' && echo \' attempt = attempt+1\' && echo \' try:\' && echo \' s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)\' && echo \' s.connect(("15.185.113.120", 48656))\' && echo \' s.send("ECHO FROM 15.185.115.21")\' && echo \' starttime = time.time()\' && echo \' bytes_to_receive = 1024*10240\' && echo \' while(bytes_to_receive > 0):\' && echo \' data = s.recv(bytes_to_receive)\' && echo \' bytes_to_receive = bytes_to_receive - len(data)\' && echo \' endtime = time.time()\' && echo \' s.close()\' && echo \' sec = endtime - starttime\' && echo \' print 1024*10240, "bytes received from 15.185.113.120 in", sec, "secs, throughput", 1024*10240*8/(1024*1024*sec), "Mbps"\' && echo \' done = True\' && echo \' except socket.error, (errno, errmsg):\' && echo \' print "(Attempt #%d) Host 15.185.115.21 could not send a message to 15.185.113.120: " %(attempt) + str(errmsg)\' && echo \' time.sleep(1)\' && echo \' except Exception, mye:\' && echo \' print "(Attempt #%d) Host 15.185.115.21 could not send a message to 15.185.113.120: " %(attempt) + str(mye)\' && echo \' except ValueError, vye:\' && echo \' print "(Attempt #%d) Host 15.185.115.21 could not send a message to 15.185.113.120: " %(attempt) + str(vye)\' && echo \'if not done:\' && echo \' print "Host 15.185.115.21 could not send a message to 15.185.113.120: " + str(errmsg)\') | python ']

Comments

  • What are you trying to do? Install vertica? What version of Vertica? If you are trying to install vertica maybe the ports are not open? Please send details of what are you doing. Thanks, Eugenia
  • Yes, I'm trying to install vertica cluster at HP Cloud with 3 nodes, follow details: vertica-ce-6.0.1-0.x86_64.RHEL5.rpm CentOS 5.8 Server 64-bit 20120828 (b) 2 vCPU / 4 GB RAM / 120 GB HD Security group: tcp 22 22 IPs 0.0.0.0/0 tcp 4033 4033 IPs 0.0.0.0/0 udp 4033 4033 IPs 0.0.0.0/0 tcp 5433 5433 IPs 0.0.0.0/0 tcp 5444 5444 IPs 0.0.0.0/0 tcp 5450 5450 IPs 0.0.0.0/0 tcp 4803 4803 IPs 0.0.0.0/0 udp 4803 4803 IPs 0.0.0.0/0 udp 4804 4804 IPs 0.0.0.0/0 udp 4805 4805 IPs 0.0.0.0/0 Thanks
  • [root@host01 sbin]# ./install_vertica -s host01,host02,host03 -r /root/vertica-ce-6.0.1-0.x86_64.RHEL5.rpm -u dbadmin Vertica Analytic Database 6.0.1-0 Installation Tool Upgrading admintools meta data format.. scanning /opt/vertica/config/users Starting installation tasks... Getting system information for cluster (this may take a while).... backing up admintools.conf on 15.185.115.21 Installing rpm on 2 hosts.... installing node.... 15.185.113.21 installing node.... 15.185.113.120 NTP service is not running on the hosts: ['15.185.113.21', '15.185.113.120', '15.185.115.21'] Vertica recommends that you keep the system clock synchronized using NTP or some other time synchronization mechanism to keep all hosts synchronized. Time variances can cause (inconsistent) query results when using Date/Time Functions. For instructions, see: * http://kbase.redhat.com/faq/FAQ_43_755.shtm * http://kbase.redhat.com/faq/FAQ_43_2790.shtm Info: the package 'pstack' is useful during troubleshooting. Vertica recommends this package is installed. Info: the package 'sysstat' is useful during troubleshooting. Vertica recommends this package is installed. Checking/fixing OS parameters..... Setting vm.min_free_kbytes to 4096 ... Info! The maximum number of open file descriptors is less than 65536 Setting open filehandle limit to 65536 ... Info! The session setting of pam_limits.so is not set in /etc/pam.d/su Setting session of pam_limits.so in /etc/pam.d/su ... Info! Parameter vm.max_map_count is less than 253184 Setting vm.max_map_count to 253184 ... Info! Parameter vm.max_map_count is less than 253184 Setting vm.max_map_count to 253184 ... Info! Parameter vm.max_map_count is less than 253184 Setting vm.max_map_count to 253184 ... Creating/Checking Vertica DBA group Creating/Checking Vertica DBA user Password for dbadmin: Installing/Repairing SSH keys for dbadmin Password:Password: Creating Vertica Data Directory... Testing N-way network test. (this may take a while) All hosts are available ... Verifying system requirements on cluster. IP configuration ... IP configuration ... IP configuration ... Testing hosts (2 of 3).... Running Consistency Tests LANG and TZ environment variables ... Running Network Connectivity and Throughput Tests... Waiting for 1 of 3 sites...
  • After about 1h the process stops and the output below was shown: Waiting for 1 of 3 sites... ... Test of host 15.185.115.21 (FAILED) ======================================= IP configuration (FAILED) ----------------------------- Cannot determine the IP address of the host See the Vertica Installation Guide for more information. Test of host 15.185.113.21 (FAILED) ======================================= IP configuration (FAILED) ----------------------------- Cannot determine the IP address of the host See the Vertica Installation Guide for more information. Test of host 15.185.113.120 (FAILED) ======================================== IP configuration (FAILED) ----------------------------- Cannot determine the IP address of the host See the Vertica Installation Guide for more information. Consistency Test (FAILED) ============================= IP and Netmask consistency (FAILED) --------------------------------------- Hosts 15.185.115.21 and 15.185.113.21 have different subnet masks [] and [] See the Vertica Installation Guide for more information. All hosts are on the same subnet (FAILED) --------------------------------------------- Could not determine the broadcast address of host 15.185.115.21 See the Vertica Installation Guide for more information. Warning: The $LANG (en_US.utf8)environment variable is not UTF-8 on 15.185.115.21 You may have issues creating databases if this is not fixed. Info: The $TZ environment variable is not set on 15.185.115.21 Warning: The $LANG (en_US.utf8)environment variable is not UTF-8 on 15.185.113.21 You may have issues creating databases if this is not fixed. Info: The $TZ environment variable is not set on 15.185.113.21 Warning: The $LANG (en_US.utf8)environment variable is not UTF-8 on 15.185.113.120 You may have issues creating databases if this is not fixed. Info: The $TZ environment variable is not set on 15.185.113.120 Network Test (FAILED) ========================= Network communication (FAILED) ---------------------------------- Low throughput 15.185.115.21 to 15.185.115.21: 94.7833716277 Mbps; check net Low throughput 15.185.113.21 to 15.185.113.21: 94.796385613 Mbps; check netw Low throughput 15.185.113.120 to 15.185.113.120: 94.6271885111 Mbps; check n Echo server failed on host 15.185.115.21: ['', ''] Echo server failed on host 15.185.115.21: ['', ''] Echo server failed on host 15.185.113.21: ['', ''] Echo server failed on host 15.185.113.120: ['', ''] Failed to send/recieve messages from all hosts, Check your netverify log for details and make sure your firewall and SELinux are disabled for the cluster interfaces. Verification failed. Correct the above issues to proceed Installation completed with warnings. Installation completed with errors. Installation failed. =============================================== It can be a subnet configuration issue? Thanks
  • Yes, it could be network configuration. You can add to option -N -T to the install_vertica. -N will indicate that host are on different subnets and -T will configure spread to be point to point. Hope that helps. Let me know, Eugenia
  • Hi, I was getting "Cannot determine the IP address of the host " error because dbadmin user didn't have permissions to run /sbin/ifconfig.

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file