Passwordless ssh issue

carlbeech

Hi Forum I've got an issue with a test installation of community edition ce-6.0.1-0.x86_64 RHEL5, on a CENTOS 6.4 set of VM's. I wanted to set up from scratch a cluster to improve knowledge... I created initially two VM's - with static IPs, 'host1' and 'host2'. SSH is set up on both. I've done the normal rpm -Uvh vertica-ce...rpm, and then run the install command: /opt/vertica/sbin/install_vertica -s host1,host2 -r vertica-ce-6.0.1-0.x86_64.RHEL5.rpm The installation crashes out with the message: Passwordless SSH access to other hosts (FAILED) Cannot connect from 192.168..... to host 192.168... without password.. Verify that /etc/ssh/sshd_config is configured properly on 192.168.... In sshd_config I've got: RSAAuthentication yes PubkeyAuthentication yes AuthorizedKeysFile .ssh/authorized_keys2 I've checked on both VMs, and found that the .ssh and files contained at all mode 777. Thinking that there's a problem with my ssh config, I've not changed the config, but cleared out the contents of .ssh directory - used ssh-keygen, and ssh-copy-id dbadmin@. I've chmod'd the directory to 700 and files to 600, and I can do passwordless logins from one machine to the other... This makes me think that the sshd-config appears ok? Thinking that it's ok, I've re-run the /opt/vertica/sbin/install_vertica script again, and it's moved my current (working) ssh directory out of the way and re-created a new (non-working) .ssh directory and crashed out with the same error message... Can anyone think of what the issue may be, or whether the installer has an option for 'don't touch the ssh config'? :-) Many thanks in advance... :-) Carl.

[Deleted User]

Hi Carl, thanks for reporting your problems to us! We'll look into this and see what the problem may be.

Pete_Clark

Would definitely like to hear the response on this one too. I've got dbadmin passwordless ssh working on my cluster, and when I try to install vertica (with sudo) I'm getting [abc123@ip-10-0-1-235 abc123]# sudo /opt/vertica/sbin/install_vertica -s agg1,agg2,agg3 -r vertica-6.1.1-0.x86_64.RHEL5.rpm Vertica Analytic Database 6.1.1-0 Installation Tool Starting installation tasks... Getting system information for cluster (this may take a while).... 'failed to login to 10.0.1.175: EOF ERROR: Could not login with SSH. Here is what SSH said: (publickey).\r\r\n' Updating Nodes that are UP 'failed to login to 10.0.1.174: EOF ERROR: Could not login with SSH. Here is what SSH said: (publickey).\r\r\n' Vertica Analytic Database 6.1.1-0 Installation Tool Starting installation tasks... Getting system information for cluster (this may take a while).... 'failed to login to 10.0.1.175: EOF ERROR: Could not login with SSH. Here is what SSH said: (publickey).\r\r\n' Updating Nodes that are UP 'failed to login to 10.0.1.174: EOF ERROR: Could not login with SSH. Here is what SSH said: (publickey).\r\r\n' ... then eventually... Error parsing validation output: No JSON object could be decoded Validation output: Unable to obtain connection from pool Installation completed with warnings. Installation completed with errors. Installation failed.

Pete_Clark

oops, I copied the output in twice, sorry about that..

[Deleted User]

Hm... Is Vertica writing an authorized_keys2 file for you, or just a regular authorized_keys file? I'm not familiar with this particular detail of the installer off the top of my head (maybe others will be?), but my approach would be to figure out what's different between what the installer does and what your system/configuration expects.

carlbeech

Hi Vertica appears to be creating 'authorized_keys2' (along with the entire .ssh directory structure - or at least making calls to programs that are doing this). I'm not sure what the installer does - this is the reason why I created passwordless login capability manually - i.e. if I can do it manually, then does this mean that my sshd-config is working correctly? The problem is that while it's working correctly for me, when I get vertica to perform an install, then it effectively removes my .ssh directory, and replaces it with its own, and doesn't work. So, I think this means that a) I'm not setting passwordless login in the same way that the vertica installer program is doing it b) my sshd_config file works for me with the options I give it, but not for vertica installer c) as the .ssh directory I created is 'erased', this means that the only thing I can change is the sshd_conf file d) I've no idea what the vertica installer is doing, on a step by step basis, to ensure I have the correct settings for the sshd_conf file. Hopefully the above makes sense? So, to my thinking I either need an explicit list of changes I need to make to the sshd_config file to ensure I have all that's required for the installer to work, or alternatively a way to tell the installer that I've taken control of the passwordless login and not to make changes.... Hope that's ok... Thanks Carl.

[Deleted User]

I think you're correct that you will need to figure out a change to sshd_config. There is no way to tell Vertica to keep your existing .ssh directory -- this isn't a one-time thing; if you, for example, add a node in the future, admintools needs to be able to update all other nodes to know about the new node. Are you using a modified sshd_config file? You could try reverting to the default as provided with your distribution. (Which Linux distribution are you using?) What the installer does is create and manipulate the .ssh directory. You have the .ssh directory that it creates; you can read it to see what configuration it's looking for. It should have a public/private key pair and a configuration to allow passwordless auth using said key pair. You could also try running "ssh -vvv dbadmin@[host]" (with the Vertica-generated .ssh directory); that will give you more information about what, precisely, is not allowing the connection to proceed. Though it's very verbose; you would most likely want to find someone familiar with its output to review it and help you track down what's wrong.

carlbeech

Hi I've included the output of the command below. But the thing that stands out to me is the line: Trying private key: /home/dbadmin/ssh/id_dsa I've only got id_rsa type files in the .ssh directory...? Thanks Carl As mentioned at the top, I'm using Centos linux V6.4, I've modified the sshd_config file with the following: RSAAuthentication yes PubkeyAuthentication yes AuthorizedKeysFile .ssh/authorized_keys2 ssh -vvv dbadmin@host1 OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010 debug1: Reading configuration data /etc/ssh/ssh_config debug1: Applying options for * debug2: ssh_connect: needpriv 0 debug1: Connecting to host1 [192.168.44.200] port 22. debug1: Connection established. debug1: identity file /home/dbadmin/.ssh/identity type -1 debug3: Not a RSA1 key file /home/dbadmin/.ssh/id_rsa. debug2: key_type_from_name: unknown key type '-----BEGIN' debug3: key_read: missing keytype debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug2: key_type_from_name: unknown key type '-----END' debug3: key_read: missing keytype debug1: identity file /home/dbadmin/.ssh/id_rsa type 1 debug1: identity file /home/dbadmin/.ssh/id_dsa type -1 debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3 debug1: match: OpenSSH_5.3 pat OpenSSH* debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_5.3 debug2: fd 3 setting O_NONBLOCK debug1: SSH2_MSG_KEXINIT sent debug3: Wrote 792 bytes for a total of 813 debug1: SSH2_MSG_KEXINIT received debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,dif fie-hellman-group14-sha1,diffie-hellman-group1-sha1 debug2: kex_parse_kexinit: ssh-rsa,ssh-dss debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc, blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc, blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openss h.com,hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openss h.com,hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: first_kex_follows 0 debug2: kex_parse_kexinit: reserved 0 debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,dif fie-hellman-group14-sha1,diffie-hellman-group1-sha1 debug2: kex_parse_kexinit: ssh-rsa,ssh-dss debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc, blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc, blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openss h.com,hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openss h.com,hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: none,zlib@openssh.com debug2: kex_parse_kexinit: none,zlib@openssh.com debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: first_kex_follows 0 debug2: kex_parse_kexinit: reserved 0 debug2: mac_setup: found hmac-md5 debug1: kex: server->client aes128-ctr hmac-md5 none debug2: mac_setup: found hmac-md5 debug1: kex: client->server aes128-ctr hmac-md5 none debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP debug3: Wrote 24 bytes for a total of 837 debug2: dh_gen_key: priv key bits set: 119/256 debug2: bits set: 487/1024 debug1: SSH2_MSG_KEX_DH_GEX_INIT sent debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY debug3: Wrote 144 bytes for a total of 981 debug3: check_host_in_hostfile: filename /home/dbadmin/.ssh/known_hosts debug3: check_host_in_hostfile: match line 2 debug3: check_host_in_hostfile: filename /home/dbadmin/.ssh/known_hosts debug3: check_host_in_hostfile: match line 1 debug1: Host 'host1' is known and matches the RSA host key. debug1: Found key in /home/dbadmin/.ssh/known_hosts:2 debug2: bits set: 511/1024 debug1: ssh_rsa_verify: signature correct debug2: kex_derive_keys debug2: set_newkeys: mode 1 debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug3: Wrote 16 bytes for a total of 997 debug2: set_newkeys: mode 0 debug1: SSH2_MSG_NEWKEYS received debug1: SSH2_MSG_SERVICE_REQUEST sent debug3: Wrote 48 bytes for a total of 1045 debug2: service_accept: ssh-userauth debug1: SSH2_MSG_SERVICE_ACCEPT received debug2: key: /home/dbadmin/.ssh/identity ((nil)) debug2: key: /home/dbadmin/.ssh/id_rsa (0x7fc681146650) debug2: key: /home/dbadmin/.ssh/id_dsa ((nil)) debug3: Wrote 64 bytes for a total of 1109 debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password debug3: preferred gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive,password debug3: authmethod_lookup gssapi-keyex debug3: remaining preferred: gssapi-with-mic,publickey,keyboard-interactive,password debug3: authmethod_is_enabled gssapi-keyex debug1: Next authentication method: gssapi-keyex debug1: No valid Key exchange context debug2: we did not send a packet, disable method debug3: authmethod_lookup gssapi-with-mic debug3: remaining preferred: publickey,keyboard-interactive,password debug3: authmethod_is_enabled gssapi-with-mic debug1: Next authentication method: gssapi-with-mic debug3: Trying to reverse map address 192.168.44.200. debug1: Unspecified GSS failure. Minor code may provide more information Credentials cache file '/tmp/krb5cc_500' not found debug1: Unspecified GSS failure. Minor code may provide more information Credentials cache file '/tmp/krb5cc_500' not found debug1: Unspecified GSS failure. Minor code may provide more information debug1: Unspecified GSS failure. Minor code may provide more information Credentials cache file '/tmp/krb5cc_500' not found debug2: we did not send a packet, disable method debug3: authmethod_lookup publickey debug3: remaining preferred: keyboard-interactive,password debug3: authmethod_is_enabled publickey debug1: Next authentication method: publickey debug1: Trying private key: /home/dbadmin/.ssh/identity debug3: no such identity: /home/dbadmin/.ssh/identity debug1: Offering public key: /home/dbadmin/.ssh/id_rsa debug3: send_pubkey_test debug2: we sent a publickey packet, wait for reply debug3: Wrote 368 bytes for a total of 1477 debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password debug1: Trying private key: /home/dbadmin/.ssh/id_dsa debug3: no such identity: /home/dbadmin/.ssh/id_dsa debug2: we did not send a packet, disable method debug3: authmethod_lookup password debug3: remaining preferred: ,password debug3: authmethod_is_enabled password debug1: Next authentication method: password

Pete_Clark

This is where I'm stuck too. I finally had the passwordless authentication working, then vertica did its thing and it's back to not working. It seems clear to me why it's not working though unless I'm missing something, there were no keys generated in the new .ssh folder that vertica created. Is this expected? Exactly who am I supposed to be running as, do I need to run the sudo command as the dbadmin user? Or should I run the sudo command from a separate account ? Would the best way forward be to delete all of the dbadmin users we've created on all the hosts, then run the install command again (below), and then maybe it will create the dbadmin users properly with proper keys and permissions for passwordless authentication? My command is: sudo /opt/vertica/sbin/install_vertica -s agg1,agg2,agg3 -r vertica-6.1.1-0.x86_64.RHEL5.rpm -u dbadmin And the ssh -v output of ssh localhost (or by ip address, or hostname) as the dbadmin user, after vertica did its thing, is: debug3: preferred gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive,password debug3: authmethod_lookup publickey debug3: remaining preferred: keyboard-interactive,password debug3: authmethod_is_enabled publickey debug1: Next authentication method: publickey debug1: Trying private key: /home/dbadmin/.ssh/id_rsa debug3: no such identity: /home/dbadmin/.ssh/id_rsa debug1: Trying private key: /home/dbadmin/.ssh/id_dsa debug3: no such identity: /home/dbadmin/.ssh/id_dsa debug2: we did not send a packet, disable method debug1: No more authentication methods to try. Permission denied (publickey).

carlbeech

Hi I've retried the installation, but in this case I've now taken out the RSAAuthentication options - i.e. the sshd_config file now has: # RSAAuthentication yes # PubkeyAuthentication yes # AuthorizedKeysFile .ssh/authorized_keys # AuthorizedKeysFile .ssh/authorized_keys2 #AuthorizedKeysCommand none #AuthorizedKeysCommandRunAs nobody As per the original version of the installation file. Trying the installation again, I still get the error: Test of host 192.168.44.200 (FAILED) ======================================== Passwordless SSH access to other hosts (FAILED) --------------------------------------------------- Cannot connect from 192.168.44.200 to host 192.168.44.200 without password Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password). Vertica requires that SSH be configured for passwordless authentication. Verify that /etc/ssh/sshd_config is configured properly on 192.168.44.200. See the Vertica Installation Guide for more information. Test of host 192.168.44.201 (FAILED) ======================================== Passwordless SSH access to other hosts (FAILED) --------------------------------------------------- Cannot connect from 192.168.44.201 to host 192.168.44.200 without password Vertica requires that SSH be configured for passwordless authentication. Verify that /etc/ssh/sshd_config is configured properly on 192.168.44.200. See the Vertica Installation Guide for more information. +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Performing a ssh -vvv dbadmin@host2 (from host1) I get: ssh -vvv dbadmin@host2 OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010 debug1: Reading configuration data /etc/ssh/ssh_config debug1: Applying options for * debug2: ssh_connect: needpriv 0 debug1: Connecting to host2 [192.168.44.201] port 22. debug1: Connection established. debug1: identity file /home/dbadmin/.ssh/identity type -1 debug3: Not a RSA1 key file /home/dbadmin/.ssh/id_rsa. debug2: key_type_from_name: unknown key type '-----BEGIN' debug3: key_read: missing keytype debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug2: key_type_from_name: unknown key type '-----END' debug3: key_read: missing keytype debug1: identity file /home/dbadmin/.ssh/id_rsa type 1 debug1: identity file /home/dbadmin/.ssh/id_dsa type -1 debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3 debug1: match: OpenSSH_5.3 pat OpenSSH* debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_5.3 debug2: fd 3 setting O_NONBLOCK debug1: SSH2_MSG_KEXINIT sent debug3: Wrote 792 bytes for a total of 813 debug1: SSH2_MSG_KEXINIT received debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1 debug2: kex_parse_kexinit: ssh-rsa,ssh-dss debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: first_kex_follows 0 debug2: kex_parse_kexinit: reserved 0 debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1 debug2: kex_parse_kexinit: ssh-rsa,ssh-dss debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: none,zlib@openssh.com debug2: kex_parse_kexinit: none,zlib@openssh.com debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: first_kex_follows 0 debug2: kex_parse_kexinit: reserved 0 debug2: mac_setup: found hmac-md5 debug1: kex: server->client aes128-ctr hmac-md5 none debug2: mac_setup: found hmac-md5 debug1: kex: client->server aes128-ctr hmac-md5 none debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP debug3: Wrote 24 bytes for a total of 837 debug2: dh_gen_key: priv key bits set: 132/256 debug2: bits set: 508/1024 debug1: SSH2_MSG_KEX_DH_GEX_INIT sent debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY debug3: Wrote 144 bytes for a total of 981 debug3: check_host_in_hostfile: filename /home/dbadmin/.ssh/known_hosts debug3: check_host_in_hostfile: match line 2 debug3: check_host_in_hostfile: filename /home/dbadmin/.ssh/known_hosts debug3: check_host_in_hostfile: match line 2 debug1: Host 'host2' is known and matches the RSA host key. debug1: Found key in /home/dbadmin/.ssh/known_hosts:2 debug2: bits set: 503/1024 debug1: ssh_rsa_verify: signature correct debug2: kex_derive_keys debug2: set_newkeys: mode 1 debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug3: Wrote 16 bytes for a total of 997 debug2: set_newkeys: mode 0 debug1: SSH2_MSG_NEWKEYS received debug1: SSH2_MSG_SERVICE_REQUEST sent debug3: Wrote 48 bytes for a total of 1045 debug2: service_accept: ssh-userauth debug1: SSH2_MSG_SERVICE_ACCEPT received debug2: key: /home/dbadmin/.ssh/identity ((nil)) debug2: key: /home/dbadmin/.ssh/id_rsa (0x7fb516c7e650) debug2: key: /home/dbadmin/.ssh/id_dsa ((nil)) debug3: Wrote 64 bytes for a total of 1109 debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password debug3: preferred gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive,password debug3: authmethod_lookup gssapi-keyex debug3: remaining preferred: gssapi-with-mic,publickey,keyboard-interactive,password debug3: authmethod_is_enabled gssapi-keyex debug1: Next authentication method: gssapi-keyex debug1: No valid Key exchange context debug2: we did not send a packet, disable method debug3: authmethod_lookup gssapi-with-mic debug3: remaining preferred: publickey,keyboard-interactive,password debug3: authmethod_is_enabled gssapi-with-mic debug1: Next authentication method: gssapi-with-mic debug3: Trying to reverse map address 192.168.44.201. debug1: Unspecified GSS failure. Minor code may provide more information Credentials cache file '/tmp/krb5cc_500' not found debug1: Unspecified GSS failure. Minor code may provide more information Credentials cache file '/tmp/krb5cc_500' not found debug1: Unspecified GSS failure. Minor code may provide more information debug1: Unspecified GSS failure. Minor code may provide more information Credentials cache file '/tmp/krb5cc_500' not found debug2: we did not send a packet, disable method debug3: authmethod_lookup publickey debug3: remaining preferred: keyboard-interactive,password debug3: authmethod_is_enabled publickey debug1: Next authentication method: publickey debug1: Trying private key: /home/dbadmin/.ssh/identity debug3: no such identity: /home/dbadmin/.ssh/identity debug1: Offering public key: /home/dbadmin/.ssh/id_rsa debug3: send_pubkey_test debug2: we sent a publickey packet, wait for reply debug3: Wrote 368 bytes for a total of 1477 debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password debug1: Trying private key: /home/dbadmin/.ssh/id_dsa debug3: no such identity: /home/dbadmin/.ssh/id_dsa debug2: we did not send a packet, disable method debug3: authmethod_lookup password debug3: remaining preferred: ,password debug3: authmethod_is_enabled password debug1: Next authentication method: password dbadmin@host2's password: The .ssh directory contains: -rwxrwxrwx. 1 dbadmin root 1571 Apr 22 19:29 authorized_keys2 -rwxrwxrwx. 1 dbadmin root 1675 Apr 22 19:29 id_rsa -rwxrwxrwx. 1 dbadmin root 392 Apr 22 19:29 id_rsa.pub -rw-r--r--. 1 dbadmin dbadmin 798 Apr 22 19:33 known_hosts

Pete_Clark

I'm good now, what I did is enabled root login across my machines, created the keys for the root user and copied them to the other machines, deleted the dbadmin user from all the machines (userdel -r dbadmin), did a "sudo bash" to get to root on my first box then ran the installer. All is good now. Is enabling root login across the machines an option for you?

carlbeech

Hi Here's a rather lengthy transcript of an install - hopefully you can see the steps I've taken Please note - my test environment is two VM's running through VMWare player running on the same PC - i.e. the system times should be the same. I've hidden some of the security information in the transcript, but there should be enough to see... I've made comment through the transcript with >>>>>> Thanks Carl. >>>>> Starting from 'blank' virtual machines >>>>> i.e. rpm -e vertica-ce >>>>> rm -rf /opt/vertica >>>>> userdel -r dbadmin >>>>> Create connectivity for ROOT user -- HOST1 [root@host1 .ssh]# ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: XX:XX:XX:XX:XX:XX:XX:XX:bf:24:8a:08:05:48:30:04 root@host1 The key's randomart image is: +--[ RSA 2048]----+ |E=o | >>>>>>> edited | | +-----------------+ >>>>> Note I've edited the above >>>>> Copy the key to host2 [root@host1 .ssh]# ssh-copy-id root@host2 The authenticity of host 'host2 (192.168.44.201)' can't be established. RSA key fingerprint is XX:XX:XX:XX:XX:XX:a0:7a:9e:6f:62:81:70:98:6d:ca. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'host2,192.168.44.201' (RSA) to the list of known hosts. root@host2's password: Now try logging into the machine, with "ssh 'root@host2'", and check in: .ssh/authorized_keys to make sure we haven't added extra keys that you weren't expecting. >>>>> Check for passwordless login for root user to HOST2 [root@host1 .ssh]# ssh root@host2 Last login: Tue Apr 23 17:52:05 2013 from host1 >>>>> Passwordless login works - now do the other way around HOST2->HOST1 [root@host2 ~]# ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: XX:XX:XX:XX:XX:XX:XX:XX:XX:4c:12:05:3e:a7:c1:00 root@host2 The key's randomart image is: +--[ RSA 2048]----+ |E | >>>>> edited |.o. . ... | +-----------------+ [root@host2 ~]# ssh-copy-id root@host1 The authenticity of host 'host1 (192.168.44.200)' can't be established. RSA key fingerprint is XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:81:70:98:6d:ca. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'host1,192.168.44.200' (RSA) to the list of known hosts. root@host1's password: Now try logging into the machine, with "ssh 'root@host1'", and check in: .ssh/authorized_keys to make sure we haven't added extra keys that you weren't expecting. >>>>> Check connectivity HOST2->HOST1 [root@host2 ~]# ssh root@host1 Last login: Tue Apr 23 17:48:58 2013 from 192.168.44.1 >>>>> Passwordless login from HOST2 to HOST1 successful [root@host1 ~]# logout Connection to host1 closed. [root@host2 ~]# logout Connection to host2 closed. [root@host1 ~]# cd $HOME/.ssh [root@host1 .ssh]# ls -l total 16 -rw-------. 1 root root 392 Apr 23 17:55 authorized_keys -rw-------. 1 root root 1675 Apr 23 17:52 id_rsa -rw-r--r--. 1 root root 392 Apr 23 17:52 id_rsa.pub -rw-r--r--. 1 root root 402 Apr 23 17:52 known_hosts >>>>> Now, install vertica-ce RPM [root@host1 .ssh]# cd /mnt/hgfs/SHARED_FOLDERS/ [root@host1 SHARED_FOLDERS]# ls -l total 47599 -rwxrwxrwx. 1 root root 3323 Apr 23 16:50 sshd_config -rwxrwxrwx. 1 root root 79739165 Apr 18 23:37 vertica-ce-6.0.1-0.x86_64.RHEL5.rpm -rwxrwxrwx. 1 root root 17738471 Apr 18 23:36 vertica-client-6.0.1-0.x86_64.rpm [root@host1 SHARED_FOLDERS]# rpm -Uvh vertica-ce-6.0.1-0.x86_64.RHEL5.rpm Preparing... ########################################### [100%] 1:vertica-ce ########################################### [100%] Vertica Analytic Database V6.0.1-0 successfully installed on host host1 ---------------------------------------------------------------------------------- Important Information ---------------------------------------------------------------------------------- If you are upgrading from a previous version, you must backup your database before continuing with this install. After restarting your database, you will be unable to revert to a previous version of the software. ---------------------------------------------------------------------------------- To download the latest Vertica documentation in zip or tar format please visit the myvertica web site. To complete installation and configuration of the cluster, run: /opt/vertica/sbin/install_vertica >>>>> Now perform cluster install for HOST1,HOST2 [root@host1 SHARED_FOLDERS]# /opt/vertica/sbin/install_vertica -s host1,host2 -r vertica-ce-6.0.1-0.x86_64.RHEL5.rpm Vertica Analytic Database 6.0.1-0 Installation Tool Upgrading admintools meta data format.. scanning /opt/vertica/config/users Starting installation tasks... Getting system information for cluster (this may take a while).... backing up admintools.conf on 192.168.44.200 Installing rpm on 1 hosts.... installing node.... 192.168.44.201 NTP service not synchronized on the hosts: ['192.168.44.201', '192.168.44.200'] Check your NTP configuration for valid NTP servers. Vertica recommends that you keep the system clock synchronized using NTP or some other time synchronization mechanism to keep all hosts synchronized. Time variances can cause (inconsistent) query results when using Date/Time Functions. For instructions, see: * http://kbase.redhat.com/faq/FAQ_43_755.shtm * http://kbase.redhat.com/faq/FAQ_43_2790.shtm Info: the package 'pstack' is useful during troubleshooting. Vertica recommends this package is installed. Checking/fixing OS parameters..... Setting vm.min_free_kbytes to 4096 ... Detected cpufreq module loaded on 192.168.44.201 Detected cpufreq module loaded on 192.168.44.200 CPU frequency scaling is enabled. This may adversely affect the performance of your database. Vertica recommends that cpu frequency scaling be turned off or set to 'performance' Creating/Checking Vertica DBA group Creating/Checking Vertica DBA user Password for dbadmin: Installing/Repairing SSH keys for dbadmin Password: Creating Vertica Data Directory... Testing N-way network test. (this may take a while) All hosts are available ... Verifying system requirements on cluster. IP configuration ... IP configuration ... Testing hosts (1 of 2).... Running Consistency Tests LANG and TZ environment variables ... Running Network Connectivity and Throughput Tests... Waiting for 1 of 2 sites... ... Test of host 192.168.44.200 (FAILED) ======================================== Passwordless SSH access to other hosts (FAILED) --------------------------------------------------- Cannot connect from 192.168.44.200 to host 192.168.44.200 without password Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password). Vertica requires that SSH be configured for passwordless authentication. Verify that /etc/ssh/sshd_config is configured properly on 192.168.44.200. See the Vertica Installation Guide for more information. Test of host 192.168.44.201 (FAILED) ======================================== Passwordless SSH access to other hosts (FAILED) --------------------------------------------------- Cannot connect from 192.168.44.201 to host 192.168.44.200 without password Vertica requires that SSH be configured for passwordless authentication. Verify that /etc/ssh/sshd_config is configured properly on 192.168.44.200. See the Vertica Installation Guide for more information. Consistency Test (ok) ========================= Info: The $TZ environment variable is not set on 192.168.44.200 Info: The $TZ environment variable is not set on 192.168.44.201 Network Test (ok) ===================== Network communication (ok) ------------------------------ Low throughput 192.168.44.200 to 192.168.44.201: 41.6394726297 Mbps; check network interface/switch configuration Verification failed. Correct the above issues to proceed Installation completed with warnings. Installation completed with errors. Installation failed. [root@host1 SHARED_FOLDERS]# [root@host1 SHARED_FOLDERS]# su - dbadmin [dbadmin@host1 ~]$ cd .ssh [dbadmin@host1 .ssh]$ ls -l total 16 -rwxrwxrwx. 1 dbadmin root 392 Apr 23 17:58 authorized_keys2 -rwxrwxrwx. 1 dbadmin root 1675 Apr 23 17:58 id_rsa -rwxrwxrwx. 1 dbadmin root 392 Apr 23 17:58 id_rsa.pub -rw-r--r--. 1 dbadmin dbadmin 396 Apr 23 17:59 known_hosts >>>>> note here that instead of 'authorized_keys' we have 'authorized_keys2' >>>>> also, the incorrect values for permissions - they should be 600 not 777 >>>>> I've made a second run, where I changed sshd_config to: # RSAAuthentication yes # PubkeyAuthentication yes # AuthorizedKeysFile .ssh/authorized_keys AuthorizedKeysFile .ssh/authorized_keys2 <-- Added in second run #AuthorizedKeysCommand none #AuthorizedKeysCommandRunAs nobody However, the result was the same. Can someone comment on the compatibility of vertica community edition with CENTOS 6.4 and openssh-server-5.3p1-84.1.el6.x86_64? Many thanks Carl.

[Deleted User]

Not an official comment on compatibility, but a question for a little more detail: When you say this is a "blank" virtual machine, is it really blank (ie., same as a fresh CentOS install), or does it just not have Vertica? Vertica constructs the .ssh directory manually; as you can probably guess from the differing results here, it doesn't use ssh-copy-id, etc. This is somewhat subject to OS configuration. For example, if you have a default umask of 000 or a filesystem that forces files to '777' by default, that could cause the issues that you're seeing with permissions. I don't know about CentOS 6.4 in particular, though. Specific comments there would be welcome.

carlbeech

Hi True, when I say 'blank' I mean that it's a VM I've been trying to get this to work on, however, I've removed vertica by: * rpm -e vertica-ce * rm -rf /opt/vertica * userdel -r dbadmin The first two commands are from the manuals on how to uninstall vertica, the userdel is from the post above from Peter. For information: [root@host1 etc]# ssh root@host2 Last login: Tue Apr 23 18:16:28 2013 from host1 [root@host2 ~]# su - dbadmin [dbadmin@host2 ~]$ umask 0002 You can see here: * passwordless logins work using the current sshd_config file for root user * the umask for dbadmin is 0002, which is u=rwx,g=rwx,o=rx * as the umask is different from the permissions associated to the created files, then something else appears to be happening [dbadmin@host2 ~]$ cd .ssh [dbadmin@host2 .ssh]$ ls -l total 16 -rwxrwxrwx. 1 dbadmin root 392 Apr 23 18:16 authorized_keys2 -rwxrwxrwx. 1 dbadmin root 1675 Apr 23 18:16 id_rsa -rwxrwxrwx. 1 dbadmin root 392 Apr 23 18:16 id_rsa.pub -rw-r--r--. 1 dbadmin dbadmin 396 Apr 23 18:16 known_hosts This is the listing from host2's dbadmin/.ssh user/dir - they're all 777 * The user was created by the RPM installation process * the 'known_hosts' file shows the default ownership of the files as dbadmin/dbadmin - however the files are dbadmin/root? Also, I don't think it's a permissions-only issue - having set all files within ~dbadmin/.ssh to 600, and the directory to 700 (which is what I did with the ROOT user above), I still get prompted for the password. [dbadmin@host1 .ssh]$ ssh dbadmin@host2 The authenticity of host 'host1 (192.168.44.200)' can't be established. RSA key fingerprint is xx:xx:xx:xx:xx:e3:a0:7a:9e:6f:62:81:70:98:6d:ca. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'host1' (RSA) to the list of known hosts. dbadmin@host2's password: Last login: Tue Apr 23 18:59:31 2013 from host1 Looking at the authorized_keys2 file within the dbadmin user, I can see that the key is for root@host1... I would have expected dbadmin@host1, which might explain why the passwordless login for dbadmin user is failing? Is this because I'm running the install as the 'root' user on host1? Thanks Carl.

[Deleted User]

Hello, Carl. I believe what you are seeing for "root@host1" is just the default comment associated with the (public) key in the authorized_keys file. Authentication takes place based on whose home directory the authorized_keys file is in, not based on the comment associated with the key. Could you show the output of the following? It runs SSH with verbose and specifically specifying /home/dbadmin/.ssh/id_rsa as the identity to use. The remote command is 'true' so that it exits if it successfully connects. ssh -v -i ~/.ssh/id_rsa dbadmin@host2 true

carlbeech

Hi I've given the output of the command below. I've been working on this issue for some time now, trying different combinations, and even trying to examine differences in comparison to the VM appliance... I've checked that when the VM (host1) was duplicated to VM (host2), that the /etc/ssh/ssh_host_rsa_key was re-generated for each machine. I've 'cleaned' out the two VM's using the following rpm -e vertica-ce.... rm -rf /opt/vertica userdel -r dbadmin I've then done an install_vertica with option '-s host1,host2'. As before, I've had the failure, Test of host 192.168.44.201 (FAILED) ======================================== Passwordless SSH access to other hosts (FAILED) --------------------------------------------------- Cannot connect from 192.168.44.201 to host 192.168.44.200 without password Vertica requires that SSH be configured for passwordless authentication. Verify that /etc/ssh/sshd_config is configured properly on 192.168.44.200. however I can see that it's created the .ssh directory, for each of the dbadmin users: ________________________________________________________________ host1: cat /etc/ssh/ssh_host_rsa_key.pub ==================================== ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwz8FKcwJRwAE0oNr6zbxxWw9NcaNg1Yd3Xk... QMgMlQjDG90VcVaCYeoEKWfpyqcY6Q== root@host1 cat id_rsa ==================================== -----BEGIN RSA PRIVATE KEY----- MIIEogIBAAKCAQEAv7ZngieAj/reqyDbWpaUsJ1cE0OrdYf2tdPRd689Lrj9yI9S ... QlRT/3yCtoJOeBCB9CS7VkV8UcSUSnXuwfzkOcWCcOpb2S6NVeM= -----END RSA PRIVATE KEY----- [root@host1 .ssh]# cat id_rsa.pub ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv7ZngieAj/reqyDbWpaUsJ1cE0OrdYf2tdPRd.../MXRXZIxGbivvrCWLl02LXA6WK3b9SniQWTfbuvGboJfpRvOV3u9TzK0qNUtrtQ61cnB12JGqZJbuHYhnMXnGY1ofQ== root@host1 cat authorized_keys2 ==================================== ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv7ZngieAj/reqyDbWpaUsJ1cE0OrdYf2tdPRd... /MXRXZIxGbivvrCWLl02LXA6WK3b9SniQWTfbuvGboJfpRvOV3u9TzK0qNUtrtQ61cnB12JGqZJbuHYhnMXnGY1ofQ== root@host1 ________________________________________________________________ host2: cat /etc/ssh/ssh_host_rsa_key.pub ==================================== ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAxLG7IuxkZiVQzQSdoDeRwxwCxgmwqCghkA1.... /ino+YbEck5OLLXyatYtRcHzN0K70JsDZN9S1EIrf7f6I6HWaSfESw== root@host2 cat id_rsa ==================================== -----BEGIN RSA PRIVATE KEY----- MIIEogIBAAKCAQEAv7ZngieAj/reqyDbWpaUsJ1cE0OrdYf2tdPRd689Lrj9yI9S ..... QlRT/3yCtoJOeBCB9CS7VkV8UcSUSnXuwfzkOcWCcOpb2S6NVeM= -----END RSA PRIVATE KEY----- [root@host2 .ssh]# cat id_rsa.pub ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv7ZngieAj/reqyDbWpaUsJ1cE0OrdYf2tdPRd .... /MXRXZIxGbivvrCWLl02LXA6WK3b9SniQWTfbuvGboJfpRvOV3u9TzK0qNUtrtQ61cnB12JGqZJbuHYhnMXnGY1ofQ== root@host1 cat authorized_keys2 ==================================== ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv7ZngieAj/reqyDbWpaUsJ1cE0OrdYf2tdPRd .... /MXRXZIxGbivvrCWLl02LXA6WK3b9SniQWTfbuvGboJfpRvOV3u9TzK0qNUtrtQ61cnB12JGqZJbuHYhnMXnGY1ofQ== root@host1 ________________________________________________________________ As you can see, while the host rsa public keys are different, the RSA values of the two dbadmin SSH directories are the same? It appears that the contents of the .ssh directory may be being copied as part of the installation process from host1 to host2? (guessing here) Hopefully this may help to identify the location of the issue? Hope that helps Thanks Carl. [dbadmin@host1 ~]$ ssh -v -i ~/.ssh/id_rsa dbadmin@host2 true OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010 debug1: Reading configuration data /etc/ssh/ssh_config debug1: Applying options for * debug1: Connecting to host2 [192.168.44.201] port 22. debug1: Connection established. debug1: identity file /home/dbadmin/.ssh/id_rsa type 1 debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3 debug1: match: OpenSSH_5.3 pat OpenSSH* debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_5.3 debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex: server->client aes128-ctr hmac-md5 none debug1: kex: client->server aes128-ctr hmac-md5 none debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP debug1: SSH2_MSG_KEX_DH_GEX_INIT sent debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY debug1: Host 'host2' is known and matches the RSA host key. debug1: Found key in /home/dbadmin/.ssh/known_hosts:2 debug1: ssh_rsa_verify: signature correct debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: SSH2_MSG_NEWKEYS received debug1: SSH2_MSG_SERVICE_REQUEST sent debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password debug1: Next authentication method: gssapi-keyex debug1: No valid Key exchange context debug1: Next authentication method: gssapi-with-mic debug1: Unspecified GSS failure. Minor code may provide more information Credentials cache file '/tmp/krb5cc_500' not found debug1: Unspecified GSS failure. Minor code may provide more information Credentials cache file '/tmp/krb5cc_500' not found debug1: Unspecified GSS failure. Minor code may provide more information debug1: Unspecified GSS failure. Minor code may provide more information Credentials cache file '/tmp/krb5cc_500' not found debug1: Next authentication method: publickey debug1: Offering public key: /home/dbadmin/.ssh/id_rsa debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password debug1: Next authentication method: password dbadmin@host2's password: debug1: Authentication succeeded (password). debug1: channel 0: new [client-session] debug1: Requesting no-more-sessions@openssh.com debug1: Entering interactive session. debug1: Sending environment. debug1: Sending env LANG = en_US.UTF-8 debug1: Sending command: true debug1: client_input_channel_req: channel 0 rtype exit-status reply 0 debug1: client_input_channel_req: channel 0 rtype eow@openssh.com reply 0 debug1: channel 0: free: client-session, nchannels 1 Transferred: sent 1776, received 2040 bytes, in 0.2 seconds Bytes per second: sent 8218.0, received 9439.6 debug1: Exit status 0 Transferred: sent 1776, received 2040 bytes, in 0.2 seconds Bytes per second: sent 8156.3, received 9368.7 debug1: Exit status 0

[Deleted User]

Carl, You should immediately delete the authorized_keys and the id_rsa files now that you have shown a portion of your private key on the public internet. Now that part of your private key is known, the protection provided by these authentication methods is weakened. Was the following output provided with or without the permissions corrected on the SSH directory and contents?

[dbadmin@host1 ~]$ ssh -v -i ~/.ssh/id_rsa dbadmin@host2 true  OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010  debug1: Reading configuration data /etc/ssh/ssh_config  debug1: Applying options for *  debug1: Connecting to host2 [192.168.44.201] port 22.  debug1: Connection established.  debug1: identity file /home/dbadmin/.ssh/id_rsa type 1  debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3  debug1: match: OpenSSH_5.3 pat OpenSSH*  debug1: Enabling compatibility mode for protocol 2.0  debug1: Local version string SSH-2.0-OpenSSH_5.3  debug1: SSH2_MSG_KEXINIT sent  debug1: SSH2_MSG_KEXINIT received  debug1: kex: server->client aes128-ctr hmac-md5 none  debug1: kex: client->server aes128-ctr hmac-md5 none  debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024s password:  debug1: Authentication succeeded (password). 

If you did fix the permissions on the files, I suspect that something is different about your SSH configurations on client and/or server. Notice that it never offers a public/private key for connection authentication. Contrast to my output:

...debug1: Authentications that can continue: publickey,gssapi-with-mic,password  debug1: Next authentication method: publickey  debug1: Offering RSA public key: drice@computer  

In general, this appears to be an SSH configuration issue compounded with a problem which is causing Vertica to create these files with the incorrect permissions. My suggestion is that you use a different user and configure passwordless SSH between the two machines. If you can do that successfully, it makes sense to revisit what may be wrong with the Vertica installation. http://www.debian-administration.org/articles/152

carlbeech

Hi Not to worry about the keys - these are on 'sandbox' VMs running host-only networking within a single PC - and at the moment are being regularly rebuilt... The reason I showed a portion of the keys was to demonstrate that somehow, the files from host1 were being put into host2 - i.e. the private key of the two machines was the same - which to my thinking, shouldn't happen. The DBADMIN user was created by the installation process, and performing a 'useradd' doesn't create a .ssh folder and files, so the question remains as to how the files were created (on both hosts). Please have a look earlier in the post where I demonstrate that I have had passwordless login working correctly using the dbadmin user. The permissions for the files appears correct. From my 'outsider' point of view, it appears that the installer is creating the dbadmin users on both hosts, performing an ssh-keygen on host1 and copying the contents of the .ssh folder to host2, but stopping at that point instead of performing a ssh-keygen on host2? Thanks Carl.

[Deleted User]

"the private key of the two machines was the same - which to my thinking, shouldn't happen." This is intentional. In some environments it is preferred to have a single private key for a particular persona (the Vertica dbadmin). We are welcome to taking input on this particular design decision. "so the question remains as to how the files were created (on both hosts). ... From my 'outsider' point of view, it appears that the installer is creating the dbadmin users on both hosts, performing an ssh-keygen on host1 and copying the contents of the .ssh folder to host2, but stopping at that point instead of performing a ssh-keygen on host2?" You are correct. This is the goal. This should set up a working SSH environment. The private key is the same on all nodes, and is trusted by each of them.

carlbeech

Ahh - it starts to make more sense now... thanks I've not had dealings where ssh keys have played a fundamental role in the build of a database - or if they've been involved it's been for a single server - which is where I work on the idea that each node has its own SSH key... but I can see why you work the keys in this fashion... - I'm not qualified to comment on whether each machine should / should not have different keys - it's just my assumption that they would have... I have been able to bring up a cluster, but not (I think) in the way that's expected - here's the procedure: First, take host1, and copy the .ssh directory (tar) - to include id_rsa, id_rsa.pub, known_hosts and authorized_keys2 As before, create clean machines by: rpm -e vertica-ce rm -rd /opt/vertica userdel -r dbadmin Now, on each machine in turn (host1,host2,host3): useradd dbadmin copy in the 'tar'd .ssh directory. (Note: I do not set the dbadmin password - i.e. it cannot be logged into directly) Now, on host1 issue the necessary rpm and vertica_install commands (as root): rpm -Uvh vertica_ce.... /opt/vertica/sbin/install_vertica -s host1,host2,host3 -r vertica_ce.... It now manages to build the cluster, and as you mention, all nodes have the same SSH settings... My thought is that it can't log into the dbadmin user (as there's no password allocated), and therefore the installer can't attempt to overwrite the SSH settings?... As mentioned, this is the only way I can get this to work so far... I've tested that you can then allocate a password to the dbadmin user after the install is complete... - if you're adding a new node, you need to do as above (i.e. the new machine does not have a password, and it appears to add ok) If I give the dbadmin user a password (passwd dbadmin...) before issuing the rpm and install_vertica commands, it prompts for an extra password during the install, and then breaks the ssh settings, ending up with an install failure... Hope that helps? Thanks Carl

[Deleted User]

Glad you got it to work for you Carl, even if you had to use work arounds. Sorry for the inconvenience.

carlbeech

Not a problem - I am still concerned though that I'm not doing the install in the 'correct' way - i.e. that the installer should be creating the dbadmin user - and that this may end up being a problem further down the track (I know that I'm only working on tester VMs at the moment, however, if this gets deployed then I wouldn't want to use work-arounds for the reasons mentioned)... As a thought, is there any verbose logging that I can switch on with the installer so we can track down where the 'deviation from norm' is taking place? Thanks Carl.