Backup failed unexpectedly! ERROR

Fredou

Hi,
My Vertica backups were working great untill i updated to the last version.
I use vbr.py but i get the message "backup failed unexpectedly!" every time i try to use it.
I even tried to do the --setupconfig but it doesn't help...

Does someone has an idea ?

Thanks 

Raul_1

Can you post the contents of the created config file?

Fredou

Yes, here it is:

[Misc]snapshotName = fullBackup
restorePointLimit = 1

[Database]
dbName = xxxxxx
dbUser = yyyyyy

[Transmission]

[Mapping]
v_xxxxxx_node0001 = 127.0.0.1:/tmp/vbr

If i add the --debug 3 option i can see this error:

Traceback (most recent call last):  File "/opt/vertica/bin/vbr.py", line 2526, in backup
    prepareAll()
  File "/opt/vertica/bin/vbr.py", line 1888, in prepareAll
    configCheck()
  File "/opt/vertica/bin/vbr.py", line 506, in configCheck
    concurrency_upperboud = int(subprocess.Popen(['ulimit -u'], shell=True, stdout= subprocess.PIPE).communicate()[0].strip())
ValueError: invalid literal for int() with base 10: 'unlimited'
backup failed unexpectedly!

Thansk for your help

Raul_1

Is the OS Ubuntu?
Can you provide the output of ls -l /bin/sh
 

Fredou

It's Debian,

lrwxrwxrwx 1 root root 4 Dec 17 10:53 /bin/sh -> bash

Raul_1

Can you try running -
ulimit -u
 

Fredou

I get "unlimited"

id10t

Hi Fredou!


1. Can you post output for:

python -c "import json; print json.dumps('/opt/vertica/bin/validators.sh', indent=2)"

2. Modify limits.conf file, set max number of processes (lets say db super user is dbadmin):

dbadmin  -   nproc  65536

3. Validate settings:

$ ulimit -u
65536

do reboot and try backup.


PS
Try to validate it without reboot, set limit manually and if it do not hangs an error run backup.

*********

How to set limit manually?

ulimit -u 65536

if it do not throws an error so try to do a backup.

*********

Example of error:

daniel@synapse:/etc/security$ ulimit -u 131072
bash: ulimit: max user processes: cannot modify limit: Operation not permitted

Regards, idiot

Fredou

Thanks, the "ulimit -u 65536" did the trick !

id10t

Hi!

I will ask to run a first command from my post above one more time. You have to set a proper value(65536 - is an arbitrary value) for nproc (the key is max_user_proc)

daniel@synapse:~$ python -c "import json; print json.dumps('/opt/vertica/bin/validators.sh', indent=2)"
{
  "total_memory": 7844, 
  "vertica": {
    "release": "2", 
    "brand": "vertica", 
    "version": "7.1.1", 
    "arch": "iamd64"
  }, 
  "nics": [
    {
      "broadcast": "", 
      "ipaddr": "", 
      "netmask": "", 
      "speed": "unknown", 
      "name": "eth0"
    }, 
    {
      "broadcast": "255.255.255.255", 
      "ipaddr": "127.0.0.1", 
      "netmask": "255.0.0.0", 
      "speed": "locallink", 
      "name": "lo"
    }, 
    {
      "broadcast": "", 
      "ipaddr": "", 
      "netmask": "", 
      "speed": "unknown", 
      "name": "ppp0"
    }
  ], 
  "hostname": "synapse", 
"max_user_proc": "62582", 
  "cpu_info": {
    "number_of_cpus": 8, 
    "cpu_type": " Intel(R) Core(TM) i7-3612QM CPU @ 2.10GHz"
  }
}