Number of threads chosen by vioperf

Tim_1

Hi,
Running vioperf on 6 new servers that will replace an old cluster, when I run vioperf on each node of the old cluster it runs in 20 threads on each host. On the new cluster the number of threads chosen to run the test varies on each server from 32-42 cores.
Old servers 2 times10cores, with Hyper Threading 40 cores
New servers 2 times16cores, with HT 64 cores.
How does vioperf decide how many threads to run the test?

Bryan_H

In recent releases of vioperf, you can set the thread count. Run vioperf --help for specifics.

Bryan_H

--thread-count=
From vertica.com docs: "The number of execution threads to use. By default, vioperf uses all threads available on the host machine."

Tim_1

Thanks, will rerun the tests with that set, but any idea why different servers of identical spec would choose different thread counts?

Bryan_H

Does the new cluster have a restored database and config from the old cluster? I will check internally whether we know why this happens.

Tim_1

Ran vioperf before installing vertica and after a database was replicated onto it.
Of the 6 servers, 2 ran 32 threads, 2 ran 40 threads and one each on 38 and 44 threads. This appears consistent, always chooses the same number of threads on the same server.

Bryan_H

It looks like vioperf chooses number of threads based on /proc/cpuinfo using "core id" and "physical id" entries attempting to compute physical (non-HT) cores by deduplicating on physical ID. Can you check to see if the /proc/cpuinfo on each node matches what vioperf is using?

Tim_1

/proc/cpuinfo is the same on all 6 servers:-
grep "core id" /proc/cpuinfo | wc -l
64
grep "physical id" /proc/cpuinfo | wc -l
64

So can understand why 2 have them are running 32 threads, the actual core count, but don't understand why the other 4 servers have chosen a different number of threads.

Bryan_H

I've no idea either, but have raised a request to look into it. What version of Vertica did the vioperf binary come from?

Tim_1

vsql --version
vsql version 09.03.0108, built for Linux64, contains support for command-line editing
Sorry, just rechecked the thread count in the report, none of them chose 32 threads.
Two chose 38 threads, two chose 40 threads and one each for 42 and 44 threads.
Thanks for raising the request, all the servers should have been built the same, so a bit of a concern as to why they defaulted to different values.