How to set HadoopFSBlockSizeBytes ?
ardeks
- Select Field -
in my hdfs-site.xml dfs.blocksize=536870912, but Vertica do not use this value.
-- hdfs_cluster_config_check --
Hadoop Conf Path : [/etc/hadoop/conf]
[OK] HadoopConfDir verified on all nodes
Connection Timeout (seconds) : [60]
Token Refresh Frequency (seconds) : [0]
HadoopFSBlockSizeBytes (MB) : [64]
[OK] Found [1] hadoop cluster configurations
0
Best Answer
-
SruthiA
Vertica Employee Administrator
You can set it using the following query
dbadmin=> SELECT get_config_parameter('HadoopFSBlockSizeBytes');
get_config_parameter
67108864
(1 row)dbadmin=> SELECT set_config_parameter('HadoopFSBlockSizeBytes',671088000);
set_config_parameter
Parameter set successfully
(1 row)dbadmin=> SELECT get_config_parameter('HadoopFSBlockSizeBytes');
get_config_parameter
671088000
(1 row)dbadmin=>
5
Answers
@ardeks -
Fyi ... Per the output of the hdfs_cluster_config_check function, "Connection Timeout", "Token Refresh Frequency" and "HadoopFSBlockSizeBytes" are all database configuration parameters that can be viewed by querying the CONFIGURATION_PARAMETERS system table.
Example:
dbadmin=> SELECT parameter_name, default_value, description FROM configuration_parameters WHERE parameter_name IN ('HadoopFSTokenRefreshFrequency', 'HadoopFSConnectionTimeout', 'HadoopFSBlockSizeBytes'); parameter_name | default_value | description -------------------------------+---------------+-------------------------------------------------------------------------------- HadoopFSConnectionTimeout | 60 | Timeout for opening TCP connections to HDFS nodex (in seconds) HadoopFSTokenRefreshFrequency | 0 | Time in seconds between renewing HDFS tokens; if 0, renews whenever it expires HadoopFSBlockSizeBytes | 67108864 | HDFS blocksize for Vertica data (3 rows)