How to set HadoopFSBlockSizeBytes ?

ardeks

in my hdfs-site.xml dfs.blocksize=536870912, but Vertica do not use this value.
-- hdfs_cluster_config_check --

    Hadoop Conf Path : [/etc/hadoop/conf]
  [OK] HadoopConfDir verified on all nodes
    Connection Timeout (seconds) : [60]
    Token Refresh Frequency (seconds) : [0]
    HadoopFSBlockSizeBytes (MB) : [64]

  [OK] Found [1] hadoop cluster configurations

SruthiA

You can set it using the following query

dbadmin=> SELECT get_config_parameter('HadoopFSBlockSizeBytes');

get_config_parameter

67108864
(1 row)

dbadmin=> SELECT set_config_parameter('HadoopFSBlockSizeBytes',671088000);

set_config_parameter

Parameter set successfully
(1 row)

dbadmin=> SELECT get_config_parameter('HadoopFSBlockSizeBytes');

get_config_parameter

671088000
(1 row)

dbadmin=>

Jim_Knicely

@ardeks -
Fyi ... Per the output of the hdfs_cluster_config_check function, "Connection Timeout", "Token Refresh Frequency" and "HadoopFSBlockSizeBytes" are all database configuration parameters that can be viewed by querying the CONFIGURATION_PARAMETERS system table.

Example:

dbadmin=> SELECT parameter_name, default_value, description FROM configuration_parameters WHERE parameter_name IN ('HadoopFSTokenRefreshFrequency', 'HadoopFSConnectionTimeout', 'HadoopFSBlockSizeBytes');
        parameter_name         | default_value |                                  description
-------------------------------+---------------+--------------------------------------------------------------------------------
 HadoopFSConnectionTimeout     | 60            | Timeout for opening TCP connections to HDFS nodex (in seconds)
 HadoopFSTokenRefreshFrequency | 0             | Time in seconds between renewing HDFS tokens; if 0, renews whenever it expires
 HadoopFSBlockSizeBytes        | 67108864      | HDFS blocksize for Vertica data
(3 rows)