How to set HadoopFSBlockSizeBytes ?

in my hdfs-site.xml dfs.blocksize=536870912, but Vertica do not use this value.
-- hdfs_cluster_config_check --

    Hadoop Conf Path : [/etc/hadoop/conf]
  [OK] HadoopConfDir verified on all nodes
    Connection Timeout (seconds) : [60]
    Token Refresh Frequency (seconds) : [0]
    HadoopFSBlockSizeBytes (MB) : [64]

  [OK] Found [1] hadoop cluster configurations

Best Answer


  • Jim_KnicelyJim_Knicely - Select Field - Administrator

    @ardeks -
    Fyi ... Per the output of the hdfs_cluster_config_check function, "Connection Timeout", "Token Refresh Frequency" and "HadoopFSBlockSizeBytes" are all database configuration parameters that can be viewed by querying the CONFIGURATION_PARAMETERS system table.


    dbadmin=> SELECT parameter_name, default_value, description FROM configuration_parameters WHERE parameter_name IN ('HadoopFSTokenRefreshFrequency', 'HadoopFSConnectionTimeout', 'HadoopFSBlockSizeBytes');
            parameter_name         | default_value |                                  description
     HadoopFSConnectionTimeout     | 60            | Timeout for opening TCP connections to HDFS nodex (in seconds)
     HadoopFSTokenRefreshFrequency | 0             | Time in seconds between renewing HDFS tokens; if 0, renews whenever it expires
     HadoopFSBlockSizeBytes        | 67108864      | HDFS blocksize for Vertica data
    (3 rows)

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file