Convert to UTF8 and replace original file

AnuskaWRAnuskaWR
edited November 2019 in General Discussion

I am trying to convert source file to UTF-8 format and save in the same directory (replacing original file) or to a new directory (saving file with the same "original" name ). Can someone help? Please note that source file does not have a static name. I can save the source file to a static name (as in the code below), but I need to retain the value of the original source file.
iconv -f ISO88599 -t utf-8 input_*.txt > /UTF8/input.txt
Your thoughts are appreciated.
-Anna

Answers

  • moshegmosheg Vertica Employee Administrator

    The following script example can handle none empty files.

    #!/bin/bash
    for orig_file_name in $(ls ./SOURCE_DIR); do
       iconv -f UTF-16 -t UTF-8 ./SOURCE_DIR/${orig_file_name} -o ./TARGET_DIR/${orig_file_name}
    done
    
  • Mosheg, thanks! Can I incorporate this code in bat file? How do I make it execute on demand when called from an ETL process?

  • moshegmosheg Vertica Employee Administrator

    In general yes, but the syntax mentioned is for Linux.
    You can load files from Windows client, however, for better performance to have parallel load you need to put your files on shared storage where each node on the cluster can access the files.

  • Thanks mosheg. I think this example can give me direction on how to resolve the issue. The source directory can have multiple files and I need to convert one specific file.

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file