The following bash script calls iconv on each text file in the input directory and converts them from shift-jis (sjis) to Unicode UTF-8 (utf-8).
#! /bin/bash
inputDir="danu-summaries"
outputDir="danu"
for file in ../$inputDir/*; do
if [ -f $file ]; then
fname=`echo "$file" | cut -d '/' -f3`
echo $fname
iconv -f sjis -t utf-8 ../$inputDir/$fname > ./$outputDir/$fname
fi
done
This blog covers various topics from technical tweaks to latest in computational linguistics, Artificial Intelligence, Machine Learning and Wed Data Mining.
Showing posts with label bash. Show all posts
Showing posts with label bash. Show all posts
Thursday, September 2, 2010
Subscribe to:
Posts (Atom)
Continuously monitor GPU usage
For nvidia GPUs do the follwing: nvidia-smi -l 1
-
If you used Adobe acrobat to annotate a PDF file and you had Acrobat crashed or had to restart the computer abruptly during the annotation p...
-
The following set of commands will plot two data series in a file using point markers and also write the output to a eps file and further co...
-
EXPLORING WEB SCALE LANGUAGE MODELS FOR SEARCH QUERY PROCESSING Jian Huang, Jiangbo Miao, Xiaolong Li, Jianfeng Gao and Kuansan Wang CROSS...