The following bash script calls iconv on each text file in the input directory and converts them from shift-jis (sjis) to Unicode UTF-8 (utf-8).
#! /bin/bash
inputDir="danu-summaries"
outputDir="danu"
for file in ../$inputDir/*; do
if [ -f $file ]; then
fname=`echo "$file" | cut -d '/' -f3`
echo $fname
iconv -f sjis -t utf-8 ../$inputDir/$fname > ./$outputDir/$fname
fi
done
This blog covers various topics from technical tweaks to latest in computational linguistics, Artificial Intelligence, Machine Learning and Wed Data Mining.
Subscribe to:
Post Comments (Atom)
Continuously monitor GPU usage
For nvidia GPUs do the follwing: nvidia-smi -l 1
-
If you used Adobe acrobat to annotate a PDF file and you had Acrobat crashed or had to restart the computer abruptly during the annotation p...
-
The following set of commands will plot two data series in a file using point markers and also write the output to a eps file and further co...
-
EXPLORING WEB SCALE LANGUAGE MODELS FOR SEARCH QUERY PROCESSING Jian Huang, Jiangbo Miao, Xiaolong Li, Jianfeng Gao and Kuansan Wang CROSS...
No comments:
Post a Comment