My girlfriend is a statistician and acquired some data in the form of 200 text files. The problem is that each file had two lines of non-informative description which needed removing. The goal was to process the data to remove the descriptive header. Instead of removing the lines from every file, I wrote this for her
mkdir news
for file in *;
do awk '{if (NR!=1 && NR!=2) {print}}' "$file" > new_"$file";
mv new_"$file" news/"$file";
done
cd news;
rm script;
rm news;
This uses awk to remove lines 1 and 2 from all of the files in the current directory and store the reduced files in the newly created directory “news”. Just run the shell script in the directory containing all the files.