COMMENTS ARE SHADED IN YELLOW.
Making a new directory called dms and copying files from another directory to dms.
i01:~ {996}$ mkdir dms
i01:~ {997}$ ls dms
i01:~ {998}$ cd dms
i01:~/dms {999}$ ls
i01:~/dms {1001}$ cp ~/data/raw/general/*.txt .
i01:~/dms {1002}$ ls
Feb_04_20_16_47_2009_1.txt Feb_04_20_16_47_2009_5.txt
Feb_04_20_16_47_2009_2.txt Feb_04_20_16_47_2009_6.txt
Feb_04_20_16_47_2009_3.txt Feb_04_20_16_47_2009_7.txt
Feb_04_20_16_47_2009_4.txt
Change to the directory of programs and list the program names in it.
i01:~/dms {1003}$ cd /u/local/bin
i01:/u/local/bin {1004}$ ls | more
abaqus
abaqus67
abaqus.q
abaqus.q.parallel
abaqus.q.serial
abaqus.queue
abaqus.queue.parallel
abaqus.queue.serial
amber.q
amber.q.parallel
amber.q.serial
amber.queue
[q] stop displaying the list.
Changing back to dms directory.
i01:/u/local/bin {1005}$ cd ~/dms
Create a file with consecutive numbers from 1 to 10000.
Using more and less commands to view the data.
Using head and tail commands to view only a few lines.
i01:~/dms {1007}$ seq 10000 > large.dat
i01:~/dms {1008}$ wc large.dat
10000 10000 48894 large.dat
i01:~/dms {1012}$ more large.dat
1
...
33
34
35
36
37
38
39
[q]
i01:~/dms {1013}$ less large.dat
1
...
34
35
36
37
38
39
[spacebar]
41
42
43
44
45
...
75
76
77
78
[ctrl-b]
40
39
38
...
4
3
2
1
[q]
i01:~/dms {1014}$ head large.dat
1
2
3
4
5
6
7
8
9
10
i01:~/dms {1015}$ tail large.dat
9991
9992
9993
9994
9995
9996
9997
9998
9999
10000
Remove a file.
i01:~/dms {1016}$ rm large.dat
i01:~/dms {1017}$ ls
Feb_04_20_16_47_2009_1.txt Feb_04_20_16_47_2009_5.txt
Feb_04_20_16_47_2009_2.txt Feb_04_20_16_47_2009_6.txt
Feb_04_20_16_47_2009_3.txt Feb_04_20_16_47_2009_7.txt
Feb_04_20_16_47_2009_4.txt
Paste files to a single file.
i01:~/dms {1018}$ more Feb_04_20_16_47_2009_1.txt
-0.018
1.738
0.013
1.348
-0.021
i01:~/dms {1019}$ more Feb_04_20_16_47_2009_2.txt
-0.614
1.307
0.230
-0.198
0.605
i01:~/dms {1020}$ paste F* > all_cols.txt
i01:~/dms {1021}$ more all_cols.txt
-0.018 -0.614 -1.601 0.683 -0.105 -0.604 -0.768
1.738 1.307 -0.482 1.234 -1.434 -0.368 2.539
0.013 0.230 0.119 0.299 0.451 1.977 0.717
1.348 -0.198 0.533 -0.232 0.656 0.542 -0.619
-0.021 0.605 -0.209 1.310 -1.514 1.733 -0.214
i01:~/dms {1022}$ paste -d"," F* > all_cols.csv
i01:~/dms {1023}$ more all_cols.csv
-0.018,-0.614,-1.601, 0.683,-0.105,-0.604,-0.768
1.738, 1.307,-0.482, 1.234,-1.434,-0.368, 2.539
0.013, 0.230, 0.119, 0.299, 0.451, 1.977, 0.717
1.348,-0.198, 0.533,-0.232, 0.656, 0.542,-0.619
-0.021, 0.605,-0.209, 1.310,-1.514, 1.733,-0.214
i01:~/dms {1024}$ paste -s F* > all_rows.txt
i01:~/dms {1025}$ ls
all_cols.csv Feb_04_20_16_47_2009_3.txt
all_cols.txt Feb_04_20_16_47_2009_4.txt
all_rows.txt Feb_04_20_16_47_2009_5.txt
Feb_04_20_16_47_2009_1.txt Feb_04_20_16_47_2009_6.txt
Feb_04_20_16_47_2009_2.txt Feb_04_20_16_47_2009_7.txt
Copying two files to the current directory and performing merging.
i01:~/dms {1026}$ cp ~/data/raw/general/grade.txt .
i01:~/dms {1027}$ cp ~/data/raw/general/demo.txt .
all_cols.csv Feb_04_20_16_47_2009_1.txt Feb_04_20_16_47_2009_5.txt
all_cols.txt Feb_04_20_16_47_2009_2.txt Feb_04_20_16_47_2009_6.txt
all_rows.txt Feb_04_20_16_47_2009_3.txt Feb_04_20_16_47_2009_7.txt
demo.txt Feb_04_20_16_47_2009_4.txt grade.txt
i01:~/dms {1041}$ more grade.txt
john 79 80
mary 88 56
mark 90 87
nick 82 85
jenny 87 83
zoe 86 90
i01:~/dms {1042}$ more demo.txt
john 24 male
mary 22 female
jenny 20 female
i01:~/dms {1043}$ sort grade.txt >g1.txt
i01:~/dms {1044}$ sort demo.txt >d1.txt
i01:~/dms {1045}$ join g1.txt d1.txt > final.txt
i01:~/dms {1046}$ more final.txt
jenny 87 83 20 female
john 79 80 24 male
mary 88 56 22 female
Use command cat to stack files.
i01:~/dms {1047}$ cat Feb_04_20_16_47_2009_1.txt Feb_04_20_16_47_2009_2.txt >
two.txt
i01:~/dms {1048}$ more two.txt
-0.018
1.738
0.013
1.348
-0.021
-0.614
1.307
0.230
-0.198
0.605
Split files and adding .txt file extension to the newly created smaller files.
i01:~/dms {1050}$ seq 10000 > large.dat
i01:~/dms {1051}$ split large.dat sf
i01:~/dms {1052}$ ls sf*
sfaa sfab sfac sfad sfae sfaf sfag sfah sfai sfaj
i01:~/dms {1053}$ wc sfaa
1000 1000 3893 sfaa
i01:~/dms {1054}$ wc sfaj
1000 1000 5001 sfaj
i01:~/dms {1055}$ for F in sf*; do mv $F $F.txt; done;
i01:~/dms {1056}$ ls sf*
sfaa.txt sfac.txt sfae.txt sfag.txt sfai.txt
sfab.txt sfad.txt sfaf.txt sfah.txt sfaj.txt
Using cut to select columns or fields.
i01:~/dms {1057}$ more all_cols.txt
-0.018 -0.614 -1.601 0.683 -0.105 -0.604 -0.768
1.738 1.307 -0.482 1.234 -1.434 -0.368 2.539
0.013 0.230 0.119 0.299 0.451 1.977 0.717
1.348 -0.198 0.533 -0.232 0.656 0.542 -0.619
-0.021 0.605 -0.209 1.310 -1.514 1.733 -0.214
i01:~/dms {1059}$ cut -c9-12 all_cols.txt >c9_12.txt
i01:~/dms {1060}$ more c9_12.txt
0.61
1.30
0.23
0.19
0.60
i01:~/dms {1080}$ cut -f3 all_cols.txt >c3.txt
i01:~/dms {1081}$ more c3.txt
-1.601
-0.482
0.119
0.533
-0.209
Searching for strings and files.
i01:~/dms {1084}$ grep "jenny" *.txt
d1.txt:jenny 20 female
demo.txt:jenny 20 female
final.txt:jenny 87 83 20 female
g1.txt:jenny 87 83
grade.txt:jenny 87 83
i01:~/dms {1088}$ grep "[a-zA-Z?]" *.txt col_char.txt
-0.01a -0.614 -1.601 0.683 -0.105 -0.604 -0.768
1.348 -0.19S 0.533 -0.232 0.656 0.542 -0.619
i01:~/dms {1091}$ find . -name "*.txt" > all.txt
i01:~/dms {1093}$ more all.txt
./Feb_04_20_16_47_2009_1.txt
./Feb_04_20_16_47_2009_2.txt
./Feb_04_20_16_47_2009_3.txt
./Feb_04_20_16_47_2009_4.txt
./Feb_04_20_16_47_2009_5.txt
./Feb_04_20_16_47_2009_6.txt
./Feb_04_20_16_47_2009_7.txt
./all_cols.txt
./all_rows.txt
./grade.txt
./demo.txt
./g1.txt
./d1.txt
./final.txt
./two.txt
./c3.txt
./c9_12.txt
./all_col_char.txt
./all.txt
Creating and running a batch file. Here batch file has been created to contain a set of commands.
i01:~/data {996}$ more test.bat
#! /bin/bash
seq 10 > small10.txt
more small10.txt
date
Checking if test.bat is executable and changing the mode.
i01:~/data {1002}$ ls -l test.bat
-rw-r--r-- 1 jingy1 jingy1 71 Feb 3 22:46 test.bat
i01:~/data {1003}$ chmod 744 test.bat
i01:~/data {1004}$ ./test.bat
1
2
3
4
5
6
7
8
9
10
Thu Feb 4 12:36:03 PST 2009