Shell Lessons for Librarians: Reference

Key Points

What is the shell?
  • The shell is powerful

  • The shell can be used to copy, move, and combine multiple files

  • Use the command pwd to find out where you are in on your computer

  • Use the command ls to list directory contents

  • Use flags -l and -lh to guide the output of the ls command

  • Use the command cd to move around your computer

  • Use the man command to check the manual page

  • Use the command mv to rename and move files.

  • Use the command cp to create a file from an existing file.

  • Use the command cat to combine more than one file of the same file type.

  • Use the wildcards * and ? as place holders that delimit which files are to be manipulated by a given an action.

  • Use the command echo to output text to the screen.

  • Use the rm command to delete unwanted files, with caution.

Counting and mining with the shell
  • wc is a command that counts

  • Use the wc command with the flags -w and -l to count the words and lines in a file or a series of files

  • Use the redirector and structure > subdirectory/filename to save results into a subdirectory

  • Use the grep command to search for instances of a string inside files

  • Use grep with the -c flag to count instances of a string, the -i flag to return a case insensitive search for a string, the -v flag to exclude a string from the results, and -w to return a whole word only search

  • Use --file=list.txt to use the file list.txt as the source of strings used in a query

  • Combine these commands and flags to build complex queries in a way that suggests the potential for using the Unix shell to count and mine your research data and research projects

Working with free text
  • Shell tools can be combined to powerful effect

Shell Cheat Sheet

Shell: Basics

pwd print working directory

ls list directory

cd change directory

Shell: Interacting with Files

mkdir make directory

cat send file or files to output (in most cases, print to shell)

head output first 10 lines of a file or files

tail output last 10 lines of a file or files

mv rename or move a file or files. Syntax for renaming a file: mv FILENAME NEWFILENAME

cp copy a file or files. Syntax: cp FILENAME NEWFILENAME

> redirect output. Syntax with cat: cat FILENAME1 FILENAME2 > NEWFILENAME

rm remove a file or files. NB: USE WITH EXTREME CAUTION!!!

Shell: Wildcards

? a placeholder for one character or number

* a placeholder for zero or more characters or numbers

[] defines a class of characters


Shell: Counting and Mining

wc word count

sort sort input

grep global regular expression print