Some shell tricks for more efficient command line use

In this post some Shell / Unix techniques I use over and over again to be more efficient at the command line.

• Aliases (.bashrc) save time. When I type "lt", shell runs "ls -lrth", "ht" lets me go to "/Applications/XAMPP/htdocs". All those seconds add up over time.
• Stay in the same terminal, putting programs in the background. I use this often in Vim: ctrl-z to suspend Vim, do stuff at the command line, go back to Vim with "fg" (when suspending > 1 process, use "jobs" and fg #number to toggle between them)
• Diff two command outputs without creating files. For example, taking out the "title" of one of my pages I could do a diff between the local version and the remote site:

$ diff <(cat header.php) <(ssh bob 'cat ~/public_html/fbreadinglist/header.php')
42a43
>   <title><?php  echo $title; ?></title>

In the parenthesis (subshell) can go any command, quite a powerful technique.

• I use sed daily to search and replace patterns from the command line, but its regex engine is archaic: you have to escape capturing parenthesis (), you don't have +, {}, non-greediness (*?). You can achieve a richer regex engine by using a perl oneliner with: perl -pe 's/pattern/replace/g'. So same syntax but replacing "sed" with "perl -pe". This is a great alternative to have when text parsing from the command line gets more complicated.
• Search history: press Ctrl-R and you get (reverse-i-search)`': -> here you can type a string to search through your shell history. This saves a lot of time compared to hitting many times the up-key.
• Navigation: use "cd -" to go to the previous directory (like a back button). If I need to sidetrack, I type bash to do work in a new shell, when exiting this new shell, I can continue where I left.
• "set -e" in a shell script will cause it to die inmediately when any command does not return 0 (= ok status). This is useful to spot errors soon and not continue with bad assumptions.
• Execute in combination with find: change permission on all 664 files to 664: $ find . -perm 644 -exec chmod 664 {} ;

Another example is to recursively search project files for "todo" actions: $ find . -name '*.py' -exec grep -inH "todo" {} ;

A variant on this is the use of xargs which reads items from pipes / stdin: $ find *py |xargs grep -l todo 2>/dev/null

• Command substitution: same as last example but with a for loop and using this technique: $ for i in $(find . -perm 644); do chmod 664 $i; done

In the () can go any command, I use this often to loop over a the output of a command or script.

• Brace expansion allows you to quickly copy files: $ cp file{,.org}
• Sometimes the "ls" command is too verbose, to see only subdirectories use the -d switch: $ ls -d */
• Use shortcuts to move around on the command line. You can set bash in emacs or vi mode, see here. I use ctrl+a/e/u/l the most to respectively go to start/end line, delete all before cursor, and to clear the screen.
• Change you prompt. Mine: PS1='[u@h W$(__git_ps1 " (%s)")]$ ' == [bbelderbos@Bob-Belderboss-MacBook-Pro posts (master)]$

Note that I use git-prompt.sh to show the current git branch in my prompt.

• Useful to know about mkdir: the -m lets you specify the permissions of the dir: $ mkdir -m 777 dir_with_777_perms ; the -p option lets you create directory trees: $ mkdir -p /deep/file/tree/level4/level5/level6
• No cat is needed for grep:

# extra process
$ cat tmp/a/longfile.txt | grep string 
# this is better / faster
$ grep string tmp/a/longfile.txt

• Redirect stderr to stdout is easy: add 2>&1 to your command: $ find / -name foo 2>&1

What are your favourite Shell / Unix tricks?

Of course the tips above are just a fraction of what is possible. Please share your time-savers in the comments below ...