Kira Grogg · this is what i do

fb g+ tw pin

Useful commands that I forget

Python basics

Some Mac commands

Basic Linux Commands

Words in brackets should be replaced by an instance of that word, e.g. [username] for me would be replaced with grogg

Commands and filenames can often be auto-completed by pressing the tab key. If there are multiple possible completes, pressing tab twice gives a list of the options.

The * symbol acts as a wildcard, so that any sequence of characters can replace it, while the ? symbol acts as a single character wildcard.

A more explanatory tutorial can be found here linkout

Some of these commands can do bad things (I'm looking at you, rm), so be careful. There aren't always checks before premanently removing or overwriting files.

  • Get manual for a given command
     man [command]
  • Go to $HOME directory
    cd
  • Go to directory
    cd [directory]
  • Go to previous directory
    cd -
  • Go up one directory
    cd ..
  • Add current directory (.) to stack
    pushd .
  • Retrieve directory from stack (i.e. go back to saved directory)
    popd
  • Get present working directory, list contents (long form)
    pwd
                ls -l
              
  • I like to create an alias for listing files in my .bashrc file. The alias ll lists files in current (or specified) directory, with long listing, by time stamp in reverse order (newest at bottom), human readable file sizes, colorized (can use -G on Macs)
    alias ll='ls -ltrh --color=tty'
  • Alias can be created for all sorts of commands
  • To run the original command, use a backslash before it:
    \ll
  • To undo an alias
    unalias [aliased command]
  • List files by size, largest at bottom
    ls -lrS
  • Copy file to another directory
    cp [file1] [path/to/newdir/]
  • Move file to another directory
    mv [file1] [path/to/newdir/]
  • Change file name (rename)
    mv [file1] [newfilename]
  • Copy file and append to the file name
    cp [file1]{,[appendName]}
  • Delete a file (permanently!)
    rm [file1]
  • Delete a folder (permanently!)
    rm -r [file1]
  • Delete a file with an interative "are you sure" first (safer)
    rm -i [file1]
    type y for "yes, delete", and n for "no, don't delete"
  • View text file
    more [filename]
    or
    less [filename]
    • q to quit, g to go to top of file, G to go to the bottom,
    • spacebar or f or z to page down, b or w to page up
    • Search forward within the file using /[pattern] and backwards using ?[pattern]. Type n for the next instance of [pattern].
  • View last part of file, with updating (e.g. to see updates to log file for a running program)
    tail -f [filename]
  • Edit a file
    vi [filename]
    or
    emacs [filename]
    The keyboard commands for vi and emacs take some time to get used to!
     
  • Update the modification and access timestamps of a file (will also create an empty file if it does not exist)
    touch [file]
  • Set the modification and access timestamps of a file
    touch -c -t [[CC]YY]MMDDhhmm[.SS] [file]
    bracketed parts are optional, use -h instead of -c for symbolic links
  • Show pathname of possible locations of app
    which [command]
  • Show pathname from which of a command
    which [command]
  • Show quick description of what a command does
     whatis [command] 
  • Find files with a particular extension (or name)
    find -name '*.txt'
  • Find files with a particular phrase in directory home (recursively)
    grep -r Reconstruction /home/
    grep -r "Reconstruction parameters" /home/
  • Resursively find all files with extention *.jpg within the current directory and delete (use carefully!)
    find . -name "*.jpg" -type f -delete
  • Find file by with [word] in the name through the database
    locate [word]
    This command requires the database to be up-to-date, updatedb, (this step is CPU intensive), but locate is much faster than find
  • Symbolic link (like a shortcut)
    ln -s /path/to/largeDataset /other/path/largeDatasetLink

    More info on links linkout

  • Get realtime list of running processes by CPU usage
     top -s 
  • Snapshot report of process statuses for user username
     ps -fu [username]
  • Kill a job by process ID (pid)
    kill [pid]
    kill -9 [pid] (to really make sure it dies)
    Process ID can be found from the top or ps commands
  • Piping |: send output of one command to another, e.g. list files and show in less
    ls | less
  • Get command history
    history
  • Get command history, search for "mv" within
    history | grep mv
  • From command line press keys Ctrl+r and type [word] to recursively search command history for [word]
  • Connect to a remote machine
    ssh -XY [username]@[hostname]
    The -XY includes X-forwarding (to get windows locally)
     
  • Copy a file to/from server
    scp [localfilename] [username]@[hostname]:[remotefilepath/remotefilename]
    scp [username]@[hostname]:[remotefilepath/filename] [localfilename] 
  • Copy a file to e.g., backup drive, archival (i.e., save timestamps)
    rsync -varP --exclude=[names*tobe*excluded] [localfilename] [username]@[hostname]:[remotefilepath/remotefilename]
  • SOCKS tunnel linkout
  • Concatinate files and print
    cat file1.txt file2.txt 
  • Concatinate files and append to 3rd file
    cat file1.txt file2.txt >> file3.txt 
  • Concatinate files and redirect to new file (>), including errors (>&), and run in background (&)
    cat file1.txt file2.txt >& newfile.txt &
  • Show disk space
    df -h
  • Show file sizes and total within current directory
    du -ch
  • Show total size of files within current directory
    du -hs
  • Show total size of files and sort by size (human-readable)
    du -hs | sort -h
  • Show total size of files/folders within current directory but not within subdirectories
    du --max-depth=1 -ch
                du -d 2 -ch  (for mac osx)
              
  • Find all files within pwd that exceed 100MB:
    find . -type f -size +100M
  • Get line-by-line differences between two text files
    diff [file1] [file2] 
  • VNC server (change 1280x1024 to the resolution of your display, change :27 to an unused instance number)
    /usr/bin/vncserver -geometry 1280x1024 -depth 24 :27
              
  • Change permissions of file to "user can execute"
     chmod u+x [filename]
    Typical options are read (r), write (w), and execute (x) for user (u), group (g), or others (o). Use (+) to add permission, and (-) to remove.
  • Change permissions of all files in local directory to "other cannot write":
    find . -type d -exec chmod o-w {} \;
  • Change the owner of a file
    chown [newuser]:[newgroup] [filename]
    You can change all files in a directory by including -R
     
  • Get current user name
     whoami 
  • Get current date in yyyymmdd format
    date +"%Y%m%d"
    There are many other options for displaying the date, see the man page for the syntax.
  • Run the previous command again: !!
  • Run a new command with same arguements as previous command:
    [new_command] !*
  • Run a command with no hangup (so that closing the terminal doesn't terminate the process)
    nohup [command] &> output.txt &
  • In terminal where command is running, Ctrl+z to pause, then to put into background and disown:
    bg
                disown %[jobnumber]
    where [jobnumber] shows up in brackets when pausing (can also use jobs)

More Linux Commands

  • Quickly look up your IP address on this execellent site, plus many other cool tools
  • Automatic ssh instructions
  • Lots of useful command usages: https://www.shell-fu.org/linkout
  • Go to beginning of line Ctrl+a
  • Go to end of line Ctrl+e
  • Search recursively through command history
    Ctrl+r [previously used command]
  • Put running process on pause: ctrl+z
  • Put procces in background: bg
  • Bring (most recent) background process to foreground: fg
  • Show all running jobs: jobs
  • Bring job #2 to foreground: fg %2
  • Kill running job #2: kill %2
  • Run job in background (can then exit without stopping)
    • Run job as usual (esp useful for rsync or something that needs input)
    • Pause it with ctrl+z
    • disown -h %[jobid]
    • bg %[jobid]
  • Find files matching *.m that were modified in the last 50 minutes, skipping the subfolders protonpet and backups:
    find . -path './protonpet' -prune -o -path  './backups' -prune -o -cmin -50 -name '*.m'
  • Find files matching *.m that were modified in the last 50 minutes, skipping the subfolders protonpet and backups, copy to backup/date folder:
    mkdir backups/$(date +"%Y%m%d")
                find . -path './protonpet' -prune -o -path './backups' -prune -o -cmin  -7250 -name '*.m' -exec cp {} backups/$(date +"%Y%m%d")/ \;
              
  • Find files in current directory older than 60 days and move them to subdirectory 2012
    find . -mtime +60 -exec mv {} 2012/ \;
  • find and replace oldword with newword in multiple files
    grep -lr -e '<oldword>' * | xargs sed -i 's/oldword/newword/g'
  • find and replace oldword with newword in multiple files, using a different separator for sed (e.g., if there is a slash in "oldword")
    grep -lr -e '<oldword>' * | xargs sed -i 's#oldword#newword#g'
  • version that works on Mac OSX:
    grep -lr -e '<oldword>' * | xargs sed -i '' 's/oldword/newword/g'
  • Tar
    tar -czf [tarfilename].tar.gz [directory]/ 
  • Untar
    tar -xvf [tarfilename].tar.gz &
  • Supertar
    tar -cjf [tarfilename].tar.bz2 [dirToTar]/ &
  • Unsupertar
    tar -xvjpf [tarfilename].tar.bz2 &
  • Zip a directory
    zip -r [filename].zip [dir/to/zip/to/]
  • Unzip to a directory
    unzip [filename].zip -d [dir/to/unzip/to/]
  • List contents of zip file
    unzip -l [filename].zip
  • Zip all subdirectories of current directory
    for i in */; do zip "${i%/}.zip" -r "$i" ; done
  • Make nested directories all at once
    mkdir -p some/nested/directories/
  • Change symbolic link
    ln -sf /new/target /path/to/symlink
  • One tab will show all options if ambigious (put in .bashrc)
    bind 'set show-all-if-ambiguous'
  • Kill VNC server session #27
    vncserver -kill :27
  • Tunnel
    ssh -ND 1081 [username]@[location.totunnel.to]
  • Quick math command (to put in bashrc)
    math () { /bin/echo "scale=4; ${1}" | bc;}
  • Save history from multiple terminal windows
    shopt -s histappend
  • Save lots of history
    HISTFILESIZE=25000
                HISTSIZE=5000
              
  • Function for quick copy to remote computer (put in .bashrc)
    function scpToOther () { scp -r "${@: 0:$#}" [user_name]@[other.computer.name]:"${@: $#}" ;}
    scpToOther [file names] [/remote/destination/path/]
  • cron instructions linkout
    • Setup: define editor and edit user cron jobs
      export EDITOR=emacs
                      crontab -e
                    
    • List
      crontab -l
    •                 01 * * * * root echo "This command is run at one min past every hour"
                      17 8 * * * root echo "This command is run daily at 8:17 am"
                      17 20 * * * root echo "This command is run daily at 8:17 pm"
                      00 4 * * 0 root echo "This command is run at 4 am every Sunday"
                      * 4 * * Sun root echo "So is this"
                      42 4 1 * * root echo "This command is run 4:42 am every 1st of the month"
                      01 * 19 07 * root echo "This command is run hourly on the 19th of July"
                    
  • Query find and replace for emacs: Esc,Shift+5
    M-%
  • Block selection (column selection, region selection) in emacs;
    • First select the beginning of the section Ctrl+Spc
    • Move cursor to end of selection and kill region with
      Ctrl+x, r, k
      paste/yank with
      Ctrl+x, r, y
  • My .bashrc aliases
    # color and info for the prompt
                export PS1='\[\033[1;34m\]\t \[\033[0;35m\]\W \[\033[1;36m\]> \[\033[0m\]'
              
    # default editor
                export EDITOR=/usr/bin/emacs
              
    # quick open-and-run for .bashrc
                alias ebash='emacs ~/.bashrc ; source ~/.bashrc'
              
    # my favorite alias
                alias ll='ls -ltrh --color=tty'
              
    # -h is for "human readable"
                alias df="df -h"
                alias du="du -h"
              
    alias path='echo -e ${PATH//:/\\n}'
              
    alias tf='tail -f'
                # quick access to remote computer
                alias serv='ssh -XY [user]@[server.com]'
                function scpToServ () { scp -r "${@: 0:$#}" [user]@[server.com]:"${@: $#}" ;}
                function scpFromServ () { scp -r [user]@[server.com]:"$1" "$2" ;}
                alias servTunnel='ssh -ND 1080 [user]@s[erver.com]'
                alias servTunnel2="ssh [user]@[server.com] -L 1081:localhost:2700 \"ssh -ND 2700 [user]@[server2.com]\""
              
    # just a quick way to do simple math
                math () { /bin/echo "scale=4; ${1}" | bc;}
              
    # one [tab] to show possibile completions
                bind 'set show-all-if-ambiguous'
              
                # must save _all_ the history!
                HISTFILESIZE=25000
                HISTSIZE=5000
                shopt -s histappend
                shopt -s cmdhist
              

Mac Stuff


Terminal commands

  • what-are-some-tips-or-tricks-for-terminal-in-mac-os-x linkout
  • Open current directory in Finder:
    open
  • Open file Finder:
    open [filename]
  • Open file Finder with specified app:
    open -a [appname] [filename]
  • Inspect file info (e.g., get the size of an image):
    file [filename]
  • Search for a file with [word] (using spotlight):
    mdfind [word]
  • Use option+click to move cursor to the clicked position
  • pbcopy and pbpaste to copy and paste to/from clipboard
  • Pop into an editor and work on a command line command there:
    Ctrl+x, Ctrl+e
  • Command line calculator
    bc
  • Function for quick math in Terminal
    math () { /bin/echo "scale=4; ${1}" | bc;}
    math 2^3
    math "2^(3+1)"
  • In Terminal Preferences, Settings, Keyboard: change control cursor left (right) to \033b (\033f) to be able to skip back and forth along a command quickly
  • To set the editor (e.g. emacs, vi, etc) for SVN:
    export SVN_EDITOR=vi
  • Password protect a folder linkout
  • Create a disk image (.dmg) of the folder:
    • Open up "Disk Utility"
    • File → New → Disk Image from Folder…
    • Choose the folder you want to protect
      • If you want to be able to write to the folder, "Image Format" must be changed from "compressed" to "read/write"
    • Choose "AES-128" encryption and press Save
      • Be sure to uncheck "Remember password in keychain"
    • Enter your desired password twice
      • Do not forget it, it is not recoverable.
    • Check the created .dmg file for the correct contents
    • Delete the original (unprotected) folder
  • Command line tricks linkout
    • Copy email addresses as `[foo]@[example.com]` instead of `Foo Bar <[foo]@[example.com]>` in Mail.app
    • defaults write com.apple.mail
                      AddressesIncludeNameOnPasteboard
                      -bool false
                    
    • Use plain text mode for new TextEdit documents
      defaults write com.apple.TextEdit
                      RichText -int 0
                    
    • To launch/unlaunch a daemon:
      launchctl (un)load -w ~/Library/LaunchAgents/[name].plist
    • Show/hide hidden files (i.e. those beginning with .)
                      defaults write com.apple.Finder AppleShowAllFiles false
                    
                      defaults write com.apple.Finder AppleShowAllFiles true
                    
  • My (OSX specific) .bash_profle aliases
                alias ebash='emacs ~/.bash_profile ; source ~/.bash_profile'
                alias ll='ls -ltrhG'
                alias ls="ls -G"
                alias ql='qlmanage -p 2>/dev/null'
                alias goWeb='cd /Library/WebServer/Documents;pwd'
                function eject() { diskutil unmount /Volumes/"${1}" ;}
              
Other Issues
  • Problems with X11 forwarding in Mountain Lion???
    • Inside
      ~/.ssh/config
      specify
      XAuthLocation /opt/X11/bin/xauth
                    
      Also, you may have to specifically open XQuartz before connecting.

Other Misc