Category Archives: Bash

Bash functions to change to a directory, and immediately list its contents

Add these to your ~/.bashrc file and then load it with either . ~/.bashrc or source ~/.bashrc.

:<<COMMENT
  Change to a directory, and immediately list it's contents, (with "ls -lAF --group-directories-first"). No prompt.

  Short description: Stored in CL_DESC

  Examples
    cl ../venv
    cl ~/myproject/

  See ca
COMMENT
#CL_DESC: For "aliaf" command (with an 'f'). Must end with a newline.
CL_DESC="cl [path]: cd [path], ls\n"
cl()  {
  cd "$1"; ls -lAF --group-directories-first
}

:<<COMMENT
  Change to a directory, and immediately list it's contents, short-form (with "ls -AF --group-directories-first"). No prompt.

  Short description: Stored in CA_DESC

  Examples
    ca ../venv
    ca ~/myproject/

  See cl
COMMENT
#CA_DESC: For "aliaf" command (with an 'f'). Must end with a newline.
CA_DESC="ca [path]: cd [path], ls\n"
ca()  {
  cd "$1"; ls -AF --group-directories-first
}
Advertisements

Bash alias-function for deleting all items from history that match a search

Add this to your ~/.bashrc file and then load it with either . ~/.bashrc or source ~/.bashrc.

:&lt;&lt;COMMENT
   Deletes all lines from the history that match a search string, with a
   prompt. The history file is then reloaded into memory.

   Examples
      hxf &quot;rm -rf&quot;
      hxf ^source

   See:
   - http://unix.stackexchange.com/questions/57924/how-to-delete-commands-in-history-matching-a-given-string
COMMENT
#The unalias prevents odd errors when calling&quot;. ~/.bashrc&quot; (May result in
#&quot;not found&quot; errors. That's okay).
unalias hxf
hxf()  {
   read -r -p &quot;About to delete all items from history that match \&quot;$1\&quot;. Are you sure? [y/N] &quot; response
   response=${response,,}    # tolower
   if [[ $response =~ ^(yes|y)$ ]]
   then
      #Delete all matched items from the file, and duplicate it to a temp
      #location.
      echo -e &quot;grep -v \&quot;$1\&quot; \&quot;$HISTFILE\&quot; &gt; /tmp/history&quot;
      grep -v &quot;$1&quot; &quot;$HISTFILE&quot; &gt; /tmp/history

      #Clear all items in the current sessions history (in memory). This
      #empties out $HISTFILE.
      echo &quot;history -c&quot;
      history -c

      #Overwrite the actual history file with the temp one.
      echo -e &quot;mv /tmp/history \&quot;$HISTFILE\&quot;&quot;
      mv /tmp/history &quot;$HISTFILE&quot;

      #Now reload it.
      echo -e &quot;history -r \&quot;$HISTFILE\&quot;&quot;
      history -r &quot;$HISTFILE&quot;     #Alternative: exec bash
   else
      echo &quot;Cancelled.&quot;
   fi
}

Bash alias-function that accepts a parameter: “srf” for “sudo rm -rf”

Add the following to your .bashrc file, and “reload” it with either of the following:

  • source ~/.bashrc
  • . ~/.bashrc
:&lt;&lt;COMMENT
   Alias-like function for recursive deletion, with are-you-sure prompt.
   Example call:
      srf /home/jeffy/django_files/rest_tutorial/rest_venv/

   With the following setting, this is *not* added to the history:
       export HISTIGNORE=&quot;*rm -r*:srf *&quot;
   - http://superuser.com/questions/232885/can-you-share-wisdom-on-using-histignore-in-bash

   See:
   - Y/N prompt: http://stackoverflow.com/a/3232082/2736496
   - Alias w/param: http://stackoverflow.com/a/7131683/2736496
COMMENT
srf()  {
    #Actual line-breaks required in order to expand the variable.
    #- http://stackoverflow.com/a/4296147/2736496
    read -r -p &quot;About to
sudo rm -rf $1
Are you sure? [y/N] &quot; response
    response=${response,,}    # tolower
    if [[ $response =~ ^(yes|y)$ ]]
    then
        echo -e &quot;sudo rm -rf $1&quot;
        sudo rm -rf &quot;$1&quot;;
    else
        echo &quot;Cancelled.&quot;
    fi
}

One day bash crash course: Functions that return, using echos with newlines in those functions, and converting *nix paths to Windows

I’m in the middle of writing a long, multi-part tutorial on Django authentication. Each part is a completely separate installation of Django, with its own virtualenv and Git repository. The only exception is the singular database, which all parts use (only one is started at a time, and I’m the only user).

After finishing one part, everything must be duplicated into the folder of the next part. This includes the code, the virtualenv and all its installed tools (and config files, like for Gunicorn), as well as things only needed by me (such as the Sublime Text project file, Git directory, and some scripts).

This got me started writing bash scripts to do some of the grunt work. Today I needed to translate a Unix path:

/home/jeffy/django_files/djauth_lifecycle_tutorial/part_04/djauth_root/django_auth_lifecycle/settings.py

to its corresponding Windows path:

Q:\django_files\djauth_lifecycle_tutorial\part_04\djauth_root\django_auth_lifecycle\settings.py

(I edit everything on Windows, and am connected to this Ubuntu machine via SFTP. The goal is to echo a Windows command that I can copy and quickly execute on my local machine.)

The first step was to determine how to search and replace a string, and then set the result to a variable. After many failed attempts, it turns out there are three ways to do it (with thanks to Stack Overflow user Cyrus, and Ask Ubuntu users Seth, and terdon):

pathOnNix="/home/django_auth_lifecycle/urls.py"

#Parameter expansion (alternate separators not permitted)
#https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html#Shell-Parameter-Expansion
path=${pathOnNix//\//\\}
echo "PX: $path"

#Command Substitution, via pipe and sed. Can also be s:'\/':'\\
#https://www.gnu.org/software/bash/manual/html_node/Command-Substitution.html
path="$(echo "${pathOnNix}" | sed -e 's/\//\\/g')"
echo "CS1: $path"

#Can also be s:'\/':'\\
path=$(echo "$pathOnNix" | sed s/'\/'/'\\'/g)
echo "CS2: $path"

#A "Here string": Can also be s:\/:\\:g
#http://en.wikipedia.org/wiki/Here_document#Here_strings
path="$(sed -e 's/\//\\/g' <<< "${pathOnNix}")"
echo "HS: $path"


Output:

PX: \home\django_auth_lifecycle\urls.py
CS1: \home\django_auth_lifecycle\urls.py
CS2: \home\django_auth_lifecycle\urls.py
HS: \home\django_auth_lifecycle\urls.py

This also covers the first step in translating Unix to Windows: Replacing '/' with '\'. In fact, it’s not far from a fully working *nix-to-windows translator:

#!/bin/bash
pathOnNix="/home/django_auth_lifecycle/urls.py"
path=${pathOnNix//\/home\//Q:\\}
path=${path//\//\\}
echo $path

Output: Q:\django_auth_lifecycle\urls.py

However, since I’m doing this multiple times, I decided to place it into a function. Specifically, a function that returns a value, which is very unlike the languages that I’m used to (Java and Python).

#!/bin/bash
nix_to_win() {
   #echo "a $1"
   pathOnC=${1/\/home\/jeffy\//Q:\\}
   #echo "b $pathOnC"
   pathOnC=${pathOnC//\//\\}
   echo "$pathOnC"
}

sourceNixFile="/home/jeffy/django_files/djauth_lifecycle_tutorial/part_03/djauth_root/django_auth_lifecycle/urls.py"
destNixDir="/home/jeffy/django_files/djauth_lifecycle_tutorial/part_04/"

winSrcFile=$(nix_to_win "$sourceNixFile")
winDestDir=$(nix_to_win "$destNixDir")

echo "File to copy: $winSrcFile"
echo "Destination: $winDestDir"

Output:

File to copy: Q:\django_files\djauth_lifecycle_tutorial\part_03\djauth_root\django_auth_lifecycle\urls.py
Destination: Q:\django_files\djauth_lifecycle_tutorial\part_04\

This was not too hard. What was really hard was when I uncommented out those two debugging statements in the function. As mentioned, “returning” a value from a bash function is very different than in Java or Python. Every echo-ed line is returned. So the new output is

File to copy: a /home/jeffy/django_files/djauth_lifecycle_tutorial/part_03/djauth_root/django_auth_lifecycle/urls.py
b Q:\django_files/djauth_lifecycle_tutorial/part_03/djauth_root/django_auth_lifecycle/urls.py
Q:\django_files\djauth_lifecycle_tutorial\part_03\djauth_root\django_auth_lifecycle\urls.py
Destination: a /home/jeffy/django_files/djauth_lifecycle_tutorial/part_04/
b Q:\django_files/djauth_lifecycle_tutorial/part_04/
Q:\django_files\djauth_lifecycle_tutorial\part_04\

Critically, if you change those last four lines to:

echo "File to copy:"
echo $winSrcFile
echo "Destination:"
echo $winDestDir

Then the output becomes:

File to copy:
a /home/jeffy/django_files/djauth_lifecycle_tutorial/part_03/djauth_root/django_auth_lifecycle/urls.py b Q:\django_files/djauth_lifecycle_tutorial/part_03/djauth_root/django_auth_lifecycle/urls.py Q:\django_files\djauth_lifecycle_tutorial\part_03\djauth_root\django_auth_lifecycle\urls.py
Destination:
a /home/jeffy/django_files/djauth_lifecycle_tutorial/part_04/ b Q:\django_files/djauth_lifecycle_tutorial/part_04/ Q:\django_files\djauth_lifecycle_tutorial\part_04\

Now there’s only four lines of output, with the “a” and “b” debugging lines on the same line as the returned value (scroll to the right). What confused me was, just the action of debugging the function caused all the lines to be jumbled together (in my actual situation, I didn’t have those “File to copy”/”Destination” headers which make things clearer…what’s that concept of how looking at a problem changes it?).

Adding newlines explicitely to the debugging statements:

nix_to_win() {
   echo "a $1\n"
   pathOnC=${1/\/home\/jeffy\//Q:\\}
   echo "b $pathOnC\n"
   pathOnC=${pathOnC//\//\\}
   echo "$pathOnC"
}

does not change the output. Trying every possible solution in this Stack Overflow question makes no difference–the lines are still jumbled together.

The problem is, of course, that the debugging and return value are sharing the same output “pipeline”: The return variable itself. Moreso, the nature of bash is, when what is being echoed is not in double quotes, that all newlines and tabs are each replaced with a single space. Unless you really *really* know what you are doing, variable references ($varname) should always be placed in double-quotes. In fact, not doing so results in the ShellCheck linter warning:

SC2086: Double quote to prevent globbing and word splitting.

This is why

echo $winSrcFile

is all on one line, and

echo "$winSrcFile"

is on multiple lines (meaning the newlines are recognized). However, there’s still a problem. While each echo is placed on its own line, explicit newline characters ('\n') are printed literally. To print actual newlines in their place, the "-e" option is required.

nix_to_win() {
   echo "a $1\n"                    #Prints "a [value]\n"
   pathOnC=${1/\/home\/jeffy\//Q:\\}
   echo -e "b $pathOnC\n"           #Prints "b [value][an actual new line]"
   pathOnC=${pathOnC//\//\\}
   echo "$pathOnC"
}

As far as how to actually debug a function without messing up the return value, you can instead output to stderr. All stdout printed in the function (such as via echo or printf), *IS* its return value. stderr still defaults to displaying on the console, but since it is a different “pipeline” than stdout, it does not interfere with the return value.

(If you want the nitty gritty details, check out this fantastic answer by user John1024 to my Stack Overflow question.)

The final version:

#!/bin/bash
pathOnNix="/home/django_auth_lifecycle/urls.py"

nix_to_win() {
   #Print debugging to stderr, to *not* concat onto return value
   #echo "a $1" >&2

   pathOnC=${1//\/home\//Q:\\}
   #echo "b $pathOnC\n" >&2    #debug to stderr

   pathOnC=${pathOnC//\//\\}
   echo "$pathOnC"             #Return value to stdout
}

pathOnNix="/home/django_auth_lifecycle/urls.py"
pathOnWin=$(nix_to_win $pathOnNix)
echo "$pathOnWin"

Output: Q:\django_auth_lifecycle\urls.py

Phew.

With thanks also to Mateo_, and Ademan on askubuntu.com.

(Here’s an XYplorer script that does just the opposite.)

[reddit]