Monthly Archives: August 2014

JavaScript form validation for file-uploads–enforcing the extension to be in an approved list.

Building on my last post, here is some new html form-checking functionality, this time for enforcing file uploads to have certain extensions. For example, when uploading a profile picture, you may want to restrict the file-type to be “.gif”, “.jpg”, or “.png”. Although this is no substitution for checking on the server (such as restricting a maximum file size in the nginx configuration), having some client-side protection is a nice supplement.

This function accepts an array of file-types. The file’s name must be one of those types, or a red message is presented, and the submit button is optionally disabled. This required a reorganization of the check-password code–and it’s now-not private function. They are in three separate files, as listed below.

(Known issue: The disable-submit button won’t behave properly if you tell both checkFileExtension and passwordsMatchAlert to disable/enable it. If the current field is valid, the button will be enabled. Same with disabling. While you won’t be prevented from submitting a totally valid form, it means the button will be made pressable if only the current field is valid.)

check_file_extension.js

/**
   Enforces the name of an uploaded file to have a valid extension, as
   contained in an array of extension types. If the extension is
   invalid, the submit button may be optionally disabled.

   This function requries
   `disable_submit_button.disableSubmitButtonIfIdNonNull`

   Derived from (downloaded 8/29/2014)
      http://stackoverflow.com/a/5796710/2736496

   @param  {string[]}  allowed_extensionArrayNoDot  An array of
   extension types, not including the dot prefix. For example:
   ["gif", "jpg", "png"]. *Should* contain unique, non-`null`, and
   non-empty values.
   @param  {string}  message_spanID  ID of the span in which to display
   the message. If `null`, defaults to "message_span_id".
   @param  {string}  submit_buttonIdNullIfDontDisable  If the submit
   button should be disabled unless the passwords match. See
   `disable_submit_button.disableSubmitButtonIfIdNonNull`.
   @param  {string}  message_spanId  ID of the span in which to display
   the "do not match!" message. If null, defaults to "message_span_id".
   @param  {string}  bad_color  The hex color for both the message and
   file-upload element, when the extension is bad. If `null`, defaults to
   `"#ff6666"`.

   @example
      <HTML><HEAD>
         <TITLE>Upload image</TITLE>
         <script src="js/disable_submit_button.js"></script>
         <script src="js/check_file_extension.js"></script>
      </HEAD><BODY>

         <h1>Upload an image</h1>

         <P>Must have an extension of <CODE>.gif</CODE>, <CODE>.jpg</CODE>, or <CODE>.png</CODE>.</P>

         <form id="user_form" method="post" action="/accounts/register/"
                enctype="multipart/form-data">

            <!-- Needed by check_file_extension.js -->
            <span id="message_span_id" class="message_span_id"></span>

            <p><label for="id_profile_picture">Profile picture:</label> <input id="id_profile_picture" name="profile_picture" type="file" /></p>

            <P><I>(To limit the extensions displayed in the open dialog, add</P>

            <BLOCKQUOTE><PRE>accept=".gif,.jpg,.png"</PRE></BLOCKQUOTE>

            <P>to the element)</I></P>

            <!-- Provide a button to click to submit the form. -->
            <input id="submit_button_id" type="submit" name="submit" value="Upload your picture" />
         </form>

      <script language="JavaScript">
         var idFile = "id_profile_picture";  //id of file-upload element
         var idSubmit = "submit_button_id"; //id of the submit button
         function triggerCheckExtension()  {
            //See documentation for additional optional parameters
            checkFileExtension("profile picture", idFile, ["gif", "jpg", "png"], idSubmit);
         }
         document.getElementById("id_profile_picture").addEventListener('change', triggerCheckExtension);
      </script>
      </BODY></HTML>
 **/
function checkFileExtension(file_description, upload_elementId, allowed_extensionArrayNoDot, submit_buttonIdNullIfDontDisable, message_spanID, bad_color) {

   if(message_spanID == null)  {
      message_spanID = "message_span_id";
   }

   var fileElement = null;
   var message = null;  //Message Object span element
   try  {
      fileElement = document.getElementById(upload_elementId);
      message = document.getElementById(message_spanID);
   }  catch(err)  {
      throw  "Attempting to obtain elements by id: upload_elementId (\"" +
      upload_elementId + "\"), message_spanID (\"" + message_spanID + "\"): " + err;
   }

   //Assume valid.
   disableSubmitButtonIfIdNonNull(false, submit_buttonIdNullIfDontDisable);
   fileElement.style.color = null;
   message.style.color = null;
   message.innerHTML = "";

   if(fileElement.value.length == 0)  {
      return;
   }

   var fileExtension = "";
   if (fileElement.value.lastIndexOf(".") > 0) {
      fileExtension = fileElement.value.substring(
         fileElement.value.lastIndexOf(".") + 1, fileElement.value.length);
   }

   var len = allowed_extensionArrayNoDot.length;
   for (var i = 0; i < len; i++) {
      if (fileExtension == allowed_extensionArrayNoDot[i])  {
         return;
      }
   }

   var extensions = "";
   var len = allowed_extensionArrayNoDot.length;
   var lenMinus1 = len - 1
   for (var i = 0; i < len; i++) {
      extensions += allowed_extensionArrayNoDot[i];
      if(i < lenMinus1)  {
         extensions += ", ";
      }
   }

   if(bad_color == null)  {
      bad_color = "#ff6666";
   }

   fileElement.style.color = bad_color;
   message.style.color = bad_color;
   message.innerHTML = "Invalid file-extension for " + file_description +
      ". Only allowed extensions are " + extensions;
   disableSubmitButtonIfIdNonNull(true, submit_buttonIdNullIfDontDisable);
}

passwords_match_alert.js

/**
   @function

   For use on HTML forms containing two password fields, where both must
   match. Provides a green alert when equal, and a red alert when not.
   If either password contains no characters, no alert is given.
   Optionally disables the submit button unless passwords match.

   This function requries `disable_submit_button.disableSubmitButtonIfIdNonNull`

   Derived from (downloaded 8/27/2014)
      http://keithscode.com/tutorials/javascript/3-a-simple-javascript-password-validator.html

   Jeff Epstein: Added the following:

      - all optional parameters
      - The "don't print any alert if one or both fields are empty" feature.
      - The optional "disable submit button feature"
      - Throwing more helpful messages when form elements do not exist.
      - This JSDoc

   With thanks to zkanda on #django irc, for the idea of moving event
   listeners out of form elements.

   @param  {string}  password1_id  "id" identifier (ID) of the
   first-password element.
   @param  {string}  password2_id  ID of the second-password element
   (the "confirm").
   @param  {string}  submit_buttonIdNullIfDontDisable  If the submit
   button should be disabled unless the passwords match. See
   `disable_submit_button.disableSubmitButtonIfIdNonNull`.
   @param  {string}  message_spanId  ID of the span in which to display
   the "do not match!" message. If null, defaults to "message_span_id".
   @param  {string}  good_color  The hex color, including the initial
   hash ('#') to change both the password confirmation, and message-text
   to, when the passwords match. If `null`, defaults to `"#66cc66"`.
   @param  {string}  bad_color  The hex color when the passwords do not
   match. If `null`, defaults to `"#ff6666"`.

   @example
      <HTML><HEAD>
         <TITLE>Create account</TITLE>
         <script src="js/disable_submit_button.js"></script>
         <script src="js/passwords_match_alert.js"></script>
      </HEAD><BODY>
         <h1>Create account</h1>

         <form id="user_form" method="post" action="/accounts/register/">

            <p>Username: <input id="id_username" maxlength="30" name="username" type="text" /></p>
            <p><label for="id_password1">Password:</label> <input id="id_password1" name="password1" type="password" /></p>
            <p><label for="id_password2">Password confirmation:</label> <input id="id_password2" name="password2" type="password" /></p>

            <!-- Where passwordsMatchAlert() writes its message -->
            <span id="message_span_id" class="message_span_id"></span>

            <P><input type="submit" id="id_submit_button" name="submit" value="Create account" /></P>
         </form>

      <script language="JavaScript">
         var idPass1 = "id_password1";  //id of password field 1
         var idPass2 = "id_password2";  //id of password field 2
         var idSubmit = "id_submit_button";  //id of the submit button
         function triggerCheckPass()  {
            //See documentation for additional optional parameters
            passwordsMatchAlert(idPass1, idPass2, idSubmit);
         }
         //"keyup" must be lowercase!
         document.getElementById(idPass1).addEventListener('keyup', triggerCheckPass);
         document.getElementById(idPass2).addEventListener('keyup', triggerCheckPass);
         document.getElementById("id_username").focus();
         document.getElementById(idSubmit).disabled = true;
      </script>
      </BODY></HTML>
 */
function passwordsMatchAlert(password1_id, password2_id, submit_buttonIdNullIfDontDisable, message_spanId, good_color, bad_color)  {
   //Disable submit button by default.
   disableSubmitButtonIfIdNonNull(true, submit_buttonIdNullIfDontDisable);

   if(message_spanId == null)  {
      message_spanId = "message_span_id";
   }


   var pass1 = null;    //Password field elements
   var pass2 = null;
   var message = null;  //Message Object span element
   try  {
      pass1 = document.getElementById(password1_id);
      pass2 = document.getElementById(password2_id);
      message = document.getElementById(message_spanId);
   }  catch(err)  {
      throw  "Attempting to obtain elements by id: password1_id (\"" + password1_id + "\"), password2_id (\"" + password2_id + "\"), message_spanId (\"" + message_spanId + "\"): " + err
   }

   //Compare the values in the password field
   //and the confirmation field
   if(pass1.value.length === 0  ||  pass2.value.length === 0)  {
      //One or both passwords are empty
      pass2.style.backgroundColor = null;//"#FFFFFF";
      message.style.color = null;//"#FFFFFF";
      message.innerHTML = "";
      return;
   }

   if(pass1.value === pass2.value){
      //Set the colors we will be using ...
      if(good_color == null)  {
         good_color = "#66cc66";
      }

      //The passwords match.
      //Set the color to the good color and inform
      //the user that they have entered the correct password
      pass2.style.backgroundColor = good_color;
      message.style.color = good_color;
      message.innerHTML = "Passwords Match!"
      disableSubmitButtonIfIdNonNull(false, submit_buttonIdNullIfDontDisable);

   }  else  {
      if(bad_color == null)  {
         bad_color = "#ff6666";
      }

      //The passwords do not match.
      //Set the color to the bad color and
      //notify the user.
      pass2.style.backgroundColor = bad_color;
      message.style.color = bad_color;
      message.innerHTML = "Passwords Do Not Match!"
   }
}

disable_submit_button.js

/**
   @function
   Disable or enable the submit button if the provided id is non-null.

   @param  {boolean}  is_disabled  If `true`, the button is disabled. If
   `false`, enabled.
   @param  {string}  id_nullIfDontDisable  "id" identifier of the submit
   button, that should be disabled (or enabled). If `null`, this
   function does nothing.
 **/
function disableSubmitButtonIfIdNonNull(is_disabled, id_nullIfDontDisable)  {
   if(id_nullIfDontDisable == null)  {
      return;
   }
   try  {
      document.getElementById(id_nullIfDontDisable).disabled = is_disabled;
   }  catch(err)  {
      throw  "Attempting to get disable element id_nullIfDontDisable (\"" +
      id_nullIfDontDisable + "\"): " + err;
   }
}
Advertisements

Password/password confirm JavaScript form checker, triggered via getElementById().addEventListener()

A javascript function for validating two passwords, which optionally disables the submit button unless the passwords are equal. Source code, including an example-use html page, is below.

empty password
passwords do not match
passwords match

/**
   @private

   Nothing to see here. Move along. Private function needed by the main
   function, just below.
 **/
function _disableSubmitButtonIfIdNonNull(is_disabled, id_nullIfDontDisable)  {
   if(id_nullIfDontDisable == null)  {
      return;
   }
   try  {
      document.getElementById(id_nullIfDontDisable).disabled = is_disabled;
   }  catch(err)  {
      throw  "Attempting to get disable element id_nullIfDontDisable (\"" + id_nullIfDontDisable + "\"): " + err;
   }
}
/**
   @function

   For use on HTML forms containing two password fields, where both must
   match. Provides a green alert when equal, and a red alert when not.
   If either password contains no characters, no alert is given.
   Optionally disables the submit button unless passwords match.

   Derived from (downloaded 8/27/2014)
      http://keithscode.com/tutorials/javascript/3-a-simple-javascript-password-validator.html

   Jeff Epstein: Added the following:

      - all optional parameters
      - The "don't print any alert if one or both fields are empty" feature.
      - The optional "disable submit button feature"
      - Throwing more helpful messages when form elements do not exist.
      - This JSDoc

   With thanks to zkanda on #django irc, for the idea of moving event listeners out of form elements.

   @param  {string}  password1_id  "id" identifier (ID) of the
   first-password element.
   @param  {string}  password2_id  ID of the second-password element
   (the "confirm").
   @param  {string}  submit_buttonIdNullIfDontDisable  ID of the submit
   button that should be disabled, unless the passwords match. If
   `null`, the button is never disabled (or abled).
   @param  {string}  passwordsMatch_messageSpanId  ID of the span in
   which to display the "do not match!" message. If null, defaults to
   "passwords_match_message_span".
   @param  {string}  good_color  The hex color, including the initial
   hash ('#') to change both the password confirmation, and message-text
   to, when the passwords match. If `null`, defaults to `"#66cc66"`.
   @param  {string}  bad_color  The hex color when the passwords do not
   match. If `null`, defaults to `"#ff6666"`.

   @example
      <HTML><HEAD>
         <TITLE>Create account</TITLE>
         <script src="js/passwords_match_alert.js"></script>
      </HEAD><BODY>
         <h1>Create account</h1>

         <form id="user_form" method="post" action="/accounts/register/">

            <p>Username: <input id="id_username" maxlength="30" name="username" type="text" /></p>
            <p><label for="id_password1">Password:</label> <input id="id_password1" name="password1" type="password" /></p>
            <p><label for="id_password2">Password confirmation:</label> <input id="id_password2" name="password2" type="password" /></p>

            <!-- Where passwordsMatchAlert() writes its message -->
            <span id="passwords_match_message_span" class="passwords_match_message_span"></span>

            <P><input type="submit" id="id_submit_button" name="submit" value="Create account" /></P>
         </form>

      <script language="JavaScript">
         var idPass1 = "id_password1";  //id of password field 1
         var idPass2 = "id_password2";  //id of password field 2
         var idSubmit = "id_submit_button";  //id of the submit button
         function triggerCheckPass()  {
            //See documentation for additional optional parameters
            passwordsMatchAlert(idPass1, idPass2, idSubmit);
         }
         //"keyup" must be lowercase!
         document.getElementById(idPass1).addEventListener('keyup', triggerCheckPass);
         document.getElementById(idPass2).addEventListener('keyup', triggerCheckPass);
         document.getElementById("id_username").focus();
         document.getElementById(idSubmit).disabled = true;
      </script>
      </BODY></HTML>
 */
function passwordsMatchAlert(password1_id, password2_id, submit_buttonIdNullIfDontDisable, passwordsMatch_messageSpanId, good_color, bad_color)  {
   //Disable submit button by default.
   _disableSubmitButtonIfIdNonNull(true, submit_buttonIdNullIfDontDisable);

   if(passwordsMatch_messageSpanId == null)  {
      passwordsMatch_messageSpanId = "passwords_match_message_span";
   }


   var pass1 = null;    //Password field elements
   var pass2 = null;
   var message = null;  //Message Object span element
   try  {
      pass1 = document.getElementById(password1_id);
      pass2 = document.getElementById(password2_id);
      message = document.getElementById(passwordsMatch_messageSpanId);
   }  catch(err)  {
      throw  "Attempting to obtain elements by id: password1_id (\"" + password1_id + "\"), password2_id (\"" + password2_id + "\"), passwordsMatch_messageSpanId (\"" + passwordsMatch_messageSpanId + "\"): " + err
   }

   //Set the colors we will be using ...
   if(good_color == null)  {
      good_color = "#66cc66";
   }
   if(bad_color == null)  {
      bad_color = "#ff6666";
   }

   //Compare the values in the password field
   //and the confirmation field
   if(pass1.value.length === 0  ||  pass2.value.length === 0)  {
      //One or both passwords are empty
      pass2.style.backgroundColor = "#FFFFFF";
      message.style.color = "#FFFFFF";
      message.innerHTML = "";
      return;
   }

   if(pass1.value === pass2.value){
      //The passwords match.
      //Set the color to the good color and inform
      //the user that they have entered the correct password
      pass2.style.backgroundColor = good_color;
      message.style.color = good_color;
      message.innerHTML = "Passwords Match!"
      _disableSubmitButtonIfIdNonNull(false, submit_buttonIdNullIfDontDisable);

   }  else  {
      //The passwords do not match.
      //Set the color to the bad color and
      //notify the user.
      pass2.style.backgroundColor = bad_color;
      message.style.color = bad_color;
      message.innerHTML = "Passwords Do Not Match!"
   }
}

My first custom Django template filter   :)

This is getting fun.

from  django import template
from  django.conf import settings

register = template.Library()

@register.filter(is_safe=True)
def multival_to_str(value, arg):
    """
        Given multiple values in "find-what" and "replace-with" lists,
        returns a replacement string from the "replace-with" list that
        has the same index as its corresponding "find-what" element.

        RETURNS
            multival_to_str_base(value, arg)
    """
    #test_multival_to_str_base()
    return  multival_to_str_base(value, arg)

def _get_value_or_crash_for_error(value, bad_args):
    if(settings.DEBUG):
        raise  ValueError("Attempting multival_to_str_base: value=\"" + str(value) + "\", args=\"" + str(bad_args) + "\"")
    else:
        return  value

def multival_to_str_base(value, arg, item_sep=',', list_sep="->", do_ignore_case=False):
    """
        Given multiple values in "find-what" and "replace-with" lists,
        returns a replacement string from the "replace-with" list that
        has the same index as its corresponding "find-what" element.

        arg
            Contains both lists, which must separated by `list_sep` and
            elements within each list must be separated by `item_sep`.
            Each element *should* be unique and at least one character
            in length.

            The replace-with list must either be exactly the same length
            or exactly one greater than the find-what list. If equal in
            length and the `value` is *not in the find-what list*, the
            value itself is returned. However, when the replace-with
            list is one greater, then the final item is returned as the
            "default value".

            It is expected but not verified, that each replace-with
            element contains only HTML-safe content.

            If arg is improperly formatted and DEBUG in settings.py is

                - `True`: This raises a ValueError
                - `False`: `value` is returned.

        item_sep
            The separator required between each element in each list.
            *should* be exactly one character in length.
        list_sep
            The separator that required between the what and with lists.
            *should* not contain item_sep, or be less than one character
            in length.
        do_ignore_case
            If `True`, then case is ignored. If `False`, case is
            required.

        EXAMPLE USAGE
            `<P><IMG SRC="/static/images/{{ album.officiality|multival_to_str:"J,I,U->major,minor,unofficial,broken_image"}}.jpg" height="20"/></P>`

        RETURNS
            Example with default params                       RETURNS
            ------------------------------------------------------------
            B|multival_to_str:A,B,C->one,two,three            two
            D|multival_to_str:A,B,C->one,two,three            D
            D|multival_to_str:A,B,C->one,two,three,four       four

    """
    if(arg is None):
        return  _get_value_or_crash_for_error(value, arg)
    bits = arg.split(list_sep)
    if(len(bits) != 2):
        return  _get_value_or_crash_for_error(value, arg)

    find_whats = bits[0].split(item_sep)
    rplc_withs = bits[1].split(item_sep)

    what_len = len(find_whats)
    with_len = len(rplc_withs)               #max eclusive
    if(what_len not in range((with_len - 1), (with_len + 1))):
        return  _get_value_or_crash_for_error(value, arg)

    if(value not in find_whats):
        if(with_len == what_len):
            return  _get_value_or_crash_for_error(value, arg)

        #what_len is one greater than with_len
        return  rplc_withs[-1]

    #value is in find_whats
    return  rplc_withs[find_whats.index(value)]


def test_multival_to_str_base():
    crash_if_test_fails("two", multival_to_str_base("B", "A,B,C->one,two,three"))
    crash_if_test_fails("D", multival_to_str_base("D", "A,B,C->one,two,three"))
    crash_if_test_fails("four", multival_to_str_base("D", "A,B,C->one,two,three,four"))

def crash_if_test_fails(expected, actual):
    if(expected != actual):
        print("FAILURE: expected=" + str(expected) + ", actual=" + str(actual))

Installing Django on DigitalOcean — Their step-by-step–ahem!–instructions, annotated

This post contains a complete and unedited duplicate of

https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-django-with-postgres-nginx-and-gunicorn (downloaded 8/6/14)

It was extremely difficult following these instructions. Many steps did not work.

The grey italicized boxes

explain how I got through it. The significant change I made was to configure my virtualenv to use Python 3.4 exclusively, although I can’t know if this is what made it so difficult.

(I’m using "Ubuntu 14 x32", and Django 1.7c2.)

I’m on a Windows machine, using Putty/SSH to shell into the server, and SFTP Net Drive Free so I can edit its files on Windows (see the middle of step 8 for details).



How To Install and Configure Django with Postgres, Nginx, and Gunicorn

Prerequisites

This tutorial assumes you have already set up your droplet (VPS) with Debian 7 or a similar distro of Linux (such as Ubuntu). If you have not already done this, please follow the tutorial on setting up a droplet here.

At the end of these pre-requisite instructions, I followed the Ubuntu path, to login as root exactly once, for the purpose of creating the secondary user account.

For convenience, I’ve broken this tutorial into two parts. The first part (steps 1 – 6) covers installation only. If you are a more advanced Django user who just needs help getting things installed you can stop at step 6. If you already have everything installed and just want to know how to configure everything, then skip ahead to step 7. If you feel like you need help from start to finish, then just go through the steps in order and you should have no problems. Let’s get started!

Step One: Update Packages


Before we do anything, we need to make sure that all the packages installed on our VPS are up to date. In order to do this, connect to your desired VPS via SSH and run the following commands:

sudo apt-get update
sudo apt-get upgrade

The first command downloads any updates for packages managed through apt-get. The second command installs the updates that were downloaded. After running the above commands if there are updates to install you will likely be prompted to indicate whether or not you want to install the updates. If this happens, just type “y” and then hit “enter” when prompted.

Step Two: Install and Create Virtualenv


Installing virtualenv is very simple. Just run the command below:

sudo apt-get install python-virtualenv

That’s all there is to it! Now let’s create our virtualenv so we can install Django and other Python packages within it:

    sudo virtualenv /opt/myenv

I’ve changed the previous step to

    sudo virtualenv -p /usr/bin/python3.4 /opt/myenv

in order to force everything to use Python 3.4. Both Python 2.7 and 3.4 are both installed by default but, according to Digital Ocean support, 2.7 cannot be removed. But I need/want 3.4 only. The p switch makes everything installed in the virtualenv use that Python version. Use ls /usr/bin/py* to see which versions are available.

An example of good virtualenv practice, is for each project to have a peer virtualenv folder

    /home/username/myproject
    /home/username/myproject_venv

where most everything is installed redundantly–even generically useful utilities like the IPython shell. This makes it possible, when used along with the above -p option, for IPython to launch with Python 3.4 in one virtutalenv-project, and 2.7 in another.

I’ve also been told to consider using virtualenvwrapper, although I’ve not gotten around to it. It’s for managing a lot of virtualenvs.

Notice that a new directory “myenv” was created in the “/opt” directory. This is where our virtualenv will live. Make sure to replace “/opt/myenv” with the path to where you want your virtualenv installed. I typically put my env’s in /opt, but this is strictly preference. Some people create a directory named “webapps” at the root of the VPS. Choose whatever method makes the most sense to you.

Step Three: Install Django


Now we need to activate our virtualenv so that when we install Python packages they install to our virtualenv. This is how you activate your virtualenv:

source /opt/myenv/bin/activate

You should now see that “(myenv)” has been appended to the beginning of your terminal prompt. This will help you to know when your virtualenv is active and which virtualenv is active should you have multiple virtualenv’s on the VPS.

With your virtualenv active, we can now install Django. To do this, we will use pip, a Python package manager much like easy_install. Here is the command you will run:

pip install django

This step was a bit of an adventure. I’ve documented the details in this post:

http://askubuntu.com/questions/507320/starting-with-ubuntu-in-digitalocean-pip-failing-with-permission-denied

It boils down to this:

    pip install django

is not good enough for two reasons. First, the sudo prefix is required, since (if you’ve followed their instructions) you are not logged in as root. Attempting this without sudo fails with

    Permission denied: ‘/opt/myenv/build’

Second, using pip without an explicit path, installs Django outside of the virtualenv. While it installs without error, it also ignores the virtualenv’s Python version, as discussed in the previous box.

The correct command is


    sudo /opt/myenv/bin/pip install django

You now have Django installed in your virtualenv! Now let’s get our database server going.

Step Four: Install PostgreSQL


Most Django users prefer to use PostgreSQL as their database server. It is much more robust than MySQL and the Django ORM works much better with PostgreSQL than MySQL, MSSQL, or others.

Since we don’t need our virtualenv active for this part, run the following command to deactivate:

deactivate

This will always deactivate whatever virtualenv is active currently. Now we need to install dependencies for PostgreSQL to work with Django with this command:

sudo apt-get install libpq-dev python-dev

Now that you have done this, install PostgreSQL like so:

sudo apt-get install postgresql postgresql-contrib

PostgreSQL is now installed on your machine and ready to roll.

Step Five: Install NGINX


NGINX is an incredibly fast and light-weight web server. We will use it to serve up our static files for our Django app. To install it just run this command:

sudo apt-get install nginx

Keep in mind that you still need to start NGINX, but we will go over this in when we start configuring our VPS.

Step Six: Install Gunicorn


Gunicorn is a very powerful Python WSGI HTTP Server. Since it is a Python package we need to first activate our virtualenv to install it. Here is how we do that:

source /opt/myenv/bin/activate

Make sure you see the added “myenv” at the beginning of your terminal prompt. With your virtualenv now active, run this command:

pip install gunicorn

The correct command is

    sudo /opt/myenv/bin/pip install gunicorn

Gunicorn is now installed within your virtualenv.

If all you wanted was to get everything installed, feel free to stop here. Otherwise, please continue for instructions on how to configure everything to work together and make your app accessible to others on the web.

Step Seven: Configure PostgreSQL


Let’s start off our configuration by working with PostgreSQL. With PostgreSQL we need to create a database, create a user, and grant the user we created access to the database we created. Start off by running the following command:

sudo su - postgres

Your terminal prompt should now say “postgres@yourserver”. If this is the case, then run this command to create your database:

createdb mydb

Your database has now been created and is named “mydb” if you didn’t change the command. You can name your database whatever you would like. Now create your database user with the following command:

createuser -P

The correct command is

    createuser -P nameofuser

and, despite the next sentence, there are no prompts aside from the password.

You will now be met with a series of 6 prompts. The first one will ask you for the name of the new user. Use whatever name you would like. The next two prompts are for your password and confirmation of password for the new user. For the last 3 prompts just enter “n” and hit “enter”. This just ensures your new users only has access to what you give it access to and nothing else. Now activate the PostgreSQL command line interface like so:

psql

Finally, grant this new user access to your new database with this command:

GRANT ALL PRIVILEGES ON DATABASE mydb TO myuser;

You now have a PostgreSQL database and a user to access that database with. Now we can install Django and set it up to use our new database.

Exit out of the database with \q

When exiting, I got this error, but I’m not certain it’s critical:

    could not save history to file “/var/lib/postgresql/.psql_history”: No such file or directory


Before proceeding to step eight, exit out of the postgres user (which you switched to at the start of step seven) with exit.

Step Eight: Create a Django Project


In order to go any further we need a Django project to test with. This will allow us to see if what we are doing is working or not. Change directories into the directory of your virtualenv (in my case /opt/myenv) like so:

cd /opt/myenv

Now make sure your virtualenv is active. If you’re unsure then just run the following command to ensure you’re activated:

source /opt/myenv/bin/activate

With your virtualenv now active, run the following command to start a new Django project:

django-admin.py startproject myproject

Again, must be preceded with sudo, and the full path to django-admin.py must be provided:

    sudo /opt/myenv/bin/django-admin.py startproject myproject


(I can’t get it to work with just django-admin.py (without the path), even if /opt/myenv/bin is in $PATH.)

You should see a new directory called “myproject” inside your virtualenv directory. This is where our new Django project files live.

In order for Django to be able to talk to our database we need to install a backend for PostgreSQL. Make sure your virtualenv is active and run the following command in order to do this:

pip install psycopg2

Another rough patch.

This should be the correct command:

    sudo /opt/myenv/bin/pip install psycopg2

But it fails with

    ./psycopg/psycopg.h:30:20: fatal error: Python.h: No such file or directory

If I had to guess, I would say this is because I started virtualenv with the -p option.

The command that ultimately worked:


    sudo apt-get install python-psycopg2

Change directories into the new “myproject” directory and then into it’s subdirectory which is also called “myproject” like this:

cd /opt/myenv/myproject/myproject

Edit the settings.py file with your editor of choice:

nano settings.py

The correct command is, no surprise,

    sudo nano settings.py

If you are more comfortable editing files on Windows, you can instead use SFTP Net Drive (as mentioned at the top of this document) to mount this remote folder as a drive. On my computer, it’s drive Q:\. Using this, I edit the files on Windows and, every time a file is saved, it is silently FTP’d back to the server. I understand it “works beautifully, or fails miserably”. So far, I’ve only seen the beautiful.

There is one change necessary to make this possible.

All of these files are owned by root. But as per the pre-requisite instructions, you log in as root exactly once, in order to create a secondary user account that has the same permissions as root, with the exception that sudo must be prefixed before every command.

SFTP Net Drive has no way to pass through the sudo prefix. It is therefore necessary to either change the files’

– Permisions (chmod) such that the secondary user has write permissions without sudo, or
– Ownership (chown) to the secondary user.

The first option opens up the files to more users than just the secondary user, so option two is the obivous choice:


    chown -R secondaryuser /opt/myenv/myproject/

Find the database settings and edit them to look like this:

DATABASES = {
    'default': {
        # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
        # Or path to database file if using sqlite3.
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'mydb',

        # The following settings are not used with sqlite3:

        'USER': 'myuser',
        'PASSWORD': 'password',
        'HOST': 'localhost',  # Empty for localhost through domain sockets or
                              #'127.0.0.1' for localhost through TCP.
        'PORT': '',           # Set to empty string for default.
    }
}

The values of NAME, USER, and PASSWORD must be as set in previous steps.

Save and exit the file. Now move up one directory so your in your main Django project directory (/opt/myenv/myproject).

cd /opt/myenv/myproject

Activate your virtualenv if you haven’t already with the following command:

source /opt/myenv/bin/activate

With your virtualenv active, run the following command so that Django can add it’s initial configuration and other tables to your database:

python manage.py syncdb

Another mess, which I’ve documented in this post:

http://stackoverflow.com/questions/25169939/python-manage-py-syncdb-fails-with-improperlyconfigured-no-module-named-psyco/25169940#25169940

The end result is this: Before running syncdb, execute the following:

sudo apt-get build-dep python-psycopg2
sudo /opt/myenv/bin/pip install psycopg2

You should see some output describing what tables were installed, followed by a prompt asking if you want to create a superuser. This is optional and depends on if you will be using Django’s auth system or the Django admin.

Step Nine: Configure Gunicorn


Gunicorn configuration is very specific to your applications needs. I will briefly go over running Gunicorn here with some different settings.

First lets just go over running Gunicorn with default settings. Here is the command to just run default Gunicorn:

gunicorn_django --bind yourdomainorip.com:8001

In Django 1.7c2, this command is both deprecated and fails. Same with the two below. In addition, because of how the browser request makes its way to gunicorn and ultimately Django, the url and port is wrong.

The correct command/url/port is

    gunicorn -b 127.0.0.1:8001 django_test.wsgi:application

In all my 47 minutes of experience with nginx and gunicorn, this is my understanding: The browser url, yourdomainorip.com is first handled by nginx. nginx then passes the request through to the localhost (127.0.0.1), at port 8001. After running the above command, that is being watched by gunicorn. Gunicorn then translates the request as necessary for Python and Django.


Since nginx is configured with no port, the browser can actually call just http://yourdomainorip.com.

Be sure to replace “yourdomainorip.com” with your domain, or the IP address of your VPS if you prefer. Now go to your web browser and visit yourdomainorip.com:8001 and see what you get. You should get the Django welcome screen.

If you look closely at the output from the above command however, you will notice only one Gunicorn worker booted. What if you are launching a large-scale application on a large VPS? Have no fear! All we need to do is modify the command a bit like so:

gunicorn_django --workers=3 --bind yourdomainorip.com:8001

Now you will notice that 3 workers were booted instead of just 1 worker. You can change this number to whatever suits your needs.

Since we ran the command to start Gunicorn as root, Gunicorn is now running as root. What if you don’t want that? Again, we can alter the command above slightly to accomodate:

gunicorn_django --workers=3 --user=nobody --bind yourdomainorip.com:8001

If you want to set more options for Gunicorn, then it is best to set up a config file that you can call when running Gunicorn. This will result in a much shorter and easier to read/configure Gunicorn command.

You can place the configuration file for gunicorn anywhere you would like. For simplicity, we will place it in our virtualenv directory. Navigate to the directory of your virtualenv like so:

cd /opt/myenv

Now open your config file with your preferred editor (nano is used in the example below):

sudo nano gunicorn_config.py

Add the following contents to the file:

command = '/opt/myenv/bin/gunicorn'
pythonpath = '/opt/myenv/myproject'
bind = '127.0.0.1:8001'
workers = 3
user = nobody

As stated in the comments below the original document (in October 2013), the nobody value needs to be surrounded by single (or double) quotes:

    user=’nobody’

I have also added the following (and created the necessary directory):

    error_logfile = “/opt/myenv/myproject/logs/gunicorn_errors.log”
    log_level = “debug”

Save and exit the file. What these options do is to set the path to the gunicorn binary, add your project directory to your Python path, set the domain and port to bind Gunicorn to, set the number of gunicorn workers and set the user Gunicorn will run as.

In order to run the server, this time we need a bit longer command. Enter the following command into your prompt:

/opt/myenv/bin/gunicorn -c /opt/myenv/gunicorn_config.py myproject.wsgi

Must be prefixed with sudo. It can be executed in or out of the virtualenv.

You will notice that in the above command we pass the “-c” flag. This tells gunicorn that we have a config file we want to use, which we pass in just after the “-c” flag. Lastly, we pass in a Python dotted notation reference to our WSGI file so that Gunicorn knows where our WSGI file is.

Running Gunicorn this way requires that you either run Gunicorn in its own screen session (if you’re familiar with using screen), or that you background the process by hitting “ctrl + z” and then typing “bg” and “enter” all right after running the Gunicorn command. This will background the process so it continues running even after your current session is closed. This also poses the problem of needing to manually start or restart Gunicorn should your VPS gets rebooted or were it to crash for some reason. To solve this problem, most people use supervisord to manage Gunicorn and start/restart it as needed. Installing and configuring supervisord has been covered in another article which can be found here.

Lastly, this is by no means an exhaustive list of configuration options for Gunicorn. Please read the Gunicorn documentation found at gunicorn.org for more on this topic.

Step Ten: Configure NGINX


Before we get too carried away, let’s first start NGINX like so:

sudo service nginx start

Since we are only setting NGINX to handle static files we need to first decide where our static files will be stored. Open your settings.py file for your Django project and edit the STATIC_ROOT line to look like this:

    STATIC_ROOT = "/opt/myenv/static/"

This path can be wherever you would like. But for cleanliness, I typically put it just outside my Django project folder, but inside my virtualenv directory.

Now that you’ve set up where your static files will be located, let’s configure NGINX to handle those files. Open up a new NGINX config file with the following command (you may replace “nano” with your editor of choice):

sudo nano /etc/nginx/sites-available/myproject

You can name the file whatever you would like, but the standard is typically to name it something related to the site that you are configuring. Now add the following to the file:

    server {
        server_name yourdomainorip.com;

        access_log off;

        location /static/ {
            alias /opt/myenv/static/;
        }

        location / {
            proxy_pass http://127.0.0.1:8001;
            proxy_set_header X-Forwarded-Host $server_name;
            proxy_set_header X-Real-IP $remote_addr;
            add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"';
        }
    }

In the location block, "alias" can be changed to "root".
Both work.

If the url in your browser is http://yourdomainorip.com, with no url directory, this is received by nginx, which forwards it to the localhost, http://127.0.0.1:8001. The gunicorn server, as configured in step 9, is listening to that address. Since gunicorn was started with

    /opt/myenv/bin/gunicorn -c /opt/myenv/gunicorn_config.py myproject.wsgi

It knows to pass this request onto the Django project named "myproject". Django generates the page, passes it back: through Gunicorn, nginx, and then to the browser.

If, however, the url ends with /static, it instead is totally handled by nginx, which gets the appropriate file from the hard-coded static directory, as configured into the location block.


(If you’re not seeing the page you’re expecting, flush the browser’s cache!)

Save and exit the file. The above configuration has set NGINX to serve anything requested at yourdomainorip.com/static/ from the static directory we set for our Django project. Anything requested at yourdomainorip.com will proxy to localhost on port 8001, which is where we told Gunicorn to run. The other lines ensure that the hostname and IP address of the request get passed on to Gunicorn. Without this, the IP address of every request becomes 127.0.0.1 and the hostname is just your VPS hostname.

Now we need to set up a symbolic link in the /etc/nginx/sites-enabled directory that points to this configuration file. That is how NGINX knows this site is active. Change directories to /etc/nginx/sites-enabled like this:

cd /etc/nginx/sites-enabled

Once there, run this command:

sudo ln -s ../sites-available/myproject

This will create the symbolic link we need so that NGINX knows to honor our new configuration file for our site.

Additionally, remove the default nginx server block:

sudo rm default

We need to restart NGINX though so that it knows to look for our changes. To do this run the following:

sudo service nginx restart

And that’s it! You now have Django installed and working with PostgreSQL and your app is web accessible with NGINX serving static content and Gunicorn serving as your app server. If you have any questions or further advice, be sure to leave it in the comments section.

Easy peasy lemon squeezy!

(Thanks to a whole lot of people on irc.freenode.net#django and #ubuntu for the hand-holding and head-patting)