
Admin / 2109 views
Contents:
- Spinning Ubuntu VPS
- Fixing locales
- Installing common programs
- Setting PostgreSQL
- Setting up users and permissions
- Virtualenv
- Starting sample Django project
- Configuring PostgreSQL to work with Django
- Setting up folders
- Gunicorn
- Supervisor
- NginX
- Useful commands
Intro
Hello. Today we will be learning how to set up usual Django project.
We will be using such tehnology stack Ubuntu 16.04 x64, Django, PostgreSQL, Gunicorn, Supervisor, Virtualenv and NginX.
For hosting we will be using DigitalOcean
Spinning Ubuntu VPS
Visit DigitalOcean and register account , you can use my invite link (you'll get $10 in credit)
Than add your droplet for example you can use the cheapear one (5$ droplet)
After choosing region where your droplet will be located (choose where your customers will be located) set up name and you've done - you've created your first droplet.
Now check your email in a few minutes there will be a mailed with login info.
Then we will connect to Ubuntu
ssh root@ourServerIpAdress
Fixing locales
Let's start with ensuring that our system is up to date. PS `-y
` means automatic yes to prompts
$ sudo apt-get update
$ sudo apt-get upgrade -y
There is common problem with locales to fix it use this
$ nano /etc/environment
and add this line add the end
LC_ALL="en_US.UTF-8"
Now reboot to finish fixing locales
$ sudo reboot
Installing common programs like top, mc, tree and python-dev etc
Now wait at least 30 seconds and login again to our server and run below command to install all `bone structure` packages
$ sudo apt-get install htop mc tree python3-dev libpq-dev python-dev libxslt1-dev libxml2 -y
Setting PostgreSQL
Installing postgresql
$ sudo apt-get install postgresql postgresql-contrib -y
Now we will create PostgreSQL user and set up database for him.
You need to change `USERNAME
`, `USERPASSWORD
` and `DATABASE_NAME
` to your own values. Note that `USERNAME
` and `DATABASE_NAME
` should be lovercase
$ sudo -u postgres psql --command="create user USERNAME"
$ sudo -u postgres psql --command="alter role USERNAME with password 'USERPASSWORD'"
$ sudo -u postgres psql --command="create database DATABASE_NAME with owner USERNAME template template0 encoding 'UTF8' "
$ sudo -u postgres psql --command="grant all privileges on database DATABASE_NAME to USERNAME"
If you have problems with postgresql try to start cluster, because in some system cluster won't start automatically (remember to change `9.5`
to your postgresql version)
$ pg_createcluster 9.5 main --start
Setting up users and permissions
Creating project directories
$ sudo mkdir -p /var/webapps/testproject/
Here we will create a user for our website, named `testuser
` and assigned to a system group called `webapps`
.
$ sudo groupadd -r webapps
$ sudo useradd -mr -g webapps -s /bin/bash -d /var/webapps/testproject/home testuser
Adding permissions to folders
$ sudo chown -R testuser:users /var/webapps/testproject/
$ sudo chmod -R g+w /var/webapps/testproject/
Virtualenv
Installing package
$ sudo apt-get install python-virtualenv -y
Now login to our `testuser
`, note that `-
` will move you to user's home directory
$ su - testuser
go one directory up
testuser@server:~$ cd ..
and create virtualenv with python 3 (for python 2 use '=python2')
testuser@server:/var/webapps/testproject$ virtualenv --python=python3 env
Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /var/webapps/testproject/env/bin/python3
Also creating executable in /var/webapps/testproject/env/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
than activate it to use all packages from it
testuser@server:/var/webapps/testproject$ source env/bin/activate
and finally create directory where we will be storing source code, in this directory you can either clone your code from git or follow next step
(env) testuser@server:/var/webapps/testproject$ mkdir code
Starting sample Django project
Assuming that you have done all previous steps than we will create sample Django project
Installing Django is as easy as
(env) testuser@server:/var/webapps/testproject$ pip install django
Collecting django
(...)
Installing collected packages: django
Successfully installed django
Starting testproject, note that dot at the end means that we will re use current directory
(env) testuser@server:/var/webapps/testproject$ cd code
(env) testuser@server:/var/webapps/testproject/code$ django-admin startproject testproject .
And test it by running
(env) testuser@server:/var/webapps/testproject/code$ python manage.py runserver 0.0.0.0:8000
Performing system checks...
System check identified no issues (0 silenced).
You have 13 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): admin, auth, contenttypes, sessions.
Run 'python manage.py migrate' to apply them.
March 26, 2017 - 12:57:32
Django version 1.10.6, using settings 'testproject.settings'
Starting development server at http://0.0.0.0:8000/
Quit the server with CONTROL-C.
And you well be able to access it from http://ourServerIpAdress:8000
Configuring PostgreSQL to work with Django
Installing PostgreSQL database adapter
(env) testuser@server:/var/webapps/testproject/code$ pip install psycopg2
You can now configure the databases in your `settings.py
`:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'DATABASE_NAME',
'USER': 'USERNAME',
'PASSWORD': 'USERPASSWORD',
'HOST': 'localhost',
'PORT': '', # Set to empty string for default.
}
}
Than we need to create tables for our newly created database
(env) testuser@server:/var/webapps/testproject/code$ python manage.py makemigrations
(env) testuser@server:/var/webapps/testproject/code$ python manage.py migrate
Setting up folders
Go one directory up
(env) testuser@server:/var/webapps/testproject$
For now if you were following all steps your project directory should look like this
├── code
│ ├── manage.py
│ └── testproject
├── env
└── home
But for real life project you will need folders for logs, configs and etc
For example in `media
` and `static
` we will be storing media files and files from `collectstatic
`
(env) testuser@server:/var/webapps/testproject$ mkdir logs conf
(env) testuser@server:/var/webapps/testproject$ mkdir -p www/media
(env) testuser@server:/var/webapps/testproject$ mkdir -p www/static
And for now it should look like
├── code
│ ├── manage.py
│ └── testproject
├── conf
├── env
├── logs
└── www
├── media
└── static
Gunicorn
In production we won’t be using Django’s single-threaded development server, but a special application server called gunicorn.
Installing it via
(env) testuser@server:/var/webapps/testproject$ pip install gunicorn
Collecting gunicorn
(...)
Installing collected packages: gunicorn
Successfully installed gunicorn
And test it by running
(env) testuser@server:/var/webapps/testproject$ cd code
(env) testuser@server:/var/webapps/testproject/code$ gunicorn testproject.wsgi:application --bind ourServerIpAdress:8001
[2017-03-26 13:01:34 +0000] [11414] [INFO] Starting gunicorn
[2017-03-26 13:01:34 +0000] [11414] [INFO] Listening at: http://ourServerIpAdress:8001 (11414)
[2017-03-26 13:01:34 +0000] [11414] [INFO] Using worker: sync
[2017-03-26 13:01:34 +0000] [11417] [INFO] Booting worker with pid: 11417
Gunicorn is set up and ready to serve your website. Now we will create config file and save it in `conf/gunicorn.conf.py
`.
Note: don't change bind
, we will later use it for NginX
proc_name = 'testproject'
bind = '127.0.0.1:8000'
workers = 3
user = 'testuser'
group = 'webapps'
loglevel = 'debug'
errorlog = '/var/webapps/testproject/logs/gunicorn.error.log'
accesslog = '/var/webapps/testproject/logs/gunicorn.access.log'
As a rule-of-thumb set the workers
according to the following formula: 2 * CPUs + 1. The idea being, that at any given time half of your workers will be busy doing I/O. For a single CPU machine it would give you 3.
Be careful not to forget to change paths and filenames to match your project in `gunicorn.conf.py`
Supervisor
We need to ensure sure that gunicorn is starting automatically with the system and that it can automatically restarts if for some reason it exits unexpectedly.
These is a job for a service called supervisord.
$ sudo apt-get install supervisor -y
Specifically on Ubuntu 16.04 we need to execute this in order to work with supervisor correctly
$ sudo systemctl enable supervisor
$ sudo systemctl start supervisor
In order to force Supervisor to run gunicorn we will need to create `/etc/supervisor/conf.d/testproject.conf
` in `/etc/supervisor/conf.d
` directory with this content
[program:testproject]
environment=LANG="en_US.utf8", LC_ALL="en_US.UTF-8", LC_LANG="en_US.UTF-8"
directory = /var/webapps/testproject/code
command = /var/webapps/testproject/env/bin/gunicorn testproject.wsgi:application -c /var/webapps/testproject/conf/gunicorn.conf.py
user = testuser
loglevel = debug
stdout_logfile = /var/webapps/testproject/logs/supervisor.log
stderr_logfile = /var/webapps/testproject/logs/supervisor.error.log
autostart = true
autorestart = true
redirect_stderr = true
And create log files
$ touch /var/webapps/testproject/logs/supervisor.log
$ touch /var/webapps/testproject/logs/supervisor.error.log
In order to recognise file and run it you should execute this
$ sudo supervisorctl reread
testproject: available
$ sudo supervisorctl update
testproject: added process group
You can now check status of our site
$ sudo supervisorctl status testproject
testproject RUNNING pid 11987, uptime 0:00:35
Your website should now be able to automatically start and restart in the case of failure or system reboot.
NginX
Here we will be setting up Nginx to work with our static files and media
To install
$ sudo apt-get install nginx -y
$ sudo service nginx start
Check status of setup by visiting page http://ourServerIpAdress . Nginx should greet you with words "Welcome to nginx!"
Here we will create Nginx virtual server configuration for our Django website. Every Nginx virtual server should be configured by a file in the `/etc/nginx/sites-available
` directory and enabling them by making symbolic links in the `/etc/nginx/sites-enabled
` directory.
Create file `/etc/nginx/sites-available/testproject
` with following content (remember to adapt it to your project variables and paths)
server {
listen 80;
server_name ourServerIpAdress;
client_max_body_size 15m;
access_log /var/webapps/testproject/logs/nginx-access.log;
error_log /var/webapps/testproject/logs/nginx-error.log;
# Enable Gzip compression
gzip on;
# Compression level (1-9)
gzip_comp_level 5;
# Don't compress anything under 256 bytes
gzip_min_length 256;
# Compress output of these MIME-types
gzip_types
application/atom+xml
application/javascript
application/json
application/rss+xml
application/vnd.ms-fontobject
application/x-font-ttf
application/x-javascript
application/x-web-app-manifest+json
application/xhtml+xml
application/xml
font/opentype
image/svg+xml
image/x-icon
text/css
text/plain
text/x-component;
location /static/ {
alias /var/webapps/testproject/www/static/;
expires 30d;
add_header Pragma public;
add_header Cache-Control "public";
}
location /media/ {
alias /var/webapps/testproject/www/media/;
expires 30d;
add_header Pragma public;
add_header Cache-Control "public";
}
location / {
proxy_pass http://127.0.0.1:8000;
proxy_set_header X-Forwarded-Host $server_name;
proxy_set_header Host $server_name;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header REMOTE_ADDR $remote_addr;
add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"';
# uncomment below to enable custom 404 (and other error pages)
# proxy_intercept_errors on;
}
# uncomment this and create file 404.html in /var/webapps/testproject/www/static
# directory to create custom 404 error page
# error_page 404 /404.html;
# location = /404.html {
# root /var/webapps/testproject/www/static;
# internal;
# }
}
You can also check syntax for NginX via
$ nginx -t
And create symbolik link to enable it in NginX
$ sudo ln -s /etc/nginx/sites-available/testproject /etc/nginx/sites-enabled/testproject
And restart NginX
$ sudo service nginx restart
As for now you should achieve up and running website on http://ourServerIpAdress
Final directory structure in `/var/webapps/testproject
` will be
├── code
│ ├── manage.py
│ └── testproject
├── conf
│ └── gunicorn.conf.py
├── env
├── home
├── logs
│ ├── gunicorn.access.log
│ ├── gunicorn.error.log
│ ├── nginx-access.log
│ ├── nginx-error.log
│ ├── supervisor.error.log
│ └── supervisor.log
└── www
├── media
└── static
Useful commands
To refresh changes in project use this, it will restart gunicorn and supervisor and as a result your changes to source files will be visible on site.
$ sudo service supervisor restart
How to backup database (unsure about this, help wanted)
This command wil create db.json file with your database backup
python manage.py dumpdata --exclude auth.permission --exclude contenttypes > db.json
When you want to load your backup you need to drop database and load in freshly created database
So in Django you need to migrate before loading backup and then
python manage.py loaddata db.json
SIdenotes: this can't be used for large databases, backup and restore databases should be identical in structure.
If you know about Ansible
and Vagrant
you can check my Django Seed Project which is deployable via ansible
.