Serving Django through Werkzeug at a subpath

Written by Johannes on April 25, 2013 Categories: Django, Python, uwsgi, Webdevelopment, Werkzeug

Your ads will be inserted here by

Easy AdSense.

Please go to the plugin admin page to
Paste your ad code OR
Suppress this ad slot.

If you want to mix a Django-Application and another WSGI-Application you can serve them through Werkzeug (http://werkzeug.pocoo.org/).

The following snippet serves Django at a subpath and “Hello World” at root.

import os
import sys
 
# save the file at your django project root path for this to work
os.environ["DJANGO_SETTINGS_MODULE"] = "chemocompile.settings"
 
from django.core.wsgi import get_wsgi_application
from werkzeug.wsgi import DispatcherMiddleware
 
def app(environ, start_response):
    status = '200 OK'
    headers = [('Content-type', 'text/plain')]
    start_response(status, headers)
    return ['Hello world']
 
django_project = get_wsgi_application()
 
app = DispatcherMiddleware(app, {
    '/subpath': django_project,
})
 
# this starts the server at port 8080 if the script is run from command line
if __name__ == '__main__':
    from werkzeug.serving import run_simple
    run_simple('', 8080, app)
No Comments

Django View classes and thread safety of mutable class attributes

Written by Johannes on December 4, 2012 Categories: Django

Your ads will be inserted here by

Easy AdSense.

Please go to the plugin admin page to
Paste your ad code OR
Suppress this ad slot.

I found this intriguing bug (or is it a feature? it is somewhat documented in the docs) in the implementation of Djangos View classes.

I wrote the following little class (somewhat simplified):

class TherapyView(TemplateView):
    template_name = "therapy.html"
    medication_form = MedicationForm
 
    medication_initial = []
 
    def prepare_data(self):
        qs = Therapy.objects.get(pk=self.kwargs.get('pk')).medication_set.all()
        for med in qs:
            self.medication_initial.append({
                'substance' : med.substance.title,
                'dosage' : med.dosage,
            })
 
    def get_medication_formset(self):
        """ return an instance of the medication formset """
        from django.forms.formsets import formset_factory
        return formset_factory(
                    self.medication_form,
                    formset=MedicationBaseFormSet
                )(initial=self.medication_initial)
 
    def get(self, request, *args, **kwargs):
        """ handle get request, return HttpResponse object on success """
        context['formset'] = self.get_medication_formset()
        return self.render_to_response(context)

The strange side effect of this was that on every call to the view, medication_initial was still filled with data from the former call. Subsequent calls to the view stacked the formset with more and more entries. This obviously could lead to some unwanted results and I would consider this to be a bug in Django.

But on reading the docs I found out that similar behavior is documented for class attributes that are initialized by View.as_view(). Here the docs warn against passing lists as arguments to as_view() as this isn’t thread safe (see https://docs.djangoproject.com/en/dev/ref/class-based-views/#specification). Apparently references to the class attribute are shared between instances of the View class.

My temporary workaround for The solution to this issue (note the lower discussion: this behavior is inherent in the way Python works) is to initialize the list to None and to overwrite it with a list created in local scope:

class TherapyView(TemplateView):
    # [...]
    medication_initial = None
 
    def prepare_data(self):
        qs = Therapy.objects.get(pk=self.kwargs.get('pk')).medication_set.all()
        medication_initial = []
        for med in qs:
            medication_initial.append({
                'substance' : med.substance.title,
                'dosage' : med.dosage,
            })
        self.medication_initial = medication_initial
 
    def get_medication_formset(self):
        # [...]
 
    def get(self, request, *args, **kwargs):
        # [...]

I haven’t had the opportunity yet to test this for concurrent access to the same view, but for serial access this appears to be stable.

2 Comments

Deploying Django behind nginx with uwsgi and virtualenv

Written by Johannes on November 27, 2012 Categories: Django, Python, uwsgi

Your ads will be inserted here by

Easy AdSense.

Please go to the plugin admin page to
Paste your ad code OR
Suppress this ad slot.

UPDATE (2013-06-10): On a more recent deployment I got errors of the “Python application not found”-kind. It took me a while to figure it out, apparently the current Django stable (1.5 at the point of writing) requires the settings module to be passed to the uwsgi process environment variables. END OF UPDATE

We use nginx to deploy Django on our staging-server and keep Django running in a virtualenv. This allows us to easily separate different projects or release versions of our Software. We use CentOS 6-systems for deployment.

This guide won’t cover deployment on SE-Linux, you have to figure that out for yourself or disable SE-Linux if your application isn’t security critical.

You won’t find nginx in the official CentOS 6 repositories, but there is an official repository provided by the nginx-developers. Add the following file to your /etc/yum.repos.d/ directory:

[root@host ~]$ cat /etc/yum.repos.d/nginx.repo 
[nginx]
name=nginx repo
# base url for 64 bit cent os
baseurl=http://nginx.org/packages/centos/6/x86_64/
# base url for 32 bit cent os
# baseurl=http://nginx.org/packages/centos/6/i686/
gpgcheck=0
enabled=1

and install nginx.

Now add a new user home directory to run your django instance in, and add the nginx-User to the corresponding group:

# create user
[root@host ~]$ useradd -m projectname
# give nginx group level access to user directory
[root@host ~]$ usermod -a -G projectname nginx
# make home directory group readable and executable
[root@host ~]$ chmod g+rx /home/projectname

With that work done, let’s install Django. First install virtualenv, if you haven’t already done so.

# install setuptools if you don't have them installed yet
[root@host ~]$ yum install python-setuptools
[root@host ~]$ easy_install virtualenv

Now switch to your project user, create and activate a virtual environment.

[root@host ~]$ su projectname
# switch to home dir if necessary
[projectname@host root]$ cd
[projectname@host ~]$ virtualenv env
[projectname@host ~]$ source env/bin/activate

You can now install Django into the virtual enviroment with pip. We use git to obtain the latest source code, but you can also use the tarballs provided by the Django project or the internal pip installer.

# obtain latest stable code via git, use -b to check out relevant branch
[(env)projectname@host ~]$ git clone git://github.com/django/django.git -b stable/1.5.x django
# install code from repository with pip
[(env)projectname@host ~]$ pip -e django

pip needs to compile some sources to install uwsgi, so you need development tools (gcc, make, …), libxml2 header and python headers.

# open root shell
[root@host ~]$ yum groupinstall "Development Tools"
[root@host ~]$ yum install python-devel libxml2-devel

Now install uwsgi.

# install uwsgi in virtual environment
[(env)projectname@host ~]$ pip install uwsgi

Copy your Django project code to the home directory and create an uwsgi.ini file.

(env)[project@host ~]$ cat uwsgi.ini 
[uwsgi]
# path to where you put your project code
chdir=/home/project/project
 
# python path to the wsgi module, check if you have one
module=project.wsgi:application
 
# this switch tells uwsgi to spawn a master process,
# that will dynamically spawn new child processes for
# server requests
master=True
# uwsgi stores the pid of your master process here
pidfile=/home/project/master.pid
vacuum=True
# path to your virtual environment
home=/home/project/env/
# path to log file
daemonize=/home/project/log
# this is where you need to point nginx to,
# if you chose to put this in project home make
# sure the home dir is readable and executable by
# nginx
socket=/tmp/uwsgi.sock
 
### SEE UPDATE NOTICE FOR THIS ONE
env = DJANGO_SETTINGS_MODULE=project.settings

Now you need to point nginx to the uwsgi socket:

# make sure /etc/nginx/conf.d/default.conf gets loaded in /etc/nginx/nginx.conf
# in the http section
[root@host ~]$ cat /etc/nginx/conf.d/default.conf 
server {
    listen       80;
    server_name  yourserver.example.org;
 
    location / {
        uwsgi_pass unix:///tmp/uwsgi.sock;
        include uwsgi_params;
    }
 
    # static code has to reside in /home/project/project/static
    # for this to work
    location /static {
        root /home/project/project;
    }
 
    error_page  404              /404.html;
 
    # redirect server error pages to the static page /50x.html
    #
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }
}

Start the uwsgi process and restart your nginx server.

# start uwsgi from virtual environment
(env)[project@host ~]$ uwsgi --ini uwsgi.ini
 
# restart nginx as root
[root@host ~]$ service nginx restart

You can kill the uwsgi master process and its child processes by issuing

[project@host ~]$ uwsgi --stop master.pid
No Comments

Understanding Django Class Based Views – part 2: the TemplateView class

Written by Johannes on September 22, 2012 Categories: Django

You can find the first part of this series here.

Welcome to our second installment of the series. As we are not satisfied with our view merely stroking our ego we want to go a step further. After reading this post we want to be able to have our view stroke our ego with a nicely formatted (standards compliant) HTML-page, and possibly add some context to the whole thing. For this purpose we will progress a little further into django/views/generic/base.py and see what other classes we can find there.

Django provides us with three more classes we will take a look at right now (there’s a fourth one, a RedirectView, which we will pay no attention to for now):

  • ContextMixin(object)
  • TemplateResponseMixin(object)
  • TemplateView(TemplateResponseMixin, ContextMixin, View)

Now, the nomenclature of these classes already suggests that there are different kinds of classes used in Django’s class based views. We have View classes, and Mixin classes. So let’s take a view at what methods these classes provide, to understand their purpose.

We start out with the very simple class ContextMixin:

class ContextMixin(object):
    """
    A default context mixin that passes the keyword arguments received by
    get_context_data as the template context.
    """
 
    def get_context_data(self, **kwargs):
        if 'view' not in kwargs:
            kwargs['view'] = self # add view instance to context
        return kwargs

This little class does nothing more than provide a method that takes a number of keyword arguments and adds the class instance to the values (under the key 'view') before returning the keywords again. We will take a look at how nicely this plays into the framework of View classes, later. First we want to check out the second Mixin class TemplateResponseMixin. This class is a little bit more elaborate and provides some more magic that needs understanding:

class TemplateResponseMixin(object):
    template_name = None
    response_class = TemplateResponse # inherits from HttpResponse
 
    def render_to_response(self, context, **response_kwargs):
        return self.response_class(
            request = self.request,
            template = self.get_template_names(),
            context = context,
            **response_kwargs
        )
 
    def get_template_names(self):
        if self.template_name is None:
            raise ImproperlyConfigured # this will scold us if we
                                       # forget to name a template
        else:
            return [self.template_name]

One thing that should be noted is the use of the TemplateResponse class as return value of render_to_response. I will not go into the details of this class, but note that it inherits from the HttpResponse class with which we already made brief acquaintance in the last part of this tutorial.

We observe, that the Mixin classes provide us with additional functionality on top of the basic handling of requests provided by the View class. The ContextMixin is responsible for preparing the context dictionary, whereas the TemplateResponseMixin provides us with an interface to template rendering as well as returning the appropriate HttpResponse object.

So let’s take a look at the TemplateViewClass and see how it works:

class TemplateView(TemplateResponseMixin, ContextMixin, View):
    def get(self, request, *args, **kwargs):
        context = self.get_context_data(**kwargs) # prepare context data
        return self.render_to_response(context)   # return rendered http
                                                  # response object.

As TemplateView inherits from TemplateResponseMixin, ContextMixin and View we know how the view is dispatched: on a get request to an url pointing to TemplateView.as_view() TemplateView().get() is called. Any keywords arguments passed to the method (e.g., in urls.py) are passed into the context and sent to the initiate a TemplateResponse object in the TemplateResponseMixin. Therefore TemplateView().get() returns the according TemplateResponse.

With this knowledge, let’s put the TemplateView class to some use. Again, we start by inheriting our own personal view class from TemplateView (we could also call TemplateView.as_view() directly from urls.py with some configuration parameters):

# myapp/views.py
class MyTemplateView(TemplateView):
    pass
 
# myproject/urls.py
from django.conf.urls import patterns, include, url
from myapp.views importMyTemplateView
 
urlpatterns = patterns('',
    url(r'^mytemplateview/', MyTemplateView.as_view(), name='mytemplateview'),
)

Firing up our browser to point at http://myserver/mytemplateview results in a very disappointing Exception “ImproperlyConfigured at /mytemplateview/”. But we should have anticipated that if we read the code closely. Obviously, a class named TemplateView should require a template to work. So we’ll do our view the favor and provide a little template and register it with our class:

<html>
    <head>
        <title>My Template View</title>
    </head>
    <body>
        <h1>My Template View</h1>
        <p>Hello Handsome!</p>
    </body>
</html>
# myapp/views.py
class MyTemplateView(TemplateView):
    template_name = "mytemplateview.html"

If we point our browser to /mytemplateview/ again we will be greeted by the familiar uplifting welcome, complimenting our looks. So now, let’s go and add some context to that. For this we simply override the get method of TemplateView (don’t forget to explicitly call the parents get method, though):

# myapp/views.py
class MyTemplateView(TemplateView):
    template_name = "mytemplateview.html"
 
    def get(self, request, *args, **kwargs):
        kwargs['greeting'] = 'Bonjour' # add a keyword
                                       # and value to the context
        return super(MyTemplateView, self).get(request,  # call get
                                               *args,    # method of
                                               **kwargs) # TemplateView
                                                         # class

We can now access that context in the template just as we’re used to:

<html>
    <head>
        <title>My Template View</title>
    </head>
    <body>
        <h1>My Template View</h1>
        <p>{{ greeting }} Handsome!</p>
    </body>
</html>
No Comments

Understanding Django Class Based Views – part 1: the View class

Written by Johannes on September 21, 2012 Categories: Django

Starting with Version 1.3, Django ships with a very powerful class based view system. Django’s (generic) class based views provide a lot of functionality (e.g. List views and Detail views) out of the box. Adapting these views to your own purposes may seem a bit challenging and overwhelming as Django’s documentation throws you headfirst into the water and hides a lot of the magic involved in Django’s class based views and mixin system. Trying to backtrace all the class inheritances involved and the methods flows of a request to the view may confuse you if you are trying to start from understanding what’s going on in the ListView.

Therefore, to understand Django’s system of class based views we start out with the View class, the base class from which all other class based views arederived and work our way up to the more complex classes.

A little disclaimer: I’m writing this tutorial to teach myself just as much as to explain CBV to others. So if you find something to be incomplete or even wrong, don’t tear into me but give me a friendly comment and I will improve it.

This class can be found in django/views/generic/base.py and exhibits the following methods and attributes (read the documentation here: https://docs.djangoproject.com/en/dev/ref/class-based-views/base/):

class View(object):
    """
    Intentionally simple parent class for all views. Only implements
    dispatch-by-method and simple sanity checking.
    """
 
    http_method_names = ['get', 'post', 'put',
                         'delete', 'head', 'options', 'trace']
 
    def __init__(self, **kwargs):
        """
        Constructor. Called in the URLconf; can contain helpful extra
        keyword arguments, and other things.
        """
        # [...]
 
    @classonlymethod
    def as_view(cls, **initkwargs):
        """
        Main entry point for a request-response process.
        """
        # [...]
 
        def view(request, *args, **kwargs):
        # [...]
            return self.dispatch(request, *args, **kwargs)
 
        # [...]
 
        return view
 
    def dispatch(self, request, *args, **kwargs):
        # [...]
        handler = getattr(self,
                          request.method.lower(),
                          self.http_method_not_allowed)
        return handler(request, *args, **kwargs)
 
    def http_method_not_allowed(self, request, *args, **kwargs):
        # [...]
 
    def options(self, request, *args, **kwargs):
        # [...]
 
    def _allowed_methods(self):
        # [...]

This is obviously a simplified and abbreviated version of the class. But let’s see what happens when we take the class to the test and use it as the base of our view class:

# myapp/views.py
 
from django.views.generic import View
 
class MyView(View):
    pass
 
# myproject/urls.py
 
from django.conf.urls import patterns, include, url
from myapp.views import MyView
 
urlpatterns = patterns('',
    url(r'^myview/', MyView.as_view(), name='myview'),
)

View.as_view() returns a callable view function that forwards the request to View.dispatch(). View.dispatch() will try to provide an appropriate handler for the request method.

With the current configuration a request to http://myserver/myview/ is met with an Error 405 – METHOD NOT ALLOWED response.

This may seem disappointing right now, but it is actually just what we should have expected. View.dispatch() will look for an attribute of the View class that coincides with the request type. In our case we have a get request, so View.dispatch() tries to deliver an attribute MyView.get. As this does not exists it falls back and delivers the http_method_not_allowed method.

We’ll get our view to respond more upliftingly very soon. All we need to do is provide an appropriate handler for our MyView class. So let’s try something very simple.

# myapp/views.py
 
class MyView(View):
 
    def get(self, request, *args, **kwargs):
        pass

Point your browser to the view’s url again and you will see Django’s pleasantly colored and nicely formatted debug display scolding you for not returning a HttpResponseObject with your view. A cure for this remedy can easily provided as we still remember HttpResponseObjects from our day one of working with Django:

# myapp/views.py
 
from django.http import HttpResponse
 
class MyView(View):
 
    def get(self, *args, **kwargs):
        return HttpResponse('Hello handsome!')

So we now got our MyView class to respond nicely and upliftingly to our requests. But of course we want our web apps to do more than just tap our egos by complimenting our good looks. For this purpose we will look at the TemplateView and TemplateResponseMixin in the next installment of this series.

From this point on we can already use this view similarly to the method-based view we know from earlier versions of Django, even though this will deprive us of many of the advantages offered to us by Django’s class based view system.

1 Comment

“Oops, an error occurred!” is not a valid error message

Written by Johannes on June 19, 2012 Categories: extbase, fluid, TYPO3, Webdevelopment

I just spent an hour trying to debug my implementation of a Fluid ViewHelper. Using tag-based notation: <namespace:viewhelper arg="val" /> resulted in no rendering of the ViewHelper. Using the more convenient inline-style notation {namespace:viewhelper(arg: “val)} threw the completely non-descript error message: “Oops, an error occurred!”

The answer to the problem: I registered the namespace in the wrong fashion. Instead of writing {namespace mf=Tx_UserMailform_ViewHelpers} I wrote {namespace mf="Tx_UserMailform_ViewHelpers"}.

This is something I would expect a smart templating language to recognize and throw at least a warning…

UPDATE: Read the comment below by Steffen: make sure to have a BE-session running and you’ll get more informative error messages. My rant was somewhat over the top. The syntax error I made actually throws the error message: “Namespace could not be resolved. This exception should never be thrown!” This gives at least a pointer towards the right direction, even though it probably would still have taken me a while to catch the quotes.

One more thing: make sure to register namespaces in partials, too. Apparently it is not enough to register them in the root template.

6 Comments

PHP and non-existing constructors.

Written by Johannes on June 3, 2012 Categories: php

I just found a nice (or lets say not so nice) unexpected behavior in PHP. Apparently anything goes with constructors.

Calling a constructor that hadn’t been defined did not throw any kind of exception, error, warning, whatsoever.

So the following example executes:

class TestClass {
  public function testFunction() {
    return "Hello World!";
  }
}
 
$testClass = new TestClass ("blabla"); // should report an error
echo $testClass->testFunction(); // Hello World

Update: Apparently there is a rationale behind this: the PHP function func_get_arg() allows shell style retrieval of function arguments.

No Comments

How to create a dynamic submenu via a userfunction in TypoScript

Written by Johannes on January 27, 2012 Categories: TYPO3 Tags: , , ,

Let’s assume we have an extension, e.g. a news extension my_news that provides a list view of entries that can be filtered via categories via the query index.php?id=listViewID&tx_my_news['category']=categoryID. In our navigation we want to create a submenu under the list view page, so that our navigation looks like this:

  • Home
  • News
    • Category 1
    • Category 2
    • Category 3
  • About us
  • etc…

We can achieve this effect using stdWrap.cObject = CASE on our TMENU_ITEM:

lib.nav = HMENU
lib.nav {
#this is all basic knowledge to you
  entryLevel = 0
  special = directory
  special.value = 1
  1 = TMENU
  1 {
    wrap = <ul>|</ul>
    expAll = 1
    NO = 1
    NO.wrapItemAndSub = <li>|</li>
    ACT < .NO
  }
 
  2 < .1
  2 {
#from hereon it gets interesting
    wrap = <ul>|</ul>
    NO {
      stdWrap.cObject = CASE
      stdWrap.cObject {
        key.field = uid
 
#default: render default menu
        default = TEXT
        default {
          field = title
          typolink.parameter.field = uid
        }
 
#overwrite menu for news categories
        news = HMENU
        news {
          special = userfunction
          special.userFunc = Tx_MyNews_UserFuncs->news_menu
          special.targetPid = {$listViewID}
 
          1 = TMENU
          1 {
            NO = 1
            NO.ATagTitle.field = title
 
            ACT = 1
            ACT < .NO
          }
          2 < .1
          2 {
            wrap = <ul>|</ul>
            NO = 1
            NO.wrapItemAndSub = <li>|</li>
            ACT < .NO
          }
        }
      }
    }
    ACT < .NO
  }
}
 
# stdWrap.cObject = CASE requires hardcoded PID... so inherit the template here for configuration ###
lib.nav.2.NO.stdWrap.cObject.235 < lib.nav.2.NO.stdWrap.cObject.news
lib.nav.2.ACT < lib.nav.2.NO

The userfunction looks like this

class Tx_MyNews_UserFuncs {
  var $cObj;
 
  /**
  * Returns an array to render an HMENU for the news categories
  *
  * @param   string $content
  * @param   array $conf TypoScript configuration array
  * @return  array news categories menu
  */
  function news_menu ($content,$conf) {
    $GLOBALS['TYPO3_DB']->store_lastBuiltQuery = 1;
 
    // prepare db statements
    $table = 'tx_mynews_domain_model_category';
    $fields = 'title, uid';
    $where = '';
    $res = $GLOBALS['TYPO3_DB']->exec_SELECTquery($fields, $table, $where);
    $title = $GLOBALS['TYPO3_DB']->debug_lastBuiltQuery;
 
    $category_menu = array( );
 
    while ( ($category = $GLOBALS['TYPO3_DB']->sql_fetch_assoc($res)) ) {
      // build category link
      $linkConf = array (
        'useCacheHash' => true,
        'parameter' => $conf['targetPid'],
        'additionalParams' => '&tx_my_news[category]=' . $category['uid']
      );
      $link = $this->cObj->typolink_URL( $linkConf );
 
      // add to menu
      $category_menu[] = array (
        'title' => $category['title'],
        '_OVERRIDE_HREF' => $link
      );
    } 
 
    $link = $this->cObj->typolink_URL( array( 'parameter' => $conf['targetPid'] ) );
    return array ( 
      array ( 
        'title' => 'News',
        '_OVERRIDE_HREF' => $link,
        '_SUB_MENU' => $category_menu
      ) 
    );
  }
}
3 Comments

How to create multiple anonymous Chromium browser session

Written by Johannes on January 15, 2012 Categories: bash-Scripts Tags: , ,