LAPTOP i7

Django Reminder Delete File when deleting object

For Django FileField objects and object of its subclasses, when using the objects delete() method, it will only delete the object record from the database, the file will remain in the appointed location in your servers filesystem.

After a few research, it is a deliberate design in Django to prevent unwanted mess in database when rolling back an record (not sure how this is doable though, leave comments if you know how :D).

So, we have to explicitly delete the file from the filesystem ourselves using

FileFieldObject.file_field.storage.delete(filepath)

A simple implementation to delete the file when delete the file object from db would be like this:

f = FileFieldObject #replace with your models .objects.get() here

storage = f.file_field.storage

#field_field is the property in your model which is an object of class model.FileField

path = f.field_field.path

storage.delete(path) #delete file from disk
f.delete() #delete record from db

Django Remind Image File upload handling and retrieving

The handling of image/file in Django is a bit complicated than PHP, since PHP basically uses HTML syntax to display a file.

Django speed up Sqlite 1000x!

Im working on a project analyzing large code bases. For just messing around, Im using Sqlite. A strange thing happened when updating ~10,000 rows: they were really slow! Sqlite was updating about 10 records a second.

It turns out Sqlite correctly cares about your data, so it makes a transaction around the UPDATE statement, verifying that each and every bit of your data hits disk before returning. For me, if disaster strikes and I lose my tiny database I can recreated in a few seconds, so I decided to live life dangerously, and tell Sqlite to go ahead and try to update data, but dont wait around for it to hit disk.

Heres the magic. Put this before your bulk INSERT/UPDATE statements. Note that it affects the entire session, so you dont want to do this before your valuable data-manipulation commands.

from django.db import connection
if connection.vendor == sqlite:

    connection.cursor().execute(PRAGMA synchronous=OFF)

Three commands sped up my program a zillion percent. The entire 10K rows updated in a fraction of a second, vs minutes. Yay!

Django with Python 3 and MySQL database

I read many folks are having problems using MySQL db driver with Python 3, especially when setting up a Django app. The default Django 1.6.5 is only supporting the MySQLdb driver and that only works with Python 2.

I have been using mysql-connector-python package with Python 3 and it has built-in django support as well. I had ran into trouble like this http://bugs.mysql.com/bug.php?id=73232, but it is fixed now with the latest mysql-connector-python 1.2.3 release. The mysql-connector-python also works with Python 2.7 as well, and its a pure python library without native code, which makes the install much easier.

When installing mysql-connector-python, ensure you allow external hosted files like this

pip install --allow-all-external mysql-connector-python

If you are behind a firewall, use the proxy option

pip install --proxy my_proxy_server --allow-all-external mysql-connector-python

With these, now you can set your Django settings.py file with MySQL engine

DATABASES = {
    default: {
        ENGINE: mysql.connector.django,
        NAME: mydb,
        USER: myuser,
        PASSWORD: mypassword,
    }
}

PS: My initial testing with Django 1.7 also works pretty well with mysql-connector-python. How sweet!

UPDATE 07/15/2015:
The django 1.8 documentation now recommends using mysqlclient package if you want to use Python3 with MySQL backend. It should be a drop in replacement for MySQLdb.

Django Learning Notes

Learning the Python Web Framework, Django, will be posting a series of reflections and notes on the framework in the future.

Django Database Modeling

from django.db import models


class Question(models.Model):
 question_text = models.CharField(max_length=200)
 pub_date = models.DateTimeField(date published)


class Choice(models.Model):
 question = models.ForeignKey(Question)
 choice_text = models.CharField(max_length=200)
 votes = models.IntegerField(default=0)

#1 import the models class from django.db module

#3 defining a class here means defining a new table, the tables will be created if we run "python manage.py migrate"

#3 the Question class inherits from the Model subclass of the models class we imported

#4 attributes defined under a model.pys class will become a column in the generated table
#4 the models.Charfield() is not a method, but a class, by setting up the attribute on line 4, we are instantiating a new CharField object
#4 the "max_length = 200" is an attribute required by the class in order to instantiate the object, I think it is defined in the classes __init__ method.

#5 the DateTimeField is a similar class with an optional attribute to be set as its column name, in here, we are setting it to be "data published" so it is more human-friendly

#7 the Class Choice will create another table named Choice in the Database

#8 This line is very special, it creates a column of Foreign Key, and it will be mapped with the Question table
#8 I am not sure how the mapping is actually done, but I am guessing we will have to specify this field whenever we insert something into the Choice table.
#8 It is because the tutorial is trying to build a polling app, an option without a question will be meaning less...
#8 technically thinking, I suppose the foreign key can be missing, but it would render that row meaningless in our app.
#8 assuming that we are going to use SELECT * from Choice WHERE question == questionID
#8 An option row without a foreign key will never be useful because it will never be selected and displayed into a question polling interface.

#9 Nothing Special Here

#10 similar to other model.field class, but its for integer instead of character.
#10 not sure if the default attribute is required when instantiating an integerField object... I suppose it isnt?

Done for now, will add more in the future.

And yes, Ive abandoned the web.py framework - its nice to use "Pure Python" code as templating language, but well, its not exactly the same, and it is not really that supported.

Django, on the other hand, is a framework that is actively under development and heavy usage by a much larger community - I guess I am going with the Majority this time.

Think of which - by using jQuery, I have actually chosen something that is used by the majority as well - if I want to be special, I wouldve used vanilla javascript, right?

I love the experience and simplicity of jQuery, it saved me so many time, I hope Django would bring a similar experience!

I am kind of excited.

UPDATE:

SO-- Yes, I have just checked the 0001_initial.py(under "poll/migrations) which is generate by running "python manage.py makemigrations polls" in the project directory root, from there, we have a better view to the data structure generated by Django.

Codes are as follow

        migrations.CreateModel(
            name=Choice,
            fields=[
                (id, models.AutoField(verbose_name=ID, serialize=False, auto_created=True, primary_key=True)),
                (choice_text, models.CharField(max_length=200)),
                (votes, models.IntegerField(default=0)),
            ],
            options={
            },
            bases=(models.Model,),
        ),

From here, we can see that the method used to create a database column here added the ID automatically to the model it creates, and it is also used as primary key automatically by default (its not like we can change that without changing the source code anyway)

Also, relational columns, a.k.a foreign keys are added into the model after all the tables are created, which kind of make sense, and it is added with the following code.

        migrations.AddField(
            model_name=choice,
            name=question,
            field=models.ForeignKey(to=polls.Question),
            preserve_default=True,
        ),

So, mystery solved, moving onto next stage!

Django celery tasks timezone error

For some reason when I access the celerys tasks page in the Djangos admin (.../djcelery/taskstate/), I got this error:

Database returned an invalid value in QuerySet.datetimes(). Are time zone definitions for your database and pytz installed?

I have to do the following to get rid of the error:

1. Uncomment these settings:

# CELERY_TIMEZONE = UTC
# CELERY_ENABLE_UTC = True

2. Add this under mysqld section in /etc/mysql/my.cnf:

default-time-zone = "+07:00"

3. Run the following command (without sudo):

$ mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root -p  mysql

4. Restart mysql:

$ sudo service mysql restart

Django Reminder Handling Custom Exceptions

In Django, if we try to get an object that does not exist, the DoesNotExist exception will be raised.

However, the following code does not work out of the box, albeit being the obvious solution to handle such exception:

try:
    img = Image.objects.get(pk = 42)
except DoesNotExist:
    print "Image Does Not Exist"

This is because the DoesNotExist exception is actually an attribute of an model object.

Therefore, we would need to use this exception express to handle it (so we dont have to import the error manually)

except Image.DoesNotExist:
    print "Image Does Not Exist"

To handle more than one possible exception, use the standard Python way to add more exception cases:

except (ValueError, Image.DoesNotExist):
    #do something.

Django ImageField rename file on upload

By default Django keeps the original name of the uploaded file but more than likely you will want to rename it to something else (like the objects id). Luckily, with ImageField or FileField of Django forms, you can assign a callable function to the upload_to parameter to do the renaming. For example:


from django.db import models
from django.utils import timezone
import os
from uuid import uuid4

def path_and_rename(instance, filename):
    upload_to = photos
    ext = filename.split(.)[-1]
    # get filename
    if instance.pk:
        filename = {}.{}.format(instance.pk, ext)
    else:
        # set filename as random string
        filename = {}.{}.format(uuid4().hex, ext)
    # return the whole path to the file
    return os.path.join(upload_to, filename)


class CardInfo(models.Model):
    id_number = models.CharField(max_length=6, primary_key=True)
    name = models.CharField(max_length=255)
    photo = models.ImageField(upload_to=path_and_rename, max_length=255, null=True, blank=True)

In this example, every image that is uploaded will be rename to the CardInfo objects primary key which is id_number.


Reference: http://stackoverflow.com/questions/15140942/django-imagefield-change-file-name-on-upload

django Show form validation error with AJAX and jQuery

I once had a situation when I submited form with AJAX and if there were validation errors I had to put them at the same place as form was normally submited.

Assume that we have edit_user_data method in the views.py. If edit_form is valid we do some really useful things. If its not, we should show user form validation errors (btw, I use uni_forms in that example).

Here is our code:



error_message is a dictionary which contains all form errors. We use JSON so we could have access to the data like server_response.result[key]. Key is the name of the field.

Here is our javascript code:



#edit_form - id of the submited form, url - where we submit the data to. If AJAX request successfuly was proceed by server (it doesnt matter if user submited valid data or not. It means he does submited something) we now could show user if his form is valid or not. If it is not valid, we remove old errors from form (or we will have multiple errors under the fields, probably duplicates). Let also assume that our form contains 5 fields: first_name, last_name, gender, sex, phone. Here you can see nice trick how to check if a key is related to one of our fields:

if (key in {first_name: 1, second_name: 1, gender: 1, sex: 1, phone: 1, }) {}

We also want to show user only one error per field (If there are more, he will be able to see them next time):

error = server_response.result[key][0];

To show error message above the field itself we need to find that field:

field = $("#edit_form").find("#div_id_" + key)

Luckly our key names are the same as field names :) And then attach a message before it:

field.before(

+ error +

);

Where error is description of error, for example, "This field is required". Done. Errors looks nice and native ")