Tech Incent
django

How to optimize your Django application for more speed with ORM?

how-to-optimize-your-django-application-for-more-speed-with-orm

when people use web applications, they want to see fast loaded pages and efficient performance. As a result, speed has become one of the top metrics that people use to determine the design value of a web application.

So here is more teacher and example why is perform is matter?

  • According to Kissmetrics, 47% of consumers expect a web page to load in 2 seconds or less.
  • This same study also found that 40% of people abandon websites that take more than 3 seconds to load.
  • A just one-second slowdown in page load time cost Amazon company $1.6 billion in sales
So how can the Django application load faster? Here is some ways we can optimize …

Fast of all, We need an example of the Django model

class Product(models.Model):
    name = models.CharField(max_length=250)]
    description = models.TextField(blank=True)

    category = models.ForeignKey(Category, on_delete=models.SET_NULL, blank=False, null=False)
    tags = models.ManyToManyField(Tag)
    
    created_at = models.DateTimeField(auto_now_add=True)

    def __str__(self):
        return self.name


class Photo(models.Model):
    image = models.ImageField(upload_to='product')
    product = models.ForeignKey(Product, related_name="photos" on_delete=models.CASCADE)

#1 Optimize SQL query with select_related and prefetch_related

Optimization SQL query / Prefetching most common technique in Django. It’s everywhere in Django. There is two option to prefetch in Django select_related and prefetch_related.

Product.objects.select_releated('category')
...

Product.objects.prefetch_related('photos')
...

Select_related: Select_related working for OneToManyField/ForeignKey, OneToOneField. Category is an OneToMany/ForeignKey Field. So imagine a query set of products. but you have to access the Category of each product. So you looping through the Product. you are doing the Product instance Category. if you are not doing with select_related then you are doing every single Product query set. you have 100 products so you have a query. but you are doing with select_related, django will prefetch all the category of every single product instance.

Prefetch_releated: Prefetch_related is working exactly the same way as select_related except OneToMany/ForeignkeyField and OnoToOneField. And Prefetch also works with reverse ForeignKey. So there are scenarios, Photo has ForeignKey of Product. So Product has reverse ForeignKey relation with Photo as related_name=’phtotos’

Checkout Django Documentation Click Here

#2 Bulk_create

Bulk_Create is another common. Imagine you have a list of Product instances but there are not in the database yet. So you have to save them. So you could that you iterate all the Product, you know for loop with Product and save. So that will insert every single Product with one to one.

So Django gives you bulk_create. So you pass all instances/lists of products that haven’t save in the database yet. Then Django executes a single query of bytes size. Byte size depends on the database(SQLITE_MAX_EXPR_DEPTH=1000,). you can also pass the Byte size parameter. In my opinion, you can create up to 300 records without any problem.

## run test create_product_v1 in 7.621 seconds
## run test create_product_v2 in 0.981 seconds

# Bad
def create_product_v1(products)
    for product in products:
        product.save()

# Good
def create_product_v2(products):
    Product.objects.bulk_create(products)

Might be different from every other machine. but it will huge different load time.

#3 Using _id in Foreign Key fields

Imagine you have a list of products and you want to know quickly if the product has category. Category is nullable and blankly field, So it can be null. So you are going to loop if product has category. but if you only know product has category, you don’t necessarily gonna use category yet. if you wanna know the product have category. So If you do product has category, it will make a query to product table to category table and fetch it but you don’t really need it at the moment. os it”s really inefficient. if you have hundreds of product than if make hundreds of category query.

So normally in Django, you define the field “name” in the case of Product model I define name=models.CharField.  Infield definition, it will create name column in the database table. But ForeignKey Field is different, my field call category in the database column is category_id of product table. Category is represented more like a virtual field. So this is why instances automatically get a foreign key field _id attribute. which you can then use quickly check if this instance has category or none.

How foreign key store in database

ATTACHMENT DETAILS how-foeignkey-field-represent-in-database

Check product has Category

def has_category(product):
    return bool(product.category_id)

#4 Save and Update

Normally when your product.save(), Django will automatically grab all the columns in product table and do an update everything. and you get a lot of fields and a lot of columns it’s gonna do that every single time. It can be quite efficient, Here is Django give you safe update_fields, which you give the fields and django will only update the field, Here is an example

# Bad
def set_new_category_v1(product, category):
    product.category = category
    product.save()
# Good
def set_new_category_v2(product, category):
    product.category = category
    product.save(update_fields=['category])

# Good
def set_new_category_v3(product, category):
    # Call queryset_update to avoid tiggering
    # save signals. It will only run on SQL UPDATE
    Product.objects.fitler(pk=product.pk).update(cattegory=category)

Set_new_category_v1: So I call save on the first version, Django will not just save category it will set name that all other fields are going to be updated.

Set_new_category_v2: But version 2 only sets the category. if the model has 50 fields but the model is only saved category fields.

Set_new_category_v3: There is a third option. The third option is you can use that object query set have one updates method which you can call with the fields you want to update and the values and that will be more efficient than saying because you bypass all the signal

#5 Fetch only what you need

Normally we use Product.objects.all(). and that would also give us all the fields for the product. But normally you don’t necessarily need all the data for the product, you just want to price the name of the product. you don’t need when was the last modified you don’t need the category any of that So Django give you Only method
Example:

# Ran test print_product_name_v1 in 0.1781524542
# Ran test print_product_name_v2 in 0.1715216745

def print_product_name_v1():
    for product in Product.objects.all():
        print(product.name)


def print_product_name_v2():
    for product in Product.objects.only('name'):
        print(product.name)

So here you got two functions, first one print_product_name_v1 function fetches all the data of this product model but al the other hand print_product_name_v2 fetches only the product name field.

But Django gives us two more options

# Run test print_product_name_v3 in 0.07765642537
# Run test print_product_name_v4 in 0.06642537344

def print_product_name_v3():
    for data in Product.objects.values('name):
        # Values: it's return list of dictionary
        print(data['name])

def print_product_name_v4():
    for name in Product.objects.values_list('name', flat=True):
        # values_list: it's return list of Tuple 
        # but if you add flat=true, it's true list of string of name
        print(name)

Values: you pass in the field name, in this case, name. And it returns a list of each object dictionary. So it’s not a normal query set it’s a values query set. As you can see it’s significantly faster than the others (print_product_name_v1 or print_product_name_v2) are 0.17 and v3 is 0.7 also v4 is 0.06. So there is a big difference in performance

Values_list: Value list is a return list of tuple or an interval values list query set of tuples and this tuple includes the values for the name that you are pass in a parameter. we can tell Django we only one single value we can tell Django flat=true. and they return a flat list instead of a list of tuples with only one item name.

Actually, values and valus_list is the same at the database level and ORM Level

Iterator Method

with Open(product.csv, 'wb') as productFile:
   quertset = Product.objects.values_list(
        'pk', 'name', 'description', 'created_at'
    )
   csv_writer = csv.writer(productFile)
   csv_writer.writerow(['ID', 'Name', 'Description', 'Created At'])
   csv_writer.writerows(queryset.iterator())

So here is an example for values_list and iterator to export product. in case I wrote values_list (pk, name, description, crated_at) create CSV write pass in the header row and then iterator of the values of this queryset. So it’s only fetching those four fields and it’s using the iterator of a query set loading the entire thing in memory.

Related posts

How to deploy django and react in heroku ?

Tech Incent

How to setup django static and media file in aws s3 ?

Sajal Mia

How to work with django ajax requests ?

Sajal Mia