Tech Incent
django

How to setup django static and media file in aws s3 ?

In this tutorial, we are setting up aws s3 for your Django static and media files. So we can store static files and upload media files more effective manner. Django settings up is the best way that I found to set up

All over world developers highly recommend aws. Using aws s3 for django static files is a really good way to serve and more flexible. When the user comes to website, static files are better loaded for the user. Also, aws s3 has a lot of good reasons.

So it’s two-part of setup…

  1. Aws and s3 setup
  2. Django setup

#1. Aws and s3 setup

  1. Login aws or Create an account here

  2. Create aws user credentials

    • Go to IAM service and create a user with programming access
      s3 adduser
    • Now create a new group
      added-group.
    • Skip next steps of add user in aws and complete create user
    • The note access key and secret key as that are needed for future steps Or download to store key for backup. that will be used as
      AWS_SECRET_ACCESS_KEY = "AKIA6IVWV2ZUUJEZGCLK"
      AWS_ACCESS_KEY_ID = "R5wkbHaPjCewwAFiffT26m1SgStiJ7ziHtRX3HgD"

      downalod keysNote: this aws user, access, secret keys will be removed after publishing this article for aws safety security.

  3. Create new S3 Bucket

    • Go to s3 service and create a bucket. it’s important to choose your region name. Aws s3 is fast but Closes region visitor will browse faster. So if you have a large scale application, it’s matter to choose a bucket region.
      Note: if you want to create same name bucket what I was created or another developer created, you will fail because you can’t create the same name bucket
      create bucket
    • Select default settings for the next steps of the creation process of bucket
    • You need to make permission of cors configuration for this bucket
      <?xml version="1.0" encoding="UTF-8"?>
      <CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
      <CORSRule>
          <AllowedOrigin>*</AllowedOrigin>
          <AllowedMethod>GET</AllowedMethod>
          <AllowedMethod>POST</AllowedMethod>
          <AllowedMethod>PUT</AllowedMethod>
          <AllowedHeader>*</AllowedHeader>
      </CORSRule>
      </CORSConfiguration>

       

    • You must be in your-bucket and add this in cors configuration
      cors-configuration
    • Crate static and media folder in your bucket
      create-folder-in-bucket
    • Great now you are done configuration of your bucket.
  4. Add custom policy or permissions for your IAM user or group

    Custom policy or permissions for django allow uploading. delete, change files.
    I am talking about manage permission or policy for this project. So we can add policy for this user. yes, I can. but is it good? Ans: it’s not. why because if another developer wants to work on this project, he has no permissions for access or maintains s3.
    Now question is, how to permissions to access or modify this project? how to attach policy? don’t worry I will explain. We will attach a policy with the IAM Group. Why? Because if another developer wants to work this project. he will simply add in the group.add-to-group
    We will use this policy, this policy allows to upload, change and delete things

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "s3:ListAllMyBuckets"
                ],
                "Resource": "arn:aws:s3:::*"
            },
            {
                "Effect": "Allow",
                "Action": [
                    "s3:ListBucket",
                    "s3:GetBucketLocation",
                    "s3:ListBucketMultipartUploads",
                    "s3:ListBucketVersions"
                ],
                "Resource": "arn:aws:s3:::<your_bucket_name>"
            },
            {
                "Effect": "Allow",
                "Action": [
                    "s3:*Object*",
                    "s3:ListMultipartUploadParts",
                    "s3:AbortMultipartUpload"
                ],
                "Resource": "arn:aws:s3:::/<your_bucket_name>"
            }
        ]
    }
    
    Add a policy to IAM user Group
    Go to Iam User section and find “Policies” from left sidebar menu
    policy-dashboard
    add-policy-json
    policy-create-confirm
    Great you just create another step is successfully created policy, now we can attach this policy to user or group.
    • Selected your new policy to attach to groupattach-policy-1
    • Now attach with group
      attach-policy-2

So that great. we had created IAM User, Group, S3 Bucket, and Policy or permissions for bucket. that’s it. Now we can set up our django application for s3.

#2 Django setup

  1. First need two packages for setup Django s3 configuration.

    1. boto3
      boto3 is an object-oriented API client to access and manage such as AWS S3 resources and EC2. boto is an official distribution maintained by Amazon.
    2. Django-storages
      Django-storages is an open-source package to manage custom backend storage such as AWS S3, Digital Ocean, Google Cloud Storage,  Apache Libclod, Azure Storage, DropBox etc.it’s make easy to use or maintain backend storage
  2. Install two package

    pip install boto3 django-storages

    In settings.py update INSTALLED_APP

    INSTALLED_APPS = [
       ...
       'storages',
       ...
    ]
  3. Run Migrations

    python manage.py migrate
  4. Configuration for AWS

    You can simply add this configuration to your settings.py file

    AWS_ACCESS_KEY_ID = '<your_access_key>'
    AWS_SECRET_ACCESS_KEY = '<your_secrect_key>'
    AWS_STORAGE_BUCKET_NAME = '<Your_bucket_name>'
    AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
    AWS_S3_OBJECT_PARAMETERS = {
        'CacheControl': 'max-age=86400',
    }
    AWS_LOCATION = 'static'
    
    STATICFILES_DIRS = [
        os.path.join(BASE_DIR, 'static'),
    ]
    
    MEDIA_URL = '//%s.s3.amazonaws.com/media/' % AWS_STORAGE_BUCKET_NAME
    MEDIA_ROOT = MEDIA_URL
    DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage' 
    STATIC_URL = 'https://%s/%s/' % (AWS_S3_CUSTOM_DOMAIN, AWS_LOCATION) 
    STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
    Collectstatic file to AWS
    python manage.py collectstatic

    collected-static files
    It will work. But I configure with separate folder call aws.

  5. best way configure AWS or How I configure AWS settings with a separate folder

    • Add ‘aws’ folder in django_s3(my django_s3, your case may be different with your project name) directory.
      SAJAL-ubuntu:~/Programming/test/django-s3$ tree -d
      .
      ├── core
      │   └── migrations
      ├── django_s3
      │   └── aws   ## crated new aws Configuration folder
      ├── static
      │   ├── css
      │   ├── images
      │   └── js
      └── templates
    • In django_s3 directory make three files calls “__init__.py”, “conf.py”, “utils.py”

      SAJAL-ubuntu:~/Programming/test/django-s3$ tree django_s3
      django_s3
      ├── asgi.py
      ├── aws
      │   ├── conf.py  ## new File
      │   ├── __init__.py  ## new File
      │   └── utils.py  ## new File
      ├── __init__.py
      ├── settings.py
      ├── urls.py
      └── wsgi.py
    • Add configuration in conf.py files
      import datetime
      import os
      
      AWS_USERNAME = 'django_s3' # not matter
      AWS_GROUP_NAME = 'django_s3_group' # not matter
      # Aws config
      AWS_SECRET_ACCESS_KEY = '2IF3Z/k4gni+YIl4RVIA4eOXp0ncBqFsrncqdHqf'
      AWS_ACCESS_KEY_ID = 'AKIA6IVWV2ZU5HKXT2VV'
      AWS_FILE_EXPIRE = 200
      AWS_PRELOAD_METADATA = True
      AWS_QUERYSTRING_AUTH = True
      AWS_DEFAULT_ACL = None
      
      DEFAULT_FILE_STORAGE = 'django_s3.aws.utils.MediaRootS3BotoStorage'
      STATICFILES_STORAGE = 'django_s3.aws.utils.StaticRootS3BotoStorage'
      AWS_STORAGE_BUCKET_NAME = 'djang-s3-bucket'
      S3DIRECT_REGION = 'us-east-1'
      S3_URL = '//%s.s3.amazonaws.com/' % AWS_STORAGE_BUCKET_NAME
      MEDIA_ROOT = S3_URL + 'media/'
      STATIC_URL = S3_URL + 'static/'
      ADMIN_MEDIA_PREFIX = STATIC_URL + 'admin/'
      AWS_S3_FILE_OVERWRITE = False
      two_months = datetime.timedelta(days=61)
      date_two_months_later = datetime.date.today() + two_months
      expires = date_two_months_later.strftime("%A, %d %B %Y 20:00:00 GMT")
      
      AWS_HEADERS = {
          'Expires': expires,
          'Cache-Control': 'max-age=%d' % (int(two_months.total_seconds()),),
      }
      
    • Add Lambda expression for storages in utils.py file
      from storages.backends.s3boto3 import S3Boto3Storage
      StaticRootS3BotoStorage = lambda: S3Boto3Storage(location='static')
      MediaRootS3BotoStorage  = lambda: S3Boto3Storage(location='media')
    • Add conf.py in settings.py
      from django_s3.aws.conf import *

       

    • Great, you configured you aws s3. now final step collectstatic
      python manage.py collectstatic

Get github django s3 code

Conclusion

I found the best way to set up django AWS. in this article I explain what I set up in my real project. please comment if you have other better ways.

Related posts

How to deploy django and react in heroku ?

Tech Incent

Django Send Email with AWS Simple Email Service(SES)

Sajal Mia

How to set up django react together ?

Sajal Mia

Leave a Comment