How to setup django static and media file in aws s3 ?

In this tutorial, we are setting up aws s3 for your Django static and media files. So we can store static files and upload media files more effective manner. Django settings up is the best way that I found to set up

All over world developers highly recommend aws. Using aws s3 for django static files is a really good way to serve and more flexible. When the user comes to website, static files are better loaded for the user. Also, aws s3 has a lot of good reasons.

So it’s two-part of setup…

1. Aws and s3 setup
2. Django setup

#1. Aws and s3 setup

1. Login aws or Create an account here

2. Create aws user credentials

   • Go to IAM service and create a user with programming access
     
   • Now create a new group
     
   • Skip next steps of add user in aws and complete create user
   • The note access key and secret key as that are needed for future steps Or download to store key for backup. that will be used as

     AWS_SECRET_ACCESS_KEY = "AKIA6IVWV2ZUUJEZGCLK"
     AWS_ACCESS_KEY_ID = "R5wkbHaPjCewwAFiffT26m1SgStiJ7ziHtRX3HgD"

     Note: this aws user, access, secret keys will be removed after publishing this article for aws safety security.

3. Create new S3 Bucket
   

   • Go to s3 service and create a bucket. it’s important to choose your region name. Aws s3 is fast but Closes region visitor will browse faster. So if you have a large scale application, it’s matter to choose a bucket region.
     Note: if you want to create same name bucket what I was created or another developer created, you will fail because you can’t create the same name bucket
     
   • Select default settings for the next steps of the creation process of bucket
   • You need to make permission of cors configuration for this bucket

     <?xml version="1.0" encoding="UTF-8"?>
     <CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
     <CORSRule>
         <AllowedOrigin>*</AllowedOrigin>
         <AllowedMethod>GET</AllowedMethod>
         <AllowedMethod>POST</AllowedMethod>
         <AllowedMethod>PUT</AllowedMethod>
         <AllowedHeader>*</AllowedHeader>
     </CORSRule>
     </CORSConfiguration>

      

   • You must be in your-bucket and add this in cors configuration
     
   • Crate static and media folder in your bucket
     
   • Great now you are done configuration of your bucket.

4. Add custom policy or permissions for your IAM user or group

   Custom policy or permissions for django allow uploading. delete, change files.
   I am talking about manage permission or policy for this project. So we can add policy for this user. yes, I can. but is it good? Ans: it’s not. why because if another developer wants to work on this project, he has no permissions for access or maintains s3.
   Now question is, how to permissions to access or modify this project? how to attach policy? don’t worry I will explain. We will attach a policy with the IAM Group. Why? Because if another developer wants to work this project. he will simply add in the group.
   We will use this policy, this policy allows to upload, change and delete things

   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Effect": "Allow",
               "Action": [
                   "s3:ListAllMyBuckets"
               ],
               "Resource": "arn:aws:s3:::*"
           },
           {
               "Effect": "Allow",
               "Action": [
                   "s3:ListBucket",
                   "s3:GetBucketLocation",
                   "s3:ListBucketMultipartUploads",
                   "s3:ListBucketVersions"
               ],
               "Resource": "arn:aws:s3:::<your_bucket_name>"
           },
           {
               "Effect": "Allow",
               "Action": [
                   "s3:*Object*",
                   "s3:ListMultipartUploadParts",
                   "s3:AbortMultipartUpload"
               ],
               "Resource": "arn:aws:s3:::/<your_bucket_name>"
           }
       ]
   }
   

   Add a policy to IAM user Group
   

   Go to Iam User section and find “Policies” from left sidebar menu
   
   
   
   Great you just create another step is successfully created policy, now we can attach this policy to user or group.

   • Selected your new policy to attach to group
   • Now attach with group
     

So that great. we had created IAM User, Group, S3 Bucket, and Policy or permissions for bucket. that’s it. Now we can set up our django application for s3.

#2 Django setup

1. First need two packages for setup Django s3 configuration.

   1. boto3
      boto3 is an object-oriented API client to access and manage such as AWS S3 resources and EC2. boto is an official distribution maintained by Amazon.
   2. Django-storages
      Django-storages is an open-source package to manage custom backend storage such as AWS S3, Digital Ocean, Google Cloud Storage,  Apache Libclod, Azure Storage, DropBox etc.it’s make easy to use or maintain backend storage

2. Install two package

   pip install boto3 django-storages

   In settings.py update INSTALLED_APP

   INSTALLED_APPS = [
      ...
      'storages',
      ...
   ]

3. Run Migrations

   python manage.py migrate

4. Configuration for AWS

   You can simply add this configuration to your settings.py file

   AWS_ACCESS_KEY_ID = '<your_access_key>'
   AWS_SECRET_ACCESS_KEY = '<your_secrect_key>'
   AWS_STORAGE_BUCKET_NAME = '<Your_bucket_name>'
   AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
   AWS_S3_OBJECT_PARAMETERS = {
       'CacheControl': 'max-age=86400',
   }
   AWS_LOCATION = 'static'
   
   STATICFILES_DIRS = [
       os.path.join(BASE_DIR, 'static'),
   ]
   
   MEDIA_URL = '//%s.s3.amazonaws.com/media/' % AWS_STORAGE_BUCKET_NAME
   MEDIA_ROOT = MEDIA_URL
   DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage' 
   STATIC_URL = 'https://%s/%s/' % (AWS_S3_CUSTOM_DOMAIN, AWS_LOCATION) 
   STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'

   Collectstatic file to AWS
   

   python manage.py collectstatic

   
   It will work. But I configure with separate folder call aws.

5. best way configure AWS or How I configure AWS settings with a separate folder

   • Add ‘aws’ folder in django_s3(my django_s3, your case may be different with your project name) directory.

     SAJAL-ubuntu:~/Programming/test/django-s3$ tree -d
     .
     ├── core
     │   └── migrations
     ├── django_s3
     │   └── aws   ## crated new aws Configuration folder
     ├── static
     │   ├── css
     │   ├── images
     │   └── js
     └── templates

   • In django_s3 directory make three files calls “__init__.py”, “conf.py”, “utils.py”
     

     SAJAL-ubuntu:~/Programming/test/django-s3$ tree django_s3
     django_s3
     ├── asgi.py
     ├── aws
     │   ├── conf.py  ## new File
     │   ├── __init__.py  ## new File
     │   └── utils.py  ## new File
     ├── __init__.py
     ├── settings.py
     ├── urls.py
     └── wsgi.py

   • Add configuration in conf.py files

     import datetime
     import os
     
     AWS_USERNAME = 'django_s3' # not matter
     AWS_GROUP_NAME = 'django_s3_group' # not matter
     # Aws config
     AWS_SECRET_ACCESS_KEY = '2IF3Z/k4gni+YIl4RVIA4eOXp0ncBqFsrncqdHqf'
     AWS_ACCESS_KEY_ID = 'AKIA6IVWV2ZU5HKXT2VV'
     AWS_FILE_EXPIRE = 200
     AWS_PRELOAD_METADATA = True
     AWS_QUERYSTRING_AUTH = True
     AWS_DEFAULT_ACL = None
     
     DEFAULT_FILE_STORAGE = 'django_s3.aws.utils.MediaRootS3BotoStorage'
     STATICFILES_STORAGE = 'django_s3.aws.utils.StaticRootS3BotoStorage'
     AWS_STORAGE_BUCKET_NAME = 'djang-s3-bucket'
     S3DIRECT_REGION = 'us-east-1'
     S3_URL = '//%s.s3.amazonaws.com/' % AWS_STORAGE_BUCKET_NAME
     MEDIA_ROOT = S3_URL + 'media/'
     STATIC_URL = S3_URL + 'static/'
     ADMIN_MEDIA_PREFIX = STATIC_URL + 'admin/'
     AWS_S3_FILE_OVERWRITE = False
     two_months = datetime.timedelta(days=61)
     date_two_months_later = datetime.date.today() + two_months
     expires = date_two_months_later.strftime("%A, %d %B %Y 20:00:00 GMT")
     
     AWS_HEADERS = {
         'Expires': expires,
         'Cache-Control': 'max-age=%d' % (int(two_months.total_seconds()),),
     }
     

   • Add Lambda expression for storages in utils.py file

     from storages.backends.s3boto3 import S3Boto3Storage
     StaticRootS3BotoStorage = lambda: S3Boto3Storage(location='static')
     MediaRootS3BotoStorage  = lambda: S3Boto3Storage(location='media')

   • Add conf.py in settings.py

     from django_s3.aws.conf import *

      

   • Great, you configured you aws s3. now final step collectstatic

     python manage.py collectstatic

Get github django s3 code

Conclusion

I found the best way to set up django AWS. in this article I explain what I set up in my real project. please comment if you have other better ways.