Introduction of AWS S3, a Simple Storage Service

Introduction of AWS S3, a Simple Storage Service

AWS Series – 005

Hello CTT Friends, welcome to this AWS series.  In this topic, we will discuss about AWS S3 cloud offering. Primarily, we will talk about what is S3, different types of feature, characteristics etc. then, we will do hands on demonstrations. There will be multiple labs so I shall create those one after another and will make available for you in CTT YouTube Channel.

AWS S3 is really useful & simple in concept, you may want to use it regularly as well for storing files that you frequently access from different devices. Also, if you are planning for AWS certification, you need to understand S3 concept thoroughly.

I wish, this course will help you to be completely familiar with S3, though it is simple in concept but there are many things that need your close attention to pass AWS certification.

So, friends, let’s begin …

What is S3?

Full form of S3 is Simple Storage Service. And, in AWS language, the definition of S3 is as following

Amazon S3 is really convenient to store and retrieve any amount of data at any time, from anywhere on the web. S3 provides amazingly durable, highly available, secured storage to store any type of objects(files).

So, you can consider AWS S3 as a cloud based hard drive, AWS platform provides you a nice interface, APIs & command line tools to interact with S3 service.

You use your USB or external hard disk to store files, right? You can use S3 for the same purpose, “storing any files” and get free from carrying external hard drives … TAADAA 😊

There are now multiple companies who provide services based on AWS S3. So, you can avoid using USB/external hard drive all together. They have created a client software which creates a special folder in user’s computer. The content of which then synchronized with cloud storage in S3. So, if you put a file from home computer, you can access the same file from your office, university or any of your devices, even from your Mobile wherever you have internet.

AWS S3 is different from external or USB drive in one aspect that S3 is Object based storage. This means you can store any type of data in this as Object but cannot use it as high-performance storage used for OS installation, Database installation or any software installation like we can do for external or USD drive which are Block based storages.

Let’s now discuss about different terminologies in S3 paradigm before deep diving other S3 features.

There are few terminologies that you should be familiar with when learning about S3.

Object:  Whatever you put in S3 is called object. Object is combination of data and metadata of your actual content. Metadata is set of name value pairs that describe the object.

Bucket: you can compare it as logical partition of your hard disk. If you are using Windows, it could be C, D, E etc. for Linux these are /root /opt etc. and so on. The same way, you can create your files, folders under S3 Bucket.

Bucket: This is equivalent as logical partition in your computer hard disk where you can put files or create folders. Bucket name must be unique across all AWS S3 buckets from all AWS regions. You cannot create new bucket with a name if someone already created another bucket using same name. It is similar behaviour like your computer, you cannot have save drive letter in Windows system or mount point in Linux.

Regions: You can select a specific AWS region while creating S3 Bucket, so all you content will stay in that region only, unless you explicitly transfer then to another region. If you want to explore what is AWS region and other AWS terminology, please visit this post. For example, if you are storing your contents/files in a S3 bucket created in Mumbai region, then AWS will not move those files from Mumbai region to other AWS region, but if you want to transfer the files to London region, then you must do the necessary activities.

Keys: Key is the identifier for an object within a bucket. Every object has one and only key that is used to interact with the object stored in S3. The combination of a bucket name, key and version-id uniquely identify each object.

Version ID: You can put multiple versions of same file, mean same file name, if you enable versioning, AWS S3 will keep all those different files for same file name with version maker.

Metadata: As mentioned before that S3 object consist of METADATA which is basically special meaningful summary date of your actual content/object. For example, different type of attributes such as created by, updated by, last update timestamp etc.

Data Consistency Model:  This is basically when you put any files in S3 or try to read it from S3, how S3 will respond to your request.

  • read-after-write consistency for PUTS of new objects – What does it mean?! So when you write a New object in S3 bucket, you can read the object right after the writing is complete. That means it is immediate consistency. You can read as soon as the file writing is finished.
  • Eventual Consistency But while you want to modify or delete any exisiting data/object from a S3 bucket, the operation may not perform immediately. This consistency model is not immediate, so there is a slight possibility that if you make a retrieval request right after you put or delete the object, you may receive old data. Though, AWS guarantees that there is no chance the data will be corrupted or partial, either you will get old data or latest data.

Friends, I have captured few major highlights of S3 as following, it will help you to understand the potential of S3 for your possible Usecases.

  • It is designed to deliver 99.999999999% durability: This is called 11 9s durability, that means it is so reliable service that you will hardly loss your data.
  • S3 Standard is designed for 99.99% availability and Standard – IA is designed for 99.9% availability: S3 is designed for 99.9% availability. It is almost all the time running & available for your use. If I calculate, it is only circa 53 minutes when S3 may be unavailable throughout a year.
  • Amazon S3 was carefully engineered to meet the requirements for scalability, reliability, speed, low-cost, and simplicity
  • 0 byte to 5TB file size can be uploaded, no limit of numbers
  • Supports SSL encryption of data in transit and at rest
  • Secure data using ACL & Bucket policies
  • Lifecycle management for automatic migration of objects
  • The Amazon S3 SLAprovides for a service credit if a customer’s monthly uptime percentage is below our service commitment in any billing cycle.

Types of S3:

Okay friends, lets now explore different types of S3 storage, comparison of different characteristics, Pricing Model etc.

Standard –  This S3 type is backbone of any cloud solution running on ASW platform. You cannot architect without using S3 standard storage. You will find Usecase that require storing files while architecting or designing cloud solution. It is 11 9s durable, that means if you put 10000 objects in S3, there is a possibility that you may loss single object in another 10000 years.

Infrequent Access Storage – IA Storage helps optimizing storage cost. If you have use case/ requirement for objects/files that need limited number of access and may not be always available. Less availability isn’t affect overall solution, then you can leverage this type of storage to minimize cost. Probable Usecase could be to store monthly reports / analytical statistics / Annual Tax data which are not often required but when needed, data should be immediately available.

Reduce Redundancy Storage –  What is this type? This type of storage offers less disability. That means there is possibility of losing your files. Let’s assume if you can recreate files if gets corrupted or lost, then you may want to use RRS to minimize your COST further. AWS maintains only 1 concurrent facility for RRS service hence providing less fault tolerance. It does not replicate objects in another service instance, hence reduced redundancy. Therefore, it is bit cheaper. Probable Usecase would be for storing Thumbnails, Video files with different sizes for different devices when you can create them from the original Video file which is stored in Standard storage.

Glacier – This is a Very important storage type.  You can archive any date for long term reference. Such as old audit data, you may need it for future reference if asked. But no point of storing it in regular storage without having need of accessing immediately. Glacier has a very cheap pricing model; thus, it will save you further money if you can leverage it correctly for different applicable Business Usecases. You can also leverage this service from outside scope of S3, because AWS offers Glacier as a standalone separate service. Though, Glacier is highly integrated with S3 via life cycle management policy. This means , you can set appropriate policy to transfer files from S3 standard storage to Glacier after meeting certain rules e.g. after 90 days.

I am providing a comparison chart below for your better understanding. You can easily understand that how these S3 storage types are different from each other. All the data is taken from AWS website.

S3 Pricing Model:

Friends, AWS provides 5GB of cloud storage for free if you just sign up with AWS platform. I would suggest start using it and you may not even need USB anymore 😊 If you are beyond 5GB limit then AWS will change you as per defined pricing model. Also, there are different type of factors that you should understand how AWS charges are applied.

Storage: It is bit straight forward, right? AWS will charge against what you consume at rate of per-GB storage space.

Read & Write Requests: Read request signifies that whenever you are trying to read any object from S3 that is one request whether it is via REST call, CLI or SDK request. Similarly, when you are trying to write or store any new object or want to update any existing object than that will be considered as a Write request.

Storage Management PricingAmazon S3 Storage Management is priced per feature e.g. Monitoring 7 Audit matrix using CloudWatch & CloudTrail. For pricing on S3 CloudWatch Metrics, visit the CloudWatch pricing page. For pricing on S3 Data Events in CloudTrail, visit the CloudTrail pricing page.

Data Transfer:  This category of pricing is applicable when you transfer data either “in” or “out” from S3 over public Internet. If you transfer your data to another region then also this type of charge will apply. Though, Transfers between S3 buckets or from S3 to any service(s) within the same region are free.

Transfer Acceleration: This is a feature provided by AWS. If you want to leverage this feature there will be charge, however in return you can transfer data from one region to another region very quickly. Specifically, end user performance will drastically increase. Let me explain a bit further,

Let’s assume a file is stored in S3 bucket located in India Mumbai region. Users from USA want to access that file. What will happen when end users from USA make the request? The file will be transferred through normal internet which has bandwidth limitation, hence slower & there will be latency. However, if Transfer acceleration is enabled then the file will be transferred from Mumbai region to nearest AWS Edge location (visit this post for more info)  which is internally connected to USA region Edge location. Bandwidth of AWS internal network is high in the range of 10 Gbps to 25 Gbps, hence time taken for file transfer from India Edge location to USA Edge location will be very less compare to normal internet network. This will accelerate Data transfer; hence it is called Transfer Acceleration.

Now, let’s understand S3 Pricing model. For example, I am providing a snapshot of pricing for Mumbai, India region. However, please visit AWS pricing page for elaborate pricing detail.Guys, I am also preparing few Hands-on Labs and will keep posting those Videos in CTT YouTube Channel, please make sure you subscribe the Channel and hit the bell icon, so you will be notified immediately when those are available.

I hope, this is helpful to kick start your AWS S3 learning. Please comment below if you want to address any specific area, I shall update the blog to cover those areas as well.

Keep visiting CTT and be updated about AWS cloud offerings. 😊

Lastly, if you prefer that someone reading through for you and that will give you better understanding, please watch it in YouTube

Video Tutorial Part 1

Video Tutorial Part 2

Author: Dhrubo

I am passionate about sharing Knowledge , Information and wisdom what ever way is possible. This is small contribution to our society from my part. I am a Programmer and love to architect and modernize IT infrastructure / solution / applications for my clients.

I am sharing my experience what I have gained so far via CTT and will keep doing so in regular basis. This is the whole purpose of founding this Knowledge Center for Cloud Technologies.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.