AWS S3 stands for Simple Storage Service – one of the most commonly used AWS services. It was first introduced in 2006, and since then, Amazon S3 has emerged as a popular platform for object-based storage usage. In S3, data is stored as objects within resources called buckets that offer storage space where a single object can take up to 5 terabytes in size. It is a cost-effective storage service and designed for 99.999999999 percent durability.
S3 also provides access to manage data across websites, mobile applications, backup and restore big data analytics, and many other applications. If you are also using S3 to store your data, then in one case or another, you may have come across a scenario or may need in the future where you need to copy your files between two different AWS S3 buckets within the same account.
In this post, I will show you how to copy files between one S3 bucket to another S3 bucket without having to write any scripts or write any piece of code at all. And the good thing is all of this is going to be directly through the console. So, without any delay, let’s dive deep into our topic.
Note: Keep in mind that this won’t affect files in the source bucket, so it’s effectively a copy command from one location to another.
If you are interested in watching a video instead then click here to view it now
Step 1 : Defining your buckets
In the S3 console, and I have two different buckets that are already pre-created here. For demonstration purposes, I already created them, one of them is named aws-simplified-source-bucket, and the other is the aws-simplified-destination-bucket. These are the two buckets where I want to transfer my files from the source bucket to the destination bucket.
So if we take a look inside the above source bucket, I have uploaded two small JSON files in a folder just for demonstration purposes.
If we just go to our destination bucket, we should see this as empty, which of course, should be as we want to transfer files between these two buckets from the source to the destination.
In fact, as I already mentioned, this is going to be through the same AWS account, and in case if you are to do this in a cross-account way, there’s a great blog post that you can check from here.
Step 2 : Data Sync
In the next step, we will use a service called AWS Data Sync; this is a new feather in the hat of AWS that lets you sync data from source bucket to destination bucket comfortably. You can just type Data Sync or AWS Data Sync up in the search bar, there you can find the tool. With this, you can automate the acceleration of data from on premise to the cloud or vice versa or between AWS Storage services.
You can see the wizard from below, where it is showing us what option we want to choose for the data transfer between on premise storage or AWS storage services. Here in this demonstration, as we are going to transfer between two AWS S3 buckets, I tend to choose the option as “Between AWS storage services” and click on Get Started.
Step-3 Setting up Configurations
In the next step, we need a whole bunch of settings regarding where our files are located. It involves 4 steps in setting up things for you.
Sub-step 3.1 : Configuring the source location
Here in the first sub-step of step 3, we need to configure source location. So we want to pick choose a new location, and then we have different other parameters to configure as well. You can refer to the below screenshots to configure the source location.
