Web Analytics

FireOak Strategies Blog

Insights and articles related to knowledge management, information security, technology, data and analytics, business process automation, platform management, and other related topics, from our experienced team of consultants.

Part 3: Automate the Creation and Export of EC2 Volume Images

In Part One, I reviewed the steps required to create and export an image of an AWS EC2 volume.  This process works well, but it is somewhat cumbersome and time-consuming for sysadmins, especially when waiting for large images to compress.


AWS EC2

This blog post is part three of a 3-part series describing a technique for how to export and import Amazon Web Services (AWS) instance volume images. Read the full introduction for more details.

In Part One, I reviewed the steps required to create and export an image of an AWS EC2 volume.  This process works well, but it is somewhat cumbersome and time-consuming for sysadmins, especially when waiting for large images to compress.  Fortunately, the entire process can be automated through a bit of shell scripting and the AWS Command Line Interface (CLI) toolkit

I won’t be covering the installation and setup of the AWS CLI tool in this post. 

Amazon’s documentation will walk you through the setup.  You’ll need to create an IAM policy with allows the following actions:

  • ec2:AttachVolume
  • ec2:CreateSnapshot
  • ec2:CreateVolume
  • ec2:DeleteSnapshot
  • ec2:DeleteVolume
  • ec2:DescribeSnapshots
  • ec2:DescribeVolumes
  • ec2:DetachVolume

Assign this policy to an IAM user and configure your AWS CLI instance with that user’s access keys.

I’ve created exportEC2VolumeImage which automates the process of creating a compressed image of an Amazon EC2 volume. This tool is designed to be run on a Linux server in your AWS account. The exportEC2VolumeImage tool can be downloaded from my GitHub repository.

Usage: $ sudo exportEC2VolumeImage volume-id availability-zone instance-id scratch-path

volume-id:                          volume ID of the image to be exported
availability-zone:              availability zone of the build server
instance-id:                         instance ID of the build server
scratch-path:                     location of scratch folder where images will be stored

Since the script will be accessing virtual hardware devices, it must be run as root or a user with read permissions to the virtual device.

Sample Run:

$ sudo exportEC2VolumeImage vol-02500000000000000 us-west-2 i-06b00000000000000 /mnt/scratch/
Volume vol-02500000000000000 is in availability zone us-west-2c
Creating a snapshot of vol-02500000000000000..............[OK]
Creating a new volume from snapshot snap-0f42e0a4b891b1d01........[OK]
Waiting for volume attach operation to complete......[OK]
Creating image...
1053818880 bytes (1.1 GB, 1005 MiB) copied, 38.0045 s, 27.7 MB/s
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 38.8194 s, 27.7 MB/s
 
Image File Contents:
 
Disk /mnt/scratch/ec2VolImg.vol-025f10e3b9335c04d.20190204_170057.img: 1 GiB, 1073741824 bytes, 2097152 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xee521bb7
 
Device                                                             Boot Start     End Sectors  Size Id Type
/mnt/scratch/ec2VolImg.vol-02500000000000000.20190204_170057.img1       2048 2097151 2095104 1023M 83 Linux
Compressing image...
Image creation complete.
Detaching volume.......[OK]
Deleting the snapshot...[OK]
Deleting the temporary volume...[OK]
Moving /mnt/scratch/ec2VolImg.vol-02500000000000000.20190204_170057.img.gz to Google Drive...
Done.

The exportEC2VolumeImage tool is available on github: https://github.com/ericpskl/exportEC2VolumeImage

 


From the FireOak Blog

News and insights from the FireOak team about managing, securing, and sharing knowledge