Adding AWS S3 Cloud Disk
About AWS S3 Cloud Disks
Amazon S3 is storage for the Internet. It is designed to make web-scale computing easier. Amazon S3 and SoftNAS S3 Cloud Disks provide access to store and retrieve any amount of data, at any time, from anywhere on the web. It gives anyone access to the same highly scalable, reliable, secure, fast, inexpensive infrastructure that Amazon uses to run its own global network of web sites. The service aims to maximize benefits of scale and to pass those benefits on to customers.
As shown below, AWS S3 Cloud Disks are block devices created from Amazon S3 storage.
Each S3 Cloud Disk device can store up to 4 petabytes (PB) of data. An unlimited number of S3 Cloud Disks are supported. Each S3 Cloud Disk is thin-provisioned, so storage space is only consumed when data is actually written to the device and actually used.
S3 Cloud Disks are attached to SoftNAS Storage Pools and provide unlimited cloud storage. Each cloud disk is encrypted and authenticated to provide added security.
S3 Cloud Disks can be created and accessed on-premise from VMware ESXi or Hyper-V systems, as well as within the Amazon EC2 cloud environment.
Cloud disks benefit from SoftNAS features, including RAM caching, SSD caching, compression, deduplication, scheduled snapshots and read/write clones. This means you get the best balance of performance and NAS features combined with the off-site data storage redundancy of S3.
Amazon S3 storage costs start at just $10 per TB per month in lower terabyte quantities and are available as low as $5 per TB per month in higher quantities. Consult Amazon S3 product information pricing for latest details and pricing.
S3 Cloud Disks can also be copied for long-term archive storage into AWS Glacier (functionality that is built into the AWS console).
Adding AWS S3 Cloud Disks
To add a new AWS S3 Cloud Disk to SoftNAS
1. Launch SoftNAS StorageCenter™ and click on Disc Devices.
The Disk Devices panel appears.
2. From Disk Devices, click on Add Device.
3. From Add Device screen, click on Next.
4. Complete the Add S3 Cloud Disk form.
Enter your AWS Access Key ID and Secret Access Key values.
Next, enter a Bucket Basename. This name is used to automatically generate unique S3 bucket names to host the S3 Cloud Disk. You can also enter any unique bucket name comprised of all lower-case letters that you prefer for an automatically generated name.
Region - use the pull-down menu to choose a regional data center where the S3 Cloud Disk and it's corresponding S3 bucket will be created and maintained.
About Regional S3 Cloud Disks - S3 Cloud Disks can be created in any of the worldwide AWS regional data centers. By specifying a specific region for S3 Cloud Disks, the data can be maintained within only that specific data center. Benefits of regional S3 Cloud Disks include: a) the data remains within the country where the regional data center is located, to meet certain regulatory compliance requirements, and b) S3 Cloud Disks are located nearby the SoftNAS EC2 instance, reducing latency and improving throughput and performance.
By default, S3 Cloud Disks and their associated S3 buckets are created in the "U.S. Standard" region (US East, Virginia) and replicated to other U.S. regions in Oregon and N. California, and provide eventual consistency for all requests. This region automatically routes requests to facilities in Northern Virginia or the Pacific Northwest using network maps.
With the addition of region-specific support, S3 Cloud Disks and their associated S3 buckets provide read-after-write consistency for new disk writes in your Amazon S3 Cloud Disk (since ZFS is a copy-on-write filesystem, this means you get immediate read-after-write consistency on disk writes).
Next, choose cloud disk options:
Maximum Disk Size - this value can be between 1 GB and 4095 TB (4 petabytes). This is the maximum cloud disk size for the device. As cloud disks are thin provisioned, there are no Amazon S3 storage costs until data is actually stored in a SoftNAS storage pool and volume.
When choosing a Maximum Disk Size, please keep in mind that you will need a SoftNAS license for the maximum about of storage you plan to use.
Encrypted disk: check this option and provide a Disk Password to encrypt the contents of the S3 cloud disk to ensure its contents cannot be accessed, except via this S3 Cloud Disk.
5. Click on Create S3 Cloud Disk.
The S3 Cloud Disk is created and automatically partitioned and ready for use.
Note: The Extra Disk Info column shows the S3 bucket name and region where the bucket is located.
The next step is to create a Storage Pool which uses the S3 cloud disk.
Note in the above screenshot the device names are in the form /dev/s3-NNN, where NNN is the cloud disk device number from 0 to NNN (there is no upper limit). In the Extra Disk Info column you will see the S3 bucket name. Once assigned to a storage pool, the device will show which pool the device is used by.
6. Create the SoftNAS Storage Pool that uses the S3 Cloud Disk.
Choose the Storage Pools menu item and use the Create button to create a storage pool that uses the new SoftNAS S3 Cloud Disk.
After creating the Storage Pool, you will see the available storage which is ready for use, as shown above. The above example shows 2 million GB of available storage (2 PB) with a single S3 cloud disk in RAID 0 configuration.
Note: Ensure you have sufficient licensed capacity in SoftNAS to accommodate the amount of storage capacity that will be added.