Storage Trifecta

This lab provides hands-on experience with the three primary types of cloud storage: Block, File, and Object. You will configure each type locally to understand their access mechanisms, metadata characteristics, and common use cases.

Learning Objectives

• Differentiate between block, file, and object storage through hands-on configuration.
• Understand the access mechanisms and use cases for each storage type.
• Compare metadata overhead and scalability constraints.

Prerequisites

• Linux environment (Ubuntu/Debian recommended)
• Root or sudo privileges
• Docker installed
• Basic familiarity with the Linux CLI

Step 1: Block Storage Simulation

Block storage provides raw storage volumes to systems. It is the foundation for filesystems. In this step, we will simulate a block device using a loopback file.

1. Create a sparse file: Create a 1GB file that doesn't actually consume 1GB on your disk yet.

truncate -s 1G lab_block.img

1. Setup a loop device: Map this file to a virtual block device.

sudo losetup -fP lab_block.img
# Find which loop device was used
losetup -a | grep lab_block.img

Assume the device is /dev/loop0 for the following steps.

1. Format the device: Create an ext4 filesystem on the "raw block" device.

sudo mkfs.ext4 /dev/loop0

1. Mount and use:

sudo mkdir -p /mnt/block_lab
sudo mount /dev/loop0 /mnt/block_lab
echo "Writing directly to block-backed storage" | sudo tee /mnt/block_lab/test.txt

Analysis: Note how the OS treats this exactly like a physical hard drive. You have full control over the filesystem and low-level formatting.

Step 2: File Storage Setup (NFS)

File storage (Network Attached Storage) provides a higher-level abstraction where the storage system manages the filesystem and shares it over a network.

1. Install NFS Server:

sudo apt update && sudo apt install -y nfs-kernel-server

1. Configure an export:

sudo mkdir -p /srv/nfs_share
sudo chown nobody:nogroup /srv/nfs_share
echo "/srv/nfs_share  127.0.0.1(rw,sync,no_subtree_check)" | sudo tee -a /etc/exports
sudo exportfs -ra

1. Mount the share:

sudo mkdir -p /mnt/nfs_client
sudo mount -t nfs 127.0.0.1:/srv/nfs_share /mnt/nfs_client

1. Verify permissions: Create a file in the mount point and check its existence in /srv/nfs_share.

Analysis: Unlike block storage, the client does not format the drive. It interacts with an existing directory structure over the network.

Step 3: Object Storage Deployment (MinIO)

Object storage stores data as "objects" with unique IDs and metadata, typically accessed via REST APIs rather than a mount point.

1. Start MinIO via Docker:

docker run -d -p 9000:9000 -p 9001:9001 \
  --name minio_lab \
  -e "MINIO_ROOT_USER=admin" \
  -e "MINIO_ROOT_PASSWORD=password" \
  quay.io/minio/minio server /data --console-address ":9001"

1. Configure MinIO Client (mc):

# Download mc if not present
curl https://dl.min.io/client/mc/release/linux-amd64/mc \
  --output mc && chmod +x mc

./mc alias set myminio http://localhost:9000 admin password

1. Interact with Objects:

# Create a bucket
./mc mb myminio/lab-bucket

# Upload a file
echo "Object storage data" > object_data.txt
./mc cp object_data.txt myminio/lab-bucket/

# List objects
./mc ls myminio/lab-bucket/

Analysis: Notice how we don't "mount" the bucket. We use a client or API to GET/PUT data.

Step 4: Comparative Analysis

Based on your observations, complete the following table in your notes:

Reflection Questions

1. Why is block storage preferred for high-performance databases?
2. What happens to the metadata in Object storage when you rename a file compared to File storage?
3. Which system scales most easily to petabytes of data? Why?