Introduction to EPFS

Overview

EPFS (E2E Parallel File System) is a highly scalable and reliable parallel file system designed for large-scale High-Performance Computing (HPC) environments. It delivers exceptional performance by supporting parallel data access, making it ideal for handling massive datasets and numerous clients.

  • High throughput: Supports multiple simultaneous clients.

  • Scalability: Can handle petabytes of data and thousands of clients.

  • Reliability: Uses robust mechanisms to ensure data integrity.

How EPFS Works

  1. Data Distribution:

    • Data is striped across multiple OSSs, enabling parallel read/write operations.

    • The striping pattern can be customized for specific use cases.

  2. Metadata Management:

    • The MDS manages metadata, keeping operations like file creation and deletion fast.

  3. Parallelism:

    • Clients access data in parallel from OSSs, minimizing bottlenecks.

Advantages

  • High Speed: Parallel operations increase performance.

  • Fault Tolerance: Data is replicated across nodes.

  • Scalability: Designed for massive datasets and numerous clients.

Note

This feature is currently available only on AlmaLinux.

Managing EPFS

Creating an EPFS

  1. Navigate to the Storage > EPFS section in the left sidebar of the dashboard.

../../_images/dashbard.png

  1. Click Create.

../../_images/create_epfs.png
  1. Provide the following details:

    • Name: Enter a name for your EPFS.

    • Plan: Select an appropriate plan.

    • VPC: Choose the desired Virtual Private Cloud.

../../_images/select_plan1.png
  1. Click Create EPFS.

    Your newly created EPFS will appear in the list.

../../_images/list_epfs.png

Accessing EPFS

Note

Ensure that EPFS and the Virtual Machine (VM) are on the same network.

Follow these steps to set up the client and mount EPFS:

  1. Download the Client Setup Script:

    Download the scripts.tar file.

    ../../_images/download_client.png

  2. Transfer the script to the server:

    Use the scp command to copy the file into your server.

    scp <File-Location>/scripts.tar <username>@<server_ip>:/root/

  3. Extract the tar file:

    Run the following command to extract the script:

    tar -xvf <file_name>

  4. Run the Client Setup Command:

    Execute the following script to set up the client:

    ./client_setup.sh tcp1056 10.5.17.... tcp
    ../../_images/run_clientscript.png

  5. Mount the EPFS:

    Use the following command to mount EPFS:

    mount -t lustre 10.20.56.8@tcp:/pfs0 <path_to_your_folder>
    ../../_images/mount_point.png

Upgrading EPFS

Enhance your EPFS by increasing its memory capacity through the Upgrade EPFS feature. 1. Click Upgrade under the Action menu.

../../_images/action_upgrade.png

  1. Select the desired upgraded plan and Confirm by clicking the Upgrade button.

../../_images/update.png

  1. Your EPFS will be upgraded to the selected configuration.

../../_images/upgrading_show.png ../../_images/upgraded_list.png

Deleting an EPFS

  • Select Delete from the Action menu.

../../_images/action_delete.png

  • A warning message will notify you of permanent data loss.

  • Confirm and proceed by clicking Delete. The selected EPFS storage will be removed.

../../_images/delete_epfs.png