Skip to main content

Introduction to EPFS

Overview​

EPFS (E2E Parallel File System) is a highly scalable and reliable parallel file system designed for large-scale High-Performance Computing (HPC) environments. It delivers exceptional performance by supporting parallel data access, making it ideal for handling massive datasets and numerous clients.

  • High throughput: Supports multiple simultaneous clients.
  • Scalability: Can handle petabytes of data and thousands of clients.
  • Reliability: Uses robust mechanisms to ensure data integrity.

How EPFS Works​

  1. Data Distribution:

    • Data is striped across multiple OSSs, enabling parallel read/write operations.
    • The striping pattern can be customized for specific use cases.

  2. Metadata Management:

    • The MDS manages metadata, keeping operations like file creation and deletion fast.

  3. Parallelism:

    • Clients access data in parallel from OSSs, minimizing bottlenecks.

Advantages​

  • High Speed: Parallel operations increase performance.
  • Fault Tolerance: Data is replicated across nodes.
  • Scalability: Designed for massive datasets and numerous clients.
Note

This feature is currently available only on AlmaLinux.

Managing EPFS​

Creating an EPFS​

  1. Navigate to the Storage > EPFS section in the left sidebar of the dashboard.

    Dashboard

  2. Click Create.

    Create EPFS

  3. Provide the following details:

    • Name: Enter a name for your EPFS.
    • Plan: Select an appropriate plan.
    • VPC: Choose the desired Virtual Private Cloud.

    Select Plan

  4. Click Create EPFS.
    Your newly created EPFS will appear in the list.

    List EPFS

Accessing EPFS​

Note

Ensure that EPFS and the Virtual Machine (VM) are on the same network.

Follow these steps to set up the client and mount EPFS:

  1. Download the Client Setup Script:
    Download the scripts.tar file.

    Download Client

  2. Transfer the script to the server:
    Use the scp command to copy the file into your server.

    scp <File-Location>/scripts.tar <username>@<server_ip>:/root/

  3. Extract the tar file:

    Run the following command to extract the script:

tar -xvf <file_name>

  1. Run the Client Setup Command:

    Execute the following script to set up the client:

./client_setup.sh tcp1056 10.5.17.... tcp

  1. Mount the EPFS:

    Use the following command to mount EPFS:

mount -t lustre 10.20.56.8@tcp:/pfs0 <path_to_your_folder>

Upgrading EPFS​

Enhance your EPFS by increasing its memory capacity through the Upgrade EPFS feature.

  1. Click Upgrade under the Action menu.

  1. Select the desired upgraded plan and Confirm by clicking the Upgrade button.

  1. Your EPFS will be upgraded to the selected configuration.

Deleting an EPFS​

  • Select Delete from the Action menu.

  • A warning message will notify you of permanent data loss.
  • Confirm and proceed by clicking Delete. The selected EPFS storage will be removed.