Introduction
The AI.Panther cluster provides multiple storage types to support a wide range of research and computing workflows. Each storage type is designed for a specific purpose and differs in performance characteristics, access model, and data-retention characteristics.
This article describes the five primary storage types available on AI.Panther:
- Network Home Directories
(/home1)
- Local Scratch
(/localscratch)
- Project Storage
(/shared/projects)
- Shared Scratch Storage
(/shared/scratch)
- Archive Storage
(/archive)
Storage Overview
| Storage Type |
Path |
Intended Use |
Backups |
Retention |
| Network Home |
/home1/ |
User home directories, configs, scripts (all users) |
Snapshots (on-pool) |
Persistent |
| Local Scratch |
/localscratch |
Temporary, high-churn, node-local workloads (all users, H200 nodes) |
No |
Auto-purged |
| Project |
/shared/projects |
Shared research project data (requires project approval) |
No |
Time-limited |
| Shared Scratch |
/shared/scratch |
Temporary, high-churn workloads across nodes (all users) |
No |
Auto-purged |
| Archive |
/archive |
Cold storage for inactive datasets (placement by request) |
Snapshots (on-pool) |
Persistent |
Network Home Directories
Overview
User home directories are provided via NFS and mounted at: /home1
This is where all user home directories reside. The filesystem is mounted and available on all nodes.
Intended Use
- Shell configuration files
- Source code
- Job scripts
- Small-medium datasets
- Results that need long-term persistence
Access Model
- Each user has a private home directory
- Users cannot access other users' home directories
Characteristics
- Mounted via NFS on all nodes
- Rolling ZFS snapshots on the storage server (frequent, hourly, daily)
- Persistent (no automatic deletion)
- Not optimized for large-scale or high-throughput I/O
Notes
Large datasets (50-100 GB+), model checkpoints, and high-throughput I/O workloads should not be stored in home directories. Consider requesting Project Storage or using Scratch instead.
Local Scratch
Overview
Local scratch storage is provided on individual H200 GPU nodes and is intended for temporary, node-local workloads that benefit from NVMe storage. This storage is mounted at: /localscratch
Local scratch is backed by node-local NVMe drives configured in RAID10 to provide high performance while tolerating single-disk failures.
Intended Use
- Temporary job working directories
- Intermediate files
- Short-lived, node-local data
- Performance-sensitive workloads that do not require persistence
Access Model
- Available to all users running jobs on a given node
- Data is only visible on the node where it was written
- Only available on H200 nodes (partitions
h200 and h200_mig)
Characteristics
- Node-local NVMe storage (RAID10)
- No backups
- No quotas
- Automatically purged
- Highest locality and lowest latency for node-local workloads
Purge Policy
Files and empty directories in local scratch are automatically deleted after 60 days from the last modification time.
Notes
Local scratch data should be treated as ephemeral. Do not store data that cannot be easily regenerated.
Project Storage
Overview
As of 2026, AI.Panther cluster includes a new, high-capacity DDN ExaScaler (Lustre) storage system providing approximately 1 petabyte (PB) of shared research storage. This storage is designed for research group project storage (not individual home directories).
Intended Use
- Shared datasets
- Simulation outputs
- Model checkpoints
- Collaborative research data
- Performance-sensitive, large-scale workloads
Access Model
Project storage is group-based:
- A project corresponds to a group
- Access is granted via group membership
- Project data resides under a project directory:
/shared/projects/<project_name>
Characteristics
- High-performance parallel filesystem (Lustre)
- No backups
- Shared across all nodes
- Designed for large capacity and high throughput
Allocation and Term
- Minimum allocation: 100 GB per project
- Default term: 1 year
- Renewals: renewed annually; renewal requests are submitted by the faculty sponsor (PI)
Notifications and Expiration
To avoid unintentional data loss, the project team will receive notifications for two scenarios:
- 30 days before the project storage is set to expire (data deletion)
- 30 days before the renewal deadline (data deletion)
Failure to renew may result in data deletion.
Deletion and Recoverability
⚠ Warning: Deleted files are not recoverable at this time. Treat all deletions as permanent. Please maintain a backup policy for your data.
Requesting Project Storage
Project storage is requested through TeamDynamix (TDX). Requests should include:
- Project name
- Project description
- Justification for request
- Amount of storage requested
- Expected duration of storage
- List of users in the group
Common User Commands
The following commands may be used by project members to view storage usage, quotas, and ownership:
df -h /shared/projects/<project_name>
lfs quota -h -g <project_group> /shared
ls -l /shared/projects/
ls -l /shared/projects/<project_name>
Shared Scratch
Overview
Shared Scratch storage is provided through the DDN ExaScaler system and is mounted at: /shared/scratch
This storage is intended for temporary, high-churn workloads that require shared access across nodes.
Intended Use
- Temporary simulation outputs
- Intermediate files for large jobs
- Shared working directories for short-lived workflows
Access Model
- World-writable
- Available to all users
- Users may not delete files owned by other users
Characteristics
- High-performance parallel filesystem
- No backups
- Automatically purged
- Optimized for throughput, not persistence
Purge Policy
Files in shared scratch are automatically deleted after 60 days from the last modification time.
Notes
Shared Scratch should never be used for long-term storage or as the sole copy of important data. All data stored here should be considered temporary.
Archive Storage
Overview
Archive storage is a cold-storage tier for inactive research data that must be retained but no longer needs to live on the active /home1 or /shared/projects filesystems. Archive is mounted at /archive on the login node (approximately 330 TB usable). Each user's data lives in a private subdirectory (/archive/<username>) that only that user can enter.
Intended Use
- Completed project datasets
- Older model checkpoints and training outputs
- Reference data that is rarely read but must be preserved
- Data being migrated off
/home1 to free capacity
Access Model
- Mounted at
/archive on the login node
- Each user has a private subdirectory (
/archive/<username>, mode 700, owned by the user)
- Users can read and write within their own subdirectory from the login node
- Creating a new user subdirectory or moving data into
/archive from /home1 requires an email request to the HPC team
Characteristics
- High capacity, optimized for retention rather than throughput
- Rolling ZFS snapshots on the storage server (daily, weekly, monthly)
- No automatic purge; data is retained until removal is requested
- Not suitable for running jobs or interactive access
Requesting Placement in Archive
To move data into /archive (or to have a new subdirectory created), send an email:
Include in the request:
- Source path(s) on
/home1 or /shared/projects to move to /archive
- Approximate size
- Whether the source copy should be deleted after the transfer completes
Retrieving Your Own Data
If you already have data in /archive/<your-username>, you can copy it to any writable location (/home1, /shared/projects, /shared/scratch) directly, using standard tools like cp or rsync from the login node. No email request is needed to read from your own archive subdirectory.
Notes
Archive is for inactive data. If you still access the data regularly or need it available to running jobs at scale, request Project Storage (/shared/projects) instead. If the data is short-lived and regeneratable, use Shared Scratch (/shared/scratch).