Data quota
When you have logged into the server, you usually start in your home (~) folder. Move to the root of the server (cd /) and you’ll see that is a large number of other directories. A few of these directories contain space that is assigned to your user account. Here is an overview:
| Path | Data quota | Max. files | Backup | Note |
|---|---|---|---|---|
/afs/ihep.ac.cn/users/${USER:0:1}/$USER |
500 MB | yes | home (~) |
|
/besfs5/users/$USER |
no | deprecated | ||
/besfs10/users/$USER |
50 GB (per directory!) | not documented | no | newer, faster storage |
/home/bes/$USER |
same as /besfs10 |
same as /besfs10 |
yes | symlink to /besfs10/homebes |
/scratchfs2/bes/$USER |
500 GB | 200,000 | no | large data files |
/workfs2/bes/$USER |
5 GB | 50,000 | yes |
Previously existing /scratchfs is retired by July 31st, 2026 and replaced by /scratchfs2.
/besfs5 will no longer be available from 2027. Please migrate to /besfs10 instead.
/besfs10 quota is applied per directory
On /besfs10, the quota is set on the directory, not on your user account. As a result, lfs quota -u $USER /besfs10 reports no limit, while writing to /besfs10/users/$USER still fails once you exceed 50 GB. Use lfs_dir instead (see below) to see the limit that actually applies to you.
Since there is no warning by email (as there is for /afs), you typically notice this only when jobs start to die halfway through with messages like:
cat: write error: Disk quota exceeded
SysError in <TFile::WriteBuffer>: error writing to file /besfs10/users/$USER/....rtraw (-1) (Disk quota exceeded)If you need more space, write your output to /scratchfs2/bes/$USER instead.
/home/bes/$USER is not a file system of its own: it is a symbolic link to /besfs10/homebes/$USER. IHEP recommends referring to your files through /home/bes, because that path is expected to keep working when the underlying file system is migrated again (as happened with /besfs5 → /besfs10).
Only /afs, /workfs2, and /home/bes provide backup protection (see the IHEP Lustre page). Note that /home/bes is backed up even though it points into /besfs10, which is not.
Two consequences:
- Anything that you cannot easily reproduce and that is not under version control should live on a backed-up file system. For your BOSS packages, the better protection is to keep your
workareain a Git repository and push it to a remote such as GitHub orcode.ihep.ac.cn. - Conversely, keep caches, build output, and large data files off the backed-up file systems: there is no point in backing those up, and they fill up the small quota of
/afsand/workfs2quickly.
Official information on the quota can be found here (BESIII) and here (IHEP). You can check your quota by running the following commands:
fs listquota /afs/ihep.ac.cn/users/${USER:0:1}/$USERlfs_dir /besfs10/users/$USERThe quota on /besfs10 is set per directory, so this is the command that shows the limit you actually run into. The user-based query,
lfs quota -u $USER /besfs10reports no limit and is therefore misleading.
lfs quota -u $USER /scratchfs2lfs quota -u $USER /workfs2The best setup is to use /besfs10 for your code and small data files. You can use symbolic links to link from your home directory to /besfs10 and avoid quota issues under /afs. Example:
ln -s /besfs10/users/$USER/.cache ~/.cache