Filter
Top Products
Chapter 7 HPSS User Interface Configuration
442 September 2002 HPSS Installation Guide
Release 4.5, Revision 2
Meetings with Transarc and IBM Austin have taken place to discuss the issue. The above restriction
may be fixed in the future.
7.6.2.7.2 Migration and Purge Algorithms
Currently, the HDM reads through all the anodes in an aggregate to determinemigration and purge
candidates. Using empirical data we determined that the HDM reads approximately 70 entries per
second (this is disk hardware dependent). The optimum number of objects stored on an aggregate
for migration purposes can be determined using the following parameters:
• The amount of time it takes to read through all objects on the aggregate.
• The frequency of migration runs (purge runs when space is needed).
• The amount of time it takes to move files from DFS to HPSS.
If migration runs once per day, and data can be moved in 4 hours, the current suggestion is to allow
approximately 500,000 objects on an aggregate. This means that migration will take approximately
6 hours(7142 seconds = ~2 hours for headerrumble + 4 hours fordata movement). Obviously, these
numbers vary dramatically and are dependent on file size and the number of files that need to be
migrated. Remember, only new files and files whose data has been altered need to be migrated into
HPSS. Files that have already been migrated into HPSS, but have been read need to update XDSM
attributes, but this can be done quickly (approximately 60 per second).
Episode XDSM(DMAPI) changes have been discussed between HPSS and Transarc. It is believed
that with these changes, the Episode XDSM(DMAPI) rate to read all objects in an aggregate would
be increased to approximately 400 per second. This would greatly reduce the time taken to
determine migration and purge candidates. Then the suggested limit for the number of objects on
an aggregate could be raised to 2 million objects.
7.6.2.7.3 Fileset Quotas
The Episode fileset quota is currently stored in a 32-bit field (actually only 2^31 – 1 can be stored).
The quota represents 1K allocations. So, the current maximum quota per fileset is about 2TB. The
Episode quota reflects the amount of data stored in the fileset plus the amount of anode space. The
quota can not be increased beyond 2TB. When the quota limit is reached the only way to store more
files is to remove old files.
When the HDM purges data from Episode the quota is not affected. The quota reflects all data ever
stored through the DFS interface. If a file is written to a mirrored fileset through the HPSS
interfaces, the quota is not altered (except for anode space) until the data is actually read through
the DFS interface. So, it is possible to write more than 2TB to a mirrored fileset, but only if the data
is written through a non-DFS interface and never read through the DFS interface (This may occur
if large data dumps are made directly to HPSS, but the client wishes to use DFS to manage the name
space).
There are no current plans to change the Episode fileset quota implementation.