HPC Allocations


This document defines the policies governing allocations for compute time and storage on the DesignSafe platform, a comprehensive natural hazards research environment for experimental, theoretical, and computational science. The allocation policy for DesignSafe considers that users may have a range of use cases, from short term analysis work to long term data storage, for very large or very small experiments. All registered users are provided access to the Tools & Applications in a controlled manner that limits the amount of computing capacity, and they may request an additional Allocation for various purposes such as the need for larger scale computation or to run applications that are not provided via the portal. New use cases for DesignSafe may emerge—please don’t hesitate to contact the project team if you have additional needs.

Who is eligible for an Account?

DesignSafe is broadly available to any researcher or practitioner working in open Natural Hazards Engineering and Social Science research. By “open,” the expectation is that any research performed on DesignSafe is unclassified, and will result in publication in a broadly available journal or conference.

Who is eligible for Allocations?

All registered users are provided access to the Tools & Applications in a controlled manner that limits the amount of computing capacity, and they may request an additional Allocation for various purposes such as the need for larger scale computation or to run applications that are not provided via the portal. An additional Allocation via DesignSafe is headed by a project PI, typically a faculty member or research scientist at a US-based academic institution (private sector users are also welcome, see below). The PI may then add additional researchers, including students, to their allocation at their discretion. PIs are responsible for ensuring that any users added to their allocation comply with the terms and conditions for use of the DesignSafe resources. Collaborative projects involving non-U.S. researchers are encouraged as long as they include substantive intellectual participation by the U.S. researchers. In joint research projects, foreign collaborators are eligible to make use of that allocation in a manner consistent with the request. While a PI is typically a faculty member or staff researcher at a U.S. academic institution, there are a number of other cases where an individual may be eligible to be a PI:

  • NSF Graduate Student Research Fellowships: While in most cases, a graduate student is ineligible to be PI of an allocation request, an exception is made for recipients of NSF Graduate Student Fellowships. Recipients of these NSF awards can submit requests for Startup allocations. Include supporting documentation (grant number or an award letter) as part of the request submission.
  • Other federal agencies: Research staff employed by federal agencies or non-NSF FFRDCs are eligible to apply for an allocation if their agency or center does not typically provide research staff with access to resources of adequate scope for the planned research.
  • State educational offices or organizations and local school districts may submit allocation requests intended to broaden the impact, accelerate the pace, and increase the effectiveness of improvements in science, mathematics, and engineering education in both K-12 and post-secondary levels. A teacher or educator at an accredited public or private K-12 school is eligible to apply for an allocation as PI.
  • Independent museums, observatories, libraries, research laboratories, professional societies and similar organizations in the United States that are directly associated with educational or research activities are eligible.
  • U.S. commercial organizations, especially small businesses with strong capabilities in scientific or engineering research or education may apply for an allocation. The DesignSafe project is interested in supporting projects that couple industrial research resources and perspectives with those of universities; therefore, it especially welcomes requests from cooperative projects involving both universities and the private commercial sector. It is necessary for these projects to submit their work in an open forum, and make the work readily available to the public.

What are the units of an allocation and how am I charged?

DesignSafe provides access to a variety of computing and storage resources, accessible through several different interfaces: via the web-based Tools & Applications portal, or via API's, or via direct command-line access to TACC's HPC systems. Web-based access via Tools & Applications has built-in constraints that limit the amount of compute time that is automatically provided to each user at the time they create an account. For users requiring access via API or direct command-line access to HPC resources, they can request additional Allocations of computation time and/or data storage. Compute time is allocated in Service Units (SUs), which equates to one hour of wall clock time on a single compute node.

How do I apply for an Additional Allocation?

Additional allocations may be requested via the DesignSafe Portal by submitting a help ticket. An application for an additional allocation includes information about the eligibility of the requestor, a description of the research to be performed and its sources of support, and a justification for the number of SUs requested.

Allocation Sizes and Limits

There are three types of additional allocations available via DesignSafe:

  • Startup Allocation — Startup projects target new users exploring the use of DesignSafe beyond the level of automatic allocations via Tools & Applications and/or planning to submit more substantial requests in the future as well as users who have modest requirements for the testbed that nonetheless can’t be met by local or institutional resources. Startup projects receive a fast track internal review. A Startup Allocation may request up to 20,000 node-hours on Frontera, and up to 1 terabyte of storage.
  • Research Allocation -- Research projects are designated for projects that have progressed beyond the startup phase and are conducting production usage of the infrastructure in pursuit of their research goals. Research allocations are subject to the full review process. A Research allocation has no mandatory maximum size; however, a maximum size of 100,000 node-hours annually on Frontera and 100TB of data storage is recommended. Requests above this limit will be considered only in exceptional circumstances with additional justification, and it is recommended the requestor contact the project team to discuss the request before submitting.
  • Educational Allocation — Education projects target faculty or teachers intending to use DesignSafe resources for classroom instruction or training classes related to cloud computing technologies. Educational Allocations receive fast track review. An Educational allocation may request up to 10,000 node-hours on Frontera and 1 terabyte of storage.

Each of these allocation types is provided through NSF funding at no direct cost to the end user. Users who need additional allocation above what they are allocated, or who are not eligible for an NSF-provided allocation, may choose to purchase additional compute time or storage capacity from DesignSafe. These services will be provided based on the underlying cloud services rates in effect at the Texas Advanced Computing Center at the time of purchase. As of this writing, storage is available for approximately $100/TB/year, and compute time is available for $0.40 per node hour.

Allocation Duration

Allocations for DesignSafe resources are made for up to a 12-month period. PIs can continue their activities in subsequent years through annual renewal requests. At the end of each 12-month allocation period, projects forfeit any unused compute SUs. Users with a Startup Allocation need not wait the full 12 months to apply for a Research Allocation, and can apply as soon as they receive preliminary results suitable for their request. Educational Allocations will typically be limited in duration to the academic semester of the class in which they take place.

Review Criteria for Additional Allocation Proposals

DesignSafe supplemental allocations are reviewed for merit by a committee consisting of members of the Allocation Advisory Board. The board makes recommendations to the project team based solely on the merit of the proposal, and not on overall availability of the resource. The board will review the proposal and make a recommendation to the project team based on the following criteria:

  • Scientific merit and significance of the proposed experiment.
  • A methodology appropriate to the use of the DesignSafe resources.
  • Success of prior or other existing allocations (for renewals) in terms of published research results and new funding.
  • Use of other NHERI-funded facilities.
  • Any funded support for the project (not required, but preference may be given to NSF supported projects using NHERI infrastructure; startup allocations can be made for exploratory research that has not yet attracted a source of funding).

Proposals that are deemed competitive by the Allocation Advisory Board are then reviewed by the project team for technical feasibility (i.e., can the project be implemented in the DesignSafe environment?) and for the availability of time on the resource. For Research Allocations, proposals will be reviewed at least once per quarter. Startup and Educational Allocation requests receive a fast track review, bypassing the Allocation Advisory Board. DesignSafe project staff will review these requests according to the merit review criteria above, and will act on these requests within two weeks of their submission.

Conflict of Interest and Confidentiality

Every effort is made to avoid conflicts of interest. Reviewers are not allowed to review or be present for the discussion of requests from their home institution, former students, postdocs, advisors, or current and recent collaborators. If in the opinion of a PI a certain individual has a conflict of interest, the PI may request that the individual not act as reviewer on their request. All reviews remain confidential and are made available only to the PI and Co-PIs submitting the request, assigned reviewers from the Allocation Advisory Board, DesignSafe project staff involved in the allocation review process, and NSF program officers for the DesignSafe project. While the contents of reviews and details of allocation requests and experiments remain confidential, a list of projects receiving allocations and the general area of research may be made public on the DesignSafe web site and used in project reports and presentations.

Supplemental Requests

A supplement is a request for additional resources during an existing allocation's one year time frame. Its purpose is to support changes in the original computational research plan that are required to achieve the scientific goals of the project. This may include altered or new goals, or support for projects proceeding more rapidly than anticipated or that require more resources than anticipated. Supplement awards are highly dependent upon availability of resources and limited to when allocation awards at the previous allocation period have been reduced to eliminate oversubscriptions. Supplements are not a mechanism to acquire additional resources for awards that were recommended for less than the amount originally requested. Supplemental requests require the filing of a progress report that documents what has been done with the originally awarded allocation, and makes the case for additional resources.

Extension Requests

Extensions of allocation periods beyond the normal 12-month duration can be requested. This request brings no new allocation, but keeps unused allocations from expiring. A brief reason is required for not using the awarded allocation, but no formal documentation is needed. Note that granting extension requests reduces the amount of allocation that can be given to users in the next year.

Allocation Proposal Format

A Startup Allocation request consists of the following components:

  • PI Name, Title, and Organization
  • (optional) Co-investigator names, titles, and organizations
  • Summary of the proposed experiments (1 page maximum)
  • Estimated resources required (include a short justification of the number and type of nodes you will request, and any specific software, hardware, or networking requirements. (1 page maximum)
  • List any sources of funding (or pending funding) that support the proposed research, including funding agency and grant name(s) (Startup Allocations may be for unfunded exploratory research)

An Educational Allocation request consists of the following components:

  • PI Name, Title, and Organization
  • (optional) Co-investigator names, titles, and organizations
  • Summary of the proposed course or training session to be supported, including the expected number of students who will participate (1 page maximum).
  • Estimated resources required (include a short justification of the number and type of nodes you will request, and any specific software, hardware, or networking requirements. (1 page maximum)
  • If applicable, list any sources of funding that support the proposed course, including funding agency and grant name(s) (Educational Allocations do not require any external funding, but in cases where it does exist, the project would like to record any grants that are impacted by Chameleon).

A Research Allocation request consists of the following components:

  • PI Name, Title, and Organization
  • (optional) Co-investigator names, titles, and organizations
  • Summary of the proposed research (2 pages maximum).
  • Results from any prior Allocations, including Startups.
    • Include a list of any publications resulting from these allocations, including conference presentations, technical reports, in preparation manuscripts, etc.
    • Include a list of any broader impacts, including students or postdocs trained, etc.
  • Estimated resources required, which must include:
    • A justification of the number and type of nodes you will request, and any specific software, hardware, or networking requirements. Wherever possible, justify this request with performance data from smaller scale experiments that validates the scope of the need (2 pages maximum)
  • List any sources of funding (or pending funding) that support the proposed research, including funding agency and grant name(s).
  • CVs (no page limit)
  • References (optional, no page limit)

Document Formatting

While readability is of greatest importance, documents must satisfy the following minimum requirements. Documents that conform to NSF proposal format guidelines will satisfy these guidelines.

  • Margins: Documents must have 2.5-cm (1-inch) margins at the top, bottom, and sides.
  • Fonts and Spacing: The type size used throughout the documents must conform to the following three requirements:

Use one of the following typefaces identified below:

  • Arial 11, Courier New, or Palatino Linotype at a font size of 10 points or larger;
  • Times New Roman at a font size of 11 points or larger; or
  • Computer Modern family of fonts at a font size of 11 points or larger.

A font size of less than 10 points may be used for mathematical formulas or equations, figures, table or diagram captions and when using a Symbol font to insert Greek letters or special characters. PIs are cautioned, however, that the text must still be readable.

  • Page Numbering: Page numbers should be included in each file by the submitter.

Allocation Limit per PI

An individual may be a PI on only one active DesignSafe allocation request at a given time. Several distinct research activities can be combined in a single allocation request, however, the allocation request for each activity must be justified, and any allocation-size limits apply to the aggregate request.

The single-allocation rule is designed to minimize the effort required by PIs for submitting allocation requests and the overhead to the process for reviewing those requests. While PIs may have several different funded grants that require computational support, these should be included as sub-sections within a single allocation request.

There are several exceptions to this rule:

  • A PI may submit a Research request at any time during an active Startup allocation. A successful Research allocation will supersede any Startup allocations and start a new 12-month allocation period.
  • If a PI is leading an individual research activity as well as a community consortium activity or community service activity, separate requests for these distinct activities will be considered.
  • A PI with active Startup or Research projects may submit requests for any number of Educational allocations to keep classroom and training users separate from research activities.

Similarly, to minimize the effort required to gain access to DesignSafe, closely collaborating researchers should submit a single collaborative allocation request rather than several individual requests. For example, a PI and associated post-doctoral researchers; investigators supported by the same funding grant; and researchers in the same lab group should consider submitting a request describing and justifying the various sub-activities. One of the collaborators is designated as the PI, and others can be designated as co-PIs.

Acknowledgement of Support

An acknowledgement of support from the DesignSafe project and the National Science Foundation should appear in any publication of material, whether copyrighted or not, that describes work which benefited from access to DesignSafe cyberinfrastructure resources. If you wish to acknowledge DesignSafe, the following paper can be used as a reference:

Rathje, E., Dawson, C. Padgett, J.E., Pinelli, J.-P., Stanzione, D., Adair, A., Arduino, P., Brandenberg, S.J., Cockerill, T., Dey, C., Esteva, M., Haan, Jr., F.L., Hanlon, M., Kareem, A., Lowes, L., Mock, S., and Mosqueda, G. 2017. “DesignSafe: A New Cyberinfrastructure for Natural Hazards Engineering,” ASCE Natural Hazards Review, doi:10.1061/(ASCE)NH.1527-6996.0000246.

Guidelines for a Successful Allocation Request

Well-written research allocation requests will meet the following guidelines:

  • The research is summarized in context of the current state of the art; outlines the computational methods to be used; and relates those methods to subsections of the request.
  • Provide sufficient information, without overwhelming details to the reviewers.
  • The justification for the request is clear, and closely coupled to the proposed experiments and needs, so that if the committee needs to reduce the original request, it can be done rationally with minimum disruption to the investigator.
  • Summarize results from relevant previous allocations, including manuscripts published, accepted, submitted, or in preparation, and relate these results to the current request.

The purpose of DesignSafe is to accelerate research progress in natural hazards engineering and to provide cyber services to NHERI awardees. The purpose of the allocations policy is to insure the public investment in DesignSafe is used in the best and fairest possible way across the open research community, while making the request process as simple as possible. Following the principles above helps insure fast and fair decision making for this large public investment.

The project team is interested in receiving as many excellent requests as possible, and encourages any potential investigators who wish to use the resource to not hesitate in contacting the team for additional help in preparing this request, with questions about the process, or to discuss any type of project that may not fit the structure described here. The project team also welcomes input in improving this process, and encourages feedback on both this document and the process.