Rules of the final stage

1.The use of optimization methods specific to certain parameters or input data sets is strictly forbidden.

2.If any changes are made to the algorithm, the revised version must maintain mathematical equivalence to the original.

3.Violation of any rule mentioned above will result in a zero score being assigned for the corresponding task.

4.Properly constructing the cluster is essential. Any damage to the server may incur a penalty of up to 20 points for the team, as determined by the ASC24 Committee.

Performance Optimization

I. HPL performance optimization:

1.Platform requirement: The runtime power consumption must remain under 3 KW. Failure to comply will result in the invalidation of the current task.

2.Goal: Obtain the correct results while achieving the highest performance.

3.Software download: http://www.netlib.org/benchmark/hpl/

II. Performance optimization of HPCG:

1.Platform requirement: The runtime power consumption must remain under 3 KW. Failure to comply will result in the invalidation of the current task.

2.About run time: The runtime of HPCG (version 3.0) must be a minimum of 1800 seconds (30 minutes), as reported in the output file. The Quick Path option is not permitted.

3.Software download: https://github.com/hpcg-benchmark/hpcg

III. Performance optimization of OpenCAEPoro:

1.Platform requirement: The power restriction of the test platform is 3 KW. If the power consumption exceeds 3 KW, the results of the current task will be invalid.

2.Goal: The OpenCAEPoro challenge shares a similar objective with the preliminary round. The ASC24 committee will announce several OpenCAEPoro workloads during the finals. Each workload's results must pass correctness checking, and the objective is to minimize OBJECT TIME. It's important to note that modifying any code related to the method parameters is prohibited, and all parameters in the input files must remain unchanged.

3.Software download: https://github.com/OpenCAEPlus/OpenCAEPoro_ASC2024

IV. Performance optimization of GoMars:

1.Platform requirement: The power restriction of the test platform is 3 KW. If the power consumption exceeds 3 KW, the results of the current task will be invalid.

2.Goal: GoMars is a novel global open planetary atmospheric model designed for Mars. It offers insights into Martian meteorological conditions crucial for landing operations. The ASC24 committee will introduce various GoMars workloads during the finals. The results of each workload must undergo correctness checking, and the objective is to minimize the runtime of the GoMars application. Please note that the provided link only contains the dynamic core of GoMars, whereas additional codes for physical processes may be included in the final version.

3.Software download: https://gitee.com/dongli85/GMCORE

V. Performance optimization of the Mystery Application:

1.Platform requirement: The power restriction of the test platform is 3 KW. If the power consumption exceeds 3 KW, the results of current task will be invalid.

2.Goal: The ASC24 committee will announce several Mystery Application workloads during the finals. The results of each workload must pass correctness checking, and the objective is to minimize runtime.

VI. Performance optimization of LLM inference:

1.Platform requirement: The power limit for the test platform is 3 KW. Any power consumption exceeding this threshold during the contest will result in the invalidation of the current task's results.

2.Goal: The LLM inference challenge shares a similar goal with the preliminary round. AquilaChat2-34B will be used in the final round and the datasets used for evaluation will be disclosed on site. Besides, it is required to utilize 4bit or less quantization for inference acceleration during the final and the baseline quantization method is 4bit bitsandbytes. Moreover, the overall accuracy rather than only output length of dataset results will be taken into account, which should be controlled within a maximum error of 1.5% compared to the BF16 precision baseline.

3.Download: Model weight and the baseline code for bitsandbytes download link https://huggingface.co/BAAI/AquilaChat2-34B/

Team Presentation

1.Each team is required to present their results using PowerPoint (PPT) slides. The presentation order for each team will be determined by a draw. English must be used in both the PPT slides and in the presentations delivered by up to two student speakers.

2.The presentation must not exceed 7 minutes in duration. Any additional time will result in a reduction of your score accordingly. Following the presentation, judges will have approximately 3 minutes to ask questions.

3.The evaluation committee will assess the presentation of each team.

4.The team advisor is welcome to observe her/his team's presentation session.

Q1: What's the 'final version' of GoMars? [GoMars]

Answer: Follow the URL of the notice, and download the code in master branch updated after Feb 20th, 2024.

Q2: Can you provide a method for verifying correctness? [GoMars]

Answer: We have a set of standards and methods for the accuracy verification, but cannot be provided directly. Very slight errors are acceptable (for example, around n% margin of error).

Q3: How to use mars data? Is it necessary for us to build GoMars with capabilities in CAM physics and chemistry based on GPTL, PIO, LAPACK, ESMF, MCT to complete the tasks on the final site of the competition? [GoMars]

Answer: The final task only contains the dynamic core, which means no extra data, physics or chemistry processes are needed in the final competition.

Q4: Is it mandatory to use the BitsAndBytes quantization method for LLM challenge？[LLM challenge]

Answer: BitsAndBytes is not mandatory, you can apply 4-bit or 4-bit lower quantization methods as long as the requirement of 1.5% error threshold is satisfied compared to the BF16 precision baseline.

Q5: Could you provide some details about the evaluation of the overall accuracy of the model? [LLM challenge]

Answer: The evaluation is like a scoring on a MMLU benchmark. The datasets and methods used for evaluation will be disclosed on site during the final round. Before that, you can use MMLU for test.

Q6: Is there any restrictions for speculative decoding, fine-tuning and pruning methods in the final round?[LLM challenge]

Answer: Speculative decoding, fine-tuning and pruning are still not permitted during the final.