AAIC Workload

This is a workload set of jobs run in GPU queues of AIST Artificial Intelligence Cloud (AAIC). As AAIC is dedicatedly used for AI research, many jobs in the workload are expected to be run for training deep learning applications.

The current workload contains jobs submitted from July 14th, 2017 to December 31st, 2018.

For detail about AAIC, visit AAIC Homepage.

The workload is also published on GitHub.

AAIC System Overview

AAIC Architecture

AAIC consists of 50 compute nodes equipped with GPUs (GPU nodes) and 68 CPU only compute nodes. Workloads published on this page are obtained from a batch job scheduling system that manages GPU nodes. Although there are 50 GPU nodes, the scheduling system manages 32 to 40 nodes depending on utilization. Remaining GPU nodes are used for other purposes.

The table shows the resource quantity of a GPU node.

AttributeValue
# CPU cores20
# GPUs8
Memory256 GiB
SSD Capacity480 GB

Here are the basic scheduling policy.

Workload Files

Workloads published in this page are formatted in CSV and distributed by gzip compressed.

There are two kind of workload files.

Workload Format

All workload files are formatted in CSV and have the same format. Each line in a workload file is a job. Each job record has the following 13 attributes.

OrderAttributeMeanings
1Job IDA unique sequential number assigned to each job. The number is unique only in a workload file.
2UserID of a user who submit the job. User ID is unique only in a workload file.
3GroupID of a group where a user who submit the job belongs. Group ID is unique only in a workload file.
4QueueA queue where the job submitted. Queue name is an integer number.
5NodesNumber of used nodes by the job.
6GPUsNumber of used GPUs within a node by the job.
7Requested WalltimeWalltime requested by job submitter.
8TermcodeJob termination code: success(0), fail(1), walltime exceeded(2), canceled by user(3) or canceled by system(4).
9Submit TimestampTime when job is submitted in UNIX timestamp.
10Start TimestampTime when job starts running in UNIX timestamp.
11Finish TimestampTime when job ends running in UNIX timestamp.
12Wait TimeDuration between time job submitted and time job starts in seconds.
13WalltimeActual walltime of the job in seconds.

Usage and Acknowledgment

AAIC workload is copyrighted by AIST, National Institute of Advanced Industrial Science and Technology (国立研究開発法人 産業技術総合研究所 知財管理番号: H31PRO-2307).

If you use this workload in your work, please add the following or a similar acknowledgement.

If you cite this workload, please add the following or a similar reference.

Publications

Papers and reports that use this workload.

Links