How to consume EphemeralStorage in an AWS Batch job's container?

Hi,

Baiscally, ephemeral storage is disk space that you get only for the duration of you AWS batch job. It gets scratched after your job completes. You can get up to 200 GB of such storage with this featue.

This KB article explains how to configure and make use of this ephemeral storage in your AWS Batch setup: https://repost.aws/knowledge-center/ecs-fargate-increase-disk-space,

It may also help to read the announcement: https://aws.amazon.com/about-aws/whats-new/2023/03/aws-batch-configurable-ephemeral-storage-fargate/

Best,

Didier

Hello,

To consume ephemeral storage in an AWS Batch job's container, you can follow these steps:

Specify Ephemeral Storage in Job Definition:

• Add the ephemeralStorage parameter in your job definition.

For example:

"containerProperties": {
    "image": "your-docker-image",
    "resourceRequirements": [
        {
            "type": "VCPU",
            "value": "2"
        },
        {
            "type": "MEMORY",
            "value": "4096"
        }
    ],
    "ephemeralStorage": {
        "sizeInGiB": 100
    }
}

Access Ephemeral Storage in the Container:

• Ephemeral storage is automatically mounted to the container at /scratch.
• You can directly read from and write to /scratch within your container.

docker run your-docker-image sh -c "echo 'Hello World' > /scratch/hello.txt"

This will utilize the specified ephemeral storage location in your AWS Batch job's container.