1.4.2.2 Submitting Docker Jobs

I. Overview of Steps:

1. Upload the docker image with the prepared environment to the local repository: 10.120.24.15:5000/jhinno

2. Submit an LSF job, pull the previous image on an HCP node, and run this docker on an HPC node

3. Use docker cp to transfer files needed for running and retrieve results

II. Example Explanation:

1. Assuming a locally created image named gem5-test:v1 has been made using docker commit, it needs to be tagged and uploaded to the local repository: 

        docker tag gem5-test:v1 10.120.24.15:5000/jhinno/gem5-test:v1 

    Upload:

        docker push 10.120.24.15:5000/jhinno/gem5-test:v1


2. Write a script for running docker (gem5.sh) and a job submission script for LSF:

     2.1 This job will perform a simple gem5 compilation: run.sh

     cd /hpc/home/simonyjhe/docker-data

     vim run.sh

        #!/bin/bash

        cd /opt/gem5

        scons build/X86/gem5.opt -j 100 /opt/mydata/output.log 2>&1

     Make gem5.sh executable: chmod a+x run.sh

     2.2 Write the LSF job script myjob.lsf:

        #BSUB -J gem5-test-job1

        #BSUB -n 56

        #BSUB -q gpu

        #BSUB -Is

        docker pull 10.120.24.15:5000/jhinno/gem5-test:v1

        docker run -it --rm --name gem5-yjhe -v /hpc/home/simonyjhe/docker-data:/opt/mydata 10.120.24.15:5000/jhinno/gem5-test:v1 /opt/mydata/run.sh 


Explanation:

        #BSUB section is for LSF job parameters: request 56 cores, submit to the gpu queue, as an interactive job

        docker section: pull -> docker run

             -it: interactive mode with terminal

             --rm: remove container upon exit; does not delete the image

             -v /hpc/home/simonyjhe/tutorials/docker/gem5:/opt/mydata: maps the working directory to /opt/mydata in the container. Data written into this directory in the container, such as output.log which is intended to be saved, can be accessed.

     
     2.3. Submit the job: bsub < myjob.lsf &