Skip to main content

Environment Migration

Due to the network access restrictions of the HPC Phase III platform, users can set up the development environment locally or on the HPC Phase II platform and then migrate it to the HPC Phase III platform for actual training or computing work.

Conda Environment Migration​

Conda Environment Packaging​

The following conda environment packaging operations are carried out locally or on the HPC Phase II platform.

View and Activate the Environment​

conda env list

conda activate <env>

alt text

alt text

Install conda-pack​

conda install -c conda-forge conda-pack

Package the Specified Environment​

conda pack -n your_env_name -o your_env_name.tar.gz

Conda Environment Upload​

The following conda environment upload operations are carried out on the HPC Phase III platform.

Upload the Packaged File​

Upload the packaged file to the HPC Phase III platform through file upload operations or the scp command. Taking the upload from the HPC Phase II platform as an example:

scp username@hpc2login.hpc.hkust-gz.edu.cn:<tar_filepath> /data/user/username/

Unpack the Packaged File​

tar -xzf <tar_file> ~/<unpack_path>

Restore the Conda Environment​

cd <unpack_path>

python3 <unpack_path>/bin/conda-unpack

Activate the Conda Environment​

source <unpack_path>/bin/activate

Container Image Migration​

Image Packaging​

The following image packaging operations are carried out locally or on the HPC Phase II platform. The following takes the HPC Phase II platform as an example.

Save the Image​

Save the job image in the development center.

alt text

Export the Image​

Export the saved image to the storage of the HPC Phase II platform in [My Images] (a new namespace needs to be created).

alt text

Image Upload​

The following image upload operations are carried out on the HPC Phase III platform.

Image File Transfer​

Upload the image file from the HPC Phase II platform to the HPC Phase III platform through the scp command.

For operation references: Data Management --- Data Migration --- Data Migration of HPC Phase II Platform

Upload the Image via the Client​

For operation references: AI Development (Container) --- Basic Services --- Image Management --- Upload Image

Image Building​

The image uploaded from the HPC Phase II platform needs to be built before it can be used normally.

alt text

alt text

Create the Development Environment​

Start the development environment using the built image.

alt text