Commit 4daabde7 authored by Patrick Michalsky's avatar Patrick Michalsky
Browse files

Update README.md

parent 367281cc
# justus2-demo
This repository serves as an example for running machine learning algorithms on the [Justus2](https://wiki.bwhpc.de/e/Category:BwForCluster_JUSTUS_2) bwForCluster located at the *University of Ulm*. The algorithm included corresponds to the official beginner's tutorial for Keras-based Tensorflow2 network architectures, which is provided and explained on the tensorflow [homepage](https://www.tensorflow.org/tutorials/quickstart/beginner?hl=en).
For the execution of the algorithm, the offered *Singularity* container platform is used. Further information about Singularity Images can be found at the following [link](https://sylabs.io/guides/3.6/user-guide/introduction.html).
### Instructions
1. Enter the *Docker* folder and execute the containing *create_image.sh* file locally. The call generates a compressed Docker container that contains all the necessary packages and dependencies to run the machine learning algorithm on a *High Performance Cluster*. The generation may take some time.
```bash
$ sudo bash create_image.sh
```
<br></br>
2. Copy or move the resulting *image.tar.gz* file to the *Server* folder.
```bash
$ cd ..
$ mv Docker/image.tar.gz Server/image.tar.gz
```
<br></br>
3. If a workspace has already been set up on the Justus2 server, this step can be skipped. Otherwise, a workspace must be created manually. This step is necessary to ensure the generation and execution of Singularity Images. A server-side tutorial for creating a temporary workspace can be accessed on the [Hardware and Architecture](https://wiki.bwhpc.de/e/Hardware_and_Architecture_(bwForCluster_JUSTUS_2)#Workspaces) page of the bwForCluster JUSTUS 2 documentation.
<br></br>
4. Transfer the *Server* folder including all contents to the Justus2 workspace. An easy way to transfer data to an external server can be achieved via *scp*:
```bash
$ scp -r Server <user>@<justus2>:<workspace>
```
<br></br>
5. Login to the Justus2 server and enter your workspace.
```bash
$ ssh <user>@<justus2>
$ cd <workspace>/Server
```
<br></br>
6. Execute the containing *queue.sh* file to submit the *jobscript.sh* file to the server's task list. During execution, the jobscript generates an independent Singularity image from the compressed Docker image and runs the embedded machine learning algorithm. Both the accuracy and the trained neural network model are saved in the workspace. The number of resources required for execution, such as the number of GPUs, can be defined in the *queue.sh*. The configuration options are managed by [Slurm](https://wiki.bwhpc.de/e/Slurm_JUSTUS_2).
```bash
$ bash queue.sh
```
<br></br>
7. The results can be found under the *results* folder in the workspace.
```bash
$ ls <workspace>/Server/results
```
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment