Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
Batch Job Submission
####################

Overview
========

Batch job submission from containers is designed to work exactly like on a login
node for the cluster. The workload will need to be annotated in order to get the
Expand All @@ -11,8 +13,8 @@ necessary configuration injected at runtime.
:header: "Cluster", "Annotation", "Value", "Schedulers"
:widths: 5, 10, 5, 5

"Marble", "ccs.ornl.gov/batchScheduler", "true", "Slurm, LSF"
"Onyx", "ccs.ornl.gov/batchScheduler", "true", "LSF"
"Marble", "ccs.ornl.gov/batchScheduler", "true", "Slurm"
"Onyx", "ccs.ornl.gov/batchScheduler", "true", "Slurm"

You can add the required annotations to any workload object such as a Pod, Deployment,
or a DeploymentConfig. Submitting a batch job from a container requires access to
Expand All @@ -23,15 +25,54 @@ the OLCF shared filesystems so that annotation is also included.
metadata:
annotations:
ccs.ornl.gov/batchScheduler: "true"
ccs.ornl.gov/fs: olcf
ccs.ornl.gov/fs: orion


Deployment Example
==================

Full example of a deployment using a base image provided by OLCF.

.. note::
Batch job submission from containers uses SSH to access the submission host. If you
use your own image you must install the **openssh client** package in your image.


Marble
------

.. code:: yaml

apiVersion: apps/v1
kind: Deployment
metadata:
name: test-jobsubmit
annotations:
ccs.ornl.gov/batchScheduler: "true"
ccs.ornl.gov/fs: orion
spec:
replicas: 1
selector:
matchLabels:
app: test-jobsubmit
template:
metadata:
labels:
app: test-jobsubmit
spec:
containers:
- name: test-jobsubmit
# this image was chosen because it has openSSH installed, please build your own image with openSSH for production use
image: "linuxserver/openssh-server:latest"
command: ["/bin/sh","-c"]
args:
- cat
tty: true
stdin: true

Onyx
----

.. code:: yaml

apiVersion: apps/v1
Expand All @@ -40,7 +81,7 @@ Full example of a deployment using a base image provided by OLCF.
name: test-jobsubmit
annotations:
ccs.ornl.gov/batchScheduler: "true"
ccs.ornl.gov/fs: olcf
ccs.ornl.gov/fs: wolf2
spec:
replicas: 1
selector:
Expand All @@ -53,11 +94,17 @@ Full example of a deployment using a base image provided by OLCF.
spec:
containers:
- name: test-jobsubmit
image: "image-registry.openshift-image-registry.svc:5000/openshift/ccs-rhel7-base-amd64:latest"
# this image was chosen because it has openSSH installed, please build your own image with openSSH for production use
image: "linuxserver/openssh-server:latest"
command: ["/bin/sh","-c"]
args:
- cat
tty: true
stdin: true
stdinOnce: true


Additional Information
======================

The annotation will install wrappers into /usr/bin:

Expand Down
92 changes: 83 additions & 9 deletions services_and_applications/slate/access_olcf_resources/mount_fs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@
Mount OLCF Filesystems
######################

Overview
========

OLCF shared filesystems can be mounted into a container running in Slate. The mountpoints
will be the same as a cluster node. The Kubernetes object will need to be annotated in order
to get the necessary configuration injected into the container at runtime.
Expand All @@ -10,58 +13,125 @@ to get the necessary configuration injected into the container at runtime.
:header: "Cluster", "Annotation", "Value", "Mounts"
:widths: 5, 8, 5, 25

"Marble", "ccs.ornl.gov/fs", "alpine2", "/ccs/sw, /ccs/home, /ccs/sys, /ccs/proj, /gpfs/alpine2"
"Marble", "ccs.ornl.gov/fs", "orion", "/ccs/sw, /ccs/home, /ccs/sys, /ccs/proj, /lustre/orion"
"Marble", "ccs.ornl.gov/fs", "kronos", "/ccs/sw, /ccs/home, /ccs/sys, /ccs/proj, /nl/kronos"
"Onyx", "ccs.ornl.gov/fs", "themis", "/ccsopen/sw, /ccsopen/home, /ccsopen/proj, /nl/themis"
"Onyx", "ccs.ornl.gov/fs", "wolf2", "/ccsopen/sw, /ccsopen/home, /ccsopen/proj, /gpfs/wolf2"

For each of the following examples, replace "ccs.ornl.gov/fs=olcf" or "ccs.ornl.gov/fs: olcf" with an annotation from the above list.

If you already have a Deployment running you can add the annotation with the client

.. code:: bash

oc annotate deployment web ccs.ornl.gov/fs=olcf
oc annotate deployment web ccs.ornl.gov/fs=orion

.. code:: bash

# or, on Onyx

oc annotate deployment web ccs.ornl.gov/fs=wolf2

.. warning::
You cannot annotate an existing pod because the injection happens at pod creation time

.. pull-quote::

Annotating a pod not managed by a deployment with ``oc annotate pod test ccs.ornl.gov/fs=olcf``
Annotating a pod not managed by a deployment with ``oc annotate pod test ccs.ornl.gov/fs=[orion,wolf2]``
will not work. Instead delete the pod and add the annotation to the metadata and recreate it.

Examples
========

General
-------

You can also add the annotations to any workload object's YAML such as a Pod, Deployment,
or DeploymentConfig.

Marble
^^^^^^

.. code:: yaml

kind: Deployment
metadata:
annotations:
ccs.ornl.gov/fs: olcf
ccs.ornl.gov/fs: orion
---
kind: DeploymentConfig
metadata:
annotations:
ccs.ornl.gov/fs: olcf
ccs.ornl.gov/fs: orion
---
kind: Pod
metadata:
annotations:
ccs.ornl.gov/fs: olcf
ccs.ornl.gov/fs: orion

Onyx
^^^^

.. code:: yaml

kind: Deployment
metadata:
annotations:
ccs.ornl.gov/fs: wolf2
---
kind: DeploymentConfig
metadata:
annotations:
ccs.ornl.gov/fs: wolf2
---
kind: Pod
metadata:
annotations:
ccs.ornl.gov/fs: wolf2

Deployment Example
------------------

Full example of a deployment mounting the OLCF shared filesystems:

Marble
^^^^^^

.. code:: yaml

apiVersion: apps/v1
kind: Deployment
metadata:
name: test-fs
annotations:
ccs.ornl.gov/fs: orion
spec:
replicas: 1
selector:
matchLabels:
app: test-fs
template:
metadata:
labels:
app: test-fs
spec:
containers:
- name: test-fs
image: busybox:latest
args:
- cat
stdin: true
stdinOnce: true

Onyx
^^^^

.. code:: yaml

apiVersion: apps/v1
kind: Deployment
metadata:
name: test-fs
annotations:
ccs.ornl.gov/fs: olcf
ccs.ornl.gov/fs: wolf2
spec:
replicas: 1
selector:
Expand All @@ -80,6 +150,10 @@ Full example of a deployment mounting the OLCF shared filesystems:
stdin: true
stdinOnce: true


Additional Information
======================

.. note::
There are no requirements in the container image in order to mount OLCF filesystems

Expand Down