diff --git a/_data/navigation.yml b/_data/navigation.yml index 18f98c7..9d2b05c 100644 --- a/_data/navigation.yml +++ b/_data/navigation.yml @@ -21,6 +21,16 @@ usemenu: url: /guide-e4s-bot/ - title: "E4S Quick Start" url: /quick-start/ + - title: "PyTorch Quick Start" + url: /quick-start-pytorch/ + - title: "TensorFlow Quick Start" + url: /quick-start-tensorflow/ + - title: "NeMo Quick Start" + url: /quick-start-nemo/ + - title: "BioNeMo Quick Start" + url: /quick-start-bionemo/ + - title: "vLLM Quick Start" + url: /quick-start-vllm/ - title: "HYPRE Quick Start" url: /quick-start-hypre/ - title: "LAMMPS Quick Start" @@ -41,6 +51,8 @@ usemenu: url: /container-installation/ - title: "E4S Container Launch" url: /container-launch/ + - title: "Frontier E4S Container Demo" + url: /frontier-container-demo/ - title: "Test with E4S" url: /test-with-e4s/ - title: "ParaTools Pro for E4S™" diff --git a/frontier-container-demo.md b/frontier-container-demo.md new file mode 100644 index 0000000..8643004 --- /dev/null +++ b/frontier-container-demo.md @@ -0,0 +1,113 @@ +--- +layout: single +title: "Frontier E4S Container Demo" +permalink: /frontier-container-demo/ +classes: wide +sidebar: + nav: "usemenu" +--- + +{% include e4s-page-actions.html %} + +# Frontier E4S container demo + +This example demonstrates a realistic E4S workflow on OLCF Frontier using the ROCm-enabled E4S container image. The goal is to show how to: + +- fetch a prebuilt E4S Apptainer/Singularity image into a scratch area, +- build and run a small example inside the container using Spack provided software, +- allocate a GPU job on Frontier, +- and execute a real GPU application from the container environment. + +This is a good fit for users who want to prototype GPU workloads on Frontier without maintaining a separate host build environment for every dependency. + +## Prerequisites + +Before starting, make sure you are logged into Frontier with a valid project allocation and that the standard OLCF environment is active. A typical workflow uses the container and MPI modules that are already provided by the system. + +The commands below assume you have a project allocation and the ability to write to a scratch area under `$MEMBERWORK`. + +## 1) Download the E4S ROCm image to scratch + +Use a scratch directory under `$MEMBERWORK` so the image is stored on a location appropriate for large temporary data. + +```bash +mkdir -p ${MEMBERWORK}/e4s-demo +cd ${MEMBERWORK}/e4s-demo + +wget https://oaciss.nic.uoregon.edu/e4s/images/26.06/e4s-rocm90a-x86_64-26.06.sif +ls -lh e4s-rocm90a-x86_64-26.06.sif +``` + +This image is approximately 24 GB and may take around 20 minutes to download, depending on network conditions. + +> There is also a CPU-only E4S image for MPI/thread-oriented workloads that do not rely on GPU-accelerated software. This ROCm image is the right choice for the GPU benchmark shown here. Other ROCm images are available for different GPU hardware versions. + +## 2) Build a PETSc example inside the container + +This step uses the container to source Spack, load a package, and compile a small PETSc example. + +```bash +cd ${MEMBERWORK}/e4s-demo + +apptainer exec --rocm e4s-rocm90a-x86_64-26.06.sif bash -c ' + source /spack/share/spack/setup-env.sh 2>/dev/null || true + spack load petsc + PETSC_DIR=$(spack location -i petsc) + cp ${PETSC_DIR}/share/petsc/examples/src/snes/tutorials/ex19.c . + cp ${PETSC_DIR}/share/petsc/examples/src/snes/tutorials/makefile . + make ex19 +' +``` + +This compiles a representative example from PETSc into the working directory inside the container. Critically, the build is executed within the container environment where the correct Spack packages and toolchain are already set up. + +## 3) Request an allocation on Frontier + +Use `salloc` to reserve compute resources before running the actual GPU workload. + +```bash +salloc -A -N 1 -t 60 -q batch +``` + +Once the allocation is active, move back into the demo directory and load the container support modules needed for GPU-aware containerized execution. + +```bash +cd ${MEMBERWORK}/e4s-demo +module load apptainer-enable-gpu apptainer-enable-mpi +``` + +## 4) Run the workload in the container + +The final step uses a single node with 8 tasks and one GPU per task. This matches the Frontier GPU setup and launches the benchmark inside the E4S container with the correct MPI and ROCm configuration. + +```bash +srun -N 1 -n 8 --ntasks-per-node=8 --gpus-per-task=1 --gpu-bind=closest --mpi=pmi2 \ + apptainer exec --rocm e4s-rocm90a-x86_64-26.06.sif \ + bash -c "source /spack/share/spack/setup-env.sh && spack load slate && unset CUDA_VISIBLE_DEVICES HIP_VISIBLE_DEVICES; slate_tester --origin d --target d --dim 10240 --nb 512 gemm" +``` + +This command is intentionally simple to illustrate the overall pattern: + +- the image is launched with ROCm support enabled, +- the Spack environment is loaded inside the container, +- and the GPU benchmark runs with the container-resident BLAS/LAPACK stack and host MPI/GPU scheduling. + +## Why this workflow matters + +This demonstration shows an E4S usage pattern that is common on HPC systems: + +- container images provide a known-good scientific software stack, +- Spack within the container resolves dependencies consistently, +- and the host scheduler still controls the job placement and GPU assignments. + +That combination makes it easier to reproduce performance studies and to run complex GPU software stacks without rebuilding a full environment on the host. + +## Related E4S container pages + +For additional container guidance and broader installation patterns, see: + +- [E4S Container Installation](/container-installation/) +- [E4S Container Launch](/container-launch/) +- [Test with E4S](/test-with-e4s/) + +Please contact the E4S team if you want help adapting this workflow to a different machine, MPI stack, or benchmark. diff --git a/use-bionemo-quick-start.md b/use-bionemo-quick-start.md new file mode 100644 index 0000000..b1420ad --- /dev/null +++ b/use-bionemo-quick-start.md @@ -0,0 +1,86 @@ +--- +layout: single +title: "BioNeMo Quick Start" +permalink: /quick-start-bionemo/ +classes: wide +sidebar: + nav: "usemenu" +--- + +{% include e4s-page-actions.html %} + +This page shows a BioNeMo workflow in the E4S 26.06 CUDA container. The examples here are intended to be run inside the Docker image shown below. + +```bash +docker run --gpus all -it --rm \ + --entrypoint bash \ + -v "$PWD:/work" \ + -w /work \ + ecpe4s/e4s-cuda:26.06 +``` + +## 1. Verify BioNeMo + +The E4S container includes `bionemo-core` and related BioNeMo packages. + +```bash +python3 - <<'PY' +import bionemo.core +import bionemo.moco +import bionemo.noodles +import bionemo.scdl + +print('bionemo.core OK:', bionemo.core.__file__) +print('BioNeMo package imports OK') +PY +``` + +## 2. Run an ESM-2 inference example + +This example loads an ESM-2 checkpoint and computes a sequence embedding with the container's GPU-enabled deep-learning stack. + +Create a script: + +```bash +cat > bionemo_esm2_recipe_demo.py <<'PY' +from transformers import AutoModel, AutoTokenizer +import torch + +device = 'cuda' if torch.cuda.is_available() else 'cpu' +model = AutoModel.from_pretrained( + 'nvidia/esm2_t6_8M_UR50D', + trust_remote_code=True, +).to(device) +tokenizer = AutoTokenizer.from_pretrained( + 'nvidia/esm2_t6_8M_UR50D', + trust_remote_code=True, +) + +sequence = 'MSKGEELFTGVVPILVELDGDVNGHKFSVSGEGEGDATYGKLTLKFICTTGKLPVPWPTL' +inputs = tokenizer(sequence, return_tensors='pt').to(device) +with torch.no_grad(): + output = model(**inputs) + +print('Output embedding shape:', output.last_hidden_state.shape) +PY +``` + +Run the example: + +```bash +python3 bionemo_esm2_recipe_demo.py +``` + +The first run downloads the checkpoint and may print model-loading details. The `UNEXPECTED` entries for `lm_head` weights can be ignored for this embedding model. A successful run ends with output similar to: + +```text +Output embedding shape: torch.Size([1, 62, 320]) +``` + +The exact sequence length can vary with the input sequence; the final dimension is the model's embedding size. + +## 3. Typical next steps + +After the container is running, you can adapt the example to your own protein sequences and use a mounted working directory for scripts and outputs. + +For additional container guidance, see [E4S Container Installation](/container-installation/) and [E4S Container Launch](/container-launch/). \ No newline at end of file diff --git a/use-nemo-quick-start.md b/use-nemo-quick-start.md new file mode 100644 index 0000000..4f3efec --- /dev/null +++ b/use-nemo-quick-start.md @@ -0,0 +1,98 @@ +--- +layout: single +title: "NeMo Quick Start" +permalink: /quick-start-nemo/ +classes: wide +sidebar: + nav: "usemenu" +--- + +{% include e4s-page-actions.html %} + +This page shows a NeMo workflow in the E4S 26.06 CUDA container. The examples here are intended to be run inside the Docker image shown below. + +```bash +docker run --gpus all -it --rm \ + --entrypoint bash \ + -v "$PWD:/work" \ + -w /work \ + ecpe4s/e4s-cuda:26.06 +``` + +## 1. Verify the NeMo packages + +The container includes the NeMo packages used by this workflow: `megatron.core`, `megatron.bridge`, and `nemo_run`. + +A quick import check: + +```bash +python3 - <<'PY' +import megatron.core +import megatron.bridge +import nemo_run +print('megatron.core import OK') +print('megatron.bridge import OK') +print('nemo_run import OK:', getattr(nemo_run, '__version__', 'OK')) +PY +``` + +This confirms the NeMo stack is available for use inside the container. + +## 2. Minimal Megatron-Bridge smoke test + +The core pattern in this workflow is converting a Hugging Face model to Megatron's native format through `AutoBridge`. + +Create a script: + +```bash +cat > megatron_bridge_demo.py <<'PY' +import torch +from megatron.bridge import AutoBridge + +print('Loading a small HF model and converting it to Megatron format...') +bridge = AutoBridge.from_hf_pretrained('Qwen/Qwen3-0.6B') +model_provider = bridge.to_megatron_provider(load_weights=True) + +print('Bridge + Megatron provider constructed successfully.') +print(type(model_provider)) +PY +``` + +Run it with the GPU selected: + +```bash +python3 megatron_bridge_demo.py +``` + +This loads a small model and demonstrates the bridge and model-conversion workflow. + +## 3. Run a local NeMo-Run experiment + +Create a small task and launch it locally with `nemo_run`: + +```bash +cat > nemo_run_demo.py <<'PY' +import nemo_run as run + +task = run.Script( + inline="echo 'NeMo-Run experiment completed successfully.'" +) + +with run.Experiment('e4s-quick-start') as experiment: + experiment.add(task, executor=run.LocalExecutor(), name='hello') + experiment.run(detach=False) +PY +python3 nemo_run_demo.py +``` + +The command should print `NeMo-Run experiment completed successfully.`. + +## 4. Typical next steps + +After the container is running, you can: + +- test model conversion and bridge workflows, +- use `nemo_run` to launch experiments, +- and adapt the same pattern to your own training or evaluation scripts. + +For additional container guidance, see [E4S Container Installation](/container-installation/) and [E4S Container Launch](/container-launch/). diff --git a/use-pytorch-quick-start.md b/use-pytorch-quick-start.md new file mode 100644 index 0000000..8ece3c9 --- /dev/null +++ b/use-pytorch-quick-start.md @@ -0,0 +1,95 @@ +--- +layout: single +title: "PyTorch Quick Start" +permalink: /quick-start-pytorch/ +classes: wide +sidebar: + nav: "usemenu" +--- + +{% include e4s-page-actions.html %} + +This page shows a minimal PyTorch workflow in the E4S 26.06 CUDA container. The examples here are intended to be run inside the Docker image shown below. + +```bash +docker run --gpus all -it --rm \ + --entrypoint bash \ + -v "$PWD:/work" \ + -w /work \ + ecpe4s/e4s-cuda:26.06 +``` + +## 1. Verify that PyTorch sees CUDA + +Inside the container: + +```bash +python3 - <<'PY' +import torch +print('PyTorch version:', torch.__version__) +print('CUDA available:', torch.cuda.is_available()) +if torch.cuda.is_available(): + print('Device:', torch.cuda.get_device_name(0)) +PY +``` + +Expected result: `CUDA available: True` and a GPU name such as `NVIDIA A100 ...`. + +## 2. Run a small training example + +Create a small file: + +```bash +cat > pytorch_demo.py <<'PY' +import torch +import torch.nn as nn + +print('PyTorch version:', torch.__version__) +print('CUDA available:', torch.cuda.is_available()) +if torch.cuda.is_available(): + print('Device:', torch.cuda.get_device_name(0)) + +device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') + +x = torch.randn(4, 4, requires_grad=True, device=device) +y = (x ** 2).sum() +y.backward() +print('Gradient check OK:', torch.allclose(x.grad, 2 * x.detach())) + +torch.manual_seed(0) +X = torch.linspace(-1, 1, 200, device=device).unsqueeze(1) +Y = 3 * X + 2 + 0.1 * torch.randn_like(X) +model = nn.Linear(1, 1).to(device) +opt = torch.optim.SGD(model.parameters(), lr=0.1) +loss_fn = nn.MSELoss() + +for _ in range(200): + opt.zero_grad() + loss = loss_fn(model(X), Y) + loss.backward() + opt.step() + +w, b = model.weight.item(), model.bias.item() +print(f'Learned: y = {w:.3f}x + {b:.3f} (target: y = 3x + 2), loss={loss.item():.4f}') +PY +``` + +Run it with the GPU selected: + +```bash +python3 pytorch_demo.py +``` + +You should see the model learn a line close to `y = 3x + 2`. + +## 3. Typical next steps + +Once the container is running, you can: + +- import `torch`, `torchvision`, or `torchaudio` directly, +- create your own model files in the mounted working directory, +- and run the same workflow on a different node or a different E4S image variant. + +For short, reproducible demos, a mounted host directory is the simplest way to keep your scripts and outputs outside the container. + +For additional container guidance, see [E4S Container Installation](/container-installation/) and [E4S Container Launch](/container-launch/). diff --git a/use-tensorflow-quick-start.md b/use-tensorflow-quick-start.md new file mode 100644 index 0000000..a11cebf --- /dev/null +++ b/use-tensorflow-quick-start.md @@ -0,0 +1,79 @@ +--- +layout: single +title: "TensorFlow Quick Start" +permalink: /quick-start-tensorflow/ +classes: wide +sidebar: + nav: "usemenu" +--- + +{% include e4s-page-actions.html %} + +This page shows a minimal TensorFlow workflow in the E4S 26.06 CUDA container. The examples here are intended to be run inside the Docker image shown below. + +```bash +docker run --gpus all -it --rm \ + --entrypoint bash \ + -v "$PWD:/work" \ + -w /work \ + ecpe4s/e4s-cuda:26.06 +``` + +## 1. Verify TensorFlow and GPU visibility + +Inside the container: + +```bash +python3 - <<'PY' +import tensorflow as tf +print('TensorFlow version:', tf.__version__) +print('GPUs visible:', tf.config.list_physical_devices('GPU')) +PY +``` + +Expected result: one or more GPU devices are visible. + +## 2. Run a small linear regression example + +Create a small script: + +```bash +cat > tensorflow_demo.py <<'PY' +import tensorflow as tf +import numpy as np + +print('TensorFlow version:', tf.__version__) +print('GPUs visible:', tf.config.list_physical_devices('GPU')) + +np.random.seed(0) +X = np.linspace(-1, 1, 200).astype('float32').reshape(-1, 1) +Y = 3 * X + 2 + 0.1 * np.random.randn(*X.shape).astype('float32') + +model = tf.keras.Sequential([tf.keras.layers.Dense(1, input_shape=(1,))]) +model.compile(optimizer=tf.keras.optimizers.SGD(learning_rate=0.1), loss='mse') +history = model.fit(X, Y, epochs=50, verbose=0) + +w, b = model.layers[0].get_weights() +print(f'Learned: y = {w[0][0]:.3f}x + {b[0]:.3f} (target: y = 3x + 2), loss={history.history["loss"][-1]:.4f}') +PY +``` + +Run it with a pinned GPU: + +```bash +python3 tensorflow_demo.py +``` + +This trains a simple model and prints the learned slope and intercept. + +## 3. Typical next steps + +After the container is running, you can: + +- use `tf.keras` for model development, +- fit models on the pinned GPU, +- and adapt the same workflow to your own data or scripts. + +As with other E4S containers, the main operational difference from a normal host environment is that GPU access and the mounted working directory are managed by the container runtime. + +For additional container guidance, see [E4S Container Installation](/container-installation/) and [E4S Container Launch](/container-launch/). diff --git a/use-vllm-quick-start.md b/use-vllm-quick-start.md new file mode 100644 index 0000000..5350eff --- /dev/null +++ b/use-vllm-quick-start.md @@ -0,0 +1,92 @@ +--- +layout: single +title: "vLLM Quick Start" +permalink: /quick-start-vllm/ +classes: wide +sidebar: + nav: "usemenu" +--- + +{% include e4s-page-actions.html %} + +This page shows a minimal vLLM workflow in the E4S 26.06 CUDA container. The examples here are intended to be run inside the Docker image shown below. + +```bash +docker run --gpus all -it --rm \ + --entrypoint bash \ + -v "$PWD:/work" \ + -w /work \ + ecpe4s/e4s-cuda:26.06 +``` + +## 1. Verify the vLLM install + +Inside the container: + +```bash +python3 - <<'PY' +import vllm +print('vLLM version:', vllm.__version__) +PY +``` + +This confirms that the package is present and importable. + +## 2. Run a small inference example + +Create a script: + +```bash +cat > vllm_demo.py <<'PY' +from vllm import LLM, SamplingParams + +def main(): + prompts = ['The capital of France is', 'PyTorch is a framework for'] + sampling_params = SamplingParams(temperature=0.7, max_tokens=30) + + llm = LLM(model='facebook/opt-125m') + outputs = llm.generate(prompts, sampling_params) + for out in outputs: + print(out.prompt, '->', out.outputs[0].text) + + +if __name__ == '__main__': + main() +PY +``` + +Run it with the GPU selected. Setting `CUDA_DEVICE_ORDER` keeps CUDA's device numbering consistent when the host has different GPU models: + +```bash +export CUDA_DEVICE_ORDER=PCI_BUS_ID +python3 vllm_demo.py +``` + +This launches a small language model and prints generated text from each prompt. + +## 3. Start the OpenAI-compatible server + +You can also launch an OpenAI-compatible API server: + +```bash +export CUDA_DEVICE_ORDER=PCI_BUS_ID +vllm serve facebook/opt-125m --port 8000 +``` + +In another shell or another container on the same host: + +```bash +curl http://localhost:8000/v1/completions \ + -H 'Content-Type: application/json' \ + -d '{"model": "facebook/opt-125m", "prompt": "Hello,", "max_tokens": 20}' +``` + +## 4. Typical next steps + +After the container is running, you can: + +- swap in a different model for inference, +- use the OpenAI-like API for serving, +- and adapt the same container workflow to a GPU-specific or batch job environment. + +For additional container guidance, see [E4S Container Installation](/container-installation/) and [E4S Container Launch](/container-launch/).