Video Tutorials
ARC provides a number of video tutorials on our channel on video.vt.edu. In particular, the following sequence walks a user through the fundamentals of ARC usage in less than an hour:
Login
These videos will walk the user through accessing our systems for the first time (and streamlining access for subsequent logins):
Accessing Software
The following videos will walk the user through accessing software that ARC has installed or through setting up your own packages:
Scheduler interaction (job submission)
The following will walk the user through the process of submitting interactive jobs for testing/development and batch jobs for production research runs:
Note that these videos require a VT Login to access. Also, each video has a table of contents that can be used to skip between sections; this can be accessed by clicking the “hamburger” (three horizontal bars) button at the top left of the video.
We are actively updating this page.
There are many types of videos linked on this page. Generally, they fall into two classes:
General overviews.
Narrow focus on how to accomplish one thing.
With respect to bullet #2, sometimes, one video suffices for a topic. In other cases, multiple videos are required. An example of the latter are virtual environments (VEs). There are videos on why they are needed, how to use modules (in setting up general environments), how to build and use VEs of various types, and how to use VEs across programming languages. Rather than throw all of these topics into one (longer) video, we carve the issues up into several videos so one can more directly satisfy their needs. You might, for example, only want to know how to build a VE and use it in a Jupyter notebook.
We try to make very few assumptions about the experience of the user so that videos are of value to novices and those with experience. All users of all types are welcome on ARC resources and are encouraged to use them.
General Overview and Tips
This video provides a general overview of use of ARC systems and some tips on helping you to understand how your codes are performing.
Accessing ARC Clusters
This video covers ssh, Open OnDemand, and VPN as tools to access ARC clusters.
Different Drives on ARC Clusters
This video describes the various mounts/directories for doing different types of work. For example, these drives vary in the desired longevity of files contained in them.
Compute Nodes on Clusters
Each ARC cluster has multiple partitions. A partition is a set or collection of compute nodes. Often partitions (sets of compute nodes) are all of one type (i.e., architecture). However, sometimes one partition has nodes with different architectures. This video shows how to understand the partitions and compute node types on ARC clusters, which is a requirement for making good choices about how to get your computations done on the clusters.
Environments
Environments are collections of software that tailor an otherwise “basic” computing ecosystem into one that supports your particular computing needs. There is a series of videos here on
motivation,
how to use modules,
how to structure directories to house your virtual environments,
how to construct and use virtual environments (VEs) for a. command-line execution of code and b. use with applications like Jupyeter notebooks.
Motivation
This video explains the software environment that exists when a user logs on and why this is not typically sufficient for most user needs. Environments are described as a way for users to customize their context to do work. Several videos below explain how to construct environments and make use of them.
Modules
Modules are software that enables you to customize your environment to get your work done. Modules are fundamental to clusters. This video explains how to use them.
Virtual environments (VEs)
There is a section below with several videos on how to structure directories where you put your VEs, how to build VEs, and how to use VEs.
How to Create and Use Virtual Environments
How to structure your VEs.
Because you can access all files created on any of the main three clusters (TC, Owl, Falcon), it is useful to structure your virtual environments so that you know, for a given VE, which cluster, partition, and compute node type you can run the VE on. (A VE must be used on the same [cluster, partition, compute node type] triple as it was constructed.)
This video shows how you might structure your directories where your VEs are located.
How to Create and Use a Python Conda Virtual Environment (VE) on Owl
This video shows you how to create a conda virtual environment on Owl. The processes for other clusters are very similar.
How to Create and Use a Python Pip-Venv Virtual Environment (VE) on Owl
This video shows you how to create a pip-venv virtual environment on Owl. The processes for other clusters are very similar.
How to Create and Use a Python Conda Virtual Environment (VE) for use with Jupyter notebooks.
LEFT OFF.
For Python.
These videos are how to build VEs for various clusters and VE types when working in Python.
conda VE on Owl.
conda VE on Falcon.
pip VE on Owl.
pip VE on Falcon.
conda VE for Jupyter notebooks.
For Julia.
These videos are how to build VEs for various clusters and VE types when working in Python.
virtual environment
How to Use Virtual Environments (VEs)
Given the VEs built in the previous section, the videos in this section use those VEs to execute computations.
For Python.
Using a conda VE on Owl.
Using a conda VE on Falcon.
Using a pip VE on Owl.
Using a pip VE on Falcon.
Using a conda VE with Jupyter notebooks.
For Julia.
These videos are how to build VEs for various clusters and VE types when working in Python.
Using a VE specified outside Julia source code.
Using a VE specified inside Julia source code.
Running Jobs on ARC Clusters
The two dominant ways to run jobs on ARC clusters are:
interactive jobs
batch jobs.
Interactive Jobs
This video provides an example of an interactive job.
Batch Jobs
This video provides an example of a batch job.