Instructor Guide to Teaching on Chameleon Cloud
This page includes some notes for instructors who want to use these materials in their own courses.
Overview of available lab materials
Some lab materials are available in multiple versions tuned for different hardware types. If you have a large class, you can spread capacity across the available hardware by assigning students to different versions.
Instructors can refer to the “Before the class begins” section to understand how to get started on Chameleon and prepare for their class.
Instructors are advised to test each lab shortly before assigning it to their class, since the software and hardware on Chameleon and the software stack used in the lab are regularly updated. Feel free to contact ffund@nyu.edu with any questions or problems.
| Lab Instructions | Trovi Artifact | Compute Instance Type | GitHub Repo |
|---|---|---|---|
| Hello, Chameleon (Intro) |  | m1.small |  |
| Hello, Linux (Intro) | (Use "Hello, Chameleon" artifact) | m1.small | |
| Cloud computing on Chameleon (Cloud computing) | | 3 x m1.medium | |
| Build an MLOps pipeline on Chameleon (DevOps and continuous X for ML systems) | | 3 x m1.large | |
| Data platforms on Chameleon (Large scale data systems) | | m1.xlarge | |
| Persistent storage on Chameleon (Large scale data systems) | | m1.medium (block), m1.large (object) | |
| Large-scale model training on Chameleon (Model training at scale) | A100 Version: | A100 Version:compute_gigaio (single GPU), gpu_a100_pcie (multi-GPU) | |
| H100 Version: | H100 Version:g1.h100.pci.1 (single GPU), g1.h100.pci.4 (multi-GPU) | ||
| ML experiment tracking with MLFlow (Model training infrastructure and platforms) | NVIDIA GPU Version: | NVIDIA GPU Version:compute_liqid or compute_gigaio | |
| AMD GPU Version: | AMD GPU Version:gpu_mi100 | ||
| NVIDIA VM Version: | NVIDIA VM Version:g1.h100.pci.1 | ||
| Building a model training cluster with Ray (Model training infrastructure and platforms) | NVIDIA GPU Version: | NVIDIA GPU Version:compute_liqid, with 2 GPUs | |
| AMD GPU Version: | AMD GPU Version:gpu_mi100, with 2 GPUs | ||
| Model optimizations for serving (Model serving) | NVIDIA GPU Version: | NVIDIA GPU Version:compute_liqid or compute_gigaio | |
| AMD GPU Version: | AMD GPU Version:gpu_mi100 | ||
| Serving on edge devices (Model serving) | | rpi5 | |
| System optimizations for model serving (Model serving) | | gpu_p100 | |
| Offline evaluation of ML systems (Monitoring and evaluating ML systems) | | m1.medium | |
| Online evaluation of ML systems (Monitoring and evaluating ML systems) | | m1.medium | |
| Closing the feedback loop (Monitoring and evaluating ML systems) | | m1.medium | |
Overview of available lecture materials
| Topic | Links |
|---|---|
| Intro to ML Systems | Chapter 1 |
| Cloud computing | Chapter 2 |
| DevOps and continuous X for ML systems | Chapter 3 |
| Large scale data systems | Chapter 4 |
| Model training at scale | Chapter 5 |
| Model training infrastructure and platforms | Chapter 6 |
| Model serving | Chapter 7 |
| Monitoring and evaluating ML systems | Chapter 8 |
Things that are known to be broken
- No notes at the moment.
Before the class begins
Before the class begins an instructor should:
Set up a project
Create an account on Chameleon Cloud, and create a project for the course.
From the project page, click “Add multiple users” and then copy the “request to join” link which you can distribute to your students.
Set up security groups
Some lab assignments assume that certain security groups are already configured within the project. These lab assignments have an x_security_groups.ipynb notebook as part of the lab materials.
When you are reviewing each lab that you plan to assign in preparation for use by students, you can run that notebook to configure the necessary security groups within your course project.
Reserve GPU nodes
Certain node types, especially
gpu_a100_pcie, may be fully reserved far in advance. Make your reservations well before you will need the resources.
Some resources on Chameleon, especially certain types of bare-metal GPU nodes, are heavily utilized. To ensure capacity for your course, so that your students do not have to contend with other Chameleon users for scarce resources near a deadline, you should pre-reserve these scarce resources for the time intervals in which your class will need them.
You should anticipate roughly 3 hours of use per student for each lab assignment. Then, use the table above, with information about compute instance types, and your enrollment numbers to determine how many of each bare-metal instance type you will need, and for which time intervals.
Make an advance reservation yourself for these resources at the times your class will need them, to block reservations by other Chameleon users.
Then, use the “Help Desk” feature on Chameleon. Give the list of reservations you have made and the project number associated with your course, and ask for these resources to be allocated for exclusive use for your course during the times you have reserved.
At the beginning of each reservation, confirm via the Help Desk that the resources have been allocated for exclusive use by your project. Then, you can delete the “placeholder” reservations you made, and your students will be able to make their own reservations.
If students will do open-ended projects that require GPU, you may want to make additional advance reservations to support this.
During the course
Communication to students
- Give students explicit instructions about expected resource usage, and what they can expect to happen if they ignore these instructions (e.g “if you make a reservation that is longer than 4 hours for X resource, course staff will delete it”). Also remind students that the infrastructure cannot support all of them doing the assignment at the same time in the last few hours before the deadline.
- It is strongly encouraged to give students a preliminary deadline for each lab assignment by which they must have made a reservation for the resources required for that lab. This helps avoid students waiting until the day before the lab is due and then not having resources available.
- If you have a large class, you may want to assign days to smooth peak usage for lab assignments, e.g. “if your student ID ends in an even number you can use the infrastructure on Monday, Wednesday, Friday, or Saturday; if your student ID ends in an odd number you can use the infrastructure on Tuesday, Thursday, Friday, or Sunday”.
Managing resources
- Keep an eye on resource usage, to make sure nobody has excessive use and to make sure resources are available to students who need them.
- At the beginning of your advance reservations, after Chameleon staff have re-configured the resource to be exclusively available to your project, you will delete your “placeholder” reservation so that students can then make their own reservations.