CS 342 - Neural Networks

When / Where: Friday 13:00-16:00 / GDC 2.216
Links: ED (TBD), Canvas (TBD)
Instructor: Philipp Krähenbühl (OH Friday 16:00 - 16:30 GDC 4.816)
TA TBD

Schedule

Week	Topic	Homework
Aug 29	Welcome to Deep Learning Welcome to Deep Learning What Is Deep Learning? Modern GPU architectures Tensors Tensors in PyTorch Gradients	Homework 1 - Out
Sep 05	First Example First Example Regression and Classification Datasets and Losses Loss Functions in PyTorch Optimization
Sep 12	Computational Graphs + Regression in PyTorch Computational Graphs Computation Graphs in PyTorch Linear Regression in PyTorch Binary Logistic Regression in PyTorch Multi-Class Logistic Regression in PyTorch First Example - Summary Deep Networks Nonlinearities
Sep 16		Homework 1 - Due
Sep 19	Deep Networks Activation Functions Output Representations Loss Functions Stochastic Gradient Descent Stochastic Gradient Descent in PyTorch Training a Deep Network in PyTorch Hyperparameters Deep Networks in PyTorch A Practical Guide to Deep Network Design	Homework 2 - Out
Sep 26	Residuals and Normalizations Residuals and Normalizations Vanishing and Exploding Gradients Normalizations Normalizations in PyTorch Residual Connections Residual Connections in PyTorch Residuals and Normalizations - Summary
Sep 30		Homework 2 - Due
Oct 03	Transformers Transformers Attention Multi-Head Attention Multi-Head Attention in PyTorch Positional Embeddings Positional Embeddings in PyTorch The Transformer Architecture The Transformer Architecture in PyTorch Applications	Homework 3 - Out
Oct 10	Training a Transformer in PyTorch + Making It Work Training a Transformer in PyTorch Transformers - Summary Making It Work Data and Advanced Network Design Advanced Training
Oct 14		Homework 3 - Due
Oct 17	Making It Work Overfitting Making It Work in PyTorch Making It Work - Summary	Homework 4 - Out
Oct 24	Generative Models Generative Models Auto-regressive generation Vector Quantization
Oct 28		Homework 4 - Due
Oct 31	Flow-Matching DALL-E Flow-Matching Latent Diffusion and State-of-the-Art Models	Homework 5 - Out
Nov 07	Large Language Models I Large Language Models Architectures Generation Instruction Tuning RLHF
Nov 11		Homework 5 - Due
Nov 14	Large Language Models II DPO Tasks and Datasets Tool Use Structured Outputs	Homework 6 - Out
Nov 21	Large Language Models III Constrained Decoding Long Context Structured Dialogues
Dec 02		Homework 6 - Due
Dec 05	Large Language Models IV + End of Class Limitations of LLMs Reinforcement Learning and LLMs Current Vision Language Models End of Class

FAQ

Why is this class offered on Fridays three hours straight?
Because the instructor is on 75% leave. This also means that the class is only offered every 2 years.
Why is the class called “Neural Networks” and not “Deep Learning”?
“Neural Networks” is the old name for “Deep Learning” (pre 2005), and nobody bothered to change the class name.
What programming language will we use in this class?
The class will primarily use Python, especially with PyTorch for deep learning assignments.
Are lectures recorded or available online?
Please check with the instructor or course website for information about lecture recordings.
Is attendance mandatory?
It’s strongly encouraged. The TA will reserve the right to not answer questions that have been answered in class.
How do I get help if I’m stuck on a homework problem?
Ask GenAI, ask on the course forum, attend office hours.
I sent the instructor an email, and didn’t get a reply.
That’s normal. Contact the TAs (course forum or in-person), or find the instructor in person on Fridays.
Can I work on homework assignments with classmates?
No, all homework must be completed individually as per the academic honesty policy.
What is the policy for regrading requests?
Grading is automated, there should generally not be any need for regrading requests. If there is, we will regrade the entire homework even if the score on some parts reduces.
What background knowledge is required for this class?
Familiarity with Python and a basic background in machine learning is recommended.
What if I want to learn more?
This class is a subset of two online classes the instructor teaches: Deep Learning and Advances in Deep Learning
Can I post my homework solution on GitHub or share it publicly?
No. If someone copies your homework during class this is considered plagiarism for both parties. If you post it after class concludes, you’re just spoiling the fun for future generations.
Will I need a GPU?
Yes. However, there are free options such as colab or MLX for M1+ MacBooks.

Grading

6 x Homework (1/6 of grade each)
Due midnight anywhere on earth (07:00 central time next day)
- 1 day late: -10%
- 2 days late: -30%
- 3 days late: -100%
  - Solution will be released. No exceptions. Plan accordingly.

Slip days

Everybody gets 3 slip days. You may use them to waive

1 day late penalty (cost: 1 slip day)
2 day late penalty (cost: 2 slip day)
convert 2 into 1 day late (cost: 1 slip day)
we cannot waive a 3 day late penalty.

Slip days are applied automatically and greedily. For example: if you’re one day late on every homework. Late penalties on the first 3 homeworks are waived irrespective of your score.

Submissions

All submissions are code only and auto-graded. You have a total of 5 submissions per homework. If you plan to go over this limit talk to a TA first. The most recent submission counts, not the highest scoring one.

Use of GenAI tools

The use of GenAI tools is permitted in this class (including for homeworks). As a student you should receive free access to GitHub Copilot, which we recommend you make use of. There are a few ground rules to the use of AI in class:

You need to clearly mark code that was written by a GenAI tools as # This code was written by XXXX where XXXX is the name of the model you used. Failure to do so might lead to plagiarism issues. This has to be done per-function. It is not sufficient to write this in a comment in the header of a file or a common readme.
You need to be able to explain every piece of code you submit. If there is duplicate code submitted, we may ask you to come into office hours to explain what you wrote to make sure it is yours.

These ground rules are firm and not following them can severely impact your grade in this class.

How to use GenAI tools responsibly

It’s easy to delegate large parts of your work (as a student or professional) to GenAI. However, this is a certain way to automate yourself and encourages being replaced by more capable GenAI tools in the future. Instead, use GenAI tools to help you grow:

Ask Questions: Use GenAI tools to explain concepts you don’t understand, code that is hard to read, or even code generated by AI tools. Examples include:
- What does torch.as_tensor do?
- How do I put a torch.Tensor on a GPU?
- How does a PyTorch training loop work?
Make sure you understand all code generated by AI tools.
- GenAI tools are not perfect and can make mistakes. Understanding their limitations will make you a better user.
- Use code generated by GenAI as a learning opportunity to improve your coding and code-reading skills.
Use GenAI as a source of motivation.
- If you’re stuck, GenAI can often help you get unstuck by proposing solutions. Even if the solution isn’t perfect, it might help you move closer to your goal. GenAI can also help you prototype different ways to solve a problem.
Use the homework assignments to familiarize yourself with GenAI tools.

How to not use GenAI

Don’t simply ask a GenAI tool (e.g., Hey Claude, can you do my homework for me?) to complete your assignments for you.

You might finish your homework this way, but it adds no real value to your learning or future career.

You don’t grow professionally
This approach won’t work for more complex, real-world problems
Most companies conduct coding interviews where GenAI tools are not allowed (or strictly monitored)
- Hey Claude, can you do my coding interview for me? will not work. Use the homework to prepare for real job market challenges.

Prerequisites

Familiarity with Python, basic ML background

Course Communication

See the Course Communication page for more information about how to contact the Teaching Team.

When corresponding any method used in this course, be mindful that your comments remain respectful of others and relevant to the course content. We don’t want to discourage students from posting any comments that are related to this course, even if not directly covered in the course materials. We do, however, expect that your comments will be respectful and considerate of others.

The course discussion forum is not a place for secondary discussions about culture, politics, sports, etc. It is also not appropriate to use the discussion forum as a place to post personal attacks on fellow classmates or the instructional team, profanity, or other individual complaints about the course, including grading issues. If you have a specific issue with the course or a grade you received, you are free to discuss it privately with the instructor or a teaching assistant.If you post something that the instructor determines to be either disrespectful or off-topic, your post may be removed. If you continue to post inappropriate content, you may be banned from the platform and/or subject to other disciplinary action(s).Students are expected to abide by a code of collegial respect and sensitivity towards their classmates and the instructional team. Please respect the right of everyone in this class to ask questions and discuss their thoughts about the subject matter of our course without fear or concern.

Accommodations

The University of Texas at Austin provides, upon request, appropriate academic accommodations for qualified students with disabilities. To determine if you qualify, please contact the Disability and Access Office. By law, we cannot provide academic accommodations until you present us with an official letter from them. If that office certifies your needs, we will work with you to make appropriate arrangements. Use the course email to provide us with necessary documentation once you receive it.

Academic Honesty

Collaboration is not allowed for homework or quizzes. Every homework and quiz needs to be solved individually. All assignments will be checked for plagiarism against your fellow classmates, submissions from previous semesters, as well as online resources. The plagiarism check will be conducted by the course lead and the results will be sent to the professor. The online course format allows for multiple methods of identity verification, collusion, collaboration and plagiarism monitoring and detection. A violation of the course policy may include (but is not limited to) the following:

Providing your UT EID to any other person
Collaborating or sharing information with another person regarding the material on any quiz, assessment or assignment, before, during and/or after any quiz, assessment or assignment
Recording any quiz, assessment or assignment material in any format
Failing to properly cite language, ideas, data, or arguments that are not originally yours. The creation of artificial intelligence tools for widespread use is an exciting innovation. These tools have both appropriate and inappropriate uses in classwork. The use of artificial intelligence tools (such as ChatGPT) in this class is permitted for students who wish to use them, provided the content generated by AI is properly cited
The public (such that it can be viewed by more than one person) posting of any form of a test bank or group of questions from any assignment
Consulting forbidden materials or sources of information

As per University Policy:

“A student found responsible for academic misconduct may be assigned both a status sanction and a grade impact for the course. The grade impact could range from a zero on the assignment in question up to a failing grade in the course. A status sanction can range from probation, deferred suspension and/or dismissal from the University. To learn more about academic integrity standards, tips for avoiding a potential academic misconduct violation, and the overall conduct process, please visit the Student Conduct and Academic Integrity website at: http://deanofstudents.utexas.edu/conduct.”

Behavior Concerns

If you are worried about someone who is acting differently, you may use the Behavior Concerns Advice Line to discuss by phone your concerns about another individuals behavior. This service is provided through a partnership among the Office of the Dean of Students, the Counseling and Mental Health Center (CMHC), the Employee Assistance Program (EAP), and The University of Texas Police Department (UTPD). Call 512-232-5050 or visit https://besafe.utexas.edu/behavior-concerns-advice-line