CSE 509 Computer System Security

Fall 2011

Course Description/Topics Lecture Notes Grading Class Hours
Important Dates Instructor and TA Texts Special needs

Course Description

In the class, we will discuss the principles and practice of computer system security. We will not address network security in this course, since it is the topic of CSE 508.

The first objective of this course is to provide a broad overview of issues and approaches in building and administering secure systems. The second objective of this course is to expose students to some of the latest research in computer and software security. This course is thus ideal for any one considering research or a career in cyber security.

Students seeking a career in software development should be aware that the vast majority of security problems today can be traced to software vulnerabilities. This course will help you understand the nature of the threats posed by these vulnerabilities, and ways to mitigate them. A survey of software vulnerabilities, and more generally, cyber threats, can be found by clicking on the links for the first one or two topics in the course.

Course Topics

The course will consist of two parts, the first of which will last about 4 to 5 weeks. The rough list of topics covered within each part is given below.

Part I. Foundations

Part II. Contemporary Threats, Vulnerabilities and Defenses

Lecture Notes

The entire set of slides and notes are now available as single PDF files. This section starts out with lecture notes from previous offerings of the course. These will be updated on an as-needed basis as we go through the semester.
Description/Reading Slides Notes
Introduction: Overview of Security Threats
Emerging threats and research directions
Cryptography Basics
Reading: Who is guarding the guardians, or how secure are the CAs
Identification and Authentication
Reading: Lamport's One-Time Password Scheme
Reading: How anonymous hacked into a security firm
Discretionary Access Control
Reading: Revisiting "Setuid Demystified"
Capabilities, Mandatory Access Control
Reading: The Confused Deputy (or why capabilities might have been invented)
See Prev Topic PDF
DTE and SELinux. POSIX Capabilities. Commercial Security Policies
Reading: Confining Root Programs with Domain and Type Enforcement
See Prev Topic PDF
OS Security, UNIX Security, Database Security
Reading: Linux capabilities (alternative link)
Reading: SELinux
Principles and practices for secure system design
Reading: The Protection of Information in Computer Systems
Background: Runtime memory organization   TXT
Stack-smashing, Heap overflows and Format string attacks
Reading: Smashing the stack for fun and profit
Integer overflows
Memory corruption defenses: guarding, ASR, DSR, ...
Reading: Memory exploitation defenses in Windows
Optional Reading: (Not so) Recent advances in exploiting buffer overruns
Optional Reading: Basic Integer Overflows
See Prev. Week PDF
Memory-error detection: Bounds-checking, etc. See Prev. Week PDF
Injection Attacks, Taint-tracking
Taint-enhanced policies
Reading: Taint-Enhanced Policy Enforcement
Race conditions and other Software vulnerabilities
Reading: Top 25 Software Vulnerabilities
Evasion, obfuscation, Software tamper-resistance
A very short article from 2011 on specific malware trends.
Securing Untrusted Code: System-call interception,
Inline-reference monitoring
Securing Untrusted Code: Inline-reference monitoring,
Software-based fault isolation, Control-flow integrity
Binary analysis and transformation: Disassembly, static binary rewriting
Dynamic translation
Untrusted Code: Java, Javascript and Web security PDF PDF
Untrusted Code: Virtual Machines PDF
Intrusion detection overview
Host-based/Application layer Intrusion detection
Intrusion detection models
Reading: A sense of self for Unix processes
Vulnerability analysis: Program analysis overview,
Abstract interpretation
Course summary PDF  

Class Place and Time:

Lectures: Mon, Fri 12:50pm to 2:10pm     Room CSE 2120


R . Sekar
Office: 2313E Computer Science
Office Hours: MF 11am to 12:15pm


Bhushan Jain
Office Hours: Tue 12:30pm to 2:00pm, Wed 11am to 12:30pm CS 2110


There is no official textbook for this course. We will rely on class notes and some papers. Some of the lectures will draw on material from the following books.

Important Dates:

First day of classes Aug 29
No classes Sep 5, 30; Nov 25
Extra classes Sep 28 Classes follow Friday schedule
Last day of classes Dec 12
Mid-term I October 7 4:00pm to 5:30pm
Mid-term II November 7 or 14 12:50pm to 2:10pm
Final Thursday, Dec 15 2:15-4:45 PM, Room 2120


Your final grades will be computed as follows. You should expect some changes to the weightages over the semester.

You will get full credit for written homeworks and quizzes as long as you

More generally, if you score m_1 through m_k in written homeworks, your score for these assignments will be given by (4/3)*average(min(75, m_1),...,min(75, m_k)). In effect, there is no benefit in scoring above 75 points on written homeworks and quizzes. This has been done so as to remove the main incentive for copying and other unethical practices in homeworks. The exact same formula will be used for quizzes, but the intent here is to reduce student anxiety about quizzes.

Copying homework solutions from a fellow student or from the Internet, and all other forms of academic dishonesty, are considered serious offenses. They will be prosecuted to the maximum extent permitted by university policies.

Special Needs

If you have a physical, psychological, medical or learning disability that may impact on your ability to carry out assigned course work, I would urge that you contact the staff in the Disabled Student Services office (DSS), in the ECC building, 632-6748v/TDD. DSS will review your concerns and determine, with you, what accommodations are necessary and appropriate. All information and documentation of disability is confidential.