36-217

Probability Theory and Random Processes

Summer 2016

Instructor: Sangwon (Justin) Hyun

Office: FMS 328
Email: shyun@stat.cmu.edu
Office Hours: Monday-Fridaylass up to 2:00pm,
and by appointment (in BH 132M)

Teaching Assistant: Zongge(John) Liu

Office: Wean Hall 8419
Email: zonggel@andrew.cmu.edu
Office Hours: Monday Wednesday 4:00pm-5:00pm

Textbooks: Required: Mathematical Statistics with Applications, D. D. Wackerly,

W. Mendenhall, R. L. Scheaffer
Duxbury Press, 2008 (7th edition)

Introduction to Probability Models, S. M. Ross,

Academic Press, 2009 (10th edition)

Supplementary Textbook: Introduction to Probability, D. P. Bertsekas,

J. N. Tsitsiklis
Athena Scientific, 2008 (2nd edition)

Lectures M Tue W Th F, 12:00pm 1:20pm, Porter Hall A18C.

Course Materials: Blackboard page.

Weekly Homework: due on Tuesdays and Thursdays at the beginning of class,

unless otherwise instructed

Midterm Exam: Friday, June 3th, 2016

Final Exam: Friday, June 24th, 2016

Prerequisites: 21-112 or 21-122 or 21-123 or 21-256 or 21-259

Not open to students who have received credit for 36-225, or 36-625
C or better is required in order to use this course
as a pre-requisite for 36-226 and 36-410.

1
Course Description
The theory of probability and random processes provides the mathematical tools and formalism
needed to model uncertainty in virtually all scientific areas. Nowadays, probabilistic models are key
for the development and analysis of scientific theories and of many algorithms, with countless appli-
cations, ranging from the analysis of network dynamics of circuit failure rates to the development
of algorithms for computer vision, statistics, machine learning, image processing, cryptography,
system performance assessment, business inventory, marketing, finance, medicine, etc.
This is a first course in probability theory that is designed to prepare you to develop and analyze
basic probabilistic models for describing and studying uncertain or random phenomena and to make
better predictions and decisions. By the end of this course, students will

1. possess an adequate background and understanding of basic concepts in probability theory;

2. be able to apply probability terminology and formalism correctly to represent elementary

random experiments and quantify uncertainty;

3. be proficient in the calculus-based mathematical skills needed to solve problems in basic

probability.

Course Objectives
1. Basic Probability.

Describe the sample space of an experiment using set notation.

Find the probabilities of complements, unions, and intersections of events.
Use counting tools to enumerate the number of equally likely outcomes of simple exper-
iments.
Use the law of total probability and Bayes Rule to calculate probabilities.
Define and identify independence of events.
Calculate conditional probabilities of events.

2. Random Variables.

Describe the random variable associated with a question of interest.

Describe and identify discrete and continuous random variables.
Use PMFs, PDFs, and CDFs to derive probabilities and quantities of interest.
Describe and identify Binomial, Poisson, Geometric, Uniform, and Normal distributed
random variables.
Calculate expectations for a random variable.
Derive quantities of interest from joint distributions of pairs of random variables.
Calculate conditional distributions of random variables.
Describe and determine independence of random variables.
Derive the distributions of functions of random variables.

3. Advanced Probability Topics.

Describe a Bernoulli process.

2
 Calculate the probability of an outcome from a Bernoulli process.
Describe a Poisson process.
Calculate the probability of an outcome from a Poisson process.
Describe a Markov chain.
Create a probability matrix for a Markov chain.
Use a probability matrix to calculate the probability of an outcome from a Markov chain.
Describe the Central Limit Theorem, Weak Law of Large Numbers and Chebyshevs
Inequality.

Course Components
1. Lectures. The main topics of the course will be covered during the lecture. You are also
responsible for any additional material covered in the assigned readings and homework. Lec-
ture attendance is mandatory. At the discretion of the instructor and the TA,
you may incur in a full letter grade penalty for repeatedly missing lectures. If
you miss class, you are responsible for the material covered during the lectures
you have not attended.
It is your responsibility to print and bring to class a copy of the lecture notes,
which you can download here:

http://www.stat.cmu.edu/~shyun/teaching/

2. Readings. For some lectures, there may be recommended readings from the textbook. These
readings may contain additional material not covered in class. I recommend obtaining a
copy of the textbooks and reading the relevant sections when advised: you will
understand the concepts and examples much better.

3. Homework. Homework problems provide you with the opportunity of learning, practicing,
and testing your knowledge and understanding of the material. All material found in the
homework may show up in later homework and/or exams. If you would like to get more
practice, please try the problems from the book by Bertsekas and Tsitsiklis: the
solutions are available here:

http://athenasc.com/prob-solved_2ndedition.pdf
Late homework is not accepted!

4. Exams. You are evaluated based on your performance on the homework and on the exams.
There is a midterm exam and a final exam. The date of the final exam is set by the
university: you are strongly advised against scheduling any traveling during the
entire period of classes and the final exam period.
The university calendar for the academic year 2015/2016 is available here:

http://www.cmu.edu/hub/calendar.html

Your feedback regarding the content, level of difficulty and pace of the lectures is very
important and I will ask for it in class.

3
Grading Rules
All numeric grades are on a scale from 0 to 100.

Midterm grades will be computed with the following weights:

Midterm Exam 40%

Average Homework Score 60%

A missed midterm exam will receive a zero score, in which case the midterm grade will simply
be computed based solely on the average homework score.

Final grades are based on exams and homework, and will be computed according to the
following weights:

Final Exam 30%

Midterm Exam 30%
Average Homework Score 40%

Letter grades will be determined according to the following rules (non-integer scores will be
rounded):

A > 90
B 8089
C 7079
D 6069
R < 60

Depending on the performance of the class, the scale used to determine the final
letter score may be curved.

To account for grading mistakes, missing homework or lectures, badly worked exam questions,
administrative glitches, etc. etc., 5 points will be added to everyones final grade.

Administrative Procedures and Logistics

Prerequisites. If you dont satisfy the prerequisites for taking this class, you cannot enroll
and take the class.

Audit. Not allowed. You must be regularly enrolled to take this class.

Conflicts with Other Classes. If the lecture times of this class conflict with the lecture
times of another class that you are taking, you must decide which of the two classes you want
to take and drop the other class.

Lectures. Use common courtesy: arrive on time; do not leave early; no cell-phone, etc.
The use of laptops is allowed only for the purposes of note taking and downloading of course
material.

4
 Class Web Page. The syllabus, homework assignments, and any supplementary materials
are available here:

http://www.stat.cmu.edu/~shyun/teaching/
Homework solutions, exam solutions, grades and announcements for this course can be
found on the course web page on Blackboard:
http://www.cmu.edu/blackboard
Please check the website and blackboard regularly.
Communication. If you have any questions related to the class material, homework prob-
lems and exams, feel free to ask the instructor during class or ask the instructor and the TA
during office hours. Please try to use the email only to address administrative and
logistic issues. You should not expect a reply within 24 hours. You are required
to use your andrew email account to communicate via email with the instructor
and the TA. Emails sent from other accounts will be considered unidentifiable
and consequently ignored. Email communications between you and the instructor/TA
must be polite, respectful, and written in clear English.
Homework. Homework assignments are due on Tuesdays and Thursdays in class,
unless otherwise communicated. Late homework submissions will not be accepted
Assignments must be completed and submitted on paper. Electronic versions will not
be accepted, unless previous arrangements are made. Write your homework and exam
solutions neatly. You will receive a null grade to unintelligible solutions. Homework will be
graded for correctness and not effort, but partial credit will be given for legitimate attempts.
Always show your work. No credit will be given for a correct but unjustified answer.
You are encouraged to discuss homework problems and collaborate with classmates, however
the work you submit must be your own and you are required to clearly acknowledge
any collaborations with your classmates on your homework solutions (failing to
do so is considered cheating). In particular, each student must independently write up
each problem. Instances of identical or nearly identical or copied homework will be
considered cheating and plagiarism. The use of material from previous semesters
of this course or from any other source to solve homework and exam problems is
regarded as unauthorized assistance and therefore as a violation of the Carnegie
Mellon University code of academic integrity.
You are required to write your name and andrew ID on the first page of your
homework.
You can retrieve your graded homework from the instructor at the end of the class (upon
signing a privacy waiver) or from the instructor/TA during office hours.
Homework Extensions. There will not be individual homework extensions. The only type
of homework extension that may be granted is when the homework deadline is extended for
the entire class.
Missing Homework and Grading Errors. As noted above, 5 points will be added to
everyones final grade at the end of the semester. In nearly every case, this more than
compensates for missing homework and grading errors. You should contact the instructor or
the TA if you have any grievance concerning homework assignments and exams.
Missed Exams.

5
 1. Midterm Exam. A missed midterm will receive a zero score. This rule admits no
exceptions other than documented medical emergencies. A makeup midterm
exam will be arranged only in case of a documented medical emergency which prevents
the student to take the midterm exam on the scheduled day.
2. Final Exam. Do not miss the final exam or you will get a null grade to it. Once again,
this rule admits no exceptions other than documented medical emergencies.
A makeup final exam will be arranged only in case of a documented medical emergency
which prevents the student to take the final exam on the scheduled day.

Integrity. Cheating, plagiarism and unauthorized assistance on homework or exams will

be dealt with in accordance with the academic integrity policy in place at Carnegie Mellon
University, as described here:

http://www.cmu.edu/academic-integrity/index.html

Disability Services. If you have a disability and need special accommodations in this class,
please contact the instructor immediately. Special accommodations for exams must be
requested no later than one week prior to the exam. You should also contact the Dis-
ability Resources office and request the appropriate documentation. For more information,
see the Carnegie Mellon Equal Opportunity Services and Disability Resources webpage

http://www.cmu.edu/hr/eos/index.html