GENERAL INFORMATION
This is the website for CS 420/520 - Theory of Computation offered in Spring 2019 and taught by Jeff Kinne. This class will be held Tuesdays and Thursdays 11am - 12:15pm in Root Hall A-012.
Office hours are: MWF 9am-noon in Root Hall A-140D.
Click on the links on the left for information about the course.
COURSE ANNOUNCEMENTS
-
Here I'll keep notes on announcements for the course.
Skip to the latest -
Jan 15 plan ...
- Administrivia - Syllabus and website. CS course policies. Regular schedule for assignments/quizzes.
- Course overview.
- Note - you need to take notes, I am not doing it for you this time.
- Proof is - start with known facts (aka axioms), use already proven statements/theorems, use rules of logic, derive new things that are true. Note - you must follow the rules - only use known facts and already proven statements/theorems. Note - proof is certainty, no guessing, no probably, no "I think so". You only say things that are true. You never make anything up. You never hope what you said is correct (either it is or isn't).
- Proof examples - direct proof (97 is prime), contradiction (sqrt(2) is irrational, there are infinitely many prime #'s), induction (arithmetic sum, geometric sum). See others - https://en.wikipedia.org/wiki/Mathematical_proof
- Proof mistakes - proof by example (all odd integers are prime), fallacies, ...
- Assignment - read Sipser chapter 0 (or something similar). Note on reading - read before we talk about it in class, or after, which ever works for you.
- You should be able to - demonstrate the proofs yourself that we do in class. First read through again what we did. Then start with a blank piece of paper or a blank board, and go through the proof. Last, demonstrate the proof to someone else - one of the people on duty in the lab for example.
-
Jan 17 plan ...
- Quiz on Sipser chapter 0. See the Assignments page for more information.
- Grading the quiz in class.
- Regular languages - DFAs. Note - see HW 2 on the assignments page, due next class.
-
Jan 22 plan ...
- HW 2 due today.
- Regular languages - NFAs.
-
Jan 24 plan ...
- Regular languages - RE's
- Maybe - RE's in python.
-
Jan 29 plan ...
- Quiz 3 on DFA's, NFA's, RE's. See assignments for details.
- Note that HW 4 is due on Jan 31.
-
Jan 31 plan ...
- Non-regular languages, proving languages are non-regular with the pumping lemma.
- Note - we will have a "big quiz / exam" over regular languages once we are done. We will count this under quiz points to hopefully raise some of your quiz grades. We will also count it under exam points. FYI.
-
Feb 5 plan ...
- Quick intro/refresher for Python
- Python code for simulating DFA, NFA. See links for python3 tutorial. Note - use python3 command to start, input to get string input.
- RE's in python, see links
- HW 5. Work on it before next time so you can ask questions. Due 12-Feb AOE.
-
Feb 7 plan ...
- RE's in python. Examples - phone #, email address, IP address, wc. See also stack overflow. And more practice - hackerrank.
- RE's in python. Beware... Where you put your () and order of operations. Matching the whole or just part of a string. More than one way to match (and it doesn't choose the way you had in mind).
- RE's - developing and debugging. Start simple, test often.
- Python code for simulating NFA
-
Feb 12 plan ...
- Question - "big quiz / small exam" on regular languages. When? Feb 21. Sample exam next time.
- Revisit code for NFA in python
- HW 5 problems, questions. Things to remember... In RE's in python, if you want to use \1 this needs to be \\1 in your string (and \w would need to be \\w, etc.), unless you use r''.
- Theorems/proofs - construction of DFA for any NFA, construction of NFA for any RE, exercises from end of Ch 1 Sipser
-
Feb 14 plan ...
- Sample exam
- Quiz 6. With 2/3 of vote of registered, we don't count it. Probably we count it.
- Extra/advanced problems - FYI to the class in case anyone else wants to follow along. Each HW assignment will have some extra/advanced problems for honors conversion students to work on. We'll pick an hour each week to discuss.
- Attendance - I started keeping track as of last week of absences. If you miss 6 times you fail.
- Note - HW 5 and Quiz 6 graded for next time. Any questions already?
- Maybe - proofs that - regular languages are closed under union, intersection, complement, concatenation, star; can convert between NFA and RE;
- Honors time - 11am Fridays
-
Feb 19 plan ...
- Note - honors hour (11am-noon) - meet without me this time, I'll be out of town at that time. (Yes, it is lame to do this for the first meeting, but them's the breaks.)
- hw5 grade - in BB, and a hw5-grade.txt file in your account. If you think I made any mistakes let me know.
- Quiz 6 solutions
- Note on grading - if your solution is not correct, I might just mention an input that your solution is wrong on. Then it's up to you to look at what your solution does with that input, and think about other inputs your solution might be wrong on.
- Sample exam questions/answers
- Note new honors problems
-
Feb 26 plan ...
- Exam 1 graded. Counted it as Quiz 7 as well if it was better than your average for quizzes. Exam 1 answers - in some videos (so you can look back at later when studying for final exam, and we can start new material today).
- Interim grade put in - what you would get if term ended now. For most of you, this is lower than you would like.
- New content - CF languages.
- How are things in the course? We end class by noon, you all discuss and let me know.
-
Feb 28 plan ...
- CFG's - Chompsky normal form, ambiguity.
- PDA's - definition, examples, equivalent with CFG's.
- Soon - Exam 1 - video answers
- Note - grammar for Python - https://docs.python.org/3/reference/grammar.html.
-
Mar 5 plan ...
- Notes from your discussion - you want small quizzes and/or assignments to get more practice, some don't always follow things in Sipser. Okay. Thanks for the feedback. Small things - got it...
- HW 8 solutions, grading
- Non context free languages, pumping lemma for CF's.
- HW 9 questions
- Soon - application of CFG's - parsing, or something else (probably PLY or SLY).
-
Mar 7 plan ...
- HW 8 and HW 9.
-
Mar 12 plan ...
- HW 9. No name on one of them. Yours?
- Quiz next time CF's.
- Grade formula. The class decides what formula you want?
-
Mar 14 plan ...
- Note exam1 solution videos linked from links page.
- Quiz 10 taken.
- HW 11 - correct answers to quiz 10. Due Tuesday.
- HW 12 - SLY something. Due in one week.
- Grade formula change? 1 vote for no, rest split on "don't care" or "please change".
- Late HW - due by 3/27 AOE. Half of the points that you missed.
- Exam 2 on CFL's - April 4. Probably exam 3 is the final. Maybe exam 3 is on the rest. Final exam includes all.
-
Mar 19 plan ...
- More SLY - sly-calc.py - **, if then, turning on/off printing.
- SLY something from scratch - language G2 below Figure 2.1 in Sipser 3rd edition.
- Quiz 10 / HW 11 - video answers or in class today?
- HW 12 - see the assignment, due right after break. Try out some test files.
-
Apr 2 plan ...
- Returned graded work - HW 11. Questions on the quiz 10 / HW 11 problems? Note that it looked to me like many of you didn't perform up to your abilities on this one - work on the problems early, look at similar examples in the book, try to "break" your solution.
- A few new things with CFL's - Chomsky normal form, is regular language pumping lemma true for CFL's?, are CFL's closed under intersection (no, and this is different than regular languages), are CFL's closed under union / star / concatenation (yes), naive algorithm for recognizing CFG, what is LR(k). bullet points.
- Things we won't discuss but would if we spent more time on CFL's - CFG and PDA equivalent (CFG for simulating PDA, and PDA for simulating CFG). Ambiguous CFG, ambiguous CFL. Deterministic CFL's - does not include all CFL's, but is useful (programming languages). DPDA can be assumed to read its whole input. DCFL's are closed under complement (note that CFL's are not). DCFL's not closed under union, intersection, star, reversal. DCFG defined based on reductions, equivalent to DPDA's if assume endmark. DCFG - leftmost reduction where reduction step doesn't need the rest of the string. DCFL's are LR(0), LR(k) slightly easier to work with and an equivalent class. Whatever we need about LR parsers, DCFL's.
- SLY, HW 12 due. Wrapping up CFL's.
- HW 9 - pick it up tomorrow or Monday.
- Exam on Tuesday.
Any questions? Review chapter 0 as well - note terms and theorems
that you need to study. Rough outline -
- Give a DFA, NFA, RE for a regular language
- Give a CFG for a CFL language
- Give a PDA for a CFL language
- Prove that a non-CFL language is not CF
- Given a CFG, answer questions about it - describe what it is, give strings that are both in the language and not in the language, is the grammar ambiguous
- Given a PDA, answer questions about it - similar as last question, also write out the transition table
- For a statement about CFL's that is true, give the proof idea. Possible statements that are true - closed under union, concatenation, star. Check also the book sections 2.1-2.3.
- For a statement that is not true in general for CFL's, give a counter-example that shows it is not true in general, and possibly also give an example where the statement happens to be true. Possible statements that are not true in general for CFL's - closed under intersection, closed under complement. No gaurantee that questions are only from that list.
- Starting third part of the course - complexity theory, selected topics in remaining chapters of Sipser.
- Exam 2. Good luck. Solutions will be posted in the usual place (file names exam2_solutions.docx) once the exam starts. Note that there were a few mistakes in the solutions to the sample exam; corrections have been made to problems 1, 3, and 5.
- Note - hw12 graded, exam2 graded for next time probably.
- Exam2 solutions - questions, if you looked at the solutions?
- Note - interview questions that are from theory of computing - see links in the Links page
- Today's definitions and concepts - Turing Machine, Turing-complete, Church-Turing thesis, complexity class P (polynomial time), complexity class EXP (exponential time), running time of a Turing machine, memory space used by a Turing machine, fact that most choices you can make in defining a TM "don't matter" in the sense that different types of TM's can simulate each other, fact that TM is roughly equivalent to any of our normal programming languages (with some "polynomial slowdown"). Note - we didn't prove anything; the imporant thing is to have some intuition about these things, and remember what the terms mean.
- The goal of theory of computing - exact characterization of the resources needed to solve problems of interest. Characterization in terms of different types of resources - time, memory, parallel-ness, verification, proof-ness, approximability, ... Exact characterization requires two parts - an algorithm that is the best possible (and a proof that it is correct and works as efficiently as claimed), and a proof that no algorithm can do better. Former is called "upper bound" proof, latter "lower bounds" proof.
- HW - read. Start reading through parts of the rest of the book that look the most interesting to you.
- Reading for today - (1) TM for language B = {w#w| w ∈ {0,1}∗} given at the beginning of section 3.1 in the text, and formal description of that TM in example 3.8. (2) Example 3.7 of TM for language {0^(2^n) | n >= 0}. (3) As much of chapter 3 as you can pay attention to without falling asleep or throwing your book (especially if the book is on your computer, don't throw your computer). (4) We'll start in Chapter 4 on Tuesday, the big question is - is the Halting Problem solvable?
- Today's definitions/results - TM configuration, Turing recognizable versus Turing decidable, undecidable, algorithm, halting problem, countable and uncountable, the set of real #'s is not countable
- Note - interview questions that are from theory of computing.
- Exam2 grades. Note - fixed solutions for problems 1, 2. Check again.
- Estimated letter grade - based on minimum formula - minimum of HW total, quiz total, exams total. You should work on improving whichever of these is significantly lower than the others. I put in a total weighted grade as well (exams 50%, HW 25%, quiz 25%) so you can see what that would look like; most would receive the same grade either way.
- Exam3 will be the final and commulative.
- See assignments page for plans for HW, quiz
- HW 14 due Monday noon - hand in physically by then, or electronically as described in the assignment.
- Note - from last time, next up is size of the real numbers, then undecidable problems.
- Reductions and decidability.
- A_TM is undecidable (chapter 4). Note that basically the same proof also shows that the halting problem is undecidable.
- Once we have one undecidable problem (A_TM) we can show others are as well using reductions. Did this for an example language. Can also do this with halting problem.
- Note - HW 16 is to write up correct solutions of Quiz 15 problems and turn in by the start of class on 4/25.
- Fall classes - questions?
- Most languages are undecidable (probabilistic method).
- Selected solutions from recent HW's, quizzes.
- HW 16 - due today, or tomorrow?
- HW 17 problems are up, due Monday by noon.
- Quiz 18 is on Tuesday, problems will be similar-ish to HW 17 problems.
- Quiz 18 not today, moved to Thursday. Turn in HW 17 Thursday as well.
- Late HW's - half credit, make an appointment. Do this if - 0's or really low.
- Grades for things... HW 14 - 0 means I didn't get it from you, check what I marked you wrong for (as always)...
- New - complexity theory, complexity classes, complete problems.
- HW 17 turned in, and solutions.
- Quiz 18 handed out, take home due 1pm tomorrow.
- Final exam - Thursday May 6 at 10am here. Cumulative, points will be roughly 2/7 exam1, 2/7 exam2, 3/7 since exam2.
- Next week - I will not be here Thursday/Friday, if you want to come to office hours do that in the first half of the week.
- Grade formula - we'll use the weighted total, that's better for most people. I updated letter grades based on the weighted total right now.
- Course evals - need to be completed by tomorrow, right?
- New today - complexity theory - P, NP, PSPACE, EXP, NP-complete. See chapter 7 in Sipser. A few selected topics from chapters 8, 9 have been discussed in class, but those will not be on the exam.
- Note - sample exam is in the code directory, model solutions are there as well, and videos of model solutions are linked from theory page in CS wiki
- Definitions and examples - computable/recursive, Turing machine, Turing-complete, (mapping/oracle) reduction, uncomputable/nonrecursive, (strong) Church-Turing thesis. Complexity classes - L, P, NP, PSPACE, EXP. Complete languages. Complexity zoo.
- Lower bounds proofs - hierarchy theorems.
- Lower bounds proofs - counting arguments / probabilistic method.