What is Computation? From Turing Machines to Black Holes and Neurons

Harvard GSAS Mini-Course, January 2022

Update [01/30/2022]: This mini-course had been over but you can access all the recordings here. Comments and suggestions are welcomed!

Meeting Time: January 10-14 & January 17-21 10:00am-11:50am Eastern Time.

Instructor: Chi-Ning Chou (he/him/his; they/them/theirs)

• Website: http://cnchou.github.io
• Email:

Guest Speakers:

Teaching Staff: Avantika Agarwal, Kartikeya Arora, Salvador Buse, Aaron K. Clark, Erick Galinkin, Simone Maria Giancola, Aman Goel, Reijo Jaakkola, Pawan Mishra, K Prahlad Narasimhan, Tirukkovalluri Sri Sowmya, Mostafa Touny.

Course Description

Overview: We are in the era of computation. Everything from a spacecraft to your digital toothbrush has a computer inside it. Meanwhile, physicists told us that physical systems such as black holes are doing some kind of computation and neuroscientists told us our intelligence is built on top of small computations by neurons. What do they really mean by computation therein? Does computation mean the same thing in different fields? Furthermore, do we really understand what computation is?

In this mini-course, we are going to revisit the concept of computation through three different perspectives starting from mathematics, to physics and biology. Through theories, examples, and experiments, we are going to see the similarities and differences between these disciplines. The aim of this mini-course is to remind people of the diverse nature of computation and envision the possibilities of future interdisciplinary research.

There will be three modules and each contains three 50-minute interactive lectures and 1-3 guest talk(s). The first module focuses on the mathematical foundation of computations in which the students will learn the concept of Turing machines, computability, reductions, and most importantly, the underlying philosophy. The second module is about the computations in the physical world where we will launch from classical mechanics, to statistical mechanics, quantum mechanics, and gravity. The third module looks into the computations in biology, ranging from genomes, evolution, to neuroscience.

Prerequisites: No math or science prerequisites are required but we expect the students to be comfortable with seeing simple equations. We are particularly welcoming students from any background. The only requirement is having the willingness to communicate with and being respectful to people from different backgrounds.

Policies and expectations: We expect students to attend as many lectures and activities as possible. We will send out a survey one week prior to the mini-course for students’ availability and potentially adjust the times accordingly. Another survey for students to commit on their attendance (don’t need to commit to all the activities!) will be sent out once the lecture & lab times have been finalized.

Learning outcomes: By the end of the course, we hope that the students will all be able to:

• Understand the high-level story of each topic and be able to explain to non-experts.
• Appreciate and respect the different ways of thinking from different fields and perspectives.
• Comfortably communicate with people from different backgrounds.
• Build up interests in some of the topics and perhaps be willing to study further in the future.

Course Format

The main components of the course are as follows:

• Main class meetings: These will be interactive lectures, with the exposition and discussion guided by student questions and comments.
• Labs: These will be simple experiments demonstrating the key concepts learned in the class. We also welcome creative and artistic elements brought by the students! (The in-person labs got cancelled due to the latest COVID regulations)
• Guest talks: These will be invited talks given by Ph.D. students or postdocs from relevant fields. The aim is to let the students have exposure to frontier research and thinking in each field.
• Coffee chats: These will be informal casual chatting sessions with free coffee and snacks made by the instructor. Due to the latest COVID regulations, this might not happen.
• The Panel discussion (new): Near the end of the mini-course, we will host a panel discussion for the students to interact with every guest speaker and the lecturer.
• Advanced sections (new): Outside the main lectures, the teaching staff will offer several 1-hour advanced sections covering a wide range of topics related to the course materials.

Topics

The course is composed of one introductory lecture, one concluding lecture, and three modules: (I) The mathematical foundation of computation; (II) Computations in the physical world; (III) Computations in the biological world. Each module will contain three 50-minute lectures where two of them (indexed by a,b) focus on theory and examples while the other (indexed by c) talks about philosophy and methodology. Each module will also have a lab and a guest talk.

• Introductory Lecture: Motivation; Overview; Goal; Logistics. [slides] [video].
Module I: The mathematical foundation of computation

Is there any problem that a mathematician cannot solve? This is known as Hilbert’s 10th problem and was one of the most important mathematical problems in the beginning of the 20th century. Intriguingly, the answer turned out to be negative. In fact, most of the problems in the mathematical world are unsolvable! To put this into a bigger context, the resolution of Hilbert’s 10th problem was a corollary of Turing’s great achievement of establishing the mathematical foundation of computation.

In this module, students will learn Turing’s mathematical formalization of computation as well as the two other central notions in the modern study of computation: asymptotics and reductions. Other computational notions such as non-determinism, randomness, and communications will also be discussed. The goal is to provide intuitions for students to think about the mathematical formulation of computation and be able to compare with the materials covered in the next two modules.

• Lecture I.a (Departure: Reasoning about Computation via Mathematics): Computational problems; Hilbert’s problems; Turing machines; Uncomputability Theorem. [slides] [video].
• Lecture I.b (Modern Developments: Models, Resources, Reductions): Other computational models; Circuits; Communication; Other computational resources; Nondeterminism; Randomness; Reductions; Cook-Levin Theorem; KW Games. [slides] [video].
• Lecture I.c (Reflection: Turing Machine and Reality): Mathematical Church-Turing Thesis; Philosophy of computation; Philosophy of science. [slides] [video].
• Lab I: Have fun with reductions.
• Guest talk I.a: Lijie Chen on “Efficiency of Algorithms and the Sorites Paradox”. [video] [ ]

Why do theoretical computer scientists think of an algorithm with running time $100000n$ being efficient while another with running time $2^{n/100000}$ being inefficient? In this talk, I'm going to introduce you to a philosophical foundation of this central formalism in theoretical CS. I will guide you to rethink about the definition of "efficient algorithm" through the lens of Sorites paradox. Furthermore, we will touch on connections to Moore's law, cosmology, and beyond. Hope that after the talk, you won't feel that you are old when you wake up tomorrow! (if you didn't get this joke, you will after attending my talk!)

Gödel’s incompleteness theorems are milestone results in mathematical logic and they have deep implications to foundations of mathematics. The first incompleteness theorem essentially states that there does not exists a Turing machine which can enumerate all the true statements of number theory, while the second incompleteness theorem demonstrates that no formal theory which covers all of mathematics (such as ZFC) can prove its own consistency.

A surprising and not well-known aspect of the first incompleteness theorem is that it can be derived by simply using the fact that the Halting problem of Turing machines is an undecidable problem. Furthermore, the resulting proof is more transparent (and accessible) than Gödel’s original proof.

In this lecture we will indicate how Gödel’s first incompleteness theorem can formalized and give an outline for an argument that proves it. In the course of formalizing it we will also take a closer look at first-order logic and proof systems. We will also briefly discuss the background of Gödel’s incompleteness theorems, Gödel’s original proof for the first incompleteness theorem and the second incompleteness theorem.

Consider a map of a country. For aesthetic reasons, it makes sense to strive for neighboring states (or provinces) to be colored using different colors. At the same time, using too many colors might be expensive to print and confusing. Therefore, we ask the following question: what is the minimum number of colors that I need to color a certain map? Intuitively, you might expect that the larger the map the more colors you would need. In this talk, we take a look at a remarkable result called the "Four-Color Theorem" which states that any map can be colored with at most four colors. We explore the historical significance of the problem and dip our toes into the key ideas behind the proof that had evaded mathematicians for over a century. Finally, we will look at the ripples that this proof created in the mathematical circles; as the first major result highly reliant on computers, it presented a watershed moment in mathematics: What is a mathematical proof? Can computers prove theorems? Does mathematics lose its credibility if it uses computers?

Module II: Computations in the physical world

Well before the apple falling on to Newton’s head, physicists had been searching for the underlying laws that govern reality. These beautiful mathematical descriptions explain how the stars move in a certain pattern, how small particles interact with each other, and so on. But “why” are these physical laws governing the universe rather than others? “What” are these physical laws doing and is there any underlying “problem” they are solving?

In this module, we are going to explore these questions through the lens of computation. Students will learn the basics of classical mechanics, statistical physics, quantum mechanics, and theory of gravity and peek into the connection to computation via “energy minimization” and “entropy maximization”. The goal is to (i) learn how physicists think, (ii) compare the concept of computations in the physical world to the mathematical formulation discussed in module 1, and (iii) build up sufficient language and knowledge for the next module.

• Lecture II.a (After the Falling Apple: Classical and Statistical Mechanics): Lagrangian and Hamiltonian mechanics; Statistical physics [slides] [video].
• Lecture II.b (Two Extremes: Quantum and Gravitational Theories): Quantum mechanics; Gravity and black holes [slides] [video].
• Lecture II.c (The Road to Reality: New Insights from Computation?): Pancomputationalism; Computation in physical systems; Physical Church-Turing Thesis; MIP*=RE [slides] [video].
• Lab II: Gravity circuits.
• Guest talk II.a: Xun Gao on “Quantum Correlations: the Resources Making Quantum Machines More Powerful”. [ ]

In this lecture, I will give an overview about the topic of quantum correlations. Quantum correlation is the statistical results produced by measuring quantum system which cannot be produced by classical physics (in general, stochastic process, equivalently hidden variable theory in terms of quantum foundation research or ontological model in terms of philosophy). Concretely, I will discuss very briefly about quantum entanglement, quantum non-locality (the correlation caused by entanglement) and quantum contextuality (a generalization of quantum non-locality). Then I will mention very briefly about a potential application of quantum correlation to machine learning models. Knowledge of quantum mechanics/quantum information is not required.

The computational paradigms of machine learning and quantum computing are different; machine learning tries to find the optimal solution in a heuristic way, while quantum computing harnesses the quantum laws allowing for more efficient and powerful computation as compared to classical computers. With the recent progress both in quantum computing and machine learning, we have entered into a realm that has not been explored before. In particular, while reaching the fault tolerance quantum computing requires another milestone, the question of whether we can gain an advantage by merging these technologies remains crucial. Thus within the realm of current noisy hardware and machine learning which are known to be robust to noise, in this talk, we first discuss three generations of quantum machine learning, each harnessing a particular aspect of quantum computers and targeting particular problems. The first scrutinize the power of quantum computers to work with high dimensional data and speed up algebra. The second generation circumvents the input/output problem by using a hybrid architecture, performing optimization on a classical computer while evaluating parameterized states on a quantum circuit, chosen based on a particular problem. Subsequently, the third generation is inspired by brain-like computation and uses the natural interaction and unitary dynamic of a given quantum system as a source for learning. After introducing each category, we provide examples from those algorithms and show the learning of several cognitive tasks such as multitasking, decision making, and long-term memory by taking advantage of several key features of quantum hardware.

The scientific community has long focused on finding efficient methods for solving problems. However, certain instances present a structure that does not allow finding an exact solution with the present knowledge and technology. Building on this, the attempts moved towards finding a probabilistically acceptable solution, with a change of orientation towards error analysis and reliability. Simulated Annealing is a method that achieves such a philosophy of thought by approximating in a clever way highly complex problems. It is a perfect example of inspiration from nature and thinking out of the box. In the lecture, we will show why such a system might be needed, and then derive directly an instance of the Algorithm step by step.

• Advanced section II.c: Tirukkovalluri Sri Sowmya on “Basic of Quantum Computing & Future Directions”. [video].
• Advanced section II.d: Avantika Agarwal on “Quantum Complexity Theory”. [video]. [ ]

Quantum computers are known to provide exponential speedups in solving certain problems, compared to the best known classical algorithms. While we know interesting results in this direction, for example, Simon's algorithm, we have also seen Grover's algorithm, which provides a "much less exciting" polynomial speedup. Is this the best possible improvement?

To address this question, in this talk, we will look at the "limitations" of quantum computers. In particular, we see a technique for proving that indeed some problems are "difficult" to solve on a quantum computer.

I have used many informal terms in this abstract, which will be formalised during the talk. It is recommended to attend the previous talk which introduces quantum computing, before this talk. If you are however unable to attend (or you know the basics of quantum computing already), I will introduce the required basics again at the start of my talk. So as long as you know high school mathematics, you are good to go!

• Advanced section II.e: Kartikeya Arora on “Quantum Computing from a Condensed Matter Perspective”. [video]. [ ]

Condensed Matter Physics is ubiquitous in almost all fields of science and technology. From superconductivity to topological phases to actual quantum computers, condensed matter covers it all. In this talk, I'll cover the basics of Quantum Mechanics, Condensed Matter Physics, and Tensor Networks. I'll also try to motivate the need for quantum computers from a condensed matter physics perspective. As a bonus, I am going to talk about time crystals and give you a recipe to create one for yourself too. Although I'll try to review the basics, it'll be better if you have attended the previous talks on quantum computing (or have some previous experience).

• Advanced section II.f: Chi-Ning Chou on “When Black Holes Meet Computational Complexity”. [video]. [ ]

In Lecture II.b, it was hinted that there are several computational aspects of black holes relevant to resolving paradoxes in quantum gravity. In this advanced section, we are going to give a more formal treatment on these connections. I will start with setting up the mathematical language and formally describe the information paradox as well as the firewall paradox. Then, I'll explain a proposal of resolving these paradoxes through the lens of computational complexity and the critics to this approach. Although there will be more math involved in this talk, the high-level story should still be very approachable to the general audience.

Module III: Computations in the biological world

Biology is the study of life. What is life? How does life form? What is the purpose of life, if there’s any? Unlike physics and mathematics, the challenges of reproducibility in biology, e.g., emergence properties and the precision in experiments, make it an extremely difficult subject to study. Statistical methods have been a prominent approach to establish biological knowledge while in recent years computational methods have arisen as a new angle to understand the biological world. Through the tango between biology and computation, how can the two inspire each other?

In this module, students will see various examples of the computational aspects of biology, including genomes, evolution, neuroscience, etc. The goal is to envision how the computational view can enrich understanding in biology and vice versa. In particular, we will be focusing on the open-ended nature of the living world and compare it with the previous two modules.

• Lecture III.a (Entering the Living World: Algorithms and Computations in Biology): What is Living Science; Computational Biology; Biological Computation [slides] [video].
• Lecture III.b (Two Mysteries: Evolution and Neuroscience): Evolution; Neuroscience [slides] [video].
• Lecture III.c (Challenges and Hopes: A Tango between Biology and Computation): Open-endedness; Emergence; The challenge and difficulty of studying biology; Mathematical ineffectiveness in biology [slides] [video].
• Lab III: Computation and intelligence in slime mold.
• Guest talk III.a: Angel Hsing-Chi Hwang on “Into the Unknown: (De)constructing Creativity in the Age of Human-Machine Partnership”. [video]. [ ]

It has become clear that machines — and specifically artificial intelligence — will play a critical role in the future of work. When we vision the blueprint of future human-machine collaboration, should our bot partners get involved in innovation or should we guard creativity as “the last territory of humanity”? In this lecture, we introduce the various constructs of human creativity and how they have been studied, measured, quantified, and even sequenced in multiple disciplines. Together, we discuss how, why, and whether machines can learn and practice creativity.

What is art? What is the nature of beauty? Is beauty in the eyes of the beholder or is beauty eternal? If we regard each individual’s experience in viewing or creating art as a computational process, what insights can we bring to this elusive question through a different angle? In this talk, I will give you a tour from caveman’s arts to Duchamp’s urinal and explore the potential connections between art and computation.

What’s intelligence? Is it memory, abstract reasoning or creativity? In this talk, we will dive in on the philosophical aspects of intelligence and attempt to give a working definition of intelligence. We will also give a few examples on different approaches in neuroscience to study intelligence.

• Advanced section III.a: Salvador Buse on “Computing with Chemistry: Turing Machines, Graph Colouring, and DNA”. [video]. [ ]

In this talk, I hope to convince you that we can think of chemical reactions as a programming language, which we can use to write any algorithm we choose. I will introduce two different models of chemical reactions — bulk reactions and stochastic reactions — and show how these can execute analog and digital logic, including the emulation of Turing machines. Finally, I will explain how we can compile chemical reactions to DNA, and introduce the field of DNA computing.

• Concluding Lecture: Summary; Students’ Sharings; A Warning and a Wish. [slides] [video].

Materials & Resources

The instructor will provide slides and additional notes available on the course website after each lecture. Optional readings and resources will also be posted on the course website. Before the course begins, we also encourage students to take a look at some of the references listed below.

Module I: The mathematical foundation of computation

Articles:

• Horswill, Ian. “What is computation?”, 2008, link.
• Wigderson, Avi. “The Godel Phenomenon in Mathematics: A Modern View.” Kurt Gödel and the Foundations of Mathematics (2011): 475, link.
• Davis, Philip J. “Toward a Philosophy of Computation.” For the Learning of Mathematics 3.1 (1982): 2-5, link.
• Knuth, Donald E. “Computer science and its relation to mathematics.” The American Mathematical Monthly 81.4 (1974): 323-343, link.

Introductory Books:

• Moore, Cristopher, and Stephan Mertens. The nature of computation. OUP Oxford, 2011, link.
• Barak, Boaz. Introduction to Theoretical Computer Science. Online, link.
• Rautenberg, Wolfgang. A concise introduction to mathematical logic. New York, NY: Springer, 2006, link.

• Wigderson, Avi. Mathematics and computation. Princeton University Press, 2019, link.
• Arora, Sanjeev, and Boaz Barak. Computational complexity: a modern approach. Cambridge University Press, 2009, link.

• Aaronson, Scott. “Why philosophers should care about computational complexity.” Computability: Turing, Gödel, Church, and Beyond 261 (2013): 327, link.
Module II: Computations in the physical world

Articles:

• Preskill, John. “Quantum computing 40 years later.” arXiv preprint arXiv:2106.10522 (2021), link.
• Gualtiero Piccinini and Corey Maley. “Computations in Physical Systems.” Stanford Encyclopedia of Philosophy, 2010, link.
• Yao, Andrew Chi-Chih. “Classical physics and the Church–Turing thesis.” Journal of the ACM (JACM) 50.1 (2003): 100-105, link.
• Piccinini, Gualtiero and Corey Maley, “Computation in Physical Systems”, The Stanford Encyclopedia of Philosophy (Summer 2021 Edition), Edward N. Zalta (ed.), link.

Introductory Books:

• Feynman, Richard P., Tony Hey, and Robin W. Allen. Feynman lectures on computation. CRC Press, 2018, link.
• Penrose, Roger. The road to reality: A complete guide to the laws of the universe. Random house, 2005, link.
• Mezard, Marc, and Andrea Montanari. Information, physics, and computation. Oxford University Press, 2009, link.

• Mézard, Marc, Giorgio Parisi, and Miguel Angel Virasoro. Spin glass theory and beyond: An Introduction to the Replica Method and Its Applications. Vol. 9. World Scientific Publishing Company, 1987, link.
• Sakurai, Jun John, and Eugene D. Commins. “Modern quantum mechanics, revised edition.” (1995): 93-95, link.
• Alber, Gernot, et al. Quantum information: An introduction to basic theoretical concepts and experiments. Vol. 173. Springer, 2003, link.
Module III: Computations in the biological world

Articles:

• Schnitzer, M. Biological computation: Amazing algorithms. Nature 416, 683 (2002), link.
• Chelly Dagdia, Z., Avdeyev, P. & Bayzid, M.S. Biological computation and computational biology: survey, challenges, and discussion. Artif Intell Rev 54, 4169–4235 (2021), link.
• Piccinini, Gualtiero. “Computationalism.” The Oxford handbook of philosophy of cognitive science. 2012, link.

Books:

• Nowak, Martin A. Evolutionary dynamics: exploring the equations of life. Harvard university press, 2006, link.
• Jones, Neil C., Pavel A. Pevzner, and Pavel Pevzner. An introduction to bioinformatics algorithms. MIT press, 2004, link.
• Gillespie, John H. Population genetics: a concise guide. JHU Press, 2004, link.

• Stanley, Kenneth O., and Joel Lehman. Why greatness cannot be planned: The myth of the objective. Springer, 2015, link.
• Schrödinger, Erwin. What is life?: With mind and matter and autobiographical sketches. Cambridge university press, 1992, link.
• Mayr, Ernst. This is biology : the science of the living world. Harvard University Press, 2001, link.
• Banatre, Jean-Pierre, et al., eds. Unconventional Programming Paradigms: International Workshop UPP 2004, Le Mont Saint Michel, France, September 15-17, 2004, Revised Selected and Invited Papers. Vol. 3566. Springer Science & Business Media, 2005, link.

FAQs

Q: If I don’t have a strong math and science background, what will I learn and will there be any difficulties?
A: Most of the lectures will focus on the high-level stories and the underlying philosophy so will be very accessible to anyone! The instructor will also make sure more knowledgeable students won’t dominate the discussion and create a friendly environment.

Q: What if I already know most of the materials, what will I get from this course?
A: If this is the case, we encourage you to talk to us in advance. In general, the course materials won’t touch the details of each subject and instead focus on high-level stories and comparison with each other. This should provide you with new angles and insights into the subjects you originally were familiar with. Advanced and extra reading/reference materials will be provided on request.

Q: Is there homework for the mini-course?
A: No, but we will provide food-for-thought questions and brainstorming problems at the end of each lecture.

Q: Will all the lectures be recorded?
A: Yes, the recording link will be sent out once available and in long term will be posted somewhere publicly.