When you’re converting Celsius to Fahrenheit, a single machine with a single thread of execution can get the job done. However, many of the computational problems that will earn you a living will demand more of you and your computer. Perhaps you will have to enlist many computers to get the job done. Or perhaps you will need to run a task in the background while the user keeps interacting with your software. Or maybe you will have to serve many users at once. Whatever the cause, more than one thing is happening at a time in the software you will write. Today we look at how our languages can help us deal with concurrent flows of execution.

Concurrency, Parallelism, and Other Terms

You’ve touched upon the ideas of concurrency in previous courses, and I sure wish I knew what you learned. Some of this might be review, and you might feel patronized. I’m more inclined to think that you’ve forgotten what you learned or you never learned it in the first place.

Computer scientists talk about two different but related ideas: concurrency and parallelism. Not everyone agrees on their distinction. In general, we can say that we make a program concurrent because there are many things happening at once, and we want our software to be able to handle it all. We make portions of a program run in parallel in order to improve performance. So, concurrency is more about what’s going on outside of our program, and parallelism is about optimizing what’s going on inside our program. The ideas are orthogonal; they can appear together or in isolation.

Each task in a program is handled by a sequential, logical flow of execution. When multiple flows appear to be running at the time, we have concurrency. When we hit the play button on our media player but then continue to navigate through our music library, we have concurrency. The program is not waiting for the music to stop before showing us our Daft Punk collection. When we enter a URL in the web browser that triggers a slow download and we impatiently hit the stop button, we have concurrency. The program is not waiting for the download to finish before letting us stop it.

There may be only a single processor with a single core running these tasks. However, the user feels that the program is interactive and responsive because multiple flows take turns getting the hardware. Interleaved execution on non-parallel hardware is called multitasking. Our generation doesn’t really know the world that existed before multitasking, where one clicked a button and waited for the task to finish.

When our computer has multiple cores or processors, we can physically execute flows at the exact same time. If execution is not just logical but physical, we have parallelism.

We make our programs execute with concurrency and parallelism for several reasons:

• For increased interactivity.
• More efficient use of hardware.
• Because Moore’s law may not hold forever, as transistors reach their physical limits. The smaller the chips, the harder it is to dissipate heat and the harder it is to keep electrons following their intended paths. Improved software performance cannot rely solely on hardware improvements.

The benefits are not without costs, however. Our software becomes more complex when it executes multiple flows at a time, and we encounter a whole new class of vulnerabilities that are hard to test.

Levels of Concurrency

Concurrent or parallel execution may be applied at different levels of execution:

• Instruction-level: machine code instructions are executed simultaneously, as in a pipelined architecture.
• Statement-level: a high-level language might include syntax for scheduling statements to execute in parallel. For example, one might write a parallel-for loop to have each iteration handled by a separate flow of execution.
• Unit-level: a subprogram is called and is run in a separate flow of execution.
• Program-level: each program is run separately in its own process, with scheduling and communication handled by the operating system.

Operating systems generally provide two vehicles for concurrency: processes and threads. A process has its own memory space with its own code section, globals, stack, and heap. Processes are said to be heavyweight tasks. Threads run within a process. All threads share the memory of their process. Threads are more lightweight compared to processes since they don’t need their own resources.

Where programming languages can help is in statement-level parallelism and unit-level concurrency. Because threading has low overhead, it’s the preferred vehicle that languages use to achieve speedups.

As an example of statement-level parallelism through threading, consider this C function to sum an arbitrary number of vectors:

void add_vectors(int nvectors, int length, double *vectors[], double sum[]) {
  for (int i = 0; i < length; ++i) {
    sum[i] = 0.0;
    for (int v = 0; v < nvectors; ++v) {
      sum[i] += vectors[v][i];
    }
  }
}

The OpenMP library provides a set of preprocessor flags that we use to make the outer loop run in parallel:

void add_vectors(int nvectors, int length, double *vectors[], double sum[]) {
  #pragma omp parallel for
  for (int i = 0; i < length; ++i) {
    sum[i] = 0.0;
    for (int v = 0; v < nvectors; ++v) {
      sum[i] += vectors[v][i];
    }
  }
}

When I compile both the serial and parallel versions and time them on my MacBook Pro, I see a speedup of around 4:

$ clang -oserial -O0 sum.c
$ clang -oparallel -Xpreprocessor -fopenmp -lomp -O0 sum.c
$ time ./serial; time ./parallel
./serial    1.48s user 0.65s system   91% cpu 2.333 total
./parallel  4.92s user 1.57s system 1051% cpu 0.617 total

OpenMP isn’t strictly a language construct, but it exploits the preprocessor’s pragma directive, which is designed to steer the compiler’s behavior. In this case, the compiler will automatically insert code to spawn a number of threads and distribute out the work of the loop evenly among them.

Ruby provides unit-level parallelism through its Thread class, which expects a block of code to execute:

threads = (0...4).map do |threadIndex|
  Thread.new do
    puts threadIndex
  end
end

threads.each { |thread| thread.join }

Java provides similar unit-level parallelism, but its Thread class expects an object instead of a block of code:

Thread[] threads = new Thread[4];

for (int i = 0; i < 4; i++) {
  int j = i;
  threads[i] = new Thread(new Runnable() {
    public void run() {
      System.out.println(j); 
    }
  });
  threads[i].start();
}

for (int i = 0; i < 4; ++i) {
  while (true) {
    try {
      threads[i].join();
      break;
    } catch (InterruptedException e) {
    }
  }
}

Lambdas in Java effectively hide the fact that the unit is an object rather than a subprogram:

for (int i = 0; i < 4; i++) {
  int j = i;
  threads[i] = new Thread(() -> System.out.println(j));
  threads[i].start();
}

for (int i = 0; i < 4; ++i) {
  threads[i].join();
}

Under the hood, however, the lambda is still an instance of the Runnable class.

Single-threading

Not all concurrent flows use multiple threads. Co-routines, for example, are two or more flows who are cooperating in some computational task. Since they are not intended to run in parallel, they execute in the same thread.

For instance, suppose I want to iterate through a 3D grid in JavaScript. Writing a triple-nested loop over and over again would clutter up my code, so I turn to a generator, which allows me to write my iteration code once. The generator looks like a function definition, but it’s got a * symbol in front of the function name and a yield statement in the innermost loop:

function *griderator(width, height, depth) {
  for (let z = 0; z < depth; z += 1) {
    for (let y = 0; y < height; y += 1) {
      for (let x = 0; x < width; x += 1) {
        yield [x, y, z];
      }
    }
  }
}

That yield statement effectively pauses the generator and hands control back to this code that’s consuming its values:

for (let p of griderator(4, 3, 5)) {
  console.log(p);
}

Many of our event-driven systems like browser-based JavaScript, Android, and the Unity game engine using a single-threaded execution model to avoid the dangers of threads clobbering shared resources. Such systems typically use an infinite event-handling loop that looks something like this:

while (event = q.dequeue) {
  dispatch event to handler  
}

The runtime adds new events to the queue as the user interacts with the software, and developers can trigger their own events.

Multi-Threading on a Single Thread

Some of our interpreted languages, including Python and some implementations of Ruby, give the appearance of having multiple threads, but the runtime ensures that only a single thread has the CPU at a given time. A global interpreter lock (GIL) is used as a conch. Whichever thread has the GIL has the CPU.

Let’s see what this form of multi-threading means. Consider this Ruby code which sums up vectors in serial fashion:

nvectors = 300
length = 100000

vectors = Array.new(nvectors) {Array.new(length) {0}}
sum = Array.new(length)

for i in 0.step(sum.size - 1, 1)
  sum[i] = 0
  for v in 0...vectors.size
    sum[i] += vectors[v][i]
  end
end

Now we add threads to distribute out the task just as we did with OpenMP:

nthreads = ARGV[0].to_i
nvectors = 300
length = 100000

vectors = Array.new(nvectors) {Array.new(length) {0}}
sum = Array.new(length)

threads = (0...nthreads).map do |ithread|
  Thread.new do
    for i in ithread.step(sum.size - 1, nthreads)
      sum[i] = 0
      for v in 0...vectors.size
        sum[i] += vectors[v][i]
      end
    end
  end
end

threads.each {|thread| thread.join}

When we time these, we find that the threaded version is significantly slower than the serial version:

$ time ./sum_serial.rb; time ./sum_parallel.rb 4
./sum_serial.rb      3.55s user 0.07s system 99% cpu 3.628 total
./sum_parallel.rb 4  4.31s user 0.07s system 99% cpu 4.378 total

The threads do not execute in parallel, and they add overhead. Many systems programmers look down upon scripting languages because they lack “real threads.”

Scheduling

The job of the operating system is to arbitrate between the many concurrent tasks who are all vying for the finite resources of the machine. Memory, disk, and network cards are all shared resources. So is the CPU itself. A scheduler is used to ensure that each task gets its share of the CPU.

The scheduler puts each task in one of these states:

• new: a task that has been created but not started
• ready: a task that is waiting to be executed
• running: a task that is currently executing
• blocked: a task that is waiting on I/O
• dead: a task that has finished executing

A task goes from ready to running when the scheduler schedules it. After a certain of time elapses, the scheduler pauses it and gives another ready task a chance. A running task might pause itself by requesting input or emitting output. The I/O will run in the background, and the task will be blocked while another task is scheduled. A blocked task will return to the ready queue when the I/O finishes.

How does a scheduler decide what task to pull out of the ready queue? In round-robin scheduling, each task gets a fixed amount of time. Tasks may instead be assigned priorities, with higher-priority tasks being given more turns or more time. In real-time systems, tasks announce their deadlines, and the scheduler does its best to keep every entity satisfied. Life may depend on it.

Dangers

A concurrent program that is progressing on schedule has liveness. But a program’s liveness can be threatened. If two threads are codependent, each waiting on a resource that the other currently holds, the program is in a state of deadlock. Someone will have to give up their resource to restore liveness.

If a concurrent program expects a particular scheduling in order to produce a correct result, we have a race condition. This can happen when access to shared resources is not controlled. Consider this Ruby program which creates two threads, both of which increment a shared i:

#!/usr/bin/env ruby

x = 1
y = 1

a = Thread.new do
  x += y
  # sleep 1
  y += x
end

b = Thread.new do
  sleep 1
  x += y
  y += x
end

a.join
b.join

puts "#{x} #{y}"

If all of a executes before b, then we expect to see this sequence of state changes:

initial   -> x = 1, y = 1
a[x += y] -> x = 2, y = 1
a[y += x] -> x = 2, y = 3
b[x += y] -> x = 5, y = 3
b[y += x] -> x = 5, y = 8

But if a gets paused between its instructions, then we get this sequence of state changes:

initial   -> x = 1, y = 1
a[x += y] -> x = 2, y = 1
b[x += y] -> x = 3, y = 1
b[y += x] -> x = 3, y = 4
a[y += x] -> x = 3, y = 7

We use sleep calls to simulate a thread getting interrupted by the scheduler. If we didn’t expect x to change in the middle of the execution of a, then this code has a race condition.

Synchronization

To avoid race conditions, we control access to shared resources through synchronization. In cooperative synchronization, one task depends on resources generated by another task. Together they form a pipeline. The archetypal producer/consumer problem is an example where cooperative synchronization is needed. The producer wants to add new elements to a shared queue. The consumer wants to remove elements from the shared queue. Their access to the shared queue must be controlled so they don’t leave the queue in a broken state. In competitive synchronization, independent tasks contend with each other for resources. Perhaps your search engine is crawling the web. You have a queue of links that must be crawled, and threads will draw from the queue. Access to the queue must be synchronized.

In 1965, Djikstra proposed the semaphore as a mechanism for control access. A semaphore bundles together an integer counter, a task queue, and two operations that your textbook calls wait and release. The semaphore’s counter is seeded with the number of tasks that may access the shared resource concurrently. Before a task may access a shared resource, it first calls wait on the semaphore. If the counter is above 0, the counter is decremented and the task proceeds. If the counter is 0, then the task gets thrown into a queue and some other task is run. When a task finishes accessing the shared resource, it calls release. If there’s a task in the queue, it is run. If not, the counter is incremented. When the counter is seeded with 1, we have a mutex. Only one entity can access the shared resource. The mutex makes it mutually exclusive.

Semaphores are a low-level synchronization primitive. Djikstra recommended that such synchronization be encapsulated inside a data structure. A data structure with builtin synchronization is called a monitor. Some of the collections in Java are monitors. You can tag a method as synchronized if you want only one entity to access the collection at a time.

Exceptions

Now we make a sharp pivot to a discussion of dealing with errors in programs. Many different kinds of errors can happen. Perhaps an expected file is missing. Perhaps the caller gave us a null object. Perhaps there’s no more disk space. Perhaps the string that we’re try to convert to an integer only has letters in it. What should we do when we detect an error? A range of choices lies before us:

• We could do nothing and silently continue. This is a terrible, inhumane strategy.
• We could print an error message. This is slightly better, but output only helps the user and only if the user has a way of seeing the output. The rest of the code knows nothing about the error.
• We could terminate the program. This is pretty severe. Some errors are to be expected and can be recovered from.
• We could expect the detector of the error to handle the error, but this requires considerable duplication of effort.
• We could return a special error flag. This isn’t bad, but it assumes there’s a special value that can be returned. indexOf often returns -1 when the target object cannot be found in a list, but what does max return when you give it the empty list? If you say 0, then how do you distinguish that from max([0, 0, 0])?
• We could pass an error handler to each subprogram, but the caller cannot be expected to know what errors might be encountered by subprogram code.
• We could pass an extra out parameter to hold the error flag, as in max(list, &status). This is getting better, but the burden is on the caller to check the status and propagate the status to its caller.
• We could have each function return a value of a special type that can be in one of two states: Ok and Error. Haskell has optional types Maybe and Either that support this kind of error propagation.
• We could stop the flow of execution and send a special object called an exception to the caller describing the exceptional event. The error-detecting code throws or raises an exception. After an exception is raised, it goes through a binding phase. The surrounding code may have opted-in to handling the exception by wrapping the dangerous code in a try/catch block. If so, the exception is bound in the catch statement. If the immediate scope does not attempt to catch it, the active subprogram is terminated, its stack frame is wiped, and the exception propagates up to the next caller. If no handler is found, the program terminates. If a handler is found, then we proceed to continuation. Where do we pick up execution? If the handler itself threw an exception, then that exception goes through the same propagation process. Otherwise we pick up execution on the first line of code after the handler. In all cases, we can wrap dangerous code up in a try statement with a finally block that will be executed before continuation or propagation proceeds. We use finally to deallocate heap blocks, close files, and other cleanup tasks.

Many of our modern languages support the final of these strategies. C doesn’t have exceptions. It expects callers to inspect a special global error flag after each function call. Bjarne Stroustrup would have none of this. C++ got exceptions. In C++, exceptions can be of any type, an instance of a class or even an int.

Java has exceptions too. Only subclasses of Exception may be thrown. Language critics complain that Java forces you to acknowledge certain kinds of exceptions, like FileNotFoundException. You must either catch these checked exceptions or declare that your method throws them. A lot of folks like to write code like this:

try {
  return list[0];
} catch (IndexOutOfBoundsException e) {
}

Don’t do this. Exceptions are for exceptional events that you can’t prevent. A bad index is something you can prevent. Use an if statement instead:

if (list.length > 0) {
  return list[0];
}

But up-and-coming Rust uses a special Result type, which may be either Ok with a value or Err with an error object. Go is another language without exceptions. This blurb appears in its FAQ:

We believe that coupling exceptions to a control structure, as in the try-catch-finally idiom, results in convoluted code. It also tends to encourage programmers to label too many ordinary errors, such as failing to open a file, as exceptional.

Go takes a different approach. For plain error handling, Go’s multi-value returns make it easy to report an error without overloading the return value. A canonical error type, coupled with Go’s other features, makes error handling pleasant but quite different from that in other languages.

If we find ourselves in a language without exceptions we might be able to hack something like them together using goto statements. This really only works within the context of a single scope, which is quite a departure from full-blown exceptions. Consider this Java-ish code, which I don’t like:

bool addButCapAtFive(ArrayList<String> choices, String item) {
  if (choices.contains(item)) {
    return false;
  }

  try {
    if (choices.length == 5) {
      throw new OverflowException();
    }
  } catch (OverflowException e) {
    System.err.println("Too many");
    System.exit(1);
  }

  choices.add(item);
  return true;
}

We can recast it in C with goto like this:

int add_but_cap_at_five(list *choices, char *item) {
  if (choices->contains(item)) {
    return 0;
  }

  if (choices->length == 5) {
    goto overflow_handler;
  }

  choices->add(item);
  return 1;

overflow_handler:
  System.err.println("Too many");
  System.exit(1);
}

In general, exceptions are favorable because they are highly structured and some aspects (but not all) of their proper handling can be checked by our tooling.

Conclusion

See you next time!

P.S. It’s time for a haiku!

John beat that steam drill
But his heart stopped soon after
A race condition

Software developers spend considerable time tinkering under the hood of their systems. Clients, on the other hand, are happy to only have a few buttons to push to make those systems work. A simpler view of a complex system is called an abstraction. When a humane abstraction is available, clients can ignore irrelevant and low-level details and focus on the features that matter to their task. Today we examine forms of abstraction that organize and simplify our code for the benefit of our clients.

Types of Abstraction

Our code can be abstracted in several ways. We’ve already examined functions. A function is a procedural abstraction. In a clean procedural abstraction, the caller of a function is only concerned with the parameters and return values, not with the underlying process. Programs themselves are procedural abstractions that can be composed in the shell or through a process API.

When we clump data and its related functions together, we have a data abstraction. Classes are a prime example of a data abstraction. Polymorphic functions also rely on data abstractions in order to serve many different types of data.

Some features of our languages are meant to help organize our code. Let’s call such a feature a grouping abstraction. Examples include packages, namespaces, static classes, and modules. A grouping abstraction gathers together types, data, and subprograms related to a theme. If we don’t need any of a group’s entities, we can ignore it.

Most of our discussion will be centered around data abstraction, but some of the ideas we hit upon will apply to the other forms as well.

Abstract Data Types

Computer scientists have a toolbox of useful data abstractions called abstract data types (ADTs). These include lists, stacks, sets, maps, and even floating point numbers. An ADT defines a set of values and the operations that it supports, but no mention is made of its representation or how its operations are implemented.

When we realize an ADT by choosing a representation and implementing its operations, we have a concrete data type. Some of the ideals of the ADT might be lost since we’re dealing with the constraints of memory and numeric precision. Both ArrayList and LinkedList are concrete data types that implement the list ADT. Both HashMap and TreeMap are concrete data types that implement the map ADT. Both float and double are concrete data types that implement the floating point number ADT.

Many of our languages allow us to define our own data types and advertise their interfaces as ADTs. Let’s discuss a few features our languages provide to help us form ADTs.

Encapsulation

The grouping together of related code and data is called encapsulation. Some groupings are based on syntactic structures. Examples include namespaces in C++ and C#, packages in Java, modules in Ruby and JavaScript. Classes, interfaces, structs are also grouping mechanisms. Other groupings are based on files. Examples include C header files and ad hoc modules based only on the file system.

Encapsulation offers several advantages across the design space of our languages:

• It organizes our code and focuses our attention.
• It allows separate compilation, which improves build speed.
• It makes us less vulnerable to name collisions.

Programming linguists categorize encapsulation schemes on several dimensions:

• Physical encapsulation schemes put all grouped code into a contiguous syntactic unit. Logical encapsulation allows non-contiguousness. For example, the definition of a class may be spread across many files.
• In a naming encapsulation, the grouping is named and the name is used to access the encapsulated items. In a non-naming encapsulation, no name is assigned to the grouping.
• Grouping-only encapsulation schemes bundle items together but leave the bundled items public. Information hiding schemes allow some items to be private.

Information Hiding

Some of the implementation of an ADT should not be available to clients. Making these implementation details private or inaccessible is called information hiding.

Keeping an object’s state private should be our default behavior. If the client has direct access to the state, the client may put our object into a situation that should not be legal.

For example, consider this Circle class:

class Circle {
  constructor(radius) {
    this.radius = radius;
    this.area = Math.PI * radius * radius;
  }
}

If the client reaches in and modifies radius, the area property will no longer be accurate. JavaScript doesn’t give us many tools for hiding information. Ruby forces all instance variables to be private. Other languages allow us to mark individual items as public, private, or protected.

To control access to an item that has been marked private, we provide accessor methods. Getters allow clients to read the value of a property. Setters allow clients to update the value. Some accessors don’t even have a backing variable, like area in this JavaScript code.

class Circle {
  constructor(radius) {
    this.radius = radius;
  }

  get area() {
    return Math.PI * this.radius * this.radius;
  }

  set area(newArea) {
    this.radius = Math.sqrt(newArea / Math.PI);
  }
}

const c = new Circle(10);
c.area = 30;

Examples

To test the soundness of a taxonomy, we should fit the systems with which we’re familiar into it.

	Physical	Logical
naming	Java classes and packages	Ruby classes and modules, C++ namespaces
non-naming	.c, .cpp, .h files, JavaScript modules
grouping only	.h files	Ruby modules, C++ namespaces
information hiding	.c or .cpp files, Java classes and packages	Ruby classes

The Charade of Classes

Did you know that in the early days of C++, it was just a thin veneer over C? Classes were just syntactic sugar. We might have had a class that looked like this:

class Hero {
  public:
    Hero(char *name, int hp) {
      this->name = strdup(name);
      this->hp = hp;
    }

    void cure(int boost) {
      hp += boost;
    }

  private:
    char *name;
    int hp; 
}

This code would get compiled down to C, where classes weren’t a thing. But C does have structs, so Hero would get turned into one of those:

struct Hero {
  char *name;
  int hp;
}

What about the methods of Hero? They’d get turned into regular C functions:

void cure(struct Hero *hero, int boost) {
  hero->hp += boost;
}

Notice how the invoking object became the first parameter. But it wasn’t called hero. It was called this:

void cure(struct Hero *this, int boost) {
  this->hp += boost;
}

When the compiler uses the compile-time type to figure out what method to call and wires the call into machine code, that’s called static dispatch.

What happened when we instantiated an object, like this?

Hero *frank = new Hero("Frank", 260);

It turned into code like this:

void initialize_hero(struct Hero *this, char *name, int hp) {
  this->name = strdup(name); 
  this->hp = hp;
}

int main() {
  struct Hero *frank = (struct Hero *) malloc(sizeof(struct Hero));
  initialize_hero(frank, "Frank", 260);
  // ...
}

Those two lines in main were particularly dear to Bjarne Stroustrup, the inventor of C++. In C, it was very easy to allocate space for an instance of a data structure but then forget to initialize it. Stroustrup believed that allocation and initialization should not be separate steps, that “resource acquisition is initialization” (RAII). Many other languages adopted a similar approach. To construct an object is to allocate memory for it and assign the object’s initial state.

Dynamic Dispatch via Vtables

Eventually, C++ became distinct enough that Stroustrup developed a compiler that took C++ directly to machine code. One of the features of C++ that warranted a custom compiler was subtype polymorphism. For subtype polymorphism to work in languages like Java and C++, we need to figure out what version of a method to call. In a polymorphic setting, we don’t exactly know what the type of the object is until runtime, so we call the process of figuring out the method dynamic dispatch.

Computer graphics pioneer Jim Blinn wrote a column in 1990s when C++ was gaining in popularity, and he dismissed its polymorphism, saying he’d been doing something like it in Fortran for the longest time. Here’s his polymorphism in Fortran:

Structure OBJTBL is a pair of a type tag and a pointer. Subroutine DRAW is the polymorphic conductor code. It walks through a conditional ladder, inspects the type tag, and dispatches accordingly.

If this was how dynamic dispatch was implemented today, its designer would have been eaten alive on Reddit and Stack Overflow. Why? For one, the conductor code is full of conditional statements. We saw in computer architecture that conditional statements thwart pipelining. For two, every time we add a new type to our hierarchy, we have to go back to the conductor code and update it. We’d really like to avoid touching code once it’s working. We tend to mess stuff up. There’s that saying, “separate that which changes from that which doesn’t.” Thankfully, dynamic dispatch is implemented differently.

Let’s look at the memory footprint of some C++ classes. We’ll start with an int wrapper:

#include <iostream>

class A {
  public:
    A(int a) :
      a(a) {
    }

    void foo() {}

  private:
    int a;
};

int main(int argc, char **argv) {
  A a(3); 
  return 0;
}

Let’s compile this and inspect a with the debugger. Here’s the g++/lldb magic that makes that possible:

g++ -std=c++11 -g footprint.cpp
lldb ./a.out
(lldb) breakpoint set --name main  # break  on main
(lldb) run                         # run until breakpoint
(lldb) n                           # step through next statement
(lldb) print sizeof(a)             # Probably 4 or 8
(lldb) x/4ub &a                    # Examine the 4 bytes of a

When I run this on my macOS laptop, I get this output:

(lldb) print sizeof(a)
$1 = 4
(lldlldb) x/4ub &a
0x7fff5fbfe918: 3       0       0       0

What’s the endianness of my machine?

What happens when we add some more state? Like an array of characters:

class A {
  public:
    A(int a) :
      a(a) {
    }

  void foo() {}

  private:
    char cs[3] = {'a', 'b', 'c'};
    int a;
};

When I run this on my macOS laptop, I get this output:

(lldb) print sizeof(o)
$1 = 8
(lldb) x/8ub &o
0x7fff5fbfe918: 97      98      99      0       3       0       0       0

The char[] is padded so that a has a word-aligned address.

Let’s add a subclass:

class B : public A {
  public:
    B(int a, float b) :
      A(a),
      b(b) {
    }

  private:
    float b;
};

int main(int argc, char **argv) {
  A a(3); 
  B b(10, 3.14159); 
  return 0;
}

Class A hasn’t changed. But let’s inspect our instance of B:

(lldb) print sizeof(b)
$0 = 12
(lldb) x/12ub &b
0x7fff5fbfe908: 97      98      99      0       3       0       0       0
0x7fff5fbfe910: -48     15      73      64

This output tells us that the first bytes of a B instance are its A foundation. We also see the wild bytes of the float that come after.

Notice that no space in these objects is reserved for code. Code doesn’t live in the object; it is shared across all objects and lives in the text segment of the process’s memory space. We package code and data together in an object-oriented programming language, but the code and data are separated within the memory of the machine.

What do you suppose happens if we have this assignment?

A ab = b;

You’re right. The extra bytes of b get chopped off. This is called object slicing.

Let’s make foo a virtual function:

virtual void foo() {}

What effect does this have on a and b? Let’s check:

(lldb) print sizeof(a)
$0 = 16
(lldb) print sizeof(b)
$1 = 24
(lldb) x/16ub &a
0x7fff5fbfe910: 32      16      0       0       1       0       0       0
0x7fff5fbfe918: 97      98      99      0       3       0       0       0
(lldb) x/24ub &b
0x7fff5fbfe8f8: 72      16      0       0       1       0       0       0
0x7fff5fbfe900: 97      98      99      0       3       0       0       0
0x7fff5fbfe908: -48     15      73      64      0       0       0       0

The objects are bigger. Both have 8 new bytes at their beginning. That, my friends, is a pointer to each class’s virtual table or vtable, the vehicle by which fast dynamic dispatch happens. Object b also gets an extra 4 bytes at the end. I believe this is used to ensure that the vtable pointers are aligned to 8-byte boundaries when we have a sequence of B instances back to back, as in an array or struct.

Let’s examine a class hierarchy (courtesy of Stroustrup) and see how these virtual tables are laid out:

class A {
  public:
    virtual void f();
    virtual void g(int);
    virtual void h(double);
    void i();
    int a;
};

class B : public A {
  public:
    void g(int);
    virtual void m(B*); 
    int b;
};

class C : public B {
  public:
    void h(double);
    virtual void n(C*);
    int c;
}

int main() {
  A o;
  B p;
  C q;
  return 0;
}

The vtable of A looks like this:

index	Function	Address
0	f	&A::f
1	g	&A::g
2	h	&A::h

Method i gets no entry, because it is not virtual.

The vtable of B looks like this:

index	Function	Address
0	f	&A::f
1	g	&B::g
2	h	&A::h
3	m	&B::m

Class B overrides g and adds m. Its other entries just point to the versions from A.

The vtable of C looks like this:

index	Function	Address
0	f	&A::f
1	g	&B::g
2	h	&C::h
3	m	&B::m
4	n	&C::n

These vtables are stored once per class—they’re static. Each instance of A, B, or C has an implicit instance variable pointing to its class’ vtable. We’ll call that member vptr. An instance of A is laid out like this in memory:

vptr

a

An instance of B is laid out like this in memory:

vptr

a

b

An instance of C is laid out like this in memory:

vptr

a

b

c

Here’s a more graphic representation of memory:

Now, suppose we have a piece of conductor code that takes in a pointer to A, but under the hood the object is an instance of C:

void conduct(A *p) {
  p->g(11);
}

What exactly has to happen here to get the right code running?

1. We must conjure up p‘s vptr.
2. In the table pointed to by p->vptr, we must locate the address of the code for g, which has index 1.
3. We must call that code with both the invoking object and 11 as parameters.

Under the hood, this invocation turns into a crazy indirect memory lookup. We get a function pointer from the vtable pointed to by the object and call it:

(*(*p->vptr)[1])(p, 11);

So, subtype polymorphism is made fast at runtime through a table lookup. No conditionals are needed. Is this indirection free? No, because there are memory reads. Will these memory reads be fast? The vtables will likely be stored in cache if they are frequently accessed. They are small and there’s only one per class. The object is also likely to be in cache if we’re operating on it.

C++ isn’t the only language to use vtables. Consider this statement from the Java Virtual Machine specification:

The Java Virtual Machine does not mandate any particular internal structure for objects.

In some of Oracle’s implementations of the Java Virtual Machine, a reference to a class instance is a pointer to a handle that is itself a pair of pointers: one to a table containing the methods of the object and a pointer to the Class object that represents the type of the object, and the other to the memory allocated from the heap for the object data.

Multiple Inheritance

Many of our object-oriented languages only allow a class to have one superclass. With only one superclass, the structure of the subclass is easy to determine: just tack its state after the superclass state in the class instance record.

What if you wanted a class to have two superclasses? Something like ClickableAnimatedSprite? Well, the class instance record would have to hold both parents. Instead of a simple linear structure, the class instance record would have to hold a hierarchy. Additionally, there is the possibility that both superclasses might have a method of the same name. When the subclass calls the method, which superclass’s version is called? In C++, you must qualify the call with the intended superclass:

ThatParent::foo();

Java forbids multiple inheritance. One can implement multiple interfaces, however. In early Java, interfaces didn’t have any method implementations, so a shared method wasn’t an issue. In Java 8, interfaces can have default implementations. You must disambiguate through qualified access, just as with C++.

What if both superclasses themselves have a common superclass? Your inheritance structure will look something like this:

A   A
|   |
B   C
 \ /
  D

What we want is this:

  A
 / \ 
B   C
 \ /
  D

This is called the dreaded diamond pattern, and C++ is one of the few languages that allows you to achieve it. You unite the shared base object by using virtual inheritance:

class Grandparent {}
class ParentA : virtual Grandparent {}
class ParentB : virtual Grandparent {}
class Child : public ParentA, ParentB {
  // construct the whole hierarchy manually
  Child() : Grandparent(), ParentA(), ParentB() {}
}

The compiler expects the subclass to manually steer the construction of the superclasses.

Templates vs. Generics

Another way we abstract data is by writing one structure that serves many types. Here’s how we might make a 2-tuple in C++ using templates:

class Tuple<typename T, typename U> {
  public:
    Tuple(T first, U second) : first(first), second(second) {}

  private:
    T first;
    U second;
}

The C++ compiler will generate a new version of this class for every type combination that it encounters. There’s no mechanism for compiling the class ahead of time. It needs to stay in its source form if others are to be able to access it.

The Java folks decided to take a different approach, one that didn’t sprinkle so many similar versions of the templated structures. Here’s our 2-tuple written using Java generics:

class Tuple<T, U> {
  private T first;
  private U second;

  public Tuple(T first, U second) {
    this.first = first;
    this.second = second;
  }
}

Behind the scenes, the compiler generates one bytecode representation of Tuple in which the type parameters have been replaced with real types determined by their bounds. If the type parameters are not bounded, the parameters are replaced with Object. This erasure leads to this non-generic version of Tuple:

class Tuple {
  private Object first;
  private Object second;

  public Tuple(Object first, Object second) {
    this.first = first;
    this.second = second;
  }
}

Java piggybacks on subtype polymorphism to implement parametric polymorphism. This means that only classes can be type parameters. We can’t have a Pair<int, boolean>, for example. But we can have Pair<Integer, Boolean>.

Reflection

The book sneaks in a discussion of reflection, though it doesn’t really fit in with the other topics we’ve looked at today. Reflection is the ability for code to ask questions about itself. One can ask a method with its parameters are, what exceptions it throws, and how visible it is. One can ask a class what its instance variables and methods are. Reflection allows us to do some interesting things, like run unit test runner. Here we ask the class for all its methods that start with the string "test" and automatically run them:

import java.lang.reflect.Method;

public class Tester {
  public static void testA() {
    System.out.println("a");
  }

  public static void testB() {
    System.out.println("b");
  }

  public static void testC() {
    System.out.println("c");
  }

  public static void helper() {
    System.out.println("helper");
  }

  public static void main(String[] args) {
    for (Method method : Tester.class.getMethods()) {
      try {
        if (method.getName().startsWith("test")) {
          method.invoke(null);
        }
      } catch (Exception e) {
        e.printStackTrace();
      }
    }
  }
}

Conclusion

See you next time!

P.S. It’s time for a haiku!

I thought I was free
But I’m the same as I see
I’m more we than me

Today we take a foray into a technical topic: the mechanics of calling functions. We might think this is information that is pretty well hidden from us when we’re writing in a high-level language. But abstractions leak. The way that functions are called bubbles up into the design of our languages. Having some awareness of what’s happening under the hood will help you write safe and faster code and will help you empathize with the designers of a language.

Where We’ve Been

But first, let’s just situate ourselves in the flow of this course. At the very beginning of the semester we discussed syntax, learning how a grammar influences what our programs look like. Then we moved on to discussing the meaning of things we utter in our programs, things like variables, types, operations, control structures, and parameters. These topics form the semantics of a programming language.

Now we move on to implementation details. For the next several weeks, we’ll be looking at how certain features are made to work. In particular, we’ll be looking at how functions are called, how data and code is combined into classes, and how programs are made to run concurrently.

Managing Function State

Your mind is consumed by the latest programming assignment. Then someone rings your doorbell. You panic. Dirty laundry carpets the floor. Dirty dishes fill the sink and counter. Your roommate’s pizza crusts have been squirreled away in random places. Immediately you begin stashing things away and switching over your mind to accommodate this new activity of having company over. The exact same thing happens when you call a function. Let’s talk about the mad scramble that accompanies a function call using the ubiquitous C calling conventions.

The calling function will be paused while the called function executes, but the calling function’s state must be preserved so it can be restored when the called function finishes. Only one function is active at a time, so the memory can be represented as a stack structure. The current function’s local state is at the top of the stack. In many operating systems, a process is laid out like this in memory:

address	segment
high	stack
	…
	heap
	globals
low	code

Technically, the stack grows downward, with its top at the bottom. A called function will get its own state below the caller’s on the stack. That state is called a stack frame or activation record.

In x86-64, two registers are used to indicate the extent of a function’s stack frame. Register %rbp is the base pointer or frame pointer. It points at the start of the function’s memory. Your book calls it the environment pointer (EP). This pointer is fixed in place during the function’s execution. The register %rsp points at the last allocated memory cell of the stack. It will grow and shrink as the function allocates or deallocates stack memory. We change %rsp either directly with a subq $NBYTES, %rsp instruction or indirectly via pushq RVALUE or popq LVALUE instructions.

Calling Without Parameters

Let’s see what happens when a call to a 0-arity function is encountered. Consider this C code:

int rand();
int f() {
  return 3 + rand();
}

We compile this down to assembly, not machine code, to see what the machine does to implement the function call. This invocation of gcc will produce the assembly:

gcc -fno-asynchronous-unwind-tables -o assembly.s -S -O0 file.c

And here’s a portion of the assembly that results:

callq _rand
addl  $3, %eax

What can you tell about how function calls are implemented by looking at this assembly?

The return value from rand is available in %eax. This is generally how simple data is returned from functions. Sometimes the return value might be sent along as an out parameter. Other times the return value might be sent back using other registers.

One thing that you can’t tell from this code is that the callq instruction implicitly does something like this:

pushq ADDRESS-OF-NEXT-INSTRUCTION
movq ADDRESS-OF-FUNCTION, %rip

The return address is pushed onto the stack so the CPU can navigate back to the caller when rand finishes. Sometimes malicious programs will take advantage of this and try to write a different address into the stack, one that leads to code the malicious program has inserted somewhere else in memory.

Register %rip is the instruction pointer or program counter. We can’t write to it it directly in x86-64. Instead we use call or jmp.

Here’s the code in its fuller context:

# Prologue
pushq %rbp
movq  %rsp, %rbp

callq _rand
addl  $3, %eax

# Epilogue
popq %rbp
retq

The first few lines constitute the prologue, the setup code that all functions include. We push the caller’s base pointer onto the stack. Your book calls this the dynamic link. With that safely archived, we update the base pointer for f so that it points at the location of the caller’s base pointer.

We call the rand function and get its result in %eax, as we have seen.

Then we reach the epilogue, the sequence of instructions that all functions follow to close out the function call. We grab the caller’s base pointer off the stack and restore it to the register. The retq instruction grabs the caller’s return address off the stack and jumps to it. Effectively it is this instruction:

popq %rip

Calling With Parameters

We’ve seen how control is passed to a parameterless function and how its return value is managed. Let’s now consider this call to a function with parameters:

void g(int a, int b, int c);
void f() {
  g(5 + 10, 20, 30);
}

This code compiles down to this assembly:

# Prologue...

movl  $15, %edi
movl  $20, %esi
movl  $30, %edx
callq _g

# Epilogue...

There are several things to observe in this code:

• Even with no optimizations, gcc will add the 5 and 10 together. A more conservative compiler would insert an add instruction.
• The parameter expressions are evaluated eagerly in left-to-right order before the function is called.
• The parameters are passed in registers.

What happens if we add more parameters? Let’s have 8 of them total:

void g(int a, int b, int c, int d, int e, int f, int g, int h);
void f() {
  g(5 + 10, 20, 30, 4, 5, 6, 7, 8);
}

This code compiles down to this assembly:

# Prologue ...

subq  $16, %rsp
movl  $15, %edi
movl  $20, %esi
movl  $30, %edx
movl  $4, %ecx
movl  $5, %r8d
movl  $6, %r9d
movl  $7, (%rsp)
movl  $8, 8(%rsp)
callq _g
addq  $16, %rsp

# Epilogue ...

We can make some new observations:

• Only the first six parameters are passed in registers.
• The stack pointer is adjusted by 16 bytes.
• The remaining actual parameters are placed on the stack in reverse order. The first parameter is nearest to the dividing line between stack frames.

Accessing Parameters and Local Variables

We’ve seen the mechanics of setting up and tearing down a function. Let’s look now at how parameters and local variables are accessed. Consider this C function which accepts eight parameters and allocates two local variables:

int sum(int a, int b, int c, int d, int e, int f, int g, int h) {
  int x = a + h;
  int y = x + 13;
  return y;
}

The compiler might translate this function into this assembly:

# Prologue...

movl  24(%rbp), %eax
movl  16(%rbp), %r10d
movl  %edi, -4(%rbp)
movl  %esi, -8(%rbp)
movl  %edx, -12(%rbp)
movl  %ecx, -16(%rbp)
movl  %r8d, -20(%rbp)
movl  %r9d, -24(%rbp)

movl  -4(%rbp), %ecx
addl  24(%rbp), %ecx
addl  $13, %ecx

movl  %ecx, %eax

# Epilogue...

We can make the following observations about how local data is accessed:

• The base pointer acts as a foothold whereby all parameters and stack variables are accessed. Recall that %rbp itself points to the location of the caller’s archived base pointer. 8 bytes above this is the caller’s return address.
• 16 bytes above the archived base pointer is the first parameter. 24 bytes above is the second parameter. And so on.
• 4 bytes below the archived base pointer is the first local variable. 8 bytes below is the second local variable. And so on.
• The stack pointer is never adjusted, even local variables are being added to the stack. Since this function doesn’t call any other functions, it doesn’t need to worry about making sure %rsp is accurate.

Interestingly, there are a bunch of unnecessary movl instructions. Parameters b through f are never used, but yet they are spilled into local memory. This is probably done so that the registers can be used by other instructions.

Only so many registers available, and functions have to coordinate how they change them so they don’t clobber each other’s state. In x86-64, the convention is that a function is free to modify %rax, %rcx, %rdx, %r8, %r9, %r10, and %r11 because the caller has preserved its copies on the stack before the call. These are the caller-saved registers. If the function needs to use other registers, it must push the caller’s values of these registers on the stack and then restore them as the function’s finish. These other registers are callee-saved registers.

Calling Conventions in Brief

In summary, the following steps happen on a function call:

1. In the caller, evaluate the actual parameters in left-to-right order.
2. In the caller, push the actual parameters onto the stack in reverse order or drop them into designated parameter registers.
3. In the caller, push the return address on the stack.
4. In the callee, push the caller’s base pointer on the stack.
5. In the callee, allocate space for local variables by adjusting %rsp or just writing into memory below %rbp.
6. Do the work of the callee.
7. In the callee, deallocate the local variables.
8. Pop the caller’s base pointer.
9. Pop the caller’s return address and jump to it.
10. Release the memory consumed by the parameters.

Somewhere near the beginning of this process, the caller will need to spill its caller-saved registers to the stack. Somewhere near the end, the callee will need to unspill any callee-saved registers.

Static Scoping

All of the discussion above is focused on C and x86-64. C takes a very simple view of functions: they are all defined at the top-level. Many languages allow functions to nest. If an inner function references variables from an outer function, we must have a scheme for accessing another function’s stack frame.

When the compiler encounters a free variable, it works outward through the chain of surrounding scopes to look for the variable’s declaration. The number of scopes it takes to find the declaration is recorded as the chain offset. The variable’s offset within that function’s stack frame is recorded as the local offset. This all happens at compile time.

At runtime, every time a function is called, a static link is inserted in the called function’s stack frame. The static link points to the stack frame of the called function’s outer function—which is not generally the same as the caller. When a free variable is referenced, the chain offset is used to traverse up these static links (the static chain). Then the local offset is used to find the outer function’s variable. The code to do this traversing is all generated at compile time, so the lookups aren’t as slow as they might be.

Dynamic Scoping

Recall that if we look for free variables in the caller rather than in the surrounding scope, we have dynamic scoping. One might implement this flavor of scoping by walking up the dynamic links (the caller’s base pointer) until one encounters a stack frame that has a declaration for the referenced variable. This scheme is called deep access. Since a function’s caller is only known at runtime, the compiler can’t automatically generate code to walk up the dynamic chain. Deep access is slow.

An alternative that allows faster lookup is to maintain a stack for each active variable name in the program. When a function accesses foo, the value is pulled installed off the top of the stack for foo. No traversal of the dynamic chain is necessary. However, when the function returns, the stacks for all of its local variables must be popped.

To be perfectly honest, I don’t know why we still talk about dynamic scoping. One of its benefits is that we can effectively pass data to functions without using parameters, but even this is not a clear benefit.

Exercises

To illustrate how these scope lookups might be implemented, let’s work through a couple of exercises and draw the changing stack. We’ll assume that each activation record has the following structure:

local variables

static link

dynamic link

return address

parameters

A

01  program P() {
02
03    var x = 'p'
04  
05    function A() {
06      println(x)
07    }
08  
09    function B() {
10      var x = 'b'
11      function C() {
12        var x = 'c'
13        println(x)
14        D()
15      }
16      function D() {
17        println(x)
18        A()
19      }
20      C()
21    }
22    B()
23  }

B

01   function main() {
02     var x
03     function bigsub() {
04       var a, b, c
05       function sub1() {
06         var a, d
07         a = b + c             // 1
08       }
09       function sub2(x) {
10         var b, e
11         function sub3() {
12           var c, e
13           sub1()
14           e = b + a           // 2
15         }
16         sub3()
17         a = d + e             // 3
18       }
19       sub2(7)
20     }
21     bigsub()
22   }

See you next time!

P.S. It’s time for a haiku.

My function just broke
I took it back to the store
No returns, they said

Programs are recursive. They are built out of smaller chunks of code that are themselves programs. We call these smaller chunks of code subprograms. These subprograms organize our code into coherent pieces, with each piece solving a small computational task and deferring to other pieces as needed. We expect that we’ll write better code if we use subprograms, because our mind will be focused on the subprogram’s task as we write it and the subprogram can be tested in isolation. Today we discuss how programming language designers have incorporated subprograms into our modern languages.

Warm-up Questions

Let’s start our discussion with a few high-level questions. You can be a perfectly competent programmer without really knowing the answers to simple questions like these.

What’s a procedure? A function? A method?
A procedure is a function that returns no value. Rather, it achieves some side effect, like printing, allocating memory, or performing I/O. Functions return a value, and in some languages, they may produce side effects. A method is a subprogram defined in the context of an class.

How does a subprogram begin executing? How does it stop executing?
A subprogram is called by some other subprogram or perhaps even by itself. It stops executing when a return statement is encountered or when the subprogram body is fully executed.

Where does execution start within a subprogram? Where does it end?
In most of our modern language, execution begins at the first statement of the subprogram’s body. Fortran had a way of establishing multiple entry points, like this:

program main
  call foo()
  call foo_without_setup()
end program main

subroutine foo()
  call setup()
entry foo_without_setup()
  call process()
end subroutine foo

But Fortran 2008 threw it out. Many of our languages allow multiple return statements, though some programmers think there should only be one exit point. If the return statement is within a try/catch statement, the exiting will happen after any finally statement.

When subprogram f is called, how does one know what subprogram definition to invoke?
This is complicated. The subprogram’s name is one distinguishing factor. Some languages, like Ruby and JavaScript, consider only the name. In other languages, the number, types, and ordering of parameters are also considered. Your book calls this the parameter profile. Usually, only the name and parameter profile are used to uniquely identify a subprogram and resolve overloading ambiguities. Interestingly, return values are often not included in this signature. See Java’s Class.getMethod documentation for an example.

The types of the actual parameters, which are the expressions sent to the subprogram in the call, are compared to the types of the formal parameters, which are the variables declared as holes that the caller must fill. Only subprograms whose types are compatible are eligible to match a call. When we have multiple subprograms with the same name but which can serve different types, we have ad hoc polymorphism. The subprogram has many shapes.

When the subprogram is a method, the set of eligible subprograms is constrained by the host object. If the host type is known at compile time, one can figure out the exact method. This is called static dispatch. If the host’s type is not quite known, then we can’t figure out what method is run until runtime using dynamic dispatch. For example, Java’s PrintStream.println for objects might be implemented like this:

public void println(Object o) {
  println(o.toString());
}

We have no idea what toString method is going to be invoked. We say that println supports subtype polymorphism, as any object that is a subclass of Object can be passed in.

Parameters

The fallout of giving things identity is that we also increase the separation between things. So it is with subprograms. We slap a meaningful name on an algorithm, and then it becomes its own entity with its own names and its own memory. Our vehicle for bridging that gulf of separation is parameters and return values.

We express the identity of a function in its header, which is generally the top line or lines of the function definition. The header establishes the protocol, the interface of parameter and return types between the subprogram and the caller. In C and C++, the header is sometimes declared separately in a prototype. These prototypes are collected up in a header file. The definitions themselves are compiled down to machine code and distributed. So that other code that calls into the compiled code can be type checked, the header file is consulted since there’s nothing in the compiled code that communicates the interface.

Semantic Models

In theory, there are really only three types of transactions that can occur between a caller and a subprogram:

• The caller can send along read-only data to the subprogram. These are called in parameters.
• The subprogram can send data that it generated back to the caller. These are called out parameters. Return values are a form of out parameters.
• The caller can send along modifiable data to the subprogram, and the subprogram’s modifications will be seen when the caller resumes control. These are called inout parameters.

Some languages like GLSL (OpenGL Shading Language) have syntax that allows the semantic intent of a parameter to be expressed directly using the modifiers in, out, or inout. In GLSL, we could write this function to compute both the quotient and remainder in an integer division:

void divmod(in int dividend, in int divisor, out int quotient, out int remainder) {
  quotient = dividend / divisor;
  remainder = dividend % divisor;
} 

Out parameters can be used to return multiple values. They have no incoming value. Inout parameters are meant to be both read from and written to, as in this function:

void complement(inout float x) {
  x = 1.0 - x;
}

Implementation Models

GLSL is relatively unique in that it supports the theoretical semantic models verbatim. Most of our languages take different approaches to achieve the same semantics. Let’s examine these approaches.

Pass-by-value
The subprogram gets its own copy of the actual parameter values. Because any changes to the parameters will be applied to the copied value and not the caller’s values, pass-by-value implements in parameter semantics. C, Java, JavaScript, and Ruby are all pass-by-value.

Pass-by-value is a simple scheme to implement and understand. It pervades modern languages. The copying of values can incur extra runtime costs if structs and arrays themselves are cloned. In Java, Ruby, and JavaScript, we have implicit pointers to any non-primitive, and only the pointers are copied, which tend to be 4- or 8-bytes.

Pass-by-result
The subprogram handles an out parameter by storing the value to be returned in a local variable and copying the value over to the caller’s memory space when the subprogram finishes.

Pass-by-copy or pass-by-value-result
The subprogram handles an inout parameter by copying the passed value to a local variable, which is referenced in the body of the subprogram, and copying the value back into the caller’s memory space with the subprogram finishes.

Pass-by-reference
The caller shares its actual parameters with the subprograms. Assignments to the formal parameters will effect changes in the caller’s memory. C++ is one of the only modern languages that truly supports this passing scheme. Some folks will try to tell you Java or Ruby are pass-by-reference, perhaps claiming that a method can change an object that’s sent as a parameter, like this:

public void budge(ArrayList<String> names) {
  names.add(0, "Chris");
}

They are right that the method is changing memory beyond itself. But the parameter is not being changed. The object the parameter refers to is what’s changing. If you tried to change the parameter, that change would be local:

public void budge(ArrayList<String> names) {
  names = new ArrayList<String>();
  names.add("Chris");
  names.add("Chris");
  names.add("Chris");
  names.add("Chris");
}

In C++, we can see true pass-by-reference at work in the canonical swap routine:

void swap(int &a, int &b) {
  int tmp = a;
  a = b;
  b = tmp;
}

Those assignments to a and b modify the caller’s actual parameters. That makes pass-by-reference an implementation of inout parameter semantics.

Most of this argument is due to terminology, as reference is an overloaded term. We need better distinction between C++ references and the implicit pointers of Java, Ruby, and JavaScript that we call references.

Pass-by-name
The caller shares an unevaluated form of its actual parameters with the subprogram. Each time the subprogram refers to the parameter, the unevaluated form is evaluated. Not many modern languages support this scheme; Scala is a notable exception. One benefit of pass-by-name semantics is that they allow the user to define their own control structures as functions, perhaps like this:

function repeatAround(n, outerBlock, innerBlock)
  for i to n - 1
    outerBlock()
    innerBlock()
  outerBlock()

// prints xoxoxox
repeatAround(4, print "x", print "o")

We want the second and third parameters to get evaluated each time they are called in the repeatAround function, not just once.

In most of our languages, the parameters are evaluated eagerly—before the subprogram takes over. With pass-by-name, we delay their evaluation. We can simulate delayed evaluation by manually turning the second and third parameters into lambdas. Here’s how we might write repeatAround in JavaScript:

function repeatAround(n, outerBlock, innerBlock)
  for (let i = 0; i < n - 1; i += 1) {
    outerBlock();
    innerBlock();
  }
  outerBlock();
}

// prints xoxoxox
repeatAround(4, () => console.log("x"), () => console.log("o"))

Exercise

Let’s examine these various parameter passing schemes with a few exercises. Imagine we have this code that supports four of the schemes mentioned above:

foo(a, b, c, d):  # 0
  a = a + 1  # 1; a is passed by value
  b = b + 1  # 2; b is passed by copy
  c = c + 1  # 3; c is passed by reference
  d = d + 1  # 4; d is passed by name

x = [1, 2, 3, 4]
y = 2
foo(x[0], x[1], y, x[y])  # 5

What is the state of memory after each of the numbered lines? Here’s my answer:

0: x=[1, 2, 3, 4]  y=2  a=1  b=2  c=&y  d=x[y]
1: x=[1, 2, 3, 4]  y=2  a=2  b=2  c=&y  d=x[y]
2: x=[1, 2, 3, 4]  y=2  a=2  b=3  c=&y  d=x[y]
3: x=[1, 2, 3, 4]  y=3  a=2  b=3  c=&y  d=x[y]
4: x=[1, 2, 3, 5]  y=3  a=2  b=3  c=&y  d=x[y]
5: x=[1, 3, 3, 5]  y=3  a=2  b=3  c=&y  d=x[y]

Changes to c automatically propagate to y since c is a reference. Changes to b propagate to x[1] once the function finishes.

Positional vs. Keyword

Many of you grew up on Java, and you therefore have an irrational fondness for it. I similarly have an irrational fondness for toast because it got me through childhood. When you see a function call in Java, you automatically understand that the first actual parameter drops into the function under the name given by the first formal parameter. The second actual parameters drops in under the name given by the second formal parameter. And so on. When actuals are mapped to formals simply by their order of appearance, we have positional parameters. But that’s not the only way.

An alternative is a scheme of keyword parameters in which each actual parameter is labeled with the formal parameter’s name. Consider this Python function for calculating the cost of a taxable good:

def tax(price, rate)
  return price * (1 + rate)

With positional parameters, we must retain the incidental knowledge of the parameter order. With keyword parameters, the ordering becomes unimportant, and the name imbues semantic meaning on our parameter expressions. Compare these three calls:

tax(10, 0.07)
tax(price = 10, rate = 0.07)
tax(rate = 0.07, price = 10)

Which do you like better? Imagine the function had 11 parameters. Does your answer change? Keyword parameters enhance readability, provided the API designers picked meaningful names. To satisfy the maximum number of opinionated programmers, some languages allow a mix of both positional and keyword parameters even within the same function call.

Python, Ruby, C#, Swift, and PHP all support named parameters. Several of these languages expect : instead of =, presumably because assignment is a legal expression that may appear as an actual parameter.

Default Parameters

Occasionally we want to leave a hole in an algorithm, but we expect it usually to be filled by a particular value. We can assign a default value to the parameter. For example, if the tax rate is usually 7.5%, then we’d define tax like so:

def tax(price, rate = 0.075)
  return price * (1 + rate)

We can then call it in abbreviated form:

tax(price = 10)

Python, C++, C#, Ruby, and JavaScript support default parameters, along with many others.

Variadic Functions

Suppose you are organizing a raffle. You might write the following code:

class Raffle {
  private ArrayList<String> entrants = new ArrayList<>();

  public void add(String name) {
    entrants.add(name);
  }

  public static void main(String[] args) {
    Raffle raffle = new Raffle();
    raffle.add("Nuna Toodle");
    raffle.add("Gary Fodmother");
    raffle.add("Molly Coddle");
  }
}

Making those three separate add calls is not a large burden, but it’d be convenient to have a way to add names en masse. We could write a helper function that takes in an array of names and then pass in a gangly array literal:

class Raffle {
  private ArrayList<String> entrants = new ArrayList<>();

  public void add(String name) {
    entrants.add(name);
  }

  public void add(String[] name) {
    for (String name : names) {
      entrants.add(name);
    }
  }

  public static void main(String[] args) {
    Raffle raffle = new Raffle();
    raffle.add(new String[] {"Nuna Toodle", "Gary Fodmother", "Molly Coddle"});
  }
}

But there’s something better. Java, along with JavaScript, C, C++, Ruby, PHP, and many other languages allow us to define functions that can take in an arbitrary number of parameters. Functions whose arity is not fixed are called variadic functions. Generally, the actual parameters are bundled up in a collection like an array that can be iterated through inside the function. Here’s Java’s take on variadic functions, which includes annotating the parameter type with ...:

class Raffle {
  private ArrayList<String> entrants = new ArrayList<>();

  public void add(String... names) {
    for (String name: names) {
      entrants.add(name);
    }
  }

  public static void main(String[] args) {
    Raffle raffle = new Raffle();
    raffle.add("Nuna Toodle", "Gary Fodmother", "Molly Coddle");
  }
}

Inside add, the parameter names is just an array. The variadic definition is no different than the explicit array definition above. However, we can call add without bundling up the array literal ourselves.

For those trivia nights, here are a few more terms to class functions with particular arities. Functions that take 3 parameters triadic, 2 parameters dyadic, 1 parameter monadic, and 0 parameters niladic.

Nesting

As you write C, C++, and Java, you always write functions and methods at the top level of their file or class. Sometimes it’s convenient to define a function inside the context of another function, especially if its just a helper function that doesn’t need to have a broad scope. For example:

function rollTwoDice() {
  function rollDie() {
    return Math.floor(Math.random() * 6) + 1;
  }
  return rollDie() + rollDie();
}

Nothing outside of rollTwoDice can see rollDie.

Closures

Interesting things can happen when we define a function that references data that’s in its surrounding scope. Consider this JavaScript code and note how the nested enclose accesses start and end, which are parameters to the outer function:

function encloser(start, end) {
  function enclose(content) {
    return start + content + end;
  }
  return enclose;
}

const parenthesize = encloser('(', ')');
const embrace = encloser('{', '}');
const bracket = encloser('[', ']');

What are the types of parenthesize, embrace, and bracket? They are functions. Will these three functions do what we think they should? Yes. JavaScript forms what’s called a closure when we define a function. The closure is a marriage between the function and its scope. Even if the defining context goes out of scope, as it does here, the closure hangs onto the values of start and end so that the concatenation can use the right values.

Closures are an incredibly convenient tool that I simply cannot fully motivate in our brief time together. They allow you to tailor the behavior of a function without burdening the caller to add a bunch of extra parameters.

Generic Subprograms

Some languages allow a parameter’s type to also be a parameter. Type parameters can be attached to either a subprogram or a class structure. We call constructs with a type parameter in C++ templates and in Java generics. Using generics, we can add tuples to Java:

class Tuple<T, U> {
  private T first;
  private U second;

  public Tuple(T first, U second) {
    this.first = first;
    this.second = second;
  }
}

The types T and U are not fixed. They will be passed in at the time a tuple is instantiated.

Coroutines

Some subprograms can pause themselves and cede control to some companion subprogram. Such subprograms all called coroutines. We use them when we have two or more cooperating threads of execution, but we don’t necessarily need them to execute simultaneously. Coroutines are generally cleaner and use fewer resources than real threads.

Kotlin supports several forms of coroutines. Their Sequence type can be used to implement iterators for complex or infinite data structures. For example, this sequence can be used to iterate through a list in a round-robin fashion forever:

fun roundRobin(vararg items: String): Sequence<String> {
  return sequence {
    var i = 0
    while (true) {
      yield(items[i])
      i = (i + 1) % items.size
    }
  }
}

You can iterator through the sequence as many times as you like. If you reach the end, the sequence wraps back to the beginning. This code runs 20 iterations through a stoplight:

val spinner = roundRobin("Green", "Yellow", "Red").iterator()
repeat(20) {
  println(spinner.next())
}

You could achieve a similar effect in Java by writing a subclass of Iterator, but you’d have to maintain the iterator’s state in instance variables, which is sometimes unnatural. A coroutine lets you pause the subprogram in the middle of a loop and return back to it later.

Conclusion

There’s a whole lot going on with subprograms and parameters. You likely don’t know what you are missing when you use only your favorite language, and you may be totally unaware that programmers who use other languages have it better than you in some ways. Just as you should travel the world to see how other social systems work, you should visit these other languages to learn how other programmers think and operate. Maybe you can steal their good ideas.

See you next time!

P.S. It’s time for a haiku!

Pass-by reference
That’s when you quote a movie
And Grandma just blinks

The programs we write do many things: they facilitate communication, they walk a user through an interaction, they automate drudgery, they control physical artifacts, they store and retrieve data, and so on. Whatever a program’s final goal, many of its intermediate steps are concerned with producing and consuming information. This information is combined with other information using mathematics and logic. Code that yields information that be can consumed by other code is called an expression. We evaluate an expression to produce a value.

As the program goes about the task of producing and consuming values, it may reach a point where it should only run certain code under a certain condition. Or it needs to go back and re-execute code it just executed. This dynamic navigation of the code is called control flow, and the syntactic features of the language to guide control flow are called control structures.

Today we’ll have a closer look at the design choices and mechanics of expressions and control structures.

Anatomy of Expressions

Syntactically, an expression is made up of operators, operands, and parentheses.

Operators

An operator can be classified along several dimensions. It may be a unary operator taking one operand, a binary operator taking two operands, a ternary operator taking three, and so on. Interestingly, most languages have only a single ternary operator: the conditional operator. We call it “the ternary operator,” even though this communicates nothing about its semantic purpose.

When an operator precedes its operands, we say it is a prefix operator. What prefix operators have you used? Perhaps -, ~, *, &, ++, --, sizeof, and new. The last two of these are not function calls. Languages with prefix operators may allow traditional binary operators to accept more operators, as in Racket’s (+ 1 2 3 4). A postfix operator trails its operands. What postfix operators have you used? Perhaps ++ and --. An infix operator appears between its operands.

The grammar of the language decides the precedence and associativity of the operators.

Most languages use very few symbols for operators, relying on convention to communicate an operator’s semantic meaning. A binary * means multiplication, and + means addition. Many of of these operators are overloaded, with separate versions of multiplication for integers and floating point numbers. In Python 3, the language designers decided to remove overloading for /. Single-slash division yields a floating point result. For example, 5 / 6 is 0.833. Double-slash division yields a whole number result. For example, 5 // 6 yields 0.

Overloading is nice when the different underlying computations have roughly the same semantic meaning. But what does << mean? That may be Ruby’s push method for arrays or C++’s insertion operator for output to an ofstream or a left shift.

Some languages allow the builtin operators to be overloaded in user-defined ways. We write functions named after the overloaded operation for the types we want to have support the operation. One of the Bjarne Stroustrup’s goals in C++ was to make user types be indistinguishable from builtin types. He called this type parity, and operator overloading was crucial to achieving his goal. In Java, we don’t have type parity. Strings get +, but our classes don’t. Instead, we give our types an add method.

Ruby also has operator overloading. We can give our strings a / operator:

class String
  def /(n)
    chars.each_slice(n).map { |slice| slice.join }
  end
end

alphabet = "abcdefghijklmnopqrstuvwxyz"
puts alphabet / 5

I’ve got a language I’ve been working on for making 2D shapes. It’s called Twoville. One day I was writing a program and I wanted to treat an integer like a bitfield, but I didn’t have a shifting operator or bitwise and. Then I thought it’d be a whole let cleaner if I just gave integers a subscript operator to get out bit i:

  

    a = 9
print(a[0])
print(a[1])
print(a[2])
print(a[3])

    
    
    
  
  
    

    a = 9
print(a[0])
print(a[1])
print(a[2])
print(a[3])

  
  expand

This operator made it easier to program up this diagram of Venn diagrams:

  

    // Parameters
width = 150
height = 100
trueColor = [1, 0.9, 0.4]
falseColor = [0.55, ~, ~]
separationFactor = 0.15
padding = [4, 20]
radiusProportion = 0.4

// Derived
radius = height * radiusProportion
separation = separationFactor * width
intersectionY = sqrt(radius * radius - separation * separation)
intersectionAngle = asin(intersectionY / radius) * 2
cellDimensions = [width, height] + padding

with view
  corner = [0, 0]
  size = [width * 4 + padding.x * 3, (height + padding.y) * 4]

to venn(aMinusB, bMinusA, both, neither, column, row)
  origin = cellDimensions * [column, row] + [0, padding.y]
  centroid = origin + [width, height] * 0.5
  left = centroid - [separation, 0]
  right = centroid + [separation, 0]
  
  // neither
  rectangle
    corner = origin
    size = [width, height]
    color = if neither then trueColor else falseColor
  
  // aMinusB
  circle
    center = left
    radius = radius
    color = if aMinusB then trueColor else falseColor
    with stroke
      size = 1
      color = :black
  
  // bMinusA
  circle
    center = right
    radius = radius
    color = if bMinusA then trueColor else falseColor
    stroke
      size = 1
      color = :black
  
  // both
  path
    stroke
      size = 1
      color = :black
    color = if both then trueColor else falseColor
    go
      position = centroid - [0, intersectionY]
    arc
      center = left
      degrees = intersectionAngle
    arc
      center = right
      degrees = intersectionAngle

for i to 16
  venn(i[0], i[1], i[2], i[3], i % 4, i / 4)

    
    
    
  
  
    

    // Parameters
width = 150
height = 100
trueColor = [1, 0.9, 0.4]
falseColor = [0.55, ~, ~]
separationFactor = 0.15
padding = [4, 20]
radiusProportion = 0.4

// Derived
radius = height * radiusProportion
separation = separationFactor * width
intersectionY = sqrt(radius * radius - separation * separation)
intersectionAngle = asin(intersectionY / radius) * 2
cellDimensions = [width, height] + padding

with view
  corner = [0, 0]
  size = [width * 4 + padding.x * 3, (height + padding.y) * 4]

to venn(aMinusB, bMinusA, both, neither, column, row)
  origin = cellDimensions * [column, row] + [0, padding.y]
  centroid = origin + [width, height] * 0.5
  left = centroid - [separation, 0]
  right = centroid + [separation, 0]
  
  // neither
  rectangle
    corner = origin
    size = [width, height]
    color = if neither then trueColor else falseColor
  
  // aMinusB
  circle
    center = left
    radius = radius
    color = if aMinusB then trueColor else falseColor
    with stroke
      size = 1
      color = :black
  
  // bMinusA
  circle
    center = right
    radius = radius
    color = if bMinusA then trueColor else falseColor
    stroke
      size = 1
      color = :black
  
  // both
  path
    stroke
      size = 1
      color = :black
    color = if both then trueColor else falseColor
    go
      position = centroid - [0, intersectionY]
    arc
      center = left
      degrees = intersectionAngle
    arc
      center = right
      degrees = intersectionAngle

for i to 16
  venn(i[0], i[1], i[2], i[3], i % 4, i / 4)

  
  expand

Time for a riddle. What’s the output of this Java program?

public class Riddle {
  public static void main(String[] args) {
    getTrue() || getFalse();
  }

  public static boolean getTrue() {
    System.out.print("T");
    return true;
  }

  public static boolean getFalse() {
    System.out.print("F");
    return false;
  }
}

The || is satisfied by its left operand alone. The side effect of the right operand is never seen, which means the right operand is never even considered. We say that && and || are short-circuit operators. Do you know any others? The ternary conditional operator ?: is also a short-circuit operator. It only evaluates one of its second and third operands. Our modern languages are also gaining a couple of other handy short-circuit operators. One is the optional chaining operator. In languages with pointers or nullable references, we have to be careful not to invoke a method or access a property on a null object. We write an if statement to guard against the null pointer exception, like this:

if (pet !== null) {
  console.log(pet.name);
}

The optional chaining operator lets us write this shorter form:

console.log(pet?.name);

Ruby, C#, JavaScript are some of the languages that support this operator.

If pet is null, the expression evaluates to undefined instead of raising an exception. If we want to fall back on a default value, we can use the null coalescing operator:

console.log(pet?.name ?? 'NPE: no pet exception');

This operator checks the left operand. If it’s not null, it evaluates to that and is done. If it is null, it evaluates to the right operand. The expression x ?? y is shorthand for x ? x : y. Some languages use ?: instead of ?? and call this operator the Elvis operator.

Operands

Speaking of operands, let’s talk more about them. When you see an expression of the form f() + g(), how do you expect it will be evaluated? Probably f will be called first, right? Maybe. The C and C++ standards do not actually specify the order in which operands are evaluated. In a left-to-right scheme, we’d evaluate f and then g. In a right-to-left scheme, we’d evaluate g and then f. There may be still other schemes.

Sometimes operands do not have the same type, as in double x = 1. There’s a double on the left and an int on the right. What should happen here? In some languages, the compiler introduces an implicit cast to turn the twos-complement 1 into the equivalent floating-point 1. What if our assignment was int x = 1.0? In some languages, this requires an explicit cast. Java is one of the languages that behaves like this. It considers numeric types in the following ladder:

double
float
long
int
short
byte

When you move from a lower type to a higher type, this is called a widening conversion. Java does not require casts for widening conversions. When you move from a higher type to a lower type, you have a narrowing conversion. Java does require casts for narrowing conversions. Other languages are more picky. In Kotlin, this code does not compile:

val f : Float = 6

One must use an explicit cast when converting between types. Why does Kotlin not adopt the same practice as Java? Aren’t widening conversions safe? No. There’s still a potential for information loss even when going to a “bigger” type. Consider this Java code:

float f = 16777217;
System.out.printf("%8f%n", f);

What will the output be? It’s 16777216. In single-precision floating point numbers, we have 1 bit for the sign, 23 for the the fraction after the floating point, and 8 for the exponent, with the value computed as sign * 2 ^ exponent * (1 + fraction). The numbers 0 through 16777215 can be faithfully represented in the 23 bits of the fraction. The number 16777216 = 2^24 can be represented by setting the exponent to 24. Past that, not every whole number can be represented. Once you have to start involving the exponent, the numbers are only going to be even.

Parentheses

The last anatomical structure of expressions is parentheses. We use these to change or emphasize the predecence and association of operations. But they are overloaded. We also use them to group parameter lists, to create tuples in Haskell and Python, and as subscript operators in MATLAB. Lisp uses parentheses to surround entire expressions.

Expressions vs. Statements

Not every structure in a program yields a value. We also have statements, which produce side effects. Common side effects include output and changes to variables. Calls to void functions are statements. Our forebears sometimes preferred to call void functions procedures rather than functions. The mathematical notion of a function implies a mapping from a domain to a range, and void functions do no such mapping.

Some languages do not have a notion of statements. Haskell, for example, disallows side effects. The code is made up entirely of definitions and expressions. In general, removing side effects makes our programs safer, faster, and easier to reason about. When a function has no dependencies on external mutable state, then its operations will be predictable. Such a function has what we call referential transparency. Generally, a function is referentially transparent if multiple calls to it can be replaced with a single call whose value is stored in a variable, as in this code:

// Multiple calls
print f(3) + f(3)

// Cached call
value = f(3)
print value + value

If a function is referentially transparent, we can optimize how it compiles, cache its results, and not be concerned with multiple cores needing to synchronize their mutable data.

Many languages support a hybrid of statements and expressions. The assignment statement, for example, has a side effect of updating memory, and it evaluates to the assigned value. A function may both return a value and produce output. A call to generate a pseudorandom random number that uses a linear congruence algorithm will evaluate to a random value but also store its value to help generate the next number.

Assignment Statements

Programming language designers generally don’t seem to agree on assignment statements. The crux of the dispute is that = of mathematics is not assignment. Rather, it’s an observation that two values are the same. The idea of using = to store values in memory and update them later is heresy to some. In ALGOL and Pascal, they chose to use :=. In R, they use id <- expr or expr -> id or id = expr.

When we update a variable by applying an operation to it, we can use the compound assignment operators: ++, --, +=, and so on. Several languages do not support the increment and decrement operators. I don’t blame them. These operators generate confusion. What’s the output of this code?

int a = 3;
int b = 3;
System.out.println(a++);
System.out.println(++b);
System.out.println(a);
System.out.println(b);

If you write your code so that side-effects and expressions are cleanly separated, then there’s no practical difference between pre- and post-increment. Yet because the designers of B, the predecessor to C, wanted to write really short code, we are stuck with this confusion until enough languages abandon them.

Some languages allow several variables to be assigned at once. For example, we can swap two variables with this multiple assignment in Ruby:

a, b = b, a

Evaluation Algorithm

The parser turns the flat source code of our program into a tree whose structure reflects the operator precedence and associativity. This tree can be evaluated directly through a traversal algorithm, or it can be turned into equivalent machine code. Let’s work through an example of constructing a tree and evaluating it using the mechanics we described above.

Suppose k is 5, n is 2, and a is {1,3,6,8,11,14,16}. What is a[k--] + 8 * n - k?

To build up the tree, we can work from the least predecent operator at the top. In this case, the operators following this precedence order:

+ -
*
--
[]

So, we start with the arithmetic operators. Since these are left-associative, we find the rightmost instance and put it at the top of our tree. That’s the -. Its left child will be the next weakest operation +. The addition operation’s left child will be the [] operation and its right child will be the * operation.

Control Structures

We pivot now to a discussion of control structures. In the early days of computing, our programs were a flat sequence of machine instructions. To execute out of sequence, we wrote the address of the out-of-sequence instruction into the program counter. Usually we did this with a jump/branch instruction. When high-level languages started to appear, language designers wondered if they should expose the low-level jump instructions in their “evolved” syntaxes. Some said yes, some no. In C, we still have the goto instruction. We can add labels to our code and execute instructions out of order, like this:

int main(int argc, char **argv) {
  int value = atoi(argv[1]);
  goto sub;

post:
  printf("value: %d\n", value);
  return 0;

sub:
  value += 1;
  goto post;
}

C’s goto only jumps to labels within the same function. But it also supports non-local jumps using setjmp and longjmp. In the 1960s, before C was developed, computer scientists advocated for structured programming. The execution flow should be predictable, they said. Edsgar Dijkstra was an advocate for structured programming, and he wrote down his position in his letter Go To Statement Considered Harmful, published in the Communications of the ACM. The structured programming side ultimately won out, but I doubt this made Dijkstra happy. High-level language designers invented syntaxes that papered over the underlying jumping mechanics with structures that matched the semantic intent: conditional statements, loops, and subroutines (which we’ll talk about another day).

These days most of us know little about low-level branching. We use conditional statements and loops exclusively to navigate the CPU through our source code. However, each language has a slightly different take on what these structures look like. For conditional structures, we have these choices:

• How many choices do we permit? Just 2, as in an if-else statement? Or do we allow more choices?
• Is the structure an expression or a statement?
• Is the condition a boolean expression? Or is it a value whose possible values we want to partition into cases?
• If we find a matching case, do we bail from the decision structure? Or do we fall through to the next case?

For iteration structures, we have these choices:

• Is the termination logic implicit or explicit?
• Is there an iterator? If so, what is its scope? Can it be modified?
• Is the termination logic tested before or after the loop body? Or somewhere in between?

A language doesn’t need loops. Pure functional languages don’t have loops, as loops imply the notion of mutable state. These languages use recursion for iteration.

Conclusion

Our programming experience is shaped by the choices that designers have made around expressions and control structures. When these tools fit how we think, we feel productive and smart. On other days, we curse the tools or our brains.

See you next time!

P.S. It’s time for a haiku!

What was wrong with ×
Now this frilly asterisk
A sign of the times

Today is a lab day. Find the lab specification in How to 3D.

Here’s your TODO list:

• Complete the checkpoints for this lab before next Thursday.
• Follow a Blender tutorial and model something. Upload your model to SketchFab, and share a link in the #friddday channel before the end of Friday.
• Read chapter 6 on input systems.
• Complete quiz 5 on Canvas before Tuesday at noon.
• Keep working on the projects.

See you next time!

Today we’ll write some code together to shade a collection of shiny annuluses.

Here’s your TODO list:

• Complete the checkpoints for the current lab before Thursday.
• Follow a Blender tutorial and model something. Upload your model to SketchFab, and share a link in the #friddday channel before the end of Friday.
• Submit the Boxels homework by the end of Friday, March 4.

See you next time!

Today is a lab day. Find the lab specification in How to 3D.

Here’s your TODO list:

• Complete the checkpoints for this lab before next Thursday.
• Follow a Blender tutorial and model something. Upload your model to SketchFab, and share a link in the #friddday channel before the end of Friday.
• Read chapter 5 on lighting.
• Complete quiz 4 on Canvas before Tuesday at noon.
• Keep working on the projects.

See you next time!

Today we’ll write some code together using projection transformations.

Here’s your TODO list:

• Complete the checkpoints for the current lab before Thursday.
• Follow a Blender tutorial and model something. Upload your model to SketchFab, and share a link in the #friddday channel before the end of Friday.
• Submit the Boxels homework by the end of Friday, March 4.

See you next time!

You know how in English nouns and verbs are independent? You can pick any noun and mix it with any nearly any verb to form a grammatical sentence. Butter dances. Fish sneeze. We have a similar situation in programming languages. The grammars of our languages allow us to mix data and operations in ways that don’t make sense. "abcdefghijklmnopqrtuvwxyz" / 7. 62.length. We generally can’t change the grammars to prevent the expression of this nonsense. Instead, we add a notion of types to our data and our operations. And then our compiler or interpreter performs type checking.

Imagine you didn’t have taste buds. Could anything go wrong? Taste buds are generally designed to help us eat good things and avoid bad things, much like a type system is designed to make sure operations are digesting good operands and not incompatible ones.

Today we’ll discuss types and how they affect programming languages and the humans who write code in them.

Type

A type describes a set of legal values and the set of operations that the data supports. In some instances, the set of values might be fully and explicitly enumerated, as in an enum. In other cases, the set of values is implicit and determined by rules made about the interpretation of memory, as with double or int. The set of operations may be builtin, as with int and double division. In other cases, you as a developer may be the one deciding what operations are supported. This is especially true in object-oriented programming. When you create a class, you are creating a new type. When you add a new method to that class, you are giving that type a new operation.

In explicitly typed languages, types help document our code. Human readers get hints about function’s purpose by looking at the types of its return value and parameters. Folks have proposed adding implicit typing to Java, which makes types less visible to the programmer. Those proposals were rejected for years. Consider Gilad Bracha’s response to one of them:

Humans benefit from the redundancy of the type declaration in two ways. Humans benefit from the redundancy of the type declaration in two ways. First, the redundant type serves as valuable documentation—readers do not have to search for the declaration of getMap() to find out what type it returns. Second, the redundancy allows the programmer to declare the intended type, and thereby benefit from a cross check performed by the compiler.

In Java 10, implicit typing was added. Implicit typing is handy when you don’t really care about the type, as in this code:

for (var element : items) {
  System.out.println(element);
}

Keeping types explicit is not a bad idea, however.

Types are also useful in our development environments. If we are using autocomplete to fill in a parameter in a function call, the IDE can filter the list of possible completions to just those variables that are in scope and of the appropriate type.

Primitives

The simplest types we call primitives. These include the various integer types. Generally they are encoded using twos-complement, in which negative numbers are expressed as their positive form, but inverted and incremented. Some types allow negative numbers, some don’t. Java, for example, has no unsigned integer types. However, Java 8 introduced some methods in the Integer class for performing unsigned operations.

When we need numbers with fractional components, we turn to float or double. These numbers are encoded using a scientific notation called the IEEE-754 floating point standard. Floating-point numbers are decomposed into three fields: a sign, a base, and an exponent. Any finite representation of real numbers is prone to numerical drift, but floating-point numbers are tricky because the drift is not uniform. There are more representable numbers close to 0 than farther away. Some languages like SQL support simpler fixed-point numbers, where the number of digits after the decimal is predetermined.

Some languages have logical types ranging from a one-bit Boolean value to a multi-valued bitfield. Other languages assume the user can just use integers to represent true and false values.

Logical and numeric types are directly supported in the CPUs of today, which have circuitry dedicated to arithmetic, inversion, shifting, and comparison. Sometimes primitive types don’t have special hardware support. For example, enums in C do not have hardware support. They just piggyback off ints.

Though computers started as an appliance that mathematicians and physicists used to process numbers, characters got their own types in the 1950s. There were various standards for mapping the arrangement of bits to alphabetic symbols. The 7-bit ASCII standard emerged as the standard in the United States, but was superceded by Unicode, which allowed for more than just 127 characters by growing the number of bytes used to represent a symbol. Many of our systems use UTF-8, a standard that is compatible with ASCII. However, if the first bits of the byte are 110, then the first byte is combined with the second byte. If the first bits of the byte are 1110, then three bytes are combined. If the first bits are 11110, then four bytes are combined. Through this scheme, all 1.1 million Unicode characters can be represented.

Compound Types

As atoms combine to form molecules, primitive types combine to form compound types. These compound types are found in today’s mainstream languages:

• If we order values in a sequence of arbitrary length, we have an array or list.
• If we order values in a sequence of fixed length, we have a tuple.
• If we have a dynamic mapping between names and values, we have an associative array or dictionary.
• If we have a static mapping between names and values, we have a record or struct or class.
• If we have a container for values of several possible types, we have a union.

Programming languages people sometimes talk about product types and sum types. A product type is a type with multiple independent fields. The number of possible values is the product of the number of possible values of each field. A sum type is a union, which has a single shared field in which values of several types can be stored. The number of possible values is the sum of the number of possible values of each type.

Arrays and Lists

Arrays are generally contiguous in memory, with all of their elements right next to each other. Processing them in a loop tends to be fast because of caching. Accessing one element from RAM automatically brings in neighboring elements into the CPU’s cache. Lists are generally scattered across RAM, with each element storing an explicit link to the next element.

At their simplest, arrays are lines of data. But they can also be two-dimensional tables or three-dimensional volumes, and so on. Multidimensional arrays can take on several forms in memory:

• Array of arrays: each element of the outer array points to that element’s inner array. Inner arrays are not likely to be adjacent in memory, meaning that caching is less effective.
• Row major: the first row of the array is laid out contiguously in memory, followed by the second row, followed by the third, and so on. Caching is effective if the inner loop advances from column to column.
• Column major: the first column of the array is laid out contiguously in memory, followed by the second column, followed by the third, and so on. Caching is effective if the inner loop advances from row to row.

When all the inner arrays are the same size, we have a rectangular array. When the inner arrays differ in size, we have jagged arrays. Only arrays of arrays can be jagged.

With lists, the programmer maintains links to each element. To reach the fifth element, one must iterate the through five links, making subscripting a linear time operation. With arrays, subscripting is constant time. The address of an element i in a 1D array is computed as base_address + i * sizeof(element). The address of an element (c, r) in a row-major 2D array is computed as r * width + c.

Strings

The product type of characters is strings. Different design decisions have been made regarding strings. They are generally implemented as arrays of characters. Sometimes this implementation is thinly veiled, as in C. Sometimes the backing array is hidden, as in Java. Which approach is better? There’s no easy answer to this kind of question. The transparency of C strings can makes them fast, versatile, and vulnerable. The opacity of Java strings makes them slower, constrained, and safe.

Additionally, we have several choices for representing the length of a string. In C, we terminate the string with the null terminator. At least, we are supposed to. Many say this was a bad design decision. It’s easy to obliterate the null terminator and write into memory that we don’t own. When malicious users do this, they are exploiting a buffer overflow vulnerability. Null termination turns certain operations, like calculating the length, into a linear time algorithm. An alternative is store the length as an integer.

The designers of Java made a bold move and decided to make strings immutable. James Gosling explained this choice in an interview:

From a strategic point of view, they tend to more often be trouble free. And there are usually things you can do with immutables that you can’t do with mutable things, such as cache the result. If you pass a string to a file open method, or if you pass a string to a constructor for a label in a user interface, in some APIs (like in lots of the Windows APIs) you pass in an array of characters. The receiver of that object really has to copy it, because they don’t know anything about the storage lifetime of it. And they don’t know what’s happening to the object, whether it is being changed under their feet.

You end up getting almost forced to replicate the object because you don’t know whether or not you get to own it. And one of the nice things about immutable objects is that the answer is, “Yeah, of course you do.” Because the question of ownership, who has the right to change it, doesn’t exist.

One of the things that forced Strings to be immutable was security. You have a file open method. You pass a String to it. And then it’s doing all kind of authentication checks before it gets around to doing the OS call. If you manage to do something that effectively mutated the String, after the security check and before the OS call, then boom, you’re in. But Strings are immutable, so that kind of attack doesn’t work. That precise example is what really demanded that Strings be immutable.

Alas, Java library developers and their clients must continue to be paranoid about other objects being mutated outside of their control. Some of the clients’ fear could have been mitigated by a mechanism of declaring parameters as constant.

Pointers and References

We can also have values that are links to other values. These links are likely implemented using the linked value’s address. When this addressing scheme is transparent, we have pointers. A value that holds an address has a pointer type. When the addresses are hidden or the linking is implemented in some other way, we have references.

Pointer types support incrementing and decrementing to advance through memory. When we dereference a pointer, we are taking away it its address nature and reaching in for the linked value. References generally don’t have any special operations beyond their linked value’s type.

One of the legal values for pointers in C and C++ is NULL, which is just a synonym for integer 0. C++11 introduced nullptr, which has a proper pointer type.

To revisit the behavior of pointers, we’ll walk through this code together:

int x = 1;
int y[4] = {2, 3, 4, 5};
int *p = &x;
*p = 6;
p = y;
*p = 7;
p++;
*p = 9;

Pointers are a feature of languages with addressable memory. Dynamic allocation of memory is technically a separate feature. C and C++ happen to support both features, so we lump them together. There’s no dynamic allocation in the code above, but there are pointers.

When pointers combine with dynamic allocation and explicit deallocation, pointers can be dangerous. Suppose p points to a block of memory. When we free(p), we release the block of memory but do nothing to p. It still points to the block. If we dereference p, we’ll access memory that is no longer ours. It might have been reallocated to some other object, or it might not have. In this situation, we say that p is a stale pointer. We’d prefer code to reliably crash than to exhibit nondeterministic behavior.

There are at least two strategies for dealing with stale pointers. One is to use tombstones. Instead of a pointing directly at the block of memory, a pointer points at another pointer, which points at the block. When you free the pointer, it reclaims the block and nulls out the tombstone. One can achieve a similar effect without the indirection of tombstones by exclusively using this version of free in C:

void freenull(void *p) {
  free(*p);
  *p = NULL;
}

We pass in the address of the pointer so that it can be modified by the function.

The other strategy is to use locks and keys, in which each allocation generates an identifier called a lock. The lock is stored in both the allocated block and in the pointer (the key). When a pointer is dereferenced, the locks are compared. If they match, the access is allowed. If not, an exception is thrown. When a block is freed, its lock is set to some other value, ensuring that future dereferences will have mismatched locks.

There’s another danger with pointers and dynamic allocation. If p gets overwritten or goes out of scope before we’ve called free, we have a memory leak.

One strategy for preventing memory leaks is to take away the responsibility of freeing memory from the programmer. Instead, some mechanism of automatic garbage collection is used.

A predictable approach to garbage collection is reference counting, in which a counter is stored with each dynamically allocated object. Every time the object is assigned to a new reference, its counter is incremented. Every time a reference is lost, as in a reassignment or by the reference going out of scope, the counter is decremented. If the counter ever goes to 0, the memory is freed. C++ provides optional reference counting via several smart pointer types available in its standard library.

A less predictable approach to garbage collection is the mark-and-sweep algorithm, in which the executing process is occasionally paused and the memory examined. The runtime has a list of all the objects that have been allocated on the heap. All references currently on the stack are traversed to see which of these objects are pointed to, recursing on any references found in the objects. Whatever objects are found not to have references are freed. Java has several garbage collection schemes available. Mark-and-sweep is one of them.

Type Systems

Once we’ve built up this notion of classifying data into types, we build up a bunch of rules about how this data may be operated on. This set of rules is called a type system. Language designers have a number of decisions to make about the type system.

Explicit vs. Implicit

Should the types be explicit, present in a variable’s declaration? Or should they be implicit? Both systems burden and benefit the programmer in different ways. Not having to write statements like this seems like a win:

Person person = new Person();
// Implicit typing can save a Person.
// var person = new Person();

But a type system with implicit types will likely lead to situations where the programmer has a different view of the program than the compiler.

Type Checking

How should types be compared when determining what is legal? The act of verifying a program’s adherence to the rules of the type system is called type checking. Any violation is a type error.

The type checker first examines the types to see if they are equivalent. Some type systems require that two types must have the exact same named type, a scheme that we call name equivalence. Under name equivalence, strings can be assigned to strings, booleans to booleans, and URL to URL. If instead of comparing names, the checker compares the underlying shape of the type, we have structure equivalence. Consider this C code:

typedef float celsius;
typedef float fahrenheit;

celsius a = 25.7f;
fahrenheit b = a;

The types have different names. Does C allow this code? It does. We can think of it as allowing structural equivalence because both types are shaped like a float.

However, consider this C code:

typedef struct { int x; } box;
typedef struct { int x; } bin;

box c;
c.x = 5;

bin d;
d = c;

The C compiler rejects this code. For record types, C requires name equivalence.

One can persuasively argue that C only allows name equivalence, as the typedef command just provides an alias to an existing named type, not a new type. The only way to get a completely new type in C is by defining a struct.

If two types are not equivalent, should we reject the code? Sometimes mismatches are clearly wrong, such as when we assign a Random to a String variable or null to an int. Other mismatches are not errors. For example, there may be rules for translating data of one type to another, such as passing an int to a function that expects a long. The int is automatically promoted to a long, and no type error is issued. This translation is called coercion. When there’s a sanctioned path to go from one type to another, we say the first type is compatible with the other.

In other situations, one type is a subtype of another. When we have a polymorphic hierarchy, we can write a function that targets the supertype, but any of its inheritors can be handled too. The types are neither equivalent nor in need of coercion.

In still other situations, code is written in which a type is left as a parameter. A placeholder type is used instead of an actual type. We call these templates in C++ or generics in Java. It’s up to the client of the code to decide what the actual type should be. Once the type parameter is known, the type checker does its usual job of ensuring the operations match the data.

Static vs. Dynamic

If we check perform type checking at compile time, we have static type checking. If we check types at runtime, we have dynamic type checking. Static type checking leads to much better runtime performance because we don’t have to keep asking questions about the types. But compilation is a lot slower. With dynamic typing, we have to continually inspect our data and ask whether an operation is legal.

Strong vs. Weak

There are several labels we put on type systems: static vs. dynamic, explicit vs. implicit, and strong vs. weak. The last of these is ill-defined. If you say that a language is strongly or weakly typed, you have said nothing. Generally, strong typing means that the type rules are enforced, that type errors are detected.

Aren’t type errors detected in every language? JavaScript checks for type errors. So do C++, C, Java, Ruby, Python, Haskell, and so on.

But some languages allow the type checking to be subverted. In C, you can cast a pointer to type A to a pointer of type B and perform B-ish operations on something that is not of type B. Similarly, you can reach inside a union and access the value through the lens of a different type, violating the actual type. That’s weak typing.

P.S. It’s time for a haiku!

Black, woman, or young
voter_t‘s no static type
Amendments tweaked it