Make

class: title, smokescreen, shelf, bottom, no-footer
background-image: url(images/gnu.png)

# 181U Spring 2020
### Make

---
layout: true
.footer[
- Geoffrey Brown, 2020
- 181U
]

.smokescreen h1 {
    border-bottom: none;
}

.small {font-size: 80%}
  .smaller {font-size: 70%}
  .small-code.remark-slide-content.compact code {font-size:1.0rem}
  .very-small-code.remark-slide-content.compact code {font-size:0.9rem}

.line-numbers{
  /* Set "line-numbers-counter" to 0 */
  counter-reset: line-numbers-counter;
}

.line-numbers .remark-code-line::before {
  /* Increment "line-numbers-counter" by 1 */
    counter-increment: line-numbers-counter;
    content: counter(line-numbers-counter);
    text-align: right;
    width: 20px;
    border-right: 1px solid #aaa;
    display: inline-block;
    margin-right: 10px;
    padding: 0 5px;
}

</style>

---
class: compact
# Agenda
* Independent compilation
* Compilation stages
* Modules, definitions, declarations
* Make to track dependencies
  * Dependency graphs
  * makefile
---
class: compact
# Independent Compilation

* As a matter of style, source code files should rarely be more than a few pages long.
* Complete compilation of large programs (which may have millions of code lines!) is very time consuming.
* For both reasons, it is important to break all but very small programs up into multiple files and compile them separately. 
* Compiled versions of each file are saved so when one source file is changed, only it needs to be recompiled before the compiled versions of each file can be efficiently combined (“linked”) to obtain a new executable version of the program.

---
class: compact
# Compilation Stages

* **Preprocessing**: source (.c) files filtered through c preprocessor
* **Compilation**: preprocessed files are compiled to assembly (.o) files
* **Linking**: program object files and library files are combined into an executable (e.g. a.out)
  * Statically linked libraries are added to executable
  * Dynamically linked libraries are represented by "stubs" in executable

* By default, c compiler deletes all intermediate files

---
class: compact
# Modules and Exporting

* Source files are often called modules. 
* It does not make sense for the same  global variable, or function to be defined in more than one module of the same program.
* A definition that is used outside of its defining module is said to be **exported**.

---
class: compact
# External References

* If a module makes use of something defined in another module, this is called an external reference.
* Each C module *should* contain declarations of all functions, variables, and data types to which it makes external references, so the compiler has the type information it needs.
  * These are best handled through *include* (.h) files
  * In the absence of declarations, the compiler will issues warnings, but may still compile the module
* In many other statically-typed languages, such as Java, compilers will look in other modules for missing definitions, but C compilers do not.

---
class: small-code,compact,hljs-tomorrow-night-eighties,line-numbers
# Definitions vs. Declarations

* Variable definitions and declarations can often look the same in C.
* The general rules for distinguishing definitions from declarations can be confusing.
* **Definition** of a variable informs the compiler of -- the name of the variable, its type, and (optionally) its initial value
```c
int a = 33;
```
* **Declaration** of a variable (or function) specifies its type, but leaves it undefined
```c
extern int a;
```
```c
int add(int, int);
```
* If a variable is *used* in a module but *defined* in a different module, then it should be declared before use -- this can be from a definition in the module where it is used or, better, through an included header file.

---
class: small-code,compact,hljs-tomorrow-night-eighties,line-numbers,col-2
# Header files (.h)

* For each module that exports variables or functions, it is customary to create a **header** that contains declarations for all exported variables or functions.

```c
// foo.c
#include "foo.h"
int x = 33;

int incfunc(int x) {
    return x + 1;
}
```
<br>

```c
// foo.h
#ifndef _FOO_H
#define _FOO_H

extern int x;
int incfunc(int);
#endif
```

```c
// preprocesor output
extern int x;
int incfunc(int);

int x = 33;

int incfunc(int x) {
    return x + 1;
}
```
---
class: compact
# Notes
![](images/ccompilation.png# w-40pct fr)

* The C compiler enforces almost nothing, and even what it enforces can be circumvented with type casts
* The C compiler only knows about declarations that are in the source file or header files included by the preprocessor
* The C compiler does not check for consistency in other modules
* The Linker only checks to make sure everything is declared (once).  It knows nothing about types -- even the difference between variables a functions ! 
* Notice that the inputs to the linker are binary files -- all of the type information is gone gone gone.
https://www.microforum.cc/blogs/entry/37-about-translation-units-and-how-c-code-is-compiled/

---
class: small-code,compact,hljs-tomorrow-night-eighties,line-numbers,col-2
#Independent Compilation Example

```c
// test_inc .c
#include <stdio.h>
#include "inc.h"
int main(void){
    inc(); inc();
    printf("%i\n", value);
    return 0;
}
```
```c
// inc.c
#include "inc.h"
int value = 0;
static int increment = 2;
void inc(){
    value += increment;
}
```
<br>
```c
// inc.h
extern int value;
extern void inc(void);
```
```bash
$ gcc inc.c test_inc.c
$ ./a.out
4
```
```bash
$ gcc -c inc.c 
$ gcc -c test_inc.c
$ gcc inc.o test_inc.o
$ ./a.out
4
```
---
class: small-code,compact,hljs-tomorrow-night-eighties,line-numbers,col-2
#  Independent Compilation
Suppose we change the types in inc.c, but don't recompile test_inc.c
```c
// inc2.c
#include "inc.h"
float value = 0;
static int increment = 2;
void inc(){
    value += increment;
}
```
<br>
```c
// inc.h
extern float value;
extern void inc(void);
```
```shell
$ gcc -c inc.c
$ gcc test_inc.o inc.o
$ ./a.out
1082130432
```

What happened ?

---
class: compact
# Build Programs

* There are substantial opportunities to mess up the dependency information in C programs
* To help minimize these errors we:
  * Religiously use header files for defining shared variables and functions
  * Use tools to help us ensure that files are recompiled as needed
* The traditional build tool for c is called make
  * make is driven by a "makefile" which is essentially a set of rules that codify the file dependencies and the processes for compiling programs
  * makefiles are (unfortunately) challenging to write correctly
* There are better tools, but 
  * The simple case is harder and
  * Many programs that you might like to use are built with make

---
class: compact
# A Simple Compilation

![](images/simplecompilation.png# w-30pct fr)

Compiling a small C program requires at least a single .c file, with .h files as appropriate. Although the command to perform this task is simply cc file.c, there are 3 steps to obtain the final executable program, as shown:

1.	**Compiler** stage: All C language code in the **.c** file is converted into a lower-level language called Assembly language; making **.s** files.
2.	**Assembler** stage: The assembly language code made by the previous stage is then converted into object code which are fragments of code which the computer understands directly. An object code file ends with **.o**.
3.	**Linker** stage: The final stage in compiling a program involves linking the object code to code libraries which contain certain "built-in" functions, such as **printf**. This stage produces an executable program, which is named **a.out** by default.

---
class: small-code,compact,hljs-tomorrow-night-eighties,line-numbers
# Compiling with Several Files

![](images/2019-12-11-14-10-53.png# w-40pct fr)

When your program becomes very large, it makes sense to divide your source code into separate easily-manageable .c files. The figure above demonstrates the compiling of a program made up of two .c files and a single common.h file. The command is as follows:
```
gcc green.c blue.c
```
where both .c files are given to the compiler. Note that the first two steps taken in compiling the files are identical to the previous procedure for a single .c file, but the last step has an interesting twist: The two .o files are linked together at the Linker stage to create one executable program, a.out.

---
class: compact
# Separate Compilation
![](images/2019-12-11-14-12-44.png# w-40pct fr)

The steps taken in creating the executable program can be divided up in to two compiler/assembler steps circled in red, and one final linker step circled in yellow. The two .o files may be created separately, but both are required at the last step to create the executable program.

You can use the -c option with cc to create the corresponding object (.o) file from a .c file. For example, typing the command: cc -c green.c will not produce an a.out file, but the compiler will stop after the assembler stage, leaving you with a green.o file.

---
class: small-code,compact,hljs-tomorrow-night-eighties,line-numbers
# Separate Compilation Steps

![](images/2019-12-11-14-14-00.png# w-40pct fr)

The three different tasks required to produce the executable program are as follows:

*	Compile green.o: `gcc -c green.c`
*	Compile blue.o: `gcc -c blue.c`
*	Link the parts together: `gcc green.o blue.o`

* Note that in order to create the file, **green.o**, the two files, **green.c** and the header file **common.h** are required. 
* In order to create the executable program, **a.out**, the object files **green.o** and **blue.o** are required.

---
class: compact
# Dependency

* **make** operates on the principle of *dependencies*  
  * For example, *program.o* depends upon *program.c*
  * If *program.c* is modified, then it is necessary to rebuild *program.o* and then any components that depend upon *program.o*

* These dependencies must be form an acyclic graph.

---
class: compact
# Dependency Graph
![](images/2019-12-11-14-20-40.png# w-40pct fr)

* The graph in the figure is a program that consists of 5 source files -- *data.c*, *data.h*, *io.c*, *io.h*, and *main.c*
* The lines radiating downwards from a file are to the files upon which it depends.  
* For example, *main.o* depends upon *data.h*, *main.c* and *io.h*.

---
class: compact
# How Dependency Works

![](images/2019-12-11-14-23-32.png# w-40pct fr)

* Suppose you have compiled the program and subsequently edit *io.c*.
* In the dependency graph, *io.o* must be regenerated as well as the project executable

---
class: small-code,compact,hljs-tomorrow-night-eighties,line-numbers
# How Dependencies are defined in make
![](images/2019-12-11-14-20-40.png# w-40pct fr)

```make
project1: data.o main.o io.o
  cc data.o main.o io.o -o project1
data.o: data.c data.h
  cc -c data.c
main.o: data.h io.h main.c
  cc -c main.c
io.o: io.h io.c
  cc -c io.c
```

* The **make** program gets its dependency information from a *makefile*
* A makefile defines dependency rules and production rules
* Whenever a dependency is not satisfied, make re-runs the associated production rule

---
class: compact
# Make is driven by file timestamps

![](images/space.png# w-10pct)
![](images/2019-12-11-14-30-13.png# w-80pct)

---
class: small-code,compact,hljs-tomorrow-night-eighties,line-numbers
# Translating the Dependency Graph

![](images/space.png# w-30pct)
![](images/2019-12-11-14-35-48.png# w-40pct)

Each dependency shown in the graph is circled with a corresponding color in the *Makefile*
```makefile
   target: source file(s)
      command (must be preceded by a \t)
```

A target in the *Makefile* is a file that will be created or updated when any of its source files are modified.

---
class: small-code,compact,hljs-tomorrow-night-eighties,line-numbers
# Running Make

* By default, **make** looks for a file named **Makefile** or **makefile** in the current directory

```bash
$ make
   cc -c data.c
   cc -c main.c
   cc -c io.c
   cc data.o main.o io.o -o project1
```

---
class: small-code,compact,hljs-tomorrow-night-eighties,line-numbers
# Macros in Make

The **program** allows you to use macros, similar to variables, to store information

```makefile
OBJECTS = data.o io.o main.o
project1: $(OBJECTS)
  cc $(OBJECTS) -o project1
```

---
class: small-code,compact,hljs-tomorrow-night-eighties,line-numbers
# Don't Repeat Yourself

```makefile
results.txt : isles.dat abyss.dat last.dat
  python testzipf.py abyss.dat isles.dat last.dat > results.txt
```
* Replace the target with an "automatic" variable `$@`

```makefile
results.txt : isles.dat abyss.dat last.dat
	python testzipf.py abyss.dat isles.dat last.dat > $@
```

* Replace the dependencies with `$^`

```makefile
results.txt : isles.dat abyss.dat last.dat
	python testzipf.py $^ > $@
```

---
class: small-code,compact,hljs-tomorrow-night-eighties,line-numbers
# Special Macros

There are a number of predefined macors

* **CC**  : the current C compiler, defaults to **cc*
* **CFLAGS** : flags for the C compiler
* **$@** : name of the current target
* **$?** : list of files for current dependency
* **$<** : Source file (single) of current dependency

---
class: small-code,compact,hljs-tomorrow-night-eighties,line-numbers
# Pattern Rules

Suppose we have the following:

```makefile
isles.dat : books/isles.txt
	python countwords.py $< $@

abyss.dat : books/abyss.txt
	python countwords.py $< $@

last.dat : books/last.txt
	python countwords.py $< $@
```

* We can replace these with a single "pattern" rule

```makefile
%.dat : books/%.txt countwords.py
	python countwords.py $< $*.dat
```

---
class: small,very-small-code,compact,hljs-tomorrow-night-eighties,line-numbers
# An example makefile

```makefile
# Generate summary table.
results.txt : testzipf.py isles.dat abyss.dat last.dat
	python $< *.dat > $@

# Count words.
.PHONY : dats
dats : isles.dat abyss.dat last.dat

%.dat : books/%.txt countwords.py
	python countwords.py $< $*.dat

.PHONY : clean
clean :
	rm -f *.dat
	rm -f results.txt
```

* `clean` doesn't build anything (hence a "phony rule") but it can be used to remove the detritus of a build

```bash
make clean
```

---
class: small-code,compact,hljs-tomorrow-night-eighties,line-numbers
# Using Variables

```makefile
  python countwords.py $< $*.dat
```

```makefile
COUNT_SRC=countwords.py
LANGUAGE=python
COUNT_EXE=$(LANGUAGE) $(COUNT_SRC)
...
  $(COUNT_EXE) $< $*.dat
```

---
class: small-code,compact,hljs-tomorrow-night-eighties,line-numbers
# Including files

* Suppose we put the rules in a separate file `config.mk` such as

```makefile
COUNT_SRC=countwords.py
LANGUAGE=python
COUNT_EXE=$(LANGUAGE) $(COUNT_SRC)
```

Then in our makefile we *include* this

```makefile
include config.mk
```

---
class: small-code,compact,hljs-tomorrow-night-eighties,line-numbers
# Predefined rules

**make** knows how to create a *.o* file from a *.c* file using the **CC** and **CFLAGS** macros, thus we can simplify the make file to a list of dependencies and one production rule:

```makefile
OBJECTS = data.o main.o io.o
project1: $(OBJECTS)
  cc $? -o $@
data.o: data.h
main.o: data.h io.h
io.o: io.h
```

```bash
$ make
cc    -c -o data.o data.c
cc    -c -o main.o main.c
cc    -c -o io.o io.c
cc data.o main.o io.o -o project1
```
---
class: small-code,compact,hljs-tomorrow-night-eighties,line-numbers
# Self Documenting Make files

```bash
make help
```
```bash
results.txt : Generate Zipf summary table.
dats        : Count words in text files.
clean       : Remove auto-generated files.
```
* this can be generated as in

```makefile
.PHONY : help
help :
	@echo "results.txt : Generate Zipf summary table."
	@echo "dats        : Count words in text files."
	@echo "clean       : Remove auto-generated files."
```

---
class: small-code,compact,hljs-tomorrow-night-eighties,line-numbers
# Auto-dependency generation

```makefile
SRCS = a_list_of_source_files
DEPDIR := .deps
DEPFLAGS = -MT $@ -MMD -MP -MF $(DEPDIR)/$*.d

COMPILE.c = $(CC) $(DEPFLAGS) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c

%.o : %.c
%.o : %.c $(DEPDIR)/%.d | $(DEPDIR)
        $(COMPILE.c) $(OUTPUT_OPTION) $<

$(DEPDIR): ; @mkdir -p $@

DEPFILES := $(SRCS:%.c=$(DEPDIR)/%.d)
$(DEPFILES):

include $(wildcard $(DEPFILES))
```

---
class: small-code,compact,hljs-tomorrow-night-eighties,line-numbers
# Auto-dependency generation
```makefile
DEPDIR := .deps
DEPFLAGS = -MT $@ -MMD -MP -MF $(DEPDIR)/$*.d
```

* Place dependency files in subdirectory named `.deps`

`DEPDIR = ...`

* GCC flags to generate dependencies

`DEPFLAGS = ...`

* `-MT $@` : set the name of the target in the generated file
* `-MMD` : generate dependency information a side-effect of compilation
* `-MP` : add a target for every prerequisite
* `-MF $(DEPDIR)/$*.d`  write the dependency file to this destination

---
class: small-code,compact,hljs-tomorrow-night-eighties,line-numbers
# Auto-dependency generation

```makefile
%.o : %.c
%.o : %.c $(DEPDIR)/%.d | $(DEPDIR)
        $(COMPILE.c) $(OUTPUT_OPTION) $<
```

* `%.o : %.c `  Delete the built-in rule
* `... $(DEPDIR)/%.d` : make the generated dependency rule a prerequisite of target
* `... | $(DEPDIR)`  : create the dependency directory if needed

---
class: small-code,compact,hljs-tomorrow-night-eighties,line-numbers
# Auto-dependency generation
```makefile
$(DEPDIR): ; @mkdir -p $@

DEPFILES := $(SRCS:%.c=$(DEPDIR)/%.d)
$(DEPFILES):
include $(wildcard $(DEPFILES))
```

* `$(DEPDIR): ; @mkdir -p $@` : create dependency directive
* `DEPFILES := $(SRCS:%.c=$(DEPDIR)/%.d)` generate a list of dependency files from source files
* `include $(wildcard $(DEPFILES))` include any dependency files that exist (wildcard prevents errors if they don't exist)
---
class: compact
# Summary

* Independent compilation
* Compilation stages
* Modules, definitions, declarations
* Make to track dependencies
* Dependency graphs
* makefile
* Cover: By Aurelio A. Heckert, <a href="https://creativecommons.org/licenses/by-sa/2.0" title="Creative Commons Attribution-Share Alike 2.0">CC BY-SA 2.0</a>, <a href="https://commons.wikimedia.org/w/index.php?curid=18203">Link</a>
* http://web.eng.hawaii.edu/Tutor/Make/
* http://swcarpentry.github.io/make-novice/