CSCI A348/A548
Lecture Notes 5

Fall 1999

In which we start developing CGI scripts with Perl.

To understand how scripts interact with web browsers and servers we begin by reviewing a simpler interaction: how static HTML files are requested by and displayed by users. Let's say that I have the following simple, basic HTML file in my DocumentRoot and its name is hello.html.

<html>
<head>
<title> Hello world! </title>
</head>
<body>
<h1> Hello world! </h1>
<p> How are you doing? </p>
</body>
</html>

The path to the file is

/u/dgerman/httpd/htdocs/hello.html

and it has to be made readable by the world for it to be accesible over the web:

tucotuco.cs.indiana.edu% pwd
/nfs/paca/home/user2/dgerman/httpd/htdocs
tucotuco.cs.indiana.edu% vi hello.html
tucotuco.cs.indiana.edu% cat hello.html
<head>
<title> Hello world! </title>
</head>
<body>
<h1> Hello world! </h1>
<p> How are you doing? </p>
</body>
</html>
tucotuco.cs.indiana.edu% ls -l hell*     
-rw-r--r--   1 dgerman  students     126 Sep  5 18:39 hello.html
tucotuco.cs.indiana.edu%

Once we've created the HTML text, it may seem that the process of delivering it to a web browser should be a trivial task. But serving even a simple page like this one requires that a lot of coordination occur between the browser and the web server on which the page is stored.

By web server we mean a program residing on a host machine that uses the Hypertext Transport Protocol (HTTP) to communicate with the browser. The program that stored in my burrow account in

/u/dgerman/httpd/httpd

is such a program, and you are using similar servers. The web is based on a client-server model. This means that there is a server (that provides resources) and a client (which requests them).

Here's what we need to keep in mind about them:

both the server and the client are programs
they may reside on the same machine or on different ones
both the client and the server have agreed in advance on some mechanism that allows them to communicate with each other (such a mechanism is called a protocol).

For the World Wide Web the role of the client is played by a web browser (such as Netscape) and the web server is the software that delivers resources such as: computer files, images, movies, sound files, to one or more web browsers.

There are thousands of web servers throughout the world (wide web) but they are all acessible from any browser because they have all agreed to use a common protocol - the Hypertext Transfer Protocol (HTTP). HTTP is based on an exchange of requests and responses.

Each request can be thought of as a command, or action, which is sent by the browser to the server to be carried out. The server performs the requested service and returns its answer in the form of a response.

The components of a simple WWW interaction are the user, the client, and the server. The client acts as an intermediary between the user and the server.

Steps 1-7 detail the basic information flow in a simple HTTP transaction. Essentially the client requests a file and the server delivers it. The entire HTTP process takes place as a result of simple transactions of requests and responses.

The user sees an interesting URL
http://tucotuco.cs.indiana.edu:19904/hello.html
and clicks the hyperlink or types the URL into the browser.
The browser interprets this command: It is different from printing, creating the bookmark, saving a file, changing any preferences, etc. This command (the equivalent Open Location in Netscape) says that the computer
tucotuco.cs.indiana.edu
needs to be contacted on port 19904 and that the /hello.html file is needed.
For this the browser sends the HTTP GET command to the server (not shown here - we'll look at how this works when we simulate this request process using telnet). The path to the requested file is relative to the server's document root).
The browser sends the GET request to the server, indicating what file it needs. This request travels over the Internet, going from computer to computer until it reaches the web server's host: tucotuco in the CSCI's burrow cluster.
There's a network security aspect here that we will need to address later.
The server receives and parses the request. It uses the file extension (.html) to determine the type of information in the file. The .html means that it will send back to the browser the file but it will first say: the file's Content-type: text/html.
You do not have to write this in the file, it is inferred by the server from the file's extension. But the server does send this information to the browser as part of the header, followed by the data (the actual file) as explained below.
An HTTP response goes from server to the client. The headers that are part of the message indicate that the request was OK and that the data returned is of
Content-type: text/html
The headers are then followed by (a blank line and then by) the HTML data itself.
The Content-type: part of the header tells the browser that the data is text formatted in HTML, so the browser renders the text appropriately, highlighting hyperlinks, etc.
User views the HTML output and has the opportunity to select another hyperlink, starting the cycle over again

Of course you want to achieve more than this limited functionality. You may want to query a database and return the result to the user. The HTTP server can't do this directly and instead it delegates the reques to an external program to which it has access, which acts as a gateway (or intermediary) between the server and the data repository.

When the server receives a request to access the database it passes the request to a gateway program which does whatever is necessary to get the data and return the results to the server.

The server then repackages the information from the script, and forwards the information back to the client. (In a sense the server acts as a sort of translator, taking data from either a file or script and providing it to the browsers in a consistent and uniform manner).

We make two observations now:

Passing data in the right format to the script is crucial.
Typically clients (web browsers) do not know (or care) whether the server handled the request internally or offloaded the work to other programs; they just interpret the returned result and display it to the user.

Clearly, in order to make all this relationship work, the gateway programs and the server must communicate with each other. The details of this interaction are specified by the Common Gateway Interface (CGI).

The CGI protocol

defines the input that a script can expect to receive from the server,
as well as the output it must return to the server in order to be understood.

What the script does in between the input and the output, is entirely up to the script.

So the process of servicing the

http://tucotuco.cs.indiana.edu:19904/cgi-bin/hello

request is different, because by the shape of the request the server realizes that it needs to execute the script specified by that address (or path) rather than simply retrieving the file.

Upon starting the script, the server provides it with a variety of potentially useful information (such as the name of the machine from which the request originated, type of browser used, etc.) and then starts the script. What follows is of no concern to the server, other than the output of the script, which the server will send back to the requesting browser.

You take a lot of responsibility this way if you're writing the script. While the server doesn't care how the script generates its output, it does need to know the format of the output - the script's output is, after all, the server's input (on the path back to the user). Recall that when the web server delivers a static file to the browser, it uses a filename extension to determine what to return in the Content-type header. This technique doesn't work for scripts, because a script's filename is unrelated to the type of information it returns. A script named getpic, for example, may return an image as its data (Content-type: image/gif) while the similarly named getinfo might return HTML text (Content-type: text/html). It is even possible for a single script to output different sorts of data depending upon the context in which it is called. Therefore it is absolutely essential that the script notify the server of the type of data it is generating, so that the server can pass this information on to the client.

The hello script is presented below:

#!/usr/bin/perl

print qq{Content-type: text/html\n\n<html>
  <head>
  <title> Hello world! </title>
  </head>
  <body>
  <h1> Hello world! </h1>
  <p> How are you doing? </p>
  </body>
  </html> 
};

The program has one statement only, which prints a few lines.

The first line starts by specifying the media type that identifies the data in the body. The \n\n that follow the header information are translated by Perl into newlines. The first ends the line with the content type, while the second inserts a (mandatory) blank line that separates the header from the rest of the message.

The remainder of the script simply outputs HTML text that looks suspiciously similar to the contents of the static hello.html shown earlier, beginning with the familiar <html> tag and ending with </html>. All of this text output by the print statement is sent to the server which executed the script.

The server captures the output, constructs a set of HTTP message headers (including the Content type returned from the script), and sends these headers and the rest of the script's output to the browser. Upon receiving and interpreting the data, the browser is left with the HTML shown in blue in the code fragment above.

This is rendered by the receiving browser in the exact same way as the hello.html file that we started with. So if we were to look only at the output we couldn't make any difference between the two approaches.

CGI scripts are quite flexible precisely because the server itself is not really involved in the process. The server's primary responsibilities are to

start the script
send it the proper information, and
pass the output back to the browser

It is up to the script to do most of the work. This division of labor leads to a protocol that is simple yet powerful.

To summarize, the information flow is as follows:

The user selects a URL that references a script:
http://tucotuco.cs.indiana.edu:19904/cgi-bin/hello
The client interprets the URL in the usual way:
GET
message requesting
/cgi-bin/hello
is sent to
tucotuco.cs.indiana.edu
on port
19904
Server receives request. Using location on the server (the path starts with /cgi-bin/) it determines it should run the script instead of simply retriving the file.
The server starts the script and sends it information in the form of environment variables and standard input
The gateway program (the script) undertakes any necessary processing and then produces output. The program suplies a Content-type header to indicate the format of the data to the server, for example:
Content-type: text/html
The headers are then followed by the HTML text generated by the script. From here on it's the same story as before:
The headers (including Content-type) and data go directly from program to the server.
Having successfully received data from the gateway program the server simply relays the information to the client.
An HTTP response goes from server to browser. Included in the response are a status indicating the request was OK and the Content-type header from the script. Following the headers is the actual HTML script.
The Content-type header tells the browser that the data is HTML, so the browser formats and renders the text appropriately, including highlighting links.
The user views the HTML output and has the opportunity to select another hyperlink, starting the cycle again.

Knowing these we will finish the Perl primer of last time and explain the following script that comes with the server:

#!/usr/local/bin/perl
 
print "Content-type: text/html\n\n";
while (($key, $val) = each %ENV) {
       print "$key = $val<BR>\n";
}

Its name is printenv and is our entry point to CGI.

Last updated: September 13, 1999 by Adrian German

CSCI A348/A548 Lecture Notes 5

CSCI A348/A548
Lecture Notes 5