Remote Shell - Version 2

Introduction

This is a continuation of the "remote shell" project started in Assignment 3. The goals for this version of the assignment are:

The Protocol

Clients and the server communicate by exchanging messages. In general, clients send request messages to the server, and the server sends reply messages to the client. Each message consists of a fixed-length message header followed immediately by a variable-lengthmessage body. The length of a message body is specified in the "msg_length" field of the message header, and may be zero.

A message header consists two 32-bit integer values, the msg_length field mentioned above, followed by a msg_type field to be described below. Because the C/C++ languages do not specify the size of an int, we will use a new type, called int32_t for these integers to be sure they are 32 bits long on all platforms. The following code will provide a definition of this data type on all platforms that I have tested: lnx0006 (Linux on IBM S/390), spinoza (Linux on IA_32), forbin (OSF/1 on Alpha), and babbage (Solaris on UltraSPARC): 

  #include <stdlib.h>
  #include <sys/types.h>

  #ifndef int32_t
  typedef int int32_t;
  #endif
That is, some of these systems predefine int32_t in stdlib.h, some in sys/types.h. On others, the type is not defined in any header file, but the size of an int is 32 bits anyway.

The msg_type field is to be an integer constant with the following values and meanings defined. The names listed should be declared as int32_t constants, and you must be sure to convert these values (as well as the msg_length values) to network byte order before sending them over a socket connection.

Client to Server Messages

Name Value Description
CS_SHUT 101 Message from Client to Server telling the server to shut down. The message body has a length of zero. The server responds by sending a SC_SHUT message (see below) to all clients (including the one that sent the CS_SHUT message to it, writing an appropriate message to the log file, closing it, and exiting. Note: This type code is unnecessary, and may be omitted if you wish.
CS_EXIT 102 Message from Client to Server to indicate that the client is exiting. The server responds by writing an appropriate message to the log file and releasing all resources associated with this client. The message body has a length of zero. This type code should be used when the client encounters end of file on stdin (because the user typed Control-D at the beginning of a line after being prompted to enter a command).
CS_USER 103 Message from Client to Server giving the login name of the local user. This message is sent in response to a SC_LOGIN message from the server. The message body contains the user's login name. You can determine a user's login name by using the nested function calls getpwuid( geteuid() ). But the user may override this name using the -u or --user command line option.
CS_CMD 104 The client sends a complete command line in the message body.
CS_STDIN 105 The client uses this message type to send anything the user types between the time a command has been sent to the server and the receipt of the next SC_PROMPT message from the server.

Server to Client Messages

Name Value Description
SC_SHUT 201 The server sends this message to each connected client just before it exits.
SC_LOGIN 202 The server sends this message to a client as soon as the client connects to it. The message body is a message telling the server's name, current version, host name, and port number, which the client prints for the client to see. The client responds with the user's login name, as described above.

Sample message body: 

QC Remote Shell version 2.3 On spinoza.qc.edu port 0x8000
SC_DENY 203 The server sends this message if the effective user id for the user name sent in response to a SC_LOGIN message does not match the euid of the server process. Note that actually attempting to authenticate users is beyond the scope of this project. The body of this message is empty. The client responds by sending a CS_EXIT message, and exiting.
SC_PROMPT 204 The server sends this message to a client to indicate that it is ready to accept a command line for processing. The client prints the message body on the user's console, waits for the user to enter a command line, and replies to the server with a CS_CMD message.
SC_STDOUT 205 The server uses this message to send standard output generated by a command being executed. The client prints the output on the user's console.
SC_STDERR 206 The server uses this message to send error output generated by a command being executed. The client prints the output on the user's console.

Starting the Assignment

All files for this assignment are to have RCS revision numbers in the form 2.x. Lock all your source files from Assignment 3 for editing, and use the command, ci -f -r2 *.h *.cc Makefile to check them in with revision numbers 2.1. The log file message can simply be something like, "First revision for Assignment 4," for all the files.

All new files that you add to the project (such as qprotocol.h described below) must also be checked in the first time with the "-r2" option. You will be prompted for both a file description and a log message for each new file checked in this way, so be prepared to enter proper information for both prompts.

To start the assignment, modify qclient.cc so it handles the -u and --user options.

The next step is to define all the protocol constants listed above in a header file that is to be included in both qclient.cc and qserver.cc. I suggest that you name this header file qprotocol.h. Be sure you document it and code it to handle nested includes of itself in the usual way. This header file is also the proper place to define int32_t.

At this point you should modify your client and server to make sure they exchange protocol messages properly. Still using an iterative server as in Assignment 3, code your server and client so the client connects to the server, and exchanges a proper set of messages with the server. But for this version of the project, there will be no SC_STDOUT, SC_STDERR, or CS_STDIN messages. The server will not actually exeute commands entered by the user. But you can test the other messages. The body of SC_PROMPT messages should be the value of the server's PS1 environment variable if it is defined; if PS1 is not defined, use the prompt string, "qserver> ". For this version of the project, you do not need to handle concurrent input from different sources. The client and server should simply alternate sending messages to each other. The client, however, will have to recognize two special commands that the user might type, exit and shut, and send the proper types of messages to the server in those cases.

Test this version of the assignment carefully and check all files into the RCS database before proceeding further in the project.

Design Changes

In prototyping this project, I made two changes to the foregoing description that you may want to incorporate in your project. You are not required to make these changes, but you may find them useful.

The first change is to remove the recognition of the shut and exit commands from the client; simply pass all command lines on to the server, and let them be recognized and processed at that end. However, I programmed my client to send a CS_EXIT message to the server when it encounters end of file while reading a command from the user. This change means that the client always actively informs the server when it is exiting, which makes the server a bit easier to implement. This reasoning could be extended so that the client sets up an on_exit() function that sends an SC_EXIT message to the server in all situations, including receipt of a signal.

The second change is to have the prompt string not be the value of the PS1 environment variable. Rather, have the server generate a prompt string that includes the user's name, the server's host name, and the current working directory (call getcwd() for the latter). This prompt string will be less distracting and more informative that the usual value of PS1, which needs have various environment variables recognized and substituted in order to be readable. Note that the current working directory will change when you implement the cd builtin command (see below), so you will need to generate this string each time you are going to send it.

Messaging Utility Functions

I found that writing the following functions helped in developing the project.

Function Prototype Description
int read_header( int, msg_header_t & ); The first parameter is the fd of a socket from which to read a 4-byte header. The second parameter is a reference to a struct (defined in qprotocol.h) with two fields of type int32_t, which this function fills in with integer values in proper byte order.
int read_body( int, msg_header_t, char * ); Given a socket and a header, this function reads a message body. I decided to have the caller allocate the memory for the message body, but you could have this function do it instead.
int recv_msg( int, msg_header_t &char ** ); Given a socket, a reference to a header, and a pointer to a variable of type char *, this function reads a message header from the socket, fills in the header, allocates memory for the message body, and reads in the message body. The caller is responsible for freeing the message body using delete[].
int write_msg( int, const msg_header_t, const char * ); Write a message header and message body to a socket.
int send_msg( int, const int32_t, const char * ); Given a socket, a message type, and a message body, construct a message header, and call write_msg() to transmit it.

Source code for these functions are available if you want to use them:

Managing Concurrent Clients

As described in class, the server is to be a concurrent server, and the design you are to use is to fork a separate "manager" process for each concurrent client. Managing these manager processes is the topic of this section.

Every time the server accepts a connection from a client, it forks a child to manage the client. If any manager's client enters a "shutdown" command, that manager must notify the server process, which in turn must notify all the manager processes so they can send SC_SHUT messages to each of their clients and exit. We will use signals to handle the interprocess communication (IPC) both from the managers to the server and from the server to the managers.

Technical Note

There is a technicality that needs to be cleared up before proceeding to how to work the signals. You have already used on_exit() to set up an exit handler to make sure the server always shuts down cleanly, which continues to be a good idea for this assignment.

However, when the server forks a manager process, the manager gets a complete copy of the server's memory, its open fds, etc. This "etc." includes both the server's signal handlers and exit handlers.

If the manager process called any of the exec() functions, the signal handlers and exit handlers would get reset to their default values (because the signal and exit handler functions would no longer be in the manager's memory). But for this project, the manager processes do not call exec because of the programming overhead that would be needed to pass parameters from the server to a manager. [Actually, you can write the code to be executed by a manager as a separately executable file if you like; it's not all that difficult.] Here I'm assuming the manager simply calls the following function, instead of exec, to do its work: 

    void manageClient( int fd, const char *client_host );
This function should be defined in a separate .cc file to keep it logically separate from the code of the manager process.

The manager process can cancel the signal handling set up by the server process by using SIG_DFL for the second argument it passes to signal(): 

    signal( SIGTERM,  SIG_DFL );
    signal( SIGINT,   SIG_DFL );
    signal( SIGCHLD,  SIG_DFL );
Figure 10.1 of the textbook lists all signal names and their default actions.

However, there is no way (that I know of) to unset on-exit functions. You can call on_exit() again to add another function to the list of functions to be called when the process exits, but you can't remove items from this list.

It's not elegant, but a solution to this problem is to have the manager processes always call _exit() when they exit and never call exit(). See section 7.3 of the textbook for more details about the differences between exit() and _exit().

Whenever a process terminates, the kernel sends SIGCHLD to its parent, but the default action is to ignore this signal. So the server must call signal() in order to have its signal handler called when a manager exits. This signal handler uses getpid() to find out which child is exiting, then uses waitpid() to get the child's exit code and to let the child leave the zombie state.

The manager process should call _exit(0) to exit normally, or _exit(1) if it encounters an error. But it uses a special exit code, SHUTDOWN_EXIT (minus 2) when it exits because of a client's shutdown command. The manager's SIGCHLD handler checks for this exit code after calling waitpid() and kills all the other manager processes when this occurs. I used a vector<pit_t> list in the server to keep track of the active manager processes, and killed each manager individually so I could be sure each one completed and was harvested by waitpid(). You could also create a process group for the manager processes as described by a student in class, but you would't be able to tell when all the managers have exited.

Note:

The exit code returned by the WEXITSTATUS() macro will be an unsigned integer between 0 and 255 (0x00 and 0xFF). If you define SHUTDOWN_EXIT to be -2 and the manager passes SHUTDOWN_EXIT to _exit() (see below) it will appear to have a value of 254 (0x000000FE) instead of -2 (0xFFFFFFFE). Don't let this bug you!

The manager processes should catch whatever signal you use to kill them (SIGINT or SIGTERM), and write a SC_SHUT message to their clients before exiting.

You should verify that all of the process management issues described here are working before proceeding to actual command line processing. You could use the following command interpreter while testing this part of the assignment: 

    int processCommandLine( int fd, const char *command_line )
    {
      if ( 0 == strcmp( "exit", command_line ) )
        _exit( 0 );
      if ( 0 == strcmp( "shut", command_line ) )
        _exit( SHUTDOWN_EXIT );
      send_msg( fd, SC_STDOUT, "Unrecognized Command" );
      return 0;
    }

Command Line Processing

There are three types of commands to process: builtin commands that must execute from the same process as the client manager, builtin commands that must execute as a separate process, and external commands, which always must execute as a separate process.

The builtin commands that must execute from the same process as the client manager are exit, shutdown, and cd. The exit and shutdown commands must run from the manager process because the manager process must be the one that exits! It's exit code (SHUTDOWN_EXIT or not) will tell the server process whether to shut down or not. The cd command must also execute from the manager's process because it calls chdir(2) to do its work, and chdir changes the current working directory of the process that called it; if it were executed from a child process, it would have no effect once the child process exits.

There is one other builtin command you are to implement, and that one should execute from a child process in order to enable I/O redirection. (I/O redirection is an optional part of this assignment.) That command is echo. For this assignment, have your echo command exit with an exit code equal to the number of command line arguments it received; you can use that value in your prompt string to help verify that it is being handled correctly.

External commands are to be executed using the execvp(3) library call. This function is easier to use than the execve(2) kernel call because execve requires you to specify the full pathname to the command file, whereas execvp will search your process' PATH to find the executable file. So, by using execvp you avoid having to tokenize PATH and search each directory in the list for an executable file. The tradeoff is that execvp takes only two arguments, so you cannot change a command's environment. (You can add code to do that in the "next assignment" for this course!)

Tokenizing a Command Line

You can use strtok() to separate a command line into whitespace-separated tokens, as you did in the wc assignment at the beginning of the semester. Here is a sketch of some code that builds an argument vector out of a command line, treating I/O redirection tokens (< and >) specially: 
  static vector token_vec;
  const char *token = strtok( buf, " \t\n\r" );
  while ( token != 0 )
  {
    token_vec.insert( token_vec.end(), token );
    token = strtok( 0, " \t\n\r" );
  } 
  int num_tokens = token_vec.size();

  for ( vi = token_vec.begin(); vi < token_vec.end(); vi++ )
  {
    if ( 0 == strcmp( *vi, "<" ) )
    {
      redirect_stdin = *(++vi);  // Need to check for null pointer.
    }
    else
      arg_vector[ ++arg_count ] == *vi;
  }
  //  Add null pointer to end of argument vector (required when calling
  //  execvp).
  arg_vector[ arg_count ] = 0;

Dispatch Tables

By making the prototypes for builtin commands as similar to main() as possible, we can accomplish two things: (1) We can use the same data structures and procedures for parsing the command line no matter which of the three types of commands we are working with, and (2) we can use a single function prototype for all builtin commands, allowing us to use a dispatch table to call the functions that implement the builtin commands. Dispatch tables use the function pointers, which do not exist in Java.

Here is the prototype we want to use for one of our builtin commands, in this case, the function that implements the echo command: 

  int do_echo( int argc, char *argv[] );
The dispatch table is to consist of an array of structs, each of which contains the name of a builtin command as a string, and a pointer to the function that will handle that command name. The struct we want needs to include a field for a "pointer to a function that receives an int and a char*[] and returns an int." Here is a typedef for such a beast: 
  typedef int builtin_func_t( int argc, char *argv[] );
(If you want your builtin commands to be able to send messages directly to clients, you could add another int paramter for the fd of the socket connecting to the client.)

Now you can write prototypes for all your builtin functions like this: 

  builtin_func_t do_exit, do_shut, do_chdir, do_echo;
You can define the struct of command names and function pointers like this: 
  struct dispatch_t
  {
    const char            *func_name;
    const builtin_func_t  *func_ptr;
  };
And you can build your array of command names and pointers to functions like this: 
  dispatch_t  no_fork[] =
  {
    { "exit",     do_exit   },
    { "q",        do_exit   },
    { "shut",     do_shut   },
    { "shutdown", do_shut   },
    { "cd",       do_chdir  },
  };
const int num_no_fork = sizeof( no_fork ) / sizeof( dispatch_t );
As the name of the array implies, there should be separate dispatch table arrays for those builtins that require a fork() and those that don't. Note also that you can easily set up multiple command names pointing to the same function, which act essentially like builtin command aliases. Finally, note that the constant num_no_fork gives the number of entries in the array, even if the number of entries grows or shrinks in future versions of the project.

Finally, here is code that checks if a command name (arg_vector[0]) is one of the do_fork[] commands, and calls the corresponding function if it is: 

  for ( int i = 0; i < num_no_fork; i++ )
  {
    if ( 0 == strcmp( arg_vector[ 0 ], no_fork[ i ].func_name ) )
    {
      command_status =
            no_fork[ i ].func_ptr( client_fd, arg_count, arg_vector );
      return;
    }
  }
If execution falls through the for loop instead of returning, it means that the command name wasn't found in the no-fork dispatch table, and the program has to fork a child process, which will then look for the command name in the do-fork dispatch table, or try to use execvp to execute the command if it is not there.

Global Variables

Global variables can make program management very error-prone. It's difficult to keep track of what functions modify the global variables, what ones access their values, and what functions don't use them at all. But global variables can be useful because they avoid a lot of parameter passing, which can be difficult to keep track of. For example, client managers must keep track of exit code of each command that it executes because that is the value of the "$?" shell variable (to be implemented in Assignment 8 of this project). So it makes sense to use a global variable, named command_status in the previous code, to keep track of each command's exit code. It might be set by a no-fork builtin, by a do-fork builtin, or by an external command (the code you get by calling wait() or waitpid()). And in this assignment, you can use its value as part of the prompt string sent to the client for each command.

You should minimize the scope of global variables by declaring them static wherever possible. A static variable (or constant) declared outside of any function definition is visible only to the functions defined in the .cc file where it appears.

The bottom line here is that global variables are sometimes necessary, but should be used carefully.

Overview of Command Processing

Before looking at how to manage I/O for commands being executed, here is a birds-eye view of what we want to accomplish:

Submitting the Assignment

The assignment is due by midnight on the last day of exams (Friday, May 24.) Create a tar file of your clean project directory, and add to it a text file named README (spelled and capitalized just like that) that tells how much of the project you finished so I can tell what parts to test and what not to test. This is not a documentation file, just a note that tells what parts of the project do and don't work.

There will be no extensions for the project and no INC grades for the course.

The final exam will be based in large part on the concepts involved in doing the project (managing sockets, processes, signals, pipes, etc.). Rather than have the project due before the final, I am providing you with my solution for you to study from. If you decide to copy any of my code in the final version of your project, you may do so, but indicate what code you copied in your README file. If you use my code as a model and write your own code based on that model, you don't have to mention that in your README file. Warning: My code works (Mostly! See "Known Bugs" below.), but my model for doing the project is probably different from what you have already started. Trying to change your code to use parts of mine will probably cause more problems than if you just write your own code. And you're not allowed just to submit all my own code back to me!

Links to Dr. Vickery's Code

Known Bugs in Dr. Vickery's Code