ISO8859-2 etc, cover other European regions plus Russia, Israel, etc.
Each side should ideally be able to understand messages for its own version and all earlier ones. It should be able to write replies to old style queries in old style response format.
The ability to talk earlier version formats may be lost if the protocol changes too much. In this case, you need to be able to ensure that no copies of the earlier version still exist (imposible, of course...).
Part of the protocol setup should involve version information.
The advantages are compactness and hence speed. The disadvantages are caused by the opaqueness of the data: it may be harder to spot errors, harder to debug, require special purpose decoding functions.
The start of the message is typically a line that represents the message type. String handling functions such as strcmp() may be used to decode this. Successive lines contain the data. Line-oriented functions and line-oriented conventions are used to manage this. For example, an array of longs may be sent as one string per line, terminated by a blank line.
Character formats are easier to setup, easier to debug, but carry higher overheads: plus other problems.
The issues of internationalisation can affect the character set used.
The "standard" ASCII set allows some variations: characters such as `[' are not required to be present and may be substituted for others. The ISO 646 character set is a subset of full ASCII and is totally portable.
The European character sets are larger than ASCII, and vary across the
continent. The most common set is ISO8859-1, covering Western Europe
ISO8859-2 etc, cover other European regions plus Russia, Israel, etc.
The IBM PC uses "code pages" to handle multiple regions. This does not seem to be well supported outside of the PC world, or even on network protocols.
Many Asian languages are based on hieroglyphics. They require 16-bit
character coding
Unicode is the principal encoding at the moment. It is a pure 16-bit code,
large enough to cover all existing languages.
There is the problem of computer language and environment support for wide character sets. C has a "locale-dependant" wide character set, so C programs can be written to use them. The local C implementation may not support your particular locale, though. Java uses Unicode characters. Motif and X support wide characters. Windows NT supports Unicode characters. Windows 95 has wide character functions (inherited from Windows NT), but because the window functions come from Windows 3.1 all these functions are broken and will hang your program.
Some other character encodings exist. ISO 2022 allows a means of switching between 8 and 16 bit representations on the fly to save space.
Message identifiers should be in the ISO 646 format or in Unicode for international environments. If a change of character set is used, there must be some of signalling this, and also of handling failure.
The header consists of an indefinite number of (logical) lines. The header is terminated by a blank line
From: jan To: you // body starts after the blank line
The line terminator in Unix is LF, under DOS is CR-LF. The Internet standard is CR-LF. Special rules exist for s/w that breaks this. The standard allows long lines to be broken into multiple lines that still form one logical line: if the CR-LF is immediately followed by a space or tab, then it is still part of the logical line
some of a logical line split over several physical linesStages may split/reassemble lines for local convenience in the header using this rule. Some s/w doesn't understand this mechanism, apparently.
Extra fields are added to a message header field:
Content-Type: <toplevel-type/specific-type> Content-Transfer-Encoding: <encoding>The standard toplevel types are application, audio, image, message, multipart, text, video. All non-standard types must begin with x-, e.g. x-compress. For each toplevel type there is a set of minor types, such as image/jpeg, image/gif. Non-standard minor types must also begin with x-, such as image/x-portable-bitmap.
The encoding is to tell whether it is sent in e.g.7bit, case-insensitive, quoted-printable, etc.
To decide when a message part is finished, this RFC uses a terminating string. I have also seen a header Content-Length in the Web protocol - not sure where that fits in.
Money formatting is awful. The currency symbol can appear before, after or in the middle. Negative amounts can have the minus sign in lots of places. I don't know a standard.
type; external-programas in
image/jpeg; xv %s application/x-latex; latex %s
telnet localhost 13will result in output such as
Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. Mon Apr 1 19:44:18 1996 Connection closed by foreign host.The message from the daytime server "Mon Apr 1 19:44:18 1996" appears among the telnet messages.
For daytime, the server sends the time and closes the connection. An httpd server will wait for a request, send a reply and then close the connection:
telnet jan.newmarch.name 80 GET http://jan.newmarch.name/will result in index.html on pandonia being delivered
HTTP/1.0 200 Document follows MIME-Version: 1.0 Server: CERN/3.0 Date: Monday, 01-Apr-96 09:48:59 GMT Content-Type: text/html Content-Length: 4832 Last-Modified: Wednesday, 11-Oct-95 01:30:14 GMT <HTML> <HEAD> <TITLE> ... </BODY> Connection closed by foreign host.
Ohter procotols will accept a different set of messages, maintain the connection for longer, etc. With all text-based protocols, telnet gives a good way of interactive debugging.
Non-character protocols are harder to debug. One needs to be able to analyse the byte stream, which generally requires a separate program. To manage this, you usually need to insert a "filter" program between client and server. This filter will read messages from, say, the client and forward them to the real server. The client will connect to the filter on one port while the server is on a different one. Then the filter can print the messages in readable format while still relaying them.
In a connection-oriented system, generally authorisation will only need to be performed at connection time. Applications with multiple security access levels may need finer-grained security access, granted to individual transactions or to a session of transactions.
Security may be done by passwords, private key exchange, public key messages, or by using a token from a security server.
The important point is to whether one process is keeping state information about itself or about the other process. One process may keep as much state information about itself as it wants, without causing any problems. If it needs to keep information about the state of the other process, then problems arise: the process' actual knowledge of the state of the other may become incorrect. This can be caused by loss of messages (in UDP), by failure to update, or by s/w errors.
An example is reading a file. In single process applications the file handling code runs as part of the application. It maintains a table of open files and the location in each of them. Each time read or write is done this file location is updated. In the DCE file system, the file server keeps track of a client's open files, and where the client's file pointer is. If a message could get lost (but DCE uses TCP) these could get out of synch. If the client crashes, the server must eventually timeout on the client's file tables and remove them. In NFS, the server does not maintain this state. The client does. Each file access from the client that reaches the server must open the file at the appropriate point, as given by the client, to perform the action.
If the server maintains information about the client, then it must be able to recover if the client crashes. If information is not saved, then on each transaction the client must transfer sufficient information for the server to function.
If the connection is unreliable, then additional handling must be in place to ensure that the two do not get out of synch. The classic example is of bank account transactions where the messages get lost. A transaction server may need to be part of the client-server system.
An example use of this is for a teacher/student environment, where a teacher is using an instance of Mosaic, and the student instances are all mirroring the master instance. One possible architecture for this is where the master is told about the students (somehow) and then sends information to each of them whenever it does anything. The student instances in turn will have to listen and obey the master. This is a case of point to multipoint.
While it solves the problem, it has a lot of ``special case'' feel to it. It means that the master must be able to maintain a list of clients which have to be instances of Mosaic for it to work (why must it be limited in this way?). Similarly, the student instances must be listening to something that must be Mosaic again. How should they find each other given these types of constraints.
The requirements also rule out another related use: an instance of Mosaic displaying documents as told to by a script file, for automated demonstrations. Should this mechanism also know about files as well as networks?
A more general solution is for an instance of Mosaic to be able to tell an arbitrary application which documents it is fetching, and for an arbitrary application to be able to tell Mosaic to fetch some documents. The teacher/student will then be solved by a special-purpose application sitting between teacher and student instances of Mosaic.
The protocol is an ASCII based one, compliant with RFC 822. Data is passed in compliance with the MIME 1.0 specification. Command lines are terminated with a CR-LF. The browser (Mosaic) is regarded as the server, any application talking to it is the client. The primary components are
The xwebteach application sends a SEND ANCHOR to the teacher Mosaic. Each time it receives an anchor by a 301 ANCHOR <url>, it sends this on to each student Mosaic that is connected by GET <url>.
The changes made to Mosaic are quite general. It needs to maintain a list of clients that it is connected to and that are active in wanting URL information sent. Each time a URL is selected, it uses this list to send the anchor. In addition, it must also receive messages from each port that used by an application.
There is no registration part to the protocol. It is assumed that the user of xwebteach and all servers will set up the common ports and machines that are used. Connections are set up within the CCI library and all error checking is done internally to this library.
The security side of CCI at present is very weak - almost non-existent. Mosaic by default starts off by not listening on any port. The user selects by menu which port to listen on. It will only listen on one port so can be sent messages from only one client. Because the user has choice of port number and whether listening is enabled, the chances are low that an intruder will get in. At worst though, it will only be able to monitor your choice of URL's, or select its own.
The heart of this protocol is the SaveYourself command sent from the session manager (SM) to any client (application) that it is managing. It can also send a Die command to terminate any client, which it will do when the session is finishing. It must also be able to query the client for how to start it up again later, for its RestartCommand.
Clients are expected to locate the server in some unspecified manner. The sample implementation uses a Unix environment variable set by the session manager, and then read by each client that is forked by the session manager.
When a client finds the session manager it informs it that it wants to be managed by sending a RegisterClient. The SM replies with a RegisterClientReply. There are actually two possibilities by the time we get to this point: the client is starting for the first time, or is being restarted. How they tell the difference is that the client is expected to maintain state: it keeps an ID that will be null on first startup but non-null afterwards. The server sends this ID as part of the RegisterClientReply. This ID is expected to be part of the RestartCommand.
When a client is started the first time, no state has been saved about it, so the SM sends it a SaveYourself message straight away.
When a client receives a SaveYourself message, it is expected to save state in such a way that it can be restored. Suppose it is an editor, with an unnamed file in the buffer. It should be able to perform an interaction with the user to name this file first. Many applications may want to do this. To organise the order of these requests, the SM will tell a client to have an interaction, but only if the client has asked for it.
Thus there may be an InteractRequest from the client to SM, and an Interact message from SM to client. Only after the client sends InteractDone will the SM continue. Finally, a client should say that its save is complete by SaveYourselfDone.
This is all complex enough that a state transition diagram is useful on both client and SM sides. On the client side it is
There is a byte-level encoding of the protocol, with a fixed range of data types, such as one-byte, two-byte, four-byte unsigned integers, and sequences of integers.
Security is managed by the ICE library (see later). This is able to implement any security scheme, but the default is the MIT MAGIC-COOKIE scheme.
Examples of one-off are telnet, ftp. In these cases the protocol is a quite visible part of the applications, and there will be loops that poll the connection port as part of the applications.
For a library, there will need to be some interface between the library and the application in which it used, so that when certain protocol events occur appropriate application-specific code can be called.
In the X Window system all of the network interface is hidden behind a call XNextEvent(). This returns each time a message is received, with a data structure filled in with information about the message. The different data structures all have a first field set to the type of message. The application sets up a loop around XNextEvent() and within that loop branches to application code on the type of event. This is similar to the Microsoft Windows event loop, but of course that does not use a client-server architecture.
An alternative, that goes well with an object-oriented approach, is to use callback functions. An application registers a function to a message handler. Each time the message comes in, the library sorts out which message handler to call internally, and the message handler then calls each function on its callback list. this reduces the visibility of the protocol details, and just means that the library interface is reduced to registering callbacks for each message type.
Some functions on the client side are
MCCIInitialize(void) MCCIConnect(serverAddr, port, callback, callbackData) MCCIPoll(serverPort) MCCISendAnchor(serverPort, on, callback, callbackData) MCCIDisconnect(serverPort)Two of these functions take callback functions as arguments. The callback to MCCIConnect() is called when either the server or client disconnects. This can be at some arbitrary time, and allows the client to clean up properly.
MCCISendAnchor() registers a callback with the library that is called whenever the server fetches a new anchor and the client wants to know about it. The server sends a message 301 ANCHOR <url>. The CCI library gets this. When MCCIPoll() executes, it sees that there is a message and calls the callback function which must be of type
void callback(char *anchor, void *callbackData)
The server side of the library is not public, since it was only meant to be used by Mosaic. It is pretty grotty, heavily tied up with Mosaic GUI code, etc.
ICE gives connection setup, consisting of byte order exchange, authentication and connection information exchange. Then protocol negotiation is done. This agrees on the protocol used, and may also include authentication (multiple protocols may share a connection). This also registers a callback function that is used to handle any of the messages of that protocol. When a message arrives this callback function, belonging to the application, is called to process the message. There are a variety of functions to read message data of various sizes.
Look at a simple non-client-server program that allows you to list files in a directory, change directory and copy files. The pseudo-code would be
read line from user
while not eof do
if line == dir
list directory
else
if line == cd <dir>
change directory
if succesful
print new directory name
else
if line == copy <file>
copy file
else
if line == quit
quit
else
complain
read line from user
In a CS situtation, the client would be at the user end, talking to a server
somewhere else. Aspects of this program belong solely at the presentation
end, such as getting the commands from the user. Some is messages from the
client to the server, some is at the server end.
For a simple file transfer, assume that all files are at the server end, and we are only transferring ASCII files from the server to the client. The transferred file is to have the same name as the original file. The client side (including presentation aspects) will become
read line from user
while not eof do
if line == dir
list directory
else
if line == cd <dir>
change directory
if succesful
print new directory name
else
if line == copy <file>
copy file
else
if line == quit
quit
else
complain
read line from user
where the italicised lines involve communication with the server.
change dir button:
if there is a selected file
change directory
if successful
update directory label
list directory
update directory list
get file button:
if there is a selected file
copy file
The client needs to be able to ask for a directory change, to a directory that it specifies. If the client asks the server to something based on server state, then the request may be honoured or fail. The client needs to be told of this. So this needs a reply, of success or fail.
The purpose of this CS setup is to transfer files. So the client needs to be able to send a "get" message. The file may or may not exist, and may have restrictive permissions. So there is a question of whether or not the transfer will go ahead, and if ok, how it takes place.
It may be convenient for the client to be able to determine which is the current directory on the server.
In the first mechanism, any state would have to be maintained by the client, and sent anew on establising the connection. In the second, the server could maintain state that lasts for any one session. Is there any state info required? Yes - the current directory on the server.
If connections are made and dissolved on a per-message basis, then multiple clients could talk "simultaneously" to a single server. If a client holds the connection for a session, then other clients cannot use that port until the session is over. There is no queuing mechanism. If this is a problem, then each session should be done using a slave server.
client server
------ ------
dir
send list of files
terminate with blank
line
cd <dir>
change dir
send blank if failed
send newdir if succeed
get <file>
send file
Here is the client program
/* TCP client that finds the
time from a server */
#include <stdio.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <sys/param.h>
#define SIZE 1024
char buf[SIZE];
#define TIME_PORT 2013
#define MAX 1024
char *GetLine(int fd)
{
static char line[MAX];
static char netread[MAX] = "";
int n, len;
char *p;
len = strlen(netread);
/* look for \r\n in netread buffer */
p = strstr(netread, "\r\n");
if (p == NULL) {
/* fill buff - no \r\n found */
n = read(fd, netread+len, MAX-len);
len += n;
netread[len] = '\0';
p = strstr(netread, "\r\n");
if (p == NULL) {
/* still cant find line terminator,
discard this one and try again
*/
netread[0] = '\0';
return GetLine(fd);
}
}
*p = '\0';
strcpy(line, netread);
/* copy rest of buf down */
memmove(netread, p+2, strlen(p+2)+1);
return line;
}
void SendDir(int fd)
{
int n;
char *s;
write(fd, "dir\r\n", 5);
/* read response */
while (1) {
s = GetLine(fd);
if (*s == '\0')
break;
printf("%s\n", s);
}
}
void ChangeDir(char *dir, int fd)
{
int n;
strcat(dir, "\r\n");
write(fd, dir, strlen(dir));
/* read response */
n = read(fd, buf, 1024);
write(1, buf, n);
}
void GetFile(char *file, int fd)
{
int n;
strcat(file, "\r\n");
write(fd, file, strlen(file));
/* read response */
n = read(fd, buf, 1024);
write(1, buf, n);
}
void HandleMsgs(int fd)
{
char buff[MAXPATHLEN + 4];
while (gets(buff) != NULL) {
if (strncmp(buff, "quit", 4) == 0) {
exit(0);
}
if (strncmp(buff, "dir", 3) == 0) {
SendDir(fd);
} else
if (strncmp(buff, "cd ", 3) == 0) {
ChangeDir(buff, fd);
} else
if (strncmp(buff, "get ", 4) == 0) {
GetFile(buff, fd);
} else {
fprintf(stderr, "illegal command\n");
}
}
}
int main(int argc,
char *argv[])
{
int sockfd;
int nread;
struct sockaddr_in serv_addr;
if (argc != 2) {
fprintf(stderr,
"usage: %s IPaddr\n",
argv[0]);
exit(1);
}
if ((sockfd =
socket(AF_INET,
SOCK_STREAM, 0))
< 0) {
perror(NULL);
exit(2);
}
serv_addr.sin_family =
AF_INET;
serv_addr.sin_addr.s_addr =
inet_addr(argv[1]);
serv_addr.sin_port =
htons(TIME_PORT);
if (connect(sockfd,
&serv_addr,
sizeof(serv_addr))
< 0) {
perror(NULL);
exit(3);
}
HandleMsgs(sockfd);
close(sockfd);
exit(0);
}
Here is the server
#include <stdio.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <ctype.h>
#include <sys/dir.h>
#include <limits.h>
#include <unistd.h>
#include <sys/param.h>
#define SIZE 1024
char buf[SIZE];
#define TIME_PORT 2013
#define MAX 1024
char *GetLine(int fd)
{
static char line[MAX];
static char netread[MAX] = "";
int n, len;
char *p;
len = strlen(netread);
/* look for \r\n in netread buffer */
p = strstr(netread, "\r\n");
if (p == NULL) {
/* fill buff - no \r\n found */
n = read(fd, netread+len, MAX-len);
len += n;
netread[len] = '\0';
p = strstr(netread, "\r\n");
if (p == NULL) {
/* still cant find line terminator,
discard this one and try again
*/
netread[0] = '\0';
return GetLine(fd);
}
}
*p = '\0';
strcpy(line, netread);
/* copy rest of buf down */
memmove(netread, p+2, strlen(p+2)+1);
return line;
}
void SendDir(int client_fd)
{
DIR *dirp;
struct direct *dp;
char buff[MAXPATHLEN + 3];
dirp = opendir(".");
for (dp = readdir(dirp); dp != NULL; dp = readdir(dirp)) {
strcpy(buff, dp->d_name);
strcat(buff, "\r\n");
write(client_fd, buff, strlen(buff));
}
strcpy(buff, "\r\n");
write(client_fd, buff, 2);
}
void ChangeDir(char *dir, int client_fd)
{
char buff[MAXPATHLEN + 3] = "";
/* remove leading blanks */
while (isspace(*dir))
dir++;
if (chdir(dir) == 0)
/* success */
getcwd(buff, MAXPATHLEN+1);
strcat(buff, "\r\n");
write(client_fd, buff, strlen(buff));
}
void SendFile(char *filename, int client_fd)
{
/* remove leading blanks */
while (isspace(*filename))
filename++;
}
void HandleMsgs(int sockfd, int client_sockfd)
{
char *s;
s = GetLine(client_sockfd);
if (s == NULL)
return;
if (strncmp(s, "dir", 3) == 0) {
SendDir(client_sockfd);
} else
if (strncmp(s, "cd ", 3) == 0) {
ChangeDir(s+3, client_sockfd);
} else
if (strncmp(s, "get ", 4) == 0) {
SendFile(s+4, client_sockfd);
} else {
/* error stub */
}
}
int main(int argc,
char *argv[])
{
int sockfd, client_sockfd;
int nread, len;
struct sockaddr_in serv_addr,
client_addr;
time_t t;
if ((sockfd =
socket(AF_INET,
SOCK_STREAM, 0))
< 0) {
perror(NULL);
exit(2);
}
serv_addr.sin_family =
AF_INET;
serv_addr.sin_addr.s_addr =
htonl(INADDR_ANY);
serv_addr.sin_port =
htons(TIME_PORT);
if (bind(sockfd, &serv_addr, sizeof(serv_addr)) < 0) {
perror(NULL);
exit(3);
}
listen(sockfd, 5);
for (;;) {
len = sizeof(client_addr);
client_sockfd =
accept(sockfd,
&client_addr,
&len);
if (client_sockfd == -1) {
perror(NULL);
continue;
}
for (;;)
HandleMsgs(sockfd, client_sockfd);
close(client_sockfd);
}
}
D. H. Crocker Standard for the Format of ARPA Internet Text Messages IETF RFC 822
N. Borenstein A User Agent Configuration Mechanism for Multimedia Mail Format Information IETF RFC 1524
N. Borenstein and N. Freed MIME Part One: Mechanisms for Specifying and Describing the Format of Internet Message Bodies IETF RFC 1521
CCI specification http://www.ncsa.uiuc.edu/SDG/Software/XMosaic/CCI/cci-spec.html
The IETF RFC's may be obtained from ftp://ietf.org/internet-drafts/ or http://www.garlic.com/~lynn/rfcietf.htm