1. The different types of clients
1.1. The DOC client
1.2. The YAWC CLient
2. Connecting
2.1. DOC
2.2. YAWC
2.3. Syncronization (YAWC and DOC)
3. Local modes (YAWC and DOC)
3.1. Basics
3.2. Getting names
3.3. Getting lines
3.4. Getting posts
3.5. Getting X messages and profiles
3.6. Client configuration
3.7. File editing (YAWC-only)
4. Online userlist handling
4.1. Basics
4.2. DOC
4.3. YAWC
5. Other BBS-client communication (YAWC and DOC)
5.1. eXpress message marking
5.2. Post marking
5.3. Idle / cutoff detection
6. Disconnecting (YAWC and DOC)
7. YAWC protocol extentions
7.1. Basics
7.2. Implemented extentions
A. Appendix A: DOC and YAWC color codes
B. Appendix B: A short history of the clients
B.1 DOC
B.2 YAWC
Chapter 0: introduction
Since nobody bothered to make a protocol description for DOC (ISCA) and YAWC
type BBS clients here's mine. It's re-engineered from the code so I can't
guarantee any correctness but promise to try to do this as accurate as
possible.
I'll start with a short introduction about the several clients, their backgrounds and their basic differences as far as the protocol is concerned. After that I'll describe what happens when a client tries to connect, when it is connected and when the connection is being closed. The last part will describe some extensions to the YAWC protocol and show how new features can be implemented without breaking older clients.
- Sven Winnecke aka. Flint, Chemnitz, Germany, october 1997
Chapter 1: The different types of clients
DOC style BBSs that support special clients usually catch them at port 23 which is also used for normal telnet connections. Once the connecting program is recognized as BBS client instead of a plain telnet client the BBS goes into a special client mode which among other things will send certain control sequencies to the client so it could act upon them in a semi-intelligent way.
Another type of BBSs that has been designed to be as compatible as possible to the DOC look and feel is the ABC code. It also supports DOC compatible clients but listens for them on port 4123. this allows to run the BBS on a host where the standard telnet daemon cannot be changed to an own daemon which would log the user straight into the BBS and also recognizes BBS clients. Apart from that all that will be said about DOC clients applies to ABC as well.
1.2. The YAWC CLient
In 1994 (or 1995?) Kenneth Haglund aka. KHaglund, author of the YAWC code,
took Serendipity's client and made some changes on it to fit it to the needs
of a YAWC BBS. It lost some functionality (like friend and enemy lists) and got
some new ones (like multi session capability).
After him Peter Roozemaal aka. MathFox from Monolith BBS took over, he removed
some bugs, added improved color support and generally made the code run
smoother a bit. After him Dave Cridland (Diamond White) made some improvements
including cursor support for the line input routine and finally after him I
made the most recent changes including making it compatible to DOC and ABC
again (without loosing YAWC CLient capabilities of course ;). MyLittlePonyTail
wrote a client for MS Windows. Some other people like Flex Able and the
programmers at SonCrest did some client coding as well.
For some odd reasons the YAWC BBS client is often referred to as CLient and
the CLient derived from MathFox/Diamond White/myself was called CLientPro as
of version 1.3.
CLients usually connect to port 1976 instead of 23. this makes it easier to install the BBS with only a minimum of root admin involvement. At the moemnt KHaglund is working on using port 23 for CLients again so the latest code might require that again finally.
The client _should_ respond to these negotiation with a corresponding answer (it should answer a IAC DO option with a IAC WONT option and to a IAC WILL option with a IAC DONT option), but most DOC clients just ignore the negotiation sequences completely. ISCA seems to work okay with that but normal telnet daemons won't. The CLientPro will do the responses, as do other DOC clients that also support telnet mode.
Before this all starts the client sends some bytes on it's own. The sequence looks as follows: IAC, CLIENT2, IAC, SB, TELOPT_ENVIRON, 0, 1, "USER", 0, <user>, IAC, SE This will identify the program as client compatible to v1.5 and pass the user name as stored in the environment variable USER to the BBS.
After that the terminal size should be send in the NAWS format:
IAC, SB, TELOPT_NAWS, 0, 0, 0, rows, IAC, SE.
If that is sent the client will process the telnet IACs as described above.
Finally the BBS will send a IAC START which the client has to answer with a
IAC START3 to identify itself as DOC client compatible to version 1.5 earlier
clients would send IAC START or IAC START2 but those will be rejected by the
BBS for being too old. All those values are also to be found in the telnet.h
file of any client source.
After the client received a IAC START it also sets it's synchronization byte
to 0. This and it's function will be explained in chapter 2.3.
After that the BBS will send the login screen and then the login prompt. It
will then send IAC codes to get username and password, this will be described
in chapter 3.
The ABC code _always_ sends this sequence first:
IAC DO 0x24 IAC DO 0x1 IAC DO 0x1F IAC DO 0x2C
IAC WILL 0x3 IAC WILL 0x1 IAC DO 0x2E
and it _always_ expects a IAC CLIENT2 as first bytes to be sent by the client
or it will just hang!
Summary of the network traffic: |
2.2. YAWC
YAWC's CLient connections usually go to port 1976. Usually there's no telnet
daemon sitting at that port so the CLient wouldn't need to worry about telnet
IAC sequences. But since it's derived from the ISCA client it usually still
has the code in it that ignores those codes. Some clients however are well
capable to deal with them properly.
Right after connecting the BBS expects a IAC CLIENT to be sent by the CLient. After that a NAWS sequence like this: IAC, SB, TELOPT_NAWS, 0, 0, 0, rows, IAC, SE should be sent to the BBS. Older YAWCs won't require that (but accept it silently), but younger YAWC BBS versions will hang until you hit a key if they don't get that sequence.
That's it. Now also the YAWC BBS will send IAC START to do the synchronizazion stuff and the login screen and then will ask the user for username and password.
Summary of the network traffic: |
2.3. Synchronization
To avoid problems that can be caused by net lag and bad modem lines a simple
but efficient synchronization method has been implemented into the BBS client;
it works the same way for both DOC and YAWC clients.
Both the BBS and the client have a sync variable which holds the current idea of the BBS and the client about how many bytes have been sent by the client to the BBS. it's an 4 byte although only the lower 3 of them are used. 3 bytes provide a number > 16 million, so someone would need to transfer more than 16 MB of data to the BBS in one session which should be highly unlikely.
(on the other hand if someone would manage to do so then he'll be in trouble anyway because the sync bytes can't be syncronized anymore. and he'll probably also have a lot of BBS admins running and shouting after him too ;)
At some time during the login procedure the BBS will send a IAC START sequence. This causes the client to set it's sync value to 0. The BBS does the same right after sending that sequence. Now with each byte the client sends and the BBS receives both ends increment their couters _EXCEPT_ the data sent are IAC sequencies. They are just sent but not counted.
The check if both synchronization bytes (BBS and client) are still identical is done every time the BBS sends an input request that will cause the client to enter a local mode. The next chapter does a further description about that.
To be able to correct some possible sync problems the BBS holds a buffer of a certain amount of the most recent data bytes that have been sent. most clients use 1 kB (1024 bytes) for that.
After the request for the client to go into such a local input mode the BBS sends it's idea of the current sync value which the client then compares with it's own value. Depending of the result the client does the following:
byte1 + 256*byte2 + 65536*byte3. (bytes 1-3 are ordered as received).
Summary of the network traffic: |
Chapter 3: Local modes (YAWC and DOC)
3.1. Basics
Both DOC and YAWC have the same basic mechanism to let the clients know when
to do something locally. they differ in some details though.
One of the big advantagdes of having a BBS client instead of plain telnet is the client's ability to do some stuff on it's side without bothering the BBS. This cuts network load, lag and BBS machine load as well.
In order to do so the BBS needs to tell the client when to go into such a local input mode and what to do and how to do it. This is done by using IAC sequencies again. These sequencies look like this:
IAC <what_to_do> <option> <sync_low> <sync_med> <sync_high>
After sending these bytes the BBS goes into a waiting position ignoring every data from the client until the client sends a IAC BLOCK to tell the BBS that it is now ready to transmit the results of the input. This technique solves some problems that arise from net lag and any data that might be on it's way back to the BBS while the BBS already sent the client into local mode.
The client should NOT send IAC BLOCK if it should do client configutation (G_CONFIG).
After sending IAC BLOCK and the data that has been read from the user the client needs to let the BBS know that it's done with sending for now. the way the client lets the BBS know this depends on which routine has been called, see the subsequent chapters.
Summary of the network traffic: |
3.2. Getting names
To read a user name the BBS sends this sequence:
IAC G_NAME <quit_priv> <sync_low> <sync_med> <sync_high>
quit_priv can have the following values:
It returns the name and appends a newline (\n) character to mark the end of the name.
Summary of the network traffic: |
3.3. Getting lines
To read a single line the BBS sends this sequence:
IAC G_STR <length> <sync_low> <sync_med> <sync_high> -or-
IAC G_STRWR <length> <sync_low> <sync_med> <sync_high> (YAWC-only)
After the client has received that command it will read a line of <length> characters locally. if the length is negative then the absolute amount of the value will be taken for the maximum length of the input and the entered characters won't be printed on the screen but replaced by "*"'s (shadowed) so that this is used for getting passwords.
Like G_NAME the client will send a \n character to mark the end of the line.
The described behaviour applies to G_STR. G_STRWR (a YAWC-only feature) will nearly act the same way but will "wrap" the last word into a buffer. This means that if the user enters a line and the line limit is reached the last word will be removed again (except the last chacter was a space character) and stored in a buffer. now if G_STRWR is called anew the wrapped word is being placed on the start of the new line.
Note: The code for G_STRWR is 0xa6, the same that DOC uses for S_WHO. Don't mix that. Also, YAWC's WHO_S code (183) is not the same as DOC's S_WHO.
Summary of the network traffic: |
3.4. Getting posts
To get a post the BBS sends this sequence:
IAC G_POST <upload> <sync_low> <sync_med> <sync_high>
upload can have th following values:
After finishing the post the client sends back to the BBS
Summary of the network traffic: |
3.5. Getting X messages and profiles
To get a post the BBS sends this sequence:
IAC G_LINES <which> <sync_low> <sync_med> <sync_high>
On YAWC BBSs <which> can have the following values:
It seems to be safe to assume that (which & 1) == 1 means "x message" and (which & 1) == 0) means "profile".
On DOC BBSs the which value is a series OR'd bits to allow or disable certain X overrides but I have been assured that most of them are obsolete. So it seems that relying on the values 1 and 2 will be sufficient.
Another difference between DOC and YAWC is how lines are handled. DOC _always_ only accepts 5 lines for both X messages and profiles. YAWC's limitations are more flexible:
Summary of the network traffic: |
3.6. Client configuration
To cause the client to do client configuration the BBS sends this sequence:
IAC G_CONFIG <option> <sync_low> <sync_med> <sync_high>
<option> will be ignored.
After finishing the client just sends a newline (\n) back to the BBS to indicate that it finished configuring. Note that _no_ IAC BLOCK is sent although at least YAWC BBSs will usually accept (and ignore) IAC BLOCK here.
Summary of the network traffic: |
IAC FILE_S <file contents> IAC FILE_E3.7. File editing (YAWC-only)
To allow a client to edit a file (for example a roominfo locally on the
client side this sequence will be sent:
IAC EDIT_S
The client will read <file contents> into a temp file and upon EDIT_S it will start the editor to allow the user to edit the uploaded data.
After finishing the client sends IAC BLOCK back, then sends the file contents (which mustn't contain NULL (\0) bytes and appends a \0 to mark the end of the file.
Only up to 47800 bytes are accepted by the BBS.
Summary of the network traffic: |
4. Online userlist handling
4.1. Basics
The client is able to download a online user list from a DOC or YAWC BBS.
This can be used for several purposes like displaying friends (and enemies)
complared to local lists, for TAB name expansion and so on.
DOC and YAWC differ a lot at this point. This is mainly because KHaglund understood the way the client processes wholists differently than it was intended.
4.2. DOC
The main purpose of the online user list download is to compare the names with
friends listed in the .(yawc)bbsrc file to be able to spot them.
Some clients already offer TAB expansion of usernames in the friend list, the
CLientPro also supports to expand _any_ name after calling a <ctrl-w> list.
after the client sends a <ctrl-W> to the server the BBS answers with an IAC
S_WHO and then a series of one-byte flags and null-terminated user names.
The flags mean:
Note: The code for S_WHO is 0xa6, the same that YAWC uses for G_STRWR. Don't mix that. Also, YAWC's WHO_S code (183) is not the same as DOC's S_WHO.
Summary of the network traffic: |
4.3. YAWC
YAWC's approach is very different. The main purpose here is to fill a TAB list
so that the user should be able to do TAB name expansion for all users online
at that time after reading a wholist. Since many YAWC BBSs have their internal
friendlists that function of the CLient had been deacticated for a long time.
(I finally partly reactivated them to allow the user to always expand friend's
names even before pulling a wholist and also after that.)
YAWC doesn't have just a name transferring feature as DOC uses when the user hits <Ctrl-W> to see who of his friends is there. Instead the real <w>holist (usually the long version) is used. This means the user _has_ to hit <w> once to be able to TAB expand any online user names.
The BBS will tell the CLient that a wholist is to follow now by sending IAC WHO_S.
The name marking in the wholist lines has changed with time so there are now two possibilities. right before the username there can either be a - <Ctrl-D> or a - \r (carriage return) to mark the beginning of the name. The end of the name is determined as follows: The CLient will scan 20 characrers starting after <ctrl-D> and <\r> and then remove all trailing spaces. So the BBS _has_ to transmit at least ((username length) - 20) spaces after the user name. After that the rest of the wholist line will follow. After the last wholist line has been transmitted the BBS will send IAC WHO_E.
Summary of the network traffic: |
5. Other BBS-client communication (YAWC and DOC)
5.1. eXpress message marking
DOC and YAWC BBSs let the clients know when an X message is being transmitted.
This may be used to filter out unwanted messages by having the names of the
senders on the client's enemy list. YAWC doesn't make much use of it but the
'x killing' feature now works again since the latest version of the CLientPro.
The format is pretty simple: the BBS will send an IAC XMSG_S right before transmitting header and body of the x message and IAC XMSG_E right after it. no reaction from the client is expected by the BBS.
Summary of the network traffic: |
5.2. Post marking
DOC BBSs mark the transmitted posts similar to the express messages as
described in the previous chapter. The reason also is to allow the client to
do some filter function.
The protocol is only slightly more complicated than that one for x messages: the BBS sends a IAC POST_S before and IAC POST_E after the post. The client notifies the BBS if it kills a post. for that every received post is counted (starting value set by IAC START is 1) and the lowest byte of that number (numposts&0xff) is being sent back to let the BBS know which post has been killed. This is to tell the BBS that if it's on a -more- prompt of a killed post it should abort the post transfer and not wait for key strokes for the -more- prompt. The whole sequence is
IAC POST_K (numposts&0xff) 17.
Most YAWC BBSs don't send the IAC POST_S and IAC POST_E seqeuncies so the CLient can't kill posts. The CLientPro is however prepared for that so that as soon as those IACs will be implemented into the BBS that feature will also work for them (some YAWCs already did so).
The CLient should _not_ send the killing notification to a YAWC BBS, it doesn't expect one and will most likely act weird if it receives one.
YAWC BBSs should _only_ send POST_S and POST_E after that's negotiated with the CLient to not break older CLients - see chapter 7 for details.
Summary of the network traffic: |
5.3. -More- prompt marking
To allow the client to capture posts and other data but excluding any --more--
prompts that may turn up they are enclosed by IAC MORE_M bytes. This is true
for both DOC and YAWC. The BBS doesn't expect any special answer by the client.
Summary of the network traffic: |
5.4. Idle / cutoff detection
DOC BBSs send a IAC CLIENT sequency to the client every 10 minutes to see if
the client is still connected the client should response with a IAC CLIENT too.
YAWC BBSs don't have a standard way to detect cutoffs, no IAC CLIENTs are sent usually. Thanatos once made an attempt to detect if the client is idle by sending IAC CLIENT and the CLient would only respond if the user wasn't idle for more than 6 minutes but this hasn't be used anywhere else, especially since this way of doing it wasn't very pretty. Another way has been found, see chapter 7 to find out more about that.
Summary of the network traffic: |
Summary of the network traffic: |
5.6. DOC and YAWC colors
Some DOC BBSs implemented a feature to allow users to use own colors at some
places like doings and posts. The color notation is a two character sequency.
The first is a '@', the second is usually the first letter of the desired
color. upper case letters indicate foreground colors and lowercase letter
background colors. to actually print a '@' two @'s ('@@') have to be used.
For a complete listing of those codes see appendix A.
This is only remotely interesting from the client perspective because the BBS generates the ANSI color code and sends it to the client. YAWC BBSs however usually sent the YAWC color code to the CLient and that client then transforms that into a color. that's why the CLient can switch on and off colors itself. The YAWC color sequency looks as follows: the first character is a <Ctrl-A>. after that the color code follows, as well usually the first letter of the color. Unlike DOC the _lowercase_ letters change the foreground colors and the _uppercase_ letters change the background. Also see appendix A for details.
Summary of the network traffic: |
5.7. Terminal size change
If the BBS client's terminal is being resized the terminal should generate a
SIGWINCH signal which is catched by the clients which then send a NAWS
sequency to the BBS to tell it about the size change. This NAWS code complies
to RFC 1073. This is what is sent:
IAC SB TELOPT_NAWS 0 0 0 <rows> IAC SE
row is the current (new) number of rows the terminal has.
So it's the same code that is also sent at connecting time by the client. The BBS shouldn't answer to this message except changing the screen size setting for the user.
Summary of the network traffic: |
5.8. YAWC name marking
To be able to fill the CLient user's TAB list even if no <w> is pressed
every time the CLientPro supports name marking. Fake YAWC color codes are
used for that: Right before a username a <Ctrl-A><n> is sent and a <Ctrl-A><N>
right after it. The text between them is considered a username by the CLient
and added to the TAB list. If names in post headers and X message headers
this enables to TAb expand the names of users that just sent messages to the
recipient and who posted messages.
This is a later extension of the YAWC protocol, only few BBSs support it but
it won't break older CLients in any case.
Summary of the network traffic: |
6. Disconnecting (YAWC and DOC)
There's no real standard logout negotiation. If either side closes the sockets
so that the other side will get an error reading or writing that side will
just close the session as well.
However newer YAWC CLients will send a IAC DO TELOPT_LOGOUT seqeuncy to the
BBS right before closing the connection, as specified in RFC 727. This might
give the BBS the chance to close the connection properly but after Thantos BBS
went down I don't think any BBS makes use of that at the moment.
Summary of the network traffic: |
7. YAWC protocol extentions
7.1. Basics
To make it possible to extend the YAWC CLient protocol without breaking older
clients a feature negotiation has been developed. It goes as follows:
The BBS sends a IAC DO CLIENT_OPTIONS to the client. old clients will just
ignore this, negotiation-aware ones will respond with IAC WILL CLIENT_OPTIONS.
The BBS can then go on asking the CLient to enable (or disable) certain
extentions by issuing
IAC SB CLIENT_OPTIONS <feature> ON|OFF
where feature is the item of negotiation and ON means "enable the feature" and
OFF means "disable feature".
The CLient should now decide it the requested extention is implemented and can be processed and it responds with IAC SB CLIENT_OPTIONS <feature> OK|ERROR where OK means that the client is ready to support the feature and ERROR that it is not. The BBS should not enable that feature in that case.
Note: You might notice that the usual IAC SE is missing that would be necessary according to the telnet RFCs. we just omit it because we do know exactly how many bytes are sent after the IAC SB CLIENT_OPTION sequency: 2.
Summary of the network traffic: |
7.2. Implemented extentions
These extentions are in use at some places right now:
Summary of the network traffic: |
Appendix A: DOC and YAWC color codes
Note: As mentioned earlier some DOC BBSs (including ISCA and Gestalt), don't currently
support color codes so this chapter doesn't apply for them.
| Color | DOC color code | YAWC color code |
|---|---|---|
| Foreground colors | ||
| dark (black) | @D | Ctrl-A d |
| red | @R | Ctrl-A r |
| green | @G | Ctrl-A g |
| yellow | @Y | Ctrl-A y |
| blue | @B | Ctrl-A b |
| purple | @P | Ctrl-A p |
| cyan | @C | Ctrl-A c |
| white | @W | Ctrl-A w |
| Foreground attributes (YAWC only) | ||
| reset all attributes | Ctrl-A a | |
| bright colors | Ctrl-A f | |
| underlined | Ctrl-A u | |
| inverted | Ctrl-A i | |
| Background colors | ||
| black | @d | Ctrl-A D |
| red | @r | Ctrl-A R |
| green | @g | Ctrl-A G |
| yellow | @y | Ctrl-A Y |
| blue | @b | Ctrl-A B |
| purple | @p | Ctrl-A P |
| cyan | @c | Ctrl-A C |
| white | @w | Ctrl-A W |
Appendix B: A short history of the clients
DOC unix clients:
Thanks to Ayourk for contributing this paragraph