Status of the Ghosts System's Development

This document was updated Fri Sep 9 13:10:27 MET DST 1994
fk@dsv.su.se

Diary

Overview

The Ghosts system is written in C++ and uses the X11R5 window system with the Motif widget set. In order to enable the full use of C++ throughout the code, a C++ wrapper for the Motif and X11 instrinsics called Motif++ is used. Ghosts was originally written under SunOS and has been recompiled under Ultrix with no (major) ill effects.

The Ghost system consists of four distinct pieces:

mailghost
An email daemon, similar in spirit to procmail, which filters incoming mail according to the user's specification. The filtering specifications are in the form of rules.
nnghost
Either an interface between the user and the nn netnews reader, or a hacked version of the nn reader itself.
ruled
A rule editor. The rules are supposedly used by the mailghost and the nnghost programs.
grouped
A graphically based newsgroup editor. Neither sources nore binaries are yet available.

State of Development

Sources

The sources for the three entities were sequestered from the home directory of ollep@sics.se (Olle Palmgren). C and C++ sources for the three sources were found to be mixed in the same directory. The Makefile, however, contains enough information to separate the components.

Include-files were found in separate directories, and were therefore not always immediately recognized as important.

Auxilliary Software

All three modules rely heavily on the use of code generators for the scanning and parsing of messages. Hence, flex (fast lex) and bison (gnu's version of yacc) are required. It was discovered that non-standard versions of these tools had been used and an initial effort had to be spent on the compilation and installation of flex++ and bison++. Apparently, the versions used generate code for the C++ compiler, while claiming to be fully backwards compatible with the extant versions of flex and bison.

Since the special versions of flex and bison have not yet gained wide-spread recognition they must be considered as essential parts of the Ghost sources.

Platform Dependencies

The prototype system was developed for SunOS and the Motif library and window manager. When Fredrik Kilander took over the sources Motif was only available on the Ultrix system. As a result, the software was ported to Ultrix. The necessary modifications were few:

Sound Play

The Ghost system has provisions for playing a sampled sound through the workstation's loudspeaker. Under SunOS this is almost trivial. In the DEC Ultrix environment the ability to play a sound at the console requires the linking of a multimedia library, purchaseable as a separate product, and not unlikely with considerable inherent complexity. All in all, the option of playing a sound may well be considered a redundant feature.

Makefile

The Makefile is built for gnu make. It originally used automatically generated dependencies, but not to the extent recommended by gnu for full automatic maintenance. Instead they must be cut from the output of the make dependencies command and pasted in at the end of the makefile.

The auto-dependencies contained references to directories which were inaccurate on the new system and were therefore removed. With their absence emerged the fact that goals in the makefile were sequenced in the wrong order. Certain goals were rearranged to reflect the needed compilation sequence. In particular this happened for targets which employed code generators, such as flex++ and bison++.

Compilation Issues

Target MAILGHOST

The target MAILGHOST has compiled successfully.

Target NNGHOST

The Makefile specifies -traditional to gcc. This clashes with certain keywords in later version of C, located in the system's header files. The explanation for the conflict lies in the fact that these keywords are missing from the SunOS version of the file. The conflict appears with the first file, active.o. In conclusion, you can't use -traditional under mips-dec-ultrix.

Compiling without traditional then stumbled on conflicting definitions of ftell. The file global.h appears to be part of other software, the author is stated as one Kim Fabricius Storm. In any event, the gcc stdio.h header declares ftell() to be of type long while global.h says that it should be off_t which is defined as an int. The solution here is to make the declaration of ftell() in global.h conditioned on the symbol __sun__.

These were only warnings. NNGHOST compiled successfully.

Trying to run it, it complained about the MASTER file. Must try out what it is supposed to contain. Documentation? None. I am fetching nn version 6.4.18 just to get something to read...

The main() procedure is located in the file nnghost.cc. It just calls nn.c:cmain().

Ruled

Ruled has compiled successfully using the X11R5 libraries supplied by DEC and a Motif++ library wrapper for Motif 1.1.

Diary

Mon Aug 15 15:18:52 1994 edit by Fredrik Kilander

I am trying to recompile the Ghost system after having built a version of XMlib.a, the C++ wrapper library for Motif. The library was built for X11R5 and Motif 1.1.

Anyway, the ghost is tricky to get through compilation. It appears that I have added /usr/include to the include-path explicitly and this creates a number of collisions as the NULL symbol is being redefined. I must try to remove this reference.

The Makefile also states that for MAILGHOST, emailLexer.o should be built before emailParser.o. However, this can't be since emailParser.o generates a header file emailParser.hh which is needed by in order to compile emailLexer.cc. I have reversed their order in the Makefile and commented out the culprit line. It also appears to be the case that emailRuleParser.o should be achieved after emailRuleLexer.o.

As a side note, comments appears to be persistent accross quoted newlines.

Mailghost compiled (with a bucketful of warnings). The next target, nnghost, went belly up on a syntax error involving var_args. I will now try to remove the explicit specification of /usr/include.

Things appears to go better without it. Well, appears was the word. Nnghost was built without too much of an ado.

For ruled things was worse. Things came to a stop in ruled.o. A lot of complaints about the X11 header files. I am trying to add /usr/local/X11/include to the include path as well. Yes, that worked better. Next stop was readEmailEditRule.cc who tried to include emailRuleEditLexer.hh which didn't exist. Hmm, all this crap about targets in the wrong order probably stems from me deleting the automatically generated dependencies.

I tried to reinsert them, but of course it doesn't work until the headerfiles who some component is dependent upon is present. There were also a few warnings. Its a bootstrap process and I think that for the generated headers the dependencies cannot be properly inferred. Either that, or I've munged the Makefile beyond repair. Now I'll try to remove the auto-dependencies and put the targets in another order.

Ok, that appears to have worked. Now I'm back where I started this afternoon, with a bunch of undefined symbols from the loader. Something is missing...and it quickly turned out to be a lowercase m, as in -lXm which should be -lXM. Now only two undefined symbols remain, and these are XtShellStrings and XtStrings.

These symbols appear in /usr/local/X11/include/X11 Shell.h and StringDefs.h. They do NOT appear in /usr/include/X11 Shell.h and StringDefs.h. There's a confusion here:

The symbols DO appear in the MIT distribution of X11R5.

/src/Packages/X11R5/dec/mit/lib/Xt/libXt.a	1.863.844 Mar 27  1993

--------------------

/usr/lib/libXt.a				1.732.204 Aug  8  1992

/usr/lib/DXM/lib/Xt/libXt.a			1.645.962 Aug 12  1991

/usr/local/X11/lib/libXt.a			1.861.024 May  3  1992

I am trying with the last one. Didn't work:
Undefined:
XUniqueContext
_XtAddCallbackForOtherWidget
I18nGetLocaleString
I18nGetLocaleCharset
I18nGetLocaleCharsets
I18nCreateDefaultFontList
I18nHasSubstring
I18nCSConstructInit
I18nCSConstructEnd
I18nCSConstructSegment
I18nCSConstructLine
I18nCvtCStoFCInit
I18nCvtCStoFCEnd
I18nCvtCStoFCSegment
I18nCvtCStoFCLine
I18nCvtCStoOSInit
I18nCvtCStoOSEnd
I18nCvtCStoOSSegment
I18nCvtCStoOSLine
I18nCvtFCtoCS
I18nCvtOStoCS
_Xt_LoadQueryFont
LowerCase
_XtAddPDDestroyCallback

Trying to recompile the motif++ wrapper using the headers in /usr/include and libs in /usr/lib although I have a gut feeling this won't work.

Well the library built (with some formal complaints). Lets see how the ruled is coming along. It still doesn't find those symbols.

I will now try to rebuild it using /usr/include instead. This loses quite horribly. Maybe because the header files in /usr/include/X11 are unprepared for C++. But how come the wrapper compiled ok?

Other probable causes for the missing symbols are missing source files in the Motif++ library, or that it should be built for Motif version 1.2 and not 1.1 which is what appears to be on /usr/include/Xm. The files on /usr/lib/DXM/lib/Xm doesn't state their version.

With a little help from Patrik it finally compiled. The secret was to use the DEC libraries and headers. Silly that.

Ruled appears to be an empty shell. Nothing happens and the panes are emtpy. This may be the state of the art. Mailghost starts and then terminates after a few seconds without any diagnostic. Nnghost terminates at once with the following message: Cannot open file /tmp/nn/MASTER, mode=0x80, errno=2 This error is probably caused by the fact that it expects to be on a system where nn is installed.

ruled.cc

This short file contains the entry point main(). It simply creates an application shell and a main window beneath it. A menubar and a work area are created under the main window and the menu bar registered with the main window. Then the shell is realized and application's main loop entered. Very clean.

create.cc

This file appears to create the widgets in the main window as well as establishing accellerator bindings and so on.

Rules

The editor reads rules from two files, ~/.mailghostrc and ~/.nnghostrc. It does this on startup and when requested. When rules are saved they are saved in the files ./nn.rules and ./email.rules respectively.

Syntax

The syntax for News-rules is shown below (according to nnRuleEditParser.y). Spaces and newlines should not be significant. Case is not significant except in strings.
RULE STRING
IF stmt
SET variables
DO actions
END

stmt := '(' stmt ')'
        stmt AND stmt
	stmt OR  stmt
	expr EQ  expr
	expr NEQ expr
	expr CONTAINS expr
	FLAG string

expr := string
	query

string := "..."

query := FIELD string

variables := variable variables

variable := FLAG STRING

actions := action actions

action := KILL
	  SELECT
Here's an example file:
rule	"ONE"
if
	"FOO" == "BAR" 
do
	kill
end

rule	"two"
if
	field "From" contains "Arthur" 
do
	select
end