I. 03f522 Project Ideas - RJL 031021

(RJLRef; ~/03f522RefactoringProjects.doc/.htm)

State models are now on our schedule. I require projects involving State Models because modeling behavior is essential to achieve MDA goals. One of these projects is upgrading bde/src code. Another is upgrading genv12 to generate the many changes I made to bde/pr_util/pr_*.c. A third project is to finish the bdeState project of Paul Kingston, which did not have an alpha- or Linux-compatible pr_util library from chgen (genv12 was developed concurrently with bdeState).

Other projects are applications of bde and/or LCP. They move the Hominid project toward two-person game-playing by allowing two actors to take turns editing a bde diagram (the gameboard) by moving, adding or deleting pieces (icons) at appropriate cells. Possible examples are tic-tac-toe, checkers and chess. Note that two bde processes

can communicate by connecting each one's logging output to the partner's replay input.

This requires SIMULTANEOUS logging and replay by each participant, instead of the separate log and replay phases of the Hominid project.

Other projects might involve the VC++ or Java version of bde (bde2vcpp or bde2java).

These are more or less data-compatible with bde, but they do NOT reuse code from bde, so their maintenance has always been a problem. E.g., bde2java uses ASCII pfkeys internally; it does not encode them to/from integers during import/export (equivalent to pr_load/pr_dump). Bde2vcpp does not support logging and replay. The more recent genjava code generator supports logging and replay, so bde2java could be refactored to use it. The more recent gencpp produces C++ classes and STL-lke containers from an arbitrary schema, so COOL components BDE and LCP could be refactored to use gencpp. Then gencpp would replace chgen for bde use and bde2vcpp could be refactored or merged to achieve bde code reuse.]

II. Analysis for refactoring of bdeReplay.cc and pr_log.c:

This week I have been trying to resolve bugs in bde's use with genv12 -log

with BDELOG, GENLOG and GENV12 #defined in configure.h for Imakefile use.

This requires a continuing analysis (reverse engineering) of bdeReplay and pr_log.c.

The replay_log function was designed as a state machine in bdeReplay.cc. However, its two core functions encode_token and do_command turned out to be independent of bde and Xwindows and were soon moved to pr_log.c . Now I believe the rest of replay_log can also be moved to pr_log.c so the database update process is entirely independent of the GUI (X11 or MSWin :-) and can be completely auto-generated by genv12. (Right now, bde/pr_util/pr_log.c was manually edited after genv12 generation, so more work like the project sjaganat did between genv11 and genv12.)

The source code refs below are all to $RL/pr_util and $RL/src, where $RL is ~lechner/bde2alpha_rl/sandbox/bdecheckout/bde.

(Line #s are approximate because I am gradually changing this code.)

Function pr_replay (lines 1978-2094 of pr_log.c) begins the logfile replay process. It strips off any trailing .typ suffix and appends "DB1.dat" to the logfile name.

(The logfile to be replayed was designed to have type .txt, and the checkpoint file (below) has type .dat, but I want both file type extensions to become ".bde".)

Let's call the logfile 'anyfile.txt for now. In bde, pr_init and pr_load are called first with anyfile.* as argument. Pr_load does its work in load_data (lines 2455-2554 of pr_load.c); load_data ignores the .txt or .dat suffix.

For ordinary .dat files load_data calls pr_parse and pr_link (once per fkey) on every input record to load the database. However, iff it detects the "SR" command on line 1, load_data calls pr_log to handle the replay process.

Function pr_log does its work by making the first call to function replay_log, currently in bdeReplay.cc lines 147-438 . Replay_log has a state machine model with 3 states:

STATE0, STATE1, and STATE2. (These states are cases of switch(replayState), and should have better, more meaningful enumerated names.) STATE0 does initialization, STATE1 interprets each command line, and STATE2 does cleanup. Pr_log enters replay_log only once, at its initial state; thereafter, STATE1 is entered from the Keypress callback function. Keypress (lines 440-501 of bdeReplay.cc) is registered as an Xwindows callback function at bde/src/init.cc (line 220). Keypress is small enough to be printed in its entirety: (Note Keypress is a no-op unless #defined BDEOG.)

[For a summary of Keypress (its STDiagram, state actions, and its bdeReplay client in bdeReplay.cc), see 03f522RefactoringSlides.htm and KeypressSTD.htm.]

Keypress does nothing unless BDELOG is #defined at compile time and the X11 event-type is 'KeyPress'. The pressed key (from the X11 event structure) is restricted to be the letter 's'. If this 'guard condition' on the event from X11 is satisfied:

(strcmp (XKeysymToString(keysym), "s") == 0) && (ReplayReady == 1)),

then Keypress calls replay_log() in pr_log.c to interpret the command and return replayStatus.

[Think of 's' as a SINGLE-STEP key for advancing the replay process one update transaction at a time - other ways to make X11 process log file inputs are documented in Paul Kingston's OOAD project.]

[Compare this method to the single-step operation of the BridgePoint demo URL, in which humans enter the data and step thru the prototype. Instead, here bde logs the data or other data generators like the Hominid project generate the log file. (Think of this process as simulating the environment that generates messages to LCP state machines.)]

In other words, replay_log executes ALL commands BEFORE Keypress enters its state model below. If replay_log fails (replayStatus < 0), the switch() below is skipped over, NOT entered. Any remaining actions inside the case switch that are NOT bde-specific, but apply to EVERY application that #defines GENLOG, should be moved into replay_log. This includes the switch(state) if possible.

III. Source code of Keypress callback function at end of bdeReplay.cc:

void Keypress(Widget w, XtPointer input, XEvent *event,

Boolean *continue_dispatch)

{

#ifdef BDELOG

extern int ReplayReady; //in init.cc; used in dialog.cc, bdeReplay.cc

KeySym keysym;

Modifiers dummy;

int replayStatus; /* local var? also in fileio.cc */

if(event -> type == KeyPress)

{

XtTranslateKeycode(XtDisplay(w), event->xkey.keycode,

event-> xkey.state, &dummy, &keysym);

dprintd("Replay Ready : %d\n", ReplayReady);

if ( (strcmp (XKeysymToString(keysym), "s") == 0) && (ReplayReady == 1) )

replayStatus = replay_log();

if (replayStatus < 0) { //0 if PASS, negative if FAIL

printf("Keypress: replay_log returns error code %d,ignored\n", replayStatus);

}

else {

switch(replayState) { /*New - RJL 031019: State sequence: (0 1* 2) */

case STATE0: /* "SR" command */

replayState = STATE1;

case STATE1: /* while not EOF do_command under Keypress control */

/* HERE is where bde's ReDraw should be called, after replay_log - RJL 031018 */

/* THis may be mandatory to make DBUpdate and HG-ReDraw an atomic operation */

if (command == StopCmd)

replayState = STATE2; // stop the iteration at EOF

break;

case STATE2 : /* "SP" command or EOF */

ReplayReady = 0; // ignore Keypress from now on

// or until a new "SR? (to concat. mult. replay logs?)

dprint("bdeReplay: Keypress: STATE2: End of Replay \n");

break;

default:

dprintd("bdeReplay.cc:Keypress: Invalid STATE %d \n", replayState);

}; // end switch(replayState)

} // end if replayStatus

} // end if event->type==KeyPress

#endif /* ifdef BDELOG */

} // end KeyPress

IV. Simplifiying the Keypress calback (can we move its state model to pr_log.c?)

To simplify Keypress, we must find another place for any bde-specific actions done in the early part of Keypress or inside its state model switch(replayState) which executes one of the state action code blocks on each logfile command line.

STATE0: Keypress does nothing when it enters this first state action case block. No bde-specific functions are done in STATE0. STATE0 is the value assigned to replayState by replay_log as a post-condition of detecting the command "SR", and its action is empty except to assign ReplayReady = 1. It also updates replayState to STATE1; this 'state transition' should be the last thing done in a state's action code.

STATE1: The second action block is case STATE1=1. No bde-specific functions are done in STATE1. STATE1 becomes the value of replayState on all subsequent Keypress calls except the last one. ReplayReady==1 is a pre-condition to doing anything when Keypress is called, and this is a result of finding an "SR" command on the first non-comment line of the input file.

The only state change in Keypress is to assign replayState = STATE2 iff (command ==StopCmd). The database update responsibilties of STATE1 have already been performed by calling do_command in replay_log. One performance and human factors problem remains in State1: a brute-force approach is used to update the canvas window by calling updatedisplaylist and ReDraw after each database update transaction. This is slow and it flickers the display because the interface level is too fine-grained. ReDraw could be called less often if multiple updates were performed before the screen is updated.

STATE2: This state is responsible for calling updatedisplaylist after a database update to convert the world coordinate data to device coordinates and then to call ReDraw if the visible diagram changes. Right now we use the brute-force technique of calling both of these no matter what database update command is received. Only commands that do NOT affect the display of the diagram are ignored.

V. Generic (bde-independent) actions in pr_log.c:

This is the place to find out the 'official' format of the log file from genv12.

Function do_command (lines 1838-1975 in pr_log.c) interprets one command line (arg1) of the replay logfile. It calls encode_token(char* token, int command). Both functions are at the end of pr_log.c.

The allowed commands are enumerated in the encode_token function, which converts a string token into an enumerated command. Command effects are documented in do_command case blocks. The enumerated list of command values is defined at line 82 (approx) of $RL/bde/pr_util/pr_log.c. Encode_token is in lines 1803-1835 (approx) of pr_log.c.

Function do_command enters one big switch(command) for the enumerated case names. (So does the switch(command) inKeypress. Command tokens fall into two groups based on their length (either 2 or 10 char tokens).

These commands do replay, not logging. Sample cases include:

Set_type commands call log_do_set_type;

Init/Load/Free/Dump call pr_init/pr_load/pr_free/pr_dump;

Delete calls log_do_delete;

StopCmd sets replayState = STATE2 (thereby exiting from STATE1)

The AddRowCmd and default: cases both check valdity of the row as a new table record, and then call pr_add_row. This means the AddRowCmd prefix to an ordinary table row is optional; the data row on the command line will be pr_added in either case.

The rest of the command interpretation is in the switch(command) block of Keypress. Do_command returns the command value to Keypress so it may be further processed in bde-specific ways. My goal is to simplify the Keypress function and move tt from bdeReplay.cc to pr_log.c. This requires moving its two remaining functions elsewhere in bde/src (not into pr_util, because genv12 produces generic database maintenance code and doesn't know about bde's X11 GUI editing features).

(1) STATE0: move this initialization part into Keypress (STATE0) or to pr_replay().

(2) STATE3: move the 'finalization' part of replay_log (STATE3) to Keypress.

VI. Refactoring BDE to use LCP and further evolution:

Bde contains embedded code for its state machines. When bde is re-programmed to use the LCP interpreter, genv12 will generate the next-state-assignment code in an event dispatcher based on the STDiagram database. The event dispatcher performs the actual state transition just BEFORE calling the next state action. The action itself evaluates guard conditions (synchronous evaluation of pre-conditions for a transition when a particular asynchronous event is dispatched.)

There are several parts to re-factoring bde to use LCP:

(1) LCP must be upgraded to alpha and Linux platforms.

(2) Bde must be refactored so the state codes become more object-oriented and the actions become single function calls.

There are two long-range goals for bde:

(1) Refactor bde and LCP to use gencpp not genv12 as their code generator.

(2) Refactor bde to localize and wrap all its X11-dependencies into functions that are GUI-independent; then we can conditionally compile alternate GUI functions for MSWindows or Java.

VI.