SUBJ: bde log/replay with an X11 work proc and WAIT timer. RJLRef; $CASE/bdelog2ks/bdelogWithX11.2k0419.hlp NTansalarak and I considered the lack of pr_set commands in her log files in $CASE/bdelog2ks/ntansala/bde/src/ and in her bare-bones test $CASE/bdelog2ks/ntansala/testlog/test.c This note is about logging and replay of pr_set* actions. Logging and replay pose two very different problems for bde. Logging is controlled by two flags. One of them (RFLAG) tells the pr_set* macros whether the update is within this context: pr_create...{pr_set*}+...pr_add as when new objects are pr_loaded or created, or outside it, as in canvas update operations which are logged as field changes. Replay has the problems of breaking into X-11's event loop control and simplifying bde's complex database/canvas interdependencies. How the X-architecture permits this is discussed later below. =============================================================== I. Log file writing in bde: ------------- Logging is enabled by tbl_row_ptr->RFLAG and hcg_log, both of which must be 1 ; one bug is that RFLAG is not set in the current version of bde. For example, see pr_set_int below, defined early in schema.h (circa line 300): Note that tbl is misnamed - it is really a table-row-pointer, not a ttabbrev. ---------------------------------------------------- /* BEGIN MODIFICATION, ctwang, 12/20/97 */ #define pr_set_int(tbl,fld,value) \ do{\ if((tbl)->RFLAG==1 && hcg_log ==1 )\ {\ fprintf( hcg_logfileptr, "PR_SET_INT %s %s %d\n",\ decode_retstr( (hcg_key*)(tbl) ),"fld", (value) );\ }\ (tbl)->fld = (value); \ }while(0) ---------------------------------------------------- Where is RFLAG? Every row of every table has a hidden non-persistent RFLAG varible. Table row structs are declared at the end of schema.h, and RFLAG is declared last in every such struct; When bde objects get a persistent OLC state variable, it may evolve from this RFLAG. For example, here is RFLAG in struct HG: --------------------------------------- hcg_extern struct HG /* DFD: a single dataflow diagram with multiple internal nodes (added 91/3/12-RJL) */ { hcg_key HGid; /* unique internal id for this table row */ . . . struct HG *next_ptr; int RFLAG; /* for pr_utility macro log */ } hcg_declare(HG); ----------------------------------------------------------- The problem with the bde/genv11 version that we use, is that RFLAG does not get set to 1 as it should be for logging during non-Create canvas operations. We need to go back thru rrassmann's version and trace whether RFLAG for ctwang's pr_set_* logging function got lost or (more likely) was never implememted for bde. =========================================================== II. replay of log files in bde: ---------------- The problem with replay is much more serious. It requires the X11 event loop to pay attention to a timer inturrupt or to a flag (set by a file or stream reading process and/or an optional WAIT timer). This it can do by executing an action routine callback every time it finds no events in its que. Because it is important for the future of [Distributed] bde, I am quoting from Paul E Kimball: The X Toolkit Cookbook, P-H '95 Chapter 8. Section 8.2 Who calls callbacks? says this [p.166]: --------------------------- "Events fgrom the User Interface are not the only things that trigger callbacks. XtAppMainLoop may also calback for one or more of the following reasons: o A specified interval of time has passed. o Another input source such as a socket or pipe is ready for reading or writing; o There are no input events waiting and a low-priority "work" task can be run. o The application has detected a POSIX-style signal that needs to be handled (R6 only)." -------------------------------------------- If the log file did not include WAIT commands, the work proc could simply read the next log file record and interpret it. This would execute regression tests as fast as the event dispatcher would allow. (See Part III below re: WAIT commands that set timers which generate later timeout events to XtAppNextEvent(app,&event).) Section 8.9, Work Procedures [pp206-212] begins with: ---------------------------- "When XtAppMainLoop finds no timers or input sources to handle, and no X events waiting in the input queue, it blocks program executaion and waits until somethong happens. ... You can accomplish work while XtAppMainLoop is waiting by registering 'work procedures'. An XtWorkProc is a callback that is executed when there is nothing else to do, and is one of the few types of callbacks that returns a [Boolean] value. .... Work procedures are executed only when there are no other events to be handled, and therefore have lower priority than the user interface proper." Multiple work procs can be added to a que for each application context and processed in FIFO order. They execute to completion before returning so they can't take too long. (Think of State Transition Diagrams with an Action procedure in each state.) This call adds a work proc to a queue: ------------------------------------------- workprocid = XtAppAddWorkProc(context, MyWorkProc, mydata); -------------------------------------- If MyWorkProc saves its own state and returns false, X11 suspends it and requeues it for later resumption; when it returns true, it is automatically removed (forever, unless re-added); to remove it explicitly, call XtRemoveWorkProc(workprocid). Any synchronization is up to the programmer. For example, bde needs this type of alternate processing to replay log files or DBDE Master rebroadcast events. If the WorkProc detects pr_add or pr_set inputs and a WAIT flag is not set, it update the database; X11 is suspended so local edits will not conflict. If the WorkProc reads a WAIT command from the log file, it sets the WAIT flag (WFLAG? we already have an RFLAG :-) and calls a process timer that will reset the WAIT flag at timeout. [Refs: , man setitimer, and Kimbal's Example 8.8] Section 8.9 ends with these words of wisdom: ------------------------------------------------- "8.9.1: Caution - Work Procedures" "A work procedure is not 'background' processing; it runs in the same thread of control as all other callbacks, and is not interrupted by events that occur while it is executing. Work procedures, like other callbacks, return voluntarily. ... Any state that must be maintained between successive calls to a work proc must be stored in globalstructures or client data. This not always simple. Because of these issues, it may sometimes make more sense to fork a chld process to perform a lengthy computation, and communicate with the child through pipes as shown in the last example[8.8]." ------------------------------------------------------ ========================================================== III. WAIT commands and timeout events ------------------------------ THe log file by default also contains WAIT commands. These require timer activation which generates later timeout events. As discussed below, timeouts are also processed by XtAppNextEvent(app,&event). As long as I am at it, I might as well quote from Kimball's Chapter 9, Advanced Event Handling which is the context for the three ways to get X11 to do something besides the GUI input devices. Section 9.2.2 describes XtAppMainLoop(app) which loops forever callng two routines: XtAppNextEvent(app,&event) and XtDispatchEvent(&event): "XtAppNextEvent does much of the work. First, it checks for expired timeouts registered with XtAppAddTimeOut and input sources registered with XtAppAddInput, calling back if necessary. In X11/R6 it also checks for signal callbacks (Section 8,10). Then it retrieves the next X event of any type from a dispay in the application context, removing it from the input queue. If no X events are waiting, XtAppNextEvent flushes any X output queues and waits for input. While waiting, it invokes the last work procedure registered with XtAppAddWorkProc, if any. XtAppNextEvent repeats this process until an X event is received." XtDispatchEvent processes the next event after deciding which widget should deal with it. (This is complicated since the event dispatcher can re-map an event to some other widget than the one that detected it [Ch. 9.8,9.9].) Redundant (obsoleted) events are filtered out before the widget is asked to handle an event [Figure 9.1 and 9.2]. V. Conclusions: My conclusion is that pr_set logging must be enabled during bde canvas update operations but disabled during new object creation (between pr_create and pr_add), both inside and outside of pr_load. (Another mode bit is needed to distinguish pr_add processing inside pr_load,which parses ASCI input, from pr_add outside pr_load (after bde canvas ops make a new table row from graph-object [display] list data). I also conclude that RFLAG should be cleared by (probably all) callers of pr_create, and RFLAG should be reset by (probably all) callers of pr_add. Pr_set calls between pr_add and the next pr_create would then be logged since RFLAG==1 then. For replay, bde must be modified by (1) adding a work proc (XtAppAddWorkProc) that processes a log file record, and later by (2) adding a timeout event handler (XtAppAddTimeOut) to implement WAIT delays during log replay. R Lechner 2k0419