GEN Meta-Schema and LCP/SM  extensions for Fwd/Rev Engineering
RJLRef: $PH/COOL-LCP/GEN_LCP_metaschemaRevs061122.txt
	(was $PH/05f523/GEN_LCP_metaschemaBibliog051012.txt)

CONTENTS:
	1. Bibliography
	2. LCP's relation to Soukup
	3. Possible extension of LCP
	4. New Insights
	5. Reverse Engineering
        6. Forward Engineering

1. Bibliography:
--------------------
Pattern Languages of Program Design, A-W 1995 
This first PLoP Conf Proceedings
contains these two refs on STDs in LCP:

Jiri Soukup: Implementing Patterns [as interface classes) (395-412)  

Panu Viljamaa: Client-Specified Self(CSS),  (495-504)
(compares CSS to dynamic classes, instance-methods, delegation)

Other Books: 
-------------
James Rumgaugh: OMT Insights, SIGS Books 1996. 
This is a collection of short articles from JOOP.
Two chapters are of special interest to BDE and LCP:
Modeling Models and Viewing Views (MVC framework) May 1994 (pp 297-308)
Controlling Code (How to implement Dynamic Models) May 1993 (pp.309-313)

J Wierenga: Design Methods for Reactive Systems, M-K '2003
(Yourdon, Statemate and the UML)

Miro Samek: Practical Statecharts in C/C++, CMPBooks 2002.
(CD with lots of source code for Samek's Quantum Framework,
plus Eval. Version of a RTOS-32 for x86 CPUs
[RTKernel-32 from On-TIme SW (p. 266 fn1)] )

Samek Ch. 9, Implementations of QF, opens with this 
DLParnas quote:
"There is no trick  in building large systems quickly;
the quicker you build them, the larger they get!"

Raistrick et al (Kennedy-Carter): MDA with xUML.
CambridgeUPress, 2004 (with CD of iUMLite from K-C).

Binder: Testing O-O Systems, A-W 1999
A complete theoretical exposition of State Machines
is in one small (~100pp) part of this 1200 pp book.
Plus an extensive bibliography.


Articles:
---------
M. Ward: "Derivation of Data Intensive Algorithms by Formal
Transformation --The Schorr-Waite Graph Marking Algorithm",
IEEE Transactions on Software Engineering, 22(9), Sept 1996, pp 665-686"
(This paper inspired my Ghost Variables slides at $PH/ghostvarsRev3
and Prachee Sharma's MS thesis at UML/CS.)

Others TBD

-------------------------------------

2. LCP's relation to Soukup:
-----------------------------
PLoP/Soukup implements class Composite as an interface
with methods that link other classes containing data,
without making the data-containing classss inter-dependent.

Does this resemble how LCP's application subclasses 
inherit from generic Active Classes, using delegation 
of BOTH method execution AND state model flow control 
from application objects to their counterpart Active 
Instances?

The app-objects are persistent table-row instances 
containing data (and embedded pointers) with no other
inerdependence besides fkeys from which pr_util
auto-generates embedded list-traversal pointers.

The methods which inter-relate the objects 
(the functional interfaces between classes)
are all contained in Soukup's Composite class.

In LCP, they are the responsibility of 
Active Classes with their method-specific State Models 
and Action Routines.


3. Possible extension of LCP:
-----------------------------

[RJL061122: In $CASE/04s522/lcp04s (linked from 
$PH/COOL-LCP/04s522LCPupdateProject/lcp04s/),
the definition of table ST has changed on relative path
ajlopez/lcp/schema/schemaRev051129.sch.
The change consists of reinterpreting the field ActFunc
as an index into the table FuncTbl[]  of action routine pointers:
------------
State ST                                            /* states */
{
    STid    StateId                         c8  1   /* the primary key       */
    SMid    StateAStateOfThisStateMachine   c8  1   /* The state machine     */
                                                    /*  containing the state */
    ActName StatesActionRoutineName         t32 0   /* Action Routine Name   */
    ActFunc StatesActionRoutinePointer      i4  0   /* index of function */
    Name    StatesName                      t80 0   /* The name of the state */
}
This is required because persistent data cannot include 
pointers to action routines. Before lcp04s, other work-arounds
were required, such as if elseif... ActName comparisons
or switch(ActName enumeration type) to invoke ActName().

Lcp04s uses makeHead.c in this way:

When a StateModel for some XXmethod is declared textually 
in XXsetup.c, makeHead.c defines FuncTbl[i] = STcurr->(*ActName). 
Then the linker can replace this address expression 
by the State's ActName funcion entry point address. 
makeHead.c also inserts this value of i in the
ActFunc field of the same State instance. 

FuncTbl[] is genrated during the lcpsetup phase; 
When lcpsetup is executed the jump table
is written to HMFunxcPtr.h. Then the lcprun phase 
(with HMFuncPtr.h  #included) is relinked.
The linker resolves the FuncTbl[] pointers 
in file HMFuncPtr.h.

In $PH/04s522LCPupdateProject/lcp04s/ajlopez/lcp/include/,
file HMFuncPtr.h is named after the Hominid application.
A generic naming convention should be used,
such as appRootFuncTbl.h or appSchemaFuncTbl.h. 
----------

A future change to lcp/olc might be a FunctionTable FT that 
contains the ActName and FuncTbl index pair.
THen ST and FT can have a many-many association SF.
Then a sequence of sub-methods can be executed by 
child_loop(ST,SF,SFid,STid) 
{
  (*FuncTbl[i=(SFcurr->FTid_pp->ActFunc)(EIid)])
}

LCP has AC as parent=container=aggregate of 
both ST-row and FT-row types of objects.
All these are static properies of the AC.

FT_rows are 1:1 related to FuncTbl[] pointers declared 
by makeiHead.c via StateModelCreateCompletely
and initialized by the linker.

State Models can also nested, if a State Action can 
invoke a sub-StateModel in its state action, 
or (preferably) referenced in its Guard Condition.

The Composite pattern has two types of components:
the leaf/atomic sub-type and the Composite sub-type.
The later is an aggregate of lower-level components.
A non-leaf STate can be recognized as a Composite 
(by bde2SM) if it has a non-Null fkey SMid2 
that references another StateModel as its expansion.

More generally, a node HN of a diagram HG can have 
a second HGid2 fkey that refers to its expansion.
In bde2SM, a diagram HG maps to a State Model SM,
and its nodes HN map 1:1 to states ST.
The second HNid2 to a nodes' expansion 
naturally maps to an SMid2 which is the SM
expansion of a STate. 

This expansion is unique if it exists; furthermore,
it may be declared as the expansion of 
more than one STate in one or more client SMs.
Sharing a sub-model among multiple STate components
and diagrams may be risky and may violate encapsulation.
However, it is done all the time in non-OO code 
by means of shared procedures and calls to 
library routines, These routines must be  
re-entrant if they can  be executed concurrently.

Execution of AI methods under LCP control
goes on concurrently, by inter-leaving their atomic
STate action routine calls, whose order is 
determined by the event dispatching discipline.

[This order may be FIFO or otherwise, including future
of Timer Service events in prioriy-queued
order of their timeout deadlines.]

[In particular, it is important to note that pr_add
is NOT re-entrant: chgen uses many globals.
Parallel threads must treat pr_add as an atomic
procedure with mutex constraints. I.e. they must
lock it for exclusive access as a critical section.
This effectively serializes pr_add transactions
which include insert and pr_link calls to update
the table and parent-to-child linked lists in which
each object instance participates.]


4. New insights: 
------------------
4.1. Suppose SM inherits from an FT superclass. 
SM is composite because it is an aggregate
of states with their action routines.

4.2. Suppose FT has a second subclass of type AR for 
'ActionRoutine' (an action ro:utine pointer to executable code).
(In an earlier draft, AR was referred to as EP = EntryPtr.
AP and FP are reserved for actual and formal parameter resp.)
Note that an AR object might as well be used 
as the target ActionRoutine for an Atomic or 
indecomposable STate. The index in FuncTbl[ActFunc] 
locates the routine address correspoding to ActName in ST.

Now each STate in a StateModel can have an FTid fkey
that refers (by down-casting if necessary) 
to EITHER of its two subclasses: 
(1) an atomic AR (a compiled function) (was ActName).
OR 
(2) an SM Composite (fkey = SMid2, address SMid2_pp).
[See below for an alternate interpretation of FT: 
as a Guard COndition returning int Pass/Fail value.]

Now the LCP dispatcher would have an additional
responsibility: Decide whether to call an AR routine
which is compiled directly into executable code, 
or to [create a new ActiveInstance AI which can] 
descend into the Composite STcurr->SMid2 SM recursively,
and follow its event-driven control flow model,
traversing its STates until a PASS/FAIL return
action is reached. This 'recursive descent'
requires a pushdown stack that saves the 
client StateModel SMid and its current state STid.
[Note that this is inefficient because it simulates
the compiled call/return stack functionality.]

The SM instance of the Client AIid is suspended 
until the SM instance of the lower-level AIid 
completes its operation and returns.


To fully  emulate low-level program semantics,
the Server ActiveInstance must return at least 
non-void status code like C expression semantics:
(e.g., a PASS (TRUE) exit, or a FAIL (FALSE) exit 
with an int error code). [Ref: TRAIL paper]
An expansion into a submachine model could be modeled 
as a Guard Condition (rather than an ActionRoutine) 
in the client, with possible side effects  as well as 
this PASS/FAIL return value (like a first-class 
function call) rather than a void action method 
with a single exit point.

Upon exit from the lowest-level SM, 
LCP pops its history stack of AI-activations 
and re-enters its 'caller' StateModel
I.e., it returns to the current STate 
of the client SM which invoked it.

After return from a 'Server' SM to a 'Client' ST,
a PASS or FAIL verdict must be anticipated.
[FAIL implies an associated error code value.]
This submachine either becomes the entire 
Action Method of that client State, or conditionally
calls the local 'PASSEXIT' action ActName 
of that State. [Ref: TRAIL paper]

I conjecture that this provides semantics for  
descending into a sub-StateModel (subclass of FT) 
that is equivalent to evaluating a 'Guard 
Condition' (GC) predicate in the Client's STate,
since the SM returns the GC's error status code.

As a matter of fact, this GC could be implemented 
by conditionally invoking Actname as an instance 
of the AR subclass of FT.  
Then the LCP interpreter becomes equivalent to
TRAIL's interpreter: each ST becomes a model
for this if...then statement block:
	if P then {do A; goto PASSexit;} 
	else {goto FAILExit(errorcode);}.
These can be sequenced as in 
	if ...elseif...elseif...else.

Thus, version (4) of LCP  extends TRAIL by
interleaving EventInstance-triggered State Transitions.
with a sequence of Guarded Action calls.

The semantics of a State Action call 
to A = (*FuncTbl[ActFunc]) is unchanged 
from the current LCP implementation: It is (always) 
an Atomic or compiled action, with a single exit 
point and no return value. 
However, it can have side effects on data members of 
local (and friend) class (and instance) variables, 
and it can generate event messages to other AI's.

P is a (new) GC that can be implemented in two ways:
(P1) Call a predicate-evaluation function 
via its EntryPoint, which is an index to 
a second address in the merged Action-or-Guard 
FuncTbl[GuardFunc] to a compiled function, 
OR 
(P2) Suspend this SM, descending into a nested SM
(one-or-many-state composite SM, interpreted as the
STD-expansion of this State's action).

This implies that a ST object must now have either 
a second SMid2 for its (Guard or Action) expansion 
or merge both GuardFunc index and ActFunc index
types into its merged FunctTbl.

With the further complication of subclassing ST
into two subtypes: current ST with atomic ActFunc (or FTid) 
or composite ST with decomposition at fkey sub-SM SMid2,
we can have it both ways. 

Now there are 4 possibilities:


(1) Atomic subtype of ST with no expansion
------------------------------------------
This is the method currently used in LCP04f,
and (as commented switch statements) in bde/src
Unfortunately this obfuscates all lower-level
control flow which is not visible on STate Diagrams.

In this case, STcurr->SMid2 == 0 and STcurr->SMid_pp ==NULL.
The State's action routine STcurr->ActName
is executed unconditionally as a compiled method
via
        (*FuncTbl[STcurr->ActFunc])(ETid)

In this case, either no Guard Condition is required,
or all guard conditions are hidden inside the
state action routine STcurr->ActName.
This case is not compatible with extended Mealy
State Machines, whose State Transition labels
include Guard Expressions as well as EventTypes.


(2) A Commposite subtype of ST with nested StateModel:
-----------------------------------------------------
This approach replaces State Action at STcurr->ActName 
(indexed by ActFunc) by all actions of the nested 
StateModel with fkey STcurr->SMid2 at STcurr->SMid2_pp.
It also requires adding a new fkey SMid2 to the State table ST.
(Note that bde's schema already includs a second HGid2
that anticipates node expansion to a nested sub-diagram.)

The current state of the client SM's ST (at STcurr) 
has a non-NULL fkey SMid2 which defines its expansion 
as a sub-StateModel. A new ActiveInstance may be created 
to execute this expansion SM instance, located at 
STcurr->SMid2; this AI holds the current initial state
for this sub-SM.

On exit from this 'Server' SM, the LCP interpreter
restores the (stacked) state STcurr and client SMcurr.
No use is made of the local ActName or ActFunc fields 
at STcurr.


(3) Trivial SMid2 expansion PLUS local ActName.
----------------------------------------------
Interpret the action of a trivlal one-state SM 
at SMid2 as a one-state GuardCondition evaluator, 
and when it returns, execute a conditional local 
state action ActName.

By trivial SM we mean that the 'guardSM' located 
at STcurr->SMid2_pp has a singleton child ST at 
    guardSTptr = (ST*) STcurr->SMid2_pp->STid_fcp. 
Upon entering that state, the LCP interpreter executes 
a guard condition function named guardSTptr->ActName.
ActName was mapped by makeHead.c in lcp04s into the index 
guardSTptr->ActFunc. The corresponding FuncTbl entry 
is directly callable as: 
	guardSTptr = (ST*) STcurr->SMid2_pp->STid_fcp;
    	(*FuncTbl[(int)guardSTptr->ActFunc])(ETid);

Upon return, local function ActName can also be called 
as:
	(*FuncTbl[STcurr->ActFunc])(ETid);

This second call can be made conditional 
on the PASS/FAIL exit from the guardST of SMid2 
at guardSTptr.
(The SM schema may not need another field for this 
return value if LCP directly consumes it.)

(4) Generalize (3) to an arbitrary multi-state SM. 
-----------------------------------------------
This case is a combination of (2) and (3).


The most interesting thing about approachs (3) and (4)
is that code migration between the atomic and 
composite types of If-conditioni predicates is 
relatively easy; in fact it should permit both:  

(1) Reverse engineeering: from code to State Model,
for any (refactorable) procedure-level code fragment. 

(2) Forward engineering: from State Model to
its (refactored) compiled code replacement.

One way forward engineering can be done is to translate
the interpreter's SM database into a two-level 
switch(state) and switch(event) case hierarchy,
with direct calls to predicate evaluation and
action routines. Other ways may be more O-O:
(Viljamaa's PLoP-95  article has 4 alternatives).


5. Reverse Engineering: 
-----------------------
Conjecture (to be verified or constrained):
Reverse engineering can move any procedure hierarchy 
into the LCP framework by simply defining the 
top-level entry point as an AR, and moving its argv[] 
parameter list into an Event Type with the same 
signature as the AR. The dispatcher then calls main()
with this argv[].
 
Then the sub-programs of main are decomposed one by one 
into State Models. Each call to a decomposed procedure
is implemented as a GC predicate of some STate of the 
client program.

The STate invokes the GC Predicate via its FTid1
fkey (or FTid_pp parent-poiner) data member.
LCP must build an Event Instance (EI)
which contains the actual argument list of the call.

The ET of this EI must have the same data member type sequence
(same signature) as the called procedure declaration did.
(The EIcurr pointer replaces the stack-based  pointer 
to these actual argument values.) 

[TBD: 
(1) extend chgen's meta-model to include 
a larger extensible set of AttributeTypes (AT).
This can be done by providing an ATid fkey in each TA. 
(ATid either augments or replaces the format field  
=I/F/Cnn/Tnn and is_key field =0/1/s in table TA.)
Associative entity TA now relates TT to AT.

(2) extend LCP's schema to include an M to N
association EA between ET and AT, analogous to
the M to N association TA between TT and AT in (1). 
Table AT will then support both the TA-list 
of TableAttribute types and the EA-list of 
EventType fields types for actual argument expressions.

[Conjecture: ET and TT can be merged, as can TA and EA.
Then each ET becomes just another TT object, albeit
at a different level of granularity in space/time.]


6. Forward Engineering:
-----------------------
Forward Engineering can now proceed by refactoring,
with prototype transformation and testing at 
the State Model level, by decomposing the one-state 
procedure AR into substates connected by state transitions:
i.e., by migrating the AR (atomic procedure) subclass of FT 
towards the other (Composite or State Model) subclass SM. 

After testing the LCP prototype if it turns out to 
to improve performance, iwe could generate a new 
compiled version of the code, 
by translating the nested State Model prototype
back into compilable and directly executable code.

Question: Can Reverse and/or Forward Engineering be automated? 
Probably. For insight into this, see Martin Ward's UK website: 
http://www.dur.ac.uk/martin.ward/

Martin's group developed "The FermaT Transformation Engine,
an industrial-strength program transformation system"
(free software under the GNU General Public License GPL).
(Scheme and Perl lovers will love this too. :-)

Conjecture: FermaT may be a good way to do forward and 
reverse engineering of code to/from state models.