On Tue, 14 Feb 2006, Bob Lechner wrote: Subject: Re: DTrace - SW tracing from app into OS kernel - Solaris---FreeBSD; FYInfo RJLRef: $PH/06s524/DTrace4OSDebugging.060214.txt Scot - this is a good article, related to SWF Ch. 8, section on trace-bassed semantics (pp 312-318, 320,325). (printable from link at end - I've done some reading ahead.:-) Bob Lechner From: ** ACM TechNews; Monday, February 13, 2006 (=acmtechnews060213) "Hidden in Plain Sight" - Queue (02/06) Vol. 4, No. 1, P. 26; Cantrill, Bryan A lack of software observability is responsible for lengthy and often needless diagnosis of serious performance problems that usually have a relatively simple root cause, according to Bryan Cantrill with Sun Microsystems' Solaris Kernel Development Group. Software cannot manifest itself physically, which means modifying the software is the only way to observe the software being executed; but such constructs can slow down the system by their very presence. Avoiding this scenario supports the paradox that addressing performance problems seen in production requires replicating the problems in either a development or test environment. Cantrill also notes that performance problems are usually introduced at the highest software abstraction layers, but are often first seen at and blamed on the lowest abstraction layers. Therefore, it stands to reason that unintentional or unnecessary work could be eliminated by moving up the software stack to uncover underlying performance problems instead of focusing on their cascading symptoms in the lower stack. Shifting the observability infrastructure's focus from software development to production and from programs to systems is Cantrill's solution, and it requires the optimization of software when it ships, dynamic instrumentation of the production system, and, above all, absolute safety of the infrastructure. Sun's DTrace system was designed with dynamic production system instrumentation in mind, and it offers the flexibility of such instrumentation without sacrificing safety by drawing a line between the system's manner of instrumentation and the framework that consumes the data. "The architectural elements of DTrace--safe and heterogeneous dynamic instrumentation, arbitrary actions and predicates, and scalable, in situ data aggregation--allow for unprecedented observability of production systems," Cantrill writes. http://www.acmqueue.com/modules.php?name=Content&pa=showpage&pid=361 NOTEWORTHY: Ref to APL p.7; Experience Section - pp 1 and 8-10 on print-friendly version. A FreeBSD version being worked on part-time - Ref. 1, p. 11