.\" Gunnar Gotshalks
.ds ]W FlexOr 2.0
.TH \fBFlexOr\fP 1 "1994 July"
.na
.SH INTRODUCTION
FlexOr technology combines the programming and documentation ideas of the Web
system developed by Donald Knuth with frame processing technology developed by
Gunnar Gotshalks. The Web system was developed by Knuth to document his
typesetting program TeX. His book, "TeX the Program", is the first published
example of a large program. Frame processing technology enhances the writing
of general purpose frames (frames are similar to templates) which can be
adapted, at the programming language source text level, to many different
environments.
A \fBFlexOr\fP frame is a sequential text file of ASCII characters with
embedded FlexOr commands which is the union of two logical files.
The \fBdocumentation file\fP contains user, designer and programmer
documentation, and program source text. The physical order of the text
presents the information is in the order in which the designer wants the
reader to view the information.
The \fBprogram file\fP contains the program source text that is to be
compiled. The source text is not in the order the compiler requires, but in
the order in which the designer expects the reader to best understand the
program and way it works.
.SH The documentation file
A FlexOr file is physically partitioned into sections just as there are
chapters in a book or sections within a chapter. Each section can contain
documentation text, program source text, or a mixture of both.
The sections can be physically in any order regardless of the order that the
compiler requires. The semantics of the program contained within the FlexOr
frame do not depend upon the physical order of the sections. The physical
order selected by the designer is the best order, in the opinion of the
designer, in which the information can be presented and understood by a
reader.
.SH The program file
Sections in a FlexOr frame that contain program source text represent nodes in
a directed acyclic graph of the program contained in the FlexOr frame.
.SH FlexOr frame processing
Since a FlexOr frame contains two logical files, two programs are needed that
can extract each logical file. The two programs are called \fBWeave\fP,
which produces typesetting commands for the FlexOr frame and \fBTangle\fP,
which restructures all the program text segments according to the designer's
commands.
.SH What the Weave program does
Weave produces, from the FlexOr frame, a file containing sufficient document
markup commands to typeset the FlexOr frame. Each version of weave inserts
commands for a different documentation markup language.
Weave programs produce the following for the user:
- A table of contents listing the section numbers of
selected sections
- Typesetting of the program source text
- A cross-reference of variables by section (single
letter variable names are not cross-referenced because
they are often used as local temporary indices and
counters)
- An alphabetic list of all the section names with
cross-reference information on the location of section
definition and use
- Section cross-reference information at the end of each
section containing program source text.
.SH What the Tangle program does
Tangle collects all the program source text segments within each section and
restructures them according to the designer's commands. The designer must
embed the appropriate \fBsection references\fP in the program source text,
creating an acyclic graph, so that an in-order pass of the acyclic graph
processes the program source text in the order required for the compiler. At
least one section must be named "The Program Begins Here". Tangle begins
processing the program source text from "The Program Begins Here".
Tangle programs do the following for the user:
- Produce a compilable program source text file
(implementation file)
- Produce a header file (declaration file)
- Expand macro definitions
- Apply adaptations.
.SH Language dependencies
So that the maximum benefits from the FlexOr methods are achieved, some
programming language dependencies are built into the weave and tangle
programs. Programming language dependencies provide macro expansion and
variable cross-references. While variable cross-references can be produced by
other tools, their output would not reference sections in FlexOr frames,
consequently reducing their effectiveness.
Programming language dependencies that should be built into the weave and
tangle programs are the following.
- The definition of a variable name, so that variables
can be put into the cross-reference tables and
identified for macro expansion
- The list of keywords in the language, so that the
keywords can be typeset in a different font (making
the weave output more readable) and can be removed
from the variable cross-reference tables
- The definition of a comment, so that macro expansion
cannot occur within comments
- The definition of a quoted string, so that macro
expansion cannot occur within a quoted string and
string characters are not interpreted as FlexOr
commands.
Weave is highly dependent upon the typesetting language, because it must
output typesetting commands. Output of appropriate commands can involve
customized processing for each language, because characters or symbols may
have to be encoded and typesetting commands can have complex syntactic
structures depending upon the amount of text or context in which typesetting
takes place.
.SH Benefits of using FlexOr
FlexOr combines literate programming with adaptation and frame processing.
Literate programming provides the following advantages:
- Documentation and code are in one file, which
minimizes redundancy, simplifies maintenance,
and encourages the designer to write documentation,
which includes the section headings and section
references
- Code is introduced in a logical sequence: the order is
not compiler driven; it is driven by human factors,
such as the consideration of the best presentation and
development order
- Documentation is improved because it occurs as part of
the design and programming effort, not as something to
be done later
- Programming style is improved because the focus is on
the current task, stubs are created more easily,
programs can be developed incrementally, and use of
macros can make programs more readable.
Adaptation and frame processing provide the following advantages:
- Time and energy are saved in producing better
documentation and programs.
- Adaption encourages reuse of designs and program
source text through customization of existing
solutions instead of "reinventing the wheel".
- The designer has only to specify what is unique about
the application and does not have to design the entire
solution from scratch. A major source of errors is
eliminated.
- In the development of new frames, algorithms, software
maintenance and reuse of existing designs are improved
because designers know that they and others will be
using and adapting the program at the source level.
.SH ===================================
.SH Single character commands.
Each of the following characters begins a FlexOr command sequence. There are
three such characters because it is more convenient for the writer to have
\fB|\fP and \fB~\fP as single character commands.
.IP \fB@\fP
Begins almost all FlexOr commands. Because FlexOr programs read character by
character, the characters immediately following the \fB@\fP determine a
particular FlexOr command. See the command \fB@@\fP in the next section.
.IP \fB~\fP
Ties together or concatenates. The tilde is used in program
source text to partition a variable name so that a subname can be modified by
macro substitution. For example, in stack~Push you can separately
rename either "stack" or "Push" producing such names as "characterPush" or
"stackWrite".
.IP \fB|...|\fP
Called a \fBprogram snippet\fP, it is used as a temporary escape from
documentation text mode into program mode. The \fB...\fP represents phrases
in program source text. The phrase is typeset in program mode and variable
names, comprising two or more characters, are put into the variable cross
reference table.
.SH Character escape sequences
Character escape sequences are used to alter the standard meaning of FlexOr
command characters (see the previous section).
.IP \fB@@\fP
Represent \fB@\fP in FlexOr files. Because \fB@\fP begins a FlexOr command,
you double \fB@\fP whenever you need an \fB@\fP in your source text.
\fBException:\fP Within quoted
items in program source text (command
\fB@tp\fP) you cannot double the \fB@\fP. Program text can be cut and
pasted from any source to any destination without modification.
When using the command \fB@tv\fP, because this mode does not analyze program
source text, you must double \fB@\fP, even within quoted strings.
.IP \fB@~\fP
Represents \fB~\fP in FlexOr program source text. In documentation text ~ has
no special meaning and, hence, can represent itself.
.IP \fB@|\fP
Represents \fB|\fP in FlexOr documentation text because of the special meaning
of the command \fB|...|\fP (see the previous section). You do not have to
treat \fB|\fP in a special way in program source text. Because the command
\fB|...|\fP is meaningless in that mode, FlexOr programs do not attach a
special meaning to \fB|\fP.
.SH Text modes
Text mode is the type of text with which FlexOr programs are working. Each
text type has different rules as to how FlexOr programs interpret the text.
The \fB@t\fP indicates a change in text mode. The characters following
\fB@t\fP denote the new text mode, which permits the evolution of FlexOr
programs so that they can handle new sets of programming languages.
If the first character following the command is white space, the tangle
programs do not use it. The ASCII file is easier to read and unwanted white
space is not introduced in the tangle output. The white space character
remains, however, because it gives finer control of the weave output.
The text mode commands are described as follows.
.IP \fB@td\fP
Starts documentation text mode. Text characters can follow the command
without intervening white space. Until the next change in text mode, all
succeeding characters are interpreted as documentation text.
.IP \fB@tf\fP
Starts format mode. This command causes weave programs to not embed the
following lines within format commands but simply echo them to the output.
Tangle programs treat the text following the @tf command as documentation
text. An example of using the command is to embed encapsulated postscript,
such as a diagram, into a FlexOr document. See flexor/examples directory for
ideas on how the command can be used.
.IP \fB@tp\fP
Starts program source text mode. Text characters can follow the command
without intervening white space. Until the next change in text mode, all
succeeding characters are interpreted as program source text.
.IP \fB@tv\fP
Start program source text verbatim mode. FlexOr programs do not parse
the program text. Be aware that you must escape all instances of the FlexOr
command character \fB@\fP, and that the FlexOr command character
\fB~\fP is not detected as a command character.
This mode handles programming languages that are not explicitly supported
by FlexOr, which gives designers and programmers the opportunity to use
the literate programming and adaptation features of FlexOr.
.SH Section-related commands.
These commands specify how FlexOr is to identify the start of a new section,
identify the start and end of a section name, alter the meaning of a section
reference, and add a section prefix to cross-references.
In FlexOr, section names can be any length. Many program comments and other
documentation text can be placed in the section headings, which can improve
program documentation when used wisely, and reduce the need for explicit
documentation text. White space (blanks, tabs, and newlines) is reduced to
one blank between words in a section name, while leading and trailing white
space is removed. Section names must match exactly (FlexOr is case sensitive)
in the section definition and the section reference. As a result, the rules
for reducing white space remove a potential source of errors.
.IP \fB@h...@<......@>\fP
Defines a new section.
.IP \fB...\fP
Represents typesetting commands, which is specific to the target
typesetting language.
.IP \fB@<\fP
Ends the typesetting commands and starts the name of the section being
defined.
.IP \fB......\fP
Represents the section name.
.IP \fB@>\fP
Ends the section name.
Text mode is the default after the start of a new section.
The following is an example definition for a new section using BookMaster
markup commands.
@h3 stitle='Short title' id=xrefShort
@
where the section definition is typeset as a level-3 heading with a
cross-reference id, |xrefShort|, and a short title, |Short title|, for a
running foot.
.IP \fB@<......@>\fP
References a section in program source text. Referencing a section differs
from defining a section name in that you do not include a preceding \fB@h\fP
and typesetting commands..
.IP \fB@<\fP
Starts the name of the section being referenced.
.IP \fB......\fP
Represents the section name.
.IP \fB@>\fP
Ends of the section name.
A section reference changes the text mode to program source text if the
user has not already done so.
The following is a reference to the section defined in the previous example.
The syntax of a section name is the same whether the section name is written
as a definition or a reference. This reduces potential errors when cut and
paste is used at program construction time.
@
.IP \fB@a\fP
Alters the meaning of a section reference and is used with \fB@<\fP.
With adaptation commands, whether a section reference refers to a section in
the adapted frame or in the adapting frame is potentially ambiguous. FlexOr
rules specify that a section reference refers to a section in the current
frame (adapting frame) if no adaptation commands occur before the reference
within the same section. Otherwise, the reference is to a section in the
adapted frame.
Thus, if \fB@<......@>\fP by default references a section in the adapting
frame, then \fB@a@<......@>\fP references a section in the adapted frame; and
vice-versa.
.IP \fB@e\fP
Begins the epilogue section. Text following the command is not used by the
tangle process and is output after everything else by the weave process. The
epilogue section has no name. It cannot contain program source text because
it cannot be referenced.
The epilogue section should be physically last in a FlexOr file because it
terminates file parsing. It contains any wrapup typesetting commands the user
wants.
.IP \fB@n\ string\fP
Changes the prefix text for section headings.
The default prefix is the empty string. The \fB@n\fP changes the prefix
to the first nonwhite space character sequence, \fBstring\fP, following \fB@n\fP.
If the sequence is a quoted string beginning with a double quotation mark, ",
the quoted string, without the quotation marks, is used as the prefix, so that
you can set a null prefix or embed blanks in the prefix. Within a quoted
string, use \" to embed the quotation mark and use \\ to embed the \
character.
Because the first white space character after the prefix is required to
delimit the prefix, the white space character cannot be a significant
character in a FlexOr frame, and the character is not used by tangle
programs.
This command is useful when FlexOr files contain more than one logical
document that the writer wants to differentiate in the printed document. The
prefix string you specify is prefixed, at weave time, to definition section
headings and cross references, until you specify another prefix string.
.SH Adaptation, copying and including.
These commands all deal with adaptation, copying and including FlexOr frames.
.IP \fB@.A\ \fP[\fIenvironVariable\fP]\ \fIfilename\fP\ [\fI\ @<...@>\fP]
Adapts the named file. If the adaptation command set is empty, this is
equivalent to \fB@.C\fP.
The optional \fBenvironVariable\fP is an environment variable that you can
define to be the path to the directory where the file to be adapted is stored.
The path must end in a /.
If the environVariable is undefined, it is treated as the file name, which
will usually not exist so you will detect the error at tangle time. If the
environVariable happens to also be a file name then the following file name
will be in error because there can be no text after the file name before
and adaptation command such as prefix, replace, suffix, and insert.
The optional \fB@<...@>\fP represents the name of the
section at which the tangle program is to begin. The default section is "The
Program Begins Here".
Text mode is forced to program source text.
The \fB@.>\fP command terminates the adapt command.
All white space between the adapt command, the optional change in root, and
the first occurrence of other adaptation commands is not used. You must not
have program or documentation text before an insert, prefix, replace, or
suffix command, because there is no logical location at which the text can be
placed in the adapted file.
.IP \fB@.>\fP
Ends adaptation commands for a file and matches an \fB@.A\fP
.IP \fB@.C\ \fP[\fIenvironVariable\fP]\ \fIfilename\fP\ [\fI\ @<...@>\fP]
Copies the named file without adaptation. The copied text is included at
tangle time but not at weave time. This command is used to describe the
construction of a program without duplicating common information, such as a
standard set of macro definitions.
The optional \fBenvironVariable\fP is an environment variable that you can
define to be the path to the directory where the file to be copied is stored.
The path must end in a /. If the environVariable is undefined, it is treated
as the file name, which will usually not exist so you will detect the error at
tangle time. If the environVariable happens to also be a file name then the
following file name will become part of the program text and not detected
until compile time.
The optional \fB@<...@>\fP represents the name of the section at which the
tangle program is to begin. The default section is "The Program Begins Here".
.IP \fB@i\ \fP[\fIenvironVariable\fP]\ \fIfilename\fP\fP
Imbeds a file. Similar to copy at tangle time, except that at weave time
the contents of the file appear in the weave document. This command permits
a large document to be split into multiple physical files.
The optional \fBenvironVariable\fP is an environment variable that you can
define to be the path to the directory where the file to be included is stored.
The path must end in a /. If the environVariable is undefined, it is treated
as the file name, which will usually not exist so you will detect the error at
tangle time. If the environVariable happens to also be a file name then the
following file name becomes a part of the document as either documentation
text or program text (in whichever mode the @i command occurred).
.IP \fB@.I\ ...@>\fP
Inserts a new section, named \fB...\fP, in the adapted file
with the program text following the insert command. Insert text continues
until the next adaptation command, section definition or end of file is found.
Text mode is forced to program source text, which makes it easier for you to
enter short adaptations. If the first character following the \fB@>\fP is
a white space character, it is not used. Text mode change commands can be
used as needed.
.IP \fB@.P\ ...@>\fP
Prefixes the program text in the section named by \fB...\fP
with the program text following the prefix command. Prefix text continues
until the next adaptation command, section definition or end of file is found.
Text mode is forced to program source text, which makes it easier for you to
enter short adaptations. If the first character following the \fB@>\fP is
a white space character, it is not used. Text mode change commands can be
used as needed.
.IP \fB@.R\ ...@>\fP
Replaces the program text in the section named by \fB...\fP
with the program text following the replace command. Replacement text
continues until the next adaptation command, section definition or end of file
is found.
Text mode is forced to program source text, which makes it easier for you to
enter short adaptations. If the first character following the \fB@>\fP is
a white space character, it is not used. Text mode change commands can be
used as needed.
.IP \fB@.S\ ...@>\fP
Suffixes the program text in the section named by \fB...\fP
with the program text following the suffix command. Suffix text continues
until the next adaptation command, section definition or end of file is found.
Text mode is forced to program source text, which makes it easier for you to
enter short adaptations. If the first character following the \fB@>\fP is
a white space character, it is not used. Text mode change commands can be
used as needed.
.IP \fB@.N\ \fIfilename\fP\fP
Starts a new output file as named to contain the tangle program text
.IP \fB@.Nh\ \fIfilename\fP\fP
Starts a new header file (also known as a declaration file). All definitions
beginning with the export keyword, that are processed after the new header
command, are added to the list to be sent to the new header file.
The export keyword can be used for the following structures:
1. export #include ...
The #include statement is written to the declaration file and not to the
definition file (.c file). The text to be output terminates with the end of
line. Section references are not followed.
2. export #typedef ... ;
The #typedef statement is written to the declaration file and not to the
definition file. The output text terminates with the first ";" found outside
of nested parenthesis, braces and brackets. Section references are followed.
3. export #define ...
The #define statement is written to the declaration file and not to the
definition file. The output text terminates with the first newline not
protected with a backslash. Section references are not followed.
4. export
The keyword export is changed to extern. The declaration file receives the
text from extern to either a ";" (for variable definitions) or a "{" for
function definitions. The ";" is included in the output and the "{" is
replaced with a ";". The definition file receives the original text except
for the export keyword and the following white space.
5. export header ... endExportToHeader
The text following "export header up to but not inlcuding
"endExportToHeader" is output to the declaration file. None of the text,
including the key phrases are output to the definition file.
.SH Other Commands.
.IP \fB@m\fP
Starts a macro definition. The scope of the macro definition continues
until the next FlexOr command. Leading and trailing white space is removed
from the definition.
.IP \fB@{...@}\fP
FlexOr program meta comments. Text within the braces is not used by
FlexOr programs. Meta comments can be nested making is easy to logically
remove and restore parts of a FlexOr file during development.
.IP \fB@'...@'\fP
Specifies a verbatim quote. Program source text and documentation text
are included without interpretation and output as is. The command is added as
an escape mechanism for special needs, and is most useful when sections of
text contain FlexOr commands and it would be tedious to escape the special
characters.
.IP \fB@s\ attr=value\ ...\ .\fP
Set a collection of attribute values which affect weave programs. Note the
final '.' is required to terminate the @s command. Every instance of the
command will change the settings. Currently the list of supported attributes
are:
partid =
Set the part identifier for a frame. Section
references use this as a secondary key for a
reference to match. Setting this attribute enables
a designer to combine logically separate frames
into one file and have sections with the same name
be treated as different sections.
tab = n
Set the distance between tab stops to be n for
WeavBmC. Has no effect for other programs. Default
value is 8.
tanglePass = yes | no
To enable weave programs to format identifiers used
as macro calls and to produce all interface
declarations written to header files, a weave
program must make a tangle pass over the frame.
in some frames there no need to make this tangle
pass (e.g. there is no root node). In this case
use the attribute @s tanglePass=no. The default
is yes.
toc = d
0 <= d <= 9, set the maximum level of header to
appear in the table of contents for WeavPlC. Has
no effect for other programs. Default value is 3.
Examples: @s tab=4 toc=2.
@s tab=5.
@s tanglePass=no.
@s partid=stringADT.
Below partid is the empty string.
@s partid= tanglePass=no.
.IP \fB@l\fP
Specifies the direction for log file output when the input file is being read.
The log direction command is useful only for debugging FlexOr scanners and
parsers.
Cmd Description
@l0 Do not log input characters
@l1 Log to SCREEN
@l2 Log to LOG file
@l3 Log to both SCREEN and LOG file.
.SH Special notes for Tangle and Weave
Tangle programs do not emit the last character of a section, if the last
character is an newline character. This makes the ASCII file easier to read
while simultaneously, not introducing unwanted new lines in the tangled
output. The character is not thrown away because it gives finer control for
weave output. If the character was thrown away, then weave programs would not
know whether or not the writer had inserted a newline character and the
spacing of the weaved output would not reflect the actual input.
.SH HOW TO GET THE FLEXOR PACKAGE
FlexOr is available through anonymous ftp.
1 Ftp to ftp.cs.yorku.ca
2 Log in as anonymous
3 cd to pub/FlexOr
4 The file FlexOr.InstallInstructions tells you the rest.