Lightweight literate programming

The idea behind literate programming is to erase the distinction between source code and documentation. The author writes a single narrative, describing algorithms, data structures, and design philosophy, and the source code is embedded at appropriate locations in this narrative.

The primary early proponent of literate programming was Dr. Donald E. Knuth. For a good summary of the history and current work in literate programming, see the Literate Programming website.

The author was inspired by recent work of his colleague, Dr. Allan M. Stavely, in devising lightweight approaches to literate programming. In the author's current toolchain, the input is written in a slightly customized dialect of DocBook. Automated tools produce the two primary forms of output:

• The ordinary DocBook toolchain processes the source file as a document, producing both Web and PDF formats. For details of this toolchain, see Writing documentation with Docbook-XML 4.2.
• A small Python script extracts one or more source files from the document, and these are the executable form of the program. See A source extractor for lightweight literate programming. This script is itself done in the literate style, and the link here is to the generated document. See also the source for litlxml, the generated source file.

Following is a list of completed projects done by the author in the literate style, with comments about points of interest in that product. All these projects also use The Cleanroom software development methodology.

• PathInfo: A file information object in Python. Includes scripts to show the files in a directory by size (for space management) and by modification time. Demonstrates object-oriented system programming, including examples of inheritance.
• A skip list container class in Python: An alternative to balanced trees: an interested abstract data type for storing sorted sequences.
• huey: A color and font selection tool. A substantial graphical user interface application using Python and its Tkinter widget set. Examples of abstract classes and inheritance. The literate program is in a separate document, huey: Internal maintenance specification.
• FontSelect: A Tkinter widget to select fonts. Used in huey, and comes with a standalone front end.
• ScrolledList: A Tkinter scrollable list widget: Illustrates the technique of building compound widgets out of the basic set.
• PageTurner: A Tkinter compound widget that allows the display of multiple content pages in a single area with "Next page" and "Previous page" buttons.
• hwscan3: TCC processor inventory report. External specification for a script that summarizes hardware configurations for user-available servers and workstations. Includes literate exposition of a small XML schema. A production script: see the generated daily report (not available to off-campus users).
• hwscan3: Internal maintenance specification. Code for the script described above.
• homelist.py: A Python script to generate a list of user homepages. Interacts with the LDAP server using the standard Python ldap module. A production script: see the generated daily report (not available to off-campus users).
• sidereal.py: A Python package for astronomical calculations. Object-oriented scientific programming. The literate program is in the separate internal maintenance specification.
• A sudoku solver. A script to solve Sudoku puzzles. Illustrates recursion.
• Python and the XML Document Object Model. This is a document discussing various techniques for reading and writing XML in Python scripts. Included is a literate exposition of a new module, xmlcreate.py, to make it easier to generate XML files from scratch.
• manweb: A CGI script to display man pages.
• tccwebstats 3.0: TCC web server statistics. This is the external specification for a script that summarizes page fetch data from Apache web server logs. It is a production script at the Tech Computer Center: see the daily report.
• tccwebstats: Internal maintenance specification. The code for the above. Examples of regular expressions.
• tccpage.py: Dynamic generation of TCC-style web pages. A module that allows scripts to generate pages in the organization's standard style.
• Customization of the DocBook XSL Stylesheets for the TCC. Documents a "customization layer" written in XSLT (Extensible Stylesheet Language, Transforms), a programming language with XML syntax.
• iband7: IBP banding data entry system. This is my largest literate project to date, with two literate modules:
  • The Internal maintenance specification contains over 1300 lines of Python (not counting comments and blank lines). The literate exposition runs over 160 pages.
  • emacs customizations contains over 600 lines of emacs LISP.
• A system for representing bird taxonomy. Foundation for many of the author's bird data management tools. External reference. Includes literate exposition of an small XML schema.
• Bird taxonomy system: Internal maintenance specification. Good examples of object-oriented programming and Python-XML interfaces.
• A system for encoding bird field notes. Literate exposition of a somewhat larger XML schema.
• noteweb: Rendering bird field notes for the Web. See the author's online bird notes, produced from this system.
• The abalist program: computing American Birding Association list totals. An application built on the two preceding modules.

Last updated: 2008/12/23 18:25:10
URL: http://www.nmt.edu/~shipman/soft/litprog/homepage.html