::::: : the wood : davidrobins.net

My name is David Robins: Christian, lead developer (resume), writer, photographer, runner, libertarian (voluntaryist), and student.

This is also my son David Geoffrey Robins' site.

CSEP501: the compiler is done

News, School ·Tuesday December 15, 2009 @ 22:17 EST (link)

I had my final report for my CSEP 501: Compiler Construction course today; it was a 20-minute scheduled block with the professor; just a fairly informal discussion about how the project went. The project was to build a "MiniJava" compiler in Java targeting Intel x86 (32-bit). Variations were permitted with permission of the instructor (e.g., we used C++; others used F#, Python, etc., or targeted MIPS or Intel x86-64). For the first two parts (scanner and parser) I was in a group of two; we parted amicably after my partner realized I wanted do to several language extensions which (although I said I'd do them myself) and didn't think he'd have time that quarter; so I worked alone on type-checking, semantics, and the big final code generation project. My final project archive is available.

Now that I'm done, it's strange that my (GNU) Screen window #2 is no longer "501" and window #4 no longer "test"; I've had those windows set that way for months now, flipping back and forth as I worked on the source and ran tests.

Speaking of tests, one of the things that helped the most was having a lot of test programs that allowed gauging the state of the compiler at any given time. Toward the end, when code was generated, I had my test framework assemble, link, and run the tests that were intended to pass (I had another class that was intended to ensure the compiler detected errors when appropriate). The framework also compared the test output with known good output.

Another great help was using source control (Subversion in this case). Looking over the logs, I can find a few interesting points where good testing caught some bad problems which were all eventually fixed: Much of these were found by tests as said, and by having about 10 different debug options to allow dumping intermediate compiler states (the AST, IR before and after optimization, register assignment, add IR statements to generated code as comments, etc.).

As the project went on I tended to mark areas that needed fixing (where it was "good enough" now, or I'd used a temporary hack) with either "TODO" or "XXX" comments, which worked well: it kept the work items obvious and easy to find, and local to where they needed to be done. But in larger projects, they can become overused and get ignored: Word has many TODOs, most of which just don't matter any more (fixed another way; current algorithm is sufficient, etc.).

This is the final report I submitted, detailing the stages of my compiler.


CSE P 501 course MiniJava compiler project, part V - Report.

Tools (versions used in parentheses, although other versions may also work): C++ (GCC 4.4.1), GNU Flex (2.5.35), GNU Bison (2.4.1), SCons (1.2.0), Perl (5.8.8), Boost (1.39).

FEATURES

All features are believed to work, plus certain extensions (see below, EXTRA FEATURES).

TEST PROGRAMS

The test programs available from http://www.cambridge.org/resources/052182060X/#programs were used, as well as a few of my own, both programs designed to pass (test/*.java) and programs designed for the compiler to catch errors (test/error/*.{java,txt}). All programs in the main test folder compile, assemble, link (with the included mj library), and run to completion without errors (not all return 0; some with void main functions return nonzero, but none return error codes indicating premature termination). For the programs from the above site, I examined the source and ensured that the right results were printed; they were very helpful in diagnosing problems. Each program also has a '.out' file which is the verified output; the test/runtests script does a 'diff' against it and the actual output, which provides another level of security against bad changes.

PrintArgs.java illustrates the passing of command-line arguments and the handling of strings (an array of strings, in this case). PrintArray.java shows more array handling, including an array of base pointers to various derived classes.

IfNot.java illustrates an optimization to remove a 'not' instruction and reverse the jcc/setcc condition.

LongAdd.java and DeepArray.java were unsuccessful attempts to cause register spills.

VoidDivide.java illustrates optimizing out (most of) a divide in void context (we don't care about the numberator, just if it divides by zero).

Or.java tests the boolean or (||) extension, using System.assert(), which would cause test/runtests, the test host, to fail if any asserts fail.

SpillTest.java has a deeply nested hierarchical comparison, which ensures we run out of registers, and exercises the spill code.

EXTRA FEATURES

Most extra features were completed for the parser assignment, thus they are described in README.2. Optimizations are described below in the appropriate stage under DESCRIPTION.

DESCRIPTION

The compiler runs several stages and passes and each does particular transformations on the input tree or list, or produces a new representation.

0. Driver: The process is driven via the argument parser in main.cc and the driver class (SPD, driver.h) which runs the pipeline (SPD::Parse) and provides context (e.g., command-line options) and global functionality (e.g., error reporting). To avoid unnecessary error checking, if one stage fails the pipeline is usually aborted (e.g., after type checking the code would bloat enormously if later stages had to account for undefined types).

1. Scanner: (see scanner.l and README.1; extensions include "void", other comparison operators besides "<", divide ("/"); also "this", "super", and "System.out.println" are not keywords; they're general identifiers handled later). Scanned tokens have location information embedded.

2. Parser: (see parser.y and README.2 for language extensions). The parser creates an AST of nodes inheriting from the base syntax node (SN, node.h), the main types being expressions (XN, exp.h), statements (CN, stmt.h), and declarations (DN, decl.h, which also includes classes, SNC/class.h, and methods, SNM/method.h, and the top-level program, SPD/driver.h). Nodes contain a generic list of children; most have specific accessors to improve readability (e.g., binary expressions, XN2, have XnLhs and XnRhs child accessors). If the number of children is indeterminate (e.g., parameters), a secondary container node is used.

3. System: SPD::SPD creates the static methods System.out.println, and a few handy extensions: System.out.print (takes a string parameter), System.abort (calls C library abort()), and System.assert (calls abort if passed boolean is false). The String built-in class is also created here. System classes may not be inherited from, and only String may be instantiated. The implementation is provided in the target mj library (see mjlib.cc).

4. Debug: along the way there are several possible debug print-outs, controlled by flags (see main.cc, or mj --help). --debug-scanner, --debug-parser, --debug-gen, --debug-reg print debug output during scanning, parsing, code generation, and register allocation respectively. --print-ast, --print-sym, --print-ir-pre, --print-ir-post, and --print-live output the AST, symbol tables, IR (before and after optimizations), and live ranges. --debug-code inserts some debug comments into the generated code.

5. Fixups: (fixup.h) Since classes can be used before declaration, this pass links type information to named classes (whether base classes, declarations, etc.). This is aided by a declaration reference type (DR, ref.h) which uses a boost::variant to store either a class name or a pointer to a declaration (DN).

6. Type-checking: (typecheck.h; see README.3) Each expression node (XN) is assigned a type, which it gets from its content (constants) or children (operators). The TI class (type.h) stores type information and has an FSubtype method to check if one type is a subtype of another. TCV::PtiTypeCheck ensures that argument types match when they need to, even when an operator can take different types (e.g., my ==/!= can also compare boolean values or class values). Method calls take on the type of their return value, and arguments are checked against declared parameters. The destination of an assignment must be a subtype of the source (e.g., Base b = new Derived() is allowed). If a method name matches a base, its parameters and return type must match too. And a non-void method must end with return (required check due to my extension making return a general statement).

7. Contexts: (context.h) Borrowing a page from perl, three contexts are defined: void, boolean, and value. Void context occurs when the result of an expression is not used; boolean context is an optimization to allow certain comparisons to pass their results in flags. Expressions in void context can be removed from the AST entirely except for method calls and division (since it could be a division by zero which has the side effect of terminating).

8. Measuring: (quant.h) Determines sizes for classes, vtable offets for non- static methods, and frame offsets for parameters and local variables. If local variables aren't used, they don't take up any space (very basic check).

9. IR: (ir.h, irv.h) The IR visitor builds a linear, 2-operand IR. (In retrospect, a 2-operand IR is simpler which is both good - it's a lower-level representation in which the universe of instructions is more restricted - and bad: some complex expressions require multiple IR instructions.) IR operations are basic (set, store, load, arithmetic, call, jump, label) and the allowed operands are constants, temporaries (future registers), labels, the frame or stack pointers, "this", and a conceptual return value. We also build a string table in this step; duplicate strings are coalesced.

10. Constants: (const.h) (This series of IR manipulators, inheriting from class IM in ir.h, is evoked from CGV::GenerateCode in code.h) This IR manipulator replaces temporaries set to constant values with the constants themselves, and also replaces series of constant expressions with the evaluated result. It also eliminates identity operations (add/subtract 0, multiply/divide by 1), and either removes or removes the condition from conditional jumps based on a constant.

11. Jumps: (jump.h) A few IR manipulators here snap jumps to jumps, remove jumps to immediately following code, and remove dead code (code between an unconditional jump and the next label, after removal of unused labels).

12. Frame: (frame.h) This IR visitor checks if the frame pointer is used. If it never is, we may be able to optimize it out later.

13. Instruction selection: (insn.h, low.h) This final IR visitor generates machine instructions (a list of LL objects). It is primarily a direct translation of the IR, with some optimizations (e.g., t0 <- FP, t0 += const, t0 <- (t0) can be matched to a single instruction). A few marker instructions are emitted to help with register allocation: start of call, end of call, epilog, prolog.

IR temporaries are still temporaries, but some register names are emtited (see e.g., LloFromOpr in insn.cc): "this" becomes ECX, return value is EAX, frame/stack EBP/ESP. Some registers are already marked as being required, defined, or used by certain instructions (see LL::DefUse in low.cc). Some register affinities are set: e.g., "idivl" uses EDX:EAX and outputs to EAX, "cdq" requires EAX and outputs EDX:EAX, and "set<cc>" requires (the low 8-bits of) EAX, EBX, ECX, or EDX on the x86 (restrictions loosen with x86_64).

14. Peephole optimizer: (peep.h) Doesn't do very much, although I had hoped to do more by storing symbolic values of the expressions in live registers and eliminating temporaries that reuse these values.

15. Register assignment: (reg.h) Manipulates the low-level instruction list by "painting" temporaries with register names. It first walks the instruction list and finds live ranges, by tracking when a temporary or register is first assigned to, when it's read, and when it's reassigned (it is an error to use an undefined temporary or register; we don't generate such code; this is not related to undefined variables, which are at a higher level, although similar code could be used to detect them earlier). ECX is artificially added as live for the whole method for non-statics.

Next, an interference graph is built, and registers are assigned in order (a cleverer algorithm could be used, but I never found a case where it was necessary, although this is not to say they don't exist; I posted to the discussion board to ask for input here). All live registers are saved/restored around function calls. This is not optimal, since the platform only requires EBX, ESI, and EDI to be saved, but I decided the internal conventions would be that only EBP need be saved, since at this point saving registers in the prolog would be difficult (it would either require modifying EBP offsets or later compound statements); this is another case where a 3-operand IR could have passed along more information and made it easier to adjust after the fact.

If no registers are available, a temporary's adjacent registers are examined; we pick the least-used candidate that doesn't collide with us, and save it around our definition/use ranges. (Collision means that the register's definition and use is interleaved with the temporary.)

E.g., in this hypothetical code segment, t3 denoting an unassigned temporary:
... (%ebx populated earlier)
    1 movl $5, %edx
    2 movl $10, t3
    3 addl %ebx, t3
    4 movl -4(%ebp), t3
... (%edx used later; %ebx and t3 not used again)
%edx is live in [1, >4], %ebx is live in [<1, 3], and t3 is live in [2, 4], so %edx is a good candidate for t3 (since its definition is before t3's and its use is after, i.e. it encloses t3 completely) but %ebx is not, since its use is interleaved with t3's.

16. Peephole optimizer: (peep.h) Run the optimizer again to see if it can do anything after register assignment or other modifications.

17. Clean: (CLN, low.h) remove any marker instructions (they're emitted as comments if they do get to the listing, so for tidiness only) unless --debug-code was specified.

18. Output: generated code is written either to stdout, or to a file specified by the --output (-o) parameter. We don't do any assembling or linking, although the Python program test/build will, as will test/runtests, assuming the GNU toolchain (as to assemble, and g++ to link, since libmj is built as C++).

There is a list of optimizations I considered (but lacked time for or the design didn't match well with them) in the file src/OPTIMIZE.

To build and run the test programs, cd to test and run ./runtests. It will just show success/failure, but it leaves the output so assembly/binaries can be examined/run afterward. I have included the assembly in the archive.

XBMC: hide reboot and shutdown options

News, Technical ·Tuesday December 15, 2009 @ 05:57 EST (link)

Since I run XBMC from MythTV, I just want a way to return to MythTV; I don't need the extra reboot/shutdown/hibernate/etc. options on the home menu. Fortunately this is easy to change. Find the directory defining your theme (skin); in my case, /usr/share/xbmc/skin/PM3.HD, for Project Mayhem III (HD), and find the file Home.xml (most likely in a subdirectory such as 720p). Edit it and find <onclick>ActivateWindow(ShutdownMenu)</onclick>; change it to read <onclick>XBMC.Quit()</onclick>, and save. If you just want to modify the shutdown menu, rather than having clicking the red power button exit XBMC, edit DialogButtonMenu.xml. If you also want to get rid of the Favorites menu, find <description>Favourites push button</description> and add <visible>false</visible> just below it. (Thanks to cptspiff in Freenode #xbmc for help with this.)

DVDs finished: Star Trek: Deep Space Nine - Season 7, The Karate Kid, The Karate Kid Part II.

Vacation until the end of the year

News, Work, Photography, Guns ·Sunday December 13, 2009 @ 07:18 EST (link)

I'm on vacation until the end of the year (actually January 2nd, since New Year's Day is a Microsoft holiday). It wasn't exactly by choice; we can only keep twice our annual vacation time; anything over that is subject to forfeiture. I calculated what I'd lose and worked backward from the end of the month, skipping company holidays, and ended up with the 11th.

So far I've been spending most of my vacation on the "MiniJava" compiler for my Master's class, so haven't had much time to relax.

Given the length of vacation (and the price of plane tickets and discomfort of flying in general), we'd considered driving across the country to both our parents' for Christmas, but my camera died, and I sure wasn't going to drive all the way across the nation without a way to take pictures.

The camera, my Nikon D300 was found dead on Thanksgiving, when I brought it in from my car: it alternated between the display not showing anything, to showing the normal shots remaining count that appears when it's off, but not turning on. I don't know what caused the problem; possibly condensation from being in the car (it was just in a soft case). Glazer's, where the camera was bought in December 2007 (not by me; I got it used a few months later when the seller upgraded to a D700), recommended I either send it back to Nikon (long wait) or try nearby repair shop Photo-tronics. Their estimate was just under $500; I decided to go ahead with the repair. (Note: parked at nearby "green" shop parking lot, since said lot was nearly empty.) When I go back to pick it up I may try to sell Glazer's my D100 or F90X, and maybe my Tamron 28-200 lens.

We got a membership (silver annual "family") at West Coast Armory's new indoor range in Bellevue (brochure with rates). It's disappointing that they don't allow steel-cased ammo; many indoor ranges don't because it interferes with the machines that their shell casing pickup company uses. So I either need to re-up my SVRC membership, go to the Sultan pit a lot, or try to sell my steel-cased 7.62x54R and .223. In any case I need to try to find brass-cased in both calibers (I suspect 7.62x54R will be less common, since most surplus is steel-cased).

Books finished: Unfair Competition: The Profits of Nonprofits.

Discovering XBMC; connecting it to MythTV

News, Technical, Media ·Friday December 4, 2009 @ 19:10 EST (link)

Even with the 0.22 release I'm unhappy with MythTV's UI, especially for videos. Although, for example, most of my movies are found so their thumbnails show up in the selection screen, the images are tiny unless you go to the trouble of selecting one. Its TV episode lookup apparently won't find my episodes (since they're not named according to its rigid conventions). And even with the new 0.22 themes it's not as slick as one would hope with all the graphical abilities available. And it's not that configurable. For example, recently I thought I'd like to allow the user to select which movie poster image is used (rather than using the first one available on TMDB). So I went browsing for something else, and came across XBMC (originally XBox Media Center, now adapted to other systems including Linux, and renamed to the somewhat redundant "XBMC Media Center"). The graphics are very slick and it's extremely configurable, with an embedded Python (2.6) interpreter (looks like it would be easy to upgrade to 3.1 if I wanted to) that can be used for scripts or "plugins", which are sort of like virtual file systems (e.g., there is one that makes MythTV's recorded programs appear as a folder), with some useful built-in functions, including ones for windowing and dialogs.

I found a way to add a button to MythTV's "Media Library" menu to launch XBMC, but it was a little harder to determine how to give that button an icon (turns out you set the <type> to something unique, like XBMC and specify an icon for it in the theme's menu-ui.xml file; or rather, two, at least for the default Terra skin: one for when it's active (visible), and one for when it's selected. I couldn't find any images of the right size, so I grabbed one from the XBMC site, scaled it to match the other icons, and made an "active" version in Photoshop with a transparent lightning overlay, as seen here.

You are welcome to use them if you need XBMC buttons for MythTV. We'll probably still use MythTV for recording TV programs, so we need a way to go back and forth.

(Paths are for Gentoo; modify to suit.) To add the button, copy /usr/share/mythtv/themes/defaultmenu/library.xml to the .mythtv directory of the home directory of the user that MythTV runs as (usually /home/mythtv), and add the following right above the closing </mythmenu> tag:
    <button>
        <type>XBMC</type>
        <text>XBMC</text>
        <description>Launch XBMC</description>
        <action>EXEC /usr/bin/xbmc -q</action>
    </button>
(Alternatively, grab this file, but there's a risk of missing intervening updates.) As written, this will give you a button with the unsightly white on black text "No image available" (note that you'll need to leave and re-enter the menu to see the new button, but you don't need to restart MythTV). To give it an image, save the two images here as (left) xbmc_off.png and (right) xbmc_on.png in /usr/share/mythtv/themes/Terra/watermarks. To have them show up, add the following to both the end of the <state name="active"> / <statetype name="icons> and the <state name="selected" from="active"> / <statetype name="icons> blocks (i.e. after the <state name="ZM_EVENT_VIEWER" from="ZOOMFINDER"> block):
    <state name="XBMC" from="default">
        <imagetype name="icon">
            <filename>watermarks/xbmc_on.png</filename>
        </imagetype>
    </state>
Alternately (again risking missing an intervening update) you can use this file. Note that you unfortunately can't use a local version of this file as was possible with library.xml. You need to restart MythTV (exit it and depending on your setup, it should auto-restart, if not, relaunch it by restarting X, usually with startx; if you've disabled exiting, you may need to kill X with ctrl-alt-backspace; if that's disabled (as newer X.org servers do), you may need to kill X remotely. The image should now show up in Media Library, with the lightning bolt overlay when selected; clicking it will launch XBMC. Default ("confluence" skinned) XBMC can be exited by moving left to the "Power" button and selecting the red "X" (my remote "just worked" for moving and selecting; YMMV).

Next step: add MythTV as a video (and perhaps music) source. That will be fine for now, but eventually I'll want to have XBMC track videos only, so I can used its improved scripting and plugins without going through MythTV. I hope this was helpful—I didn't find anywhere else online with all the steps. No doubt I'll have more to say about XBMC as I use it: hopefully all good.

Books finished: Ayn Rand Answers, The Cat Who Walks Through Walls.

MythVideo upgrade: better metadata

News, Technical, Media ·Friday November 27, 2009 @ 23:30 EST (link)

I recently upgraded the MythTV box (to the current Gentoo ebuilds, including 0.22 the MythTV application). I did this to get some new features of MythVideo including improved metadata storage and lookup for videos and TV shows. The new themes look nice too, for the most part; sometimes text is too small, and since I don't have cover art/posters for most of my movies/shows there are a lot of very big images of question marks (or blanks). But that will be remedied. The included scripts in /usr/share/mythtv/mythvideo/scripts (mainly tmdb.pl and ttvdb.py) that find episodes (using TheTVDB.com and themoviedb.org) are a reasonably good place to start, but they don't conform too well to my naming conventions. Fortunately they are both in languages I know extremely well: open source means you can make things better, and fairly easily (even if it's just a local benefit); it's a knowledge economy.

Having posters for most movies makes the MythVideo browsing UI pretty nice. By default, the shortcut key W will by default invoke one of the two scripts above (configurable, of course; so I could fix the scripts or replace them with my own) which will pull metadata from one of those two free sites (there are other scripts that will use IMDb or other sites as sources). They have a well-defined interface as to how information is communicated between them and MythTV.

For instance, it would be nice to be able to select the poster used for a movie visually. Is it possible to pop up a UI from a script within an existing X session? I don't see any reason why not; and indeed, the PyGTK tutorial's hello world script works great. (I went with GTK because Tk and Motif and wx and the older ones looks butt-ugly and have severely limited functionality.) It can even be done from an ssh session as long as it's prefaced by export DISPLAY=:0.0 (as appropriate). (It's lousy that it segfaults if DISPLAY isn't set, though.)

So what's wrong with the tmdb.pl and ttvdb.py scripts? Why not use them and be happy? Perhaps I can, but these are the hurdles:
  1. They don't understand my naming convention. For example, I group some movies into directories and remove redundancies, e.g. Harry Potter/1 The Philosopher's Stone (the 1 keeps the order correct; there isn't a sort override field in the videometadata database). The script tries to find a movie called "1 The Philosopher's Stone" rather than keeping the directory and stripping the numeric prefix. If I used the full names (Harry Potter and …) they'd still be out of order (alpha-sorted). Possibly fixable by modifying the script itself, but then I'm doomed when I upgrade.

  2. It picks the first movie poster available; I'd like to be able to select one (see above regarding PyGTK). E.g., the first poster for Transformers: Revenge of the Fallen is pretty dark and gloomy and uninformative.

  3. When the database is rescanned, old metadata is destroyed and has to be re-fetched from scratch, meaning unnecessary and slow network traffic, and items with multiple matches have to be user-selected again. Granted, one fix is to just not run "Scan for changes", although I don't know what MythTV will do to the database in future. But this can be overcome by automated database backups.

  4. I want to tie it in with my DVD watch list.

There's a useful new utility called Jamu (Just Another Metadata Updater—really, those names aren't clever any more, people, although at least it provides a recognizable and unique program name). It can do lookups for various metadata and artwork, so I'll certainly make use of it in my own utilities.

Aside from the MythTV stuff there was the usual collection of blocking package issues which eventually got sorted. The Myth box needed 430 packages; then I decided to emerge -uDNv world on the server too, which also had its share of issues. For a while I had some corruption on torrents created by one machine on a (Samba) shared directory on another. When I upgraded both to kernel 2.6.31 the problem went away; I'd narrowed it down to Samba (vs. LVM or something else) being the culprit, and there were fixes for CIFS corruption between my old (2.6.28) kernel and this one.

Speaking of database backups above, I borrowed a script from another machine and started automatic backups of the MythTV database. I suppose I can just use the videometadata table if I can be confident that it won't randomly disappear, or if it does, I can get it back with minimal hassle. (I never need to actually rescan for new content, since I already have programs that monitor my video directory with inotify and add videometadata entries appropriately, and a non-destructive scanner.

Given the current cost of HD space (e.g. an external USB 1.5T drive is a little over $100 today at Newegg.com), it's almost as cheap to record data (show episodes, or whole DVDs) on a (potential chain of) USB HDs (possibly linked as a logical volume, or symlinked from a common directory) than it would be to use DVD±Rs or (given the stage of development—8x and 12x burners are rare and expensive, especially external—and burn speed, and low penetration compared to DVDs) Blu-Ray disks. Perhaps Bob is onto something, at the current price point.

I also installed MythMusic, but need a tiny utility to set ID3 properly (just something simple that uses the directory and filename) (Mutagen, a promising Python ID3 library). When the TV suspends or powers down it also turns off the music (since it goes through the TV to the amp), so I increased the blanking time in xorg.conf.

Useful modules: IMDb lookup in Perl (IMDB::Film; tested, works) and Python (imdbpy); Python 3 (what I'm using but modules can be hard to find since it's relatively new) PostgreSQL module.

Thanksgiving 2009

News, Photography ·Friday November 27, 2009 @ 18:36 EST (link)

We had two of Honey's friends (Jenna and Heba) over on Thanksgiving; they cooked, I was happy (and fairly busy working on the compiler for my course). We had a turkey breast (with stuffing and cranberry sauce of course), Brussels sprouts, mashed potatoes, and some vegetables that were cooked with the turkey. And apple pie and ice-cream for dessert. It was great, except when I went to get my camera from the car it was dead (no readout at all—not even shots remaining count, which usually shows up even without a battery). So that soured the day for me.

The camera revived earlier today (Friday), died again, revived again, and died… it seems to think it's Easter, not Thanksgiving. Things were looking good, but the present prognosis is negative.

Books finished: How the States Got Their Shapes.

DVDs finished: Harry Potter and the Order of the Phoenix, Ice Age, Forgetting Sarah Marshall, M*A*S*H: Season Ten.

The 28 principles of The 5000 Year Leap

Political ·Friday November 20, 2009 @ 23:40 EST (link)

I recently finished W. Cleon Skousen's The 5,000 Year Leap: A Miracle that Changed the World. In my review I said:
The author means well, and most of the principles are good: but there are some few contradictions where he fails to fully realize the definition of a free people, and goes back on his support for natural law and property rights and espouses the tyranny of a majority: however on the whole it the book is a good compendium of the American founder's ideals in a modern context.
I'd like to expand on that and examine the 28 principles listed in the book as they relate to libertarian philosophy in general and mine in particular.
  1. The only reliable basis for sound government and just human relations is Natural Law.

    "Natural law" is a bit vague but the theory is of a common law that civilized people recognize: of course, nobody agrees entirely as to what it is. Certainly non-initiation of aggression would be included, though, which would also cover things like respect for another's property. As examples he lists unalienable rights (see #8), unalienable duties (#9), habeas corpus, limited government (how limited? the non-aggression principle (NAP) is sufficient here too!), separation of powers, self-preservation, right to contract, and others; in short, it gets your whites whiter and gets bloodstains out of carpet. But it is all unnecessary with (or flows from) the non-aggression principle, a much more succinct axiom on which to build a civilized (voluntary) society.

  2. A free people cannot survive under a republican constitution unless they remain virtuous and morally strong.

    Also somewhat vague. The converse is certainly true: following Tyler's dictum, envious and greedy people of little virtue feel entitled to the product of their hard-working betters. I'd like to live among moral people—specifically, those that follow my morals, as would most people. But I'd also be happy to live among people that followed the NAP: they are then moral by definition.

  3. The most promising method of securing a virtuous and morally stable people is to elect virtuous leaders.

    Better still to have leaders that can't do any harm—i.e., can't initiate aggression against anyone either by direct application of force (assault, confiscation) or threat thereof (taxation).

  4. Without religion the government of a free people cannot be maintained.

    Many nations manage to stay "free" in today's terms (i.e., they can vote to change the names of their overlords every few years) without particular virtue (in fact scandals involving politicians are legion) or religion. Certainly one would hope that faith in Christ and a devotion to sound Biblical teachings would inspire a leader to do well, but there have been a number of public failures even of Christian leaders.

  5. All things were created by God, therefore upon Him all mankind are equally dependent, and to Him they are equally responsible.

    I believe this, but he does a disservice to Christians everywhere with his hand-waving in this section; furthermore, it is not a principle but a statement.

  6. All men are created equal.

    This chapter is excellent: it emphasizes that people should be treated equally by the law and have equal rights, but that people are not born with equal faculties, and are not entitled to equal influence, property, or other advantages. To that I add people make themselves unequal by their choices in life: someone with a better credit record will legitimately be treated better than a profligate debtor, or a felon. But some of the examples he gives look good but are wanting: he says that people's right should be protected equally… "at the print shop", for example, without regard to the rights of the owner of said shop, who should not be required to do business with anybody and should be free to discriminate as he wishes in his private business (and suffer the consequences if he does so unfairly).

  7. The proper role of government is to protect equal rights, not provide equal things.

    An extension of the previous.

  8. Men are endowed by their Creator with certain unalienable rights.

    Most of his examples are true and good, and are derivable from the NAP and self-ownership: rights to self-government, bear arms, own, develop, and dispose of property, make personal choices, free conscience, choose a profession, choose a mate, beget one's kind, assemble, petition, free speech, free press*, enjoy the fruits of one's labors, barter and trade, invent, explore*, privacy, provide personal security, nature's necessities (air, food, water, clothing, shelter)*, fair trial, free association, contract. I would take issue with the starred positive rights listed, e.g. there is no right to explore if it requires trespassing, or to food via theft.

  9. To protect man’s rights, God has revealed certain principles of divine law.

    Also vague.

  10. The God-given right to govern is vested in the sovereign authority of the whole people.

    No! This is the same tyranny of the majority that subjects us to so much redistributive theft today, to pay for so many programs that are not only unused and unwanted, but also unhelpful, e.g., welfare introducing a generational spiral of dependency and disincentives to ambition and work.

  11. The majority of the people may alter or abolish a government which has become tyrannical.

    True, with the caveat that it would be immoral to attempt to replace a NAP-following minimal government that efficiently handles national defense (not military adventurism), police to prevent or stop initiation of force or fraud (but not to hassle people for victimless crimes) with one of another type, that will initiate force against people and steal from them.

  12. The United States of America shall be a republic.

    Although this enhances scalability it didn't manage to stop legalization of slavery, passage of the Federal income tax, Japanese internment, Social Security/Medicare, and massive taxation, spending, and debts. Since we have the technology, direct democracy could hardly be worse. Presence or absence of the NAP as a governing principle would vastly overshadow republicanism.

  13. A constitution should be structured to permanently protect the people from the human frailties of their rulers.

    Indeed. Include the NAP; it's the only way to be sure.

  14. Life and liberty are secure only so long as the right to property is secure.

    Property rights are key. The NAP extends to others not violating your property rights: trespassing and theft are aggressive acts.

  15. The highest level of prosperity occurs when there is a free-market economy and minimum of government regulations.

    Amen!

  16. The government should be separated into three branches – legislative, executive, and judicial.

    It seems to work. This is more about the deck chairs than the course of the ship.

  17. A system of checks and balances should be adopted to prevent the abuse of power.

    Better to use one big gun: the NAP. Checks and balances haven't always worked out so well.

  18. The unalienable rights of the people are most likely to be preserved if the principles of government are set forth in a written constitution.

    Absolutely. But be careful who you allow to interpret it.

  19. Only limited and carefully defined powers should be delegated to government, all others being retained in the people.

    Very limited. No initiation of aggression, for example. Not far enough, Skousen; go further!

  20. Efficiency and dispatch require government to operate according to the will of the majority, but constitutional provisions must be made to protect the rights of the minority.

    False; see above. If there's a threat, people will react: if, for example, the limited defensive military detected a great threat, patriots would, as in the days of the Minutemen, rally to provide "blood and treasure."

  21. Strong local self-government is the keystone to preserving human freedom.

    Subsidiarity is a sound principle, all the way to the individual.

  22. A free people should be governed by law and not by the whims of men.

    Of course.

  23. A free society cannot survive as a republic without a broad program of general education.

    Maybe, maybe not: not government's business to rob people to provide it. Diligent parents and virtuous citizens will make it available; and if not, then it is not desired and robbery is no legitimate way to source it.

  24. A free people will not survive unless they stay strong.

    Agreed. While initiation of aggression is frowned upon by libertarians, tempered self-defense constitutes a valid response to aggression.

  25. "Peace, commerce, and honest friendship with all nations – entangling alliances with none."

    Definitely. He uses a good term, too: separatism, not isolationism. Read Ron Paul's speeches; you'll see it all the time; see, for example, his books A Foreign Policy of Freedom and Pillars of Prosperity.

  26. The core unit which determines the strength of any society is the family; therefore, the government should foster and protect its integrity.

    False. The government should protect everybody, and should not be in the business of favoring any special interest group, even one as basic as the family. Not everyone should marry or have children; some may not want them or cannot provide for them, and should not be maltreated because of it. Marriage or any union should be a contract; the government's only role is to provide courts to enforce it.

  27. The burden of debt is as destructive to freedom as subjugation by conquest.

    Overly dramatic, perhaps, but I never consented to any debt: it is aggression visited upon us and our descendants.

  28. The United States has a manifest destiny to be an example and a blessing to the entire human race.

    Sure—but so does everyone else. For a while, though, we shone the brightest: may we continue to do so.

Books finished: The Fires of Heaven, Between Planets, The 5000 Year Leap.

DVDs finished: Harry Potter and the Goblet of Fire.

SCons: adding a post-processing step

Technical ·Monday November 16, 2009 @ 22:58 EST (link)

I got annoyed by this warning generated by compiling my project's Flex scanner:
flex --header-file=scanner.h -DECHO= -t scanner.l > scanner.cc
g++ -o scanner.o -c -ggdb -Wall scanner.cc
<stdout>: In function 'int yy_get_next_buffer()':
<stdout>:1340: warning: comparison between signed and unsigned integer expressions
I looked it up and it turns out that although it's not fixed in Flex itself, it is logged and fixed as Debian bug 466793 in the Debian package: the YY_INPUT macro in the generated code (scanner.cc here) should have size_t in one place where it has int. Since my system is Gentoo, not Debian, I couldn't (consistently) install the patched Flex, but I could patch the generated scanner.cc file before compilation in the SCons build file, by adding a post-processing step to the action list for .l (Flex lexer definition) files as follows:
c_file, cxx_file = SCons.Tool.createCFileBuilders(env)
cxx_file.cmdgen['.l'] += [
 r"perl -pwli -e 's/^(\t\t)int( n; \\)$/$1size_t$2/" +
 r" if /^#define YY_INPUT/ .. /^#endif/' $TARGET"]
Now the relevant build steps become:
flex --header-file=scanner.h -DECHO= -t scanner.l > scanner.cc
perl -pwli -e 's/^(\t\t)int( n; \\)$/$1size_t$2/ if /^#define YY_INPUT/ .. /^#endif/' scanner.cc
g++ -o scanner.o -c -ggdb -Wall scanner.cc
I have, of course, glossed over exactly how I knew to call SCons.Tool.createCFileBuilders, and that adding to cxx_file.cmdgen['.l'] would work and do what I wanted; as it happens, I had to browse the sources for that, although it's all in the inline documentation. On my system the SCons modules are under /usr/lib64/scons-1.2.0/SCons. It turns out that cxx_file is a SCons Builder.CompositeBuilder object, whose cmdgen field is a Builder.DictCmdGenerator whose values are SCons action lists. It might be better if it were not necessary to access the un(py)documented cmdgen field (one might assume that the add_action method would do what I wanted, but unfortunately it replaces the old action, that of running Flex in this case).

It's sufficiently elegant for a postmodern high-level language.

Books finished: Night of Power.

DVDs finished: Harry Potter and the Prisoner of Azkaban.

CCW training with "University Defenders"

News, Guns, School ·Sunday November 15, 2009 @ 17:07 EST (link)

Some UW students were being robbed around the University District area, both in their residences and on the street, and Stanley Luong and a few others had decided to carry concealed weapons to prevent being robbed again (article from The Daily). I don't blame him in the least. Where he got into trouble was starting a Facebook group with "vigilante" in the title and description, talking about going after or attempting to decoy potential robbers; it was deleted by Facebook and later reappeared with the less provocative title University District Defenders. Richard Walker, an NRA-certified instructor with his own firearms training company, Black Dawg Partners, offered the group a free CCW (concealed carry weapon) training class, and Champion Arms in Kent donated classroom time. I decided to attend to meet the group and see what the training was like. The range isn't hard to find, and it's not that far away: I-405S to WA-167S, get off on E. Valley Hwy., and follow it to 18801: about 30 minutes, but surely double that in traffic. I got there exactly at the start time, 1230, but most people were late. There were six UW students and another interested party, and two trainers (Richard and David).

Richard is extremely well qualified and was friendly and a good teacher, making his points clearly and without talking down to anyone. I confess I had my doubts about him before we met—I was worried that he would encourage the group in their early "vigilante" ideas, or try a hard sell of more advanced courses (this class was free—"free" always makes me suspicious), or would put the group down unnecessarily. None of those happened: he came across as serious and knowledgeable and any "vigilante" ideas melted away as he went over firearms safety, best practices, local laws, and some court cases. The rest of the group went down to the range afterward, but the class took four hours, longer than I'd expected (and told Honey), so I left right after it finished and got home at around 1700.

This is the message Stanley Luong sent to the Facebook group after the event:
Thanks to everyone who showed up and helped out. The day began at 12:30pm with Richard teaching safety guidelines and handling techniques. We had a break to give an interview with KIRO 7 and the Seattle Times. After that, we continued our meeting with Richard leading the discussion on legal issues, ramifications, and several case studies of self-defense scenarios. Overall, yesterday was a great forum in raising our awareness of gun rights and safety, and ended with us shooting at the range. Special thanks to Black Dawg for their pro-bono training and Champion Arms in Kent for hosting us.

Next time, we will try to have an event closer to campus. If we had one on campus, there would definitely be a large turnout but we would all have to go unarmed because of the regulations there.

This is what I sent to msgun Monday:
I attended—free training, not too far way (~30 mins), what's not to like?, plus I wanted to meet some of the people in the group.

First, the group has indeed toned down. There was some hare-brained scheme proposed in the Facebook group to cruise around with a "decoy" wearing an iPod and several other shadowy figures with nonlethal weapons ready to pounce, so I did my best to shoot that down (no pun intended) and point out its many flaws (you're not the police, you're not trained, if you're in fear of grave bodily injury you don't want to bring anything less than deadly force, you could be hurt/killed, you could hurt someone else, jail, civil court costs, even with a "good shoot", etc.). The original poster agreed that his was a foolish idea.

Second, this training outfit looked legit—Rick Walker (http://www.blackdawgpartners.com - not the most pro site in the world though) is a class act and knows his stuff, ex-military (weapons expert), certified as instructor by several reputable groups, etc. He teaches various courses including the Utah permit course. He went over some basics of weapons and CCW with a PowerPoint deck in a classroom at Champion Arms (they offered the room free)—starting from Cooper's 4 rules, then things like where you may or may not carry, Castle doctrine, reacting to scenarios—run if you can, call 911 if you're not in immediate danger; importance of practice; all stuff I've seen but nice to have it all in one place. He went over some cases, what people did, what they were convicted of, and their civil costs. I can reproduce the quiz he passed out before the class and discussed if people want. I believe it was sufficiently sobering to cause the "Defenders" to change their course, which is great: more armed good guys on the streets that know the law and practice with their particular defense weapon is a good thing.

I also talked to him about doing training for a Microsoft group; he said he could do something similar (a basic CCW class) for just cost of gas and materials; perhaps if we were interested he could give us a rate on doing a Utah permit course or something else for those interested.

The session Sunday got the group some better coverage; the image now should be of a group of serious people armed and able to protect themselves and others from grave bodily harm.

DVDs finished: Butterfly Effect: Revelation, Harry Potter and the Sorcerer's Stone, Harry Potter and the Chamber of Secrets.

A comfortable violence

Political ·Friday November 13, 2009 @ 18:56 EST (link)

This piece is by Elliott Prechter, via the Austrian Economics Discussion list at work. I'm forever curious as to what makes non-libertarians, especially big-government supporters, tick. Why are they not libertarian? What do they have against freedom? I tend to get two kinds of answers: one involves greed, and the other involves fear, sometimes for themselves as Elliott describes so well below, and sometimes for hypothetical groups, frequently "poor people", who will imaginably (usually counter to fact) be so much worse off in a truly free society.

I think a lot of people don't want liberty. At a deep level, many voters want good, kind, motherly totalitarianism—so they can feel safe and secure. I understand where they're coming from on an emotional level, but it's just that I don't believe such a thing as "good despotism" exists, or has ever existed. Freedom isn't perfect, but history has shown it to be the best system we've got—as Winston Churchill said "Democracy is the worst form of government except all those other forms that have been tried from time to time."

As an example of many people's natural draw towards big government, many survivors of Communism—who faced imprisonment, poverty, starvation, death—said they still believed the system to be good, but just that the people in charge just happened to be evil and greedy. In their minds, with the right people in charge, despotism would have been paradise!

I think one fundamental reason that despotism has never produced paradise is that it is a system based on coercive violence rather than voluntary action. Violence will tend to escalate due to the perverse incentives in such a system. For example, in a voluntary system starting a business, employing people, and producing wealth is beneficial to the self. Under Communism, this incentive does not exist. However, the incentive to seek bureaucratic control over others—by entering the government—is the only path towards wealth and individual "success". So such a system will encourage more and more corruption because corruption is so immensely rewarded, and virtuous activity (i.e. wealth producing) is punished. It is ironic that the insatiable desire of individuals to better their own lot (which is the common cry against freedom and capitalism) is what causes despotism to fail, and conversely it is this same incentive that powers the immense wealth-producing capacity of capitalism. The only difference is that in the former case force is chosen as the primary means of dealing with one another, whereas in the latter all human interaction is mutual.

The U.S. is very far from despotism of course, but we can still clearly see elements of the "escalation of violence" already in effect as government power grows. Take Goldman Sachs as an example. Under a voluntary, non-coercive system, they would be spending their time innovating and researching to produce better products, which would in turn help everybody and increase net wealth. However, since the US has a large non-voluntary component to our society (namely the massive U.S. government), they are instead using resources, time and effort to get "friends" into government that can bend the rules in their favor; such rule bending (and even outright theft) destroys wealth rather than creates it. This trend—rewarding those who seek to use violent means to control others, rather than rewarding those who work hard and produce wealth—is dangerous for the long-term health of any country. I don't particularly mean to single out Goldman (it's just a great example at present) since this type of corruption occurs all over the world in every government—some more than others.

My main point here has nothing to do with any particular specifics of any political party's agenda—the battle between freedom and despotism is not primarily about details such as healthcare, or firearms, or abortion, or education, or any other specific debates and issues that divide the nation, but rather primarily that the long-term effects of institutionalized violence are to destroy wealth and prosperity, thereby rendering all specific political debates about how to "split the pie" meaningless once everyone is equal in poverty.

A key element of this piece is 'I don't believe such a thing as "good despotism" exists.' Whenever people rule over others, even elected by a system such as ours, they will use violence for advantage. The best system is one where power is fully distributed to individuals and all transactions voluntary, with use of force limited to stopping or punishing initiation of force. Rather than a central government guaranteeing a Republican form of government, how much better if the states made a confederation to mutually guarantee to each other a non-coercive form of government?

DVDs finished: The Puppet Masters.

<Previous 10 entries>