Q:

How to organize a Bento - Veewee - Vagrant project

I'm trying to use Bento, Veewee and Vagrant to automate the process of provisioning VMs.  This approach has a lot of promise for simplifying development, dev-testing and QA processes.
(For those who haven't encountered these -- Vagrant makes it easy to quickly set up / tear down VMs in Virtual Box, VMWare, EC2 etc.; Veewee simplifies the process of building base boxes for Veewee; and Bento simplifies the process of defining base boxes and then automating the operation of Veewee).
The problem is, the interrelationship of Bento, Veewee and Vagrant is confusing.  Many definition files, and some operations, are duplicated across layers and it isn't clear which layer should get the updates that're specific to my project.  The Ruby environment also complicates this quite a lot -- seems like Bento commands all need to be run from the Bento directory, so boxes & other files end up in illogical places.
Can anyone suggest a file layout & workflow for using this combination of tools?

A:

I'm responding to my own question to capture what I discovered.

There's a way to make Bento, Veewee, Vagrant and Ruby work together in a semi-sensible way, but it's complicated.  It's better to use the latest version of Bento that replaces Veewee with the Packer tool.
Packer does more or less what Veewee did, but made some great improvements including: 

Each Packer box definition is in a single JSON file that refers to a flat list of script files -- no longer three layers of files with symlinks; easier to understand and better for source code control.
The Packer binaries are OS-native executables (no longer in Ruby).  You deploy them by putting them somewhere on your path (like /usr/local/bin on Linux).  Packer avoids the Ruby craziness.

Here is a simple workflow:

Install VirtualBox.  Install Bento.  Bento will also install Vagrant as a Ruby gem; but we won't use that instance of Vagrant.  Install Vagrant separately using its native installer for your operating system.
Choose a name for your project to use in dir & file names.   (For this example, I'm using "CUSTOM").  Create an empty sub-directory within the bento / packer / scripts directory.  I did mkdir CUSTOM.
Review the .JSON files in .../ bento / packer.  You can alter one in-place, or it's probably wiser to copy one & alter that.  I did cp centos-6.4-x86_64.json centos-6.4-x86_64-CUSTOM.json.
Edit the "builders" section of that file to change the the VM's disk size, RAM size, ssh port etc.
Edit the "provisioners" section, "output" line to provide a meaningful name for the box file to be created.  (For this example, I'm using "BOXFILE").  Note that you can embed {{timestamp}} to add a Unix timestamp to the file name -- helps to make the file name unique.
Edit the "provisioners" section of that file to refer to any scripts (in order) you want to run in the Packer build sequence.  This would typically be a combination of the scripts already provided by Packer, plus whatever other scripts you build for your custom purposes.

The approach I used looked like this:
"provisioners": [
{
  "execute_command": "echo 'vagrant' | {{.Vars}} sudo -S -E bash '{{.Path}}'",
  "scripts": [
    "scripts/common/sshd.sh",
    "scripts/common/vagrant.sh",
    "scripts/common/vmtools.sh",
    "scripts/CUSTOM/reso_1024x768.sh",
    "scripts/CUSTOM/prompt_cwd.sh",
    "scripts/CUSTOM/standard_dirs.sh",
    "scripts/CUSTOM/standard_utils.sh",
    "scripts/CUSTOM/python27.sh",
    "scripts/CUSTOM/supervisord.sh",
    "scripts/CUSTOM/ruby.sh",
    "scripts/centos/cleanup.sh",
    "scripts/common/minimize.sh"
  ],
  "type": "shell"
}

Note my custom scripts all appear before the cleanup and minimize steps -- this is important.

From the bento / packer directory, run packer build -only=virtualbox centos-6.4-x86_64-CUSTOM.json. If disk images (ISOs) for VirtualBox guest add-ons and base OS haven't been downloaded before, they will be now; these files are big so this step can take a while.
Packer uses VirtualBox to create a temporary VM, and runs all your specified scripts on it.  When this process is done, a copy of the VM is captured in the bento / builds / visualbox directory using the BOXFILE name you specified above, and Packer discards the VirtualBox VM.
Register the box you just built with Vagrant.  Using the separately installed version of Vagrant (e.g. you may have to specify /usr/local/bin/vagrant or whatever's appropriate for your OS), do vagrant box add CUSTOM path/to/BOXFILE.  This creates a new sub-directory in ~/ .vagrant.d / boxes / CUSTOM / VirtualBox with the files Vagrant needs to create VMs from this box definition.
Create a Vagrant project directory and cd into it.  I did mkdir ~/test; cd ~/test.
Initialize a Vagrant project using this box: vagrant init CUSTOM
Start the VM: vagrant up
Connect to the VM: vagrant ssh

At this point, you're using Vagrant normally and Bento / Packer are no longer involved -- their combined job is just to create the basebox, and once that's been registered with Vagrant, Vagrant stands alone.

There are lots of other interesting things to look at re: Vagrant, like the special vagrant-aws plugin, and how to use it with Chef, Puppet or Ansible ... but I will stop here since my original question is answered.

