DVS: Deployable Versioning System - Ver 0.1

Copyright (c) 2007 by Bruce Cropley

All rights reserved

History

  • xx Mar 2007 Commenced
  • 22 Mar 2007 First commit to local CVS
  • 3 Jun 2007 First commit to SourceForge CVS
  • 12 Jun 2007 First commit to git
  • 4 Nov 2007 Version 0.1 tar.gz file distribution uploaded to SF
  • 17 Mar 2009 Converted SF repository to git, uploaded git history

DVS Introduction

DVS is designed to automate and track the development, testing and installation workflow of a set of deployables. The command line interface is intended to be similar to those of CVS, SubVersion and similar version control tools.

Purpose

DVS was written to address some common problems of in-house development and deployment, including:

  • The executable or package that was tested may not be the exactly the same one that was intended to be released by the developers
  • The executable or package that was deployed may not be the same one that was accepted by QA
  • Rolling back to previous versions is time consuming as the original binary may need to be rebuilt, or someone needs to manually track it
  • It is hard to know which machines have had which versions installed.
  • A different combination of people may need to know about the state transitions that a deployable makes
  • If queues of work are building up or releases are being pushed back to previous stages, it is useful to know where these are occuring and how much time is being taken
  • DVS is intended to be easily integrated into version control, build and bug/issue/planning systems

Concepts

DVS tracks binary “deployables” in a central repository. A deployable can be a installable binary package such as a Solaris package, or it could potentially be a change to a configuration file or database. (TODO)

Deployables are tracked with an ascending integer id that is allocated as the deployable is entered into the system. The deployables can go through states and transitions including:

         | create
         V
Backlog
         | develop
         V
Development
         | build
         V
Building
         | register
         V
Registered
         | qa
         V
QA
         | accept
         V
Accepted
         | deploy    |
         V           |
Installed            |
                     | reject
                     V
Rejected

There is no protection against any particular transition being made.

DVS can be configured to automatically send emails to those who are subscribed to one of:

  • the deployable instance (e.g. this particular binary for app1)
  • the purpose of the deployable (known as its “module”)
  • the state that is being changed to (E.g. accepted by QA)
  • or just to everything

Usage info

There is some online help (this copy is probably out of date):

$ dvs --help

Usage: dvs [dvs-options] command [command-options-and-arguments]
  where dvs-options are -q, -n, etc.
        (specify --help-options for a list of options) TODO
  where command is add, admin, etc.
        (specify --help-commands for a list of commands)
  where command-options-and-arguments depend on the specific command
        (specify -H followed by a command name for command-specific help)
  Specify --help to receive this message

$ dvs --help-commands
DVS commands are:

                accept       Accept by QA
                add_host     Add a deployment target host
                build        Mark a module as being ready to build for release
                config       Configure the DVS database
                del_host     Remove a deployment target host
                deploy       Deploy to production on the current machine
                help         Show the general help
                log          Show transitions made
                qa           Move a binary into QA and install on the current machine
                reject       Reject by QA
                stat         Show deployables per state
                subscribe    Subscribe to transition emails
                totals       Show total times spent in each state
                unsubscribe  Unsubscribe from transition emails


$ dvs --help accept
accept: Accept by QA

Usage: dvs <general_options> accept <accept_options>

Options:
  -h MACHINES, --host=MACHINES
  -r REASON, --reason=REASON
                        The reason for the transition, viewable by dvs log
  -M, --merge           Merge from existing deployables in related and
                        destination states.
  -O, --overwrite       Replace existing deployables in destination state.

Quick Start

  1. Make sure a recent version of python is in your PATH (I’m using 2.5)

  2. sudo python setup.py install

  3. Create a new shared directory that is writeable from build, qa and production machines by all users that need to use the dvs system.

  4. All users of dvs must set their DVSROOT environment variable to this directory

  5. They must also have the same version of python in their PATH; the dvs executable should have been installed into that path.

  6. Decide where you want to search for candidate deployables by default (autobuild_path)

  7. Determine your smtp host and default email domain

  8. dvs init --autobuild_path=my_build_path \

    --smtp_host=mail.something.com --mail_domain=something.com

  9. Use the other dvs commands, starting with dvs --help build

Running the tests

To run the unit tests, go into the test_dvs directory and run:

./unit_tests.py

The system tests require the DVSROOT and PYTHONPATH environment variables to be set up. There is a shell script that you can source that will do that first:

. ./source_me.sh
./system_tests.py

Be careful not to run the system tests without temporarily setting your DVSROOT, or you’ll end up with rubbish in your dvs logs. Once you have run the system tests, you can use that environment to experiment with the dvs commands.

Developers

If you have installed dvs, you will need to override where you get the modified source from, e.g.:

export PYTHONPATH=TheTopLevelDVSWorkspaceDirectory

To build the documentation into html, you will need Sphinx

Implementation

DVS is written in Python. The main classes are:

VersionManager
The central high level coordinator class.
Deployable
A deployable instance wraps a version of a module that has been built into a binary that is going through the release process. DeployableComponent derived classes represent types of deployables, such as SolarisPackages. TODO: in the process of splitting Deployables from DeployableComponents Deployable will need renaming to Release.
Module
Each deployable is an installable instance of a particular module type. The module is typically the name or handle of an application in the version control system.
State
DVS comes with some standard states that a deployable can be go through, such as Accepted, In QA etc.
Command
Each action that can be performed by DVS is represented by an instance of a specific Command derived class.
TransitionEntry
DVS logs all the state transitions that have been made by each deployable. These are stored as instances of TransitionEntry.

Unit Tests

Some of the unit tests depend on the head of the python-mock sourceforge project: python-mock.cvs.sourceforge.net If you make any changes to dvs, please maintain and extend the unit tests, and send me a patch to incorporate into future releases.

Inspiration & Further Reading

  • “Lean Software Development - An Agile Toolkit” by Mary & Tom Poppendieck
  • “Toyota Production System” by Taiichi Ohno
  • “The Cathedral and the Bazaar” by Eric Raymond

License

The DVS system and associated files are released under the FreeBSD license. Essentially, you can do what you like with it except pretend you wrote it yourself.

Copyright (c) 2007, Bruce Cropley

All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

  • Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
  • Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
  • Neither the name of this library nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.