The technical side of Commons v1.0

As Matt @admin announced earlier today, we’ve just tagged the CUNY Academic Commons “version 1.0”.

The 1.0 milestone represents a couple of big changes in our development processes.

  1. Versioning

    The most obvious change is the use of version numbers. It might seem at first like a relatively minor and superficial alteration in the way we operate. In truth, it’s a big change.

    Prior to the switch, our development cycles were haphazard at best. Bugs would be fixed and features implemented whenever someone on the development team happened to get around to it. That system (or lack thereof) works pretty well for a small team. But the team behind the Commons has been growing steadily over the past year, both in sheer number and in breadth (of geographical distribution, of areas of technical expertise, of responsibility). Greater distribution requires more thoughtful coordination and communication, and releasing discrete versions is a tool that we hope will help us achieve these goals.

    There are many benefits to discrete versioning. Here are a few prominent ones, off the top of my head:

    • Improved ticket management · Our ticketing system works best when tasks can be clearly assigned to individual developers, and organized according to priority. Version numbers provide a framework for this kind of management, as important tickets get assigned to earlier versions, and developer workload balance for a given release can be easily assessed.
    • Improved roadmapping · Roadmaps – software development language for “schedule of upcoming features” – are easier to understand and to implement when they’re organized by release numbers instead of thrown into a single pile.
    • Improved testing · When features and bugfixes are rolled out one-at-a-time, it can mean near-nonstop testing for the development and community team. Having controlled releases means that testing for a whole set of changes can be managed at the same time, in the days leading up to the release itself. That’s a better use of everyone’s time.
  2. Better distinction between development, staging, and production

    In those halcyon days early in the life of the Commons, nearly all development was done in the live production environment. When fixing a bug or launching a feature, I’d edit the site while it was up and running. Danger, Will Robinson! So many things can go wrong when you’re editing a live site, it’s hardly worth enumerating.

    The dev team eventually did set up a development environment. But it was running on a different server from the production site, and so you could never be guaranteed that a fix on the dev site would translate to the live site. Moreover, all those little fixes that would get sneaked directly onto the production site meant the dev site and the production site got progressively more out of synch as time went on, until the dev site was sufficiently different from the live site to be next to useless as a development site.

    Our new setup is much more sophisticated and, we hope, more conducive to quality development. We have a staging environment running in exactly the same operating environment as the production environment. Development environments are all local, which is to say that each member of the development team has a functional copy of the site on his personal computer. To whatever extent possible, development takes place in those local environments, is tested in our staging environment to look for environment-specific bugs, and only then pushed to the production environmnet. The (lofty) goal is that we’ll never have to make tweaks directly to the production site.

  3. Version control

    By “version control” I mean something a bit different from (though related to) “versioning” as mentioned above. Version control is a way to keep track of every change made to the files. It used to be that, when I tweaked a template or plugin file, unless I made a backup, the previous version would be overwritten and lost. Now that we’re using Git, we can always revert changes if need be (or browse the source out of mere historical curiosity!).

    The need for version control is all the more pressing as our development team grows. It’s important to keep track of who is changing what. And when you have local, independent development environments, it’s imperative that you have a way of making sure that the changes that I make on my version are not overwritten by the changes made by another member of the team when we merge them together in the staging environment. Git is designed for this kind of distributed development, as it keeps careful track of who edited what (‘git blame’, appropriately enough) and allows graceful merging of disparate edits.

    Using Git with WordPress and MediaWiki posed something of a technical challenge. For one thing, we wanted to track the entire codebase, but not files that users had uploaded to the system. Furthermore, MW and WP store crucial configuration information in the database, which is not tracked by Git. Finally, in order for local development environments to work properly, some environment-specific variables (database passwords, for instance) needed to be abstracted out of tracked files like wp-config.php. Zach at Cast Iron Coding forged a path through this jungle, and we’ve come up with a system that allows us all to keep our versions in sync using Git. In the future, I hope to write in a little more techinical detail just how this works.

    We use Git not just in our local development environments, but on the staging and production servers as well. For instance, when a release candidate has been vetted in the staging environment and is ready to go live, it’s extremely easy to launch the version on the production site: I simply shell into the production server and git pull or checkout the tagged version from the repository.