The technical side of Commons v1.0

As Matt @admin announced earlier today, we’ve just tagged the CUNY Academic Commons “version 1.0”.

The 1.0 milestone represents a couple of big changes in our development processes.

  1. Versioning

    The most obvious change is the use of version numbers. It might seem at first like a relatively minor and superficial alteration in the way we operate. In truth, it’s a big change.

    Prior to the switch, our development cycles were haphazard at best. Bugs would be fixed and features implemented whenever someone on the development team happened to get around to it. That system (or lack thereof) works pretty well for a small team. But the team behind the Commons has been growing steadily over the past year, both in sheer number and in breadth (of geographical distribution, of areas of technical expertise, of responsibility). Greater distribution requires more thoughtful coordination and communication, and releasing discrete versions is a tool that we hope will help us achieve these goals.

    There are many benefits to discrete versioning. Here are a few prominent ones, off the top of my head:

    • Improved ticket management · Our ticketing system works best when tasks can be clearly assigned to individual developers, and organized according to priority. Version numbers provide a framework for this kind of management, as important tickets get assigned to earlier versions, and developer workload balance for a given release can be easily assessed.
    • Improved roadmapping · Roadmaps – software development language for “schedule of upcoming features” – are easier to understand and to implement when they’re organized by release numbers instead of thrown into a single pile.
    • Improved testing · When features and bugfixes are rolled out one-at-a-time, it can mean near-nonstop testing for the development and community team. Having controlled releases means that testing for a whole set of changes can be managed at the same time, in the days leading up to the release itself. That’s a better use of everyone’s time.
  2. Better distinction between development, staging, and production

    In those halcyon days early in the life of the Commons, nearly all development was done in the live production environment. When fixing a bug or launching a feature, I’d edit the site while it was up and running. Danger, Will Robinson! So many things can go wrong when you’re editing a live site, it’s hardly worth enumerating.

    The dev team eventually did set up a development environment. But it was running on a different server from the production site, and so you could never be guaranteed that a fix on the dev site would translate to the live site. Moreover, all those little fixes that would get sneaked directly onto the production site meant the dev site and the production site got progressively more out of synch as time went on, until the dev site was sufficiently different from the live site to be next to useless as a development site.

    Our new setup is much more sophisticated and, we hope, more conducive to quality development. We have a staging environment running in exactly the same operating environment as the production environment. Development environments are all local, which is to say that each member of the development team has a functional copy of the site on his personal computer. To whatever extent possible, development takes place in those local environments, is tested in our staging environment to look for environment-specific bugs, and only then pushed to the production environmnet. The (lofty) goal is that we’ll never have to make tweaks directly to the production site.

  3. Version control

    By “version control” I mean something a bit different from (though related to) “versioning” as mentioned above. Version control is a way to keep track of every change made to the files. It used to be that, when I tweaked a template or plugin file, unless I made a backup, the previous version would be overwritten and lost. Now that we’re using Git, we can always revert changes if need be (or browse the source out of mere historical curiosity!).

    The need for version control is all the more pressing as our development team grows. It’s important to keep track of who is changing what. And when you have local, independent development environments, it’s imperative that you have a way of making sure that the changes that I make on my version are not overwritten by the changes made by another member of the team when we merge them together in the staging environment. Git is designed for this kind of distributed development, as it keeps careful track of who edited what (‘git blame’, appropriately enough) and allows graceful merging of disparate edits.

    Using Git with WordPress and MediaWiki posed something of a technical challenge. For one thing, we wanted to track the entire codebase, but not files that users had uploaded to the system. Furthermore, MW and WP store crucial configuration information in the database, which is not tracked by Git. Finally, in order for local development environments to work properly, some environment-specific variables (database passwords, for instance) needed to be abstracted out of tracked files like wp-config.php. Zach at Cast Iron Coding forged a path through this jungle, and we’ve come up with a system that allows us all to keep our versions in sync using Git. In the future, I hope to write in a little more techinical detail just how this works.

    We use Git not just in our local development environments, but on the staging and production servers as well. For instance, when a release candidate has been vetted in the staging environment and is ready to go live, it’s extremely easy to launch the version on the production site: I simply shell into the production server and git pull or checkout the tagged version from the repository.

Sys Admin Appreciation Day

Yesterday I learned, via a tweet from Bethany Nowviskie, that Friday, July 30th is System Administrator Appreciation Day. While I hadn’t heard about this holiday before, it strikes me as one of the best holidays yet invented — because, really, we can’t thank our Sys Admins enough. And that is particularly true for the CUNY Academic Commons, where we are lucky enough to have the extremely bodacious André Pitanga, Lead Systems Administrator at the CUNY Graduate Center, running our show — or, rather, running the servers that run the software that runs our show.

Photo of André's keyboard by catcubed (http://www.flickr.com/photos/headlouse/1484615917/)

The Sys Admin Appreciation Day website details a number of reasons why all members of the Commons should be grateful to André for his work. And they are all true — André designed and set up our server racks, got the servers running, configured our networks, installed our software packages, performed needed system upgrades, troubleshooted thorny permissions issues, secured the site during malicious attacks, and made himself available for consultation on all sorts of issues. If you’re lucky enough to get a tour of the server room from him –I’ve done it five or six times now; it never gets old — you’ll see his eyes light up as he describes the racks he designed for this project. Sure, those eyes take on a slightly maniacal glimmer as he discusses server cooling systems, but it’s all good! Right?

There are a few things, though, that make André’s work truly special: his contagious enthusiasm for collaborative projects, his passion for open-source platforms and the communities that support them, and (most importantly), his humbleness, sincerity, and openness. The Commons is extremely lucky to have André behind it, not just because of his formidable technical expertise, but also because of the creative enthusiasm that he adds to Development Team.

And, so, on this day, I want to express my gratitude to André, my colleague and my friend, for everything he has done for the Commons. Thank you, André. You rule!

New BuddyPress plugin: BP External Activity

Here on the CUNY Academic Commons, we have an installation of MediaWiki which runs alongside our WordPress/BuddyPress installation. Without some additional coding, edits on the wiki don’t show up in the Commons News feed, and as a result they tend to get a little lost in the shuffle. This new plugin, BP External Activity, addresses this issue by allowing the administrator of a BuddyPress installation to specify an arbitrary RSS feed whose items will be regularly imported into the activity stream.

The plugin imports RSS feeds every hour. You may find that you need to decrease your Simplepie cache time to make it work:
add_filter( 'wp_feed_cache_transient_lifetime', create_function('$a', 'return 600;') );
reduces the RSS cache to ten minutes, for example. Put that in your bp-custom.php file if you are having problems with the plugin.

At the moment, the plugin uses the Author field from the RSS feed to look for a matching author in your WP database. If it doesn’t find one, it uses the unlinked text ‘A user’, as in ‘A user edited the wiki page…’.

Thanks to Andy Peatling, whose External Group Blogs served as the inspiration for a good part of the code.

Visit the plugin homepage for more details.

BuddyPress plugins running on the CUNY Academic Commons

A few people have asked recently for a list of the plugins installed on the CUNY Academic Commons. In the spirit of Joe’s post, here I thought I’d make it public. I’m going to limit myself to the BuddyPress plugins here, for the sake of simplicity. (I’d like to write a series of posts on the anatomy of the CUNY Academic Commons; maybe this will be the first in that series.) Here they are, in no particular order other than the order in which they appear on my plugin list.

  • BP TinyMCE. This plugin is messed up, and I have part of it switched off, but I still use the filters that allow additional tags through, in case people want to write some raw HTML in their forum posts, etc.
  • BP Groupblog. Allows blogs to be associated with groups, displaying posts on that group’s activity feed and automatically credentialing group members on the blog. I did some custom modifications to the way the plugin works so that clicking on the Blog tab in a group leads you to subdomain address rather than the Groupblog custom address (thereby also ensuring that visitors see the intended blog theme rather than the BP-ish theme).
  • BP MPO Activity Filter. This plugin works along with More Privacy Options to ensure that the new privacy settings are understood by Buddypress and that blog-related activity items are displayed to the appropriate people.
  • BuddyPress Group Documents. This one is crucial to our members, who often use the plugin to share collaborative docs.
  • BP Include Non-Member Comments makes sure that blog comments from non-members are included on the sitewide activity feed.
  • BP External Activity – an as-yet unreleased plugin I wrote that brings in items from an external RSS feed and adds them to the sitewide activity feed. We’re using it for MediaWiki edits.
  • BP Group Management lets admins add people to groups. Very handy for putting together a group quickly, without having to wait for invites.
  • BP System Report. We’re using this one to keep track of some data in our system and report it back to members and administrators.
  • BuddyPress Group Email Subscription allows users to subscribe to immediate or digest email notification of group activity. Right now we’re running it on a trial basis with a handful of members, in order to test it. (Here’s how to run it with a whitelist of users, if you want)
  • BuddyPress Terms of Service Agreement, another as-yet-unreleased plugin (this one by CAC Dev Team member Chris Stein) that requires new members to check TOS acceptance box before being allowed to register.
  • Custom Profile Filters for BuddyPress allows users to customize the way that their profile interests become links
  • Enhanced BuddyPress Widgets. Lets the admin decide the default state of BP widgets on the front page.
  • Forum Attachments for BuddyPress. Another of our most important BP plugins, this one allows users to share files via the group forums.
  • Group Forum Subscription for BuddyPress. This is our legacy email notification system, which is going to be in place until I get back from my honeymoon and can replace it 🙂
  • Invite Anyone lets our users invite new members to the community and makes it easier to populate groups.

Questions about any of these plugins or how they work with BuddyPress? Ask in the comments.

New BuddyPress plugin: BP System Report

One of the goals of the Community Team behind the CUNY Academic Commons is to figure out how members are using the site, so that we can make it a better place for meeting and collaborating with each other. With a system like BuddyPress, though, it’s a bit hard to get a general sense of what’s going on on the site. BP System Report is a new plugin meant to address this issue.

BP System Report records regular summaries of statistics related to your BuddyPress installation. You can then compare any two snapshots using the built-in comparison tool, which calculates percentage differences. The information currently gathered by the plugin:

BP System Report
BP System Report
  • Members: total number, number active during report interval, percent active, total friendship connections, average friendships per member
  • Groups: total number, number active, percent active, total group memberships, average group membership
  • Public/private/hidden groups: total number, number active, percent active, total membership, average membership
  • Blogs: total number, number active, percent active

The plugin is quite beta, so still might be buggy. Please feel free to report problems.

Future versions of this plugin will include:

  • more analytical data collected
  • CSV export
  • better admin control over report frequency

Regarding this last point: the BP System Report defaults to twice-daily reports. If you’d like to adjust it manually, deactivate the plugin, edit the line
wp_schedule_event( time() + 30, 'twicedaily', 'bp_system_report_pseudo_cron_hook' );
in bp-system-report.php to say ‘hourly’ or ‘daily’ or ‘weekly’ (or some custom time you define in bp_system_report_more_reccurences() )

Download BP System Report here.

Adding an “email to members” checkbox to the BuddyPress group activity stream

During the recent upgrade from BuddyPress 1.1.x to BuddyPress 1.2.x, and the subsequent move away from group wires to interactive group activity streams, one thing that some users on the CUNY Academic Commons missed was the “Notify members by email” checkbox of the old wire.

This morning I wrote a bit of code to add that kind of functionality to group activity streams. There are three functions, each of which goes in your plugins/bp-custom.php file.

First, adding the checkbox to the activity box. Notice that it only shows up when you’re on a group page.


function cac_email_activity_checkbox() {
	if ( !bp_is_groups_component() )
		return;
	?>
	
	
	<?php
}
add_action( 'bp_activity_post_form_options', 'cac_email_activity_checkbox' );

Second, handling the data when it gets to the server and sending the emails. Obviously, you’ll want to change the text of the email to match your own site and your own preferences.


function cac_email_activity_handler( $activity ) {
	global $bp;
	
	if ( $_POST['mailme'] == 'mailme' ) {
		
		$subject = sprintf('[CUNY Academic Commons] New update in the group "%s"',  $bp->groups->current_group->name );
		
		$message = strip_tags($activity->action);
		$message .= '

';
		$message .= strip_tags($activity->content);
		
		$message .= '

-------
';
		
		$message .= sprintf('You recieved this message because you are a member of the group "%s" on the CUNY Academic Commons. Visit the group: %s', $bp->groups->current_group->name, $bp->root_domain . '/' . $bp->groups->current_group->slug . '/' . $bp->groups->current_group->slug . '/' );
		
		//print_r($message);
	
		if ( bp_group_has_members( 'exclude_admins_mods=0&per_page=10000' ) ) {
			global $members_template;
			foreach( $members_template->members as $m ) {
				wp_mail( $m->user_email, $subject, $message );
			}
		}
	}
	
	remove_action( 'bp_activity_after_save' , 'ass_group_notification_activity' , 50 );
}
add_action( 'bp_activity_after_save', 'cac_email_activity_handler', 1 );

Finally, you’ll need some Javascript to make the AJAX activity submission work correctly. This is really just a copy of what’s in the bp-default JS file, with a few added lines to make it work.

function cac_email_activity_js() {
	if ( !bp_is_groups_component() )
		return;
	?>
	

	var jq = jQuery;
	jq(document).ready( function() {
jq("input#aw-whats-new-submit").unbind('click');
			/* New posts */
	jq("input#aw-whats-new-submit").click( function() {
		var button = jq(this);
		var form = button.parent().parent().parent().parent();

		form.children().each( function() {
			if ( jq.nodeName(this, "textarea") || jq.nodeName(this, "input") )
				jq(this).attr( 'disabled', 'disabled' );
		});

		jq( 'form#' + form.attr('id') + ' span.ajax-loader' ).show();

		/* Remove any errors */
		jq('div.error').remove();
		button.attr('disabled','disabled');

		/* Default POST values */
		var object = '';
		var item_id = jq("#whats-new-post-in").val();
		var content = jq("textarea#whats-new").val();
		var mailme = jq("#cac_activity_mail:checked").val();

		/* Set object for non-profile posts */
		if ( item_id > 0 ) {
			object = jq("#whats-new-post-object").val();
		}

		jq.post( ajaxurl, {
			action: 'post_update',
			'cookie': encodeURIComponent(document.cookie),
			'_wpnonce_post_update': jq("input#_wpnonce_post_update").val(),
			'content': content,
			'object': object,
			'mailme': mailme,
			'item_id': item_id
		},
		function(response)
		{
			jq( 'form#' + form.attr('id') + ' span.ajax-loader' ).hide();

			form.children().each( function() {
				if ( jq.nodeName(this, "textarea") || jq.nodeName(this, "input") )
					jq(this).attr( 'disabled', '' );
			});

			/* Check for errors and append if found. */
			if ( response[0] + response[1] == '-1' ) {
				form.prepend( response.substr( 2, response.length ) );
				jq( 'form#' + form.attr('id') + ' div.error').hide().fadeIn( 200 );
				button.attr("disabled", '');
			} else {
				if ( 0 == jq("ul.activity-list").length ) {
					jq("div.error").slideUp(100).remove();
					jq("div#message").slideUp(100).remove();
					jq("div.activity").append( '
    ' ); } jq("ul.activity-list").prepend(response); jq("ul.activity-list li:first").addClass('new-update'); jq("li.new-update").hide().slideDown( 300 ); jq("li.new-update").removeClass( 'new-update' ); jq("textarea#whats-new").val(''); jq("#cac_activity_mail").removeAttr('checked'); /* Re-enable the submit button after 8 seconds. */ setTimeout( function() { button.attr("disabled", ''); }, 8000 ); } }); return false; }); }); <?php } add_action( 'bp_activity_post_form_options', 'cac_email_activity_js', 999 );

New BuddyPress plugin: BP Group Management

BuddyPress has great group administrator functions – the ability to invite members to groups, to promote them to different statuses, the ability to ban certain member, and so on. But unless the sitewide administrator is also the administrator of the group, the site admin does not have the same abilities. On some sites – like here on the Academic Commons, where it’s frequently desirable to add members manually to groups – this limitation for sitewide admins can be somewhat restricting.

This plugin, BP Group Management, creates a new administration panel to the Dashboard, accessible only by the sitewide administrator, which does the following:

  • provides a sortable list of all groups (public, private, and hidden) with their created-on dates and ID numbers
  • allows admins to delete groups easily
  • allows admins to view lists of current members, and to promote/demote/ban them
  • allows admins to add any member of the site directly to the group, skipping the need for invitations

Download the plugin here.

The version of the plugin in the repository only works for versions of BuddyPress 1.2 and greater. For a mostly functional version of the plugin that works with BP 1.1.3 (no guarantees on any other versions, but it should work down to BP 1.1 at least), click here.

New BuddyPress plugin: BP Include Non-Member Comments

By default, BuddyPress does not include comments from non-members (or non-logged-in users more generally) in the sitewide activity stream. For some communities, this default behavior is probably just fine. But in a community like the CUNY Academic Commons, which aims to attract readership and conversation from all sorts of folks, whether or not they’re signed in, the sitewide activity stream will be much more meaningful if it includes comments from everyone.

My new plugin, BP Include Non-Member Comments, was built with this purpose in mind. The plugin is activated here on the CUNY Academic Commons, so that all comments on your blog – whether from logged-in users or not – will show up on the news feed.

BP Include Non-Member Comments

The plugin has been tested on version 1.1.3 of BP, as well as the 1.2 release candidate. If you want to use the plugin for 1.1.3 or lower, you will need to uncomment the first few add_action and add_filter lines in the plugin file.

Technical caveat: Non-logged-in commenters have BP user_id 0. When BP creates the activity stream, it decides whether or not to show the Delete button by checking to see whether the user_id for the currently logged in user is the same as the user_id of the person to whom the comment belongs. Presumably, though, you don’t want non-logged-in viewers of the activity stream to be able to delete items from the activity stream at all. BP’s core code is not currently set up to make it easy to remove these buttons, so I employed an ugly fix. If you have changed your theme significantly from the default, you might have to adjust the filter bp_nonmember_comment_content (near the end of the plugin) to remove the button properly.

Download the plugin here.

New BuddyPress plugin: BP-TinyMCE

I threw a little something together today to add WYSIWYG editing to BuddyPress, using TinyMCE. I want to be careful about the tags I allow, so I’m whitelisting, which is a bit tedious. As a result, there are only a few buttons available: a, em, strong, ul, ol, li. It’s a start, though.

Seems to work everywhere in BP: forums, wire, messages, profile pages.

A note about TinyMCE: WP ships with TinyMCE, and I thought it made sense to use that version instead of attaching one to this plugin. I think that the path to TinyMCE (line 18 of the plugin) should work on all installations, but you may have to tinker if you don’t see it popping up in the head of your BP pages. Moreover, the language files for WP’s version of TinyMCE are misnamed, which means that they don’t work right out of the box (at least for me they don’t). You may need to change the name of wp-includes/js/tinymce/langs/wp-langs-en.php to en.php in order to get the hover and help text in the TinyMCE box to work.

Download the plugin here. Don’t use in a production environment unless you are very certain that you are satisfied with the security of this plugin!

New BuddyPress plugin: Invite Anyone

In my view, the most powerful feature of BuddyPress – the feature that powers the core goal of the CUNY Academic Commons, that of collaboration – is groups. By default, BuddyPress only lets you invite other members of the community to your group if you and the member are already friends within BuddyPress. In some communities, this feature probably prevents a lot of spam. But in other communities, like the one here at the Commons, the friendship requirement adds a sometimes inconvenient extra step to the process of getting a productive group up and running.

Invite Anyone in action
Invite Anyone in action

This new plugin, Invite Anyone, does just what its name claims: it alters the group invitation process to allow group creators and administrators to invite anyone from their BuddyPress installation, not just their friends.

Like so often happens, though, solving this one problem made another one pop up: Scrolling through a list of your friends to find potential invitees is one thing, but scrolling through a list of every member of the site is another thing entirely. Most communities, including the CUNY Academic Commons, will simply have too many members. To make things easier, I’ve taken the autosuggest feature from the Compose Message screen in BuddyPress and retooled it to work on the Send Invites screen as well. Start typing the name of the user you’d like to invite, and with each letter you type, BuddyPress will make better and better suggestions as to who you mean. Just hit enter or click to add the suggested user to the invited list.

Download the plugin here. As the plugin is built using the BP Group Extension API, you’ll need at least BP 1.1.

After I’ve done a bit more testing with our custom theme here on the Commons, I’ll activate the plugin on this site, so that members of the Commons community can take advantage of the new feature.

A technical note: I tested the plugin in a variety of different environments (different browsers, different themes, different servers) and stumbled upon a few issues, in particular with the autosuggest AJAX in Chrome for Mac. If you find similar issues, or have any other feedback, please leave a comment.