Building our own house

16 April 2018

From the beginning, newCardigan wanted to do things differently. We're all volunteers, and need newCardigan administration to be relatively painless. But we also value our members' privacy and security, and wanted to do the right thing by them. With this month's GLAM Blog Club theme being 'control', I thought it might be interesting to run through the newCardigan tech stack to explain how we've set things up, and why.

newCardigan uses software for two basic functions: promotion, and event management. I'm excluding @ausglamblogs here, because that's it's own separate thing. Initially, we had a website that was literally one html file and a css file on a webserver. Push communications were achieved via a monthly newsletter using Tiny Letter, and event bookings were managed by asking people to email us if they were intending to come to the next cardiParty. Obviously, this wasn't going to be sustainable in the long term, but it was low-maintenance, and relatively easy to manage as we were getting started.

It's safe to say that newCardigan quickly became rather bigger, rather faster, than the founders anticipated. Our string-and-chewing-gum arrangement quickly became unweildy. The first thing to fix was the website. I set up a Ghost site, the same software I use for this blog. This enabled us to have multiple authors, and a proper publishing platform. We used Universe to manage cardiParty bookings for a while, but then moved to Eventzilla because it looked like it would allow us to use waiting lists. We also wanted a way for the newCardigan community to talk to each other in 'our' space, so we launched a Discourse instance and plugged it into the Ghost site to allow for comments directly in the same page as our cardiParty posts. When we launched our cardiCast podcast, we used Soundcloud to host and publish. Internally, we experimented with Trello to plan things, and Loomio and then Slack to help the 'cardiCore' get organised and stay in touch. We also moved from TinyLetter to Mailchimp to enable automatic posting of event emails, by intergrating Mailchimp with the website RSS feed.

On the one hand, we were being a bit agile by trying things out. On the other hand, it started to feel like we were churning through a lot of systems that weren't working for us. The main problems were a lack of flexibility, particularly in the freemium platforms like Slack and Eventzilla, as well as looming subscription costs for Slack and nervousness about the future of Soundcloud. But we also realised it was time to formalise newCardigan, which meant we needed a proper membership database and more sophisticated way to manage records. Through all of this, we were ultimately looking for systems that were easy to use (for us and our community members), but also privacy-friendly, controlled by us, and able to be customised to our needs. Inevitably, we ended up with an open source stack.

The core of our system is CiviCRM, a constituent relationship management system specifically designed for non-profits. This allowed us to get rid of Eventzilla and Mailchimp, and manage contacts in a more sophisticated way. CiviCRM can be used by itself, but mostly works as a plugin for Drupal, Joomla! or WordPress. Since WordPress is what we were all most familiar with, we decided to migrate the website from Ghost to Wordpress with a CiviCRM integration. As part of the move, we ditched comments completely, and shut down the Discourse server: nobody had really used it, with comments and chat basically happening on Facebook and Twitter. Moving from Soundcloud to a self-hosted podcasting solution turned out to be a bit easier than I anticipated, though it was fiddly to set up initially. Conveniently for us, Digital Ocean launched their Spaces product just before we migrated everything. Spaces works basically like AWS's S3 product: in fact, you can use the S3 API to interact with Spaces. We use Cyberduck to upload files, but the compatability of everything allowed us to use the Blubrry WordPress plugin to host cardiCast using Digital Ocean Spaces, and publish it to all the usual podcast platforms using WordPress and good old RSS.

We added Stripe integration for taking payments (thanks donors!), and then to complete the move to self-hosting, we migrated our internal discussions from Slack to RocketChat. Rocketchat was a pleasant surprise for me as the system administrator of all this. Whereas I had a bit of an argument with CiviCRM, taking a couple of weeks to iron out all the weird, confusingly-documented intricacies and google some PHP snippets, when I installed RocketChat it was so easy I didn't quite believe there was nothing else to do. Rocketchat utilises Caddy for webserving and automatic HTTPS, and the newish Linux snaps system, so it's more or less set-and-forget.

The last thing we haven't quite managed to move onto our own stack is Google Drive. We used Google Hangouts for meetings a few times, but RocketChat's videochat functionality (built on Jitsi) is actually pretty good, and easier to jump in and out of, so the last Google thing we use is Drive for scheduling and document sharing. Ideally we'll move off this too soon: possibly to an ownCloud instance, or maybe just using Digital Ocean spaces.

So there you go. It's not the simplest thing I've ever done, but now that everything is set up, it's fairly straightforward to manage. Importantly, we've reduced the number of external services that can see any data from our members and others who interact with our platforms. Whilst we are still using Google Drive, there isn't any user data there: it has a bit of documentation for the Committee to manage things, and our scheduling documents, and that's basically it. We're not quite running our own servers in the spare room, but I'm pretty happy with how far we've managed to move towards running our own systems so we don't force members and participants to hand over data to third parties just so they can socialise with other GLAM people. As much as possible, it's newCardigan members, or at worst, newCardigan as an organisation, in control.