By The University of Kent has long needed a centralised system of creating events and displaying them on the main website. So a major piece of work we’ve been engaged with for a while now has been to build an events calendar system.
It will go live on 7th November at www.kent.ac.uk/calendar (currently restricted access)
Because you won’t yet be able to see the system working, I’ve included a few screenshots below. (They also help make this blog post look prettier.) Please do note that the events in the images below are just for testing purposes! So don’t rely on them being accurate until you seen the actual live system up and running at www.kent.ac.uk/calendar
Centralised and federated
A key feature of the new system is that it will allow users to add their own events to display on their own calendar sites, as well as submitting those events for inclusion in the central university events calendar. Once events have been created, approved, and marked for publishing, they then appear on the appropriate calendar on the university website. This could be on the central /calendar site, on a departmental site, or both. This is important, because for the first time users around the university will be able to manage their own events while allowing them to be easily reused elsewhere on the website.
Blogging events?
Some of you might be thinking “I have a wordpress blog and that has a really nice events plugin”, etc, etc. Good for you. Well, we did consider that as an option, but while such plugins are great, they just didn’t allow the power and flexibility we wanted for this product. In particular, they typically lack the ability to have a range of events calendars for separate centres and departments at the university, each of which could offer up event suggestions to the central university events calendar. WordPress-like events plugins work really well for small-scale, localised requirements, but less well across an entire organisation.
Sharepoint administration
The centralised/federated requirement entailed a relatively rich permissions system for administering and creating events. One which would know as soon as you log in who you are and what group, team, department you’re in, and would control the level of access you have for creating and approving events.
This got us to thinking about Microsoft Sharepoint. It’s used across the university as an intranet and document sharing tool. It integrates with our managed desktop PCs so it knows who you are and what you do. In fact it already has a bunch of powerful permissions and workflow tools and settings built in. What if we were to use that as the admin area for creating and managing events?
What indeed. After a few weeks’ playing around with the intricacies of Sharepoint, RDFa, SPARQL, ARC, our own Pantheon publishing framework, and a range of other technologies, we decided that it would work! So started a phase of agile development (more on this below) to produce a Sharepoint-based admin interface and a Pantheon-based frontend interface.
Slurp!
We decided after some initial testing and trials that the best way to hook up Sharepoint with Pantheon was to build an intermediary system which pulls data out of Sharepoint and put it into a relational database – in this case MySQL. We gave this intermediary the not unimaginative title of “The Slurper“. The Pantheon frontend code would then use the MySQL db as its data source directly. Nginx caching would then help sort out any performance issues that might arise from using direct db calls.
The slurper was actually key to the whole process because it had to be lightweight enough to run very frequently (every minute) to allow the frontend event listings to keep up to date with backend edits. Obviously with potentially thousands of events in Sharepoint, it just wouldn’t be practical to pull out every single event every minute into a database without slowing Sharepoint down for all the other tasks it has to perform in our organisation. The trick then was to make sure the only events to get pulled out would be those that had been added, deleted, or modified since the last update a minute ago. Also, in case a lot of events got added simultaneously the slurper was throttled to pulling 10 events a minute. Although this means it might get behind when lots of new events are added, it should still catch up with itself fairly quickly, and it would never put unrealistic demands on Sharepoint.
Agile scrums
I mentioned the term agile development above. I really don’t want to go into the details of agile methodologies here, but I do want to mention it briefly because I feel it was important to the development process for the events calendar and what I feel is its overall positive outcome.
Suffice it to say that the events calendar product was our first foray into the world of an agile project management methodology – specifically Scrum (or perhaps more accurately Scrum-but). The idea is that we work not on projects but on products; we work in very carefully timeboxed two-week sprints; and we work with the aim that at the end of each sprint we deliver the most important new features of the product at that time.
Although we were learning a lot as we were going along, the end result is I think a much more efficient process of producing and maintaining a product. Agile does not mean easy. It does not mean “make cool stuff up as you go along”. It’s a very tightly controlled and transparent approach. But it works very well when done properly, and you have good buy-in from your customers.
Future plans
The initial deployment of the events calendar will be relatively limited in scope. Once we’re more confident that it’s all working well and – most importantly – easy for people to use, it’ll start to get opened up to a wider audience. As with all products developed in an agile framework, they are never fully completed, and development on the product is always ongoing.