Virtual Greg

Our second-year Database Concepts students recently wrapped up their group projects in the course I co-lecture with subject coordinator Sherah Kurnia. Every year we assign students a semester-long project to design and build a data-driven website. This year we asked the twenty groups of five students to build a social networking site (SNS).

In our final lecture last week, we asked students to fill in a survey about the project. All those present responded, which is good in a week in which students are constantly asked to fill in surveys about their courses. Their responses were mostly very positive, and the students’ work was mostly excellent, so we consider the project a success. However SNS are a fairly new phenomenon – making this arguably a cutting-edge” undergrad projects – and not everything went the way we expected. In this post I’ll describe the thinking that went into our choice of project, and the decisions and changes of direction that happened along the way. Using the survey results, I’ll test which of our expectations panned out. If I get time I hope later to polish this up for a conference – perhaps this year’s Ascilite.

Developing the project theme

I started ruminating about social software as a project theme a couple of years ago, when I wrote a simple blog for myself (not this blog, another blog), and hosted it on my website, using PHP and MySql. Like all coding projects it was sometimes interesting and sometimes tedious. I noticed that it became more fun as the system approached completion and I could use it to send messages to friends, and (after I got comments working) they could reply. This was not a fully-featured blog, but I got a buzz from using the system to interact with people in other places and seeing their messages appear on the screen. Being web-based, it was a snap to email friends a link and ask them to post a comment. Reasoning that students would also find this fun, I thought a blog would make a good student project. The only trouble was, while the user interface could be made into a reasonably complex piece of web programming, the data model was very simple. In our database course projects, students have to model the database before they build the web front-end, and we want to give them ten or fifteen tables. A simple blog can be done with four, and getting the table count up to fifteen would, I think, require boring complication of the basic blog idea.

Last December I went on a family holiday in Asia. We covered a lot of ground in two weeks, visiting six cities in three countries. This rather ridiculous schedule meant that we rarely spent more than a couple of days in any one place, and were constantly pondering basic questions like: Where is the airport, Where is the hotel, Where is public transport and how do you use it, and What are the best places to visit during our brief stay? In Seoul we had the good fortune to be shown around by Larissa Hjorth, who spends a lot of research time in that city. Having a local friend made such an improvement to our too-brief visit that I wished I knew someone at every destination. Wouldn’t it be great to ask a local helper to spend an afternoon showing us around, or even to simply swap lunch for travel advice? I’d certainly be glad to return the favour if they ever came to Australia. Of course, this is just the kind of thing that social networking services can be used for: making online friends in distant places for mutual aid while traveling. Indie bands organize tours this way all the time, using MySpace.

I saw Facebook go through our undergrad cohort like a virus during 2007, and had started to think about assigning an SNS to students, although, like blogs, the core functions of social networking can be implemented with just a few database tables. But adding a “traveler” theme would increase the table count, and might itself be of interest to our undergrads, many of whom are either visitors to Australia or like to travel during their holidays. If we asked students to build such a site, they might be motivated by seeing how popular social networking sites work, and by the fun of using their creations to interact with each other.

I spent some of my airport time sketching out a design. The basis of SNS is a social graph: a list of people with a list of who is friends with whom, along with the ability to send and accept friend-requests, and to send regular messages. Adding a “travel” theme implied extra functions like storing travel plans, displaying a list of interests and hobbies, and arranging to be a fellow user’s host during a visit. It should allow users to offer spare rooms for their fellow users to stay in, to bid for rooms being offered by others, and to record agreements to let rooms. This allowed the project to incorporate auction-style features similar to eBay, a web site that many students are also interested in. After interacting during a visit, users might be given the ability to rate each other’s services, and check the ratings of prospective friends – adding another web-ism that is currently popular. It wasn’t too hard to think of sufficient services to bring the table count up to a size that is reasonable for our student project.

I built a prototype, “TravelNet”, to try out ideas for how the system should be modeled, and to allow Sherah to decide if she thought it suitable for this year’s project. In the past few years we have given our students e-commerce websites as projects: retail being a well-understood, business-focused problem domain. After some thought, we agreed that the attractions of the social networking theme outweighed the risk of giving up our familiar e-commerce theme.

I searched around the web to see if such a system was already available. There are many “themed” social network sites, and many travel websites, though the latter are mostly geared to people who stay in hotels, rather than young people looking for cheap accommodation and a social experience. A couple of sites came close to the spec we had in mind: Couchsurfing in particular, and one or two Facebook apps, were very close. Oh well, we wouldn’t be able to patent the idea – but could we still use it as a student project? Is it ok to ask students to build something that is already “out there” on the web? I figure it’s ok to be unoriginal, as undergrad assignments are invariably problems for which a solution already exists. But would students copy an existing website design? Again, not too much of a problem: we teach database and web programming, not visual layout, so we shouldn’t mind if students copy a commercial site’s look-and-feel, as long as they couldn’t get access to the underlying data model or application code. But could they access an existing data model or code? The Facebook developers’ object model is publicly available, and gives some hints about social networking entities and attributes. But I think it is sufficiently different to the TravelNet data model that we didn’t need to worry about this either.

Social networking has some interesting problems that would add a little spice to the project. Friendship between two users is a symmetric relationship, which relational databases model awkwardly. If Mary is a friend of Jack, then Jack is also a friend of Mary. This would have to be dealt with using Selects containing “or” or “union”, or by inserting two “FriendOf” rows for each pair of friends: neither of these being ideal solutions. Travel plans and lists of hobbies are conceptually simple but would require careful UI programming.

Assigning the project to students

These preliminaries happened in Jan/Feb 2008, so that by the time the semester started in March, we were more-or-less ready to assign the TravelNet project. After we employed our sessional tutors Neven and Fei in February, I made them users on my TravelNet prototype, and they helped do more testing. Sherah and I wrote the first project handout, and we gave it to students it in March.

Our projects run in two parts, lasting about ten weeks all up. In the first, we give students a textual description of the system to be built, and they each design a relational data model to support it. In the second, we give students the “correct” data model, and, in groups of five, they implement it as a MySql database with a web user-interface.

The data-modeling phase was unremarkable. The majority of students came up with good designs that would support a working system. In only one area did a lot of student designs differ from ours: We modeled Friend-requests and Friendships as different entities, whereas many students quite reasonably made Friendship equivalent to a Friend-request that was marked as “accepted”. (If many Friend-requests were rejected, our solution might make for more efficient searches.) Many students had problems modeling Ratings and the arranging of Accommodation. Ratings can be modeled straightforwardly as one or two extra attributes of the event being rated: for example, adding “buyer rating” and “seller rating” columns to a rental-agreement table. This forces there to be only one pair of ratings per interaction, and makes it easy to track the rating to the interaction. Many students did not see this, and modeled a distinct Rating entity, which would be difficult to manage and constrain. With regard to accommodation arrangements, many students had trouble differentiating bids and offers from a resulting agreement or transaction. We insisted that, rather than follow strict auction rules, this system should allow house owners to give accommodation to anyone they chose, not necessarily the user who offered the highest bid. This meant that Agreements could not simply be an Offer marked as “accepted”.

After individual data models are assessed by teaching staff, we then give students the “best” model, to implement in MySql. We take pains at this stage to explain to students that there can be more than one correct and workable model, in the sense of Graeme Simsion‘s PhD thesis. We also usually cut down the system functionality (and underlying data model), between the data-modeling and implementation phases. This is purely for workload reasons: there is far more work involved in implementing a working application than devising the underlying data model.

Workload is difficult to judge. It is notoriously hard for experienced professionals to estimate the time they will need to implement a system. It can be harder to judge how long a group of students will need. Based on my experience writing the TravelNet prototype, I knew that even the core social-networking functionality took some effort to implement as a website. In the end we decided to demand little more than this. We eliminated accommodation and ratings from the spec, and asked students to implement a system that allowed people to: register as users, create and edit a profile including a list of hobbies, create travel plans consisting of several destination cities, search for and befriend other users, and send text messages.

Another difficult decision at this stage was whether to make my prototype TravelNet site available to students to play with. A concern of ours was whether our project spec was sufficiently clear for students to understand what they had to build. Because it was the first time we ran a SNS themed project, we did not have the benefit of prior experience. There was insufficient time for students to recover if they misinterpreted the spec and went off on the wrong tangent. We were also concerned about difficulty and workload. We reasoned that giving students a prototype would clarify the system requirements and provide immediate hints about how the user interface could be handled. The danger was that students might slavishly copy the prototype, stifling creativity and perpetuating any design flaws I had made.

Less likely, but with a greater penalty, was the danger that students might get hold of my source code. A web server will always hide PHP code from a browser. But could the server be hacked into? I don’t believe that it was, but I did get a scare one night. I read through the Message table and realized that a couple of students were practicing SQL injection and CSS attacks. I knew SQL injection wouldn’t work, but I hadn’t anticipated cross-site scripting. I checked what the hackers were doing, and later spoke to them. They were able to get hold of PHP session cookies, which might have allowed them to log in as a different student, however they didn’t get source code. The only way to get source code that I am aware of involves being able to store extra files on the server (containing functions that send existing files to the hacker) however no extra files appeared.

There was some minor drama when the prototype was first released, as some students sent “abusive” messages to each other. This seems inevitable in a class of teenagers, and knowing the students involved it was highly unlikely that the messages were intended to cause real offence. But after consultation with my department manager, it was decided to remind students of their obligation not to harass others when using University-provided messaging systems. This reminder appeared to have the desired effect in that no more nasty messages were sent. However I feel it may also have killed off students’ use of the prototype for genuine group communication beyond their initial exploration of the system.

The students, working in groups of five, had four weeks to build working websites. This phase of the project was very successful, with teaching staff pleasantly surprised by the quality of the students’ work. There was also great variety in UI design, and none looked too similar to my prototype.

Judging whether the project was successful

The only infallible way to evaluate a piece of teaching would be to run an experiment. One might give project A to one group of students and project B to a second group of students, all of whom go to the same lectures and tutorials in the same year, then compare the two groups’ success on a task after the project, such as an exam. Though I’ve heard of people doing this, it requires a lot of work, and might be ethically questionable.

Alternatively one might compare students year-to-year. If this year’s students do better on the exam than last year’s, and given the class teaching was substantially the same, does that mean this year’s students learned more during the project? It might, but it might also mean that the sessional tutors were better, or that this year’s students had received better teaching during their first year, before they did our course.

We could also compare the project marks year-to-year. There was a lot of very good work this year, and it looks as though the overall grades will be higher. This suggests that this year’s theme worked better, but it could also mean that it was simply easier, or that we were less tough in marking it. I don’t think either of these is the case, but one couldn’t rule it out on the available data.

About the only other measurement we can make of the utility of our project theme is to ask our students what they thought of it. To this end, during our final lecture, we asked students to fill in a survey, with a numerical score and comment for each of a collection of questions, designed to test the following hypotheses:

1. Because students like social networking sites, they would be interested in and motivated by a development project based on SNS.
2. The quantity and difficulty of work required by our SNS project was appropriate.
3. The project helped students learn useful skills.
4. The prototype system provided to students was useful to them.
5. Students would use their own projects, or my prototype, as group collaboration tools during development.

From both the numerical scores and comments made on the survey sheets, we can say that:

Hypothesis 1 is strongly supported. Students were overwhelmingly enthusiastic about the social-networking theme. Students rated the question “The SNS theme was interesting and motivating” an average score of 4.2/5. There were many positive comments, including:

• Yes. I really wanted to know how sites like Facebook work.
• It was interesting due to the popularity of Facebook.
• Very much. It’s definitely interesting to learn how Facebook works.
• Many people use Facebook so we could relate to the assignment.
• The most interesting project I have in my Uni life! Can’t imagine that we can actually do a SNS.
• About as interesting as a project can get.

On other hand, there were one or two negative comments:

• I don’t have Facebook, Myspace, Frienster accounts. So yeah it just does not appeal to me.

Hypothesis 2 is supported. Most students felt that the workload was about right. A few complained it was too hard, a few complained it was too easy, and a few said they preferred individual to group work. Average score for this question was 3.4/5.

Hypothesis 3 is supported. Students felt they had learned skills that would be valuable for future study and work. Average score for this question was 4.1/5.

Hypothesis 4 is supported. Many students appreciated receiving the prototype. A few didn’t use it. None complained about it being made available. Average score for this question was 3.6/5.

Hypothesis 5 is NOT supported. Students did not use the SNS they were building for “real” communication. Likewise they used my prototype for testing, but not for communication beyond that. Average score for this question was 2.4/5. Comments included:

• Most of the use was for testing our project. But sometimes we would jokingly send messages to each other.
• Mainly as a demo, but did try messaging partners.
• We tested it briefly but we didn’t really use its functions.
• A few friends and I actually used it in school to msg each other while we are working in different labs.

We also asked students to suggest other “real world systems” they would like used as project themes. The most popular suggestions were:

• Messaging systems such as blogs, discussion forums, or IM (7 votes)
• Online banking (6 votes)
• Online broking or auctioning like eBay (5 votes)
• e-Commerce sites like Amazon (3 votes)
• Large multiplayer game (2 votes)

And there were one vote each for

• a dating service
• a search engine
• a teaching system like Blackboard
• a pirate download site