Google Summer of Code — how proposals are evaluated

In the previous post I explained the motivation behind GSoC and the state of things with regards to how it helps FreeBSD.

I guess I must explain a bit more on how GSOC proposal evaluations are done, since it can be not entirely clear to outsiders.

Introduction

Below you’ll find an insight on how the evaluation of Google Summer of Code took place for the FreeBSD project, but please not that people and projects vary in how they treat their students and participants. Having said that, over the years of GSoC participation I’ve had a chance to interact with other teams, and their experiences and the approach are similar to FreeBSD’s in how the communicate, help and judge students.

Please note that even within FreeBSD we always have several developers with administrator’s roles and they review proposals as well. Their process of looking at these proposals might have differed to the one described below.

Skimming

Quick glance at the list is the first think I do (according to Jacob Nielsen this is how we all do it. So you skim over 33 items, you see those which doesn’t make sense at all. Just to make sure you’re right in mentally nuking these that you think aren’t decent, I opened it in separate tabs in the browser. I had 99.99% success rate in judging irrelevant ones. This is due to simple reason: some of the proposals seem to be generated from Google. If you want to be accepted in **any project DON’T pick anything from Google. Don’t re-paste anything from the Internet. Don’t republish stuff other people did. **This really must be your proposal. So the irrelevant ones got marked as “non-acceptable” right away. By the way: they stayed that way, since most people reviewed those too. So let me make it clear once again: Don’t waste your time on submitting junk: it for SURE will be ignored. This means: no money for junk.

Quick selection based on my knowledge

Nobody is an expert on everything. I’m no different. Basically I know nothing about BSD text processing tools, and since this proposal showed up, I knew I’m the bad person to review it. However, we’ve had 2 of those, and since we decided not to accept duplicates (not to have two people working on the same project) I glanced at both of them quickly.

One was brief and lacked details. Had poor formatting. Was sort of done very quickly. Was a hack. Obviously, if you want to show your real initiative, I expect you provide something in a readable form. So this really doesn’t have to be PDF produced with QuarkXPress – just do your best, use Google Melange formatting and you’ll be just fine.

Deeper understanding

Some projects really interested me. I started to study them in more detail. I skipped student information, since it wasn’t relevant for me at this stage. I was interested if the idea is OK and whether there are some practical steps and implementation schedule for the proposal.

Unfortunately some had smarter form of spam embedded too. These projects got nuked too, thus no further time wasted for me.

We’re basically talking about around 22 project proposals already. It’s 66% of submitted proposals in total.

Typical proposal must have couple of elements (I think these are imposed by the proposal submission form provided by Google). I must said Google did a good job, since the form had enough information to make reading worth its time.

Every student provided:

1. Name, Surname
2. E-mail and phone number
3. Short bio
4. Resume
5. Availability

So Name and Surname got skipped, so did the e-mail and phone. Short bio was important for me, somehow.

And I want to put emphasis on short. I’d really like Google to put some form of a limit (100 words) on what should show up here. Some were several A4 pages long. Well, honestly, I didn’t read it. I also skimmed over the keywords in your text.

Some proposals struck my attention: why on Earth people submitting proposals to the FreeBSD Project mention Linux only? I mean some of them had not a single word about FreeBSD. For me this is the sign you didn’t study too well for this years GSOC and you don’t want to be accepted, since it looks like you’ve picked an idea from the FreeBSD ideas list (which you have managed to find somehow) and without knowing what FreeBSD is you’ve submitted this stuff.

Anyway, we had busy student-professionals applying too. Personally, I would have picked people like but. My reasoning is that if I see somebody working with computers professionally, it may mean the understanding of the whole GSOC is much better, responsibility is much bigger and that the likelihood of successful project’s completion is higher. However, after some discussions we agreed that people with really limited time spent on the project are kinda risky. You know, when you can spend only 4 hours per week on the project it’s clear isn’t not going to be finished on time.

We’ve had one person, whose proposal was really good, but the gentleman applied to other projects too, with equally good proposals, and on the way of evaluating his ideas, he decided to pick another project. Google basically doesn’t let you to apply for multiple projects, so if you do so, you’ll have to decide later. So to save your time: Before picking FreeBSD, just glance at other projects and verify you really want FreeBSD This will save lots of time to everybody.

I hope this will help you understand what is important in the proposal for 2013 Google Summer of Code (there will be next Google Summer of Code, right?)