WMHACK 17 – what can possibly happen in 3 days

Long story short – I am just back from the 2017 edition of Wikimedia Hackathon, and this time I got things to say. Keeping it a bit different from previous posts on hackathons, I am going to keep this short and steady so that you do not get bored too much. Cutting drama is something which I generally dont do, but this one really needs it.

More media of the event at: Wikimedia Commons

Wikimedia Hackathons are always a reminder of how little I know about the Mediawiki software, and working live with these dinosaur-mw-mentors are more than fun. You might wonder why the Foundation should fly us to a place when you can get the same done over IRC/any other medium. I can tell you why as I complete this post.

Background: We had a Google Summer of Code 2015 project to build and deploy a newsletter extension for Mediawiki – which never hit production yet. We were trying all our lucks at Wikimania hackathons, other summits etc – but we never were close enough. I do not blame this on anybody as for eg, we were almost in beta (one step before prod after WMHACK16, but Wikimania hackathon 2016 saw a huge change of codebase (shift to contenthandler), and we were back at level 0. It had always been a dream of a few community members like Quim Gil, Me and couple of others to get this thing to production – and we were trying hard with little luck.

My aim for the hackathon: Simple, but to get all security review blockers of Newsletter extension merged and get it deployable in production.

Day 0: Arrival, and other fun

Flying from berlin on an early hour flight – this went pretty smooth and I was a bit sleep deprived as well. Sadly I couldn’t get to my room until 13:00 – but I took onto working a bit of my things on my Igalia project – and killed the time. Later that night I remember even having a short dinner with Srishti Seth and spending the rest of the night mingling with various other mentors, etc. We even had some preparatory meetings – how good.

Day 1: Mentor-newcomer program

This year we had a new test of a mentoring program by the Hackathon organizers, and we had kind of like a pitching session and breakout session to induct in newcomers to projects. I had a whole list of newcomer tasks which were pitched to the newcomers as well. We created a couple of Telegram groups to co-ordinate, and I would say that the whole thing went smooth.

Day 2-3: What was that ?

I clubbed day 2 and 3 together as it went super fast and super hacky. I found out Brian Wolff who was lurking around fixing things here and there and made him sit down with the newsletter extension hacks. This turned out pretty well, as he was happy to help and that went all the way from Day 2, 10 o clock to Day 2 night 21:00. WOW. Some funny things which happened (mostly exaggerated, technically should sound fun):

  1. At one point – I was literally dictated on writing a new maintenance script by Brian (no laptop) and Florian (no laptop) together. This was equally embarrassing and informative.
  2. Srishti was there as well, and she fixed one of her first patch sets to Gerrit, which was for her embarrassment – adding an @author tag to one of the newsletter extension classes (I’m kidding here, as my first patch was a spelling mistake).
  3. Around 20:00 Nemo Bis tells me I should contribute to Malayalam translations via the Unicode web app and I get to vote on translations.
  4.  MtDu (a past GCI student) was there as well with us on the table – and he seemed to be enjoying me being dictated as well. He fixed a couple of other bugs for us – kudos for him.
  5. At around 22:00, Brian went up to take his laptop (that would mean that sleep was cancelled for the night for the table), but then at around 22:15, Florian made a statement that made a huge impact. It was something like “The party happens only tonight, and you can fix the code tomorrow too”. I remember packing back the machines, and even pulling Srishti (who apparently pulled in Andre) and what not – we had a huge team checking out the Hackathon party. I blame Florian, and almost felt like adding “Hackathon party” as a blocker for “Newsletter deployment to production”. Kidding again, but that was fun.
  6. On the way to hackathon party we rate ourselves on a 100 (work:show instead of Howard Stern 10). I am rated 60:40 for work:show while Florian on a 90:100 and Brian on a 110:-10. (more kidding, Florian had the worse ratings).

Day 3: We still have things left

We were closing down on things, and apparently Brian and me were stuck on a huge performance issue with the Newsletter tablepager listing. To make it less technical, the current query as he mentions “should be killed with fire as it can fry the database as it uses filesort and we wanted something which uses INDEXes or something cheaper”. Technically we started on this huge thing on Day 2 around 22:10, and this stayed with us all the way till EOD. Main improvements:

  1. 13:00 – we have most of the cases working, but we still havent got anywhere with our blocker patch
  2. <Missed the group photo somewhere in the middle> as we almost thought we were almost there.
  3. 15:00 – the showcase has started, and we were trying our best (mostly Brian) but we couldn’t showcase the whole thing as done.
  4. 17:00 – showcase finished, we find a better room with cool wind coming in to work on it and by around 18:45 – we have it fixed. WOW. It was such a moment to see the thing working and Brian explained his amusement with a weary ‘yaay’ (it was literally just that long).

and yay – we have all blockers resolved for the Newsletter extension security review! and a handful finished from our huge list of tiny tasks at T159081 as well.

Stats for the nerds:

  1. Gerrit changesets submitted during the hackathon for the newsletter extension.

Key Learnings

  • Hackathons work, as you get to make the mentors sit down with you patiently to get the community things done.
  • Telegram chat groups work, but mostly needs to be communicated a bit earlier. This happened during wmhack17, and happy for that.
  • Party on the second night can delay deployments – but luckily I pulled out before 1:00 am on the third morning.

Some funny chats during the hackathon:

Alright. So thats it, Vienna was hot, clean and great! Thanks to the amazing organizers again.

Note: Even though I was a signed in mentor for the mentor-newcomer task, I mostly became a newcomer from day 2 with these people. I tried to recruit in people to work on the rest of the tasks via Telegram, which brought in Srishti and MtDu to fixing things though.

Few updates after the blog was published.

“MtDu was the one who convinced me to go to the party with his SQL argument (If you LEFT JOIN the hackathon to the party, today the party has value; tomorrow it IS NULL). Although upon actually arriving I found my favourite anti-social computer nerd in a corner with her laptop complaining about lack of WiFi, and preceeded to continue to stare at a laptop screen during the party”  – bawolff 

DetailView on Django REST – multiple methods

Long time since I wrote something on Django, but today it surprised me again with its flexibility. Let me put it down here. Remember the traditional DetailView ?

Example from: https://docs.djangoproject.com/en/1.10/ref/class-based-views/generic-display/#detailview

from django.views.generic.detail import DetailView
from articles.models import Article<img src="blob:https://wordpress.com/64cb7829-8030-43aa-91de-8e4cf958aad0" alt="Selection_407.png" class="alignnone size-full wp-image-media-2"/>

class ArticleDetailView(DetailView):

    model = Article

    def get_object(self): 
        try: 
            return Article.objects.get(pk=self.kwargs.get('pk'))

Setting this thing up in Angular to retrieve details of an Article, I used to follow something similar:

from articles.models import Article
from django.http import Http404

class ArticleDetailView(generics.RetrieveAPIView):

    model = Article
    serializer_class = ArticleDetailsSerializer
    queryset = Article.objects.all() 

    def get_object(self): 
        try: 
            return Article.objects.get(pk=self.kwargs.get('pk'))
        expect ValueError:
            return Http404

I was looking through the documentation agian , and I see that the entire thing can be done in like 3 lines. Take a look:

from articles.models import Article

class ArticleDetailView(generics.RetrieveAPIView):

    model = Article
    serializer_class = ArticleDetailsSerializer
    queryset = Article.objects.all() 

Note: This should work only if you have Article primary key received on urls.py as pk. ie, something like this in your urls.py:

from django.conf.urls import url

from article.views import ArticleDetailView

urlpatterns = [
    url(r'^article/(?P<pk>[\w]+)/$', ArticleDetailView.as_view(), name='article-list'),
]

In case you do not want to send in the primary key, you can provide the custom slug name using the attribute lookup_fields in your view definition. Read more here: http://www.django-rest-framework.org/api-guide/generic-views/#attributes

An example request to the URL would be http://127.0.0.1:8000/article/50. Django for life, Enjoy humans.

WikiToLearn India conf 2017, a late recollection

Firstly, I still can’t believe that it took almost 4 weeks for me to come up with a write-up on a conference – looks like I’m slipping. Well, it’s the exam month, and lecture slides are as boring as usual.

Food is <3
Food is ❤

So this was my first ever conference under the sponsorship of KDE, and of course, when you are invited cross-continent – you have the responsibility to make every moment count. I personally do not know how effective I was, but I can tell that this one was one of the best and most active University level conference I had been in – like the people just wanted more and more – which was Wow.

Since this post is coming late, I will have to skip few details here, which other speakers have already talked about in their own blogs. So for me, it started on Jan 16, 2017, morning – and let me tell you that the journey all the way to Jaipur was not the best I had to go through. Like if I compare the route I took with the shortest path – bah, but it was enthralling and peaceful.

Reaching back to India
This was indeed special, as it was like 4 months since I left the country, and upon reaching back I had couple of anxieties too – like things have changed. So the WTL organizing team was there to pick me up from the airport, and yay – we reached our accommodation in the Uni. I haven’t slept for like 24 hours or more, and just went to sleep. I woke up with a call that Ricardo Iaconelli, the founder of WTL is sharing the room with me. He had just flown all the way from Milan, and yeah – we had actually met during Wikimania 2016 back at Esino Lario, Italy. Both of us expressed how cold we felt even though coming from still colder countries (me here in DE) – probably we were not expecting a cold -6 deg.

0th day
Mostly spend the time roaming around with Ricardo and also trying to get connected to some form of internet to complete my slides for the talk in two days. I was super happy to have some fine Indian food at last, and yay – I could see from the face Ricardo too was. From my experience with Italians, it’s difficult to impress them with food, but I think the butter paneer got him.

Met my long lost friend R Harish that night, and wait – he didn’t notice the damage 4 months in Berlin had done to my hair until I took my cap off, and he just backed off for a moment. I need to quote it here that I forgot that cap then and there like it never reached back with me to Berlin. damn.

 

Meeting long lost friends, after long time.
Meeting long lost friends, after long time.

That night we had an online webinar with some serious guy who was bragging something how he invented something, and how he couldn’t license it properly to make it proprietary. I am not naming the things here as this bloke is famous for suing people who stand up and say he is a jerk. Ricardo and myself were deeply moved by this talk, and we had discussions about how Open things should be, and how dangerous tendencies like this bloke can be. I guess this affected most of the excellent keynote he gave the next day too.

I cannot stop the first day without mentioning a machine, Sagar Agarwall, and I wish him better health – nothing more. He’s got all the rest, as I see it from this updates. Get some sleep, man, please.

That night was pretty cold, and let me tell you the worst cold I had in like the past 5 months or so (Ricardo too as freezing).

First day, talks, keynotes and afternoon bash
Things started up, and a modest crowd turned up – yeah – it was more than what you usually expect on hackathons and stuff – probably because it was happening as part of the LNMIIT tech fest PLINTH. By the way, I have to mention Srijan, Haritha and Chaithanya – my juniors during Bachelors and was speakers and organizing as well. I was happy to see them and nice to talk Malayalam – finally.

As a last bench student, I barely listen to lectures – but I tried my best to listen wholeheartedly to all the wonderful presentations which followed, and with the good food and an amazing keynote by Ricardo, the first day was almost coming to an end.

The evening was the best thing I remember about the first day when Sagar asked the participants to start mingling with the mentors around. I had a small but nice team around me, and they had pretty nice questions and stuff about GSoC, contributions, etc. That’s my cake, and I hope I could entertain them with the stuff I had. This went on for like 45 minutes or so – and still continued, people just wanted to talk and gather information – how cool.

Evening sessions were like ...
Evening sessions were like (well)!

During the night, we had a small live hands-on hackathon session – which was again pretty nice. I think the point of the 2-hour hackathon was to make people excited about programming languages, some use cases for using a version control etc. I again got a small pack of students near me, and we really did some serious code – yeah really. With their support and asking them to think about the logic involved, we coded down a python script that would download all images from a given Wikipedia page, and store it in a folder according to the original image names – how cool is that. Personally, I was enjoying myself, and I could see from their face that they were too – don’t people entertain python jokes these days?

That night went on and on, and we had more people coming in asking new questions, and we were solving it one after the other – what a crowd.

That night was cold again, but we managed to get a heater – thanks to one of the WikiToLearn organizers (I forgot his name), but you people saved us. I have to mention the endless fun the speakers had in between, and most of my time was dedicated interacting with them and the participants – which was my original plan, of course.

Second day, my talk, photographs, evening FAQ and party
I woke up early to make sure things were set for my presentation, and with the little help from here and there, I was ready with my slides here. The presentation started with a delay of like 15 mins or so, but the crowd was fresh – and it went well (I guess)? I have to still get the video which the team was recording to analyze how it went, though. I tried my best to avoid text from the slides, adding in more graphical and logical order for important topics – like contributing to an Open Source project, which I think makes it more clear.

Serious talks going on - yeah - thats me
Serious talks going on – yeah – thats me

I don’t know what worked, but after the presentation, we still had loads of students coming in and asking in detail about things to be done, how to contribute etc – which was energizing. By the end of the day, I literally lost voice, but we still planned one more short FAQ session where we asked people to write a question if they have in mind to ask to the speaker panel. That really went well, and we did have some really funny questions and answers. I could see from their faces that all the speakers were having fun – yeah that was one of my best social days ever.

Closing in, leaving town
It was time to wind things up on 19th night – and surprisingly we had a pro show running on the stage, and we were invited! Yay! Ricardo and me sneaked out to the back where we met people who were previously in the conference and they were delighted to see us \m/ We partied together, and at some point they lifted both of us up on their shoulders, and that was the best thing I did in Jaipur that day (or was it people making me to do).

I left later to the airport with Ms. Srivastava – one among our amazing speaker pack. Delhi was foggy, but personally, I felt that I should’ve stayed a couple of days more at Jaipur.

Things that worked out
I don’t usually compare, but I think things really worked out this time with the WTL conference as the speaker panel was pretty nice – yeah it was – and no I’m not talking about me. Other things that worked out was maybe the excellent hand’s own session, and Sagar’s push to people to interact with the speakers – which made them too close with us. The keynote and the ‘guys’ talk on the 0th day too triggered things which made people think about why they should be passionate about Open Source in the first place. How nice?

Things that didn’t worked out
* Heater in our room (:P)
* couldn’t find anything more – it went perfectly as per planned, and more or less the speakers were so comfortable with each other that we were happy to make ourselves home.

I need to end this thing now, but I can assure you there are tons of things which I really missed in this post. The entire thing was written up fast, as I still have exams coming up and my scheduler is looking angrily at me. Thank you, KDE, Sagar, Srijan and the team for such a nice event – and to the amazing participants out there – much love. The speaker’s team – you people were <3.

The event website: https://india2017.wikitolearn.events/

The entire team
The entire team

Surprise from Python on new years eve, 2016

Alright, so another usual night and I was trying to figure out a strange behavior with a Python class I was working. Looks like I had OOP lectures only in C++, and things are little bit different here. So here you go!

I had this strange class method here described as:


class BenchmarkResults(object):

    @classmethod
    def _generate_db_entries(cls, tests, skip_aggregated=False, parent=None, test_table = []):
         # This function recursively calls _generate_db_entries with
         # test_table param set to a list 

and this was processing in a JSON file from one of my POST API and doing some crazy stuff. The API calls this class like

class BotReportView(APIView):
    def post(self, request, format=None):
        test_data_results = BenchmarkResults(json.load(self.request.FILES.get('test_data')))
        results_table = test_data_results._generate_db_entries(test_data_results)

Now the funny part. I found that on subsequent calls to the API, the test_data_results kept on multiplying 😮 like the data from the previous POST also got appended to the current JSON which-should-be-processed. Thinking it was a problem with the client, I tried switching to curl, but with little luck.

The problem
Finally pycharm got me figuring out what went wrong, as it showed a warning on the test_table = [] something like:
The default method argument is mutable. Bah, taking at look at http://stackoverflow.com/questions/9039191/mutable-default-method-arguments-in-python I see that when an empty list [] is kept as a default formal argument for a function, the list is not killed in between, and keeps on growing on subsequent calls – surprise :\

Quoting http://stackoverflow.com/a/9039224/3355893, when we have something like:

def status(options=[]):
    options.append('new_option')
    return options

print status()
print status()
print status()

will print something like

['new_option']
['new_option', 'new_option']
['new_option', 'new_option', 'new_option']

Tough luck finding it out, but it took couple of hours 😀 I got my code easily fixed by not allowing [] to stay as a default param, but separately passing an empty list to the function during the call. Like

class BenchmarkResults(object):

    @classmethod
    def _generate_db_entries_helper(cls, tests, skip_aggregated=False, parent=None):
        test_table = []
        self._generate_db_entries(cls, tests, test_tables skip_aggregated=False, parent=None):

    @classmethod
    def _generate_db_entries(cls, tests, test_table, skip_aggregated=False, parent=None):
         # This function recursively calls _generate_db_entries with test_table param set to a list 

Yay. that was it, and latter called in the helper function, and things started working. You can see the change here, and finally, Happy New year 2017! Make every commit count!