Endgame

As I start to write this post, it’s not over yet, but it nearly is. And I’d also have to deal with another set of challenges, like starting work in a very different environment from University of Mexico. Work is very different from studying.

There were very different types of masters, and I wish that I had closely researched what I was going to do before I came here. I only managed to earn my masters after almost two years here. If I had taken my masters part time in Labyrinth Uni, I would roughly be finished around now. It would not have made any difference. But my grades would have been a little crappy. I don’t know if I would have liked that. I would have a lot more money in the bank than now, that’s for sure.

There were a lot of other good universities which offered masters in Computer Science – all of them competitive, of course. Some unis did not offer masters’ degrees, only PhDs. Some offered only part time master’s degrees, and their classes were on evenings and weekends. Some offered degrees that could be completed in one year, but I don’t know what the stress would have been like. In a way I’m not that happy that this took that long to complete, but in a way I’m glad that I had enough time and space to raise my level of computer science knowledge to the level where it was reasonably certain that I could get employed. If I had to do it all in the space of one year, I might not have succeeded, although one of my friends was pretty dismissive of the possibility of failure: “this is a market where computer science is king. How can you not get employed with a postgrad degree in computer science? If you can’t find work, nobody can find work.”

Ultimately you can only study in one place at a time. In the end, I studied in a place much further south of Snowy Hill, and therefore very different. I had talked to people, and I found out that there were quite a few people who had problems finding work in IT nonetheless. Many of them were not English speakers. The Indians and the Americans had no problem. It was the people from China who struggled. And I’ve heard stories where people from China would employ people to pump up their credentials for them so that they could enter the masters’ programs here. The people from China were of varying levels of ability – most were at least above average students, but a few definitely lacked an extra level of insight and technical sophistication. But I have to take my hat off to them, because if all the lessons were conducted in Mandarin, I don’t know how I would have survived.

I still remember the last months in Singapore where I kept on asking people for advice, and wondering if I should make this change to my life. I didn’t know anything. I had never changed jobs before. I did not have an undergraduate degree in computer science. I had never worked in the USA. I only knew that the USA had a high unemployment rate and I thought that it was unlikely that I would find work here. I was just lucky that I was studying a recession-proof discipline. It did not really make sense to give up one or two years’ worth of earnings, and enroll in a university where I had to pay much higher tuition. And it only dawned on me six months into my studies that if I didn’t work in the US for one or two years after graduation, I would probably have wasted my money.

The master’s project was a tetchy experience for many. Many years ago, all master’s students had to submit to a comprehensive exam which tested you on the core subjects. And then it was replaced by a project, where you had to submit a report. If the master’s thesis was a mini-PhD, the project was a mini master’s thesis. So on paper, it shouldn’t have been a very stressful thing.

But it was. There were a few things that made it stressful. It wasn’t stressful if you were American. You could work with a startup starting from whenever, and you could work part-time with them and finish up your master’s whenever. For a foreigner, it would have been much more difficult because they would only give you a work permit when you completed the master’s. So the passing of the master’s degree was almost a matter of life and death.

The other thing is that the University of Mexico was a public university, and at the same time it has to maintain pretty high academic standards. So the professors are overworked and underpaid. They’re not going to enjoy having to take care of two or three master’s students knocking on their door all the time. I’ve looked after master’s students before, and I know how much work that is. You have to brief them, tell them why their ideas are stupid, tell them what could be done.

I had spent a few hours with a grad student, and then afterwards he left San Diego. We had weekly meetings over Skype, with the professor. But the professor never said anything – I think he did just enough to not be accused of negligence. The grad student (actually, the newly minted PhD) did not completely understand what I was doing. He clarified a few things, but he never really understood my work. So a lot of the time it was a one man show. Yes, the professor gave me the data, and pointed me to a supercomputer that I could use, but that was about it. I was thrown into the ocean in order to teach me how to swim.

Standards are higher now. Twenty years ago, you could reasonably say that you were stuck, and that you needed the help of somebody more knowledgeable in order to help your solve your problems. Now there would be no excuse. All your answers are googlable. Everything you will ever need to know is available somewhere else on the internet. There is absolutely no excuse for ignorance, no excuse for failure. What needs to be developed is the ability to find the answers whenever you are stuck. It’s the next closest thing to real life.

There was a form, for the professor to sign to indicate that he was satisfied enough with your project in order to graduate. And there would be a little power struggle concerning that form. I’ve come to understand the significance of that form. There were a few times when I’ve had to convince my professor to sign a few other forms and he was a little suspicious that I was sneaking in “that form”. Actually they were immigration forms, or requests to study part time.

It was a little fishy that people were reaching a point, three weeks before the end date of graduation, without really knowing for sure that they were going to graduate at that point. I had a job offer waiting for me, and I obtained it after a fair amount of hard work, and the thought that it could be taken away from me because “that form” was not signed was fairly stressful. I suppose it was their way of enforcing standards and making sure that students did not skimp on writing a good report.

But without the help of the professors, mind. Like you could send them stuff, and they wouldn’t be able to respond to you because they are running two labs, the CEO of some other start up, organising weekly talks, flying to conferences, sitting on committees, teaching classes, and raising families on top of all that shit. They work on such a brutal schedule that you feel kinda bad just asking for a few minutes of their time.

Towards the end, I was talking to a few of my fellow master’s students. Many of them, to my relief, had managed to finish the work in one term, although very typically, they had first made contact with the professor one term before that. There was one of them who was taking a fairly heavy course at the same time that he was doing the project. And he only started meeting the professor at the start of the term. We were both working on Hadoop projects, but at least he had access to a supercomputer account that had dedicated clusters set up for him to do his computing. My account was more akin to a public computer, and every single time I ran a program, I had to get the system to give me the nodes, set up the system, run it, and then shut down the system. I still remember the half a week I spent trying to unfuck that system, turning over every stone, cursing and swearing every step of the way until I got all the kinks ironed out. Unfucking a distributed system is no joke, because you don’t really know where to look for all the log files, you look at a whole bunch of scripts and you don’t really know how they work and what you’re supposed to change. Ultimately, I managed to succeed. Which is good because that was the basis for at least half of my project.

Another guy I had met on the last week told me that he still hadn’t written the paper. He was panicking over it. His professor would meet him on the last day, and go through the editing, and only after that would he be allowed to submit the paper. My professor never did that for me. But I do suspect that it has something to do with the way I work: I usually notice that my supervisors end up getting me to do all the stuff myself, and give up on managing me. There are usually two reasons for this: first, I don’t usually do what I’m told to do, and secondly when I do it my own way, I usually get fairly decent results. It is difficult to read engineering papers. The intellectual content is not that difficult. What’s really difficult is that all the elements, all the stuff that everybody is familiar with, is combined in a fairly novel way that makes it difficult to follow. It’s like opening the hood of an engine for the first time, and figuring out how things work. The mechanic who built the engine knows it right away. The poor guy who has to read that paper is your professor, and he has a million and one other things to do. That’s one reason why engineering is so difficult, and why it’s usually a bottom up effort, rather than a top down. Because it’s quite rare that a boss can master all the details from the top to the bottom.

Anyway a few days ago I bumped into a fellow masters student who was rushing through the last few experiments for his report. He told me that he was trying to solve a few problems. Obviously I had met him just as he had finished struggling with those problems, and he was nerdy enough to think that those were things that other people would be concerned about. But he was describing a simulation, and when he talked about certain things that were not initialized because some data that should have been loaded wasn’t loaded, I knew exactly what he was talking about. He also said that sometimes you change something, and it runs faster / slower. So you report that. But you very quickly run into trouble because you can’t explain why. Believe me, I’ve dealt with a great amount of that bullshit.

When I got my professor to sign that form, it was a very weird meeting. The due date was Friday. On Monday, I met him to talk about my project, and he told me to prepare a presentation, and he told me to change certain things in my report. On Wednesday, I wanted to meet him, but he wasn’t around. I decided to hang around the computer science building instead. When I went around the master’s student’s lounge to hang out, I bumped into other people and I heard about their stories. Sometimes I wondered what it would have been like to work with other people for my project work. I had deliberately closed myself off to other people to concentrate on my work. Sometimes I would lock myself up in my own room. With the blinds drawn down. But eventually I would get into a rut where I was just aimlessly surfing the internet, so I would have to get out of that room and go somewhere else to work. Other times I would go to the main library. Or some random student lounge. Or some random café. It didn’t matter where I was since I was accessing the supercomputer by remote. But I very seldom worked in the computer science building. Maybe I was sick of that place. Or maybe I just felt that meeting people distracted me from my work. But during that day I did bump into quite a few people I knew. And I wondered what working with people would be like.

I had been up since midnight. I worked on my slides from midnight until noon. I had this funny habit of just stewing on my juices, and not writing any words, until – bang, I would start writing for hours at a stretch. Then after I was exhausted, another period of apparent inactivity would follow. I don’t know if that was typical. I felt that I had to make sure that I met my prof. So I went over to check his office every half an hour. Finally, at around five, he was in, but then he was talking to the other people in his startup. He saw me and suggested that I wait outside his office. So I camped in the corridor, a little too distracted at the noise coming out of it to really work on editing my stuff. An hour passed, and then a woman walked into the room. And then walked out. The guys who were talking to him – I knew one of them – one of them came and told me that the professor just got news that his mother had died. At this point in time, he was in his office, speaking in his native language over the phone. Then he finished the conversation. He asked me to hand over the form, and then he signed it without saying anything else.

So this was totally crazy. I had just spent half a day wondering what it would have been like to have done my project in a different way, or to go through my graduate studies in a different way. There had been maybe two or three months of massive anxiety when I was always saying, “tell me how it ends”. And this is how it ends: the professor has a situation, and he’s in a bad shape, and he doesn’t want to see me anymore, and therefore he signs the form so that I leave him alone. Great!

After this, I turn to the graduate student. I wrote him an email, telling him thanks and oh by the way, the professor’s mum just died, just so you know. I also added that I had made a mistake in undertaking this project: I should not have been working on this project because there were two people – the one who was in Mexico was not interested in the project, and the one who was interested in the project was not in Mexico.

And then he wrote back to me telling me that he was also quite disappointed about how much face time he got with his professors. He thought that his professors would be more caring, but it didn’t turn out to be the case. And he felt that having to work on things on his own was a great character building exercise. He told me that it was very difficult to work with the top names in the field, because they always expected the world of you.

And I thought, OK, fine. But what I felt that he neglected, and what I also neglected was the possibility of hooking up with your peers – the other students in the lab. Getting ideas from other people, and learning from other people. I could have gone onto a mode of operations where I was mainly thinking about getting professors to sign my forms. First I would have to work on myself, in order to be good enough to discuss things with other people. Then the second step would be to work with peers, in order to be good enough to have things worth showing to other professors. Then it would be time to tackle the big guns. That was something that I had done during my work time. I strenuously avoided talking to big bosses until I was sure that I wouldn’t be found wanting in front of them. I always stayed under the radar until I became good enough. But that approach can also be rightly criticized as being too anti-social.

When you leave a great university, like University of Mexico, or Snowy Hill, there are always a thousand and one things where you think that you could have done differently. There will always be tons of things that you didn’t do. It’s just a matter of whether these amount to what people would consider “regrets” or not.

The professors who were examining me were interesting. One of them was an artificial intelligence professor, and I had taken one of his courses before. He was a tough grader. People routinely hand out As like candy in grad school, because they don't want your courses to get into the way of your research. But he was pretty demanding in that course. And I know that during academic seminars he would always ask all the questions. The other professor was even worse: I had never had much contact with him because he was in an unrelated field, but I had heard stories about how he would conduct lab meetings during Sunday nights, and how he would call somebody in the middle of the night and ask that a few experiments would be run RIGHT NOW. So I wasn't really looking forward to the examination. The most interesting thing is that the second professor's son would be serving as a summer intern under the AI professor, and he would be working on a topic that was similar to mine. Fuck my life. My advisor had told me that it's very rare that a person fails the master's exam - it almost never happens. But I was not totally looking forward to this experience.

However on the day itself, I maybe did not get as much sleep as I could have. The day before, it struck me that I was this close to passing my masters. Yes, my prof had already signed the thing off, but until then I had been tying up loose ends. So when I had all the stuff wrapped up, I just sat back and thought for a while - it was such a beautiful day in Mexico. Although my life in the University of Mexico would come to an end. I was 99% there. But not there yet. It was like running 42 km of a marathon, and the finish line is JUST THERE... but there's still the last 200m. And it involved waking up on time and turning up to the exam.

It wasn't a bad experience. It turned out that the guy whom I guessed was the first examiner was wrong. It was somebody else, and I knew that guy. I had once tried to take a big data class before, and it required me to pass an interview with the professor - I suspected that the professor did that because he had limited number of accounts on the supercomputer. But he decided to deny me access to the course.

The presentation went smoothly. I had rehearsed what I was going to say. I usually hate doing this but I did it anyway. The second professor, the Mr tough guy was in a cordial mood. He asked me if I were a Singaporean, because he could detect my accent. I was. In fact, I was wearing a red t-shirt and white pants that day, but later put on a blue sweater because it was too crass. I could have pulled off my sweater there and then and told him, "I'm wearing my national colours today! For good luck!". The presentation went smoothly. Or at least I could see that those two were not trying to give me a hard time. They asked questions that were not that difficult. I had an answer for everything. And in the end, the first professor said, OK, very good. Pass. And he was nice enough to add that he would have changed his mind about not allowing me to take that course. But of course it's too late now. So this was the end. It was really the end. Like I would not attempt to earn another academic degree ever again. My formal education, which started with my first day in kindergarten, was really over. And I could now carry on with the rest of my life.