eli james

The Two Tiers of Singapore's Tech Companies

7:14am, 19 September 2017

tl;dr: There are two tiers of companies in Singapore - good and bad. People in the bad tier have to fear outsourcing. People in the good tier don’t. Each tier finds it hard to believe the other tier exists. And so the complaints and recommendations for one tier are usually completely different from the other tier, almost as if people are living in two different worlds. This essay describes what these two tiers look like.

There’s this meme that software engineering in Singapore is a terrible career for most people, and that you should stay away from it. This idea seems particularly persistent on Reddit, Facebook and in the comments section of Tech In Asia.

See this post, for instance. shadowys says that Singapore is a bad place for software engineers, that places in Singapore don’t have good software engineering, that NTU/NUS Computer Science doesn’t help prepare you for a proper career of programming, that the tech jobs in Singapore will be outsourced. Pretty standard as far as Reddit Singapore Tech Industry rants go. The comments are then filled with people who agree with him, describing a sad future ahead.

You have to wonder if shadowys has a point, though, because he hasn’t been the only one making these complaints. See the comments in this, and this, and this. All of these objections are similar, despite being made by different people over different time periods. The majority of them are negative about the Singaporean tech ecosystem. And none of these people – I believe – are stupid. But still, nothing these people have said matches my experience of the industry today, nor that of my friends. So what’s going on?

The two tiers

Sometime back, at a career fair in Ho Chi Minh City, I told an RMIT program coordinator that we’ve had to raise salaries 40% over the past 2 years. He later told me that he talked to a large outsourcing company also present at the fair and they had looked at him like he was crazy when he mentioned my number. “No,” they said, “We haven’t raised our salaries significantly. Where are you getting this information?”

Was I wrong? Well, the numbers that I report monthly to our Singapore office have increased by around 40% over the two years. I wasn’t imagining things. But I also don’t think those outsourcing guys were lying.

I think what’s going on is this: there are two tiers of companies in Ho Chi Minh City. There are outsourcing companies and several lower-tier consulting companies at the bottom. These companies don’t care about employee welfare, don’t care about high employee attrition, and don’t pay good salaries. Let’s call this the bad tier. Then there are the well funded startups, the subset of consulting companies that care about employee welfare, the product companies that care about engineering. This is the good tier. Both tiers are equally real. Both tiers coexist. And both tiers can be vastly different experiences.

The guys at the outsourcing company weren’t lying when they said they hadn’t increased salaries. And why should they? They had a large enough brand presence that they could count on a steady stream of suckers willing to sling badly written PHP and malformed HTML at equally unsuspecting clients all day every day. And if an engineer quit, so what? They were all replaceable components in a PHP-and-HTML slinging machine anyway.

Meanwhile, in the space we operated in, I had to keep increasing the salaries of my guys because they were starting to look around at their peers and going “my friend gets paid USD$2k a month FOR INSTALLING WORDPRESS. WHY AM I STILL HERE?”

(For reference, USD$1k/month is considered 99th percentile for fresh grads in HCMC. To put that in the Singaporean context: imagine someone paying SGD$7k a month salary for writing Wordpress themes and plugins.)

So, ok, you think, that’s great. What does this have to do with Singapore?

I think Singapore is the same. I think there is a bad tier of companies at the bottom. This is where you hear stories about stagnating salaries, where people say that you are at risk of being outsourced, where engineers try to jump to management as quickly as possible, because being an engineer in the bad tier is about as great as being a garment worker in an Industrial Age shirt shop.

(I am exaggerating, I know they’re not the same. C’mon, geez).

But there’s also a good tier. In this tier, engineers get paid good salaries, get to solve interesting problems, work in ergonomic chairs with large screen monitors and eat a small mountain of snacks in the office. This tier is what my friends mostly see. In fact, when I ask them if they believe their job opportunities will increase tomorrow, to a person, they all reply “yes!” This tier is very comfortable, very nice, and is the reason they trigger when someone on Reddit says TECH IN SINGAPORE SUCKS, LISTEN TO ME, STAY AWAY!

Viki’s first day of a new office, circa June 2012

Good tier stories

I think that people in both tiers have a hard time believing that the other tier exists. I’ll go over bad tier stories later, but I think they’re more plentiful online (thanks, Reddit). So: here are some good tier stories, picked from my circle of friends.

A friend just received an unexpected raise this year. Her salary increased let’s say – some multiple between 1 and 2x. She’s a software engineer, not a manager.

I interned at Viki as an undergraduate. I told a senior who was graduating that he should apply because it was cool. He came into Viki and spent 3 years building their data infrastructure from scratch. (He is who I think of the most when someone says there aren’t jobs with interesting technical problems in Singapore). One year after I asked him to join, Viki got acquired for 200 million USD. He walked away 3 years in, after the infrastructure was mostly built, with a decent payout for no extra work.

An ex-course mate works with Haskell at a large company. Before that he worked in a bad tier consulting company, but quickly realised that the opportunities for growth were limited. Today, he tells us that “I’m so used to writing Haskell. The type checker and compiler are so powerful, it makes everything easier. I write the code, and if it compiles, I can be sure it works!” It’s a bit like saying “man, you know the problem with driving automatic is that you forget how to use the clutch” to a group of people who take the bus to work everyday. We’re all suitably jealous.

Another friend, fresh out of university, is working at a famous local startup. He tells me “it’s great, I’m happy and challenged most of the time, maybe sometimes want to be challenged even more. There’s still grunt work and legacy code of course, but overall it’s really good!” He’s working on combating fraud and bad actors in a community at scale, which is super interesting and fun from a technical perspective, and he’s paid well for it.

Consulting companies can be good tier as well, though this is rather rarer. A founder at one of the more respectable consulting companies in Singapore showed me his firm’s Code Climate page. Every single project on that page had at least 90% test coverage, and a significant number of them were client projects. I was blown away. How they accomplish that - while delivering projects on time - is beyond me.

Another friend struggled for a bit in a bad tier startup, then studied enough algorithms to make the jump to Grab. Grab has problems. But they do pay well (salaries starting at 4K) and they do have code review for all commits, plus their scale problems are at least more interesting than writing shit CRUD apps on contract for a Japanese dog insurance company or something.

The last friend worked at a bunch of good tier companies for a bit, and then his last company got bought by Google, and now he works at Google Singapore.

Software engineers walking back from lunch, Robertson Quay, 2012

I could go on with these stories, they’re so common in my circle of friends that I’m not even trying hard for examples. My point is that these experiences exist, and that the good tier is worth spending some time to get into. The salaries in these companies go up every few years. (When my Viki senior joined, years ago, it was around 3.5k starting salary. When I graduated it was 4k and above, across the top group of good-tier companies. I know of one friend who got 7k straight out of university, though this was an upper bound case where he had really optimised his resume).

For some reason people find it hard to believe when I tell them about the salary ranges in my circle of friends. “No, the average for fresh grad software engineer is 2.5-3k” one commenter said to me, several years ago.

This is probably a good time to remind ourselves that an average is rather lossy. If I expose your grandmother to -20 degrees Celsius for one minute and 100 degrees Celsius the next, your grandmother would experience an average of 40 degrees Celsius, but she’d also be severely traumatised. My theory is that the good tier companies are dragging up the mean for salaries in computing, but there’s a gap between them and the bad tier companies at the bottom. While I can’t confirm this, the mean has been moving upwards for the past few years, which is an indication that not everything is shit in Singapore.

I think most people in the bad tier see the average and think that the salary is only marginally higher than their starting salaries, and that therefore everything is still bad, just marginally better. They don’t think that maybe an average of 3.5k is because all the graduates in the good tier are receiving 4-7k starting salaries, and pulling the number up. And if they do hear of such stories – which increasingly they should; the mean has moved to 4k, and then 4.3k today – they chalk it up to overfunding of crappy Singaporean startups.

People also don’t realise the degree of goodness of the good tier companies. For instance, good tier companies are usually better places for growth. The amount they spend on employee development ranges from company to company, but nearly all my friends in good tier companies benefit in some way from their jobs. A number of them get sponsored to attend technical conferences every year. It’s hard to imagine the same can be said for an aforementioned PHP-slinging dog-insurance-website-building consultant.

Objections

At this point, there are a couple of things you could say. You could say that I’m describing a terribly biased sample. Perhaps these good companies are a rare minority in Singapore, and maybe my friends are all super smart programmers who somehow all managed to get themselves into the best places.

My response to that is, firstly: great! Recall that the original complaint was about how ALL of Singapore’s ecosystem was terrible, that all the programmers were replaceable by outsourcing, that salaries stagnate, that everyone should jump to management as quickly as possible. So if you’re asking this question, then I’ve gotten you to believe that maybe some good companies do exist in Singapore. Now is just a matter of convincing you that this isn’t some super rare thing.

When I graduated from NUS, some of my friends had rushed to find jobs after graduation, and found themselves in bad tier companies. One was unhappy at a company I’ll call Super Technologies, and another in a company that made software for – amongst other things – the ERP tolls. Both were deeply dissatisfied with their jobs.

The reason these friends moved out from those companies and found better jobs was because we met up occasionally, and word got out through the computing grapevine that good companies existed out there. They only had to figure out and practice for the interview. My contribution to the group was that I had interned at Viki, and I gained access to the engineers there, who knew the good tier landscape in Singapore better than any of us.

Today, any junior in the School of Computing who interns outside is exposed to this shared knowledge of what kinds of companies are good in Singapore. It’s only natural to share things like “don’t go to Super Technologies, they suck because the management doesn’t get technology” or “join Pivotal, because they’re seriously good”.

This is how my friends slowly migrated their way to the good tier. The majority of my circle are now in good tier companies, and I’d wager that the same story is playing out with various other cohorts after us. I don’t believe my story is unusual. It’s not too difficult to believe that people share company recommendations with each other, after all.

I guess the next thing you could say is “OK, fine, maybe a number of good tier companies exist, but they’re still such a small handful in Singapore compared to the US.”

This is a fair point. Sure, if we’re talking demographics, maybe we could have a long, abstract conversation about whether there are enough good companies for all the software engineers in Singapore. We could also have a long, thoughtful discussion about the various costs vs benefits of living in the US vs Singapore. But I’m not interested in that for the purposes of this essay. I’m approaching this purely from a “I’m an individual software engineer, I have to live in Singapore, and I’m looking for a career, what can I do?” approach, and when we look at it this way, things appear a lot better.

For starters, did I mention that the good tier companies here are starving for talent? Or, to put this another way, there is a global scarcity of people good enough to get into good tier companies. You may have noticed that I mentioned good tier companies increase the salaries of all the levels every few years. You may have also noticed the story about my friend getting an unexpected raise earlier. Now think: would companies do this if there wasn’t fierce competition for talent?

The key phrase, of course is “good enough to get into”. I’ll talk more about getting into the good tier later. But to me, that isn’t as important as convincing you that it exists.

Not all my friends are in good tier companies. Some, like my friend using Haskell, started his career in a bad tier company. The difference between my friends and the bad tier programmers who complain on Reddit is that my friends know the good tier companies exist. I think that matters a whole lot. There’s a real difference in optimism when you know good jobs await you around the corner, so long as you figure out how to get into them. It’s much better than thinking all the jobs out there are as bad as the one you’re currently in.

A final word on the good tier stories. In case you think the people in all those stories are amazing programmers who topped the school in all their courses: no. My friends weren’t top performers when we were at NUS. The best people in our cohort typically ended up in Silicon Valley. The friends in my stories above come from broken families, from good families, from poor households, from rich households, from other countries, from Singapore. Some of us, like me, failed multiple university courses. What unites us is that a) for whatever reason we are tied to Singapore and b) we have no problems spending a large amount of time programming.

And yes, by the way, you can totally say that good tier companies in Singapore aren’t as good as companies in the US. But at least now you admit that good tier companies exist in Singapore, and the comparison isn’t bad tier SG to good tier US anymore, it’s good tier SG to good tier US. And we’ve at least moved the conversation away from “STAY AWAY FROM TECH IF YOU WANT TO WORK IN SINGAPORE, YOU’LL BE OUTSOURCED” to something more nuanced.

Programming was always this r̶i̶d̶i̶c̶u̶l̶o̶u̶s̶ cool in Singapore!

What is the bad tier?

We should probably pause at this juncture and define what we mean exactly by bad tier, to avoid a bunch of problems later.

I’ll define bad tier as “could plausibly be outsourced to a cheaper country” or, as the bad tier complaints put it: “could plausibly be OUTSOURCED TO INDIA.”

(By the way, I’m overusing the “outsourced to India” meme in this essay, because that’s the bogeyman that bad tier complaints overwhelmingly use. I do not mean that programmers in India are bad. I do not mean that all Indian outsourcing companies are terrible. If you’d prefer, you can replace all mentions of India with “outsourced to a cheaper, developing country”.)

My definition allows us to exclude a couple of nonsensical scenarios. For instance, Viki didn’t always pay well - it was once a bunch of guys working out of Hackerspace SG. Could Viki be considered bad tier when it was small and scrappy?

I’d wager no. Viki’s early team wasn’t outsourceable because they were doing work that was too valuable. Therefore Viki was motivated to invest in its employees. And indeed – the kinds of problems that the early Viki employees faced were very different from the usual bad tier complaints we hear about online.

I like my definition because I think many of the negative symptoms of the bad tier are explainable by the “can you be outsourced” metric.

For instance, if software engineers at your company are treated like shit (that is, you can imagine that some day, plausibly, they lay off the team and recruit in India) then it’s likely that the company doesn’t view its programmers as particularly valuable. Contrast that with Google’s search team, where the very thought of outsourcing is impossible to even imagine today. The business term for this situation is that the company sees the programmers as a “cost centre”^[1].

There are a number of reasons you could be seen as a cost centre. It could be that the work isn’t very important to the business of the company. It could also be that management – rightly or wrongly – doesn’t see much value in maintaining in-house technology. And of course outsourcing companies and technology consulting companies are by definition cost centres – organisations that other companies hire to do work they don’t want to do in-house.

As a result, if you work at one of these places, you should see many of the complaints that Reddit commenters level at “tech companies in Singapore”. You should see very little employee training and investment (because “why train employees in a cost centre?”). You should see stagnating salaries (because “if the costs of running this department go up any further, we’re moving to India!”). You should see engineers with very little power or respect accorded to them (“you managed to keep our IT systems running without major cost increases, here’s your promotion mr manager!”) which means that software engineers will quickly learn to move into management, since that is what gets rewarded.

If “plausibly outsourceable” is the definition for bad tier, then the reverse, “implausible to outsource”, is the definition for the good tier. What’s interesting with this definition is that you don’t even have to be doing super valuable work to be considered a good tier workplace. Your department simply has to appear not outsourceable. There’s been rumours for a long time now about various government IT projects being brought in-house, after various people in the Singapore government have had a change of thinking about the value of building IT systems internally, as opposed to engaging external vendors for them. Therefore, the engineers working in GovTech are very likely experiencing a good tier environment, with many of the same benefits. And the evidence so far bears this out.

The other implication about this definition of the good tier/bad tier is that being in the good tier doesn’t guarantee high salaries or a good experience. I’ve alluded to this earlier in the section, that Viki used to pay peanuts but isn’t bad tier. I have friends who are very much in the good tier, but don’t see high salaries because they joined their friend’s startup, or they started their own startup. Some have burnt out, which isn’t exactly nice. The key bit here is really just that they’re still in a completely different world from the bad tier programmers. They don’t ever have to worry about outsourcing, they know they’re considered high value, and if the business takes off they would have to scale the salary and employee benefits aggressively.

Anyway, enough about the good tier companies. What are some concrete bad tier stories?

This piece on TODAY interviews a handful of software consulting companies, one of which eventually outsources to India. This is pretty much the definition of bad tier. I mean, if your only function is to bring in non-critical deals and pass it on to teams elsewhere, there can’t be much value that you’re creating. And if you’re not creating much value then there probably isn’t much space for a good software engineering environment.

Xavier Thay describes on Quora the life of a software engineer in a Singapore Government outsourcing firm:

(…) the lifeline or major source of income for most IT companies is winning Government Tenders, with various companies / corporate entities bidding for the right to be the project contractor.

There are only so many government agencies and projects every year and as such if a company does not try for a tender, the chance of winning it is zero. (on the flip side, if a company does try, and is not the winning bidder, loss analysis may have to be done).

Some years back, it was a more chummy picture. there are only the few IT solution providers who had the resources to handle government projects and they provide employment for many Singaporeans.

However, in recent times, Chinese and Indian companies have started coming in as well, leading to over saturation in a already small market.

(…) How does it affect the engineer on the ground? If too much is promised in the job scope and if the budget is too low, it translates into high working hours with lesser compensation at the end of the year (smaller bonus).

In extreme cases, the employee cannot even claim overtime because the budget does not allow for it.

This concurs with many of the experiences commenters in /r/singapore talk about; I’d imagine that such places would have been the majority employer for IT graduates in Singapore in the recent past. But this is no longer the only option today.

Bad advice seen at NUS Hackerspace

The Game

Now that you know that two tiers exist, we can play a game. The game is “which tier are they talking about?”

This Tech in Asia article titled The Dark Side of Tech Development in Singapore interviewed an Information Systems student predisposed to working in an MNC, a bunch of ‘tech’ companies that saw aggressive poaching, a software engineer who jumped every 4 months for $100 increments, and a bunch of non-technical startups that hired bad outsourced talent to build their MVPs. This article is almost comically bad tier, as if the author went to a startup meet-up and aggressively filtered for cluelessness.

These Reddit comments (one, two), saying that management is where it’s at? Probably bad tier. Ditto for the ‘replaced by India’ meme.

This Quora question asks how the Singapore government can say there’s a lack of engineers, while there being at the same time widespread complaints about bad salaries and jobs. With this good tier/bad tier divide, this is easy to answer: the good tier companies experience the lack of talent, while employees in the bad tier companies complain.

shadowys’s Reddit post? Here’s the interesting thing: I think he lives in the bad tier. Most good tier companies require a technical interview to get in. Technical interviews - especially ones that involve algorithms - aren’t easy to get through, and are certainly not doable by “someone who knows basic algorithms off a free online course but already know how to make a proper Android app”. If shadowys attempted in any way to get into the good tier, he wouldn’t be so quick to badmouth the value of a traditional CS education, simply because he would quickly realise the number of good tier companies that actually value a CS background.

Perhaps it’s worth it to talk a bit about why shadowys and others like him think like this. When you’re in the bad tier, companies care about what you can do for them. It’s a purely transactional relationship: “I need Android apps, I don’t want to train people in Android apps, you have Android programming skills so I’ll hire you.” So shadowys and co see these companies and conclude that what you need to do to get hired is “have skill in X”, and CS schools don’t teach you “skill in X”, so therefore CS curriculums are “outdated” and “bad”.

When you get to the good tier, however, this story changes dramatically. Put yourself in the shoes of a good tier company: you’re fighting this talent war with everyone else in your tier. You’re going to be investing in your employees to hold on to them, and your salary costs are the most significant part of your operational bottom line. In this scenario, it’s frankly ridiculous to hire people who can “just do X”. You want engineers who can adapt to new technologies, evaluate competing approaches beyond “that technology seems trendy, let’s use it” and plausibly you want them to be generalist enough to jump between teams in your company.

So what do you do? Well, it turns out there’s this proven way for future-proofing a software career. You could tweak your recruitment strategy to filter for it. It’s called: a CS education.

It’s wrong to think that only a CS degree can give you a CS education. People with non-CS degrees get hired all the time because they took the time to study. And in practice the interviews are usually mixed in with more practical tests. But good tier companies have an unusually high percentage of testing for CS fundamentals, and this is probably why. They simply value strong CS fundamentals more, because the economics favours hiring generalists programmers.

Why am I writing this?

The first reason I’m writing this is completely selfish: I want to introduce the idea of good tier/bad tier to public discussions about the Singaporean tech industry. I think too many people conflate the two, when we’re actually talking about two completely different worlds. And, worse, the people in either world can’t see that the other exists.

When people in good tier tech companies talk about the future of tech in Singapore, or when the government releases some PR piece about Smart Nation, people in the bad tier comment in droves because their reality does not fit with this vision. It seems ludicrous that the government wants to invest in tech, when they see colleagues leaving engineering to join (shit) management, and when they see salary stagnation in their workplaces.

Meanwhile when someone talks about his or her experience in a bad tier tech company, and extrapolates to the whole Singaporean tech ecosystem, the people in good tier companies trigger because they haven’t had to worry about Indian outsourcing since the 90s and “isn’t that no longer true today? Do you even know what you’re talking about?”

In the future, all such exchanges can be shortened with a link to this essay.

The second reason I’m writing this piece is to tell you that a good tier exists, and more importantly that you can get into it.

I know everything in this essay sounds vaguely elitist. “Random schmuck tells stories about the awesome jobs he and his friends have in Singapore” doesn’t sound too far off. But I really want to impress on you that these “awesome” jobs aren’t as rare as you think. They’re considered normal options amongst my friends - and amongst many of my juniors, or at least the ones I’m able to keep in touch with through NUS Hackers.

There really is no excuse to not try to get into the good tier. Computing is marvellously meritocratic: there isn’t an old boy network you have to break into, there isn’t a professional accreditation test that you have to pay someone to pass, and nobody is locking up CLRS textbooks and going “HAHA NOW YOU CAN’T STUDY FOR TECHNICAL INTERVIEWS HAHA”.

If you want to get into the good tier, the only truly painful thing you would have to do is to repeatedly fail technical interviews as you do a study-and-apply cycle. And this isn’t even some unique punishment reserved for bad tier employees — I’m not that great at algorithm interviews myself, so if I were to apply to a new good tier company tomorrow, I’d have to revise algorithms and data structures and distributed systems like everyone else, and possibly fail a lot when I apply to companies during this process.

To those of you who think that this is a really stupid essay to write, because now all the software engineers in Singapore will know about the good tier and optimise for it, which in turn intensifies the competition for the few good jobs in Singapore — maybe you’re right, and I’m doing the stupidest thing ever. I don’t think that’s going to happen soon, though — you don’t really hear tech companies going “STOP STOP WE HAVE PLENTY OF BRILLIANT CS GRADUATES!” so this feels a little premature. But assuming this does happen — a) I’m not sure writing this essay would have made much of a difference, and b) an island full of Google-worthy software engineers sounds like a totally awesome problem to have, so get back to me when that happens.

But then again maybe most people in bad tier companies don’t want to get into the good tier. I’m not all-seeing; there could be extenuating circumstances that prevents them from making the jump. In which case this essay is still good, because now they no longer get a free pass to say “ALL tech companies in Singapore suck, we’ll ALL be outsourced to India and NOBODY wants to do engineering anyway” because this simply isn’t true.

---

Footnotes

^[1] There’s a better definition of Cost Centre over at Patrick McKenzie’s blog, which I’m going to quote here:

Peter Drucker — you haven’t heard of him, but he is a prophet among people who sign checks — came up with the terms Profit Center and Cost Center. Profit Centers are the part of an organization that bring in the bacon: partners at law firms, sales at enterprise software companies, “masters of the universe” on Wall Street, etc etc. Cost Centers are, well, everybody else. You really want to be attached to Profit Centers because it will bring you higher wages, more respect, and greater opportunities for everything of value to you. It isn’t hard: a bright high schooler, given a paragraph-long description of a business, can usually identify where the Profit Center is. If you want to work there, work for that. If you can’t, either a) work elsewhere or b) engineer your transfer after joining the company.

What I realised while writing this essay is that good tier and bad tier don’t map exactly to profit centre and cost centre. It maps to whatever management sees you as.