Everything seemed normal

The air was warm, humid, and dense. I was panting after a badminton loss. I had never been good at this, or any sport for that matter. But I liked flexing the muscles after months of home confinement. Stepping out of the stadium, a rundown building surrounded by brick walls and metal sheets, the air was a lot more pleasant. The location of the court was nice, right next to a river. An adult was sleeping on a hammock under the shade hovering over the river bank. Half a dozen of kids were swimming in the grayish water. I couldn't help but notice, on the other side of the bank, a slump area where the river was, among other things, part of their toilet and sewage. Right at that moment, it was easy to forget Saigon was a cosmopolitan of ten million residents and the engine of Vietnam’s emerging economic prowess. Under that facade, Saigon was still pretty much a part of South Vietnam - a maze of rivers, canals, and delta farmlands - and the fate of the city and the land is anything but one.

Everything seemed normal except that it shouldn’t. We just struggled the whole ordeal of a four-month lockdown. Covid was still reigning across the country. Vaccine coverage was around 30%. In the rural area, agricultural produces was left to be spoiled on the field as it would cost more to ship elsewhere. Waves of laborers retreated to their hometowns without the slightest idea of what their future looked like. Meanwhile, in the city, everything cost more and the labor shortage was high. For the first time since the stats were made available in 2000 Vietnam recorded negative economic growth of 6.17%. Real estate, stock, crypto, and even gold markets were reaching their historical records, not because of the belief of a V-shape bounce back, but because of people hiding their assets for the coming inflation once the stimulus hit.

Everything seemed normal but the substance had changed. The life of the elderly was more fragile. Break-ups and divorces rose. Worst of all, young people, including kids, were robbed of 2 years in the most important period of their lives. Covid was unprecedented. It exposed many of us to our worst enemy, change. Some could brag how embracing change was part of their DNA, but let’s be real, few people turned their life upside down for the kick of it. Changes forced people to face uncertainty, make decisions without a mental model, and live with the consequences. Take an example: kids and schools. Among people my age, the number one reason for self-enforcing isolation at home was the worry that they could bring the virus home to the kids. The local government expressed the same concern, two months after lockdown measures were eased, kindergartens and schools had yet to open. For the entire of 2021, schools had open for around 4 months. While infection risk and its impact on kids were uncertain, it has already been a certainty that the lack of interaction with same-age peers damaged the well-being of kids. Which was more critical, a 0.1% chance of infection, or a 100% chance of development impact? Reports and studies were one thing, but when it came to our lives, the choice was personal, emotional, and far from statistical. Whatever the decision, the scars left on the current generation would be slow to fade.

A few days ago, Charlie Munger stated markets were even crazier than the dot-com bubble. He might be right. Statistically, he had been right more often than he was wrong. But the same statement could be found in 2006, and then in 2015. As the world got better connected, it had been inevitable that we dealt with bigger crises impacting more people. I believe that eventually covid and its crazy variants would be over, that we would treat covid without much discrimination from its influenza cousins, and everything would seem normal again. But make no mistake, the course of the world would never be on the same trajectory had covid not happened, and normalcy is anything but a wet dream.

In the river, the kids, tired of breaststrokes, had changed their game. The older kids were hopping on a floating traffic marker on the water for big jumps, betting on who could make the biggest splash. The younger ones speculating from the safety of their bright-orange life vests. A few years ago, there would have been neither markers nor life vests. Done mourning my loss, I stepped back to the court for the next game.

The case of Project Manager

I have never used a clickbait thumbnail for my post before, not even when I nuked a production database. But admit it, the thumbnail got your interest this time. Let's see if I can deliver.

The role of a project manager (PM) is somewhat controversial in the software development community. Generalization is bad. There are bad PMs and there are bad engineers. But you don’t quite see the same question posed for product and engineer people, the other two pillars of software development. Google even tried to let go of all of its PMs and had all their engineers reporting to a single VP of Engineering. What makes PM different?

The opinions about PM are more subjected to bias than others. A lot of PM work happens behind the scene. When work is running smoothly and on schedule, every day is business as usual. At times, it feels thankless and unappreciated. It’s only when a project is plagued with issues that the PM starts getting attention - usually not in a good way. The strategic values of project management tend to take place before shit hits the fan. After that, primitive instincts kick in, engineers keep their heads down and work harder, the process is out of the window, and a PM seems to only stand in the way. A great PM could still shield the engineers, prioritize work so the worst fire is put out first, communicate the impact to stakeholders, and plan for the next step. But usually, a great PM wouldn’t let the shit hit the fan, to begin with. In other words, PMs are judged after things have gone FUBAR, and everyone can afford to be smart in hindsight. The same hindsight you probably had learned from your angry parents after those end-of-year conferences with school teachers. No mom, I wouldn’t have skipped school that day had I known there was a test. 

Every good PM succeeds in his own way, but bad PMs all fail the same: they didn’t do what they are supposed to do. What are they supposed to do? Sadly there is no universal truth. Granted, each company operates in its own way. At some places, SWEs deploy their own code, while in others, SREs reign supreme power over the production system. Some product owners are supposed to help with GUI design (hopefully based on some standard design system) or help with market research. But by and large, the outcome of members in a tech team is bounded to tangible deliveries and the inter-company differences are within standard deviation.

The PM role? They are supposed to be responsible for the success of the freakin’ project. Now that’s wild, how is that translated into concrete action is anyone’s speculation. That might mean being the link between engineers and customers, making sure both sides get what they want when they want. It could be about being a master of process, fluent in a range of management methodologies, and having an eye on constant improvement. Perhaps it involves understanding the web and mobile architecture, knowing modern technologies, and being quick on the uptake. Or I might have as well just described an Account Manager, Scrum Master, Technical Consultant, and part-time Avenger. PM’s scope of work is ambiguous by design as no two projects are alike and project management is the glue pulling things together, but unclear expectations are the breeding ground of disappointments.

Another thing that gives the PMs bad rap is the disdain for technology of some of them. Software development is not rocket science but it is not exactly a pure exercise of muscles either. That is to say, the field is somewhat technical. And just as any other technical field, it is full of useless jargon, lame inside jokes, and know-hows that take years to pick up. Non-technical PMs, people who experience difficulties explaining how a website works to a 5-year-old, manage to navigate around this quagmire by materializing all units of work into tickets. And they proceed to treat the tickets as little black boxes. The meaning of the work is watered down into start and end dates, and a set of labels for convenient herding. This makes a complicated system simpler. Some’s navigation skills are good enough that they don’t need to know how the finished work would look like. Work to them is parcels to carriers, you are supposed to ship it, not knowing the dead bodies inside. As I work for Parcel Perform, this sounds great!

There is a problem though, most people in software development are not in the business of writing code, they are in the business of building software products - hopefully, great ones. The difference is that one is an isolated piece of work with a predefined outcome, the other is a process of figuring out the intersection of market fit and technical prowess. Building non-trivial software requires plenty of arguments, negotiations, and decisions. Non-technical PM can’t call out bullshit. They lack the skills and tools to affect the outcome and all important decisions were handed to engineers and product owners. Without that decision-making capability, PMs turn into administrative assistants, busy themselves with monitoring project status, sending out updates, and keeping track of who does what. It is nice but it doesn't break or make a project.

Some PMs are shit umbrellas. Some are shit funnels. And the canopy of the said umbrella is technical knowledge. Not the same knowledge that is required for engineers to write code, but the knowledge to see through a project with clarity, know what is important and what is not, and call the shots when needed. It is not unusual for an engineer to pick up a book on Agile to better align his working habits with Scrum’s sprints.  The sight of a PM reading Google’s Site Reliability Engineering to come up with a better fire fighting routine is as common as the sight of a Saola. Isn’t that having the cake and eat it too?

Are PMs useless then? The bad ones are. Bad PMs pose negative net morale and productivity for their team. Don’t believe me? Try rewriting the software for the third time because your PM failed to strong-arm a customer to put his shit together. But the whole thing about Google not needing PMs is as much of an urban myth as it is truth. It happened in July 2001, in computer time that was a century ago. It wasn’t the mighty Google where every practice seems to be deliberated, there were around 130 engineers. The layoff didn’t stick. The engineers themselves opposed it. The whole thing lasted for less than a month and was pretty much Larry Page throwing a tantrum against Eric Schmidt’s adult supervision.

As long as software products are written by humans, the role of project management in coordinating a bunch of professionals toward the common goals is always needed. People who think project management is a good way to get into the tech scene without the background need to get their reality check. PMs are responsible for the success of a project even though they have little control over it - how exactly is that easy? It takes a lot to be a good engineer, and it takes even more to be a good PM. The more people on board with this thought, the better it is for everyone in software development. Don't let a bad apple spoils the bunch, and the best way is to work on being a good one.

The engineer/manager spectrum

When I was younger, I was given many forms of the same advice: at some point, I would be better off to double down on either being an engineer or a manager if I wanted to progress in my career. I was booted from my first job while pretending to be good at everything, naturally, I thought this was a good idea.

Life looked at my choice with as much consideration as my parents when I told them I wanted to join the infantry in kindergarten - which was not at all. I joined a startup, and subsequently another startup. I never got to choose. Working at an early-stage startup is like conducting an old locomotive on a half-done railway. You juggle between keeping the engine running, because just like your code it breaks a lot, and building the railway with whatever you find along the way, you are selling a product before it is fully ready. The game is over when either stops.

Along that journey, I realize that career choices are not complicated, they are complex. Complication is when a system consists of multiple parts with intricate interactions. As overwhelming as the parts and interactions could be, they follow some universal laws that eventually make the system knowable. Complexity is when the parts and interactions are not fully knowable and the best we can do is some reasonable prediction. A car is complicated, the traffic is complex. I think people who gave me advice earlier understood this. But in an attempt to simplify a complex subject, half of the truth was lost.

I think when people talk about the engineer/manager choice, they probably mean that at the point you have to weigh your options as being an engineer or a manager, you have collected all the low-hanging fruits in your career. To proceed, you need a level of focus you haven’t experienced earlier. And as it is hard to focus on two hard things at the same time, you would be better off focusing on only one. Which is also true for my parents’ expectations of me.

While that is true, to perform at the peak level you cannot spread your effort all over the place, there is another half of the truth, that is the choice is neither binary nor permanent. You are probably overthinking if you view the management as a critical, life-changing decision. The move to management is not a promotion, it is a change in scope of work in which you drop existing responsibilities to pick up others. And if you hate the new responsibilities, you can always go back to the engineering track. Most companies have had sufficient understanding on this matter and will be glad to provide some rotation options for their employees.

The higher you advance in your career, the stronger the element of leadership exists in your scope of work. People in the technical track get promoted for working on hard and impactful problems. You wouldn’t get to work on the hard problems unless the easier ones were effectively delegated to someone else. The most impactful problems tend to be ones that once solved provide sustainable advantages for the business. Such problems tend to involve people of different background and function. Someone would need to rally them and align their interests to the shared goal. And yet delegation, communication, and goal alignment aren’t common practices among engineers. Although technical leadership isn’t exactly the same as people leadership, I think you will navigate better if you have an opportunity to learn about how people think, how larger projects are prioritized , and how organizations are run.

From my own experience, alternating between the two roles makes me better at both. In a hard project that challenges my technical prowess, I am more of an engineer. In a more relaxing project, my manager role is stronger so I can oversee things go to the right places. I am a better manager because I understand the morale friction caused by a poorly planned project, and I am a better engineer because I know the red flags of a poor project and when to fire alarms.

The final caveat is that engineer and manager roles exist in a spectrum and it curbs your professional and personal development by forcefully align your options along with the extremes. Not only do most people find themselves somewhere in the middle, but they also move back and forth as their careers progress. People who can turn a technical advantage into a business advantage usually have a blend of engineer/manager in them. You can advance without spending time in management. But if you want to give management a try, you should. Do not let your identity be simply defined by a title. 

Lessons from a promotion

My tech team is organized into squads - cross-functional teams owning end-to-end feature development. The squad leadership is a joint collaboration of a product owner, a project manager, and a tech lead. As the business expands, more squads are needed and it falls to me to fulfill these new tech lead vacancies. To facilitate professional growth, internal promotion is favored over external hires. Along the way, I learned a few lessons.

It starts with a job description

The one thing that makes or breaks the promotion is the job description. A vertical promotion from junior to senior involves performing harder and grander tasks with tighter deadlines, in other words, being more proficient a what you have already been doing. A promotion to a leadership position is more “horizontal” in that sense. Pretty much like the time you left high-school, I don’t think any amount of prior experience can truly prepare one for what comes next.

Our tech leads work with people across all principles to provide a cohesive technical vision for the squad, contribute to the product strategy, and coach their members. In that role, many activities are new to them. They will be working with people whose functions they haven’t fully comprehended, like a tech lead with FE background working with DevOps for a deployment plan. They are asked for estimations while given far less details than what they received pre-promotion. They are exposed to HR matters around the well-being of their crew, not all of which make everyone happy. And just sometimes they have the trauma of having their handcrafted solution taken out of context for an entirely different thing and 3 days to deliver. Given the drastic change in scope of work (and the PTSD), it is understandable that post-promotion, some feel like a fish out of water. Unfortunately if there is a structural approach to eliminate this sense of disorientation, I haven’t found it yet.

While it is tempting to propose a five-page long job description listing out all little details one is supposed to perform and hence solve the challenge once for all, the managerial wet dream is nothing more than a motivational debt. Software development rewards people for their creative prowess and that in turn attracts great problem solvers to the craft. Practically spelling out what one needs to do is the opposite of that. The job description should enable the person to picture the boundaries of her authority and the impact she has on the team without resorting to dictating the specific activities. Everyone will have different responses to “make tactical moves to ensure successful deliveries”, or “look after the career development of team members”, and that’s part of the growth. Take that, Tiger Mom!

Strength in diversity

In the previous year, the squad model had some initial successes. The first two squads jelled and performed well, relatively uneventfully. Structurally. both were the mirroring image of each other: BE-heavy, big data focus, led by old-timers. So when it came to the next new squad, there was a strong urge to copy the earlier success: same leadership profile, same structure, and same kind of work. That should be easy, the management knows what to do, the promoted people have existing role models to follow, and things probably fall into the right place like they had done before. That was as close to a squad printer as I could think of.

In reality, my third squad was FE-heavy, had a strong interest in UI/UX topics, and had a product owner stationed away from the main body of the team. It couldn’t be any more different from the former two. I am glad that this happened.

A parthenogenetic offspring of a squad would have been an easy choice down a slippery slope.

I didn’t realize at the time, but collectively the technical discussion had already leaned towards the server side of thing more than it should. It is normal that individually each of us turns our face towards what we know and against what we don’t. But it gets dangerous when we all turn towards the same thing, we get ignorant of our faults and prejudices. In OOP, that is known as closed for modification, and closed for extension too.

The identical leadership profile would also send the wrong kind of signal, that one has be X and work in Y to get promoted. Everyone with a different profile probably feels unappreciated like a 40-year-old on Snapchat and take their chance elsewhere.

With the birth of the third squad, I got to learn the importance of a design system, the vast untapped advancement of browser technologies, and the bias in BE-FE collaboration. All these are areas of improvement that wouldn’t have surfaced if we had gone down the easy path and promoted yet another BE engineer. It itches me to sound like a social justice warrior, but we did find new powers in diversity.

The support structure

No, the new squad was not released to the wild to fend for itself. That would have been bogus.

In fact, the support structure was the one area received the most attention back in the squad formation. We defined the 3-prong structure where product, technology, and agenda support each other. The right people were hand picked for the backbone and the remaining vacancies received the highest recruitment priority. Meeting plan was laid out so everyone had multiple outlets to discuss their opinions. 

The support structure was least of my concern, till something hit me in the face, something technical yet also... sociological.

The new squad got its people and work split from the two existing ones, like a cell division. Hence found itself co-contributing a number of code repos with the others. That led to some confusions where its realm of existence started and ended. The team operated with a constant fear of stepping on someone else’s foot. The organizational structure was changed and had not been reflected in the software interface. Wham! It was such a classic case of the Conway’s law that I was awed to observe it first hand, yet hurt for not seeing it coming earlier. The law was one of my favorite engineering observations, right up there with Murphy’s.

The following rectification was relatively straightforward. We educated people about the boundaries of squads, brought in service contract to strengthen the interface between them, and proceeded to splitting shared services into smaller ones where it made sense.

A personalized journey

Accompanying the tech leads on their way through aforementioned obstacles was a rewarding experience but easy it was not. There are many questions yet few definite answers, how many tech debts are too many, when an internal tool should be made. Much variation in preference, some are more than happy to deal with abstraction where others are keen on a transparent view. And much uncertainty ahead for that no plan can account, how one accounts for spending a month waiting for an engineer to onboard just to have the guy quit a day before his start.

It is probably apparent now that I haven’t done enough of this to actually know what I am doing. But I am experienced enough in software development to deal with uncertainties. And I am invested in getting it work for my team.

The Agile Manifesto has it that

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

They are good guidelines to drive our weekly catch-ups. We keep a very experimental approach at what we are doing, and maintain a close feedback loop. What work are replicated elsewhere, what don’t are studied. But most importantly, I always try to be a thought partner through out this journey. Interactively growing a team where people are collaborative and open about their problems today is more important than having it down to a science with a rigid plan tomorrow.

If one day I have a toddler, I ain’t no need for books.

Onwards.