Stories of the underdogs II

The internet as we know today started in the late 1980s. By mid 1990s, it had already had a revolutionary impact on culture, commerce, and technology. Around that time Vietnam had its first commercial internet connection. The tipping point was 2004 when internet usage grew by 200% and gave 10% of the population access to the highway of information. A young population, and a network of locally connected game centers, the first profound impact of the internet in Vietnam was the gaming industry. There were some competitions and when the dust settled, two dominating players emerged: Singapore-based Garena and homegrown VNG. When Garena came to Vietnam in 2010, the press described the event as the doom of VNG, that the little Vietnamese startup would soon be devoured. And there was foundation to that, Garena was led by a Stanford MBA graduate, with a seed fund of $2M and operated at 5 different countries before entering Vietnam. On the other hand, VNG founder was originally a bank officer by day and ran his game center in an alley at night. The startup got $80k seed fund and was non-existential outside of the country. Seven years later, both are still around and extremely successful being the first two SEA startups worth more than $1B. There is a little twist to that though. VNG is still virtually non-existential outside of Vietnam. It manages to get on the same tier with Garena with only one market, while the later has 8. Within Vietnam, Garena is solely known as a game publisher, VNG has transformed into an Internet Giant whose products touch every Vietnamese internet users.

The story of VNG is inspiring, against all odds, and exactly the kind of folk tales you suppose to hear in the world of startups. Most startups suffer from inexperience, understaffed, under financed, and overloaded. Of course, once in a while, there are exceptions, like Color, a photo-based social network led by veteran Vietnamese American founders with $41M initial funding. But for the majority of startups, those 4 horsemen of entrepreneurship continue to chase them till exit. Yet despite being disadvantaged, startups succeeded and disrupted markets from time to time. Again, except if you were Color, the startup was a flop and sold to Apple at $7M. So what if we have read the startup stories wrong. That is, what if instead of succeeding despite of all the challenges, great startups succeed exactly because of the challenges they face?

To listen at these stories differently, first we need to redefine what is an advantage. The same qualities that make big companies appear to be fearsome are often the source of great weakness. For example,
when it comes to technology startups, being big isn't exactly an advantage. While a bigger developer team in theory can produce software faster, they are also significantly harder to manage, might suffer from low productivity because of communicating and bureaucracy overhead, and more expensive to keep. Instagram was acquired for a billion dollar with a team of 13. Whatsapp has 50 engineers for 900 million users. With the power of software, hardware, and automation, size is more a liability than an advantage.

On another point, it's tartarus for a company to expand its core competencies to other areas. Once its core competencies are formed, much of the company resources is spent on extending the scope of features so the product is applicable to more use cases, refining development process, and organizational structure to sustain all those activities. Innovations are still cultivated, usually in form of specialization of teams. which results in a stream of smaller, continuous improvement over an existing product or service. That also makes disruptive innovation, like creating products or services that did not exist before, harder. Google for example is an extremely successful company, yet despite its size and the number of projects, much of Google revenue surrounds its core competencies as a search engine. Most of the projects whose role is to ensure Google's relevancy in the next decade such as Android, self-driving car, or AI, are from acquisition and not in-house.

While big corporates focus on regional and global scale, Vietnam startups, even the successful ones, think and act local. This in turn results in products solving very specific problems existing only in the country, and virtually unheard of for anyone else. As a game distributor, Garena did and still aces VNG in every perspectives. They are well-tuned towards international partnership, multi-nation online game operation, and game tournament hosting. What left for VNG were games that are much less popular and operate in only one country. And it was meant to be that way. VNG resources were spent on another problem of Vietnam in the late 2000s. Back then, to run a game center, you needed to be some sort of computer/game enthusiast because the hardware constantly ran outdated, latest games kept popping up, and your computer needed to be protected against these children who were too eager to try whatever tricks they found on the internet. But that wouldn't scale when game center became a business and investment. So VNG came up with this business development team that would consult small business owners with new location, hardware and software setups, and on top of that, install a home grown management software that not only protected the computers against threats, updated with latest games, but also enabled a VNG membership country-wide. So all of sudden, credit player accommodated in one center, can be used in another, as long as the other place also installed VNG management software. The software spread like wild fire and so did VNG presence in every corner of the country.

The trend in Vietnamese startup in recent years can be summed up as following:

• Solve local problems
• Avoid direct confrontation with bigger players
• Disproportionately abundant in Entertainment and Lifestyle segment, due to the size of it
• Much fewer B2B effort even among each other, which sometimes unintentionally leads to fewer strategic partnership enhancing value proposition and creating win-win situation

In other words, many startups here find guerrilla tactic fits them right. But then, if such can be formulated, why aren't we seeing more successful startups from Vietnam? Because guerrilla tactic is hard, to the point of desperation. Niche local problems aren't always obvious, they more often lurk in a corner, hiding in plain sight. Finding them is like shooting arrow to a bullseye you cannot see. Sometime, the bullseye doesn't even exist. And you have to go on like that days after days, with marginal profit, investors and employees show doubt, sometimes very expressively, none of the effort seems worth it. You only turn to guerrilla tactic when nothing else seems to work. Even startups succeed in this tactic, drop it as soon as they can. Right when VNG revenue from game distribution was stable, they started looking for extension in conventional market, like social network, media streaming, and messaging, all of which eventually replace game distribution as VNG's cash cow.

Doing startup is one of the many ways to gain perspectives of the world. A great one. Much of what we consider valuable in our world arise out of these kinds of challenges, because the act of facing overwhelming odds produces greatness and beauty. But just as important, the nature of these challenges might not be what they appear to be, giants have weakness, and underdogs over and over accomplish the unexpected with right preparation.

Bye Chung Cư Nguyễn Huệ. Bye Saigon.

Bạn phải đọc cái tin này, thì mới biết tôi đang lầm bầm gì. Nên đọc đi, tôi chờ. Lười lười đọc tiêu đề được rồi.

http://www.yan.vn/thien-duong-ca-phe-chung-cu-42-nguyen-hue-xoa-so-gay-nhieu-tiec-nuoi-121754.html

Giải tán mấy quán này, chắc chung cư Nguyễn Huệ chỉ còn cái xác bê tông. Từ lâu người ta không còn ở đây nữa, tại vì, như trong bài, hạ tầng xập xệ quá rồi. Nói cách khác, những hàng quán này giờ là thứ duy nhất giữ sức sống của chung cư. Dẹp quán đi, người dân không (muốn) quay lại ở, chung cư chắc nhường chỗ cho một toà cao ốc văn phòng phù hợp hơn với khuôn mặt thành phố.

Ở Saigon, các hộ kinh doanh nhỏ là xương sống của nền kinh tế, họ không có chỗ trên khuôn mặt thành phố? Đã lâu ở Saigon không có những công trình như Hồ Con Rùa, không nhầm mục đích quần què gì hết, đẹp thì làm. Ngày xây Hồ Con Rùa, ông kiến trúc sư chắc không nghĩ đến việc bán bánh tráng trộn, hay công viên 30/4 thành chỗ gom giấy báo lót đít. Cũng như chung cư Nguyễn Huệ, những địa danh này trờ thành môi trường để các mô hình kinh doanh phát triển một cách tự nhiên (organic). Đó là dạng công việc cần đến chính phủ, tạo ra môi trường phát triển tự nhiên cho xã hội, và nhường công tác quản lý vi mô (đổ đầy shop vào một cái mall trống) cho khối tư nhân. Khi chính phủ hiệp lực với tư nhân bự, tư nhân nhỏ thành người làm thuê, ở thuê.

Chung cư Tôn Thất Đạm. Cái vẻ chất chất nghệ sĩ bạn nhìn thấy mà không gọi tên được là nhờ tính tự nhiên trong tổng thể. Như một ngôi làng Thuỵ Sĩ.

Trà Lạc Đình, Hồ Con Rùa cũng đi luôn? Cô chú ở đây từ hồi chung cư mới dựng.

Hồ Con Rùa nhìn từ Lạc Đình.

Từ sau ngày 25/3/2015, Saigon hay được so sánh với (quá khứ) Singapore. Bằng vai phải lứa với Singapore thành KPI. Ở Sing, khu Tiong Bahru có hơn chục chung cư cũ, được xây từ thời lập quốc nên chính phủ quyết định phân hàng di sản quốc gia, mọi thay đổi, sửa chữa đều phải xin phép. Giờ thành thánh địa cho tụi hispter ghét mainstream. Booksactually nơi tôi mua được bản Sorrow of War ở ngay đây. Những căn đầu tiên ở Tiong Bahru xây hồi 1920, nhưng phần lớn những gì ta nhìn thấy giờ đây là từ 1960. Saigon đâu thiếu di-sản-quốc-gia-singapore.

Tài sản quốc gia Singapore. Nhân tiện, Marina Bay Sands không phải.

Nghị định số 99/2015 có thể tránh được nếu toà nhà 42 Nguyễn Huệ từ bỏ chức năng chung cư của mình? Tôi không biết, tôi là dân tay ngang, và bạn đừng có mà tin một thằng ất ơ trên Internet. Nhưng trung-tâm-mua-sắm-nhìn-như-chung-cư / mall-that-look-like-apartment-building thành trào lưu mới của Saigon? I can live with that. Heck, after a selfie stick is decided to be called "gậy tự sướng", I can live with anything.

Don't work after 7PM

I have had crunch times where my day basically consists of only one thing: work, randomly interrupted by periods eyes close, mouth drools, and mind goes blank, all by themselves. And those times can be healthy, like a couple of days before a release give me a boost of motivation. But doing so over any extended period is just bad. Its bad for me and bad for the very business I thought I was helping.

Enough talks about burnout, about there is more to life than just work (because work somehow gets the bad rap), about working on the same set of problem over and over is damn boring, lets talk about an engineer's values.

As an engineer, my values consists of 2 components.

• My absolute value, defined by skills, methodologies, and mindsets. These values are the stuff I bring to meetup, write about in my blog, and are applicable to whatever I do. They are mine.
• My relative value, defined by my knowledge of the project I am building, the sector my company is in, and how technical decisions were made. These determines my productivity, and ultimately my worth to the company.

Of all the jobs I have had, I always start like this. I don't know anything about what I am supposed to build, but I have my skills as an engineer (hopefully qualified after a legit interview). As I stay around, I acquire more the domain knowledge and know how things are done. I apply what I have learned from the last job to this and in turn sharpen those skills. Both my absolute and relative values increase.

If I neglect picking up new skills, of course my absolute value stays still. Or rather, it slightly decreases as everyone else is forwarding. But if I stay longer, spend more time writing the code that somehow a repetition from the previous job, (same old CRUD), I become the person to provide backstory of features, explain the tweaks and customization we have in code, and set the pace for newbies. My relative value increases, and as a whole, my worth to the company also does.

Alternatively, because I am more than a lazy-ass, occasionally I learn something. New knowledge is hard and immediate reward is scarce so in the beginning I stumble. But those technical advancements are important never the less, like efficient asynchronous model, container orchestration, or architecting data-intensive system. A lot of the things I read, practice, put into pet projects are just out of curiosity and at some point go to the attic for not being practical. But once in a while, it turns out to be useful, I can contribute back to the product I am building. My absolute value enjoys a slight hump and because I still contribute to the project, my relative value also increases.

*Both are extreme scenarios made up to illustrate my point, most of us are somewhere in the middle of the spectrum.

Every night, every weekend I spend completing what I should have done between 9 to 5 fosters my relative value and postpones development of absolute value. At some point, there is a chance my worth as a data archiver wins over my worth as an individual contributor. I get my benefits for being valuable to my company, if not even more. That though makes me as an engineer less appealing to other organizations who derive benefits from my absolute value. A golden handcuff.

I can understand where the idea of focusing on your job 24/7 coming from. Picked a course. Realized I didn't know shit. Sucked it up. Put in a lot of hard work. Submit assignment. Remembered I had 3 other courses that semester. Repeated. This was the vicious circle I went through in college. Basically what worked for college was to spend more time till I am better at it. Which I had plenty.

And for a while, career seems like an extended version of college where the semester is longer. So the same trick should work, right? If I work day and night on my job, I should be better. Except career and college aren't the same. The rules are different, a product is an ultimate goal, whereas an assignment is merely a means to an end, more responsibilities from society and family are waiting, and the problems we face are not only harder, but also demand to be done in less time. What got you here won't get you there*.

The keys to succeed in different phases of my life can be arguably sum up as:

• High school: just be smart and don't do drugs.
• College: lot of hard work and occasionally try something totally irrelevant.
• Career: still hard work but learn to know where your time should be spent (up till this point, but I guess you can slice one's career to a zillion of different stages).

Effective on-the-job training is rare, most desirable skills like machine learning, distributed computing, or business intelligence take years to build up. In this part of the world, sabbatical leave isn't the norm. Many people actually haven't heard about it. So being a top-notch engineer is anything but a personal investment. Who I am in the next 5 years is defined less by the company I am in, and more by what I choose to do after 7.

[Agile Games] Jenga Builders

Summary

The core idea of this game is to have teams build wooden structures with jenga sticks. The game is played by rounds, where each round is supposed to deliver an Agile lesson. In the context of this post, attendees can expect to learn about the darkness principle, quick feedback and communication, and being adaptive to change.

Each round after the first is essentially optional, the host can choose to add/remove rounds depends on available time. The host is also welcomed to pick, or customize her own round to deliver the desired lesson.

Team size

Minimum 4, ideally not more than 6.

Timing 

20 - 60 minutes depending on team size and number of rounds.

Materials:

• A deck of Jenga
• Photos of pre-designed structures, 3 x no. of team / round
• A whiteboard to record scores

Instructions

This game will be run over several rounds, with each round introducing a new concept.

Setup

• Split up the group into team of at least 4, and ideally not more than 6.
• Choose one person from each team to be the Analyst. The Analyst knows the core of the structure, but not which colors to be used. The Analyst is not allowed to touch Jenga sticks.
• Choose one person from each team to be the Designer. The Designer knows the required colors for a specific part of the structure. The Design is also not allowed to touch Jenga sticks.
• Choose one person from each team to be the Tester. The Tester knows the whole structure, but can only answer questions and is not allowed to give instructions or touch Jenga sticks. 
• The rest of each team will play Developers. Developers are the only ones who can actually use Jenga sticks and build stuff.

• The Analyst is given a photo of the structure needed to be built. The color of the sticks in this photo does not matter, in practice, team can use whatever colors available.
• The Designer is given another photo of a part of the structure needed to be built. The color of the sticks in this photo is crucial. The part must be a component of the final structure, with the exact color depicted in the photo.
• The Analyst and Design have to combine their knowledge of the system and convey that to the Developers, who build stuff.
• The Tester is given the photo of the final structure and has the right to accept or reject the work of the team.
• The initial flow is that The Analyst and The Designer will discuss and agree on the structure, convey the idea to The Developers and finally the work is given to The Tester to accept or reject.
• No one is allowed to show her photo to others.

Round 1:

This round teaches The Darkness Principle.

Each element in the system is ignorant of the behavior of the system as a whole, it responds only to information that is available to it locally. This point is vitally important. If each element 'knew' what was happening to the system as a whole, all of the complexity would have to be present in that element.

After the team has formed and roles assigned, give the below photos to relevant team members. And the game begins!

Analyst #1

Designer #1

Tester #1

After all the teams have completed their structure, ask questions on what happened, what went well, and what went wrong.

Learning:

What we learn from The Darkness Principle is that nobody on a team has a complete picture of all that's happening in the entire team. Each team member can only have an incomplete mental model of the whole project. That is why they have to plan and decide together.

Round 2:

This round teaches Quick Feedback and Communication.

Give the below photos to relevant team members.

Analyst #2

Designer #2

Tester #2

The trick here is that there are many different possible outcomes from the photos given to The Analyst and The Designer. Only The Tester knows the correct one. But he is at the bottom of the chain. 

After all the teams have completed their structure, ask questions on what happened, what went well, and what went wrong. And if they want to make any change to the work flow.

Learning:

The work will be carried on in usual manner, then given to The Tester to be rejected. We should note half the amount of work was wasted, and that the position of The Tester isn't optimized for quick feedback loop.

Round 3:

This round teaches Being Adaptive To Change.

Give the below photos to relevant team members.

Analyst #3.1

Designer #3.1

Tester #3.1

The trick here is that the outcome is almost possible to do. (At least for me, I had to put a finger on top of the whole thing to prevent it from falling apart.)

Somewhere half way, when the teams keep failing to build the structure, hand out this second set of photos.

Analyst #3.2

Designer #3.2

Tester #3.2

This time, the structure should be possible. (I managed to make that.)

After all the teams have completed their structure, ask the same retrospective questions.

Learning:

What we learn from this is that goal and target are not always rigid, and in order to deliver, sometimes, reasonable compromises are needed.

Round 4:

Freestyle. This round is for the team to summary what they have learned in previous round and as a whole, feel good about the learning curve. "Finally, a proper sprint!"

Analyst #4

Designer #4

Tester #4

All structure photos can be found here https://www.dropbox.com/s/np96ltzrdznpmxc/JengaBuilders.zip?dl=0

Observations

The game had its first debut at Silicon Straits 18th Feb 2016. Below are the points I observed from that first try.

• As suggested by The Darkness Principle, the key to win this game is communication, communication, and communication. A good team did a lot more talking than their lesser counterparts. Members in a good team were more willing to listen to each other and showed less tendency to suppress minority's ideas. The existence of an alpha role was subtle. All are desirable behaviours for brainstorming sections. The good team also utilized the role of Tester better (raise more questions and shorter feedback circle). 
• Whereas in the bad team, where The Analyst and The Designer had troubles expressing their vision, the team stumbled trying to get the sub-components right, left alone combining them together. With bad vision, the team fought for an alpha role, a process in which team members just went ahead trying to prove them right, spent less time listen to each other, and suppress more "silly" ideas (unfortunately, a few of them were really good). And in general, the bad team were more distant to leadership (members with photos) and treated them as checkpoints, rather than equal collaborators. 
• The gap of speed between a good team and a bad team can be quite big, up to 100% (my subjective observation). This had a couple of by-effects. The good team had way too much free time. The bad team is demoralised. It was suggested to have not 01 host for the entire game but 01 host for each team, constantly checking with each other and dropping hints to keep teams on pace.
• Can consider switching roles between rounds, so that people can try different positions, have time to slow down themselves and do internal retrospective.
• Stronger emphasis on mini retrospective meetings. The debut was run with participants all standing. While this was good for the game play and team dynamics, it thwarted the necessary pause to retrospect what happened, what went well and bad, any improvement to make.
• At some point, The Testers would try to manage the whole project, because they have the complete view of the structure. But as they aren't allowed to give instructions, team needed to have good strategy to query the correct requirements. So far, none of the teams spent time on improving the art of asking questions.

2015 - I don't trust my plans any more

In contrast to the unpredictable 2014, 2015 was a year full of plans and projects. I planned my life around the sparks of changes in 2014, hoping that I could keep the momentum and better myself every day gone by. And of course, like every single time I make any sort of plan, life tossed it into all weird, twisted, yet still amazing, directions.

1. I moved to Singapore, and left.

Open door concert, which I have missed dearly

2015 indeed started big. I was relocated to Singapore and tasked to lead an effort to produce a gaming smartphone. A freaking actual smartphone. An odd request. But why not?

The original idea was nothing short of moon shooting: an Android-based phone with hardware customized for gaming and bulky design as eye candy. The learning curve was steep, gave me butterflies in the stomach for months. I digested books from hardware design to manufacturing techniques, became proficient at navigating through the dissatisfying interface of Alibaba, and went from trying to pronounce CyanogenMod correctly to a walking wikipedia of Android forks.

As a city, Singapore is extremely modern, convenient, and safe. Hate to admit but Singapore's tech community was way more active than Vietnam's with events basically every day. There was much to think about those events. On one hand, they were trendy with all the latest hypes, people were talking about trips to Silicon Valley as if they were weekend getaway (sorry, I grew up on a mountain), and all the big shots have offices there. On the other hand, a few were just sales pitches in disguise, some were fuelled by consultant bullshit (especially anything "IoT"), and the rest managed to remain quite abstract and high level. Actual coding could still be found at hackathons though, which were plenty. But the most important thing was that everything was there and very well-established. So much that I didn't feel like there was a need for anything new.

It took 5 months (or 6?) to present two working prototypes with a detailed production plan. Just to be rejected by management. For the same reasons raised since the very beginning: huge upfront investment, nonexistent margin, and unless one is an Ivy League member, consumer would not bother. ¯\_(ツ)_/¯

Without going any further to the misalignment between me and management, that was the background of the decision that, after half a year living in Singapore, it was time for me to go home for more adventures, and bigger social impact.

2. I (still) organized BarCamp Saigon.

Meet the team

First thing I committed to upon returning to Saigon was to pull together BarCamp Saigon 2015. After the nightmare of 2014, I thought I had learned a thing or two. Well, guess who managed to surprise himself :D

All jokes aside, the preparation of BarCamp Saigon 2015 was a lot smoother, the team had one event under their bell, which boosted confidence, and I received superb support from professor Edouard, to whom BarCamp Saigon 2015 also happened to be his first public event as the manager of RMIT's IT program.

And thank to that joint effort of everyone, I was given some room to think about the future of BarCamp. BarCamp Saigon has been around since 2008, and during that, a community has grown around it. From a tech event, every year we are observing a more diverse crowd attracted to the openness and democracy of BarCamp. The challenge the BarCamp team needs to face is to maintain the same factors that have kept BarCamp an one of a kind event in Vietnam, and at the same time, evolve to meet the quality requirement of niche segments, an important factor if we want to set the standard for a community event.

In the foreseeable future, BarCamp will remain to be an umbrella event for all sort of topics as long as someone want to share, and someone willing to listen. We are looking for more focused events, like TechCamp for hard-core tech, and, say, BulbCamp for creative people. But an event like BarCamp is an intensive effort. we need 3 months of preparation for each. The majority of time goes into looking for sponsors, and coordinating sponsors and vendors both before and on event day. And most importantly, we don't have enough people to sustain more than an event per year, and we are subject to terrible terrible voluntarism. What works in the tech circle might not work elsewhere, and we might end up solving problems that don't exist, or generating irrelevant values. It works way better if someone with their industry inside decide if their industry can use an unconference like BarCamp to spread a fresh air, and we contribute our expertise from many events in the past, and build something new from that combination. So folks, if you think BarCamp makes sense for your industry, please spread the word, we will take 2016 together!

3. I started training to be a long-distance runner.

Who could have known this pearl in the heart of a mega city?

Singapore's national sport is windows shopping. But it was there that I was blessed to meet Deny. Deny started off as the CTO of my then company for a very short period of time, but remained to be my mentor ever since.

By introducing me to MacRitchie reservoir, Deny unfolded a hidden part of the mega city. MacRitchie is an ever-green tropical forest growing around a lake which was meant to serve as a stable fresh water supply for colonial Singapore, a hundred years back. There are traits through the woods and around the lake shore. Some of these traits link with other upper North reservoirs, requiring no less than 8 hours to explore. I am weak, I am always content with 4.8km and 11km traits :P

Deny is a long-time runner and had completed a full marathon a few years ago. I believe it was none of his intention to squeeze the living breath out of the poor me, but he often held conversations as we jogged under forest shades, those that always started with work (and I felt compulsory to respond) but soon expanded over various topics of life. Needless to say, the first few trips were brutal. It called for mammoth effort to both keep up with his pace and follow the on going convo. I have to say, desperately grasping some air to feed my brain with oxygen and restraining not to split any stupid nonsense is an experience very close to drowning. I am pretty sure there were occasions all I could do was mumbling senselessly. Sorry Deny!

HCMC 21K!

But those early days were crucial for my foundation, and bred a new hobby that, by now, would very likely to stick with me for a few years. The whole running and exercising thing has taken me to a number of interesting events, including Bali 10K, Fansipan trek, and HCMC 21K just a few weeks ago. But the most significant one is to be able to run around Xuan Huong lake, the heart of my home city, 6 days in a row. A task I couldn't do back in high school (we boys typically needed to short break in the middle). The thought that 26-yo me is doing something a bit better than 18-yo me is really cheerful.

I am the kind of person who finds being alone enjoyable (which needs to be differentiated from being lonely, which sucks really hard). Running alone an hour or two, or programming 4-5 hours straight is neither boring or difficult, but pleasant and productive. The essence of running is to exert oneself to the fullest within one's individual limits and I hope this can serve as a metaphor in my life too. (Though recently Trang has managed to find her way into my lonesome solace, I am really glad about this disturbance. Perhaps more on this next year)

4. I wrote vividly.

I wrote a fucking blog post every month in 2015.

There, I said it, such an achievement. And I wouldn't have been able to accomplish that without shredding nonexistent constraints (and a delusion) I put on myself.

• That I shouldn't stray away from engineering topics. While it is true that I am strongest at engineering and in the past has met some instances where engineering principles made great sense in life. But the other way also works. One's senerity is reflected in his work and being only one-sided one runs the risk of building his own personal problems right into the product itself. Open up to all sort of topics gave me new sources of inspiration and seed diversity into my career.
• That my Vietnamese was lame. That is actually true too. 

Tiếng Việt của tôi nghe lộc cộc như ngựa chạy. Hồi còn học văn, tôi nghễnh ngãng, lại không đánh giá đúng tầm quan trọng của ngôn từ. Tôi chỉ nhớ được, là lần đầu tôi phải dừng lại, nói lên điều mình suy nghĩ bằng con chữ, là một bài luận TOELF, đương nhiên là bằng tiếng Anh. Cái này kéo dài trong nhiều năm liền, từ phổ thông đến sau đại học. Chả vẻ vang gì, dở tiếng mẹ đẻ của mình tôi lại còn thấy nhột. Dù vẫn còn khoảng cách về chất lượng mỗi khi dịch bài từ tiếng Anh sang tiếng Việt, và ngược lại, mọi thứ đang trở nên tự nhiên hơn tí tẹo.

I was very unskillful at using Vietnamese. I was an absent-mind kid and has always regretted that I didn't develop an appreciation for literature when I was younger. As far as I can remember, the first time I looked into myself and tried to recreate my stream of thoughts in words, it was an TOEFL essay, obviously in English. And that continued to be my dominant form of communication for many years. But being suck at my mother tongue is not something I am proud of and I don't like the feeling of being less than proficient at something as important as communication. Though there is still a degrade in quality when I translate a post from one language to another, it is coming a tiny bit more naturally now.

• That one day I could turn writing as a hobby into a means of monetization. Writing as a hobby is great. And if I can live by writing, it is even greater. The core idea of making money from writing is that someone must find my writing enjoyable. And nothing is wrong with that. People enjoy a whole lot of thing, there must be someone enjoying my writing. But the trouble was that once I had that idea in mind, I found myself writing less as a way to express myself, and more as a way to please others, the almighty "audience" (who might not even exist). And that was bad. I kept circling around basic topics, known facts, and nothing controversial, because those were low-handing fruits. I was neither productive, nor proud of my individuality. And having an agent nagging about what I should and should not write (though more like must, if I wanted to be published), was annoying as fuck. So I just dropped.

I only want to write because I am a human being with thoughts and feelings, and because no one can stand me being emo and pouring my thoughts on them so randomly.

5. I found these people.

I still remember I found this group of people a hot, sweaty night of April, in Saigon, during one of the many trips between SGP and SGN I had. Though till now, the exact reason the group formed and clicked and jelled so well is still beyond my understanding, and therefore obviously beyond words. Though I am suspect it is Thanh's good and cheerful nature that makes everyone at ease!

We have already had a lounge as our hub of activity, gone for a few trips together, and given each other names that would embarrass the shit out of anyone in public. Out of sudden, I realize that "Damn, these weird crazy people are my greatest dose of anti depression now". Dũng, in case you fucker are reading this, you are important too, but you're not a new dot in 2015. You are a rat's ass dot since junior high!

---

If there is anything 2015 has taught me well, it is that planning is necessary and it gives you a long-term vision how your life plays out, but it is also important to listen to yourself and the omens of life. Often when we are too focused on the goal (and plans really help that), we underestimate our natural ability to navigate complex system and improvise, forget the original reason of all these plans, and lose track of the journey.

My plan to make a smartphone was dropped like a sack of potatoes, but I got to go through a learning period of hardware design and manufacturing that wasn't available elsewhere.

My plan to organize BarCamp Saigon 2015 didn't turn into a disaster, though lot of unexpected happened. We still haven't missed a single year of BarCamp.

My plan to take on HCMC 21K run scattered as I failed to follow a rhythm and both over stressed my muscles and gave them too much rest. I crawled back to the finish line with cramped legs, but finish the race I did.

I can go on like this for a while, but it is just rant, you get the idea.

At the end of the day, it doesn't matter what happened or whether a plan went well, but that I don't give up on becoming a more okay person and, this is new, who I go through stuff with.

Hello 2016!!!

2016 started off with great photos!!!

Run like you stole it!

How my first half marathon went, broken down to km (don’t mind Nike+, it was just giving me ego boost with 30K)

0 km: Whoa there must be a few thousands of people here. And we do smell like thousands of people. EH! Second wave? I have to wait for another 5 minutes? ... It must have been more than 5 minutes now, any chance they forgot there was a second wave? Oh okay, here we go!!!

1 km: Alright, easy, I got this, keep my speed steady and don’t get caught up in a race this early. Did I just see a girl in seifuku?

2 km: Hoà? Where are you Hoà?

3 km: Cool water! Okay lets have a gulp or two and splash some on the head. Oops, too much. I think I missed the last trash bin, I have to keep this half empty bottle now? I will just take a paper cup next time.

4 km: My legs are still alright. Looks like the muscle tension fear is a hoax!

5 km: So that’s the track for 5K runners? Dũng gonna be fine.

6 km: Another water station, time to drop this bottle and ops for a paper cup. Lets splash some over my back, the last one was very refreshing. Oh shit, this isn’t water, it’s electrolyte drink. Oh crap. Oh crap. Bad mistake.

7 km: Lets have some high fives with volunteers, everyone loves high fives. Hope my hands aren’t too sweaty.

8 km: What the fuck? The professionals are on their way back already?! Keep calm, keep calm, gotta reserve stamina, I haven’t even reached the bridge yet.

9 km: Better not mix up water and electrolyte drink this time. Whoa, splashes of cool water feel really good. I wouldn’t mind an ice bucket challenge now.

10 km: Yay half way! I am still in good shape. Lets climb up this bridge!!! Wait, holy fuck, the bridge towers are miles away, I can’t even get a good look at it from here, are we fucking running over there?

11 km: My legs are not alright! The tendons over my heels are tensed. I really should have taken more uphill running on the treadmill. And yes, the girl in seifuku is real, she even has her red backpack on, and she is dashing like crazy on her way back!

12 km: Is there no end to this god damn bridge? This slope must be 15%. Gee, I am as slow as walking now. And why don’t they mark the distance on the bridge?

13 km: Yes! Turn around! Yes! There is just downhill now.

14 km: Try. Not. To. Walk. Talking to myself it is easier to keep running than multiple small stops. Murakami said he never walked in a running competition. Be like Murakami.

15 km: I don’t think Murakami said anything about stopping for a break. Have to stop now. Why I am getting sleepy?

16 km: The water station should be just around the corner right? I believe I can run till there. Nope, nope. Too tired. Have to stop now.

17 km: I can’t run anymore. My mouth tastes weird. I want something sweet, like, diabetes-guaranteed sweet. Did Hoà say something about choc last night? Where the fuck is Hoà? Screw Murakami, I am walking!

18 km: I think the can of red bull I just got from a street vendor is kicking in. My lung has stopped burning, and I can feel my legs again. Yosh, about time, all these ladies are passing me!

19 km: Arrggg my right calf cramped. Freak it is twisting and moving on its own. Oh shit, not you too left thigh! Oh hey random runner, thanks for your spray.

20 km: I bet that if I stopped pulling either of my feet, my legs would tangle each other like two pythons. Come on guys, don’t you dare failing me!

21 km: Holy guacamole! Stay cool. Don’t scream like a teenage girl. Smile for the cameras. I crossed the finish line!!!

All jokes aside, I did have to walk more a bit more than a km, but other than that, I got to the finish line in 2:34:47, around what I expected, and really enjoyed the race. Such a blast.

Phú Mỹ bridge is a beast. I felt really good entering the bridge, and really bad exiting it. I understood there would be quite some uphill, but didn't know such a tremendous impact on my stamina. And seriously, some runners came back on ambulance.

Exiting the bridge, my energy went on a spiral downward. Despite all will power, I simply couldn't move my legs, and had to walk between km 17 - 18. Not promoting a dose of energy drink in a race, but practically, the decision to get a can of red bull saved my ass there. It prompted to me that I was barely qualified for a half marathon and training alone wouldn't be enough to better the result. I have to watch my diet, and build up more muscles for all the energy my body needs for long distance running. Tough!

And the last 3K with both legs cramped was really brutal. In normal condition, I would have freaked out seeing my muscles twisting and moving like they have their own brains. But I was desperate. Painful as it was, I am glad I neither stopped or walked on that part, it helped ease the guilty of walking earlier.

I am giving myself a break over the next couple of weeks (with Tet in the middle), and will be soon back on the track. I feel a strong need to hone my skills on 21km track before ever planning for anything higher. Gee, I have already given this running thing much more effort than I ever did for university entrance exam :)

What people did not tell you about microservices

The context

Over the last few months, I was commissioned to build an analytics system for logistics. Think of Google Analytics for shipment. The project consists of an engine to collect shipment information, analytics logics to normalize, combine and produce subsets of data through various stages, and a few different interfaces to represent the data to targeted users. That nature suggests a system with discrete components that should be planned and built in plug-and-play manner.

Having each component as a standalone, independent module allows me to choose the proper tool for the task. Java made up the core of the system where we need processing speed, stability, and off-the-shelf client support. Nodejs is used where either web socket or massive asynchronous tasks involves. And Python where we want a quick way to glue things together and iterate on ideas. The communication hub is built around Apache Kafka. Up till this point, all development environment is orchestrated with Docker and Docker Compose. We are planning to bring the Docker containers to production soon, once the monitoring is in place.

The project is considered strategic and therefore a new team has been built around it. The goal is to have this team dedicate to the project, grow with it and form the intellectual core in the long run. The project is run in 2-week sprints with committed deliveries at the end of each sprint. Though more often delivery schedule is always welcomed.

So that's the background, lets look at expensive lessons I have learned along the way.

1. Good onboarding is crucial

For many project, onboarding a new team member is an important activity but often neglected and instead, the newbie is given a stack of obsolete document. If she is lucky, she would be given the source code and have to figure out how to bring up the system.

The main reason for the neglect is because during onboarding period, the team's productivity slows down as key people are pulled away from their tasks to explain (parts of) the system to the newbie. And if your team follow a rigid delivery schedule like my team does, onboarding can be a stressful experience as more balls have to be juggled: maintain committed deliveries, and spend decent time for thorough explanation.

I have learned that with such many balls, go for good onboarding and negotiate delivery schedule if needed. While it is tempting to spend less time on the newbie (who, after all, is supposed to kick ass), there are too many reasons not to:

• There are just so many works a good onboarding requires. Over the course of few days, one has to go through the project overview, business values, and core competencies. he also needs to set up a development environment and interact with the system one would later help building. During which, he would need to get familiar with all components, what they do, and their scope of responsibility. And before writing any line of code, he must understand project standards, toolings, and enforced and unenforced conventions. It is a major, haunting task.
• Plus a microservice system is almost always complicated by its nature with all those moving components. A newbie, no matter how good he is, won't be productive till he has a good grab on both the big picture and the little piece of lego he will be working on. As a universal law of knowledge worker, a "just code the thing as spec'd" is a lose-lose situation.
• Given that, no matter how much night oil you burn, you aren't very likely to meet your deadline anyway, not in a decent way. Goofing around for a demo, or pull together an immature onboarding would only result in expensive technical debt and bad morale, both definitely bite you back at some point, might as well, the very next sprint.

A well-informed newbie would always be productive in no time and make up for the delayed delivery. Always.

2. You can't possibly write enough document.

Now, every time I start a new project, I talk to myself "This is more documentation than I have ever done before. Should be enough for even my mother to understand." and it's never enough.

But that is even more true with a system with much dynamic as a microservies system. Seriously, the number of documentation I write this time is just ridiculous.

• On highest level, there is architecture and infrastructure documentation. One focuses on the logical components, while the other on the actual machines things run on.
• Integration across services are captured in flow diagrams, which then come with message format (Kafka, MQ, remember?).
• Every component then has its own set up guide (besides one whole-system set up guide) and those whose service is used by others have their interactive API doc.
• Specifically complicated components get their own flow diagram (though the rest should be able to interact with them in black box manner.

But the reality is that not only the documentation alone isn't enough to understand the system (a process which, to be fair, can be overflowed with data and decent time for self exploration should be given), I couldn't keep up with the amount of doc, the amount of work is justifiable for a full-time technical writer.

That situation calls for a change in practice. While major high-level document (like the architecture and infrastructure) should continue to drive the implementation, finer detailed documentations that involve multiple modification each sprint should be derived from the code itself. The goal is to have the code serve as one reliable source of truth and everything else to make sense of it is generated when needed. For example, a business analyst needs a database schema? Generate it from the database schema. A developer needs to know how to use the latest API? Comment block is extracted, combined with a simple interactive form to try out, and you have API doc.

3. Automate your environment setup

Now that you have automated document generation, it is a good time to automate environment setup too, starting from dev and all the way to production. My project is using Docker, but this can be applied to all tools, like Chef, Puppet, or Ansible.

At the beginning of the project, with a handful of services, it is simple enough to announce a change in service Docker with a group chat message, or a poke. You can also easily update the new setup requirements to the service's README file. But by the time the project gets a dozen of services, if a developer has to either keep a eye on chat messages for setup changes, or iterate that many README files, he would commit a suicide.

Be it bash script or python script, or whatever else, automate environment setup as much as you can. A few things you can consider to begin with are:

• auto update code of all services and restart Docker containers
• auto remove obsolete Docker images and containers (you get lot of these when building new services)
• auto update configuration files based on last edited/updated timestamp.

I can't stress how relieving it was for my team to have a smooth, reliable automation. No more waste of half an hour every morning trying to figure out why what worked yesterday before stops today. Seamless automation is worth all effort.

4. Standardize everything

The upside of microservices is that developers have lot of freedom to do their work, as long as service contract is respected. The downside of microservices is that developers have too much freedom in their work.

Aside from the choices of technology, a few things I kept running into

• Mix used of `CMD` and `ENTRYPOINT` in Docker (hint, they aren't supposed to compete, but complement each other)
• Inconsistent log formats. The log level fluctuated from standard output to log rotator, from random debug log to nginx-style. The Wild West can only so crazy.
• Bash scripts with different names, in different directories, but all do the same thing. `run.sh`, `start.sh`, in a `scripts/` directory, or at root level. And so on.

Independently, these are just a bunch of little harmless variations of some right things to do to keep a software system maintainable. But together, they make the life of whoever whose job involves jumping in and out services miserable (i.e. everyone but developers). Now of course monolithic codebases have such issue too, but it can be ruled out easily even without a quality police because everyone looks at such thorn all the time. I didn't realize how different personal preferences can be (but hey, after all, we are a new team).

Well, eventually the variations pissed me off so much I had the development paused so we could sync up the conventions and made an oath to keep them. For the watch!

5. Collective ownership doesn't work

By the book, Agile management encourages every to share responsibilities for code quality, anyone can make necessary changes anywhere and everyone is expected to fix problems they find.

Whereas the share responsibilities part still rings every bits and bytes and we encourage that with tight feedback circle, rigid definition of done, and tests as safety net, the two later parts go from bad to really bad. In a microservices system that employs various technologies, anyone can still make change anywhere, but it is probably for one's best interest that he doesn't. Lets take a quick look at the technologies I mentioned upfront: nodejs is asynchronous by nature and one must forgo traditional threading model (I got many eyebrows saying that nodejs runs on only one big ass thread, but it does, people!). Python is a dynamically-typed language  and Java is a statically-typed one. Each language calls for a different mindset. Makes change to a codebase one does not understand the philosophy behind and technical debt is probably the best outcome.

And fix problems they find? Typical management's hopeless optimism.

A more practical model is to let a developer takes a complete ownership over his service(s). Others might chip in for help, but within the service's scope, he is the technical lead and in charge of maintain quality standard, code convention, and whatever else he considers important. That might sound too rigid for an Agile team, and it might be true. But screw the Agile label, I need reliability. With no monitor system in place, and a dozen services running, each has its own set of runtime problems, I need each to know one thing inside out, not a bit of everything.

6. Keep an eye on everything that moves

Ok, saying that people didn't tell me about the need of monitoring a microservices system is an overstatement. The topic of monitoring appears one way or another in books and articles I have read about microservices. What wasn't seemed to be stressed enough is the sheer amount of work required to get monitoring up to the level you can be confident about the system without fearing something would fall apart the moment you turn away.

Right on the development environment, having all the log gathered in one place that you can later on `tail -f *.log` is a huge time saver. Depends on the 3rd party libraries that you use, your log might be populate with mumbo jumbo of bullshit. Take effort to filter those out of your log, the investment is paid back every time you inspect an inter-service bug. If possible, slice your log into 3 groups:

• Activity log (or debug log) to monitor the flow of data between services
• Error log, so you can find the most critical thing right away when something goes awry
• Third-party libraries log, in case you want to play safe

Once the system is in production, the focus is less on the flow of data, but more on performance and health checking. You want to have the ability to know how many nodes has a request had to travel through before a user can see anything, how much time did it spend at each node, and set up programming to trigger escalating actions when certain thresholds are passed. For this purpose, we are using ELK stack where Logstash crawls log from distributed servers, feeds ElasticSearch, and put a presentation layer of Kibana on top. Formatting log and organize Kibana report is then an ever going job.

While ELK would tell you about the performance of a system, it shows little about the health, e.g. a service can be serving request at less than 100ms, but its RAM is  whooping 90% up and CPU utilization is always above 70%. That introduces a different set of system monitor tools, like AWS CloudWatch, Navigos, or NewRelic.

From time to time, by doing house keeping work such as migrating service into a bigger machine, scale out a service to a few instances, or deploying new service instances while iteratively shut down old ones to achieve zero down time, you would get really tired of constantly checking whether a service is up, and still at the same IP address or not. Well, that is service discovery like Consul or etcd you are longing for.

My point here is to illustrate two points. First, it is very crucial to keep a close eye on the system as a whole and optimize the flow of data. Second, it is very tempting to apply all the bells and whistles, and surrounded yourselves in dashboards and get distracted from the only thing that matters: the system itself.

7. Whatever you estimate, multiply by 3

There is probably no other project where my estimation has been off by 100% and, unfortunately, nothing intrinsically stops it from going to 150% or even 200%. Most original estimation could only barely cover coding and unit testing. While equal amount of time is required for gluing a service to others (integration test), and environment setup (Docker, monitoring). The experience learned from this is absolutely valuable, but it is costing an arm and a leg.