RMIT Internship Q & A

• Should I choose an enterprise or a startup?
• Should I apply for a paid or unpaid internship? If it is a paid internship, what is a good offer?
• How can I find an internship abroad?

Question: Should I choose an enterprise or a startup?

Answer:

I choose this to be the first question to answer because I can tightly relate to it, back when I was preparing for my internship. Unfortunately, with all the small talks and experience I have collected, this is the type of question you have to look inside yourself to find an answer.

An enterprise is an established business. It has a stable business model, long-term business partners and sufficient resources to overcome market glitches. Being able to survive in the ferocious industry is an indicator that an enterprise knows a lot of things and has done a number of things right. Derived from that experience are a number of methodologies, policies and procedures that cover at the very least core-business activities. Joining an enterprise is a secured career option. It is not very likely that the company will be out of business in the matter of months, usually that happens over the course of couple of years. Look at Nokia or BlackBarry for example. You will be benefited from the experience of the company through various forms, e.g. training, documentation and mentorship. Your working environment is likely to be more luxury than that of your startup colleagues (though in Vietnam and over the world, the gap is narrowing down). A standard enterprise would provide you with hardware devices, free lunch, medical care, social insurance coverage, Tet bonus and everything else that is required by the labour law.

However, enterprises are no utopia. Due to its size, an enterprise is structured in ways that make most of its employees replaceable, even at manager level. A set of lego is my favorite metaphor for this. The number of employees in an enterprise might also be an issue. On one hand, your existence will be so insignificant that no one would know if you were eaten by a lion (good-old IBM joke). On the other hand, you might get crushed under peer pressure. Unilever is very notorious for this, every year it recruits hundreds of management trainees and lets them fight till death. Every enterprise would have a handful of long-term partners, which provide a stable source of income. But these long-lasting partnerships are placed on top of legacy systems, developed, who know, 20-30 years ago. For example, Cobol is a very ancient programming language, which was responsible for the infamous Y2K. By the time I took my internship in 2010, CSC was still known for providing technical support to such systems.

Compared to enterprises, startups are on the other extreme. Startups are on hype all over the world. Everyone talks about it. Startups attracts a lot of talents because it gives people a chance to visualize their own universe where imagination is the only limits. For anything you do, there might be a chance that you are the first one ever set out to do it, solving a unique problem for mankind. Oh Jesus Chris, the feeling that you are pushing the edge of the world, that is good. For experienced developers, startups hit a sweet spot when bureaucratic policies are replaced by the freedom to establish new standards. If you noticed, most of the new things in software industry came from startups, e.g Agile for methodology or RoR for development. Last but not least startups have been making countless founders and core-members millionaires. When Instagram was acquired with ONE-BILLION USD, it had around 12 members, effectively everyone of them can have their own yacht and island now.

But startup is a risky path. For every successful startup, 99 others fails. The working environment and benefit package in most of startups are every limited. Many founders and startup employees live on minimum wage or below until the business takes off, or fail miserably. Oh hey, but too much funding in an early stage can kill a startup too (Color of Bill Nguyen, or Skunkworks right in Vietnam are bold examples). Working in a startup is very stressful. It is a common belief that if you are working less than 100h/week, your startup is not going to success and investors won’t be convinced. While it sure feels good pushing the edge of the world, the same situation could be described as crawling in the dark if you failed, lost your direction and had no idea where  you were heading to.

There are pros and cons, and learning opportunities in both enterprises and startups. And always remember that this is just an internship. You can fail, you can make a bad choice and you should. What is the worst thing that can happen? You wasted 4 months of your life in a crappy company. But that experience, either sweet or leaves a bad taste, is priceless. He who never fails never succeeds. “You can’t connect the dots looking forward; you can only connect them looking backwards. So you have to trust that the dots will somehow connect in your future. You have to trust in something — your gut, destiny, life, karma, whatever. This approach has never let me down, and it has made all the difference in my life.” Steve Jobs, 2005.

Question: Should I apply for a paid or unpaid internship? If it is a paid internship, what is a good offer?

Answer:

An offer in a job, not only an internship, is the mutual commitment between employer and employee. By providing benefit packages and working environment stated in the offer, an employer has the right to expect its employee to legally hold responsibility for his work. Being paid in an internship is not only an indicator that you are valuable to your employer but also a call for your professionalism in being responsible for your impact to the organization performance.

In RMIT’s official internship agreement, it is stated that monthly allowance is optional. This is a move of the university to lower the entry barrier for some students who have not yet been excellent in their profession (the word “monthly allowance” itself is also to lower the barrier as it indicates that a legal contract is not required). Based on my pure assumption, students who are concerned about paid/unpaid internship are ones that haven’t got any previous experience in the industry. And by considering unpaid internship, students are trying to reduce the pressure they are about to face during the internship. That seems to be a reasonable option, but if you are serious about your career, I suggest not to. An unpaid position is an open message from the organization that your work, not matter how good or bad, is not going to matter, it doesn’t provide value to the organization, it is junk. You have spent a little fortune for your education, you better work on something matters. And this is kinda just for fun but receiving a paycheck after all of the hard work is part of the internship experience too.

Talking about offer, it is important to acknowledge that salary is a small part of an offer. For an internship, that portion is especially small. Ultimately  what an organization has to provide is its working environment, corporation culture and other experience coming along. A few items to notice when considering offers would include

• The culture the organization is embracing. Refer to the first question to know which type of company fits your taste.
• The people you are working with. Sometimes, it doesn’t matter what you do, it is who you do it with. An offer receives its incremental value if it comes from an organization where employees are top-notches in particular fields in the industry. Working with these gurus is an priceless experience (which might not always be positive, I know a handful of gurus who are truly sore losers). And the fact that they are gathering is one place is kinda an indicator on its own already.
• The type of projects you are going to work with. Is that some kind of project that you can freely write about in your school report or is it something you have to sign Non Disclosure Agreement (whether or not you pass the “course” depends on the quality of your reports, it is an important piece)? Is that the type of project you want to work on in the next few years? How do those projects align with your personal development?
• What are the growth opportunities the company provides? Do them align with your goals in the next few years?
• And finally salary, aka monthly allowance. As stated earlier, I suggest taking a paid internship rather than an unpaid one. The salary should be sufficient for you to live on it independently. Given the current state of the market, a salary of $300-700 is considered well-paid. You can refer to this survey to get a sense of what is going on in the industry.
  • Don’t jump into the conclusion that a salary is low or high with just the number. As an intern, there is a chance that you won’t be productive in the first few months (read, your entire internship) and that your salary together with other benefits you are receiving are nothing but cost to the company. Investing in an intern is usually a long-term investment, and no fool places all eggs in one basket. Carefully consider what you are receiving in big picture, not just a paycheck.
  • Many companies pay its interns the same amount as entry-level employee. It might mean that the company has a deep pocket. But it is more likely that the company doesn’t provide any special training/treatment to interns and technically you *are* a normal employee under the eyes of your direct boss (monetary award is mutually commitment between two parties, remember?). This does not necessarily mean good or bad, just something as a green horn you should look out for.

Question: How can I find an internship abroad?

Answer: 

Hard work and a lot of luck.

Firstly you should know that you are competing with a lot of native students whose education is tailored for the need of the local market. If your language is not fluent, your CV is not shiny, you have little to no working experience and all you have is a cold contact, your chance to win this competition is extremely slim. The interview processes of different companies are quite identical and if you have been attending WPP workshop, you should know about the art of answering interview questions better that I do now  So in this answer, I will only focus on the parts before the interview.

Spend time on it, decently. There is no place for luck in this competition. It is not luck that brings you a reference to an overseas internship, it is years of hard work for a nice transcript and reputation among fellow students. It is not luck that your CV passes the screening step, a decent and well-written CV does. And little can luck help to boost your English for an interview, on phone. Furthermore the whole visa a truly a pain in butt. Internship visa is especially troublesome. In many countries, the only visa types that are available are Resident, Work and Visitor visa. Work visa are for professional only and there is a quota on how many expats a company can employ. Visitor visa usually lasts for only 1 month for Vietnamese citizen and visa bearer can’t be employed (legally). Due to its nature, this is something you need to discuss and coordinate closely with your employer. It can easily eat up 2 months of your time, and if it mixes up with exam period, ain’t gonna be nice.

Choose a company. All companies expect long-term value  from investments they make. A fact you should know is that in RMIT Melbourne, there are many Vietnamese students who can’t find an internship over there and have to reluctantly travel back to Vietnam for an employment opportunity. The reason is that companies are concerned about the work permit for the student once his/her education in Australia is over. Compared to native students who speak a faultless language and have no problem with visa, investing in a Vietnamese student is too risky. Same logic applies when you are seeking for an internship overseas.

However, there are two types of companies where being a Vietnamese engineer is an advantage. Firstly, those are companies looking at Vietnam as a potential business partner. The purposes can be various, ranging from extending market to talent acquisition. These companies are in great need to understand about from multiple aspects and a Vietnamese employee is an useful resource. The second type are young (relatively) technology giants, like Google, Facebook or Twitter. These companies succeeded in the new waves of technology and greatly appreciate the power of diversity. They are always try to balance the demographics of employees. When you look into companies of this type, the company size matters. Acquiring visa that permits work in most countries is are troublesome and time consuming process. Unless the company is big enough to have its own legal department, most will pass the burden.

Get a reference. From my experience in recruitment, most companies value hiring opportunities coming from its internal network, aka word-of-mouth reference. So once you have a good idea which company you want to apply to, you need to approach the company from different direction, and as soon as possible. You can approach the company HR’s via social network such as LinkedIn. If your LinkedIn profile is attractive, you will receive many offers, good and bad ones. This might not be common in Vietnam (yet), but quite popular in neighbor countries, like Singapore. Even more effective, you approach the company’s employee through open source project. I really love this approach because not only you gain a valuable internal reference, your application profile can also be pre-filtered. Similarly approach the group of Vietnamese people working in the company  . Lastly approach the company through developer group, competition and event held by the company. There is actually one last approach which is to become excellent in the field the company is starving for talents, but that is quite hard for students so I don’t really want to stress on.

I assume I should talk about closing an offer as well. But wait, if you are a bright student, you have been studying hard and now you are getting an overseas internship, it is no time for QnA, it is time for party \m/ So perhaps another time, or you can always email me 

Cogini Skill Sets - Perceive and Control your own career development.

Ever since the early day, our vision about the company has been clear, to build a fun place to work. Moving towards that direction, countless of remarkable improvements have been made, with joined effort from everyone in the company, from the management board to new interns. The two most significant milestones in our journey of building a fun place to work are our recruitment process and our current office. Believing that hiring is the most important decision a manager can make, we have always been religious about recruitment. For many times, we have said "no" to talented people who could bring immediate effect to the work place because we didn't think it was a good culture fit and would eventually be a long-term harm. The new office on District 7 and all the nice benefits are high fixed costs for us that to a certain extend makes revenue flow management more troublesome. But that is a trade off we are willing to make as we can see how much it has been improving the professional and personal lives of us. Today, I would like to introduce the next big thing at Cogini, which, if done right, would bring employee development to the next level and provide a substantial long-term competitive advantage over everyone else. Cogini Skill Sets - Perceive and Control your own career development.

For a long time, our weakness in seniority has made employee development resort to self-study. And that isn't a smooth trip. The number of new technologies is blooming. Practical application of certain skills or technologies can be vague without industry insight. Without a roadmap, self development can be ineffective at best and crawling in the dark at worst. The idea of Cogini Skill Sets (CSS) is to provide a skill sets system that covers multiple roles (e.g developer, designer, system admin, etc) across multiple levels from interns, people in probation to people who have spent their entire career here. CSS serves as a checklist for employees to decide which skill sets to attain and control their career development. The progress one makes in CSS also contributes to his/her performance reviews (which are done once a year or twice). This is a exciting move that we expect to bring perceived control and progress to everyone's career.

Looking at the bigger picture, CSS is the first stepping stone in building a platform necessary for Cogini to be a long-term enduring and growing business. With the CSS roadmap laid out, we can focus on building a pipeline of people in every department with varying levels of skills and experience, ranging from entry level all the way up to senior and leadership positions. The vision is that for almost all of our hires from entry level, the company provides all the training and mentorship necessary so that any employee has the opportunity to become a senior leader within 5 years. 

I would like to make the roadmap a joined effort of everyone. It is acknowledged that it is hard to know what we need to know, to learn and to research to make us more valuable to the company. But that is where our diversity kicks in. There will be certain things about which you have a better idea than the person sitting next to you and the other way around. The Vietnam branch of Cogini has been in operation in two years and a half, which means at the minimum we have the experience to lay out a 2-year-long roadmap for a newbie. The number will be even more significant if everyone contributes his/her experience and ideas. There is always something we can learn from each other, so CSS will be beneficial to not only newbies but also experienced employees.

In the following week, we are collecting people's ideas about virtually everything you think an employee would need to attain to become successful at Cogini. Each idea should be written in a condensed noun phrase on one of the stickies you can find everywhere in the office and submit to the "CSS Box" on each floor. For those who cannot make it to the office, feel free to email me your ideas.

Ask yourself:

• How do you grow personally? How do you grow professionally? What is your vision for where you want to go?
• What skills that make you a better person today than you were yesterday?
• What skills you have learned / want to acquire to differentiate yourself from everyone else?
• What skills you have learned / want to acquire that would potentially to make work more fun?
• What skills that you have pushed yourself out of your comfort zone to attain?
• What skills you think would get the company as a whole grow?
• How can you make people enjoy working with you?
• How can you do what you are doing more efficiently? How can your team become more efficient? How can the company as a whole become more efficient?
• What make you passionate about the company, about your work?
• And more...

I am very excited about this new change we are introducing. We have always been keeping a flat organization in mind so that everyone's ideas are treated equally importantly. Now we are making it to the next level where together we are shaping up the type of employees we are raising, now and coming generations. In many ways, you are having the chance to leave your personal trait on the direction Cogini are walking in the next few years.

Software Estimation - Techniques and process

Tl;dr

In software estimation, counting and computing are more reliable than judgement alone. Try to start your estimation by counting something closely related to the project size, available early, consistent between projects and statistically meaningful. To make sure as few miscounts would be done as possible, check the work against some sort of Work Breakdown Structure. Best Case and Worst Case make a good start for judgement, but there is a hidden problem of adding them up. The Mostly Like Case and Expected Case formular were introduced to help the situation. Estimation can also be made by comparing to similar projects in the past.
Not everything in a software project can be counted (easily), a family of techniques known as proxy-based estimation helps overcome this challenge. The techniques rely massively on the ability to process raw historical data of an organization.
Software estimation can also take advantage of wisdom of the crowds by creating the right environment to support this idea.

Count, Compute, Judge

People have the tendency to mistake estimation with judgement (or, in another word, guess). However, researchers have found that judgement alone is the most inaccurate form of estimation. Estimation experts are more likely to perform better than newbies with judgement alone, but in fact, that result comes from a wide range of historical data, experience and painful stories the experts have been through in their career. Was the estimation made in an inexperienced field, no different in performance was observed.

Under the light of statistical science, counting and computing are proven to be more reliable. You should always count related things first, then compute what you can't count and finalize the estimation with calibration data. Only use judgement as the last resort.

There are many things you can count in a software project. Per the cone of uncertainty, the later in the project, the finer the level of granularity you can count at.

In order to avoid being paralyzed by choices, there are a few rules of thumb we can use to decide what to count.

• As size is the strongest influence in a software project, count something that closely reflects the project size
• Thing that can be count early in the project is better than something we need to wait till later
• To get benefits from historical data (like pro estimators), we better count something consistent between projects
• Count something that woud result in a relatively large number (like 20 or more) so that we can take advantage of the law of large numbers. The law states that the erros on the high side and errors on the low side cancel each other out to some degree.

Decomposition by Work Breakdown Structure

In the last post, we have already listed omitted activities as one of the major sources of estimation error. Omitted activities consists of forgotten features and forgotten tasks. Forgotten features can be reduced by thorough requirement engineering and experience. In general, this requires much practice and retrospection to improve. Forgotten tasks, on the other hand, can be improved dramatically by checking our work against an activity-based work breakdown structure.

Justify Probability Statement

Even though, judgement is very subjective, we cannot avoid that. After all the counting and computing, judgement is needed for the actual numbers. Educated guess is better than blind guess, and here is how can we make one. We have learned that estimate is a probability statement, we should stop using single-point number as reliable estimate. Best Case and Worst Case make a good start, it is more likely to catch actual hours somewhere in the middle and makes us more comfortable.

But there is a problem with adding up best and worst cases. Lets say that each of the individual Best Case estimates is 25% likely, meaning that you have only a 25% chance of doing as well or better than the estimate. The odds of delivering any individual task according to a Best Case estimate are not great: only 1 in 4 (25%). But the odds of delivering all the tasks are vanishingly small. To deliver both the first task and the second task on time, you have to beat 1 in 4 odds for the first task and 1 in 4 odds for the second task. Statistically, those odds are multiplied together, so the odds of completing both tasks on time is only 1 in 16. To complete all 10 tasks on time you have to multiply the 1/4s 10 times, which gives you odds of only about 1 in 1,000,000, or 0.000095%. The odds of 1 in 4 might not seem so bad at the individual task level, but the combined odds kill software schedules. The statistics of combining a set of Worst Case estimates work similarly. (McConnell, 2006)

Due to that, we introduce Most Likely Case with the hope that the sum will be closer to the actual. Still, developers' "most likely" estimates tend to be optimistic. A technique called the Program Evaluation and Review Technique allows us to calculate the expected case. The formula is derived from statistical studies.

Expected Case = [Best Case + (4 x Most Likely Case) + Worst Case] / 6

Or if the organization has a history of consistent optimism

Expected Case = [Best Case + (3 x Most Likely Case) + (2 x Worst Case)] / 6

Estimating by Analogy

The basic idea is to create new estimates by comparing the new project to a similar project in the past. Again, old rule applies, count first, then compute and use judgement

Break similar previous project into pieces using requirements and WBS	Count
Compare the size of new project and the old one piece by piece	Judge
Build up the estimate for the new project's size as a percentage of the old project's size	Compute
Create an effort estimate based on the size of the new project compared to the size of the previous project	Compute
Calibrate the result	Judge

There are areas where analogy doesn't work, like business rule. But still

"One contrast between the estimate created using analogy + decomposition, and un-decomposed approach is that in the later uncertainty in one area can spread to other areas." (McConnell, 2006)

Proxy-based Estimate

Not all activities in software development process result in code, nor everything can be counted, for instance how many test cases a feature needs, how many defects should be expected, how many pages of user documentation would be written. A family of estimation techniques known as proxy-based techniques helps to overcome these challenges. Find another metric that is correlated to what we want to estimate ultimately. Once the proxy is found, we estimate or count the number of proxy items and then use a calculation based on historical data to convert from the proxy count to the estimate we really want. The basic idea behind this kind of technique is that developers cannot estimate exactly accurately, but can estimate relatively accurately pretty well. Which means it is hard to tell if a task takes 4 or 6 hours, but relatively easy to state a task is two times harder than another. By making relative comparison to the past, we tell the future.

Where can we find the proxy? There are three main sources: industry average data, organization historical data and project specific data, in the order of increasing accuracy.

• The data of different organizations within the same industry differentiate variously, by a factor of 10. And if we use the average productivity for our industry, we won't be accounting for the possibility that our organization might be at the top end of the productivity range or at the bottom. (McConnell, 2006)
• The majority of projects in an organization are often similar in size and also developed under similar organizational influences. The estimates hence will not be subject to much error.
• Project specific data is useful in the same way historical data is. Further more using data from the project itself will account for the influences that are unique to that specific project. The sooner on a project we can begin basing our estimates on data from the project itself, the sooner our estimates will become truly accurate.

A few popular approaches in this family are Story Points, Fuzzy Logic, T-shirt Sizing and Standard Components. Due to the lack of processing historical data, we do not use proxy-based estimate very often. Lets take a quick look anyway.

Story Points is very well-known across Scrum teams. It is used to measure the relative effort for a story, in numeric units. Story Points only starts to become useful after the first few iteration, when the team can count the number of story points it delivered and compute its velocity. You can easily find articles about Story Points on the Internet.

Fuzzy Logic is exactly similar to Story Points but instead of using numeric measurements, people classify size as Very Small, Small, Medium, Large, and Very Large. The favorite argument point when it comes to "Fuzz Logic vs Story Points" is that the use of numeric scale implies that you can perform numeric operations on the numbers, multiplication, addition, subtraction, and so on. But if that relationship isn't valid, a 12-point story doesn't require 4 times the effort a 3-point story needs, then the number 12 isn't any more valid than the Large and Very Large sizes.

T-shirt Sizing is a derivation of Fuzzy Logic where business value is brought to the table. Sales and marketing staff will say, "How can I know whether I want that feature if I don't know how much it costs?" And a good estimator will say, "I can't tell you what it will cost until we've done more detailed requirements work." It would appear that the two groups are at an impasse. By representing both business value and development cost, nontechnical stakeholders can make decision based on net business value. (Numeric values are for illustration purpose)

	Development Cost
Business Value	Extra Large	Large	Medium	Small
Extra Large	0	4	6	7
Large	-4	0	2	3
Medium	-6	-2	0	1
Small	-7	-3	-1	0

Standard Components is the most straight forward one. If we have developed a number of program that are architecturally similar to each other and possess a certain amount of historical data, we can estimate the number of standard components we have in the new program, and compute the size of the new program based on past sizes.

When proxy-based estimate is not effective.

When using proxy-based estimate, it's important to remember the Law of Large Numbers, that the rolled-up number has a validity that the underlying numbers do not have. If you don't have a big enough number of items to estimate, the statistics of this approach won't work properly, and you should look for another method.

Collect historical data

The weather today won't always be the same as it was yesterday, but it's more likely to be like yesterday's weather than like anything else (Beck and Fowler 2001)

The most important reason to use historical data from our own organization is that it improves estimation accuracy. Historical data takes into account organizational influences. Estimating these influences one by one is difficult and error-prone. Historical data adjusts for all these influence even though identifying the specifics is hard. The data also helps us avoid subjectivity and unfounded optimism. There is an effect known as The Second Project Effect where a lot of assumptions are made from what you learned from the last project, "We know the business logic better this time", "There was a lot of turnover last time, we won't have it this time (?!)" or "We will do a better job at requirement management". With historical data, we use a simple assumption that the next project will go about the same as the last project did. Because in fact productivity is an organizational attribute that cannot easily be varied from project to project (Putnam and Myers 1992).

Group Review

This is actually not a technique at all, but rather a set of rules of thumb to conduct an estimation review in group. The goal of doing the review in group is to obtain the wisdom of the crowd, so before looking into the set of rules, lets talk about this effect.

Wisdom of the crowd is the belief that the aggregation of information in groups, resulting in decisions that are often better than could have been made by any single member of the group. Not all crowds (groups) are wise. Consider, for example, mobs or crazed investors in a stock market bubble. These key criteria separate wise crowds from irrational ones:

Criteria	Description
Diversity in opinion	Each person should have private information even if it's just an eccentric interpretation of the known facts.
Independence	People's opinions aren't determined by the opinions of those around them.
Decentralization	People are able to specialize and draw on local knowledge.
Aggregation	Some mechanism exists for turning private judgments into a collective decision.

http://en.wikipedia.org/wiki/The_Wisdom_of_Crowds

When the decision making environment is not set up to accept the crowd, is that the benefits of individual judgments and private information are lost and that the crowd can only do as well as its smartest member, rather than perform better

Extreme	Description
Homogeneity	The need for diversity within a crowd to ensure enough variance in approach, thought process, and private information is stressed
Centralization	The hierarchical management bureaucracy limits the advantage of the wisdom of low-level engineers.
Division	Information held by one subdivision was not accessible by another.
Imitation	Where choices are visible and made in sequence, an "information cascade"[5] can form in which only the first few decision makers gain.
Emotionality	Emotional factors, such as a feeling of belonging, can lead to peer pressure, herd instinct, and in extreme cases collective hysteria.

http://en.wikipedia.org/wiki/The_Wisdom_of_Crowds

Based on that study, Steve McConnell suggested this set of rules in group review.

• Have each team member estimate pieces of the project individually, and then meet to compare your estimates Discuss differences in the estimates enough to understand the sources of the differences. Work until you reach consensus on high and low ends of estimation ranges.
• Don't just average your estimates and accept that You can compute the average, but you need to discuss the differences among individual results. Do not just take the calculated average automatically. Convergence among the estimates tells you that you probably have a good estimate. Spread tells you that there are probably factors you have overlooked and need to understand better.
• Arrive at a consensus estimate that the whole group accepts If you reach an impasse, you can't vote. You must discuss differences and obtain agreement from all group members.

---

Beck, Kent, and Martin Fowler, 2001. Planning Extreme Programming, Boston, MA: Addison-Wesley.
Putnam, Lawrence H., and Ware Myers, 1992. Measures for Excellence: Reliable Software On Time, Within Budget, Englewood Cliffs, NJ: Yourdon Press.
Steve McConnell. (2006). Calibration and historical data. In: Software Estimation - Demystifying the black art. Redmond: Microsoft Press.
Steve McConnell. (2006). Estimation by Analogy. In: Software Estimation - Demystifying the black art. Redmond: Microsoft Press.
Steve McConnell. (2006). Individual expert judgement. In: Software Estimation - Demystifying the black art. Redmond: Microsoft Press.

Interview at Vietnam's most successful internet company

There are tons of rumor about top tech corporations in Vietnam, but how is it actually like in the nutshell? I got tired of rumors already so I decided to take a look into that world by myself, I applied to the most successful Internet company in Vietnam.

The recruitment information on the Internet in general was very smooth. From the website I was able to learn about open vacants, benefits and perks, and the hiring process. However the sheer size of a 2000-people corporation shows signs of departments stepping on each other foot. The company favors project teams over feature teams. Each team is then responsible for its own vacants. This ends up in numerous job descriptions with the same title "Senior Software Developer" and different human-incomprehensible ID such as 12-WBM-1369 or 13-WTE-1504. Perhaps for insiders, these two codes are as different as e-commerce and social game development, but what I perceived was a confusion (and I didn't fully understand there were different IDs until after the interview).

The application required a CV and a cover letter, which are pretty standard. Mine were written 3 years ago and had never been actually used before (I got my first job via, uhm, word of mouth network). I quickly revised them and submitted the application around midnight. Late afternoon the next day (Thursday), I have got my interview scheduled and confirmed for 9AM next Monday. My phone was accidentally out of battery and I really appreciated that the HR girl patiently tried to reach me 4 times before I could pick up. Though it wasn't a job hooping season, I was still pleasantly surprised.

The night before the interview, I got unconsciously excited. Given that I had interviewed close to a hundred of applicants at the point of time, the excitement was hard to understand. In fact, I got too jumpy that barely could I sleep and that upset my stomach the next morning. 

Couldn't enjoy my breakfast much I came to the company 15 minutes early. When I arrived, the motorbike park and elevator were both crowded. I guess these people don't start a day at 10 like I do. I managed to find a place in the elevator. Standing in a box with other strangers, don't know what to say and what to do with your body was really awkward. I never enjoy sharing the elevator with strangers and I would have took the stairs if the appointment hadn't been on the 13th floor. As the elevator went up and people got in and out, I could see the company offices occupying not one but several floors in this building.

The half of the 13th floor that I was in seemed to be a big meeting area, it was packed with multiple glass-wall rooms named after major rivers over the world. Mekong was the first and Yangtze the last. The company logo and slogan printed on transparent decal were on every walls. I proceeded to meet the receptionist and grabbed a chair next to a few coffee tables in the hall, waiting for my interviewers to come. A big monitor was showing latest K-Pop hits meanwhile. Brochures and posters were scatted every where around the hall. They look really professional. Yet it reminded me of Valve's employee handbook. Of all organizational artifacts, an employee manual served as such a compelling form of global PR for the shift from an industrial biz model to a knowledge management/humanistic model. Brilliant awesomeness was still hiding.

When I was about to finish the third brochure, one of my interviewer, Ms. Tuyen, showed up and took me to a meeting room. It was a little room at the end of an lobby running across the hall. Walking down the dark lobby, I asked whether I would be interviewed in English. "Vietnamese", she replied. "So why the email was in English?". Many Vietnamese companies, most of them, practice this half-baked communication style, English for writing and Vietnamese for the rest. People ended up with some sort of Vietlish that I am allerged to ("Khang oi, can you help me", "Regards em nhe", etc.) Tuyen redirected my question elegantly, but I could tell that Vietnamese was the only language here. I wasn't surprised. In fact it reminded me of Summer, my former employee who couldn't blend into Cogini English-speaking culture.

There were places for 6 in the room. The first impression was the noise of the AC attached to the outer wall of the room. I don't think the noise was that bad, but when I am worried, every external signal seems to be amplified ten fold. The AC was no white noise generator and somehow I started to like the waiting hall better. There was instruction to use VoIP for conference in the room, but I didn't find any devices. The room has a good view over the main street and flower boxes on the pavement but I wasn't left alone in the room to explore the view. The interview happened right afterward. 

Tuyen was an HR staff, she needed someone else to test my technical skills. So she started the interview by talking about the potential project that I would join if hired. "Not launch yet", "Similar to what is happening out there". I couldn't help but think about an e-commerce system, which this corp hasn't succeeded yet, despite of its number of attempts every year. 

Before she finished her last sentence, two men entered the room. The older looked quite casual, actually his outfit looked a bit slipshod and his hair obviously needed some touch. The other guy had tan skin and looked quite sporty. He was very quiet during the interview, in fact, he didn't ask me a single question.

I was asked to give a short talk about myself. Ha! Just like Cogini! And yet I managed t deliver a below average introduction. Knowing the question and listening to countless answers don't make your answer better. The interviewers showed concerns when I expressed my interest in getting a masters degree and being a lecturer in the next couple of year. For a moment I saw my reflection from the other side of the table. 

We moved on through some technical questions, from data structure to database and web server engines, and scaling techniques. The questions had nothing to do with my CV. I didn't mention these skills in my CV and technically database was the only part I know thoroughly and put into my CV (rule of thumb, only put what you really know into the CV). My technical interviewer clearly was asking his concerns, not checking the skills I possessed. I couldn't help but wonder, how can these people detect a candidate that doesn't fit for the position he is applying for (due to the confusing job descriptions) but a true gem for another team right within the company.

The questions were randomly selected, I think, because he didn't have any note with him, just paged through my CV. That gave me an impression that he didn't read my CV before hand. Despite of his randomness, all questions and explanations focused on only one single thing: scalability. For every question, he wanted to know if I knew the implementation and algorithm beneath. Having its success root in online game distribution, the company has a vast number of loyal users. So focusing on scalability make a perfect sense to me. Though, my interviewer had the tendency to go a bit too extreme, I believe he knew what he was talking about. Anyway, not being a big fan of revert engineering, I must have passed 3 questions that related to things behind the curtain. However, after all the interview that I did, I developed a thick skin and defended myself through his questions quite well.

As prospect products and vision are important assets here, we weren't allowed to talk much about those. We went on to have some discussion about software development process and daily activity. The point of view of my interviewer was that processes (he didn't state which) are helpful for outsourcing companies as they indicate what are the steps and what need to be done in each; for in-house projects, processes served little value and yet seemed to create too much bureaucracy overhead (?!). He then went on explaining why multitasking is normal. Their work come from multiple sources, on-going projects and support for live products. The code base also goes through constant rework as "This is the Internet, thing changes fast. The right user experience is unknown and experiments are needed", he said. As we continued to talk about technical work, it appeared to me that his team biggest achievement was to be capable of implementing their own version of world-class libraries and frameworks such as SQLite or jQuery. Couldn't restrain myself, I asked for his opinion about the open source community, about Q&A platform like StackOverflow and Quora. The situation sounded just like the movie "300" to me, just that the Spartan were no better than Persian.

The last couple of minutes at the end of the interview was some casual chit chat about working environment and job description. As far as I could understand, engineers are only given really technical work. My seek for a position with the balance of management and technical work was blocked by the bureaucracy of the 2000-people organization. There weren't many new facts in this last minute talk, but enough for me to confirm that the most successful Internal company in Vietnam has a firm hold on its human resources. Though my time to work for a corporation hasn't come yet, I wish it live long and prosper.

Software Estimation - meaning and sources of error

Do you remember when did you make your first software estimate? I could not. But it must have come very naturally and unconsciously. Perhaps when I was examining my first assignment requirement, looking for room for an extra feature. Or when we divided work in my first group project. Or when I was asked "how long will it take?" during my internship. Nowadays, the projects I am estimating have gotten more complicated that I can no longer depend on my intuitive judgment for reliable estimates. It is an obvious need to upgrade techniques and mindset accordingly to meet the new situation. Reliable software estimate is not a black magic nor a fictional story, it is a skill that can be improved through retrospection, discipline and practice. In this post, I will briefly capture the meaning of accurate estimate to an organization and identify sources of estimation errors. (Slides are available)

Estimation in software development

Estimation is considered the building block of many activities in software development life circle. These include schedule planning (detailed schedule, complete work breakdown structure), budget planning (prioritize functionality, divide iterations) and resources planning. Accuracy of estimate affects these activities and in general the project ability to hit target. So when a consultant company like us is asked for an estimate, we are not asked for a tentative judgement that we can change our mind later on. We are asked for a commitment or a plan to meet a target.

Given that responsibility, developers' attitude towards estimation is still neglected. If I ask you how long would it take to finish your coming release, there is a huge change that I would get a single-point number answer. In fact a project outcomes follow a probability distribution (McConnell, 2006). A project might be done faster, or later. And the chance the project will be done in the middle of the distribution is most likely. Moreover, there is always a limit on how well a project can be done so the part on the left will be truncated.

What we have now is a representation of the probability distribution of project outcomes. The single-point "estimate" we had earlier is actually a target.

What is estimate used for in your company? If you don't really care how software development works, estimation is all about charging customer. Actually there is much thinking following that probability statement. Take estimate as a means of visualization, project planners will be able to find the gap between a project's business targets and its estimated schedule and cost.

A good estimate is an estimate that provides clear enough view of the project reality to allow the project leadership to make good decisions about how to control the project to hit targets - McConnell, 2006

As it is hard to get 100% accurate estimate and error is inevitable, what is better, overestimate or underestimate? In overall, the penalty for underestimation is more severe than that of overestimation. However

The focus of the estimate should be on accuracy, not conservatism. Once you move the focus away from accuracy, bias can creep in from different sources and the value of the estimate will be reduced - McConnell, 2006

Why your estimate is not accurate

The cone of uncertainty

The amount of uncertainty during software development process varies at different moments and typically narrows down over time. This implies that estimate made at the beginning of the project is less accurate that estimates made at later stages. At the beginning of a project, the targets haven't been fully conceptualized yet. Decisions at this level are broad and subject to changes in future. On the other hand, most decisions in development phase are significantly smaller and focus on implementation details. Do not expect the cone to narrow itself though. Unless decisions to remove uncertainty are made, the variation does not go away.

Organization structure that kills both productivity and predictability.

An environment that

• Makes room for multitasking to creep in
• Employs incompetent technical skills
• Apply incomplete/unskilled project management

Is a great condition to surpress our productivity. It suffers and goes down, rarely linearly but usually exponentially. The key is, the parameters of this exponential function are highly subject to human nature and context. In other words, they are a myth. Thank to this, there gone our predictability.

Unstable requirements

Everyone in the industry knows how destructive requirement changes can be. They prevent us from narrowing down the cone of uncertainty. They are also not well tracked and the project isn't re-estimated when it should be. Project control strategies are out of the scope of this post. But in term of estimation strategy, we can incorporate an allowance for requirement growth and changes (McConnell, 2006). 20%, 30% or 50% depends on your own judgement.

Omitted activities

Include time in your estimates for stated requirements, implied requirements and non-functional requirements. Nothing can be build for free and your estimates shouldn't imply that it can.

Unfounded optimism

Developers are hopelessly optimistic creatures. Yes, we are. That's a fact. It's a thing that we can't deny. This behavior has a close relationship with omitted activities. Our mind is not prepared for multi tasking. Instead of juggling things simultaneously, people have a tendency to focus on only one thing that they are most interested in, they know best or they perceive as being most important to the project and discard the rest. The moment you fail to conceive the project as a whole picture, subjectivity and bias creep in.

Estimate Influences

Because everything that helps you understand the nature of software development improve estimation accuracy.

Project size.

Project size implies the number of potential variations in a project. The bigger the project, the harder it is to be estimated. If building a 10-story building take 20 man-months, how long would it take to build a 100-story? Much more than 200 man-months. During the construction, many issues would emerge. Architecture to support the height, wind and natural disaster. Logistic so that materials are always ready and idle time is minimized. Power plan to keep the structure habitable. That's a perfect metaphor for software development. As the project gets bigger, there are more modules. These modules need to communicate with each other. The number of these communication paths is corresponding to the effort required for to complete the project. And it grows exponentially. This phenomena is widely known as diseconomy of scale.

One of the biggest problem I had related to this was that I failed to anticipate the rich interaction between components. The initial estimate was usually correct, but then missing pieces of how it interact with others kicked in and overflew the estimate.

Type of software

Different software types have different focus and difficulty. Developing an e-commerce website is generally easy and focuses on reusability to maximize profit. Developing a life-critical system is not only complicated on its own but also constrained by industrial or legal regulations. Although not all software development activities produce code, these differences are somehow reflected into LOC over time and demonstrate a strong different in velocity across different project types.

Personnel

Researchers have found a difference of 10-fold in productivity and quality between good and bad programmers with the same levels of experience and also between different teams in the same industries (McConnell, 2008). This degree of variation is not unique to the software industry but more a common behavior among occupations, including professional writing, sport and police work. The top 20 percent of the people produced about 50 percent of the output. On the other hand, 10 percent of the people contribute negatively to the output, i.e. can't fix the problems they created.

---

Steve McConnell. (2006). What Is an "Estimate"?. In: Software Estimation - Demystifying the black art. Redmond: Microsoft Press.

Steve McConnell. (2006). Value of Accurate Estimates. In: Software Estimation - Demystifying the black art. Redmond: Microsoft Press.

Steve McConnell. (2006). Estimate Influences. In: Software Estimation - Demystifying the black art. Redmond: Microsoft Press.

Steve McConnell. (2008). 10x Software Development. Available: http://blogs.construx.com/blogs/stevemcc/archive/2008/03/27/productivity-variations-among-software-developers-and-teams-the-origin-of-quot-10x-quot.aspx. Last accessed 26th Mar 2013.

[Lightning talk] Group by time interval in SQL

The other day I needed to write a query that grouped entries by time intervals (week, month, etc.)

I thought of PSQL's generate_series first as it is capable of creating series of numeric values with precise step. So my first attempt would be to select start and end dates of a series of one-week intervals, starting from the first case creation date.


It was not easy as it seems. I went through a hell of type casting. Then the easy step would be to group entries whose created date fails into these intervals.


In fact, I didn't even get to this step. I, in fact, went to the wrong direction, given the context of the feature I was implementing, it made more sense to group entries by calendar's week, not any random 7-day interval.


In order to achieve that, the most critical step was to be able to trace back the beginning of the week from any given date. Thank god, that was easy.


The price I had to pay for going the wrong way was high, but for this case, I guess I would eat it

3 Days of Lean and Kaizen

This week, from March 4th to 6th, the first Lean Mindset Workshop and Kaizen Camp Gathering in HCMC was held. The Agile Vietnam guys did an truly awesome job in bringing together top-notches around the globe the event. That included experts on Lean Software Development, Mary and Tom Poppendieck as well as Jim Benson and Tonianne Demaria Barry, founders of Personal Kanban.

Joining the event was both a last-minute decision and a bet to me. Three days before the event, Saigon Service Design Jam was held. It was also the first of its kind in Vietnam, and also lasted for three days. I couldn't afford the two events together. Selecting the Lean and Kaizen one was truly a leap of faith as I had stopped attending the events of Agile Vietnam a while back. Although I had to commuted 30km in Saigon's heat for the event, it was well chosen.

What was there?

The Lean Mindset Workshop occupied the whole March 4th.

Mary and Tom were an admirable 70-something couple who were spending their retired time travelling around the world giving Lean Mindset workshops and writing books. Honestly, I was overwhelmed by the amount of information. The usually-two-day workshop was compressed into one so that was a lot of talking and for a short period in the afternoon I was lost.

The workshop started by examining the death of companies that were driven under the name of productivity, developed securing manual and policy which ironically prevented innovations and finally failed to focus on its customer's values. The stability periods between economy crisis are getting shorter, the market is more fluctuating than ever and yet disruptions remain extremely hard to forecast. There is a rising need for organizations to structure themselves to be flexible against changes from time to time. The ideas of Lean Mindset are evolved around this concept.

We moved on to identify the seven flow disrupters in software development. Together in a group of 7-8, we discussed about the source, effects, and solution of each. The next step was to give our organization a health check based on the basic disciplines of a healthy organization. The list had a lot to do with quality control and as expected Cogini failed hard on testing disciplines (though we got most of the rest right).

Due to the time limit, we could only afford wrapping up Iteration, Kanban, and Continuous Delivery quickly. Kanban was still a pretty new concept in Vietnam software industry so it was important to stress that Kanban doesn't ask an organization to change anything that its currently does. Positioning itself as a supporting framework to visualize processes and policies, Kanban can be integrated smoothly with organization's current process. The three processes covered in the workshop are tools serving the purpose of making organizations more Lean and agile.

To end the day, we had an interesting talk about improving productivity

The legendary 10,000 hours

Balancing challenge and skill

The science of motivation

Kaizen Camp took over the last two days of the event. 

I found the way the workshop and the camp lined up and supported each other fascinating. If there had been only the workshop, most attendants would have came out like me, overwhelmed and confused by the massive amount of information. If there had been only the Kaizen Camp, there wouldn't have been enough topics to feed two days full of action. During the camp, we discussed work, life, and community. And under the spirit of kaizen, we also discussed improving all of them. The camp shared the same format with Barcamp and even though there were facilitators, the topics were community-centric and highly diverse. Fascinating!

The topics gave me the opportunities to challenge and practice what I was introduced to during the workshop. One of the reasons I stopped attending events of Agile Vietnam was that even though the sharing there was interesting, I found that each organization was unique in certain ways. How could I apply something that had been successful else where was always blur. Facilitating a group of motivated people isn't about teaching them or telling them what to do but more about asking triggering questions to get the participants to open up and speak out what is on their mind. And Mary, Tom, Jim and especially Toni made awesome facilitators. Many questions were answered specifically to Cogini context and the fog was just lifted. I ended up facilitating a session or two myself.

Nice people

I have attended a number of Barcamp, locally and internationally. Yet the attendants to this event managed to be the most diverse. I didn't met as many managers as I would like to, but there was a gentleman from TMA that really knew what he was talking. I met a girl from the North that seemed to have full of doubts in life yet decided to join a startup. I made friends with two engineers with tremendous experience from KMS, and a head of department of a game company. There was also a startup guy who seemed to be making similar mistakes as I did when getting to know Agile methodologies. And who could have known that I would run into a high-school friend there. Behind each attendant was a more sophisticated story. A startup with bureaucracy problems of a Forbes 500. A company that sent only programmers and no manager to the event. A sponsor that didn't mind that its name couldn't show up on media prints. Despite all of those differences, I was moved by their genuine desire to learn, share and make friends. Once again, my belief that engineers are nicest people on earth was strengthen. Thank you.

---

* Special thank to Huy Tran for allowing me to use his photos to illustrate for the post