Homework - Bài tập

Learn as much detail about the domain you are working on as possible. Talk with potential clients about their business, product and trouble, though 80% you are an introvert. Visit end-users in their "natural habitat". Those are the homework of a software engineer. Doing homework is meant to improve your understand of the business logic in your current system.

Of all engineering professions, software engineer is one of a kind. No one would possibly ask a traffic engineer to build her house. But a software engineer who built a social network yesterday and is implementing a mobile health care system today, is an everyday story.

In short term, the domain-specific knowledge allows you to work and communicate with your clients in their own language. Which of these below codebases would you rather be working?

if employeeID in PayCheck.get(paycheckID).getEmployeeIds:

if paycheck.include(employee):

Coding in the domain language means you can represent a business concept right in the code. Whenever you skip a domain term, you are introducing a secret understanding that this int over here means the way to identify an employee while that int over there is the way to identify the paycheck this month. And the testers and next engineers inheriting the codebase aren't happy about that.

Programmers "talk" the programming language: classes, objects, and databases. Clients talk in the language of their domain: payroll, social insurance and tax evasion. You could try to translate the two languages, but sooner or later, bugs will creep in. Eventually programmers need to speak the language of their client, not the other way around. A client isn't going to pick up Python 101 and tell you that the payroll is containing some duplicated employeeID, she is more likely telling you she is paying double for a freeloader.

But more importantly, it is about your value as an engineer. You are worth more than your code. Don't accept a job where you're told exactly what to build and how to build it. You need to work somewhere that appreciates your insights into the product as well as your ability to build it. Don't be a guy who has worked in e-commerce for 3 years but can't tell what e-commerce is rather than a shopping cart. I once worked on a mobile health care project. The team leader, despite of having 2 master's degrees from MIT and CMU, knew nothing about a health care system. But she tackled that head on, learned everything she could find about existing systems and legal regulations of the US government and talked to countless physicians and specialists in every conference she happened to attend. To an extend she could tell the difference in procedure between two stages and probably had enough material to write a book about US health care industry.

Nowadays, tools and technologies are advanced and abundant, but more often than not, we often hear of projects in big words that are at the end of the day, empty and meaningless because of the lack of commitment.


=====================================================

Tỉ mẩn tìm hiểu mảng ngành của dự án mình đang tham gia. Trò chuyện với khách hàng tiềm năng về công việc, sản phẩm và những mối lo của họ, dù 80% bạn là người hướng nội. Khảo sát đối tượng của sản phẩm. Đó là bài tập của nghề kỹ sư phần mềm. Làm bài tập đầy đủ là hiểu thêm về hệ thống bạn đang triển khai.

Trong hầm bà lằng thứ kỹ sư, kỹ sư phần mềm nó hơi lạ (không phải vì mềm). Chẳng ai đi hỏi anh kỹ sư cầu đường về xây nhà. Nhưng chuyện anh phần mềm sáng làm mạng xã hội chiều mần hệ thống y tế lại bình thường như cân đường hộp sữa.

Ngày một ngày hai, hiểu biết về mảng ngành đang tham gia giúp bạn làm việc và trao đổi với khách hàng bằng "tiếng mẹ đẻ". Thay kệ bạn biết python hay không, bạn thích làm việc với đoạn code nào hơn?

if employeeID in PayCheck.get(paycheckID).getEmployeeIds:

if paycheck.include(employee):

Tái hiện được khái niệm của mảng ngành qua những dòng lệnh có rất nhiều giá trị. Bất cứ khi nào bạn từ chối ăn nằm với thứ ngôn ngữ thực tế này là bạn đang tích góp một bí mật nho nhỏ "chỉ có hai ta" kiểu như số ở đây là kí kiệu của nhân viên còn số ở kia là kí hiệu bản lương tháng này, lộn là tháng này nhịn. Mà testers và các bạn lập trình viên về sau này thì chúa ghét cái của để dành này, của cho là của nợ mà.

Dân kỹ sư suy nghĩ theo ngôn ngữ lập trình, đủ thứ âm binh classes, objects, và databases, đối với người ngoài thì rất biến thái, vẹo vọ. Còn khách hàng, họ nói bằng ngôn từ của ngành nghề họ: bảng lương, bảo hiểm xã hội và trốn thuế, kiểu vậy. Bạn có thể gắng gọng mà dịch hai thứ ngôn ngữ vốn chả có gì chung chạ này, nhưng không chóng thì chày, bọ cũng bò vào thôi, hồng nào hồng chẳng có gai. Chuyện này dù đúng hay sai, vẫn là bạn phải học thứ ngôn ngữ lạ lẫm kia của khách hàng thôi. Vì sẽ chẳng có chuyện chị client vác sách Python 101 lên và bảo bạn cái bảng lương có mấy mã số nhân viên bị trùng. Chị sẽ cất giọng nam cao 5 quãng 8 mà bảo rằng mình đang phải trả lương gấp đôi cho một thằng ất ơ nào ấy.

Đùa chút thôi, chuyện này còn quan trọng hơn, nó can dự đến giá trị của bạn. Gọi mình là kỹ sư phần mềm, bạn phải có biết rằng bạn đáng giá hơn cái máy chuyển hoá pizza và cà phê thành code. Đừng dễ dãi nhận một công việc mà bạn được dắt tay từng thứ một. Để nuôi lớn khả năng của mình, hãy làm việc với những người biết đáng giá cao kiến thức và ý kiến của bạn. Đừng gọi mình là kỹ sư nếu sau 3 năm ăn nằm với thương mại điện tử, bạn vẫn chả biết thương mạng điện tử khác với cái giỏ hàng thế nào. Ở thì quá khứ chưa xa xôi lắm, tôi từng theo đuổi một dự án theo dõi sức khoẻ qua di động. Chị trưởng nhóm, dù dắt lưng 2 tấm bằng thạc sỹ, ở cả MIT và CMU, lơ ngơ như bò đeo nơ về hệ thống chăm sóc sức khoẻ cũng như các chế tài tại Mỹ, thị trường chính của sản phẩm. Nhưng chị dấn thân lắm, đọc tất cả những gì tìm được về những hệ thống hiện hành và các qui định pháp luật. Chị gặp những người làm trong ngành y tế để trò chuyện còn nhiều hơn gặp ba mẹ mình. Đến độ chị giờ kể vanh vách chế tài của 2 bang khác nhau ra sao, và có lẽ đủ tư liệu để viết sách về nền y tế Huê Kỳ.

Ngày nay, công cụ và kỹ thuật tiên tiến đầy rẫy, nhiều khi đến thừa mứa, 1 việc mà đến hai ba tools. Nhưng vẫn đầy ra những dự án đao to búa lớn và thất bại thậm tệ chỉ vì biếng nhác cống hiến với nghề.

Software Estimation - meaning and sources of error

Do you remember when did you make your first software estimate? I could not. But it must have come very naturally and unconsciously. Perhaps when I was examining my first assignment requirement, looking for room for an extra feature. Or when we divided work in my first group project. Or when I was asked "how long will it take?" during my internship. Nowadays, the projects I am estimating have gotten more complicated that I can no longer depend on my intuitive judgment for reliable estimates. It is an obvious need to upgrade techniques and mindset accordingly to meet the new situation. Reliable software estimate is not a black magic nor a fictional story, it is a skill that can be improved through retrospection, discipline and practice. In this post, I will briefly capture the meaning of accurate estimate to an organization and identify sources of estimation errors. (Slides are available)

Estimation in software development

Estimation is considered the building block of many activities in software development life circle. These include schedule planning (detailed schedule, complete work breakdown structure), budget planning (prioritize functionality, divide iterations) and resources planning. Accuracy of estimate affects these activities and in general the project ability to hit target. So when a consultant company like us is asked for an estimate, we are not asked for a tentative judgement that we can change our mind later on. We are asked for a commitment or a plan to meet a target.

Given that responsibility, developers' attitude towards estimation is still neglected. If I ask you how long would it take to finish your coming release, there is a huge change that I would get a single-point number answer. In fact a project outcomes follow a probability distribution (McConnell, 2006). A project might be done faster, or later. And the chance the project will be done in the middle of the distribution is most likely. Moreover, there is always a limit on how well a project can be done so the part on the left will be truncated.

What we have now is a representation of the probability distribution of project outcomes. The single-point "estimate" we had earlier is actually a target.

What is estimate used for in your company? If you don't really care how software development works, estimation is all about charging customer. Actually there is much thinking following that probability statement. Take estimate as a means of visualization, project planners will be able to find the gap between a project's business targets and its estimated schedule and cost.

A good estimate is an estimate that provides clear enough view of the project reality to allow the project leadership to make good decisions about how to control the project to hit targets - McConnell, 2006

As it is hard to get 100% accurate estimate and error is inevitable, what is better, overestimate or underestimate? In overall, the penalty for underestimation is more severe than that of overestimation. However

The focus of the estimate should be on accuracy, not conservatism. Once you move the focus away from accuracy, bias can creep in from different sources and the value of the estimate will be reduced - McConnell, 2006

Why your estimate is not accurate

The cone of uncertainty

The amount of uncertainty during software development process varies at different moments and typically narrows down over time. This implies that estimate made at the beginning of the project is less accurate that estimates made at later stages. At the beginning of a project, the targets haven't been fully conceptualized yet. Decisions at this level are broad and subject to changes in future. On the other hand, most decisions in development phase are significantly smaller and focus on implementation details. Do not expect the cone to narrow itself though. Unless decisions to remove uncertainty are made, the variation does not go away.

Organization structure that kills both productivity and predictability.

An environment that

• Makes room for multitasking to creep in
• Employs incompetent technical skills
• Apply incomplete/unskilled project management

Is a great condition to surpress our productivity. It suffers and goes down, rarely linearly but usually exponentially. The key is, the parameters of this exponential function are highly subject to human nature and context. In other words, they are a myth. Thank to this, there gone our predictability.

Unstable requirements

Everyone in the industry knows how destructive requirement changes can be. They prevent us from narrowing down the cone of uncertainty. They are also not well tracked and the project isn't re-estimated when it should be. Project control strategies are out of the scope of this post. But in term of estimation strategy, we can incorporate an allowance for requirement growth and changes (McConnell, 2006). 20%, 30% or 50% depends on your own judgement.

Omitted activities

Include time in your estimates for stated requirements, implied requirements and non-functional requirements. Nothing can be build for free and your estimates shouldn't imply that it can.

Unfounded optimism

Developers are hopelessly optimistic creatures. Yes, we are. That's a fact. It's a thing that we can't deny. This behavior has a close relationship with omitted activities. Our mind is not prepared for multi tasking. Instead of juggling things simultaneously, people have a tendency to focus on only one thing that they are most interested in, they know best or they perceive as being most important to the project and discard the rest. The moment you fail to conceive the project as a whole picture, subjectivity and bias creep in.

Estimate Influences

Because everything that helps you understand the nature of software development improve estimation accuracy.

Project size.

Project size implies the number of potential variations in a project. The bigger the project, the harder it is to be estimated. If building a 10-story building take 20 man-months, how long would it take to build a 100-story? Much more than 200 man-months. During the construction, many issues would emerge. Architecture to support the height, wind and natural disaster. Logistic so that materials are always ready and idle time is minimized. Power plan to keep the structure habitable. That's a perfect metaphor for software development. As the project gets bigger, there are more modules. These modules need to communicate with each other. The number of these communication paths is corresponding to the effort required for to complete the project. And it grows exponentially. This phenomena is widely known as diseconomy of scale.

One of the biggest problem I had related to this was that I failed to anticipate the rich interaction between components. The initial estimate was usually correct, but then missing pieces of how it interact with others kicked in and overflew the estimate.

Type of software

Different software types have different focus and difficulty. Developing an e-commerce website is generally easy and focuses on reusability to maximize profit. Developing a life-critical system is not only complicated on its own but also constrained by industrial or legal regulations. Although not all software development activities produce code, these differences are somehow reflected into LOC over time and demonstrate a strong different in velocity across different project types.

Personnel

Researchers have found a difference of 10-fold in productivity and quality between good and bad programmers with the same levels of experience and also between different teams in the same industries (McConnell, 2008). This degree of variation is not unique to the software industry but more a common behavior among occupations, including professional writing, sport and police work. The top 20 percent of the people produced about 50 percent of the output. On the other hand, 10 percent of the people contribute negatively to the output, i.e. can't fix the problems they created.

---

Steve McConnell. (2006). What Is an "Estimate"?. In: Software Estimation - Demystifying the black art. Redmond: Microsoft Press.

Steve McConnell. (2006). Value of Accurate Estimates. In: Software Estimation - Demystifying the black art. Redmond: Microsoft Press.

Steve McConnell. (2006). Estimate Influences. In: Software Estimation - Demystifying the black art. Redmond: Microsoft Press.

Steve McConnell. (2008). 10x Software Development. Available: http://blogs.construx.com/blogs/stevemcc/archive/2008/03/27/productivity-variations-among-software-developers-and-teams-the-origin-of-quot-10x-quot.aspx. Last accessed 26th Mar 2013.

3 Days of Lean and Kaizen

This week, from March 4th to 6th, the first Lean Mindset Workshop and Kaizen Camp Gathering in HCMC was held. The Agile Vietnam guys did an truly awesome job in bringing together top-notches around the globe the event. That included experts on Lean Software Development, Mary and Tom Poppendieck as well as Jim Benson and Tonianne Demaria Barry, founders of Personal Kanban.

Joining the event was both a last-minute decision and a bet to me. Three days before the event, Saigon Service Design Jam was held. It was also the first of its kind in Vietnam, and also lasted for three days. I couldn't afford the two events together. Selecting the Lean and Kaizen one was truly a leap of faith as I had stopped attending the events of Agile Vietnam a while back. Although I had to commuted 30km in Saigon's heat for the event, it was well chosen.

What was there?

The Lean Mindset Workshop occupied the whole March 4th.

Mary and Tom were an admirable 70-something couple who were spending their retired time travelling around the world giving Lean Mindset workshops and writing books. Honestly, I was overwhelmed by the amount of information. The usually-two-day workshop was compressed into one so that was a lot of talking and for a short period in the afternoon I was lost.

The workshop started by examining the death of companies that were driven under the name of productivity, developed securing manual and policy which ironically prevented innovations and finally failed to focus on its customer's values. The stability periods between economy crisis are getting shorter, the market is more fluctuating than ever and yet disruptions remain extremely hard to forecast. There is a rising need for organizations to structure themselves to be flexible against changes from time to time. The ideas of Lean Mindset are evolved around this concept.

We moved on to identify the seven flow disrupters in software development. Together in a group of 7-8, we discussed about the source, effects, and solution of each. The next step was to give our organization a health check based on the basic disciplines of a healthy organization. The list had a lot to do with quality control and as expected Cogini failed hard on testing disciplines (though we got most of the rest right).

Due to the time limit, we could only afford wrapping up Iteration, Kanban, and Continuous Delivery quickly. Kanban was still a pretty new concept in Vietnam software industry so it was important to stress that Kanban doesn't ask an organization to change anything that its currently does. Positioning itself as a supporting framework to visualize processes and policies, Kanban can be integrated smoothly with organization's current process. The three processes covered in the workshop are tools serving the purpose of making organizations more Lean and agile.

To end the day, we had an interesting talk about improving productivity

The legendary 10,000 hours

Balancing challenge and skill

The science of motivation

Kaizen Camp took over the last two days of the event. 

I found the way the workshop and the camp lined up and supported each other fascinating. If there had been only the workshop, most attendants would have came out like me, overwhelmed and confused by the massive amount of information. If there had been only the Kaizen Camp, there wouldn't have been enough topics to feed two days full of action. During the camp, we discussed work, life, and community. And under the spirit of kaizen, we also discussed improving all of them. The camp shared the same format with Barcamp and even though there were facilitators, the topics were community-centric and highly diverse. Fascinating!

The topics gave me the opportunities to challenge and practice what I was introduced to during the workshop. One of the reasons I stopped attending events of Agile Vietnam was that even though the sharing there was interesting, I found that each organization was unique in certain ways. How could I apply something that had been successful else where was always blur. Facilitating a group of motivated people isn't about teaching them or telling them what to do but more about asking triggering questions to get the participants to open up and speak out what is on their mind. And Mary, Tom, Jim and especially Toni made awesome facilitators. Many questions were answered specifically to Cogini context and the fog was just lifted. I ended up facilitating a session or two myself.

Nice people

I have attended a number of Barcamp, locally and internationally. Yet the attendants to this event managed to be the most diverse. I didn't met as many managers as I would like to, but there was a gentleman from TMA that really knew what he was talking. I met a girl from the North that seemed to have full of doubts in life yet decided to join a startup. I made friends with two engineers with tremendous experience from KMS, and a head of department of a game company. There was also a startup guy who seemed to be making similar mistakes as I did when getting to know Agile methodologies. And who could have known that I would run into a high-school friend there. Behind each attendant was a more sophisticated story. A startup with bureaucracy problems of a Forbes 500. A company that sent only programmers and no manager to the event. A sponsor that didn't mind that its name couldn't show up on media prints. Despite all of those differences, I was moved by their genuine desire to learn, share and make friends. Once again, my belief that engineers are nicest people on earth was strengthen. Thank you.

---

* Special thank to Huy Tran for allowing me to use his photos to illustrate for the post