Wisdom:

This article appeared in an Australian magazine

Inside the self-tuning “Genetic” Linux

In genetic-library updates: I/O Workload fingerprinting should be released to LKML soon. Brandon Philips has a running version of the 0(1) plugin, w/ a small positive improvement. That should be following shortly.

So, I’ve been pretty busy lately with MBA school, book writing, full work plate, businesses to run, personal issues, and attempting a life on top of that. And thus the genetic library has suffered due with a lack of updates. But enough of my excuses…what are the future plans to revive this patchset.

I have some good news. IBM has generously given me 1 PY of help to work on the genetic library which I will be overseeing. Unfortunately, I will still have my normal work responsibilities to attend to, so only my “free work time” will be available. This PY will attempt to create a plugin for the O(1) scheduler in order to increase the likelihood that the genetic lib will ever make it into the kernel. Resumes are being accepted as we speak. :)

In other good news, I have a “I/O Workload Fingerprinting” feature coming out on the genetic library, which will enable faster convergence. This feature is the main topic of my 2006 OLS paper, and thus I will actually have to do it.

Stay tuned for updates.

It strikes me that too many of America’s youth idolize sports stars as their heroes in life. Unfortunately, this idolization extends past the star’s abilities in the arena to their personal life. Needless to say, stars like Kobe Bryant and Mark McGwire are not model citizens.

A hero should be someone who you want to emulate, and has done something to be proud of.

My question is, why do not more children see their parents as heroes. From fathers who are over in Iraq, to mothers who work three jobs to support their family. These are America’s heroes.

While I do not believe kids should grow up thinking that working three jobs is something to emulate, I do believe the mentality of doing what it takes is something to be proud of and something to strive for.

Sometimes the best way to prove a point is through an example. It is a little long, but intriguing until the end. Take a look at a simplier life:

   Body Ritual among the Nacirema

Read the comment below after reading the study.

Only 6 days until I start B-School at University of Texas. The first week I go down to San Antonio for a Business Immersion Course (BIC). It’s basically business boot camp for us non-business types.

I’ll soon find out if my summer of business-book reading actually helped.

I’m excited about it right now, but I’m sure my tune will change once I’m studying for final exams again. My first classes are accounting and operations management. How hard can accounting be for an engineer? (famous last words)

After much work w/ Peter Williams and Con Kolivas, I finally have a decent performing genetic-lib implementation on interbench. It does orders of magnitude better in the burn & compile pieces. What really screws Zaphod and the genetic-lib up is when there is one interactive child running in the background w/ a bunch of threads that are doing while(1) {}. The interactive task does not get classified as such and they miss many of their checkpoints while the cpu hogs run. This latest version has a few bug fixes and detects interactive tasks sooner. The only thing to watch is that with the quicker detection of CPU hogs, a burst of CPU activity on X will get it classified as a CPU hog. Please let me know if you see as such.

I’ve been using interbench a lot lately, and it does an excellent job of catching the “bad children”. I can see almost immediately when some child gets mutated off into the woods and performance tanks. Con did a great job with the benchmark.

Mine is time. With more and more people vying for bits of it, I have realized that time is a finite resource, and should be cherished. In the last few years, I have noticed a fundamental difference in how I think. I the past I valued money above time. I would rather spend a day putting in a garage door opener rather than paying someone $60 to do it. Now, $60 for a free day sounds like a great trade off.

That’s not to say I just throw money around to save my time. I don’t mind spending an extra hour negotiating a car salesman to save a $1,000 (plus I find it enjoyable). The trade off of how much money saved versus how much time it took must always be made.

There are other in-tangible commodities that the time trade off should be made. Mainly, relationships. It is very important to take time to maintain relationships as they are what will get you through when times are tough. They are also one of the greatest assets. Having a friend that knows a friend goes a long ways in developing a career or a business.

The final time trade off that I am always making is an emotional investment. To keep from burning out and experiencing the time-loss that goes along with it, methods of decompressing must be constituted. Some people use television, others use sports. Whatever method is used, the recreation trade off goes a long way.

Update: As one co-worker pointed out, I am probably not old enough yet, but in time, my most valued commodity will probably change to health.

I have managed to do it again. I’ve overloaded myself w/ too much to do, with too little time. Thankfully my biggest near-term concern is nearly complete. My OLS paper is almost finished. The paper is on the genetic-library in the Linux kernel. It will be very useful for people to learn about the genetic-library and hopefully show how beneficial it is.

The biggest problem I have encountered is a regressions in performance results on the Zaphod CPU plugin. I was getting only 1%. Thankfully, my co-author, Peter Williams, was able to track down a bug in the run-delay calculation, and get the results back up to the >3% that I was originally seeing. I also noticed a trend towards the Zaphod plugin doing very well in higher CPU loads.

The best news I have gotten from doing this paper is stellar results in the Anticipatory I/O scheduler. On average, I was seeing 8.72% improvement, and up to 23.22% improvement in the random write workload.

The next thing I need to look at is a combination of RAID’d and unRAID’d disks. I was seeing huge fluxations in results on Anticipatory (which should not be used on RAID’d disks in the first place). I am theorizing that because the genetic-lib is per I/O scheduler, that it bounces back and forth for tuning for the individual SCSI disk, and then tries tuning for the RAID’d disks. Which one is tunes for depends on which one is getting more I/O. I will have to granularize the genetic-lib to operate on a per-disk basis. Thankfully there has been some support put into the kernel recently.

Watch http://linuxsymposium.org for updates.

I will hopefully have some time to do some more development work on the genetic-library soon, and a new version coming out soon.

After listening to Jon “Maddog” Hall give a speech on open source in emerging countries, I thought about a point that he touched on that was very interesting. In the future, the current model of closed sourced proprietary software will not work. Because of open source software, business models will have to change.

Before we look at the future, we need to look at the past. Back in the 1960’s all software was essentially open. It was developed by small software houses creating very specific software tailored to the customer’s needs. The customer kept the source code, and could make changes to it as needed. It was not until the 1980’s and the development of micro-computers when software companies started packaging general software that all customers might be able to use. Because the software was not for any specific customer, they closed the source to protect trade secrets and other IP. This practice of general software has continued to the present day. The problem with this current closed-source model is that if a customer wants a specific change, a bug fix, a new feature, or a new language, it is very difficult to get closed source software companies to make the changes in a cost effective and timely manner. Most of the time, it’s not in their “best business interest” to make the change. So customers have no option to get the change they need.

Now bring in open source software. If a customer uses open source software they have the freedom to make a change if they want. They have the freedom to get a timely bug fix, to support a obscure language, or tailor the software to their specific needs. To hire a developer may cost them money, but they have the business option to do so. Most of the time, it is far cheaper to hire a developer to tailor open source software to their needs then to buy the proprietary software in the first place. Now the customer has software that does exactly what they want.

I believe that open source software will continue to get better and better to rival anything that Microsoft puts out (arguably already does). Why would customers want to pay exorbitant amounts of money for sub par in-flexible software. I believe that as more and more customers move to using open source software, that the software model will move back to the one that existed in the 1960’s. Customers will hire software houses to take open source code, and tailor it to their specific needs at a fraction of the cost. They can keep the changes and update the code base as needed. The current closed-source software model will not be able to exist.