Archive for the 'Programming' Category

LoseThos is a small 64bit open source free operating system available for IBM PC compatible computers. It is written and released by Terry A. Davis and is an operating system developed from ground up. The primary focus of this operating system seems to be video games (really simple ones!) and is intended for "programming as entertainment". Interestingly it also introduces a new language very much similar to C/C++!

The latest version available as of November 11th 2008 is V4.19 and the ISO file is less than 20MB. The distribution is a "live cd" and you can boot into the OS by burning it into a CD. Since LoseThos is a pure 64bit OS, it will work only on those systems or virtual machines which support 64bit. For example, Linux qemu emulator is not supported.

What makes LoseThos interesting is that the whole of the OS is written by Terry over a period of 5 years. It has over 100,000 lines of code. Programmers may find LoseThos as an interesting toy! It allows complete access to all the hardware and there is minimal layering.

LoseThos Hardware Requirements

• IBM PC Compatible (EM64T)
• VGA Graphics
• Minimum 256MB RAM and a maximum of 128GB RAM!
• CD-ROM but no SCSI
• ATA/SATA hard drives
• Gameport Joystick/RS232 serial port but no USB

LoseThos is currently just a hobby OS which can be fun for programmers. But who knows, it might even inspire someone to create an exceptional  64 bit open source operating system in future!

Don’t waste your time trying to get this running on Vmware Workstation. After trying for a few minutes I realized that LoseThos uses non standard stuff and hence Vmware cannot support it.  In fact I was also unable to get this running on my latest machine which is Intel Quad core CPU (Q9300).

Grails is an open source Web application framework written for Java platform. It is based on the Groovy programming language and is a competitor to Ruby on Rails.

Grails boost programmer productivity by eliminating configuration effort using the "coding by convention" paradigm.

Grails eliminates the need for downloading and configuring multiple sets of applications for starting development of a Java Web application. Grails provides a single package which contains test application server, ORM tool (Hibernate) etc. So in a nutshell, Grails is J2EE on steroids.

Books on Groovy

In order to be an expert on Grails, you need to learn Groovy. Groovy is a scripting language based on Java. Following are some of the good books available in Groovy.

Groovy in Action - This takes a detailed look at the Java scripting language Groovy. This is the favorite book on Groovy.

Groovy Recipes: Greasing the Wheels of Java- Groovy recipes one programming problem at a time and explores how it can be done in Groovy. This is a must have book if you want to be quickly productive in Groovy. This book also looks at Grails.

Programming Groovy: Dynamic Productivity for the Java Developer - This book is not yet released. This is intended for new programmers and will contain both Groovy fundamentals and advanced Groovy topics such as meta programming, unit testing and domain specific language creation.

Books on Grails

The Definitive Guide to Grails (Definitive Guide) - This is a comprehensive reference to Grails. It takes you through the complete Grails development lifecycle such as project configuration and automatic ORM mapping. This also contains a quick introduction to Groovy.

Getting Started with Grails - This book is available as a free Grails book download (PDF). This is a quick introduction to Grails platform. It takes you through the framework step by step. It also explores how a Web application is turned into concrete code from abstract code.

Ruby on Rails is the hot new Web application framework based on Ruby language. It offers unmatched programmer productivity and has taken over the Web2.0 application development. In this post, I will look at some of the free tutorials and free ebooks on Ruby on Rails available for download.

1. Tutorial on Ruby on Rails - This is a beginner’s tutorial on RoR. Hosted by RoR official wiki, this is the best place to start on Ruby on Rails.

2. Rolling with Ruby on Rails Part I and Part II - This tutorial written by Curt Hibbs is a good introduction to Ruby on Rails. It takes you through a sample application end to end.

3. Learn Ruby on Rails in 4 days! - This is a PDF book on Ruby on Rails. It gives you a toolbox of Rails techniques and can be used as a bridge from Rails basics to Advanced rails.

4. Fast track Web application development with Ruby on Rails - This article from IBM takes a quick look at developing a Rails Web application.

5. Beginner’s guide to Ruby on Rails - Another introductory tutorial on Ruby on Rails.

It is almost a year since I quit my regular job and started my freelance career. It turned out to be much more tough than I thought. Though I enjoy much more freedom, I am yet to pull off the same income I had with my previous job. But the good thing is that there are no limits as to what I can do!

Following tips are from my experience. I work as a programmer and hence it will be more useful to those who work as a freelance programmer.

1. Be your own boss! - This means that you need to have discipline. Ensure that you have a time schedule. For example, you can set aside 8 hours daily for your work and then ensure that you work those hours.

2. Create a work enviornment - You need a place where you can work without distractions. You don’t want your daughter come up you and say “daddy, let us go to the beach” when you are neck deep in a programming problem. do you?

3. Networking - In order to have a successful freelance career, you need to have good networking skills with people. Sometimes you may have work that you need to outsource. Most of the freelancers work with a very small set of clients. The relationship is so good that the client won’t think of changing you even for a lower bid by somebody else.

4. Be honest and assertive - No one likes a guy who tells lies. So ensure that you are 100% honest with your client. If it causes a client to leave, it is perhaps good for you. Also don’t work under the client, but work with the client.

5. Develop a good portfolio - Ensure that you have a decent Web site up and running with testimonials and details of your previous work. You can also a run a blog which will cover the challenges you face daily. For example, I document all the programming challenges I face. You may get new clients through this medium.

6. Be part of the community - There are a lot of forums/websites for freelancers. You also have sites like rentacoder or getafreelancer which can be used to get programming work. Have a solid profile on these with “professional looking” photo!

7. Be organized - This is vey important. Organize your work and create a plan. The plan need not be elaborate. But it should help you in identifying whether you have too much work in hand.

8. Outsource the subtasks - You may not be good in web design or graphics design. But you have project to rewrite a website with PHP. What would you do? In such cases, outsource your work. There are a lot of good design guys out there who can be contracted for the web design work! A good programmer need not be a good graphics designer and vice versa.

9. Use tools - One of the most important things in freelance career is to enhance your productivity. More work = more money. Investigate which tools can make you productive and then deploy them in your regular work.

10. Enjoy life! - The reason why you went freelance is probably because you wanted more freedom/choice. Hence don’t forget to enjoy your life!

Many popular Websites use captchas to prevent automated signups. The idea is to have a test which only humans can answer. Almost all captchas used now are image based. The user is asked to type in a word displayed on the screen. These days many of these sites offer alternate mechanisms such as audio for people with visual disability.

This week I was on vacation and I thought I will just go through the captchas of most popular Websites. What I found was a wide range of visual captchas. Some of these are difficult even for a human! These are listed in alphabetical order.

The following sites were selected from the list of Top 500 Websites by Alexa.

alibaba.com - This is a simple captcha and I think it will be easy to write a program to decode this!

bebo.com - Another simple captcha with plain background. Note that some letters are incomplete, but still readable.

craigslist.com - A captcha easy for humans, difficult for programs.

del.icio.us - Simple captcha with dizzying background.

digg.com - This is a tough one even for humans. Well, a lot of guys will be trying to break this!

facebook.com - Can it be more difficult than this? Tough for humans and machines.

fotolog.com - Note how the background makes automatic detection almost impossible. Good one.

friendster.com - Huh? Fun even in captchas? Nice one.

gmail.com - The captcha looks very simple. The trick is that they use different fonts and styles for every pageload!

icq.com - Interesting background! Note that some letters are grouped while others are not.

livejournal.com - Another tough one, even for humans! Note that typing in ‘Audio’ enables an audio test!

lycos.com - What the? can you read it? They even have their site name on the captcha!

metacafe.com - Ugly, but very effective.

live.com - Typical microsoft stuff. Lot of bells and whistles, but difficult to read!

netlog.com - Note how the thickness variation makes it difficult even for humans.

piratebay.org - Even the pirates have got a nice captcha with a threatening black background.

reddit.com - Another difficult captcha. Sometime almost impossible to read.

rediff.com - A simple captcha which is easier to read, but I guess can be cracked with code.

xanga.com - Clever one with random lines connecting letters.

yahoo.com - Another captcha using the random lines connecting letters.

Ruby language is one of the most elegant computer languages out there. You can write highly efficient code with Ruby. It is also very popular and bagged the programming language of the year award in 2006! In this post, I will look at the free ebooks available for Ruby.

1. Programming Ruby, The Pragmatic Programmer’s Guide - This is one of the best free ebooks available for Ruby. The first edition of this book is available online. A copy of this book is also included in the Ruby language download. I would suggest this as the first book for learning Ruby.

2. Mr. Neighborly’s Humble Little Ruby Book - This is a short treatise on the Ruby language, covering language basics and some library functions. PDF version of this ebook (144 pages) can be downloaded after registering at the site.

3. Why’s (Poignant) Guide to Ruby - This is an interesting Ruby book and is available online. This e-book covers Ruby in deep, but not everyone may like the novel type style used. It is also the funniest book I have read so far. You can also download a PDF version here.

4. The Little Book of Ruby - Supposed to be the “fastest and the easiest way to learn Ruby by example”, this book is available as a PDF download. You can also download sample code used in the book. Data structures are explained in detail.

5. Learning Ruby - This book is still in development and covers the basics. You can read the book online or download a zipped version.

6. Ruby user’s guide - The ultimate reference to the Ruby language and covers the language in depth. This can be read online.

7. Ruby Quick Reference - This is a single page quick reference guide to Ruby. This is an ideal reference to keep beside your desk.

8. A little ruby, a lot of objects - This is a Ruby book under construction. The chapters are available as PDF. This book covers the language from the object oriented viewpoint.

9. Data Structures and Algorithms with Object-Oriented Design Patterns in Ruby - This is a book on data structures and algorithms. The language used for examples is Ruby. Use this book once you have learned the basics of Ruby language. You can read this book online.

1. Mythical man month Frederick P Brooks - This classic book looks at the human elements of software engineering. I know some of you will scream, “not this book again!”. But this is a must read book if you are a programmer and many of the conclusions in the book are still very valid after all these years.

2. The pragmatic programmer: From Journeyman to Master by Andrew Hunt and David Thomas - This book explores the “apprentice” way of becoming a master programmer. I found this very informative look at programmer evolution.

3. Code Complete by Steve McConnell - An instructive book on programming by a Microsoft engineer. But there is nothing specific about Microsoft in this. It is a pleasure to read this book and you will find yourself reading it again and again.

4. Object-Oriented Software Construction, Second Edition by Bertrand Meyer - This book is a complete reference on the object oriented software design. I don’t think there is any other exhaustive book on the object oriented topic. Only drawback is that this book is too academic. Still a must have item for a programmer.

5. Design Patterns: Elements of Reusable Object-Oriented Software - This is a gold mine of programming wisdom. But you need some solid coding experience before you can appreciate this book. Don’t just memorize the pattern names in the book, but try to understand the real motivation for each of the patterns presented in this book.

Here is a funny Javascript hack I across in digg.

1. Open any Website which contains images. Google image search or a Flickr page is best.

2. In the Address bar, replace URL with the following text.

javascript:R=0; x1=.1; y1=.05; x2=.25; y2=.24; x3=1.6; y3=.24; x4=300; y4=200; x5=300; y5=200; DI=document.images; DIL=DI.length; function A(){for(i=0; i-DIL; i++){DIS=DI[ i ].style; DIS.position=’absolute’; DIS.left=Math.sin(R*x1+i*x2+x3)*x4+x5; DIS.top=Math.cos(R*y1+i*y2+y3)*y4+y5}R++}setInterval(’A()’,5); void(0);

3. Click on Go!

Don’t get scared if things looks pretty scary This basically rotates all the images in the current Webpage.

Recently I was trying to install Rational Application Developer, a popular Java software development platform. During installation I got an error saying that integrated Websphere application server installation failed. The installation did continue and I was able to use RAD, but without the built in WAS server. But to do anything useful I needed WAS server!

I looked at the logs generated. The rad_install.log file under logs folder of RAD installation contained the following error,

(Jan 4, 2007 12:37:20 PM), Install, com.ibm.rational.cdi.common.CustomExec, err, Operation timed out: connect
(Jan 4, 2007 12:37:20 PM), Install, com.installshield.wizard.RunnableWizardBeanContext, err, java.lang.NullPointerException
STACK_TRACE: 5
java.lang.NullPointerException
at com.ibm.rational.cdi.common.CustomExec.parseConfigurationFile(CustomExec.java:444)
at com.ibm.rational.cdi.common.CustomExec.execute(CustomExec.java:294)
at com.installshield.wizard.RunnableWizardBeanContext.run(RunnableWizardBeanContext.java:21)

To resolve this I used the following hack. I still don’t know the root cause of the problem.

1. Uninstalled RAD.
2. Installed RAD again, but kept the install integrated WAS 6.0 button unchecked.
3. From the disk4 folder, I have manually installed WAS server. Selected custom install and then choose the directory for installation as [RAD directory]\runtimes\base_v6. The base_v6 folder was manually created.

When I started RAD again, voila! the integrated WAS server was enabled.

My Machine settings
Windows XP service pack 2 with all latest patches.
All hard disks were NTFS formatted.

References
IBM on the WAS issue in RAD
Similar problem listed on IBM forum without any solution

Plone is a content management system built on Zope application server. Learning Plone is tough. What I have learned so far is that in order to be a successful Plone developer, first you need have to solid understanding of Zope concepts and Python programming language. Once that is taken care, you can jump into Plone!

One of the best references out there for Plone is the Plone Book by Andy McKay. This is a bit outdated (2004) but is ok to learn Plone.

Another good Plone resource is the Plone Howtos. Common questions such as “How do I remove footer or header” is answered there.

I have a nagging feeling that even Plone is getting outdated!

Following code in Java illustrates how Jakarta HttpClient package can be used to tunnel through NTLM proxy. This is handy if you want to download a page periodically and then serve it from your local web server. For this code to compile packages needed are httpclient 3.0, logging 1.0.5 and codec 1.3 (Later versions are also fine). All the jars for these packages must be in classpath while running the code below.

import java.io.*;
import java.net.Socket;
import org.apache.commons.httpclient.*;
import org.apache.commons.httpclient.auth.AuthScope;
import org.apache.commons.httpclient.methods.GetMethod;
 
public class NTLMFetch
{
  public static void main(String[] args) throws Exception 
  {
    // Enable proxy to host logging
    System.setProperty("org.apache.commons.logging.Log", 
               "org.apache.commons.logging.impl.SimpleLog");
    System.setProperty("org.apache.commons.logging.simplelog.
                       showdatetime", "true");
    System.setProperty("org.apache.commons.logging.simplelog.
                        log.httpclient.wire.header", "debug");
    System.setProperty("org.apache.commons.logging.simplelog.
                        log.org.apache.commons.httpclient", "debug");
 
    // Update this to the actual URL
    String url = "http://mysite.com/index.html"; 
    HttpClient client = new HttpClient();
    HttpMethod method = new GetMethod(url);
    HostConfiguration hostConfig= client.getHostConfiguration();
    // Update this to point to NTLM enabled proxy
    hostConfig.setProxy("192.168.0.0", 80); 

    // Authenticate using NTLM
    client.getState().setProxyCredentials(AuthScope.ANY,
          new NTCredentials("userid","password","",""));

    int statusCode = client.executeMethod(method);
    System.out.println("Stat=" +HttpStatus.getStatusText(statusCode));
    String strOutput = method.getResponseBodyAsString();

    // Write the output to a local file
    PrintWriter writer = new PrintWriter(new FileWriter("c:/down.html"));
    writer.println(strOutput);
    writer.close();
  }
}

WSAD from IBM is one of the best tools available out there for enterprise Java development. It is a perfect IDE where everything in Java (EJB, servlets, JSP, webservices etc.) can be developed with ease. Here are some WSAD tips which my fellow programmers find quite useful.

How do I change the workspace in WSAD?
Normally when you start WSAD, you get an option to change workspace. If you had checked the “Do not show this dialog” option, you will not get get this option again. To enable workspace selection again,
Right click on WSAD shortcut and insert -setworkspace in the target field of the shortcut. In my case it is “D:\Program Files\IBM\WebSphere Studio\Application Developer\v5.1.2\wsappdev.exe” -setworkspace

Why is VSS plugin not working in WSAD?
To enable VSS plugin with WSAD 5.1 modify “wsappdev.ini” in WSAD working directory and comment VMArgs=-Xj9, for example #VMArgs=-Xj9

Why is my JDBC settings not working in WSAD?
You need to add classes12.zip to the classpath of the JDBC provider.

Can I install WSAD 5.1.2 on Windows 2003?
Yes you can! But you need enable compatibility mode for the installer. Right click on the installer and then select properties->compatibility->compatibility mode for windows 2000.

Which shortcut keys you use in WSAD?

Here are some interesting facts about www.ebay.com, the leading internet auction site. Being in the Web application development industry, these figures appear mind boggling to me. I knew they must be having heavy traffic and used wonder how they manage it…

• eBay manages over 212,000,000 registered users!
• eBay contains over 1 billion photos!
• eBay manages over 26 billion SQL executions per day!!
• eBay gets over 1 billion page views per day!!
• eBay stores over 2 petabytes of data!!

If you want to read more on eBay’s architecture evolution, click here (PDF). This is a presentation on eBay architecture by Randy Shoup and Dan Pritchett at the SD forum.
eBay’s growth chart

Following are the key architecture decisions which helped eBay to achieve such scalability.

Scaling the data tier
1. The database load is spread by segmenting the data by function (user, item, account etc.)
2. Horizontal split of databases by access path.
3. Minimization of database resources (no stored procedure, no sorting, extensive use of prepared statements)
4. Minimize database transactions (This is achieved by careful ordering of database operations, running of reconciliation batch in case of failure etc.)
Â
Scaling the application tier
1. Again spreading the load by segmentation (segmentation by function and further horizontal segmentation inside functions)
2. Everything is stateless (state maintained in cookies or scratch database), no EJB, minimal J2EE and maximum use of cache.
3. Tiered application and eBay’s own pure Java OR mapping.
4. Maximum code partitioning with minimum dependencies.

Further Reading
Presentation on eBay’s architecture evolution

I have been programming for over 10 years. Even now I am still learning a lot about good programming. Programming is a skill which is improved by practice and experience. True, people are born with different programming capabilities, still many techniques can be learned only with experience.

What is that distinguishes a good programmer from others? Here is a list from my experience.

1. He thinks - A good coder never jumps straight away into coding when he gets a work. He first *thinks* about the problem and decides on how to approach the solution. He may try out some code snippets and write something which typically thrown out later. But once he starts the actual coding he proceeds at lighting speed.

2. He never duplicates code - A good programmer never, I repeat never duplicates code. To me code duplication is one one of the worst sins for a programmer. In the initial coding some duplication may be there, but an expert removes them gradually.

3. He writes extensible code - Writing extensible code requires considerable experience. This is what makes coding a creative activity. You need to be aware of the full impact of what you are writing in the code’s future life cycle. Design patterns is a collection of techniques which helps in writing extensible code. A good programmer may not know the names of all patterns but he is aware of them nonetheless.

4. He is not afraid to re-factor - No one can write software in a perfect way at the first try. You write a good amount of code, analyze it and then re-factor it. This is a continuous process during coding. I have seen that re-factoring requires considerable amount prior experience.

5. He comments for a purpose - A good programmer comments what is relevant. If he has spend sometime thinking about a portion of code, he will comment his thoughts in the code. This helps another programmer to understand the thoughts and design decisions behind the code he is reading.

Obviously there are other qualities (such as code optimization) which can be seen in a good programmer. But to me the above 5 items are the most important.
Further Reading
What are design patterns
What is refactoring?

Nowadays many software service providers have dedicated testing teams to carry out software testing. One problem with this is that the testing team becomes disconnected from the developer team. It also brings inter team politics as a new issue in software development.

Here is a list of top 10 programmer excuses I hear almost regularly at my workplace. This happens when a tester reports a defect or bug to the programmer.

1. “It works on my machine!”
2. “It is not in the software spec.”
3. “It is not in my test plan.”
4. “It worked yesterday.”
5. “There is something wrong with your data.”
6. “Is it deployed or is the server restarted?”
7. “I can’t reproduce the problem.”
8. “You have the wrong version.”
9. “There was no time to test it.”
10. “Somebody must have changed my code.”