Why AI Failed - My Two Cents

This post is inspired by the late Dr. Push Singh's article titled "Why AI Failed". When I say "failure" of AI, I am referring to the fact that we still do not have machines with human-level intelligence. Such machines were expected by 2001 itself. We know that science is not even on the brink of it. Dr. Singh's article is comprehensive and succinct in its explanation. In this post, I talk about the same thing with an analogy. In the analogy, I shall describe a problem and analyze two possible methods of approach. I shall end by drawing parallels and explain what is going wrong, in my opinion, with the current direction of AI research.

Before I detail the task, there are somethings I would like to say. The task I am going to talk about is an "ideal" case. It cannot be reproduced in real life. When I use terms like examination, please do not form a vision of a silent hall with an invigilator and a time limit of 3 hours. The whole thing is totally abstract. Analyze this article from 50 feet above - think about everything in general, do not nitpick and confine yourself to the material words I use. They are just there to make this article take a form in English. In fact, I myself do not have concrete pictures about what I am say. Think of this post as a description of something I see far away without my spectacles on!

The task - We are to train a 10 year old in mathematics and make him fit for an exam in integral calculus. Details are as follows:-
(1) You are God! You are perfect. Once again, Please do not think of God as a magician with a halo around his head who can do anything. I just mean you are "ideal". You know the perfect optimal way to do and solve anything.
(2) This 10 year old that you are assigned is just an average 10 year old. Nothing extra-ordinary about him. Nothing bad about him either. He is "human".
(3) The exam is one like no other. The score you get is a "perfect" indicator of your knowledge about integral calculus. Only God(you) can get 100%. The "human" cannot. Also, knowing even the most basic and elementary formulae of integral calculus will mean that you score more than 0. Think of this exam as a compilation of the all the questions that can exist about integral calculus. Right from the most elementary and trivial ones, to the most complex ones.
(4) Your sole objective is to maximize his score in the exam. You do not care about anything else.

The first question that you would now have would be - "how much time do we have?". I am not going to given an answer to this right now. Please push that question to the back of your head. All I shall say now is that the time is not enough to turn him into a God, i.e. make him score a 100%. 100% needs "infinite" time (if you do not understand why I said "infinite" time, please think about it. Getting a vision of "ideal", "perfect", etc. is essential to proceed. Such concepts are difficult to put in words. They can only be felt). You only have "finite" time on hand. Since I say the 10 year old is average, our task at hand is not a trivial one. It will take time and careful planning. Your planning would not only involve deciding what mathematics to teach him, but also a plan of the boy's daily routine. Right from the games he plays, the food he eats, etc. It is like he is totally in your control.

I do not know the perfect solution to this problem myself. I am sure no one reading this post will either! I can say that confidently because I am talking about "ideal" cases here. The exam itself is not conceivable. (A lot of people have exagerrated views about their teaching and mathematical abilities, I myself have ;). The fact that you were good at solving integration problems in college and could do them faster than all your friends does not mean you are the "ideal" person I am talking about here. We are all equally idiotic in front of "perfection". Someone may be a 10, someone else maybe a 200, 10000, or whatever. In comparison with infinity, all of them are nothing(zero) because anything divided by infinity is zero). If you think you know the perfect solution to this problem, I suggest you stop reading the post. It is useless for you to proceed.

If you are reading this paragraph and further, you agree that you do not know the perfect solution to this problem. I don't know it either. There are a lot of approaches we can talk about. Small changes in one might cause improvements. I do not want to get into that. I just want company as I attempt to analyze the "kind" of solutions possible. Hopefully one day we will know the perfect solution. I think the approaches can be broadly categorized as below:-

Approach 1: The drilling-in approach. Since the sole objective is to maximize the score in the exam, make the kid learn as much as possible by-heart. Begin with the basic formulae. Drill them in! Remember, this will help because the "exam" has the most basic and elementary problems as well as complex problems. Get a few basic text books and make him learn all the solutions by-heart. This will also push the score higher. Keep doing such things. Make him work overtime, to the exclusion of everything else, even sleep. Think about the scores you can reach with this kind of an approach...
Approach 2: The normal approach. Draw the kid a proper routine. Start with the basics. Make sure the kid develops in all respects. Now, think about the scores you can reach with this kind of approach...

Below is the graph of scores vs time taken of both the approaches.


Note:- I do not have any real concrete reasons for why I think the graph should look like that.

Observation from the plot: If the time given is less, approach 1 will lead to better results. If time given is more, approach 2 will lead to better results. I am sure this conclusion is not counter-intuitive by any means. Basically approach 2 has a higher rate of growth after a certain period of time. Approach 1 is good if the time of reckoning is small. I don't know if either of them will reach a score of 100. But all I can say is that if they do, approach 2 will beat approach 1 to the finish line. In fact, I believe approach 1 has no hope. It will plateau after a while. Approach 2 is the only one with hope.

Let us now get back to what I wanted to talk about - AI and its failure. I think the kid is our computer, the computer scientists and programmers are the gods. The integral calculus examination is just a way to see if the computer has achieved human-level intelligence. Scientists today are using approach 1. That is because it gives "immediate" results. Grants are given with deadlines. People giving the money want to see results quickly. So they do not care how the results are obtained. I am sure if you were to apply approach 2 in real life with a kid, it would take a lot of heart and tenacity. Just like that, for computer scientists to move towards the goal of human-level intelligence using approach 2 will require a lot of courage and rigour.

It is sad to see that very very few are prepared to do that. It is this which I think is the fundamental reason for the failure of AI - our "short-sighted" (desire of immediate results) attitude. I call out to the scientific community - please broaden your minds and vision!!! Please look beyond your immediate materialistic needs. Do not forget them!!! Just look beyond them once in a while.

I am very inspired by the following people - Marvin Minsky, Chris McKinstry and Push Singh. I hardly know anything myself. In fact, my first step into the field of AI was 2-3 months ago! I know nothing! It is just that I am interested and very inspired by the idea of human-level intelligence on a computer. It is like "hacking into God's computer" - I am borrowing Chris' words. I believe computers with human-level intelligence will solve all of the worlds problems.

...subtle is the lord, but he does not play dice with the universe... My views about determinism later...

Reward = 2 * Effort

As children, we are told by our parents that when we do a good deed, we get twice of what we do in return. Below is, what I think, mathematical proof of that.

T.P.T:
Reward = 2 * Effort

Assumptions:
(1) Life is an equation of the form L.H.S = R.H.S; where the LHS is the summation of all the efforts we put and RHS is the summation of what life gives back to us. Basically the = sign is put based on the assumption that we always get out of life exactly what we put into it.
(2) If we do a deed 'x', and it is good, '+x' is added to the LHS. If the deed is bad, we add a '-x' to the L.H.S.
(3) I we get a positive value on the right hand side, it is a reward. If we get a negative value, it is pain.

Proof: If a person does a good deed, a +x is added to the LHS. Now, during the process of doing the good deed, he goes through pain (ex. - Sitting up all night and helping someone do his homework is a good deed. But when you do it, you miss your sleep (pain)). The magnitive of this pain is exactly equal to the magnitude of the good deed. Therefore |pain| = |good_deed|. Let the magnitude of both of them be equal to 'x'.

If we write it in equation form, we get

x = -x + expected_reward

Here the x on the LHS is the good deed. The -x on the RHS is due to the pain felt while doing the good deed. The only way to balance the equation would be if a positive term were added on the RHS of the equation. Clearly, this would be the expected reward after having done the good deed and having endured the pain of doing the good deed. We can clearly see that expected_reward = 2x. Hence Proved :)

Clearly, the most critical assumption here is that we always get out of life exactly what we put into it. Do you believe that???

Abhiram.

History of Sorting Algorithms

The story behind an invention/discovery is always fascinating. Be it Newton having an apple fall on his head, or serendipitous happenings which led to discoveries like Penicillin, Radioactivity, etc - there is always something intriguing about stories of such a kind. Often, many inventions/discoveries either do not have such a story or they have a story and it is not well known.

In this post, I would like to share with you the history of how various sorting algorithms were developed.
(1) Selection Sort - Invented by Mr. Selection
(2) Bubble Sort - Invented by Mr. Bubble
(3) Merge Sort - Invented by Mr. Merge
(4) Counting Sort - Invented by Mr. Count

Story of Selection Sort:-
Mr. Selection's mother was an English teacher. Once every 3 months, she bought home examination papers to evaluate. Once the valuation was complete, she liked the papers arranged highest marks first and lowest last. She had a practice to praise the highest while handing out the paper, and at the same time, rebuke the person who scored the least. Having the highest scorer's sheet handed out first ensured that everyone got to know who he/she was; it would be correct kind of boost required for the student. Also, having the person who scored least to be called out last would help because by then, everyone else would have got their papers and would be busy with their own sheets, so least attention is drawn to it.

His mother gave him the task of arranging the papers. He decided to follow this strategy to arrange the papers.
(1) Place all the sheets side-by-side.
(2) Scan from left to right and find out the guy who got the highest.
(3) Take his sheet and place it face-down on another table.
(4) Among the leftover papers, scan again from left to right and find out the highest in the remaining set.
(5) Place his paper too, face-down, on the paper already placed on the table.
Repeat (4) and (5) until the entire set is complete.

10 years later, when he had to write a program to sort values, he recalled arranging the papers. He realised that in every pass, he selected the maximum and then placed it aside. He did the same thing in the program and named the algorithm after himself :)
Story of Bubble Sort:-
Mr. Bubble was a student of Pineapple High School Convent. During the daily prayers, all students of the same class stood together in a line and it was required that the students stood in ascending order of their heights. During the first ever time the students had to make a line, there was utter chaos. Everyone was shouting... "hey you are shorter than me, come and stand in front of me", "hey lambu, go and stand behind re", etc... The teacher called for silence!!! She thought of a way to arrange all of them.

The first sorting process that struck her was to find the shortest one and place him first; find the next shortest one and place him second and so on... She realised that the process of finding the shortest student among a set of students was very tricky. When she scanned from the top to the bottom of the line often she would not be able to decide whether a guy near the end was shorter than a guy she saw in the beginning. She could only decide who was shorter if they stood beside each other. So she went to the head of the line. She started swapping two adjacent students if they were not in order. She realised that after the first pass, clearly the tallest guy was at the end. She then decided to do the same to the remaining set. She kept performing the same and soon the line was sorted.

Mr. Bubble was impressed with what his teacher did. This incident was etched in his mind. Years later, he read a paper authored by Mr. Selection about a sorting algorithm called selection sort. He somehow happened to recall his teacher's method of sorting and realised it was different from the selection sort. He realised how the teacher bubbled the elements to the ends of the array. He decided to try programming it. He programmed it and it worked. This was how the Bubble Sort was born!!!

Story of Merge Sort:-
Mr. Merge was a scientist working in basketball research laboratories. The lab was far away from his home and so he chose to rent a place closeby. While shifting, he packed all his books into small cartons. He was an avid reader, because of which there were a really high number of cartons.

Once he reached, he wanted to take out the books and arrange them on his shelf in order of their heights. He realised that opening all the cartons and then sorting the books would make too much of a mess. He decided that he would break his problem into smaller bits. He first opened the cartons and arranged the books in the carton according to height. This was not much of a problem because the cartons could not hold too many books. Once arranged, he merged the cartons. He realised that having already sorted them made his job a bit easier. Merging small sorted sets of elements to get one full big sorted set was easier than sorting the entire large set together!!! The next thing you know - Merge Sort was born!!!

Story of Counting Sort:-
Mr. Count's little brother studied at samsung kindergarten school. Count would go to pick his brother up every afternoon. One afternoon, he was a bit early, and he could overhear what was going on in his brother's class. The teacher had listed a huge array of alphabets (randomnly and with repetition). Each kid was asked to sort the list. The teacher believed that this exercise would help kids get familiar with the alphabets as well as develop their memory (for a human to sort a list, it does require a little bit of memory usage on his part). Mr. Count's little brother could not do the exercise. The bell rung and before the teacher dismissed the class, he told Count's brother - "I will not send you home tomorrow if you do not complete this exercise in front of me".

Mr. Count then started wondering how he could make his little brother's job easy. The real problem for the kid was that there were too many alphabets on the board. The kid indeed knew the order of the alphabets when he was asked to chant it - "abcdefghijklmnop...". But to re-arrange them when they were in random fashion was a little difficult for the kid. Everytime, he had to recite the alphabet in his head. This was taking a lot of time.

Mr. Count then came up with a strategy:-
(1) He asked his brother to make a table with two colums and 26 rows
(2) In the first column, write down numbers alphabets from a to z
(3) Now, start scanning the array left-right and as you encounter each alphabet, put a line against the corresponding letter in the table. Once done, forget about the array.
(4) Now just scan the table from top to bottom. The number of times each letter had to be listed was obviously determined by the number of dashes for that particular alphabet.
(5) Hence, a top to bottom scan; with simultaneous listing of the alphabet would give the sorted list the teacher wanted.

Count's brother caught on with this idea and could complete the exercise next day with significant ease. The brother excitedly mentioned to count that the teacher was particularly impressed about the speed with which he completed the task.

When count was a university student, he had to write a program to sort a similar kind of list. The range of values was limited and known. There were no memory constraints and the process of sorting had to be as fast as possible. It goes withough saying that this was the birth of the counting sort.

Things to notice:-
(1) Mr. Merge had the same problem of sorting a large number of items and that too as efficiently as possible. A Merge sort is indeed used in that kind of a situation.
(2) Mr. Count had a "limited range of values". That is indeed the kind of situation a counting sort is used in!
(3) Both Mr. Selection and Mr. Bubble did not have too many things to handle at the same time. Selection and Bubble Sort are used for small arrays!

It goes without saying that these stories are pure crap. They are just things I made up to satisfy myself. However, I somehow feel the real stories could not have been much different; maybe different situations and also the scientists' parents being a little more creative in naming their children :P.

This is not an attempt from me to trivialise any matter. I have a feeling that making up stories like this for everything new you learn aids your understanding. I try and do it for everything I learn and I find that it really helps me.

Another thing I would like to point out to people. Successful computer scientists are not people who sit in front of the computer all day. They are people who observe everything around them and find inspiration to translate human language into a language the computer can understand. Edsger Dijkstra once said "...computer science is no more about computers than astronomy is about telescopes...". Often I hear dialogues like "...I don't want to get into computer science because I hate sitting in front of the computer all day man...". I hope this post shows those people that computer science does not need you to stare at the screen all the time; studying computer science does not mean you will spend a life building meticulous GUIs for Grocery Applications at Infosys! There is a difference between the terms Information Technology and Computer Science. Please do not use them interchangeably!!! Computer Science is as evergreen a science as any other. Booms and busts in IT have no effect on the science and its sacredness. This realization does not dawn easily on computer engineers even, let alone the others. I just wish that people did not have this misconception.

Abhiram.

Be Careful when you ask for a Boon!!!

Just an idea...

Lot has been talked about excessive honking. For me, it is so irritating that I forget to get irritated by autos and other general traffic!!!

It looks like most Bangaloreans think that honking will blow out a hole in everything ahead of them. Honking, for them, is a way to make people ahead of them move faster. For the non-honkers, it is terribly irritating and causes more stress than bad traffic!

Honking is just to alert the person ahead of you. It is a way to say "Someone is behind you". I think it is appropriate in the following situations:-
(1) You have a very slow vehicle ahead of you who is holding you back even though there is ample of space
(2) You are scared that the vehicle ahead of you might suddenly chase lane and that this might cause an accident
(3) General corners, curves, etc.
.. some more....

Anyways, I have an idea to reduce this menace. It is this - CONNECT THE HORN OF A VEHICLE TO THE PETROL TANK SUCH THAT PRESSING THE HORN CONSUMES A LITTLE PETROL... The amount of petrol should not be too much that people don't use it even when necessary, at the same time it should not be too less to make it un-noteworthy. Just optimal.

I think this would be perfect way to make people conscious. It is just like a small pinch. The small pinch will not hurt the first time, or the second time... But if done incessantly, it will turn the skin raw and cause hurt....

Abhiram...