“You should never be surprised by or feel the need to explain why any physical system is in a high entropy state.”
― Brain Greene, The Fabric of the Cosmos

Adhyayam – 2

In the previous part, i gave a brief introduction about Artificial Intelligence, and Where do we stand in it currently.

Heuristics are techniques defined for a particular problem for solving it. They are a major part of algorithms that instruct machines to solve problems and make decisions.

Entropy is the name people came up with to describe why nothing is 100% perfect or 100% efficient. Technically, Entropy is the number of ways stuff can be arranged. And orderliness is not an universal definition, is it? What seems to be ordered to one can be disordered for another. so in my own formulation, I would define ordered system is one which has clear relationship among its various parameters or components. I can throw 10 red balls inside a box, and jiggle them. Every ball in that lot does not have the same relationship with other balls. There can’t be a perfect generic rule, whereas they can be arranged in a particular fashion so that there is some pattern followed if not a completely generic one. This we can all agree that is in a more ordered state than the previous. Now I pose a question – how many ways are there to achieve this pattern with the given system and how many messy ways?

It is reasonable to assume for a simplistic argument that each time we jiggle the box there is a new disordered arrangement of balls in the box and I can jiggle ‘n’ number of times every time coming up with a new arrangement, whereas we can arrange the balls in patterns only a certain number of times which is much less than ‘n’ . Thus more the number of ways of arrangement, higher is the entropy.

The common misconception is that Entropy is the measure of disorder but it is not. It is true that for most of the systems the number of disordered states is much higher that the number of ordered states, implying that higher the entropy higher the disordered state; leading to the above misconception. If so, we can argue this way If we make a system in which the number of ordered states are much higher that the number of disordered states, then the entropy will still be high implying a highly ordered state of the system. Yes, this is perfectly true and recently scientists have come up with such systems. Those are beyond the  scope of this post but I shall try to put up a post about them a little later.

After the brief description of entropy, we shall come back to AI. In most of the two-player games, the most favourable moves are the ones which has the maximum number of moves in the subsequent turn for that player compared to his opponent. If a move yields 10 different moves for your opponent and 15 moves for you in the next turn then it is a good move. Logically speaking it implies that there are more ways in which you can move than your opponent thus giving you the control of the game and a series of a controlled moves guarantees you a win. This is a general strategy followed by most of the two player algorithms for games. The most efficient one to me is the Alpha-Beta pruning algorithm which follows a similar logic. The general statement is ‘to maximise the diversity of future possible states’.

In the above example, the number of moves is one heuristic for evaluating the worthiness of a particular move. This is used in games like Othello, chess. There can be many such heuristics depending on the need and accuracy. Recently, people have used ‘Entropy’ as such a heuristic for making a system intelligent.

Having linked all the terms in the title, we shall proceed to the explanation. Alex Wissnergross proposed that entropy can be used as a heuristic. There is a law in Thermodynamics which I believe the most fundamental law in the whole of universe, which states that ‘Entropy of a  spontaneous process always increases in forward time’. Here, we need not worry about thermodynamic spontaneity but just get the idea. In our previous example, when we jiggle the box of balls, each event has more number of states to be arranged than the previous one. This is the implication of the Entropy Law.

Now we see the connection between Entropy and AI algorithms being, ‘Increase in the number of possible states’. That is what AI needs and that is exactly what Entropy provides. Simply Elegant!! Alex argues that Entropy is the “Underlying Mechanism of Intelligence”. He describes a force which drives systems to increase their entropy (called as entropic force) is the required heuristic of AI.

Entropic force

The above equation states that the force F seeks to maximize the possible diversity S with a strength T till a given time t (tau). Alex and his colleagues built a software engine called ‘Entropica’ which visualises and simulates the above equation in the given system. The main characteristic of Entropica is that only the system and the equation are fed in and the program by itself does intelligent actions without any goal or instruction, which is amazing and definitely the highest level of intelligence. In the video below, this is clearly demonstrated:

It is amazing how seemingly unrelated topics converge together and produce an astonishing result. If entropy can produce intelligent behaviour in systems on its own, is it also the reason behind human intelligence? This is an amazing question to investigate and might also provide answers why other animals are not as intelligent as humans.

Like, comment and share!!

Links & References:

Part 1 

Alex’s Page :  http://www.alexwg.org/

Entropica : http://www.entropica.com/

Alex’s TED talk : http://www.ted.com/talks/alex_wissner_gross_a_new_equation_for_intelligence




Entropy – The Next Heuristic for AI – Part 1

“Maybe the only significant difference between a really smart simulation and a human being was the noise they made when you punched them.”        ― Terry Pratchett, The Long Earth

Adhyayam – 1 [Chapter – 1 (Sanskrit)]

                                                       Artificial Intelligence is the future! It is an undeniable verity. Most of us would have heard or read about the annexation of Deep Mind Technologies , a mysterious AI company researching on Machine Learning by Google.inc early this year for $400 million. It was started in just 2011 and rose up to be one the most advanced AI companies albeit its clandestine work. So, there is an immense scope and need for AI.

What is AI?

I would state it this way : ” Any Algorithm or a procedure, which is capable of making rational decisions for various external stimuli. Well, that was short and precise wasn’t it? Just for a comparison, i will give you the widely accepted definition :

“The automation of activities that we associate with human thinking, activities such as decision making, problem solving, learning, …”  – Bellman, 1978.

Now the reader should observe that  in my definition, i was completely generic.Since the definition of AI by Bellman in 1978, there has been many fields introduced in AI such as visual perception, speech recognition, language processing, game playing. And notice the word automation. AI need not be an automation. We have loads of softwares that come under AI. The most important aspect of my definition is that i adhere to AI being capable of making only decision and that suffices. All others are just another way of saying the same. In game playing, you make decisions as to what the next optimal move should be, in language processing, you make decisions as to how to interpret the phrase and what is the suitable reply. In machine learning, you again choose the aspects that are to be incorporated in the next generation trials and their utility.  I also believe that the various other aspects of AI like Logic, reasoning, are also specific cases of decision making process.

Now the world has forged ahead from just pondering over the idea of the evolution of human intelligence to imparting it to machines. Each machine is a brain child of its creator. Now scientists try to assay the development of human intelligence through imparting intelligence to machines. Now a question arises, how close are the current AI algorithms to human cognition?, well not quite close but there has been tremendous furtherances in AI.

Okay.. so where are your fancy AI machines in the market?? Well, does Siri ring a bell? Google Now?. These are the products of natural language processing researches, one of the main domain in AI. AI is extremely vast and new kinds of intelligence are constantly budding up.

I would like to share some of the AI projects, which i feel are amazing and each is a work of art. I shall share 4 things which are jargon but everyone needs to know

  • Kilobots :  These are tiny mobile robots which are engineered to demonstrate collective behaviours algorithms. kilobot4Literally there are 1024 of them. These are useful for studying the swarm behaviour and implementing them in robotics for doing tedious tasks through intercommunicating bots. Developed by Self-Organizing systems research group at Harvard University. Their main aim is to control these 1000+ bots for various procedures like shape formation, contour tracing and localisation.
  • Wolfram Alpha : Some of you would have known about it but still, alphaThis is a computational knowledge engine. Well that’s how they call it. To put it in simpler terms, it is a search engine, which apart from from googling the answer (oops!) i meant searching the answer, it also computes the answer. Have a tough calculus problem? just put it in the wolfram alpha search bar and viola!! get the answer, the plot and various other relevant data! Compared to it, Google search is… just lol! Well, try it till you feel the amazement. Here are some sample pictures of it.


Wolfram Alpha does not just add the directories of the websites . but it relies on licensed data catalogues by its employees, that are as accurate it can ever be. Unlike a normal search engine, it does not give you pictures of celebrities but rather gives you factual answer as well as its visual interpretation. (Phew!! Google is out of danger!..hmmm..maybe).

  • Blue Brain Project : It is a project aiming to reverse engineer the brain. It reconstructs the brain ‘piece by piece’. The cortex in the brain is the most evolved and advanced technology created by nature, and this project aims to virtually simulate that. They initially started with a rat’s brain and have successfully using 100 mesocircuits, and by 2023, they have planned to recreate the human brain which is equivalent too 1000 rat brains.To further the understanding they have planned to simulate at a molecular level to study the chemistry behind cognition.
  • A.L.I.C.E : It is an acronym for “Artificial Linguistic logoInternet Computer Entity” is a chatterbot, means you can have a conversation with a computer that uses natural language processing. It is an open source software. The researches created a XML-compliant language called AIML (AI markup language) in which even users can customize their chatterbox.

I don’t think i need to explain about IBM’s two most mind-blowing machines Deep Blue and Watson.

So, after this quite a lengthy introduction to AI, we shall come to the topic, what is entropy? and How is it even related to AI?.. That will be in the next Adhyayam!!

“You should never be surprised by or feel the need to explain why any physical system is in a high entropy state.”
― Brain Greene, The Fabric of the Cosmos

For Further Readings :

Kilobots :        http://spectrum.ieee.org/automaton/robotics/robotics-hardware/a-thousand-kilobots-self-assemble                 http://www.eecs.harvard.edu/ssr/projects/progSA/kilobot.html

Wolfram Alpha : http://computer.howstuffworks.com/internet/basics/wolfram-alpha1.htm                                                         http://www.wolframalpha.com/

Blue Brain Project : http://bluebrain.epfl.ch/page-52063.html

Religious Science or Scientific Religion?? – A New Quest

              Actually, I am fed up of people asking me whether am a theist?? Being a Brahmanan (Brahmin), they expect me to be one. But Nope, i am not!! Well, an atheist? Nope! Actually i don’t fit into anyone of the religious inclinations popularized by Dr.Richard Dawkins, and don’t want to fit into any.I don’t believe in the conventional definition of GOD nor Atheism. So what am i?? I just have some reasonable questions.


Religion and Science started when man wanted to understand nature in the first place, from then religion deviated into some balderdash and philosophy (all religions for that matter). Lets take the example of Christianity, when those ancients started writing the bible, They assumed that Ptolemy’s view of our solar system was true, which was done using scientific methods, though incorrect. I don’t want to go into further disputable arguments which may hurt other’s feelings, but when science keeps itself updating, why cant religion? The science my father studied is not the same which i study, nor will be the same which my son will study, But the religious preachings remain the same forever. Why is that?

 “Religion is flawed, only because men are flawed”. There is nothing wrong in being flawed, but it is felonious to remain flawed. As a supporter of science, i admit that in science, there were, and in some cases still contain flaws, but they immediately strived to solve it. Why can’t religion do that? Fine, we all respect our ancients and their views, (Even science does, taking the direction of current to be the direction of flow of positive charges in a conductor) but when they are wrong, they are wrong! “Kuttram kuttrame!!” I agree that there has been some paradigm shifting changes in our lives from those of our ancients, which has changed our way of life from theirs. But the scientific facts seem to remain the same. We are not yet that old for the universe to expand so much that it loses all the evidence of Big Bang and other things, Atleast till then, religious leaders could be broad-minded enough to realize and accept the flaws in their religion, and try to modify their views if possible. That kind of world would be amazing.

 But in a way i do support religion. Throughout history religion has been the vital reason for the creation and development of various arts. When there is no religion, then those would go meaningless. Maybe new styles of art may occur, but i have a crossed mind about those. They may not carry the “essence” that present arts have, And not to forget the “Tradition”.And here i do not mean the stupid “parigaram, and venduthal”( After all, no religion defines god as a business-man to do dealings) These traditions have created our identity, losing these causes a big damage. Yes, i do try to follow my tradition, just because its my identity and i love it, but i never do it because i want a supernatural power to bestow me with something.This is what our elders used to advice us as “Kadanai sei; Palanai Edhirparkade” [Do your duty, never expect the outcome] which describes that our world is completely uncertain, which are invariantly the results of Quantum Mechanics and Chaos Theory.

So were those the only reasons that i support religion? No! The notion of god may vary in different religions, which are just different interpretations, just like Quantum Mechanics having different interpretations. I have come across a very few people, who give an alternative explanation for the preachings in their religions analogous with scientific principles. The vacuous atheists, mock them. As for me, what on earth is wrong with such type of interpretations.

   Personally, i feel that religion cannot and should not be erased totally. Can science provide peace? or confidence? uplift our state of mind? Well, Religion atleast attempts to do that, and we must appreciate that. After all, what’s the point of just living in a mechanistic way? Lets assume you have understood the whole universe, and have realized Einstein’s dream of a single equation explaining the entire universe/Multiverse whatever it may be.

“Then what??”

The only logical answer, that my feeble mind could think of is that, the visionaries at that time, would use science to formulate ways to live our lives. Since they could understand everything, they could use science to their advantage to have a better life. So why cant Religions all over the world use the current science to formulate a better way??

Why cant they understand this?

Or atleast the scientists, the so called geniuses, instead of proving highly non-intuitive not very useful theorems like 1+2+3+4+… = -(1/12) !!!!, should foresee the future of humanity and not the human intelligence. I don’t know about the reader, but i really see a crisis here. Maybe we are using science to our advantage even now like weather satellites, seismographs etc., but i want science to have an impact on the virtues, that every human must posses. I don’t know if this could be done or not, but i really wish for it.

 I am not against religion nor science, i just have the view that when both were created for the same cause, why cant they just merge? Why cant we view science and religion as different manifestations of the same thing i.e., to realize the beauty of nature and the purpose of our lives in it.

 I haven’t done much research about this topic, but i just wrote what i feel. If this kind of ideas already prevail, i would love to read more on those. As for the Title, I am more inclined to Scientific Religion rather than Religious Science.

 Thank you for your time, Critical, non-critical comments and views invited.