Tag Archives: NLP

An alternative search engine

I hate Google. Not because I think they are stupid, but because I believe that they are brilliant, or at least their search engine is. As a follow-up to the previous post, what Google does is provide us with the most popular information, which is partially based on how often the source is mentioned in other places on the web(links), partially how well it’s is presented and favoured by other users(page-ranking), and partially how relevant it is to the search keywords and your personality(the actual search algorithm, and monitoring of your search history).

So what Google does, is blindly tell us what we like, compared to what everybody else likes, thus defining our place in life and society, and we love Google for it. However, it does so in a rather decent manner, making that the only decent thing about it. The information is presented fast and clean, in theory presented as something bias. Only on the side or at the top of the page do you actually see the sponsored links, seperated, and simply formatted, indifferntiating from each other, except the catch-phrase and the name of the product, limiting the bureaus to only natural language to manipulate the consumer. In comparison to Times Square, this is perhaps as decent as it will get in the global generation, when it comes to information representation.

Nough said, there is no competition to Google. Yahoo is dead, and Live Search never got through puberty, Google own’s the web, and the web own’s what you know, and what you know own’s what you think, believe and live by. 

One thing important to understand about a single provider of a service, is that using it, becomes a religion. And when a phenomenon like religion occurs, the human mind dies, and when that happens, innovation stops, and we get a “middle age”, where we believe whatever we have is good enough, and don’t try to build new technologies.

The IT industry is exactly the opposite of this, you might argue, but then Google is as much a parasite in the system, as it is a building stone. Just think about it. You are SATISFIED with Google, a better solution seems firstly unneeded, and secodnly impossible, so you entrust all you privates to this giants’ posetion, and you feel good about it, becuase that just makes your search results better!

Well, let’s not get too synical, and abstract here, the world is full of conspiracy theories already, besides, they make your mind go ka-boom(whereas religion, as the sense of stability, calms it down). The simple point that I want to get across here is that someone needs to stand up to Google, and provide them some sort of competition, before we all end up like we have, with the white earbuds in our ears.

So how do we beat Google? Well breaking down a giant and rebeuilding it from the ground up is seldomly the answer, the best is always to innovate (a world all you marketing people are very fund of). Take whatever exists, with all it’s problems, and solve some of them. Where Google is weak, is accuracy. To find accurate and legitimate information via Google can be like finding a needle in a haystack, all depending on what sort of information it is. So we end up taking the top-most results, which are innevitably the most popular ones, but the most popular answer is not always the most accurate one.

This popularity aspect on the other side, brings us much closer to the global village that Marshall McLuhan so vitiously proclaimed. We can connect trends across continents thanks to Google, which provides same information to all of us, and we personally rate what is better and what is worse, and filter the bull from the truth. 

Einstein once said:

“I believe, there are 2 things that are infinite in this world – the universe and human stupidity, and I’m not sure about the first one.”

We as humans have a problem, and that is we are easy to manipulate to believe something sensational, popular, or trendy. That’s the way we went into any war in history, the most clear examples being Vietnam and Iraq, no real problem persisted, but we all believed it was the right thing to do, until proven otherwise. Please notice that it was not the governments that went to war, but us, because none of us could to see outside the box of society, beyond the popular trends, untill it was too late 🙂

So, what does this have to do with Google? Well, it means that Google provoked the war in Iraq, Bush told us one thing, and every top ranking page on Google said the same. Only further down could you ever find contradicting information, which we all know today was more accurate. At the same time, now that we have realised that we were mistaken, it is only thanks to Google that we have realised this, and have put Obama in the White House. So this popularity dimension has it’s pluses, but more importantly, it has it’s minuses, and this is the problem to tackle.

In the course of the past century we have all finally grown accustumed to the fact that the human mind is prominant to make mistakes, but somehow we relate that only to mathematical computations and leave complex decision-making up to the dumb human and it’s feelings. If we no longer trust the human mind to make the complex calculations, than why do we still rely upon the stupid mind to wheigh the different facts and their accuracy before we make an important decision?

A search engine that didn’t simply tell us what was most popular, but read, understood, weighed the information, and presented it in a decent, relevant to us manner, so that we then as people could make most rational decisions based on accuracy, and not on simple popular belief. Imagine how many wars could be avoided, how much new successful business could be built, how many market crashes could be avoided, and what a better place the world would be to live in!

Such a search engine seems distant, but it isn’t quite as far away as we may think. It is only half an AI, since it hasn’t to make any real decisions, but simply sort everything that has already been decided, into right and wrong decisions, not simply popular and unpopular truth. Then it can levae it up to us, to believe this machine or not.

The question then of course is, why  would we believe it then? We don’t trust a machine to even drive our cars, why would we entrust it our political decisions? Well, imagine the indexing capacity of this monster, it is the slave, no human could ever be made into, or could duplicate. Reading the immense amounts of information fed into the internet 24 hours a day, 7 days a week. If done right, this guy can make rational explanations way beyond any human’s possible ability, simply because of the bigger “brain capacity”.

There is another aspect to this search engine that makes it brilliant, and that is the fact that it actually helps us make decisions, something we are becoming extremely bad at because of the huge amount of choices and decisions that have to be made, our mind is simply way to occupied with the everyday decisions to make right choices at work, or perhaps the other way around, depending on the person of the case. Explaining the difficulty of this sea of choices would take another 1000 words, so I simply suggest watching this Google talk instead, and imagine thereafter how such a decision-sorting engine could help, if it knew enough about the world and you: http://video.google.com/videoplay?docid=6127548813950043200&ei=5ZskSaGnE6f22gKj1LX8Bg&q=choice+google+tech+talk

No great conclusion for you guys this time, enough to consider already 🙂

Scale is Verb && Verb is Scale

First off, I’ve found a word for my points and dimensions, and feel very stupid that I had to spend so much trying to explain a multi-dimensional semantic net

Prior to realizing the former, I, at some point, took a skim through my newly bought book, Graphic Design – The New Basics, and got caught by two interesting subtitles, Scale is relative and Scale is a verb. Now in design that makes perfect sense, objects must be in harmonic scale with everything else, or not, if it the intention that they stand out, but I saw sth different here.

Now previously I said that everything is a dimension, and every dimension is a point, or put it the other way around, basically everything is intertvined dimensions. But the term scale is relative, made me think that perhaps verbs lie outside these dimensions, and more precisely they prioritise the links between the dimensions, the title scale is a verb only enforced that thought. If you take a look at the diagram of a simple semantic net of a few terms, this is exactly what is symbolised.

Semantic Network

Semantic Network

Or well, almost.

NLP aka. AI

Well, it’s been a while since I posted something, but I guess I’ve been thinking..

Thinking about my highschool project, now named, Natural Language Processing(which changed the goal, as a general, quite a bit). After a few weeks of research, I’ve become familiar with the known problems of bringing NLP to life, and I must admit, these problems look much alike problems within machine visual, aquistic, taste and sensory recognition. This really leaves me wondering, why are these areas of research, regarded as so completely different, when all of them basically require – observing, storing, prioritising and querying the data?(Excuse me if I missed some, but you get my point 🙂 ).

Why are we trying to achieve each one of these goals on their own(where they do seem extremely complicated), instead of creating a unified solution which can be used across the different areas of concerns? I guess the reason for this is the frightful name of such a solution -> artificial intelligence.

However, if you look at the world one detail at a time, as we do with the different aspects of AI, it seems extremely complex and chaotic, if we could find one simple truth, one simple rule, everything becomes simple and harmonic. In just the same way I have come to believe that recognising words and language is exactly the same as recognising shapes and colors, the only difference is the observing channel, what we have now is different terms for essentially the same thing, and that, in my mind, seems utterly stupid.

This leads me to my scary conclusion -> The easiest way to make something that trully can understand and speak like a human, is to create something that is fundamentally same as a human, a so-called artificial intelligence.

On babies

Another aspect that concerns me in AI development, is how advanced we expect the systems to be already at their starting point.

It is simple to understand, that a 1-year-old baby cannot understand a Stanford professor, first it needs to learn, observe and understand, and it must do so in a context close enough to the context of the Stanford professor, if we want it follow along the same lines of thought as the professor. So, why is it so tough to understand, that without having an idea of the world, the AI cannot come with complete and meaningful to us answers?

What we do nowadays is create agents that take a read through the Oxford dictionary, create a semantic net of the meanings of words in the English language, and rely on statistical calculations for everything else. This is like locking up a kid in basement and forcing him/her to read and memorise the dictionary. That creates a relatively good and smart source of information, but it has no understanding outside of that dictionary, which scares us, we are social beings, and send people that lock their kids up to prison(well most societies do at least). And one small detail, how does the kid know how to read in the first place?

Recently a Googler had posted on the official Google blog, that she belived that Google has acieved a rough 90% efficiency in finding what we want, however the last 10% require 90% of the work, and here she’s essentially talking about correctly and ligitimately understanding the human and the knowledge that we’ve preserved on the world wide web. However NLP in information querying is a whole other post 🙂

So perhaps babies hold the answer to the worlds problems. If developers want to create a true AI, they should create babies, not professors 🙂