Sub Conscious Learning System

I had a thought, today. It was for a type of learning which seemed quite natural, but I had not even imagined being able to implement, previously. It is a system for self-scoring. As you might know, in general evolutionary algorithms, a computer program is given a score based on how well it functions, and it's children carry on that score, and build onto it. Eventually, after many generations, it gets smarter and smarter. However, with this system, if forces the entire species to lean as a whole, rather then individually. This works perfectly fine if the algorithm is designed to only master a single task, even if it's complex. But what about more complex tasks?

How could an artificial intelligence agent be designed, so that it can make it's own choices. Score it's own actions and abilities. This is an important trait for agents who want to have any hope in thriving in a complex environment, where more then a single action must be scored. This is a critical base for higher level thinking machines. Machines that learn strategy, and can plan out actions ahead of time. Taking into consideration events which only they specifically have encountered, not their previous generations.

Here is my proposal. For this system, the original generation based learning algorithm is still in place. This judges actions solely around things such as survival of the individual, and expanding the population. But there is also another learning system in play, here. The second learning system is per agent, and effects only their brain. This is constantly updating, as well.

In this system, imagine we have an entire AI brain network. Neurons and all. Now, take a chunk of it. Say, 25% or so, and mark this as the subconscious. This part is unable to be edited by the agent. In addition, this chunk is close to the end or the network, and all linked together closely. Yes, there are many connections going in, and also several going out. Though, in the middle of these, there are "emotion" neurons. These release emotional responses based on their input. Positive input, results in a positive output. Negative input, results in a negative output. Highly positive input, results in a highly positive output. And so on. Now, these emotional responses are the "score" values for the agent. It wants these to be as high as possible. So it is constantly weighing in different neuron values for the rest of the brain to try to influence the emotions. Now, the main evolutionary algorithm still wants the brain to function a certain way, to accomplish it's overall tasks. Survive, and thrive.

When a new agent is born, it's default brain state is given to it by the evolutionary algorithm. It also decides which parts of the brain are editable, or not. (This network is also assuming the same setup as the one I described in my article "Smoother Node Map Learning".)

By doing it this way, the agent is allowed to learn and adapt to it's environment at it's own pace, while the main algorithm gives a general path for everyone to follow. Both algorithms fight each other, and also work together to achieve a goal.

As each agent is it's own brain, and learning path, each one will have it's own personality, goals, and so on. Each will behave in their own way, but still follow the same overall instincts given to them when they are born.

I am quite excited to see how this plays out, so I will be playing with the code for it. (Though most likely I'll just get lazy and never finish it, as is the case for most of these ideas.)

Why can't machines feel emotion?

This is a heavy topic, in my opinion. This is something that has been thrown around as a artificial intelligence stereotype since the topic of artificial intelligence via computer was thought up.

Maybe I should start with why this bias against machines was first used. In early days of computers, it was assumed that no matter how much math a computer could do, a human would always be more intelligent. Eventually, a computer was designed specifically to play chess. After beating a professional chess player with little effort, computer scientists began to see that there really was potential in computer programs surpassing human level intelligence. The idea of AI has been thrown around for centuries. Even Leonardo Da Vinci once built a robot (armor, that was controlled by a series of ropes and pulleys), to simulate AI. But now, this seemed like it was actually within reach.

Years later, the first "chat bot" was a created. A program that was designed to respond to human sentences, in a was they was comprehensible. Many people even thought that there was just talking to a regular person over a computer. However, though the program formed correct speech, and was able, to an extent, respond correctly to human input, the responses all seemed emotionless and heartless. This is where the famous serotype that all machines were "forever emotionless."


I disagree with this, completely. Though, to explain my logic, I should explain what "emotion" is. Or at least what it is in my opinion. I believe that what we think of as emotion, is nothing more then a complex scoring system. Whether you believe in evolution or not is irrelevant for this point. Simple adaptation, a quick and observable event, is all the proof that is needed to convey this reasoning. People who are loving are more likely to have families, kids, and relationships, etc. Thus, they can pass down this mental trait to their children, grandchildren, and further down the line. So imagine if a person was unable to feel. They would have nothing to motivate them to live. To encourage them to have families, or take care of those families. Their chances of having children are significantly low. Even if they did, their children would share this trait and the pattern would continue until the trait finally woven out of their family tree, from the mental traits of the other parent.

So emotion is definitely important to keep a family tree going. It can also been observed that more emotional people can also strengthen this point, as they are even more encouraged to have families, and similar. So emotion is a key factor in creating families. Ok, simple enough.

Let's move on to another point. Emotions such as fear, happiness, anger. With fear, fear keeps people alive. People with little fear are more likely to do stupid stuff, and get themselves killed. Thus, fear keeps you alive longer, and gives you more time to start a family and carry on that trait of fear to your children. Anger, gives you a boost of adrenaline to fight off threats and protect yourself, or loved ones from what your brain sees as danger. (Mental or physical.) Happiness, pleasure, and similar, gives you an objective to work towards; a purpose in life. A feeling of pride when you hold your child, or a gleam across your face when you see someone else you care about smile. That motivation also keeps us alive as a society. The drive to build better tools, stronger shelter, faster means of travel. Each of these helping the survival of humans immensely. Sadness, tells us points or things to avoid. Pain causes sadness, and pain hurts the body. So stay away from what causes you pain. Better, and longer life. Etc. I could keep going for hours, but I feel I've made my point.

Emotion has given us the ability to learn as well. Imagine if a baby was born without the ability to feel emotion in any way. It wouldn't be able to learn even simple tasks, as it would be unable to answer the simple question, "why?"

"Why should I walk? Why should I read and write? Why should I do this instead of that?"

Without being able to understand why it needs to do something, it has no reason to do it. No push. No encouragement. No remorse, no regrets, nothing to aim for, or avoid. Pain would mean nothing. Pleasure would mean nothing. It would uncaring do dangerous things without caring if it was hurt in the process. Simple just because it could. There's also a good chance it wouldn't even move. As why bother wasting the energy?

Emotion is critical in learning. And in social situations, more emotion is much stronger for learning, and survival as a whole. So is it not sort of like a goal? People always seek out to please their emotions. Avoid pain, seek pleasure, try to be happy, protect yourself with anger or fear as necessary, etc. In simple adaptation, traits that are better for a being's survival are more likely to be traced to their kids, grand kids, and enviably everyone generations down the road. Almost like a score on how helpful that trait was.


AI is no different. It has a goal, and gets a score based on that goal. If it gets a low score, it's sad. A higher score makes it happy. It is constantly surviving towards whatever makes the program said makes it happy. (I.e. Gives it a higher score.) Using multiple score situations, the AI basically can feel multiple emotions. The current score being it's current emotion. Tracking scores over long term could even translate towards long term goals, hopes, or dreams. As each AI learns in their own way, they each have their own personalities, and ideas of what makes them happy, and what doesn't. Depending on how the scores are set up, AI can have the exact same emotions as a human can. Or more, or less. Or a set that's completely different from ours in every way.

So why can't an AI feel emotion? It very well, can.



So, did I screw up somewhere? Disagree with something, or simply want to add your own input? Go ahead and leave a comment. :)

Pathfinding as Better AI

When most people think of pathfinding algorithms, such as A Star, they think of simply finding the shortest distance between two locations on a map. Although this was it's original purpose, it can be so much more then that. A lot of implementations of pathfinding take advantage of the movement cost to make the paths more realistic. Such as avoiding wall-crawling, or preferring to walk on the sidewalk instead of grass unless necessary. These are very simple operations, but yet have already rocketed the simple find-the-shortest-path to a more natural, actual walking pattern.

But why do we have to stop there?

Let's build onto this a little bit. Instead of messing with solely the cost to move to a specific node, let's add more nodes. In addition to nodes like "move north," "move south," etc. Let's add "sprint forward," and "jump forward." Then adjust the cost to include time taken, and energy consumed, and demand to get to the end as quickly as possible. By doing this, more complex paths are available. Enemies will walk around normally, but when a player starts to walk by, they will rush towards you. Possibility breaking into full sprint to get to the player if they need. Adding a maximum energy could even persuade the enemy to give up, if it becomes too difficult to reach the player. Imagine how realistic monster movement would be now!

We can go even deeper.

What if we add even more complex functions. Such as "wait," "swing sword, " "shoot bow, " "drink healing potion." At this point, imagine how much more complex the creature's intelligence will become. It could actually hide behind a pillar out of the player's sight, and shoot a bow when they're not looking. Or charge up and hit you with a sword, then run. The possibilities quickly become immense. Simple enemies are become something to fear, and computer controlled pets become the most important companions.

Pros and Cons?

Well, there are a few of each. For pros, it adds a very high intelligence simulations. And it's highly flexible.

Cons, wells, balancing the point values can be a little tricky. I'll explain how to work with scoring, below. But the biggest issues are calculating heuristics, if applicable, and performance.

Performance is something that becomes a huge resource on this. As so many more dimensions are being explored. Working even on a simple grid instantly becomes a massive chore for the CPU. In addition, if heuristics, which are already very difficult to calculate, are done incorrectly, the computer could be wondering around for quite a while before finding an acceptable path. As for heuristics, it's much harder to calculate, as now multiple paths are being explored, such as "should I sprint or not?" This effect can hugely break the outcome for algorithms that depend on it. A star can get away with it, by always setting the value to 0. Though it will take longer to find a path. (Though it will always be the most accurate path to meet your needs.

Scoring is a little weird, but easy if you use square mean. Basically, make sure all values you're scoring, raise about the same way. (Such as the all have roughly the same starting and ending values. And none level up faster then the others.) Now, for each node, get all scores for it. Raise all of them to the power of 2. And find the average. That is the cost for this cell. :)

Now, much sure for make each cell store as little memory as possible. In really difficult situations, it may take many, many nodes to find a result.

In summary, this is my proposal. It sounds great in theory, but I'll have to test it some to see how practical it would actually be in real AI. Thanks for reading, and I hope this gives you some ideas for your game/program/whatever. xD

Learning to Learn

One really interesting thing about node map learning, it it's flexibility. Given the right functions, and enough time, the node map can be theoretically capable of learning to preform any task. Though this can be very slow, and very daunting process. So why not teach itself?

A quick Google search will show how powerful learning algorithms are. Amazing tasks can be taught to the AI. Some preforming calculations far more powerful then previously thought possible. Other things can be taught to. Such as creating the fastest car, or the best fighting stick figure. Well, that's awesome. But I'm sure it took a while to get to that state using only self learning methods. And it does. And when thrown into a new environment, it can take a very, very long time to make the correct adaptations. Our algorithms are good, but not that good.

So let's let the AI figure it out. AI algorithms are already able to define the perfect algorithms for specific tasks, to a point that far surpases that of any human. Why not let the AI find a perfect learning algorithm?

The concept behind this is to have a "master" node map which is constantly learning and expanding it's knowledge of the functions it has, and how to use them. And use that to generate possible learning algorithm maps. Then send these maps out into a large series of varying tests. (There must be a large number of them, and they must be very diverse in order to minimize pattern finding and exploiting.) It wouldn't do every test, but a random selection of them. This helps to prevent the node map from excelling in a specific test to boost it's score. Then return the results over time for each test. Take all of the results from each test, and find the average learning curve. The idea for the master node map is find and create an algorithm that generates the highest learning curve. This process is very time consuming, and takes many, many, many more tests and passes then usual. That's because all of the learning is done ahead of time. More tests will have to constantly be added, and do to randomness, and test weaknesses, learning should not cease. This new learning algorithm would be constantly getting better over time, until it can finally surpases all of our current learning algorithms by far.

Node Map Learning, and Other Learning Functions

Now, node map learning is functional, and is capable of learning on it's own. Though, through the trial-and-error-like method of learning, any function that has non-static data could screw up the whole learning process, right? Surely something as constantly changing as another learning function would slow all learning to a crawl, forcing the node map to take many more passes to travel the same distance, right? Well, yes and no. Though this logic is correct, to an extent, it actually would really help the AI.

The best way to think of this is like a business. The AI represents the entire company. The node map is the CEO of that company. Each function you hand the node map is another employee type working under the CEO. The types in this case are not actual employees, but are "job titles" which can be filled by multiple people, or simply not used at all. The CEO's job is to get the business operating as well as possible with only these job titles and no more. The only thing the node map (CEO) is able to do is choose where in the company to put each job title. So when the process first starts, he'll look at things, and make logical guesses; passing data from person to person. The thing about learning algorithms in this: they are adaptable. Employees like this have no special talent in a single field, which is difficult for the company to find them a position, but they can adapt to any position, given enough time. Like a business has managers, and people working below them, and other managers, and even more people working below them, node maps can be nested. All forms of learning algorithms are able to be placed inside of a node map and nested. Though, how these are nested is up to the node map.

As stated above, a potential issue with using functions that operate in a non-static manner are likely to throw off the node map's calculations. This is true, and will always slow down the node map's learning curve. In the example of a business, the CEO is putting different people in positions everywhere, where they function poorly. So it moves them. This repeats, a lot. And because a learning algorithm can only learn if it is not moved from it's current position, the curve only has a change to grow when the node map is focusing another another area, instead of the learning algorithm. When the node map places a learning function higher up in the chain, then if has a less chance of being changed, and thus given a much larger chance to learn and grow.

At this point, all of the functions and learning algorithms, including the main node map, are all attempting to work together in order to master their environment. Each one will often trip over each other, or misread data from each other. But, they are help each other. A single algorithms success significantly improves the success of the others. A single algorithms failures can be patched by the others. It is important to use learning algorithms inside a node map, because of this. It may slow down things at first, (by a lot) but it really speeds things up in the long run, and gives the node map much more flexibility.

Node Map Learning

Artificial Intelligence is an amazing topic. The whole concept of a computer program learning about it's environment and adapting to it is amazing. I've always loved AI, and have made many attempts to reproduce it. Though I have run into numerous problems along the way. Mostly due towards my lack of knowledge on the subject, and because of the ridiculous amount of work involved to make such a program. Though this is to be expected. AI is not a simple subject. There's so much involved in it, that it's almost necessary to use large teams to design anything of a decent scale. Though it can be done. So why don't we have AI around us commonly if it possible? Well, in a small sense we do. Neural Networks for example, are used at banks to help read off hand writing into computer text, and can do so with over 99% accuracy. But this isn't the type of machine learning I am referring to. What about emotions? Self set goals? These things are possible, and have been created in several instances. But the design behind them almost seem primitive compared to even small animals. This is because even though we have many different algorithms to aid in machine learning, they all have one thing in common. They are slow. It will often take many, many, many passes before the computer finally gets anywhere logical. Yes, in a sense the human mind is the same way, taking the first several years of our lives to learn about how to interact with the world around us. But we can't wait that long. For a computer, an environment on a scale such as ours would take almost centuries to understand, and lots of storage space. Not to mention a ridiculous CPU to calculate decisions in real time. Resources we don't have to spare. That's why AI is such a largely studied field. It's constantly coming up with new ways to speed things up, and make it more efficient. That's why I'm here. I proposing a new type of learning method which I hope will make learning on a mass scale much easier. Though I do not fully understand every part of the algorithm yet, I hope to keep fleshing it out more and more over time. This information right here is a basic summary of the algorithm, rather then any detailed math behind it.


The Node Map Learning method is machine learning concept which is hopefully better at understanding much larger and more complex environments then regular algorithms such as Neural Networks and Genetic Algorithms. It works by basically creating it's very own node map, (which functions similar to a program's code, shown right) then filling this node map with many, many functions. Anything that can be given to it. This includes stuff like distance between two points, sorting a list, adding numbers, and of course, actual functions for interacting with the environments. The AI takes all this data, and tries to organize it into a complex tree of data processing to work with such. The tree is constantly changing and evolving. Some of the functions given to the AI might be things such as Neural Networks or Genetic Algorithms, in which case it would also use these functions, and as the AI operated, would also learn to make better use of these functions. One last thing to note, is that each limb of the tree, (the data that comes out of functions, and gets thrown into other ones) must have a "type" assigned to it. Such as "number". This means that a function which accepts 1 input in the form of a number can only accept data that is assigned as a type of number. Data types can extend other data types. Such as "integer" would extend "number". Thus, all integers can be used as numbers, but not all numbers can be used as integers. The tree grows constantly as well, becoming more and more complex as needed. This process occurs in a similar method to genetic algorithms. The AI will semi-randomly (see paragraph below) start testing out different tree designs by adding, removing, or replacing branches on the tree. If the new design functions better, use that tree instead. If not, revert to previous tree. This process continues infinity, until there is absolutely no better way of designing the tree, without completely remaking it.

It chooses a base design by randomly creating X number of completely random tree designs initially. (X is determined by the complexity of the environment, though is usually around 100 or so. Though can be much higher for intricate environments.) Then it tests all X concepts, and looks for patterns in each one, and compares these patterns to how well it did. For example, the AI may test 250 designs, and looking at the designs, trees the used the pattern "Function 1 > Function 2 > Function 3 and 4" tended to have better results, while trees that use the pattern "Function 1 > Function 4" seemed to function very poorly. Taking all the patterns into account, it designs a new tree a sees how it functions. Constantly testing it, changing it, and making it better. Each and every new tree that is tested is broken down into a list of patterns and given a numerical value based on how well it functioned. This builds up after a while, and eventually will give the AI a very clear idea of what kind of designs work better then others.

This is my concept. I hope to have a working proof of concept out later. Thanks for taking the time to read this, and follow to keep updated. I will be posting more on the subject soon.