How can we apply artificial intelligence to the financial markets
What is artificial intelligence and machine learning and can we apply it to the financial markets?
It took us 3 and a half years of research and development until we finally reached a point we can trust our software.
Obviously you can find all sort of information on the internet about machine learning and AI, like these articles on Wikipedia for example, but the concept is quite simple: You run an algorithm (there are many) on the set of data, and once the algorithm is finished, the software will know how to run by itself on new sets of data, even if it’s never been seen.
There are 2 types of algorithm methods –
- Supervised – Similar to training a dog: if it does good you pet them, if it does wrong you scold at them. After a while, they will learn how to behave
- Unsupervised – This is the most interesting algorithm out there. This means you give the algorithm a set of data but you DO NOT tell it what is wrong and what is good. It does it by itself.
So, can you apply those algorithms in the financial markets?
First, let’s start by learning a bit about how ML (Machine Learning) and AI (Artificial Intelligence) work and its purposes.
To create simple computer software, we need to insert some scenarios we want it to handle, we add the way we’d like the software to act, and let it run.
A “stupid” software will ONLY KNOW HOW TO WORK according to the scenarios we entered and taught it.
An AI software will take the same scenarios we entered and ways to behave we told it to, and will be able to do it NOT only on the ones we told it to but also on SIMILAR scenarios.
This is basically why AI and ML are the future in any way you can imagine – Because it’s not limited to what the programmer writes in the code, but also it can adjust and act to things that aren’t inside its code and also, over time, will be smarter in handling situations only by itself.
OK let’s go back a bit
Scenarios? Ways to behave? WHAT??
Say we got a lifetime doctor’s records of some people. They are anonymous, of course, because we don’t care who they are. We only care about their DATA.
Now we want to find something, like, maybe, can we find cancer disease BEFORE the person knows it’s happening – or in other words – Can we predict cancer?
We can check – are they the cigarette smokers? If yes, how many had cancer?
This has been the way until now.
You probably can already guess why it’s not merely enough.
If they don’t smoke – does that mean they won’t have cancer? We already know it’s not true.
And sadly there’s a variety of cancers to almost every organ in the human body (cancer is when some cells of our own body stop dying unlike the other cells and the body starts to attack them. Basically, nature makes our body suicide from inside).
So what can we do if we want to predict cancer?
It’s simple – We take into consideration as many parameters we can. Like:
Age, gender, place of living, place of working, family history, doctors’ appointments, and medical record, food and drink habits, etc.
Those are the objective data.
We need also subjective data such as happiness in life, the scale of pressure, type of person, etc.
Once we have ALL this data for every person, we need to do 3 things:
- Check which one of the parameters can, in fact, be some kind of prediction to cancer
- Run a statistics machine learning algorithm (like Naïve Base)
- Use the results to solve a worldwide problem
We wish, right?
Now we get on to the problems of artificial intelligence (AI) and ML:
- Data
Data is extremely difficult to collect, and then to manipulate. In our example to get these data, we need to cooperate with medical services to get their clients’ data, create a questioner, and send it to all the clients and analyze the data. Though there is such cooperation around the world, it’s still not easy to also get subjective data.
- Analyzing big data
Big data has become a known word around the world.
There was a time companies said they work with big data and clients threw the money at them.
But it’s not that simple. Every data you add for the algorithm to learn from – increases exponentially the time for the software to analyze…
Inefficient software may take a very LONG period of time to run.
Funny personal anecdote, our first AI software we developed to learn how to predict price changes in the stock market looked so genius at first, but after we started running that artificial intelligence and measuring the time it will take to finish, we saw it will take no less than 27,000,000,000,000,000 years from now(!!) Obviously, we couldn’t wait, and in future articles, I will explain how we lowered it to only a few hours running time.
Let me give you an example of the difference between Big Data and just simple data with a game:
I chose a number between 1-1000. You have to guess which one is it. But there’s a catch – you need to find the number in as little time possible. How would you do it?
Think about it for a second.
Got a solution?
If you guessed that you should ask me “Is it higher than 500?” and then according to my answer (If I chose the number “990”), the answer is yes. Then your next question will be “Is it higher than 750″… You get the point.
That’s easy, right?
What if you got a number with 80 digits? Then it might take a long long time until we break this number, maybe even months. And that’s only one running time. What if we need it to create strategies for trading and investing and we need it to go over millions of possible strategies?
It will take a lot of time.
As humans, we can’t really comprehend really big (or small) numbers. Like these two questions, I like to ask people once I talk about large numbers.
- If 1 million seconds is 12 days, how much time is 1 billion seconds?
- And, if your salary is $100,000 each month, how long will it take until you reach 1 billion dollars (say you can save all of it each month)?
You can easily calculate it, but it’s an intuition question, not a math one. Think for yourself, what’s your intuition answers are? The answers will be later on in this article.
So we’ve talked about what’s machine learning algorithm and a bit on big data problems.
Now, can we apply artificial intelligence to the financial markets?
In short, yes.
But it’s easier said than done.
It took us 3 and a half years of research and development until we finally reached a point we can trust our software.
Because other than the ML and big data problems, we face a whole different problem in the field of financial markets, since they act like in a chaotic environment it makes predicting a lot harder.
And, (and it’s the most important and) because of the spread whenever you enter a position you face an average of 56% against you.
That’s probably the time to say there are two kinds of players in the financial markets:
- Investors – They invest their money for years ahead and they gain the average rate the market makes (around 8% a year). By the way, according to decades of studies, there’s one stock that if you’re an investor you should put all your money on, and that’s the S&P500 stock (Symbol SPY). In another post, I’ll prove this fact.
- Traders – They usually use time limit (options) or profit/loss lines (if it reaches +X get out with a profit and if it reaches -Y get out in a loss)
We are on the traders’ side.
We want to gain more money, faster, and more chances of getting out in time.
But unlike investors who buy now and then forget about it, as traders we must beat not only the commissions our broker offers us but also the spread (the difference between the lowest price a seller is willing to sell and the highest price a buyer is willing to buy). The spread is usually set by the broker and it’s one of the best ways for a broker to gain profits.
So, we also know that like in gambling the house always wins, so as in the financial markets – the broker’s always gaining profits.
Back to our financial algorithm – we found a broker service that lets us collect the financial data, and we’re saving it. Now, we need to analyze it to find patterns. But how?
In an everyday changing environment, how can we rely on anything?
We solved that problem by relying on our algorithm on behavior analysis. We figure that even though the market can change, the forces that control it (the investors and traders) will stay the same (Obviously, they change too, but way slower).
So we’re talking about collecting on average millions of data and parameters a day for each stock. Once we try to collect 1000 stocks for a few years time you can imagine how much data is inside, so it’s just a matter of creating a super-fast unsupervised machine learning algorithm with only one rule: The most money you can make is the better – and let it run and find the best way to trade by itself.
Creating artificial intelligence
In conclusion, it is possible to create an automatic software or some artificial intelligence to trade for you in the financial markets, but it’s EXTREMELY difficult. You need to overcome many problems in serval fields in order to do it. And after you do it, it’s unlikely that you will let anyone use it.
But we’re different. We will let our subscribers use our algorithm for free, just to have a sense of how it works.
Subscribe now to get more information about artificial intelligence in the financial markets and to get informed once our algorithm is ready for outside users.
Our software will let you choose which assets you want to buy, and when – and it will tell you when to get out. Simple, yet important.
By the way, the answers to the question before are:
- One billion seconds are 32 years
- It will take 830 years to gain one billion dollars if your salary is 100K per month
Was that your intuition?
Sign up below to our newsletter for a free test drive on our trading algorithm! Find more about artificial intelligence.
Top Image Credit: Photo : iStock/MF3d
This is not something you can do over the weekend and let run happily ever after.