I joined IBM as a Summer Intern 2018 in the Data Science Elite Team under Aleksandr Petrov. I remember, on my first day of internship, I was asking Aleksandr, which project does he want me to look into. He smiled at me and said, do what you wish to, research on what your interest is, and build what you know can be a industry product; I was surprised. In all my internship experiences, I used to be assigned to a specific project, and had a very streamlined thought process; whereas, I knew this is going to be a completely different experience.
I finally had the opportunity to work on my long sought-after research area of Reinforcement Learning. I started reading more upon the business use-cases of Reinforcement Learning, and decided to implement the critical and complex use-case of the financial industry - TRADING ! The Chief Data Scientist at IBM - Avijit Chatterjee, agreed to mentor me, with his expertise in the financial sector.
With about a week's research, understanding the intricacies of the finance sector, analyzing the use-case from a traders perspective and scrutinizing it as a Data Scientist, I started to code my first Reinforcement Learning Model. My mind was zealous and adrenaline rush was at its peak, I was excited and eager to see what this piece of code could do.
With 5 weeks of continual effort, the model was ready. The result were looking too good to be true; the model gave a 33% profitability on a 10,000$ of initial investment; whereas, a Buy-and-Hold strategy would have made loss in this year. The best way to observe how the model is making decision was to look at the trades made by the model on unseen data.
Fig: Reinforcement Learning Trading model applied on S&P 2011 data
The graph above shows the action taken by the trading model on each day of the trade. When the model predicts an upward trend in the market, it Buys to Cover any Shorted trades and Buys more shares. Similarly, when the model predicts a downward trend in the market, it Sells the shares it has in hand and goes for a Short Sell. If seen attentively, the RL trading model is able to take accurate decisions in peaks and troughs; taking into consideration the volatile stock market.
Another interesting thing to notice with the agent's actions is that, it is able to capture the head-and-shoulder trends in the market. The model seems to be promising and will be a property of IBM. With this completion of the project, I am really grateful to my manager- Aleksandr Petrov, the Chief Data Scientist - Avijit Chatterjee and my mentors - Grace Yang and Jesus Rios.
To know more about the project, follow Inside Machine Learning- IBM, and read the following posts of the Reinforcement Learning- The Business Use-cases series.
- Introduction to Reinforcement Learning
- Reinforcement Learning in Finance - Trading
Follow me on Medium!