Classifying financial time series using Discrete Fourier Transforms

A financial time series represents the collective decisions of many individual traders; it seems reasonable to me that the nature of these decisions may differ based on the underlying asset. For example, a company with a higher market cap may be more liquid, and subject to larger individual buy/sell orders including institutional investment. Thus, there is a case to be made that information such as the market cap of a company should be ‘encoded’ into its price movements. While these characteristics may be difficult to pinpoint on a chart, in principle it may be possible for a machine learning algorithm to find statistical relationships between the time series and the market cap of the company. This post will investigate that claim. Read more...

DIY MachineLearningStocks

I recently released a machine learning stock prediction project on GitHub, unimaginatively named MachineLearningStocks. It is a project that I’ve put quite a lot of time into, and is in fact a simplified version of a system that I’ve been using to live trade. This post doesn’t really offer anything on top of the existing readme, but I figured it would be good to have a copy (with some minor changes) here as well. Read more...

Creating a stock price database with MariaDB and python

In this post, I describe my thought process in designing a simple relational database to store historical OHLCV for equities, including why I chose MariaDB and code for populating the database. Read more...

Learning Machine Learning

In this post, I present my opinion to a question I frequently get asked: how do I get started with machine learning? Read more...

Gradient tree boosting and XGBoost

Decision trees make for pretty vanilla classifiers: they do an unspectacular job with most machine learning tasks, and you’d be forgiven for overlooking them when deciding on a classification algorithm. But decision trees happen to be the cornerstone of a powerful class of learning algorithms: gradient tree boosting methods. I will try to elucidate the (short) history of gradient tree boosting, starting with the pioneering implementation of boosted trees and ending with the state-of-the-art. I’m afraid there will be a hearty amount of mathematics, but it is really a very intuitive topic at its core. Read more...

8-bit Julia set art in python

You may have heard a mathematician or physicist (or more likely your maths teacher) describe mathematics as beautiful. What could they mean by this? There is just something mysteriously attractive about the purity, complexity, interconnectedness, and underlying truth of it all (“Beauty is truth, truth beauty” - Keats). Read more...

Retrieving historical stock prices from Yahoo Finance with no API

Yahoo Finance has long been an excellent free financial resource with a wealth of data and a convenient API, allowing open source programming libraries to access stock data. But not any more. As of May 2017, they have discontinued their API, probably as a result of Yahoo’s pending acquisition by Verizon. This means that excellent tools like pandas-datareader are now broken, much to the dismay of many amateur algorithmic traders or analysts. It turns out that there is a rather hackish workaround which allows us to download the data as CSV (i.e spreadsheet) files, which of course can then be read into excel, pandas dataframes etc. Read more...

The Leibniz integral rule in electrodynamics

I have been slowly working through David Griffith’s widely-used textbook, Introduction to Electrodynamics. It is known for having a large number of reasonably difficult exercises, which are instrumental in conveying some concepts not directly addressed in the main text. I found an elegant shortcut to one of the questions, which was not noted in the solution manual. The shortcut involves (ab)using a very versatile, if slightly overlooked theorem in calculus – the Leibniz integral rule. I will briefly discuss this, before working through the problem. Read more...

Intuiting the gamma function, part 3

We now pick up where we left off and fully elucidate the link between the factorial and the gamma function. Read more...

Intuiting the gamma function, part 2

In Part 1, we showed that repeated differentiation gives rise to a factorial. In this second post of Intuiting the gamma function, we are going to show that integration by parts can also produce a factorial – an instrumental step in generalising the idea of a factorial to non-integers. Read more...