See Demo

Go to the Demo Web Site

Linear Regression with Gradient Descent

• Linear Regression, a machine learning technique borrowed from statistics, refers to the process of fitting a linear equation (y = mx + b) to a set of training data (x being the input, y the output). Predictions can be made for new input data by feeding x into the equation and solving for y.
• Multiple Linear Regression refers to the scenario where there are multiple input variables, typically listed as x0, x1, x2, and so on. In this case the equation to solve for is: y = m0x0 + m1x1 + m2x2 + b.
• Polynomial Regression refers to the process of fitting a polynomial equation to the data. This is useful when there is not a linear (straight line) relationship between x and y. An example of a polynomial equation (of order 3) is y = a * x^3 + b * x^2 + c * x + d. There is also exponential, power law, logarithmic and more types of regression!
• While most regression problems can be solved using statistics, for example with a method known as "ordinary least squares", in the case or large data sets with many input variables, the equation cannot easily be estimated in one fell swoop. This is where the machine learning technique known as "gradient descent" comes in. Gradient descent is an algorithmic process which adjusts the parameters of a model according to errors to minimize errors. Performed iteratively overtime, a linear or polynomial regression equation can be estimated. This technique will serve as a fundamental piece of building a neural network simulation and so while it is unnecessary in the simple linear regression examples for this week, it's the perfect place to learn and practice the algorithm.
• Rebecca Fiebrink's Machine Learning for Musicians and Artists Session 2
• Chapter 3 of An Introduction to Statistical Learning (ISL)
• Week 1 of Andrew Ng's Machine Learning Coursera course
• Tom Alexander's JS Regression Library
• A video tutorial of the math and implementation by Jyo Pari