Linear Regression

Great article! Reblogging it for future reference.

Math ∩ Programming

Machine learning is broadly split into two camps, statistical learning and non-statistical learning. The latter we’ve started to get a good picture of on this blog; we approached Perceptrons, decision trees, and neural networks from a non-statistical perspective. And generally “statistical” learning is just that, a perspective. Data is phrased in terms of independent and dependent variables, and statistical techniques are leveraged against the data. In this post we’ll focus on the simplest example of this, linear regression, and in the sequel see it applied to various learning problems.

As usual, all of the code presented in this post is available on this blog’s Github page.

The Linear Model, in Two Variables

And so given a data set we start by splitting it into independent variables and dependent variables. For this section, we’ll focus on the case of two variables, $latex X, Y$. Here, if we want…

View original post 2,899 more words

The Fast Fourier Transform

Math ∩ Programming

It’s often said that the Age of Information began on August 17, 1964 with the publication of Cooley and Tukey’s paper, “An Algorithm for the Machine Calculation of Complex Fourier Series.” They published a landmark algorithm which has since been called the Fast Fourier Transform algorithm, and has spawned countless variations. Specifically, it improved the best known computational bound on the discrete Fourier transform from $latex O(n^2)$ to $latex O(n \log n)$, which is the difference between uselessness and panacea.

Indeed, their work was revolutionary because so much of our current daily lives depends on efficient signal processing. Digital audio and video, graphics, mobile phones, radar and sonar, satellite transmissions, weather forecasting, economics and medicine all use the Fast Fourier Transform algorithm in a crucial way. (Not to mention that electronic circuits wouldn’t exist without Fourier analysis in general.) Before the Fast Fourier Transform algorithm was public knowledge, it simply…

View original post 2,410 more words

How to know your python code is PEP8 compliant?

Everyday, I am learning new things about this awesome language. One amongst the great things it offers is its readability and how it is ingrained in its syntax. It really makes newbies write readable code. This post is about how to know whether your code is PEP8 compliant. PEP8 is the standard code styling guide followed by python developers worldwide.

  1. Install the command-line program pep8
    $ pip install pep8
  2. Execute it for the python file, you want to check, to get the code violations. E.g.
    $ pep8 test.py
    test.py:7:1: W191 indentation contains tabs
    test.py:9:1: W191 indentation contains tabs
    test.py:10:1: W191 indentation contains tabs
    test.py:14:80: E501 line too long (91 > 79 characters)

That’s it! Its as simple as that (simplicity – another great feature of python). You can run it on all of your files together (or on a series of files) :

  $ pep8 *.py

Disadvantage: It will give a long list of violations if you are new to python. I recommend running it after every python file you write. Its more manageable that way. Once you are experienced enough, then the above technique will be helpful since the number of violations would be lesser.

References
Python Guide: Code Style

How to give a great research talk

Here is a summary (read: notes) on the talk given by John Krumm and three other researchers from Microsoft Research back in 2007.

This video gives a good overview about the things which make up a great talk. The Q&A at the end also has a good set of points which are beneficial. It is given by four researchers including John; each speak for approx. 10 minutes. Target audience is a group of researchers working at Microsoft. So, its good for experienced researchers too.

John Krumm starts with why is it important to give a good research talk; goes on to talk about what to say, how to say it, how to chair a session (9th minute) and how to end a talk. Important takeaway: practice; make good eye contact. Definitely listen to his talk in this video.

The second speaker (Patrick) talks about the importance of visuals. Important takeaways: use images/demo/videos in the beginning of the talk, use one image per slide to show related work, use images from the paper to explain concept and use visual tricks to emphasize most important things in an image and make sure you have some interesting takeaway to give e.g. something for the audience to think/learn from the talk. If you don’t want to go through his entire talk, skip to the 25th minute; he gives a summary of all the important points.

The third speaker (Rick) (from the 30th minute) talks about visuals from a different perspective. To illustrate, he uses one of his talks from a conference he presented. I skipped the beginning of his talk since I didn’t find anything useful but listen to the Q&A for his session. Some takeaways: acknowledge your collaborators in the first slide. Important thing to note: He is a great speaker. Notice the way he organizes his speech, keeps a balance and the manner in which he speaks.

The fourth speaker (Mary) (from the 45th minute) gives a list of pointers to keep in mind when giving a talk. She starts with the need to have stellar results to talk about, practical implications of the work being presented; she also gives some tips to prepare for rude Q&A (52th minute)Important takeaway: practice! practice! practice!; proper voice projection; good eye contact; be energetic; good body language; good body posture and balance; prepare for the worst – get questions from the people to whom you give your practice talks.

Some key questions from the Q&A at the End:  What is the difference between job talk and conference talk? (from 57th minute) When to memorize and when not to? There are other good ones too. So be sure to listen to them.

Lastly, pick up tips from all the speakers – from the way they give this talk on “giving a great talk”! 😀

l0-Norm, l1-Norm, l2-Norm, … , l-infinity Norm

Helpful for beginners

Rorasa's blog

I’m working on things related to norm a lot lately and it is time to talk about it. In this post we are going to discuss about a whole family of norm.

What is a norm?

Mathematically a norm is a total size or length of all vectors in a vector space  or matrices. For simplicity, we can say that the higher the norm is, the bigger the (value in) matrix or vector is. Norm may come in many forms and many names, including these popular name: Euclidean distance, Mean-squared Error, etc.

Most of the time you will see the norm appears in a equation like this:

$latex \left \| x \right \|$ where $latex x $ can be a vector or a matrix.

For example, a Euclidean norm of a vector $latex

a = \begin{bmatrix}
3
\\
-2
\\
1
\end{bmatrix}$ is $latex \left \| a \right…

View original post 1,669 more words