A Geometric Analysis of Lagrangian, Dual Problem, and KKT Conditions

Preface

This is an article providing another perspective on understanding Lagrangian and dual problem. These two topics are essential to convex and non-convex optimization. Since it is a blog post, the proper background to understand this article is kept rather low. If you need to brush up on convex...

Read More

A Brief History of RNN Encoder Decoder

RNN encoder-decoder has become the next heated field in Natural Language Processing, here is a history of its development. Papers are organized in a timely manner.

Ian Goodfellow’s book has a great overall summary to it: http://www.deeplearningbook.org/contents/rnn.html but sadly since it is a book, it lacks any recent development and...

Read More

A Collection of Numpy Tricks

Numpy Array overrides many operations, so deciphering them could be uneasy. Here are a collection of what I would consider tricky/handy moments from Numpy.

Trick 1: Collection1 == Collection2

The == in Numpy, when applied to two collections mean element-wise comparison, and the returned result is an array....

Read More

Understand Numpy Reshape, Transpose, and Theano Dimshuffle

When you work with Numpy, you work with multidimensional arrays (or tensors). I have to admit such concept was not too easy for me to grasp in the beginning, but after some delibration, it became relatively easy. This post uses the term tensor/multidimensional array interchangeably.

for a tensor of shape...

Read More

Numpy and Theano Indexing and Slicing

As some of you might know, I’ve been working on Numpy and Theano for a while. I can’t claim any expertise, but have found many interesting facets and had to synthesize them together, leading to this post.

Python Double Colon Slicing

The normal kind of slicing can be found...

Read More