Posts by Tags

argmax

backpropagation

Reverse Mode Differentiation with Vector Calculus

5 minute read

Published:

This is something back in 2016 when I was struggling with my first serious neural model. Previously I mostly use Scala for NLP. But there was no auto differentiation locally in Scala, so we ended up thinking to write our own package. The task, naturally, turned out to be waaaaay too non-trivial, and we soon gave it up days after. Well, anyway, here is the wrap-up.

conceptnet

differentiation

Reverse Mode Differentiation with Vector Calculus

5 minute read

Published:

This is something back in 2016 when I was struggling with my first serious neural model. Previously I mostly use Scala for NLP. But there was no auto differentiation locally in Scala, so we ended up thinking to write our own package. The task, naturally, turned out to be waaaaay too non-trivial, and we soon gave it up days after. Well, anyway, here is the wrap-up.

gradient

Reverse Mode Differentiation with Vector Calculus

5 minute read

Published:

This is something back in 2016 when I was struggling with my first serious neural model. Previously I mostly use Scala for NLP. But there was no auto differentiation locally in Scala, so we ended up thinking to write our own package. The task, naturally, turned out to be waaaaay too non-trivial, and we soon gave it up days after. Well, anyway, here is the wrap-up.

manual

Reverse Mode Differentiation with Vector Calculus

5 minute read

Published:

This is something back in 2016 when I was struggling with my first serious neural model. Previously I mostly use Scala for NLP. But there was no auto differentiation locally in Scala, so we ended up thinking to write our own package. The task, naturally, turned out to be waaaaay too non-trivial, and we soon gave it up days after. Well, anyway, here is the wrap-up.

reimplementation

Model Reproduction

7 minute read

Published:

Model reimplementation is both a pain and a great source of learning.

reproduction

Model Reproduction

7 minute read

Published:

Model reimplementation is both a pain and a great source of learning.

reverse mode

Reverse Mode Differentiation with Vector Calculus

5 minute read

Published:

This is something back in 2016 when I was struggling with my first serious neural model. Previously I mostly use Scala for NLP. But there was no auto differentiation locally in Scala, so we ended up thinking to write our own package. The task, naturally, turned out to be waaaaay too non-trivial, and we soon gave it up days after. Well, anyway, here is the wrap-up.

softmax

sparsemax

vector calculus

Reverse Mode Differentiation with Vector Calculus

5 minute read

Published:

This is something back in 2016 when I was struggling with my first serious neural model. Previously I mostly use Scala for NLP. But there was no auto differentiation locally in Scala, so we ended up thinking to write our own package. The task, naturally, turned out to be waaaaay too non-trivial, and we soon gave it up days after. Well, anyway, here is the wrap-up.