# Unsorted mathematical (or otherwise technical) links, June 2017: Numba, Julia, NNs

A monthly collection of random links and commentary, now split to two parts: math/tech-heavy (this post) and separate post for less-technical items. Time frame covered: the past June and older items.

## Links

### More Gradient Descent

[applied.optimization, applied.resources]

Related to a previous link: Sebastian Ruder has a comprehensive listing of gradient descent algorithms. Link, arxiv.

### Numba, Cython

[software.numba, software.cython, software.python]

I found about Numba. This looks amazingly fun way to write fast, performant (just-in-time) compiled Python. I was previously aware that things like Cython, CPython C extensions, and various tricks exist, but apparently unlike them, Numba should play well along with NumPy (which is a deal-breaker).

Lately I’ve been writing mostly Matlab … but going to try Numba next time I’ll write numerical / data / stats stuff in Python.

Note. See also this.

Note. Cython docs.

### Julia

[software.julia]

Meanwhile, Julia received funding. I’ve been intending to take a look at Julia for some time now: It looks cool (and also very high-performing), but still apparently heavily in development … maybe I’ll wait until they release a version that will be stable.

### On GANs

[applied.AI, applied.neural]

“Generative Adversarial Networks (GANs), Some Open Questions”.

### SVM fun

[applied.ML, applied.SVM]

### Reference: List of Neural Network Architectures

[applied.AI, applied.neural]

Neural network architecture overdose?! via Twitter.

### More NNs

[applied.AI, applied.neural]

Santoro et al. 2017. “A simple neural network module for relational reasoning”.

Relational reasoning is a central component of generally intelligent behavior, but has proven difficult for neural networks to learn. In this paper we describe how to use Relation Networks (RNs) as a simple plug-and-play module to solve problems that fundamentally hinge on relational reasoning.

### Programming for Scientists

[science.programming, software.science, software.best_practices]

Wilson et al. 2017, “Good enough practices in scientific computing”, Plos Comp. Biol. link, doi:10.1371/journal.pcbi.1005510

I like the ideas! Except Google Docs sounds terrible for serious scientific writing with lots of math notation. For occasional group / lab projects I’ve used collaborative-latex-editing-as-a-service tools like Overleaf successfully.

### …

[software.php]

I’ve been ‘learning’ a tiny bit of PHP.

Note. Concerning sanitizing input.

### Non-linear Dynamics and Chaos

[notes.to_self, math.references, math.chaos, math.nonlinear_dynamics]

References for future (and reminder that this thing exists):

Series of Lectures on YouTube by

Book reference: Link.

### Philosophy of Real Numbers

Curious math.SE discussions on real numbers.

“Why do we study real numbers?”

“Why are all structures I study based on real numbers?”

See also “How real are real numbers”, by G. J. Chaitin, 2004.

We discuss mathematical and physical arguments against continuity and in favor of discreteness, with particular emphasis on the ideas of Emile Borel (1871-1956).

### Math of Voting

[math.general, random.elections]

When can a majority of voters find common ground, that is, a position they all agree upon?

Deborah E. Berg, Serguei Norine, Francis Edward Su, Robin Thomas, Paul Wollan, 2008. “Voting in agreeable societies”.

### URI / URL / URN?

[software.general, internet,stuff]

I call them all URLs but Stackoverflow has details.

### Signals and Systems reference

[notes.to_self, applied.resources]

Wikibook looks okay, but not all-encompassing treatment.

### German Tank Problem, meet MCMC

[notes.to_self, random.tank_problems, things.fortune_cookies]

2015, so I probably should check if the APIs have changed.

See also related post concerning fortunate cookies.

### Python and R Plotting Tools References

[software.resources, stats.software, stats.plots, software.python, software.R]

### R and Data Science Textbook

[stats.edu, stats.resources, notes.to_self]

Garrett Grolemund,Hadley Wickham: R for Data Science.

### Not-So-Awesome Math Notation

[math.notation, math.general]

### Law of the Iterated Logarithm

[notes.to_self, math.probability]

Note. On the bounds of i.i.d. random walks. Wikipedia.

https://en.wikipedia.org/wiki/Law_of_large_numbers

https://en.wikipedia.org/wiki/Central_limit_theorem

### Apollo 11 Source Code on Github

[software.github, things.apollo11]

This made rounds on the internet some time ago.

### Reference: Intuitive Intro to Non-Standard Analysis

[notes.to_self, math.non_standard_analysis]

### Reference: Convex Optimization

[notes.to_self, applied.optimization, applied.resources]

More convex optimization resources. Course organized here in HY 2017. Somehow managed to miss it. Oh well, at least the materials are still online.

### Visual Intuitive Introductions to Stuff

[math.resources, math.visualizations]

link.

See for example very nice visualization for PCA.

### Reference: Linear Algebra Textbook

[notes.to_self, math.linalg, math.textbooks, math.resources]

Sergei Treil, Linear Algebra Done Wrong.

See also blog post and this list I linked in 2016.

### Fermi Paradox Dissolved with Bayesian priors?

[space.aliens, applied.bayesian, stats.bayesian]

See slides by Anders Sandberg, Eric Drexler & Toby Ord: pdf.

### Simulation: Everyone In the Room Gives Randomly Dollars to Each Other

[applied.stuff, applied.compusoc, notes.to_self]

…a most curious distribution arises. Simulation credit to Uri Wilensky.

Note. It appears that there exist fields called agent-based social simulation and computational economics, which are related to investigation of real-life implications of the phenomena such as this.

Search term: complex adaptive systems

### Musings of Math Major On Becoming Data Scientist

[applied.datasci, stats.general, math.careers]

By Tim Hopper, 2015. Link

### Einstein’s Derivation of

[random.einstein]

By Terry Tao, 2007. I’m not myself a physics major (or even minor), but reading this was a fun exercise to rehash various bits of undergrad maths and become more acquainted 20th century physics in the process. link.

### Efficiency of Morse Code

[random.morse_code]

Brief analysis by John D. Cook.

### On p-Hacking

[stats.p_hacking, stats.general]

Statistics reference.

Andrew Gelman and Eric Loken, 2013. “The garden of forking paths: Why multiple comparisons can be a problem”

### Statistical-Inference-in-a-Box

[stats.software, stats.resources, stats.bayesian]

Not just in any box, in SQL query-like box. BayesDB, in public alpha. Looks interesting, but not sure if this is really a tool that allows non-statisticians see new depths in their data without statistics knowledge, or a just a useful tool for statisticians who know SQL?

### WikiCFP

[academia.tools]

### Why We Use Quicksort?

[theorcompsci.complexity, theorcompsci.algo]

…when other algorithms such as mergesort and heapsort exist?

### Computer Vision Learning Resources

[applied.CV, applied.resources]

### …

[software.bash]

### …

[software.stuff, software.careers]

When choosing your tech stack, remember that you are not Google, but maybe you want to get hired by them.

### …

[software.linux, software.editors, software.resources]

A Practical Guide to Linux Commands Editors and Shell Programming from University of Missouri AEM 3100.

### Caffe2

[notes.to_self, applied.ML, applied.neural, applied.deep, applied.software, software.libraries]

There is simply too many of these libraries, but I’m making a note of this anyway.

### More Statistics References

[notes.to_self, stats.resources, stats.edu]