A monthly collection of random links and commentary, now split to two parts: math/tech-heavy (this post) and separate post for less-technical items. Time frame covered: the past June and older items.
More Gradient Descent
[software.numba, software.cython, software.python]
I found about Numba. This looks amazingly fun way to write fast, performant (just-in-time) compiled Python. I was previously aware that things like Cython, CPython C extensions, and various tricks exist, but apparently unlike them, Numba should play well along with NumPy (which is a deal-breaker).
Lately I’ve been writing mostly Matlab … but going to try Numba next time I’ll write numerical / data / stats stuff in Python.
Note. See also this.
Note. Cython docs.
Meanwhile, Julia received funding. I’ve been intending to take a look at Julia for some time now: It looks cool (and also very high-performing), but still apparently heavily in development … maybe I’ll wait until they release a version that will be stable.
Reference: List of Neural Network Architectures
Santoro et al. 2017. “A simple neural network module for relational reasoning”.
Relational reasoning is a central component of generally intelligent behavior, but has proven difficult for neural networks to learn. In this paper we describe how to use Relation Networks (RNs) as a simple plug-and-play module to solve problems that fundamentally hinge on relational reasoning.
Programming for Scientists
[science.programming, software.science, software.best_practices]
I like the ideas! Except Google Docs sounds terrible for serious scientific writing with lots of math notation. For occasional group / lab projects I’ve used collaborative-latex-editing-as-a-service tools like Overleaf successfully.
Note. Concerning sanitizing input.
Non-linear Dynamics and Chaos
[notes.to_self, math.references, math.chaos, math.nonlinear_dynamics]
References for future (and reminder that this thing exists):
Book reference: Link.
Philosophy of Real Numbers
Curious math.SE discussions on real numbers.
See also “How real are real numbers”, by G. J. Chaitin, 2004.
We discuss mathematical and physical arguments against continuity and in favor of discreteness, with particular emphasis on the ideas of Emile Borel (1871-1956).
Math of Voting
When can a majority of voters find common ground, that is, a position they all agree upon?
Deborah E. Berg, Serguei Norine, Francis Edward Su, Robin Thomas, Paul Wollan, 2008. “Voting in agreeable societies”.
URI / URL / URN?
I call them all URLs but Stackoverflow has details.
Signals and Systems reference
Wikibook looks okay, but not all-encompassing treatment.
German Tank Problem, meet MCMC
[notes.to_self, random.tank_problems, things.fortune_cookies]
2015, so I probably should check if the APIs have changed.
See also related post concerning fortunate cookies.
Python and R Plotting Tools References
[software.resources, stats.software, stats.plots, software.python, software.R]
R and Data Science Textbook
[stats.edu, stats.resources, notes.to_self]
Not-So-Awesome Math Notation
Law of the Iterated Logarithm
Note. On the bounds of i.i.d. random walks. Wikipedia.
Apollo 11 Source Code on Github
Reference: Intuitive Intro to Non-Standard Analysis
Reference: Convex Optimization
[notes.to_self, applied.optimization, applied.resources]
More convex optimization resources. Course organized here in HY 2017. Somehow managed to miss it. Oh well, at least the materials are still online.
Visual Intuitive Introductions to Stuff
See for example very nice visualization for PCA.
Reference: Linear Algebra Textbook
[notes.to_self, math.linalg, math.textbooks, math.resources]
Sergei Treil, Linear Algebra Done Wrong.
Fermi Paradox Dissolved with Bayesian priors?
[space.aliens, applied.bayesian, stats.bayesian]
See slides by Anders Sandberg, Eric Drexler & Toby Ord: pdf.
Simulation: Everyone In the Room Gives Randomly Dollars to Each Other
[applied.stuff, applied.compusoc, notes.to_self]
Search term: complex adaptive systems
Musings of Math Major On Becoming Data Scientist
[applied.datasci, stats.general, math.careers]
By Tim Hopper, 2015. Link
Einstein’s Derivation of
By Terry Tao, 2007. I’m not myself a physics major (or even minor), but reading this was a fun exercise to rehash various bits of undergrad maths and become more acquainted 20th century physics in the process. link.
Efficiency of Morse Code
Andrew Gelman and Eric Loken, 2013. “The garden of forking paths: Why multiple comparisons can be a problem”
[stats.software, stats.resources, stats.bayesian]
Not just in any box, in SQL query-like box. BayesDB, in public alpha. Looks interesting, but not sure if this is really a tool that allows non-statisticians see new depths in their data without statistics knowledge, or a just a useful tool for statisticians who know SQL?
Why We Use Quicksort?
…when other algorithms such as mergesort and heapsort exist?
Computer Vision Learning Resources
[software.linux, software.editors, software.resources]
[notes.to_self, applied.ML, applied.neural, applied.deep, applied.software, software.libraries]
There is simply too many of these libraries, but I’m making a note of this anyway.
More Statistics References
[notes.to_self, stats.resources, stats.edu]