Okay, let’s try to get started with this idea of topic-based-links-posting.

First thing I read today: an article about feature visualization in (deep) neural nets aka. Deep Dreaming.

The key reference I happened to read today is:

Olah, et al., “Feature Visualization”, Distill, 2017. Linky.

As always with Distill articles, they have very high-quality visualizations and graphics.

The basic idea of “dreaming” is quite simple (let’s see if I’ve understood it correctly i.e. can explain it:)

Usually NN models are learned by optimizing with (stochastic) gradient descent (or any widley avaialble derivative algorithms): In the context of supervised feed-forward networks, given inputs (e.g. images) and desired outpus (e.g. classification results for those images), we try to find the NN parameters that minimize some loss function that measures how well our NN (in this example) classifies images. The networks have layered, convolutional structure; correctly trained, the neurons manage detect features in the images (they ‘activate’ when the NN is given an image with a certain pattern). So far so good.

In the “dreaming mode”, we reverse the polarity keeping the network fixed, try instead to find an image that produces maximal activation for feature-detecting neurons of interest. Except it isn’t that simple. See the article for details (this is new material to me after all, I may misunderstand something) but the main problem is that naively optimizing results in finding images of particular kind of noise that maximally activate the neurons but does not necessary correspond to any phenemonon in the training dataset.

What I consider the ‘‘main beef’’ of the article are the various ways to correct for this behaviour: regularization and (gradient) preconditioning.

But also another interesting point about the architecture of NNs and how they can be thought as way to learn “representations” of images in some very high-dimensional space (with the coordinate basis given by individiual neurons).

[..] Examples like these suggest that neurons are not necessarily the right semantic units for understanding neural nets.

If neurons are not the right way to understand neural nets, what is? In real life, combinations of neurons work together to represent images in neural networks. Individual neurons are the basis directions of activation space, and it is not clear that these should be any more special than any other direction.

Szegedy et al.[11] found that random directions seem just as meaningful as the basis directions. More recently Bau, Zhou et al.[12] found basis directions to be interpretable more often than random directions. Our experience is broadly consistent with both results; we find that random directions often seem interpretable, but at a lower rate than basis directions.

I’m slightly unclear about this.