Up to Speed on Deep Learning: July Update

Sharing some of the latest research, announcements, and resources on deep learning.

By Isaac Madan (email)

Continuing our series of deep learning updates, we pulled together some of the awesome resources that have emerged since our last post. In case you missed it, here are our past updates: June (part 1, part 2, part 3, part 4), May, April (part 1, part 2), March part 1, February, November, September part 2 & October part 1, September part 1, August (part 1, part 2), July(part 1, part 2), June, and the original set of 20+ resources we outlined in April 2016. As always, this list is not comprehensive, so let us know if there’s something we should add, or if you’re interested in discussing this area further.

Research & Announcements

Apollo by Baidu. Newly launched source platform for building autonomous vehicles.

Neural Network Libraries by Sony. Sony demonstrates its interest in deep learning by releasing their own open source deep learning framework.

CAN (Creative Adversarial Network) — Explained by Harshvardhan Gupta. Facebook researchers propose a new system for generating art. The system generates art by looking at art and learning about style; and becomes creative by increasing the arousal potential of the generated art by deviating from the learned styles. This post walks through the paper and explains it. Original paper here.

‘Explainable Artificial Intelligence’: Cracking open the black box of AI by George Nott. One current downside, and ongoing research area, of deep neural networks today is that they are black boxes, meaning their decision making & outcomes can’t be easily justified or explained. Article discusses various attempts and ongoing work in this area, including work by UC Berkeley & Max Plank Institute described in this original paper here.

Interpreting Deep Neural Networks using Cognitive Psychology by DeepMind. In a similar vein to the article above, DeepMind researchers propose a new approach to interpreting/explaining deep neural network models by leveraging methods from cognitive psychology. For example, when children guess they meaning of a word from a single example (one-shot word learning), they are employing a variety of inductive biases, such as shape bias. DeepMind assesses this bias in their models to improve their interpretation of what’s happening under the hood. Original paper here.

Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapes by Wu et al. Digs into the question, why do deep neural networks generalize well?

Resources, Tutorials & Data

Under the Hood of a Self-Driving Taxi by Oliver Cameron of Voyage. A helpful overview of the tech stack powering a self-driving car, digging into Voyage’s compute, power, and drive-by-wire systems.

How HBO’s Silicon Valley built “Not Hotdog” with mobile TensorFlow, Keras & React Native by Tim Anglade. A walk-thru of how the Silicon Valley TV show built their app that famously identifies hotdogs and not hotdogs.

Machine UI, a new IDE purpose-built for machine learning with visual model representation. Video.

A 2017 Guide to Semantic Segmentation with Deep Learning by Qure.ai. Overview of state-of-the-art in semantic segmentation. As context, semantic segmentation is understanding an image at pixel level i.e, we want to assign each pixel in the image an object class.


By Isaac Madan. Isaac is an investor at Venrock (email). If you’re interested in deep learning or there are resources I should share in a future newsletter, I’d love to hear from you.

Up to Speed on Deep Learning: June Update, Part 4

Sharing some of the latest research, announcements, and resources on deep learning.

By Isaac Madan (email)

Continuing our series of deep learning updates, we pulled together some of the awesome resources that have emerged since our last post. In case you missed it, here are our past updates: June (part 1part 2part 3)MayApril (part 1part 2)March part 1FebruaryNovemberSeptember part 2 & October part 1September part 1August (part 1part 2)July(part 1part 2), June, and the original set of 20+ resources we outlined in April 2016. As always, this list is not comprehensive, so let us know if there’s something we should add, or if you’re interested in discussing this area further.

Research & Announcements

Grounded Language Learning in a Simulated 3D World by Google DeepMind. Teaching an AI agent to learn & apply language. Here we present an agent that learns to interpret language in a simulated 3D environment where it is rewarded for the successful execution of written instructions. The agent’s comprehension of language extends beyond its prior experience, enabling it to apply familiar language to unfamiliar situations and to interpret entirely novel instructions.

One Model To Learn Them All by Google. Getting a deep learning model to work well for a specific task like speech recognition, translation, etc. can take lots of time researching architecture & tuning. A generalizable model that works well across various tasks would thus be quite useful — Google presents one such model. In particular, this single model is trained concurrently on ImageNet, multiple translation tasks, image captioning (COCO dataset), a speech recognition corpus, and an English parsing task.

Tensor2Tensor by Google Brain. An open-source system for training deep learning models in TensorFlow. T2T facilitates the creation of state-of-the art models for a wide variety of ML applications, such as translation, parsing, image captioning and more, enabling the exploration of various ideas much faster than previously possible. This release also includes a library of datasets and models, including the best models from a few recent papers. GitHub repo here.

TensorFlow Object Detection API by Google. Last year Google demonstrated state-of-the-art results in object detection and won the COCO detection challenge, and featured their work in products like the NestCam.They’re now open sourcing this work: a framework built on top of TensorFlow that makes it easy to construct, train and deploy object detection models. Our goals in designing this system was to support state-of-the-art models while allowing for rapid exploration and research.

MobileNets: Open-Source Models for Efficient On-Device Vision by Google. It’s hard to run visual recognition models accurately on mobile devices given limitations in computational power and space. As such, MobileNets, a family of mobile-first computer vision models for TensorFlow, designed to effectively maximize accuracy while being mindful of the restricted resources for an on-device or embedded application.

deeplearning.ai by Andrew Ng. A new project by Andrew Ng coming up in August 2017. No details provided on the site yet.

Me trying to classify some random stuff on my desk:)

Resources, Tutorials & Data

Building a Real-Time Object Recognition App with Tensorflow and OpenCV by Dat Tran. In this article, I will walk through the steps how you can easily build your own real-time object recognition application with Tensorflow’s (TF) new Object Detection API and OpenCV in Python 3 (specifically 3.5). The focus will be on the challenges that I faced when building it. GitHub repo here.

What Can’t Deep Learning Do? by Bharath Ramsundar. A tweetstorm listing some of the known failures behind deep learning methods. Helpful in understanding where future research may be directed.

Generative Adversarial Networks for Beginners by O’Reilly. Build a neural network that learns to generate handwritten digits. GANs are neural networks that learn to create synthetic data similar to some known input data. GitHub repo here.

Measuring the Progress of AI Research by Electronic Frontier Foundation. Tracking what’s state-of-the-art in ML/AI and understanding how a specific subfield is progressing can get complicated. This pilot project collects problems and metrics/datasets from the AI research literature, and tracks progress on them.


By Isaac Madan. Isaac is an investor at Venrock (email). If you’re interested in deep learning or there are resources I should share in a future newsletter, I’d love to hear from you.

Up to Speed on Deep Learning: June 11–18 Update

Sharing some of the latest research, announcements, and resources on deep learning.

By Isaac Madan (email)

Continuing our series of deep learning updates, we pulled together some of the awesome resources that have emerged since our last post. In case you missed it, here are our past updates: June (part 1, part 2), May, April (part 1, part 2), March part 1, February, November, September part 2 & October part 1, September part 1, August (part 1, part 2), July (part 1, part 2), June, and the original set of 20+ resources we outlined in April 2016. As always, this list is not comprehensive, so let us know if there’s something we should add, or if you’re interested in discussing this area further.

Research & Announcements

Learning to Speak via Interaction by Baidu Research. Teaching an AI agent to speak by interacting with a virtual agent. This represents an advancement in more closely replicating how humans learn, as well as advancing our goal to demonstrate general artificial intelligence. Our AI agent learns to speak in an interactive way similar to a baby. In contrast, the conventional approach relies on supervised training using a large corpus of pre-collected training set, which is static and makes it hard to capture the interactive nature within the process of language learning. Original paper here.

Deep Shimon: Robot that composes its own music by Mason Britan of Georgia Tech. The robot Shimon composes and performs his first deep learning driven piece. A recurrent deep neural network is trained on a large database of classical and jazz music. Based on learned semantic relationships between musical units in this dataset, Shimon generates and performs a new musical piece. Video here.

Curiosity-driven Exploration by Self-supervised Prediction by Pathak et al. UC Berkeley researchers demonstrate artificial curiosity via an intrinsic curiosity model to control a virtual agent in a video game and understand its environment faster — which can accelerate problem solving. Original paper here and video here.

Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour by Facebook Research. Deep learning benefits from massive data sets, but this means long training times that slow down development. Using commodity hardware, our implementation achieves ∼90% scaling efficiency when moving from 8 to 256 GPUs. This system enables us to train visual recognition models on internet-scale data with high efficiency. Original paper here.

Resources

A gentle introduction to deep learning with TensorFlow by Michelle Fullwood at PyCon 2017. This talk aims to gently bridge the divide by demonstrating how deep learning operates on core machine learning concepts and getting attendees started coding deep neural networks using Google’s TensorFlow library. 41 minute video. Slides here and GitHub here.

Deep Reinforcement Learning Demystified (Episode 0) by Moustafa Alzantot. Basic description of what reinforcement learning is and provide examples for where it can be used. Cover the essential terminologies for reinforcement learning and provide a quick tutorial about OpenAI gym.

Neural Networks and Deep Learning by Michael Nielsen. Free online book that introduces neural networks and deep learning.

You can probably use deep learning even if your data isn’t that big by Andrew Beam. Article argues and explains how you can still use deep learning in (some) small data settings, if you train your model carefully. In response to Don’t use deep learning your data isn’t that big by Jeff Leek.

Posting on ArXiv is good, flag planting notwithstanding by Yann LeCun. In response to, and refuting, An Adversarial Review of “Adversarial Generation of Natural Language” by Yoav Goldberg of Bar Ilan University, which takes issue with deep learning researchers publishing aggressively on Arxiv.

Tutorials & Data

Computational Neuroscience Coursera course by University of Washington. Starts July 3, enroll now. Learn how the brain processes information. This course provides an introduction to basic computational methods for understanding what nervous systems do and for determining how they function. We will explore the computational principles governing various aspects of vision, sensory-motor control, learning, and memory.

Core ML and Vision: Machine Learning in iOS 11 Tutorial by Audrey Tam. iOS 11 introduces two new frameworks related to machine learning, Core ML and Vision. This tutorial walks you through how to use these new APIs and build a scene classifier.

Deep Learning CNN’s in Tensorflow with GPUs by Cole Murray. In this tutorial, you’ll learn the architecture of a convolutional neural network (CNN), how to create a CNN in Tensorflow, and provide predictions on labels of images. Finally, you’ll learn how to run the model on a GPU so you can spend your time creating better models, not waiting for them to converge.

Open-sourced Kinetics data set by Google DeepMind. Annotated data set of human actions — things like playing instruments, shaking hands, and hugging. Kinetics is a large-scale, high-quality dataset of YouTube video URLs which include a diverse range of human focused actions. The dataset consists of approximately 300,000 video clips, and covers 400 human action classes with at least 400 video clips for each action class.

Let’s evolve a neural network with a genetic algorithm by Matt Harvey of Coastline Automation. Applying a genetic algorithm to evolve a network with the goal of achieving optimal hyperparameters in a fraction of the time required to do a brute force search.


By Isaac Madan. Isaac is an investor at Venrock (email). If you’re interested in deep learning or there are resources I should share in a future newsletter, I’d love to hear from you.

Up to Speed on Deep Learning: June Update

Sharing some of the latest research, announcements, and resources on deep learning.

By Isaac Madan (email)

Continuing our series of deep learning updates, we pulled together some of the awesome resources that have emerged since our last post. In case you missed it, here are our past updates: May, April part 2, April part 1, March part 1, February, November, September part 2 & October part 1, September part 1, August part 2, August part 1, July part 2, July part 1, June, and the original set of 20+ resources we outlined in April 2016. As always, this list is not comprehensive, so let us know if there’s something we should add, or if you’re interested in discussing this area further.

Continue reading “Up to Speed on Deep Learning: June Update”

Up to Speed on Deep Learning: May Update

Sharing some of the latest research, announcements, and resources on deep learning.

By Isaac Madan (email)

Continuing our series of deep learning updates, we pulled together some of the awesome resources that have emerged since our last post. In case you missed it, here are our past updates: April part 2, April part 1, March part 1, February, November, September part 2 & October part 1, September part 1, August part 2, August part 1, July part 2, July part 1, June, and the original set of 20+ resources we outlined in April 2016. As always, this list is not comprehensive, so let us know if there’s something we should add, or if you’re interested in discussing this area further.

Continue reading “Up to Speed on Deep Learning: May Update”