MSc Computer Science group project proposal

November 2014


The aim of this project is to make sophisticated machine learning tools so easy to use that kids (and adults with little knowledge of computer science) can use machine learning algorithms in their creations.  In fact, the aim is to make machine learning so easy that kids and adults can play with it.  The idea is to wrap machine learning tools as simple ‘building blocks’ which can be bolted together with existing rapid prototyping tools to allow users to quickly throw together sophisticated inventions.  e.g. Lego robots which can classify objects using a video camera, or can understand some natural speech, or learn some task through trial and error (e.g. balancing on one leg).

Machine Learning

Over the last few years, machine learning has developed to the point where machines can perform very well on a variety of tasks such as image classification (“does this image contain a dog?”), natural language processing (e.g. “does a sentence express a positive or negative opinion?”), machine translation (e.g. translate from French to English) etc.  Some of these tools are now sufficiently easy to use that a skilled software developer can start making use of them in a day or two, without requiring a deep understanding of the underlying algorithms.  But these tools are still a little too tricky to use for non-programmers.

Rapid prototyping


Another relatively recent trend has been to take existing tools and technologies and package them in such a way as to make them accessible to a much wider audience.  The Arduino (pictured above) is an excellent example of this trend.  The Arduino was not a fundamental breakthrough in computer science; instead it ‘just’ made it easier to build embedded computer systems.  But, by lowering the barrier to entry, the Arduino allowed artists, architects, biologists, teachers, kids etc etc to use fairly sophisticated electronics in their creations.  This resulted in an explosion in the use of embedded computer systems, and a huge variety of fascinating projects.

The aim of our project is to lower the barrier of entry for machine learning.  The aim is to do for machine learning what the Arduino did for embedded computing!

Teaching kids to code

Another trend over the last few years has been a renewed focus on getting young kids excited about computer science.  For example, the new UK national curriculum (released September 2014) requires that primary school kids get some hands-on experience with coding.  There are visual programming environments such as MIT’s Scratch which allow young children to start messing about with programming from a very young age and there are extensions to Scratch to allow it to control hardware such as Lego NXT robots.  For example, here’s the Scratch code to get a robot to draw a square:


(image from

One potential direction you could take this project would be to modify Scratch (or similar) to add ‘blocks’ which provide a very simple interface to sophisticated machine learning algorithms.  For example, you could add a “Wait until <object> detected” block.  The user would see a drop-down box to pick which object class to detect (dog, cat, human, cup, car etc).  The resulting code would continually monitor a video input to see if a particular object is detected.  This would allow kids to write code to, for example, create a robot which can react to the family pet, adults, toy cars etc.  Or, if you’d prefer, you could look at other machine learning tools such as natural language processing.

(One thing to note is that Lego NXTs have no where near enough compute power to run image classification on the NXT.  So you’d either have to run the image classification on a PC, or use a mobile GPU such as nVidia’s Jetson mobile GPU board (see below).  You can communicate with the Lego NXT 2.0 brick via Bluetooth or USB).

Kids today will grow up with increasingly sophisticated AI

Terms like ‘digital generation’ are often used to describe today’s kids.  The idea is that kids who have been born in roughly the last decade will grow up fully immersed in a ‘digital’ world where wireless Internet connections and fast, powerful, mobile compute devices are pervasive.  But what is often missed in these conversations is that today’s kids will also grow up in a world where AI will become more and more sophisticated.  Already, today, we take for granted that face recognition works very reliably and that tools like Siri or Google Now or Amazon Echo can do a good job of understanding natural spoken language.  These tools will only get more powerful and more pervasive.  Hence kids today might expect their toys to also have some element of AI; and, furthermore, it would seem a good idea to allow kids to ‘see behind the curtain’ (i.e. to get hands-on experience of using AI in their own creations) so they can develop an informed opinion of both the benefits and the limitations of AI.



Caffe is an open source deep learning framework which comes with a pre-trained neural network capable of detecting many classes of object.

nVidia’s Jetson TK1 mobile GPU board

Deep learning algorithms are quite compute-intensive and are well suited to running on a GPU. Hence, if you wanted to build an autonomous robot, you might need to look at interfacing a Lego NXT robot with something like this nVidia Jetson TK1 GPU mobile board:


Luckily, Caffe already runs on the TK1!

Lego NXT Mindstorm robots



Enchanting is basically a version of Scratch for programming Lego NXT robots.

Enchanting is based on Snap! (a re-implementation of Scratch)


Update 19/11/2014: nVidia’s deep learning library, cuDNN, now runs on the Jetson TK1