1 - Numba Basics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# GTC 2018 Numba Tutorial Notebook 1: Numba Basics\n",
    "\n",
    "## What is Numba?\n",
    "\n",
    "Numba is a **just-in-time**, **type-specializing**, **function compiler** for accelerating **numerically-focused** Python.  That's a long list, so let's break down those terms:\n",
    "\n",
    " * **function compiler**: Numba compiles Python functions, not entire applications, and not parts of functions.  Numba does not replace your Python interpreter, but is just another Python module that can turn a function into a (usually) faster function. \n",
    " * **type-specializing**: Numba speeds up your function by generating a specialized implementation for the specific data types you are using.  Python functions are designed to operate on generic data types, which makes them very flexible, but also very slow.  In practice, you only will call a function with a small number of argument types, so Numba will generate a fast implementation for each set of types.\n",
    " * **just-in-time**: Numba translates functions when they are first called.  This ensures the compiler knows what argument types you will be using.  This also allows Numba to be used interactively in a Jupyter notebook just as easily as a traditional application\n",
    " * **numerically-focused**: Currently, Numba is focused on numerical data types, like `int`, `float`, and `complex`.  There is very limited string processing support, and many string use cases are not going to work well on the GPU.  To get best results with Numba, you will likely be using NumPy arrays.\n",
    "\n",
    "## Requirements\n",
    "\n",
    "Numba supports a wide range of operating systems:\n",
    "\n",
    " * Windows 7 and later, 32 and 64-bit\n",
    " * macOS 10.9 and later, 64-bit\n",
    " * Linux (most anything >= RHEL 5), 32-bit and 64-bit\n",
    "\n",
    "and Python versions:\n",
    "\n",
    " * Python 2.7, 3.4-3.6\n",
    " * NumPy 1.10 and later\n",
    "\n",
    "and a very wide range of hardware:\n",
    "\n",
    "* x86, x86_64/AMD64 CPUs\n",
    "* NVIDIA CUDA GPUs (Compute capability 3.0 and later, CUDA 8.0 and later)\n",
    "* AMD GPUs (experimental patches)\n",
    "* ARM (experimental patches)\n",
    "\n",
    "For this tutorial, we will be using Linux 64-bit and CUDA 8."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## First Steps\n",
    "\n",
    "Let's write our first Numba function and compile it for the **CPU**.  The Numba compiler is typically enabled by applying a *decorator* to a Python function.  Decorators are functions that transform Python functions.  Here we will use the CPU compilation decorator:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from numba import jit\n",
    "import math\n",
    "\n",
    "@jit\n",
    "def hypot(x, y):\n",
    "    # Implementation from https://en.wikipedia.org/wiki/Hypot\n",
    "    x = abs(x);\n",
    "    y = abs(y);\n",
    "    t = min(x, y);\n",
    "    x = max(x, y);\n",
    "    t = t / x;\n",
    "    return x * math.sqrt(1+t*t)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above code is equivalent to writing:\n",
    "``` python\n",
    "def hypot(x, y):\n",
    "    x = abs(x);\n",
    "    y = abs(y);\n",
    "    t = min(x, y);\n",
    "    x = max(x, y);\n",
    "    t = t / x;\n",
    "    return x * math.sqrt(1+t*t)\n",
    "    \n",
    "hypot = jit(hypot)\n",
    "```\n",
    "This means that the Numba compiler is just a function you can call whenever you want!\n",
    "\n",
    "Let's try out our hypotenuse calculation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5.0"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hypot(3.0, 4.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first time we call `hypot`, the compiler is triggered and compiles a machine code implementation for float inputs.  Numba also saves the original Python implementation of the function in the `.py_func` attribute, so we can call the original Python code to make sure we get the same answer:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5.0"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hypot.py_func(3.0, 4.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Benchmarking\n",
    "\n",
    "An important part of using Numba is measuring the performance of your new code.  Let's see if we actually sped anything up.  The easiest way to do this in the Jupyter notebook is to use the `%timeit` magic function.  Let's first measure the speed of the original Python:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "657 ns ± 6.96 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\n"
     ]
    }
   ],
   "source": [
    "%timeit hypot.py_func(3.0, 4.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `%timeit` magic runs the statement many times to get an accurate estimate of the run time.  It also returns the best time by default, which is useful to reduce the probability that random background events affect your measurement.  The best of 3 approach also ensures that the compilation time on the first call doesn't skew the results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "154 ns ± 1.82 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)\n"
     ]
    }
   ],
   "source": [
    "%timeit hypot(3.0, 4.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Numba did a pretty good job with this function.  It's more than 4x faster than the pure Python version.\n",
    "\n",
    "Of course, the `hypot` function is already present in the Python module:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "108 ns ± 0.803 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)\n"
     ]
    }
   ],
   "source": [
    "%timeit math.hypot(3.0, 4.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Python's built-in is even faster than Numba!  This is because Numba does introduce some overhead to each function call that is larger than the function call overhead of Python itself.  Extremely fast functions (like the above one) will be hurt by this.\n",
    "\n",
    "(However, if you call one Numba function from another one, there is very little function overhead, sometimes even zero if the compiler inlines the function into the other one.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How does Numba work?\n",
    "\n",
    "The first time we called our Numba-wrapped `hypot` function, the following process was initiated:\n",
    "\n",
    "![Numba Flowchart](img/numba_flowchart.png \"The compilation process\")\n",
    "\n",
    "We can see the result of type inference by using the `.inspect_types()` method, which prints an annotated version of the source code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hypot (float64, float64)\n",
      "--------------------------------------------------------------------------------\n",
      "# File: <ipython-input-1-2d77e48c1ef7>\n",
      "# --- LINE 4 --- \n",
      "# label 0\n",
      "#   del x\n",
      "#   del $0.1\n",
      "#   del $0.3\n",
      "#   del y\n",
      "#   del $0.4\n",
      "#   del $0.6\n",
      "#   del $0.7\n",
      "#   del $0.10\n",
      "#   del y.1\n",
      "#   del x.1\n",
      "#   del $0.11\n",
      "#   del $0.14\n",
      "#   del t\n",
      "#   del $0.17\n",
      "#   del $0.19\n",
      "#   del t.1\n",
      "#   del $const0.21\n",
      "#   del $0.24\n",
      "#   del $0.25\n",
      "#   del $0.20\n",
      "#   del x.2\n",
      "#   del $0.26\n",
      "#   del $0.27\n",
      "\n",
      "@jit\n",
      "\n",
      "# --- LINE 5 --- \n",
      "\n",
      "def hypot(x, y):\n",
      "\n",
      "    # --- LINE 6 --- \n",
      "\n",
      "    # Implementation from https://en.wikipedia.org/wiki/Hypot\n",
      "\n",
      "    # --- LINE 7 --- \n",
      "    #   x = arg(0, name=x)  :: float64\n",
      "    #   y = arg(1, name=y)  :: float64\n",
      "    #   $0.1 = global(abs: <built-in function abs>)  :: Function(<built-in function abs>)\n",
      "    #   $0.3 = call $0.1(x, func=$0.1, args=[Var(x, <ipython-input-1-2d77e48c1ef7> (7))], kws=(), vararg=None)  :: (float64,) -> float64\n",
      "    #   x.1 = $0.3  :: float64\n",
      "\n",
      "    x = abs(x);\n",
      "\n",
      "    # --- LINE 8 --- \n",
      "    #   $0.4 = global(abs: <built-in function abs>)  :: Function(<built-in function abs>)\n",
      "    #   $0.6 = call $0.4(y, func=$0.4, args=[Var(y, <ipython-input-1-2d77e48c1ef7> (7))], kws=(), vararg=None)  :: (float64,) -> float64\n",
      "    #   y.1 = $0.6  :: float64\n",
      "\n",
      "    y = abs(y);\n",
      "\n",
      "    # --- LINE 9 --- \n",
      "    #   $0.7 = global(min: <built-in function min>)  :: Function(<built-in function min>)\n",
      "    #   $0.10 = call $0.7(x.1, y.1, func=$0.7, args=[Var(x.1, <ipython-input-1-2d77e48c1ef7> (7)), Var(y.1, <ipython-input-1-2d77e48c1ef7> (8))], kws=(), vararg=None)  :: (float64, float64) -> float64\n",
      "    #   t = $0.10  :: float64\n",
      "\n",
      "    t = min(x, y);\n",
      "\n",
      "    # --- LINE 10 --- \n",
      "    #   $0.11 = global(max: <built-in function max>)  :: Function(<built-in function max>)\n",
      "    #   $0.14 = call $0.11(x.1, y.1, func=$0.11, args=[Var(x.1, <ipython-input-1-2d77e48c1ef7> (7)), Var(y.1, <ipython-input-1-2d77e48c1ef7> (8))], kws=(), vararg=None)  :: (float64, float64) -> float64\n",
      "    #   x.2 = $0.14  :: float64\n",
      "\n",
      "    x = max(x, y);\n",
      "\n",
      "    # --- LINE 11 --- \n",
      "    #   $0.17 = t / x.2  :: float64\n",
      "    #   t.1 = $0.17  :: float64\n",
      "\n",
      "    t = t / x;\n",
      "\n",
      "    # --- LINE 12 --- \n",
      "    #   $0.19 = global(math: <module 'math' from '/Users/sseibert/anaconda/envs/gtc2018/lib/python3.6/lib-dynload/math.cpython-36m-darwin.so'>)  :: Module(<module 'math' from '/Users/sseibert/anaconda/envs/gtc2018/lib/python3.6/lib-dynload/math.cpython-36m-darwin.so'>)\n",
      "    #   $0.20 = getattr(value=$0.19, attr=sqrt)  :: Function(<built-in function sqrt>)\n",
      "    #   $const0.21 = const(int, 1)  :: int64\n",
      "    #   $0.24 = t.1 * t.1  :: float64\n",
      "    #   $0.25 = $const0.21 + $0.24  :: float64\n",
      "    #   $0.26 = call $0.20($0.25, func=$0.20, args=[Var($0.25, <ipython-input-1-2d77e48c1ef7> (12))], kws=(), vararg=None)  :: (float64,) -> float64\n",
      "    #   $0.27 = x.2 * $0.26  :: float64\n",
      "    #   $0.28 = cast(value=$0.27)  :: float64\n",
      "    #   return $0.28\n",
      "\n",
      "    return x * math.sqrt(1+t*t)\n",
      "\n",
      "\n",
      "================================================================================\n"
     ]
    }
   ],
   "source": [
    "hypot.inspect_types()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that Numba's type names tend to mirror the NumPy type names, so a Python `float` is a `float64` (also called \"double precision\" in other languages).  Taking a look at the data types can sometimes be important in GPU code because the performance of `float32` and `float64` computations will be very different on CUDA devices.  An accidental upcast can dramatically slow down a function."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## When Things Go Wrong\n",
    "\n",
    "Numba cannot compile all Python code.  Some functions don't have a Numba-translation, and some kinds of Python types can't be efficiently compiled at all (yet).  For example, Numba does not support dictionaries (as of this tutorial):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'value'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "@jit\n",
    "def cannot_compile(x):\n",
    "    return x['key']\n",
    "\n",
    "cannot_compile(dict(key='value'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wait, what happened??  By default, Numba will fall back to a mode, called \"object mode,\" which does not do type-specialization.  Object mode exists to enable other Numba functionality, but in many cases, you want Numba to tell you if type inference fails.  You can force \"nopython mode\" (the other compilation mode) by passing arguments to the decorator:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypingError",
     "evalue": "Failed at nopython (nopython frontend)\nInternal error at <numba.typeinfer.ArgConstraint object at 0x11307c898>:\n--%<-----------------------------------------------------------------\nTraceback (most recent call last):\n  File \"/Users/sseibert/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/errors.py\", line 259, in new_error_context\n    yield\n  File \"/Users/sseibert/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/typeinfer.py\", line 189, in __call__\n    assert ty.is_precise()\nAssertionError\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/Users/sseibert/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/typeinfer.py\", line 137, in propagate\n    constraint(typeinfer)\n  File \"/Users/sseibert/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/typeinfer.py\", line 190, in __call__\n    typeinfer.add_type(self.dst, ty, loc=self.loc)\n  File \"/Users/sseibert/anaconda/envs/gtc2018/lib/python3.6/contextlib.py\", line 99, in __exit__\n    self.gen.throw(type, value, traceback)\n  File \"/Users/sseibert/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/errors.py\", line 265, in new_error_context\n    six.reraise(type(newerr), newerr, sys.exc_info()[2])\n  File \"/Users/sseibert/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/six.py\", line 658, in reraise\n    raise value.with_traceback(tb)\n  File \"/Users/sseibert/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/errors.py\", line 259, in new_error_context\n    yield\n  File \"/Users/sseibert/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/typeinfer.py\", line 189, in __call__\n    assert ty.is_precise()\nnumba.errors.InternalError: \n[1] During: typing of argument at <ipython-input-9-42c374763781> (3)\n--%<-----------------------------------------------------------------\n\nFile \"<ipython-input-9-42c374763781>\", line 3\n\nThis error may have been caused by the following argument(s):\n- argument 0: cannot determine Numba type of <class 'dict'>\n",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypingError\u001b[0m                               Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-9-42c374763781>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      3\u001b[0m     \u001b[0;32mreturn\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'key'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0mcannot_compile\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'value'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/dispatcher.py\u001b[0m in \u001b[0;36m_compile_for_args\u001b[0;34m(self, *args, **kws)\u001b[0m\n\u001b[1;32m    328\u001b[0m                                 for i, err in failed_args))\n\u001b[1;32m    329\u001b[0m                 \u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpatch_message\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmsg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 330\u001b[0;31m             \u001b[0;32mraise\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    331\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    332\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0minspect_llvm\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msignature\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/dispatcher.py\u001b[0m in \u001b[0;36m_compile_for_args\u001b[0;34m(self, *args, **kws)\u001b[0m\n\u001b[1;32m    305\u001b[0m                 \u001b[0margtypes\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtypeof_pyval\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    306\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 307\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcompile\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtuple\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margtypes\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    308\u001b[0m         \u001b[0;32mexcept\u001b[0m \u001b[0merrors\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mTypingError\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    309\u001b[0m             \u001b[0;31m# Intercept typing error that may be due to an argument\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/dispatcher.py\u001b[0m in \u001b[0;36mcompile\u001b[0;34m(self, sig)\u001b[0m\n\u001b[1;32m    577\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    578\u001b[0m                 \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_cache_misses\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0msig\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 579\u001b[0;31m                 \u001b[0mcres\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_compiler\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcompile\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mreturn_type\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    580\u001b[0m                 \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd_overload\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcres\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    581\u001b[0m                 \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_cache\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msave_overload\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msig\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcres\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/dispatcher.py\u001b[0m in \u001b[0;36mcompile\u001b[0;34m(self, args, return_type)\u001b[0m\n\u001b[1;32m     78\u001b[0m                                       \u001b[0mimpl\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     79\u001b[0m                                       \u001b[0margs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mreturn_type\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mreturn_type\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 80\u001b[0;31m                                       flags=flags, locals=self.locals)\n\u001b[0m\u001b[1;32m     81\u001b[0m         \u001b[0;31m# Check typing error if object mode is used\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     82\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mcres\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtyping_error\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mflags\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0menable_pyobject\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/compiler.py\u001b[0m in \u001b[0;36mcompile_extra\u001b[0;34m(typingctx, targetctx, func, args, return_type, flags, locals, library)\u001b[0m\n\u001b[1;32m    777\u001b[0m     pipeline = Pipeline(typingctx, targetctx, library,\n\u001b[1;32m    778\u001b[0m                         args, return_type, flags, locals)\n\u001b[0;32m--> 779\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0mpipeline\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcompile_extra\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfunc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    780\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    781\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/compiler.py\u001b[0m in \u001b[0;36mcompile_extra\u001b[0;34m(self, func)\u001b[0m\n\u001b[1;32m    360\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlifted\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    361\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlifted_from\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 362\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_compile_bytecode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    363\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    364\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mcompile_ir\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfunc_ir\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlifted\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlifted_from\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/compiler.py\u001b[0m in \u001b[0;36m_compile_bytecode\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    736\u001b[0m         \"\"\"\n\u001b[1;32m    737\u001b[0m         \u001b[0;32massert\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfunc_ir\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 738\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_compile_core\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    739\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    740\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_compile_ir\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/compiler.py\u001b[0m in \u001b[0;36m_compile_core\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    723\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    724\u001b[0m         \u001b[0mpm\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfinalize\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 725\u001b[0;31m         \u001b[0mres\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpm\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstatus\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    726\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mres\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    727\u001b[0m             \u001b[0;31m# Early pipeline completion\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/compiler.py\u001b[0m in \u001b[0;36mrun\u001b[0;34m(self, status)\u001b[0m\n\u001b[1;32m    246\u001b[0m                     \u001b[0;31m# No more fallback pipelines?\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    247\u001b[0m                     \u001b[0;32mif\u001b[0m \u001b[0mis_final_pipeline\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 248\u001b[0;31m                         \u001b[0;32mraise\u001b[0m \u001b[0mpatched_exception\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    249\u001b[0m                     \u001b[0;31m# Go to next fallback pipeline\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    250\u001b[0m                     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/compiler.py\u001b[0m in \u001b[0;36mrun\u001b[0;34m(self, status)\u001b[0m\n\u001b[1;32m    238\u001b[0m                 \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    239\u001b[0m                     \u001b[0mevent\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstage_name\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 240\u001b[0;31m                     \u001b[0mstage\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    241\u001b[0m                 \u001b[0;32mexcept\u001b[0m \u001b[0m_EarlyPipelineCompletion\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    242\u001b[0m                     \u001b[0;32mreturn\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mresult\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/compiler.py\u001b[0m in \u001b[0;36mstage_nopython_frontend\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    452\u001b[0m                 \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    453\u001b[0m                 \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreturn_type\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 454\u001b[0;31m                 self.locals)\n\u001b[0m\u001b[1;32m    455\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    456\u001b[0m         with self.fallback_context('Function \"%s\" has invalid return type'\n",
      "\u001b[0;32m~/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/compiler.py\u001b[0m in \u001b[0;36mtype_inference_stage\u001b[0;34m(typingctx, interp, args, return_type, locals)\u001b[0m\n\u001b[1;32m    879\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    880\u001b[0m         \u001b[0minfer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbuild_constraint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 881\u001b[0;31m         \u001b[0minfer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpropagate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    882\u001b[0m         \u001b[0mtypemap\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrestype\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcalltypes\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0minfer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0munify\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    883\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/typeinfer.py\u001b[0m in \u001b[0;36mpropagate\u001b[0;34m(self, raise_errors)\u001b[0m\n\u001b[1;32m    844\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0merrors\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    845\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mraise_errors\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 846\u001b[0;31m                 \u001b[0;32mraise\u001b[0m \u001b[0merrors\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    847\u001b[0m             \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    848\u001b[0m                 \u001b[0;32mreturn\u001b[0m \u001b[0merrors\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mTypingError\u001b[0m: Failed at nopython (nopython frontend)\nInternal error at <numba.typeinfer.ArgConstraint object at 0x11307c898>:\n--%<-----------------------------------------------------------------\nTraceback (most recent call last):\n  File \"/Users/sseibert/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/errors.py\", line 259, in new_error_context\n    yield\n  File \"/Users/sseibert/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/typeinfer.py\", line 189, in __call__\n    assert ty.is_precise()\nAssertionError\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/Users/sseibert/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/typeinfer.py\", line 137, in propagate\n    constraint(typeinfer)\n  File \"/Users/sseibert/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/typeinfer.py\", line 190, in __call__\n    typeinfer.add_type(self.dst, ty, loc=self.loc)\n  File \"/Users/sseibert/anaconda/envs/gtc2018/lib/python3.6/contextlib.py\", line 99, in __exit__\n    self.gen.throw(type, value, traceback)\n  File \"/Users/sseibert/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/errors.py\", line 265, in new_error_context\n    six.reraise(type(newerr), newerr, sys.exc_info()[2])\n  File \"/Users/sseibert/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/six.py\", line 658, in reraise\n    raise value.with_traceback(tb)\n  File \"/Users/sseibert/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/errors.py\", line 259, in new_error_context\n    yield\n  File \"/Users/sseibert/anaconda/envs/gtc2018/lib/python3.6/site-packages/numba/typeinfer.py\", line 189, in __call__\n    assert ty.is_precise()\nnumba.errors.InternalError: \n[1] During: typing of argument at <ipython-input-9-42c374763781> (3)\n--%<-----------------------------------------------------------------\n\nFile \"<ipython-input-9-42c374763781>\", line 3\n\nThis error may have been caused by the following argument(s):\n- argument 0: cannot determine Numba type of <class 'dict'>\n"
     ]
    }
   ],
   "source": [
    "@jit(nopython=True)\n",
    "def cannot_compile(x):\n",
    "    return x['key']\n",
    "\n",
    "cannot_compile(dict(key='value'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we get an exception when Numba tries to compile the function, with an error that says:\n",
    "```\n",
    "- argument 0: cannot determine Numba type of <class 'dict'>\n",
    "```\n",
    "which is the underlying problem.\n",
    "\n",
    "We will see other `@jit` decorator arguments in future sections."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise\n",
    "\n",
    "Below is a function that loops over two input NumPy arrays and puts their sum into the output array.  Modify this function to call the `hypot` function we defined above.  We will learn a more efficient way to write such functions in a future section.\n",
    "\n",
    "(Make sure to execute all the cells in this notebook so that `hypot` is defined.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "@jit(nopython=True)\n",
    "def ex1(x, y, out):\n",
    "    for i in range(x.shape[0]):\n",
    "        out[i] = x[i] + y[i]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "in1: [0. 1. 2. 3. 4. 5. 6. 7. 8. 9.]\n",
      "in2: [ 1.  3.  5.  7.  9. 11. 13. 15. 17. 19.]\n",
      "out: [ 1.  4.  7. 10. 13. 16. 19. 22. 25. 28.]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "in1 = np.arange(10, dtype=np.float64)\n",
    "in2 = 2 * in1 + 1\n",
    "out = np.empty_like(in1)\n",
    "\n",
    "print('in1:', in1)\n",
    "print('in2:', in2)\n",
    "\n",
    "ex1(in1, in2, out)\n",
    "\n",
    "print('out:', out)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This test will fail until you fix the ex1 function\n",
    "np.testing.assert_almost_equal(out, np.hypot(in1, in2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}