.

.
Python: Interactive computing with bpython and IPython
.
.. .

.
Netsoc

Stephen Shaw

2010

Netsoc (Stephen Shaw) <stesh@netsoc.tcd.ie> 2010 1 / 23

Getting started

SSH to one of our servers

$ ssh you@login.netsoc.tcd.ie
PuTTY: Enter login.netsoc.tcd.ie as the hostname
NX on cube if you want - all you need is a shell though
No netsoc account?
CS: macneill.scss.tcd.ie
Maths servers?
Talk to an admin before you leave so you have an account for next time

Netsoc (Stephen Shaw) <stesh@netsoc.tcd.ie> 2010 2 / 23

The plan

Schedule
This week - The bpython interpreter, the IPython interpreter, scipy,
numpy, sympy, pylab
These slides: http://www.netsoc.tcd.ie/tutorials/2011/02/08/
python-tutorial-3/

Netsoc (Stephen Shaw) <stesh@netsoc.tcd.ie> 2010 3 / 23

Interactive computing

As we've seen, the REPL is great for playing around with python quickly
Standard REPL is quite minimal
Gets a bit tedious after a while
We can do a lot better

Netsoc (Stephen Shaw) <stesh@netsoc.tcd.ie> 2010 4 / 23

bpython

bpython is a fancy interface to the REPL

Does all sorts of cool things:
syntax highlighting
tab completion
documentation viewing
recording and playback
Saving history to disk
Posting to pastebin
It is
customizable
free
easy to set up
installed on cube

Netsoc (Stephen Shaw) <stesh@netsoc.tcd.ie> 2010 5 / 23

IPython

bpython is great, but IPython is even better

A completely new REPL, designed specifically for scientific computing
Object introspection (for tab completion)
macros
System shell access - ipython -p sh
Can be embedded within other applications
Pretty exception tracebacks
Really nice interface to pdb, the python debugger

Netsoc (Stephen Shaw) <stesh@netsoc.tcd.ie> 2010 6 / 23

IPython commands

IPython adds magic functions to python's standard lexicon

Magic functions are used to control IPython itself
preceded with a %
Crack open an object with ? and ??
Not quite punching the duck
Usual tab completion
You can embed an IPython shell in your own programs
We'll see this later

Netsoc (Stephen Shaw) <stesh@netsoc.tcd.ie> 2010 7 / 23

Magic Functions

Command What it does

%Pprint toggle pretty-printing
%autoindent toggle automatic indentation
%autocall call functions without parentheses
%bg send a job to a separate thread
%cpaste paste in code from the clipboard
%debug activate the interactive debugger
%edit start your $EDITOR
%hist display your command history
%macro define a macro
%p shorthand for python's print
%prun profile a code block
%psearch search for an object in the current namespace
%pycat syntax-highlight a code file
%run run a module inside IPython

Netsoc (Stephen Shaw) <stesh@netsoc.tcd.ie> 2010 8 / 23

Introspection
In the regular REPL, can get information about an object with dir():
Python 2.6.6 ( r266 :84292 , Dec 26 2010 , 22:31:48)
[ GCC 4.4.5] on linux2
Type " help " , " copyright " , " credits " or " license " f o r more information .
>>> import sys
>>> dir ( sys )
[ ' __displayhook__ ' , ' __doc__ ' , ' __excepthook__ ' , ' __name__ ' , '�
__package__ ' , ' __stderr__ ' , ' __stdin__ ' , ' __stdout__ ' , '�
_clear_type_cache ' , ' _current_frames ' , ' _getframe ' , ' api_version ' ,�
' argv ' , ' builtin_module_names ' , ' byteorder ' , ' call_tracing ' , '�
callstats ' , ' copyright ' , ' displayhook ' , ' dont_write_bytecode ' , '�
exc_clear ' , ' exc_info ' , ' exc_type ' , ' excepthook ' , ' exec_prefix ' , '�
executable ' , ' exit ' , ' flags ' , ' float_info ' , ' getcheckinterval ' , '�
getdefaultencoding ' , ' getdlopenflags ' , ' getfilesystemencoding ' , '�
getprofile ' , ' getrecursionlimit ' , ' getrefcount ' , ' getsizeof ' , '�
gettrace ' , ' hexversion ' , ' maxint ' , ' maxsize ' , ' maxunicode ' , '�
meta_path ' , ' modules ' , ' path ' , ' path_hooks ' , ' path_importer_cache '�
, ' platform ' , ' prefix ' , ' ps1 ' , ' ps2 ' , ' py3kwarning ' , ' pydebug ' , '�
setcheckinterval ' , ' setdlopenflags ' , ' setprofile ' , '�
setrecursionlimit ' , ' settrace ' , ' stderr ' , ' stdin ' , ' stdout ' , '�
subversion ' , ' version ' , ' version_info ' , ' warnoptions ']

Netsoc (Stephen Shaw) <stesh@netsoc.tcd.ie> 2010 9 / 23

Introspection
Messy
Much better in IPython:
In [1]: import sys
In [2]: sys ?
Type : module
Base Class : < type ' module ' >
String Form : < module ' sys ' ( built - i n ) >
Namespace : Interactive
Docstring :
This module provides access to some objects used or maintained by the
interpreter and to functions that interact strongly with the �
interpreter .

Dynamic objects :

argv -- command line arguments ; argv [0] i s the script pathname i f �

known
path -- module search path ; path [0] i s the script directory , e l s e �
''
modules -- dictionary of loaded modules
...

Netsoc (Stephen Shaw) <stesh@netsoc.tcd.ie> 2010 10 / 23

Aliases

Run a command in your shell by prefixing it with !:

In [1]: ! uptime
15:10:29 up 143 days , 23:12 , 128 users , load �
average : 1.44 , 1.24 , 1.08

Can start a whole shell this way:

In [1]: ! zsh
stesh@cube : ˜ [21:14]% echo $ SHELL
/ usr / bin / zsh

Capture STDOUT by assigning to a variable:

In [2]: dirs = ! ls /

Netsoc (Stephen Shaw) <stesh@netsoc.tcd.ie> 2010 11 / 23

History
A lot of advanced functionality relies on history
In [1]: f o r i i n range (99 ,0 , -1) :
...: p r i n t '% d bottles of beer on the wall ' % i
...:
...:
# A lot of output

In [2]: % hist
1:
f o r i i n range (99 ,0 , -1) :
p r i n t '% d bottles of beer on the wall ' % i

2: _ip . magic ( " hist " )

We can modify previous lines in an editor:

In [3]: % ed 1:2
IPython will make a temporary file named : / tmp / ipython_edit_am8xPW . py
Editing ... done . Executing edited code ...

Netsoc (Stephen Shaw) <stesh@netsoc.tcd.ie> 2010 12 / 23

Macros

`Shorthand' for a sequence of lines

%macro <macro name> <history line number(s)>

In [4]: % macro bottles 1

Macro ` bottles ` created . To execute , type its name �
( without quotes ) .
Macro contents :
f o r i i n range (99 ,0 , -1) :
p r i n t '% d bottles of beer on the wall ' % i

Can now run this entire block of code by just typing bottles
Macros can have arguments too
store them in a global list called _margv

Netsoc (Stephen Shaw) <stesh@netsoc.tcd.ie> 2010 13 / 23

NumPy

IPython is ideal for once-off computations

A number of packages are especially suited to this
NumPy is a package with fast implementations of common mathematical
algorithms
Really efficient n-ary array class and associated operations
Standard operations on matrices: norm, inverse, determinant,
exponentation
Gauss-jordan elimination, Eigenvalues, eigenvectors, QR decomposition,
Cholesky decomposition
Fast Fourier Transform

Netsoc (Stephen Shaw) <stesh@netsoc.tcd.ie> 2010 14 / 23

Playing with matrices
 
1 2 3
Create the matrix A = 4 5 5, and compute its transpose AT
6 6 8

In [1]: import numpy

In [2]: from numpy import *

In [3]: A = matrix ( '1 2 3; 4 5 5; 6 6 8 ')

In [4]: A
Out [4]:
matrix ([[1 , 2 , 3] ,
[4 , 5 , 5] ,
[6 , 6 , 8]])

In [5]: A . T
Out [5]:
matrix ([[1 , 4 , 6] ,
[2 , 5 , 6] ,
[3 , 5 , 8]])

Netsoc (Stephen Shaw) <stesh@netsoc.tcd.ie> 2010 15 / 23

Playing with matrices

In [6]: B = matrix ( '4 5 7; 2 9 5; 7 2 5 ')

In [7]: A * B
Out [7]:
matrix ([[ 29 , 29 , 32] ,
[ 61 , 75 , 78] ,
[ 92 , 100 , 112]])

In [8]: numpy . linalg . solve (A , B )

Out [8]:
matrix ([[ -0.75 , -4.83333333 , -4.58333333] ,
[ -1.75 , 7.16666667 , 2.41666667] ,
[ 2.75 , -1.5 , 2.25 ]])

solve is a function with takes two non-singular square matrices A and

B, and returns the matrix x such that Ax = B
Netsoc (Stephen Shaw) <stesh@netsoc.tcd.ie> 2010 16 / 23
SymPy

SymPy lets you do symbolic computations

You declare mathematical symbols, then call functions on them

In [1]: from sympy import *

In [2]: x , y = Symbol ( 'x ') , Symbol ( 'y ')

In [3]: y = x **2 + 4* x - 12

In [4]: solve (y , x )
Out [4]: [2 , -6]

In [5]: diff (y , x )
Out [5]: 4 + 2* x

In [6]: integrate (y , x )
Out [6]: -12* x + 2* x **2 + (1/3) * x **3

Netsoc (Stephen Shaw) <stesh@netsoc.tcd.ie> 2010 17 / 23

SymPy

solve(y, x) - solve for y with respect to x

diff(y, x) - differentiate y with respect to x
integrate(y, x) - integrate y with respect to x
Declaring Symbols explicitly is a pain
isympy is a program which embeds an IPython shell, and declares all the
constants for you

Netsoc (Stephen Shaw) <stesh@netsoc.tcd.ie> 2010 18 / 23

NetworkX

High-performance graph library

Useful for anything from network topology to linguistic analysis
Supports undirected graphs, directed graphs, multigraphs, etc.
Use any hashable object as vertex
Use any hashable object as a label
Loads of algorithms:
Traversal
Centrality
Cycle detection
Shortest paths (including A∗ )
Isomorphisms1

1 Can only tell you if two graphs definitely aren't isomorphic - determining if they definitely

are is very complicated (not known to be NP complete!)

Netsoc (Stephen Shaw) <stesh@netsoc.tcd.ie> 2010 19 / 23
A simple graph

In [1]: import networkx as nx

In [2]: G = nx . Graph ()

In [3]: G . add_nodes_from ([1 ,2 ,3 ,4 ,5 ,6 ,7 ,8])

In [4]: G . add_edges_from ([(1 ,4) ,(6 ,7) ,(6 ,8) ,(3 ,4) ,(3 ,7) ,(1 ,8) ,(1 ,7) ])

In [5]: G . number_of_nodes ()
Out [5]: 8

In [6]: G [1]
Out [6]: {4: {} , 7: {} , 8: {}}

In [7]: nx . degree (G ,6)

Out [7]: 2

Netsoc (Stephen Shaw) <stesh@netsoc.tcd.ie> 2010 20 / 23

Plotting graphs

MatPlotLib and PyLab are `standard' plotting tools

Very high quality results
Several formats

In [8]: import matplotlib . pyplot as plt

In [9]: nx . draw ( G )

You can view plots on screen, or write them to any of several formats
Can customize colors, thickness, fonts, etc.

Netsoc (Stephen Shaw) <stesh@netsoc.tcd.ie> 2010 21 / 23

Plotting graphs

Netsoc (Stephen Shaw) <stesh@netsoc.tcd.ie> 2010 22 / 23

Annnyway...

We have only scratched the surface

Small-scale
IPython is really scalable: can drive entire clusters
All of these packages and modules are freely available
Questions?

Netsoc (Stephen Shaw) <stesh@netsoc.tcd.ie> 2010 23 / 23