Today: modules, how the internet works

Finally Explain Other Code Lines in /

The example file begins..

#!/usr/bin/env python3

Stanford CS106A Pylibs Example
Nick Parlante

import sys
import random

def read_terms(filename):

1. #!/usr/bin/env python3

2. import sys

Modules and Modern Coding

Modules hold code for common problems, ready for your code to use. We say that you build your code "on top of" the libraries. Modern coding is part custom, and part building on top of module code.

alt:your code built on top of modules like sys

We ♥ Modules

Module = Name + Code

Syntax 1: import math

Syntax 2: math.sqrt(2)

>>> import math
>>> math.sqrt(2)  # call sqrt() fn
>>> math.sqrt

>>> math.log(10)
>>> math.pi       # constants in module too

Quit and restart the interpreter without the import, see common error:

>>> # quit and restart interpreter
>>> math.sqrt(2)  # OOPS forgot the import
Traceback (most recent call last):
NameError: name 'math' is not defined
>>> import math
>>> math.sqrt(2)  # now it works

Module = Dependency

1. "Standard" Modules - Fine

Many Standard Modules

2. Non-Standard "pip" Modules - Depends

Other modules are valuable but they are not a standard part of Python. For code using non-standard module to work, the module must be installed on that computer via the "pip" Python tool. e.g. for homeworks we had you pip-install the "Pillow" module with this command:

$ python3 -m pip install Pillow
..prints stuff...
Successfully installed Pillow-5.4.1

A non-standard module can be great, although the risk is harder to measure. The history thus far is that popular modules continue to be maintained. Sometimes the maintenance is picked up by a different group than the original module author.

Aside: Security of Pip Module

When you install a module on your machine from somewhere - you are trusting that code to run on your machine. In very rare cases, bad guys have tampered with modules to include malware in the module. Be more careful if installing a little used module. In contrast, python itself or standard modules like urllib are very safe as so many people use them.

Module Docs

Hacker: Use dir() and help() (optional)

>>> import math
>>> dir(math)
['__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'acos', 'acosh', 'asin', 'asinh', 'atan', 'atan2', 'atanh', 'ceil', 'copysign', 'cos', 'cosh', 'degrees', 'e', 'erf', 'erfc', 'exp', 'expm1', 'fabs', 'factorial', 'floor', 'fmod', 'frexp', 'fsum', 'gamma', 'gcd', 'hypot', 'inf', 'isclose', 'isfinite', 'isinf', 'isnan', 'ldexp', 'lgamma', 'log', 'log10', 'log1p', 'log2', 'modf', 'nan', 'pi', 'pow', 'radians', 'remainder', 'sin', 'sinh', 'sqrt', 'tan', 'tanh', 'tau', 'trunc']
>>> help(math.sqrt)
Help on built-in function sqrt in module math:

sqrt(x, /)
    Return the square root of x.
>>> help(math.cos)
Help on built-in function cos in module math:

cos(x, /)
    Return the cosine of x (measured in radians).

How to Create Your Own Module?

You already have! A regular old file is a module. Is a Module

How hard is it to write a module? Not hard at all. A regular Python file we have written works as a module too with whatever defs the file has.

alt: is a module named wordcount

Consider the file in

Forms a module named wordcount

Try this demo in the wordcount directory

>>> # Run interpreter in wordcount directory
>>> import wordcount
>>> wordcount.read_counts('test1.txt')
{'a': 2, 'b': 2}

dir() and help() work on wordcount Too

>>> dir(wordcount)
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'clean', 'main', 'print_counts', 'print_top', 'read_counts', 'sys']
>>> help(wordcount.read_counts)

Help on function read_counts in module wordcount. The text here comes from the """Pydoc""" you write at the top of a function.

    Given filename, reads its text, splits it into words.
    Returns a "counts" dict where each word

Babynames Module Example

# 1. In the file
# import the file in same directory
import babynames


    # 2. Call the read_files() function                                                                  
    names = babynames.read_files(FILENAMES)

Module Example: urllib

How Does The Web Work?

alt: web client makes request to server, gets back HTML response, browser draws the HTML


Here is the HTML code for is plain text with a bolded word in it, tags like <b> mark up the text.

This <b>bolded</b> text
HTML Experiment - View Source

Go to Try view-source command on this page (right click on page). Search for a word in the page text, such as "whether" .. to find that text in the HTML code.

Thing of how many web pages you have looked at - this is the code behind those pages. It's a text format! Lines of unicode chars!

urllib Demo

>>> import urllib.request
>>> f = urllib.request.urlopen('')
>>> text ='utf-8')
>>> text.find('Whether')
>>> text[26997:27100]
"Whether you're new to programming or an experienced developer, it's easy to learn and use Python.

# without >>>, for copy/paste
import urllib.request
f = urllib.request.urlopen('')
text ='utf-8')

Data From the Web vs. Files

How Does The Internet work?

Now does the Internet work? I pulled these slides together I had laying around from another class. Neat to see how something you use every day works.

Internet - TCP/IP Standards

1. Packets

packets transmitted on the wire

Sending One Packet

IP Address

computers with ip addresses

Sandra Bullock Blooper

TCP/IP blooper in this video of "The Net"...


Blooper: in the video the IP address is shown as 75.748.86.91 - not a valid IP address! Each number should be 1 byte, 0..255


router and its computers

The most common way for a computer to be "on the internet" is to establish a connection with a "router" which is already on the internet. The computer establishes a connection via, say, wifi to communicate packets with the router. The router is "upstream" of the computer, connecting the computer to the whole internet.

IP Packet - Hopping

Lots of Packet Hops

The packet is passed from router to router - called a "hop". There might be 10 or 20 hops in a typical internet connection.

packet proceeds by multiple hops

Router Lessons

The routing of a packet from your computer is like a capillary/artery system .. your computer is down at the capillary level, your packet gets forwarded up to larger and larger arteries, makes its way over to the right area, and then down to smaller and smaller capillaries again, finally arriving at its destination.

Note: Special "Local" IP Addresses

What Does it Mean to Be On the Internet?

So what does it mean for a computer to be on the internet? Typically it means the computer has established a connection with a router. The commonly used DHCP standard (Dynamic Host Configuration Protocol), facilitates connecting to a router; establishing a temporary connection, and the router gives your computer an IP address to use temporarily. Typically DHCP is used when you connect to a Wi-Fi access point.

Demo: Your Internet

Bring up the networking control panel of your computer. It should show what IP address you are currently using and the IP address of your router. You will probably see some text mentioning that DHCP is being used. Your computer will likely have a local IP address, just used while you're in this room.

Demo: Ping

"Ping" is an old and very simple internet utility. Your computer sends a "ping" packet to any computer on the internet, and the computer responds with a "ping" reply (not all computers respond to ping). In this way, you can check if the other computer is functioning and if the network path between you and it works. As a verb, "ping" is also used in regular English this way .. not sure if that's from the internet or the other way around.

Experiment: Most computers have a ping utility, or you can try "ping" on the command line (works on the Mac, Windows, and Linux). Try pinging or Not all computers respond to ping. Type ctrl-c to terminate ping.

Milliseconds fraction of a second used for the packet to go and come back. 1 ms = 1/1000 of a second. Different from bandwidth, this "round trip delay".

Here I run the "ping" program for a few addresses, see what it reports

$ ping  # I type in a command here
PING ( 56 data bytes
64 bytes from icmp_seq=0 ttl=53 time=8.219 ms
64 bytes from icmp_seq=1 ttl=53 time=5.657 ms
64 bytes from icmp_seq=2 ttl=53 time=5.825 ms
^C                            # Type ctrl-C to exit
--- ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 5.657/6.567/8.219/1.170 ms
$ ping
PING ( 56 data bytes
64 bytes from icmp_seq=0 ttl=64 time=0.686 ms
64 bytes from icmp_seq=1 ttl=64 time=0.640 ms
64 bytes from icmp_seq=2 ttl=64 time=0.445 ms
64 bytes from icmp_seq=3 ttl=64 time=0.498 ms
--- ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.445/0.567/0.686/0.099 ms


Traceroute is a program that will attempt to identify all the routers in between you and some other computer out on the internet - demonstrating the hop-hop-hop quality of the internet. Most computers have some sort of "traceroute" utility available if you want to try it yourself (not required). On Windows it's called "tracert" in Windows Power Shell, and it does not suppor the "-q 1" option below, but otherwise works fine.

Some routers are visible to traceroute and some not, so it does not provide completely reliable output. However, it is a neat reflection of the hop-hop-hop quality of the internet.

Traceroute is housed in the east bay - 13 hops we see here. The milliseconds listed is the round-trip delay.

$ traceroute -q 1
traceroute to (, 64 hops max, 52 byte packets
 1  rt-ac68u-b3f0 (  7.152 ms
 2 (  9.316 ms
 3 (  9.638 ms
 4 (  9.775 ms
 5 (  31.753 ms
 6 (  10.273 ms
 7 (  10.570 ms
 8 (  11.344 ms
 9 (  13.555 ms
10 (  11.583 ms
11 (  11.938 ms
12 (  14.833 ms
13 (  11.549 ms

Traceroute Serbia

A random Serbian address - 31 hops - the farthest thing I could fine. See the extra delay where the packets go across the Atlantic - I'm guessing hop 16. The names there may refer to Amsterdam and France. Note that the packets are going at a fraction of the speed of light here - a fundamental limit of how quickly you can get a packet across the earth.

$ traceroute -q 1
traceroute to (, 64 hops max, 52 byte packets
 1  rt-ac68u-b3f0 (  9.136 ms
 2 (  9.608 ms
 3 (  20.184 ms
 4 (  15.058 ms
 5 (  11.050 ms
 6 (  11.294 ms
 7 (  10.420 ms
 8 (  20.021 ms
 9 (  37.200 ms
10 (  36.318 ms
11 (  49.991 ms
12 (  66.591 ms
13 (  67.178 ms
14 (  77.369 ms
15 (  86.026 ms
16 (  152.559 ms
17 (  161.324 ms
18 (  164.945 ms
19 (  172.507 ms
20 (  197.670 ms
21 (  181.075 ms
22 (  184.336 ms
23 (  189.231 ms
24 (  182.364 ms
25 (  191.607 ms
26 (  187.181 ms
27  *
28 (  204.945 ms
29 (  192.673 ms
30 (  193.978 ms
31 (  193.032 ms !Z

TCP/IP Summary Picture

packet hopping across many routers