Attributions

Usage is attributed in the speaker notes

​

1

Please do not redistribute these slides without prior written permission

2

CS 5500

Foundations of Software Engineering

Dr. Mike Shah

3

Pre-Class Warm up (1/4)

• 32- mind boggling facts about the internet: https://lifehacks.io/facts-about-the-internet/
• 4. Internet users consumed one zettabyte bandwidth in 2016.
• One Zettabyte is equal to a thousand Exabytes, a billion Terabytes, or trillion Gigabytes. By the year 2021, 82% of all IP traffic will be video, predicts Cisco.
• 13. Garfield the cartoon once offered its own Email service.
• Yes, Garfield, the cartoon character was offering an email service named GMail.com.
• Google later acquired the service, and they renamed it to Google Mail (GMail.com) as we all know today.

​

4

Pre-Class Warm up (2/4)

• By the year 2021, 82% of all IP traffic will be video, predicts Cisco.
• To be determined (TBD) if 82% of internet traffic is indeed video.
• I wonder if Cisco knew how 2020 would be the unfortunate ‘year of Zoom’

​

5

Pre-Class Warm up (3/4)

• Pretty good prediction!

6

Pre-Class Warm up (4/4)

• 2023 numbers say still say either 65% or 80%

7

Note to self: Start audio recording of lecture :)�(Someone remind me if I forget!)

8

“The Daily Scrum”

9

Last Class (“What did I do yesterday”) (1/2)

• Final Project Overview
• Basic 2D Graphics

10

Last Class (“What did I do yesterday”) (2/2)

• Questions or comments?

11

Course Logistics (“What are we going to do today”)

• Today
• Final Projects Released
• You’ll also get your team
• Assignments
• No more
• Project
• There will be a separate github repository where project will be completed.
• See spreadsheet above with new github repository -- click on the link after you get in touch with your teammates to coordinate your new repository name.

12

Blocking (“What is stopping forward progress”)

• Questions on our testing tools in DLang?
• Other open questions?

13

Getting Organized on our project

14

Husky Town - Base Requirements (1/2)

• Recall that we have a few basic requirements:
• 2D avatars
• Reasonable collisions with world
• Ability to see other 2D avatars in real-time on a network
• You have the ability to chat
• Theme: ‘sharing’ somehow has to be incorporated into your project

​

15

https://www.gather.town/

Husky Town - Base Requirements (2/2)

• Recall that we have a few basic requirements:
• 2D avatars
• Reasonable collisions with world
• Ability to see other 2D avatars in real-time on a network
• You have the ability to chat
• Theme: ‘sharing’ somehow has to be incorporated into your project

​

16

https://www.gather.town/

• Today we’ll look at a few ideas to help you get started.

2D TileMaps

Tilemap Demo (1/2)

• Provided is a brief demo on tilemaps
• You can find in the course github repository some sample code for this demo
• This demo show some ideas of moving a character around
• This demo shows some ideas of handling ‘collision’ against tiles.

18

Tilemap Demo (2/2)

• Holding the space key in the previous demo will show an idea of how to select individual tiles from a ‘tile map’
• This can be useful when building the world
• It also may present some ideas about ‘how’ to build or otherwise represent your world.

19

Tiled 2D Applications

• A common way to render a world in 2D is using tiles.
• Tiles are individual images that are drawn on a grid
• Editors for tile-based graphics are generally easy to work with -- behind the scenes there is an integer representing the tile to draw

20

2D Tile Maps

• A 2D tile map is a collection of individual textured sprites that make up a level.
• The image on the right shows several square tiles which make up a larger tile map.

Square 2D Tile maps

• Typically, a square tile map is the most common for a 2D platformer type game.
• Mario is the typical example
• A tilemap is a list of values stored in a container data structure (e.g. array)
• Typically a 2D array is the most convenient representation
• -1’s in this example could represent ‘no tile’
• Anything else represents a textured tile that is looked up in the tilemap texture atlas.

Tilemap atlas (1/3)

• Each value in the tilemap is referencing a tile value stored in a texture atlas
• It is typically divided up evenly for each tile.

Texture Atlas of many tiles

2D Tile map which references a texture atlas

Tilemap atlas (2/3)

• Each value in the tilemap is referencing a tile value stored in a texture atlas
• It is typically divided up evenly for each tile.

​

00

01

02

03

04

...

2D Tile map which references a texture atlas

Texture Atlas of many tiles

Tilemap atlas (3/3)

• Each value in the tilemap is referencing a tile value stored in a texture atlas
• It is typically divided up evenly for each tile.
• Each index in the tile map then corresponds to a ‘tile type’ (represented by an integer)
• i.e. 00 =

00

01

02

03

04

...

2D Tile map which references a texture atlas

Texture Atlas of many tiles

Another example [source] (1/2)

Another example [source] (2/2)

Utilizing Multiple Tile Maps

• Because a Tilemap is just a 2D array, you can have multiple tile maps
• Typically, we are interested in the ‘visual grid’
• Can also use it for collision detection
• Can use it for AI paths
• Can use it for game events/triggers
• Laying out items over the map

Layering (Draw Order)

• 2D applications utilize a layering system to determine the draw order
• The classic painter's algorithm works just fine
• i.e. draw the background first, then draw the foreground next. (Example below shows this)
• (Z-buffering is typically used in 3D)

​

29

Layering (2/3)

• In practice you may assign some integer for ‘order’ for your sprites.
• Then you would sort them.
• Background order = -10000
• characters and enemies = 1
• particle effects = 500
• GUI 10000

​

30

Layering (3/3)

• In practice you may assign some integer for ‘order’ for your sprites.
• Then you would sort them.
• Background order = -10000
• characters and enemies = 1
• particle effects = 500
• GUI 10000

31

10000

1

1

1

500

-10000

Tilemaps Array Access

32

Picking a Tile from an Atlas (1/3)

​

• Given this Tilemap, how can we figure out where to find in our texture atlas Tile ‘01’?

​

00

01

02

03

04

...

7 Columns

Picking a Tile from an Atlas (2/3)

• Given this Tilemap, how can we figure out where to find in our texture atlas Tile ‘01’?
• Given the tile type, say ‘01’ for example, we can select the ‘x’ and ‘y’ position of it in our array.

​

00

01

02

03

04

...

7 Columns

Picking a Tile from an Atlas (3/3)

• Given this Tilemap, how can we figure out where to find in our texture atlas Tile ‘01’?
• Given the tile type, say ‘01’ for example, we can select the ‘x’ and ‘y’ position of it in our array.
• row = (TileNumber / Col_Width)
• column = (TileNumber % Col_Width)
• ^That is our ‘selection’ function.
• 01 / 8 = 0th row
• 01 % 8 = 1st column
• i.e. Select tile at (0,1)

00

01

02

03

04

...

Col_width = 8

Nice illustration [source]

• Select the ‘row’ with division � (row increments every ‘w’ steps, where ‘w’ is width of array)
• Select the ‘column’ with %, which gives you remainder
• (column increments every step, and wraps around after surpassing ‘w’, where ‘w’ is width of array)

1D or 2D Arrays Data Structure [row-col-order]

• If you are dynamically creating an array in C++, it will look something like this:
• int* tiles = new [rows*cols]; �// Create an array
• This means we are working with a 1D array
• We need to then decide if we are using row-major or column-major order.
• (Note: It does not matter--as long as you are consistent!)

Accessing a 1D array

• Again, we need to follow some consistent behavior.
• I have provided a sample in today’s module.
• (next slide)

1D-Array example

Accessing a 1D array

12 Rows

6 Columns

Accessing a 1D array

0

1

2

3

4

5

6

7

8

9

10

11

12

0 1 2 3 4 5

Accessing a 1D array

0 1 2 3 4 5

0

1

2

3

4

5

6

7

8

9

10

11

12

Accessing a 1D array

Here is our target: �x = 3, y = 1

0 1 2 3 4 5

0

1

2

3

4

5

6

7

8

9

10

11

12

Accessing a 1D array

0 1 2 3 (Our x-axis)

Here is our target: �x = 3, y = 1

0

1�our

y-axis

Accessing a 1D array

Access array like we would on a coordinate system

Here is our target: �x = 3, y = 1

0 1 2 3 4 5

0

1

2

3

4

5

6

7

8

9

10

11

12

Accessing a 1D array

This is our ‘pitch or width’

6 is our pitch or ‘width’ or how many column items

Access array like we would on a coordinate system

Here is our target: �x = 3, y = 1

0

1

2

3

4

5

6

7

8

9

10

11

12

Accessing a 1D array

This is what ‘row’ we are on

Access array like we would on a coordinate system

Here is our target: �x = 3, y = 1

0

1

2

3

4

5

6

7

8

9

10

11

12

0 1 2 3 4 5

Accessing a 1D array

y*COLS thus moves us ‘up and down’

Access array like we would on a coordinate system

Here is our target: �x = 3, y = 1

0

1

2

3

4

5

6

7

8

9

10

11

12

0 1 2 3 4 5

Accessing a 1D array

This is what column we are on in our row (i.e. x--coordinate)

Access array like we would on a coordinate system

Here is our target: �x = 3, y = 1

0

1

2

3

4

5

6

7

8

9

10

11

12

0 1 2 3 4 5

Accessing a 1D array

[1 * 6 + 3] = 9 // the 9th tile at (3,1)

Access array like we would on a coordinate system

Here is our target: �x = 3, y = 1

0

1

2

3

4

5

6

7

8

9

10

11

12

0 1 2 3 4 5

Accessing a 1D array

0

1

2

3

4

5

6

7

8

9

10

11

12

A few more examples

0 1 2 3 4 5

Cameras

52

2D Cameras

• A virtual camera is the final part that makes up our virtual scene
• The camera defines how an object is projected onto the viewport (I.e. window)
• 2D games have an orthographic projection
• This makes sense, as there is no perspective. (Objects are as big as they are!)
• Typically they move smoothly along with the main character or advance once a player hits a checkpoint.

53

Camera Systems (2/2)

• Here is an example of a camera that follows the main character
• The camera is centered on the character, and shows a subset of the tilemap
• One strategy to achieve this, is as the character moves is to shift the tiles that are displayed.
• i.e. only a portion of a tilemap is shown at a given time.

​

Camera Positioning - Screen Coordinates

• The screen coordinates shown are labeled below.
• (See next slide for world coordinates)

0 1 2 3 4 5

6 7 8 9 10 11

12 13 14 15 ...

Camera Positioning - World Coordinates

• The world coordinates shown are labeled below for our tile map (as it is represented in memory)
• Note that these coordinates are different than what the character sees.
• Thus, some offset needs to be established.

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ...

Helper Functions

• An example function is shown here on how one can translate between world and screen coordinates.
• Good reference on tilemaps [source]
• This can help you determine which tiles to load from memory and display on the screen based on where the ‘camera is’

Tilemap Art Assets

58

Where to get royalty-free art assets?

• It’s important that you look at the license
• Using assets that are not ‘royalty free’ and available could get your company sued.
• Generally for academic and educational use there are some ‘fair use’ for non-commercial protections.
• We should try to follow best practices however.

59

https://kenney.nl/assets/tiny-town

This website had a few nice tilesets:

https://kenney.nl/assets/roguelike-modern-city

​

You might find others on your own

Some Suggestions

• In order to assist in your development efforts, it is fine to build other programs (e.g. a tiny tilemap editor) to make your life easier.

60

Short 5 minute break

• 3 hours and 15 minutes is a long time.
• I will try to never lecture for more than half of that time without some sort of ‘break’ or transition to an in-class activity/lab.
• Use this time to stretch, check your phones, eat/drink something, etc.

​

61

Husky Town - Base Requirements (2/2)

• Recall that we have a few basic requirements:
• 2D avatars
• Reasonable collisions with world
• Ability to see other 2D avatars in real-time on a network
• You have the ability to chat
• Theme: ‘sharing’ somehow has to be incorporated into your project

​

62

https://www.gather.town/

• Let’s talk a little bit about networking now

Networking Basics

A crash course into communicating data between processes (on the same or different machines)

63

Networking Crash Course

• We are working on a networked-based final project.
• So we need to learn a little a little bit about networking!
• Consider today a crash course in network programming.
• I am giving a short treatment of the subject to help you get started with writing networked applications
• You may otherwise consider this an advertisement if you want to take the Networking course (for those that have not)

64

https://catalog.northeastern.edu/graduate/computer-information-science/computer-science/#coursestext

The very basics - Sockets (1/4)

• We have previously been use to just creating ‘1’ process (i.e. an application) that we run.
• A process runs in a sense in it’s own ‘box’
• However, if we want to communicate with other processes (or other machines that otherwise host processes)--we need a mechanism to do so
• That is known as a socket

65

https://github.com/kusdavletov/socket-programming-simple-server-and-client

The very basics - Sockets (2/4)

• We have previously been use to just creating ‘1’ process (i.e. an application) that we run.
• A process runs in a sense in it’s own ‘box’
• However, if we want to communicate with other processes (or other machines that otherwise host processes)--we need a mechanism to do so
• That is known as a socket

66

https://github.com/kusdavletov/socket-programming-simple-server-and-client

The very basics - Sockets (3/4)

​

• Sockets are nothing more than a ‘file descriptor’
• That is, each process has a table of file descriptors for sockets.
• So a socket is ‘some integer’ in which I can communicate over a specific socket.
• From that socket I may read or write out data to the network
• Other sockets can then ‘listen’ for other sockets to send or receive data from.

67

https://github.com/kusdavletov/socket-programming-simple-server-and-client

The very basics - Sockets (4/4)

​

• Sockets are nothing more than a ‘file descriptor’
• That is, each process has a table of file descriptors for sockets.
• So a socket is ‘some integer’ in which I can communicate over a specific socket.
• From that socket I may read or write out data to the network
• Other sockets can then ‘listen’ for other sockets to send or receive data from.

68

https://github.com/kusdavletov/socket-programming-simple-server-and-client

If it’s useful analogy--imagine a human as a process, and a tin can as a ‘socket’. The string in between is the ‘network’ where you can listen or send data.

https://w1nnersclub.com/wp-content/uploads/2018/02/tin-can-telephone.jpg

The ‘Network’ and Network Layers

69

Network Programming

• When writing code for the network, we have to also think about the network hardware as well as the software component.
• Hardware
• Sometimes known as the physical layer
• Responsible for connecting computers (e.g. through a router) to various gateways into the internet
• Software on networking hardware devices has software that coordinates or manages traffic (in the form of ‘packets’)
• Software
• This is the software that we create, and we will utilize a socket to perform our networking.

70

(FYI) Physical Network Layers

• There are many layers in the physical network organized by geographic proximity
• SAN (System Area Network) - Spans over a cluster or machine room
• LAN (Local Area Network) Spans a building or campus
• WAN (Wide Area Network) Spans country or world
• The internetwork (internet) is an interconnected set of networks.
• The Global IP Internet (capital “I”) is the most important example.

71

Networking Crash Course:

https://www.youtube.com/watch?v=3QhU9jd03a0

Open Systems Interconnection (OSI)

• Depending on when you have taken networking (or in the future), you might have learn about ‘7’ layers.
• This is the OSI model
• Really today we just think about ‘4’ layers (next slide)

72

Simplified TCP/IP Model

• Same communication, but a different model
• TCP/IP model consisting of 4 layers
• We’re focused on the ‘host to host’ layer

73

Client-Server Model

A first model to understand networking

https://en.wikipedia.org/wiki/Client%E2%80%93server_model

​

74

Probably the most popular model is the Client-Server

• There is one server process
• The server manages resources, and provides some service for manipulating client resources
• There are one or more client processes
• The server gets alerted to do something by requests from clients.
• You can think of this sort of like a vending machine
• (The server holds all the resources, and it waits until a human makes some request)

75

How does communication take place over a socket?

(i.e., what information is being sent)

76

How is information sent?

• We are likely to have different machines (Mac, Windows, Linux, IPhone, Android, etc.), and within that hardware different networking hardware
• So even if the Local Area Networks (LANs) and Wide Area Networks (WANs) are incompatible however--bits(i.e., 1’s and 0’s) can still be sent back and forth.
• The solution to ensure these bits are understandable is to use a protocol
• A protocol is a software running on each host and router
• Protocol is a set of rules that governs how hosts and routers should cooperate when transferring data between networks

​

77

What does an Internet Protocol (IP) do?

• Provides a consistent naming scheme
• That is-- uniform format for host addresses to understand.
• Each host (and router) is assigned at least one unique internet address that identifies it.
• Provides a delivery mechanism
• The standard transfer unit is called a packet
• A packet consists of a header and a payload
• Header:
• Contains information such as packet size, source and destination addresses
• (Think of our ‘header block’ as the sender and return address on an envelope)
• Payload:
• Contains the data bits ent from source host.
• (Think of payload as whatever is inside the envelope)

78

Example Data Transfer (1/10)

​

79

Host A has some data

Example Data Transfer (2/10)

​

80

System call copies data into a kernel buffer

Example Data Transfer (3/10)

​

81

Data is fit into a packet using some standard protocol

Example Data Transfer (4/10)

​

82

The entire chunk is called a frame

Example Data Transfer (5/10)

​

83

LAN copies frame to network

Example Data Transfer (6/10)

​

84

Router reads frame using its protocol software

Example Data Transfer (7/10)

​

85

Router finds destination address from packet header

Example Data Transfer (8/10)

​

86

Router copies data to LAN2 adapter

Example Data Transfer (9/10)

​

87

Protocol software reads the frame, and strips out payload

Example Data Transfer (10/10)

88

System call is made to read data

Other Issues sending data

• What if data is lost?
• What if the data frames are different sizes?
• How do the routers forward on frames?
• more? (security?)

​

• All of these will be answered in a full networking course
• (A whole branch of computer science).
• I’m not necessarily expecting your application to be fully resilient to every type of error.
• This is a pretty decent resource to get more of an overview:
• https://wps.pearsoned.com/ecs_kurose_compnetw_6/216/55463/14198700.cw/index.html

​

89

Data Transmission Protocols

(i.e., TCP and UDP)

90

Internet (Uppercase ‘I’)

• Some notes about our most famous internet (i.e. The Internet)
• Internet is based on the TCP/IP protocol family
• IP (Internet Protocol)
• Describes the naming scheme and unreliable delivery capability of packets (datagrams) from host-to-host
• TCP (Transmission Control Protocol)
• Uses IP to provide reliable byte streams from process-to-process over connections
• Other protocols exist like UDP (Unreliable Datagram Protocol)
• Uses IP to provide unreliable datagram delivery from process-to-process
• Protocols accessed through Unix file I/O and functions from sockets interface.

91

TCP (Transmission Control Protocol)

• Ensures packets are transmitted with high reliability
• i.e. Packets must arrive, or otherwise we will be notified of an error
• Packets of data are sent in order specified
• In fact, it is guaranteed!
• Programmatically we do the following steps:

92

So per ‘client and server process’, our sockets are using a TCP/IP protocol to send and receive packets of information across the network

UDP (User Datagram Protocol) or (Unreliable Datagram Protocol)

• Packets of data may be lost or delivered out of order
• There is no buffer at either the sending or receiving end
• Unreliable but fast
• It is up to the application to deal with lost packets

93

TCP versus UDP visual

• Note the additional ‘request’ and ‘confirmation’ of connection on TCP versus UDP where we just request and send data immediately.
• Throughput on UDP thus is higher (2 data requests sent and received versus one)
• But, reliability is lower on UDP than TCP

94

https://www.educative.io/api/edpresso/shot/5669375870763008/image/5080565785034752

TCP versus UDP

Exercise: What does the audience think -- Pick which protocol may be better in the next slide

95

Should we use UDP or TCP Protocol for the following applications?

• Gmail?
• (i.e., formerly Garfield mail)

96

Should we use UDP or TCP Protocol for the following applications?

• Mario Kart?
• Online video game where you can race against your friends?

97

Should we use UDP or TCP Protocol for the following applications?

• Netflix, the popular streaming service?

98

Another visual of TCP/IP protocol

99

Hardware/Software Organization of Internet Application (1/5)

100

Hardware/Software Organization of Internet Application (2/5)

​

101

User programs using ‘sockets’ library

Hardware/Software Organization of Internet Application (3/5)

​

102

System calls are made using TCP/IP protocol

Hardware/Software Organization of Internet Application (4/5)

​

103

Network adapter takes data (previous diagram)

Hardware/Software Organization of Internet Application (5/5)

​

104

Reversed process

Programming View of Internet

IP address

105

How we (programmers) view Internet

• Hosts are mapped to a set of 32-bit IP addresses
• 23.5.219.215
• The set of IP addresses is mapped to a set of identifiers called Internet domain names
• 23.5.219.215 maps to www.northeastern.edu
• A process on one Internet host can communicate with a process on another Internet host over a connection

106

Quick Aside: IPv4 and IPv6

• Original Internet Protocol was 32-bit addressed--Internet Protocol Version 4 (IPv4)
• Now (well, in 1996) the Internet Engineering Task Force (IETF) introduced IPv6 with 128-bit addresses
• why?
• We would run out of internet addresses with only 32-bits!
• Most of the Internet traffic is still IPv4
• As of 2016, about 14% of the internet is starting to use IPv6
• Regardless, we should write code that is protocol independent.

107

Programmer View - IP Addresses

• To store a 32-bit IP Address, we can use an IP address struct
• IP Addresses are always stored in memory in network byte order
• (big-endian byte order - Most significant byte first)

​

​

• The convention is for a 32-bit IP address, the decimal value is separated by a period (Decimal dotted notation).
• IP address: 0x8002C2F2 = 128.2.194.242

108

Programmer View - Internet Connections (1/2)

• Clients and servers communicate by sending streams of bytes over connections
• Each connection is:
• Point-to-point: Connects a pair of processes
• Full-duplex: Data can flow in both directions at the same time
• Reliable: Stream of bytes sent by the source is eventually received by the destination in the same order it was sent.
• A socket is an endpoint of connection
• Socket address is an IP address:port pair
• A port is a 16-bit integer that identifies a process:
• Ephemeral port: Assigned automatically by client kernel when client makes a connection request
• Well-known port: Associated with some service provided by a server
• (e.g. port 80 is associated with web servers)

109

Programmer View - Internet Connections (2/2)

• Clients and servers communicate by sending streams of bytes over connections
• Each connection is:
• Point-to-point: Connects a pair of processes
• Full-duplex: Data can flow in both directions at the same time
• Reliable: Stream of bytes sent by the source is eventually received by the destination in the same order it was sent.
• A socket is an endpoint of connection
• Socket address is an IP address:port pair
• A port is a 16-bit integer that identifies a process:
• Ephemeral port: Assigned automatically by client kernel when client makes a connection request
• Well-known port: Associated with some service provided by a server
• (e.g. port 80 is associated with web servers)

110

How is the order enforced?

Ans: Based on the protocol

​

Well-known Service names and ports

• Popular services have permanently assigned well-known ports to these well-known services
• echo servers: echo 7
• ftp servers: ftp 21
• ssh servers: ssh 22
• email servers: smtp 25
• Web servers: http 80
• Mappings between well-known ports and services are found in /etc/services directory on linux machines

111

cat /etc/services | head -n 45

Anatomy of a connection

• A connection is uniquely identified by the socket addresses of its endpoints (socket pair)
• <Clientaddress:clientport, serveraddress:serverport>

112

You can use the Port to Identify the services

113

Attempt to talk to the web because the port is 80 (See the 80 at the end after the colon?)

(Aside:) More on portsPort

• A communication endpoint that’s tied with a service or resource
• To communicate over a network, you must know both the address and the port
• Port numbers are 16 bits wide
• 65,536 possible ports
• Ports have specific ports
• 1 - 1023 are systems ports with usually well known uses
• 1024 - 49151 are registered ports
• 49152 - 65535 are dynamic, private, or ephemeral
• You can find ports in: cat /etc/services

114

Sockets

Looking again at sockets communication at a low level

115

Sockets Interface

• Set of systems-level functions used with Unix I/O to build network applications
• Created in the early 80’s as part of the original Berkeley distribution of Unix which had early versions of Internet protocols!
• Sockets are available on all modern systems
• Unix variants
• Windows
• OSX
• IOS
• Android
• ARM
• etc

116

Sockets

• What is a socket?
• To an operating system kernel--sockets are endpoints of communication
• To an application, a socket is a file descriptor that lets the application read/write from and to the network
• Remember, all Unix I/O devices (even networks!) are modeled by files
• Note that sockets can both read and write
• So clients and servers communicate with each other by reading from and writing to socket descriptors
• The distinction between regular file I/O and socket I/O is how the application “opens” and “closes” the socket descriptors

117

fd suffix means “ file descriptor”

Server + Client Structure (1/7)

​

118

Server + Client Structure (2/7)

​

119

• Server starts up first
• Listens for requests from clients

Server + Client Structure (3/7)

​

120

Client launches

Server + Client Structure (4/7)

​

121

A tcp request is made to connect to a server

Server + Client Structure (5/7)

​

122

When connected, a data exchange can take place

Server + Client Structure (6/7)

​

123

Client disconnects when finished

Server + Client Structure (7/7)

124

Server drops client, and can disconnect

C Programmer View - Socket Address Structures

• Internet (IPv4) specific socket address:

125

Host and Service Conversion: getaddrinfo (1/2)

• getaddrinfo is the modern way to convert string representations to hostnames, host addresses, ports, and service names to socket address structures
• (gethostbyname and getservbyname were the old ones)
• All it does is translate www.stackoverflow.com to 69.59.196.211
• getaddrinfo helps us write portable protocol-independent code
• Works for both IPv4 and IPv6

​

126

Host and Service Conversion: getaddrinfo (2/2)

​

• Man page can help decipher some of the complexity
• Note that we have structs (addrinfo) that contain information we can use.

127

Host and Service Conversion: getnameinfo

• getnameinfo is the inverse of getaddrinfo, converting a socket address to the corresponding host and service

128

Sockets Interface (1/4)

129

Sockets Interface (2/4)

130

Grab address data

Sockets Interface (3/4)

131

Grab address data

Sockets Interface (4/4)

132

Then we can create a socket descriptor

Programmer View - Creating a socket (in C)

• Creating sockets is protocol specific.
• In this example we are creating one for IPV4 protocol in C code
• (You have some examples coming up)

133

Sockets Interface (1/5)

134

We created a socket descriptor

Sockets Interface (2/5)

135

Server does the same thing

Sockets Interface (3/5)

136

Server then uses ‘bind’ to associate server’s socket address to a socket descriptor

Sockets Interface (4/5)

137

listen then tells kernel to listen and accept requests from clients.

Sockets Interface (5/5)

138

Now server can ‘accept’ requests, and wait for requests from clients to connect

Short 5 minute break

• 3 hours and 15 minutes is a long time.
• I will try to never lecture for more than half of that time without some sort of ‘break’ or transition to an in-class activity/lab.
• Use this time to stretch, check your phones, eat/drink something, etc.

​

139

Networking in D

import std.socket;

140

import std.socket;

• DLang comes with it a std.socket library implementation
• This makes things relatively easy to work with.
• The sockets are based off the ‘C socket’ library
• Sample C code or Python code would be similar if you need additional resources.

141

https://dlang.org/phobos/std_socket.html

Example socket application

• Let’s take a look at a very simple socket application for retrieving our IP address to start.
• This socket program opens a socket to some known address (e.g. ‘Google’) that we are pretty sure is available.
• We can connect to that address, and then return some information about our socket:
• local address (which is our ip)
• remote address (where we connected the other end of our socket)
• This is a nice trick for also finding your ip

142

Example client/server application (1/3)

• Let’s do a quick code walk of the client and server from the github repository.
• ‘chat’ is the simplest example that opens a connection to the server and the server echos back the message sent.
• It’s not meant to be very resilient, but gives you an idea of how to use a socket.

143

Example client/server application (2/3)

• ‘chat_packet_custom’
• This example shows a pretty ‘raw’ output of a packet
• It’s actually quite useful to be able to interpret back and forth between bytes (that’s all data is sent along a socket) to your format.
• This introduces the idea of ‘serializing’ and ‘deserializing’ data
• You can do this with raw bytes
• or, you could also do this with .json (remember std.json?)

144

Example client/server application (3/3)

• ‘multithreaded_chat’
• This example shows some abstraction as well as how to handle multiple clients
• A difference from previous examples is that this example spawns a new thread to handle each client.
• Messages are also broadcast out back to all clients from the server

145

Networking Gotcha’s

146

Traps and Pitfalls (1/3)

• Firewalls may block your process
• (may need to set your application to be safe)
• Failure to close socket
• If a socket is not closed when exiting (remember a socket is essentially a file descriptor), then it may be seen as busy.
• Wait 30-60 seconds
• otherwise try a different port for communication.
• Localhost is otherwise useful for testing
• 127.0.0.1 - IPv4
• ::1 - IPv6
• In this course, we won’t check for things like overflows
• Consider a network security course

147

Traps and Pitfalls (2/3)

• Northeastern’s network/firewall may block traffic
• Why? Various ports may be reserved within the university
• Try to run your program from a hotspot or a different network
• Connection not able to be made
• Again, try a different port -- certain ports on Northeastern routers may be used for other services
• Also, make sure your program has been terminated
• ports open from previous executions may cause that port to be ‘in-use’

148

Traps and Pitfalls (3/3)

• Be careful with your ‘struct’ size of your packet.
• You want your bits tightly packed together
• One reason is performance -- less bits sent (not as much a concern in this course)
• The other thing to be aware of however, is that bits can get ‘padded’
• See here for an example: https://www.youtube.com/watch?v=0jYbgwKnNko&list=PLvv0ScY6vfd9Fso-3cB4CGnSlW0E4btJV&index=66

149

Question to Audience on TCP (1/2)

• With UDP we can sort of ‘spam’ a connection from multiple clients to a server and vice versa.
• With TCP Sockets, we have a 1 to 1 connection.
• Question to audience: How would I have a server accept multiple client connections?

150

Question to Audience on TCP (2/2)

• With UDP we can sort of ‘spam’ a connection from multiple clients to a server and vice versa.
• With TCP Sockets, we have a 1 to 1 connection.
• Question to audience: How would I have a server accept multiple client connections?
• Ans:
• Could have multiple threads. One thread spawned per collection, thread then runs an infinite loop listening for requests.
• Could also use ‘socket selector’ (SocketSet in DLang) which is a multiplexor.
• Allows you to read from multiple sockets
• See more here:
• Or maybe just create a bunch of sockets and do some bookkeeping

151

Client-Server Model For Project

• From a design perspective you’ll have to think a little about what role a ‘client’ and ‘server’ process play.
• Here’s two ideas:
• The first client to join could also be the ‘server’, but you will have to be able to handle requests as well as ‘broadcast’ responses to all clients
• Another is just to have a seperate ‘server’ process and a ‘client’ process
• This is more true to the model shown on the right

152

Should we use UDP or TCP Protocol for the following applications?

• Husky Town?
• Thoughts?
• No right answer--but you will have to choose
• Depending on your choice, may have to provide constraints on your design
• Protocol could also differ based on features...
• Thinking about undo/redo--look at google docs

153

How to find your IP?

• I recommend doing so programmatically with the program I provided earlier -- otherwise:
• Windows
• Settings / Network & Internet / Wifi / Then choose the wifi network
• Mac: ipconfig or ifconfig
• Linux
• ip addr show

154

Short 5 minute break

• 3 hours and 15 minutes is a long time.
• I will try to never lecture for more than half of that time without some sort of ‘break’ or transition to an in-class activity/lab.
• Use this time to stretch, check your phones, eat/drink something, etc.

​

155

Retrospective

• Anything unclear from the lecture before we move forward?

156

Project Time

157

Daily Wisdom

(Everyday wisdom)

158

Part of Your Project Grade

• In the past I have asked (as part of the final project grade) folks to fill out a survey.
• Here is part of it that you will fill out for yourself and your teammates.
• So I now want to talk about how to work well in a team
• (next slide!)

159

Wisdom on Teamwork! (1/2)

• Be responsive
• reply to emails--do not disappear!
• Communicate with your teammates
• “I read this and need to think about it more” is better than no answer for a week.
• Provide Constructive Criticism towards the code not the person
• (replace “You” with “the code”)
• Thank your teammates for their effort, and come into the conversation with an open mind
• “Hmm, can you explain what you were thinking here” vs “You are WRONG!”
• Ask for help from your team
• (Totally fine!)
• (Programming in pairs is expected at some level)

160

Wisdom on Teamwork! (2/2)

• Keep everyone in the loop
• (“I am struggling with this feature can we pair program”, “I have completed feature X and would like someone to review it”, “Let’s meet every X days on Teams/Skype/etc.”)
• Make everyone feel important and included
• (Speak to everyone, keep everyone cc’d on e-mails)
• Do you talk to everyone in the group when speaking or do you focus your attention intensely on one person?
• Do you pause to give others a time to ask a question?
• Do you interrupt and talk over others?

161

After teaching this class a few times...

• I have seen/experienced many team dynamics
• About 95% of the teams will work out pretty successfully
• I really liked your way of pseudo randomly assigning teams, I met new people, went out of my comfort zone to work with them and learnt a lot from them during the whole process. -- former student

162

After teaching this class a few times...

• Some things to look out for though....
• I think team’s should be encouraged to do more design upfront, perhaps include a pre-project milestone requiring a UML diagram to be done -- former student
• Words of wisdom to ‘design first’ then code.

163

Archetypes (1/3)

• The Ghost - The disappearing team member
• How to handle: Reach out to your project manager and me within the first week if you do not hear from them
• How to incorporate: Give them some responsibility so they can get back on track (sometimes stuff happens)
• ‘The Maverick/Hero Coder’ - Does work but does not communicate with team members on work done
• How to handle: Have a ‘standup’ time or regular time to report so you can communicate what was being done.
• How to incorporate: Have member pair program or teach other members what they have done.

164

Archetypes (2/3)

• The Coaster - Does not really do anything
• How to handle: Pair program. This is someone who has the skills but is otherwise going to coast by, handle at first milestone check-in
• How to incorporate: Pair program
• The Last Minute Contribution - Does nothing, gets worried, contributes a ton of code last 2-3 days.
• How to handle: Make project manager/instructor aware at project check-ins
• How to incorporate: Do need to distribute tasks to them at some point, perhaps gauge their level
• The ‘I’ll just do it all’ - Grabs a disproportionate amount of work (This is probably me...)
• How to handle: Pair program
• How to incorporate: Have them specifically distribute tasks

165

Archetypes (3/3)

• Note: Part of your grade will be from your peer evaluation to make this a pleasant experience in the sense that everyone should be invested.
• ...there are other archetypes within a team dynamic as well, perhaps you’ll have other strategies to handle these situations if they arise.
• 95% of the time things work well -- some semesters it’s 100%!
• Regardless of the ‘archetypes’ remember it’s a team.
• You win and lose together -- lift each other up.

166

Find your team

167

Action items

• Find your team
• https://docs.google.com/spreadsheets/d/1QAkHqViCW6nZ51PV3yL1WGZnjJ5kr0zRKY3I6kVOntI/edit?usp=sharing
• Do a ctrl+f to find your name -- Contact me if you do not see your name (or see your name twice)
• introduce yourself (“Hi my name is....”)
• Create an e-mail thread with your teammates
• (Decide if you want to use Teams, slack, etc. to coordinate)
• Create your team repository
• (See piazza or team spreadsheet for git invitation)
• (Teammates can join by also clicking on the git repository link and then select the team after it has been created)
• (Do not join other folks teams... :) )
• Start reading through project
• Statement of work, Milestone 0, and beyond!

168

In-Class Activity

169

In-Class Activity

• Complete the in-class activity from the schedule
• (Do this during class, not before :) )
• Please take 2-5 minutes to do so
• These make up a total of 5% of your grade
• We will review the answers shortly

170

Extra

171

Good Software Engineering

• We started off the lecture thinking about what makes a great software engineer
• One component is experience, and understanding how to tackle problems.
• Two key challenges in software engineering are
• Software is always changing.
• Many people work on the same software
• Languages (such as Dlang) have been developed to help tackle these challenges by providing useful abstractions to reason about software
• i.e. thinking about software as a collection of objects (Object-oriented programming)

172

Client-Server Model

173

Localhost

174

175

Some more code smells

• Allow caller (shown in the bottom) how to put out the data.

176

Today we’ll be working in Teams!

• Introduce yourself
• “Hello my name is”
• You don’t have to shake hands if someone has the flu...
• You should be ‘pair programming’
• That means working on one laptop to do any code implementation.
• You can simultaneously read or write responses to questions if you like.
• (At the end of lab, exchange e-mails and you can send your code files to your partners)
• If your partner is not here, just come up to the front, and I’ll reassign you.

177

Lab

• Meeting your team
• Exchange e-mails
• Try to figure out a regular time you would be free in the next few weeks to work on project
• Create a team repository on github
• click here to create: X
• (I will be pushing materials here in the next week or so with more information--including your Assignment 4)

178

C++

179

Other means of Networking

i.e., Two common protocols of well known services -- HTTP and FTP

180

HTTP Response and Request

• You can communicate directly with servers over port 80.
• These are known as HTTP requests and responses
• As an FYI, SFML provides facilities to do so
• (note: requests/responses are not encrypted using HTTPS so they are not secure)

181

FTP (File Transfer Protocol)

• Additionally, there is a specific protocol for communicating file and directory operations known as FTP
• Most often I personally use this for downloading resources from the web.

182

Networking in SFML

What is available to use in SFML

183

SFML and Networking

• We have two options for applying networking.
• 1.) Use the C-based sockets library that we have briefly seen.
• 2.) Use SFML’s C++ wrapper around the C sockets
• https://www.sfml-dev.org/documentation/2.5.1/modules.php
• For this course, I am going to point us towards SFML’s built in network library

184

SFML and Networking

• Note: This means we’ll have to link in sfml-network, so be sure to do that in your builds!

185

SFML Socket API

• https://www.sfml-dev.org/documentation/2.5.1/group__network.php#details
• SFML Provides socket-based communication using both UDP and TCP sockets
• There are also additional facilities for handling HTTP requests/responses as well as FTP for file transfer.

186

SFML Network Module

• https://www.sfml-dev.org/tutorials/2.5/#network-module
• One of the best places to get started with Networking is the SFML network module
• The previous materials we talked about in ‘C’ have been wrapped in SFML in C++ classes.

187

TCP Socket Example

• Documentation: https://www.sfml-dev.org/documentation/2.5.1/classsf_1_1TcpSocket.php#details
• ​

188

Words to the wise

• Lab today as far as deliverable is probably not too demanding time wise.
• That said, it’s worth thinking about taking and improving on the networking code as soon as possible.
• Discuss TCP versus UDP
• Thinking about building a ‘class’ for handling your network data.
• Think about what a ‘packet’ looks like.
• I would recommend browsing: https://www.sfml-dev.org/tutorials/2.5/network-packet.php
• The snippet below shows how you can create a new packet type
• ​

189