Chapter 7 Multimedia Networking Chapter 7: Goals Chapter 7

Multimedia, Quality of Service: What is it?
Chapter 7
Multimedia Networking
Multimedia applications:
network audio and video
(“continuous media”)
A note on the use of these ppt slides:
We’re making these slides freely available to all (faculty, students, readers).
They’re in PowerPoint form so you can add, modify, and delete slides
(including this one) and slide content to suit your needs. They obviously
represent a lot of work on our part. In return for use, we only ask the
‰ If you use these slides (e.g., in a class) in substantially unaltered form,
that you mention their source (after all, we’d like people to use our book!)
‰ If you post any slides in substantially unaltered form on a www site, that
you note that they are adapted from (or perhaps identical to) our slides, and
note our copyright of this material.
Computer Networking: A Top
Down Approach Featuring the
3rd edition.
Jim Kurose, Keith Ross
Addison-Wesley, July 2004.
Thanks and enjoy! JFK / KWR
All material copyright 1996-2004
J.F Kurose and K.W. Ross, All Rights Reserved
7: Multimedia Networking
network provides
application with level of
performance needed for
application to function.
7: Multimedia Networking
Chapter 7: Goals
Chapter 7 outline
❒ 7.1 Multimedia
❒ Classify multimedia applications
❒ Identify the network services the apps need
❒ Making the best of best effort service
❒ Mechanisms for providing QoS
Networking Applications
❒ 7.2 Streaming stored
audio and video
❒ 7.3 Real-time Multimedia:
Internet Phone study
❒ 7.4 Protocols for RealTime Interactive
Protocols and Architectures
❒ Specific protocols for best-effort
❒ Architectures for QoS
❒ 7.6 Beyond Best
❒ 7.7 Scheduling and
Policing Mechanisms
❒ 7.8 Integrated
Services and
❒ 7.9 RSVP
❒ 7.5 Distributing
Multimedia: content
distribution networks
7: Multimedia Networking
MM Networking Applications
Classes of MM applications:
1) Streaming stored audio
and video
2) Streaming live audio and
3) Real-time interactive
audio and video
Jitter is the variability
of packet delays within
the same packet stream
7: Multimedia Networking
Streaming Stored Multimedia
❒ Typically delay sensitive
end-to-end delay
delay jitter
❒ But loss tolerant:
infrequent losses cause
minor glitches
❒ Antithesis of data,
which are loss intolerant
but delay tolerant.
7: Multimedia Networking
❒ media stored at source
❒ transmitted to client
❒ streaming: client playout begins
before all data has arrived
❒ timing constraint for still-to-be
transmitted data: in time for playout
7: Multimedia Networking
Cumulative data
Streaming Stored Multimedia:
What is it?
1. video
2. video
Streaming Stored Multimedia: Interactivity
streaming: at this time, client
playing out early part of video,
while server still sending later
part of video
7: Multimedia Networking
❒ applications: IP telephony,
video conference, distributed
interactive worlds
❒ end-end delay requirements:
❍ audio: < 150 msec good, < 400 msec OK
• includes application-level (packetization) and network
• higher delays noticeable, impair interactivity
❒ session initialization
7: Multimedia Networking
TCP/UDP/IP: “best-effort service”
no guarantees on delay, loss
But you said multimedia apps requires ?
QoS and level of performance to be
? effective!
Today’s Internet multimedia applications
use application-level techniques to mitigate
(as best possible) effects of delay, loss
7: Multimedia Networking 7-11
how does callee advertise its IP address, port
number, encoding algorithms?
7: Multimedia Networking 7-10
Multimedia Over Today’s Internet
Interactive, Real-Time Multimedia
❒ Internet radio talk show
❒ Live sporting event
❒ playback buffer
❒ playback can lag tens of seconds after
❒ still have timing constraint
❒ fast forward impossible
❒ rewind, pause possible!
7: Multimedia Networking
Streaming Live Multimedia
VCR-like functionality: client can
pause, rewind, FF, push slider bar
❍ 10 sec initial delay OK
❍ 1-2 sec until command effect OK
❍ RTSP often used (more later)
timing constraint for still-to-be
transmitted data: in time for playout
3. video received,
played out at client
How should the Internet evolve to better
support multimedia?
Integrated services philosophy:
❒ Fundamental changes in
Internet so that apps can
reserve end-to-end
❒ Requires new, complex
software in hosts & routers
❒ no major changes
❒ more bandwidth when
❒ content distribution,
application-layer multicast
application layer
Differentiated services
❒ Fewer changes to Internet
infrastructure, yet provide
1st and 2nd class service.
What’s your opinion?
7: Multimedia Networking 7-12
A few words about audio compression
❒ Analog signal sampled
❒ Example: 8,000
at constant rate
samples/sec, 256
quantized values -->
64,000 bps
❒ Receiver converts it
back to analog signal:
telephone: 8,000
CD music: 44,100
❒ Each sample quantized,
i.e., rounded
Example rates
❒ CD: 1.411 Mbps
❒ MP3: 96, 128, 160 kbps
❒ Internet telephony:
5.3 - 13 kbps
e.g., 28=256 possible
quantized values
❒ Each quantized value
represented by bits
some quality reduction
8 bits for 256 values
A few words about video compression
❒ Video is sequence of
images displayed at
constant rate
e.g. 24 images/sec
❒ Digital image is array of
❒ Each pixel represented
by bits
❒ Redundancy
❍ spatial
❍ temporal
7: Multimedia Networking 7-13
Chapter 7 outline
adapt layers to available
7: Multimedia Networking 7-14
Streaming Stored Multimedia
❒ 7.1 Multimedia
Networking Applications
❒ 7.2 Streaming stored
audio and video
❒ 7.3 Real-time Multimedia:
Internet Phone study
❒ 7.4 Protocols for RealTime Interactive
❒ MPEG 1 (CD-ROM) 1.5
❒ MPEG2 (DVD) 3-6 Mbps
❒ MPEG4 (often used in
Internet, < 1 Mbps)
❒ Layered (scalable) video
❒ 7.6 Beyond Best
❒ 7.7 Scheduling and
Policing Mechanisms
❒ 7.8 Integrated
Services and
❒ 7.9 RSVP
Application-level streaming
techniques for making the
best out of best effort
❍ client side buffering
❍ use of UDP versus TCP
❍ multiple encodings of
Media Player
❒ jitter removal
❒ decompression
❒ error concealment
❒ graphical user interface
w/ controls for
❒ 7.5 Distributing
Multimedia: content
distribution networks
7: Multimedia Networking 7-15
Internet multimedia: simplest approach
7: Multimedia Networking 7-16
Internet multimedia: streaming approach
❒ audio or video stored in file
❒ files transferred as HTTP object
received in entirety at client
then passed to player
❒ browser GETs metafile
❒ browser launches player, passing metafile
audio, video not streamed:
❒ no, “pipelining,” long delays until playout!
7: Multimedia Networking 7-17
❒ player contacts server
❒ server streams audio/video to player
7: Multimedia Networking 7-18
Streaming Multimedia: Client Buffering
constant bit
rate video
playout at client
client playout
❒ This architecture allows for non-HTTP protocol between
server and media player
client video
constant bit
rate video
Cumulative data
Streaming from a streaming server
❒ Client-side buffering, playout delay compensate
❒ Can also use UDP instead of TCP.
for network-added delay, delay jitter
7: Multimedia Networking 7-19
Streaming Multimedia: Client Buffering
7: Multimedia Networking 7-20
Streaming Multimedia: UDP or TCP?
❒ server sends at rate appropriate for client (oblivious to
rate, d
variable fill
rate, x(t)
network congestion !)
❍ often send rate = encoding rate = constant rate
❍ then, fill rate = constant rate - packet loss
❒ short playout delay (2-5 seconds) to compensate for network
delay jitter
❒ error recover: time permitting
❒ send at maximum possible rate under TCP
❒ fill rate fluctuates due to TCP congestion control
❒ Client-side buffering, playout delay compensate
for network-added delay, delay jitter
❒ larger playout delay: smooth TCP delivery rate
❒ HTTP/TCP passes more easily through firewalls
7: Multimedia Networking 7-21
Streaming Multimedia: client rate(s)
1.5 Mbps encoding
28.8 Kbps encoding
Q: how to handle different client receive rate
❍ 28.8 Kbps dialup
❍ 100Mbps Ethernet
A: server stores, transmits multiple copies
of video, encoded at different rates
7: Multimedia Networking 7-23
7: Multimedia Networking 7-22
User Control of Streaming Media: RTSP
❒ Does not target multimedia
❒ No commands for fast
forward, etc.
RTSP: RFC 2326
❒ Client-server application
layer protocol.
❒ For user to control display:
rewind, fast forward,
pause, resume,
repositioning, etc…
What it doesn’t do:
❒ does not define how
audio/video is encapsulated
for streaming over network
❒ does not restrict how
streamed media is
transported; it can be
transported over UDP or
❒ does not specify how the
media player buffers
7: Multimedia Networking 7-24
RTSP: out of band control
FTP uses an “out-of-band”
control channel:
❒ A file is transferred over
one TCP connection.
❒ Control information
(directory changes, file
deletion, file renaming,
etc.) is sent over a
separate TCP connection.
❒ The “out-of-band” and “inband” channels use
different port numbers.
RTSP Example
RTSP messages are also sent
❒ RTSP control messages
use different port numbers
than the media stream:
Port 554
❒ The media stream is
❒ metafile communicated to web browser
❒ browser launches player
❒ player sets up an RTSP control connection, data
connection to streaming server
considered “in-band”.
7: Multimedia Networking 7-25
Metafile Example
7: Multimedia Networking 7-26
RTSP Operation
<group language=en lipsync>
<track type=audio
src = "rtsp://">
<track type=audio
e="DVI4/16000/2" pt="90 DVI4/8000/1"
<track type="video/jpeg"
7: Multimedia Networking 7-27
RTSP Exchange Example
7: Multimedia Networking 7-28
Chapter 7 outline
C: SETUP rtsp:// RTSP/1.0
Transport: rtp/udp; compression; port=3056; mode=PLAY
S: RTSP/1.0 200 1 OK
Session 4231
C: PLAY rtsp:// RTSP/1.0
Session: 4231
Range: npt=0C: PAUSE rtsp:// RTSP/1.0
Session: 4231
Range: npt=37
❒ 7.1 Multimedia Networking
❒ 7.2 Streaming stored
audio and video
❒ 7.3 Real-time Multimedia:
Internet Phone case study
❒ 7.4 Protocols for RealTime Interactive
C: TEARDOWN rtsp:// RTSP/1.0
Session: 4231
S: 200 3 OK
7: Multimedia Networking 7-29
❒ 7.6 Beyond Best
❒ 7.7 Scheduling and
Policing Mechanisms
❒ 7.8 Integrated
Services and
❒ 7.9 RSVP
❒ 7.5 Distributing
Multimedia: content
distribution networks
7: Multimedia Networking 7-30
Real-time interactive applications
Interactive Multimedia: Internet Phone
Introduce Internet Phone by way of an example
❒ PC-2-PC phone
❍ instant messaging
services are providing
❒ PC-2-phone
❒ speaker’s audio: alternating talk spurts, silent
Going to now look at
a PC-2-PC Internet
phone example in
64 kbps during talk spurt
❒ pkts generated only during talk spurts
❍ Net2phone
❒ videoconference with
20 msec chunks at 8 Kbytes/sec: 160 bytes data
❒ application-layer header added to each chunk.
❒ Chunk+header encapsulated into UDP segment.
❒ application sends UDP segment into socket every
20 msec during talkspurt.
7: Multimedia Networking 7-31
Delay Jitter
congestion (router buffer overflow)
❒ delay loss: IP datagram arrives too late for
playout at receiver
delays: processing, queueing in network; end-system
(sender, receiver) delays
typical maximum tolerable delay: 400 ms
Cumulative data
❒ network loss: IP datagram lost due to network
constant bit
constant bit
rate playout
at client
Internet Phone: Packet Loss and Delay
7: Multimedia Networking 7-32
❒ loss tolerance: depending on voice encoding, losses
concealed, packet loss rates between 1% and 10%
can be tolerated.
client playout
❒ Consider the end-to-end delays of two consecutive
packets: difference can be more or less than 20
7: Multimedia Networking 7-33
Internet Phone: Fixed Playout Delay
❒ Receiver attempts to playout each chunk exactly q
msecs after chunk was generated.
❍ chunk has time stamp t: play out chunk at t+q .
❍ chunk arrives after t+q: data arrives too late
for playout, data “lost”
❒ Tradeoff for q:
❍ large q: less packet loss
❍ small q: better interactive experience
7: Multimedia Networking 7-34
Fixed Playout Delay
• Sender generates packets every 20 msec during talk spurt.
• First packet received at time r
• First playout schedule: begins at p
• Second playout schedule: begins at p’
playout schedule
p' - r
playout schedule
7: Multimedia Networking 7-35
7: Multimedia Networking 7-36
Adaptive playout delay II
Adaptive Playout Delay, I
❒ Goal: minimize playout delay, keeping late loss rate low
❒ Approach: adaptive playout delay adjustment:
Estimate network delay, adjust playout delay at beginning of
each talk spurt.
Silent periods compressed and elongated.
Chunks still played out every 20 msec during talk spurt.
Also useful to estimate the average deviation of the delay, vi :
vi = (1 − u )vi −1 + u | ri − ti − d i |
The estimates di and vi are calculated for every received packet,
although they are only used at the beginning of a talk spurt.
For first packet in talk spurt, playout time is:
t i = timestamp of the ith packet
pi = ti + d i + Kvi
ri = the time packet i is received by receiver
pi = the time packet i is played at receiver
where K is a positive constant.
ri − t i = network delay for ith packet
d i = estimate of average network delay after receiving ith packet
Remaining packets in talkspurt are played out periodically
Dynamic estimate of average delay at receiver:
d i = (1 − u )d i −1 + u ( ri − ti )
where u is a fixed constant (e.g., u = .01).
7: Multimedia Networking 7-37
Recovery from packet loss (1)
Adaptive Playout, III
Q: How does receiver determine whether packet is
first in a talkspurt?
❒ If no loss, receiver looks at successive timestamps.
difference of successive stamps > 20 msec -->talk spurt
❒ With loss possible, receiver must look at both time
stamps and sequence numbers.
7: Multimedia Networking 7-38
difference of successive stamps > 20 msec and sequence
numbers without gaps --> talk spurt begins.
forward error correction
(FEC): simple scheme
❒ for every group of n
chunks create a
redundant chunk by
exclusive OR-ing the n
original chunks
❒ send out n+1 chunks,
increasing the bandwidth
by factor 1/n.
❒ can reconstruct the
original n chunks if there
is at most one lost chunk
from the n+1 chunks
7: Multimedia Networking 7-39
Recovery from packet loss (2)
2nd FEC scheme
• “piggyback lower
quality stream”
• send lower resolution
audio stream as the
redundant information
• for example, nominal
stream PCM at 64 kbps
and redundant stream
GSM at 13 kbps.
• Whenever there is non-consecutive loss, the
receiver can conceal the loss.
• Can also append (n-1)st and (n-2)nd low-bit rate
7: Multimedia Networking 7-41
❒ Playout delay needs to
be fixed to the time to
receive all n+1 packets
❒ Tradeoff:
❍ increase n, less
bandwidth waste
❍ increase n, longer
playout delay
❍ increase n, higher
probability that 2 or
more chunks will be
7: Multimedia Networking 7-40
Recovery from packet loss (3)
❒ chunks are broken
up into smaller units
❒ for example, 4 5 msec units
per chunk
❒ Packet contains small units
from different chunks
❒ if packet is lost, still have
most of every chunk
❒ has no redundancy overhead
❒ but adds to playout delay
7: Multimedia Networking 7-42
Summary: Internet Multimedia: bag of tricks
❒ use UDP to avoid TCP congestion control (delays)
for time-sensitive traffic
❒ client-side adaptive playout delay: to compensate
for delay
❒ server side matches stream bandwidth to available
client-to-server path bandwidth
chose among pre-encoded stream rates
dynamic server encoding rate
❒ error recovery (on top of UDP)
❍ FEC, interleaving
❍ retransmissions, time permitting
❍ conceal errors: repeat nearby data
Chapter 7 outline
❒ 7.1 Multimedia
Networking Applications
❒ 7.2 Streaming stored
audio and video
❒ 7.3 Real-time Multimedia:
Internet Phone study
❒ 7.4 Protocols for RealTime Interactive
❒ 7.6 Beyond Best
❒ 7.7 Scheduling and
Policing Mechanisms
❒ 7.8 Integrated
Services and
❒ 7.9 RSVP
❒ 7.5 Distributing
Multimedia: content
distribution networks
7: Multimedia Networking 7-43
Real-Time Protocol (RTP)
❒ RTP specifies a packet
structure for packets
carrying audio and
video data
❒ RFC 1889.
❒ RTP packet provides
payload type
packet sequence
7: Multimedia Networking 7-44
RTP runs on top of UDP
❒ RTP runs in the end
❒ RTP packets are
encapsulated in UDP
❒ Interoperability: If
two Internet phone
applications run RTP,
then they may be able
to work together
RTP libraries provide a transport-layer interface
that extend UDP:
• port numbers, IP addresses
• payload type identification
• packet sequence numbering
• time-stamping
7: Multimedia Networking 7-45
RTP Example
❒ Consider sending 64
kbps PCM-encoded
voice over RTP.
❒ Application collects
the encoded data in
chunks, e.g., every 20
msec = 160 bytes in a
❒ The audio chunk along
with the RTP header
form the RTP packet,
which is encapsulated
into a UDP segment.
7: Multimedia Networking 7-46
RTP and QoS
❒ RTP header indicates
type of audio encoding
in each packet
sender can change
encoding during a
❒ RTP header also
contains sequence
numbers and
7: Multimedia Networking 7-47
❒ RTP does not provide any mechanism to ensure
timely delivery of data or provide other quality of
service guarantees.
❒ RTP encapsulation is only seen at the end systems:
it is not seen by intermediate routers.
Routers providing best-effort service do not make any
special effort to ensure that RTP packets arrive at the
destination in a timely matter.
7: Multimedia Networking 7-48
RTP Header
RTP Header (2)
❒ Timestamp field (32 bytes long). Reflects the sampling
Payload Type (7 bits): Indicates type of encoding currently being
used. If sender changes encoding in middle of conference, sender
informs the receiver through this payload type field.
•Payload type 0: PCM mu-law, 64 kbps
•Payload type 3, GSM, 13 kbps
•Payload type 7, LPC, 2.4 kbps
•Payload type 26, Motion JPEG
•Payload type 31. H.261
•Payload type 33, MPEG2 video
instant of the first byte in the RTP data packet.
❍ For audio, timestamp clock typically increments by one
for each sampling period (for example, each 125 usecs
for a 8 KHz sampling clock)
❍ if application generates chunks of 160 encoded samples,
then timestamp increases by 160 for each RTP packet
when source is active. Timestamp clock continues to
increase at constant rate when source is inactive.
❒ SSRC field (32 bits long). Identifies the source of the RTP
Sequence Number (16 bits): Increments by one for each RTP packet
sent, and may be used to detect packet loss and to restore packet
stream. Each stream in a RTP session should have a distinct
7: Multimedia Networking 7-49
7: Multimedia Networking 7-50
RTSP/RTP Programming Assignment
Real-Time Control Protocol (RTCP)
❒ Build a server that encapsulates stored video
❒ Works in conjunction with
frames into RTP packets
grab video frame, add RTP headers, create UDP
segments, send segments to UDP socket
include seq numbers and time stamps
client RTP provided for you
❒ Also write the client side of RTSP
❍ issue play and pause commands
❍ server RTSP provided for you
❒ Each participant in RTP
session periodically
transmits RTCP control
packets to all other
❒ Each RTCP packet contains
sender and/or receiver
of packets sent, number of
packets lost, interarrival
jitter, etc.
❒ Feedback can be used to
control performance
❍ Sender may modify its
transmissions based on
report statistics useful to
7: Multimedia Networking 7-51
RTCP - Continued
❒ Statistics include number
7: Multimedia Networking 7-52
RTCP Packets
- For an RTP session there is typically a single multicast address; all RTP
and RTCP packets belonging to the session use the multicast address.
- RTP and RTCP packets are distinguished from each other through the use of
distinct port numbers.
Receiver report packets:
❒ fraction of packets
lost, last sequence
number, average
interarrival jitter.
Sender report packets:
❒ SSRC of the RTP
stream, the current
time, the number of
packets sent, and the
number of bytes sent.
Source description
❒ e-mail address of
sender, sender's name,
SSRC of associated
RTP stream.
❒ Provide mapping
between the SSRC and
the user/host name.
- To limit traffic, each participant reduces his RTCP traffic as the number
of conference participants increases.
7: Multimedia Networking 7-53
7: Multimedia Networking 7-54
Synchronization of Streams
❒ RTCP can synchronize
different media streams
within a RTP session.
❒ Consider videoconferencing
app for which each sender
generates one RTP stream
for video and one for audio.
❒ Timestamps in RTP packets
tied to the video and audio
sampling clocks
❍ not tied to the wallclock time
RTCP Bandwidth Scaling
❒ Each RTCP sender-report
packet contains (for the
most recently generated
packet in the associated
RTP stream):
timestamp of the RTP
wall-clock time for when
packet was created.
❒ Receivers can use this
association to synchronize
the playout of audio and
7: Multimedia Networking 7-55
❒ RTCP attempts to limit its
❒ The 75 kbps is equally shared
traffic to 5% of the
among receivers:
session bandwidth.
❍ With R receivers, each
receiver gets to send RTCP
traffic at 75/R kbps.
❒ Suppose one sender,
sending video at a rate of 2 ❒ Sender gets to send RTCP
Mbps. Then RTCP attempts
traffic at 25 kbps.
to limit its traffic to 100
❒ Participant determines RTCP
packet transmission period by
❒ RTCP gives 75% of this
calculating avg RTCP packet
rate to the receivers;
size (across the entire
remaining 25% to the
session) and dividing by
allocated rate.
7: Multimedia Networking 7-56