Monitor Master - Product Portfolio Stationary Testing

- Find problems before your customers do
- Get the diagnostics you need to fix them
- Keep services available
Stationary Test Portfolio
Your Network:
The Customer Perspective
Understanding, measuring, and improving the
quality of your network services is vitally important to keeping your customers happy and
ultimately, subscribed. We all know this, but
how can we achieve it?
The simple answer: Monitor Master. With
over ten years of development and experience
behind it and over 100 customers worldwide,
Monitor Master is the best Quality of Service
measurement and service monitoring tool on
the market.
Your Challenges
You want to know your subscribers are getting a good service
– and be able to prove it
n Some problems can’t be detected from inside the network
n If there’s a problem, you need to know immediately
n When there is a problem, you need detailed information to fix it
– fast
n You need to monitor a wide range of services now and even more
in the future
Our Solution
Monitor Master for end-to-end monitoring, regression testing,
content testing, roaming, and revenue assurance
n Used by regulators, operators, and content providers
n Active test probes run 24x7 to test any service on any bearer
n Supports LTE devices
n Provides detailed diagnostic information for root cause analysis
n Threshold-based alarms for automatic problem alerts
n Centralized reporting and data warehouse for short-term, longterm, and trend analysis reports
Monitor Master – Product Portfolio Stationary Testing Key Points
End-to-end service monitoring
Centrally managed
n Extensive bearer support
n Wide variety of services supported
n Central SIM server
n Highly flexible and configurable reporting
n Monitor your services from the most
important viewpoint: the customer’s
n Configure automatic alerts
n Forecast problems before they happen­
n Adapt easily to emergent technologies
without product upgrades
n Diagnose and resolve faults fast with
detailed trace information
n Triple play, IMS, and convergence testing
Monitor Master Overview
Monitor Master is a software and hardware product for measuring the
end-to-end quality and user experience of wireless and fixed line data and
voice services. Monitor Master forms part of Ascom’s Symphony network
testing product range, which includes mobile drive-testing and optimization solutions.
Monitor Master has been designed to be a highly flexible product. It
has been used by our customers around the world for a variety of activities
including large-scale revenue assurance testing, content testing, prelaunch
multi-modal service testing and roaming solutions. Monitor Master is the
perfect solution for all your testing needs.
Proprietary Hardware
The Monitor Master platform consists of stationary remote agents
which run tests over numerous different types of wireless and fixed bearers
(LTE, GSM, GPRS, UMTS, HSDPA, CDMA, PSTN, WiFi, or LAN). These agents
can be situated anywhere in the world and are centrally controlled by the
Quality Manager management system.
Central Management
Quality Manager provides easy access to test scheduling, test script
editors, data warehousing, and reporting engines – a single point of access
to control and benefit from the whole system.
SIM Multiplexing
Each modem in an agent uses a local SIM card or a SIM stored in a centrally located SIM server. Any SIM in the SIM server can be delivered to any
one of the remote agents over an IP connection and used for testing. This
approach allows SIMs and agents to be located in totally different countries
or continents, with easy SIM management from a central location.
Monitor Master – Product Portfolio Stationary Testing 3
Supported Services
Monitor Master supports a wide range of technologies and services
available to your customers.
Monitor Master has been developed with flexibility in mind. It is not
fixed in its capabilities or scope of testing, but can test almost any service
and provide a wide variety of metrics and diagnostic information at the
same time.
With its ability to simulate a user performing a wide range of activities on a mobile device, Monitor Master is the most comprehensive test and
monitoring platform on the market.
n Instant messaging
n Email (POP3, SMTP,
Web browsing
Web 2.0
n IP testing
n Video streaming
n Video quality MOS
n Mobile broadband
Voice calls
n Voice quality PESQ
n iPhone Visual
n Ringback tones
n Voice over IP
Some of the services and technologies supported by Monitor Master
Monitor Master – Product Portfolio Stationary Testing 4
Monitor Master Architecture
Monitor Master components are built using our Quality Manager
architecture­, which provides the level of flexibility, reliability, and performance demanded by production environments. Over 100 customers worldwide use this proven architecture to provide their network quality of service
The platform architecture has been designed with scalability in mind,
with a central processing and management server, Quality Manager, which
can manage and control a large number of test probes via IP.
Resource checks are performed by Quality Manager to
ensure that scheduling conflicts do not occur
Model test scenarios quickly and easily with drag-anddrop. No programming needed!
Quality Manager – Central Management System
Quality Manager forms the central hub of the system. It performs the
following functions in the system:
n System administration:
- SIM management
- Probe and modem management (including remote SW updates)
- User management (roles/permissions/access)
n Test script management
n Test creation (through web-based deployment of Story Boarder
n Reporting (through Box Office)
n Scheduling of individual test cases and complex test sequences
The Quality Manager server itself is a modular platform, built from a
number of subcomponents responsible for different tasks. Each of these
components can be placed on different servers to increase performance
and availability.
Story Boarder – Test Modeling
Story Boarder enables quick and easy test modeling through its dragand-drop interface, allowing prewritten modules (such as “Download FTP”
and “Browse Page”) to be linked together and appropriately configured,
forming a deployable test scenario.
With hundreds of prewritten modules to choose from, it’s easy to
create­a test that closely matches the interactions you wish to emulate.
Story Boarder can also run VBScript directly, allowing the test creator
to customise a script in any way they please. This also provides an interface
to control and run other third-party software.
Alarm Manager – OSS integration
Alarm Manager plugs seamlessly into Quality Manager to enable
authorized­users to create alarms based on test criteria such as a KPI
crossing­a threshold, or a particular diagnostic message being generated.
Alarm Manager supports SNMP v2c. Heartbeats are supported, as well
as multiple northbound recipients. Alarm Manager can also generate email
Monitor Master has been integrated with various OSS products such as
HP Openview, BMC Patrol, IBM Tivoli Netcool, and Vallent ServiceAssure.
Monitor Master has carrier grade alarming capabilities
that integrate easily with your existing OSS platform
Monitor Master – Product Portfolio Stationary Testing 5
Projector – Data Consolidation
Projector is a server that processes measurement data from the distributed probes and consolidates it into a data warehouse structure for
long-term storage and analysis.
Projector is a highly scalable and robust data warehouse solution
based on Microsoft® SQL Server. It can handle hundreds of incoming reports
per minute, from hundreds of agents simultaneously.
For smaller installations Projector can use a single SQL Server instance­;
for large installations there is full support for distributed SQL Server
Box Office Reporting Tool
Box Office – Reporting Platform
Box Office is our tool for creating and running reports based on test
Box Office uses a scalable, secure, and customisable framework that
allows developers to modify existing standard reports or create new reports
to suit the exact requirements of the end users.
Some of the benefits of Box Office:
n Easily customizable
n Wide variety of report types and graphs available
n View data from multiple sources
n Reports can auto-update – ideal for network monitoring dashboards
n User access and security control implemented
n Users only need Microsoft® Internet Explorer to view and modify
n Format data according to your rules (e.g. color code cells in tables
according to value)
See the separate section on Box Office (below) for more details.
Monitor Master – Product Portfolio Stationary Testing 6
Measurement Probes
Monitor Master software can run on a number of different platforms.
The simplest platform is a PC/Laptop running only a Monitor Master Software
Agent: this implementation is for customers who only require LAN/IP testing­
capabilities. To test over-the-air service, one of Ascom’s proprietary hardware agents is required.
Three different types of agent hardware are available at present, to
meet the varied requirements that you may have. Although each agent is
different in terms of capacity and size, they all support or implement the
following common technologies:
nMicrosoft® Windows XP based operating system
PSTN, and ISDN interfaces
n SIM multiplexing in conjunction with the SIM server platform
n USB interface control allowing integration of USB devices, such as
modems for mobile broadband testing
Each agent is capable of running tests developed with Story Boarder,
the test creation tool, and automatically updates the data warehouse with
test data and metrics once a test has completed.
Agent X1100
Agent X1100
The Agent X1100 is a rack-mountable (19” rack, 3U high) hardware
appliance designed for large-volume service testing. It is capable of controlling up to eight modems or phones simultaneously. The modem “blades”
(four slots) can be swapped and are easily inserted and allows for easy field
upgrades as new faster modems are available.
Agent X219
Agent X219
The Agent X219 is a 1U high hardware appliance designed for mounting­
in a standard 19” rack. It can contain up to three modem modules and a
PSTN modem, as well as WiFi connection.
Monitor Master – Product Portfolio Stationary Testing 7
SIM Multiplexing
SIM Server
The Monitor Master SIM Server is the key to a flexible and highly
scalable­SIM multiplexing solution enabling central SIM storage and distribution. A single SIM Server can store up to 336 SIM cards. The architecture
allows for easy plug-and-play expansion. There is no limit on the number of
SIM Servers and the system has been proven in live operations supporting
more than 300 probes delivering several hundreds of SIM images.
The SIM multiplexing platform supports SIM cards and Universal SIM
cards (USIMs).
Device Profiles
Monitor Master Quality Manager
Test modeling
Test planning
Ad-hoc testing
Resource allocation
Data integration
Data warehouse
Reporting & analysis
Alarm & trap management
Country A
Remote network
Country B
SIM Server
Flow of
SIM Details
Overview of the Monitor Master architecture and SIM multiplexer
SIM details are delivered to any agent via an IP connection
Monitor Master – Product Portfolio Stationary Testing 8
Voice Calling
Monitor Master has full support for peer-to-peer voice calling on and
between all available voice bearers, including public switched telephone
networks (PSTN), circuit switched dialling (CSD), and voice over IP (VoIP).
The calling and receiving parties can be on the same or different
probes, which can be in different locations. This allows you to create any
voice call situation you require. Monitor Master records a variety of KPIs (see
Typical KPIs) for each party.
Key Points
PSTN, CSD, and VoIP support
Voice Quality PESQ
n Multi-party calls across geography
n Voicemail deposit and retrieval
n IVR/DTMF capability
n Audio recording, playback, and
n Call waiting, barring, forwarding,
and holding; CLIP and CLIR support
n Ringback tones
Voice Quality
Monitor Master can be used to measure the perceived quality of a
voice call in full compliance with ITU-T standard P.862, using the Perceived
Evaluation of Speech Quality (PESQ) algorithm.
PESQ provides an objective measurement that predicts the results of
subjective listening tests on telephony systems. The PESQ algorithm compares the original unprocessed signal with the degraded version recorded on
the voice call and can return either PESQ, PESQ-LQ (MOS), or PESQ-Ie score.
Reference Sample
Typical KPIs
Call Setup Time
Call Answer Time
n Call Duration
n Call Quality Measurement (PESQ)
n Voicemail Answer Time
n Voicemail Notification Time
quality score
The PESQ system for analysis of degraded voice calls
IVR , DTMF and Voicemail
Monitor Master can automatically navigate IVR systems, sending DTMF
tones and recording and comparing audio where required.
This capability allows Monitor Master to dial up voicemail systems,
leave voicemails, wait for notifications and retrieve voicemail, checking the
automated audio played by the voicemail system along the way. It provides
KPIs for each interaction made. In a similar way Ringback tone services can
be tested, from subscription phase to verification.
Voice Supplementary Services
The platform also has full support for supplementary voice services such as: Calling Line Identification Presentation (CLIP), Calling Line
Identification Restriction (CLIR), Call Barring, Call Forwarding, Call Waiting,
and Call Holding.
Monitor Master – Product Portfolio Stationary Testing 9
Monitor Master comes with comprehensive support for SMS and EMS,
providing accurate timings and KPIs for sending and receiving messages.
SMS-based applications are also supported, such as horoscope or
lottery­updates, sport notifications, or similar content in which the platform
requests SMS-based data by sending an SMS with a particular key word.
Monitor Master also has full support for MMS scenarios such as transmission/receipt of MMS via email or Web-based applications, and can also
inject MMS traffic via the MM3, MM4, and MM7 interfaces.
Key Points
iPhone Visual Voicemail
n Instant Messaging (MSN, Skype)
n SIM Toolkit
Typical KPIs
SMS Send, Delivery Time (End-toEnd)
n MMS Send, MMS Notification, and
MMS Retrieval Times
n Email Send, Delivery Time
n Blackberry Email Round Trip Time
n Visual Voicemail Connect, Download
Graph showing the min/avg/max time over time to
receive the VVM notification SMS
Visual Voicemail (VVM)
Although branded a voicemail service, this is actually an email-based
messaging system. Apple developed VVM for the iPhone, deciding an IMAP
email system would provide greater flexibility and more features to the
iPhone user.
Monitor Master has been fully integrated with the VVM system and is
capable of sending, receiving, listening (and performing PESQ scores on),
and deleting VVM messages.
Instant Messaging
Monitor Master supports Skype, MSN, and Wireless Village instant
messaging applications. As with all Monitor Master tests, it can interact
as a user would, and record data about a variety of interactions, such as:
presence­checking, login/logout, and message delivery and receipt.
SMTP, POP3, and IMAP email protocols including IMAP SSL (secure
email) are all supported with the system.
Monitor Master can check email trip times to and from BlackBerry devices under its control. This provides accurate timing information for your
BlackBerry email network.
USSD and SIM Toolkit
There is also support for Unstructured Supplementary Services Data
(USSD). This is commonly used by pre-paid subscribers to obtain their
balance­. Monitor Master can process the balance before and after any network transactions to verify that the correct amount has been billed to the
customer. Monitor Master can also create test scenarios that interact with
the SIM Toolkit.
Monitor Master – Product Portfolio Stationary Testing 10
Monitor Master can test prepaid subscriber services. Prepay specific
services, like balance checking and prepay top-up can also be tested.
Balance Checks
Monitor Master can be used to perform prepay balance checks via
USSD, IVR/SMS combination, and Web/WAP portal. These are just some of
the common options, but many more are available.
Key Points
Balance checks
Auto top-up
n Revenue Assurance
Typical KPIs
Balance Check Duration & Amount
Automatic Top-Up Time
Auto Top-Up
Automated IVR top-up services can be tested with Monitor Master.
For example, a list of prepay voucher numbers, provided either in file
format or in a database table, can be accessed sequentially by Monitor
Master and entered as DTMF tones in a call to the automated IVR top-up
system. By checking the balance before and after this interaction, Monitor
Master can decide if the top-up has worked successfully.
Revenue Assurance
Revenue Assurance can be tested for prepay subscribers, by checking
the balance before and after a service test, to ensure that the subscriber
has been charged correctly. Any combination of the services mentioned in
previous sections can be used for this purpose.
For more ways to test revenue assurance, please see section “Revenue
Assurance” on page 20.
An example of an account balance status report. A ringtone content download test
including a balance check to verify that the correct amount was charged for each item
Monitor Master – Product Portfolio Stationary Testing 11
Key Points
Video Streaming
Video Call
n Video PING
n YouTube support
n Mean Opinion Score measurements
n Non-intrusive testing
Video Streaming
Monitor Master comes with extensive video streaming test support.
The system can connect to a variety of video stream sources, such as
YouTube, and analyze the quality of the received video stream. A range of
metrics and KPIs are calculated including buffering, bandwidth, and various
other perceived quality measurements (see Video Quality below).
Using Monitor Master you can also quickly verify the availability of all
your video feeds through Video PING (VPING).
As with all IP-based Monitor Master tests, IP packet level trace information is captured allowing detailed diagnostic work to be performed in the
event of a failure or poor performance.
Video Quality
Monitor Master has full support for Video Mean Opinion Score (VMOS)
calculations on streamed video content in order to verify the quality as
perceived by the end user.
The Mean Opinion Score (MOS) was originally defined by the ITU-T for
voice calls. The standard provides a numerical indication of the perceived
quality of the call.
Typical KPIs
Audio and Video Mean Opinion Score
Jitter %
n Blockiness %
n Blurriness %
n Loudness %
n Buffering Time
The MOS approach is now applied to video content using a number of
algorithms to calculate a video MOS based on a non-intrusive analysis of
received content.
Monitor Master dynamically accesses video stream content over
the air or via a LAN connection and calculates the video MOS as would be
experienced­by an end user.
Monitor Master measures and analyzes content using a non-intrusive
approach. Reference content is not required, making it ideally suited to
monitoring streaming services in general as well as for services where no
reference content is available, such as mobile TV.
3G CS Video Call
Using Monitor Master you can test circuit switched video calls on a 3G
network. Monitor Master can simulate a 64kbps circuit switched video call
from a UMTS modem on one Agent to a UMTS modem on another Agent.
Monitor Master – Product Portfolio Stationary Testing 12
Monitor Master is the most powerful platform on the market for testing­
and monitoring browsing services. With its user-transaction recording­
features­, auto-browse capabilities, full support for WAP and Web, and
extensive­device simulation support, measuring any portal-based application is made extremely easy.
The platform supports secure traffic and comes with low-level features
such as segmentation and reassembly (SAR) of large data transmissions,
socket reuse, and parallel HTTP fetches.
Key Points
Simulates real users accessing the
real services using real devices
n Full Web 2.0 support based on
n WAP 1.x/2.0
n Internet Explorer (IE6,7,8) support
n Digital Rights Management (DRM)
n Cascading Style Sheet support
n Auto-browsing (spidering)
n Content analysis
n Dynamic browsing – to access other
services & subscriptions
Typical KPIs
Response Time
Downloaded Bytes
n HTTP Response Codes
n Service Availability by Device
n Device Interoperability
n Content Conformance
Monitor Master simulates real handsets using a handset emulation
from our virtual device repository with hundreds of different devices. All
major characteristics of the real device are stored in the virtual profile which
allows Monitor Master to mimic the behavior of the real device.
Monitor Master will automatically select the correct WAP profile
(WAP1 or 2) for the selected device and will browse a WAP site as a real
user using a real handset.
Internet Explorer
Monitor Master can also execute Internet Explorer Web transactions
exactly like a user would – going to URLs, following links, entering information into forms and submitting data. This is done by driving Internet Explorer
directly from the operating system.
WEB 2.0
Web 2.0 is a term that encompasses user-generated content, communication, interoperability, and collaboration using a host of different access­
technologies and devices. The future will involve more and more third parties delivering services through these technologies.
Web 2.0 is here now and this translates to real challenges for testing­
and monitoring. For example, how would you test Twitter performance between a mobile handset such as an iPhone and a PC to prove that tweets
are getting through in a timely way? How would you compare Facebook
performance on an Android/Google handset with performance on a PC so
you could see how the radio network affects the end user experience?
Web 2.0 is fully implemented in Monitor Master using the Web-kit
engine and our Camaro browser, providing support for client-side JavaScript
and AJAX.
Example script browsing Facebook
Web 2.0 uses more technologies to deliver a richer experience
Monitor Master – Product Portfolio Stationary Testing 13
Content Analysis
Monitor Master is the leading platform on the market for mobile content analysis. With its unique analysis engine it can analyze all available
content using the same methodology.
In addition to the comprehensive support of content types, the flexible­
architecture of Monitor Master makes it possible to extend the analysis
engine, to deal with the latest content types.
Markup and Text Analysis
Using Monitor Master, not only can you measure the availability
and performance of content services, you can also verify that the content
returned­is correct and of the right quality.
Key Points
Automatically check downloaded
content for text and images
n Analyze document formatting
n Check location of content using
XPath or regular expressions
n Check HTTP header content
n Validate site structure
n Branding verification
Using techniques such as simple text matching, regular expressions,
and XPath you can verify the quality and positioning of your content.
In addition to markup technologies you can also parse and verify
Cascading Style Sheets.
This allows us to perform deck verification tests including brand
verification (such as is the correct logo always available in the top-right
Image Analysis
Using Monitor Master’s powerful image analysis capabilities you can
verify logos and images on your portal.
Monitor Master supports all image formats used by mobile content
today and can extract and test aspects such as: image dimensions, color
depth and animation frames.
Methods are also available for scoring an image against a reference
image to test content adaptation engines.
Using Monitor Master it is easy to analyze mobile
content for conformance, interoperability, usability,
and performance
Music and Tunes Analysis
Music and ringtone downloads are popular services for mobile users­.
Using Monitor Master’s multi-modal capabilities and music and tunes analysis­
features you can monitor and verify all of your applications regardless­of
the delivery mechanism.
Monitor Master supports MP3, WAV, MIDI, and various other music
and ring-tone formats.
Monitor Master – Product Portfolio Stationary Testing 14
Monitor Master can test a number of different IP-based services, such
as FTP, HTTP, ping, and DNS lookup. These tests can be conducted over-theair, or via a LAN connection or broadband DSL line.
Key Points
Monitor Master can connect to, upload, and download files from both
FTP and HTTP servers.
HTTP interactions can be managed through the methods described in
the Browsing section.
FTP and HTTP tests both record a number of KPIs, such as connection
time, throughput, download, and upload time.
Variety of IP based services supported such as FTP, ping, DNS, HTTP
n Can integrate with third-party tools
n Packet capture available with
n Layer 3 trace diagnostic data
Box Office report showing FTP throughput over time
Typical KPIs
Throughput rates for FTP and HTTP
Ping round trip time
n Packet loss %
n DNS Lookup response time
n Three-way handshake time
n Download/upload time
Monitor Master can also perform domain name queries on domain
name servers. For each lookup, the user can specify which connection and
which domain name server to use. Testing returns a list of IP addresses or
resolved host names and the time taken to return the information.
Ping is widely used to test both availability of your network servers
and the latency experienced in the network. Ping tests can be configured
to send any number of packets to a destination server and will calculate
the average­, minimum, and maximum round trip times along with the
percentage­packet loss.
Third-Party Integration
Due to the flexibility of Monitor Master, other data-centric services can
easily be integrated into a test scenario. Skype, SyncML, VPN connectivity,
and iTunes are just a few examples of these.
You are able to analyze the VoIP session in Wireshark
Monitor Master can test VoIP, either peer-to-peer between two VoIP
parties where no SIP server is involved, or via a SIP Server which also enables
VoIP to Mobile or fixed testing. Many interesting KPIs can be recorded including a speech quality PESQ score. You can also configure different codecs
such as G.711 u-Law and A-Law, G.723.1 / G.722 / G.726 / G.729, Speex
(8 and 16 kHz) / GSM 6.10 / iLBC. IP tracing (PCAP) is integrated into the
VoIP solution to allow detailed analysis of the a VoIP session.
Monitor Master – Product Portfolio Stationary Testing 15
Protocol Analysis
IP tracing
All IP-based Monitor Master tests can be used to capture packet data in
the PCAP format, which can be easily viewed with standard packet capture
tools such as WireShark. This allows for in-depth analysis of all the IP traffic
that passes through the modem or LAN interface.
Key Points
Example PCAP trace
PCAP – Wireshark support
Layer 3 tracing
n Chipset statistics
n Analysis with Field Analyzer
n Root-cause analysis
Wireshark IO graphs
Layer 3 Trace and Chipset statistics
GSM Layer 3, WCDMA NAS Signaling, WCDMA RRC Signaling, and CDMA
Phone Messages are available for tracing on each modem on the agent.
Chipset statistics, such as Idle Reports, Neighbor Reports, and HSDPA Channel
statistics are also recorded in the trace files. This data can be analyzed­using
Ascom’s powerful Field Analyzer tool, allowing users to drill down to pinpoint the exact cause of failed or poorly performing tests.
Field Analyzer
The signaling messages can also be processed by Monitor Master to
n Precision timing points for KPI metrics, e.g. ETSI PDP Context
Activation Time
n Error cause analysis
n Customized Box Office reports showing summarized trace data
Summarized HS-DPCCH CQI Stats
Monitor Master – Product Portfolio Stationary Testing 16
Application Areas
Due to the wide variety of tasks that Monitor Master can perform, it is
sometimes simpler to describe the value that Monitor Master can provide
by grouping its capabilities into application areas.
The main application areas for Monitor Master that we see with our customers are End-to-End Testing, Functional Testing, Roaming, and Revenue
Assurance. The diagram below outlines the types of testing associated­with
each application area:
24x7 Service
n Customer Experience
n Trend Analysis
Regression /
Ad-hoc Service
n Interactive Testing
n Regression Testing
Inbound /
n SIM Server
n Conforms to IREG
Automate billing
record checks
n Generate call
records for any
network interaction
Main application areas for Monitor Master
Our clients often require a mix of more than one application area.
For example a network monitoring team may require end-to-end service
testing­, but also require ad-hoc testing to ensure new releases of VAS platforms function correctly with particular device profiles. The flexibility of
Monitor Master makes this possible.
Application areas describe a general type of application for Monitor
Master, but Monitor Master is not limited to just these areas; its flexibility
means that more and more ways of deploying Monitor Master arise each
year, and the number of application areas and type of testing is always
Monitor Master – Product Portfolio Stationary Testing Key Points
Assess the quality of your services
from the customer perspective
n 24x7 round-the-clock automated
n Instant notification of service issues
via SNMP and auto-refresh reports
n Benchmark services against competitors
n Analyze trends in service response
times and availability
End-To-End Monitoring
Modern services are complex and often involve third parties that you
cannot control. All the parts may appear to work properly and yet the service
still fails from the customer’s point of view. So it’s not enough to monitor the
parts of the delivery chain, it’s vital to test each service from end to end.
Instant Awareness of Issues
If a service is not working correctly every second costs you money
– and subscribers. So when something does go wrong, you need to know
immediately – and preferably, you want the information to diagnose and
fix it quickly.
Even if your services are working, are they working well enough to
meet customer expectations? Does it matter if your average SMS round
trip time is 15 seconds or 10 seconds? By benchmarking against your competitors, you can better understand your position in the market in terms of
service quality and availability – and if need be, improve it.
Monitor Master is widely deployed to meet these needs and concerns
for many operators around the world. With its strong flexibility and dynamic
reporting and alarm system, Monitor Master helps our customers keep track
of their services on a 24x7 basis. In addition, Layer 3 and IP traces, combined
with Monitor Master’s analysis logic, enables the user to see where the
failure occurred and resolve the issue swiftly.
Example Box Office report highlighting service availability, with failures in
red/amber. Below is a summary of service availability by service type for the
previous 24 hours.
Monitor Master – Product Portfolio Stationary Testing Key Points
Regression Testing
Test new and upgraded services
prior to launch
n Interactively test services at any
n Tailor reports to meet the needs of
your new services
n Before and after trend comparison
n Trace level data for diagnosis of
Regression testing is essential when you need to know that new or
upgraded services are safe to launch to your customers.
Modern, next-generation services involving Web 2.0 components or
VoIP services interact in such complex ways, even simple changes can cause
faults. Even traditional, hardened services like SMS or voice can fail given
the number of elements involved in delivering them.
The only way to be sure that your service is ready is to test it from the
end user’s point of view. Manual testing is expensive, time-consuming, and
error prone, and a real handset can’t give you the technical detail needed
to diagnose a problem and fix it quickly.
Test Before and After
Either by using our prepackaged Standard Scripts, or by creating your
own, regression testing is a simple case of running your test(s) before and
after the installation/upgrade of your new service and seeing the difference
in results (easy to do with the built-in reporting tool, Box Office).
A typical regression test suite for testing an MMS-C would send different content types to all your important handsets and verify that the content
adaptation engine successfully transcoded all MMS messages­depending on
the receiving device. For example: if the receiving handset can’t handle video
has it been transcoded to an animated GIF image?
Upgrade to SMS infrastructure resulted in degraded
end-to-end performance.
Interactive Control
In addition, Monitor Master supports live interactive control of test
agents, called Interactive Mode, allowing test engineers to interact with
the network and perform voice, text, or data functions. This is perfect for
quick ad-hoc testing, to give a real, live impression of the performance of
your services.
Monitor Master is the perfect tool to test and diagnose any problems
you may have with your new and upgraded services.
Many customers have used Monitor Master for regression testing, and
have been able to isolate issues quickly and resolve them. In one particular
instance, after a customer upgraded their SMSC platform, Monitor Master
reported increased MMS notification times. On further investigation, it was
noticed that the new SMSC Push Proxy Gateway (PPG), used to handle these
notifications, had not been configured correctly after the upgrade. By correcting this before the new SMSC platform went live, the operator made
sure their subscribers did not experience the problem.
Monitor Master – Product Portfolio Stationary Testing Key Points
Centralized SIM Server allows SIMs
to be deployed to any agent
n Geographically distributed agents
n Test between different agents in
different locations
n IREG support
n Mosaic™: Ascom’s managed roaming service
Roaming scenarios add far more complexity to already complex mobile
n Multiple operators, multiple carriers, and multiple network
n Ever-changing technologies, both in the VAS and network infrastructure areas – both locally and within roaming partners
n Physical distance, language barriers, and differing time zones can
make manual methods of fault finding extremely time-consuming
With its excellent SIM-Server technology and the capability to deploy
agents to anywhere in the world, Monitor Master provides state-of-the-art
inbound and outbound roaming testing capabilities.
Managed Service
Ascom also offers MosaicTM, the managed service option for customers
who want to test outbound roaming scenarios without hosting their own
Monitor Master probes in other countries.
Mosaic offers an easy-access solution for all your outbound roaming
requirements where you can either run our standard scripts or provide your
own custom scripts.
Mosaic is available in a wide variety of countries and locations around
the globe – contact your sales representative for more details­.
Services Tested
Any service supported by Monitor Master can be tested in a roaming
scenario. The most common roaming test requirements are for voice and
messaging services, but more complex services such as VoIP, Blackberry,
and Visual Voice Mail are becoming more important.
Example IREG Conformance Report,
automatically generated by Box Office
Monitor Master can be used to test IREG scenarios, and as with all Monitor
Master tests, they can be automated and set to run on a schedule­.
IREG reports are generated automatically and can be emailed or exported­
as a PDF.
Monitor Master – Product Portfolio Stationary Testing Key Points
Revenue Assurance
Monitor Master creates CDRs for all
test interactions
n Correlate CDR records against
network­charging center CDRs
n Automatic notification of billing
n Integrated with CashBack, for
a complete turnkey revenue
assurance­ solution
Getting the bill right is important. Charging too much upsets your
customers and gets the undivided attention of regulators; charging too
little upsets your company and partners.
As networks and services evolve and expand, unforeseen errors can
introduce­charging problems, and proving that the bill is correct is not
Monitor Master can provide revenue assurance by testing your network services and generating Call Detail Records (CDRs) which can be compared against your internal network CDRs.
A Party
B Party
Content download
Mobile TV
Typical CDR KPIs
Complete Call Detail Record Reconciliation
Monitor Master test CDRs can be correlated automatically with the real
CDRs using either Monitor Master or a third-party system of your choice.
Monitor Master is integrated with the CashBack revenue assurance platform.
Combining Monitor Master and CashBack we can provide a complete turnkey
solution, where Monitor Master is used as the active component to trigger
calls according to a test plan and generate test CDRs, and CashBack as the
passive component, used to trace the calls throughout the billing chain to
identify revenue leakage and tariff consistency.
CashBack report of CDR statistics
comparison & analysis
rating & billing chain
CDR stream
GUI access
TCG control
Almost unlimited potential
measurement points
R&B process chain
generated by
charged to
CDR Reconciliation
Integration with Monitor Master
Monitor Master – Product Portfolio Stationary Testing Key Points
Control a modem anywhere using
Web browser, in real time
n Make calls
n Send and receive SMS
n Browse Web
n Generate CDR records
n Ad-hoc testing
n IREG testing
Interactive Mode
Interactive Mode is a unique feature of Monitor Master providing direct
control of up to four modems via a Web-based interface.
By selecting an agent, modem and SIM, the user can effectively
“be” any user in any location, making phone calls, sending messages, and
browsing­the Web. All of this can be done from a desk with a PC running
a Web browser.
Making a voice call using two modems
Not only is this perfect for ad-hoc testing of network services, but it
is also an excellent method for testing roaming services and IREG conformance.
When making a call in Interactive Mode, the audio is linked to your PC
microphone and speaker, allowing you to listen and speak, as if you were
using a real phone. Calls can be made between two handsets in the same
interactive session.
Once an Interactive Mode session has finished, call details are recorded
in the central server database just like for a normal test, allowing the user
to view metrics and KPIs for the interactions they performed.
Using Interactive Mode to browse Facebook
Monitor Master – Product Portfolio Stationary Testing Key Points
Provides wide variety of reports
(see below)
n Drill down capability
n Direct access to media produced
by tests, such as audio, trace, and
n User group control
n Automated report distribution
n Flexible structure allows users to
create custom reports
n Auto-refresh capability
n iPhone interface
Report Types
Standard tables
Pivot tables
n Scatter graphs
n Frequency distributions
n Bar charts and histograms
n Pie charts
n Gantt charts
n Freeform (with user defined CSS)
And more…
Box Office
Monitor Master’s reporting tool, Box Office, delivers outstanding reports­
and information based on data gathered by the test probes, including­KPIs,
metrics, messages, and trace information.
Box Office comes with a wide array of standard reports, but can also
be customized quickly and easily to produce new reports that meet any
specific requirement you may have.
Reports are edited via
the same interface as used
to view reports – there is no
need for a separate tool.
Security is managed
by granting specific rights
to users through role management. This feature can
be used to separate users
into groups that can access
specific reports relevant to
Scatter graph showing drop in Ping time after
them. It can also be used to
network upgrade
grant editing rights to Box
Office report developers.
Reports can also be viewed on the Safari browser on the iPhone, for
users on the move.
Source Data
In addition to querying and displaying data stored in the Monitor
Master data warehouse, Box Office can query data from other databases
and other database types (SQL Server and Sun’s MySQL™).
Monitor Master – Product Portfolio Stationary Testing Key Points
Meet your requirements without
hardware or software upgrades
n Develop complex scripts in-house
n Wide variety of pre-canned test
steps available for use
n Execute custom VBScript code
n Integrate with third-party tools and
Monitor Master has the unique ability to be modified to suit unusual
or complicated requirements without any additional hardware or software,
or involvement of Ascom developers.
Our test creation tool, Story Boarder, allows customers to create custom test scenarios by joining together a path of test steps, such as “Send
SMS,” “Receive Email,” and “Make Data Connection,” in an easy, visual, dragand-drop manner.
In addition to these pre-canned test steps, Monitor Master can execute
raw VBScript added using Story Boarder. This opens up a whole world of customization and third-party integration capabilities. The VBScript language
is easy to learn and write, and gives an extra edge to the test creation tool.
Here are some typical applications:
n Manipulating files in a file system
n Executing windows command line programs (which could be other
custom programs that perform specific tasks)
n Performing complex regular expression matching and XPath
This kind of functionality has allowed our customers to integrate
Monitor Master with their systems – without extra cost. For example:
n Connect to a UNIX charging server, download billing files via FTP,
parse them, and create KPIs from the results
n Read prepay voucher numbers from a text file, store them in a
local­database, then load each voucher onto a SIM card using an
IVR service and DTMF tones, checking the balance before and after
Monitor Master’s flexibility also permits you to create new KPIs, metrics­,
and messages in each test. So, not only can you integrate with a third-party
system, perform a unique Windows®-based operation, or interact with other
operating systems directly, but you can also measure how long it takes and
report it as a KPI with any name you require. If a user can do it, Monitor
Master can measure it.
Advanced Scripting Scene in Story Boarder – connect to DB
Monitor Master – Product Portfolio Stationary Testing 24
Professional and Educational Services
In addition to providing a speedy standard Monitor Master deployment using our standard scripts and reports, we also offer the option of
customizing­the system to meet any specific requirements you have.
Professional Services
Ascom can provide a turnkey solution with all the professional services­
necessary to assure a successful deployment, including design, installation
and commissioning, scripting, report configuration, training, testing and
acceptance, and documentation.
Our experienced consultants have been working with Quality of Service
testing for many years and will be able to provide all the assistance you
We can also provide experienced full-time on-site Engineers who will
remain on-site at your premises to assist in the day-to-day testing and
manage the system as one of your employees.
Training Courses
Ascom offers a variety of training courses. We can provide training at
a location of your choice, or at one of our training centers.
Standard Courses – three-day courses:
n User Training – how to use the system
n Administration & Maintenance Training – how to manage
n Reporting Training – how to develop and configure new Box
Office reports
n Advanced Scripting Training – how to develop new advanced
scripts based on VBScript
On top of these trainings we offer a huge variety of addon packages which provides training on niche topics tailored for
your needs.
We work very hard to provide a very professional and interesting training­experience to help you learn how to fully utilize
your solution and get the most out of it.
Webinars offer us a chance to get in touch with our customers to
discuss­and demonstrate the latest features and developments in our
products­. Webinars are varied and are given often, to keep you up to date
with the latest information.
Managed Services
Ascom can also provide a managed service solution where we operate
the solution for you.
Monitor Master – Product Portfolio Stationary Testing 25
Customer Reference
Vodafone Spain
“Quality is extremely important to us.
To ensure this we have implemented
Monitor Master throughout our entire
product service lifecycle, from testing
a new service in pre-launch mode to
regression testing and monitoring
on launched services. Monitor Master
has really been a key component, and
enabler, for improving our revenues,
costs and of course customer satisfaction!”
Jose Gonzalez, Director Products, Services &
Innovation, Vodafone Spain, Madrid
Based in Madrid, Vodafone Spain is a wholly owned subsidiary of
Vodafone International. At the end of 2008 Vodafone Spain served more
than 16 million customers via GSM & UMTS. Vodafone’s innovative wireless
products and services help empower people to connect to those who matter
most. Vodafone Spain is the second-largest operator in Spain and multiple
independent quality studies continue to rank Vodafone Spain as the best
operator in Spain for call and data quality.
Vodafone Spain currently has more than 20 Monitor Master probes
in their network, and has plans to add more. They use Monitor Master for
service monitoring 7x24, network performance monitoring 7x24, revenue
assurance testing, roaming testing, provisioning testing, service audits, and
quality assurance.
They test and monitor many services within the national network,
n Service monitoring, performance, and quality assurance:
- Transfer balance
- Customer care assistance menus
- Query balance
- Vodafone live (Mobile TV [radio and network interfaces],
Real tones download, Games download, MP3 download, and
- Instant Messaging
- MMS (P2P, P2P [inter-carrier] and Premium services [MO and MT])
- Web applications
- Product provisioning
- MVNOs services
n Revenue Assurance in local and roaming (calls, messaging, data
sessions and roaming services)
n Service audits (VF live and provisioning)
n Regression testing (IT releases)
Some of the specific test cases used in the above tests include:
USSD stage 2 for interactive menus
n Voice quality testing using the PESQ algorithm to produce MOS
scores in a full mesh configuration
n Video quality testing using the VMOS algorithm to produce scores
in a full mesh configuration
n VF live browsing and content audit to produce a list of failures
inside the portal
n General purpose test scenarios (voice, video, USSD Stage 1 and 2,
USSD Callback, SMS, MMS, data sessions) allow us to test most of
the services locally and roaming using different devices, A and B
subscribers, carriers, bearers, etc.
Monitor Master – Product Portfolio Stationary Testing Monitor Master is used by over 100 operators world-wide as well as regulators and major
content providers, with systems ranging from
a single probe to large-scale deployments with
over 460 distributed probes. Monitor Master‘s
unique customization capability is one of the
key factors explaining why so many have chosen Monitor Master.
Monitor Master is undoubtedly a safe investment, with native support for all basic and
complex test scenarios as well as the ability
for users to develop unique testing scenarios,
fast and easily without further investment
or product­changes. Monitor Master is a costeffective­solution which can test what you
want, when you want it – always able to adapt
to your current testing requirements.
Ascom is the market leader within Quality
of Service testing, benchmarking, and optimization, with product solutions such as QVoice
for drive testing and benchmarking and TEMS
for optimization. Box Office is used to display
data from all products providing a unique umbrella solution for all your long-term trend
analysis requirements.
Provisioning testing and audit integrates the provisioning protocols for IT systems, service platforms, and network nodes in order
to know the status of a customer
n Every test case is integrated with the Supervision Management
Centre which is in charge of issue management
Some examples of Monitor Master’s ability to diagnose and troubleshoot network related issues:
n Top Deck: Monitor Master was able to discover an issue with content control, which was then corrected in a timely manner.
n VF live (MP3 download): Monitor Master was able to discover MP3
download related issues that Vodafone Spain could then correct
and minimize network downtime.
n DSL Provisioning Testing: Monitor Master was able to identify during provisioning testing on pre-launch DSL services an issue with
data translation that was then corrected, preventing customers
from experiencing call routing problems.
Some examples of Monitor Master’s ability to adapt to specific customer needs:
VF live: When Vodafone launched their WAP portal, they integrated
the content download to allow customers to use their handsets to play
games, music, etc. They tried to supervise VF live services with their current
solution but in the end they chosed to implement it with Monitor Master.
Roaming testing: After trying a couple of other solutions, Vodafone
implemented Monitor Master for roaming testing in order to test their
service­from the customer’s point of view. Their previous solutions had only
been able to test the services from a technological perspective, providing
no insight into the customer’s experience.
Monitor Master – Product Portfolio Stationary Testing 27
About Ascom
The Ascom Group is an international solution provider with comprehensive technological know-how in Mission-Critical Communication. The
company concentrates on the core areas of Wireless Solutions (high-value,
customer-specific on-site communications solutions) and Security
Solutions (applications for security, communication, control systems for infrastructure operators, public security institutions and
the army). The company has subsidiaries in 17 countries and a
workforce of some 2300 employees worldwide.
With a wealth of experience in implementing complex projects
for discerning customers, Ascom has established itself in many key
markets. Offerings range from analysis and consulting to system
design and system integration, project management, engineering
and implementation, through to maintenance and support.
The Mission Critical Communication Company
Ascom’s corporate strategy focuses on four key elements:
n Mission-critical communication – solutions tailored to meet
individual communication requirements in particularly challenging
n Organic growth – innovative products and solutions as growth
n Strategic acquisition policy – aimed at ensuring the ongoing
technological development of the Wireless Solutions and Security
Solutions divisions
n Increased operational efficiency and consistent cost management
Mission-critical communication is the first focal point of our corporate
strategy. We see mission-critical communication as comprising client­-specific­
communication solutions that optimally address time-critical requirements­
in the daily business and function quickly, reliably and smoothly even in
extreme conditions.
The second mainstay of our strategy is our focus on organic growth.
By offering innovative products and solutions, we want to promote further
growth and thus achieve a lasting increase in Ascom’s enterprise value.
Innovation enables us to maintain our technological advantage over our
competitors. Our products and solutions offer the utmost security and reliability. They also offer the flexibility required to ensure quick and costefficient­integration in our clients’ existing systems environments. In
their development work, the specialists at Wireless Solutions and Security
Solutions use the latest communications technologies and create special
functionalities tailored to the respective client environment and requirements.
The third focal point of our corporate strategy is a diligent and clearly
defined acquisitions strategy aimed at bolstering organic growth in the
Wireless Solutions and Security Solutions divisions by way of smaller, strategically appropriate acquisitions aimed at expanding the leading market
positions we hold.
The fourth strategic mainstay focuses on consistent cost management and increasing operational performance. The streamlined organiza-
Monitor Master – Product Portfolio Stationary Testing Ascom reserves the right to change specifications without notice. Ascom acknowledges all registered trademarks appearing herein.
Ascom Network Testing Inc.
1943 Isaac Newton Square
Reston | VA | 20190-5006 | USA
07.2011 © Ascom Network Testing Inc. | tems-monitor-master_product-portfolio_Rev C
tional structure that Ascom has today, will be further optimized on both,
group as well as divisional levels. As part of the concentration on the two
divisions Wireless Solutions and Security Solutions, internal processes will
be reassessed­and additional improvements and simplifications made.
Our short and efficient information channels allow us to react quickly and
flexibly­to client needs and changes in the market environment.