LOLA Low Latency Audio Visual Streaming System

Add to my manuals
68 Pages

advertisement

LOLA Low Latency Audio Visual Streaming System | Manualzz

LOLA: Low Latency Audio Visual Streaming System

LOLA

Low Latency Audio Visual Streaming System

Installation & User's Manual

© 2005-2015 - Conservatorio di musica G. Tartini – Trieste, Italy

Version 1.4.1

http://www.conts.it/artistica/lola-project [email protected]

1

LOLA: Low Latency Audio Visual Streaming System

Index

1. Introduction......................................................................................................................................3

2. Hardware and Operating System requirements................................................................................4

2.1. Audio input/output hardware requirements.............................................................................................................................4

2.2. Video input hardware requirements.........................................................................................................................................4

2.3. Video output hardware requirements.......................................................................................................................................6

2.4. PC hardware requirements......................................................................................................................................................6

2.5. Network requirements.............................................................................................................................................................8

2.6. LOLA v1.4.x backward compatibility...................................................................................................................................10

2.7. Network Interface Cards (NICs) performances.....................................................................................................................10

3. System Setup..................................................................................................................................11

3.1. Common libraries/drivers installation....................................................................................................................................11

3.2. Drivers for BitFlow systems (Alta-AN, Karbon-CXP)..........................................................................................................12

3.3. Drivers for Ximea USB3-Vision systems..............................................................................................................................14

3.4. Drivers for Point Grey USB3-Vision systems.......................................................................................................................14

3.5. Audio hardware Setup...........................................................................................................................................................15

3.6. BitFlow Video hardware Setup..............................................................................................................................................16

3.7. Installing LOLA application..................................................................................................................................................16

3.8. Tuning the LOLA Workstation..............................................................................................................................................19

3.9. TCP/UDP ports used by LOLA.............................................................................................................................................19

3.10. Local and Remote Audio/Video Quality..............................................................................................................................19

3.11. Software you should not install on your Lola pc..................................................................................................................20

4. Running LOLA...............................................................................................................................21

4.1. Local audio and video settings (Lola Setup)..........................................................................................................................22

4.2. Local and Remote rendering windows..................................................................................................................................27

4.3. Color Menu and Bayer decoding...........................................................................................................................................28

4.4. Connection with a remote host..............................................................................................................................................31

4.5. Fine Audio/Video buffer tuning.............................................................................................................................................32

4.6. Toggling Audio/Video and Network Monitor........................................................................................................................32

4.7. Disconnecting from a remote host.........................................................................................................................................33

4.8. Debugging a connection: Bounce-Back mode.......................................................................................................................33

4.9. Debugging a connection: TaskMgr........................................................................................................................................34

4.10. Audio and Video Recording................................................................................................................................................35

4.11. Chat window....................................................................................................................................................................... 36

4.12. Audio Test signal.................................................................................................................................................................37

4.13. Windows layout...................................................................................................................................................................37

4.14. Duplicate window (remote host display).............................................................................................................................37

4.15. LolaGui process priority......................................................................................................................................................38

4.16. Audio only setup.................................................................................................................................................................38

4.17. Lola system checkup...........................................................................................................................................................38

5. LOLA Tools....................................................................................................................................40

5.1. Lola Wav Splitter..................................................................................................................................................................40

5.2. Lola Video Converter...........................................................................................................................................................40

5.3. LolaGUI Tester.....................................................................................................................................................................41

Appendix A – Old experimental digital video setup ........................................................................43

Appendix B – Compatible Video Analog Cameras ...........................................................................44

Appendix C – Cameras Cabling Guide .............................................................................................45

Appendix D – Setting Cameras in External Trigger Mode ...............................................................50

Appendix E – Audio Card internal Mixer settings.............................................................................54

Appendix F – Manual BitFlow Configuration...................................................................................57

Appendix G – Lola Hardware components........................................................................................60

Appendix H – Known Issues..............................................................................................................62

Appendix I – Remote assistance with TeamViewer...........................................................................66

Appendix J – Connection debugging hints.........................................................................................67

Acknowledgments..............................................................................................................................68

2

LOLA: Low Latency Audio Visual Streaming System

1. Introduction

Lola is an Audio Visual Streaming System developed by “Conservatorio di Musica G.

Tartini”, Trieste, Italy, in collaboration with GARR, the Italian Academic and Research Network.

LOLA goal is to provide a tool for real-time audio/video “natural” distance human interaction. It was originally conceived for distance music performances, education and production, but can be used for any other scenario where real-time interaction is required. A special attention has been paid to the optimization of the signal processing and transmission in order to keep the system latency as low as possible, below the human delay perception threshold. Both Standard Definition video (SD) and High Definition video (HD) modes are supported from version 1.4.x

The system is based on high performance audio/video acquisition hardware, and on the integration and optimization of audio/video stream acquisition, presentation and transmission. The

LOLA system also requires very high performance Wide Area and Local Area Networks: 1 Gigabit per second end-to-end connection is the minimum suggested configuration when running the system with uncompressed video signals; lower bandwidth can be used when running the LOLA system with video compression on.

NOTE: Lola depends on very specific hardware components and it doesn't work with any kind of camera, frame grabber or audio card; please carefully read the following “Hardware and

Operating System requirements” chapter.

3

LOLA: Low Latency Audio Visual Streaming System

2. Hardware and Operating System requirements

In order to achieve a low transmission and presentation latency, LOLA relies on software optimization and on high performance audio and video devices. Low latency video in particular has been achieved by adopting a family of industrial video grabbers which at the moment are only supported under the Microsoft Windows Operating System. To date, LOLA has been tested on

Windows XP, Windows 7 and Windows 8.1 (both 32-bit and 64-bit versions). Since Windows XP is being phased out, and Windows 7 offers better performances for LOLA, we recommend to use

Windows 7 as the base operating system.

Starting from ver. 1.4.0, support for Windows XP is discontinued.

Minimum OS requirements is Windows Vista with “SP2” and “Platform Update for Windows Vista”.

2.1. Audio input/output hardware requirements

Low latency audio performance requires that small in/out audio buffers can be used, that no bottlenecks are present in the data transfer between the converters and the system, and that robust and efficient software drivers are available for the OS in use. USB and FireWire (IEEE 1394) external devices do not offer such a fast data acquisition processing, thus LOLA uses a PCI/PCIe internal device and a reliable ASIO driver. To date, the following Audio boards have been successfully tested and have shown very good performances with a buffer size of 32 samples (0.7

ms):

RME HDSPe-AIO (PCI-Express internal card)

RME Hammerfall HDSP 9632 (PCI internal card)

RME Multiface II (PCI/PCIe host card + external ADC/DAC box)

2.2. Video input hardware requirements

Fast video acquisition and streaming relies on a family of video grabbers by BitFlow Inc.

and on the new USB3-Vision compliant cameras by Ximea and PointGrey, which all provide high hardware performance and a versatile API for low-level video processing control.

2.2.1. Supported BitFlow video grabbers:

BitFlow ALTA-AN1 (Analog grabber)

This card supports both B/W and color analog cameras. Tested cameras for b/w service are:

Sony HR50 (it works fine at 30 and 60 fps) and Sony HR70 (this one works fine at 29 and

30 fps but with some workarounds; see Appendix D for more details).

In case of color video acquisition the video camera must provide component analog color

4

LOLA: Low Latency Audio Visual Streaming System

output (RGB or YpbPr). For color operation the PC running LOLA should provide enough

CPU performance to deal with data processing, i.e. an i5/i7 CPU based workstation is recommended. The following color cameras have been extensively tested with LOLA (and camera files are provided):

Hitachi KP-FD30 color analog

Toshiba IK-TF5 color analog

To connect the cameras to the video grabber you need appropriate analog cables. Analog cables length can vary, and up to now cables up to 12m have been successfully used. Longer cables should work fine as well, but testing that the performance does not degrade is recommended. These cables can be easily created starting from high quality VGA cables, both for b/w and color analog cameras. A schematic of the various cables pinout is included in Appendix C.

Note: Considering that analog cameras require a lot of work (and may have possible issues) with cables and camera files configuration, we highly recommend to use the digital ones, specially to take advantage of higher video resolutions and all the new features introduced from Lola 1.4.0.

BitFlow KARBON-CXP2 (Digital grabber)

This card supports B/W, Bayer encoded and color digital cameras. Tested CoaXPress cameras are:

Imperx: CXP-B1410C, CXP-B1411C and ICX-B1410C (old Gen 1 model)

Adimec: OPAL-2000C-CXP ( to date, due to an unresolved issue at initialization time with the BitFlow grabber, this camera is not recommended )

(to properly configure digital cameras please carefully read the “System setup” chapter)

For Bayer decoding and color operations we strongly recommend an i7 cpu.

To connect CoaXPress digital cameras to the video grabber you need an high quality 75

Ohm coaxial cable (BNC connectors); cables up to 100m have been successfully used.

Camera power is provided directly by the grabber.

Useful links:

- http://www.coaxpress.com/coaxpress.php

- http://www.intercon-1.com/Uploads//CoaXPress%20Product%20Info%20Sheet.pdf

2.2.2. Ximea USB3-Vision xiQ cameras

Starting from ver. 1.4.0 Lola also supports USB3-Vision cameras; in order to achieve the best performance we currently use the xiQ camera series by Ximea which provide low latency acquisition and reliable API for low-level programming.

5

LOLA: Low Latency Audio Visual Streaming System

Ximea cameras don't require a frame grabber; they connect directly to USB3 ports.

Currently tested camera is:

Ximea xiQ line: MQ013CG-E2

( http://www.ximea.com/en/products/usb3-vision-cameras-xiq-line/mq013cg-e2 )

2.2.3. Point Grey USB3-Vision cameras

Starting from ver. 1.4.1, Lola also supports USB3-Vision cameras by Point Grey; to date we successfully tested the Grasshopper3 and Flea3 series.

Currently tested cameras are:

Grasshopper3 series:

GS3-U3-41C6C (Color camera) and GS3-U3-41C6M (Monochrome camera)

Flea3 series:

FL3-U3-13S2C-CS (Color camera)

2.2.4. Experimental configurations

LOLA also experimentally supports BitFlow digital Camera Link grabber and cameras (B/W and Color). See Appendix A for details and Appendix B for further considerations and a short list of compatible video cameras.

Currently t here is no more support for this type of video setup.

2.3. Video output hardware requirements

LCD monitors introduce additional latency w.r.t. CRT monitors. Use a fast LCD monitor with real low latency/response time (equal to or less than 2 or 3 milliseconds) or an old style, but faster, CRT monitor/projector.

The way you connect the monitor to the PC may also cause additional latency. Use VGA,

DVI-D or RGB connectors, avoid HDMI. In case you use any additional device, like video splitters, cable extenders, etc., please ensure that they do not add significant further latency to the video output chain. If you need dual video output, try to use graphic cards in the PC which already have native dual output, instead of using external splitting devices. In general beware that many video devices (including video walls, etc) are intended for broadcasting or streaming use, where latency is not important; thus the devices perform very badly when used with LOLA and should not be used.

2.4. PC hardware requirements

LOLA software runs on a typical mid/high-end multimedia PC. A suggested configuration consists in an Intel dual/quad core based system (for color and HD support we highly recommend an i7 Quad Core processor with Hyperthread support), with at least 8 GB RAM and the following specifications:

6

LOLA: Low Latency Audio Visual Streaming System

Mid/High performance motherboard providing PCI Express bus and supporting DDR3 ram technology. Intel chipsets are highly recommended.

1Gbit LAN port. LOLA currently uses only 1 LAN port, but in the near future it will support more LAN ports simultaneously for multiple nodes and multiple cameras support. See section

2.8 for important information regarding NIC performance.

Integrated graphic adapter should be enough for mid-range pc; however a PCI-Express

Video Adapter by Nvidia (GeForce GTX series cards or better) is strongly recommended, specially for color and HD video processing and for the newly introduced Cuda compression features.

Enough PCI-express free slots to host Audio card, Graphic adapter, BitFlow grabber and additionally a compatible Intel NIC adapter. About required pci-e lanes, note that analog

BitFlow cards require a x4 PCIe slot and digital cards require a x8 PCIe slot. Using a lower bandwidth PCIe slot, sometimes located besides the PCIe x4 and PCIe x16 ones, will result in bad video grabbing performances.

Mid/Full tower case with quiet design, a good air flow circulation and fans, and at least four full height expansion slots.

A really good and stable PSU (Corsair AX series, Seasonic or Antec).

When using the Ximea or PointGrey cameras, a compatible USB3 host adapter chipset.

Tested and recommended USB3 adapters are:

- Fresco Logic FL1009 (395MB/s) and Fresco Logic FL1100 (395MB/s)

- Renesas D720200AF1 (386MB/s), Renesas D720202 (365MB/s), Renesas D20200F1

(327MB/s)

- Intel Z77 (450MB/s), Intel QM77 (450MB/s)

Known incompatible USB 3.0 chipsets are:

- ASMedia AS1042

- Etron EJ168A

- Via Labs VL800-Q8

- Texas Instruments (they generally work with the Ximea cameras, but not with the PtGrey

ones)

- ASMedia xHCI (not supported by Ximea cameras, but it seems to work for PtGrey ones)

For a more comprehensive USB3 adapters list look at:

Ximea: http://www.ximea.com/support/wiki/usb3/Compatible_hardware

PointGrey: http://www.ptgrey.com/tan/10359

and for a more detailed hardware configuration list see Appendix G.

7

LOLA: Low Latency Audio Visual Streaming System

2.5. Network requirements

LOLA requires at least 100Mbps in minimal uncompressed configuration (640x480, b/w, 30 frames per second) up to 500Mbps in full SD video uncompressed configuration (640x480, color,

60 frames per second), or much more for HD uncompressed/unencoded video setups (over 2Gbps), and generates a very high Packet Per Second (PPS) rate, as it uses 1K data packets, e.g. from 100 up to 60 KPPS for SD and up to 250 KPPS for HD uncompressed/unencoded video. Thus the minimal end-to-end connectivity for uncompressed mode (excluding uncompressed/unencoded HD where it goes beyond 1Gbps) must be at least 1Gbps. This requires 1Gbps compliant cabling (Cat6 or optical), and adequate networking hardware. You should pay special attention to LAN switches performances when using LOLA, because of the high PPS rate, in particular when other data flow compete on the same switch; in general the ideal situation is when the LOLA workstation is directly connected to the Wide Area Network device (router or switch) at 1Gbps with no other network devices in between, or at least there is no competing network traffic on the LAN devices being used by LOLA data traffic. Furthermore some older routers (like for example Cisco 7200 and 7300 series with NSE 100 or NSE 300 routing engines and equivalent ones) are also known to be unable to perform correctly with LOLA in full uncompressed mode.

Starting from version 1.3.0 and on, Lola supports m-jpeg video compression; using compressed video the minimum required bandwidth is reduced to about 20Mbps (640x480, b/w or color, 30 fps). Please note that when using compression without the use of a CUDA compliant graphic card, the PC processing power will be affected heavily, and the overall latency will also increase from 3 to 10 milliseconds, depending on video resolution and CPU speed.

The following tables give an approximate estimate of LOLA bandwidth traffic:

Uncompressed 8 bit (B/W or Bayer encoded)

640x480 - 30 fps:

640x480 - 60 fps:

1024x768 - 30 fps:

1024x768 - 60 fps:

1280x720 - 30 fps:

1280x720 - 60 fps:

1280x960 - 30 fps:

1280x960 - 60 fps:

~ 85 Mbps

~ 170 Mbps

~ 210 Mbps

~ 420 Mbps

~ 245 Mbps

~ 490 Mbps

~ 325 Mbps

~ 650 Mbps

1280x1024 - 30 fps: ~ 350 Mbps

1280x1024 - 60 fps: ~ 700 Mbps

1392x1040 - 30 fps: ~ 385 Mbps

1920x1080 - 30 fps: ~ 535 Mbps

Uncompressed 24 bit (Color RGB)

640x480 - 30 fps: ~ 250 Mbps

8

LOLA: Low Latency Audio Visual Streaming System

640x480 - 60 fps:

1024x768 - 30 fps:

1024x768 - 60 fps:

1280x720 - 30 fps:

1280x720 - 60 fps:

1280x960 - 30 fps:

1280x960 - 60 fps:

1280x1024 - 30 fps:

1280x1024 - 60 fps:

1392x1040 - 30 fps:

1920x1080 - 30 fps:

~ 500 Mbps

~ 620 Mbps

~ 1240 Mbps

~ 730 Mbps

~ 1460 Mbps

~ 965 Mbps

~ 1930 Mbps

~ 1030 Mbps

~ 2060 Mbps

~ 1100 Mbps

~ 1700 Mbps

Compressed 8/24 bit (Jpeg comp. quality = 60)

640x480 - 30 fps:

640x480 - 60 fps:

1024x768 - 30 fps:

1024x768 - 60 fps:

1280x720 - 30 fps:

1280x720 - 60 fps:

1280x960 - 30 fps:

1280x960 - 60 fps:

1280x1024 - 30 fps:

1280x1024 - 60 fps:

1392x1040 - 30 fps:

1920x1080 - 30 fps:

~ 15 Mbps

~ 30 Mbps

~ 25 Mbps

~ 50 Mbps

~ 30 Mbps

~ 60 Mbps

~ 40 Mbps

~ 80 Mbps

~ 45 Mbps

~ 90 Mbps

~ 50 Mbps

~ 60 Mbps

LOLA should not run behind a Firewall and should run with a public IPv4 address (no

NAT). IPv6 addresses are currently not yet supported. Traffic Shapers and other devices which inspect/act deeply on network performances must also be avoided.

There can be network issues if the LOLA generated traffic goes via a number of Local Area Network Switches which are shared with other IP traffic: indeed LOLA generates a high rate of packets (PPS rate) as it uses 1K packet size, and this can affect performance of a number of LAN switches, resulting in a poor

LOLA end-to-end connection. A workaround can be to connect LOLA PC directly (or as close a possible) to the LAN edge routers connecting to the network backbone, to avoid switches.

As LOLA provides no error recovery procedures and a very minimal data buffering, the whole end-to-end network connection must be with very high performance: low jitter, error free and the lowest possible network latency (packets Round Trip Time, RTT); network latency adds in fact to LOLA system latency, and the sum define the upper limit when real-time human interaction is

9

LOLA: Low Latency Audio Visual Streaming System

still possible. LOLA has been developed to use the services provided by the Research and

Education hi-end networking services (like GARR, GÈANT, Internet2) and can run both on shared

IP network services (TCP and UDP) or on dedicated circuits, which provide better network performances and reliability, especially at high bandwidth configuration. It is recommended to perform end-to-end network tests also using additional tools; iperf or equivalent http://iperf.sourceforge.net/ ) can be used, trying to simulate LOLA expected traffic before using

LOLA itself, but the best possible tests you can perform are using the LOLA-Tester application, which is included into the distribution package itself: in fact it generates a traffic which is very very similar to a real LOLA session (see section 5.3 for more details). These tools are also useful as a powerful network debugging aid, in case of problems using LOLA.

2.6. LOLA v1.4.x backward compatibility

Due to some newly introduced features, LOLA v1.4.x is NOT backward compatible with

1.2.x (and before) versions. LOLA is fully compatible with all 1.3.x versions.

2.7. Network Interface Cards (NICs) performances

During various tests we detected that a number of NICs who declare a full 1Gigabit per second capability are indeed unable to deliver that, or have problems in delivering correctly the number of Packets per Second (PPS) generated by LOLA (up to 250000 in some cases). We identified issues with all the Realtek and Broadcom chipset based NICs. Also some Marvell NICs seem to have the same issues. We thus strongly suggest to get NICs which are based on Intel chipsets, because all Intel based NICs we tested perform correctly also with LOLA at high frame rates.

INTEL

NICs NOTE: if you encounter some audio or video performance issues at high video fps (greater than 30), we suggest to modify the “Interrupt Moderation Rate” value to

“Medium” (on Windows 7) or “Low” (on Windows 8). Generally the default setting is “Adaptive”, but during our tests we discovered that it's not always good for LOLA. The same problem occurs if you completely turn off the interrupt moderation rate.

10

LOLA: Low Latency Audio Visual Streaming System

3. System Setup

This chapter describe in details how to install and correctly setup a LOLA system. Please read it carefully.

3.1. Common libraries/drivers installation

These libraries are required by all Lola versions (BitFlow, Ximea and PointGrey).

3.1.1 WinPcap (ver. 4.1.3)

LOLA relies on the WinPcap windows packet capture library, which has to be installed on your PC before running LOLA. You will find the correct installer version within the Lola

package inside the “WinpcapSetup” folder.

3.1.2 Visual C++ 2010 SP1 Redistributable package (x86-x64)

Starting from version 1.4.0, Lola is distributed with all the required Microsoft libraries, so it is no longer needed to download and install the “Visual Studio C++ 2010 SP1

Redistributable” package.

However, in case you encounter some issues with the included libraries, you can manually download and install the package from: http://www.microsoft.com/en-us/download/default.aspx

Read Appendix H.10 for further details.

3.1.3 RME Audio Card updated drivers

Download and install the latest available firmware and drivers from the audio board producer site: http://www.rme-audio.de/en_downloads.php

3.1.4 Nvidia Graphics Card updated drivers (required only by Lola “_Cuda” versions)

When using LOLA compiled with the Nvidia-Cuda support (LolaGui_Cuda versions), please make sure your Nvidia graphics card drivers are greater than or equal to ver. 319.37.

If not download an updated version from Nvidia site: http://www.nvidia.com/Download/index.aspx

11

LOLA: Low Latency Audio Visual Streaming System

3.2. Drivers for BitFlow systems (Alta-AN, Karbon-CXP)

3.2.1 BitFlow SDK 5.90

Starting from version 1.4.1, you will find all the required BitFlow installers within the Lola package inside the “BitFlowSetup” folder.

In case you have installed previous SDK versions or you are upgrading an older LOLA version which runs with an SDK older than 5.90, you must first remove the old SDK and then install the new one.

You do not need a license code for the BitFlow SDK, thus just install it as “drivers only”, when prompted for the serial number.

Note: if you are using a 64-bit Windows version please be sure to leave the “Install 32-

bit Binaries on 64-bit Windows” option checked (see Fig. 1).

Figure 1: BitFlow SDK Setup

After installing drivers you need to reboot your pc !

For analog systems no other drivers or tools are required; for digital systems (Karbon-

CXP) please follow the additional steps described below (section 3.2.2, steps 1 and 2)

3.2.2 Additional steps for BitFlow Karbon-CXP card (not required for ALTA-AN)

After installing the BitFlow drivers you will also need to update the Karbon-CXP firmware

12

LOLA: Low Latency Audio Visual Streaming System

and to install the Ximilon application.

Before proceeding, please make sure to connect your CXP camera to the first grabber

input (VFG0); it's labeled as "Port 1" on the grabber rear metal bracket (Fig. 2).

Figure 2: Karbon-CXP2 rear ports

1) FWDownload command (ONLY for Karbon-CXP card)

To update the Karbon-CXP firmware, open a Command Prompt window (All programs

Accessories

 Command Prompt) and run the following command:

FWDownload

You will be prompted to confirm the operation by pressing 'd' on your keyboard (Fig. 3).

Follow the instructions of the program and when it has finished please perform a cold

reboot (note that a warm reboot is not sufficient). To do so, turn the computer off (by pressing the power switch button or using the “Shut down” button on the Windows Start menu), wait for about ten seconds to ensure that all capacitors on the motherboard have time to discharge and then turn it on again.

Figure 3: FWDownload command (Karbon-CXP grabber)

13

LOLA: Low Latency Audio Visual Streaming System

2) Ximilon version 1.7.1 for SDK 5.80+ (ONLY for Karbon-CXP card)

This tool installs all the required libraries by the standard GenICam interface and it's needed to properly configure and control your CoaXPress cameras.

After installing Ximilon you need to reboot your pc !

After all required drivers and libraries have been properly installed and you have correctly rebooted your pc, Lola should automatically configure your camera during first start-up; if not, see Appendix F for manual configuration.

3.3. Drivers for Ximea USB3-Vision systems

3.3.1 XIMEA API Software Package (ver. 4.03.14)

Starting from version 1.4.1, you will find the correct XIMEA API installer

(XIMEA_API_Installer_Beta.exe) within the Lola package inside the “XimeaSetup” folder.

In case you have installed a previous API version, you must first remove it and then install the new one.

After installing Ximea drivers and rebooting your pc, you can safely connect your USB3-

Vision camera to the computer.

3.4. Drivers for Point Grey USB3-Vision systems

3.3.1 Point Grey API Software Package (FlyCapture 2.7.3.11)

You will find the correct Point Grey API installer within the Lola package inside the

“PointGreySetup” folder.

Please note that during the installation process and after having selected “I will use USB cameras” checkbox, you can also decide whether or not to install the Point Grey USBPro driver. This is a “special” driver, optimized for having better USB performance and generally for reaching higher frame rates on Point Grey cameras; however, remember that when the USBPro driver is installed, non-PGR devices will not function on that adapter, so be careful to configure it and to select the correct USB3 port. You can also install, uninstall or reconfigure this special driver at any time using the “DriverControlGUI” application provided in the Point Grey “Utilities” folder.

14

LOLA: Low Latency Audio Visual Streaming System

Figure 4: FlyCapture2 Installation, USBPro driver configuration

During installation time you will also be asked to automatically manage the “Processor Idle

States” (just when using the Point Grey cameras); you can safely disable this option in order to manually manage your processor idle state at any time using the utilities provided in the

Point Grey “Utilities” folder (“Disable Processor Idle States” and “Enable Processor Idle

States” applications). Disabling the cpu idle states (C1E specially) can help to resolve some possible image “tearing” issues. Further details can be found at the following link: http://www.ptgrey.com/tan/10367

3.5. Audio hardware Setup

Update/install the drivers first! (see section 3.1). Before using the software, it is highly recommended to check if there is any board setup panel providing latency settings on which the software applications have no control. The RME driver usually provides a DSP Setting Panel as the

one shown in Fig. 5 (a link to it is usually available in the Windows bottom-right toolbar).

Set the hardware latency (Buffer Size) to the smallest available (32 samples or 0.7 ms).

DSP setting panel might have a different look (it depends on driver versions), but they all have the same equivalent functionality.

Since version 1.4.0, when LOLA is launched, it will check your current buffer setting and, in case it doesn't correspond to the correct value, it will alert you with a warning message. LOLA will also check for all available audio ASIO devices and if none is available, the program will provide a warning message and run without audio support.

It will be possible to select the desired ASIO audio device from the Audio Video Setup dialog

window, available in the Tools menu (Fig. 6). If you are unfamiliar with the audio card internal

mixer settings, see Appendix E.

15

LOLA: Low Latency Audio Visual Streaming System

Figure 5: audio device control panel (2 different driver versions)

3.6. BitFlow Video hardware Setup

Please update/install the BitFlow drivers first! (see section 3.2).

3.5.1 Automatic BitFlow configuration

Since version 1.4.0, Lola automatically configures your video grabber and camera during

start-up, both for analog and digital cards. Manual configuration, using SysReg and/or Ximilon utilities, is no longer needed.

In case of automatic configuration failure, please carefully read and follow the instructions described in Appendix F.

3.7. Installing LOLA application

After the WinPcap, Visual C++ Redistribute libraries and BitFlow SDK or Ximea SDK are correctly installed, to install LOLA simply unzip the .zip file provided in a directory of your choice and where you have all read/write permissions .

To run LOLA just launch the LolaGui_x.x.x_YYY.exe application (x.x.x = version number,

16

YYY = build info).

LOLA: Low Latency Audio Visual Streaming System

3.6.1 Activation code and first time run

The first time the software is launched, LOLA will ask you for a user identification and activation code. Just insert the ones you received.

Then the Audio/Video setup dialog window will pop up (Fig. 6) asking for first-time device

selection. The setup will then be stored in a LolaGui.ini configuration file in the Lola current directory and will be used in future sessions. You can always modify the Audio/Video setup starting from the Options menu, but only when LOLA is not connected to a remote node.

Figure 6: Lola Setup Dialog

LOLA initial default audio setup is for 2 stereo channels, collecting audio input from the first two channels of the RME audio card; e.g. setting the “Audio Input Offset” to 0. LOLA will also remember these settings until you change them again. Multichannel audio is always enabled since v1.4.0

17

LOLA: Low Latency Audio Visual Streaming System

You will also have the possibility to select the “N. of Audio Channels” from this menu; if you want to use more than 2 channels or you want to collect audio not from the first two channels of the RME audio card, see section 4.1 for further setup details.

TX Audio Level let you choose between “1” (no audio pre-amplification) or “2” (6dB audio pre-amplification). The default value “1” should be OK with most devices and setup, but in case you need to boost the level, you can use “2” instead.

In the “Video” section, select the camera file appropriate for your device. You can also configure a Toshiba or Hitachi color camera to run in b/w by selecting the appropriate “Mono8” files. Since version 1.2.1 LOLA supports all Frame Rates between 25fps up to 60fps. Of course the more fps the more bandwidth you will use. Standard values of 30fps or 60fps are commonly used.

If your PC has more than one NIC (Network Interface Card), in the “Network Adapter” field you can also select which one LOLA will use to send and receive data.

Please be sure to select the

correct NIC, otherwise Lola cannot properly connect to the remote node.

Multiple NICs Note: Due to an internal software design, if you have more than one NIC adapter, you have to physically disconnect the unused network cable and leave only the one connected to the selected NIC.

Starting from version 1.3.0, LOLA can also compress video data. By default compression

is disabled (“NONE” selected from the “compression” menu) since it introduces further latency and requires more processing power. It should be used only when there is not enough bandwidth available to run LOLA with uncompressed video data (see section 2.5 and section 4.1 for further details).

When using BitFlow grabbers, after you have finished configuring the available options and clicked the OK button, Lola will ask your permission to write a new value in your Windows

Registry (Fig. 7). This is needed to automatically configure your analog or digital cameras. Please

make sure you are logged in as an account with administrative rights.

Figure 7: Lola Camera File registration window

18

LOLA: Low Latency Audio Visual Streaming System

3.6.2 Changing the user identification and activation codes

If you need to change your user identification/activation code, just select the LOLA “Help” menu, and then “Update Serial”; upon confirmation your current codes will be removed. Quit the

LOLA application and restart it in order to enter the new codes.

3.8. Tuning the LOLA Workstation

It is recommended to switch off other applications that might interfere with the audio/video streaming process when the application is used in production sessions: anti-virus scanning software or similar should be carefully disabled. In case you notice some PC performance problems, it is suggested to disable the various “energy saving” options. The fine tuning of the Operating System for audio video streaming applications might also be advisable, although it is not strictly necessary, specially on Windows 7 or later . Please note that Windows 7 firewall disables by default ICMP and thus LOLA capability to perform its initial "check" function with the remote partner; you can fix this by enabling ICMP responses in the Windows 7 firewall configuration. Also remember to turn

off the screen saver.

When using Lola it's also suggested to set your Windows "Power plan" to "High performance" (Control Panel

 Power options); depending on cpu we have seen some great performance improvements.

3.9. TCP/UDP ports used by LOLA

LOLA will use by default ports TCP 7000, UDP 19788 and UDP 19798 for service communication and audio/video streaming. It is thus required that no restrictions apply for these ports (e.g. due to network or Windows firewall settings or local network administration policies). If you experience problems like “no connection”, or “video OK but no audio”, or “audio OK but no video” or “remote video window open but remains black screen” it is likely that some port filtering is affecting your connection. Please check your local network policies and also ask the remote site to do the same.

3.10. Local and Remote Audio/Video Quality

LOLA is just a tool which delivers to the remote site (in the best possible way) what it gets as input into the system locally; if you input good audio an good video, you will likely get good audio and good video delivered to the remote side. As such, remember to check locally your input before sending it to the remote system. Check your audio levels, check how microphones collect the sound, check the images the cameras are getting and the local lighting, etc. : remember the saying

“garbage in → garbage out!”

19

LOLA: Low Latency Audio Visual Streaming System

3.11. Software you should not install on your Lola pc

LOLA absolutely requires a perfectly “tuned” pc and so, in order to maintain your Windows installation as clean as possible, please don't install any Antivirus software, Skype or similar video conferencing applications, Apple software (iTunes or similar), VideoLan or other streaming systems, and in general all other tools or background services that could seriously affect the overall system and network performance.

The ideal Lola pc is based on a really clean Windows installation, with only the required drivers and tools for the audio and video cards and grabbers and no other software than Lola itself.

Please also check the correctness of your installed drivers, specially with regard to the RME card, BitFlow grabber and Network adapter.

20

LOLA: Low Latency Audio Visual Streaming System

4. Running LOLA

LOLA user interface (Fig. 8) is intended to be as simple as possible: a remote address to call

(IPv4 address), a “Check” button to make a quick and simple pre-connection test, a “Connect” and a

“Disconnect” button. A “Session name” box allows you to assign a specific name to a setup configuration for easier identification and you can also “Save” this configuration from the “File menu”. It is thus possible to recall previously saved setup configurations via the “File menu”

“Open” command; as the saved configuration file also includes the remote party IP address, you can use this feature to create your “LOLA address book”, where each entry also includes personalized setup parameters for the remote partner. Furthermore, LOLA also remembers the “last session” setup, thus it will always restart from the last session configuration. See also section 4.2 and Fig.

17.

Figure 8: LOLA main window: the control panel (left) and the local host video frame (right)

Since version 1.4.1, LOLA executable is provided in six versions and two architectures

(x86-32bit and x64-64bit):

1. LolaGui_1.4.x_BITFLOW: for BitFlow grabbers

2. LolaGui_1.4.x_XIMEA: for Ximea USB3-Vision xiQ cameras

21

LOLA: Low Latency Audio Visual Streaming System

3. LolaGui_1.4.x_PTGREY: for Point Grey USB3-Vision cameras

4. LolaGui_1.4.x_BITFLOW_CUDA: for BitFlow grabbers and Nvidia Cuda compatible graphics cards

5. LolaGui_1.4.x_XIMEA_CUDA: for Ximea USB3-Vision xiQ cameras and Nvidia Cuda compatible graphics cards

6. LolaGui_1.4.x_PTGREY_CUDA: for Point Grey USB3-Vision cameras and Nvidia Cuda compatible graphics cards

To prevent possible bitness-mismatch issues, the 64-bit versions are labeled with the _x64

suffix; 32-bit versions have no suffix. Although you can run 32-bit versions on a 64-bit OS, it's strongly recommended to use the appropriate 64-bit version to avoid possible errors.

Please, make sure to use the correct version that corresponds to your OS bitness (32 or 64 bit) and hardware setup, otherwise Lola will not work.

When using Lola with Nvidia-Cuda support (_CUDA versions), please also make sure that your Nvidia graphics card drivers are greater than or equal to ver. 319.37.

To check your current version open the Nvidia Control Panel (a link to it is usually available in the Windows bottom-right toolbar) and click on “System information”.

Using Nvidia cards with Cuda support can generally improve the overall jpeg compression speed; however it highly depends on your graphics card model. At the moment we suggest to use at least the Geforce GTX 700 series graphics cards (a GTX 770 or better is highly recommended).

4.1. Local audio and video settings (Lola Setup)

Before attempting to connect to a remote host, the user should correctly configure the local

audio and video options. The Lola Setup dialog (in the Options Menu, Fig. 6) allows to select the

desired ASIO audio device, the video camera file, the desired video frame rate (25 up to 60 fps, but

30 or 60 are the standard suggested values), the video compression (default is “NONE”) and the network interface controller.

4.1.1 Audio setup

This section allows to set the number of used audio channels and to specify from which audio input channel from the RME audio card you shall collect the local audio input. You can control from which audio input channel LOLA shall receive input by using the “Audio Input

Offset” setting. If set to 0 (default value), LOLA will receive audio starting from the first RME channel, which usually is an analog input one.

22

LOLA: Low Latency Audio Visual Streaming System

Figure 9: Lola Setup, Audio section

As an example, imagine you have multichannel audio enabled and your audio card has the following list of input audio channels:

AN1 AN2 SP.L SP.R A1 A2 A3 A4 A5 A6 A7 A8

and you want LOLA to receive audio input from the 8 ADAT (A1-A8) channels. Then you need to set “Audio Input Offset” = 4, thus LOLA will skip the first 4 channels, and “N. Audio Channels” =

8 in the Lola Setup dialog box. If you request more audio channels than available, LOLA will signal an error. An easy way to detect your RME audio card configuration is to see the HDSP Matrix from the card internal mixer (see appendix E). The “input channels” are listed as the top row in the matrix.

It is recommended to verify that audio input is working properly and the input latency is low as expected. A rough test can be done by switching on input-to-output redirection (using the “Local

audio On/Off” flag available on the control panel in the main window, see Fig. 8) and using

headphones connected to the LOLA audio card output to avoid an audio feedback effect. Further checks on the audio setup can be done using the appropriate audio card mixer/setup application (see for example Appendix E).

LOLA can also pre-amplify audio input by 6dB in case this is needed; just set “TX Audio level” to “2.0” if you want this pre-amplification to happen.

4.1.2 Video Setup

If the local video camera is connected and the correct camera file is selected, you should also see in the “Local Host” windows your local video. Also in this case you should check that the local video latency is low as expected. You should also check that the local audio and the local video are synchronized correctly.

23

LOLA: Low Latency Audio Visual Streaming System

Figure 10: Lola Setup, Video section

In this section you can control various video parameters:

"Camera Model" and "Camera Settings" specify the camera model, its resolution and its pixel depth (8 bit for monochrome-Mono8 or 24 bit for RGB8-RGB8PACKED).

"Frame Rate": here you can specify the frame per seconds of your camera. Inside the brackets on the right side of the "Fps" drop-down box, Lola indicates the minimum and maximum fps values reached by the camera at the chosen resolution.

To avoid video stuttering or tearing effects it's preferable to select a value which correspond to your monitor refresh rate or to the half; since generally the refresh monitor rate is 60Hz the suggested values are 30 or 60 fps.

"Compression": starting from version 1.3.0 it is possible to send and receive jpeg compressed video frames; this reduces the amount of transmitted/received data/packets so saving a lot of network bandwidth. In order to apply the compression, select from the drop-down menu the “M-JPEG” value; after selecting this option the compression

quality drop-down menu will be available too (in brackets is indicated the compression algorithm: "CPU" indicates cpu-simd, "GPU" indicates gpu-cuda acceleration).

"Comp. Quality" indicates the overall video compression quality:

- 100% = high video quality but more cpu/gpu usage and compression time;

- 60% (default) = good balance between video quality and cpu/gpu performance;

- 40% = low video quality but low cpu/gpu usage and overall better performance.

Although it is also possible to compress Bayer encoded images, we recommend to use compression on 24-bit RGB images only, in order to avoid highly visible video artifacts.

"Estimated TX Bandwidth (Mbit)" indicates the required network transmission bandwith; for jpeg compressed streams it is an approximation.

24

LOLA: Low Latency Audio Visual Streaming System

Options:

- "Optimize Jpeg Decompression": when activated LOLA uses a secondary thread to decompress the incoming jpeg stream in order to avoid issues with the audio buffers

(LOLA Network Monitor

 Realigned buffers). Generally the benefits depends on your cpu speed, number of cores and hyperthread support. If you see that your receiving video stream is stuttering (specially with frame rate higher than 50 fps) try to disable this option; when disable pay attention to the number of Realigned Buffers in the Network

Monitor. The default is "enabled".

- "Enable CPU-SIMD Acceleration": this option should be always enabled in order to achieve the better performance from your system. However, in the remote case you encounter some audio or video issues, you can try to disable it to resolve them. The default is "enabled".

- "Auto-Enable Bayer Decoding": when activated LOLA tries to auto-detect Bayer encoded images and automatically debayering them. In case this doesn't work properly you can manually activate Bayer demosaicing by using the Color Bayer Decoding command available in the Bayer Color menu (Local/Remote windows). The default is

“enabled”.

- “Auto-Enable Bilinear Interpolation”: when using DirectX video acceleration LOLA automatically set image interpolation to “Bilinear” for a better image quality. Since this operation is directly done by your graphic card, it doesn't add any latency to the overall video rendering pipeline.

When you disable DirectX acceleration (by using Disable DirectX Acceleration command in the local/remote window Options menu) LOLA automatically set interpolation to “Nearest Neighbor”. In this case pay attention that using a zoom

bigger than 100% may consume more CPU time and decrease the overall system

performance, specially if you reactivate bilinear interpolation; so if you hear clicks on audio or notice “stuttering” video effects set the window zoom to 100% and interpolation to “Nearest neighbor”.

- “Use CXP cameras quick startup”: when checked Lola directly configures your digital camera without using the GenICam interface. Since the actual version of the

GenICam libraries has some minor issues that randomly appear, we have preferred to configure the cameras through a direct memory access. However, if you encounter any image size or pixel format configuration issues, try to uncheck this option in order to use the GenICam standard interface.

- “Use LOLA HW Color Correction for CXP cameras (bypass Ximilon settings)”: when activated LOLA tries to directly configure your CoaXPress camera bypassing

Ximilon settings; disable it only when you encounter problems or want to manually configure your camera through Ximilon interface. This option is available only on

Karbon-CXP grabber.

- “Use VFG0 as default virtual frame grabber for CXP cameras”: when using the

Karbon-CXP2 grabber, you have the ability to connect up to two cxp cameras and to select the desired one by selecting VFG0 or VFG1 (VFG stands for Virtual Frame

Grabber; please refer to the BitFlow manual for further details). By leaving this option checked, which is the default value, Lola will not pop up a dialog to prompt you which board/port to open but instead it will automatically select the first one on the list (by

25

LOLA: Low Latency Audio Visual Streaming System

default VFG0, C1, Master, Port 1); if you want to manually configure it, uncheck this option and Lola will prompt you to select the desired VFG at every camera initialization.

- “Use GPU to decode incoming JPEG stream”: here you can specify to use or not the

GPU to decompress incoming stream; the overall decompression speed depends on your

GPU card model. You can monitor the decompression speed by using the Show Window

Info command available in the Options menu (on the Remote Video window); if it's slow you can deselect this option and use the CPU to decompress jpeg streams. This option is available only on LOLA “_Cuda” versions.

- “Incomplete frames rendering threshold”: when using uncompressed video you can

also select the incomplete frames rendering threshold (Fig. 10); 0% means that Lola

must receive ALL video packets from the remote host in order to correctly render the entire video frame. Setting a percentage between 1% to 10% means that LOLA can render the video frame also if it hasn't received all the necessary packets to rebuild the entire frame: in case of some lost network packets, this results in small black lines randomly rendered on the display surface. In normal operation conditions, there shall be no lost data packets, thus this parameter shall be left 0% (which is the default value).

Use it only if you experience some small network problems which affects video.

4.1.3 Network Setup

Network Adapter” section: inside this section you can select the desired Network

Interface Controller and apply or not the “IP and UDP Advanced Filtering” option. This last option should be always checked except in case of network connection errors or the inability to complete the negotiation between the hosts.

Video packet size” set the size in bytes of the transmitted video packet; the default value is

1000 which is a good compromise for overall CPU and network performance. However when using high resolution images (Full-HD) you can try to rise this value up to 1400 bytes in order to reduce the CPU load and to contribute for an overall better system performance. When rising it, please pay attention that larger packets could affect network performance on “non-optimal” network connections.

Figure 11: Lola Setup, Network section

26

LOLA: Low Latency Audio Visual Streaming System

4.2. Local and Remote rendering windows

The features of the local video stream (resolution / pixel depth / frame rate) are shown on

top of the video frame, as shown in Fig. 13 (i.e. “Local host: 1280x1024, 24 bit, 30 FpS”).

It is possible to enlarge the video frame by selecting the desired zoom ratio from the pop-up

menu available on the video frame (Fig. 12) or simply by resizing the window with the mouse; note

that the video aspect ratio is preserved in all cases. Since version 1.4.0 there are no more problems

about CPU usage because video rendering is completely DirectX-Gpu accelerated.

You can monitor window rendering speed and overall video performance by using the Show

Window Info command (Fig. 12) available in the pop-up menu. When you select this option, you

will see some useful information (Rendering mode, FpS, Zoom, Bayer decoding speed, Rendering

speed, Jpeg compression/decompression speed) overlaid on your display surface (Fig. 13).

When using CoaXPress cameras with 24-bit pixel format, an “RGB ↔ BGR” command will

be also enabled in the popup menu (Fig. 12). This command converts the standard GenICam RGB

format outputted by digital cameras to the BGR format used by Windows to render the images on screen; we are waiting for a new camera firmware update from Imperx which should directly perform this conversion internally and with no latency.

Figure 12: local/remote windows right-click pop-up menu

27

LOLA: Low Latency Audio Visual Streaming System

Figure 13: Rendering info overlaid on display surface

4.3. Color Menu and Bayer decoding

Color Bayer Decoding” and “Bayer Matrix” menus, available in the local/remote window

pop-up menu (Fig.12), are intended for use with digital cameras when 8 bit Bayer encoded images

are used (Raw8/Mono8): they allow to switch from B/W to Color (by software Bayer decoding) and to select the correct Bayer matrix type.

If you experience some strange color artifacts, try to change the bayer matrix type: for example the Ximea and Imperx cameras use “BGGR”, instead the PointGrey use “BGGR” on

Flea3 cameras and “GRBG” on the Grasshopper3 ones.

Enable SW Color Correction” command available in the pop-up menu provides support for color corrections (both for demosaiced and RGB images); this is a software correction and so use it carefully because it adds some amount of latency depending on your CPU speed. Note that when digital cameras with an RGB 24bit pixel depth are used, this option is automatically disabled because LOLA can directly control the color corrections through hardware interface and with no latency. In this case use the “Digital Cameras HW Color Correction” command available in the

main window “Options” menu (Fig. 14).

28

LOLA: Low Latency Audio Visual Streaming System

Figure 14: Hardware Color Correction command (main window Options menu)

Figure 15: Hardware Color Correction window for BitFlow CXP grabber (GenTL Interface)

29

LOLA: Low Latency Audio Visual Streaming System

Figure 16: Hardware Color Correction window for Ximea xiQ cameras

Depending on the camera pixel format (Mono8 or RGB), some color correction options may not be available.

For the Ximea digital cameras is also available the “Enable Bad Pixels Correction” option

(Fig. 16); in accordance with the manufacturer's specifications “this correction is done on the host

side (driver level), based on the RAW data received from the camera. This means that the processing is an additional step in the imaging pipeline, but its impact on the overall latency is really marginal (it uses SSE function set)”. However, in order to ensure the maximum LOLA performance, this option is disabled by default.

For the Imperx cameras you can also choose the “White Balance” mode: when set to “Auto” the camera automatically configure its colors gain for you, when set to “Manual” you can manually change the gain for each of the component colors, thus the camera will keep them constantly.

Generally it depends on your needs, but we suggest to set it to “Manual” in order to have a more stable colors and to avoid some unwanted automatic balance corrections under unexpected lightning conditions changes.

NOTE: In order to reduce the required network bandwidth, when using digital cameras

it's highly recommended to use Bayer encoded images; when demosaiced they have the same quality as the RGB ones and generally the overall latency added by decoding process is about 1 ms or lower (Intel Core i7, quad core, 3.0GHz).

To use Bayer encoded images on digital cameras, simply select a camera-file with “Mono8” indication in the Camera Settings drop-down list (i.e. “1280x1024Mono8 ”). Lola will automatically activate the “Color Bayer Decoding” command menu.

30

LOLA: Low Latency Audio Visual Streaming System

4.4. Connection with a remote host

The simplest way to connect to a remote host is by specifying its IP address in the Remote

IP Address box and clicking on the Connect button. You can also assign a “Session name” label for easier identification into the corresponding box.

You may also load a previously saved “Session Configuration File” in which the address information for a given host is stored. These are files with extension .ssn , and can be loaded/saved

using the File Menu (Fig. 17). These files are stored into the same folder where the LOLA

application is. The configuration stored into the “LastSsn.ssn” file is always the settings used in the last LOLA session. If you want to create your own LOLA address/configuration book, just create one .ssn file per each site you usually connect to, and then recall the appropriate one when needed.

To remove a stored session configuration, just delete the corresponding .ssn file in the LOLA folder.

Figure 17: Session Load/Save options

Before sending a connection request, it is recommended to check if the remote host is ready for the connection. After inserting the remote host IP address, use the Check button provided in the control panel below the IP address box: LOLA will issue a “ping request” the remote host at the given IP address, and will also check if LOLA is running on it. An estimate of the network round trip time (RTT) will also be provided. If the remote host is reachable, but LOLA is not running on it, you should ask your remote party to start LOLA on his local host, and wait until your Check test declares LOLA is also running remotely. The LOLA chat service (see section 4.10) works after you enter the remote IP address and can help you in coordinating the connection with your remote partner. Please note that if ICMP network protocol is filtered by network or local firewalls, the

“Check” test will fail.

If LOLA is running on the remote host, pressing the Connect button will cause the two systems to negotiate the connection and start sending their audio/video streams to each other. Only one node needs to initiate the connection. Note that while different video stream characteristics are allowed for the two hosts (e.g., different frame rate, picture resolution, bit depth, color or b/w), the audio stream parameters must be exactly the same. By default, 44100 Hz @ 16 bit, stereo, is assumed. If during the negotiation an incompatible setup is detected, the two hosts will refuse to connect. This is particularly important if audio multichannel is enabled.

31

LOLA: Low Latency Audio Visual Streaming System

4.5. Fine Audio/Video buffer tuning

Once the two hosts are connected, the user will have the opportunity to interactively change the number of audio and video buffers used by the application, by using the Buffer Tuning panel

available in the Tools menu (Fig. 18). The number of buffers can be set both using the control slider,

or by typing the value into the box below the slider. Adding buffers means also adding latency: this fact indeed also let you fine tune the audio/video synchronization, for example by adding audio buffers. A balance between optimal latency, quality and synchronization is something you should try to achieve, depending on the specific configuration situation.

For the video, you can control both the number of buffers you use before sending your video

(“Local”) and the number of buffers you use before displaying the received remote video

(“Remote”). For the audio you can only control the number of buffers you use before rendering the received remote audio.

The number of buffers can be increased if dropouts or glitches are noticed in the received audio or video streams, due to network traffic or delay jitter, otherwise it should be kept as low as possible to avoid additional latency. In a typical correct situation video buffers are both set to “0”, and audio buffers should be possibly “0” and not exceed “5”. LOLA remembers the last settings of all Audio and Video Buffers, and will also save them into all specific .ssn files you save; thus you can also save the “ideal” settings you like for any remote location you connect to.

Figure 18: Buffer tuning panel

4.6. Toggling Audio/Video and Network Monitor

During a connection, the user can also gain some information on the audio and video stream

through the Network Monitor panel, available in the Tools menu (Fig. 19). This dialog box provides

some information on the connection and on the traffic, as well as the possibility of selectively

32

LOLA: Low Latency Audio Visual Streaming System

switching off/on each one of the inbound, outbound, audio and video streams.

The Network Monitor windows Video section also show real-time values of send/received video frames: “Local window Fps” and “Sent Fps” shall always be close to “Local Fps settings” values. At the same time “Remote windows fps” and “Received Fps” shall always be close to

“Remote Fps Settings” values. If you notice quite different values, you should start investigating for transmission losses problems (differences in Remote values) or in local PC performances (Local values). For a detailed debugging help using Lola Network Monitor and TaskMgr, see Appendix I.

Figure 19: Network monitor panel

4.7. Disconnecting from a remote host

The connection can be interrupted by any of the two hosts by clicking on the Disconnect button.

When the software is shutting down, the current local configuration (i.e., audio and video setup) will be saved to a .ini file located in the same directory as the executable, and will be loaded at the next start-up of the Lola application. Also, the last session information (remote host address and info) is stored for future sessions.

4.8. Debugging a connection: Bounce-Back mode

To allow some easier debugging in case of problems, and to let you check the quality of your

33

LOLA: Low Latency Audio Visual Streaming System

transmission as received on the remote site, under the “Tools” menu you can enable the “Bounce-

back mode”. When selected, the remote LOLA host will start bouncing back your video stream instead of sending its own. This action does not require any intervention on the remote host. On the other hand, if you also want to have your audio to be bounced-back, an action is needed on the remote host, by either physically connecting the in-out audio on the card, or by telling the audio card mixer to route the audio in back to out channel. This bounce-back feature is still experimental and also requires the remote host to use quite a lot of processing power.

Note 1: connected hosts must have the same video compressed or uncompressed configuration. If you use an “asymmetric” video connection (compressed with uncompressed) you cannot use the bounce-back mode.

Note 2: Currently, Lola supports bounce-back mode on uncompressed streams only.

4.9. Debugging a connection: TaskMgr

LOLA strictly rely on an error free stable network connection. To check the status of what you are sending/receiving from your own PC, you should run the Windows “TaskMgr” application.

Figure 20: Windows Task Manager

Select the “network” tab, and under the “show” menu display all total (green), input (yellow) and output (red) real time traffic statistics. When everything is working fine (with uncompressed video - default) you should see something like the above picture: a very stable constant traffic flow, both in and out. If the 2 nodes are configured in the same way (same frame rate, same resolution, same number of audio channels, etc) the input and output traffic shall be very close to each other. If you see that the input traffic is unstable (and of course LOLA loses lot of audio and video data), your should start investigating the network setup. It means that at some point of the connection between the remote sending node and your node traffic is dropped. This can happen in the various places: the network backbone connections (check with your network supplier), or on the remote

34

LOLA: Low Latency Audio Visual Streaming System

Local Area Network (check with the remote partner) or in your own Local Area Network. In case of

LAN problems, check that you do not have any firewall of any kind in place on the LAN, because most of them cannot cope with LOLA traffic correctly. You should also check that your LAN switches can work with LOLA traffic (at 30000 Packets Per Second PPS for 30fps in color SD), and in case check if you have any IP traffic shaper in place which can affect LOLA traffic, too.

4.10. Audio and Video Recording

4.10.1 Audio recording

LOLA can record an audio session at full quality, both for the local and the remote audio. If you specify a file name and press the “Start Now” button on the main window, LOLA will create into specified folder 2 audio files, with “_Local” and and “_Remote” suffixes which contains all audio channels being send/received during a session. Please note that if you use more than 2 audio channels, the audio recording files will contain all audio channels, thus you need an appropriate audio player/editor in order to be able to open them all. Inside LOLA folder you will also find

LolaWavSplitter.exe: it's a small utility to extract channels from a multichannel .wav file (see

“LOLA Tools” chapter for more details).

4.10.2 Video recording

Starting from version 1.4.0, LOLA can also record video sessions (both local and remote streams).

Figure 21: Audio/Video Recording Section (main Lola window)

In the “Audio/Video Recording” section of the main window, you can Start/Stop recording, select which stream to record (both Audio and Video, Audio only or Video only) and set the output filename. To select the output path or to change the default recording settings select the “A/V

Recording Setup” command available in the main window “Options” menu or simply press

Ctrl+Shift+R. Please refer to the following section 4.9.3 for further details.

Note: When recording uncompressed data, please pay attention to your hdd/sdd writing

speed. For example the required bandwidth for an uncompressed 1920x1080 RGB video stream at

30 fps is about 178 MB/s and recording both local and remote streams on the same disk raises it to

356 MB/s. Using a Bayer encoded video with the same resolution and fps reduces the bandwidth demand to 59.33 MB/s or using Jpeg compression drastically reduces it to about 7 MB/s or lower

35

LOLA: Low Latency Audio Visual Streaming System

(depending on compression quality). To achieve better performance it is also suggested to use different physical disks for recording local and remote streams.

LOLA writes the video streams to disk as a sequence of numbered image files (one file for each frame) using bmp or jpg formats; this system should grant a good compatibility with general video-editing softwares.

4.10.3 Audio and Video Recording Setup Window

Inside “A/V Recording Setup” window you can change various recording settings. You can specify the output paths, compression (uncompressed or jpeg compressed images) and decide which stream to record (local, remote or both).

Figure 22: Audio/Video Recording Section (main LOLA window)

4.11. Chat window

In order to resolve possible issues with the remote host, LOLA has the ability to “chat” with the remote node. To open the chat window select the Chat window command available in the Tools menu or simply press Ctrl+L. The remote address where the message will be sent is the same as that

36

LOLA: Low Latency Audio Visual Streaming System

indicated in the main window in the Remote Session/IP Address field. To send the message press

CTRL+Return. Chat window works also when LOLA is not connected.

4.12. Audio Test signal

To test the audio connection Lola can send an internally generated audio signal (689Hz,

-3dBfs) to the remote host. To do so use the Send Audio Test Signal command available in the Tools menu.

4.13. Windows layout

When Lola is connected to the remote host, you can decide how to automatically dispose the

Local and Remote windows on the screen. Three layouts are available: tile vertically (Ctrl+Shift+1), tile horizontally (Ctrl+Shift+2) or maximize remote (Ctrl+Shift+3). This last is really useful when using HD or Full-HD setups.

4.14. Duplicate window (remote host display)

When Lola is connected to the remote host, you can duplicate the Remote host window by clicking on the Create/Destroy Duplicate Window command available in the main window Options menu. A new remote window clone will be displayed; to destroy it simply click another time on

Create/Destroy Duplicate Window command.

Figure 23: Duplicate window (remote host)

37

LOLA: Low Latency Audio Visual Streaming System

Duplicating the remote windows is useful to organize your local video setup, enabling scenarios where you need to handle in different ways the video coming from the remote site. For example imagine that you need to use a monitor dedicated for the users involved in the connection, and you need another different image to display to the audience on a different screen or projector.

As you can zoom/resize independently the 2 video windows, you can arrange them in the more appropriate way for your setup.

4.15. LolaGui process priority

Starting with version 1.4.0, you can change the LolaGui executable process priority by using the “LolaGUI process priority” menu in the Options menu (main window).

In the previous versions it was hard coded and set to “High priority” by default; but after adjusting and improving some internal thread settings, LOLA can now work with a “Normal

priority” level. This change has brought some benefits for the audio thread stability, specially when using HD video streams and/or CUDA acceleration (it primarily avoids some audio “click” issues).

You can try to raise the priority level to “Above Normal” or “High” and see if this could improve your LOLA performance or not; it highly depends on your CPU type, speed and number of cores.

4.16. Audio only setup

If you haven't installed a video grabber on your system (BitFlow cards or USB3-Vision cameras), you can use Lola in an “audio only” setup. To do so please use the XIMEA version because it doesn't require additional BitFlow SDK drivers and in this case it works better. During start up, Lola will warn you about no video devices found and ask you if you want to continue; answer “yes” and you are ready to go.

4.17. Lola system checkup

When Lola is running, please make sure to check some important things:

If you have more than one network card, make sure to select the right one (the one really connected to your network and correctly configured); otherwise Lola will fail to connect or show you a black “Remote host” window. To check or set your network adapter open the

“Lola Setup” dialog (Options menu → Lola setup).

38

LOLA: Low Latency Audio Visual Streaming System

After applying your camera “Fps” setting, always check its correctness by opening the Lola

Network Monitor (Tools menu → Network monitor) and look if the “Local FpS settings” and “Local window FpS” values match.

You can also check it by using the “Show Window Info” command in your “Local host” window context menu; when selected a “Fps” info will be overlaid on the display surface

(together with other useful information).

When selecting a camera file and the relative image size and pixel format (Mono8 or

RGB8), please take a look at the estimated TX bandwidth value in order to avoid sending too much data and packets to the remote host (this can cause video stuttering or audio crackling noise issues to the receiving site).

Make sure to use the correct Lola version that corresponds to your OS bitness (32 or 64 bit) and hardware setup, otherwise Lola will not work.

39

LOLA: Low Latency Audio Visual Streaming System

5. LOLA Tools

Provided with LOLA there are three useful tools: LolaWavSplitter, LolaVideoConverter and

LolaGui_Tester.

5.1. Lola Wav Splitter

This tool converts multichannel audio files into separate mono files.

Figure 24: Lola Wav Splitter application

5.2. Lola Video Converter

In order to optimize the overall system performance, Lola writes raw image files when using

Bayer encoded streams or Jpeg BGR files when using compressed streams. So, before importing these files into your preferred video editing software, you can use this tool to convert them into one of the following formats:

8 bit Bayer encoded to 24 bit RGB;

BGR to RGB (for 24 bit Jpeg compressed images)

Uncompressed Bmp to Jpeg;

Using "Browse" button you can specify the folder where you previously saved the A/V files; in the Files info section it will display the total number of video files, their format (bmp or jpg) and image info. It also suggests the more appropriate conversion option.

frame.

Click on the "Preview" buttons to have a conversion preview based on the first captured

40

LOLA: Low Latency Audio Visual Streaming System

Cliking on "Convert" button will start the conversion process. When finished you will find the converted files in the same "Input path" location but inside a newly created folder named

"ConvertedFiles_[DD_MM_YY]_[hh_mm_ss]" (where DD=day, MM=month, YY=year, hh=hours, mm=minutes and ss=seconds). Files inside new folder maintain their original name.

Figure 25: Lola Video Converter application

5.3. LolaGUI Tester

This tool simulates a LOLA setup which runs without audio and video equipment, but generates a nearly identical network traffic as the real LOLA does. Currently it only generates a video data stream, because is really difficult to emulate the RME Audio cards internal

41

LOLA: Low Latency Audio Visual Streaming System

clock/interrupts precision; but given the important network traffic is generated by video stream it works fine as a quick to use tester.

You can use it to connect between two "tester" installations, but also between a real LOLA application and a tester.

Since version 1.0.3 there are a jitter indication and a jitter graph in order to get more detailed info about your network performance.

Figure 26: LolaGUI tester application

42

LOLA: Low Latency Audio Visual Streaming System

Appendix A – Old experimental digital video setup

A.1 - CameraLink

A CameraLink Digital (B/W and Color) video configuration was tested but for various reasons, first of all the small maximum cable length (10m) but also for the greater latency, it is abandoned. The

Camera Link configuration under test was as follows:

BitFlow Neon-CL grabber with SDK 5.60 drivers;

Sentech STC-PoCL33A color video camera (color output is Bayer encoded, needs SW

Bayer decoding).

Digital Camera Link cameras are found on the market in a wide variety of brands and models, and a selection of cameras, both B/W or RGB and likely to be compatible with LOLA, is as follows:

Adimec (e.g., A1000 Camera Link or Opal2000-CL ), Basler (e.g., L100 or L300 series), CIS (e.g.

VCC-F32 series), Hitachi (e.g., KP-FD30CL), Imperx (e.g., ICL-B0610 ), Leutron Vision (e.g.,

PicSight P34B-PoCL), Jai (e.g., RMC-6740CL), PhotonFocus (e.g., MV-D640(C) series), Sentech

(e.g., STC-CLC33A ), Toshiba-Teli (e.g., CS6910CL ), Uniq(e.g., UP600-CL).

There is NO MORE support for this type of video setup.

43

LOLA: Low Latency Audio Visual Streaming System

Appendix B – Compatible Video Analog Cameras

Analog B/W cameras with characteristics similar to the Sony HR50 can be easily found. On the other hand, analog color cameras with frame rates > 30 fps @ 640x480 and RGB output seems to be rare on the market. A few examples of brands offering analog RGB cameras are Toshiba (e.g.,

IK-TF5), JAI-Pulnix (e.g. the

CV-M91

series), Hitachi (e.g. KP-FD30).

Using a different camera (color or b/w) usually requires adapting/creating a new “camera file” specific for the camera itself. You can start from the ones available in the distribution kit, and create a new one for the intended model. Finally, the camera file must also be compliant to external trigger mode. This can be achieved by editing a camera file with the CamEd.exe tool (available in

Start/All Programs/BitFlow/Sdk 5.90/), and setting the External trigger flag to Enabled in the

Triggering menu item (Fig. B1).

Figure B1:

The CamEd.exe utility, and the external trigger flag set to “enable”.

Other parameters which might need adjustment are the camera resolution and the video sensor offsets. Please contact [email protected]

for assistance if you plan to use a different camera from the ones in the supported list.

Work is also in progress to support generic RGB and YpbPr input video sources.

44

LOLA: Low Latency Audio Visual Streaming System

Appendix C – Cameras Cabling Guide

Although you can buy appropriate cables for many of the analog cameras to be used with the BitFlow grabber cards, this short guide can help you in building your own high quality cables.

In order to start, we suggest to use high quality VGA cables, which you can quite easily get anywhere. A good reference model are for example the Belkin F3H982, which comes in different lengths. In some of the tables you will find specific examples built using the Belkin F3H982 cables. Another very good alternate cable, which also supplies power via the BitFlow card to the cameras, is the KSI (Kontact-Systeme Inter

Gmbh) model HYB 8 + 3 Koax (nr. 75 65 380) http://www.ksi.at/online-kataloge/kat7A/7A-565/7-565.pdf.

For b/w cameras you will often need a 12 pin Hirosi connector model HA10A-10P-12S, while color cameras often use a standard 15pin VGA connector, thus you do not need to replace the one already on the cable.

For the BitFlow grabber 62 pin connectors the suggested parts are:

Plug model AMP 748367-1, Shielding

HW model AMP 748471-1, Enclosure model AMP 748476-1

Please always check the camera documentation for the correct cabling. The following are the schemas for

LOLA supported cameras. In addition here is the BitFlow 62 pin cabling full specification :

45

LOLA: Low Latency Audio Visual Streaming System

46

LOLA: Low Latency Audio Visual Streaming System

Cable for Sony HR50

The VGA connector has 3 coaxial cables for video signal, but for b/w cameras you only need one; just select, for example, the "G" (green) one. Use the twisted pairs for the other signals. Remove both VGA connectors from the cable, and connect the pins as follows:

Hirosi 62pin signal comments

1

6

7

8

9

2

3

4

5

10

11

12

43

24

47

4

23

5

3

2

44

26

25

48

Ground

+12V

Ground

VIDEO1

Ground

HD

VD

Ground

Ground

WEN_IN

TRIG_OUT

Ground

VIDEO1 shield, coax

VIDEO1 signal, coax

Cable for Hitachi KP-FD30

Remove one of the VGA connectors from the cable and connect to the 62 pin connector as shown in

Fig. C1, the other VGA connector is already correctly connected to go on the camera.

VD

HD

23 24 25

4

26

47

6 trigger wen r b g wire wire wire

11 shield shield shield

Figure C1:

The cabling for the Hitachi KP-FD30 camera.

47

LOLA: Low Latency Audio Visual Streaming System

Here is a detailed table for pinout connections, if you use the Belkin

F3H982 and the Hitachi:

(NC) = Not Connected

15 Pin Hitachi camera

1

2

3

4

R/C Output

G/Y Output

B/VBS Output

WE Output

5 GND (TXD/RXD)

6 Video GND

7 Video GND

8 Video GND

9 Unreg. +12V Input

10

11

12

13

14

15

Trig Input

GND

RXD

HD In/Out/Sync

Out

VD In/VD Out

TXD

Shield GND

Comment

For Y/C mode

For Y/C mode

Belkin cable 62 Pin DIN Comment

Red Coaxial 23 2 tap mode?

Green Coaxial

Wen in Y/C mode? Blue Coaxial

2

1

2 tap mode?

Wen?

Black 26

Brown

Red Shield

Green Shield

Blue Shield

(NC)

11

6

4

Power from

BitFlow

(No wire) 5

No wire available on Belkin cable

Red

Orange

Yellow

25

(NC)

(NC)

HD=Horizontal

Direction

VD=Vertical

Direction

Wen?

Green

Grey

White

24

47

(NC)

Shield/Copper (NC) or 62

Connct to improve

EMI/RFI

Rejection

Power Supply from the BitFlow card: the Belkin F3H982 cable does not have a wire to provide power supply from the BitFlow board card (pin 5) to the camera (pin 9). If the cable you are using has a wire available, just make the connection as above, and you will not need an external power supply for the camera.

Better grounding/shielding: even if not strictly necessary, if you also connect the shield/ground as in the table above, you may gain in image quality, specially in “noisy” area with a lot of EMI/RFI.

48

LOLA: Low Latency Audio Visual Streaming System

Cable for Toshiba IK-TF5

The cable for the Toshiba is very similar to the one for the Hitachi. Remove one of the VGA connectors from the cable and connect to the 62 pin connector as shown in the following table (the other VGA connector is already correctly connected to go on the camera), F3H982 Belkin cable:

(NC) = Not Connected

15 Pin Toshiba camera

1 R Output

2 G Output

3 B Output

4 TXD

5

9

10

11

12

13

14

15

GND

6 GND

7 GND

8 Sync

12V

GND

RXD

Trig

HD In/Out

VD In/Out

Index

Shield GND

Comment

No Y/C mode?

Wen?

Power from

BitFlow

Wen?

Belkin cable 62 Pin DIN Comment

Red Coaxial 23

Green Coaxial

Blue Coaxial

Black

2

1

(NC)

Brown 4

Shorted to Blue

Shield

Red Shield

Green Shield

11

6

Blue Shield

(No wire)

4

5

Shorted to Blue

Shield

No wire available on

Belkin cable

Red

Orange

Yellow

Green

Grey

White

(NC)

(NC)

25

24

47

26

Shield/Copper (NC) or 62

Connect to improve

EMI/RFI

Rejection

Power Supply from the BitFlow card: the Belkin F3H982 cable does not have a wire to provide power supply from the BitFlow board card (pin 5) to the camera (pin 9). If the cable you are using has a wire available, just make the connection as above, and you will not need an external power supply for the camera.

Better grounding/shielding: even if not strictly ncessary, if you also connect the shield/ground as in the table above, you may gain in image quality, expecially in “noisy” area with a lot of EMI/RFI.

49

LOLA: Low Latency Audio Visual Streaming System

Appendix D – Setting Cameras in External Trigger Mode

This section show how to correctly set in External Trigger Mode the supported cameras. If you are using a camera not listed here, contact [email protected]

for assistance.

Sony HR50 setting for external trigger mode

If using the Sony HR50 B/W analog camera, t he camera must be configured according to external trig/shutter mode 2

(DIP switches 6 and 7 set to OFF, and DIP switch 8 set to ON) and with the switch n. 4 set to ON.

Figure D1: Sony HR50 DIP switch settings

Sony HR70 setting for external trigger mode

If using the Sony HR70 B/W analog camera, t he camera must be configured according to external trig/shutter mode 2

(DIP switches 6 and 7 set to OFF, and DIP switch 8 set to ON).

Figure D2: Sony HR70 DIP switch setting s

Note: the default exposure setting for Lola is 100, but when using the Sony XC-HR70 camera this

value causes some artifacts on the image (black lines or superimposed ghost images).

50

LOLA: Low Latency Audio Visual Streaming System

As a workaround you have to manually set the "Exposure" value in the LolaGui.ini file (created by

Lola during the first start/end cycle) using the following indications:

- 640x480, 30 fps >> Exposure=31.000000

- 1024x768, 29 fps >> Exposure=30.000000

If you don't want to edit the file every time you change the camera file, you can set the following values which work for both:

- FrameRate=29.000000

- Exposure=30.000000

Hitachi KP-FD30 setting for external trigger mode

If using the Hitachi KP-FD30 color analog camera, the camera must be configured according to external trig/shutter mode: the internal DIP switches (accessed by removing the back panel of the camera) should be set as follows: SW6001 - Lower side (input sync), SW6002 - Upper side (Fig.

D3).

Figure D3: Hitachi KP-FD30 internal switch setting for external trig/shutter mode.

51

LOLA: Low Latency Audio Visual Streaming System

The camera rear switch should be set for progressive scan as follows: 1-On (VGA progressive scan), 2-Off, 3-Off, 4-Off (Fig. D4).

Figure D4: Hitachi KP=FD30 rear DIP switch setting for progressive scan mode.

Finally, the "shutter mode" field in the software setting menu of the camera should be set to "Ext. Trigger"

Main Menu -> shutter Menu -> Mode: ext trig

Trigger Mode: one trig

Trig polarity: negative

Wen Polarity: positive

Main Menu -> Sub Menu -> Item: preset

Sync/HD out: HD

Sync On G: On

Toshiba IK-TF5 setting for external trigger mode

The Toshiba IK-TF5 color analog camera should be configured according to external trig/shutter mode as follows:

1. the internal DIP switch (accessed by removing the metal chassis of the camera) should be set in the HD/VD Input 75 Ohm position (factory setting), as in Fig. D5.

Figure D5: IK-TF5 internal DIP switch setting.

52

LOLA: Low Latency Audio Visual Streaming System

2. the "shutter " Menu fields in the software setting menu of the camera (Fig, D6) should be set as follows:

Index Menu -> shutter Menu -> Mode: E. TRG

E. TRG: 1P SR

TRG P.: positive polarity

EXP: 16 msec

PART: off

Figure D6: IK-TF5 shutter menu settings

53

LOLA: Low Latency Audio Visual Streaming System

Appendix E – Audio Card internal Mixer settings

LOLA usually adopt RME audio cards for audio in/out functions. In order to work correctly, these audio cards must be setup in the correct configuration to ensure LOLA gets the correct audio input channels, and also can output the sound from the remote host correctly. You should at first display the audio card routing matrix (see Fig. E1 and E2). The top row displays the available input channels. This information is useful to set appropriately the AudioInputOffset parameter. In both examples, to use AN1 and AN2 analog channels the AudioInputOffset must be set to 0; to use the ADAT A1...A8 channels, the AudioInputOffset must be set to 8 in the Fig. E1 example, and to 4 in the Fig. E2 one.

Figure E1: the HDSP Matrix, with audio out routing for analogue and ADAT channels

Figure E2: a different HDSP Matrix, with audio out routing for the ADAT channels only

54

LOLA: Low Latency Audio Visual Streaming System

Then the output routing shall be correctly configured. Fig. E1 shows a correct routing example both for the analogue channels and for the ADAT channels. Fig. E2 shows a configuration where only the 8 ADAT channels are routed to the output.

The HDSP audio Mixer should also look like in Fig. E3 or E4. Please note that buttons link Out1 Out2 AN1

AN1 shall NOT be in red/pink color, because this means you set the audio card in local audio loop.

Figure E3: The RME main Mixer window (stereo analog in/out setup)

55

LOLA: Low Latency Audio Visual Streaming System

Figure E4: The RME main Mixer window (8 channels out setup)

56

LOLA: Low Latency Audio Visual Streaming System

Appendix F – Manual BitFlow Configuration

Please update/install the BitFlow drivers first! (see section 3.2).

F.1 Camera file installation

Before using an application relying on BitFlow video devices, a so called camera

configuration file must be installed for each camera possibly connected to the BitFlow grabber. The utility SysReg.exe, available in Start/All Programs/BitFlow/SDK 5.90/, can be used to add a new camera file to the grabber (note: for these tools to affect the system configuration in Windows 7, they must be “run as Administrator”). It is useful to also install test camera files which can generate synthetic video with a wide range of frame rate and resolution characteristics. You should install the camera files for the video camera(s) you intend to use with LOLA. The current camera files for the

LOLA tested cameras are included into the “CameraFiles” folder inside the LOLA distribution kit

(if not already included into the BitFlow standard distribution):

- Sony HR50:

- Sony HR70:

- Hitachi KP-FD30:

- Toshiba IK-TF5:

00-Sony-XCHR50-640x480-Mono8-Fps60.anlg

00-Sony-XCHR70-640x480-Mono8-Fps30.anlg

00-Sony-XCHR70-1024x768-Mono8-Fps29.anlg

Note: This camera is supported but not really suggested because it requires

some hacks inside LolaGui.ini file and can only reach 30fps.

00-Hitachi-KPFD30-640x480-Mono8-Fps60.anlg

00-Hitachi-KPFD30-640x480-RGB8-Fps60.anlg

00-Toshiba-IKTF5-640x480-Mono8-Fps60.anlg

00-Toshiba-IKTF5-640x480-RGB8-Fps60.anlg

Color cameras can also run in b/w (for example because you want to reduce network bandwidth use); you just need to load and then select the appropriate b/w configuration Camera

File, which is identified by the “Mono8” indication in the name.

.

Figure F1: SysReg.exe utility. Add button allows to choose

57

LOLA: Low Latency Audio Visual Streaming System

new camera files from the list of available ones

When the camera files are installed, you can select the appropriate camera from the Audio

Video Setup dialog window, available in the LOLA Tools menu (Fig. 4).

LOLA allows the user to select different frame rates (from 25 up to 60 fps) from its own setup dialog windows. In order to make this option work, the camera must be configured to

allow triggering by an external source. See Appendix D to correctly set the supported analog cameras. We can also supply on request camera files with internal triggering (...SYNC-ON-VID ones) but they should not be used with LOLA.

F.2 Additional steps to manually configure CoaXPress Digital cameras

Launch Ximilon, expand your camera-name node (i.e. “Imperx ICX-B1410C”), “Camera

Feature Set” node and “AcquisitionAndTriggerControls” node and then change the following parameters:

- “Imperx CXP-B1410C” and “Imperx CXP-B1411C” (Gen 2)

- Exposure:

- ExposureMode = Off;

- Trigger

- TriggerActivation = RisingEdge;

- TriggerMode = On;

- TriggerOverlap = Off;

- TriggerSource = CxpTrigger;

- TriggerType = Fast;

- ImageSizeControl

- CameraSpeed = OverClocked;

- “Imperx ICX-B1410C” (Gen 1)

- Exposure:

- ExposureMode = Off;

- Trigger

- TriggerActivation = RisingEdge;

- TriggerMode = Off;

- TriggerOverlap = Off;

- TriggerSource = TimerTrigger;

- TriggerType = Fast;

58

LOLA: Low Latency Audio Visual Streaming System

- ImageSizeControl

- CameraSpeed = OverClocked;

- “Adimec OPAL-2000C”

- Acquisition Control:

- ExposureMode = TriggerWidth

- ExposureTime = 15140

- FrameStartActivation = RisingEdge

- FrameStartSource = Trigger

- TriggerActivation = RisingEdge

- TriggerSource = Trigger

If you manually configure your camera, you must also change the Width, Height and

PixelFormat values according to your BitFlow camerafile settings.

F.3 Camera Setup Check

After the camera files have been registered, it is recommended to check that the video grabbing devices are working properly, by using CiView.exe utility provided with the BitFlow SDK in the same location as the SysReg.exe tool. Note that with this particular configuration, the camera must triggered by an external trigger. This can be achieved by running the application in “Free Run

Mode” (this option is available in the Preview menu).

When LOLA is launched, it will check for all available video BitFlow devices (if none is available, the program will provide a warning and run without video support). It will be possible to select the desired video camera, among the ones listed inside Lola “CameraFiles” folder.

59

LOLA: Low Latency Audio Visual Streaming System

Appendix G – Lola Hardware components

HIGH-END PC (Full HD resolution, 1920x1080, RGB) :

Mid/Full tower case with quiet design, a good air flow circulation and fans, and at least four full height expansion slots (Antec Sonata Solo II or similar)

Power Supply Unit: at least 860W (Corsair AX860 or better)

Processor: Intel Core i7 (4930K, 6 cores or better)

16GB RAM DDR3 (Corsair or Kingston, 4x4GB, Quad Channel DDR3 Memory Kit)

Motherboard: an high-end motherbord with Intel chipset X79 (i.e. M/B ASRock Fatal1ty X79

Professional or similar) or better with enough PCI-express slots for:

- 1 Graphics Card (16x slot required) … Note: high-end graphics cards occupy the space of two slots!

- 1 BitFlow frame grabber (for the Karbon-CXP a 8x slot is required, for the ALTA-AN a 4x slot is required) or 1 USB3.0 adapter card in case the motherboard has an incompatible USB3-Vision chipset

- 1 RME Audio Card (for the HDSPe-AIO a 1x slot is required)

SSD (suggested for OS)

2 HDD - 1TB (Raid 0 configuration) for uncompressed video recording (Western Digital WD-Black)

Nvidia GeForce GTX 770 or better (EVGA) … suggested for jpeg hardware acceleration

Intel Gigabit CT Desktop Ethernet Pci-e adapter (required when an Intel NIC is not integrated on the

motherboard)

Full-HD LCD Monitor with low response time (<= 2ms)

OS Windows 7 or 8.1 (32-bit or 64-bit)

MID-RANGE PC (SD/HD resolution, 1024x768, RGB ):

Mid/Full tower case with quiet design, a good air flow circulation and fans, and at least four full height expansion slots (Antec suggested)

Power Supply Unit: at least 500/600W (Corsair suggested); it depends on Graphics card

Processor: Intel Core i7 (3770K, 4 cores)

8 GB RAM DDR3 (Corsair or Kingston suggested)

Motherboard: a mid-range motherbord with Intel chipset and enough PCI-express slots for:

- 1 Graphics Card (a 16x slot is required) … Note: high-end graphics cards occupy the space of two slots!

- 1 BitFlow frame grabber (for the Karbon-CXP a 8x slot is required, for the ALTA-AN a 4x slot is required) or 1 USB3.0 adapter card in case the motherboard has an incompatible USB3-Vision chipset

- 1 RME Audio Card (for the HDSPe-AIO a 1x slot is required)

Nvidia GeForce GTX 650 (suggested for jpeg hardware acceleration)

Intel Gigabit CT Desktop Ethernet Pci-e adapter (required when an Intel NIC is not integrated on the motherboard)

OS Windows 7 or 8.1 (32-bit or 64-bit)

LOW-END PC (SD resolution, 640x480, RGB):

Mid tower case with qa good air flow circulation and fans, and at least four full height expansion slots

Power Supply Unit: at least 400/500W

Processor: Intel Core i5

60

LOLA: Low Latency Audio Visual Streaming System

4 GB RAM (DDR3)

Motherboard: a mid-range motherbord with Intel chipset and enough PCI-express slots for:

- 1 Graphics Card (a 16x slot is required)

- 1 BitFlow frame grabber (for the Karbon-CXP a 8x slot is required, for the ALTA-AN a 4x slot is required) or 1 USB3.0 adapter card in case the motherboard has an incompatible USB3-Vision chipset

- 1 RME Audio Card (for the HDSPe-AIO a 1x slot is required)

Intel Gigabit CT Desktop Ethernet Pci-e adapter (required when an Intel NIC is not integrated on the motherboard)

OS Windows 7 or 8.1 (32-bit or 64-bit)

- Digital cameras and frame grabbers with CoaXPress interface:

BitFlow Karbon-CXP2

Imperx B1410C-CXP camera

Imperx B1411C-CXP camera

Adimec OPAL-2000C (this is still in beta-testing)

- USB3Vision Digital Cameras (no frame grabber required)

XIMEA MQ013CG-E2 camera

- Analog cameras and frame grabbers:

BitFlow ALTA-AN1

Hitachi KP-FD30

(Toshiba IK-TF5 - Discontinued)

- Audio Cards:

RME HDSPe-AIO (Pci-e)

- Possible Audio Components

2 small Genelec or Yamaha audio monitors

1 active audio PA system (fox example Mackie)

2 condenser high quality microphones (for example Neumann KM184)

1 dynamic microphone (service voice)

1 analogue mixer (minimum 16ch in, 4 out + Main L/R, 4 AUX (for example Mackie), or one digital mixer with ADAT support

Various audio cables

61

LOLA: Low Latency Audio Visual Streaming System

Appendix H – Known Issues

H.1 No timeout on remote partner unexpected disconnection

In some rare occasions, if the remote LOLA system occurs in an unexpected disconnection (system or application problems, network disconnection, etc) and thus cannot notify the other system about its closure, the local system remains “open and active” without warning (you of course immediately see you are not receiving any more audio/video from the remote system). In such a case, just quit LOLA manually and restart it to be ready for a new connection. If the remote system resumes transmission without requesting a new “connect” (for example because it has been a network disconnection problem), both LOLA systems will realign and restart normal operations without any action needed by you on both sides.

H.2 Windows Firewall and LOLA initial “check”

Windows 7 Firewall by default completely blocks ICMP (“ping”) packets. This makes the initial

LOLA “check” function unable to work correctly. To ensure that you can correctly use LOLA, without risking to run with a Firewall completely disabled (if you are on the public Internet), you should configure

Windows Firewall with a rule enabling ICMP packets flow on your system (as well as on the remote system, of course).

H.3 LOLA and antivirus software

LOLA uses an IP network packets building engine based on WinPcap libraries. Many antivirus packages will detect this and list LOLA as a “malicious” software and try to block it. Until these antivirus software will correctly recognize LOLA, you should disable them on your system. More over, in any case, antivirus packages perform highly invasive activities on your PC, which can seriously compromise system performances and LOLA itself. Thus you should carefully disable them while using LOLA.

H.4 Packet Per Second (PPS) and LAN Switches

LOLA produces a very high UDP packet per second (PPS) rate, especially when in color mode at

60fps. A number of LAN switches might be unable to cope with such a traffic, especially if there are other applications running on the same switch which also requires a high PPS rate or packets priority handling

(like VoIP system/telephones, etc). If you see, for example via TaskMGR “network monitoring” application a very irregular traffic pattern (and you get audio/video drops, clicks, artifacts), try to disable other applications running on the same switches, or try to setup a direct connection between the LOLA system and your border router, avoiding the LAN switches which cause the problem.

H.5 Packet Per Second (PPS) and NICs

Network Interface Cards (NICs) based on Realtek, Broadcom and some on Maverick chipset cannot deliver correctly the PPS rate required by LOLA. This issue is know also when these NICs are used for other kind of applications and depends on the the way the NIC drivers are handling traffic. There is no workaround

62

LOLA: Low Latency Audio Visual Streaming System

to this issue until the manufacturer fix it either in the NIC itself of the drivers. We suggest to use Intel based

NICs which work fine. Please see some important informations about “Interrupt moderation rate” settings for the Intel NICs described at chapter 2.8. Network Interface Cards (NICs) performances.

H.6 Network Configuration on the same Virtual LAN

If both LOLA systems are on the same Virtual LAN (VLAN), for example a GigaEthernet Circuit with IP addresses in the same subnet, the current LOLA IP packet generation engine can behave not correctly and you might have the systems to connect correctly, but no audio or video or both streaming flow among them. You can see this in the Network Monitor panel, where the counters do not increase.

While a fix is in progress, a workaround to this problem is to configure the systems (PCs) Network

Connection Control Panel in a way that the “Gateway” IP address of one PC is set to the IP address of the other PC (and the same on the other side).

If system “A” has

IP: 140.2.1.10 mask 255.255.255.0

and system “B” has

IP: 140.2.1.20 mask 255.255.255.0

then the full system “A” configuration shall be:

IP: 140.2.1.10 mask 255.255.255.0 GW: 140.2.1.20

and the full system “B” configuration shall be:

IP: 140.2.1.20 mask 255.255.255.0 GW: 140.2.1.10

H.7 Sony XC-HR70 camera exposure settings

The default exposure setting for Lola is 100.000000, but when using the Sony XC-HR70 camera this value causes some artifacts on the image (black lines or superimposed ghost images). As a workaround you have to manually set the "Exposure" value in the LolaGui.ini file using the following indications:

640x480, 30fps → Exposure=31.000000

1024x768, 29fps → Exposure=30.000000

If you don't want to edit the file every time you change the camerafile, you can set the following values which work for both:

63

LOLA: Low Latency Audio Visual Streaming System

- FrameRate=29.000000

- Exposure=30.000000

H.8 Video Artifacts if LOCAL and REMOTE video windows overlay on screen.

If the LOCAL and the REMOTE video Windows overlay one to the other on the screen, the foreground window may display some artifacts in the area where the background window is laying behind it.

This issue is still under investigation. Thus avoid to have partially overlapping LOCAL and REMOTE windows on the the screen while using LOLA. The issue is not present if one of the windows uses the full area of the screen.

Fig H1: note the artifact in the area where the 2 windows overlap

(in this case a different color rendering, but it can also be a partially late frame area).

H.9 BitFlow cards not recognized after Windows 8.1 Update 1 - Workaround.

When installing Windows 8.1 Update 1 the BitFlow cards are not recognized at the first cold boot, but after restarting the pc (warm reboot), they work fine as usual. As suggested by BitFlow support, most likely the problem is related to the fact that Windows 8 (by default) never actually shuts down: when you say

"Shut down" it actually sleeps. A work-around is to turn off this behavior, so that "Shut down" really shut's down the computer. It's a little hard to find this setting, but check under the :

- Control Panel > Hardware & Sound > Power Option > System Settings > Shutdown Settings

64

LOLA: Low Latency Audio Visual Streaming System

H.10 Manual installation of the “Microsoft Visual C++ 2010 SP1 Redistributable Package (x86-x64)”

In case you encounter any issues with the included Microsoft libraries, you can manually download and install the package from: http://www.microsoft.com/en-us/download/default.aspx

and searching for "Microsoft Visual C++ 2010 SP1 Redistributable Package (x86)" if you are using a 32-bit

Windows version or “Microsoft Visual C++ 2010 SP1 Redistributable Package (64)” for a 64-bit one. In case you can also apply the appropriate security patches, if any.

H.11 Digital cameras initialization failure when using the GenICam interface

When using the GenICam interface to initialize your digital camera (i.e. when “Use CXP cameras

quick startup” option is unchecked), sometimes it is not correctly initialized (video artifacts, wrong size and pixel format or duplicated images). This is due to the fact that the actual version of the GenICam libraries has some minor issues that randomly appear; as a workaround you have to reinitialize your camera by opening the Lola Setup dialog and re-applying your settings with the “OK” button or by activating the option

“Use CXP cameras quick startup” (in this case Lola configures the camera through a direct memory access).

H.12 Digital cameras bayer encoded images (Mono8/Raw8)

If you experience some strange color artifacts, try to change the bayer matrix: for example the Ximea and Imperx cameras use “BGGR”, instead the PointGrey use “BGGR” for Flea3 cameras and “GRBG” for the Grasshopper3 ones.

65

LOLA: Low Latency Audio Visual Streaming System

Appendix I – Remote assistance with TeamViewer

In order to give you a better assistance or to configure untested digital cameras, we can also check and debug your pc and camera remotely using the TeamViewer software. Before asking for remote assistance, please download the “TeamViewer QuickSupport” executable (please make

sure to download the “QuickSupport” version and not the full TeamViewer installer; the

TeamViewerQS.exe is a simple standalone application that doesn't require any installation).

Contact us at [email protected]

for further information or details.

66

LOLA: Low Latency Audio Visual Streaming System

Appendix J – Connection debugging hints

To debug your connection it's very useful to use the Network Monitor tool in conjunction with TaskMgr. (this section is still under preparation).

67

LOLA: Low Latency Audio Visual Streaming System

Acknowledgments

We would like to thank:

Dan Nichols (NIU School of Music) for his precious work, collaboration and suggestions both for the creation of camera cables and for fine tuning of many Camera Files;

Marco d'Ambrosio (Università di Cassino) for his camera power supply suggestions;

• the GPUJPEG library authors for giving us the permission to use it inside Lola;

• the BitFlow support for the really precious help;

• the Ximea support and specially Max Larin for their useful suggestions;

Otto J. Wittner for having loaned us a PtGrey Flea3 camera, so giving the ability to include the support for Point Grey USB3-Vision cameras inside Lola;

Matt Parkin for finding an insidious cxp bug and letting us to remotely configure and debug his Imperx CXP-B1411C camera;

• the LOLA users community for helping us in testing/debugging and suggesting new features!

68

advertisement

Was this manual useful for you? Yes No
Thank you for your participation!

* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project

Related manuals

Download PDF

advertisement

Table of contents