MIR AI Camera Getting Started

MiR AI Camera enhances the workflow of MiR robots by identifying specific objects and triggering predefined actions. It can initiate missions or lock restricted areas, preventing congestion and optimizing robot movements. Ideal for well-lit environments with stable lighting conditions, it excels in monitoring localized areas and narrow doorways.

PDF

Document

Getting Started (en)

Date: 08/2020

Revision: v.1.3

Copyright and disclaimer

All rights reserved. No parts of this manual may be reproduced in any form without the express written permission of Mobile Industrial Robots A/S (MiR). MiR makes no warranties, expressed or implied, in respect of this document or its contents. In addition, the contents of the document are subject to change without prior notice. Every precaution has been taken in the preparation of this manual. Nevertheless, MiR assumes no responsibility for errors or omissions or any damages resulting from the use of the information contained.

Contact the manufacturer:

Mobile Industrial Robots A/S

Emil Neckelmanns Vej 15F

DK-5220 Odense SØ www.mobile-industrial-robots.com

Phone: +45 20 377 577

Email: [email protected]

CVR: 35251235

1. About this document

1.1 Version history

1.2 Where to find more information

2. Product presentation

2.1 External parts

2.2 Nameplate

2.3 MiR Fleet and operating requirements

2.4 Light indicator

3. Getting started

3.1 In the box

3.2 Suggested use

3.3 Mounting and powering up MiR AI Camera

3.4 Connecting to MiR AI Camera

3.5 Adding cameras to MiR Fleet

4. Commissioning

4.1 Collection phase

4.2 Pre-processing phase

4.3 Validation phase

4.4 Training phase

4.5 Upload the data model

4.6 Setting up triggers and actions

4. Troubleshooting

1. About this document

This document describes how to install and use your MiR AI Camera. MiR AI Camera is a

MiR accessory that can be implemented into your work environment to improve the flow of

MiR robots connected to MiR Fleet.

NOTICE

Save this manual. It contains important safety and operating instructions.

1.1 Version history

This table shows current and previous versions of this document and their interrelations with hardware releases.

Revision Release date

1.0

2019-11-04

1.1

1.2

1.3

2020-03-09

2020-03-20

2020-07-22

Description

First edition.

Updates for interface improvements.

Added section: Identification label.

Updated for interface improvements and renamed the section Identification label to

Nameplate.

1.0 Beta

1.0

1.2 Where to find more information

At the MiR website , you can find the following resources under the Manuals tab on each product page:

• Quick starts describe how you start operating MiR robots quickly. This document is in print in the box with the robots. Quick starts are available in multiple languages.

• User guides provide all the information you need to operate and maintain MiR robots.

User guides are available in multiple languages.

• Risk Analysis Guides include guidelines on how to create a risk assessment of your robot solution.

1. About this document

• Commissioning guides describe how to commission your robot safely and prepare it to operate in the workplace.

• Operating guides describe how to set up and use top modules and accessories, such as charging stations, hooks, shelf lifts, and pallet lifts.

• Getting started guides describe how to set up products that are mainly software-based, such as MiR Fleet.

• Reference guides contain descriptions of all the elements of the robot interface and MiR

Fleet interface. Reference guides are available in multiple languages.

• REST API references for MiR robots, MiR hooks, and MiR Fleet.

• The MiR network and WiFi guide specifies the performance requirements of your network and how you must configure it for MiR robots and MiR Fleet to operate successfully.

2. Product presentation

MiR AI Camera is intended to be used to improve the workflow of MiR robots. It provides

MiR Fleet with additional information that it can use to guide MiR robots more effectively, for example by initiating missions when carts are ready to be transported or by avoiding sending robots through obstructed, narrow doorways and into crowded areas.

In MiR Fleet, the camera is set up to trigger a specific action when the camera detects a certain object. The actions can either be the initialization of a fleet mission or locking a

Limit-robots zone.

Figure 2.1. When the AI camera detects a trigger object, it sends a signal to MiR Fleet. MiR Fleet then either locks a Limit-robots zone or schedules a mission.

2. Product presentation

2.1 External parts

This section presents the parts of MiR AI Camera that are visible on the outside.

Figure 2.2. External parts of MiR AI Camera.

Pos.

Camera lens

Reset button

Nameplate

Description

Ethernet port

WiFi antenna

Pos.

Description

Indicator light

Ventilation

Mounting bolts

Power socket

To reset the camera, you must hold down the Reset button for ten seconds.

2. Product presentation

2.2 Nameplate

MiR AI Camera is delivered with a nameplate mounted to it on the back. The nameplate of

MiR AI Camera identifies the model and serial number and includes the CE mark and the address of Mobile Industrial Robots.

Figure 2.3. Example of the MiR AI Camera nameplate.

2.3 MiR Fleet and operating requirements

MiR AI Camera is a product that must be connected to MiR Fleet to be used as intended. The following list describes the requirements for being able to use MiR AI Camera with your MiR

Fleet:

Some of these points are only applicable if you choose to use a WiFi connection to the camera instead of Ethernet.

• You must be using a MiR Fleet Server Solution. AI cameras are not supported on MiR Fleet

PC.

• Your MiR Fleet must be configured to use a host network interface. This is the default configuration for MiR Fleet.

• Ensure that port 1908 in your network is open and free for the AI camera to use.

• MiR AI Camera should be in the same physical location as MiR Fleet.

• A minimum bandwidth of 1 Mbit per AI camera with a maximum latency of 150 mS.

• A signal strength no less than -75 dBm. The signal-to-noise ratio should be at least 20 dB.

• MiR Fleet must be updated to software version 2.10.0 or higher for the AI camera settings to be available.

• MiR AI Camera performs best in well-lit areas.

2. Product presentation

2.4 Light indicator

The light indicator on MiR AI Camera is used to provide feedback on its current status. The following table describes the status linked to the various colors.

Color

White

Yellow

Green

Red

Cyan

Dark blue

Status

Busy. The camera is updating or starting up.

WiFi setup mode active. The camera is waiting for network connection.

Connected to network but no data model has been uploaded to the camera. The light blinks when changing to this state.

Error, such as connection errors. The light blinks when the camera itself requires attention and is static when the error is handled in the

MiR Fleet interface.

Object detection mode active.

Collection mode active.

3. Getting started

This section describes how to get started with MiR AI Camera.

To be able to use MiR AI Camera, your MiR Fleet Server Solution must be running software version 2.10.0.

3.1 In the box

This section describes the content of the MiR AI Camera box.

The box contains:

• MiR AI Camera

• A wall mount

• A WiFi antenna

• A Triad plug in adapter

• A folder with the following documents:

• The CE Declaration of Conformity for your product

• Passwords

• A USB drive with the following documents:

• MiR AI Camera Getting Started

• MiR Fleet Server Solution Getting Started Guide

• MiR Fleet Reference Guide

3. Getting started

3.2 Suggested use

It is important before installing MiR AI Camera to consider what you aim to use it for. This section describes some of the possible uses of MiR AI Camera, and where it should be mounted in each case.

MiR AI Camera is intended to be used to monitor a delimited area. The area of interest must be within the field of view and line of sight of the camera. It is recommended to focus only on a localized area. It is not recommended to install a camera that views the entire work area where people or machinery are performing multiple tasks and are in constant motion.

Make sure to mount your camera in a well-lit area with stable lighting conditions. Changing lighting conditions may affect the performance of MiR AI Camera negatively.

MiR AI Camera improves the workflow of MiR robots by triggering an action when it detects a specific object. An action can either be the initialization of a fleet mission or locking a

Limit-robots zone, so no MiR robots may enter. It is important to consider which action should be linked to which detected object, and to think about whether the mounted position of the camera is ideal for this. The following examples may provide insight into what MiR

AI Camera should be used for.

Preventing blockages in narrow doorways

Two MiR AI Cameras can be mounted on each side of a narrow doorway to monitor incoming traffic. If a camera detects an incoming object or person, MiR Fleet is alerted and temporarily locks the area in the narrow doorway. This prevents MiR robots from obstructing the doorway for incoming traffic and avoids that MiR robots attempt to pass an obstructed doorway. Instead, MiR robots wait at a reasonable distance to allow the other object or person through first.

If a MiR robot is already in the narrow doorway, MiR AI Camera will not make the robot stop in the middle of the doorway when it locks the doorway area.

3. Getting started

MiR AI Camera can also be configured only to lock the doorway area after detecting certain objects. For example, if the doorway is large enough to allow a person and robot through at the same time, it should be considered only to set MiR AI Camera to lock the area if carts or other robots are approaching the doorway.

Autonomously initiating robot missions

MiR AI Camera can be mounted to focus on an area where the presence of a specific object triggers a fleet mission. For example, an AI camera can be set up at a pick-up position, so when it detects an object at the position, a MiR robot is sent to pick up and transport the object to a drop-off point. It is important in this case that the pick-up position occupies most of the camera's field of view. This is to avoid that the camera detects a target object in other positions and then initiates a mission even though the object is not necessarily placed on the pick-up position.

3. Getting started

Figure 3.1. In this example, MiR AI Camera is focusing on a loading rack. In the first image, the camera detects nothing in its field of view. In the second image, it has detected a crate and initiates a mission to send a robot over to pick up the crate.

This example can also be applied to carts, shelves, and pallets where the camera is focused on a cart position, shelf position, or a pallet rack. It is also possible to use other types of missions as long as the mission trigger is an object that MiR AI Camera can identify easily in a designated area.

Foreseeable misuse

The following list includes examples where MiR AI Camera is not intended to be used:

• Monitoring heavily-trafficked areas.

• Functioning as a surveillance camera.

• Identifying specific persons.

• Controlling a MiR robot or other automated mobile robots remotely.

• Using in medical and life critical situations.

• Using in potentially explosive environments.

• Using in hygiene zones.

3. Getting started

3.3 Mounting and powering up MiR AI Camera

It is recommended to read through

Suggested use on page 11

before mounting

MiR AI Camera.

Figure 3.2

identifies the key parts of the camera mount. There are two mounting plates: one plate is mounted to the camera, and the other plate you must mount to the surface you want the cameras to be fixed to. It is possible to adjust their orientations using the ball and socket joint identified in

Figure 3.2

Figure 3.2. Parts of the AI camera mount.

Pos.

Description

Camera mounting holes

Joint tightening knob

Pos.

Description

Wall mounting holes

Ball and socket joint

3. Getting started

To mount MiR AI Camera follow these steps:

Place the camera mount in the position you want the MiR AI Camera to be mounted.

Make sure the mounting plate is placed flat against the surface.

The camera mount is adjustable, so you can always adjust the orientation of the camera after fixing the camera mount.

Mark the position of the four mounting holes on the surface you want to mount the camera to. Remove the camera mount.

Drill screw holes into the surface at the marked positions. Consider using wall plugs for a more secure mounting.

Place the camera mount back in the desired position, so the mounting holes align with the drill holes in the wall. Insert screws to fix the camera mount to the wall.

Remove the two bolt caps and washers from the mounting bolts on the camera, and fit the camera bolts through the mounting holes.

Place the washers and bolt caps back on the bolts, and tighten the two caps to ensure that the camera is fastened securely.

3. Getting started

Loosen the joint tightening knob, and adjust the angle of the camera so it is oriented approximately in the correct direction.

Connecting to MiR AI Camera below , it is described how you access a

live-stream video to position the camera accurately.

Tighten the joint knob when you are satisfied with the position.

Plug in the power cable to turn on the AI camera. It may take up to a minute for the camera to turn on. The indicator light will light up in white when the camera is starting up.

10.

Once the light turns yellow, the camera is ready to be connected to a network. There are two options for connecting MiR AI Camera to your MiR Fleet network: a.

Connect the camera to a nearby Ethernet port that is connected to the same network as MiR Fleet.

Screw in the WiFi antenna, and continue to the next section for instructions on connecting the camera to the network.

3.4 Connecting to MiR AI Camera

It is possible to connect to the MiR AI Camera directly, using an Ethernet cable or wirelessly through WiFi.

If you want to see a live-stream video from the camera to ensure that it is positioned correctly, you must first connect to the robot's WiFi. After step 4, you can then choose to connect it to MiR Fleet through Ethernet.

Connecting through WiFi

If your camera is mounted in a position that does not enable you to connect it to the network through Ethernet, the following steps describe how to connect through WiFi instead.

Turn on your MiR AI Camera. If the WiFi antenna is mounted, MiR AI Camera creates a

WiFi hotspot, and the indicator light changes to yellow.

3. Getting started

With your device, connect to the camera's WiFi. It is named in the format: MiR_AXXXX , where XXXX is the last four digits in the camera's serial number. For the WiFi password, see the MiR passwords document included with your camera.

If you cannot connect to MiR AI Camera, try to reset the camera by holding down the Reset button for ten seconds.

Once connected to the WiFi, open a browser and go to www.mir.com

3. Getting started

Select Stream to view a live-stream video from your camera. Use this to adjust the position of the camera to make sure it is correctly oriented towards the area of interest.

If you just wanted to see the live-stream from the camera, you can now connect the camera via Ethernet—see

Connecting through Ethernet on page 20 .

3. Getting started

Select Connection , select WiFi , and then select the network that your MiR Fleet is connected to. Enter the WiFi password and username (if required) to connect the camera to the network.

If you want to set a static IP for your camera, select Use static IP .

When the camera is connected to the same network as MiR Fleet, the indicator light changes to green. The AI camera hotspot vanishes, and the camera is now available in the MiR Fleet interface.

Check that MiR Fleet is connected to the camera by signing into MiR Fleet and going to

Setup > AI cameras . Your AI camera should appear under New cameras . It can take a little time for MiR Fleet to detect the camera.

If your camera does not appear, select Scan for camera , and enter the

IP address of the camera.

3. Getting started

Connecting through Ethernet

Connect MiR AI Camera to the same network as your MiR Fleet using an Ethernet cable.

Check that MiR Fleet is connected to the camera by signing into MiR Fleet and going to

Setup > AI cameras . Your AI camera should appear under New cameras . It can take a little time for MiR Fleet to detect the camera.

If your camera does not appear, select Scan for camera , and enter the

IP address of the camera.

3. Getting started

3.5 Adding cameras to MiR Fleet

The process of adding cameras to MiR Fleet is similar to adding robots to MiR Fleet. The following steps describe how to add a new MiR AI Camera to the fleet.

Make sure your MiR Fleet is updated to a software version 2.10.0 and is using a host network interface configuration. See

MiR Fleet and operating requirements on page 8

Sign in to MiR Fleet, and go to Setup > AI cameras . All cameras connected to same network as MiR Fleet should be displayed.

If your new camera is not available, select Scan for camera , and enter the camera's

IP address. The camera should appear under New cameras .

To add the camera, select + Add .

3. Getting started

Edit the camera's name. It is recommended to name the cameras based on their position or function, such as Hallway or Cart pickup . Also, verify that the serial number and software version of MiR AI Camera are correct and that Disabled is set to No .

Select Save when you are done.

3. Getting started

You can also remove cameras from MiR Fleet after adding them. To do this, select Edit for the AI camera you want to remove, and then select X Remove .

NOTICE

When you remove an AI camera from MiR Fleet, the AI camera is automatically factory reset.

4. Commissioning

This section describes how to train MiR AI Camera to detect certain objects and setup which actions should be triggered when the camera detects specific objects. There are four main phases included in the training of MiR AI Camera:

Collection phase

Sample images are collected autonomously by the AI camera. These are required for training the AI camera.

Pre-processing phase

Objects in the collected images are detected and framed by MiR AI Training Tool. Images containing similar objects are clustered into groups.

Validation phase

You must label the collected images based on the detected object in the image, and only select images with accurate framing.

Training phase

Based on the validated images, MiR AI Camera is trained to detect future observations of the labeled objects.

Figure 4.1. To create a data file, the AI cameras first collect data in the Collection phase. You then upload the data to MiR AI Training Tool to complete the Pre-processing phase. In the training tool, you must complete the

Validation and Training phases to create a model that you upload to MiR Fleet which then distributes it to the connected AI cameras.

4. Commissioning

4.1 Collection phase

For MiR AI Camera to detect specific objects, it must first be trained, using a collection of sample images of those objects. The objects you want the camera to detect are referred to as target objects. Each camera autonomously collects the necessary images over eight hours. The collection phase must happen during normal operating hours in the time span when the camera is intended to be used. Do not let the camera collect data after working hours or in a time span when it is not intended to be used.

It is important that you gather images of objects where they are alone in the camera's field of view. It is less efficient to train MiR AI Camera when there are multiple target objects in the images.

Figure 4.2

shows examples of both bad and good images to collect.

Figure 4.2. Examples of good and bad images to collect to use for training.

4. Commissioning

Image

Target object

Loaded pallet

MiR500 with pallet

Description

The target object is fully visible, and there are only a few other moving objects.

The target object is fully visible and isolated.

There are many moving objects, making it difficult to isolate a single target object.

There are many moving objects, and the target object is partially covered by other objects, making it difficult to isolate the target object.

During the Collection phase, the camera only records images whenever it detects motion. It is important during the collection phase that the desired target objects enter the camera's field of view.

If you have several AI cameras, all cameras connected to MiR Fleet record images during the Collection phase. Make sure you have added all your MiR

AI Cameras to MiR Fleet before you begin collecting data.

Follow the steps below to begin the Collection phase, and download the data necessary to begin the next phase:

Go to Setup > AI camera and select AI camera settings .

4. Commissioning

Under Collect image data , select Start collecting . All of the cameras connected to the fleet will now begin collecting images.

The remaining time of the collection phase is displayed. When the displayed time has passed, return to the MiR Fleet interface and navigate to AI camera settings again to download the data.

4. Commissioning

Download the file containing all the images by selecting Download data . Make sure to use a computer that can be connected to the internet, as you will need to upload the data to an online training server in the next step to begin training.

4.2 Pre-processing phase

The data preparation and training take place on the following website: https://training.mirrobots-ai.com

. This site is known as MiR AI Training Tool. The following steps describe how to upload the training data collected in the Collection phase:

Open your preferred web browser and go to https://training.mir-robots-ai.com

to access the MiR AI Training Tool. If you have not used MiR AI Training Tool before, select

4. Commissioning

Insert your email and create a password. Select Register when you are done, and sign in with your new user.

4. Commissioning

To start a new session, you will need to add credits. One credit is used each time you train a new data model. Credits are not used when you create a new training session, only when you have finished validation and begin to train a model from the session.

Credits are added to your account by inserting credit codes. You should have received a code for this from your distributor. If you have not received a code, contact your distributor.

Select Click here to add more credits .

Type in the code that came with your MiR AI Camera. You will receive the number of credits that are linked to that code.

To upload a new set of collected data, select Create new session .

4. Commissioning

Enter a name for your session. By default, the session is named the current date. Select

Choose file and browse for the data file you saved at the end of the Collection phase, containing all the collected images. Select it, and upload it to the site.

4. Commissioning

The server will pre-process the images. This can take up to an hour, depending on the number of images collected. The pre-processing is handled autonomously, so you can leave the site and return later once the pre-processing is complete. You will receive an email notification when the pre-processing is done.

After you have uploaded the data, if Error is displayed in your training session, delete the data and begin a new Collection phase. Make sure your

MiR AI Camera is mounted in a well-lit area, and that objects of interest enter its field of view during the Collection phase.

4.3 Validation phase

In the Validation phase, the images must be labeled based on the target object in the images. The candidate target object is framed by a green box. To start the process, you must create a new label for a target object you would like MiR AI Camera to detect. After creating a new label, you validate which images include the same target object and should be used for training. This section presents an example of this process and provides guidelines for which images to validate for training.

4. Commissioning

CAUTION

This is the most important stage when setting up your MiR AI Camera. You must follow the guide accurately to make sure your MiR AI Camera performs as expected. The following section describes how to label the training images correctly. It is highly recommended that you read the following section before continuing to the step by step instructions.

Labeling

During the Pre-processing phase, MiR AI Training Tool detects candidate target objects, frames them in the images where they appear, and attempts to cluster images with the same target objects. In the Validation phase, two actions must be performed:

Creating labels

The target objects must be given an appropriate label to categorize them under an object type. The first time you select an image with a new target object, you must create a new label. Only one target object is found per image, thus only one label must be applied per image. As a general rule, if the appearances of target objects are the same, they should be grouped under the same label. If two target objects similar in appearance are given two separate labels, it is not guaranteed that MiR AI Camera can distinguish the difference. For this reason, when creating a new label, it is important to choose a label name that correctly encapsulates the target object type that you are categorizing.

Table 4.1

provides an overview of the correct and incorrect usage of labels for some example target objects. You can train your camera to detect other target objects than those listed in the table.

4. Commissioning

Target objects

People

Correct labeling Incorrect labeling

Robots and vehicles

Carts and shelves

All people go under one collective label.

Example: person

All robots or vehicles of the same type go under a product-specific label.

Example: robot cart

MiR100 forklift

, MiR200 , AGV

All carts or shelves of similar size and appearance can be categorized under one label.

Example: medicine shelf , blue

Creating individual labels for certain people.

Example: Sarah , John , workers , intruders , guests

All robots go under the same label, or each robot receives its own label.

Example: Robots , Sarah's forklift , MiR100 1 , MiR100 2

All cart and shelves go under the same label, or each cart or shelf receives its own label.

Example: medicine shelf 1 , medicine shelf 2 , shelves and carts

Table 4.1. Overview of correct and incorrect use of labels.

Validating

Once you have labeled a target object, MiR AI Training Tool presents other detected target objects where similarities have been detected. You must validate which detected target objects should receive the same label. It is important that only images where the green box correctly frames the target object are validated for training.

Figure 4.3

illustrates examples of valid and invalid framing of target objects.

4. Commissioning

Figure 4.3. Examples of valid framing have a green border (examples 1 and 2), and examples of invalid framing have a red border (examples 3-6).

Image Validity

Valid

Invalid

Description

The frame fits tightly around the target object, and very little background is included in the frame.

The frame almost fits around the target object. Only small sections of the object are outside the frame, and little of the background is included.

The target object is not in the frame.

The target object is inside the frame, but too much background is also included.

Two different objects are in the frame. Only one target object should be in a frame.

The frame includes only very little of the target object.

6 Invalid

Table 4.2. Descriptions of the framing examples in

Figure 4.3

4. Commissioning

It is highly recommended only to include images with high-quality framing, even if it means that fewer images are accepted.

It is not necessary to complete the validation phase in one sitting. All labeling and validation is saved as you work, and you can return to MiR AI Training

Tool at any time to complete it.

Validation phase step by step instructions

The following steps describe an example of how to begin the Validation phase. While going through these steps, keep in mind that the images used in the example will not be the same images your MiR AI Training Tool displays. Follow the guidelines above to determine whether an image is valid for labeling.

In MiR AI Training Tool, select Start labeling for the session you would like to begin labeling.

4. Commissioning

The labeling session opens and displays several images where candidate target objects are framed. Browse through the images until you find one of your target objects that have been framed according to the guidelines described in

Validating on page 34 . Once

you have found a suitably framed target object, select the image.

Each image displayed in MiR AI Training Tool represents a cluster of images that are very similar. You can see how many images are included in each cluster by hovering over the image.

4. Commissioning

A dialog box with three images similar to the one you selected is displayed. Validate the images that are framing the same target object correctly. A validated image has a green checkmark to indicate it is selected. To reject an image, hover over the green checkmark to turn it into a red cross, and select the cross.

Type a suitable label for the target object. In this case, we use the label mir200_roeq .

Select Save when you have finished.

You have now validated a cluster of images.

4. Commissioning

Select a new target object image to begin the same process.

You can remove any images that are incorrectly labeled by selecting the red X that is displayed when you hover over an image.

We recommended you continue labeling and validating images until you have gone through all of the images that are framed correctly. Often, more images result in a better performance from the camera, but this also depends strongly on the quality of the labeled images. It is highly recommended only to include images with high-quality framing, even if it means that fewer images are accepted. If do not have enough highquality images in one session, you can merge a session with other sessions—see

Merging sessions on the next page

4. Commissioning

Merging sessions

If you want to train a single model based on images taken from multiple sessions, it is possible to merge the sessions together. You can merge any number of sessions together.

When you merge sessions together:

• All labels that have already been created in other sessions are included in the merge.

• All images that are labeled keep their labels after merging. If the sessions share images with the same labels, these are merged under the same label.

• Any data models trained using the merged sessions are deleted from the training tool. It is a good idea to download a backup of your date models before merging.

The following steps describe how to merge sessions:

In the selected session, select Manage data . All of the other sessions you have started are displayed.

Select Merge for the sessions you want to merge with the current session.

4. Commissioning

After merging a session, a success message is displayed and the session is listed with the currently merged sessions.

To revert a merge, select Revert to separate the sessions again. If you have labeled any images in the merged session, the labels are still applied to the images after reverting.

4. Commissioning

4.4 Training phase

The Training phase can begin once you have labeled your images. You can begin the training without labeling all images, but MiR recommends that you label as many images as possible to improve the performance of the MiR AI Camera.

To begin the Training phase, enter the training session where you have finished labeling, and select Start training .

NOTICE

Only select Start training when you have finished labeling. You use one of your credits for each training session you begin.

The Training phase does not require any more interaction. MiR AI Training Tool will automatically complete the training for the MiR AI Camera. This can take several hours to complete, but it is not necessary for MiR AI Training Tool to be open on your PC throughout the training. Once the training is complete, you will receive an e-mail and can continue to set up your MiR AI Camera through MiR Fleet.

4. Commissioning

4.5 Upload the data model

The training phase outputs a data model. This model contains the information required for

MiR AI Camera to detect the objects labeled in the validation phase. The following steps describe how to upload the file to the connected MiR AI Cameras:

Select Download mode l for the session you have created for your MiR AI Cameras.

The data model is usually approximately 20-30 MB.

4. Commissioning

Connect your PC to the MiR Fleet network, and sign in to the interface. Navigate to Setup

> AI cameras , and select AI camera settings .

Select Upload new data model and then the data model you just downloaded from MiR

AI Training Tool.

It may take a while for the data model to be uploaded and processed. Once the model is uploaded, the indicator light on the camera turns cyan and the status is displayed as Active .

The camera is now ready to detect objects. The next step is to create triggers, so certain actions are triggered when the cameras detect specific target objects.

4. Commissioning

4.6 Setting up triggers and actions

You must set up each MiR AI Camera to trigger certain actions when they detect specific trigger objects. There are two types of actions you can trigger: Lock zone or Fleet mission.

Lock zone actions are used to stop MiR robots from entering a Limit-robots zone, and Fleet mission actions initiate a fleet mission.

If you want to create a zone that MiR AI Camera can lock, but does not otherwise limit the number of robots that can be within the zone, make sure to increase the Robot limit to the number of robots you have in your fleet. You can change this value when editing a selected zone in the map editor.

4. Commissioning

The following steps describe how to set up a trigger:

Go to Setup > AI cameras , and select Edit for the AI camera you would like to set up triggers for.

Select + Add trigger to create a new trigger

Select Edit to add a target object that will trigger an action.

4. Commissioning

Select the object or objects you would like to trigger an action when the camera detects them. The available objects match the labels created in MiR AI Training Tool. In this case, new_cart_loaded is the selected target object that will trigger an action when the camera detects it.

If one target object is in the frame and has triggered an action, another target object entering the frame at the same time will not trigger the action again.

4. Commissioning

Select the type of action you would like to be triggered when the camera detects the selected object in the previous step. You can choose between two types of actions: a.

Fleet mission enables you to choose a mission within MiR Fleet that should be initialized as soon as possible when the camera detects the linked object. It can take a few seconds for new missions to be accessible in the AI camera settings.

Lock zone enables you to choose a Limit-robots zone and prevent it from being assigned to a robot, so no MiR robots may enter if the camera detects the target object. If a robot already assigned to the zone, it will finish driving through the zone even if MiR Fleet locks the zone. A zone can be assigned to a robot if it is within 10 meters of the edge of the zone.

4. Commissioning

Under Trigger action , select the Limit-robots zone or fleet mission you would like to link to the chosen object. In the example image below, the trigger initiates the PickupCart mission when the camera detects a loaded cart.

Another example, using a Lock zone action, sets the camera to lock a zone in a narrow hallway when the camera detects a person.

You can create as many triggers as you need. Once you have finished creating triggers, save your changes by selecting Save . The camera is now ready to trigger actions when it detects specific target objects.

4. Troubleshooting

This section describes possible reasons why your MiR AI Camera may not be detecting objects reliably. With each reason, there is a workaround or solution that you should apply to avoid or solve the issue.

• Not enough training data

During the Collection phase, if the target objects did not enter the camera's field of view, you may not have collected enough images for training. If the Validation phase did not include many images of the target objects, you should merge a new collection of data where you make sure that the objects enter the camera's field of view more frequently.

To merge two sessions together, see

Validation phase on page 32

If you suspect that you have discarded too many images incorrectly in MiR AI Training

Tool, you can review all of the discarded images by selecting Discarded images in the menu bar to the right.

4. Troubleshooting

• Change in lighting after the Collection phase

If the lighting has changed significantly since the Collection phase, the camera may not be able to recognize the objects. Run the Collection phase again, while the lighting conditions are the same as when the AI camera is operating.

If the lighting conditions around your MiR AI Camera change throughout the day, make sure to run the Collection phase in all the different lighting conditions.

• Incorrect labeling

If you have labeled any images incorrectly, the data model may be corrupted. Repeat the

Validation phase or review all the labeled images in the training server to ensure that all images have been labeled correctly. You can review all the images associated with a label by selecting the label in the menu bar to the right in MiR AI Training Tool.

4. Troubleshooting

• Inclusion of too many invalid image samples

If invalid images are used in the Training phase, the data model may be corrupted.

Repeat the Validation phase or review all the labeled images in the training server to ensure that the validated images follow the guidelines for valid sample images. You can review all the images associated with a label by selecting the label in the menu bar to the right in MiR AI Training Tool.

• Objects have changed appearance

If the objects you have trained your MiR AI Camera to detect have changed appearance significantly, MiR AI Camera may no longer be able to recognize them. Run the Collection phase again and make sure the new objects are included.

• MiR AI Camera has been repositioned without retraining

If you change the position of your camera so its field of view is occupied by an area that is significantly different from the areas included in the images collected in the Collection phase, the MiR AI Camera may not be able to detect target objects correctly in that area.

Run the Collection phase again where images of the new area are included.

If you are experiencing another issue, or you were not able to resolve the issue yourself, go to Setup > AI cameras > AI camera settings , and select Download error logs . This will download all the registered activity in your AI cameras into a collection of error logs. Send the error logs to your distributor for further help troubleshooting.

Hello! I am an AI chatbot specifically trained to assist you with the MIR AI Camera Getting Started. I have thoroughly reviewed the document and can help you locate the exact information you need or explain the content in clear and simple terms. Whether you’re looking for guidance on specific features, troubleshooting steps, or general usage, feel free to ask your questions. The more details you provide about your concerns or needs, the more accurately and effectively I can assist you!