vCloud Usage Meter User Guide uction

vCloud Usage Meter User Guide uction
vCloud Usage Meter
User Guide
Version 3.0
uction
Version 2.0.1
October 2011
vCloud Usage Meter
User Guide
© 2012 VMware, Inc. All rights reserved. This product is protected by U.S. and international
copyright and intellectual property laws. This product is covered by one or more patents listed at
http://www.vmware.com/download/patents.html.
VMware is a registered trademark or trademark of VMware, Inc. in the United States and/or other
jurisdictions. All other marks and names mentioned herein may be trademarks of their respective
companies.
VMware, Inc
3401 Hillview Ave
Palo Alto, CA 94304
www.vmware.com
© 2012 VMware, Inc. All rights reserved.
Page 2 of 39
vCloud Usage Meter
User Guide
Contents
1.
About the VMware vCloud Usage Meter ........................................... 5
1.1 Overview ........................................................................................................................ 5
1.2 Related Documentation ................................................................................................. 5
1.3 Requirements ................................................................................................................ 6
2.
Installation ........................................................................................ 8
2.1 Deploying through vCenter Server ................................................................................ 8
2.2 Deploying Through vSphere Host ................................................................................. 8
2.3 Configuring Virtual Appliance Details ............................................................................ 9
3.
Restarting Tomcat .......................................................................... 10
4.
Upgrading the vCloud Usage Meter................................................ 11
5.
Using the vCloud Usage Meter User Interface ................................ 12
5.1 Log into the vCloud Usage Meter User Interface ........................................................ 12
5.2 Main Menu of the User Interface ................................................................................. 12
5.3 Notifications Area ........................................................................................................ 13
6.
Managing the vCloud Usage Meter ................................................ 15
6.1 Managing Contact Details ........................................................................................... 15
6.2 Managing Email Details ............................................................................................... 15
6.3 Managing LDAP Authentication .................................................................................. 16
6.4 Managing Products...................................................................................................... 16
6.5 API Access Tokens ..................................................................................................... 19
7.
Managing Licenses......................................................................... 20
7.1 Managing License Sets ............................................................................................... 20
7.2 Managing Billing Categories ........................................................................................ 20
8.
Setting Up Automatic Reporting ..................................................... 21
9.
Monitoring ...................................................................................... 22
10.
Managing Customers ................................................................. 23
10.1 Adding a Customer Through the User Interface ......................................................... 23
10.2 Deleting Customers ..................................................................................................... 24
10.3 Importing/Exporting Customers and Rules.................................................................. 24
11.
Managing Rules ......................................................................... 26
© 2012 VMware, Inc. All rights reserved.
Page 3 of 39
vCloud Usage Meter
User Guide
11.1 Adding a Rule .............................................................................................................. 26
11.2 Deleting Customer Rules............................................................................................. 28
12.
Generating Reports .................................................................... 30
12.1 Detailed Billing Report ................................................................................................. 30
12.2 Detailed Usage Report ................................................................................................ 31
12.3 Customer Summary Report ......................................................................................... 32
12.4 License Summary Report ............................................................................................ 32
12.5 Product Report ............................................................................................................ 33
12.6 Customer Product Report ............................................................................................ 33
13.
Support ...................................................................................... 35
13.1 Runtime Information .................................................................................................... 35
13.2 Database ..................................................................................................................... 35
13.3 Logging Level .............................................................................................................. 35
13.4 Log Rolling ................................................................................................................... 35
13.5 Support Bundle ............................................................................................................ 36
14.
Using the vCloud Usage Meter's JMX Service ........................... 37
15.
Advanced Management.............................................................. 38
15.1 Configuring Memory for Environment Size.................................................................. 38
15.2 Turning On Parallel Collections ................................................................................... 38
15.3 Monitoring the Appliance ............................................................................................. 39
15.4 Managing the vCloud Usage Meter Database ............................................................ 39
© 2012 VMware, Inc. All rights reserved.
Page 4 of 39
vCloud Usage Meter
User Guide
1. About the VMware vCloud Usage Meter
1.1
Overview
The VMware® vCloud Usage Meter is a tool that monitors, measures, and reports on VMware
product consumption at a Service Provider. The vCloud Usage Meter is provided as a virtual
appliance.
The vCloud Usage Meter serves two functions: metering and reporting. The metering part of the
tool is used to collect raw usage data on VMware Service Provider Program (VSPP) bundles; the
reporting part of the tool is used to generate reports for planning, billing, general account
management, dispute resolution, and other purposes.
Information collected on an hourly basis includes:
Time at which data is collected
Host DNS name
Host RAM (physical memory)
vSphere license type
Virtual machine vCenter name
Virtual machine hostname
vRAM (allocated virtual memory)
Billing vRAM (calculation based on reserved virtual memory and per-virtual-machine memory
cap)
Virtual machine CPU (count of virtual CPUs)
Virtual machine Instance UUID (universal unique identifier)
Virtual machine location in vSphere Inventory
Data collected is stored in the PostgreSQL database of the virtual appliance itself.
1.2
Related Documentation
Table 1. Related Documentation
Subject Area
Found At
vCloud Usage Meter Release Notes
vCloud Usage Meter API Guide
VMware Community, via title search
VSPP Program Guide
VSPP Product Guide
http://www.vmware.com/partners/partners.html
Support for Service Providers and Aggregators
with vSphere or vCloud entitlements
https://vmware.com/support/contacts/file-sr.html
Customers with no entitlements
http://communities.vmware.com/community/vmtn/vcd/vcloud_
usage_meter
®
VMware vSphere documentation
http://www.vmware.com/support/pubs
© 2012 VMware, Inc. All rights reserved.
Page 5 of 39
vCloud Usage Meter
User Guide
1.3
Requirements
Before you install the vCloud Usage Meter, ensure that the minimum software and hardware
requirements are met.
Table 2. Software Requirements
Requirement
Description
®
VMware vSphere
(if deployed through vSphere Host)
VMware ESXi 4.0
VMware ESXi 4.0 Update 1
VMware ESXi 4.0 Update 2
VMware ESXi 4.0 Update 3
VMware ESXi 4.0 Update 4
VMware ESXi 4.1
VMware ESXi 4.1 Update 1
VMware ESXi 4.1 Update 2
VMware ESXi 5.0
VMware ESXi 5.0 Update 1
VMware ESXi 5.1
VMware vCenter Server™ Versions
(if deployed through vCenter Server)
VMware vCenter Server 4.0
VMware vCenter Server 4.0 Update 1
VMware vCenter Server 4.0 Update 2
VMware vCenter Server 4.0 Update 3
VMware vCenter Server 4.0 Update 4
VMware vCenter Server 4.1
VMware vCenter Server 4.1 Update 1
VMware vCenter Server 4.1 Update 2
VMware vCenter Server 4.1 Update 3
VMware vCenter Server 5.0
VMware vCenter Server 5.0 Update 1
VMware vCenter Server 5.1
VMware vCenter Operations Manager
Enterprise™ (optional)
VMware vCenter Operations Manager Enterprise
5.0.3
VMware Site Recovery Manager™ (optional)
VMware Site Recovery Manager 5.0
VMware vCloud Integration Manager (optional)
VMware vCloud Integration Manager 1.0
© 2012 VMware, Inc. All rights reserved.
Page 6 of 39
vCloud Usage Meter
User Guide
Requirement
Description
Browser
Internet Explorer 8 and later
Firefox 7 and later
Google Chrome 12 and later
Table 3. System Requirements
Requirement
Description
Hardware Requirements
2.0GHz or faster Intel or AMD x86 processor
3.6 GB or more RAM
40GB or more disk storage
10/100 Ethernet adapter (1 Gigabit recommended)
Configuration Maximums
Each vCloud Usage Meter virtual appliance supports metering of:
10 vCenter Servers
10,000 virtual machines
1000 Customers
2500 vCenter Inventory objects
Note: When working with large datasets of virtual machines and vCenter
Server inventory objects, greater than 10,000 virtual machines and 2,500
vCenter Server inventory objects, VMware recommends having at least
750MB of RAM available on the client for efficient operation.
TCP Ports
vCloud Usage Meter is accessed using predetermined TCP ports. If you
manage network components from outside a firewall, you may be required to
reconfigure the firewall to allow access on the appropriate ports.
Port
Source
Target
Purpose
80
vCloud Usage
Meter
vCenter Server
vSphere API
443
vCloud Usage
Meter
vCenter Server
vSphere API
5480
vCenter Update
Manager
vCloud Usage
Meter
Used for virtual
appliance updates
8443
Client Browser
vCloud Usage
Meter
Used for the web
application
© 2012 VMware, Inc. All rights reserved.
Page 7 of 39
vCloud Usage Meter
User Guide
2. Installation
2.1
Deploying through vCenter Server
1. Download the vCloud Usage Meter OVA installation file and save it to your local machine:
http://www.vmware.com/download/download.do?downloadGroup=UMSV3
2. From the File menu in the vSphere client, select Deploy OVF Template. The installation
wizard is displayed.
3. In the Deploy OVF Template screen, specify the location of the saved OVA file and click
Next.
4. Verify the OVF Template Details and click Next.
5. Accept the VMware End User License Agreement and click Next.
6. Specify a Name and Inventory Location for the vCloud Usage Meter virtual appliance and
click Next.
7. Select the resource pool within which you want to deploy the template and click Next.
8. Select the Datastore in which you want to store the virtual machine files and click Next.
9. Select the Disk Format in which you want to store the virtual disks and click Next.
10. Select a network for the deployed template and click Next.
11. Select the IP Address Allocation policy to allocate IP addresses and click Next.
12. Review the information in the Ready to Complete screen and click Finish. The installer starts
deploying the template.
13. Power on the virtual machine and perform the steps in Configuring Virtual Appliance Details.
2.2
Deploying Through vSphere Host
1. Download the vCloud Usage Meter OVA installation file and save it to your local machine:
http://www.vmware.com/download/download.do?downloadGroup=UMSV3
2. Ensure that you have vSphere Client installed.
3. From the File menu in the vSphere client, select Deploy OVF Template. The installation
wizard is displayed.
4. In the Deploy OVF Template screen, specify the location of the saved OVA file and click
Next.
5. Verify the OVF Template Details and click Next.
6. Accept the VMware End User License Agreement and click Next.
7. Specify a Name and Inventory Location for the vCloud Usage Meter virtual appliance and
click Next.
8. Select the resource pool within which you want to deploy the template and click Next.
9. Select the Datastore in which you want to store the virtual machine files and click Next.
10. Select the Disk Format in which you want to store the virtual disks and click Next.
11. Select a network for the deployed template and click Next.
12. Select the IP Address Allocation policy to allocate IP addresses and click Next.
© 2012 VMware, Inc. All rights reserved.
Page 8 of 39
vCloud Usage Meter
User Guide
13. Review the information in the Ready to Complete screen and click Finish. The installer starts
deploying the template.
14. Power on the virtual machine and perform the steps in Configuring Virtual Appliance Details.
2.3
Configuring Virtual Appliance Details
Perform these steps to complete the installation of your Usage Meter Virtual Appliance.
To change/create passwords
1. On the virtual machine console, navigate to Login and press Enter. The command line
interface prompts you to enter your credentials.
2. Log in as root with the default password vmware, and then press Enter.
3. To change your console password, at the #prompt, type passwd and press Enter.
4. Enter and confirm a new console password. The password is changed.
5. To create your web application password, at the #prompt, type webpass and press Enter.
6. Enter and confirm a new web application password. The password is created.
7. Type Exit to return to the virtual machine console.
To change time zone details
1. On the virtual machine console, navigate to Set Timezone and press Enter. By default, NTP
has not been configured for a virtual appliance.
2. Select your time zone and press Enter.
3. Restart the Tomcat service. For more information, see Restarting Tomcat.
To configure the network
The vCloud Usage Meter console displays a web application URL. However, network details are
not configured at OVF deployment. You must configure network parameters so that the console
displays the correct URL for running the web application.
1. Log in and run the following command:
/opt/vmware/share/vami/vami_config_net
2. Follow instructions on configuring new network settings. On completion of setting up the
network, interfaces will reset.
3. Return to the vCloud Usage Meter console and note the new web application URL address in
the format http://<IP-Address>:8443/um.
© 2012 VMware, Inc. All rights reserved.
Page 9 of 39
vCloud Usage Meter
User Guide
3. Restarting Tomcat
Tomcat needs to be restarted when:
Configuring or changing the time zone. See Configuring Virtual Appliance Details.
Resetting. Under some conditions, Tomcat may not find the hostname. When Tomcat is
started/stopped on the appliance multiple times, you may not get a response when you try
running the VCloud Usage Meter web application in your browser.
To stop and start Tomcat
Use the command
service tomcat restart
© 2012 VMware, Inc. All rights reserved.
Page 10 of 39
vCloud Usage Meter
User Guide
4. Upgrading the vCloud Usage Meter
You can upgrade to version 3.0 from version 2.3.x. Before upgrading, back up your vCloud Usage
Meter virtual machine.
To upgrade from version 2.3.x to 3.0
1. Transfer the patch file usage-meter-300-update.zip to /root in the virtual appliance.
2. Log in to the virtual appliance as root.
3. Use the command unzip /root/usage-meter-300-update.zip
4. Use the commands
cd usage-meter-300-update
./updateum.
5. The vCenter Server UUID will be blank until after the first collection, post upgrade, where all
the vCenter Server UUIDs are updated. (Optional)You can edit and save a vCenter Server
with a blank instance UUID before the first collection in order to set the UUID again.
6. If you have configured an outgoing email server and it requires a password, enter the
password again. vCloud Usage Meter 3.0 changes the way the password is encrypted in the
database and is unable to convert the pre-existing value.
© 2012 VMware, Inc. All rights reserved.
Page 11 of 39
vCloud Usage Meter
User Guide
5. Using the vCloud Usage Meter User Interface
5.1
Log into the vCloud Usage Meter User Interface
Open a browser and enter the web application URL address.
Log in to the vCloud Usage Meter User Interface with the web application password.
For more information on the web application URL address and password, see Configuring
Password, Time Zone, and Network.
5.2
Main Menu of the User Interface
Along the top right menu bar, the user interface displays the following menu choices.
Table 4. vCloud Usage Meter UI Menu
Menu
Description
Manage
Allows you to manage settings and/or credentials for:
Contact (Service Provider)
Email (Outgoing Email Server, Email Alerts)
LDAP (LDAP Authentication)
Products (vCenter Servers, vCenter Operations Manager Servers, Site
Recovery Manager Servers, vCloud Integration Manager, Collector)
API (API Access Tokens)
Note: This page is only visible to those with administrator privileges.
For more information, see the section Managing vCloud Usage Meter.
Licenses
Allows you to:
Create a label for a license set
Add licenses to a set
Assign licenses to a default category
For more information, see the section Managing Licenses.
Automatic Reporting
Allows you to set up automatic reporting.
For more information, see the section Setting Up Automatic Reporting.
Monitor
Allows you to select a month of a year to monitor.
For more information, see the section Monitoring.
© 2012 VMware, Inc. All rights reserved.
Page 12 of 39
vCloud Usage Meter
User Guide
Menu
Description
Customers
Allows you to:
View a customer list
Import customers and/or rules
Add customers singly
Edit customers
Delete customers
Export customers and/or rules
For more information, see the section Managing Customers.
Rules
Allows you to:
Create rules
View current rules
View mapped and unmapped virtual machines
For more information, see the section Managing Customer Rules.
Report
Allows you to:
Select type of report
Set a customer filter
Set a license set filter
Select a month of a year
Set a per-virtual machine memory cap
Note: This page is only visible if data is present for reporting.
For more information, see the section Generating Reports.
Support
Allows you to:
Select a logging level
View or save the virtual appliance log file
Generate support bundles
For more information, see the section Support.
Log Out
5.3
Logs you out of the vCloud Usage Meter.
Notifications Area
The vCloud Usage Meter’s user interface displays notifications about user activities and
background processes.
The notifications appear at the top left of each page.
© 2012 VMware, Inc. All rights reserved.
Page 13 of 39
vCloud Usage Meter
User Guide
Notifications persist from session to session, unless manually cleared, or if the web
application is restarted (which is infrequent).
A Clear button allows you to remove the current notifications.
To clear the Notifications list
1. Click the Clear button. The clear button operates with one click without asking for a
confirmation.
2. To view notifications as part of a log, view the virtual appliance log file. For more information,
see the section Support.
© 2012 VMware, Inc. All rights reserved.
Page 14 of 39
vCloud Usage Meter
User Guide
6. Managing the vCloud Usage Meter
The vCloud Usage Meter must be configured after installation to properly meter vCenter Servers
and report virtual machine usage.
6.1
Managing Contact Details
Service Provider details are required in order for the vCloud Usage Meter virtual appliance to
collect and report on virtual machine and customer usage. Failure to enter details will result in
failed vCenter collections and incomplete monthly reports.
To enter Service Provider details
1. Click Manage in the top right menu bar.
2. Click the Contact tab.
3. Enter a Company name.
4. Enter a Contact Name.
5. Enter a Phone number.
6. Enter an Email.
6.2
Managing Email Details
To enter email details
1. Click Manage in the top right menu bar.
2. Click the Email tab.
6.2.1 Outgoing Email Server
Configure the outgoing email server to receive automatic monthly billing reports and alerts of
failed and successful collections.
To enter Outgoing Email Server details
1. Enter the Host.
2. Enter Port (the default port is 25).
3. (Optional) If you require authentication, enter a User name and Password.
6.2.2 Email Alerts
You can set up an option for an email address to receive email alerts after successful or failed
collections.
To set up email alerts
1. Enter the From Email.
2. Enter the To Email. To enter more than one email address, separate email addresses with
commas (example: [email protected], [email protected]).
3. Select whether successful or failed collection emails will be sent.
© 2012 VMware, Inc. All rights reserved.
Page 15 of 39
vCloud Usage Meter
User Guide
4. In order to test email alerts when saving, select Send a test email alert after saving to verify
that the outgoing email server is working correctly and that the alerts will be sent. By default
this check box is not selected.
5. Click Save when completed.
6.3
Managing LDAP Authentication
vCloud Usage Meter can authenticate users against an LDAP service.
To enter LDAP Authentication details
1. Click Manage in the top right menu bar.
2. Click the LDAP Authentication tab.
3. Enter the LDAP Authentication details, such as Host, Password and Distinguished Names.
4. Click Save when completed.
6.4
Managing Products
To enter product details
5. Click Manage in the top right menu bar.
6. Click the Products tab.
6.4.1 vCenter Server
At least one vCenter Server must be added to collect virtual machine usage data.
The vCloud Usage Meter does not support directly metering vSphere hosts.
To enter vCenter Server details
1. Under vCenter Servers, click Add. The Add/Edit vCenter Server screen appears.
2. Enter Host Name or IP. You can insert multiple servers at the same time if they have the
same login credentials, by supplying multiple comma-separated names.
3. Enter User Name. Read-Only administrator privileges required.
4. Enter a Password.
5. Click Save. If there is an error, a separate message dialog will appear, and the server will not
be added to the list.
6.4.2 vCenter Operations Manager Server
If a vCenter Operations Manager Server is associated with a vCenter Server, it is listed and will
be reported on in the Product Report. You must supply user credentials.
To enter vCenter Operations Manager Server user credentials
1. Under vCenter Operations Manager Servers, click Edit.
2. Enter User Name and Password.
© 2012 VMware, Inc. All rights reserved.
Page 16 of 39
vCloud Usage Meter
User Guide
6.4.3 Site Recovery Manager Server
(Optional) Configure Site Recovery Manager (SRM) Servers and their relationships to vCenter
Servers. Only one SRM Server, located at the protected or recovery site, needs to be configured
for metering.
Multiple SRM relationships can be set up for one or more customers and each SRM setup needs to be
metered. Usage is added up with other SRM servers.
Example: NewCo and YourCo are both running private cloud backups to a Service Provider.
Each customer would need a separate encapsulated SRM installation to back up resources. The
Service Provider would register each failover SRM server/vCenter relationship for each customer.
To enter Site Recovery details
1. Under Site Recovery Manager Servers, click Add. The Add/Edit SRM Server screen
appears.
2. Enter Host Name or IP.
3. Enter User Name. Read-Only administrator privileges required.
4. Enter a Password.
5. From the dropdown, select the vCenter Server to pair to the SRM Server.
6. In order to test the SRM Server credentials when saving, select the Test Before Saving
check box. If there is an error, a separate message dialog will appear with error details, and
the server will not be added to the list. By default this check box is selected.
7. Click Save to commit changes.
6.4.4 vCloud Integration Manager Details
Configure vCloud Integration Manager details in order to collect usage data from vCloud
Integration Manager.
To enter vCloud Integration Manager details
1. Under vCloud Integration Manager, click Add. The Add/Edit vCloud Integration Manager
screen appears.
2. Enter Host Name or IP.
3. Enter Username as [email protected]
4. Enter a Password.
5. In order to test the vCloud Integration Manager credentials when saving, select the Test
Before Saving check box. If there is an error, a separate message dialog will appear with
error details, and vCloud Integration Manager will not be added to the list. By default this
check box is selected.
6.4.5 Collector Details
Set the start time for the vCloud Usage Meter to poll vCenter Servers. The default is 5 minutes
after the hour. The start time can be set at zero (the start of the hour) or to any other minute of
the hour, in order to avoid competition with other scripts working with the vCenter Servers.
1. Select the minute of the hour for the polling to start.
2. Click Save to commit changes.
© 2012 VMware, Inc. All rights reserved.
Page 17 of 39
vCloud Usage Meter
User Guide
3. (Optional) Click Test to verify that the collection process is working correctly. Data will be
collected for the purpose of the test, but will not be saved to the database. Email alerts must
be set in advance to receive results.
© 2012 VMware, Inc. All rights reserved.
Page 18 of 39
vCloud Usage Meter
User Guide
6.5
API Access Tokens
All vCloud Usage Meter APIs require authentication. If you wish to use the vCloud Usage Meter
API, generate an authentication token.
For more information, refer to the vCloud Usage Meter API Guide v3.0.
© 2012 VMware, Inc. All rights reserved.
Page 19 of 39
vCloud Usage Meter
User Guide
7. Managing Licenses
License sets allow you to associate virtual machine vRAM usage with one or more vSphere
license keys. vSphere licenses keys are updated after a successful vCenter query. If a license
key is no longer present in a vCenter Server, the license is removed from the licenses that can be
managed.
7.1
Managing License Sets
To create a License Set
1. Click Licenses in the top right menu bar.
2. Under the text License Sets, enter a license set name and click New.
3. Under the text Add Licenses to the current set, select the name of the License Set you
wish to edit. The selected set is also listed as the current set next to the Delete button.
4. To manage the licenses associated with the license set, select or deselect the checkboxes on
the right. License details (name, key, vCenters, and hosts) are displayed to aid in your
selections. Licenses can be assigned to more than one license set.
To delete a License Set
1. Select the license set you wish to delete.
2. Click Delete. A message asking ―Are you sure?‖ appears.
3. Click Yes.
7.2
Managing Billing Categories
vSphere license keys deployed by Service Providers may have different billing characteristics, so
you can indicate which vSphere license keys are billable by defining the billing category. You can
assign these categories:
VSPP. (Default) License key obtained through VSPP and all virtual machines running on
hosts with these keys are considered to be billable.
Demo. License keys that are not billable that can be used for environments such as demo
systems and View deployments.
Perpetual. License keys obtained outside of the VSPP program and virtual machines running
on hosts with these keys are not considered billable.
To assign a license category
Under the heading Billing Category, select a billing category for each license as desired. The list
provides license details (name, key, and vCenters) to aid in your category selection.
© 2012 VMware, Inc. All rights reserved.
Page 20 of 39
vCloud Usage Meter
User Guide
8. Setting Up Automatic Reporting
You can set up VCloud Usage Meter to automatically generate sets of reports and email them to
an Aggregator, or elsewhere. Service Providers who report to multiple Aggregators can set up an
automatic report for each one, using a different license set for each (for information about license
sets, see Managing Licenses).
Available reports to use with Automatic Reporting Sets are displayed by name. Reports can be
included in more than one automatic reporting set.
To create or edit an Automatic Reporting Set
1. Click Automatic Reporting in the top right menu bar.
2. Enter an Automatic Reporting Set name in the right hand box next to Name. As you type,
existing names starting with the same letter will appear. If you select an existing set, the last
time the report was sent, if available, appears at the bottom of this area.
3. Select a reporting day of the month.
4. Select a set of reports to include by using Shift + Click or Ctrl + Click. Reports can be
assigned to more than one automatic reporting set.
5. Select a license set or sets to include by using Shift + Click or Ctrl + Click. License sets can
be assigned to more than one automatic reporting set.
6. Enter an email that the reporting will be from in From Email.
7. Enter an email that the reporting will be sent to in To Email.
8. Click Save.
To delete an Automatic Reporting Set
1. On the left hand side, ensure the automatic reporting set that you wish to delete is selected.
2. Click Delete.
© 2012 VMware, Inc. All rights reserved.
Page 21 of 39
vCloud Usage Meter
User Guide
9. Monitoring
The Monitor page allows you to view times of successful (OK) or failed (Fail) collections for each
vCenter Server as set up in the Manage page (for more information, see vCenter Servers).
The default view is the current month and year, but you can select a view of any month and year
of the past four years.
To select and view a month and year
1. Click Monitor in the top right menu bar.
2. Select the year and month you wish to view.
3. Click Submit.
4. To view collection times, hover your mouse over the OK or Fail count in any column.
© 2012 VMware, Inc. All rights reserved.
Page 22 of 39
vCloud Usage Meter
User Guide
10. Managing Customers
The vCloud Usage Meter provides the ability to add, edit, delete, export, and import customers
that are using Service Provider resources.
Creating customers and also associating rules with them enables monthly customer usage
reporting. It also aids in planning, billing, general account management, dispute resolution, and
other purposes. For more information about rules, see Managing Rules.
Table 5. vCloud Usage Meter Customer Options
Option
Description
Sharing customer
details among multiple
vCloud Usage Meter
virtual appliances
Service Providers frequently have multiple vCloud Usage Meter virtual
appliances installed in different datacenters. There is no federated database or
replication between vCloud Usage Meter virtual appliances at this time. So,
customer details stored in one vCloud Usage Meter virtual appliance are not
shared among other virtual appliances.
If a customer’s details are changed, in order to share the change among
multiple virtual appliances, customer details must be exported or imported into
the other installations. VMware recommends that an administrator enter
customers into one virtual appliance and update the others independently.
Bulk upload and export
of customer details,
including associated
rules
Basic customer importing and exporting are discussed inImporting/Exporting
Customers and Rules.
Shielding customers
whose details are
confidential
If a customer’s name or location is considered sensitive, the customer can be
designated as restricted, and given a code name and/or location. An
arrangement is made with VMware VSPP on what code to use for these
customers and the location of the restricted customers is maintained at VSPP
operations.
For more information about rules, see Managing Rules.
While the customer’s code details can be either added through the UI or
imported, the restricted status can only be set manually through the UI (see
Adding a Customer).
10.1 Adding a Customer Through the User Interface
To add a customer
1. Click Customers in the top right menu bar.
2. Click Add. Add/Edit Customer screen appears.
3. Enter a unique customer Name.
4. If the customer is restricted, select the Restricted check box.
5. Select a Country.
6. Enter a Postal Code.
7. Click Save to commit changes. The new customer appears in the customer list.
© 2012 VMware, Inc. All rights reserved.
Page 23 of 39
vCloud Usage Meter
User Guide
10.2 Deleting Customers
When deleting a customer from the vCloud Usage Meter, all rules associated with the customers
are also removed from the system.
To delete a customer
1. Click Customers in the top right menu bar.
2. Select the check box for the customer or customers to be deleted.
3. Click the Delete Selected button. A message asking ―Delete all selected?‖ appears.
4. Click Yes. A message will appear in the upper left of the browser window confirming that the
customer or customers have been deleted.
10.3 Importing/Exporting Customers and Rules
Customer and rules can be imported into the vCloud Usage Meter via a tab separated file. This
file can be manually generated or consist of an exported customers and customers rules list from
another vCloud Usage Meter virtual appliance.
The vCloud Usage Meter allows:
Import of customers
Import of customers and rules together
Export of customers and rules together
For information about how to construct rules, see Managing Rules. If vCloud Usage Meter cannot
import a rule, an error message is displayed and the rule is not created.
About customer imports:
Duplicate customer names are not permitted.
Customers that are restricted are not allowed to be imported via a tab separated file and must
be entered manually.
Country code information is available from the ISO Organization, ISO 3166-1-alpha-2 code.
© 2012 VMware, Inc. All rights reserved.
Page 24 of 39
vCloud Usage Meter
User Guide
To import customers and/or rules
1. Prepare, or obtain from export, a tab separated file. A sample of the contents of a file that
would import both customers and rules is shown below.
Row
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
Column A
# vCloud
Usage Meter
Customer
Export
# Version: 1
# Customers
# Name
NewCo
YourCo
TheCo
ThisCo
# Rules
# Example rule:
Exact string
folder match for
abc for customer
CloudCo
# CloudCo
# Customer
NewCo
NewCo
NewCo
NewCo
NewCo
NewCo
TheCo
Column B
Column C
Country
AF
US
Postal Code
Column D
Column E
Exact String
Value Type
Unique ID
Unique ID
Exact string
Unique ID
Unique ID
Unique ID
Unique ID
abc
Value
vm-100
host-77
ad
resgroup-v99
vm-103
resgroup-v91
datacenter-2
44
94555
US
95555
vCenter
10.255.79.10
10.255.79.10
10.255.79.10
10.255.79.10
10.255.79.10
192.168.128.1
Folder
Object Type
VM
Host
VM DNS name
vApp
VM
vApp
Data Center
2. Click Customers in the top right menu bar.
3. Browse to the file to be uploaded and click Import:
a. If import completes without error, a new page is displayed with the message ―Importing
completed with no errors.‖
b. If import of customers fails, an error message indicating the line number, text associated
with the line producing the error, and the error message are displayed.
4. Click in the top right menu bar to return to the VCloud Usage Meter UI.
To export customers and rules
1. Click Customers in the top right menu bar.
2. Click the Export All button. A new dialog window is presented in order to save the customers
and customer’s rules as a tab separated text file (.tsv).
The export procedure exports all customers and rules whether customers are selected or
unselected (there is no selection for a subset in this version of the vCloud Usage Meter).
© 2012 VMware, Inc. All rights reserved.
Page 25 of 39
vCloud Usage Meter
User Guide
11. Managing Rules
Customer rules allow a Service Provider to associate vCenter Server objects with a customer’s
cloud infrastructure. This enables monthly customer usage reporting as defined in the VSPP
Program Guide and the VSPP Product Guide. The vCloud Usage Meter provides the Service
Provider with granular control of a vCenter Server inventory, where they are able to link objects in
the vCenter Server inventory from as high as the vCenter Server down to the individual unique ID
for a virtual machine or IP address.
During customer rules creation, if there is a conflict with another customer rule an error message
is displayed to the end user with customer and virtual machines affected and the rule is not
created.
Table 6 vCloud Usage Meter Rules Options
Option
Via
Description
Create a rule
directly, by specific
object type
Rules page
Create a rule by directly selecting the
object and value type, and then
entering a value.
Create a rule by
locating an object
type in the vSphere
inventory
Rules page
Use the filters and tree view to locate
the object type that the rule will map
to.
Import of customers
and rules together
Customers page
Import as tab separated file.
For further information, refer to these topics:
Managing Customers for more information about customers.
Importing/Exporting Customers and Rules for more information and a sample of a rule as
imported or exported.
Generating Reports for information about reports.
Customer Summary Report for information about reviewing customers’ monthly usage after
creating rules.
11.1 Adding a Rule
The features in the Rules page help you map rules to the inventory structure.
To link a rule to a customer using Create Rules
1. Click Rules in the top right menu bar.
2. Enter the Customer name. The Customer name has auto complete functionality to assist
with finding a customer. As you enter each letter there is a slight delay as the customer list is
populated based on values entered. Complete the name manually, or select the customer
from the filtered list provided.
© 2012 VMware, Inc. All rights reserved.
Page 26 of 39
vCloud Usage Meter
User Guide
3. Select Object Type and Value Type. As you select Object Type, the options for Value Type
will vary accordingly. For certain combinations of object and value type, other selections,
such as vCenter, are required.
Exact String, Substring, Regular Expression, and CIDR Notation apply to all vCenter Servers
registered with the vCloud Usage Meter virtual appliance. Refer to the table below for the
input required for combinations of supported object and value types.
Table 7. Supported Object and Value Types for Mapping Customers to Rules
Object Type
Description
Value
Types
vCenter
Needed
VM
A virtual machine is a software computer that
Unique ID
yes
VM vCenter
Server Name
A name for the vCenter Server that represents its
purpose
Exact string
no
Substring
no
Regular
expression
no
Exact string
no
Substring
no
Regular
expression
no
VM DNS
Name
The DNS name of a virtual machine.
IP V4
address
IP network address.
IP V4 CIDR
notation
no
Folder
Folders allow you to group objects of the same type so
you can easily manage them. For example, you can
use folders to set permissions across objects, to set
alarms across objects, and to organize objects in a
meaningful way.
Unique ID
yes
Exact string
no
Substring
no
Regular
expression
no
A resource pool is a logical abstraction for flexible
management of resources. Resource pools can be
grouped into hierarchies and used to hierarchically
partition available CPU and memory resources in
vSphere or organization virtual datacenter in vCloud
Director.
Unique ID
yes
Exact string
no
Substring
no
Regular
expression
no
A vSphere vApp is a format for packaging and
managing applications. A vApp can contain multiple
virtual machines.
Unique ID
yes
Exact string
no
Substring
no
Regular
expression
no
Unique ID
yes
Resource
Pool
vApp
Host
Similar to rules that are created against vCenter
Servers, only host unique IDs can be associated
against a customer.
© 2012 VMware, Inc. All rights reserved.
Page 27 of 39
vCloud Usage Meter
User Guide
Cluster
Similar to rules that are created against vCenter
Servers, only cluster unique IDs can be associated
against a customer.
Unique ID
yes
Datacenter
Similar to rules that are created against vCenter
Servers, only datacenter unique IDs can be associated
against a customer.
Unique ID
yes
vCenter
Server
Rules that are created against vCenter Servers are
specific to that vCenter Server unique ID only without
the functionality to configure based on string, substring, or regular expression.
Unique ID
yes
4. Enter a Value to be associated with the customer.
5. Click Create to add the new rule to the current rules associated with the customer.
6. Click Show customer virtual machines to view all virtual machines associated with all
customer rules created.
To link a rule to a customer using the vSphere Inventory Tree
1. Click Rules in the top right menu bar.
2. Enter the Customer name. The Customer name has auto complete functionality to assist
with finding a customer. As you enter each letter there is a slight delay as the customer list is
populated, based on values entered. Complete the name manually, or select the customer
from the filtered list provided.
3. Review the tree displayed under vSphere Inventory. The vCenter Servers, and objects
contained within them, can be expanded or collapsed by clicking the arrow pointers, or using
the buttons Collapse All or Expand All.
The vSphere Inventory filter is linked to those vCenter Servers that the vCloud Usage Meter
is collecting usage data for. By default, all objects in the vCenter Server inventory are
collected for each vCenter Server.
4. Enter text in the field to the left of Filter that matches a location in the tree that you want to
create a rule for, and then click Filter. The filter then reduces the tree display to objects
related to the text you entered.
5. Select the vSphere Inventory Object you wish to associate with the customer. A Unique
ID—or mapping between the vCenter Server ID and more—is created to link the customer to
the associated vSphere Inventory object and the details of that mapping then appear under
the Create Rules section.
6. If the mapping is correct, click Create, as in step 5 above. Then continue with step 6 above.
7. If the mapping is not correct, click Reset Filter to clear the previous search criteria.
11.2 Deleting Customer Rules
Customer’s rules can be deleted from the vCloud Usage Meter using the Rules page. When
deleting customer’s rules from the vCloud Usage Meter, only the relationship between the
vSphere Inventory object and the customer is removed. All usage data associated with the virtual
machines is still maintained.
© 2012 VMware, Inc. All rights reserved.
Page 28 of 39
vCloud Usage Meter
User Guide
To delete customer rules
1. Click Rules in the top right menu bar.
2. Enter the Customer name. The area Current Rules displays the current rules for the
customer.
3. Select the checkbox for the rule(s) to be deleted.
4. Click the Delete Selected button. A message asking ―Are you sure?‖ appears.
5. Click Yes. A message will appear in the upper left of the browser window confirming that the
rule or rules have been deleted.
© 2012 VMware, Inc. All rights reserved.
Page 29 of 39
vCloud Usage Meter
User Guide
12. Generating Reports
The vCloud Usage Meter allows the end user to generate various reports to monitor and track
resource usage for your vCenter Server and virtual machines. There are six different reports that
can be generated from the vCloud Usage Meter, as described below.
A report can be generated manually at any time and can be exported as a tab separated file or
zip, when a large number of records are returned.
The previous report executed will remain cached in your browser until another report is generated
or upon logout from the vCloud Usage Meter virtual appliances.
To generate a report
1. Click Report in the top right menu bar.
2. Select the Report Type.
3. Select:
All to return all records.
Customer to report on a specific customer.
Enter a Customer name in the provided field. This field has auto complete functionality.
As you enter each letter the customer list is populated based on values entered.
Complete the name manually, or select the customer from the filtered list provided.
Customer Country to report by country
Enter a Customer Country in the provided field. This field has auto complete
functionality. As you enter each letter the country list is populated based on values
entered and the countries of existing customers.
Customer Postal Code to report by postal code
Enter a Postal Code in the provided field. This field has auto complete functionality. As
you enter each number or letter the postal code list is populated based on values entered
and the codes of existing customers.
4. Select License Sets or leave blank to return all records.
5. Select Options:
a. Select Month and Year.
b. Specify Per VM Memory Cap (this field has a default value of 24 GB)
6. To view the report in the user interface, click the Browse button.
7. To save the report to a file, click the Export button. (Optional) Select the Zip checkbox to
export the file as a zip archive.
12.1 Detailed Billing Report
This is a generic report that provides billing information for all types of vSphere licenses and is
also used for sending usage details to the Aggregator assigned to the Service Provider. Virtual
machine usage reported is based on which vSphere license keys are selected as VSPP keys.
Please refer to License Management for more details.
© 2012 VMware, Inc. All rights reserved.
Page 30 of 39
vCloud Usage Meter
User Guide
Table 8. Detailed Billing Report Details
Report Column
Description
Time
Day of the reporting month, hour, minute, and milliseconds
Host DNS Name
Host identifier which can be IP address or FQDN depending on how the
host was configured with vCenter
Host Memory Size (KB)
Host memory size displayed in KB
vSphere License
vSphere license type associated with vSphere host
VM RAM (MB)
Configured virtual RAM for the virtual machine
Billing RAM (MB)
Billing virtual RAM for the virtual machine calculated from the amount of
reserved RAM assigned to the virtual machine
VM CPUs
Number of CPUs assigned to the virtual machine
VM Instance UUID
Unique identifier for virtual machine to assist with identification and audit
12.2 Detailed Usage Report
The Usage Data Report provides all usage-related information including virtual machine name as
listed in vCenter Server and virtual machine DNS name.
Table 9. Detailed Usage Report Details
Report Column
Description
Time
Day of the reporting month, hour, minute, and milliseconds
Customer
Customer associated with record
Host DNS Name
Host identifier which can be IP address or FQDN depending on how the
host was configured with vCenter
Host RAM (GB)
Host memory size displayed in GB
vSphere License
vSphere license type associated with vSphere host
VM VC Name
Virtual machine name as listed in vCenter Server
VM Hostname
Virtual machine DNS name
VM RAM (MB)
Configured virtual RAM for the virtual machine
© 2012 VMware, Inc. All rights reserved.
Page 31 of 39
vCloud Usage Meter
User Guide
Report Column
Description
Billing RAM (MB)
Billing virtual RAM for the virtual machine calculated from the amount of
reserved RAM assigned to the virtual machine
VM CPUs
Number of CPUs assigned to the virtual machine
VM Instance UUID
Unique identifier for virtual machine to assist with identification and audit
12.3 Customer Summary Report
This report provides information such as customer, vSphere license type, category, if a license is
billable, billed vRAM in GB, and capped billed vRAM in GB. Uncapped and capped memory totals
are based on the sum of all hourly collections for each license type.
Table 10. Customer Summary Report Details
Report Column
Description
Customer
Customer associated with usage
Country
Customer’s country
Postal
Customer’s postal code
License Type
License type being metered
Category
vSphere license category
Billable
If the vSphere license is billable
Billed vRAM (GB-Hour)
Non-Capped Memory (GB-Hour)
Capped Billed vRAM
(GB-Hour)
Capped Memory (GB-Hour) – Used in billing calculations
12.4 License Summary Report
This report provides information such as vSphere license type, category, if license is billable,
Billed vRAM in GB, and Capped Billed vRAM in GB. Uncapped and capped memory totals are
based on the sum of all hourly collections for each license type. This report replaces the
Aggregate Billing report for the vCloud Usage Meter releases previous to version 2.3 and is sent
to the Aggregator for use in monthly reporting.
© 2012 VMware, Inc. All rights reserved.
Page 32 of 39
vCloud Usage Meter
User Guide
Table 11. License Summary Report Details
Report Column
Description
Billable
If the vSphere license is billable
Category
vSphere license category
License Type
License type being metered
Billed vRAM (GB-Hour)
Non-Capped Memory (GB-Hour)
Capped Billed vRAM
(GB-Hour)
Capped Memory (GB-Hour)—Used in billing calculation
12.5 Product Report
This report provides information such the product being monitored, the metric being metered on,
and the associated monthly usage. This report will be included in monthly reporting if Site
Recovery Manager is used to protect the customer’s virtual machines.
Table 12. Product Report Details
Report Column
Description
Product
Product metered
Metric Type
Metric being used to meter a product
Value
Monthly usage for product
12.6 Customer Product Report
This report provides lists the customers and virtual machines protected by Site Recovery
Manager. Customers will be displayed in the report if the vCenter Server Folder and vCenter
Resource Pool associated with a protection group are also mapped to the customer as part of the
customers rules definitions.
Virtual machines will be displayed in this report if they are protected for one or more days during
the month.
© 2012 VMware, Inc. All rights reserved.
Page 33 of 39
vCloud Usage Meter
User Guide
Table 13. Customer Product Report Details
Report Column
Description
Customer
Customer associated with the virtual machine
Product
Product metered
Object
Metric being used to meter a product
© 2012 VMware, Inc. All rights reserved.
Page 34 of 39
vCloud Usage Meter
User Guide
13. Support
The vCloud Usage Meter Support page provides information for developer or technical support
use.
13.1 Runtime Information
Runtime Information is provided for diagnostic purposes. Technical support may request you to
relay information such as heap used and Java arguments in order to diagnose problems.
13.2 Database
This information shows all virtual machines in the Usage Meter database, along with their related
hosts and vCenter Servers. This helps diagnose problems such as duplicate hosts.
13.3 Logging Level
Access appliance log information for the vCloud Usage Meter through the console at
/var/log/usgmtr/*.
The available logging level settings are:
Info (default, includes Warn, Error, and Fatal levels)
Debug (more detail, will cause the logs to fill faster)
Warn (warnings only, not recommended)
Error (errors only, not recommended)
Trace (not currently used)
Fatal (fatal only, not recommended)
To change logging levels in the user interface
1. Click Support in the top right menu bar.
2. Under Logging Level, select the check box Detailed.
To change logging levels in the properties file
1. Access the log4j.properties file at
/usr/local/apache-tomcat-7.0.14/webapps/um/WEB-INF/classes/log4j.properties
2. Edit this line by replacing INFO with your desired logging level:
log4j.rootLogger=INFO, roll
13.4 Log Rolling
By default the Usage Meter allots 100MB for log activity, before deleting log history. VMware
recommends that this log rolling capacity not be reduced.
To change log rolling settings in the properties file
1. Access the log4j.properties file at
/usr/local/apache-tomcat-7.0.14/webapps/um/WEB-INF/classes/log4j.properties
© 2012 VMware, Inc. All rights reserved.
Page 35 of 39
vCloud Usage Meter
User Guide
2. Edit these lines to configure your desired log rolling:
log4j.appender.roll.MaxFileSize=10MB
log4j.appender.roll.MaxBackupIndex=10
13.5 Support Bundle
Technical support may request support bundles, a collection of log files, to help diagnose
problems. Support bundles take disk space and should be deleted when no longer needed.
To generate or delete a support bundle
1. Under Support Bundle, click Generate.
2. Under Support Bundle, click Delete support bundles.
© 2012 VMware, Inc. All rights reserved.
Page 36 of 39
vCloud Usage Meter
User Guide
14. Using the vCloud Usage Meter's JMX Service
Each vCloud Usage Meter server host exposes a number of MBeans through JMX (Java
Management Extensions) to provide access to internal data such as database size, customer
count, customer rules count, and other information.
You can use any JMX client to access the vCloud Usage Meter JMX service. JConsole is an
example of a JMX client.
To use the vCloud Usage Meter’s JMX service
1. Use the command
/opt/vmware/cloudusagemetering/scripts/tomcat
2. To enable local access only JMX monitoring, uncomment the lines given (for example):
#export CATALINA_OPTS='Dcom.sun.management.jmxremote.password.file=$CATALINA_HOME/conf/jmxr
emote.password Dcom.sun.management.jmxremote.access.file=$CATALINA_HOME/conf/jmxrem
ote.access -Dcom.sun.management.jmxremote Dcom.sun.management.jmxremote.port=9003 Dcom.sun.management.jmxremote.ssl=false Dcom.sun.management.jmxremote.authenticate=true'
#export JAVA_OPTS=$JAVA_OPTS" -Djava.rmi.server.hostname=`ifconfig |
grep 'inet addr:'| grep -v '127.0.0.1' | cut -d: -f2 | awk '{ print
$1}'`"
3. In the directory /usr/local/apache-tomcat-7.0.14/conf/ create two files:
jmxremote.password
jmxremote.access
4. To create a read-only user for JMX:
a.
Add the line monitorRole <password> to the file jmxremote.password , replacing
<password> with your chosen password
b. Add the line monitorRole readonly to the file jmxremote.access
5. To change the file permissions to 600, use the command
chmod 600 /usr/local/apache-tomcat-7.0.14/conf/jmxremote.*
6. Restart Tomcat with the command
service tomcat restart
7. Connect with your JMX client by entering:
IP address of the vCloud Usage Meter server
Port number 9003
User name monitorRole
Password for the user you created
© 2012 VMware, Inc. All rights reserved.
Page 37 of 39
vCloud Usage Meter
User Guide
15. Advanced Management
These options allow you to configure memory size, collection speed, monitoring, and database
backup.
15.1 Configuring Memory for Environment Size
All but the very largest environments should work well with the vCloud Usage Meter’s default
settings.
The default virtual appliance vRAM size is 3,600 MB. Service Providers with smaller
environments can lower the virtual appliance vRAM to as low as 1,536 MB.
To see how much vRAM the web application is using, look in the /var/log/usgmtr/um.log after a
collection has run , or look at the Support Page while a collection is running. Your vRAM size
should be configured to 25% higher than the vRAM usage you observe, to allow for a reserved
amount of vRAM for the database and operating system.
You do not need to modify JVM heap size settings. The virtual appliance now self-adjusts
memory, reserving 800 MB for the database and operating system, and dedicating the remainder
to the vCloud Usage Meter services.
To change the amount of vRAM for the Usage Meter virtual appliance
1. In your vSphere client, navigate to Home > Hosts and Clusters > VMs and Templates.
2. Locate the vCloud Usage Meter virtual appliance in the inventory tree, right-click and select
Power > Power Off.
3. Right-click and select Edit Settings.
4. Click the Hardware tab and Memory in the list on the left, if these are not already selected.
5. Under Memory Configuration on the right, type a larger or smaller Memory Size value, and
click OK.
6. Right-click the vCloud Usage Meter virtual appliance and select Power > Power On.
15.2 Turning On Parallel Collections
To increase collection speed if you have multiple vCenter Servers, you can turn on parallel
collections.
1. Use the command
service tomcat stop
2. Change the 1 to a 5:
/usr/local/apache-tomcat-7.0.14/WEB-INF/webapps/um/web.xml
<context-param>
<param-name>maxConcurrentCollections</param-name>
<param-value>1</param-value>
</context-param>
3. Use the command
service tomcat restart
© 2012 VMware, Inc. All rights reserved.
Page 38 of 39
vCloud Usage Meter
User Guide
15.3 Monitoring the Appliance
This provides a way to monitor the vCloud Usage Meter virtual machine for failures using HP
OpenView.
SSH is pre-installed in the appliance, but it is set to chkconfig off for sshd.
To enable SSH
As root, use these commands:
chkconfig sshd on
service sshd start
15.4 Managing the vCloud Usage Meter Database
All usage data collected by the vCloud Usage Meter is maintained in the appliance database.
Therefore, to maintain history, VMware recommends that you not delete the vCloud Usage Meter
virtual appliance.
Database administrators should have knowledge of PostgreSQL before performing any direct
database functions. VMware does not support modifying the database directly.
To back up the database
Run this command as the PostgreSQL user:
pg_dump usage_meter_db | gzip > /tmp/usage_meter_db.pgd.gz
This will auto-gzip the database into a single file that can be viewed.
© 2012 VMware, Inc. All rights reserved.
Page 39 of 39
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement