Avid Technology | Unity MediaNetwork PortServer | Avid® Unity™ MediaNetwork 5 - System Maintenance

Avid® Unity™ MediaNetwork 5 - System Maintenance
 ®
Avid Unity™ MediaNetwork 5
System Maintenance
Avid Technology EMEA | Enterprise Customer Success
Page 1 of 20 Avid® Unity™ MediaNetwork 5 System Maintenance Table of Contents
Introduction .................................................................................................................................................. 2
Daily Maintenance Procedures ..................................................................................................................... 3
Check the Monitor Tool for Error Indications........................................................................................... 3
Check the Windows Event Log.................................................................................................................. 4
The “DEAN” light....................................................................................................................................... 5
Check for Proper Metadata Verification and Saves.................................................................................. 6
Check internal RAID integrity.................................................................................................................... 7
Weekly Maintenance Procedures ................................................................................................................. 8
Checking Workspace Usage ...................................................................................................................... 8
Rebooting the MediaEngine ................................................................................................................... 10
Appendices.................................................................................................................................................. 12
Appendix 1 ‐ Identifying Bad Drives........................................................................................................ 12
Appendix 2 ‐ Replacing a Failed Drive..................................................................................................... 14
Appendix 3 ‐ Recreating Drive Labels ..................................................................................................... 18
Appendix 4 ‐ The PortServer Console ..................................................................................................... 19
Appendix 5 ‐ Making a Metadata back‐up.............................................................................................. 20
Introduction
This document covers basic maintenance procedures for Avid® Unity™ MediaNetwork 5 shared storage systems. The section “Daily Maintenance Procedures” details actions that should be taken several times a week to ensure continued operation of your Avid® Unity™ system. The Monitor tool should be checked daily, if possible to monitor system health. The section “Weekly Maintenance Procedures” describes action which, while no less important than the daily procedures, require less frequent attention. The workspace usage should be checked with the Administration tool at least once a week. Remedial procedures for issues found are covered in the Appendices. Please note that the use of Anti‐Virus solutions on the MediaEngine itself are not supported. Typical symptoms caused by AV solutions include attempts to Optimize a workspace failing and the random disconnection of Ethernet Attached Clients. Avid Technology EMEA | Enterprise Customer Success
Page 2 of 20 Avid® Unity™ MediaNetwork 5 System Maintenance Daily Maintenance Procedures
ƒ
Check the Monitor Tool error indicators ƒ
Check Metadata verification and save status ƒ
Check internal RAID integrity Check the Monitor Tool for Error Indications
The Unity™ Monitor Tool runs only on the Unity MediaEngine and is usually set to launch automatically at system start‐up. If the Monitor Tool is not running, launch it from the Start menu of the Unity MediaEngine as shown in Fig.1 below:. Fig.1 ‐ Launching the Monitor Tool from the Start menu Under normal conditions, the Monitor Tool should display 4 green indicators on the first tab of the monitor tool as shown in Fig.2 below: Normal operation is indicated by 4 “green lights” Fig.2 ‐ System running without error conditions (“All Green Lights”) Avid Technology EMEA | Enterprise Customer Success
Page 3 of 20 Avid® Unity™ MediaNetwork 5 System Maintenance Check the Windows Event Log
If the fourth indicator changes to red and is accompanied by the text “Check Event Log” this indicates that Avid® Unity™ MediaNetwork has written an event to the Windows System Event Log: Unity MediaNetwork has made an entry in the System Event Log Dismiss the indicator by clicking the “Reset Event” button Fig.3 – “Check event log” indicator The Windows System Event Log should be checked and remedial action taken. Contact Avid Customer Success for assistance if required. The indicator may be returned to normal by clicking the “Reset Event” button. Avid Technology EMEA | Enterprise Customer Success
Page 4 of 20 Avid® Unity™ MediaNetwork 5 System Maintenance The “DEAN” light
When errors have been detected with one or more Media drives, the third indicator changes to red and the text changes to “Drive Error Analysis Needed”: Errors have been detected on one or more Unity drives Dismiss the indicator by clicking the “Reset Event” button Fig.4 ‐ Drive Error Analysis Needed The file UnityClientLogs.txt should be checked using the Avid Disk Error Analyzer as described in Appendix 1 below. The indicator may be returned to normal by clicking the “Reset Event” button. Contact Avid Customer success if a replacement drive is required. The procedure for replacing a failed drive is detailed in Appendix 2. Avid Technology EMEA | Enterprise Customer Success
Page 5 of 20 Avid® Unity™ MediaNetwork 5 System Maintenance Check for Proper Metadata Verification and Saves
During normal operation Unity™ MediaNetwork will write its to disc Metadata continuously with 20 second pauses between Metadata saves. The last verification and save times can be found on the Metadata tab of the Monitor Tool. If you cannot see the Metadata Status tab, use the Tab navigation buttons to scroll through the tabs. Tab Navigation Buttons Metadata Verify Time and Save Time Fig.5 ‐ Metadata Status Verify and Save times Because the Metadata is verified and saved with 20 second pauses, the Verify and Save times should be within a few minutes of current time. As the system fills up, the metadata grows and takes longer to save, resulting in slightly longer time lags since the last successful Verify / Save. If the metadata appears not to have been updated for some time (several hours for example) then contact Avid Customer Success for assistance. NOTE: Examining the time and date stamp of the three metadata files in the D:\AVIDONLY folder will not provide an accurate indication of the last metadata save. The File Manager service writes directly into these files on a hardware level and the Windows datestamp is not update during these writes. Avid Technology EMEA | Enterprise Customer Success
Page 6 of 20 Avid® Unity™ MediaNetwork 5 System Maintenance Check internal RAID integrity
Unlike earlier versions of Avid® Unity™ MediaNetwork, to improve media read/write performance MediaNetwork 5 does not save a copy of its metadata to the media drives but instead saves three individual copies of the metadata to a separate partition on the internal system drive. This partition is mounted as “AVIDONLY” in the root of the D: drive. While it appears as a single drive to Windows, the internal system drive is actually a mirrored pair controlled by the on‐board RAID controller to provide redundant protection for the metadata. Should one of these drives fail it must be replaced immediately to ensure continued protection of the metadata. For added security, the metadata may be periodically copied to an external device. This and a further description of the metadata location are described in Appendix 5 below. To view the status of the individual hard drives, start the “Intel Matrix Storage Console” from the Start menu of the Unity MediaEngine as shown in Fig.6 below: Fig.6 – Launching the Intel Matrix Storage Console from the Start menu When the Storage Console opens, navigate down to “Volumes” and “RAID Hard Drives” to view the status of both the RAID and the individual drives as shown in Fig.7 below: Fig.7 ‐ The Intel Matrix Storage Console You may wish to leave the Storage Console open on the desktop. If a drive is found to be faulty, contact Avid Customer Success to obtain a replacement. Avid Technology EMEA | Enterprise Customer Success
Page 7 of 20 Avid® Unity™ MediaNetwork 5 System Maintenance Weekly Maintenance Procedures
ƒ
Check workspace usage ƒ
Reboot the Media Engine Checking Workspace Usage
To view current workspace usage, launch the Administration Tool from the Start Menu on either the Unity MediaEngine or any Fibre Connected client then click the “Workspace Manager” button on the toolbar to display the “Workspace Management” windows as shown in Fig.8 below: Workspace Manager Button
Total Space
Unallocated Space
Workspace is completely full Workspace less than half full Workspace is almost empty Amount by which the Workspace can be extended
Fig.8 – The Workspace Management window (graphic) Workspace Size
Workspace handle Note that: ƒ
The workspace names “everyone” is completely full and must be extended is users are to continue writing to it ƒ
The workspaces “Student1”, “Student2” and “Student3” are less than half full ƒ
The workspaces “tech2”, “tech4” and “tech7 are almost completely empty Workspaces may be increased or decreased in size at any time either by clicking on the workspace size and entering a new value or by holding down the <ALT> key and dragging the workspace handle. In the Fig.8 above there is a single Allocation Group named “PINEWOOD” having a total space of 3270.75GB and with approximately 1300GB of unallocated space. For reliable operation and effective optimisation of workspaces, Avid recommends that at least 7% of the total space in any Allocation Group remains unallocated for use as “working space”. For a more detailed view of workspace information with Workspace information displayed numerically rather than graphically, click the Workspace View button as shown in Fig.9 below: Avid Technology EMEA | Enterprise Customer Success
Page 8 of 20 Avid® Unity™ MediaNetwork 5 System Maintenance Workspace View Button
Fig.9 – Detailed Workspace view For more information on Workspace management including Optimization of Workspaces, please refer to Chapter 7 of the “Avid Unity MediaNetwork Administration Guide”. Avid Technology EMEA | Enterprise Customer Success
Page 9 of 20 Avid® Unity™ MediaNetwork 5 System Maintenance Rebooting the MediaEngine
To ensure continued reliable operation, the Avid Unity MediaEngine must be periodically rebooted. Ideally this should be performed once a week but, if operational constraints prevent this, an interval of up to a month should be acceptable. Before rebooting the MediaEngine, the File Manager service must first be stopped to ensure that the metadata is saved to disc. Click the “Stop File Manager” on the Monitor Tool to stop the File Manager service: Stop the File Manager service by clicking this button Fig.10 – Stopping the File Manager After clicking the button, the metadata is saved to disc. While this is happening the animation shown in Fig.11 below is displayed: Fig.11 – Metadata saving Avid Technology EMEA | Enterprise Customer Success
Page 10 of 20 Avid® Unity™ MediaNetwork 5 System Maintenance When the metadata has finished saving and the File Manager service has stopped, the Monitor Tool should look like Fig.12 below: Fig.12 – File Manager service not running The MediaEngine may now be rebooted from the Start menu as normal. Note that if a system restart or shut‐down is attempted without first stopping the File Manager service, Unity MediaNetwork will intercept the command and simply save the metadata and stop the File Manger service (the saving metadata animation will be shown as in Fig.11 above). The system will not shut down or restart until the command is selected a second time from the start menu. This is to safeguard the metadata against accidental shut‐downs. Note that the MEDIArrays should not require scheduled reboots but a periodic reboot during a scheduled reboot of the MediaEngine is a good idea. Three or four reboot cycles in a 12 month period would be ideal. NOTE: With continued use, Ethernet Attached Client licenses can fail to be released when an client disconnects. Rebooting the MediaEngine will free the stuck licenses but if a reboot is not possible at a time when they are required they may be freed using the PortServer console as described in Appendix 4 below: Avid Technology EMEA | Enterprise Customer Success
Page 11 of 20 Avid® Unity™ MediaNetwork 5 System Maintenance Appendices
1.
Identifying Bad Drives 2.
Replacing a failed drive 3.
Recreating Drive Labels 4.
The PortServer Console 5.
Making a Metadata back‐up Appendix 1 - Identifying Bad Drives
The File Manager service tracks drive and client errors in the file UnityClientLog.txt. To analyse the errors which trigger the “Drive Error Analysis Needed” indicator in the Monitor tool, launch the Avid Disk Error Analyzer from the Start menu of the MediaEngine. Navigate to the file UnityClientLog.txt in the folder D:\Program Files\Avid Technology\AvidUnity. When the Disk Error Analyzer loads it should look something likeFig.13 below: Fig.13 – The Avid Disk Error Analyzer From the pane on the right‐hand side you can choose to display error on either a per‐client or a per‐drive basis. If a particular drive is showing many write errors or “Long Read Command Time” errors then it probably requires replacing. Contact Avid Customer Success to arrange a replacement. Please note that Avid Customer Success may ask you to perform addition testing on the drive using Avid Storage Manager (ASM) or DiskLog. Note that the UnityClientLog.txt file can be analysed on any Windows PC by copying both the file Analyzer.exe (the executable for Avid Disk Error Analyzer)and UnityClientLog.txt from the folder D:\Program Files\Avid Technology\AvidUnity. Avid Technology EMEA | Enterprise Customer Success
Page 12 of 20 Avid® Unity™ MediaNetwork 5 System Maintenance Once the errors have been remedied then a new empty UnityClientLog.txt file should be created from the “Advanced Settings” tab of the Monitor Tool as shown in Fig.14 below: Create a new UnityClientLog.txt file by clicking this button Fig.14 – Creating a new UnityClientLog.txt file A new empty file will be created and the old file will be renamed by appending the current date and time as shown in Fig.15: Fig.16 ‐ The UnityClientLog.txt files Avid Technology EMEA | Enterprise Customer Success
Page 13 of 20 Avid® Unity™ MediaNetwork 5 System Maintenance Appendix 2 - Replacing a Failed Drive
The online drive recovery operation is designed to recover drives while your system remains online and without disconnecting clients. This significantly improves workflow because client machines do not need to unmount workspaces during a repair. When you perform an online drive recovery please remember the following points: ƒ
Clients that are not already connected to the MediaEngine will not be able to connect until after the online repair is complete ƒ
If a client disconnects during the online drive repair it will not be able to reconnect ƒ
Clients will be briefly denied write access at some point during online repair while the File Manager service locks the Data Drive Set from writes in order to finish the repair ƒ
An online drive recovery will take longer to complete than an offline drive recovery The offline drive recovery operation will take the Data Drive Set offline for the duration of the recovery. This will force all clients to disconnect but the recovery will be completed more rapidly. To perform a drive recovery, launch Setup Manager from the Avid Unity group in the Start menu of the MediaEngine. Then proceed as follows referring to Fig.16 below: 1.
Select “Data Drive Set” from the “Available Drive Groups” window 2.
Confirm that there is a spare drive of the same size as the failed drive 3.
Click to select the drive to be replaced (Drive 006, Disk 15 in the example below) 4.
Hold down <Ctrl> and select the spare drive to use (Disk 1 in the example below) 5.
Choose “Recover Failed Data Drive” from the Recover menu 1. 2. 3.
4.
5.
Fig.16 – Replacing a drive in Setup Manager Avid Technology EMEA | Enterprise Customer Success
Page 14 of 20 Avid® Unity™ MediaNetwork 5 System Maintenance 6.
A confirmation dialogue box will appear. Click Yes to recover the selected drive. 7.
A dialogue box will appear asking whether you want to leave the Drive Set online during the process. Click Yes to start an online drive replacement. Click No to perform an offline drive replacement. 8.
(Optional) If you clicked Yes in the previous dialogue, the Server Offline Control dialogue box will appear. Click OK to continue with the offline repair after the specified time. A message will be sent to all connected clients informing them that they will be disconnected after the specified time has elapsed. Click Cancel to cancel the entire process. 9.
A progress indicator will appear showing that the Setup Manager is performing the drive recovery. This will take from between several seconds to several hours depending upon the amount of data on the drive. Note that writing to the drives is blocked for a brief period at the end of online recovery. Avid Technology EMEA | Enterprise Customer Success
Page 15 of 20 Avid® Unity™ MediaNetwork 5 System Maintenance 10. When the replacement is complete, the progress indicator will close and a completion dialogue box will appear. 11. The Setup Manager window will now appear as in Fig.17. below: Fig.17 – Setup Manager following a drive replacement Disk 15 shows as “Replaced”
Drive 006 is now Disk 1 12. The drive may now be physically removed from the chassis and the replacement drive inserted in it’s place. To locate which physical drive to remove, right‐click on the drive marked as “Replaced” and select “Identify”. The light on the drive in the drive enclosure will blink for 15 seconds. NOTE: If possible, you should watch when you select Identify to determine the actual drive that needs repair. Be sure to note which drive is identified to make sure you know which drive should be replaced. (For example, you could be to place a small piece of tape on the front of the drive.) 13. Manually remove the faulty drive from the drive enclosure, replace it with a new unit, and wait for it to spin up. NOTE: With SATA drives as used in both the Avid MediaEngine and MEDIArray XT, it may happen that the drive is not automatically recognised by the drive controller. If this is the case then a reboot of the enclosure containing the replaced drive will be necessary in order for the drive to be recognised. For this reason you may wish to postpone the physical drive replacement until the next maintenance window or scheduled reboot of the system. Avid Technology EMEA | Enterprise Customer Success
Page 16 of 20 Avid® Unity™ MediaNetwork 5 System Maintenance 14. The new drive will appear in the “Raw Drives” Drive Group and must be “made Spare” in order for it to appear in the Data Drive Set. Refer to Fig.18 below: 15. Select the “Raw Drives” Drive Group 16. Select the new Raw Drive 17. Select “Make Drive Spare” from the Drives menu 15. 16. 17.
Fig.18 – Making a Raw drive Spare 18. The new drive will now appear as a Spare in the main Setup Manager window (see Fig.16 above) Following the drive recovery it is possible that some data will not be able to be recovered on the new drive. This can happen, for example, if a file was not mirrored and the drive that contained some of its data failed completely. In this case there is no way to recover the original data. When this happens, the part of any file that could not be recovered is replaced with a sequence of 0 (zero) characters, and those files are moved to the root of the workspace on which they were located. They are then renamed to BadFileNNN OriginalName, where NNN is a unique integer and OriginalName is the original name of the damaged file. You should check these damaged files to see whether anything useful can be recovered and then delete those of no use to free up space. Avid Technology EMEA | Enterprise Customer Success
Page 17 of 20 Avid® Unity™ MediaNetwork 5 System Maintenance Appendix 3 - Recreating Drive Labels
If there is a Missing Disk in the Data Drive Set and that same drive appears in either the “Raw Drives” or “Other FibreChannel Drives” Drive Groups then there is a chance that the drive is not bad, but simply has lost its assigned Unity Label. Use the following procedure when attempting to reapply the appropriate label to this drive. If it is successful, the File Manager service will once again recognise the drive and bring it back into the Data Drive Set: 1.
Launch Setup Manager (see Appendix 2) and determine that the Missing Disk is appearing in Raw or Other FibreChannel Drives 2.
Navigate to the Avid Unity directory on the MediaEngine (normally D:\Program Files\Avid Technology\Avid Unity) 3.
Locate and Launch RecoverDisks.exe 4.
Once the GUI has opened, click on the Open File button 5.
Select the file RecoveryInfo.dat file and open it 6.
Within the RecoverDisks window a list of drives will appear 7.
Check the ACTION column. When a drive is recognized, the words “LABEL OKAY" appear under this heading. When a drive has lost its label, the words "REGEN NEEDED" will appear. 8.
If REGEN NEEDED appears next to any drive, scroll down to the bottom of the window and click on the COMMIT CHANGES button. 9.
If the process is successful, the DISPOSITION column will then say “NO CHANGE” next to all drives which say Label Okay, and “UPDATED" next to the drive which said “Regen Needed”. 10. Launch Setup Manager and select REFRESH DISPLAY from under the DRIVE SET menu. The drive will no longer appear under “Raw Drives” or “Other FibreChannel Drives” and the “Missing Disk” description will also be gone. The drive has regained its label and is once again part of the Data Drive Set. NOTE: If this process does not successfully reapply the label to the Missing Disk, then there may be a more serious issue with this drive and normal Unity Drive troubleshooting procedures should be followed. Avid Technology EMEA | Enterprise Customer Success
Page 18 of 20 Avid® Unity™ MediaNetwork 5 System Maintenance Appendix 4 - The PortServer Console
On occasion, Ethernet Attached Clients that are disconnected continue to occupy a license seat. Also, the PortServer can lock files on behalf of a client, and the Administrator may need to release the lock. In order to fix this, you can do the following: 1.
On the affected MediaEngine (or any computer with ethernet access to it) open a command prompt 2.
Type "telnet <Address> 6010" where <Address> is the IP address or resolvable hostname of the MediaEngine. If you are running this on the MediaEngine itself use either "127.0.0.1" or "localhost" 3.
A connection message will appear and the window will clear. Type "cccc". When you type the fourth "c" you will see a screen asking you to enter the password for the Unity Administrator account 4.
Enter the password for the account (case sensitive) and you will then be presented with a screen showing the various command options as follows: Fig.19 – The PortServer Console (l) List – this lists all of the client threads in the following format Machine Name [IP Address] (Active, Inactive) (Connection Manager, Editor, Console) A thread is considered inactive if the client has not issued a command on that thread in more than 2 minutes. A Connection Manager is distinguished from an Editor based on whether or not a workspace mount request has been issued on this thread. (d) Drop – this also lists all of the client threads, but allows the user to type in a number and kill the chosen thread. If bandwidth, licenses, or workspace mount requests have been associated with this thread, they will each be relinquished when the thread is dropped. Likewise, open file handles and locked files associated with this thread will also be dropped. (s) Show Locks – this lists all of the files currently locked by the PortServer (along with the IP address of the client that requested and acquired the lock). (u) Unlock file – lists all of the files currently locked by the PortServer and lets you pick one to release the lock. (q) Quit – exits the console application (but leaves the PortServer service running). By using the "l" command you can view which clients are inactive and then disconnect them with the "d" command. Once you have freed up sufficient licenses you should be able to reconnect your editing client in the usual way. NOTE: A “Quick and Dirty” way to free up stale licenses is to simply restart the Avid Unity PortServer service. This will however disconnect all Ethernet Attached Clients who will then be required to reconnect manually. Avid Technology EMEA | Enterprise Customer Success
Page 19 of 20 Avid® Unity™ MediaNetwork 5 System Maintenance Appendix 5 - Making a Metadata back-up
Earlier versions of Avid® Unity™ MediaNetwork employed a mechanism whereby the metadata (the information describing the location of all files stored on the Data Drive Set) was periodically written to the data drives themselves. In order to maximise media read/write performance, MediaNetwork 5 instead saves three individual copies of the metadata to a separate partition on the internal system drive which is a redundant mirrored pair of drives controlled by a hardware RAID controller. This partition is mounted as “AVIDONLY” in the root of the D: drive as shown in Fig.20 below: Fig.20 – The “AVIDONLY” metadata partition For additional data security it may be desirable to take a copy of these three files to an external device at the end of every day’s work. Should it then happen that the metadata is lost through internal hard‐drive failure (although preventative maintenance as described in Check internal RAID integrity above should reduce this risk to practically zero) no more than a day’s work would be lost. In order to copy these files, the File Manager service must first be stopped. This not only allows access to the files but also ensures that all metadata information has been written to them. The File Manager service may be stopped and started easily from the first tab of the monitor tool as described in Rebooting the MediaEngine above. Once the file manager service has been stopped, the files may be dragged and dropped onto an external device such as a USB flash drive or a remote network location. Should it ever be necessary to restore the metadata from these files following a catastrophic internal drive failure, please contact Avid Customer Success for assistance. Avid® Unity™ MediaNetwork 5 System Maintenance Simon Dorricott ‐ Shared Storage Technical Lead Avid Customer Success EMEA February 2011 Avid Technology EMEA | Enterprise Customer Success
Page 20 of 20 
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertising