Investigating audio issues in Skype for Business is different in Office 365 versus the on-premises server product. In the on-premises world, if you’re lucky, the monitoring reports are available to look at the call or conference data. If not, you’ll need to gather client logs from the affected end user and search for the vqreportevent event to look for clues (hope you can sort through XML). Luckily in the Skype Online, this is all made easier using the Call Analytics tool (currently still in preview at the time of this writing). In this article I will cover some key elements of Call Analytics and how to navigate the tool.
The Call Analytics tool can be access via the Tools menu on the left in the Skype for Business admin center or directly using the URL https://adminportal.services.skypeforbusiness.com/analytics. Under the Call Analytics (Preview) menu on the left is User Search. Users can be searched using their name or SIP address and chosen from the result menu list. Once a user is selected, a plethora of information is presented, such as their picture, directory location, and 7-day audio call quality:
The Call History section allow quick identification of calls that had Good and Poor audio quality. This is what sets this tool apart from looking at client logs or monitoring server reports and trying to remember threshold values for good and poor calls. Let’s select one of the calls to Wendy Baker and see why it was flagged as Poor.
The tool makes it pretty clear: there was poor call quality due to network issues. The phone and video icons show that this was an audio and video call between the two users. Below each user are several different categories of information that have been collected: Device, System, Connectivity, and Network. The tools will automatically load the tab that holds the root cause of the poor quality, in this case the Network tab. Here’s what the statistics are:
Several values are color-coded based on their severity, so we’ve got several issues here. The inbound network, or the network connection coming to Gordon, averaged 7 ms of jitter, which isn’t terrible. Skype for Business Online can support less than 30 ms of jitter over any 15 second interval. However, this also shows the call was reaching a maximum of 100 ms of jitter during the call, which is well above threshold. There is some packet loss but well below the 1% required for Skype for Business Online.
The outbound network, or the connection from Gordon out to the Wendy, is where we had the most issues. Jitter had some issues but is below the 30 ms target. Average round-trip time, or latency, to Wendy was well beyond the threshold of 100 ms. This size of latency will cause delays and fragmentation in the speaker’s audio. Another issue here is packet loss. The average packet loss of 12.87% with bursts up to a maximum of 30.06% is more than the 1% target for audio for Skype for Business Online. Packet loss will cause the audio to be distorted or missing. Let’s check out the Connectivity tab and see what kind of network Gordon was using:
Gordon was using Wi-Fi and has a signal strength of 79. Wi-Fi networks are notorious for causing poor audio quality as it can be difficult to configure Wi-Fi to support real-time audio. I would suggest to Gordon to use a wired connection when possible to increase the likelihood of a successful audio call.
You may be asking why the disparity between the inbound and outbound network from Gordon’s perspective. In this case, I was using a tool on Gordon’s system during the call to add latency and packet loss to his outbound stream. Typically I have seen the inbound and outbound network have similar performance metrics. If you do see a disparity, it could mean some network device is affect the stream one way but not another, perhaps a firewall or proxy device.
In addition to Gordon’s network issues, we see that the Device tab for Wendy had a warning. Here’s what is listed for her:
Wendy was using the default microphone (capture device) and default speaker (render device) during this call. Typically this means someone was using the built-in microphone on their laptop and not using a Skype for Business certified audio device. We’ll check Wendy’s System tab in a minute. First, let’s see what values are being flagged here. We see that the Microphone Glitch Rate is 31.00 and Capture DSP Effects was a 7. Microphone Glitch Rate is the average number of microphone glitches per five minutes, ideally there is less than one glitch per five minutes. Glitches occur when the Skype for Business application was unable to have exclusive control of the microphone or device during the call, meaning another application was trying to use it. Capture DSP Effects is a bitmask representing which audio capture DSP effects were used in the hardware. DSP (digital signal processing) refers to the process of analyzing and manipulating an analog signal to a digital. To be honest, I’m not sure why this is flagged as I could not find any information or suggested values for this element.
So now we’ve established the Wendy was not using a Skype for Business certified audio device. Let’s check out the System tab to see what she was using.
In the User Agent row, we see that Wendy is using an iPhone running iOS 10.3.3 and the Skype for Business mobile app version 6.17.5.0000. This makes sense now that we only saw default input and output devices on the Devices tab, although I have seen that the Device tab still reports a default input and output device even when using a Bluetooth headset paired with an iPhone. I have also seen high Microphone Glitch Rates when making calls on an iPhone, which makes sense as a smart phone is probably competing for the microphone resource.
After reviewing the Overview section of this call, we now have a good idea of why this is a bad call. Network issues such as jitter, latency, and packet loss on the Wi-Fi network caused Gordon’s audio to be poor. Wendy may also have had issues with her microphone due to using an iPhone, although I typically see Microphone Glitch Rates when using iPhone even with a certified device. This is where talking with the user about their experience will determine if the glitch rate is an issue.
If desired, you can dig further into the details of the call using the Advanced and Debug tabs found at the top. The Advanced tab is going to give more details behind the values in the Overview tab, such as network statistics, codecs and FEC usage, speech and noise decibel levels, and echo cause events. The Debug tab is show if any events were raised by the device or client during the call. There are many different events that can be raised for a variety of reasons, I would suggest taking this information to your favorite search engine to find out more.
All the above was for a peer-to-peer call. Troubleshooting a conference or mult-party call is no different except there are multiple streams to look through:
We can see that this meeting only had two participants, and one of Gordon’s audio and video sessions was marked as poor. If you click the Session Start Time for the first one that is marked Poor, it will take you to the same screen with information on Device, System, Connectivity, and Network with the difference being that it shows Gordon connecting to the Skype Service instead of connecting to Wendy. All conferences will uses MCUs located in Skype for Business Online.
Looks like Gordon is still using that Wi-Fi network as he experienced jitter, latency, and packet loss during this call.
Let’s go back to the conference overview as I have a small suggestion to make. The default view is a list view that is sorted by the Participant and then the Session Start Time. If you want to get a better idea of how a meeting progressed, above the Audio Quality column is an option to switch to a graph view. I find the graph view easier to see when and how many times a participant joined the conference, if they were using video, and when screen sharing sessions started and stopped. Clicking on the session bars will take you to the details we’ve already looked through above.
As announced during Ignite 2017, the Call Analytics tool will soon include the ability to diagnose Microsoft Teams calls and sessions. I hope this is soon as being able to troubleshoot and investigate poor calls and meetings could be required for some organizations before adopting the platform.
References
NextHop Blog: High Glitch Rate in QoE Report
Office Support: Media Quality and Network Connectivity Performance in Skype for Business Online
