Tuesday, 20 October 2009

Number Normalization – Server Side

In OCS Number Normalization is performed in two places either on the Server if it is a Server activity or on the client if it is a client activity .

During my testing for my previous post around PSTN Conferencing I came across some difference in the way client side and server side normalization works.

These differences mean that translations which would succeed on the client will fail when performed server side.

My intial investigation have focused around a single Normalziation rule, in order to keep things simple.

So first the background, in the UK it is common for PBXs to require a 9 to be prefixed to numbers for outbound dialling so to cater for this I often use the following normalization rule, this allows users to continue to enter numbers as they would have done using a PBX phone:

Existing Normalization Rule

This rule should allow for either:

  • 01134960121
  • 901134960121

To be entered and be translated into:

  • +441134960121

This works as expected in the communicator client as shown in the following two screenshots:

Communicator Showing Translation 0Communicator Showing Translation 90

Route Helper and Test Translation within the Admin Tool both perform the translation as expected as well.

In this test there is only 1 Normalization Profile, which is assigned to the Pool and Mediation Server; there is no User Specific Profile assigned.

With regards to the logging shown during this post we will be focusing on the TranslationApplication log; as this contains the required information; in order to make the logs readable I have removed all but the explanation text; and each line is shown as a bullet point.

Before moving to the Server Side aspect, shown below is the output for when the Communicator Client logs in and downloads the Normalization Profile.

  • OnLocationProfileRequest: SERVICE, reqUri=sip:adam.gent@contoso.com;gruu;opaque=app:locationprofile:get;default
  • Phone context is user-default, using Location Profile assigned to caller URI
  • Using user-default profile: name=UK.contoso.com
  • Found default parameter, need default profile
  • Using default profile: UK.contoso.com
  • Sending 200 response to SERVICE request profile details. xml=<LocationProfileDescription xmlns="http://schemas.microsoft.com/2007/03/locationProfileDescription"><Name>UK.contoso.com</Name><Rule><Pattern>^9?0(\d{9,10})$</Pattern><Translation>+44$1</Translation><InternalEnterpriseExtension>false</InternalEnterpriseExtension><ApplicableForDeviceDialing>true</ApplicableForDeviceDialing></Rule><OptimizeDeviceDialing>false</OptimizeDeviceDialing></LocationProfileDescription>

I have removed all normalization rules from the last log line above apart for the one we are interested in.

Moving onto the Server Side, OCS uses the Translation Application to perform all Server Side Normalization requirements on the Server. For testing I have tested against the PSTN Conferencing (Conferencing Attendant Service (CAS)) and incoming calls into the Mediation Server.

Starting with Conferencing, I have called into the CAS twice from a Mobile (Cell) Phone, the first test is using 01134960121 as the phone number to identify with and the second is using 901134960121 both of these should be translated to +441134960121 as shown earlier with the Communicator Screenshots

The first test failed and the second test worked correctly as can be seen in the log outputs below:

Input: 01134960121 Expected Output: +441134960121 Actual Output: No Match

  • OnRequest: INVITE, reqUri=sip:+441134960100@contoso.com;user=phone newHostPart=
  • Processing From URI: sip:+447700900621@contoso.com;user=phone
  • Global number, no further From URI processing. FromUri=tel:+447700900621
  • Global number, no further processing. RequestUri=sip:+441134960100@contoso.com;user=phone
    calledNumber='01134960121'
  • No matching rule found

Input: 901134960121 Expected Output: +441134960121 Actual Output: +441134960121

  • OnRequest: INVITE, reqUri=sip:+441134960100@contoso.com;user=phone newHostPart=
  • Processing From URI: sip:+447700900621@contoso.com;user=phone
  • Global number, no further From URI processing. FromUri=tel:+447700900621
  • Global number, no further processing. RequestUri=sip:+441134960100@contoso.com;user=phone
    calledNumber='901134960121'
  • ruleName='PSTN Access UK - 0, 90' matched Num='901134960121', txNum='+441134960121'
  • MATCH: New request Uri='sip:+441134960121@contoso.com'

This shows that the Translation Application does not understand the optional part of the Regular Expression.

In order to see if this just occurs with the CAS, I did the same tests by providing 01134960121 and 901134960121 as the Called Number in an incoming call to the Mediation Server. Again the first test failed and the second test worked correctly as can be seen in the log outputs below:

Input: 01134960121 Expected Output: +441134960121 Actual Output: No Match

  • OnRequest: INVITE, reqUri=sip:01134960100;phone-context=UK.contoso.com@contoso.com;user=phone newHostPart=
  • Processing From URI: sip:+447700900621@contoso.com;user=phone
  • Global number, no further From URI processing. FromUri=tel:+447700900621
  • calledNumber='01134960100'
  • No matching rule found

Input: 901134960121 Expected Output: +441134960121 Actual Output: +441134960121

  • OnRequest: INVITE, reqUri=sip:901134960100;phone-context=UK.contoso.com@contoso.com;user=phone newHostPart=
  • Processing From URI: sip:+447700900621@contoso.com;user=phone
  • Global number, no further From URI processing. FromUri=tel:+447700900621
  • calledNumber='901134960100'
  • ruleName='PSTN Access UK - 0, 90' matched Num='901134960100', txNum='+441134960100'
  • MATCH: New request Uri='sip:+441134960100@contoso.com;user=phone'

Having established the same issue occurs for both the CAS and Mediation Server, I wanted to see what happened if I entered simple Normalization Rule to match 01134960100

In order to do this the following Normalization Rule was added and it was placed in the order after the existing rule.

The screenshot below shows the new rule:

New Normalization Rule

And the screenshot below shows the rule order:

Translation Ordering Location Profile

Running the two tests that failed again, 01134960100 for both the CAS and the Mediation Server both worked as expected although against the new Normalization Rule.

These are shown below:

Conferencing, Input: 01134960121 Expected Output: +441134960121 Actual Output: +441134960121

  • OnRequest: INVITE, reqUri=sip:+441134960100@contoso.com;user=phone newHostPart=
  • Processing From URI: sip:+447700900621@contoso.com;user=phone
  • Global number, no further From URI processing. FromUri=tel:+447700900621
  • Global number, no further processing. RequestUri=sip:+441134960100@contoso.com;user=phone
  • calledNumber='01134960121'
  • ruleName='PSTN Access UK - 0' matched Num='01134960121', txNum='+441134960121'
  • MATCH: New request Uri='sip:+441134960121@contoso.com'

Mediation Server, Input: 01134960121 Expected Output: +441134960121 Actual Output: +441134960121

  • OnRequest: INVITE, reqUri=sip:01134960100;phone-context=UK.contoso.com@contoso.com;user=phone newHostPart=
  • Processing From URI: sip:+447700900621@contoso.com;user=phone
  • Global number, no further From URI processing. FromUri=tel:+447700900621
  • calledNumber='01134960100'
  • ruleName='PSTN Access UK - 0' matched Num='01134960100', txNum='+441134960100'
  • MATCH: New request Uri='sip:+441134960100@contoso.com;user=phone'

While breaking down the Normalization rules into simple rules is one option, it is far from ideal as it create more rules and potentially more confusion; considering the rules work as expected client side.

Since I have only looked at a single Regular Expression which focuses around an optional character either there is an issue with the optional character within the Translation Application or there are issues with other regular expressions.

In summary just because every OCS test tool says that the normalization rule works and it works in Communicator do not assume that it works server side. Hopefully this will be fixed so that the same rule produces the same output no matter if it is run client or server side.

It would also be useful for another test tool that allows for testing against the Translation Application.

The test platform was OCS 2007 R2 Enterprise Edition running QFE2.

Tuesday, 13 October 2009

Follow-up July Updates Dial In Conferencing

A few months ago I posted about the changes in Dial In Conferencing following the July Updates also knows as QFE 2; which can be found here.

Since this posting Brian Ricks pointed out that if the Tel URI has the extension specified in a users OCS Account Settings this will be used to match against when authenticating in Dial In Conferencing; it also populates in Communicator Web Access. An example of the Line URI setting can be seen below:

Line URI Example

Someone then asked the question of what happens if someone has the same extension such as:

  • tel:+12223334444;EXT=4444
  • tel:+12225554444;EXT=4444

While OCS will prevents duplicate phone numbers it will allow duplicate extensions.

This intrigued me, so I started to look in more detail of how the conferencing login works and the associated activities such as setting a PIN.

The first step was to configure two users with the different E.164 numbers but with the same extension and one user with just an E.164 number, these were:

  • User A: tel:+12223334444;ext=4444
  • User B: tel:+12225554444;ext=4444
  • User C: tel:+12226664444

The next step was to set a password on each this is done via CWA; the system I am testing this on is set to allow passwords with a minimum length of 4 and to keep things simple I opted to set a password of 9898 for each account.

This worked for the first account, for the second account I received the error shown below.

Set PIN - Error

Knowing that the password does comply with the policy and trying for a second time, I tried a different PIN (7878) and this worked correctly.

Therefore it appears the answer to the question of if multiple people can have the same Extension assigned, they can not have the same PIN resulting in OCS being able to match an Extension and a PIN uniquely to a user account.

The problem I see with this is that with an error message of:

“The PIN you provided does not meet your company’s policy requirements. Please create a new PIN.”

Users are likely to try the same PIN a couple of times and then call the IT Support team; if the error message indicates that they had to choice a different PIN users a more likely to do so.

So now that we know how OCS keeps users unique, lets take a look under the hood.

During this we will be focusing on the UserPinService log; as this contains the required information; in order to make the logs readable I have removed all but the explanation text; and each line is shown as a bullet point.

In addition each call to the PSTN Conferencing is done using a standard phone, rather than using an OCS Endpoint. This ensures that Communicator or Phone Edition does not automatically authenticate.

First lets look at how the User PIN Service locates the user to do the PIN matching against; the logs below show me logging in with an extension of 4444.

  • Error Translating Number: No matching rule has been found in the dial plan for the called number.
  • ResolveUser Sproc - Phone Number = 4444, NPH1 = +4444 NPH2 = 4444 Phone Ext = 4444 MaxResults = 200

From the first line it is possible to see that there is no normalization rule in the location profile assigned to the Pool that matches 4444.

The User PIN Service then goes to look for a Tel URI that matches 4444, +4444 or an extension that machines 4444; this is shown in the second log line above.

If a user(s) is found the logs will show something similar to the below:

  • Found [1] pools to search across.
  • initialized with [1] branches and [2] users.

In this case two users were found and the User PIN Service continues to try and match the entered PIN against one of these two users.

Expanding on this I added a new Normalization Rule to the Profile assigned to the Pool, this contained the following:

Phone pattern regular expression: 4444

Translation pattern regular expression: +12226664444

This resulted in three users being located by the User PIN Service, which is shown below:

  • ResolveUser Sproc - Phone Number = 4444, NPH1 = +4444 NPH2 = +12226664444 Phone Ext = 4444 MaxResults = 200
  • Found [1] pools to search across.
  • initialized with [1] branches and [3] users.

In addition by configuring two additional users with a Tel URI of 4444 and +4444, the search result increased to 5; these were then removed to avoid further confusion.

It is also worth pointing out that the exact same routine would happen if a person entered their full phone number:

  • Error Translating Number: No matching rule has been found in the dial plan for the called number.
  • ResolveUser Sproc - Phone Number = 1222333444, NPH1 = +1222333444 NPH2 = 1222333444 Phone Ext = 1222333444 MaxResults = 200

Looking at what happens when the PIN entered is invalid, log entries similar to the below will appear:

  • VerifyUserPinBatchLocal - Succeeded, calling back with [3] results.
  • VerifyUserPinBatch succeeded for Pool [OCSR2.contoso.com]
  • BranchTransaction for Pool [OCSR2.contoso.com] is now complete.
  • branches pending [0]. Results so far [3]
  • no users matched the supplied pin.

If the PIN entered matches a user, log entries similar to the below will appear:

  • VerifyUserPinBatchLocal - Succeeded, calling back with [3] results.
  • VerifyUserPinBatch succeeded for Pool [OCSR2.contoso.com]
  • BranchTransaction for Pool [OCSR2.contoso.com] is now complete.
  • branches pending [0]. Results so far [3]
  • found one user [usera@contoso.com] with valid account/pin from pool [OCSR2.contoso.com].

The next aspect to look at is the PIN Change process. If the user tries to enter a PIN which is already assigned to a user, log entries similar to the below will appear:

  • Succeeded, calling back with [3] results.
  • VerifyUserPinBatch succeeded for Pool [OCSR2.contoso.com]
  • BranchTransaction for Pool [OCSR2.contoso.com] is now complete.
  • branches pending [0]. Results so far [3]
  • found two matches. User1 [usera@contoso.com] User2 [userb@contoso.com]. Failing request with pinNotUnique.
  • ResolveUserWithPin call failed with [PinNotUnique] verification code [OtherFailure]

If the PIN Change is successful, log entries similar to the below will appear:

  • Succeeded, calling back with [3] results.
  • VerifyUserPinBatch succeeded for Pool [OCSR2.contoso.com]
  • BranchTransaction for Pool [OCSR2.contoso.com] is now complete.
  • branches pending [0]. Results so far [3]
  • no users matched the supplied pin.
  • ResolveUserWithPin call failed with [OtherFailure] verification code [OtherFailure]
  • Need to increment auth failure for all users
  • Processing RecordFailureBatch sproc.
  • Deserializing request: Microsoft.Rtc.Server.UserPin.Private.updateuserpintype
  • Processing SetPin sproc.

The last line shown above shows that the PIN is being Set, the failure notices can be safely ignored these are the User PIN Service checking to see if another user with the same Extension is already using the PIN.

Now that we have gone through the main processes; it is worth looking at the issues that occur with multiple users having the same Extensions; either directly specified or through the normalization process.

As we have seen the User PIN Service searches Active Directory based on 4 criteria and pulls back records matching 1 of those 4 criteria; in addition it limits the maximum number of returned rows to 200. While this for the majority of installation will not be an issue for large installations looking to move from 100’s of PBXs to OCS issues could occur.

The other issue being the more common will be users receiving an error while changing their PIN; while a 4 digit PIN allows for 6561 combinations, not taking into account the PINs blocked by OCS for being too simple, users are still creatures of habit and try to keep things as simple as possible. Therefore it is likely users will try PINs like 1212 or 9090.

The final thing to consider is what happens if two people end up with the same PIN and the same extension. In order to get two users with the same PIN, required a bit of trickery but it is something that can happen within a live environment.

I removed the 4444 normalization rule and then configured User B and User C with an PIN of 1819. OCS allows me to do this due to 4444 no longer translates to +12226664444. I then added the 4444 normalization rule back in.

This results in two users which will be matched with an Extension of 4444 is entered and PIN of 1819. Although two users are matched  OCS handles this by refusing access to the conference since it does not know which user is trying to access the conference. This is shown in the logs as:

  • found two matches. User1 [userb@contoso.com] User2 [userc@contoso.com]. Failing request with pinNotUnique.
  • ResolveUserWithPin call failed with [PinNotUnique] verification code [OtherFailure]

It is likely that this scenario will only happen is you are utilising the extension setting in the Line URI and you modify the the Pool Normalization Profile after users start setting their PINs. This could happen if additional PBXs are brought online which were not catered for the initial Voice Plan design. The only solution would be to remove the Normalization rule or get all users to reset their passwords.

The other important part of Dial In Conferencing is the Conference Attendant (CAA) and Conference Announcement Service I stayed away from pulling logs from the Conference Attendant in an aim to keep this as simple as possible. For those of you wondering why a user is prompted for a PIN if the Extension is invalid this is due to the CAA capturing both the Extension and the PIN before passing it into the User PIN Service.

I will aim to review the Conference Attendant (CAA) and Conference Announcement Service in a later post.

One thing to point out that this has all been tested on a system with only a Single Pool; how the scenarios alter with Multiple Pools I am unsure.

And the final point is that during the testing for this post I found some strange behaviour in the Number Normalization, I will follow this up in a separate post.