Cisco Administration is Tough

I had to do a few adjustments to the phone system. The Cisco administration is an unwieldy beast. I know non-technical people who find Avaya much easier when dealing with account adds and removes.

My first task was to setup a new phone. I wasn’t assigning this phone to anyone, staff will log into and out of it as needed. So I logged into Cisco Unified CM Administration and copied an existing phone configuration and entered the phone’s MAC address. But that wasn't enough. I also needed to assign an extension to this phone, even though no one was going to be using this extension.

Since there wasn’t an unused extension to assign to this phone I had to create an extension. The best way to create an extension is to copy an existing one and then edit the important fields.

Once I created the extension I then returned to the phone configuration and assigned the extension. Done.

None of this was rocket science, but you need to know the steps involved. The administrative software does not walk you through this common function.

I also needed to delete a user account. Since this is UM 9.x I had to convert the user to from a LDAP Synchronized User to a Local User. This made sense. Previously, in UM 8.x and earlier you would remove the user from the LDAP master, be that AD or OD or some other LDAP server, do a full LDAP sync and the user account would disappear. That is no longer the case. You cannot delete a LDAP Synchronized User until they are converted to Local User, but you cannot do this conversion if this user failed to sync the last time LDAP sync was performed. So be sure to convert the user to a Local User before removing them from the LDAP server.

I decided to keep the devices assigned to this user since I will assign them to a new user in the future. But I needed to remove her voicemail account. This involves logging into another server, which has a different interface. In this case I needed to remove the user or her phone extension from the LDAP directory, sync the voicemail system with the LDAP server and then delete her account from voicemail.

I now see why the Cisco phone customers I talked with let their phone intergrators handle Adds and Removes. The processes aren't difficult but they involve multiple steps and the administrative interface doesn’t provide a guide.

Cisco needs to improve the administrator experience.

Memories

A few photos of our old AT&T Definity 75 PBX.

This shows one type of complexity that I no longer deal with, the tracing of wires. Every phone was on 1 or 3 pair of wires that had to go directly into the PBX. And every outside line also had to go to the PBX. But you don't connect wires directly to the PBX, they connect to the patchboard which in turn connects the wires to a specific PBX port. This makes it easier to manage since the ports are marked on the patchboards. Unfortunately the markings are not consistent and old wires are not removed. While logical I always found this to be a challenge when I had to move wires around, such as when a location switched between a digital and an analog phone. I won’t miss it.

Record all changes

We accidentally shut down the WindStream router. No big deal, I think, just start it up again. Well, it isn’t talking with our Cisco router. Big Deal.

I called CWPS for help, and after checking the basics they suspect that the router restarted with an old configuration that didn’t include the modifications made on the day of installation. They were right.

But it took way too long to fix. WindStream had to check the circuit and then give the case to the router support group. Everything seemed fine to them, the settings matched their records. Unfortunately their records didn't include changes made on installation day. That mistake cost us around an extra 4 hours to get it repaired. Even after CWPS gave them the updated information it took over an hour for WindStream to make those changes.

Records have been updated and I requested that the new configuration be saved in the router to the settings will  survive a reboot.

A co-worker has suggested that the problem is that Mercury is in retrograde and some people find that their phone lines go down.

And some days you can’t win [updated]

Cisco Jabber search hates large Address Books

I was setting up a user with Cisco Jabber today and he has a huge huge Address Book. Searching on a name takes around a minute.

Dialectic hates me

Dialectic worked great with my testing on January 11. I bought a few licenses today and it is now having problems authenticating to our Cisco UM, so it cannot control our phones. Working on it.

Update: A coworker got Dialectic to work on his MacBook. He went to miscellaneous preferences and turned on proxy server authentication and entered his OD username and password. Still not working for me, but this indicates that it should work. In addition Jon Nation of JNSoftware responded to my e-mail on this issue very quickly and provided information on how Dialectic works with Cisco UM.

Another Update: Dialectic is now working on my Mac after playing with the proxy preferences preferences some more. But I don’t know what I did to make it work and am now trying to duplicate my success on a 3rd Mac. But this makes me feel hopeful.

Jabber for the Mac

Cisco’s Jabber for the Mac successfully integrates the Macintosh with Cisco’s phone systems. Jabber can be used to control your Cisco desktop phone or act as a soft phone. It provides access to your Address Book as well as your Cisco UM directory. Click here to read the review.

Jabber integrates phone, chat, voicemail and other functions. While there are some shortcomings, I had some troubles at times with the contact search function, in less than a week some of our staff are using their Macs with Jabber and iPhone earbuds instead of their phones to make and place calls. Even if you aren't interested in using Jabber as a software it is the best way to listen to voicemail messages and place calls.

As of UM 9.0, the Jabber client is included with the standard license, so trying it out is a no brainer. Jabber only works with the Cisco UM phone system.

They like it, they really like it

My staff is happy with the new Cisco BE6000 phone system. Everyone has a 8961 phone and many are using Cisco Jabber to enhance their phones. And if my staff is happy, I'm happy.

The biggest win is that people found the phones easy to use. Many people were able to transfer calls and do conferences after reading the 2 page starter guide and playing with the phones. Listening to voicemail and dialing via Jabber is well liked. Keep in mind that we are jumping 20 years of technology from our previous phone system.

The staff did benefit greatly from training by CWPS.

Here is a list of observations.

Everything is great but the transfer delay

I’m not sure why but many people are seeing a delay when calls are transferred to them from the operator. They pick up the phone for 2 to 6 seconds of silence. I know this isn’t normal so I’ll be looking into the cause. Could be wiring (we still have lots of Cat 5 from over 15 years ago), or overlapping extensions, or some other network issue.

I wish Cisco Jabber could do more

Jabber is great for listening to voicemail but you have to go to your voicemail web page to download a message or recording as a .wav file. I have been told that Cisco will be adding more capability to Jabber.

Cisco Jabber works fine on the Mac

Be sure to set the preferences for Cisco Jabber for the Mac to integrate it with Address Book/Contacts. It doesn’t integrate with Outlook for the Mac but if the Outlook user stores their contacts on their mail server, such as as MS Exchange or Kerio Connect, then Address Book/Contacts will have the correct information to pass onto Jabber.

The web interfaces need improvements and consolidation

Users go to one page for presence options, another for voicemail and a third for UC setup. The multiple web interfaces need to be consolidated or migrated to apps. My impression is that Avaya's One-X interface achieves this goal. Also, the voicemail setup interface requires Java in the web browser, something I am trying to eradicate in my office.

Administration needs to be easier

Again, rethink the interface and consolidate to a single interface even though there are 3 servers. I would hope that Cisco could devise a single interface for administrating most functions.

And some functions are very awkward. Want to upload a photo to display on your phones? Edit an xml file. Upload the photo and a small version of it via TFTD. Then go to another interface to restart the TFTD server. WTF?

But it is much better than our old AT&T System 75r3. And I am more concerned over the average user’s interface than the administrator module. 

Reach Me Anywhere works great

It is much easier for anyone to reach me by setting the PBX to ring my cell phone if I don't answer my desk phone after 4 seconds (2 rings). And then hang up on my cell to put the person to my office voicemail if I still don’t pick up the call. You can get a lot more sophisticated but it isn’t necessary for most people. Please I can schedule Reach Me Anywhere so calls are not forwarded to my cell on weekends or during non-business hours.

Most phone systems have a forwarding feature similar to Reach Me Anywhere but some systems do not include the scheduling feature.

Cisco is moving forward

We purchased version 9.0 but by the time we went live it was upgraded to 9.1.1. Version 10 is expected this year. While our integrators only recommend upgrades if an upgrade provides an important feature or fix—if it ain’t broke don’t fix it—it is good to know that Cisco is aggressively developing this product. But some upgrades can cause problems, we had to downgrade the firmware on the receptionist phone in order to get it to work properly with the sidecars.

Cisco is depending on VM for the servers

We have a single appliance from Cisco that is running the 3 servers that make up the PBX. The next version of Cisco’s UC, version 10, will only work as VM, which means that offices currently running earlier version of UC on rebranded IBM and HP servers will find upgrading to 10 to be more challenging than earlier upgrades. Cisco uses a special version of VMware’s Foundation.

I should have double-checked the new phone circuit

I found out yesterday that with our codec our new T1 will support only 18 concurrent calls with the highest quality codec. I was told by our phone agent to expect 32 concurrent calls, but that would be with a lower quality codec. So perhaps we should have gone with a T1 PRI after all. So we may end up adding a 2nd T1 to bring us up to 36 concurrent calls. On the bright side this would all be on the carrier side and require changes only to the carrier’s router, the PBX will accept this 2nd T1 without having to add new cards. If I did some more homework on codecs before signing contracts I might have selected a T1 PRI connection.

And speaking of codecs

The default codecs on the Cisco system for internal calls is tinny. We quickly had it switche to the highest quality codec. This made a big difference in voice quality.

In sum, I am happy with our choice of Cisco and CWPS. While I will be critical about this system, it is a huge advance for our office.

The new system is live

I've been so busy working on the new phone system that I haven't had time to do much posting or any deep review. I will do more writing as life slows down.

I spent today putting a new phone on everyone's desk. Windstream started porting the numbers from our old circuit to our new one around 4pm. A representative from our phone agent, ARG, and our PBX installer, were involved with this step. Our new circuit has been active for 3 weeks, so we have been making outgoing calls over that time and staff with new DID numbers were able to receive calls. All calls come in on our new circuit and the old one should have been killed by now.

We did a fair bit of fine tuning today and we will have to do more over the next 2 days. But the bulk of the work has been completed.

The best part of the day was when an editor told another editor how easy it was to look up the staff directory on the phone and do various tasks. I am pleased how easy it has been for people to take advantage of the phone's features with just a 2 page quick start guide and some exploring.

Everyone I show the Cisco Jabber app are excited by it. When we did internal surveys 6 months ago no one was interested computer telephony or having office visual voicemail on their iPhones. Now that people see these features in action they are interested in them.

Tomorrow morning I'll be removing old handsets, do some tinkering and take the official user training.

Anyone interested in a AT&T Definity 75r3 system? It is free for the taking.

Finally Fax

We can receive faxes again.

The provider for our 3 analog lines finally finished the work.. We don't get many faxes but the loss of our incoming fax lines probably cost us a few ads for the March issue. The provider claims that the order was improperly made, I find that hard to believe, and their response to complaints was very slow. A Verizon tech came to finish the job yesterday (he didn't know what was the problem, he was just told that the lines weren't working) and Chris from ARG came out so he could work with the tech. Everything is now working.

As I thought our use of fax was dying we've seen a sudden increase in our use of faxing.

We have a 4-Sight Fax server. Some sales reps are contacting many doctor offices and they are faxing them since doctors prefer faxes over e-mail. These reps have e-fax accounts but wanted a way to just enter a list of doctors and have faxes with customized cover sheets go out. With the e-fax they would have manually crafted a letter to each doctor. I created a cover letter in 4-Sight Fax for this project. The subject line is fixed so they don't have to enter it every time. They can enter multiple doctors into the Fax Client and off go the faxes with the cover letter and a 1 page ad with a minimum of effort. Perhaps this could have been done more smoothly with e-fax but it was a rush job and I didn't have time to look into the e-fax capabilities.

Our classified advertising manager was asking about digital faxes last week. Rather than set her up with an e-fax account I am having all faxes that come to our 

4-Sight Fax server be routed to her via e-mail. 4-Sight Fax could route incoming faxes to appropriate e-mail accounts based on the phone number dialed.

Almost Live

Our Cisco system is almost live. This is very exciting.

The PBX is now talking with our Open Directory again. Test users can sign on Cisco services using the web and use Cisco Jabber client. I'll be writing about Jabber as soon as I have some time. I find it exciting and useful.

I was surprised that our new DID numbers are working for incoming calls. We haven't switched to the new circuit yet but we have been able to use it for outgoing calls. But someone told me that they received a call on their new phone. So we did some testing and found that staff who have the new DID numbers and extensions can receive calls from outside the office. Staff who have DID numbers can only receive calls on their old phones until we officially switch circuits on Feb 20th. But this will allow for more testing and system improvements prior to going live.

The hardest thing to get staff testing the new phones to understand is that our new PBX doesn't talk to the old one. Calls coming in to the receptionist will be transferred to their old phones. That if they want to call another staff member they need to use their old phone. Thankfully our testers will only have to live with 2 phones on their desks for a few days.

I am working on articles about Cisco Jabber for the Mac and the Cisco 8961 phones.

Some Progress

I think our fax lines have been moved over to the new analog line vendor yesterday. The 3 lines were down for a short amount of time and now when I make an outgoing call the Caller ID says "DC" instead of "Washingtonian."

Windstream and CWPS are supposed to arrive at 11am today to install the Windstream router and hook it up to the Cisco router and test it out.

And then wait a week to start getting the PBX all ready to go live. This will be a time of fine tuning and more learning for me. And then go live 4pm on Feb 20.

Today's No Progress

An installer came from the analog phone line company (note that I don't mention them by name). He checked the basement phone closet, no lines are tagged for the numbers we will be using. Checked the second floor phone closet, he didn't see extensions that were supposed to have been tagged by the tech who came 2 weeks ago.

He called his company and told them to check with Verizon to see if they did their bit. My guess is no, Verizon didn't receive an order to put in the lines. The tech said he ordered a rush job so we will have working fax machines in 3 weeks when we cut our analog T1 lines.

Progress

The Phone

I have been enjoying the phone on my desk. So nice to pick up the phone or press the speaker phone button and see a list of recently dialed and received calls. So nice to have more than a one line LCD screen.

The Carriers

Working with the carriers has been the most frustrating part of the new phone system. We are using 2 carriers, one for our 3 analog lines (fax & fax modems) and another for our SIP trunks. I don't deal with the carriers directly, we have a phone agent, ARG, which works directly with the carriers. Considering the problems encountered and steps required, I am happy to be one step away from the carriers.

The first carrier sent us bill which included charges for the December and installation charges. The problem is that they didn't install the new lines yet. So we won't be paying that bill.

I complain to ARG and then the carriers sends someone to do the installation work. He can't find any free wires going from the building's 2nd floor phone closet to the basement. I am told they need to send over 2 people with a concrete drill to install new lines. Lots of $$$.

I contact ARG about the situation. They send their own engineer and he finds lots of unused wires between the 2nd floor and basement phone closets. He even finds wires that connect to our server room so we are almost all set for the SIP trunks. So I don't need more wire installed.

Late last week WindStream, our SIP trunk provider, sends over their contractor to get the new circuit pulled up from the basement phone closet to our server room. He uses the existing wiring and does the job quickly.

We are on track for going live on Feb 20, 4PM. We will have new phone training Feb 21.

Here are some events leading up to the happy day:

Tue Jan 29: Analog phone carrier will send someone again to complete the wiring of new lines to fax machines and modems.

Tue Feb 5: Switch over fax lines to new carrier

Wed Feb 6: Windstream will complete activation of  the new SIP trunk (don't you love these tech terms) and CWPS will connect the SIP trunks to the new PBX. This is a key date. It means that the new phones connect to the outside world.

Between Feb 6 and 20th I'll be working with CWPS to do final configuration of the new system. This includes assigning new extensions to everyone with a 20xx extension. I hope to be reporting on fun stuff here.

Wed Feb 20, 4pm: Switch our lines to the new circuit. We will put new phones on staff desks either earlier in the day or the day before. Once the lines are switched we will remove the old phones. We will keep the old PBX and one phone working so staff can check on old voicemail.

Thu Feb 21: Phone training.

New Go Live Date

Some of the fun with new phone systems are changing target dates.

WindStream delayed the circuit install date from Jan 10 to Jan 18 because we added a phone line to be transferred to the circuit, bumping our order to the bottom of the list. Phone companies, 20th century customer service, 

In addition the secret service might not allow Verizon to do its work due to the inauguration. And they aren't sure if there are phone wires that go from the floor's phone closet to the building's basement phone closet. So we are having professionals come check out the wiring this week or next week. *sigh*

So CWPS engineers are book for the following 2 weeks and suggest the week of Feb 4 for going live. This is too deep into our production schedule and I am concerned that our staff will not have time for training, so we need to wait until we ship our March issue to the printers.

So the new go live date is 5pm Feb 20 and then training for everyone Feb 21.

Dialing From My Mac

Working on a Mac can be more challenging than from a Windows computer. Microsoft Provides TAPI, Telephony Applications Programming Interface, and most phone vendors take advantage of it. TAPI is the standard interface between Windows and phones. Apple doesn't offer an equivalent.

Fortunately there are a few products provide this functionality. I am currently testing Dialectic and Phone Amego. Both programs add a menu to the Mac menubar to quickly make calls. Both provide plugins so you can dial directly from Address Book/Contacts or other programs such as DayLite.

They both work with Cisco phone systems. But they share a common weakness with Cisco phones, you need to enter the phone's IP address in addition to the user's name and password. While I expect phone IP addresses to be stable since they are on for long periods of time, when the phone’s IP address changes then Dialectic or Phone Amego settings would need to be updated. I gather in the Windows world the dialer application would talk directly to the PBX and do not need to know the target phone's IP address.

But with that said, both programs succeed at automating phone dialing. They offer functionality beyond that but I have not yet explored those functions. I am looking forward to how they handle incoming calls but that will have to wait until we go live at the end of the month.

Phone Amego: http://www.sustworks.com/pa_guide/index.html#.UPBQ0KWP0g8
Available from the developer's stie and Apple's App Store for $29.95 for a single license. License packs are available at discounted prices.

Dialectic: http://www.jonn8.com/dialectic/
Available from the deleloper's site for $25 for a single license. License packs are available at discounted prices.

If you use either product or use another one please post a comment.

New Dates and Working Through Problems

I just got a new date for circuit installation, Jan 18. That should give us enough to to go live before Feb 1. On glitch is that the new DIDs will not be solid until the circuit is delivered which means I will not know the new extensions for around 50 people until then. And it puts further work on user setup and LDAP synching until then. But I think I can start testing Mac programs for letting Address Book/Contacts dial my phone.

We've been having problems with remote access for 2 servers: Kerio Connect (e-mail) and DayLite (crm). Both look like firewall rule issues.

DayLite is fixed. I got the Cisco ASA administration program today and saw that there was no rule allowing packets to go from the Internet to the DayLite server. Simple entry error, the wrong server was entered. Changed the entry and all is happy. Yeah.

We have been having problems with remote users not being able to send e-mail. We restrict all SMTP connections from outside our LAN to be secure SMTP. Secure SMTP uses port 465. I think, by default, Mail.app tries to use port 587. Since Mail.app doesn't get a response from the mail server saying "nope, port 587 is closed for" Mail.app doesn't know to switch to port 465. Or that is my guess. So why doe mail on an iPhone, which has the default port of 587 for secure SMTP, have no problem sending e-mail. So we will let traffic for port 587 reach the mail server. That should solve that problem. Or we can have people manually change Mail.app to only use port 465 for the office mail account. The latter seems to be a lot more work.

So for the next week I will be focusing on non-phone responsibilities, and then go whole hog on phones.

Quick Update

VPN via the ASA is all done. There was a problem with a Mac getting the internal DNS servers with Cisco AnyConnect but a modification to the ASA configuration fixed that. More importantly, staff will have VPN access only if their user account in Open Directory is a member of the VPN group. Joy.

DigiCert took only 15 minutes to send us a security certificate for the ASA so people can start connecting remotely.

So life will get a little simpler for me. We've eliminated a separate directory. Hooray.

Now to get the 2 analog lines hooked up to the router. Still waiting to hear about circuit schedule.

And just got a bill from Granite for our 3 fax lines that we moved to them. I don't remember them arriving to install the lines nor do they explain all of the charges on the bill.

Firefighting

This is data network day. We removed the SonicWall and replaced it with the Cisco Router and ASA firewall. It's been an interesting morning. This doesn't have to directly have anything to do with phones, just the result of changing network architecture.

The biggest difference is how the DMZ, or demilitarized zone, is handled. We have 3 public servers in the DMZ. The SonicWall and ASA handle DMZs very differently. Using the SonicWall, computers in the DMZ had public IP address. The ASA creates a subnetwork for its DMZ and you have to create NAT to connect the DMZ address. In short, I need to change the IP address of every computer in the DMZ. And then update the internal DNS. No need to worry about external DNS servers since the external IP addresses remain the same.

One temporary complication is that a computer may have the old DNS entry cached. On a Mac you can either restart it or type the following in the terminal:

dscacheutil -flushcache

In some cases dealing with these IP changes can be a pain. Our copier needed to have LDAP and SMTP addresses updated since these servers are in the DNS. Our main mail server had to have a new range of IP addresses added for accepting Mail. And our backup mail server had to have new entries for domains it sends mail to (this is limited to our main mail server). When moving computers to new IP names there is an increased chance of something going wrong because you forget to fix something.

Another change is remote access. The ASA is able to authenticate login against LDAP. The CWPS engineer is looking into a method so we can set who may and may not log in remotely. If we had Active Directory (AD) this would be no problem. LDAP is trickier. If we can't define who may and may not we may just break the connection between the ASA and OD and just enter authorized remote users separately, as we did with the SonicWall.

Which brings me to a gripe about Cisco and LDAP and Open Directory. They provide great support for AD, as they should. But LDAP is the step-child. It should work, but if it breaks don't count on Cisco for support. I am so happy CWPS is trying to figure all of this out for me.