Powershell, BES, and the GAL

One of the things that I love about IT is solving problems. It's a great feeling when you figure something out that you've been working on for a while. Especially when it's something that you are not really familiar with and you really have to dig into the subject to try and resolve the issue.

Then there is the dark side of IT. We've all been there, attempting to solve a problem on a subject that you are not familiar with, and trying to do it under a time constraint along with learning the in's and out's of the problems you are trying to solve. I recently "resolved" a problem at work much like this.

A little background. At work, we are moving to Blackberries and BES (Blackberry Enterprise Server), away from Treos and Goodlink. And we upgraded to Exchange 2007 in the middle of 2008. Everything about the BES server is great...except for the fact that you don't get a GAL (Global Address List) locally for quick finds. Instead, the blackberry has to query the GAL on the Global Catalog, and that can take sometimes take up to a few minutes. I can understand why RIM doesn't want to push the GAL to the phone by default. A lot of companies that use BES are large companies that have multiple GALs, and more than likely they are very large GALs. That can eat up a lot of memory on the phone. It would be nice if they gave you the option of pushing the GAL to the phone (I'm not sure if this is a new feature in BES 5.0 that's going to be released soon or not, I haven't had time to do much research).

Since we'd gotten a few complaints about this, I was tasked with resolving this issue (especially since the users had the GAL on their Treos). The first thing that I looked at was using EWS (Exchange Web Services, which allows developers to hook directly into Exchange to pull/modify/add items, etc). I am definitely not a developer, so most of the EWS stuff was going right over top of my head. Because I was also knee-deep in multiple other projects, I knew if I wanted to do this, I'd need to find a different approach.

The approach I took was definitely not the best, not the prettiest solution, but it's a solution none the less, and I'm going to share it with you now.

I ended up using Powershell to do a bulk of the work. Powershell is not able to plug directly into mailboxes themselves without an extra utility to invoke EWS. I was lucky enough to find someone who did a bulk of the work already, and I'd like to give a big shout out to Glen Sclaes for creating a EWS Powershell Utility. One of the scripts he posted also did a bulk of the work for creating the contacts. I was able to take the scripts that he had created and modify them to do the work that I wanted them to. (NOTE: His scripts had a way of synchronizing the contacts in a folder, but I was never able to get it to work correctly, mostly because I'm still not able to fully understand what's going on in EWS). I also had to create a user that will be used solely for the purpose of creating the GAL, and exporting the folder. This will be clear why later. You will need to make sure you create the sub-folder under contacts (or perhaps create a top level folder, it's up to you) where the GAL will be created. Create this folder in the mailbox account you created. If this is not there, the script will fail.

Here's a basic overview of my script, and I've attached it to this post so you can download and use/modify as you see fit.

The first line of the script will add the EWS utility to the script. Once that's added, the first task of the script is to create my "GAL". I couldn't figure out how to pull the GAL using Powershell, nor using EWS, so I had to create my own. I used Powershell to generate the lists I needed into .csv files. I exported the users, our resource mailboxes (equipment and conference rooms that can be checked out) and our distro lists.

The next block of code sets the variables needed to be used during the remainder of the script. It gives the URL to your CAS (Client Access Server portion of Exchange 2007), which is where the Exchange Web Service plugs into the mailbox database to make edits. It also allows you to set the email and SAMAccountName of the mailbox used to create the GAL, and sets the paths to the folder.

The script then runs through each of the .csv files, importing each line into the GAL creation mailbox. Currently each of the .csv files run separately, which means that there are 3 import-csv calls, each with all of the EWS calls that are needed. Currently, the contacts are created with name, email address, mobile phone number, and work phone number (set to the main phone number). It is easy to add new fields, but these were the only ones we were concerned with.

Once created, the GAL will be exported using export-mailbox. This is why I suggested creating a single mailbox account for this. The export-mailbox command will pull the GAL folder out into a .pst file, and after it is exported, it will delete all of the content. Another important reason why you use a single account for this is because export-mailbox has a quirk that can really bite you.

When you delete items in Exchange, they are not deleted completely by default. Deleted files, after they've been shift+deleted, or emptied from the deleted items folder, they are stored in the "dumpster". The dumpster allows you to recover items that are deleted from your mailbox for a certain amount of time before finally being fully deleted. The default "dumpster" items stay for 14 days. During these 14 days, you can recover deleted items using OWA or using Outlook. This can cause a problem because export-mailbox will also pick up items in the dumpster, convert them to regular "messages" and export those as well. This can cause duplicates as you can probably imagine. The way to resolve this issue is by turning off the dumpster for this account. This can be done by running the following Powershell command:

Set-Mailbox (account name) -ItemRetention (time, i.e 0.00:00:00)

(We use zero so that no items are retained when deleted). Obviously, you wouldn't want to do this for one of your users, as they would not be able to recover any deleted items.

Once the export is complete, you now need to clean out each of your user's GAL folder. Since not everyone in the company is using Blackberries, not everyone will need the GAL. I resolved this issue by setting a custom attribute for each of the users. The custom attribute can be set by issuing the Powershell command:

Set-Mailbox (account name) -customattribute2 (attribute, i.e. BES)

The script will create another .csv file (there's a pattern going on here), which will find any users with that custom attribute set, and put them into the .csv file.

Next is something that I'm not really proud of, and I'm sure there's a better way of doing it, but I was in a bind and needed something. In order to clear out the user's GAL, I needed to do an export-mailbox to another .pst file, once again setting the -deletecontent flag. This .pst file is useless, and will be deleted the next time it runs (I'm thinking of moving the delete for the .psts to the end). Once all of the mailboxes are exported, I can then proceed to use the same .csv file to import the exported .pst into our BES server users mailboxes, then TA-DA, after a few minutes, your users should have the GAL on their phones.

Now all you need to do is set it up as a scheduled task by invoking Powershell and running the script from a batch file. We currently update our GAL every 2 weeks. This helps avoid the users' exported pst from growing too large, as it will export the current GAL in their account, as well the dumpster. If we did this every day, the PST would grow by 13 times it original size by the end of the 2 week period, since we clear out the dumpster every 14 days, and would remain there.

If anyone has any better ways of doing this, I'm all ears... er... eyes. Also, if anyone wants to look into creating a EWS script to do this, I'd be more than happy to have that as well.

UPDATE: Still trying to find a place to host the script itself, since I can't do it using blogger. I'll post the script once I find a place to host the file.

UPDATE2: Found hosting space at Hotlink files. You can download the script Here. The zip contains the .ps1 script, which is can be called from Powershell or a batch file, a readme.txt file that explains how to use the file, and a folder containing Glen Scales EWS utillities. Note that you will need to have the Microsoft Exchange 2007 admin tools (with Powershell) and a copy of Outlook installed (for the PST import/export). You can only run this script from a 32-bit machine as well, since the import/export only works on 32-bit machines.

First Post

Welcome to my new blog. 

I am mostly setting this up as documentation for myself in things that i've done throughout my career.  If there was a particular problem that i've come across, or a script that i've had to write for a particular issue, I will try and post it here.  I hope that the information I post here will help others throughout their journey in IT.  

This will probably be a pretty random blog dealing with issues with VMWare, Automation, Powershell,  scripting, linux, random System Administration tasks, and perhpas some interspersed opinions about current events in IT.  

A little about myself, i'm a System Administrator for an Interactive design firm located in Columbus, OH.  I am a typical jack-of-all-trades system administrator, usually spending my time working on multiple projects at once.  I am proficent in multiple operating systems (Windows, Linux, Mac OS X, Unix, Solaris, etc.) and spend my time working in each environment (mostly in Windows and Linux though).  The main project I am currently working on is Cruise Control, which is a extensible framework tool, mostly used for Continous Integration in a development environment.  Cruise Control is a wrapper for Ant that allows you to write scripts that can be used in your development environment.

If you have any specific requests on what you'd like to see on this blog, feel free to post them in the comments section, and if I have any information on that subject, I'll post it.

Addendum: Also, be sure to visit the other blog, mostly updated by my wife, discussing our impending parenthood: Geeks in Love