"Transition" to Exchange 2007

So, we have officially "transitioned" as Microsoft calls it from Exchange 2003 to Exchange 2007 here at The Chapel. Exchange 03 has served us well these past 6 years. Wow, has it been that long? Our dependence on email has risen so much since we installed it and it has become one of our most mission critical applications. Only when it goes offline (like when we did our transition) do we notice just how much we depend on it! I did my best to minimize the downtime for the transition but there were a few hangups that I'm going to dive into now.

First, some of the history. We started with Exchange 2003 Standard shortly after it came out. I remember that it was before the first Service Pack. At the time I thought to myself, "We will never hit that 16 Gb database limit". Boy was I wrong! Service Pack 2 was a life saver as it let us keep the mail flowing by raising the database limit. This was good for a while but going from one campus to 4 in a year and doubling our staff was putting quite a strain on our email server and it's limits. Also with the introduction of ActiveSync for the iPhone, we were having more and more sync issues and missing appointments.

Fast forward till 3 months ago. We were hitting the max limit of Exchange 03 Standard again and needed a solution. I wanted to wait for Exchange 2010 and just skip 07 but we couldn't wait any longer so we decided to make the move to 07 and then upgrade when 2010 came out. The problem was that we needed a fix now and couldn't wait until our transition so we did an in-place upgrade from Exchange 2003 Standard to Exchange 2003 Enterprise to buy us some more time.

Then the planning and testing began. After reading up on several blogs and talking with several of my Church IT RoundTable peeps, I installed the server. My main source of documentation was a blogpost on MSExchange.org which was a huge help! There are a few things it leaves out though but Google came to the rescue. I installed Exchange 2007 as an all-in-one install like I had with Exchange 2003. Installing an Exchange front end server would have made things easier as I could have kept things running as we migrated our users and services to the new server though.

I though I would be able to keep things going with the setup I had but after much testing, I found out that with how complex our environment is, it just wouldn't work. We have a BES and rely heavily on ActiveSync and OWA and I just couldn't get everything happy. So, I made the move to the new server all at once (For us the best time window was Sunday after all the services).

There were a few "gotchas" that came up after the transition.

First, I found out that while computers on the network will magically update their server settings after their mailbox is moved to the new server, it doesn't update computers off-site connecting by RPC over HTTP. This lead to quite a few email, text messages, and tweets from people telling me that they couldn't connect. I had set up a blog prior to the cut over and had informed people that because email could be down and that I won't be able to communicate with them, that they needed to check the blog for updates. This worked well (for the people who went there) and I'll definitely use a blog again in the future. We also found out that the new "Outlook Anywhere" (RPC over HTTP) defaults to "NTLM Authentication" and by default won't accept clients configured to use "Basic authentication". We had some clients that had used Basic Authentication and to speed things up had to enable it for Outlook Anywhere. This EMC command did the trick:

Set-OutlookAnywhere -Name Server01 -IISAuthenticationMethod Basic,NTLM

TechNet Link

Second, IMAP and POP are not enabled by default. After a few people using Mac Mail and some other mail clients that rely on IMAP. Since we only had a few people using this, it took a back seat untill the MAPI and ActiveSync clients were working.

Third, Exchange 07 now manages manages what 03 called "SMTP Virtual Server" in the Hub Transport Role and Send Connectors. Read up on this if you are unfamiliar with how this works. It can save you some headaches.

Fourth, We moved the final accounts to the new server which were the few BES clients I had. This blogpost is what I used for reference and it worked out well.

What would I do differently? Hard call but I think I would set up an Exchange 2007 Front End Server so that I could take my time. A front end server would proxy the OWA requests when you are in a coexistence period. I also would have gotten the new certificate before moving my services to the new server. Our one Palm Centro user had ActiveSync problems with the GoDaddy.com certificate we had. My CITRT peeps David Szpunar and Daryl Hunter have some great blogposts on working with Certs when upgradeing so I'll just link to their posts.

Microsoft knew that ActiveSync and the AutoConnect settings can be a pain so they made a great site for testing your connection, https://www.testexchangeconnectivity.com/

So, I hope this helps someone out there but mostly I did this post so I don't forget!

Internet Failover Using EIGRP

Internet Failover Using EIGRP

If you have a mostly Cisco network like we do, you know that there is nothing you can’t do, for a price. Cisco has some great Fail-Over techniques using their ASA 5500 series Firewalls but you need to get the extra HA licensing. Also, what do you do if your other firewalls aren’t Cisco? In this case, EIGRP can get you the same basic features for free. Now before I get lots of comments on this, using dynamic weighted routes with EIGRP does not provide Stateful Failover meaning that you will drop your connection for a short period while the network converges and you won’t be able to setup BGP this way (I think).



The Setup

For this post, I’m going to use a simple scenario of having two sites with two internet connections. One is much faster than the other and it makes sense to backhaul the internet from Site B to Site A when the connection at Site A is active. For this post, let’s assume that you want to just protect against the link between sites going down and not things like the Internet modem locking up (In future posts I’ll address this).

RTRA EIGRP Config

RTRA Fa 0/0 ip address is 10.100.1.2 255.255.255.0
RTRA Fa 0/1 ip address is 10.1.1.1 255.255.255.0
FWA Fa 0/1 ip address is 10.1.1.254 255.255.255.0

RTRA# conf t
RTRA## ip route 0.0.0.0 0.0.0.0 10.1.1.254
RTRA# router eigrp 100
RTRA# network 10.1.1.0 0.0.0.255
RTRA# network 10.100.1.0 0.0.0.255
RTRA# redistribute static metric 20000 1 255 255 1500

RTRB EIGRP Config

Fa 0/0 ip address is 10.100.1.3 255.255.255.0
Fa 0/1 ip address is 192.168.1.1 255.255.255.0

RTRB# conf t
RTRB# ip route 0.0.0.0 0.0.0.0 192.168.1.254 255
RTRB# router eigrp 100
RTRB# network 192.168.1.0 0.0.0.255
RTRB# network 10.100.1.0 0.0.0.255

Summary

Both routers are participating in EIGRP group 100. They will advertise their routes to each other and calculate the metric based on bandwidth, congestion, etc. The key here is setting the “redistribute static metric 20000 1 255 255 1500” command on RTRA. This tells EIGRP to redistribute the static routes you have setup in the router with a bandwidth of 20,000 kbps, delay of 1, reliability of 255, loading of 255, and MTU of 1500. RTRB has a static route for 0.0.0.0 but with a metric of 255. The dynamic route from RTRA will have a metric of 170 (External EIGRP) and will replace the static route in the routing table. This works fine if the only static route you have is for your default route so this isn’t very flexible. You could also attach a summary default route to Fa 0/0 and accomplish the same thing.

If the link between RTRA and RTRB is active, traffic that doesn’t have a route in RTRB (Internet) will be sent to RTRA. In the event that the link between RTRA and RTRB goes down, RTRB will send traffic to its local internet connection because the static route is now the only default route.

Now the biggest downside to this type of “failover” routing is that it isn’t fully dynamic and doesn’t take into account the fact that Cable or DSL modems can lock up and will still look like they are up to the firewall or router and therefore not remove the route. This approach also doesn’t work well if you have even more locations with their own Gateways to the Internet. I will address this in the next post.

Idea's for Projects

Just got back from the Granger IT Round Table which was awesome! Some things that I heard got me thinking of some projects.

• Open Source Voice Mail - We have Cisco Call Manager and I don't really want to change to Asterisk because of our investment. I do however like the idea of free voice mail server with Voice Mail to Email features and Speach to Text that some offer. Cisco Unity Connection 1.x doesn't offer these or does for a big price. Working on getting our Unity Connection 1.2 to work with IMAP for voice messages. Almost there.
• SharePoint for our Intranet and Communications team and contractors to use so that they don't have to send huge files though email.
• Open Source hard drive imaging. Currently we are using Ghost but I don't want to buy more licenses. Something like CloneZilla.
  
• More storage for our Graphics and Video guys untill we get a long-term solution. Getting a Drobo Pro to replace Buffalo Tera Station Pro II.
• New wireless for all campuses! Looking at Aruba systems.
  

Going to be a fun year!

James is Here!

James Michael Good - Born May 13th, 2009 @ 10:00 PM

Things went almost exactly as they did with our first child, Madison. James beat her by one minute on the delivery time by coming in at 10:00 PM and Madison coming in at 10:01 PM.
Liza is doing well and recovering quickly. We are both sleep deprived because we wants to eat all the time. Especially at night :-)

Now for some pictures:





See more pictures here on FlickR.

Major Life Changes Ahead

Getting married, buying a house, moving, having a kid. All big moments in my life. Now, in as little as two days, I will get to see the miracle of birth again as I welcome our second child, James, into the world. I feel like I'm prepared as I am already a dad but everyone fears the unknown.

What will he look like? Will he have have brown eyes and brown hair like me or blond hair and blue eyes like his mom? Will he be tall or short? All these questions and more make me all the more excited for the coming days. I'm also excited that God knows all this and more about my future son and who he will become.

More details and pictures to come as we adjust from being a family of 3 to a family of 4.

Cisco Phones on HP ProCurve

Trying to save money? Aren't we all. If you have Cisco phones, you may be able to save big $$ by switching to HP ProCurve switches.

When we needed to add more PoE ports to our Grayslake campus I looked at the price tag of a new Cisco 3560-48PS starting at $4,000, I knew I should shop around. I've been looking at HP for a while now so we picked up a 2610-48-PWR for less than half. Good deal, but then I had to get them working together. This is how I got it to work.

First thing you have to know is that even though HP tries to be like Cisco and use a lot of the same terms on the command line, they're not.

What Cisco calls EtherChannel, HP calls a Trunk. This caused me much trouble early on. HP doesn't really have an equivalent to what Cisco calls a trunk port. All ports are access ports on HP and if you want to use a port like a trunk, you have to assign all the vlan's that you want to use on that port.

In my situation, I had to assign 3 vlans for my trunk. The other confusing thing is that there is no native vlan as you would think of one on a Cisco trunk. To do that, you set a vlan to the port as "untagged" wich makes it the default vlan for that port. Since you can only have one untagged vlan for a port, you have to make the others "Tagged".


If you are using Cisco phones (maybe any phone) you have to have a default vlan and then a voice vlan. The voice vlan has to be set as a static tagged port and then also defined as a voice vlan from the command line as follows:

ProCurve Switch 2610# conf
ProCurve Switch 2610(config)# vlan 10
ProCurve Switch 2610(vlan-10)# voice

After this, you should be set. Well, sort of. Cisco 7912G, 7940G, and 7960G phones are not 802.3af PoE compliant they won't power up on this switch. The solution, add a patch cord. When you do, the positive signal of the PoE gets sent on the correct pin and the phone works. Kind of a bummer but $200 of crossover cables you can have all your phones up and running.

So, about $46 a port for HP as compared to at least $83 for the Cisco, time will tell if it is worth it.

Oh, With HP you also get Lifetime Warranty and updates!

** Update **
After doing some more testing it appears that the phones are not getting an IP address from the Voice VLAN and are not using it. This will not work in our environment because it is designed to have voice traffic on the voice VLANS. If we had newer phones like the 7941G which support LLPD-MED which acts like Cisco CDP, we could make them work. Back to the drawing board :-(

**Update**
If I had listened to what @procurvehelp on Twitter had told me, I would have had a solution.

@procurvehelp "Todo this: Press Setting, choose Network, choose Admin VLAN, press **# to go admin mode, change op & admin vlans, save & reboot."

On the Cisco phone, after you change the Admin. VLAN Id and press the "Validate" soft key, press the "Save" soft key. After that, the phone reboots with the correct settings and you are now on the Voice vlan! So, it is possible to get 7912G, 7940G, and 7960G phones able to work with an HP ProCurve 2610-PWR-48.

**Update**
Last night we fully cut over to the ProCurve and everything is working great! I'll have to wait to see if we have any call quality issues but I don't think that we will. Think I might look at some more HP gear but just make sure that I have the newer phones!

Windows 7 Beta on Tablet PC

A few weeks ago I switched my hard drive back to Vista on my Lenovo X61 Tablet. Over all I think Windows 7 is a huge improvement over Vista. I couldn't get my pen to work the first time but now it appears to be working and the handwriting recognition has gotten even better!

So, back on Windows 7 Beta and I love it!

Update

Windows 7 is available to the public now but I'm still loving Windows 7 Beta on my tablet and everything is working great. Looking to get off the Beta and onto the production x64 version soon. I haven't had any issues with features working but I will document the install better this next time around.

Some Late Nights

I just had some late nights upgrading our switches and Dell SAN. Here's the scoop and a few things that may save you if you have to do the same.

We had to update our Cisco switches at all our Chapel campuses this last week to support our newest endeavor, HD Simulcasting! We are going to be sending HD video of our live service to our other campuses in only a month! We are going to send one feed for a fixed center screen and another for Imag and graphics on the side screens. We are using HaiVision 1020 encoder with a Maco HD card to encode both streams. We are still working out some decoder issues now.

*Note on updating Cisco switches though the web GUI. Even if it says that it has uploaded the .tar file and is rebooting, DO NOT DISCONNECT! If you don't wait until is says, it may not load the HTML files.

While I was in the updating mood I updated our Dell MD3000i SAN to the newest firmware. I hadn't done this before but it was pretty easy. (See Justin Moore's blogpost for details) The part that took the longest was backing up the SAN. It took about 10 hours and the sad part is that it only had about 1TB on it! I need to figure out a better backup method. Backing up to our Buffalo TeraStation Pro II is WAY too slow!

*If you add another NIC to your VMWare Server, make sure to exclude your SAN NIC's from the bridge mode.

*I never did post what I thought about the Buffalo TeraStation we got. My advice, if you want to use this for more that archival purposes, spend a little more and get something much better.

TeraStation Pros

• Cheap
• Uh...

TeraStation Cons

• One GB Ethernet port
• Slow web GUI
• Bad firmware updates and NO change log on the updates (unless you can read Chinese)
• Lots of Mac OS X issues.
• Did I mention slow?

Frozen!

We are getting closer to having our fiber network installed. For the last week we have seen many AT&T trucks around our Grayslake campus pulling fiber optics up and down the highway. The fiber finally arrived yesterday but there was one big problem that prevented it from being connected.

ICE!

We had rushed to get the conduit on our property run this fall so that we wouldn't hold up the project. We found out yesterday that that conduit had filled with water over the last two months! The record low temperatures we have had over the past month had caused a small portion of our underground conduit to freeze! Now we had to figure out how to clear it out. Hopefully without digging it up.

This is what we tried:

1. (Prayer)
2. Fish tape. Nope
3. Bigger fish tape. Nope.
4. Power rodder (for sewer pipes). Nope.
5. Garden hose run down the pipe to the blockage and hot water turned on. Success!

After leaving the hose on for about 30 minutes we had melted though the 2' section of ice and the pipe was clear. We pumped and vacuumed the water out and sucked a new rope though. Later today, AT&T came back and pulled the fiber.

Hopefully soon our fiber will be done and we can communicate at the speed of light!

Windows 7 Beta on Lenovo X61 Tablet

These are my first thoughts of Windows 7 Beta on my Lenovo X61 Tablet

• Install was great. Was able to get online right away with installed drivers.
• It seams faster than the preloaded Vista Business that I had.
• Biometric scanner was not installed but Upek has a driver for Windows 7.
• Touch screen and pen not working yet.
• Have to run some sites in "Compatibility Mode" in IE8. Mostly, they show up with real small font that the zoom doesn't help or have some overlap of controls.
• Bottom task bar is nicer with more animation.
• The new "Show Desktop" button on the right of the task bar doesn't make much sense.
• No "Quick Launch"? The new "Pin to Taskbar" feature seams to have replaced the Quick Launch.

More to come with pictures as I continue to test.

New Domain Name!

So I decided to no longer be a cheap-skate and get an actual domain name! In the next few days my blog will be switching to http://jeremygood.net . Hopefully I can get everything from Blogger to moved over. I registered the domain with Google Apps which in turn registered it with GoDaddy.com.

I'm hoping that because Blogger is by Google and I registered with Google for my domain name it will be an easy switch.