Swimming in SonicWALL

Since posting last week about my troubles treading in to SonicWALL water, I think my issues have all been resolved and things are really humming right along. Truth be told, the problem was really a combination of user-error and user-ignorance. If you follow me at all on Twitter, you already know that I had major, show-stopping issues over a couple of days last week. WAN->LAN traffic would flow perfectly fine for a while, and then at some completely arbitrary point, it would stop passing traffic to certain internal hosts. Like I said, this was show stopping.

You might like some background on our implementation. When ESI received our first IP assignment from BellSouth many years ago on our fractional T-1, it was a short and simple /29 network which yielded six usable addresses, minus one that had to be assigned to the ISP provided Cisco router, leaving us with five addresses we could use. A couple years ago (three to be exact), we outgrew that, and in an attempt to keep a 1:1 ratio on NAT rules, we acquired an additional subnet – but this time a significantly larger /28 was assigned. For whatever reason, we never bothered moving all the services to the new subnet. With Watchguard, it was never an issue and was easy enough to run them all simultaneously. The first subnet was configured on the WAN interface and the additional addresses were added as “virtual addresses” on that interface and they were then available to create rules/policies with.

Enough with the history lesson – let’s move on to the present. Personally, I’ll take 99% of the blame for the user-error part. Upon receiving the NSA 3500, I was anxious to get started, so I unboxed it and started exploring the web admin interface. It was confusing and quite different for someone like me, who prior to this, had only had experience with Watchguard gear. After questioning many SonicWALL users whom I respect, I started firing off the Public Server Wizard and creating all my NAT Policies and Firewall Rules. This was apparently mistake number one. While it does indeed work, I later found out that Mark doesn’t suggest that method. Creating them manually gives you a bit more granualar control, leaves a little less cruft in the auto-generated NAT Policies, and the real kicker – you end up gaining a much better understanding of how the Firewall Rules and NAT Policies work together.

Moreno and I made plans to work on this last Thursday morning. I went in to the office early and decided I’d put the SonicWALL NSA back in production while we finished up. Around 7:30 AM, I made the swap and rebooted the Cisco from AT&T to flush any routing related issues out. A few services came online with the NSA and Mark continued cleaning up the rest of the NAT rules and chasing rabbits. By 8:15 or so, we had everything working except one Linux server which was non-mission critical, so we decided to hold-off on that one for a while and see how things went. Things went extremely well throughout the day, but as soon as we finished watching the LOST season finale on Thursday night around 11:15 PM, my Treo started buzzing with down alerts. I started checking things, and finally at this point, it dawned on me that all the services that were failing were on the newer /28 subnet. Surely that must bare some significance. I rebooted the NSA to no avail. After about an hour, traffic magically started flowing to all hosts again so I called it a night, texted Mark to let him know about the issue, and set the alarm for 6:00 AM.

I crawled out of bed around 6:30 Friday morning and got to the office around 7:15. At this point, I had resolved to myself that it was either do or die. I was not going to revert back to Watchguard a third time. If we couldn’t fix the SonicWALL by noon, I was going to wash my hands of it and let Moreno have his gear back. I chatted with Mark around 8:00 and he was bumfuzzled and en route to meet with another client but promised to look at my logs if I’d send them over and also open a ticket with SonicWALL to get the issue resolved ASAP.

I kept hammering away. Sitting idly and waiting on Mark and SonicWALL was not part of my playbook and for some reason, neither was calling support directly. I’m just not the kind of guy most of the time. For some reason, I’d rather spend severalĀ  hours resolving an issue on my own than just calling and asking someone. Maybe it’s pride? Regardless, around 9:00 I turned to Google and the SonicWALL website and within 15 minutes, I discovered SonicWALL KBID 3726 titled “SonicOS: Configuring Multiple Subnets Using Static ARP with SonicOS Enhanced” which outlines these simple instructions:

Follow these instructions to create a second subnet on an interface:

  • Create a static ARP assignment. Enable the “publish entry” check box.
    1. Login to the SonicWALL’s Management page.
    2. Select Network > ARP.
    3. Click the ADD button under Static ARP Entries.
    4. IP Address – Specify the IP address to which the SonicWALL should be assigned on the additional subnet.
    5. Interface – Specify the interface (LAN / WAN / OPT / WLAN) where the additional subnet resides.
    6. Publish Entry – Enabling this option causes the SonicWALL to respond to ARP queries for the specified IP address with the SonicWALL’s MAC address. This box must be checked when creating additional subnets.
    7. Click OK.
  • Select Network > Routing.
  • Select Add. Create the following new route policy:
    • Source: ANY
    • Destination: Create new address object
      • Name the object for your secondary subnet
      • Zone Assignment of your secondary subnet
      • Type: Network
      • Network: Enter the Network address of the secondary subnet
      • Netmask: Enter the Subnet mask of the secondary subnet
      • Click OK
    • Service: ANY
    • Gateway: 0.0.0.0
    • Interface: Select the interface the secondary subnet resides on
    • Metric: 20
    • Comment: Label policy so it can be identified at a later date
    • Click OK
  • A ssecondary subnet on the LAN interface will use the default NAT Policy & Access Rules. Access rules & NAT policies may be added.

As soon as I created the ARP assignment, traffic started flowing but I went ahead and created the static route as well for good measure.

I’ve got a couple more SonicWALL posts to come. One about how AWESOME the hardware-based VPN is and another about exactly how to configure the Firewall Rules and NAT Policies without using the Wizard. Tomorrow (Tuesday) morning, I’m heading to Charlotte for the SonicWALL Roadshow and then in the afternoon, I’m deploying my first TZ 150 endpoint to a remote office, along with another special piece of hardware. I’ll try to get some pictures of all that and the VPN post up up by the weekend.

Overall, I’m quite happy with the NSA 3500 and my SonicWALL VPN solution. For now, I’ll withhold my formal review and recommendation on Moreno until I see his final invoice – if he cuts me some slack, I’ll cut him some on the blog.