ISP or Firewall Problem?

Count me in as a one man IT Shop where advanced networking is one of my few weak areas.

We have T1 Dedicated internet here with top speed being 3 Meg. We have been having very poor internet speeds. They come and go. Sometimes we are at .50 meg and others we are averaging close to our 3 meg at 2.64 meg.

We do have a few people who do stream but even at 5:30 when those people are gone, it can drop on its own back down to .50 meg. Horrible.

I called our ISP and over the phone and when they came out, speeds were fine. So wondering if it could be our Firewall. I can reboot it and it seems OK but sometimes I reboot and it is just as slow.

It will pick up speed and slow down off and on all day long.

We are currently on an Endian Software firewall on a Pentium 4 box with 3 Nic's.

I have a Watchguard firewall here but having difficulty configuring it to the settings we have on the current one. Support Forums for Watchguard don't want much to do with helping convert over, so haven't found where to put my Port Forwarding and Externall Access forwarders yet.

Anything I can do to see if it is the firewall or should I push the ISP more on their poor service?

Thanks...