Ping times through site-to-site VPN terrible

Hi all,

Recently something strange started happening. We have a site-to-site VPN via Cisco ASA (ASA5520 to a ASA5505). Ping times to the outside interface of the remote ASA is good (30-40ms), but if I ping anything inside the network it's consistently bad (400-1000ms). I have a router and an AP both plugged directly into the ASA inside. Pings to both of those devices are bad.

The link is good, since ping to the outside is good. But something is going awry inside the tunnel. We've tried:
-rebuilding vpn profiles on both sides
-changing IPsec proposals
-swapping out physical ASA's
-changing MTU (currently 1400)

Traffic is not the issue, as we're working on this while the remote site is closed. Any ideas?

Thanks,