Well, turns out the problem isn't memory or any of that jazz. It's the tables.
I've got code that's been running over 24 hours that uses tables.
I switched to arrays and cells and it finished in under 5 minutes. Sheesh. My computer handled 400000x400 arrays and cells without any memory issues.
I guess tables are just stupid slow for large sets.