My friend lives in Durham NH.
I would say to pay close attention to both. The X3 is definitely more nimble than the X5 which is ultimately why I decided on the X3. It was a bit more "fun" to drive which was important to me as I usually drive cars and not SUVs. If this is important to you, you might want to drive both and think about that.
Otherwise I would bring the family down (maybe minus the dog) and pile into both and see how you feel. Too cramped? Comfortable? If you put some money down they might let you take the trucks on an extended test drive for a few hours in the afternoon.