That's really weird. The wheels should have no effect, because its the overall tire diameter/circumference that will mess up the transfer case. The difference between those two sizes are within tolerances front to back.
Anyway, I was on 19x9.5 et45 and 19x10 et38 when I ran them. They were smooth as butter. Then I just put the same 275/40's on the 9.5 fronts before picking up 2 extra 10's. All option worked perfectly, so I really don't know what to tell you.
only thing I can think of (and it's a huge stretch) would be that you're rears were worn down very far while your fronts were new (causing enough of a size difference) or the tires had manufacturing defects of some kind...
I can guarantee that the staggered setup had zero problems with my transfer case. And I doubt there was any real difference between our model years that would relate to this issue (mine was a 2007 3.0si).. All that being said, I do like the 275/40R19's all around much better.
Prototype 2007 Monaco E83 'X3M' | N54 Swap | Gruppe M CF CAI | VRSF CP & DPs | CXR Intercooler | ER Dual Oil Coolers | Dual 2.5" w/X-Pipe | M3 Rear Exhaust | 20" BC Forged HB05 | Michelin PSS 265 & 305 | F10 M5 & F80 M3 Brakes | KW Variant 2 | H&R Swaybars | TMS Z4M Camber Arms | Z4 Finned Diff | BMW Perf. Grills w/Tri-Color | Custom M Aerodynamics | CF Trim | M Perf. Carbon Shift & Handbrake Knobs | BMW Perf. SS Pedals | M3 Dead Pedal | V1 Hardwired | Int LEDs | Umnitza Orion v4 AE