what size of a transformer are you talking about.
The information's in the question. He's looking at whether a single-phase transformer, supplied by two hots of a 50A circuit from a 208V Y service, can yield a more useful single-phase source than the alternative of 2 x 120V 50A supplies from the hots to neutral. Hence, the size of transformer under consideration is 208 x 50 = 10.4kVA input.
understand the theory behind calculating the 87A?
It's straightforward and intuitive.
P=VI (power = voltage x current)
Considering the voltage and current at the input and output of the transformer:
Pin = Vin x Iin
Pout = Vout x I out
If the transformer is perfectly efficient i.e. has no losses, the power output would equal the power input. Hence:
Pout = Pin = Vout x Iout = Vin x Iin.
Vout x Iout = Vin x Iin rearranges to:
Iout/Iin = Vin/Vout
I.e.
the ratio of output to input current is the inverse of the ratio of output to input voltage. Simplest example: halve the voltage, and double the current will be needed to transfer the same power.
Real transformers dissipate some power as heat in the copper and iron, therefore the power available at the output will be less than the transformer consumes at the input. Efficiencies range from 60-99% or so with the highest efficiencies only in the very largest high voltage transmission transformers. When considering the output current available from a transformer, one must check or estimate the efficiency and reduce the output by that ratio from the 'ideal' transformer's output.
In your case I merely estimated about 8% for losses and excitation as a worst case for a commercial 10kVA transformer, hence 87 x 0.92 = 80A useful output at 50A input.
Beyond this, signifincant maths is required to further understand the characteristics of real transformers.