Is this a fair punishment?

That is him getting off easy. I'm just a minor, but I do some babysitting and the reasoning is this: If you can't do the time, you can't do the crime. If he is going to physically injure you, then you need to show him who is boss and who he is messing with. Letting him have his gameboy back is letting him get away with his actions, and I can ure you he will do it again.