asymmetrical encryption

In trying to understand asymmetrical encryption, I thought of a way to do it that doesn't involve large prime numbers. If someone can explain to me why it isn't done in the way I thought of, I'll have a firmer grasp of the way it's actually implemented. So, Person A wants to send a secret email...