If “different typos” averaged out to “nearly no typos” the same logic would have different words average out to nearly no words. What actually happens is the model learns context, and can produce output which contains emojis in one context and not others. These contexts can be very far from the average context.
I’m afraid the upshot is you don’t understand how the models work. There is extensive filtering before training - they do not get “the entire internet” and average it. If you want to understand properly, there are a lot of resources that will let you, but I’m not going to try to do it here, so you’ll either have to believe me or be wrong, I’m afraid.
If “different typos” averaged out to “nearly no typos” the same logic would have different words average out to nearly no words. What actually happens is the model learns context, and can produce output which contains emojis in one context and not others. These contexts can be very far from the average context.
I’m afraid the upshot is you don’t understand how the models work. There is extensive filtering before training - they do not get “the entire internet” and average it. If you want to understand properly, there are a lot of resources that will let you, but I’m not going to try to do it here, so you’ll either have to believe me or be wrong, I’m afraid.