If one token in your pool rises sharply, traders buy it from the pool and you end up with less of the winner and more of the other token. If prices return to where they were, the loss disappears — hence "impermanent". If not, it becomes permanent when you withdraw.
Trading fees may or may not make up for it. Pools of two stablecoins have almost none.