Catching is hard. It's a quick path for advancement if you're good at it, but its a rough position. And it's not as glamorous as P/SS.
It's no accident that the catching equipment is referred to as the tools of ignorance. Despite my size, I caught when I was a young player and liked it. However, and in addition to the other excellent points made above, catching becomes incredibly harder when the ball starts moving, which I recall happened first for me as a catcher at 12 in 1972. I had an opportunity to play catch with then Alexandria Ace Randy Johnson, who many of you will remember as a diminutive (about 5'8") portsider who was a groundball machine.
Jones pitched for the Padres and Pirates as I recall, and I believe Jones won 20 games twice and lost 20 games once. His fastball topped out in Alexandria at 79. His sinker ball was thrown around 74. However, after taking the goadings of a cocky, excited 12 year old bat boy to throw the sinker, he finally relented and threw it. I got hit in the legs with the first four that he threw, but hung in there and caught the fifth toss. That ball moved downward late a good 8-12 inches. It was a really good sinker, as I would learn later. I caught all the way through, but only as the emergency catcher and sometimes bullpen catcher.
The slider is a particular bitch of a pitch to catch.