A lot of the enjoyment from performing an activity comes from developing some level of competence or mastery with that activity. This creates a positive feedback loop where you perform well, feel good about yourself, and want to continue to do it.
A lot of times I think there's an initial cliff for people trying something new where either you're naturally inclined toward it, and you have a few hours of novelty, or you're not, in which case you immediately hate it. When my ex first tried videogames (and hadn't grown up playing them) he hated it. He used to get so mad at the controls, the situations, the failures. He didn't have an intuitive sense for what to do.
His first 20 hours in Mass Effect he didn't even know he could upgrade skills.
As we played more games together (Skyrim, ME, Fallout, etc) he started to develop more of an intuitive sense for the systems and began to really enjoy playing them. This probably took 30-50 hours of total play time.
If you think about any of the activities you really enjoy, consider that you're probably pretty good at them and you probably also went through an initial learning phase at some point in your life where it wasn't as fun... but it might be so far back you've forgotten it.
I think that as we get older, we tend to forget that "the suck" (that first 30 hours or so after the novelty wears off) of any activity is a natural part of the learning process and doesn't mean you're "not good at something."
I tend to lose interest in new activities quickly when I slam into "the suck" portion because trying something new always seems more appealing than grinding out 30 hours of something unpleasant unless I'm really motivated somehow.
This probably means that finding ways to force yourself to grind through "the suck" and develop a level of familiarity and expertise with any new skill is a key part of sticking with it... after which it becomes self-reinforcing.
This, by the way, is especially pernicious if you tend to be initially pretty good at a lot of things for whatever reason... "the suck" feels more unpleasant if you sample a lot of stuff and do okay at first, because the "yay success" hit is easier to get at novice level elsewhere.
Don't give up. Keep pushing. Embrace the suck.