The biggest barrier to getting very fit is one's psychology around fitness, the biggest enabler is establishing consistent habit.
The following is an (incomplete) list of reasons explaining why exceptional fitness is hard to attain. Having these blockades in your conscious awareness will help you push through them when they are inevitably encountered:
Excruciatingly Gradual Visual Deltas: Visual progress is virtually impossible to reliably see day-to-day. Progress becomes clearly apparent on the order of weeks. You sample your results day-to-day and see no change, discouraging you.
Consistency is Hard & Boring: No one will tell you to go to the gym, you have to have a deeper reason & motivation. No one will tell you not to eat X food. You will find yourself doing the same routine for months on end (especially when losing weight) and it will just be a waiting game through the pain. Embrace the mundane.
A Specific Kind of Pain: Exercise is painful, but you must reframe the pain as good pain that teaches you lessons, builds character, and amounts to important changes in your life.
Hunger: When losing weight hunger can be unbearable until you mentally adjust to the feeling.
Exertion: When gaining muscle the pain of pushing past your limits will be uncomfortable.
Inertia: It is hard to start a new habit unless you are driven by something deeper to see it through. Surface-level motivation lasts maybe 1 week when you first start exercising due to massive habit inertia.
You don’t feel like working out until you are warmed up (often). Then the mind shuts down and the body is unlocked. You have to get to the gym and break this Catch-22.
Apathy Towards Fitness: Often, being very fit is not valuable to an individual and their goals in life. It has to fit into one’s life roadmap and you have to see the value that elevated fitness can bring into your life (higher energy levels, greater mobility, higher self-esteem, greater mental health, longevity). It has to become a priority.
Target Visualization: It is hard to imagine how fit you can become until you become that level of fit. This is especially apparent when you lose large amounts of weight, a new aesthetic high occurs every 2-4 weeks that you'd never have projected.
Consistent Diet is Hard: You have to get plenty of protein, control calories, balance other macronutrients. It is very hard to ensure you are getting the food you need in a busy life. It can seem you are eating enough, but it is very black or white whether you are hitting the macros you need to build muscle or lose fat.
Mental Presence Is Required: Heavy lifting taxes the central nervous system. It is hard to be present during the workday and fully present in the gym as well to push to new frontiers.
You have to constantly push yourself further. If training for size/strength, trying the next 5lbs up even if you fail is critical. If doing cardio, go .1 more miles or 1 more minute.
Time Investment: Exercising takes time out of your busy day.
Self-Consciousness/Learning Curve: People new to the gym often feel self-conscious or like they have to overcome a large learning curve. Realize everyone had to start somewhere and we are all learning every day. Eventually, you will be an expert from your self-learnings just trying things.
Bad Workouts Tripping You Up: You will have many “off-days” where the weight just isn't moving right, your legs feel heavy, you aren't lifting the same weights. These are normal, and as long as your diet is correct, they will pass and you will continue to new heights. These should not discourage you.
There will be days when you are in the gym and just do not want to be there. This is normal. This is where the power of habit shines, it will carry you through your routine no matter how bad you feel.
It Is Easy to Slip on Certain Isolations: If you don’t work some auxiliary muscles in isolation (like forearms) they will not grow at the same pace as other muscle groups. This can lead to you getting demoralized by "lagging" regions, but you just need to be conscious of what you are isolating & creating microtears in.
A lot of how far you will go in your fitness journey will be determined by how well you can integrate fitness into your day-to-day life. Everything from the time of day, to your mood/mindset when you exercise, will impact your rate of progress or leave you trapped in a holding pattern.
The most worthwhile things in life are often the hardest to achieve and take long spans of time to earn.