From the egoic state of mind, it really appears as if outer experiences determine happiness. The reason for that is that the egoic perspective is artificially limited. The ego lacks wholeness and knowledge of its true state. This perceived lack is projected outwards. External circumstances seem to temporarily fill that inner void. The reality though is that we only associate certain states of mind with experience. All states are always arising within ourselves, independent of experience.
How can you stop giving things the power to determine your state of mind? There is a right and wrong way to do this. If the ego mind realizes this predicament, it might try withdrawal as a strategy. You might become a minimalist and reduce your activity. You forcefully try to disengage from the world out there. This might help you to realize some of that truth, but for most people, this is going to be the wrong way and will lead to frustration. It's denial, not true asceticism, which comes from inner abundance.
The right way to stop giving experiences the power over your state of mind is to see the illusory nature of all appearance. Seeing the dreamlike nature of everything will reveal the pure potential of every moment. You stop giving the power over your state of happiness away to form. Just as a dream at night loses its power when you wake up. Waking up to the true nature of things does the same. Then reality is free to play, with no strings attached.