Yes, that is the way the built in feedback works. It will not be displayed until the user attempts to leave the page. If you wish to display the feedback in a different fashion it will require creating your own custom feedback for the question.Here is a related topic on how to do that: Question Feedback
