Depends on the selector, and how deep the center groove is between the two end deep detents.
Some are shallow'ish in V groove, so easy to walk the selector tip up the wall as you are pushing the end of the selector and turning it, while other are deeper, have more of a flat side, your never going to walk the selector tip back into the receiver channel for the selector to be push out instead.
But yes, pistol grip retainer bolt backed off, drop the grip down on the bolt to relieve the spring tension to all the selector detent to drop down to clear the selector channel, and most of the time the selector will just push right out. I say most of the time, since can have a Few FCG groups that the side of the selector will bind up on the side of the rear of trigger tail, and in those cases, may have to start to pull the FCG pins, before you are able to get the tail of the trigger out of the way.
And removing the FCG from time to time, is not a bad thing, since over time you can get fouling/debris build up in the pin channels, to cause the trigger pull to start to become gritty; especially on a new build after a few hundred rounds, where the parkerizing in the channels and on the pins starts to wear off.