Does an N connector have less loss than a PL-259? Yes.
Is there enough of a difference that you will ever notice it at CB band? No.
In the video six-Shooter linked above, and in other tests I've seen, even after stringing several PL-259/SO-239's together, and in the case of the video above specifically, you still have 0.09 dB in loss across 20 of them. Further, he is using the S11 reading, not the S21 reading (I, personally, would have done this test with the both ports). So you have 20 connections, and the signal is passing through each one twice. Because of this you have to double the number of connections to get accurate average per connection. So, get a reading for a single connection, you now take that 0.09 reading and divide by 40. You get 0.00225 dB. When I did a similar test using an AIM4170 years ago, I got a similar result per connector, (although my results were a little higher, coming closer to 0.0025 dB) and I've seen one or two other youtube videos by different people showing this as well. This basically means you would have to have over 400 PL-259/SO-239 connections to get to just 1 DB of loss.
I agree with you, the PL-259 has "more loss" than an N connector, but the important part here is the scale of the difference. It is important to factor in the actual difference, which in this case is negligible. Twice as much of nothing is still nothing...
I think that the impedance change argument should also be addressed, so I'll be referencing
this page which has plenty of calculations. Two lines from his conclusion.
The down side of an incorrect impedance are either not worth worrying about or are easy to compensate for at frequencies less than 500MHz.
Also you can buy SO-239 connectors with a mostly air dielectric. The large center pin with a relative dielectric of something like 1.3 would raise the impedance to 46 ohms and be much less of an issue.
So, at CB frequencies its well within the range of "not worth worrying over" with tons of room to spare, and if you still are worried, there is a fix that will mitigate the non-problem you are so worried over even more...
The DB