Your last included"The S/N ratio of the noise you are trying to measure against the noise inside the analyser will not improve/change with a change to a narrower RBW. This is because both the noise contribution from the analyser and the noise signal of interest BOTH have to pass through the same (narrower) RBW filter so they BOTH go down by the same ratio of 10dB as they BOTH get sliced into a narrower slice of noise by a factor of 10. A Slice of noise 10 times narrower will have 10 times LESS power. I hope this helps"
This text is directly out of the same AP note, so is in direct conflict with your position.
Resolution bandwidth also affects signal-to-noise ratio, or sensitivity. The noise generated in the analyzer is random and has a constant amplitude over a wide frequency range. Since the resolution, or IF, bandwidth filters come after the first gain stage, the total noise power that passes through the filters is determined by the width of the filters.
I think this is the same ap note (150):http://literature.agilent.com/litweb/pdf/5954-9130.pdf
Look on page 20, figure 29a, 29b, and 29c, and see how insuffiicient res BW has degraded the S/N ratios for the FM broadcast carrier (29c). The note makes a point about this in the caption. I set my FM signal generators MOD index using the process described in this app note, counting carrier nulls.
Table 11 shows that at the 3rd carrier null, using a modulating fequency of 8.67KHz, I acheive exactly +75KHz frequency deviation (standard for FM broadcasting). To really see the nulls of the carrier, I use a 300 Hz res BW, versus the 1 KHz they use in the later app note 150. My 1971 hardcopy version uses 300 Hz res BW for figure A. Using 300 Hz res BW does not degrade the carrier powers, but the null is closer to zero in the display when adjusting the modulating signal. It is a tradeoff between scan time and resolution of the null (0 energy) to the adjacent sidebands.
The power of the carrier is not changed with res BW (better not be, or we have a problem), but with too wide of res BW, the noise floor is too high to make out the carrier null events. Figure 27 shows the carrier headed to the first Bessel null. Mathematically, the null crossing goes below 0 signal and then changes sign (phase).
Aside from the signal (carrier), KTB is a product of equivalent noise temp (NF), BW, and, and Boltzman''s constant. Minimizing one of these terms (BW) has a great effect on how much noise power is allows to be detected by the analyzer's AM detector, along with the signal, as the mixer sweeps through the scan width. So yes, signal to noise improves with lower res BWs.
How would you would go about using a spectrum analyzser to set FM modulation index on a carrier modulated with a sine wave? We used this method in the '80s to set our cmd uplinks for Hughes Satellites. Cmd sensitivty verfication is a sensitive to carrier uplink power and FM deviation. The HP 8566 was always used for this work, on the ground and in orbit. We set the res BW to very low values to get the main carrier null precisely established, since it provided the best measurement of sideband power to noise ratios within the uplink signal, at the point where the Bessel function went to 0 for a specific MOD index.
