I also agree with the slight possibility of one exception. If the TV transmitter is down in VHF low band (and some still are), there is a chance the visual signal could cause RF desensing in his CB receiver. This can make signals in your receiver appear weaker then normal or weak signals disappear altogether. There is reasonable frequency separation even if the TV transmitter is on channel 2 to prevent this problem in most cases.
The problem is the visual signal is huge in terms of power when you compare it to a normal audio RF channel. The oral signal for a TV transmitter may be 100 KW but the ERP of the visual signal could be around a megawatt! At 1/4 mile away from the tower, depending on it's height and antenna element spacing, he could be approaching the area of maximum signal level.
Typical VHF broadcast antenna patterns at 1000 feet above ground level produce peak RF density on the ground at approximately 1 mile from the tower. Lower towers place this point closer. If the front end in your CB receiver is not selective enough to reject most of this signal, just add a quality low pass filter on the back of your radio to add much more attenuation of the TV signal.