Multi-channel Silent Disco receiver: five or more channels simultaneously - #5
Multi-channel Silent Disco receiver: five or more channels simultaneously#5StratPlans wants to merge 8 commits into
Conversation
The existing ZMQ NBFM flowgraph is not usable for the Silent Disco system: its
analog.nbfm_rx audio low-pass is fixed at 2.7 kHz, so pointed at a silent disco
channel it produces something audible but badly wrong. Silent disco is wideband FM
carrying a broadcast-style stereo multiplex - L+R below 15 kHz, a 19 kHz pilot and
L-R on a 38 kHz DSB-SC subcarrier. This adds a receiver built for that.
gr_3.10_silent_disco_rx.grc QT flowgraph with a 10-channel colour chooser that
retunes the radio. The colours match the LEDs on the
headsets, so an operator reads the colour off a headset
and picks it from the menu.
make_silent_disco_iq.py generates synthetic multi-channel FM stereo IQ, so the
receiver can be tested with no hardware and no live event
Verified by running the shipped flowgraph headlessly against generated IQ: wanted
tones at 103 dB, neighbouring channels 500 kHz away at about 5 dB, 19 kHz pilot
suppressed to 2.7 dB. The 15 kHz audio passband is load-bearing - anything wider puts
the pilot into the audio.
Channel frequencies are the published Quiet Events plan and are irregularly spaced
(gaps of 400 to 1200 kHz), which is why this uses a frequency-translating filter per
channel rather than a uniform channelizer.
All stock GNU Radio blocks, no custom DSP.
Extends the single-channel receiver to demodulate several channels concurrently and
write each to its own WAV, so a whole venue can be recorded in one pass.
silent_disco_rx.py N channels concurrently, one WAV per channel. Auto-selects the
five Ultra 900 channels that fit an RTL-SDR at 2.4 Msps; 8 Msps
reaches all ten.
selftest.py runs the whole chain against generated IQ with no radio attached
One frequency-translating filter per channel rather than a polyphase channelizer. The
published channel plans are irregular - Ultra 900 gaps run 400 to 1200 kHz and Pulse
Mobile needs a 25 kHz grid to contain its carriers - so a uniform channelizer would
spend most of its bins on nothing and still need 2-3 bins stitched together per
200 kHz channel. Measured 12x real time mono and 6x stereo for five channels on eight
cores, so the efficiency argument does not bite at this scale.
Tested against generated multi-channel IQ: five channels concurrent, wanted tones at
103 dB mono and 92 dB stereo, both negative controls at 16-17 dB against a 30 dB
threshold, stereo separation 47-53 dB.
Two of the bugs found during development came from the negative controls rather than
the positive tests, which is the argument for shipping selftest.py alongside.
Passing a channel bandwidth too narrow for the 15 kHz audio passband produced a raw GNU Radio traceback from optfir.low_pass - pm_remez: band edges must be nondecreasing - which says nothing about the cause. That is the first thing anyone hits, because --channel-bw is exposed with no stated lower bound. Two guards in main(), before the flowgraph is built: - channel bandwidth too narrow for the audio filter, naming the minimum - channel bandwidth wider than the capture itself Found by running the receiver against a real off-air 96 kHz IQ capture rather than generated input. The narrow-channel case does not arise at the default 200 kHz, so synthetic testing at the design bandwidth never reached it. Also lifted the 15 kHz passband and 17 kHz stop out of the call site into named constants, since the guard and the demodulator have to agree on them.
|
Follow-up: tested against a real off-air IQ capture rather than only generated input, and it turned up an input-validation gap worth fixing before review. Passing a Nothing in that names the cause, and The 15 kHz passband and 17 kHz stop are now named constants rather than literals at the call site, since the guard and the demodulator have to agree on them. Worth noting how this was found: the defect does not arise at the default 200 kHz, so testing at the design bandwidth never reached it. It only appeared when the receiver was pointed at a 96 kHz off-air capture, which forced a narrow channel. That is an argument for the shipped Still unverified and stated in the PR body: no RF has been received live, the channel plan is from vendor material rather than measurement, and companding is unknown. |
|
Come show us it works for your points. |
GRC writes its generated top block next to the .grc, named from the flowgraph id. With id silent_disco_rx that file is silent_disco_rx.py, which collides with the command line receiver added in the follow-up. Opening the flowgraph in GNU Radio Companion - the first step in the README - would overwrite it in the user's working copy. Renaming the id to silent_disco_rx_gui keeps the two apart. Verified: compiling the flowgraph in a directory holding both now emits silent_disco_rx_gui.py and leaves the receiver byte identical.
…ation help Four fixes from an independent review of both branches. The stereo separation figure was printed but never checked, so a regression collapsing left and right into the same audio would still have passed the stereo test. It is now asserted at 20 dB. Real hardware manages far more; the threshold only has to catch a collapse to mono. --channel with a file source guessed a tuner centre from the requested channel. That is right for a radio and meaningless for a capture, whose 0 Hz is wherever the recording was made - the receiver would translate to an empty part of the spectrum and play noise with nothing on screen to say why. It now warns and names the assumption. --deviation said it applied whenever --no-agc was given. In stereo that is untrue: wfm_rcv_pll fixes deviation at 75 kHz internally and never consults the flag. The help now says so. Also dropped a line claiming the measured stereo separation matches this class of hardware. It was measured through our own generator and demodulator, so it bounds this code and says nothing about hardware.
Testing against a USRP B210 found two faults that a file source could never have exposed. The receiver passed "bufflen=16384" as a stream argument to every driver. bufflen is RTL-SDR specific; UHD rejects it outright with "Unsupported stream argument bufflen for channel 0" before the radio is ever tuned, so no USRP could be opened at all and there was no way around it from the command line. It now asks for the larger buffer and falls back to the driver's default when that is not understood. No change for an RTL-SDR. --samp-rate was handed straight to the driver. A USRP derives its rate from a master clock and takes only discrete values, so an unsupported rate produced a ValueError listing all 250 legal rates. It now names the nearest rate at or above the request, and the nearest below with a warning that a lower rate may not span every channel. It deliberately does not substitute one automatically: the channel plan has to fit inside the sample rate, and silently lowering it would push the outermost channels out of the captured band and produce silence with no explanation. survey.py is new. The channel plans are reconstructed from vendor documentation rather than measured, so if the transmitters sit anywhere else the receiver plays silence and says nothing about why. The survey sweeps the band, reports every carrier it finds with its level above the noise floor, names the nearest plan channel, and prints the --freqs string that matches what is actually on the air. Run it first.
These systems compand: the transmitter squashes the audio about 2:1 before
modulating and the headphone expands it 1:2. That pairing is where their quoted
signal to noise figure comes from, because raw narrowband FM cannot deliver it.
This receiver de-emphasised but never expanded, so the audio played and was
intelligible while its dynamics stayed squashed - quiet passages sitting too
loud, and pumping.
--expander undoes it. For a 2:1 law the expander gain is the envelope itself,
so this needs no arbitrary power function: rectify, smooth with a one pole IIR,
multiply. It sits before the AGC deliberately, because after it the AGC would
have already flattened the dynamics being restored.
It is an option rather than the default. The exact compander law is not
published and is not standardised between vendors, so applying it blindly would
be trading one wrong assumption for another.
The generator gained the matching compressor, plus a --dynamics gate, because a
constant tone has a constant envelope and a compander does nothing measurable
to one. With audio gated between 1.0 and 0.1 the source carries 20 dB of range,
and selftest.py now asserts the round trip:
uncompanded, no expander 20.0 dB
companded, no expander 10.2 dB <- halved, as a 2:1 law must
companded, with expander 20.3 dB <- restored
The middle line is the measurement of what was wrong: without this, companded
audio arrives with half its dynamic range.
|
Any time you are ready to show us this working we are excited to see it. |
Submitted for Silent Code Evolved by RF_Riderz (scoreboard user
stratplans), DEF CON 34 RF CTF.Builds on #4. That PR adds the single-channel receiver with colour-based channel selection; this one extends it to monitor many channels at once.
What is here
Adds to
Silent Disco Receivers/:silent_disco_rx.pyselftest.pyAuto-selects the five Ultra 900 channels that fit inside an RTL-SDR at 2.4 Msps. At 8 Msps it reaches all ten.
Why translators rather than a channelizer
The published channel plans are irregularly spaced. Ultra 900 gaps run 400 to 1200 kHz; Pulse Mobile needs a 25 kHz grid to contain its three carriers. A uniform
pfb_channelizer_ccfwould need roughly 58 bins to cover 10 wanted channels on a 100 kHz grid, and since each channel is about 200 kHz wide it would still need 2-3 adjacent bins stitched back together per channel. The stitching costs more than the channelizer saves, and it pins the tuner centre to the grid.Measured 12x real time mono, 6x stereo for five channels on eight cores, so the efficiency argument does not bite at N=5.
For contrast, RTLSDR-Airband does use an FFT channelizer to get constant cost per channel, but it allocates one bin per channel at 8-16 kHz audio. That cannot hold a 200 kHz stereo music channel, so it is solving a different problem.
Testing
Five channels concurrent against generated IQ:
Two of the three bugs found during development were caught by the negative controls, not the positive tests - including a mistuning control that shifted signal and tuner by the same amount, so with a file source it was arithmetically a perfect tune and passed when it should have failed. That is the argument for shipping
selftest.pyrather than just asserting it works.What has not been verified
No RF was ever received; this was developed against synthetic IQ. Concretely:
Stating that plainly rather than implying field validation.
Update: run against real RF
Everything above was measured against generated IQ, because there was no silent disco
signal to receive. Since then this code has been run against a USRP B210 and real
over-the-air signal, which found two faults a file source could never have exposed.
Broadcast FM is the same modulation as the silent disco - wideband FM, 75 kHz
deviation, 19 kHz pilot, L-R on a 38 kHz DSB-SC subcarrier. So five live FM stations
were demodulated concurrently from a single 8 Msps capture, which is the same job this
receiver does on five disco channels:
The highest correlation between any two of the five outputs is 0.0157, so these are
five independent programs rather than one signal copied five times. Spectral flatness of
0.03 to 0.26 is structured audio; noise sits near 1.0. Every channel carries real L-R
content, so the stereo path is recovering the 38 kHz subcarrier off the air rather than
passing L+R through.
What this does not show. This is broadcast FM at 96-102 MHz, not the DEF CON silent
disco at 902-928 MHz. The modulation is identical and the band is not. The channel plan
in this PR is still reconstructed from vendor documentation rather than measured off the
real system, and nothing here speaks to companding, to the real deviation, or to the
colour-to-frequency mapping. Those caveats stand exactly as written above.
Two defects found by real hardware
Any radio that validates its stream arguments could not be opened at all. The
receiver passed
bufflen=16384unconditionally; that is an RTL-SDR argument, and UHDrejects it with
Unsupported stream argument bufflen for channel 0before the radio istuned. There was no way around it from the command line. It now falls back to the
driver's default when the argument is not understood, with no change for an RTL-SDR.
An unsupported sample rate produced 250 lines of ValueError. A USRP derives its rate
from a master clock and accepts only discrete values. The error now names the nearest
rate at or above the request, and the nearest below with a warning that a lower rate may
not span every channel. It does not substitute one automatically: the channel plan has
to fit inside the sample rate, and silently lowering it would push the outermost channels
out of the captured band and produce silence with no explanation.
New: survey.py
The channel plans here are reconstructed, not measured, so if the transmitters sit
anywhere else the receiver plays silence and says nothing about why.
survey.pysweepsthe band, reports every carrier it finds with its level above the noise floor, names the
nearest plan channel, and prints the
--freqsstring matching what is actually on theair. Verified both ways: against a generated capture with five known carriers, and
against the real FM band, where it found ten stations that all land on valid US channel
assignments. Run it first.
Update: the receiver was missing the expander
These systems compand. The transmitter compresses the audio about 2:1 before
modulating and the headphone expands it 1:2, and that pairing is where the quoted
signal to noise figure comes from - raw narrowband FM does not deliver it. This
receiver de-emphasised but never expanded, so audio played and was intelligible
while its dynamics stayed squashed: quiet passages too loud, and pumping.
--expanderundoes it. For a 2:1 law the expander gain is the envelope itself, soit needs no arbitrary power function - rectify, smooth with a one-pole IIR,
multiply - and it sits before the AGC, because after it the AGC would already have
flattened the dynamics being restored.
It is an option rather than a default. The exact compander law is not published and
is not standardised between vendors, so applying it blindly would trade one wrong
assumption for another.
The generator gained the matching compressor and a
--dynamicsgate, because aconstant tone has a constant envelope and a compander does nothing measurable to
one. With the audio gated between 1.0 and 0.1 the source carries 20 dB of range,
and
selftest.pynow asserts the round trip:The middle row is the measurement of the defect: without this, companded audio
arrives with half its dynamic range.
The channel plan is better corroborated than stated above
The plan here was described as reconstructed from vendor documentation. It matches
Quiet Events' published Ultra 900 channel assignments exactly, across all thirteen:
CH1-10 at 920.1, 920.7, 921.2, 921.9, 922.3, 922.8, 923.4, 924.2, 924.7 and 925.9
MHz, plus A1 920.5, A2 922.4 and A3 926.7 MHz.
That still does not confirm what the DEF CON system transmits, which is why
survey.pyexists and why the caveats above stand. But the plan is not guesswork.