Skip to content

AGENTS.md

Latest commit

 

History

History
206 lines (162 loc) · 8.09 KB

AGENTS.md

File metadata and controls

206 lines (162 loc) · 8.09 KB
Raw

NMR Quantification Analysis - Technical Summary

Project Context

Analysis of pure-shift NMR data for absolute metabolite quantification. Reference library and model mixture data provided as JCAMP-DX files. Goal: replicate results from paper's Table S5 and SI.

Critical Discoveries

1. Formula Correction (MAJOR)

Original (incorrect) formula in quantify_all_metabolites_v3_ref20.py:

calc_conc = ref_conc * scale * scale_tsp  # DOUBLE TSP CORRECTION - WRONG

Correct formula:

# For TSP-normalized spectra:
y_ref = spec_ref[mask] / tsp_area_ref
y_samp = spec_samp[mask] / tsp_area_samp

# Lorentzian fitting gives amplitudes from TSP-normalized spectra
samp_amp = result_samp.params['amplitude'].value
ref_amp = result_ref.params['amplitude'].value
scale = samp_amp / ref_amp  # This IS Scale(M)/Scale(TSP)

calc_conc = ref_conc * scale  # CORRECT - no additional * scale_tsp

Mathematical justification:

  • samp_amp = (sample_metabolite_signal / sample_TSP_area)
  • ref_amp = (ref_metabolite_signal / ref_TSP_area)
  • scale = samp_amp / ref_amp = Scale(M) / Scale(TSP)
  • Therefore: [M] = [M]_ref × Scale(M) = [M]_ref × scale × Scale(TSP)
  • But since scale = Scale(M)/Scale(TSP), then [M] = [M]_ref × scale automatically!

2. TSP Signal Variation Mystery (MAJOR)

Our calculated Scale(TSP) from magnitude spectra:

File Scale(TSP) vs File 10
File 10 1.000 reference
File 20 2.188 2.2×
File 30 2.765 2.8×
File 40 7.703 7.7×
File 50 3.469 3.5×
File 60 3.470 3.5×
File 70 7.597 7.6×

Table S5 from paper:

File Scale(TSP)
File 10 1.0000
File 20 0.9932
File 30 0.6607
File 40 0.8964
File 50 0.8992
File 60 0.9546
File 70 0.8852

Range comparison:

  • Our data: 1.0× to 7.7× (factor of 7.7 variation)
  • Table S5: 0.66 to 1.00 (factor of 1.5 variation)

This is the central unresolved issue.

Technical Details

Data Files

  • Location: raw_data/Reference_Raw_Date_JCAMP-DX/Alanine-Reference/
  • Naming: {10,20,30,40,50,60,70}.dx (main files)
  • Also odd-numbered files exist (11, 21, etc.) - purpose unclear

File Header Information (from 11.dx)

  • Alanine concentration: 40 mM
  • TSP concentration: 1 mM (from SI PDF)
  • Solvent: 90% H₂O / 10% D₂O
  • Spectrometer: 600 MHz
  • Temperature: 300K
  • Sequence: NoesypresatSAPPHIRE

JCAMP-DX Data Structure

Data is stored as complex pairs (real, imaginary). Two methods attempted for ppm scale calculation:

Method 1 - CORRECT (uses $O1/$SWH):

sfo1 = float(dic['$SFO1'][0])    # Spectrometer frequency in MHz
o1_hz = float(dic['$O1'][0])      # Carrier frequency in Hz
sw_hz = float(dic['$SWH'][0])     # Spectral width in Hz
o1_ppm = o1_hz / sfo1
sw_ppm = sw_hz / sfo1
ppm = np.linspace(o1_ppm + sw_ppm/2, o1_ppm - sw_ppm/2, n_points)

Method 2 - INCORRECT (uses $OFFSET):

# This method produces wrong ppm scale - TSP signal not found correctly
offset_hz = float(dic['$OFFSET'][0])  # Display offset, not spectral reference

Conclusion: Method 1 is correct. $OFFSET in these files is a display parameter, not the chemical shift reference.

Phase Correction Parameters

Each file contains $PHC0 and $PHC1 phase correction values, but:

  • Applying them produces spectra with 79% negative values in "absorption" mode
  • Phase-corrected absorption still has large dispersion component
  • These parameters do NOT produce proper pure-absorption spectra
  • Authors likely used automatic phase correction in TopSpin before data export

Signal Processing Pipeline

Current validated approach (in quantify_all_metabolites_v3_ref20.py):

  1. Read JCAMP-DX → complex spectrum (real + imag)
  2. Calculate magnitude → sqrt(real² + imag²)
  3. Find TSP position → max of magnitude in [-0.5, 0.5] ppm region
  4. Shift ppm axis → center TSP at exactly 0 ppm
  5. Integrate TSP area → trapz(magnitude[tsp_region]) over [-0.1, 0.1] ppm
  6. Normalize spectrum → spectrum / tsp_area
  7. Fit Lorentzian peaks → using lmfit to TSP-normalized spectra
  8. Calculate scale → sample_amplitude / reference_amplitude
  9. Calculate concentration → ref_conc × scale (NO additional TSP correction)

Peak Identification (Confirmed)

Chemical Shift Assignment Proton Count Expected Height (normalized to TSP=1)
0.00 ppm TSP singlet 9H 1.0 (by definition)
1.47 ppm Alanine CH₃ (doublet) 3H ~13.3× (40 mM / 1 mM × 3/9)
3.78 ppm Alanine CH (quartet) 1H ~4.4× (40 mM / 1 mM × 1/9)
4.7 ppm Water (suppressed) - Variable

Observed heights match expectations (CH₃ ~17×, CH ~5×), confirming TSP is correctly identified despite its much lower concentration.

Scripts Created

Script Purpose Status
quantify_all_metabolites_v3_ref20.py Main quantification with corrected formula ✅ Working, ~100% recovery
quantify_ref_vs_integration.py Compares [M]R vs [M]I methods ✅ Working
plot_tsp_comparison.py Original TSP comparison plot ✅ Uses correct method
plot_tsp_comparison_full_range.py Full spectrum (WRONG ppm method initially) ⚠️ Needs correction
alanine_full_range_CORRECT.py Full spectrum with correct ppm scale ✅ Correct method
plot_tsp_region_only.py Individual TSP region plots ✅ Shows raw variation
compare_tsp_methods.py Compares $O1 vs $OFFSET ppm methods ✅ Demonstrates correct method

Open Questions / Next Steps

  1. Why does our Scale(TSP) vary 7.7× while Table S5 shows only 1.5× variation?

    • Possible causes: a. Different TopSpin processing parameters (different phase correction) b. Baseline correction applied by authors c. Different integration method (area vs peak height vs curve fitting) d. The JCAMP export used different scaling per file e. Water suppression effectiveness varies between files

  2. How did the authors calculate Scale(TSP) in Table S5?

    • Need to find the exact processing pipeline they used
    • May need to re-export from TopSpin with consistent parameters
    • Or find additional metadata in the raw Bruker files

  3. Why do the odd-numbered files exist? (11, 21, 31, etc.)

    • Are they duplicates, different acquisitions, or processed differently?

  4. Does the TSP variation matter for final quantification?

    • Our formula calc_conc = ref_conc × scale is internally consistent
    • It gives ~100% recovery for most metabolites
    • But we cannot independently verify the absolute TSP correction

  5. Phase correction:

    • Current magnitude spectra are robust but not what authors used
    • Real absorption spectra after proper phase correction might show different TSP ratios
    • The $PHC0/$PHC1 values in headers don't produce proper absorption mode

Key Files to Continue Investigation

  • quantify_all_metabolites_v3_ref20.py - Current best working script
  • raw_data/Reference_Raw_Date_JCAMP-DX/Alanine-Reference/ - Reference data
  • SI.pdf - Supporting Information with Table S5
  • 2_Analytical Chemistry Data.xlsx - May contain additional metadata

Summary for Next Agent

The core issue is the TSP signal variation discrepancy:

  • We have proven the formula is correct (no double TSP correction)
  • We have proven the ppm scale calculation is correct (Method 1: $O1/$SWH)
  • We have proven TSP is correctly identified in the spectra
  • BUT our calculated Scale(TSP) values (1.0-7.7) don't match Table S5 (0.66-1.0)

Suggested investigation paths:

  1. Check if odd-numbered files (11, 21, etc.) are the "correct" processed versions
  2. Investigate if TopSpin processing parameters differ between files
  3. Try alternative TSP integration methods (peak height vs area vs Lorentzian fit)
  4. Check if baseline correction significantly affects TSP area
  5. Look at the 2_Analytical Chemistry Data.xlsx file for additional metadata
  6. Try to extract raw Bruker parameters if available