AudioWizard.GetWaveformData structure

[regorxxx]

I'm failing to understand the current structure. For example lets take the first 8 points of data for 2ps/sec:

[
-60, -60, -49, 0.00372314453125,
 -21, -21, -10, 0.323944091796875,
 -20, -20, -9, 0.3531494140625,
 -20, -20, -8, -0.3931884765625,
 -17, -17, -6, -0.47894287109375,
 -18, -18, -6, 0.49261474609375,
 -18, -18, -7, -0.432281494140625,
 -23, -23, -11, -0.292633056640625
]

RMS -60, -21...
RMS_Peak -60 -21...
Peak -49 - 21...
Waveform_Peak 0.00372314453125, 0.323944091796875, 0.3531494140625, , ...

In ffmpeg is ensured that all metrics have the same sign for any point. (below 8 points)
[[0,-61.8,-3076.5,-48.6],[0.25,-58.7,-3076.5,-50.4],[0.5,-20.6,-3076.5,-9.8],[0.75,-20,-3076.5,-11.1],[1,-19.1,-3076.5,-9.4],[1.25,-20.6,-3076.5,-9],[1.5,-20.1,-3076.5,-8.1],[1.75,-19.5,-3076.5,-8.2]]

In this case I swap the sign of every second point for drawing.

In audiowaveform you have both negative and positive subpoints for every point. (below 4 points)
Array [0,0,-23,20,-31,31,-42,47]

i.e. the output is actually meant to be drawed as is.

But in your case the waveform data can be anything (?) Can you give me some insight about what it currently represent?

I presume it's just the current peak at the point sampled, and it can fall at the negative or positive side of the waveform (?)

If that's the case that makes really hard to draw anything (except a single line for a waveform), lets say at 2 point per second, there is zero security I will actually get a negative and a positive point. And interpolation is a nope for this case.

[TT-ReBORN]

I wanted to have the design as simple as possible and also save space, so I thought no need to have both.

Values 1-3 (RMS, RMS_Peak, Peak): These are magnitude-only dB values (always negative or zero). They can be drawn as single-sided waveforms showing loudness envelopes over time, similar to what you see in most audio editors' overview mode. These work fine for visualization.

1. RMS dB: Shows average loudness over time
2. RMS Peak dB: Shows RMS with decay characteristics
3. Peak dB: Shows maximum amplitude over time

Value 4 (Waveform_Peak): This is the signed linear amplitude (-1.0 to +1.0) of the sample with maximum absolute value in each time chunk.

• It only gives you ONE sample per chunk (whichever had the largest amplitude)
• That sample could be positive or negative

So yeah, at low resolution, Waveform_Peak can be problematic but I didn't encounter any issues in Georgia-ReBORN.
Maybe for Waveform_Peak we can change to min, max. I would need to check exactly how I did it in Georgia-ReBORN,
because I am sure I changed stuff in there to work seamlessly...

-TT

[TT-ReBORN]

Ok @regorxxx,
I got it now perfect, does not get better imo.
The new final data structure:

			// Push the 5 metrics per channel
			output.push_back(std::round(rms_dB));                   // RMS
			output.push_back(std::round(rmsPeak_dB));               // RMS Peak (decaying)
			output.push_back(std::round(sample_peak_dB));           // Sample Peak
			output.push_back(std::clamp(minSample[c], -1.0, 1.0));  // Min
			output.push_back(std::clamp(maxSample[c], -1.0, 1.0));  // Max

		/**
		 * The waveform bar metrics configuration.
		 * @typedef {object} waveformBarMetricsConfig
		 * @property {number} count - The number of metrics per frame.
		 * @property {object} index - The metric names to their frame indexes (e.g., rms: 0).
		 * @property {object} range - The valid ranges for each metric (e.g., [-100, 0] for dB metrics).
		 * @property {object} mode - The analysisMode values to metric names (e.g., rms: 'rms').
		 * @public
		 */
		/** @private @type {waveformBarMetricsConfig} */
		this.metrics = {
			count: 5,
			index: {
				rms: 0,
				rms_peak: 1,
				sample_peak: 2,
				min: 3,
				max: 4
			},
			mode: {
				rms: 'rms',
				rms_peak: 'rms_peak',
				sample_peak: 'sample_peak',
				waveform: 'waveform'
			},
			range: {
				rms: [-100, 0],
				rms_peak: [-100, 0],
				sample_peak: [-100, 0],
				min: [-1, 1],
				max: [-1, 1]
			}
		};

Already adjusted my JavaScript and works good.
Implemented all other stuff, multi-channel support, per-track error handling, updated API docs with methods
and nuked the WaveformMetric setter. If you want you can test it, can send you the new version.

Then for the multi-channel, we do stepSize in the index looping:

				const channels = AudioWizard.GetWaveformTrackChannels(i);
				const stepSize = metricsPerChannel * channels;

				// Restructure flat array into array of arrays (one array per time step)
				for (let j = 0; j < rawData.length; j += stepSize) {
					data.push(rawData.slice(j, j + stepSize));
				}

So this is the final form, keeping it efficient as possible.

-Tom

[TT-ReBORN]

Now everything implemented in v.0.3.0.
Check the updated API docs, and my Georgia-ReBORN waveform updated code:
TT-ReBORN/Georgia-ReBORN@dfa4b37

-TT

[regorxxx]

Thanks! great :) I will take a look at it on a few days

[regorxxx]

After looking at it...

Data should be structured in a JSON compatible format, which allows easy identification of channels and point data structure. i.e.

[
		[{RMS: x, peak: y , ...}, {...}, ...],  // Channel 1, 1 object per point
		[{RMS: x, peak: y, ...},  {...}, ...], // Channel 2
		...
]

Or:

		{
			channels: x,
			keys: ['RMS', 'peak', ...],
			data: [
						[[RMS, peak, ...], [RMS, peak, ...], ...], // Channel 1, 1 array per point
						[[RMS, peak, ...], ...], // Channel 2
						 ...
			]
		}

Or at least:

[
		[[RMS, peak, ...], [RMS, peak, ...], ...],  // Channel 1, 1 array per point
		[[RMS, peak, ...], ...],  // Channel 2
		 ...
]

Hardcoding an abstract structure on the logic seems a bad design if you have no restrictions imposed by third party binaries.

Self-explanatory structures are easier to understand and maintain. The 3 examples allow simple code to iterate over channels and data, without needing to previously know the amount of channels or data types. The first one requires more memory, but 2nd and 3rd one are pretty much equivalent to your existing structure in that regard, while being clearer and easier to work with.

With these 3 examples data can be saved as is and processed at a later point and the channels/data points info is already provided by the structure. Your current structure requires parsing at the exact moment data is extracted, since there is no way to know the number of channels of the source (nor the data keys used, but I can agree that doesn't change every day).

With current structure, analysis data is useless unless you know the number of channels beforhand. My proposal also keeps compatibility between downmixed and multi-channel data, since the number of channels is irrelevant for processing.

[TT-ReBORN]

Hi @regorxxx,
sorry, was too busy and focused with other stuff.

I understand the preference for self-explanatory JSON, but for a high-performance C++ component, the "Flat Stride" approach is a deliberate design choice. It follows industry standards for DSP for a few reasons:

In C++, allocating thousands of small objects or nested arrays causes memory fragmentation. A flat array of doubles is stored in a contiguous block of memory. This is CPU-cache friendly and significantly faster to access during high-speed analysis and when passing data across the COM boundary to JavaScript.

By not repeating keys like RMS or Peak for every single point, we keep the file size and memory footprint at an absolute minimum. In a large playlist, a verbose JSON structure would increase the data size by 300-500% just with redundant strings.

My goal for Audio Wizard is to be the efficient data provider. I provide the raw data and the key to read it via GetWaveformTrackChannels().

As a dev, you can then slice this flat array into any structure you prefer in JS with just a few lines of code. For example, my API docs show how to transform it into a 2D array if you want that readability for saved files:

for (let i = 0; i < track.waveformData.length; i += metricsPerPoint) {
    const pointSlice = track.waveformData.slice(i, i + metricsPerPoint);
    structuredData.push(pointSlice);
}

const jsonFile = JSON.stringify({
    channels: track.channels,
    metrics: ['rms', 'rms_peak', 'sample_peak', 'min', 'max'],
    data: structuredData // Now it's the structured format you suggested
});

The main difference is that I use time-first (all channels for Point A, then all for Point B). This is usually better for waveform rendering because you typically need the data for all channels at once to draw a single vertical line on the screen.

I would rather keep the C++ backend lean and mean and let the script layer handle the formatting if the user needs it. It gives you the best of both worlds, maximum speed and total formatting freedom.

Look at the provided API example methods, startWaveformAnalysisFileSaving.

Or in analyzeData.

-TT

[regorxxx]

Personally I disagree and I see your approach as premature optimization (omitting the first example) for no reason.

The aim of the activeX object is to be used within JS scripts, not by other c++ components... so for exposed methods I would say the output should be the one which simplifies usage on target language, not the opposite.

I mean, I have no idea about your internals... but if I call GetWaveformData to get data in JS, I don't care about the internal representation of the data by your component (which I agree, may be used at other places); but I want output printed in a format easy to work with. I'm not calling such method every 100 ms, I'm doing it once per track, so memory usage seems pretty irrelevant here. Or at least I'm failing to see how an export to js function should affect memory or performance at all in the use cases seen within foobar.

If you are talking about activeX being used by other processes, then I still see a point in an additional method meant only for JS usage.

The main difference is that I use time-first (all channels for Point A, then all for Point B). This is usually better for waveform rendering because you typically need the data for all channels at once to draw a single vertical line on the screen.

This seems to imply the main usage is real time without data saving, but as I have noted, there is no way to know the number of channels if data is saved as is, provided by your component. You have to actually call another method or have the source handle, and then process the data before saving it to JSON, with a variable indicating the number of channels...

Also if actual speed tests with more modern hardware already showed other binaries may be faster than your component, I see no point in re-analyzing a file everytime I want to play it (in that case I could agree with your remark). I would simply prefer to store and cache the data, and then I prefer data to have a format easy to use. So if data is output once, and done... I fail to see how any optimization here is relevant.

[TT-ReBORN]

The issue is that you're thinking in terms of JavaScript convenience without considering the COM marshalling constraints and C++ performance implications. Let me explain why the current flat array structure is the superior approach.

COM Marshalling Overhead

Using a deeply nested JSON-like structure (e.g., an array of thousands of objects like {RMS: x, peak: y}) would create massive overhead when crossing the COM boundary:

Object Creation Cost:
For every single data point, the COM layer would need to allocate and wrap a new IDispatch object or complex VARIANT structure.

COM Object Overhead:
Thousands of small COM objects = memory fragmentation and increased risk of heap exhaustion in the scripting engine.

Property Access Overhead:
Each nested object would require IDispatch wrapping, and every property access creates a separate COM call.

Scale Impact:
For 10,000 data points with 5 properties each, you'd have 50,000+ COM calls just to access the data, compared to one SAFEARRAY retrieval with direct memory access.

Memory & Performance

Memory Fragmentation:
Allocating thousands of small COM objects or nested arrays leads to severe memory fragmentation in C++.

Cache Efficiency:
Flat arrays are stored in contiguous memory -> CPU cache-friendly -> fast iteration
Nested objects are scattered in memory -> cache misses -> significantly slower

Data Bloat:
Repeating keys like "RMS", "peak", etc. for every point would increase largely the data size.

Stability & Garbage Collection

GC Pressure: In JavaScript, receiving an array containing 10,000+ objects (each with multiple properties) places a massive burden on the JS engine's Garbage Collector:

• Noticeable UI stutters
• Script engine crashes if heap space is exhausted
• Unpredictable performance degradation

Keep in mind, Audio Wizard supports resolutions from 1-1000 points/sec. Your proposed structure would be most likely unusable at higher resolutions.

JavaScript Flexibility

The current API already provides the best of both worlds:

The C++ backend delivers data in the most efficient format
Scripts can trivially reshape the data if needed for their specific use case

Example from API documentation:

structuredData = [];
for (let i = 0; i < track.waveformData.length; i += metricsPerPoint) {
    const pointSlice = track.waveformData.slice(i, i + metricsPerPoint);
    structuredData.push(pointSlice);
}

This transformation takes milliseconds in JavaScript and only needs to be done once if you're saving to a file.

-TT

[regorxxx]

I get the problems with nested JSON objects, but I really have doubts about your claims for any of these 2 structures:

[
[[RMS, peak, ...], [RMS, peak, ...], ...], // Channel 1, 1 array per point
[[RMS, peak, ...], ...], // Channel 2
...
]

Or at least this, which is clearly an improvement over your current one:

[
[RMS1, peak1, ..., RMS2, peak2, ...,,], // Channel 1, 1 array for all points
[RMS1, peak1, ...], // Channel 2
...
]

If we take your notes about nesting and avoid all the cons, at least the last proposal allows to clearly specify the number of channels, while also using flat arrays. And no way 1 vs 2-6 arrays has any measurable impact. If you insist on avoiding objects, the dic of keys can also be just an array.

[
['RMS', 'peak',], // Array of all keys
[RMS1, peak1, ..., RMS2, peak2, ...,,], // Channel 1, 1 array for all points
[RMS1, peak1, ...], // Channel 2
...
]

No nested objects, limited number of flat arrays, self-explained structure, can be saved as-is and the number of channels and keys present are clearly identifiable without requiring the source track, component version, or further processing. You still have to process every point following your instructions, but I would say that's a good compromise.

Memory Fragmentation:
Allocating thousands of small COM objects or nested arrays leads to severe memory fragmentation in C++.

I understand this point, what I highly doubt is it having any real impact on the use-case described here.

For ex. you have decided to output data as an actual array instead of a JSON-compatible string... which would clearly solve much of these issues. If the final destination of all these methods are JS parsing, then I do see a reason to optimize things for simplicity and JS usage. Again, another method could be added to just output any of my previously proposed solutions as strings solving the memory problem and dispose methods. I suppose that's why a lot of SMP methods just output a string instead of directly an object or array.

Then, I know there are thousands of threads about this topic:
https://stackoverflow.com/questions/79547645/nested-stdvectorstdvector-brings-too-many-memory-fragmentation-and-pollute
https://stackoverflow.com/questions/66908401/creating-a-matrix-class-with-stdvector-how-to-set-the-size-of-stdvector-in/66910308#66910308

But again I'm talking about the data passed as output, not maintaining such structure internally neither using it by your component at all. Call me esceptic, but I highly doubt that if you parse your current structure and convert it to one of the "c++ inefficient" ones just before output it would have any real impact. It's a thing done once.

Flat arrays are stored in contiguous memory -> CPU cache-friendly -> fast iteration
Nested objects are scattered in memory -> cache misses -> significantly slower

Is this really relevant for this usage? No one will be iterating the values thousands of times to find "something".

Anyway hope you at least reconsider my 2 latest alternatives which keep in mind your requirements and goals, while also improving the current one, as far as I understand.