Notes: provider specialization as a performance direction - #204
Merged
Conversation
Emit against the concrete provider the consumer already references, rather than the provider-agnostic ADO.NET surface. The note is the measured case, the specific techniques, how detection would work, and a sequencing with exit criteria. The opportunity, measured as allocation per operation against PostgreSQL: from where Dapper.AOT is today to hand-tuned ADO.NET is a 46% cut on a single-row lookup and 20% over a hundred rows, all of it reachable from generated code within the existing contract. Going further and bypassing ADO.NET entirely is worth a further 9%, so the ceiling is close and most of what is available is in front of us rather than behind an API redesign. Seven techniques are spelled out, since they are the actual specification: one command created once and reused, prepared once, a generic provider-specific parameter, no DbType inference per assignment, typed getters on the concrete reader, SequentialAccess plus arity from the call site, and IsDBNull before a typed get. Five are provider-agnostic and can land first. The generic-parameter question is verified rather than assumed, by scanning the shipped assemblies across every version in the local caches, and the answer narrows the plan usefully: only Npgsql has one. Microsoft.Data.SqlClient, MySqlConnector and Microsoft.Data.Sqlite all route value types through DbParameter.Value, which is object, so the box is unavoidable from outside those drivers. That makes the largest single item PostgreSQL-only today, makes it a better-bounded piece of work than it looked, and turns "add a generic parameter type" into a feature request those driver teams can be shown a number for. Two things are flagged rather than concluded. Dapper.AOT currently shows no single-row allocation win over vanilla, which wants an owner's eye before any of this is sized, since it may be a bug rather than a missing feature. And the timing column says nothing on this setup: a round trip is ~475us and a hundred extra rows cost 30us, so client cost sits inside a shadow a hundred times its size. Allocation is the honest axis; saturation is the other one and is unmeasured. Non-goals are stated: the connection model is out of reach from generated code sitting on ADO.NET, specialization must never change observable behaviour, and the agnostic path must not regress.
The detection rule was too conservative. "Specialize only when the static type proves it" declines the common case, since a great deal of real code takes DbConnection or IDbConnection from DI. Where the static type does not prove it but the provider is referenced, emit a runtime type test with the agnostic path as the else -- the test is one type check against a database round trip, and monomorphic per call site, so it predicts perfectly. Three details recorded because they decide whether that is sound: test the command rather than the connection, since NpgsqlParameter<T> needs an NpgsqlCommand and CreateCommand is typed DbCommand whatever the connection is; wrapped connections such as MiniProfiler's fail the test and take the agnostic arm, which is a feature and worth saying out loud before someone asks; and the number of specialized arms wants a cap rather than an open-ended cross product. Adds a section on how the emission should be structured, which the note was missing -- it said what to emit but not how. The case for full explicit emission is not mainly that the code is more obvious: it removes a type-erasure tax the current design imposes, since an anonymous type cannot be a type argument on a runtime type and the factory degrades to CommandFactory<object?> with a cast back; and it turns provider specialization from a plumbing problem into a line of code. The payoff is already estimated rather than aspirational: the hand-tuned ADO.NET row is essentially what full explicit emission looks like, so 1,927B to 1,035B is the estimate for this proposal specifically. Costs are stated too. Code size is the serious one, mitigated by emitting one concrete method per shape and provider rather than inlining at every call site. Behaviour migrating from library to generator is the actual work, and is why this sits downstream of parity rather than parallel to it. And generated code freezes at generation time where a library fix ships by package bump. One claim is marked as needing testing rather than asserted: whether a generic TArgs interceptor with an inline body lets the anonymous argument object stop escaping and become a stack-allocation candidate. What is certain is that the current shape forecloses it.
Sketched and compiled rather than reasoned about, because the tempting next step after removing the CommandFactory<object?> erasure is to stop erasing the argument object too -- give it a TArgs of its own and hope it stops escaping. Three measurements say no. The dominant read shape cannot reach such an overload at all. Query<Customer> supplies one type argument, so a two-parameter Query<TResult, TArgs> is not even a candidate: explicit type arguments must supply every type parameter, and C# has no partial inference. Nor can the call be written explicitly, since an anonymous type has no name to give. Unreachable by construction rather than by oversight. Two side findings recorded in case a generic-args overload is ever wanted for other reasons: it binds without OverloadResolutionPriority, because an identity conversion already beats conversion-to-object; and DynamicParameters-shaped arguments would start binding to it, which Dapper handles specially today and would need a carve-out. The object is small: 24B for a one-field anonymous type, 32B for two fields. Against the ~890B separating Dapper.AOT today from hand-tuned ADO.NET that is under 3% of the gap. And the stack allocation does not happen anyway. Probed with an anonymous object created, read once, and never crossing a call boundary, against the same object passed as object: 24.0B per iteration either way. The hoped-for saving is not there to collect. So the ~890B is elsewhere -- command, parameter collection and parameter objects per execution, the reader, the state machines, and the boxing of parameter values. Items 1, 2 and 3 of the technique list, not the argument object. The step-0 anomaly points the same way, since no single-row win over vanilla is what one would expect if the command is not being reused.
The claim that a generic-args overload binds without OverloadResolutionPriority was too general: it was tested on Execute only. The line is whether the call site states a type argument, and the two sides fail differently. With an explicit type argument, Query<Customer> supplies one and a two-parameter overload is not a candidate at all, so priority is moot rather than unnecessary -- there is nothing to prioritise. Without one, the generic overload binds on ordinary rules because an identity conversion beats conversion-to-object, now verified for Execute, dynamic-returning Query and ExecuteScalar rather than inferred from one case. Which bounds the idea precisely: a generic-args overload is available for the non-generic-result methods, most of the write path plus dynamic reads, and for nothing else. null and object-typed locals keep today's overload either way.
It was described in one sentence as "an alternative for a major", and the single-method-body shape appeared only as a code-size mitigation under costs -- backwards, since that shape is the design rather than a workaround. Now its own section, with the emitted shape sketched. Working the sketch through turned up a constraint worth recording: a helper method cannot take the anonymous argument type, because that type cannot be written as a parameter type. The resolution is that the shared body takes the extracted values rather than the argument object, which is forced rather than chosen -- and it is the good outcome, because sql then becomes a parameter and fifty call sites of the same shape collapse to one emitted method instead of fifty copies of a reader loop. The obvious objection to end-to-end emission largely evaporates, which makes the major more attractive than the note previously made it sound. Also notes what splits a shape -- operation, row type, parameter names and types -- with the name question flagged as wanting a decision against real codebases. Sequencing gains a step 3b: the major is not required by steps 1-3, which can land on the current factory shape; it is what makes them cheap to write and turns the detection switch into an if. So the honest question is whether to do steps 1-3 twice or once. The args-object dead end moves under a "closed ideas" heading, since it was longer than the proposal it was a digression from.
The note claimed fifty call sites of a shape would collapse to one emitted method, which was illustration presented as fact, and used it to conclude the code-size objection to end-to-end emission "largely evaporates". Neither is established. Putting the body inline under the Cast, and using typed.id from there, runs the same code and specializes identically. The differences are narrower than the note implied: emitted source size, compile time and golden-fixture bulk; one async state machine type per call site rather than per shape, which is the most concrete of them and is an AOT metadata argument rather than a throughput one; and debuggability. Per-operation cost is unchanged either way. The inline form is also simpler -- no shape key, no decision about whether parameter names split a shape, no helper signature to design -- so it wins by default unless shape repetition is high enough to pay for the machinery. That number is unmeasured, and the note now says to measure it across real repositories before choosing rather than assuming. What survives as forced rather than chosen: if the body is factored out at all, it must take extracted values, since an anonymous type cannot be written as a parameter type.
Passing the arguments as object plus a binder that destructures them, rather than the shared body taking extracted values, erases the argument shape out of the signature -- so the shape key collapses from (operation, row type, parameter names and types) to just (operation, row type). That fixes the sharing problem both earlier forms had, and it is the strongest argument for the factored shape. Function pointers are the obvious primitive and the measurement does not support them. Neither indirection allocates, since a static readonly delegate is built once at type init. The function pointer measured slower than the delegate, 7.09 against 4.04 ns, most likely because the JIT can speculatively inline through a delegate with a stable target and cannot do so for a pointer arriving as a parameter. And all of it is noise at the scale that matters: 2-5 ns against a ~475us round trip is about 0.001%. So the delegate form is the better default rather than the fallback -- it needs no AllowUnsafeBlocks, no detection of it, and no second emission path. Recorded that function pointers would impose exactly that: a consumer without the switch gets CS0214 from generated code, and it is compilation-wide rather than something a generated file can opt into. One simplification found while checking: the binder needs no per-provider signature. Function pointer parameters are contravariant so narrowing is rejected outright, and a provider-specific pointer type would force a separate shared body per provider -- unnecessary, since the binder can take DbCommand and cast internally, the shared body having already proven the type in its type-test branch. The cost of the variant is that the binder is an opaque call mid-body, so the JIT cannot optimise across it as it can for a fully inline body. Irrelevant against a round trip, recorded because it is the one real difference.
The shape was written out by hand -- interceptor-constrained entry, binder in a static readonly delegate, shared body with the provider type test -- and measured against the same PostgreSQL workload. It reaches the target: 1,224B against Dapper.AOT's 1,971B, a 37% cut, landing within about 10% of hand-tuned. So the three-layer shape costs nothing meaningful over hand-written code. The important finding is that command reuse is roughly 70% of the entire gap. Identical code with only the command policy differing gives 1,848B fresh against 1,224B reused, about 625B of the ~890B. Typed parameters, typed getters, SequentialAccess and the provider switch share the remaining ~265B. And fresh-command lands on Dapper.AOT's number, which is evidence that Dapper.AOT is not reusing commands in this shape. That answers step 0 by measurement rather than profiling, and reframes the work: item 1 is not one technique among seven, it is most of the prize. Positional parameters were tested and do not pay in this case: 1,283B positional against 1,224B named, no benefit and marginally worse. Npgsql appears to resolve the name mapping once at Prepare and cache it, leaving nothing to save per execution. The test was the prepared-and-reused case, which is the best case for that caching, so the idea could still pay on non-prepared or fresh-command paths -- which is where much real code apparently sits. Also records a constraint found the hard way: parameters must be declared before Prepare, since the server records the parameter list at parse time. A binder that adds lazily on first execution cannot also prepare. That is exactly why AddParameters and UpdateParameters are separate concerns, and collapsing them breaks preparation.
…paration The earlier claim that command reuse is ~70% of the gap was too coarse, and it compared against Dapper.AOT's default rather than its best available. Both are now corrected by measurement. Command handling splits into two comparable, independent wins: reusing the command object is worth ~382B, and preparing it is worth a further ~242B. One item in the technique list was hiding two. CacheCommand and StrictTypes exist already, so the plain DapperAot row is the default rather than the ceiling. CacheCommand works and captures about half of what object reuse is worth -- 190B of the ~380B -- and why it does not reach the rest is worth knowing before building anything. StrictTypes showed no reliable gain on top, inside jitter and worse in that run. Nothing reaches preparation: CanPrepare is emitted as true, but the ~242B it is worth does not appear in any configuration measured, which makes it the largest unclaimed item and one that needs no new API. Also states plainly that allocation is not perf and that perf is not yet measured. Wall-clock says nothing when the round trip dominates by a hundredfold; client CPU per operation was attempted and is only good to 5-10% because process CPU time is quantised to the scheduler tick. The instrument that would answer it is throughput under saturation, with client and server pinned to different cores. And records two decisions on the positional rewrite: it should land after or as part of the emission work rather than being retrofitted onto the current library, since the rewrite belongs where the SQL is emitted; and it applies only where the SQL is a compile-time constant literal, because a generator cannot rewrite a string it cannot see.
The claim that wall-clock says nothing because a ~475us round trip dominates was wrong, and it came from a badly configured instrument rather than from the physics: every timing run behind it used BenchmarkDotNet's short job, three warmup and three iterations. That is fine for allocation, which is counted rather than timed, and useless for timing. With a normal job the error bars fall to +/-5-8us and the differences resolve cleanly. The constant round trip is what helps -- it is a fixed offset, so the difference between two stacks is the work they do. Measured properly, Prepare() is nearly the entire timing story: same code, only preparation differing, 512.2us against 479.6us. That 32.6us is about 90% of the 36us spread across every stack measured. The corollary reorders the technique list. Command object reuse buys no time at all, only bytes -- 506.9us fresh against 512.2us reused is nothing. CacheCommand likewise. Positional parameters buy no time either, agreeing with the allocation result. So the two axes rank the work differently and preparation is first on both, while being the one thing nothing currently reaches: CanPrepare is emitted as true and nothing acts on the flag. Also records why preparation is worth more than it looks: it removes per-execution parse and plan work on the server, so it is not client CPU, buys server capacity as well as latency, and does not shrink as the network gets slower.
The measured sections were added but the conclusions around them were not updated, which is exactly the drift worth catching. Step 0 still read as open when it had been answered: the anomaly is that the default configuration neither re-uses nor prepares commands. CacheCommand recovers about half of what re-use is worth and nothing reaches preparation, so the baseline for comparison is [DapperAot, CacheCommand] rather than the bare default. The opening table and its caveat now say so too. Step 1 lumped command re-use and preparation together as one agnostic item. They separate, and they rank differently: preparation is ~90% of the timing spread and ~242B and is unclaimed, while re-use is allocation-only. Preparation is now its own step, first, with the declare-before-Prepare constraint attached, and the rest of the agnostic techniques follow as 1b with the explicit expectation that they move allocation and not timing. How to measure now says to measure both axes always -- since they rank the work differently and inferring one from the other is how this plan got mis-ordered once already -- and to never use BenchmarkDotNet's short job for timing, which is what produced the retracted claim.
The note quoted numbers from experiments that lived only in a scratch directory on one machine. They now exist as test/SpecializationProbes, which prints the tables in the note: which overload an ordinary Dapper call site binds to, what an argument object costs, whether a non-escaping one gets stack-allocated, and what the two indirections cost. No dependencies, no database, and deliberately absent from Build.csproj to match test/Usage*, so CI is unaffected. Also states plainly that the database measurements are a different matter. The four-stack table, the emitted-shape validation, the CacheCommand comparison and the timings all come from a harness in a private repository, run against one machine -- so they are evidence for planning and are not independently verifiable by a reader of this note. Better to say so than to let the two kinds of claim sit side by side looking equally checkable.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Notes only — a new working note in
notes/, plus its row in the index. No code.Emit against the concrete provider the consumer already references, rather than the provider-agnostic ADO.NET surface.
The opportunity, measured
Allocation per operation against PostgreSQL 17, single-row primary-key lookup and 100-row scan, three-column POCO:
The techniques, spelled out
Seven, since they're the actual specification: one command created once and reused; prepared once; a generic provider-specific parameter; no
DbTypeinference per assignment; typed getters on the concrete reader;SequentialAccessplus arity taken from the call site;IsDBNullbefore a typed get. Five are provider-agnostic and can land first.The generic-parameter question — verified, and it narrows the plan usefully
Scanned the shipped assemblies for a generic
*Parameter\1` across every version in the local caches:NpgsqlParameter<T>So avoiding the parameter box is PostgreSQL-only today — the other three route value types through
DbParameter.Value, which isobject. Three consequences: the largest single item is smaller and better-bounded than it looked; the 46% must be measured per provider rather than assumed to transfer; and "add a generic parameter type" becomes a feature request those driver teams can be shown a measured number for.Two things flagged rather than concluded
Non-goals, stated
The connection model is out of reach from generated code sitting on ADO.NET; specialization must never change observable behaviour; the agnostic path must not regress.
Sequencing is four steps with exit criteria, ordered so the provider-agnostic wins land before any provider-specific machinery exists.