Skip to content

Fire-Aalt/BLinq

Repository files navigation

BLinq

Burst-compatible LINQ for unmanaged Unity collections, built around generic type system, IL post-processing, Roslyn validation, and source-generation.

BLinq combines practical ideas from existing LINQ libraries with an architecture designed specifically for High-Performance C# (HPC#): constrained unmanaged pipelines, ILPP delegate rewriting, and source-generated boilerplate where necessary.

Unlike LINQ, BLinq does not allocate managed iterator objects per chained operator. Unlike ZLinq, it does not execute or store managed Func<> delegates, while still exposing them in the API.

Queries are represented as nested generic value types, with each operator wrapping the previous query. This allows to store the full query shape in the type system, giving Burst a concrete pipeline with no delegate dispatch, virtual calls, or hot-path branching, while an ILPostProcessor enables the Func<> API in Burst.

using FireAlt.BLinq;

[BurstCompile]
private static void BurstedMethod(in NativeArray<int> nativeArray) 
{
    var localVar = 2;
    var sum = nativeArray
        .AsQuery()
        .Where(x => x > 0) // Supports anonymous lambdas
        .Select(x => x * localVar) // Supports capturing local vars
        .Sum(SelectBig); // Supports all LINQ overloads and static method capturing
}

private static int SelectBig(int value) 
{
    return value > 5 ? value : 0;
}

Contents

Highlights

  • Broad LINQ coverage: 97% of core LINQ operators up to .Net 10 are available, with each implemented operator covered by unit tests and benchmarks.
  • Ergonomic Func APIs: an ILPostProcessor rewrites supported LINQ-shaped Func calls into unmanaged strategy structs, so common query code remains natural while staying Burst compatible.
  • Burst-friendly query fusion: operators compose through generics all the way down, creating one concrete mega-type that represents the complete query.
  • Lazy by default: query chains are initialized lazily and do not allocate intermediate collections unless materialization is required.
  • Analyzer-guarded delegates: a Roslyn analyzer validates rewritten Func bodies so managed code cannot accidentally enter Burst paths, except for supported nested BLinq operations.
  • Source-generated numeric support: Sum() and Average() overloads are generated per numeric type with [assembly: GenerateAccumulatorFor(typeof(float2), DivisorType.Int)].
  • Manual struct path available: the delegate-free API remains available by implementing interfaces such as IPredicate<T> or ISelector<TSource,TResult> yourself.

Burst Behavior

In benchmarks, Burst showed that it can vectorize pretty complex fixed-count pipelines including Select(), Sum(), Average(), and similar non-count changing queries. Operators that can change the number of elements, such as Where(), naturally limit vectorization after that point because the resulting count is data-dependent. In practice, I found that Burst produces nearly identical assembly for queries created using BLinq, compared to manually written operators, even if the query contains multiple levels of generic depth: Burst Compiler is very good at optimizing all abstractions away.

Comparisons against regular LINQ under Burst have shown query speedups from about 2.5x to 100x depending on query shape (Benchmarks). BLinq only targets unmanaged data and unmanaged-compatible query logic, and thus it is not intended to be a drop-in replacement for managed IEnumerable<T> pipelines.

Supported Collections

BLinq generates AsQuery() extensions for the collections listed in GenerateDefaultQueryExtensions.cs, including collections from Unity.Collections and optional Entities, KrasCore, and BovineLabs.Core collections when those packages are available.

Sum and Average support the numeric accumulator types generated by GenerateDefaultAccumulators.cs.

Getting Started

Install the package via Package Manager > Install package from git URL... using:

https://github.com/Fire-Aalt/BLinq.git

Use using FireAlt.BLinq; and call AsQuery() on any Native/Unsafe unmanaged collection to use BLinq's Burst-compatible LINQ.

using FireAlt.BLinq;

var source = new NativeList(Allocator.Temp);
source.Add(1);
source.Add(2);
source.Add(3);
source.Add(4);
source.Add(5);

// Call AsQuery to apply BLinq
var seq1 = source.AsQuery().Where(x => x % 2 == 0);

// Can also be applied to Span (even inside Burst)
Span<int> span = stackalloc int[5] { 1, 2, 3, 4, 5 };
var seq2 = span.AsQuery().Select(x => x * x);

Nearly all operators up to .NET 10 are available.

You can method chain and foreach like regular LINQ, but there are some limitations. Please see Limitations for details.

Operator Reference

Operator Brief
Aggregate Applies an accumulator over the query and returns the final accumulated value.
AggregateBy Groups elements by key and accumulates a value for each key.
All Determines whether all elements of a query satisfy a predicate.
Any Determines whether the query contains any elements or any elements matching a predicate.
Append Appends a value to the end of a query.
Average Returns the average of the query or projected query values using the default accumulator.
Chunk Splits a query into contiguous temporary native chunks.
Concat Concatenates two queries.
Contains Determines whether the query contains a specified value.
Count Returns the number of elements in the query or the number matching a predicate.
CountBy Counts elements by selected key and yields key/count pairs.
DefaultIfEmpty Returns the query elements, or a singleton default value when the query is empty.
Distinct Returns distinct elements from a query using default equality.
DistinctBy Returns distinct elements according to a selected key.
ElementAt Returns the element at a zero-based index.
ElementAtOrDefault Returns the element at a zero-based index, or the default value when the index is out of range.
Except Produces the set difference of two queries using default equality.
ExceptBy Produces the set difference of two queries according to a selected key.
First Returns the first element of a query or the first element matching a predicate.
FirstOrDefault Returns the first element of a query or predicate match, or the default value when no element exists.
GroupBy Groups query elements by key and returns grouped results.
GroupJoin Correlates two queries by matching keys and groups inner matches for each outer element.
Index Incorporates the element's index into a value tuple.
Intersect Produces the set intersection of two queries using default equality.
IntersectBy Produces the set intersection of two queries according to a selected key.
Join Correlates two queries by matching keys and yields one result for each matching outer and inner pair.
JoinLeft Correlates two queries by matching keys and yields all outer elements, using default inner values for unmatched keys.
JoinRight Correlates two queries by matching keys and yields all inner elements, using default outer values for unmatched keys.
Last Returns the last element of a query or the last element matching a predicate.
LastOrDefault Returns the last element of a query or predicate match, or the default value when no element exists.
LongCount Returns the number of elements in the query as a long, or the number matching a predicate.
Max Returns the largest element in the query according to the default comparer or a supplied comparer.
MaxBy Returns the element with the maximum selected key.
Min Returns the smallest element in the query according to the default comparer or a supplied comparer.
MinBy Returns the element with the minimum selected key.
OrderBy Sorts the query in ascending order according to the element or a selected key.
OrderByDescending Sorts the query in descending order according to the element or a selected key.
Prepend Prepends a value to the beginning of a query.
Reverse Yields the elements of a query in reverse order.
Select Projects each element of a query into a new form.
SelectMany Projects each element to an inner query and flattens the resulting queries.
SequenceEqual Determines whether two queries contain equal elements in the same order.
Single Returns the only element of a query or predicate match and throws when the result is not exactly one element.
SingleOrDefault Returns the only element of a query or predicate match, or the default value when no element exists.
Skip Bypasses a specified number of elements and yields the remaining elements.
SkipWhile Bypasses elements while they match a predicate, then yields the remaining elements.
Sum Returns the sum of the query or projected query values using the default accumulator.
Take Yields a specified number of contiguous elements from the start of a query.
TakeLast Yields a specified number of contiguous elements from the end of a query.
TakeWhile Yields elements while they match a predicate.
ThenBy Adds a secondary ascending key ordering to an ordered query.
ThenByDescending Adds a secondary descending key ordering to an ordered query.
Union Produces the set union of two queries using default equality.
UnionBy Produces the set union of two queries according to a selected key.
Where Filters a query so that only elements matching a predicate are yielded.
Zip Merges two queries pairwise, optionally using a result selector.

Materialization

Operator Brief
ToNativeList Materializes a query into a NativeList<T> using the requested allocator.
ToUnsafeList Materializes a query into an UnsafeList<T> using the requested allocator.
ToNativeArray Materializes a query into a NativeArray<T> using the requested allocator.
ToNativeHashSet Materializes a query into a NativeHashSet<T> using the requested allocator.
ToNativeHashMap Materializes a query into a NativeHashMap<TKey,TValue> using selected keys and values and a requested allocator.
ToManagedList Materializes a query into a managed list.
ToManagedArray Materializes a query into a managed array.
ToManagedHashSet Materializes a query into a managed hash set.
ToManagedDictionary Materializes a query into a managed dictionary using selected keys and values.

Unsupported Core LINQ Operators

Operator Reason
Cast Unsupported because BLinq works with unmanaged value types and cannot safely model runtime reference casts.
OfType Unsupported because BLinq works with unmanaged value types and cannot safely model runtime type filtering.

Benchmarks

Benchmarks compared the relative performance of three LINQ libraries: Microsoft LINQ, zero-allocation ZLinq, and BLinq with Burst enabled/disabled.

LINQ was used as the baseline where available. Since Unity still targets netstandard2.1, some newer LINQ operators are missing: for those cases, ZLinq was used as the baseline.

Tests were run with Unity TestRunner, using 2 prewarm runs and 10 measurement runs. Safety checks were disabled for all tests, matching build defaults.

Each operator test used 10,000 elements, with multiple overloads included where relevant, such as .Count() and .Count(predicate), to stress different execution paths.

Operator LINQ ZLinq BLinq.NoBurst BLinq.Burst
Aggregate x1.00 x4.14 x3.20 x130.14
AggregateBy ----- x1.00 x2.49 x4.40
All x1.00 x3.09 x3.22 x24.58
Any x1.00 x3.61 x3.38 x19.96
Append x1.00 x1.77 x5.36 x21.01
Average x1.00 x1.06 x3.23 x118.06
Chunk ----- x1.00 x2.02 x15.37
Concat x1.00 x1.48 x4.29 x19.87
Contains x1.00 x2.90 x10.76 x14.18
Count x1.00 x4.94 x4.27 x167.50
CountBy ----- x1.00 x1.00 x4.22
DefaultIfEmpty x1.00 x1.84 x4.55 x22.92
Distinct x1.00 x1.15 x4.00 x5.87
DistinctBy ----- x1.00 x2.68 x4.05
ElementAt x1.00 x57.24 x51.21 x69.50
ElementAtOrDefault x1.00 x56.08 x51.77 x55.65
Except x1.00 x1.70 x2.47 x3.15
ExceptBy ----- x1.00 x2.61 x3.97
First x1.00 x3.39 x3.25 x17.02
FirstOrDefault x1.00 x3.29 x3.31 x24.97
GroupBy x1.00 x1.07 x1.56 x5.59
GroupJoin x1.00 x1.16 x2.03 x16.11
Index ----- x1.00 x2.93 x205.54
Intersect x1.00 x1.55 x2.37 x2.73
IntersectBy ----- x1.00 x2.41 x3.75
Join x1.00 x0.69 x1.56 x7.28
JoinLeft ----- x1.00 x2.26 x9.81
JoinRight ----- x1.00 x2.16 x9.64
Last x1.00 x283.75 x333.82 x333.82
LastOrDefault x1.00 x6.30 x5.19 x20.06
LongCount x1.00 x4.74 x4.13 x98.58
Max x1.00 x0.97 x5.81 x104.52
MaxBy ----- x1.00 x3.20 x9.92
Min x1.00 x0.86 x4.94 x101.72
MinBy ----- x1.00 x2.40 x13.35
OrderBy x1.00 x1.29 x0.94 x3.91
OrderByDescending x1.00 x1.29 x0.82 x3.77
Prepend x1.00 x1.98 x5.80 x45.11
Reverse x1.00 x2.05 x3.91 x9.63
Select x1.00 x2.09 x5.57 x158.21
SelectMany x1.00 x1.09 x3.98 x77.55
SequenceEqual x1.00 x1.42 x2.59 x21.21
Single x1.00 x3.16 x3.41 x5.41
SingleOrDefault x1.00 x4.00 x4.10 x11.87
Skip x1.00 x1.75 x4.15 x22.16
SkipWhile x1.00 x1.64 x4.32 x19.86
Sum x1.00 x1.22 x3.54 x117.36
Take x1.00 x1.74 x4.73 x24.27
TakeLast x1.00 x3.27 x9.97 x33.82
TakeWhile x1.00 x1.63 x3.87 x9.37
ThenBy x1.00 x1.16 x1.08 x3.87
ThenByDescending x1.00 x1.14 x0.99 x3.68
Union x1.00 x1.83 x2.55 x3.43
UnionBy ----- x1.00 x2.29 x4.83
Where x1.00 x1.72 x5.46 x25.45
Zip x1.00 x2.22 x4.42 x284.52

ILPP Profiling

Define BLINQ_ILPP_PROFILE to enable IL post-processor timing logs. When enabled, BLinq writes [BLinq ILPP] entries to Unity's Editor log with total ILPP time, nested stage timings, self time for parent stages, and call counts for repeated stages.

This is intended for diagnosing editor import or compilation slowdowns. Leave the define disabled during normal development if you do not need ILPP timing output.

Source Generation Attributes

BLinq exposes assembly-level attributes for extending the generated API to project-specific collection and numeric types.

GenerateQueryExtensionFor

Use GenerateQueryExtensionFor to generate an AsQuery() extension and an IQueryEnumerator for a collection type.

using FireAlt.BLinq;
using Unity.Collections;

// `LengthProperty` and `Indexer` are optional. 
// Add them for various optimizations if the collection is a continuous block of memory with appropriate `Length` and has an indexer expression `this[index]`
[assembly: GenerateQueryExtensionFor(typeof(NativeArray<>), typeof(NativeArray<>.Enumerator), LengthProperty = "Length", Indexer = true)]

The first argument is the collection type that receives AsQuery(), and the second argument is the enumerator type stored by Query<TEnumerator,T>.

Requirements:

  • The attribute must be placed at assembly scope and can be repeated for multiple source types.
  • The enumerator type must implement System.Collections.Generic.IEnumerator<T> so the generator can infer the query item type.
  • The collection type must expose a compatible GetEnumerator() method because the generated extension constructs new Query<TEnumerator,T>(collection.GetEnumerator()).
  • Generic collection and enumerator types should be passed as open generic types, such as typeof(MyCollection<>) and typeof(MyCollection<>.Enumerator).

GenerateAccumulatorFor

Use GenerateAccumulatorFor to generate the default accumulator and typed Sum() / Average() overloads for an unmanaged numeric-like type.

using FireAlt.BLinq;
using Unity.Mathematics;

[assembly: GenerateAccumulatorFor(typeof(float3), DivisorType.Int)]

The first argument is the value type to accumulate, and the second argument controls how the uint element count is used when generating Average().

Requirements:

  • The attribute must be placed at assembly scope and can be repeated for multiple accumulator types.
  • The value type must be unmanaged and support total + value and total / count expressions.
  • DivisorType.Int generates division by (int)count, which is appropriate for signed, floating-point, vector, and most custom numeric types.
  • DivisorType.UInt generates division by count, which is appropriate for unsigned scalar and vector types.
  • The generator emits a {TypeName}Accumulator : IAccumulator<T> plus overloads that allow query.Sum(), query.Average(), and selector variants without explicitly naming the accumulator type.

Limitations

BLinq has a few intentional limitations that follow from its design. These tradeoffs keep the query pipeline Burst-friendly, allocation-tight, and statically optimizable.

  • Rider/ReSharper Burst analyzer warnings: Rider/ReSharper's Burst analyzer can flag queries that use delegate-shaped APIs as managed code, even though BLinq rewrites supported calls through ILPP. I could not find a way to whitelist the declared Func method because the analyzer only evaluates the call site, so affected query blocks need local analyzer suppression comments. Disabling the analyzer globally is possible, but is not recommended for beginners in Burst.
  • Queries are temporary: BLinq queries are meant to be built and consumed locally, not stored in fields. Some operators use temporary allocation state, such as OrderBy, and the nested generic query type can become complex quickly.
  • IL2CPP generic depth: IL2CPP has a maximum generic nesting depth, which is 7 by default. Query chains should stay within that operation depth unless the IL2CPP generic depth limit is increased through IL2CPP compiler arguments: with a higher limit, longer query chains are supported.

Future C# / Unity TODO

  • CORE CLR: revise Mono.Cecil dependency.
  • CORE CLR: use static abstract interface members for aggregatable numeric types and remove the current accumulator source generator.
  • FUTURE C#: use extension operators for arithmetic so external types can participate in accumulator contracts without owning the original type.

Special Thanks to

apkdev for the idea to use Mono.Cecil to make Func<> API possible in Burst.

ZLinq for the additional architectural optimizations and certain algorithms.

About

Burst-compatible LINQ for unmanaged Unity collections, built around IL post-processing, Roslyn validation, and source-generation.

Topics

Resources

License

MIT, Unknown licenses found

Licenses found

MIT
LICENSE
Unknown
LICENSE.meta

Stars

19 stars

Watchers

1 watching

Forks

Packages

 
 
 

Contributors

Languages