Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ s = sum(A) # CNScalar
x = fetch(s) # native Julia scalar, e.g. Float32
```

Use `allowautofetch() do ... end` or `@allowautofetch` to permit host comparisons and numeric conversions of device scalars. Permission is task-local and disabled by default; arithmetic remains on the backend. See [Device scalars](https://julialegate.github.io/cuNumeric.jl/stable/api_cnscalar).
Use `allowfetch() do ... end` or `@allowfetch` to permit host comparisons and numeric conversions of device scalars. Permission is task-local and disabled by default; arithmetic remains on the backend. See [Device scalars](https://julialegate.github.io/cuNumeric.jl/stable/api_cnscalar).

**0D arrays support scalar-shaped arithmetic.** Use `+`, `-`, `*`, `/`, and `^`
with two 0D NDArrays or with a 0D NDArray and a Julia number. Results stay as
Expand Down
24 changes: 12 additions & 12 deletions docs/src/api_cnscalar.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ linear algebra coefficients use this shared storage path. `Ref(s)` in broadcast
also preserves device storage for either representation.

Host control parameters, including a norm's `p`, random distribution parameters,
and `isapprox` tolerances, require allowautofetch permission for either representation.
and `isapprox` tolerances, require allowfetch permission for either representation.
The `searchsorted` convenience function explicitly extracts its result indices
to construct a Julia range;
use `searchsortedfirst`/`searchsortedlast` to retain device results.
Expand All @@ -44,35 +44,35 @@ value = fetch(t) # explicit synchronization, always permitted
`cnscalar(a)` wraps an existing 0D NDArray; `s.value` accesses its backend array
without synchronizing. Use `fetch(s)` for explicit host extraction.
This extends Julia's `Base.fetch`; ordinary Julia values retain their existing
`fetch(x) == x` behavior. Explicit `fetch` does not require `allowautofetch`.
`fetch(x) == x` behavior. Explicit `fetch` does not require `allowfetch`.
Arithmetic, `min`/`max`, and promotion between numeric types keep results on the
backend. Showing a wrapper uses the backing 0D NDArray's display and extracts
its value, including in the REPL, without requiring allowautofetch permission. End
its value, including in the REPL, without requiring allowfetch permission. End
an expression with `;` to suppress REPL display and that synchronization.
The existing promotion policy still applies.

Device scalars can also be coefficients in `contract!`, `mul!`, `axpy!`,
`axpby!`, and TensorOperations calls without enabling allowautofetch.
`axpby!`, and TensorOperations calls without enabling allowfetch.

Value-dependent conversions, such as floating point to integer or complex to
real, require allowautofetch permission even when the target is another CNScalar.
real, require allowfetch permission even when the target is another CNScalar.
They preserve Julia's `InexactError` checks instead of silently truncating values.

## Scoped permission

Implicit host extraction is disabled by default. Comparisons, scalar predicates,
and conversion to supported host numeric types require `allowautofetch` permission:
and conversion to supported host numeric types require `allowfetch` permission:

```julia
@allowautofetch s > 0 # Julia Bool
allowautofetch() do
@allowfetch s > 0 # Julia Bool
allowfetch() do
Float64(s) # Julia Float64
end
```

The scope returns the body's result and restores the previous permission even
when the body throws. Nested `allowautofetch(false) do ... end` disables extraction
temporarily. `allowautofetch(true)` / `allowautofetch(false)` set the calling task's
when the body throws. Nested `allowfetch(false) do ... end` disables extraction
temporarily. `allowfetch(true)` / `allowfetch(false)` set the calling task's
permission until changed again. Independent tasks do not inherit permission.
This permission is separate from `allowscalar` and `allowpromotion`.

Expand All @@ -85,11 +85,11 @@ function host_residual(x)
return Float64(r) # permission checked here, inside this function
end

residual = @allowautofetch host_residual(x)
residual = @allowfetch host_residual(x)
# host_residual(x) # errors outside the scope
```

Arithmetic remains asynchronous even inside an allowautofetch scope. There is no
Arithmetic remains asynchronous even inside an allowfetch scope. There is no
general fallback that fetches arguments when Julia cannot find a method.
For example, a function accepting only `Float64` still needs `f(Float64(s))`.
Julia also requires an actual Bool in `if`: use `Bool(all(A))` within the scope,
Expand Down
6 changes: 3 additions & 3 deletions docs/src/linalg.md
Original file line number Diff line number Diff line change
Expand Up @@ -369,10 +369,10 @@ using cuNumeric, Krylov

A = NDArray([4.0 1.0; 1.0 3.0])
b = NDArray([1.0, 2.0])
x, stats = @allowautofetch Krylov.cg(A, b; rtol=1e-8)
x, stats = @allowfetch Krylov.cg(A, b; rtol=1e-8)
```

For complex systems, use `@allowpromotion @allowautofetch Krylov.cg(...)`:
For complex systems, use `@allowpromotion @allowfetch Krylov.cg(...)`:
CG computes real scalar coefficients that must promote when scaling complex
vectors.

Expand All @@ -383,7 +383,7 @@ The `HaloVector` in that example illustrates a custom vector type; this
extension defines the corresponding methods for `NDArray`.

To reuse allocations, construct `workspace = Krylov.CgWorkspace(A, b)` and call
`@allowautofetch Krylov.cg!(workspace, A, b)`. Other Krylov solvers may require
`@allowfetch Krylov.cg!(workspace, A, b)`. Other Krylov solvers may require
additional integration methods.

## Not available yet
Expand Down
44 changes: 22 additions & 22 deletions src/cnscalar.jl
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
export CNFloat, CNInt, CNUInt, CNBool, CNReal, CNComplex, CNScalar, DeviceScalar,
cnscalar, allowautofetch, @allowautofetch
cnscalar, allowfetch, @allowfetch

"A floating-point device scalar; wrapping its 0D storage does not synchronize."
struct CNFloat{T<:AbstractFloat,P} <: AbstractFloat
Expand Down Expand Up @@ -55,7 +55,7 @@ _host_isone(::CNScalar) = false
# permission-checked for both wrapped and unwrapped device scalars.
_maybe_fetch(x) = x
function _maybe_fetch(x::DeviceScalar{<:Real})
_assert_allowautofetch()
_assert_allowfetch()
return only(_scale_storage(x))
end

Expand All @@ -79,37 +79,37 @@ fill(x::DeviceScalar, dims::Int...) = fill(x, dims)
fill(x::DeviceScalar, dim::Int) = fill(x, (dim,))

"""
allowautofetch(f, allow=true)
allowautofetch(allow::Bool=true)
@allowautofetch expression
allowfetch(f, allow=true)
allowfetch(allow::Bool=true)
@allowfetch expression

Permit implicit host extraction of CNScalars for comparisons, predicates, and
conversion to host numeric types. Arithmetic stays on the backend. The do-block
and macro restore the calling task's previous permission, including on errors.
Permission is task-local and separate from scalar indexing and promotion.
This is not a fallback for arbitrary functions with unsupported argument types.
"""
allowautofetch(f::F, allow::Bool=true) where {F} =
task_local_storage(f, :cuNumericAllowAutoFetch, allow)
allowautofetch(allow::Bool=true) = (task_local_storage(:cuNumericAllowAutoFetch, allow); nothing)
allowfetch(f::F, allow::Bool=true) where {F} =
task_local_storage(f, :cuNumericAllowFetch, allow)
allowfetch(allow::Bool=true) = (task_local_storage(:cuNumericAllowFetch, allow); nothing)

macro allowautofetch(ex)
macro allowfetch(ex)
quote
local previous = get(task_local_storage(), :cuNumericAllowAutoFetch, nothing)
task_local_storage(:cuNumericAllowAutoFetch, true)
local previous = get(task_local_storage(), :cuNumericAllowFetch, nothing)
task_local_storage(:cuNumericAllowFetch, true)
@__tryfinally($(esc(ex)),
if isnothing(previous)
delete!(task_local_storage(), :cuNumericAllowAutoFetch)
delete!(task_local_storage(), :cuNumericAllowFetch)
else
task_local_storage(:cuNumericAllowAutoFetch, previous)
task_local_storage(:cuNumericAllowFetch, previous)
end)
end
end

function _assert_allowautofetch()
get(task_local_storage(), :cuNumericAllowAutoFetch, false) && return nothing
function _assert_allowfetch()
get(task_local_storage(), :cuNumericAllowFetch, false) && return nothing
throw(ArgumentError("Implicit CNScalar host extraction is disabled. Use " *
"allowautofetch() do ... end or @allowautofetch, or explicitly call fetch(x)."))
"allowfetch() do ... end or @allowfetch, or explicitly call fetch(x)."))
end

"""
Expand All @@ -118,7 +118,7 @@ end

Retrieve a native Julia scalar, waiting for the result as needed. An NDArray
must contain exactly one element. Explicit fetching does not require
`allowautofetch` or `allowscalar` permission.
`allowfetch` or `allowscalar` permission.
"""
Base.fetch(x::CNScalar) = only(x.value)
Base.only(x::CNScalar) = fetch(x)
Expand All @@ -139,7 +139,7 @@ _scalar_operand(x::Number) = x
_scalar_operand(x::NDArray{<:Any,0}) = x
_scalar_binary(f, x, y) = cnscalar(broadcast(f, _scalar_operand(x), _scalar_operand(y)))
function _scalar_compare(f, x, y)
_assert_allowautofetch()
_assert_allowfetch()
result = broadcast(f, _scalar_operand(x), _scalar_operand(y))
value = only(result)
destroy!(result)
Expand All @@ -148,7 +148,7 @@ end
_scalar_host(x::CNScalar) = fetch(x)
_scalar_host(x::Number) = x
function _scalar_compare(f::Union{typeof(isless),typeof(isequal)}, x, y)
_assert_allowautofetch()
_assert_allowfetch()
return f(_scalar_host(x), _scalar_host(y))
end

Expand Down Expand Up @@ -231,7 +231,7 @@ Base.abs2(x::CNComplex) = real(x * conj(x))
Base.:!(x::CNReal{Bool}) = _scalar_unary(!, x)
for op in (:iszero, :isone, :isfinite, :isinf, :isnan), W in _SCALAR_WRAPPERS
@eval function Base.$op(x::$W)
_assert_allowautofetch()
_assert_allowfetch()
return $op(fetch(x))
end
end
Expand All @@ -241,7 +241,7 @@ _scalar_convert(::Type{T}, x::Number) where {T} = cnscalar(NDArray(convert(T, x)
function _checked_scalar_convert(::Type{T}, x::CNScalar) where {T}
# These conversions can throw InexactError based on the value. A backend
# dtype cast would silently truncate or discard an imaginary component.
_assert_allowautofetch()
_assert_allowfetch()
return cnscalar(NDArray(convert(T, fetch(x))))
end
_scalar_convert(::Type{T}, x::CNComplex) where {T<:Real} = _checked_scalar_convert(T, x)
Expand Down Expand Up @@ -272,7 +272,7 @@ Base.float(::Type{S}) where {S<:CNScalar} = _scalar_type(float(_scalar_eltype(S)

for T in Base.uniontypes(SUPPORTED_ARRAY_TYPES), W in _SCALAR_WRAPPERS
@eval function Base.convert(::Type{$T}, x::$W)
_assert_allowautofetch()
_assert_allowfetch()
return convert($T, fetch(x))
end
@eval (::Type{$T})(x::$W) = convert($T, x)
Expand Down
38 changes: 19 additions & 19 deletions test/array/cnscalar.jl
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ struct CNScalarNumberSlot{T<:Number}
end

@testset "CNScalar storage and arithmetic" begin
allowautofetch(false)
allowfetch(false)
cuNumeric.allowscalar(false)
cuNumeric.allowpromotion() do
for T in Base.uniontypes(cuNumeric.SUPPORTED_ARRAY_TYPES)
Expand All @@ -30,7 +30,7 @@ end
@test fetch(a) == one(T)
@test only(x) == fetch(x)
@test_throws ErrorException x[]
@test_throws ErrorException (@allowautofetch x[])
@test_throws ErrorException (@allowfetch x[])
cuNumeric.allowscalar() do
@test x[] === a[]
@test x[] == one(T)
Expand All @@ -49,9 +49,9 @@ end
@test fetch(op(x)) ≈ op(one(T))
end
@test_throws ArgumentError convert(T, x)
@test allowautofetch(() -> convert(T, x)) == one(T)
@test allowfetch(() -> convert(T, x)) == one(T)
@test_throws ArgumentError x == one(T)
@test (@allowautofetch x == one(T))
@test (@allowfetch x == one(T))
@test_throws ArgumentError x == one(T)
end
x = sum(NDArray([1.0, 2.0, 3.0]))
Expand All @@ -60,9 +60,9 @@ end
@test fetch(sqrt(x)) ≈ sqrt(6.0)
@test Array(NDArray([1.0, 2.0]) .* x) == [6.0, 12.0]
@test fetch(max(x, 2.0)) == 6.0
@test (@allowautofetch x > 2.0)
@test (@allowautofetch isless(2.0, x))
@test (@allowautofetch 2x) isa CNScalar
@test (@allowfetch x > 2.0)
@test (@allowfetch isless(2.0, x))
@test (@allowfetch 2x) isa CNScalar
p, q = promote(x, 2.0)
@test p isa CNScalar && q isa CNScalar
@test fetch(p) == 6.0 && fetch(q) == 2.0
Expand All @@ -73,10 +73,10 @@ end
end
fractional = cnscalar(NDArray(1.5))
@test_throws ArgumentError CNInt{Int64}(fractional)
@test_throws InexactError @allowautofetch CNInt{Int64}(fractional)
@test_throws InexactError @allowfetch CNInt{Int64}(fractional)
imaginary = cnscalar(NDArray(1.0 + 2.0im))
@test_throws ArgumentError CNFloat{Float64}(imaginary)
@test_throws InexactError @allowautofetch CNFloat{Float64}(imaginary)
@test_throws InexactError @allowfetch CNFloat{Float64}(imaginary)
end
end

Expand Down Expand Up @@ -110,29 +110,29 @@ end
@test Array(a) == Float32[2, 2]
end

@testset "Scoped allowautofetch" begin
@testset "Scoped allowfetch" begin
x = cnscalar(NDArray(2.0))
@test allowautofetch(() -> 42) == 42
@test (@allowautofetch 43) == 43
@test_throws ErrorException allowautofetch() do
@test allowfetch(() -> 42) == 42
@test (@allowfetch 43) == 43
@test_throws ErrorException allowfetch() do
error("scope test")
end
@test_throws ArgumentError Float64(x)
@allowautofetch begin
@allowfetch begin
@test Float64(x) == 2.0
allowautofetch(false) do
allowfetch(false) do
@test_throws ArgumentError Float64(x)
end
@test Float64(x) == 2.0
# Independent tasks do not inherit this permission.
@test fetch(@async get(task_local_storage(), :cuNumericAllowAutoFetch, false)) == false
@test fetch(@async get(task_local_storage(), :cuNumericAllowFetch, false)) == false
end
@test_throws ArgumentError Float64(x)
@test_throws ErrorException @allowautofetch error("macro scope test")
@test_throws ErrorException @allowfetch error("macro scope test")
@test_throws ArgumentError Float64(x)
allowautofetch(true)
allowfetch(true)
@test Float64(x) == 2.0
allowautofetch(false)
allowfetch(false)
@test_throws ArgumentError Float64(x)
end

Expand Down
2 changes: 1 addition & 1 deletion test/array/diagonal_updates.jl
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ end
end
for p in (NDArray(2), cnscalar(NDArray(2)))
@test_throws "Implicit CNScalar host extraction is disabled" norm(D, p)
@allowautofetch @test fetch(norm(D, p)) ≈ 5f0
@allowfetch @test fetch(norm(D, p)) ≈ 5f0
@test fetch(norm(D, fetch(p))) ≈ 5f0
end
end
Expand Down
2 changes: 1 addition & 1 deletion test/array/tensoroperations.jl
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,7 @@ using TensorOperations: TensorOperations as TO
reduced = sum(C)
@test reduced isa CNScalar
@test ndims(reduced) == 0
allowautofetch(false) do
allowfetch(false) do
for α in (NDArray(sum(hostC)), reduced)
@tensor scaled[i, j] := α * C[i, j]
@test Array(scaled) ≈ sum(hostC) .* hostC
Expand Down
8 changes: 4 additions & 4 deletions test/gpu_only/krylov.jl
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ using Krylov
α, β = sum(NDArray([a])), sum(NDArray([b]))
x = NDArray(hostx)
# Neither helpers nor coefficient dispatch should implicitly fetch.
allowautofetch(false) do
allowfetch(false) do
for s in (α, α.value)
y = NDArray(copy(hosty))
@test Krylov.kaxpy!(3, s, x, y) === y
Expand All @@ -32,11 +32,11 @@ using Krylov
workspace = Krylov.CgWorkspace(Krylov.KrylovConstructor(rhs))
tol = 20 * eps(R)
# Complex CG has real coefficients; permit their promotion to complex.
@allowpromotion @allowautofetch Krylov.cg!(workspace, A, rhs; atol=zero(R), rtol=tol, itmax=100, history=true)
@allowpromotion @allowfetch Krylov.cg!(workspace, A, rhs; atol=zero(R), rtol=tol, itmax=100, history=true)
@test workspace.stats.solved
@test norm(hostA * Array(workspace.x) - hostb) / norm(hostb) <= 5tol
@test !isempty(workspace.stats.residuals)
solution, stats = @allowpromotion @allowautofetch Krylov.cg(A, rhs; atol=zero(R), rtol=tol, itmax=100)
solution, stats = @allowpromotion @allowfetch Krylov.cg(A, rhs; atol=zero(R), rtol=tol, itmax=100)
@test stats.solved
@test solution isa NDArray
@test norm(hostA * Array(solution) - hostb) / norm(hostb) <= 5tol
Expand All @@ -45,7 +45,7 @@ using Krylov
nonsymmetric = Matrix(Tridiagonal(fill(T(-0.3), n - 1), fill(T(4), n), fill(T(-0.8), n - 1)))
B = NDArray(nonsymmetric)
biworkspace = Krylov.BicgstabWorkspace(B, rhs)
@allowpromotion @allowautofetch Krylov.bicgstab!(biworkspace, B, rhs; atol=zero(R), rtol=tol, itmax=100)
@allowpromotion @allowfetch Krylov.bicgstab!(biworkspace, B, rhs; atol=zero(R), rtol=tol, itmax=100)
@test biworkspace.stats.solved
@test norm(nonsymmetric * Array(biworkspace.x) - hostb) / norm(hostb) <= 5tol
end
Expand Down
Loading