Skip to content

Commit b70ceb4

Browse files
grynspangrynspan
committed
[WIP] Swift Testing image attachments (Windows)
1 parent 23517a4 commit b70ceb4

File tree

1 file changed

+377
-0
lines changed

1 file changed

+377
-0
lines changed

‎proposals/testing/NNNN-image-attachments-in-swift-testing-windows.md

Lines changed: 377 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,377 @@
1+
# Image attachments in Swift Testing (Windows)
2+
3+
* Proposal: [ST-NNNN](NNNN-filename.md)
4+
* Authors: [Jonathan Grynspan](https://github.com/grynspan)
5+
* Review Manager: TBD
6+
* Status: **Awaiting review**
7+
* Implementation: [swiftlang/swift-testing#1245](https://github.com/swiftlang/swift-testing/pull/1245), [swiftlang/swift-testing#1254](https://github.com/swiftlang/swift-testing/pull/1254), _et al_.
8+
* Review: ([pitch](https://forums.swift.org/...))
9+
10+
## Introduction
11+
12+
In [ST-0014](https://github.com/swiftlang/swift-evolution/blob/main/proposals/testing/0014-image-attachments-in-swift-testing-apple-platforms.md),
13+
we added to Swift Testing the ability to attach images (of types `CGImage`,
14+
`NSImage`, `UIImage`, and `CIImage`) on Apple platforms. This proposal builds on
15+
that one to add support for attaching images on Windows.
16+
17+
## Motivation
18+
19+
It is frequently useful to be able to attach images to tests for engineers to
20+
review, e.g. if a UI element is not being drawn correctly. If something doesn't
21+
render correctly in a CI environment, for instance, it is very useful to test
22+
authors to be able to download the failed rendering and examine it at-desk.
23+
24+
We recently introduced the ability to attach images to tests on Apple's
25+
platforms. Swift Testing is a cross-platform testing library, so we should
26+
extend this functionality to other platforms too. This proposal covers Windows
27+
in particular.
28+
29+
## Proposed solution
30+
31+
We propose adding the ability to automatically encode images to standard
32+
graphics formats such as JPEG or PNG using Windows' built-in Windows Image
33+
Component library, similar to how we added support on Apple platforms using Core
34+
Graphics.
35+
36+
## Detailed design
37+
38+
### Some background about Windows' image types
39+
40+
Windows has several generations of API for representing and encoding images. The
41+
earliest Windows API of interest to this proposal is the Graphics Device
42+
Interface (GDI) which dates back to the earliest versions of Windows. Image
43+
types in GDI that are of interest to us are `HBITMAP` and `HICON`, which are
44+
_handles_ (pointers-to-pointers) and which are not reference-counted. Both types
45+
are projected into Swift as typealiases of `UnsafeMutablePointer`.
46+
47+
Windows' latest[^direct2d] graphics API is the Windows Imaging Component (WIC)
48+
which uses types based on the Component Object Model (COM). COM types (including
49+
those implemented in WIC) are C++ classes that inherit from `IUnknown`.
50+
51+
[^direct2d]: There is an even newer API in this area, Direct2D, but it is beyond
52+
the scope of this proposal. A developer who has an instance of e.g.
53+
`ID2D1Bitmap` can use WIC API to convert it to a WIC bitmap source before
54+
attaching it to a test.
55+
56+
`IUnknown` which is conceptually similar to Cocoa's `NSObject` class in that it
57+
provides basic reference-counting and reflection functionality. As of this
58+
proposal, the Swift C/C++ importer is not aware of COM classes and does not
59+
project them into Swift as reference-counted classes. Rather, they are projected
60+
as `UnsafeMutablePointer<T>`, and developers who use them must manually manage
61+
their reference counts and must use `QueryInterface()` to cast them to other COM
62+
classes.
63+
64+
In short: the types we need to support are all specializations of
65+
`UnsafeMutablePointer`, but we do not need to support all specializations of
66+
`UnsafeMutablePointer` unconditionally.
67+
68+
### Defining a new protocol for Windows image attachments
69+
70+
A new protocol is introduced for Windows, similar to the `AttachableAsCGImage`
71+
protocol we introduced for Apple's platforms:
72+
73+
```swift
74+
/// A protocol describing images that can be converted to instances of
75+
/// ``Testing/Attachment``.
76+
///
77+
/// Instances of types conforming to this protocol do not themselves conform to
78+
/// ``Testing/Attachable``. Instead, the testing library provides additional
79+
/// initializers on ``Testing/Attachment`` that take instances of such types and
80+
/// handle converting them to image data when needed.
81+
///
82+
/// The following system-provided image types conform to this protocol and can
83+
/// be attached to a test:
84+
///
85+
/// - [`HBITMAP`](https://learn.microsoft.com/en-us/windows/win32/gdi/bitmaps)
86+
/// - [`HICON`](https://learn.microsoft.com/en-us/windows/win32/menurc/icons)
87+
/// - [`IWICBitmapSource`](https://learn.microsoft.com/en-us/windows/win32/api/wincodec/nn-wincodec-iwicbitmapsource)
88+
/// (including its subclasses declared by Windows Imaging Component)
89+
///
90+
/// You do not generally need to add your own conformances to this protocol. If
91+
/// you have an image in another format that needs to be attached to a test,
92+
/// first convert it to an instance of one of the types above.
93+
public protocol AttachableAsIWICBitmapSource {
94+
/// Create a WIC bitmap source representing an instance of this type.
95+
///
96+
/// - Returns: A pointer to a new WIC bitmap source representing this image.
97+
/// The caller is responsible for releasing this image when done with it.
98+
///
99+
/// - Throws: Any error that prevented the creation of the WIC bitmap source.
100+
func copyAttachableIWICBitmapSource() throws -> UnsafeMutablePointer<IWICBitmapSource>
101+
}
102+
```
103+
104+
And we add a conformance to this protocol to `UnsafeMutablePointer` when its
105+
`Pointee` type is one of the following types:
106+
107+
- [`HBITMAP.Pointee`](https://learn.microsoft.com/en-us/windows/win32/gdi/bitmaps)
108+
- [`HICON.Pointee`](https://learn.microsoft.com/en-us/windows/win32/menurc/icons)
109+
- [`IWICBitmapSource`](https://learn.microsoft.com/en-us/windows/win32/api/wincodec/nn-wincodec-iwicbitmapsource)
110+
(including its subclasses declared by Windows Imaging Component)
111+
112+
> [!NOTE]
113+
> The list of conforming types may be extended in the future. The Testing
114+
> Workgroup will determine if additional Swift Evolution reviews are needed.
115+
116+
A type in Swift can only conform to a protocol with **one** set of constraints,
117+
so we need a helper protocol in order to make `UnsafeMutablePointer`
118+
conditionally conform for all of the above types. This protocol must be `public`
119+
so that Swift Testing can refer to it in API, but it is an implementation detail
120+
and not part of this proposal:
121+
122+
```swift
123+
public protocol _AttachableByAddressAsIWICBitmapSource {}
124+
125+
extension HBITMAP.Pointee: _AttachableByAddressAsIWICBitmapSource {}
126+
extension HICON.Pointee: _AttachableByAddressAsIWICBitmapSource {}
127+
extension IWICBitmapSource: _AttachableByAddressAsIWICBitmapSource {}
128+
129+
extension UnsafeMutablePointer: AttachableAsIWICBitmapSource
130+
where Pointee: _AttachableByAddressAsIWICBitmapSource {}
131+
```
132+
133+
See the **Future directions** section (specifically the point about COM and C++
134+
interop) for more information on why the helper protocol is excluded from this
135+
proposal.
136+
137+
### Attaching a conforming image
138+
139+
New overloads of `Attachment.init()` and `Attachment.record()` are provided:
140+
141+
```swift
142+
extension Attachment {
143+
/// Initialize an instance of this type that encloses the given image.
144+
///
145+
/// - Parameters:
146+
/// - image: A pointer to the value that will be attached to the output of
147+
/// the test run.
148+
/// - preferredName: The preferred name of the attachment when writing it
149+
/// to a test report or to disk. If `nil`, the testing library attempts
150+
/// to derive a reasonable filename for the attached value.
151+
/// - imageFormat: The image format with which to encode `image`.
152+
/// - sourceLocation: The source location of the call to this initializer.
153+
/// This value is used when recording issues associated with the
154+
/// attachment.
155+
///
156+
/// The following system-provided image types conform to the
157+
/// ``AttachableAsIWICBitmapSource`` protocol and can be attached to a test:
158+
///
159+
/// - [`HBITMAP`](https://learn.microsoft.com/en-us/windows/win32/gdi/bitmaps)
160+
/// - [`HICON`](https://learn.microsoft.com/en-us/windows/win32/menurc/icons)
161+
/// - [`IWICBitmapSource`](https://learn.microsoft.com/en-us/windows/win32/api/wincodec/nn-wincodec-iwicbitmapsource)
162+
/// (including its subclasses declared by Windows Imaging Component)
163+
///
164+
/// The testing library uses the image format specified by `imageFormat`. Pass
165+
/// `nil` to let the testing library decide which image format to use. If you
166+
/// pass `nil`, then the image format that the testing library uses depends on
167+
/// the path extension you specify in `preferredName`, if any. If you do not
168+
/// specify a path extension, or if the path extension you specify doesn't
169+
/// correspond to an image format the operating system knows how to write, the
170+
/// testing library selects an appropriate image format for you.
171+
public init<T>(
172+
_ image: T,
173+
named preferredName: String? = nil,
174+
as imageFormat: AttachableImageFormat? = nil,
175+
sourceLocation: SourceLocation = #_sourceLocation
176+
) where AttachableValue == _AttachableImageWrapper<T>
177+
178+
/// Attach an image to the current test.
179+
///
180+
/// - Parameters:
181+
/// - image: The value to attach.
182+
/// - preferredName: The preferred name of the attachment when writing it
183+
/// to a test report or to disk. If `nil`, the testing library attempts
184+
/// to derive a reasonable filename for the attached value.
185+
/// - imageFormat: The image format with which to encode `image`.
186+
/// - sourceLocation: The source location of the call to this initializer.
187+
/// This value is used when recording issues associated with the
188+
/// attachment.
189+
///
190+
/// This function creates a new instance of ``Attachment`` wrapping `image`
191+
/// and immediately attaches it to the current test.
192+
///
193+
/// The following system-provided image types conform to the
194+
/// ``AttachableAsIWICBitmapSource`` protocol and can be attached to a test:
195+
///
196+
/// - [`HBITMAP`](https://learn.microsoft.com/en-us/windows/win32/gdi/bitmaps)
197+
/// - [`HICON`](https://learn.microsoft.com/en-us/windows/win32/menurc/icons)
198+
/// - [`IWICBitmapSource`](https://learn.microsoft.com/en-us/windows/win32/api/wincodec/nn-wincodec-iwicbitmapsource)
199+
/// (including its subclasses declared by Windows Imaging Component)
200+
///
201+
/// The testing library uses the image format specified by `imageFormat`. Pass
202+
/// `nil` to let the testing library decide which image format to use. If you
203+
/// pass `nil`, then the image format that the testing library uses depends on
204+
/// the path extension you specify in `preferredName`, if any. If you do not
205+
/// specify a path extension, or if the path extension you specify doesn't
206+
/// correspond to an image format the operating system knows how to write, the
207+
/// testing library selects an appropriate image format for you.
208+
public static func record<T>(
209+
_ image: T,
210+
named preferredName: String? = nil,
211+
as imageFormat: AttachableImageFormat? = nil,
212+
sourceLocation: SourceLocation = #_sourceLocation
213+
) where AttachableValue == _AttachableImageWrapper<T>
214+
}
215+
```
216+
217+
> [!NOTE]
218+
> `_AttachableImageWrapper` was described in [ST-0014](https://github.com/swiftlang/swift-evolution/blob/main/proposals/testing/0014-image-attachments-in-swift-testing-apple-platforms.md#attaching-a-conforming-image).
219+
> The only difference on Windows is that its associated `Image` type is
220+
> constrained to `AttachableAsIWICBitmapSource` instead of `AttachableAsCGImage`.
221+
222+
### Specifying image formats
223+
224+
As on Apple platforms, a test author can specify the image format to use with
225+
`AttachableImageFormat`. See [ST-0014](https://github.com/swiftlang/swift-evolution/blob/main/proposals/testing/0014-image-attachments-in-swift-testing-apple-platforms.md#specifying-image-formats)
226+
for more information about that type.
227+
228+
Windows does not use Uniform Type Identifiers, so those `AttachableImageFormat`
229+
members that use `UTType` are not available here. Instead, Windows uses a
230+
variety of COM classes that implement codecs for different image formats. We
231+
provide conveniences over those COM classes' `CLSID` values:
232+
233+
```swift
234+
extension AttachableImageFormat {
235+
/// The `CLSID` value corresponding to the WIC image encoder for this image
236+
/// format.
237+
public var clsid: CLSID { get }
238+
239+
/// Construct an instance of this type with the given `CLSID` value and
240+
/// encoding quality.
241+
///
242+
/// - Parameters:
243+
/// - clsid: The `CLSID` value corresponding to a WIC image encoder to use
244+
/// when encoding images.
245+
/// - encodingQuality: The encoding quality to use when encoding images. For
246+
/// the lowest supported quality, pass `0.0`. For the highest supported
247+
/// quality, pass `1.0`.
248+
///
249+
/// If the target image encoder does not support variable-quality encoding,
250+
/// the value of the `encodingQuality` argument is ignored.
251+
///
252+
/// If `clsid` does not represent an image encoder type supported by WIC, the
253+
/// result is undefined. For a list of image encoders supported by WIC, see
254+
/// the documentation for the [IWICBitmapEncoder](https://learn.microsoft.com/en-us/windows/win32/api/wincodec/nn-wincodec-iwicbitmapencoder)
255+
/// class.
256+
public init(_ clsid: CLSID, encodingQuality: Float = 1.0)
257+
}
258+
```
259+
260+
For convenience, we also provide an initializer that takes a path extension
261+
and tries to map it to the appropriate codec's `CLSID` value:
262+
263+
```swift
264+
extension AttachableImageFormat {
265+
/// Construct an instance of this type with the given path extension and
266+
/// encoding quality.
267+
///
268+
/// - Parameters:
269+
/// - pathExtension: A path extension corresponding to the image format to
270+
/// use when encoding images.
271+
/// - encodingQuality: The encoding quality to use when encoding images. For
272+
/// the lowest supported quality, pass `0.0`. For the highest supported
273+
/// quality, pass `1.0`.
274+
///
275+
/// If the target image format does not support variable-quality encoding,
276+
/// the value of the `encodingQuality` argument is ignored.
277+
///
278+
/// If `pathExtension` does not correspond to an image format that WIC can use
279+
/// to encode images, this initializer returns `nil`. For a list of image
280+
/// encoders supported by WIC, see the documentation for the [IWICBitmapEncoder](https://learn.microsoft.com/en-us/windows/win32/api/wincodec/nn-wincodec-iwicbitmapencoder)
281+
/// class.
282+
public init?(pathExtension: String, encodingQuality: Float = 1.0)
283+
}
284+
```
285+
286+
For consistency, we will expose `init(pathExtension:encodingQuality:)` on Apple
287+
platforms too. (This is the only part of this proposal that affects platforms
288+
other than Windows.)
289+
290+
### Example usage
291+
292+
A developer may then easily attach an image to a test by calling
293+
`Attachment.record()` and passing the image of interest. For example, to attach
294+
an icon to a test as a PNG file:
295+
296+
```swift
297+
import Testing
298+
import WinSDK
299+
300+
@MainActor @Test func `attaching an icon`() throws {
301+
let hIcon: HICON = ...
302+
defer {
303+
DestroyIcon(hIcon)
304+
}
305+
Attachment.record(hIcon, named: "my icon", as: .png)
306+
// OR: Attachment.record(hIcon, named: "my icon.png")
307+
}
308+
```
309+
310+
## Source compatibility
311+
312+
This change is additive only.
313+
314+
## Integration with supporting tools
315+
316+
None needed.
317+
318+
## Future directions
319+
320+
- Adding support for projecting COM classes as foreign-reference-counted Swift
321+
classes. The C++ interop team is interested in implementing this feature, but
322+
it is beyond the scope of this proposal. **If this feature is implemented in
323+
the future**, it will cause types like `IWICBitmapSource` to be projected
324+
directly into Swift instead of as `UnsafeMutablePointer` specializations. This
325+
would be a source-breaking change for Swift Testing, but it would make COM
326+
classes much easier to use in Swift.
327+
328+
In the context of this proposal, `IWICBitmapSource` would be able to directly
329+
conform to `AttachableAsIWICBitmapSource` and we would no longer need the
330+
`_AttachableByAddressAsIWICBitmapSource` helper protocol. The
331+
`AttachableAsIWICBitmapSource` protocol's `copyAttachableIWICBitmapSource()`
332+
requirement would likely change to a property (i.e.
333+
`var attachableIWICBitmapSource: IWICBitmapSource { get throws }`) as it would
334+
be able to participate in Swift's automatic reference counting.
335+
336+
- Adding support for managed (.NET or C#) image types. Support for managed types
337+
on Windows would first require a new Swift/.NET or Swift/C# interop feature
338+
and is therefore beyond the scope of this proposal.
339+
340+
- Adding support for other platforms. See [ST-0014](https://github.com/swiftlang/swift-evolution/blob/main/proposals/testing/0014-image-attachments-in-swift-testing-apple-platforms.md#future-directions)
341+
for further discussion about supporting additional platforms.
342+
343+
## Alternatives considered
344+
345+
- Doing nothing. We have already added support for attaching images on Apple's
346+
platforms, and Swift Testing is meant to be a cross-platform library, so we
347+
should make a best effort to provide the same functionality on Windows and,
348+
eventually, other platforms.
349+
350+
- Using more Windows-/COM-like terminology and spelling, e.g.
351+
`CloneAttachableBitmapSource()` instead of `copyAttachableIWICBitmapSource()`.
352+
Swift API should follow Swift API guidelines, even when extending types and
353+
calling functions implemented under other standards.
354+
355+
- Making `IWICBitmapSource` conform directly to `Attachable`. As with `CGImage`
356+
in [ST-0014](https://github.com/swiftlang/swift-evolution/blob/main/proposals/testing/0014-image-attachments-in-swift-testing-apple-platforms.md#alternatives-considered),
357+
this would prevent us from including additional information (i.e. an instance
358+
of `AttachableImageFormat`). Further, it would be difficult to correctly
359+
manage the lifetime of Windows' 'image objects as they do not participate in
360+
automatic reference counting.
361+
362+
- Using the GDI+ type [`Gdiplus.Image`](https://learn.microsoft.com/en-us/windows/win32/api/gdiplusheaders/nl-gdiplusheaders-image)
363+
as our currency type instead of `IWICBitmapSource`. This type is a C++ class
364+
but is not a COM class, and so it is not projected into Swift except as
365+
`OpaquePointer` which makes it unsafe to extend it with protocol conformances.
366+
As well, GDI+ is a much older API than WIC and is not recommended by Microsoft
367+
for new development.
368+
369+
- Designing a platform-agnostic solution. This would likely require adding a
370+
dependency on an open-source image package such as [ImageMagick](https://github.com/ImageMagick/ImageMagick).
371+
Such a library would be a significant new dependency for the testing library
372+
and the Swift toolchain at large.
373+
374+
## Acknowledgments
375+
376+
Thank you to @compnerd and the C++ interop team for their help with Windows and
377+
the COM API.

0 commit comments

Comments
 (0)