Windows support

[viktorerlingsson]

It would be nice if LavinMQ worked on Windows.

I have done some testing, and unfortunately it's not super straight-forward. Below are some of my findings.

Some info on porting to Windows can be found here https://github.com/crystal-lang/crystal/wiki/Porting-to-Windows#running-crystal-compiler-on-windows-bootstrapping

Current state

Branch: https://github.com/cloudamqp/lavinmq/tree/windows-support

LavinMQ can be started with a fresh datadir. HTTP server and mgmt GUI is working.
Queues, vhosts, exchanges can be created. Messages can be published and read.

A lot of things are not working though.
LavinMQ can not be started when there is data in the datadir.
Most specs are faliing.

Findings

Flag

You can use {% if flag?(:windows) %} or {% if flag?(:win32) %} to do Windows-specific things.

MFiles / Files

Memory mapped files exists in windows as well, see https://learn.microsoft.com/en-us/windows/win32/memory/file-mapping . But I haven't gotten that to work properly yet, so I have used fallback to regular files instead. Crystal's own file handling seems to be enough.

That also comes with a bunch of problems though, mainly because of how we read (and write) data to IO's in different places in both LavinMQ and AMQProtocol.cr. Performance is also pretty bad using regular files.

I've had some problems with reading existing messages on startup, where we read the wrong bytes in some places. I think this has to do with how and when we update the pos when reading data.
There might also be some differences with how uninitialized slices work, but I'm not sure.

Files has to be opened with a or a+ mode to ensure we write to the end of the file.

Paths

Paths are not as standardized in Windows. They are generally formatted like C:\dir\file.ending. Should work if we just use windows paths.

System info

Below follows some ways of getting system info like disk stats, memory usage, etc, on Windows.

Disk stats

fun GetDiskFreeSpaceExA(lpRootPathName : Char*, lpFreeBytesAvailableToCaller : Void*, lpTotalNumberOfBytes : Void*, lpTotalNumberOfFreeBytes : Void*) : Bool
bavail = blocks = c = 0_i64
LibC.GetDiskFreeSpaceExA(path.check_no_null_byte, pointerof(bavail), pointerof(blocks), pointerof(c))

Memory stats

    fun GlobalMemoryStatusEx(Void*) : Void*
    struct MEMORYSTATUSEX
      dwLength : Int
      dwMemoryLoad : Int
      ullTotalPhys : Int64
      ullAvailPhys : Int64
      ullTotalPageFile : Int64
      ullAvailPageFile : Int64
      ullTotalVirtual : Int64
      ullAvailVirtual : Int64
      ullAvailExtendedVirtual : Int64
    end

      memory = LibC::MEMORYSTATUSEX.new
      memory.dwLength = sizeof(LibC::MEMORYSTATUSEX)
      ptr = pointerof(memory)
      mem = LibC.GlobalMemoryStatusEx(ptr)
      memory.ullTotalPhys

PAGE_SIZE =

fun GetSystemInfo : SYSTEM_INFO
PAGE_SIZE = LibC.GetSystemInfo.@dwPageSize

set_rlimit

fun _setmaxstdio(max : Int) : Int

getrusage

times = Process.times
ResourceUsage.new(times.@utime, times.@stime, 0_i64, 0_i64, 0_i64, 0_i64, 0_i64, 0_i64, 0_i64)

Other

SystemD - fixed by this

Sockets - Haven't looked into this, but can not bind to unix sockets.

[viktorerlingsson]

Probably not putting that much more work into this at the moment, but we might pick it up later.

[carlhoerberg]

Yes, in MFile we only update pos when reading, not when appending, which is different from how normal File works. If you have opened a File in a+ mode you can read at any pos, but when you write to that file the pos will be reset to the end of the file, in MFile we skip that part for performance reasons.

[carlhoerberg]

I thought that UNIX domain sockets wasn't available in Windows, but it looks like i was wrong: https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/

[carlhoerberg]

For cross platform compatible paths we https://devdocs.io/crystal/path