File descriptor flag support

[marcbachmann]

Not sure whether it would be simpler passing just a fourth arg 😄
The backwards compatibility is probably also not needed.

This pr

• changes the arguments to an object
• adds a flags param
• adds a cache param

[marcbachmann]

We could also remove the cache fallback as there is one in the aligned-block-file module

[marcbachmann]

I'm currently experimenting with the flags.
Maybe it's better to explicly set read & write mode instead of passing the flags.

[marcbachmann]

I wonder why we have to use truncate instead of just resetting the in the blocks file. https://github.com/flumedb/flumelog-offset/blame/master/frame/recoverable.js#L51-L65

That way it would read-only compatible and also restorable from there.

[marcbachmann]

The flag itself is really usable. But additionally I was trying to refresh an already opened file. That didn't work because of the truncate and the other process which was writing to the file.

[marcbachmann]

I slightly changed the args again to use the filename as first arg and the options as second.
That way it's similar to the api of fs.createReadStream and pull-file

[marcbachmann]

Damn, this is still wrong. Opts could be a codec.

[dominictarr]

no truncate: #4 (comment)

That is quite a good idea... hmm, it may mean that append mode doesn't work, though. It could just check if it's read only, set the offset and skip truncating. are you expecting to read from a file that is being appended to by someone else? cos that will have other edge cases.

[dominictarr]

thanks, merged into 3.1.0

[marcbachmann]

Hmm. we still need to update the arg check in to make it completely backwards-compatible. #4 (comment)

typeof obj !== 'object' || obj.decode would do it

are you expecting to read from a file that is being appended to by someone else? cos that will have other edge cases.

Yes. I'm currently writing my own read-only version. I don't need all the caching in the block-file and reads are already optimized. We might need to modularize the module a bit.

e.g.

var file = require('pull-file')
pull(
  file(`${__dirname}/index`, {live: true}),
  pull.map(decode(codec)),
  pull.log()
)

[marcbachmann]

I'll write views in a separate process to ensure uptime and reliabilty of the first process.
Views not needed to ensure writes, aka indexes will stay in the main process.

[marcbachmann]

Next time I should add a WIP annotation somewhere. 😄
Fix in #5

[marcbachmann]

The pull stream version also doesn't work of course because it can't restore if the last block gets written. So I'll use pub/sub on the master process.

Skipping the truncate function would be nice if flags[0] == 'r'.
I'll keep you updated.