Use proper Reader and Writer semantics

[tgulacsi]

Why do you encode/marshal into a freshly allocated []byte, and do you require a []byte for unmarshal? Even proper encodings/csv is hidden behind this memory-hungry, nonscalable interface...

[matryer]

Good question.

We wanted to provide a common interface for all codecs, and since (at the time of initial writing) the JSON encoder used simple marshal/unmarshal methods (i.e. no Encoders which I believe were added later) we went with that.

If you'd like to contribute to designing codecs2, we'd love your input.

On 3 Sep 2013, at 22:11, Tamás Gulácsi [email protected] wrote:

Why do you encode/marshal into a freshly allocated []byte, and do you require a []byte for unmarshal? Even proper encodings/csv is hidden behind this memory-hungry, nonscalable interface...

—
Reply to this email directly or view it on GitHub.

[tgulacsi]

Thanks for the positive response!

My use case is a niche, but I have to send unlimited data (records from database query resultset) to the client.
I use a hand-made encoder which marshals the result records one after another, and writes them in sequence - thus never more is in memory than one marshaled record.

The generalization is a little bit harder: how should we represent such an iterator-like interface?
Maybe we could define a

type Iterable interface {
Next() interface{}
}

and specialize the marshaler for it - but first a real-world use case would be needed.

The usage of io.Reader / io.Writer is easier, though.

[matryer]

Even without the io.Reader and io.Writer interface, you could use codecs, and just write each response out like you do with your hand-made encoder. At least that way, you could still get the benefit of multiple formats.

An interesting problem.

On 3 Sep 2013, at 23:36, Tamás Gulácsi [email protected] wrote:

Thanks for the positive response!

My use case is a niche, but I have to send unlimited data (records from database query resultset) to the client.
I use a hand-made encoder which marshals the result records one after another, and writes them in sequence - thus never more is in memory than one marshaled record.

The generalization is a little bit harder: how should we represent such an iterator-like interface?
Maybe we could define a

type Iterable interface {
Next() interface{}
}

and specialize the marshaler for it - but first a real-world use case would be needed.

The usage of io.Reader / io.Writer is easier, though.

—
Reply to this email directly or view it on GitHub.

[tylerstillwater]

@tgulacsi @matryer has this issue been resolved? Does it need further discussion? I would like to close it if possible.

[matryer]

I think it should go to designing codecs2 - streaming is definitely a better interface.

On Feb 5, 2014, at 11:31 AM, Tyler [email protected] wrote:

@tgulacsi @matryer has this issue been resolved? Does it need further discussion? I would like to close it if possible.

—
Reply to this email directly or view it on GitHub.