parser: Incomplete() returns false for unclosed double quotes and true for unclosed single quotes, while it is expected to return true for both

[Rossil2012]

Hello,

I have encountered a problem with the javascript mvdan/sh library, specifically with the parser's InteractiveStep and Incomplete methods.

In my understanding, when InteractiveStep encounters a string like echo " or echo ', InteractiveStep returns an empty array. And when the Incomplete method is subsequently invoked, it returns true, indicating that InteractiveStep is awaiting the end of the current statement, i.e., the closing double or single quote.

However, my tests indicate that while this behavior is as expected for single quotes echo ', it does not hold for double quotes echo ". In the latter case, the Incomplete method returns false instead of true.

I have written a test that emulates the style of __js/testmain.js:

const assert = require('assert').strict

const sh = require('mvdan-sh')

const syntax = sh.syntax
const parser = syntax.NewParser()

{
  // using the parser interactively with steps
  const lines = [
    "foo\n",
    "bar; baz\n",
    "\n",
    "foo; 'incom\n",
    " \n",
    "plete'\n",

    // testing quote and new line
    // single quote
    "echo \'\n",  // expect: {"count": 0, "incomplete": true}
    "123\n",      // expect: {"count": 0, "incomplete": true}
    "\'\n",       // expect: {"count": 1, "incomplete": false}

    //double quote
    "echo \"\n",  // expect: {"count": 0, "incomplete": true}
    "123\n",      // expect: {"count": 0, "incomplete": true}
    "\"\n",       // expect: {"count": 1, "incomplete": false}
  ]
  const wantResults = [
    { "count": 1, "incomplete": false },
    { "count": 2, "incomplete": false },
    { "count": 0, "incomplete": false },
    { "count": 1, "incomplete": true },
    { "count": 1, "incomplete": true },
    { "count": 2, "incomplete": false },
    // single quote expectation
    { "count": 0, "incomplete": true },
    { "count": 0, "incomplete": true },
    { "count": 1, "incomplete": false },
    // double quote expectation
    { "count": 0, "incomplete": true },
    { "count": 0, "incomplete": true },
    { "count": 1, "incomplete": false },
  ]
  var gotResults = []
  for (var i = 0; i < lines.length; i++) {
    var line = lines[i]
    var want = wantResults[i]

    var stmts = parser.InteractiveStep(line)
    gotResults.push({
      "count": stmts.length,
      "incomplete": parser.Incomplete(),
    })
  }
  assert.deepEqual(gotResults, wantResults)
}

The test fails with the following error:

AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:
+ actual - expected ... Lines skipped

[
...
{
  count: 0,
+ incomplete: false
- incomplete: true
},
{
  count: 0,
+ incomplete: false
- incomplete: true
},
...
  incomplete: false
}
]

{
  generatedMessage: true,
  code: 'ERR_ASSERTION',
  actual: [
    { count: 1, incomplete: false },
    { count: 2, incomplete: false },
    { count: 0, incomplete: false },
    { count: 1, incomplete: true },
    { count: 1, incomplete: true },
    { count: 2, incomplete: false },
    { count: 0, incomplete: true },
    { count: 0, incomplete: true },
    { count: 1, incomplete: false },
    { count: 0, incomplete: false },
    { count: 0, incomplete: false },
    { count: 1, incomplete: false }
  ],
  expected: [
    { count: 1, incomplete: false },
    { count: 2, incomplete: false },
    { count: 0, incomplete: false },
    { count: 1, incomplete: true },
    { count: 1, incomplete: true },
    { count: 2, incomplete: false },
    { count: 0, incomplete: true },
    { count: 0, incomplete: true },
    { count: 1, incomplete: false },
    { count: 0, incomplete: true },
    { count: 0, incomplete: true },
    { count: 1, incomplete: false }
  ],
  operator: 'deepStrictEqual'
}

I am currently using version "0.10.1" of the mvdan-sh library.

Could you please help with this issue?

Thank you in advance.

[mvdan]

Apologies for the slowness here. The reality is that I don't have the time to properly maintain a JS port of this shell library, or to learn enough JS to be a good steward for it. Moreover, I built mvdan-sh on top of GopherJS, which is still somewhat maintained, but is struggling to support the latest versions of Go - which this library uses.

The good news is that someone else built a new JS library on top op of this one, which is actively maintained and uses WASM instead of GopherJS, seemingly bringing a speed-up of over 4x: https://github.com/un-ts/sh-syntax

I'd suggest that you give that a try, and re-raise any issues on that repo if they are still relevant. For now, given that I don't intend to further develop mvdan-sh, I am closing existing issues and deleting the code, but leaving notes in the README and the npm package's page so that any users get pointed in the right direction.

For more, see #1145.

Thanks for understanding :)