`err` Not Working as (I) Expected

[kevinmeredith]

In the following Parser:

object Foo extends JavaTokenParsers { 

  def word(x: String) = s"\\b$x\\b".r

  lazy val expr  = aSentence | something

  lazy val aSentence = noun ~ verb ~ obj

  lazy val noun   = word("noun")
  lazy val verb   = word("verb") | err("not a verb!")
  lazy val obj    = word("object")

  lazy val something = word("FOO")
}

It will parse noun verb object.

scala> Foo.parseAll(Foo.expr, "noun verb object")
res1: Foo.ParseResult[java.io.Serializable] = [1.17] parsed: ((noun~verb)~object)

But, when entering a valid noun, but an invalid verb, why won't the err("not a verb!") return an Error with that particular error message?

scala> Foo.parseAll(Foo.expr, "noun vedsfasdf")
res2: Foo.ParseResult[java.io.Serializable] =
[1.6] failure: string matching regex `\bverb\b' expected but `v' found

noun vedsfasdf
     ^

credit: Thanks to Travis Brown for explaining the need for the word function here.

@dcsobral provides a thorough explanation here - http://stackoverflow.com/questions/25147833/using-err-in-a-child-parser as to why this behavior is occurring.

I don't completely understand his answer (re-reading it a few more times), but shouldn't err(... in my above example result in that particular Error?

[gourlaysama]

Indeed, err should probably consume whitespace.

It doesn't always matter what err consumes since, well, it produces an error. But when two alternatives both return an error, the parser has to pick one, and it picks the one that consumes the most input (favoring the second one if they consume the same amount). And err can lose when the alternative is a regex or literal parser, since those are little cheaters that consume additional whitespace.

See #25 (comment) for more reading.

As a workaround, as shown in @dcsobral's answer, you can manually consume whitespace before err every time you use it, or possibly override err yourself to do the right thing once and for all:

// just call ws(someParser) to have it handle whitespace
private def ws[T](p: Parser[T]): Parser[T] = new Parser[T] {
  def apply(in: Input) = {
    val offset = in.offset
    val start = handleWhiteSpace(in.source, offset)
    p(in.drop (start - offset))
  }
}

override def err(msg: String) = ws(super.err(msg))

...

lazy val verb   = word("verb") | err("not a verb!") // will now do the right thing

[gourlaysama]

I'm reopening this since the fix has been reverted. See #41 for more info.

[hrj]

Hmm... If the fix was reverted because of binary incompatibility, re-applying it for 1.1.0 should be possible?

I ask because this seems to be the only issue blocking a 1.1.0 release.

[hrj]

@gourlaysama Does this require more than a reapplication of the previous patch?

[Philippus]

Moved the milestone for this one, as 1.1.0 was already released.