v4.0.1 tokenizer treats Vue template text as a string and silently omits later messages

[hbl18]

In vue3-gettext@4.0.1, valid text in a Vue template can put the tokenizer into string-reading mode even though the character is not a JavaScript string delimiter.

When a straight quote immediately follows text that the tokenizer recognizes as ParenLeft or Comma, the tokenizer calls readString(). If no matching quote follows, it reads to EOF, prints this error, and loses later translation calls:

parsing error, string literal is not closed until end of file

The extraction command still exits successfully and writes an incomplete POT file. This makes the failure easy to miss and can cause existing translations to become obsolete or disappear during the following catalog merge.

This appears related to #66, but the behavior below is reproducible with stable 4.0.1: it does not throw an exception, and the command reports success despite omitting a valid message.

Minimal reproduction

Create an empty directory with these files.

package.json

{
  "private": true,
  "type": "module",
  "scripts": {
    "gettext:extract": "vue-gettext-extract"
  },
  "dependencies": {
    "vue3-gettext": "4.0.1"
  }
}

gettext.config.cjs

'use strict';

module.exports = {
  input: {
    path: '.',
    include: ['src/**/*.vue'],
    exclude: [],
    parserOptions: {
      mapping: {
        simple: ['$gettext'],
      },
      overrideDefaultKeywords: false,
    },
  },
  output: {
    path: './i18n',
    potPath: './messages.pot',
    jsonPath: '../translations.json',
    locales: ['en'],
    flat: true,
    linguas: false,
  },
};

src/App.vue

<template>
  <p>('</p>
  <p>{{ $gettext(`Hello`) }}</p>
</template>

Install and extract:

npm install
mkdir -p i18n
npm run gettext:extract

Actual behavior

The command prints:

parsing error, string literal is not closed until end of file

It nevertheless exits with status 0, prints Extraction successful, and writes a POT file without Hello:

msgid ""
msgstr ""
"Content-Type: text/plain; charset=UTF-8\n"

The exported parser API demonstrates the same result:

import { parseSrc } from 'vue3-gettext';

const source = `<template>
  <p>('</p>
  <p>{{ $gettext(\`Hello\`) }}</p>
</template>
`;

console.log(parseSrc(source)); // []

Removing the first <p> makes Hello extract normally.

Expected behavior

• Text nodes and other non-script regions must not be interpreted as JavaScript string literals.
• $gettext(`Hello`) must be extracted.
• If extraction cannot safely continue, the command must exit non-zero and must not replace catalogs with incomplete output.

Expected POT entry:

#: src/App.vue:3
msgid "Hello"
msgstr ""

Root cause indication

The v4.0.1 tokenizer scans the complete SFC as undifferentiated text. It emits ParenLeft for ( in the template text. The following ' then satisfies the condition used to start readString(), even though it belongs to a Vue text node. readString() consumes the remainder of the file while looking for another ', so it never tokenizes the later $gettext(`Hello`) call.

Impact

A successful exit with incomplete extraction is destructive for established catalogs:

1. CI and local scripts treat extraction as successful.
2. Missing messages are merged as removed/obsolete.
3. Subsequent cleanup can delete translations and translator comments.
4. Large generated diffs appear unrelated to the source change that triggered them.

Suggested direction

Possible safeguards:

1. Parse Vue SFC regions before tokenizing, or otherwise prevent template text/markup from opening JavaScript string tokens.
2. Restrict string scanning to arguments of recognized gettext keywords rather than every ( or , token in the file.
3. Treat an unterminated string as a fatal extraction error and avoid writing/merging partial output.

Suggested regression assertions:

• parseSrc() returns the Hello message for the reproduction above.
• No diagnostic is emitted for valid Vue template text.
• CLI exits non-zero and preserves prior output if any source file genuinely cannot be parsed.

Environment

• vue3-gettext: 4.0.1
• Node.js: reproducible with Node.js 24.20.0
• OS: macOS
• GNU gettext tools installed (msgmerge, msginit, msgattrib)

---

Disclaimer: AI assistance was used in diagnosing, isolating the minimal reproduction, and drafting this issue.