Skip to content

fix(expansion): keep empty fields when IFS delimiters repeat - #1282

Open
luantaraschi wants to merge 1 commit into
reubeno:mainfrom
luantaraschi:fix/ifs-empty-fields
Open

luantaraschi wants to merge 1 commit into
reubeno:mainfrom
luantaraschi:fix/ifs-empty-fields

Conversation

@luantaraschi

Copy link
Copy Markdown
Contributor

Field splitting treated every IFS character the same way: end the current field, and keep it if it has content. That is right for the whitespace characters, which collapse into a single separator. The other ones each end a field, even when the field is empty, so a:b::c is four fields.

With IFS=: and var holding a:b::c:

bash before after
set -- $var; echo $# 4 3 4
printf '[%s]' "$@" [a][b][][c] [a][b][c] [a][b][][c]

The same collapse hit ${arr[@]}, $(...) output and for loops, so a colon-separated record with an empty column came out one column short and the rest shifted left. read splits elsewhere and was already right, which is probably why this went unnoticed.

ifs.yaml already had six cases for it, all marked known_failure. They pass now, so the marker is gone from all six.

The rule the splitter follows, each part checked against bash:

  • IFS whitespace before the first field is dropped, and a run of it ends one field.
  • A non-whitespace delimiter ends a field. Whitespace on either side of it belongs to the same run, so a : b with IFS=': ' is two fields. A second non-whitespace delimiter in the same run ends a second, empty field.
  • A leading delimiter leaves an empty first field. A trailing one does not add an empty last field.

is_ifs_whitespace spells out the six characters instead of calling char::is_whitespace, because bash goes by isspace in the C locale. The vertical tab counts, which char::is_ascii_whitespace leaves out, and the non-breaking space does not, which char::is_whitespace counts. There is a test case for the first half of that.

Six new cases cover the parts that had no coverage: a run of three or more delimiters, leading against trailing, whitespace around a delimiter, the whitespace character set, a delimiter split across two expansions in one word ($a$b where a ends in : and b starts with one), and an array expansion whose fields each split on their own.

One thing this does not fix: literal text in a word is still split, so IFS=:; set -- a::b gives three fields where bash gives one. That is #295, and the two cases covering it stay marked as known failures.

@reubeno reubeno added the state: needs review PR or issue author is waiting for a review label Aug 22, 2026
Field splitting ended a field on every IFS character but only kept the field
when it had content, so a run of delimiters always collapsed into one. That
is right for the whitespace characters and wrong for the others: with IFS=:
the value a:b::c is four fields, and brush gave three.

The six cases in ifs.yaml that covered this were marked known_failure. They
pass now, so the marker is gone.

Assisted-by: Claude Code:claude-opus-5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

state: needs review PR or issue author is waiting for a review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants