Question regarding scroll margins and SIXELs

[jerch]

Sorry, if you already stated it elsewhere and I overlooked it - I wonder how a VT340 deals with scroll margins in SIXEL phase.

My idea so far - imho the 300+ line only supported top/bottom restrictions and left/right was a 400+ thing (plz correct me, if thats already wrong).

Now regarding SIXELs - do scroll margins affect SIXEL printing by any means? Does DECOM change here anything? Has DECAWM any influence on SIXEL printing width?

My current expectations from applying basic text mechanics are (given SIXEL scrolling is set):

• printing SIXELs within an active text scroll region cannot leave the region, thus SIXELs would not overlap a bottom scroll border, instead the image gets scrolled within the scroll region, if it happens to be higher. I would assume the same for left/right borders, but that one prolly cannot be tested on a VT340.
• SIXEL printing outside of an active text scroll region would dump the pixels to full viewport extend, until it may cross scroll borders into an active region (then the rule above gets applied for image parts within the scroll region)

Following your findings in other threads and what you wrote so far about SIXEL mechanics my assumption above is certainly wrong. From your writings I would conclude, that SIXELs are not scroll border aware at all, due their more or less direct screen buffer printing skipping most text buffer indirections.

Could you shed light on that mystery on my end? Ofc if needed, I can create a few test cases.

[j4james]

Scroll margins definitely do have an effect - there 's a margin clipping test case here:
https://github.com/hackerb9/vt340test/blob/main/j4james/margin_clipping.sh
And you can see the results here:
https://github.com/hackerb9/vt340test/blob/main/j4james/margin_clipping.png

This only tests top/bottom margins, because that's all that is supported on the VT340, but I'd expect left/right to work the same way (except for scrolling).

I haven't tested anything related to DECOM or DECAWM, because I can't actually think of any difference they would make to sixel output (other than their effect on the text cursor position, which ultimately determines the sixel starting position). What exactly did you have in mind?

[hackerb9]

From your writings I would conclude, that SIXELs are not scroll border aware at all, due their more or less direct screen buffer printing skipping most text buffer indirections.

Please let me know where I wrote that as I should definitely make it clearer. I would say that sixels typically work as if they were part of the text. The illusion is only broken in the corner cases (as @j4james has documented).

Could you shed light on that mystery on my end? Ofc if needed, I can create a few test cases.

You are correct that there are no side margins on a VT340 and @j4james is correct that top and bottom margins do affect sixels.

DECAWM does not impact sixels. The documentation says that anything outside of graphics window is to be simply truncated, not wrapped. (Note that the graphics window is not necessarily the same as the what the text window is displaying. By moving the page, it is possible to draw sixels that appear to be truncated offscreen at the time of drawing and then scroll them onto the screen later. Even so, sixels are never wrapped.)

DECOM should have exactly the impact you'd expect, positioning the cursor relative to the screen or the scrolling region, but I have not tested it.

If you would send a few test cases, I would happily run them for you.

[jerch]

Thx for the answers.

About DECOM:
The question comes from the fact, that if DECOM is set, the text cursor cannot move outside of margins, thus I was wondering if sixel painting would be restricted likewise (e.g. cannot paint pixels below bottom margin on vt340).
And yes, I would also assume to apply the same behavior to left/right borders (xtermjs currently does not support those either, but I want to add support as it is a very convenient text output mode for complicated screen setups.)

About DECAWM:
This was just to make sure, DECAWM would never wrap sixel image parts, that would get truncated at the right border, to the next line below (in the same sense as it wraps text content). So the always truncating graphics window behavior answers that.
This question might look weird in the first place, but we kinda have that issue on almost all TEs implementing sixels with a tile-to-textcell abstraction under reflow logic. It gets evident when you try to resize those terminals with a sixel dump in the viewport.

Prolly will make another test case with scroll margins set and text scrolling happening, because its still not quite clear to me, how sixels pixels (from sixels) on the graphics window will exactly "morph" under these conditions.

[j4james]

I think the margin_clipping test I linked above should actually cover everything you're asking. It tests the rendering of an image starting above the top margin (which is allowed), and extending into the margin area until it overflows the bottom margin, which triggers a scroll.

The segment of the image that was above the margin remains static, while the segment inside the margin boundaries is scrolled up. This basically results in the middle segment of the image being erased as it is scrolled under the top margin.

This first scroll is then followed by an RI (reverse index) below the top margin to test scrolling the image down again (an additional segment of the test pattern is output to fill the line gap that is created by that movement).

Finally an image is output below the bottom margin, large enough that it should actually overflow the bottom of the page. But this doesn't display anything, and doesn't cause the page to scroll, because images below the bottom margin are ignored.

If there is an error in your implementation you are likely to see areas of red in the image where something hasn't been appropriately clipped or scrolled out of view.

This is all in the test pattern on the left, btw. The one on the right is testing how the margin effects the background fill.

The question comes from the fact, that if DECOM is set, the text cursor cannot move outside of margins, thus I was wondering if sixel painting would be restricted likewise (e.g. cannot paint pixels below bottom margin on vt340).

As mentioned above, even without DECOM set you can't paint below the bottom margin, so I don't think setting DECOM is likely to make any difference.

[hackerb9]

Prolly will make another test case with scroll margins set and text scrolling happening, because its still not quite clear to me, how sixels pixels (from sixels) on the graphics window will exactly "morph" under these conditions.

Sure, send it on over and I'll try it out.

[hackerb9]

I was curious about scrolling regions and wrote up a little script to try it out myself. I'm not sure that I'm testing exactly what is being asked for, but the results are pleasingly unsurprising. Nothing weird going on here:

Without scrolling regions:

With a scrolling region:

With both a scrolling region and origin mode enabled:

[jerch]

Wow thx a lot - yepp, thats exactly what I would have tested.

[jerch]

To summarize your test results, thats what I get from it (plz correct any point that I got wrong to avoid the next broken implementation detail, haha):

1. if the text cursor cannot leave the scroll region (DECOM on), a sixel dump within would scroll rows like text, thus scroll the sixels up in the scroll region, truncating when reaching the top border (as seen in 3rd picture)
2. if a scroll region smaller than the viewport is set, things are conditional:
   • if the initial text cursor was outside upon sixel start, it would print sixels normally (upper half of 2nd picture)
   • once the cursor reaches the scroll region, it gets caught and cannot leave it anymore, treated as under rule 1. (lower part of 2nd picture). Bottom border cannot be crossed by normal sixel printing progression.
3. No or full viewport scroll region is just a special case of the 1. rule applied to all cols/rows.

If thats right, things can be applied in a similar fashion to left/right borders. What might be surprising there is the switch from outside to inside scroll borders handling. Example:

EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
EEEEEEEEEEEEEXEEEEEEEEEEEEEEEEEEEEEEEEEE
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
EEEEEEEEEEEEEEEEE#########EEEEEEEEEEEEEE
EEEEEEEEEEEEEEEEE#########EEEEEEEEEEEEEE
EEEEEEEEEEEEEEEEE#########EEEEEEEEEEEEEE
EEEEEEEEEEEEEEEEE#########EEEEEEEEEEEEEE
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE

Where '###' denotes the scrolling window and you start outputting sixels from position X. Following the rules above I'd assume the following output:

EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
EEEEEEEEEEEEEa0sssa1sssa2sssa3sssa4sssa5
EEEEEEEEEEEEEb0sssb1sssb2sssb3sssb4sssb5
EEEEEEEEEEEEEc0sssw1sssw2sEEEEEEEEEEEEEE
EEEEEEEEEEEEEEEEEsx1sssx2sEEEEEEEEEEEEEE
EEEEEEEEEEEEEEEEEsy1sssy2sEEEEEEEEEEEEEE
EEEEEEEEEEEEEEEEEsz1sssz2sEEEEEEEEEEEEEE
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE

'a0' denotes here the row ('a') and the col('0') within the sixel data, as they would have mapped to text cells on a big flat canvas. The scrolling happens only in the window (see how 'w' follows 'b'), futhermore the cursor cannot leave anymore the scrolling window (truncated on left/right borders).
Would that be the an acceptable handling for left/right borders, what you think?

(This is kinda offtopic in a repo for vt340 tests. I can move it elsewhere, if you want.)

[j4james]

if the text cursor cannot leave the scroll region (DECOM on), a sixel dump within would scroll rows like text, thus scroll the sixels up in the scroll region, truncating when reaching the top border (as seen in 3rd picture)

Just to be absolutely clear, DECOM has no effect on the image itself in that test - only the text position. If you replace every occurrence of echo -n ${CSI}H with echo -n "${CSI}${top}H" instead of setting DECOM, you'll get the exact same result. All DECOM is doing is changing the cursor position, which in turn effects the position at which the image is output.

Would that be the an acceptable handling for left/right borders, what you think?

I don't think I'd agree with this. Lines a and b are definitely wrong, because even if the image could start off outside the left margin, it can't extend past the right margin, so the first two lines should also be truncated at that point. And the more I think about, the more I think that images shouldn't even be allowed outside the left margin.

If that's not the case, then I'd expect the image to always be visible outside the left margin, and truncated on the right, even once inside the top/bottom margins. But then it's going to get really weird once you start scrolling, because only the center portion of the image would scroll, but the last sixel line on the left would still continue to be redrawn.

By that logic, I'd expect your test pattern to look like this (note the z0 in the bottom left):

EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
EEEEEEEEEEEEEa0sssa1sssa2sEEEEEEEEEEEEEE
EEEEEEEEEEEEEb0sssb1sssb2sEEEEEEEEEEEEEE
EEEEEEEEEEEEEc0sssw1sssw2sEEEEEEEEEEEEEE
EEEEEEEEEEEEEd0sssx1sssx2sEEEEEEEEEEEEEE
EEEEEEEEEEEEEe0sssy1sssy2sEEEEEEEEEEEEEE
EEEEEEEEEEEEEz0sssz1sssz2sEEEEEEEEEEEEEE
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE

Only in practice it would be even weirder, because the sixel height is not likely to be an exact multiple of the line height, so the z0 area would be a weird amalgamation of sixel segments overriding each other at different y offsets.

In short, I think the most sensible approach would be to disallow images starting outside the left margin. This follows a similar pattern to the ICH and DCH controls, which also aren't allowed outside the left/right margins, but can be used outside the top/bottom margins.

[jerch]

@j4james I see your points, and I think we only differ in 2 aspects:

• never cross right border
  While my version allows that, if the cursor is not within the scroll window (basically testing for both dimensions at once), your version would check (and forbid) border crossing separately. I want that to be as close as possible to text behavior and gonna do some tests on the VT525 regarding scroll borders, once it is up and running.
• continue left sixels even if outside
  Hmm this looks not right to me. Why should the image disregard the left border and always start at initial offset? It looks to me like a violation of the border constraints, that I tend to weigh higher. Also it is not much better in terms of "image preservation" as it introduces more nasty "pixel amalgamation" side effects with that e-z line succession and in the lines above (c-w, d-x, e-y).

In short, I think the most sensible approach would be to disallow images starting outside the left margin. This follows a similar pattern to the ICH and DCH controls, which also aren't allowed outside the left/right margins, but can be used outside the top/bottom margins.

Yes, this seems like a good way to circumvent most of those ugly pixel amalgamation effects.

Edit:
About your DECOM remark - yes thats totally clear.

[j4james]

I want that to be as close as possible to text behavior and gonna do some tests on the VT525 regarding scroll borders, once it is up and running.

I completely agree. I thought what I was proposing was close to the text behavior, but I'm beginning to have my doubts now. I just did a couple of tests on the 5 terminals I know of that support left/right margins, and I got 4 different results! At least most of them agreed on not ever crossing the right border, but I wouldn't necessarily trust that that is correct. I'm happy to wait and see what your VT525 really does.

Btw, I was thinking it might be nice to setup a documentation/testing repo similar to this one for your VT525. Beside the DECPS audio, and this margin discussion, I'm sure there are a lot more features that aren't fully documented that would be worth testing.

[hackerb9]

To summarize your test results, thats what I get from it (plz correct any point that I got wrong to avoid the next broken implementation detail, haha):

1. if the text cursor cannot leave the scroll region (DECOM on), a sixel dump within would scroll rows like text, thus scroll the sixels up in the scroll region, truncating when reaching the top border (as seen in 3rd picture)

Yes, that is the way sixels work when DECSDM (Sixel Display Mode) is disabled.

2. if a scroll region smaller than the viewport is set, things are conditional:
   
   * if the initial text cursor was outside upon sixel start, it would print sixels normally (upper half of 2nd picture)
   * once the cursor reaches the scroll region, it gets caught and cannot leave it anymore, treated as under rule 1. (lower part of 2nd picture). Bottom border cannot be crossed by normal sixel printing progression.

Correct, but the cursor starting outside or inside the scroll region does not need to be handled specially, other than ensuring it is never below the bottom of the region.

3. No or full viewport scroll region is just a special case of the 1. rule applied to all cols/rows.

Again, correct, but I think it is actually simpler than that. No matter whether DECOM is on or off and whether a scrolling region is defined some things are always true (presuming DECSDM is disabled):

1. Sixels scroll exactly like text.
2. An unset scrolling region is equivalent to all lines being the scrolling region.
3. If the cursor is below the scrolling region before printing output, it is moved to the last line of the region.
4. When the region scrolls up, the topmost line is replaced with the line below it, which in turn is replaced by the line below that, and so on. That continues until the bottommost line, which is simply cleared.

Given the above, the only contingency is whether DECOM is set before output. If it is on and the cursor is above the region, then the cursor is moved to the first line of the region. Note that this is the same for text and sixel output, so it is likely already implemented.

If thats right, things can be applied in a similar fashion to left/right borders. What might be surprising there is the switch from outside to inside scroll borders handling.

The VT525 is beyond my knowledge, but I can only say that, whatever you do, make sure sixel scrolling in regions works exactly the same as text scrolling. (Unless DECSDM is set, in which case, sixels never trigger scrolling and output is truncated to the region).

(This is kinda offtopic in a repo for vt340 tests. I can move it elsewhere, if you want.)

It's fine here and is kind of interesting to me, although I cannot help very much.

As for a setting up a special repository for in-depth VT525 documentation & testing, I think that'd be cool. But, until you do, feel free to use these pages to discuss or even submit pull requests with documentation and I'll publish them here.

[jerch]

@j4james Yes thats a good idea, I am sure there are a lot of things to test, the sheer amount of sequences of the late VT devices is overwhelming compared to whats supported in all common TEs these days (which makes them all look like poor clones). I am not advocating for full vt500+ emulation (imho some things are just dead and not worth to be revived), yet it is a good opportunity to fix emulation details for existing stuff, and to dig for concepts and sequence gems, that are worth to be implemented (like left/right borders, some mass actions like rect area functions etc. pp, funny - most things are even older from 300/400 line).

@hackerb9 Yepp, your list of contraints is the better overall summary, thx.

And about DECSDM, what I shamelessly left out of considerations above - can you explain more in detail, how that behaves? (sorry for being a slowpoke) Given DECSDM is set:

• Is DECOM respected for sixel painting start, thus would implicitly start painting at scroll region offset (DECOM on) or viewport home (DECOM off)?
• If the above is true, how does a smaller scroll region affect sixel output, that started outside (DECOM off) at the bottom border? Would it just stop there without scrolling (effectively truncating lower image parts)? I'd assume it never will pass the bottom border from normal image progression, no matter how DECSDM or DECOM is set. If thats true, scroll border constraints would always be active for sixel output, independent of any other setting.

Edit: About no scroll region being explicitly set - the vt520 docs state somewhere, that the margins are set to the viewport margins by default (lines per screen for bottom, page borders for left/right). If thats also true for vt340, an implementation could be simplified to always respect scroll margins, which happens to be the same as the full viewport in that case. (Note that the page concept is not reflected on most TEs, but kinda reduced to a single one.)

[dankamongmen]

if someone has a VT525, i'd love to know the story on #16 (does DECERA apply to sixels?)