Z-order for some strokes is wrong, wrong, wrong.

[Fweeb]

A little background is in order here.

In SWF, every edge can have a line style and two fill styles. This is how SWF handles overlapping geometry. Taken from the SWF specification:

For example, if a shape consists of two overlapping squares, and only FillStyle0 is defined, Flash Player renders a ‘hole’ where the paths overlap. This area can be filled using FillStyle1. In this situation, the rule is that for any directed vector, FillStyle0 is the color to the left of the vector, and FillStyle1 is the color to the right of the vector

Now, the PYSWF library that has been pulled into this add-on helps a bit with its SWFShape object. In particular, if you run the _create_edge_maps() function, you populate the object's fill_edge_maps and line_edge_maps arrays which, in turn, are used by the _create_path_from_edge_maps() function. That last function is particularly useful because the resulting path is a sequence of edges with one line style and just one fill style.

The problem is that the library doesn't seem to do a good job of picking the correct fill style in the cases where you actually have a fill style index (indices start at 1, so an index of zero is considered not having a style). In fact, in those circumstances, you end up with two paths: one for each fill style. One of those paths is correctly filled. The other one has a fill style that matches the fill of larger stroke surrounding it (because, really, it's supposed to be a hole).

Granted, Grease Pencil strokes don't really support holes and you have to make due with a special holdout material... but that's a separate discussion.

The point here is that right now, imported paths get two strokes... and the wrong one is often on top. So you get a result like this (artwork courtesy of Stephen Brooks and his book Tradigital Animate CC):

Notice the fingers, Adam's apple, and ears.

I think the place to fix this is in the _process_sub_path() function in the SWFShape object definition here in data.py... but I could be wrong.

[Fweeb]

Just a note on something that I tried, using @vxlcoder's help. I modified _process_sub_path() to be the following, but it seems that there's still stuff going on that I don't understand, so the wrong fill style gets chosen.

def _process_sub_path(self, sub_path, linestyle_idx, fillstyle_idx0, fillstyle_idx1, record_id=-1):
        path = None

        if fillstyle_idx0 != 0 or fillstyle_idx1 != 0 and len(sub_path) > 1:
            # Reasonably sure this is a fill 
            # Find point closes to origin and associated edges
            a, b = None, None
            for c, d in zip(sub_path, sub_path[1:] + sub_path[:1]):
                if a:
                    if c.to[0] > a.to[0]:
                        continue
                    elif c.to[0] == a.to[0] and c.to[1] > a.to[1]:
                        continue
                a, b = c, d
            # Get values for calculating
            ax = a.to[0] - a.start[0]
            ay = a.to[1] - a.start[1]
            bx = b.to[0] - b.start[0]
            by = b.to[1] - b.start[1]
            # Calculate the dot product
            z = (ax * by) - (ay * bx)
            # Now we can figure out which style to use
            if z > 0: # Counter-clockwise geometry, use fill style 0
                # Reusing orginal PYSWF code here... still not entirely sure why it reverses direction for fs0
                if not fillstyle_idx0 in self.current_fill_edge_map:
                    path = self.current_fill_edge_map[fillstyle_idx0] = []
                else:
                    path = self.current_fill_edge_map[fillstyle_idx0]
                for j in range(len(sub_path) - 1, -1, -1):
                    path.append(sub_path[j].reverse_with_new_fillstyle(fillstyle_idx0))
            else: # Clockwise geometry, use fill style 1
                if not fillstyle_idx1 in self.current_fill_edge_map:
                    path = self.current_fill_edge_map[fillstyle_idx1] = []
                else:
                    path = self.current_fill_edge_map[fillstyle_idx1]
                self._append_to(path, sub_path)

        if linestyle_idx != 0:
            if not linestyle_idx in self.current_line_edge_map:
                path = self.current_line_edge_map[linestyle_idx] = []
            else:
                path = self.current_line_edge_map[linestyle_idx]
            self._append_to(path, sub_path)

In particular, I'm not entirely clear on why it's necessary to reverse the direction of the path if it's found to use fillstyle_idx0.

[Fweeb]

Thinking more on this, it's worth considering that I'm going about this the wrong way. The main challenge is that Grease Pencil strokes need to know when they're located within the space of another stroke. The simplified pseudocode for this would go something like this:

path = None
for edge_sequence in all_edges:
    container = find_container(edge_sequence) # Returns None if the sequence isn't in a container
    if container is not None:
        path = set_fill_color(edge_sequence, container) # Returns a path with the appropriate fill style (including 0; a hole)
    else:
        path = set_fill_color(edge_sequence) # Returns a path with whichever fill style isn't 0

I think that's right. The hard part is figuring out the logic for find_container() and picking the right fill style index in set_fill_color().

[Fweeb]

Got this a bit closer with the commit c973ac2, with @vxlcoder's help. Now we have a true z-order issue (before it was more than z-order... before we were generating duplicate geometry with two different styles, based on edges with both style indices populated). This commit fixed that, but now we we have the real z-order problem.

I suspect that the problem comes from here in _process_sub_path() where a reversed sub_path is appended to the edgemap. Here are the lines (minus snarky comments) so you don't have to click the link:

            if fillstyle_index_use is fillstyle_idx0:
                for j in range(len(sub_path) - 1, -1, -1):
                    path.append(sub_path[j].reverse_with_new_fillstyle(fillstyle_idx0))

I've tried putting this condition after the self._append_to(path, sub_path) function, but it doesn't appear to give favorable results.