Research

Features:

1. spatio-temporal information can be used to form 3-D neighborhood granules in videos with natural partitioning,
2. neighborhood granular rough rule-base can be formed on 3-D granulated space to reduce the computational complexity,
3. granular rough flow graph can be defined for rule-base adaptation,
4. flow graph can be used for the tasks of testing and validation of the rule-base with two added features, namely, expected flow graph and mutual dependency

Neighborhood Granular Rough Rule-base and Flow Graph for Unsupervised Tracking (NRBFG)

Overview:

The initial P frames are given to the rule-base for initial labeling, marked as I1 in the figure below. The current frame (ft) and its previous P frames (ft−p) in a video, marked as I2, are the input for their processing. The spatiocolor and spatio-temporal granules are formed over these frames, shown in blocks ‘A’ and ‘B’, respectively. The decision-making regarding the object–background separation in ft is performed according to the rule-base. The foreground segmentation is then done (block ‘C’) depending on the output of the rule-base. The coverage of the rule-base is checked (block ‘D’) afterwards over the segmented output. If the coverage is satisfactory, then the foreground segment will be tracked, otherwise the rule-base will get updated with the granular flow graph (block ‘E’).

Details

Formation of Neighborhood Granules

1. Spatio-color Granules:

A granule $N(x_i)$ around a point $x_i$ in universe $U$ is represented as:

$$N(x_i) = \cup x_j \in U$$

where $x_i$ and $x_j$ are binary connected over the condition $|color(x_j) − color(x_i)| < Thr$. In other words, a region growing operation is performed around $x_i$ with threshold value $Thr$. $Thr$ is the color nearness threshold. That is, how much of a similar color will fall into a bin.
Note: $x_i$, around which a granule is formed, is such a point which is not already contained by any other granule.

2. 3-D Spatio-temporal Granules:

Note: All the changed information is computed with respect to the current frame.
If the current frame ($f_t$) is of size $M×N$ and its previous P frames ($f_{t−p} : p = 1, ... , P$) are considered, then the changed information between $f_t$ and $F_{t−p}$ (denoted as $\tau_{p}$) is computed according to:

$$\tau_p = |f_t − f_{t−p}| \space \forall p \in P$$

The matrix ($\tau_p$) is of size $M×N$ and there are P such matrices. The median of $\tau_p : p = 1, . . . , P (\tau_{med})$ is a matrix of size $M × N$ and is computed as:

$$\tau_{med} = Median(\tau_1, . . . , \tau_P )$$

Let $x_i$ be the position of a pixel in the current ($t^{th}$) frame, then the 3-D granules around it are formed according to:

$$N_{sp−tmp}(x_i) = \cup x_j \in U$$

where $x_i$ and $x_j$ are binary connected over $|\tau(x_j) − \tau(x_i)| < Thr_t$ and $x_j \in f_p : p = t, . . . , t − P$.
Note: the value of P depends on the speed of the moving object(s) from frame-to-frame.

3. Color Neighborhood Granules

These granules are formed with the RGB-D values of the points present in each $N_{sp−tmp}$.
Let $c_i$ be the color value of a point in $N_{sp−tmp}$. Then the granule around it is defined as:

$$N_{clr}(c_i) = \{ c_j \in N_{sp-tmp}: \Delta (c_i, c_j) \le Thr_c \}$$

The color and spatio-temporal granules, thus formed from initial P video frames, are used for rule-base generation and adaptation. This adaptive rule-base, formed over the granulated space, is used for tracking unknown input frames.

Rough rule-base Creation

Colour and Depth Granules formed from initial P video frames together with Spatio-Temporal Gra- nules serve as foundational elements for the creation of rule-base.

This rule-base, serves as classificator for all spatio-colour granules ℵx generated over current frame and consists of four attri- butes for each feature:

• Belongs (Be): All of $\aleph_x$ pixels values (either coordinates, colour or depth) are encapsulated in corresponding granule $\aleph_{sp−tmp x}$, $\aleph_{RGB x}$, $\aleph_{D x}$.
• Does Not Belong (NB): None of the pixels belong to rule-base granules.
• Partially Belongs PB: Some of the pixels belong to the rule-base granules.
• Contained In (CC): All of the rule-base granule pixels belong to $\aleph_x$.

Decision:

• O - $\aleph_x$ represents an object
• B - $\aleph_x$ represents the background

Generation of Granular Flow Graph

...

Object tracking

...