Image forming system, apparatus and method of discriminative color features extraction thereof

US 20090087088A1
Filed: 07/29/2008
Published: 04/02/2009
Est. Priority Date: 09/28/2007
Status: Active Grant

First Claim

Patent Images

1. An image forming system, comprising:

a photographing device to photograph an image of an object;

a color feature extracting device receiving the image from the photographing device, extracting a discriminative color feature from the image, generating a final color model of the object, extracting a color blob of the object based on the final color model, performing blob analysis on the extracted color blob, and generating parameters to control a posturing of the photographing device according to the blob analysis;

a control device receiving from the color feature extracting device the parameters to control the posturing of the photographing device, and controlling the posturing of the photographing device;

a storage device storing the photographed image of the object; and

a display device displaying the photographed image of the object.

View all claims

1 Assignment

Timeline View

Assignment View

0 Petitions

Accused Products

Abstract

Provided are an apparatus and method of extracting a discriminative color feature, and an image forming system including: a photographing device to photograph an image of an object; a color feature extracting device receiving the image from the photographing device, extracting a discriminative color feature from the image, generating a final color model of the object, extracting a color blob of the object based on the final color model, performing blob analysis on the extracted color blob, and generating parameters to control a posturing of the photographing device according to the blob analysis; a control device receiving from the color feature extracting device the parameters to control the posturing of the photographing device, and controlling the posturing of the photographing device; a storage device storing the photographed image of the object; and a display device displaying the photographed image of the object.

9 Citations

View as Search Results

25 Claims

1. An image forming system, comprising:
- a photographing device to photograph an image of an object;
  
  a color feature extracting device receiving the image from the photographing device, extracting a discriminative color feature from the image, generating a final color model of the object, extracting a color blob of the object based on the final color model, performing blob analysis on the extracted color blob, and generating parameters to control a posturing of the photographing device according to the blob analysis;
  
  a control device receiving from the color feature extracting device the parameters to control the posturing of the photographing device, and controlling the posturing of the photographing device;
  
  a storage device storing the photographed image of the object; and
  
  a display device displaying the photographed image of the object.
- View Dependent Claims (2, 3, 4, 5)
- - 2. The system of claim 1, further comprising:
    - a labeling device providing an object area to the color feature extracting device, the object area being manually labeled on the image by a user.
  - 3. The system of claim 1, wherein the control device controls at least one of operations that include a pan, a tilt, a zoom, and a selection of a light condensing area according to the parameters to control the posturing of the photographing device.
  - 4. The system of claim 3, wherein the control device controls the selection operation of the light condensing area to select a new area where the object is located, as a light condensing basis, to condense light in the new area.
  - 5. The system of claim 4, wherein the control device controls the selection operation of the light condensing area to select a central area of the image as a default light condensing area or to dynamically select a new image area where the object is located as a light condensing area to form the image, and to dynamically control a zoom rate, a focal distance, a pan parameter, or a tilt parameter based on image data information of the light condensing area.

6. A color feature extracting device, comprising:
- a main color extracting unit dividing an object area and a non-object area of an input image into a plurality of rectangles, generating a color histogram for each color channel, and extracting a main color of an object rectangle from the color histogram;
  
  a weight calculating unit calculating a minimum distance between a color histogram bin of the object rectangle and a color histogram bin of a non-object rectangle with respect to the main color, a bin weight of the color histogram of the object rectangle, a color channel weight of the object rectangle, and a weight of the object rectangle to extract a discriminative color feature of the object; and
  
  a final color model generating unit generating a final color model of the object based on a result of re-weighting the color histogram bins of the object rectangle.
- View Dependent Claims (7, 8, 9, 10, 11, 12, 13, 14, 15, 16)
- - 7. The color feature extracting device of claim 6, wherein the main color extracting unit comprises:
    - an area divider dividing the object area and the non-object area of the input image into the plurality of rectangles;
      
      a histogram calculator generating the color histogram of the object rectangle and of the non-object rectangle and the color histogram of the non-object rectangle using the color channel, each color histogram being divided into a plurality of discrete bins; and
      
      a main color extractor extracting the main color of the object rectangle and of the non-object rectangle based on the generated color histogram of the object rectangle.
  - 8. The color feature extracting unit of claim 6, wherein the weight calculating unit comprises:
    - a minimum bin distance calculator calculating the minimum distance between the color histogram bin of the object rectangle and the color histogram bin of the non-object rectangle with respect to the main color;
      
      a bin weight calculator calculating the weight of the color histogram bin of the object rectangle based on the calculated minimum distance;
      
      a color channel weight calculator calculating the color channel weight of the object rectangle based on the calculated bin weight;
      
      a rectangle weight calculator calculating the weight of the object rectangle based on the calculated bin weight; and
      
      a bin re-weight calculator re-weighting the color histogram bin of the object rectangle based on the color channel weight and the weight of the object rectangle to extract the discriminative color feature of the object.
  - 9. The color feature extracting device of claim 8, wherein the bin minimum distance calculator calculates the minimum distance between the color histogram bin of the object rectangle and the color histogram bin of the non-object rectangle according to, $J$
    - [ H r c 
      
      ( i ) , BG c 
      
      ( j ) ] = arg 
      
      
      
      min j 
      
      
      
      b i - b j ′
      
      
      
      , 
      
      D r c 
      
      ( i ) = 
      
      b i - b j ′
      
      
      
      where c denotes the color channel, r denotes the object rectangle, b_idenotes an i^thbin, J denotes one bin of the color histogram of the non-object rectangle of the color channel with a minimum distance to the i^thbin of the color channel from a main color histogram of the object rectangle, H_r^c(i) denotes a color histogram of the i^thbin of the color channel of the object rectangle, BG^c(j) denotes a non-object color histogram of a j^thbin of the color channel of the non-object rectangle, b′
      
      _jdenotes the j^thbin, and D_r^c(i) denotes a minimum distance between the i^thbin of the color histogram bin of the object rectangle and a non-object rectangle color histogram with a same color channel.
  - 10. The color feature extracting device of claim 8, wherein the bin weight calculator calculates the weight of the color histogram bin of the object rectangle according to,
    w_r^c(i)=T(D_r^c(i),s)×
    - |H_r^c(i)−
      
      BG^c(J)|,where s=sign(H_r^c(i)−
      
      BG^c(J)), T(x, y)=e^x×
      
      k×
      
      y, and k is a constant.
  - 11. The color feature extracting device of claim 10, wherein the color channel weight calculator calculates the weight of the color channel of the object rectangle according to, $w^{c} = \sum$
    - m = 1 M 
      
      w r c 
      
      ( m ) $W^{c} = \frac{w^{c}}{\sum_{k = 1}^{K} w^{k}},$ where M denotes a number of bins of an object rectangle histogram, K denotes a number of color channels, the weight of each bin of the color channel of the object rectangle is accumulated in $w^{c} = \sum_{m = 1}^{M} w_{r}^{c} (m),$ and the weight of the color channel is calculated in $W^{c} = \frac{w^{c}}{\sum_{k = 1}^{K} w^{k}} .$
  - 12. The color feature extracting device of claim 11, wherein the rectangle weight calculator calculates the weight of the object rectangle according to, $w_{r} = \sum$
    - k = 1 K 
      
      ∑
      
      m = 1 M 
      
      w r k 
      
      ( m ) , 
      
      W r = w r ∑
      
      l = 1 L 
      
      w l , where L denotes a number of object rectangles, a sum of bin weight of the object rectangle histogram is calculated in $w_{r} = \sum_{k = 1}^{K} \sum_{m = 1}^{M} w_{r}^{k} (m),$ and the weight of the object rectangle is calculated in $W_{r} = \frac{w_{r}}{\sum_{l = 1}^{L} w_{l}} .$
  - 13. The color feature extracting device of claim 12, wherein the bin re-weight calculator re-weights the color histogram bin of the object rectangle, based on the color channel weight and the weight of the object rectangle, according to,
    W_r^c(i)=w_r^c(i)×
    - W_r×
      
      W^cand thereby calculates the re-weighted color histogram bin of the object rectangle W_r^c(i).
  - 14. The color feature extracting device of claim 13, wherein the final color model generating unit generates the final color model of the object according to,
    N={N^r,N^g,N^b},where, with respect to the color channel, $N^{c}$
    - ( i ) = ∑
      
      r = 1 R 
      
      ∑
      
      m = 1 M 
      
      W r c 
      
      ( b m = i ) , R denotes a number of object rectangles, and M denotes the number of bins of the object rectangle histogram.
  - 15. The color feature extracting device of claim 6, further comprising:
    - a color blob extracting unit extracting a color blob of the object from a new input image based on the final color model of the object.
  - 16. The color feature extracting device of claim 15, further comprising:
    - an object locating unit performing blob analysis on the extracted color blob and calculating a centroid and a size of an object in the input image based on the blob analysis to locate and track the object.

17. A method of extracting a discriminative color feature, comprising:
- dividing an object area and a non-object area of an input image into a plurality of rectangles, and generating a color histogram for each color channel to extract a main color of an object rectangle from the color histogram;
  
  calculating a minimum distance between a color histogram bin of the object rectangle and a color histogram bin of the non-object rectangle to identify a discriminative color histogram bin of the object rectangle from the histogram bin of the non-object rectangle;
  
  calculating a bin weight of the color histogram of the object rectangle based on the calculated minimum distance;
  
  calculating a color channel weight of the object rectangle and a weight of the object rectangle based on the calculated bin weight of the color histogram of the object rectangle;
  
  re-weighting the color histogram bin of the object rectangle based on the calculated color channel weight and the weight of the object rectangle to extract a discriminative color feature of the object; and
  
  generating a final color model of the object based on the result of re-weighting the color histogram bins of the object rectangle.
- View Dependent Claims (18, 19, 20, 21, 22, 23, 24, 25)
- - 18. The method of claim 17, wherein the calculating of the minimum distance calculates the minimum distance between the color histogram bin of the object rectangle and the color histogram bin of the non-object rectangle according to, $J$
    - [ H r c 
      
      ( i ) , BG c 
      
      ( j ) ] = arg 
      
      
      
      min j 
      
      
      
      b i - b j ′
      
      
      
      $D_{r}^{c} (i) = \langle b_{i} - b_{j}^{'} \rangle,$ where c denotes the color channel, r denotes the object rectangle, b₁denotes an i^thbin, J denotes one bin of the color histogram of the non-object rectangle of the color channel with a minimum distance to the i^thbin of the color channel from a main color histogram of the object rectangle, H_r^c(i) denotes a color histogram of the i^thbin of the color channel of the object rectangle, BG^c(j) denotes a non-object color histogram of a j^thbin of the color channel of the non-object rectangle, b′
      
      _jdenotes the j^thbin, and D_r^c(i) denotes a minimum distance between the i^thbin of the color histogram bin of the object rectangle and a non-object rectangle color histogram with a same color channel.
  - 19. The method of claim 17, wherein the calculating of the bin weight calculates the weight of the color histogram bin of the object rectangle according to,
    w_r^c(i)=T(D_r^c(i),s)×
    - |H_r^c(i)−
      
      BG^c(J)|,where s=sign(H_r^c(i)−
      
      BG^c(J)), T(x, y)=e^x×
      
      k×
      
      y, and k is a constant.
  - 20. The method of claim 19, wherein the color channel weight and the weight of the object rectangle is calculated according to, $w^{c} = \sum$
    - m = 1 M 
      
      w r c 
      
      ( m ) $W^{c} = \frac{w^{c}}{\sum_{k = 1}^{K} w^{k}},$ where M denotes a number of bins of an object rectangle histogram, K denotes a number of colors channels, the weight of each bin of the color channel of the object rectangle is accumulated in $w^{c} = \sum_{m = 1}^{M} w_{r}^{c} (m),$ and the weight of the color channel is calculated in $W^{c} = \frac{w^{c}}{\sum_{k = 1}^{K} w^{k}} .$
  - 21. The method of claim 20, wherein the calculating of the color channel weight and the weight of the object rectangle calculates the weight of the object rectangle according to, $w_{r} = \sum$
    - k = 1 K 
      
      ∑
      
      m = 1 M 
      
      w r k 
      
      ( m ) , 
      
      W r = w r ∑
      
      l = 1 L 
      
      w l , where L denotes a number of object rectangles, a sum of bin weight of the object rectangle histogram is calculated in $w_{r} = \sum_{k = 1}^{K} \sum_{m = 1}^{M} w_{r}^{k} (m),$ and the weight of the object rectangle is calculated in $W_{r} = \frac{w_{r}}{\sum_{l = 1}^{L} w_{l}} .$
  - 22. The method of claim 21, wherein the color histogram bin of the object rectangle is re-weighted based on the color channel weight and the weight of the object rectangle, according to,
    W_r^c(i)=w_r^c(i)×
    - W_r×
      
      W^c,and thereby calculates the re-weighted color histogram bin of the object rectangle W_r^c(i).
  - 23. The method of claim 22, wherein the final color model of the object is generated according to,
    N={N^r,N^g,N^b},where, with respect to the color channel, $N^{c}$
    - ( i ) = ∑
      
      r = 1 R 
      
      ∑
      
      m = 1 M 
      
      W r c 
      
      ( b m = i ) , R denotes a number of object rectangles, and M denotes the number of bins of the object rectangle histogram.
  - 24. The method of claim 17, further comprising:
    - extracting a color blob of the object from a new input image based on the final color model of the object.
  - 25. The method of claim 24, further comprising:
    - performing blob analysis on the extracted color blob and calculating a centroid and a size of an object in the input image based on the blob analysis to locate and track the object.

Specification

Resources

Litigation Campaign Assessment

Current Assignee
Samsung Electronics Co. Ltd.
Original Assignee
Samsung Electronics Co. Ltd.
Inventors
Chen, Maolin, Jeong, Moon Sik

Granted Patent

US 8,331,667 B2
Time in Patent Office

Days
Field of Search
US Class Current

382/165
CPC Class Codes

G06F 18/2321   using statistics or functio...

G06V 10/56   relating to colour

G06V 10/763   Non-hierarchical techniques...

G06V 40/10   Human or animal bodies, e.g...

Image forming system, apparatus and method of discriminative color features extraction thereof

First Claim

1 Assignment

0 Petitions

Accused Products

Abstract

9 Citations

25 Claims

Specification

Solutions

Use Cases

Quick Links

Image forming system, apparatus and method of discriminative color features extraction thereof

First Claim

1 Assignment

Subscription Required

Subscription Required

0 Petitions

Subscription Required

Accused Products

Subscription Required

Abstract

9 Citations

25 Claims

Specification

Subscription Required

Solutions

Use Cases

Quick Links