III. Proposed System Model:
system model is classified into four Parts:
& processed the images of sign taken from video using Image Processing
features from images using feature extraction technique
and Indentified word of sign on the base of extracted features using
NLP to collected words and format the sentence
A. Extract & processed the
images of sign taken from video using Image processing techniques
model only one person will be considered. Taking video by camera and input into
the system. In this paper both one hand and two hand gesture including face are
and process the image of sign
in figure-1 frame Extraction will do by Video Capture Object. Then frames will be saved in jpeg image. Image will be converted into gray scale image. Apply otsu’s
thresolding technique on gray scale image to convert into the binary image.
Then crop the target part from the image using skin color based segmentation.
B. Extract features from images using feature
Figure.2 Find Features of sign
shows that feature extraction method is apply on processed and target cropped
images to extract features. This paper mention that we will going to use convex
hull method to extract features like shape, area, angle, curve, exterior coordinates,
and orientation for accurately recognize the word.
Classified and Indentified word of sign using classification technique
Figure.3 Recognize word using
shown in figure-3, extracted features will be taken as inputs into
classification technique which will perform reorganization and output should be
work will use Neuro-fuzzy classifier as classification technique to recognize
the word by using of extracted features. A Neuro-fuzzy approach is termed as Neuro-fuzzy
system. It is a hybrid of neural network and Fuzzy Logic (If-Then-Else Rules)
shown below in figure-4.
As shown in following
figure-4, a Neuro-fuzzy system can be viewed as a 5-layer feed forward neural
network. The first layer represents input variables, the middle (hidden) layer
(combine by 3 layers) represents fuzzy system and the third layer represents
output variables. Fuzzy sets are encoded as (fuzzy) connection weights.
D. Apply NLP to collected words
and format the sentence
Figure-5 Tagging and Parsing to
format the sentence)
per above figure-5 Collected words will input into NLP engine then apply POS
tagging and parsing.
Stanford POS tagging tool will be
used to tag the word 10. POS tagger tags the words by one of the part of
speech like noun, pronoun, verb, adjective, adverb, preposition, conjunction,
Tagged words will apply to format
the sentence. This model will use LR parser to parsing the meaningful sentence.