Machine Learning an Artificial algorithm tend to be pretty
sophisticated. It gives the computers the ability to learn from the surrounding
data and make decisions. Instead of building heavy machines we have built such
algorithms that eventually helps to decrease the number of complex algorithms
and helps the computer become independent. In such cases pattern recognition
becomes one the most important challenge. It is used by most of the algorithms
to make optimized decisions. It is mainly a study of how to observe the
environment, distinguish between what should be considered amongst the whole
environment and to take particular decision based on the observations. This
report talks about different machine learning techniques. Also the pattern
recognition process, design cycle, applications and models.
Different types of machines have different machine learning
algorithms, building these algorithms is a challenge for the scientists.
Different algorithms give different learning experience to the machines. It
certainly doesn’t depend on the nature completely but also the data structures
used as well as the theories of cognitive and genetic structures. Many of them
are borrowed from the current neural networks and cognitive sciences. Overall, learning
is to improve performance based on some measure defined to know if the machine
has learned something. We have two main types of algorithms i.e supervised and
unsupervised algorithm. Humans have
developed high abilities to sense the environment like recognizing handwriting,
taste, colour, faces etc. We need to make the machines analyze the same.
Pattern recognition was developed in the 1960s. But in spite of all these years
of research the goal of designing a general purpose pattern recognizer is still
Supervised algorithm perceives both the input as well as the
output and generalizes in a way that it can be used by all possible inputs.
After analyzing the training data it produces an immediate example which can be
used to map new examples. It follows the following steps:
1) Determine the type of training examples – The
user should be aware of what kind of data should be used as training set.
Gather a training set-A set of input and
output is gathered.
the input feature representation of learned function – The accuracy of the
learned function depends on the input representation. The input object is
transformed into a feature vector containing features describing the input
Determine the structure of learning function
& corresponding learning algorithm.
the design and run the learning algorithm on the gather set of data.
Evaluate the accuracy of the learned function.
Supervised learning are of 2
algorithm applies to just nominal responses with few values.
for responses that are a real number.
The supervised algorithms are as
SVM- SVMs are models used to
analyze data for classification and regression analysis. It has good speed and
memory usage when the vectors are few. Even when the default linear scheme is
easy to interpret while using a kernel it is difficult to know how o=the data
is being classified.
Naïve Bayes – It has good speed and
memory usage for simple distributions but is poor for large datasets and kernel
Nearest Neighbor – Nearest neighbor
can have either of categorical or continuous predictors at a time. It has poor
predictions with high dimensions and also does not perform fitting with linear
Discriminant Analysis – It is
accurate when the modelling assumptions are satisfied else the accuracy varies.
The machine receives input but does not obtain the target
output nor the rewards from the environment. But it can develop a framework based
on the knowledge it develops from the environment.
The unsupervised algorithms are as
clustering – Vectors are given as input and a dendogram is
returned as output. It creates a multilevel cluster tree.
K-means structuring – It is more efficient than hierarchical
clustering. In this algorithm each observation is classified into different
clusters depending on it’s nearest mean.
VS UNSUPERVISED LEARNING
supervised algorithm the classes are predetermined whereas in unsupervised the
basic task is to develop classification labels.
supervised algorithm the data can be divided into segments and then the machine
searches patterns and mathematical models based on the data. In unsupervised
algorithm the data is divided into clusters based on their similarities.
supervised algorithm the models are evaluated on the basis of their predictive
capacity in relation to measures of variance in the data itself. Whereas in
unsupervised the machine is told in advance how many clusters should be formed.
FACED IN LEARNING
Learning completely depends on the
machine and the algorithm Since the machines relies on the information it
perceives from the environment, the machines should be ready to face the
challenges it comes across. Such problems affects the learning process of the
machine. As different input gives different output it becomes important to take
into consideration appropriate and optimize output by the machine. The problems
faced during learning are:
BIAS- The machine
tends to prefer one hypothesis over another. Say for example we have two agents
N ad P. Since both the agents have their own hypothesis predicted by taking all
the data into consideration, it becomes difficult for the learning agent to
distinguish between which one is the best. Till the learning agent cannot
choose between the two hypothesis, the agent cannot resolve the disagreement.
In order to come to a conclusion, the agent needs a bias. A good bias is the
one which works best in the practical environment asking which hypothesis suits
the best to the data.
NOISE- In the real world,
data can never be perfect(without noise) . Noise is created when some of the
attributes have missing values, have been assigned inappropriate values.
Handling these noises becomes important for the learning algorithm
RECOGNITION – Pattern recognition is used in the
classification of objects (2D or 3D) or
multidimensional patterns into categories. There are many pattern recognition
systems for character and handwriting, speech and speaker recognition, document,
fingerprint, white blood cell classification, military target recognition. The
machines train the pattern recognition techniques to identify objects for
sorting, inspection, and assembly. The design of a pattern recognition system
requires the following modules:
feature extraction and selection, decision making, and system performance
of low cost and high resolution sensors (e.g., CCD cameras, microphones and
sharing over the Internet have resulted in huge repositories of digitized
and video). Need for efficient archiving and retrieval of this data has
pattern recognition algorithms in new application domains.
OF PATTERN RECOGNITION
Hypothesize the models that describe the two
the data to get rid of the noise in it.
the model that best represents the pattern.
OF PATTERN RECOGNITION
Template matching:- The
pattern to be recognized is matched against a stored template while taking
into account all the
translation, rotation and scale changes.
Statistical pattern recognition:-
It focuses on the statistical properties of the patterns
Artificial Neural Networks:- It focuses on
biological neural models.
Syntactic Pattern Recognition:- It’s decisions
are based on logical rules and grammars.
IN PATTERN RECOGNITION
acquisition and sensing: Measurements of physical
variables, Important issues: bandwidth, resolution, sensitivity, distortion,
SNR, latency, etc.
2) Pre-processing: Removing
noise from the data, separate the patterns of interest from the background.
Feature extraction: Finding a new
representation in terms of features.
Model learning and estimation:
Learning to map between features, pattern and categories.
Using features and learned models for assigning pattern to a category
Post-processing: Evaluating confidence
in decisions, Exploitation of context to improve performance,
Combination of experts.
IN DESIGNING THE PATTERN RECOGNITION SYSTEM
of pattern classes.
– Feature extraction
– Selection of
training and test examples.
OF A PATTERN RECOGNITION SYSTEM:
Designing the pattern has the
Step 1) Data collection: First step
is to collect our training and test data and the question arises
if the data collected has adequate
set of values or not.
Step 2) Feature selection: In this
step we study the data in terms of it’s
domain dependence and prior information, it’s computational cost and
feasibility, values having patterns, values having different
patterns, invariant features with respect to translation, rotation and scale,
robust features with respect to occlusion, distortion, deformation, and
variations in environment..
Step 3) Model Selection:- In this
step we select the model based on the following criteria:- It’s domain
dependence and prior information, Design criteria, parametric and
non-parametric models, handling features with missing values and also it’s
The various models are:- Templates,
theoretic or statistical decision, syntactic or structural, neural, and hybrid. Using these
models we can identify hoe close we are to the final model having the
Step 4) In this phase we decide how
to learn the rules from the provided data.
Learning being of 2 types:-
Supervised learning – Here a
categorical label is provided for each and every pattern in the training set.
Unsupervised learning – The machine
itself forms clusters and groups based
on the input patterns.
Reinforcement learning – Here the
agent provides a feedback of the decision is right or wrong even when the
category is not initially designed.
Step 5) Evaluation – This is the
final step in which we need to evaluate how we can estimate the performance of the training dataset in the
present and also in the near future. And also evaluate the problems faced due
to over fitting.
IN PATTERN RECOGNITION
Techniques for analyzing multidimensional data
of various types and scales along with
algorithms for projection,
dimensionality reduction, clustering and classification of data is
explained. Pattern recognition
models can be designed using the following steps:
matching – For template matching the patterns are represented in the form of pixels,
curves etc. and the recognition function used is correlation between the
patterns and the distance measure. The typical criterion for this approach is
the classification error.
pattern recognition – For statistical pattern recognition the patterns are represented
in the form of features of the patterns and the recognition function used is the
discriminant function. The typical criterion for this approach is the classification
Syntactic or Structural – For statistical
pattern recognition the patterns are represented in the form of primitives of
the patterns and the recognition function used are the rules and the grammar. The
typical criterion for this approach is the acceptance error.
network – For
statistical pattern recognition the patterns are represented in the form of pixels,
features of the patterns and the recognition function used is the network
function. The typical criterion for this approach is the mean square error.
PATTERN RECOGNITION APPLICATIONS
has it’s application in the following areas:
Some examples of pattern recognition applications
are as follows:
Bioinformatics – It is used in sequence
analysis with DNA/Protein sequence as the input. Here the pattern classes are
known types of genes.
Data mining – It is used in searching
meaningful patterns with points in the multidimensional space as the input.
Here the pattern classes are Compact and well as separated
Document Image Analysis – It is used in optical
character recognition with document image as the input. Here the pattern classes
are alphanumeric characters, word.
Document classification – It is used in the
internet search with text document as the input. The patterns are classified in
Industrial automation – It is used in printed
circuit board inspection with intensity image as input. The pattern classes are
either defective or non-defective depending on the nature of the pattern.
Multimedia database retrieval – Internet searching
is one of the major application having video clips as input and patterns classified
on the basis of video genres.
Biometric recognition – Personal identification
uses biometric recognition and has fingerprints, iris, face as input and the
patterns are classified based on authorized users with access control to those
Remote sensing – Remote sensing applies in forecasting the
crop yields with a multi spectral image as an input and classes in the form of
land usage and growth patterns of the crop.
Speech recognition – The telephone directory
uses speech recognition after receiving the speech wave form and forms classes
based on the spoken words.
Medical – Computer aided diagnosis use pattern
recognition with microscopic images.
Military – Automatic target recognition has
classes in the form of target type and optical / infrared image as input.
Natural language processing – It is used in
information extraction with sentences as input and pattern classes as parts of
recognition is used to cover all stages of an investigation from problem formulation
and data collection through to discrimination and classification, assessment of
results and interpretation.
terminologies are described below:
Steps in statistical pattern recognition:
the problem – Understanding completely the aim of investigating and also
planning the remain stages in the entire process.
collection – Recording details of the data collection procedure and measuring all
the appropriate variables.
examination of the data – Verify the data, calculate the summary statistics and
produce the plots in order to get the structure.
selection / feature extraction – Select variables from the sets that are appropriate
for the given task which are gained from the either linear of non linear
transformation of the original set. This feature extraction is artificial to
pattern classification / clustering – We analyze the data and provide a successful
conclusion to our study and also it acts as a pre procesing for the supervised learning.
discrimination or regression procedures as appropriate – Here the classifier is
designed using the training set.
of results – The trained classifier is
applied to the independent test set of patterns that are labeled.
– To analyze the results we need further hypothesis that need further data
collection. This cycle can be terminated at different stages: The hypothesis
posed can be answered at the initial study of the data or may be it is later
discovered that the data cannot answer the stated hypothesis and hence it has
to be reformulated.
pattern recognition Approach
approach all the patterns are represented in the form of d features that are
viewed as a point in the d-dimensional space. The main aim is to select the
features in different categories having pattern vectors so that they can capture
compact and d-dimensional feature space. The separation of different patterns from
the classes determine how effective the representation space is. After
obtaining training data from different classes the main aim is to generate
decision boundaries that separate the patterns that belong to different
classes. In statistical decision theory approach we generate the decision
boundary depending on the probability distribution of the patterns belonging to
different classes and these boundaries should be either specified or learnt.
The discriminate analysis approach can also be used for classification where we
first form a decision boundary in the parametric form and then based on the training patterns we choose the
best decision boundary. These boundaries are created using the mean square error
criterion. According to Vapnik’s philosophy “If we give a restricted amount of data
to solve some problem and try to solve such problem but never try to solve a
more generic problem then we can never conclude based on the information
provided as it is insufficient.”
recognition is a field of study developing significantly from 1960s. It was
very much an
subject, covering developments in the areas of statistics, engineering,
computer science, psychology and physiology, among others. It has huge number
of applications in the field of Bioinformatics, Data Mining, Document
Classification, Document Image Analysis, and Industrial Automation, Multimedia,
Database retrieval, Biometric recognition, Remote sensing, Speech recognition,
Medical, Military, Natural language processing.
Measuring the performance
of learning algorithms and some classifiers have been seen and analyzing the
evaluation methods with metrics they use to measure the performance by defining
formal framework. We have concluded that the performance of the classifiers are
measured on the basis of the classification accuracy. Some methods can be used
to evaluate classifier or algorithm in general while some others are applicable
only to few algorithms. We have also seen how pattern recognition is important
in the field of artificial intelligence. It is emerging as human beings have
their own limits in recognizing patterns. The report also shows how the statistical
approach covers various stages of investigation
from formulating the data to interpreting the results.