Token Specification
Compiler is a power tool to find errors and give a good output.Lexical analyser is one phase of the compiler.
Lexical analyser is nothing but a set of tokens.Tokens are specified as follows:
Alphabet or Character Class:
Alphabets denotes any number of finte set of symbols.
Eg:{0,1} is the binary alphabet.
String:
String is nothing but a sentence or word,in other words it is a finite sequence of symbols drawn from the alphabet.
Length of the String:
It is nothing but the number of occurence of symbols in string,s denoted by |s|.
Eg:s=abc|s|=3
Empty String:
The empty string is a special string of length zero.
Language:
It denotes any set of strings over some fixed alphabet.
Parts of a String:
Prefix of S:
A string which is obtained by removing zero or more trailing symbols of string S.
Eg:ban is a prefix of banana.
Suffix of S:
A string which is formed by deleting zero or more of the leading symbols of S.
Eg:nana is a suffix of banana
SubString of S:
A string which is obtained by deleting both a prfix and a suffix from the string S.
Proper Prefix:
Any non-empty string X that is a prefix of S,such that S not equal to X.
Subsequence of S:
Any string formed by deleting zero or more not necessarily contiguous symbols from S.
Ex:baaa is a subsequence of banana.
These are the most specified tokens in lexical analyzer of compiler.
Like it on Facebook, Tweet it or share this article on other bookmarking websites.