Tokens in C language

Tokens in c are the basic building blocks in C language. We can also define a token in c as the smallest individual unit in a program through which a program is structured. The meaning of tokens is already predefined by the C compiler. Therefore it is not possible to create a program without using tokens. So let’s start with tokens.

Introduction

Tokens are the basic building blocks in C. In other words, we can say a token is the smallest individual unit in a C program. In other words, we can say that a program is created using a combination of these tokens. There are 6 main types of tokens.

Types of tokens

Keywords

Identifiers

Constants

Operators

Special Characters

Strings

Keywords: Keywords are those reserved words whose meaning has already been explained to the C compiler. All keywords are a sequence of characters with fixed meanings. These keywords are combined with a formal syntax in C.

Rules for Keywords:

• All keywords are lowercase.
• Keywords can’t be used as a variable or function name.
• All the names of data types are keywords.
• All the names of control statements & looping controls in C are keywords.
• There is a total of 32 keywords in the C language.

int	short	for	continue	enum	return	case
float	unsigned	while	default	volatile	void	volatile
char	signed	do	typedef	register	extern
long	if	goto	struct	auto	const
double	else	break	union	static	short

Identifiers: It refers to the name of variables, functions and arrays f. It helps us to identify data and other objects in the C program. These are user-defined names and consists of alphabets(A-Z,a-z), digits(0-9) & underscores ( _ ) .

Rules for declaring identifiers:

• Both uppercase letters & lowercase letters are permitted although lowercase letters are commonly used.
• Identifier names are case sensitive i.e FIRST is different from first & First
• Identifiers cannot include any special character except underscore “_”.
• We can’t use two consecutive underscores in identifier names.
• Keywords can’t be used as identifier names.
• Underscore is generally used as a link between two words in long Identifiers. example: first_name
• Identifier names can’t start with digits. i.e it begins only with letters or underscores as the first character. But some compiler conflicts using underscore as the first character.
• We can’t use space in Identifier names.
• Identifier name should be unique. i.e we can’t use the same Identifier names in a single program.
• Identifier names can’t contain more than 31 characters. Identifier names can be longer than 31 characters but the compiler looks only at the first 31 characters.
• it’s good practice to use meaningful Identifier names but it’s not compulsory.

Constants: Constants are fixed values assigned to a variable whose value doesn’t change. it is used to define fixed values in a C program.

There are 2 ways to define a variable as constant in C language.

1. Using const keyword Syntax: const data_type var_name=value;
2. Using #define pre-processorSyntax: #define var_name value;

Terms in Constants:

• Values of constants are assigned once at the time of declaration of a variable in the program.
• We cannot change the value of a variable using the const keyword once the value is assigned to that variable.
• Changing the value of a constant in a variable causes a compilation error because we can’t change the value of a constant in the C program.

Types of Constants:

There are 2 types of constants in the C language.

Types of Constants
Primary Constants
Secondary Constants

1. Primary Constants: There are 3 types of Primary Constants in the C language.

• Numeric Constant: A constant of numeric type consists of a sequence of digits or numbers with a fractional value. Example: 0,1,34,567, 1234L,12U, 012,-123,+123,0.02, -0.23, +0.34, 0.02F, 0.5e2,14E-2, -5.6e-2 etc.Invalid: 123 456, 12,34,567, $123 etc.
• Character Constant: A character constant consists of a single character enclosed within single quote. In computers, characters are stored using the machine’s character set using ASCII values. all escape sequences are also character constants. Example: ‘a’, ‘@’, ‘&’, ‘\n’ etc.
• Logical Constant: A logical constant consists of logarithmic constants in a variable.Example: const x=log 10;

2. Secondary Constants: There are 5 types of Secondary Constants in C.

Arrays: An array is a collection of homogenous types of variable values.

Terms in Arrays:

• It is a collection of similar data items & these data items may be int, float or all char. 

• An array is also known as the subscripted variable • Each member of an array is identified by a unique index or subscript assigned to it. 

• An array is a +ve integer enclosed in [ ] placed imminently after the array name. 

• An index holds the integer value starting with zero. Syntax: data_type array_name[size of array];For example, int marks[5], float temp[5] etc.

Structure: A structure is a user-defined data type that can be used to group elements of different data types into a single type. it is declared using the struct keyword followed by a structure name. The difference between a structure & an array is that an array contains related information of the same data type but structures have the features to store related information even of different data types. structure helps us to save our time & memory by declaring a single variable instead of multiple variables. this helps a programmer to write efficient & fast executable codes. Syntax:struct structure_name{data_type var_name;data_type var_name;};For example, Suppose, there is a school in which we have to create a record of students’ data like name, roll no, date of birth, father’s name, class, address etc. so storing students’ records using arrays is not possible because we can’t store records of different data types in an array.

• Union: As similar to structure, a union is a collection of variables of different data types. the only difference between structure & a union is that in the case of unions, you can only store information in one field at any one time. unions are used to save memory. Syntax:union union_name{data_type var_name;data_type var_name;};
• Pointer: A pointer provides access to a variable by using the address of that variable.Syntax:data_type *ptr_name;
• Enum: The enum data type is a user-defined type based on the standard integer type. it consists of a set of named integer constants. The enum the keyword is basically used to declare and initialize a sequence of integer constants. Syntax:enum enumeration_name{identifier1,identifier2,identifier3....identifierN};

Operators: An operator is a symbol that tells the computer to perform certain mathematical or logical operations for meaningful purposes. it is used in programs to manipulate data & variables. C languages are reached by the operators.

Categories of operators:

1. Unary operators: It is an operator which is used to operate on one operand. For example, increment and decrement operators are unary operators.var_name++ a++ a-- --b ++b etc.
2. Binary operators: It is an operator which is used to operate on two operands.
   • Arithmetic Operators: This operator is used to perform arithmetical tasks i.e addition(+), subtraction(-), multiplication(*), and division(/) along with the modulus operator(%).

Operator	Description
+	Addition
–	Subtraction
*	Multiplication
/	Division
%	Modulus

• Relational Operators: C supports 6 relational operators. it returns either true or false & in C language, true is represented by 1 and false is represented by 0.

Operator	Description
==	Is equal to
!=	Is not equal to
>	Greater than
<	Less
>=	Greater than or equal to
<=	Less than or equal to

• Logical Operators: C supports 3 Logical operators. it is used when we want to perform operations on more than one condition.

Operator	Description
&&	If both conditions are true then the result is true.
।।	If both conditions are opposite then the result is true.
!	If both conditions are opposite then the result is true.

• Increment Operators: It means increasing the value of a variable by 1.

Operator	Description
++	Increment

• Decrement Operators: It means decreasing the value of a variable by 1.

Operator	Description
—	Decrement

• Assignment Operator: it is used to assign the result of an expression to a variable.

Operator	Description
=	Assign
+=	Increments, then assign
-=	Decrements, then assign
*=	Multiplies, then assign
/=	Divides, then assign
%=	Modulus, then assigns
<<=	Left shift and assigns
>>=	Right shift and assigns
&=	Bitwise AND assigns
^=	Bitwise exclusive OR and assigns
।=	Bitwise inclusive OR and assigns

• Bitwise Operators: C supports bitwise operators for the manipulation of data at the bit level. it is not applied to integers only & works on binary values.

Operator	Description
<<	Binary Left Shift Operator
!=	Is not equal to
>>	Binary Right Shift Operator
~	Binary One’s Complement Operator
&	Binary AND Operator
^	Binary XOR Operator
।	Binary OR Operator

• Conditional Operators: C supports a ternary operator i.e conditional operator.

Operator	Description
?:	Conditional Operator

• Special Operators: C supports the following special operators.

Operator	Description
,	Comma operator
sizeof()	Returns the size of a memory location.
&	Returns the address of a memory location.
*	Pointer to a variable.
. & ->	Member selection operator

Special Characters in C:

• Square brackets [ ]: Used for single and multi-dimensional arrays.
• Simple brackets(): Used for function declaration.
• Curly braces { }: Used for opening and closing the code.
• The comma (,): Used to separate variables.
• Hash/pre-processor (#): Used for the header file.
• Asterisk (*): Used for Pointers.
• Tilde (~): Used for destructing the memory.
• Period (.): Used for accessing union members.
• Colon(:): Used for labelling or in some operators.
• The semicolon (;): Used to terminate the sentence in a program
• The assignment operator(=): Used to assign the value to the variable.

Strings: it is a collection/array of characters enclosed within double quotes. it has a null character ‘\0’ at the end of the string.

Syntax: data_type var_name="value";

For example, char a[5]="Harsh";

 

Examples to implement tokens in c

Example 1 – Keywords

#include<stdio.h>
int main()
{
    int a;
    float b;
    char c;
    char d[5];
    const float pi=3.14;
    a=3;
    b=2.05;
    c='A';
    d[5]="Harsh";
    printf("int a=%d\n",a);
    printf("float b=%f\n",b);
    printf("char c=%c\n",c);
    printf("string d=%s\n",d[5]);
    printf("pi=%f",pi);
}

Output:

int a=3 float b=2.050000 char c=A string d=(null) pi=3.140000

Example 2 – Switch

#include<stdio.h>
void main()
{  
    int a,b, result;
    char ch;
    printf("enter first number:");
    scanf("%d",&a);
    printf("enter second number:");
    scanf("%d",&b);
    printf("enter the choice(+,-,*,/):");
    scanf("%c,&ch");
    switch(ch)
    {
     case'+':
      result=a+b;
      printf("result=%d",result);
      break;
     case'-':
      result=a-b;
      printf("result=%d",result);
      break;
     case'*':
      result=a*b;
      printf("result=%d",result);
      break;
     case'/':
      result=a/b;
      printf("result=%d", result);
      break;
      case 5:
     default:
      printf("you entered invalid choice");
    }
    return 0;
}

Output:

enter first number:10 enter second number:12 enter the choice(+,-,*,/):+ result=22

Example 3 – Typedef

#include <stdio.h>
#include <string.h>
typedef struct Courses {
char courseName[60];
float CourseFee;
char companyName[100];
int loginID;
} Courses; 
int main()
{
Courses course;
strcpy( course.courseName, "C Programming");
strcpy( course.companyName, "EDUCBA");
course.CourseFee = 5000.00;
course.loginID=2452;
printf( "Course Name : %s\n", course.courseName);
printf( "Company Name : %s\n", course.companyName);
printf( "Course Fee : %f\n", course.CourseFee);
printf( "Login ID : %d\n", course.loginID);
return 0;
}

Conclusion

• Tokens in C language are the basic building blocks of a program.
• This is the smallest individual unit in a C program. It means a C program is constructed using a combination of these tokens.
• The meaning of keywords is defined in the c compiler.
• Operators are used to performing certain mathematical & logical tasks.