Tokens in C language

Job Updates Join
YouTube Watch

Tokens in c are the basic building blocks in C language. We can also define a token in c as the smallest individual unit in a program through which a program is structured. The meaning of tokens is already predefined by the C compiler. Therefore it is not possible to create a program without using tokens. So let’s start with tokens.

Table of Contents

Introduction

Tokens are the basic building blocks in C. In other words, we can say a token is the smallest individual unit in a C program. In other words, we can say that a program is created using a combination of these tokens. There are 6 main types of tokens.

Types of tokens

Keywords
Identifiers
Constants
Operators
Special Characters
Strings

Keywords: Keywords are those reserved words whose meaning has already been explained to the C compiler. All keywords are a sequence of characters with fixed meanings. These keywords are combined with a formal syntax in C.

Rules for Keywords:

  • All keywords are lowercase.
  • Keywords can’t be used as a variable or function name.
  • All the names of data types are keywords.
  • All the names of control statements & looping controls in C are keywords.
  • There is a total of 32 keywords in the C language.
int short for continue enum return case
float unsigned while default volatile void volatile
char signed do typedef register extern  
long if goto struct auto const  
double else break union static short

Identifiers: It refers to the name of variables, functions and arrays f. It helps us to identify data and other objects in the C program. These are user-defined names and consists of alphabets(A-Z,a-z), digits(0-9) & underscores ( _ ) .

Rules for declaring identifiers:

  • Both uppercase letters & lowercase letters are permitted although lowercase letters are commonly used.
  • Identifier names are case sensitive i.e FIRST is different from first & First
  • Identifiers cannot include any special character except underscore “_”.
  • We can’t use two consecutive underscores in identifier names.
  • Keywords can’t be used as identifier names.
  • Underscore is generally used as a link between two words in long Identifiers. example: first_name
  • Identifier names can’t start with digits. i.e it begins only with letters or underscores as the first character. But some compiler conflicts using underscore as the first character.
  • We can’t use space in Identifier names.
  • Identifier name should be unique. i.e we can’t use the same Identifier names in a single program.
  • Identifier names can’t contain more than 31 characters. Identifier names can be longer than 31 characters but the compiler looks only at the first 31 characters.
  • it’s good practice to use meaningful Identifier names but it’s not compulsory.

Constants: Constants are fixed values assigned to a variable whose value doesn’t change. it is used to define fixed values in a C program.

There are 2 ways to define a variable as constant in C language.

  1. Using const keyword Syntax: const data_type var_name=value;
  2. Using #define pre-processorSyntax: #define var_name value;

Terms in Constants:

  • Values of constants are assigned once at the time of declaration of a variable in the program.
  • We cannot change the value of a variable using the const keyword once the value is assigned to that variable.
  • Changing the value of a constant in a variable causes a compilation error because we can’t change the value of a constant in the C program.

Types of Constants:

There are 2 types of constants in the C language.

Types of Constants
Primary Constants
Secondary Constants
  1. Primary Constants: There are 3 types of Primary Constants in the C language.
  • Numeric Constant: A constant of numeric type consists of a sequence of digits or numbers with a fractional value. Example: 0,1,34,567, 1234L,12U, 012,-123,+123,0.02, -0.23, +0.34, 0.02F, 0.5e2,14E-2, -5.6e-2 etc.Invalid: 123 456, 12,34,567, $123 etc.
  • Character Constant: A character constant consists of a single character enclosed within single quote. In computers, characters are stored using the machine’s character set using ASCII values. all escape sequences are also character constants. Example: ‘a’, ‘@’, ‘&’, ‘\n’ etc.
  • Logical Constant: A logical constant consists of logarithmic constants in a variable.Example: const x=log 10;

2. Secondary Constants: There are 5 types of Secondary Constants in C.

Arrays: An array is a collection of homogenous types of variable values.

Terms in Arrays:

• It is a collection of similar data items & these data items may be int, float or all char. 

• An array is also known as the subscripted variable • Each member of an array is identified by a unique index or subscript assigned to it. 

• An array is a +ve integer enclosed in [ ] placed imminently after the array name. 

• An index holds the integer value starting with zero. Syntax: data_type array_name[size of array];For example, int marks[5], float temp[5] etc.

Structure: A structure is a user-defined data type that can be used to group elements of different data types into a single type. it is declared using the struct keyword followed by a structure name. The difference between a structure & an array is that an array contains related information of the same data type but structures have the features to store related information even of different data types. structure helps us to save our time & memory by declaring a single variable instead of multiple variables. this helps a programmer to write efficient & fast executable codes. Syntax:struct structure_name{data_type var_name;data_type var_name;};For example, Suppose, there is a school in which we have to create a record of students’ data like name, roll no, date of birth, father’s name, class, address etc. so storing students’ records using arrays is not possible because we can’t store records of different data types in an array.

  • Union: As similar to structure, a union is a collection of variables of different data types. the only difference between structure & a union is that in the case of unions, you can only store information in one field at any one time. unions are used to save memory. Syntax:union union_name{data_type var_name;data_type var_name;};
  • Pointer: A pointer provides access to a variable by using the address of that variable.Syntax:data_type *ptr_name;
  • Enum: The enum data type is a user-defined type based on the standard integer type. it consists of a set of named integer constants. The enum the keyword is basically used to declare and initialize a sequence of integer constants. Syntax:enum enumeration_name{identifier1,identifier2,identifier3....identifierN};

Operators: An operator is a symbol that tells the computer to perform certain mathematical or logical operations for meaningful purposes. it is used in programs to manipulate data & variables. C languages are reached by the operators.

Categories of operators:

  1. Unary operators: It is an operator which is used to operate on one operand. For example, increment and decrement operators are unary operators.var_name++ a++ a-- --b ++b etc.
  2. Binary operators: It is an operator which is used to operate on two operands.
    • Arithmetic Operators: This operator is used to perform arithmetical tasks i.e addition(+), subtraction(-), multiplication(*), and division(/) along with the modulus operator(%).
Operator Description
+ Addition
Subtraction
* Multiplication
/ Division
% Modulus
  • Relational Operators: C supports 6 relational operators. it returns either true or false & in C language, true is represented by 1 and false is represented by 0.
Operator Description
== Is equal to
!= Is not equal to
> Greater than
< Less
>= Greater than or equal to
<= Less than or equal to
  • Logical Operators: C supports 3 Logical operators. it is used when we want to perform operations on more than one condition.
Operator Description
&& If both conditions are true then the result is true.
।। If both conditions are opposite then the result is true.
! If both conditions are opposite then the result is true.
  • Increment Operators: It means increasing the value of a variable by 1.
Operator Description
++ Increment
  • Decrement Operators: It means decreasing the value of a variable by 1.
Operator Description
Decrement
  • Assignment Operator: it is used to assign the result of an expression to a variable.
Operator Description
= Assign
+= Increments, then assign
-= Decrements, then assign
*= Multiplies, then assign
/= Divides, then assign
%= Modulus, then assigns
<<= Left shift and assigns
>>= Right shift and assigns
&= Bitwise AND assigns
^= Bitwise exclusive OR and assigns
।= Bitwise inclusive OR and assigns
  • Bitwise Operators: C supports bitwise operators for the manipulation of data at the bit level. it is not applied to integers only & works on binary values.
Operator Description
<< Binary Left Shift Operator
!= Is not equal to
>> Binary Right Shift Operator
~ Binary One’s Complement Operator
& Binary AND Operator
^ Binary XOR Operator
। Binary OR Operator
  • Conditional Operators: C supports a ternary operator i.e conditional operator.
Operator Description
?: Conditional Operator
  • Special Operators: C supports the following special operators.
Operator Description
, Comma operator
sizeof() Returns the size of a memory location.
& Returns the address of a memory location.
* Pointer to a variable.
. & -> Member selection operator

Special Characters in C:

  • Square brackets [ ]: Used for single and multi-dimensional arrays.
  • Simple brackets(): Used for function declaration.
  • Curly braces { }: Used for opening and closing the code.
  • The comma (,): Used to separate variables.
  • Hash/pre-processor (#): Used for the header file.
  • Asterisk (*): Used for Pointers.
  • Tilde (~): Used for destructing the memory.
  • Period (.): Used for accessing union members.
  • Colon(:): Used for labelling or in some operators.
  • The semicolon (;): Used to terminate the sentence in a program
  • The assignment operator(=): Used to assign the value to the variable.

Strings: it is a collection/array of characters enclosed within double quotes. it has a null character ‘\0’ at the end of the string.

Syntax: data_type var_name="value";

For example, char a[5]="Harsh";

 

Examples to implement tokens in c

Example 1 – Keywords

#include<stdio.h>
int main()
{
    int a;
    float b;
    char c;
    char d[5];
    const float pi=3.14;
    a=3;
    b=2.05;
    c='A';
    d[5]="Harsh";
    printf("int a=%d\n",a);
    printf("float b=%f\n",b);
    printf("char c=%c\n",c);
    printf("string d=%s\n",d[5]);
    printf("pi=%f",pi);
}

Output:

int a=3 float b=2.050000 char c=A string d=(null) pi=3.140000

Example 2 – Switch

#include<stdio.h>
void main()
{  
    int a,b, result;
    char ch;
    printf("enter first number:");
    scanf("%d",&a);
    printf("enter second number:");
    scanf("%d",&b);
    printf("enter the choice(+,-,*,/):");
    scanf("%c,&ch");
    switch(ch)
    {
     case'+':
      result=a+b;
      printf("result=%d",result);
      break;
     case'-':
      result=a-b;
      printf("result=%d",result);
      break;
     case'*':
      result=a*b;
      printf("result=%d",result);
      break;
     case'/':
      result=a/b;
      printf("result=%d", result);
      break;
      case 5:
     default:
      printf("you entered invalid choice");
    }
    return 0;
}

Output:

enter first number:10 enter second number:12 enter the choice(+,-,*,/):+ result=22

Example 3 – Typedef

#include <stdio.h>
#include <string.h>
typedef struct Courses {
char courseName[60];
float CourseFee;
char companyName[100];
int loginID;
} Courses; 
int main()
{
Courses course;
strcpy( course.courseName, "C Programming");
strcpy( course.companyName, "EDUCBA");
course.CourseFee = 5000.00;
course.loginID=2452;
printf( "Course Name : %s\n", course.courseName);
printf( "Company Name : %s\n", course.companyName);
printf( "Course Fee : %f\n", course.CourseFee);
printf( "Login ID : %d\n", course.loginID);
return 0;
}

Conclusion

  • Tokens in C language are the basic building blocks of a program.
  • This is the smallest individual unit in a C program. It means a C program is constructed using a combination of these tokens.
  • The meaning of keywords is defined in the c compiler.
  • Operators are used to performing certain mathematical & logical tasks.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top