Tokens in Python

Tokens in Python are the smallest units of a Python program, such as keywords, identifiers, literals, operators, and delimiters.

Types of Tokens:

• Identifiers
• Keywords
• Literals
• Operators
• Punctuators/Special Symbols

1.) Identifiers

An identifier is the name given to variables, functions, or classes. It helps uniquely identify them.

Rules:

• Must start with a letter (A-Z or a-z) or underscore (_).
• Can contain letters, digits (0-9), and underscores.
• Cannot use Python keywords as identifiers.
• Case-sensitive.

Examples:

# Valid Identifiers
my_variable = 10
_myFunction = "Hello"
value123 = 50

# Invalid Identifiers
123value = 50   # Starts with a digit
my-variable = 10  # Contains special character '-'
class = 30       # Uses a keyword

2.) Keywords

Keywords are reserved words in Python that have predefined meanings and cannot be used as identifiers.

• Example Keywords:
  • if, else, while, for, def, return, True, False, None, class, try, except.

Examples:

# Example of keywords in use
if True:
    print("This is a keyword example")

3.) Literals

Literals are constants used in Python programs that represent fixed values.

Types of Literals:

1. String Literals: Enclosed in single, double, or triple quotes.

name = "John"
multiline = """This is 
a multiline string."""

2. Numeric Literals: Integer, float, or complex numbers.

age = 25  # Integer
pi = 3.14  # Float
complex_num = 3 + 5j  # Complex

4.) Punctuators (Special Symbols):

Punctuators are symbols used for various purposes like grouping and separating code elements.

Examples:

• Parentheses () – Used for function calls
• Brackets [] – Used for lists and indexing
• Braces {} – Used for dictionaries
• Colon : – Used in loops and functions
• Comma , – Used for separating values
• Hash # – Used for comments

Example Code:

# Using parentheses
print("Hello, World!")

# Using brackets for lists
my_list = [1, 2, 3]

# Using braces for dictionaries
my_dict = {"name": "Alice", "age": 25}