Tokens in Python

Tokens in Python are the smallest units of a Python program, such as keywords, identifiers, literals, operators, and delimiters.

Types of Tokens:

• Identifiers
• Keywords
• Literals
• Operators
• Delimiters

1.) Identifiers

An identifier is the name given to variables, functions, or classes. It helps uniquely identify them.

Rules:

• Must start with a letter (A-Z or a-z) or underscore (_).
• Can contain letters, digits (0-9), and underscores.
• Cannot use Python keywords as identifiers.
• Case-sensitive.

Examples:

# Valid Identifiers
my_variable = 10
_myFunction = "Hello"
value123 = 50

# Invalid Identifiers
123value = 50   # Starts with a digit
my-variable = 10  # Contains special character '-'
class = 30       # Uses a keyword

2.) Keywords

Keywords are reserved words in Python that have predefined meanings and cannot be used as identifiers.

• Example Keywords:
  • if, else, while, for, def, return, True, False, None, class, try, except.

Examples:

# Example of keywords in use
if True:
    print("This is a keyword example")

3.) Literals

Literals are constants used in Python programs that represent fixed values.

Types of Literals:

1. String Literals: Enclosed in single, double, or triple quotes.

name = "John"
multiline = """This is 
a multiline string."""

2. Numeric Literals: Integer, float, or complex numbers.

age = 25  # Integer
pi = 3.14  # Float
complex_num = 3 + 5j  # Complex