How to Read a File Line By Line into a List in Python

Python programming frequently involves different tasks such as reading a file line by line and adding it to a list. Therefore, understanding the supported approaches and strategies can substantially simplify the process of extracting required information or processing large data files.

In today’s guide, we will check out multiple built-in tools, including readline(), readlines(), and for loop, as well as more advanced tools like iter() and the mmap module for reading a file line-by-line in Python.

Basic Python File Reading Methods

First of all, we will discuss these basic methods of file reading in Python.

• open()
• readline()
• readlines()
• Simple for loop
• for loop with list comprehension

1. Using Python open() Function

The open() is a built-in Python function that can be utilized for opening a file and returning the relevant file object. This function takes two parameters, the filename and the relevant defined mode in which the file will be opened.

For example, we have used the “open()” function to open the “testfile.txt” in the read mode, which is specified as “r“. Then, we have assigned the file object to the “file” variable.

As the next step, the “read()” method is invoked on the file object to read all lines of the file and save it in the “file_content“.

Lastly, the file content will be printed on the console using the print() function and the file will be closed.

file = open("testfile.txt", "r")
file_content = file.read()
print(file_content)
file.close()

2. Using Python readline() Method

The readline() is another built-in Python method of the file object. This method reads a single line from the specified file and outputs it as a string. It reads the next line on each subsequent call until the file end is reached.

In the provided code, we have invoked the readline() for reading the file lines one by one. The added “while” loop continues until no more lines exist. Inside this loop, each line is displayed with the print() function.

file = open("testfile.txt", "r")
line = file.readline()
while line:
    print(line)
    line = file.readline()
file.close()

3. Using Python readlines() Method

The readlines() method reads all file lines and then outputs a list of strings. In this list, each line comprises the newline character at the end.

Here, the readlines() method will read all lines of the “testfile.txt” and the lines are output as a list. This list will be then assigned to the “lines” variable. Next, a “for” loop is added for iterating through each line in the newly created lines list. Lastly, the file will be closed.

file = open("testfile.txt", "r")
lines = file.readlines()
for line in lines:
    print(line)
file.close()

4. Using Python for loop Method

You can also utilize a for loop directly on the file object. In this scenario, each iteration will assign the current line to a variable, permitting further printing or processing as follows.

file = open("testfile.txt", "r")
for line in file:
    print(line)
file.close()

5. Using Python for loop With List Comprehension

List comprehension can be also used for iterating over the file object and then implementing any particular transformation or operation to each line.

For instance, the following program will open “testfile.txt” in the read mode. Then, the for loop is used with the list comprehension for creating a list named “lines“.

As the next step, each file line is stripped of starting and ending whitespace with the help of the “strip()” method and added to the lines list. Lastly, the lines list will be displayed on the console.

file = open("testfile.txt", "r")
lines = [line.strip() for line in file]
print(lines)
file.close()

Advanced File Reading Methods in Python

This section will discuss some of the advanced file-reading methods in Python.

• rstrip() method
• splitlines() method
• iter() with next() method
• fileinput module
• mmap module

1. Using Python rstrip() Method

rstrip() is a built-in Python method of string that is utilized for removing or eliminating the whitespace characters from the right end of the given string. However, it can be also used for reading lines of a file.

Here, we have used a list comprehension for creating a list named “lines“. Each line from the specified “testfile.txt” is processed using the rstrip() method. This method will remove the trailing whitespaces.

Then, the updated line is added to the lines list. Lastly, the lines list will be displayed on the console and the file will be closed.

file = open("testfile.txt", "r")
lines = [line.rstrip() for line in file]
print(lines)
file.close()

2. Using Python splitlines() Method

splitlines() python method splits a string into a list of lines. This method can recognize different line break representations and removes the line break characters.

In the given code, we have utilized the read() method for reading the file content. The splitlines() method is then applied to the “content” variable. It will split the content into a list of lines.

file = open("testfile.txt", "r")
content = file.read()
lines = content.splitlines()
print(lines)
file.close()

3. Using iter() With next() in Python

You can also implement the approach of creating an iterator object with the help of the “iter()” function. Then, call the “next()” function for fetching the next line from the iterator and assign it to the “line” variable. Lastly, print the file lines on the console.

file = open("testfile.txt", "r")
file_iterator = iter(file)
line = next(file_iterator)
print(line)
file.close()

4. Using Python fileinput Module

Python fileinput module offers an easy way for iterating over the lines from multiple input sources, including files. Its “input()” method is responsible for accepting the file name as input and outputting an iterator that generates lines from the files.

import fileinput

for line in fileinput.input(["testfile.txt"]):
    print(line)

The for loop iterates over each line from the specified “testfile.txt” and displays it.

5. Using Python mmap Module

In Python, the mmap module permits you to map a file into memory and retrieve its content directly. For instance, in the following code, we will use the “with” statement to open the “testfile.txt” in the read mode. Then, the “mmap()” method of the mmap module maps the given file into memory and assigns the resultant memory-mapped file object to “mmapped_file“.

Lastly, the “read()” method is used for reading the entire file content and it will be displayed by calling the print() function.

import mmap

with open("testfile.txt", "r") as file:
    with mmap.mmap(file.fileno(), length=0, access=mmap.ACCESS_READ) as mmapped_file:
        data = mmapped_file.read()
        print(data)

Read a File line-by-line in Python

Have a look at the enlisted best practices for reading a file line-by-line into a list.

• Validate file name and accessibility before reading.
• Utilize a context manager like the “with” statement for automatic file closure.
• Implement error handling for file-related issues.
• Use list comprehension for concise code.
• Clean up data using string manipulation methods like “strip()” and “rstrip()“.
• Keep in mind the memory constraints for large files.

That’s all from today’s informative guide related to reading files in Python.

Conclusion

Knowing how to read a file line-by-line into a list is a fundamental skill in Python. By utilizing several built-in methods such as readline(), readlines(), and for loop, developers can efficiently process or read file content.

On the other side, string manipulation techniques like rstrip() and strip() further enhance data cleaning. Last but not least, advanced techniques like iter() with next(), the fileinput module, and the mmap module can be used for additional flexibility.

Want to explore and learn more related to Python, do check out our dedicated Python Tutorial Series!