Python Mutable Default Arguments: Avoiding Surprises

• Introduction
• Step-by-Step Guide
• Code Example
• Additional Notes
• Summary
• Conclusion
• References

Python functions can have optional arguments with default values, but using mutable objects like lists or dictionaries as defaults can lead to unexpected results. This is because default argument values are created only once during function definition, not each time the function is called. Let's see an example to understand this behavior and how to avoid it.

In Python, functions can have default arguments, making them optional during calls. However, using mutable objects (like lists or dictionaries) as default arguments can lead to unexpected behavior. This is because default argument values are created only once when the function is defined, not every time it's called.

Let's illustrate with an example:

def add_item(item, my_list=[]):
  my_list.append(item)
  return my_list

print(add_item(1))  # Output: [1]
print(add_item(2))  # Output: [1, 2] 

You might expect add_item(2) to return [2], but it returns [1, 2]. This is because the empty list [] assigned to my_list is created only once when the function is defined. Subsequent calls to add_item modify this same list object.

To avoid this, it's best practice to use None as the default argument and create the mutable object inside the function:

def add_item(item, my_list=None):
  if my_list is None:
    my_list = []
  my_list.append(item)
  return my_list

print(add_item(1))  # Output: [1]
print(add_item(2))  # Output: [2]

Now, each call to add_item creates a new list if my_list is not provided, ensuring the function behaves as expected.

The code demonstrates the common pitfall of using mutable default arguments in Python functions. The first function shows how using a list as a default argument can lead to unexpected behavior, as the list is shared across multiple function calls. The second function provides a solution by using None as the default argument and creating a new list inside the function if no list is provided, ensuring that each call operates on a separate list.

# Example demonstrating the issue with mutable default arguments
def add_item(item, my_list=[]):
  """Adds an item to a list.

  Args:
    item: The item to add.
    my_list: The list to add to (defaults to an empty list).
  """
  my_list.append(item)
  return my_list

print("Demonstrating the issue:")
print(add_item(1))  # Output: [1]
print(add_item(2))  # Output: [1, 2]  <-- Unexpected behavior!

# Solution: Using None as default and creating the list inside
def add_item_fixed(item, my_list=None):
  """Adds an item to a list.

  Args:
    item: The item to add.
    my_list: The list to add to (defaults to a new empty list).
  """
  if my_list is None:
    my_list = []
  my_list.append(item)
  return my_list

print("\nDemonstrating the fix:")
print(add_item_fixed(1))  # Output: [1]
print(add_item_fixed(2))  # Output: [2]  <-- Expected behavior!

Explanation:

1. The Problem: The first function add_item uses an empty list [] as the default for my_list. This list is created only once when the function is defined. So, every time you call add_item without providing a list, it modifies the same list object created initially.

2. The Solution: The second function add_item_fixed uses None as the default argument. Inside the function, it checks if my_list is None. If it is, a new empty list is created. This ensures that each call to the function either uses the list provided as an argument or creates a brand new list if none is given.

This pattern of using None as the default argument and then creating the mutable object inside the function is a common practice in Python to avoid unexpected behavior with default arguments.

• Impact on Code Readability: While using mutable defaults might seem convenient initially, it can make your code harder to understand and debug later. Unexpected behavior from shared mutable defaults can lead to subtle bugs.
• Other Mutable Objects: This issue isn't limited to lists. It applies to any mutable object used as a default argument, including dictionaries, sets, and even custom classes.
• Alternatives to None: While None is the most common and recommended way to handle this, you can use a sentinel value (a unique, immutable object) to indicate the absence of a provided argument.
• Importance of Documentation: Clearly document the expected behavior of your functions, especially when it comes to optional arguments and potential side effects. This helps other developers (and your future self) understand how the code works.
• Python's Design Choice: This behavior is a consequence of how Python evaluates default arguments. Understanding this design choice can help you write more predictable and maintainable code.
• "Least Astonishment" Principle: This issue highlights the importance of the "Least Astonishment" principle in programming. Your code should behave in a way that is intuitive and predictable for anyone who uses it.

This article highlights a common pitfall in Python when using mutable objects (like lists or dictionaries) as default arguments in functions.

The Problem: Default argument values are created only once when the function is defined, not every time it's called. This means that if a mutable object is used as a default argument, subsequent calls to the function can unintentionally modify the same object, leading to unexpected behavior.

Example:

Using a list as a default argument:

def add_item(item, my_list=[]):
  my_list.append(item)
  return my_list

Calling add_item multiple times will result in the same list being modified:

print(add_item(1))  # Output: [1]
print(add_item(2))  # Output: [1, 2]  # Unexpected!

Solution:

To avoid this issue, use None as the default argument and create a new mutable object inside the function if the argument is not provided:

def add_item(item, my_list=None):
  if my_list is None:
    my_list = []
  my_list.append(item)
  return my_list

This ensures that each call to the function with a default argument will work with a fresh, independent mutable object:

print(add_item(1))  # Output: [1]
print(add_item(2))  # Output: [2]  # Expected!

Key Takeaway: Be cautious when using mutable objects as default arguments in Python. To avoid unexpected behavior, create new instances of mutable objects within the function if they are intended to be independent for each call.

In conclusion, while default arguments in Python offer convenience, developers should exercise caution when using mutable objects like lists and dictionaries as default values. Due to Python's behavior of evaluating default arguments only once during function definition, using mutable objects can lead to unintended data sharing and unexpected results across multiple function calls. To mitigate this risk, it's best practice to use None as the default argument and create new instances of mutable objects within the function body if the argument is not explicitly provided. This approach ensures that each function call operates on independent objects, preventing unexpected side effects and promoting code clarity and maintainability.

• Understand the behavior of default params in python classes - Anvil ... | What I’m trying to do: Understand the behavior of default params in python classes. I made a sort of random change to the class init, which achieves the behavior I want, but I don’t understand why it does. What I’ve tried and what’s not working: Originally in the Week Custom component the init looked like: def init(self, week_number, days = [], **properties): self.days = days self.week_number = week_number self.init_components(**properties) When printing the length of the ...
• Why is the empty dictionary a dangerous default value in Python ... | Oct 12, 2014 ... "Least Astonishment" and the Mutable Default Argument (34 answers). How can I update an attribute created by a base class' mutable default ...
• Insert Method for Lists - Python Help - Discussions on Python.org | Hi. A Python newbie here. I like to ask how did the below code produce numbers in reverse? I don’t see a reverse method, so how come it resulted to [5,4,3,2,1]? Thank you. </> my_list = [ ] for i in range (5): my_list.insert (0, i + 1) print (my_list) </> [5, 4, 3, 2, 1] Thank you.
• Python list should be empty on class instance initialisation, but it's ... | Dec 26, 2010 ... "Least Astonishment" and the Mutable Default Argument (34 answers) ... Least Astonishment” in Python: The Mutable Default Argument · Share.
• Do not use mutable objects as default arguments in Python : r/Python | Posted by u/[Deleted Account] - 1,320 votes and 221 comments
• python function default parameter is evaluated only once? - Stack ... | Oct 26, 2012 ... "Least Astonishment" and the Mutable Default Argument · 1 · python get new random name for every call of class · -1 · default value for the ...
• Python class function default variables are class objects? - Stack ... | Jul 27, 2011 ... "Least Astonishment" and the Mutable Default Argument (34 answers) ... Why does using arg=None fix Python's mutable default argument issue?
• Recursion, memoization and mutable default arguments in Python ... | Mar 8, 2019 ... ... Python's mutable default arguments. This behavior explains the ... "Least Astonishment" and the Mutable Default Argument · 116 · How does ...
• Python function with default list argument - Stack Overflow | Aug 9, 2013 ... "Least Astonishment" and the Mutable Default Argument · 24 · Python optional parameters · 0 · Append changing all elements from list, python ...